Plugin with two datasets parameters, will always pop-up gui in macro runs

Dear image.sc community,

I really need your expertise!

TLDR:

  • We have a plugin to run ilastik which has two parameters of type DataSet (the auto-generated goi shows dropdown lists).
  • Macro recording works :white_check_mark:, however
  • Running the macro again with the same recorded Parameters will still pop up the Gui and effectively make it impossible to run in batch mode.
  • A “minimal” running example of this plugin can be found here: https://github.com/k-dominik/minimal-fiji-plugin-issue. I’ve also pre-build the plugin here so you can just install it by placing it into your plugins folder

So, how to make this work in batch?

longer version:

we, at the ilastik team have an (imagej2) plugin to bridge the gap from ilastik to ImageJ/Fiji: ilastik4ij. It allows users to run ilastik workflows from within fiji. In the simple case, the user supplies a few parameters and a dataset and gets back an image. This already works beautifully using using the scijava magic to gather user input. Macro recording/playback also works nicely in the simple case.
But for more complex things, we don’t use one, but two input images of type net.imagej.Dataset. In this case, when the macro is recorded, the parameters are resolved correctly. When running the plugin, however, there will be a popup asking for the parameters again. This effectively renders batch processing in this case impossible.
I have boiled down our plugin to the most simple thing that still reproduces the error in this repository. This thing is so simple I’m almost inclined to post the source code here :wink: (the only code can be found in …/DemoCommand.java). It has two parameters of type net.imagej.Dataset, and when run, it does nothing but print a debug message. It shows the same behavior with respect to macro recording/playback as our original plugin.
You can try it our yourself, a binary can be found here.
There is even an example folder in the repository that contains a recorded macro along with two images.

So it would be really great if someone has an idea how to get this running, how to debug this better, or point me to someone who might know something.

Thanks a lot!

Disclaimer:

I am not a java developer, leave alone familiar with all the fanciness of the scijava plugin auto-magic that is at play here. A lot of great people have contributed to our Fiji plugin in the past that allows ilastik to be run from within Fiji. So it would be great to resolve this remaining issue and make our plugin fly :rocket:

There is an original thread that is concerned with this behavior: Object Classification using the ilastik plugin for Fiji, but it was very specific to the ilastik plugin, and I believe that this thread has greater chances of finding the cause.

2 Likes

Sorry to bump this, but @ctrueden, could you please have a quick look?

Hi @k-dominik ,
This is not exactly a solution but maybe a workaround :
You ask the user to select just one folder, with a constrain architecture like :
SelectedFolder
|_subfolderA
|_subfolderB
Cheers,
R

2 Likes

The problem here is that when running the macro command, the framework needs to convert from a String (the name of the image) to your required input (of type Dataset), which is internally achieved using the SciJava ConvertService.

Currently, Fiji supports conversion from String to ij.ImagePlus, but not directly to net.imagej.Dataset, as illustrated by this little Groovy script runnable from the script editor:

#@ ConvertService convertService

import net.imagej.Dataset
import ij.ImagePlus

println convertService.supports("someImageName", ImagePlus.class) // true
println convertService.supports("someImageName", Dataset.class) // false

I added a StringToDatasetConverter (among other converters) to the SCIFIO library in December last year:

The pull request was merged last April, but the scifio versions >0.39.0 containing it didn’t make it to an update site yet. You might still try if depending on scifio-0.41.0 solves your issue.


Another option would be to add another (delegate) converter to imagej-legacy to do:

  • String => ImagePlus => Dataset
1 Like

Another option is to replace Datasetto ImagePlus, but I do not know to which extent this affect the rest of the repo:


    @Parameter(label = "Image 1")
    public ImagePlus inputImage1;

    @Parameter(label = "Image 2")
    public ImagePlus inputImage2;
1 Like

Of course, but I was hesitant to suggest a backwards-oriented change towards ImageJ 1.x classes. But since ilastik4ij depends on net.imagej:ij anyways already, that’s a fair suggestion :slight_smile:.

1 Like

@imagejan
Do you have the full GAV of scifio ? I wanted to try your suggestion but I don’t know which one from the multiple repo @ maven.scijava.org I should use.

1 Like

This does not seem to exist:

        <dependency>
            <groupId>scifio</groupId>
            <artifactId>scifio</artifactId>
            <version>0.41.0</version>
        </dependency>
1 Like

The groupID is io.scif (corresponding to the scif.io URL):

<dependency>
  <groupId>io.scif</groupId>
  <artifactId>scifio</artifactId>
  <version>0.41.0</version>
</dependency>

See: https://maven.scijava.org/#nexus-search;gav~io.scif~scifio~~~

2 Likes

Very weird things are happening with scifio v0.41.0:

It looks like it tries to open the png file (but there’s no need because the file is already opened in imagej):


[ERROR] Error loading file
java.io.IOException: io.scif.img.ImgIOException: java.io.IOException: io.scif.FormatException: Invalid PNG signature.
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:163)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:133)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:138)
	at io.scif.convert.FileToDatasetConverter.convert(FileToDatasetConverter.java:66)
	at org.scijava.convert.AbstractConvertService.convert(AbstractConvertService.java:128)
	at org.scijava.convert.AbstractDelegateConverter.convert(AbstractDelegateConverter.java:25)
	at org.scijava.convert.AbstractConvertService.convert(AbstractConvertService.java:128)
	at net.imagej.legacy.plugin.MacroPreprocessor.process(MacroPreprocessor.java:82)
	at org.scijava.module.ModuleRunner.preProcess(ModuleRunner.java:105)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:157)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: io.scif.img.ImgIOException: java.io.IOException: io.scif.FormatException: Invalid PNG signature.
	at io.scif.img.ImgOpener.createReader(ImgOpener.java:489)
	at io.scif.img.ImgOpener.openImgs(ImgOpener.java:242)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:152)
	... 16 more
Caused by: java.io.IOException: io.scif.FormatException: Invalid PNG signature.
	at io.scif.AbstractReader.setSource(AbstractReader.java:279)
	at io.scif.services.DefaultInitializeService.initializeReader(DefaultInitializeService.java:91)
	at io.scif.img.ImgOpener.createReader(ImgOpener.java:483)
	... 18 more
Caused by: io.scif.FormatException: Invalid PNG signature.
	at io.scif.formats.APNGFormat$Parser.typedParse(APNGFormat.java:408)
	at io.scif.formats.APNGFormat$Parser.typedParse(APNGFormat.java:391)
	at io.scif.AbstractParser.parse(AbstractParser.java:244)
	at io.scif.AbstractParser.parse(AbstractParser.java:314)
	at io.scif.AbstractParser.parse(AbstractParser.java:53)
	at io.scif.AbstractReader.setSource(AbstractReader.java:271)
	... 20 more
[ERROR] Error loading file
java.io.IOException: io.scif.img.ImgIOException: java.io.IOException: io.scif.FormatException: Invalid PNG signature.
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:163)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:133)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:138)
	at io.scif.convert.FileToDatasetConverter.convert(FileToDatasetConverter.java:66)
	at org.scijava.convert.AbstractConvertService.convert(AbstractConvertService.java:128)
	at org.scijava.convert.AbstractDelegateConverter.convert(AbstractDelegateConverter.java:25)
	at org.scijava.convert.AbstractConvertService.convert(AbstractConvertService.java:128)
	at net.imagej.legacy.plugin.MacroPreprocessor.process(MacroPreprocessor.java:82)
	at org.scijava.module.ModuleRunner.preProcess(ModuleRunner.java:105)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:157)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: io.scif.img.ImgIOException: java.io.IOException: io.scif.FormatException: Invalid PNG signature.
	at io.scif.img.ImgOpener.createReader(ImgOpener.java:489)
	at io.scif.img.ImgOpener.openImgs(ImgOpener.java:242)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:152)
	... 16 more
Caused by: java.io.IOException: io.scif.FormatException: Invalid PNG signature.
	at io.scif.AbstractReader.setSource(AbstractReader.java:279)
	at io.scif.services.DefaultInitializeService.initializeReader(DefaultInitializeService.java:91)
	at io.scif.img.ImgOpener.createReader(ImgOpener.java:483)
	... 18 more
Caused by: io.scif.FormatException: Invalid PNG signature.
	at io.scif.formats.APNGFormat$Parser.typedParse(APNGFormat.java:408)
	at io.scif.formats.APNGFormat$Parser.typedParse(APNGFormat.java:391)
	at io.scif.AbstractParser.parse(AbstractParser.java:244)
	at io.scif.AbstractParser.parse(AbstractParser.java:314)
	at io.scif.AbstractParser.parse(AbstractParser.java:53)
	at io.scif.AbstractReader.setSource(AbstractReader.java:271)
	... 20 more
[ERROR] Module threw exception
java.lang.NullPointerException
	at io.scif.services.DefaultInitializeService.initializeReader(DefaultInitializeService.java:87)
	at io.scif.img.ImgOpener.createReader(ImgOpener.java:483)
	at io.scif.img.ImgOpener.openImgs(ImgOpener.java:242)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:152)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:133)
	at io.scif.services.DefaultDatasetIOService.open(DefaultDatasetIOService.java:138)
	at io.scif.convert.FileToDatasetConverter.convert(FileToDatasetConverter.java:66)
	at org.scijava.convert.AbstractConvertService.convert(AbstractConvertService.java:128)
	at org.scijava.convert.AbstractDelegateConverter.convert(AbstractDelegateConverter.java:25)
	at org.scijava.convert.AbstractConvertService.convert(AbstractConvertService.java:128)
	at net.imagej.legacy.plugin.MacroPreprocessor.process(MacroPreprocessor.java:82)
	at org.scijava.module.ModuleRunner.preProcess(ModuleRunner.java:105)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:157)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
1 Like

Oh yes, sorry I forgot that the use case here is about already opened images. That’s of course a general issue here: by using a String in an option parameter of a macro command, how do you expect it to be treated? Should it only use open images? Should it try to open a given file path?

I guess you can argue it should first look if there’s a matching image title among the open images, and then try to open an image from disk. Or, alternatively, check if the String is a valid file path… there’s just a lot of valid options to support.


EDIT: I opened an issue here:

2 Likes

So, if I understand correctly, currently for ij1 macro:

  • if an ImagePlus object has a Parameter annotation, ij will look whether an open image matches the string given in the macro.

  • if a DataSet object has a Parameter annotation, the string in the macro is ignored, thus a window is popping asking the user which image should be selected.

@k-dominik for ij4ilastik, what’s the use case, do you expect the Dataset objects to be already opened images or are they files ?

1 Like

@imagejan @k-dominik

I believe a correct solution at the moment if to add this converter in the ilastik4ij repository:

package org.debugdemo.debug.ui;

import ij.ImagePlus;
import net.imagej.Dataset;
import org.scijava.Priority;
import org.scijava.convert.AbstractConverter;
import org.scijava.convert.ConvertService;
import org.scijava.plugin.Parameter;
import org.scijava.plugin.Plugin;

@Plugin(type = org.scijava.convert.Converter.class, priority = Priority.EXTREMELY_HIGH) // priority in order to bypass scifio converter to come...
public class StringToDatasetConverter extends AbstractConverter<String, Dataset> {

    @Parameter
    ConvertService cs;

    @Override
    public <T> T convert(Object src, Class<T> dest) {
        assert src instanceof String;
        String name = (String) src;
        // First conversion : String to ImagePlus
        ImagePlus imagePlus = cs.convert(name, ImagePlus.class);
        if (imagePlus == null) return null;
        // Second convestion : ImagePlus to Dataset
        return (T) cs.convert(imagePlus, Dataset.class);
    }

    @Override
    public Class<Dataset> getOutputType() {
        return Dataset.class;
    }

    @Override
    public Class<String> getInputType() {
        return String.class;
    }
}

1 Like

@k-dominik

You can check the workaround with the extra converter in https://github.com/NicoKiaru/minimal-fiji-plugin-issue

1 Like

In order to avoid code duplication, I strongly recommend:

  1. to extend AbstractDelegateConverter as in the StringToDatasetConverter example in my PR linked above, so something like this:
@Plugin(type = Converter.class, priority = Priority.VERY_HIGH) // don't exaggerate priorities!
public class StringToDatasetConverter extends
	AbstractDelegateConverter<String, ImagePlus, Dataset>
{

	@Override
	public Class<Dataset> getOutputType() {
		return Dataset.class;
	}

	@Override
	public Class<String> getInputType() {
		return String.class;
	}

	@Override
	protected Class<ImagePlus> getDelegateType() {
		return ImagePlus.class;
	}
}

and

  1. to contribute this to imagej-legacy instead of maintaining your own solutions (even if it is meant to be just “temporary”)
2 Likes

Temporary is not the right word, it’s more ‘should be working easily next week because of a conference’ :grin:. So a solution at the update site level is doable, while updating imagej-legacy and make it reach the normal install is involving too many people I guess.

I’ll continue the discussion on the issue

By the way:

You shouldn’t use assert in production code, as the assert statement will not be executed by default. See:

https://docs.oracle.com/javase/specs/jls/se8/html/jls-14.html#jls-14.10

1 Like

Hi @imagejan, @NicoKiaru thank you so much for looking into this!! This is really awesome.

as for the question:

So far the use case has been to already have opened images. So in macros there’s always this code to first load them, usually name them and then pass them to run… I guess I would keep it this way as the main point of the ilastik plugin is to be able to integrate in your fiji/ij pipeline.

2 Likes

Well, you can always upload a SNAPSHOT version of imagej-legacy to your own update site, right? That’s the way how @haesleinhuepf is doing it for the CLIJ update sites.

Having it in imagej-legacy from the beginning would save us from more troubles in the future (in my opinion): when others rely on a different converter in the ecosystem, but yours has higher priority, then your update site will effectively break other update sites (potentially). As long as we don’t have a super elegant dependency management across different update sites, I think it’s a bad idea.

I agree that a conference or a teaching event is a high motivation to get thing to work for a large user base. But in particular for the case of teaching: what you’ll teach here will be the expected behavior for users, and it might be difficult to change how things work after the fact.
But probably I sound a bit pessimistic here, it’s not such a great deal after all.

1 Like

I really think everything will be transparent for users, because the behaviour we put is the behaviour wanted on the long term (and which was broken until now).

Also, at the moment, I’m not sure that we agree (yet) on what should be done at the imagej-legacy level (https://github.com/imagej/imagej-legacy/issues/246). So even if we agree on WHAT should work - we are not settled on HOW to do it on the long term, so making a SNAPSHOT of imagej-legacy is not ideal I believe (and it will still break other update sites, doesn’t it ?).

Now let’s look at the bright side : I was doing my own wrapper for ilastik and now I’m (trying) to contribute to the proper ilastik one - it’s already removing quite a lot of code duplication :grimacing:.

4 Likes