Object Classification using the ilastik plugin for Fiji

Dear all,
I’m trying to write a Fiji macro to perform a pixel classification followed by an object classification with the ilastik plugin. I have been able to include the pixel classification following the example provided in the README for the ilastik’s Fiji plugin:

However, I have found no example for the object classification, and the Fiji recorder just returns this:

run(“Run Object Classification Prediction”);

Could someone please tell me how to use as inputs i) the trained ilastik project, ii) the raw image and iii) the probabilities map? (using the ImageJ macro language). Thank you very much in advance.

Hi @pau,

I’ve seen @Christian_Tischer doing this recently, could you maybe share your insights?

Cheers
Dominik

In fact, I just worked on the Java bridge from ImageJ to ilastik, but did not test the macro recording.
@wolny, did you try this recently?

Hi @pau,

simple macro recording won’t work for object classification. Same goes for pixel classification, that’s why we provided an example in our github repo. I’ll update the README with a sample macro for pixel classification followed by object classification in a spare moment. If you feel adventurous in the meantime, you may try adapting the following java example into a Fiji macro: https://github.com/ilastik/ilastik4ij/blob/master/src/test/java/org/ilastik/ilastik4ij/ObjectClassificationDemo.java

Best,
Adrian

1 Like

Hi @k-dominik @Christian_Tischer,
Thank you for your answers.

I’m using ilastik to process images of axons (A, raw image). I use the pixel classification to segment the axoplasm (B, blue). Then I classify objects as axoplasm (C, blue-yellow) and other stuff.

Since I want to continue batch processing both the probabilities map and the objects, I tried to include the ilstik steps within the macro script. I have done this for now:

//script parameters
#@ File(label="Directory", style="directory") dir
#@ File(label="Pixel Classification", description="Enter an ilastik project (ilp) file", style="extensions:ilp") projectPixel
#@ File(label="Object Classification", description="Enter an ilastik project (ilp) file", style="extensions:ilp") projectObject

//check if folder contains h5 files
dirName=File.getName(dir);
list=getFileList(dir);
count=0;
for (i=0; i<list.length; i++) {
	if (endsWith(list[i], ".h5")==true) {
		count++;
	}
}

//error message
if (count==0) {
	exit("No h5 files found in "+dirName);
}

//pixel-object classification and h5 export
for (i=0; i<list.length; i++) {
	if (endsWith(list[i], ".h5")) {
		fileNameWithFormat=File.getName(list[i]);
		indexFormat=indexOf(fileNameWithFormat, ".h5");
		fileName=substring(fileNameWithFormat, 0, indexFormat);
		run("Import HDF5", "select=["+dir+File.separator+list[i]+"] datasetname=[/data] axisorder=tzyxc");
		rename(fileName);
		run("Run Pixel Classification Prediction", "projectfilename=["+projectPixel+"] inputimage=["+fileName+"] pixelclassificationtype=Probabilities");
		rename("Probabilities");
		run("Export HDF5", "select=[Probabilities] exportpath=["+dir+File.separator+fileName+"_Probabilities.h5] datasetname=data compressionlevel=0 input=Probabilities");

		//to continue with object clasification...
	}
}

But I got stuck with the object classification. Hopefully I’ll be able to continue with @wolny 's help. Thank you very much.

Cheers
Pau

Hi @wolny,

Thank you very much! Pixel classification actually works with macro recording. I mean, not just copy-pasting, but I get the parameters so I can just add some variables to make it work.

However, I don’t get the parameters for the object classification when recording, so the command will stop the batch processing.

Do you see what I mean? I’m not a programmer, obviously :sweat_smile:
I’ll try to follow the java example. Anyway, I would really appreciate if you could let me know when you update the README with a sample macro. Thanks again.

Best,
Pau

Hi @Christian_Tischer. Apologies for bothering you with this again.
I have seen that now the macro recording for the object classification has been fixed. Thank you very much for that. With the recorder help I have been able to continue my macro. However, when Fiji runs the object classification, the macro stops (precluding batch processing) and I get this dialog box:

Since I am providing the information requested by the dialog box, I do not understand why the macro stops. I would really appreciate if you could bear a hand with this issue. I am writting my script below:

//script parameters
#@ File(label="Directory", style="directory") dir
#@ File(label="Pixel Classification", description="Enter an ilastik project (ilp) file", style="extensions:ilp") projectPixel
#@ File(label="Object Classification", description="Enter an ilastik project (ilp) file", style="extensions:ilp") projectObject

//check if folder contains h5 files
dirName=File.getName(dir);
list=getFileList(dir);
count=0;
for (i=0; i<list.length; i++) {
	if (endsWith(list[i], ".h5")==true) {
		count++;
	}
}

//error message
if (count==0) {
	exit("No h5 files found in "+dirName);
}

//pixel-object classification and h5 export
for (i=0; i<list.length; i++) {
	if (endsWith(list[i], ".h5")) {
		fileNameWithFormat=File.getName(list[i]);
		indexFormat=indexOf(fileNameWithFormat, ".h5");
		fileName=substring(fileNameWithFormat, 0, indexFormat);
		run("Import HDF5", "select=["+dir+File.separator+list[i]+"] datasetname=[/data] axisorder=tzyxc");
		rename(fileName);
		run("Run Pixel Classification Prediction", "projectfilename=["+projectPixel+"] inputimage=["+fileName+"] pixelclassificationtype=Probabilities");
		probabilities=getTitle();
		run("Export HDF5", "select=[Probabilities] exportpath=["+dir+File.separator+fileName+"_Probabilities.h5] datasetname=data compressionlevel=0 input=Probabilities");
		run("Run Object Classification Prediction", "projectfilename=["+projectObject+"] inputimage=["+fileName+"] inputproborsegimage=["+probabilities+"] secondinputtype=Probabilities");
		saveAs("tiff", dir+File.separator+fileName+"_Object Predictions");
		run("Close All");
	}
}

Best

@k-dominik, could you have a look?
I can also have a look but earliest have time end of next week.

Take care with the case of the parameter. You can have a look at https://github.com/ilastik/ilastik4ij/blob/fe7bd82fba3e02a41f2c1e9747e28d85346c5e8a/src/main/java/org/ilastik/ilastik4ij/ui/IlastikObjectClassificationCommand.java

The parameters are:

  • projectFileName -> not projectfilename
  • inputImage -> not inputimage

and so on…

Thank you for the suggestion. I have tried to do as you say:

run("Run Object Classification Prediction", "projectFileName=["+projectObject+"] inputImage=["+fileName+"] inputProbOrSegImage=["+probabilities+"] secondInputType=Probabilities");

However, the macro still stops and I get a dialog box asking for even more info:


(Before it only asked for Raw input image and Pixel Probability or Segmentation Image)

So I guess parameters are well recorded (without case letters). I would say that the issue is somehow related to the probability map name generated by the pixel classification. But I have no idea :sweat_smile:

1 Like

Hi I can reproduce this - staring at the code now to find the issue…

3 Likes

So here’s the diff between the current PixelClassification ui and the Object Classification ui:

--- src/main/java/org/ilastik/ilastik4ij/ui/IlastikPixelClassificationCommand.java	2020-03-01 15:36:28.358359688 +0100
+++ src/main/java/org/ilastik/ilastik4ij/ui/IlastikObjectClassificationCommand.java	2020-04-06 14:45:44.033165903 +0200
@@ -3,7 +3,7 @@
 import net.imagej.Dataset;
 import net.imagej.ImgPlus;
 import net.imglib2.type.NativeType;
-import org.ilastik.ilastik4ij.executors.PixelClassification;
+import org.ilastik.ilastik4ij.executors.ObjectClassification;
 import org.scijava.ItemIO;
 import org.scijava.app.StatusService;
 import org.scijava.command.Command;
@@ -18,8 +18,8 @@
 
 import static org.ilastik.ilastik4ij.executors.AbstractIlastikExecutor.PixelPredictionType;
 
-@Plugin(type = Command.class, headless = true, menuPath = "Plugins>ilastik>Run Pixel Classification Prediction")
-public class IlastikPixelClassificationCommand implements Command {
+@Plugin(type = Command.class, headless = true, menuPath = "Plugins>ilastik>Run Object Classification Prediction")
+public class IlastikObjectClassificationCommand implements Command {
 
     @Parameter
     public LogService logService;
@@ -39,8 +39,11 @@
     @Parameter(label = "Raw input image")
     public Dataset inputImage;
 
-    @Parameter(label = "Output type", choices = {UiConstants.PIXEL_PREDICTION_TYPE_PROBABILITIES, UiConstants.PIXEL_PREDICTION_TYPE_SEGMENTATION}, style = "radioButtonHorizontal")
-    public String pixelClassificationType;
+    @Parameter(label = "Pixel Probability or Segmentation image")
+    public Dataset inputProbOrSegImage;
+
+    @Parameter(label = "Second Input Type", choices = {UiConstants.PIXEL_PREDICTION_TYPE_PROBABILITIES, UiConstants.PIXEL_PREDICTION_TYPE_SEGMENTATION}, style = "radioButtonHorizontal")
+    public String secondInputType = UiConstants.PIXEL_PREDICTION_TYPE_PROBABILITIES;
 
     @Parameter(type = ItemIO.OUTPUT)
     private ImgPlus<? extends NativeType<?>> predictions;
@@ -59,17 +62,17 @@
         try {
             runClassification();
         } catch (IOException e) {
-            logService.error("Pixel classification command failed", e);
+            logService.error("Object classification command failed", e);
             throw new RuntimeException(e);
         }
     }
 
     private void runClassification() throws IOException {
-        final PixelClassification pixelClassification = new PixelClassification(ilastikOptions.getExecutableFile(),
-                projectFileName, logService, statusService, ilastikOptions.getNumThreads(), ilastikOptions.getMaxRamMb());
+        final ObjectClassification objectClassification = new ObjectClassification(ilastikOptions.getExecutableFile(), projectFileName, logService, statusService, ilastikOptions.getNumThreads(), ilastikOptions.getMaxRamMb());
+
+        final PixelPredictionType secondInputImageType = PixelPredictionType.valueOf(secondInputType);
 
-        PixelPredictionType pixelPredictionType = PixelPredictionType.valueOf(pixelClassificationType);
-        this.predictions = pixelClassification.classifyPixels(inputImage.getImgPlus(), pixelPredictionType);
+        this.predictions = objectClassification.classifyObjects(inputImage.getImgPlus(), inputProbOrSegImage.getImgPlus(), secondInputImageType);
 
         // DisplayUtils.showOutput(uiService, predictions);
     }

So I really cannot find the issue. The only really cannot spot the problem. The only real difference is seems to be that there are to parameters of type DataSet in Object Classification as to only one in Pixel Classification…

Are there any plugin experts around that might be able to have an answer, @Christian_Tischer, @imagejan?

My running hypothesis is the following:

the run command actually never sets that variable (inputImage in case of Pixel Classification), but the currently active image is magically passed. So in pixel classification it works by accident (even if this variable is recorded correctly) because there is only one DataSet type variable, but in Object Classification there are two…

2 Likes

Hi,
I am also very interested in using the ObjectClassification in a macro and have the same problem. But I also found a problem when I start the plugin manually.

If I open new images, a raw and a binary image, and start the ilastik>Run Object Classification Prediction for the first time, the plugin gives me the following options for the selection of the raw input image:

The selection for the segmentation image shows the normal image names. If I start the plugin, I get an exception. However, if I run it a second time (even if I just cancel it after the first call), it gives me the normal image names for both the raw and the segmentation image and works.

If I run it in my macro, it always shows the dialogue with the img selection. Don´t know if this is something complete different or has also something to do with the other error.
Best,
Thomas

1 Like

Hi @T-Zobel, thanks for the report. I haven’t seen this particular issue in the wild yet. Can you reproduce it consistently, so every time you open it for the first time?

Hi @k-dominik,
yes, yesterday I could reproduce it consistently. Even after a restart and with different Images. I will try later a new Fiji installation and check if I will get the same problem. I could also ask for the image ID and change the variable to this behavior, just to check if it is working then.
I will write you today late again, as I don’t have much time during the day.

Maybe one more quick question :wink:
Is it also possible with the Fiji plugin to get the tables as .csv values?
Thanks a lot for the quick reply!

@pau Sorry for hijacking your thread.

Best,
Thomas

1 Like

No problem @T-Zobel. By the way, I cannot reproduce your issue using the macro manually. It works fine within my Fiji. However, I’m doing some stuff differently: 1) I’m using HDF5 files as imput, previously imported with the Import HDF5 plugin; 2) I’m using Probability Maps as second input.

Another difference. I get the entire path of each file as filename (as a consequence of using the Import HDF5 plugin, I guess…). However, it still happens even if I rename the imported images to have a shorter filename.

Just in case some of this helps…

Hi,
I get the same error if I import h5 files (then I get the entire path of the file). I have test it with a new downloaded and updated Fiji and ilastik plugin. The images were created from Zeiss .czi files (split channels, max intensity projection and an otsu threshold for the segmentation image, then export as h5 via ilastik plugin). I hope I can try on the weekend another computer and check if I will get the same error. If so, I can share the original files.

Edit: And it is still the same. If I open the plugin the first time after I opened new images, I get the error. After this it is working. If I open new images I get the error again.

Happy easter and stay healthy,
Thomas

1 Like