Ilastik and the mysterious .h5 file creation

When I attempt to use Ilastik on a MacOS by going to Plugin > Ilastik > Export HDF5, it asks me where to save the .h5 file and then I have no idea how to open it. If I rename the file "ilastik.h5 and drag and drop it into Fiji, it doesn’t support it either. How to open something with a .h5 suffix in Mac OS?

Hi @Kristy_Wendt,

the plugin is primarily meant for exporting data from fiji to use in ilastik. So, you can open the generated h5 file in ilastik. ilastik is usually faster using this format as compared to e.g. tif. That’s the reason the plugin is there in the first place.

If you want to load it in fiji, use the import hdf5 entry from the same plugin.

Cheers

1 Like

Thanks @k-dominik! I’ve got a relatively small time series and ostensibly I’m using Ilastik to assist with segmentation for another plugin. I’m plating at low densities, with relatively high contrast, but the only thing that’s happening is conversion to an hd5 and some whacky initial processing.

So here are my cells that I’m wanting to look at morphology of the process extension in an automated way:

and here’s what ilastik does to them:

I wonder if my image is too simple for what this plugin is intended to do, or if there’s preprocessing that’s required for my preprocessing to get segmentation to work for morphology analysis.

Hello @Kristy_Wendt,

the second image - is it showing the result of pixel classification? Or what does it show? Where do the colors come from.

Your data looks like a wonderful candidate to be processed in ilastik, so …

Cheers
Dominik

Thanks @k-dominik. I just realized my beginner’s mistake. I did not type in “60” for my number of timepoints and "t"for temporal nature of stack. I do find that ilastik trainer doesn’t work out of the box in Fiji, but that I’m able to use it as a standalone after the download. I’m having trouble with my final export, so still not home yet, but closer. Going to send some error code.

Anyway, for those that are using temporal stacks, it’s well documented in ilastik that you have to put in the information for the correct number of timepoints in t and also let the prompt know that the axis is “t” and not “z.”

However, the ilastik pixel classification doesn’t seem to work in Fiji yet.

So when I ask ilastik to do pixel classification of the .h5 file I create in Fiji, it asks me for the trained ilastik project file. I don’t think there’s currently a way to do this in Fiji. Is that correct?

Here’s my code for when I try to run the pixel classification on the .h5 that I create in Fiji. I think it’s because it’s made to open something I’ve already trained in the standalone.

[INFO] Reading available sites from https://imagej.net/
[INFO] Temporary files: {tempFileRawInput=/var/folders/yw/lhbq1xkd34z4fwt_s9tmcmc40000gp/T/ilastik4j7363466176297058827_in_raw.h5, tempFileOutput=/var/folders/yw/lhbq1xkd34z4fwt_s9tmcmc40000gp/T/ilastik4j2337686873880760267_out.h5}
[INFO] Exporting image of shape (1, 60, 512, 512, 1). Axis order: 'TZYXC'
[INFO] Saving as 'uint8'. Compression level: 1
[INFO] Running ilastik headless command:
[INFO] [/opt/ilastik/run_ilastik.sh, --headless, --project=/Users/kwendt/Desktop/test2.h5, --output_filename_format=/var/folders/yw/lhbq1xkd34z4fwt_s9tmcmc40000gp/T/ilastik4j2337686873880760267_out.h5, --output_format=hdf5, --output_axis_order=tzyxc, --export_source=Simple Segmentation, /var/folders/yw/lhbq1xkd34z4fwt_s9tmcmc40000gp/T/ilastik4j7363466176297058827_in_raw.h5]
[INFO] Deleted tmp file: /var/folders/yw/lhbq1xkd34z4fwt_s9tmcmc40000gp/T/ilastik4j7363466176297058827_in_raw.h5
[ERROR] Pixel classification command failed
java.io.IOException: Cannot run program "/opt/ilastik/run_ilastik.sh": error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at org.ilastik.ilastik4ij.executors.AbstractIlastikExecutor.executeCommandLine(AbstractIlastikExecutor.java:118)
	at org.ilastik.ilastik4ij.executors.AbstractIlastikExecutor.executeIlastik(AbstractIlastikExecutor.java:61)
	at org.ilastik.ilastik4ij.executors.PixelClassification.classifyPixels(PixelClassification.java:24)
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.runClassification(IlastikPixelClassificationCommand.java:72)
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.run(IlastikPixelClassificationCommand.java:60)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 14 more
[ERROR] Module threw exception
java.lang.RuntimeException: java.io.IOException: Cannot run program "/opt/ilastik/run_ilastik.sh": error=2, No such file or directory
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.run(IlastikPixelClassificationCommand.java:63)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Cannot run program "/opt/ilastik/run_ilastik.sh": error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at org.ilastik.ilastik4ij.executors.AbstractIlastikExecutor.executeCommandLine(AbstractIlastikExecutor.java:118)
	at org.ilastik.ilastik4ij.executors.AbstractIlastikExecutor.executeIlastik(AbstractIlastikExecutor.java:61)
	at org.ilastik.ilastik4ij.executors.PixelClassification.classifyPixels(PixelClassification.java:24)
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.runClassification(IlastikPixelClassificationCommand.java:72)
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.run(IlastikPixelClassificationCommand.java:60)
	... 9 more
Caused by: java.io.IOException: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 14 more
[INFO] /t{60}: GROUP
[INFO] /t{60}/channel{1}: DATASET
[INFO] /t{60}: GROUP
[INFO] /t{60}/channel{1}: DATASET
[INFO] Done loading HDF5 file!
[INFO] /t{60}: GROUP
[INFO] /t{60}/channel{1}: DATASET
[INFO] Found dataset '/t{60}/channel{1}' of type 'uint8'
[INFO] Constructing output image of shape (512, 512, 1, 1, 60). Axis order: 'XYCZT'
[INFO] Loading HDF5 dataset took: 1363
[INFO] Done loading HDF5 file!
[INFO] Temporary files: {tempFileRawInput=/var/folders/yw/lhbq1xkd34z4fwt_s9tmcmc40000gp/T/ilastik4j2501973541918098285_in_raw.h5, tempFileOutput=/var/folders/yw/lhbq1xkd34z4fwt_s9tmcmc40000gp/T/ilastik4j8119844611818010638_out.h5}
[INFO] Exporting image of shape (60, 1, 512, 512, 1). Axis order: 'TZYXC'
[INFO] Saving as 'uint8'. Compression level: 1
[INFO] Running ilastik headless command:
[INFO] [/opt/ilastik/run_ilastik.sh, --headless, --project=/Users/kwendt/Desktop/test2.h5, --output_filename_format=/var/folders/yw/lhbq1xkd34z4fwt_s9tmcmc40000gp/T/ilastik4j8119844611818010638_out.h5, --output_format=hdf5, --output_axis_order=tzyxc, --export_source=Simple Segmentation, /var/folders/yw/lhbq1xkd34z4fwt_s9tmcmc40000gp/T/ilastik4j2501973541918098285_in_raw.h5]
[INFO] Deleted tmp file: /var/folders/yw/lhbq1xkd34z4fwt_s9tmcmc40000gp/T/ilastik4j2501973541918098285_in_raw.h5
[ERROR] Pixel classification command failed
java.io.IOException: Cannot run program "/opt/ilastik/run_ilastik.sh": error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at org.ilastik.ilastik4ij.executors.AbstractIlastikExecutor.executeCommandLine(AbstractIlastikExecutor.java:118)
	at org.ilastik.ilastik4ij.executors.AbstractIlastikExecutor.executeIlastik(AbstractIlastikExecutor.java:61)
	at org.ilastik.ilastik4ij.executors.PixelClassification.classifyPixels(PixelClassification.java:24)
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.runClassification(IlastikPixelClassificationCommand.java:72)
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.run(IlastikPixelClassificationCommand.java:60)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 14 more
[ERROR] Module threw exception
java.lang.RuntimeException: java.io.IOException: Cannot run program "/opt/ilastik/run_ilastik.sh": error=2, No such file or directory
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.run(IlastikPixelClassificationCommand.java:63)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Cannot run program "/opt/ilastik/run_ilastik.sh": error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at org.ilastik.ilastik4ij.executors.AbstractIlastikExecutor.executeCommandLine(AbstractIlastikExecutor.java:118)
	at org.ilastik.ilastik4ij.executors.AbstractIlastikExecutor.executeIlastik(AbstractIlastikExecutor.java:61)
	at org.ilastik.ilastik4ij.executors.PixelClassification.classifyPixels(PixelClassification.java:24)
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.runClassification(IlastikPixelClassificationCommand.java:72)
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.run(IlastikPixelClassificationCommand.java:60)
	... 9 more
Caused by: java.io.IOException: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 14 more

Hi @Kristy_Wendt,

The basic workflow would be, exporting

  1. do whatever pre-processing steps you have in fiji
  2. export the image you want to train the ilastik classifier to hdf5 using the plugin
  3. this is where you have to leave fiji: open ilastik, start a new (pixel classification) project (you’ll be asked to save the project right when you create it, so put it somewhere you can find it again). Then you train the classifier (see this youtube video for a quick tutorial). So yeah, you train, train, train until you’re satisfied (don’t forget to save in the end).
  4. Then you can go back to fiji, open a different image, do the same pre-processing (if any) and then use the plugin to run the pixel classifier that you have trained in the previous step. So you’d want to put in the .ilp file in there.

Please let me know if this gets you going!

Cheers :slight_smile:

1 Like