Ilastik macro ImageJ

Dear all,
I was playing with Ilastik software to create a pixel segmentation project. In this project I train Ilastik to recognice: cells, cell debris, nucleus and background.

Now I would like to load in ImageJ my 353 slides timelapse and run the Ilastik project to get a mask of my cell debris only.

First, when I run pixel segmentation ilastik plugin in ImageJ an Exception message appears saying that HDF5 file is not found. I do not why ImageJ insists in saving the HDF5 files in TEMP folder. I even wrote a macro to call the plugin, and indicate to save HDF5 files in another folder and still ImageJ keeps looking for the file in TEMP folder. Misteries of ImageJ!

So can anyone help me to write a macro to get the death cell debris mask of my 353 frames using the Ilastik project I created?

Thanks
Miguel.

Hey @mvizoso,

would you mind sharing the macro you executed and the exact error message that came up? That makes it easier for us to help you :wink:

Cheers,
Robert

Dear @haesleinhuepf,
good point, here is the code:

// set global variables
dir1 = "C:/Users/Sapito/Desktop/Ilastik/Project 20200523.ilp";
outputType = "Segmentation"; //  "Probabilities" or "Segmentation"
outputDataset = "exported_data";
axisOrder = "tzyxc";
compressionLevel = 0;
dir2="C:/Users/Sapito/Desktop/Ilastik/";

	// run pixel classification
	d="Pos702 deadcellmask.tif";	
	inputImage = d;
	pixelClassificationArgs = "projectfilename=" + dir1 + " saveonly=false inputimage=" + inputImage + " pixelclassificationtype=" + outputType;
	run("Run Pixel Classification Prediction", pixelClassificationArgs);

	// export segmentation to H5
	outputFile = dir2 + "output" + ".h5";
	exportArgs = "select=" + outputFile + " datasetname=" + outputDataset + " compressionlevel=" + compressionLevel;
	run("Export HDF5", exportArgs);

And I got this error code:

java.lang.RuntimeException: java.util.concurrent.ExecutionException: ncsa.hdf.hdf5lib.exceptions.HDF5FileNotFoundException: Path does not exit. (C:\Users\Sapito\AppData\Local\Temp\ilastik4j7753005737859870235_out.h5)

I can confirm this issue. I just tried to run a random ilastik Pixel classifier from the menu and get the same error:

(Fiji Is Just) ImageJ 2.0.0-rc-72/1.52p; Java 1.8.0_172 [64-bit]; Windows 10 10.0; 96MB of 12033MB (<1%)
 
java.lang.RuntimeException: java.util.concurrent.ExecutionException: ncsa.hdf.hdf5lib.exceptions.HDF5FileNotFoundException: Path does not exit. (C:\Users\rober\AppData\Local\Temp\ilastik4j6940197260019095592_out.h5)
	at net.imagej.legacy.LegacyService.runLegacyCompatibleCommand(LegacyService.java:307)
	at net.imagej.legacy.DefaultLegacyHooks.interceptRunPlugIn(DefaultLegacyHooks.java:166)
	at ij.IJ.runPlugIn(IJ.java)
	at ij.Executer.runCommand(Executer.java:137)
	at ij.Executer.run(Executer.java:63)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: ncsa.hdf.hdf5lib.exceptions.HDF5FileNotFoundException: Path does not exit. (C:\Users\rober\AppData\Local\Temp\ilastik4j6940197260019095592_out.h5)
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at net.imagej.legacy.LegacyService.runLegacyCompatibleCommand(LegacyService.java:303)
	... 5 more
Caused by: ncsa.hdf.hdf5lib.exceptions.HDF5FileNotFoundException: Path does not exit. (C:\Users\rober\AppData\Local\Temp\ilastik4j6940197260019095592_out.h5)
	at ch.systemsx.cisd.hdf5.HDF5BaseReader.openFile(HDF5BaseReader.java:194)
	at ch.systemsx.cisd.hdf5.HDF5BaseReader.<init>(HDF5BaseReader.java:147)
	at ch.systemsx.cisd.hdf5.HDF5BaseReader.<init>(HDF5BaseReader.java:126)
	at ch.systemsx.cisd.hdf5.HDF5ReaderConfigurator.reader(HDF5ReaderConfigurator.java:86)
	at ch.systemsx.cisd.hdf5.HDF5FactoryProvider$HDF5Factory.openForReading(HDF5FactoryProvider.java:54)
	at ch.systemsx.cisd.hdf5.HDF5Factory.openForReading(HDF5Factory.java:64)
	at org.ilastik.ilastik4ij.hdf5.Hdf5DataSetReader.read(Hdf5DataSetReader.java:51)
	at org.ilastik.ilastik4ij.executors.AbstractIlastikExecutor.executeIlastik(AbstractIlastikExecutor.java:64)
	at org.ilastik.ilastik4ij.executors.PixelClassification.classifyPixels(PixelClassification.java:24)
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.runClassification(IlastikPixelClassificationCommand.java:72)
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.run(IlastikPixelClassificationCommand.java:60)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more

I can also confirm that the file indeed doesn’t exist.

Furthermore, in the Fiji console, I read this:

[INFO] Temporary files: {tempFileRawInput=C:\Users\rober\AppData\Local\Temp\ilastik4j6797903815800985269_in_raw.h5, tempFileOutput=C:\Users\rober\AppData\Local\Temp\ilastik4j6940197260019095592_out.h5}
[INFO] Exporting image of shape (1, 1, 254, 256, 1). Axis order: 'TZYXC'
[INFO] Saving as 'uint8'. Compression level: 1
[INFO] Running ilastik headless command:
[INFO] [C:\Program Files\ilastik-1.3.3post3\ilastik.exe, --headless, --project=C:\structure\teaching\lecture_applied_bioimage_analysis_2020\08_Machine_Learning\exercise_solution\ilastik_project\rounding_assay.ilp, --output_filename_format=C:\Users\rober\AppData\Local\Temp\ilastik4j6940197260019095592_out.h5, --output_format=hdf5, --output_axis_order=tzyxc, --export_source=Simple Segmentation, --input_axes=tzyxc, C:\Users\rober\AppData\Local\Temp\ilastik4j6797903815800985269_in_raw.h5]
[ERROR] C:\Program Files\ilastik-1.3.3post3\ilastik-meta\lazyflow\lazyflow\classifiers\__init__.py:20: UserWarning: init: Could not import tiktorch classifier
[ERROR]   warnings.warn("init: Could not import tiktorch classifier")
[INFO] INFO ilastik_main: Starting ilastik from "C:\Program Files\ilastik-1.3.3post3".
[INFO] Starting ilastik from "C:\Program Files\ilastik-1.3.3post3".
[INFO] INFO ilastik_main: Configuring lazyflow RAM limit to 4.0GiB
[INFO] INFO lazyflow.utility.memory: Available memory set to 4.0GiB
[ERROR] WARNING 2020-05-24 11:11:15,773 opConservationTracking 30628 28540 Could not find any ILP solver
[ERROR] WARNING 2020-05-24 11:11:15,839 opStructuredTracking 30628 28540 Could not find any ILP solver
[ERROR] WARNING 2020-05-24 11:11:15,845 structuredTrackingWorkflow 30628 28540 Could not find any learning solver. Tracking will use flow-based solver (DPCT). Learning for tracking will be disabled!
[INFO] INFO ilastik.shell.projectManager: Opening Project: C:/structure/teaching/lecture_applied_bioimage_analysis_2020/08_Machine_Learning/exercise_solution/ilastik_project/rounding_assay.ilp
[ERROR] WARNING dataset.py(313): H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead.
[ERROR] WARNING opSimpleBlockedArrayCache.py(43): FutureWarning: Conversion of the second argument of issubdtype from `dtype` to `np.generic` is deprecated. In future, it will be treated as `np.object_ == np.dtype(dtype).type`.
[ERROR] WARNING opSlicedBlockedArrayCache.py(141): FutureWarning: Conversion of the second argument of issubdtype from `dtype` to `np.generic` is deprecated. In future, it will be treated as `np.object_ == np.dtype(dtype).type`.
[ERROR] ERROR 2020-05-24 11:11:53,695 log_exception 30628 28540 Traceback (most recent call last):
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik\applets\dataExport\dataExportApplet.py", line 284, in _configure_operator_with_parsed_args
[ERROR]     source_index = source_choices.index(export_source)
[ERROR] ValueError: 'simple segmentation' is not in list
[ERROR] 
[ERROR] During handling of the above exception, another exception occurred:
[ERROR] 
[ERROR] Traceback (most recent call last):
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik\shell\projectManager.py", line 467, in _loadProject
[ERROR]     self.workflow.onProjectLoaded(self)
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik\workflows\objectClassification\objectClassificationWorkflow.py", line 340, in onProjectLoaded
[ERROR]     self.dataExportApplet.configure_operator_with_parsed_args(self._batch_export_args)
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik\applets\dataExport\dataExportApplet.py", line 261, in configure_operator_with_parsed_args
[ERROR]     self._configure_operator_with_parsed_args(parsed_args, opDataExport)
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik\applets\dataExport\dataExportApplet.py", line 288, in _configure_operator_with_parsed_args
[ERROR]     "Valid options are: {}".format(parsed_args.export_source, source_choices)
[ERROR] Exception: Invalid option for --export_source: 'Simple Segmentation'
[ERROR] Valid options are: ['object predictions', 'object probabilities', 'blockwise object predictions', 'blockwise object probabilities', 'object identities', 'pixel probabilities']
[ERROR] 
[ERROR] ERROR 2020-05-24 11:11:53,696 log_exception 30628 28540 Project could not be loaded due to the exception shown above.
[ERROR] Aborting Project Open Action
[ERROR] ERROR 2020-05-24 11:11:59,258 excepthooks 30628 28540 Unhandled exception in thread: 'MainThread'
[ERROR] ERROR 2020-05-24 11:11:59,292 excepthooks 30628 28540 Traceback (most recent call last):
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik\applets\dataExport\dataExportApplet.py", line 284, in _configure_operator_with_parsed_args
[ERROR]     source_index = source_choices.index(export_source)
[ERROR] ValueError: 'simple segmentation' is not in list
[ERROR] 
[ERROR] During handling of the above exception, another exception occurred:
[ERROR] 
[ERROR] Traceback (most recent call last):
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik.py", line 134, in <module>
[ERROR]     main()
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik.py", line 128, in main
[ERROR]     hShell = ilastik_main.main(parsed_args, workflow_cmdline_args)
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik_main.py", line 170, in main
[ERROR]     f(shell)
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik_main.py", line 347, in loadProject
[ERROR]     shell.openProjectFile(path, parsed_args.readonly)
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik\shell\headless\headlessShell.py", line 125, in openProjectFile
[ERROR]     self.projectManager._loadProject(hdf5File, projectFilePath, readOnly)
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\lazyflow\lazyflow\utility\timer.py", line 153, in wrapper
[ERROR]     return func(*args, **kwargs)
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik\shell\projectManager.py", line 467, in _loadProject
[ERROR]     self.workflow.onProjectLoaded(self)
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik\workflows\objectClassification\objectClassificationWorkflow.py", line 340, in onProjectLoaded
[ERROR]     self.dataExportApplet.configure_operator_with_parsed_args(self._batch_export_args)
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik\applets\dataExport\dataExportApplet.py", line 261, in configure_operator_with_parsed_args
[ERROR]     self._configure_operator_with_parsed_args(parsed_args, opDataExport)
[ERROR]   File "C:\Program Files\ilastik-1.3.3post3\ilastik-meta\ilastik\ilastik\applets\dataExport\dataExportApplet.py", line 288, in _configure_operator_with_parsed_args
[ERROR]     "Valid options are: {}".format(parsed_args.export_source, source_choices)
[ERROR] Exception: Invalid option for --export_source: 'Simple Segmentation'
[ERROR] Valid options are: ['object predictions', 'object probabilities', 'blockwise object predictions', 'blockwise object probabilities', 'object identities', 'pixel probabilities']
[ERROR] 
[ERROR] ilastik error: 
[ERROR] The operation completed successfully.
[ERROR] 
[ERROR] 
[INFO] ilastik execution finished successfully!
[INFO] Deleted tmp file: C:\Users\rober\AppData\Local\Temp\ilastik4j6797903815800985269_in_raw.h5
[ERROR] Module threw exception
ncsa.hdf.hdf5lib.exceptions.HDF5FileNotFoundException: Path does not exit. (C:\Users\rober\AppData\Local\Temp\ilastik4j6940197260019095592_out.h5)
	at ch.systemsx.cisd.hdf5.HDF5BaseReader.openFile(HDF5BaseReader.java:194)
	at ch.systemsx.cisd.hdf5.HDF5BaseReader.<init>(HDF5BaseReader.java:147)
	at ch.systemsx.cisd.hdf5.HDF5BaseReader.<init>(HDF5BaseReader.java:126)
	at ch.systemsx.cisd.hdf5.HDF5ReaderConfigurator.reader(HDF5ReaderConfigurator.java:86)
	at ch.systemsx.cisd.hdf5.HDF5FactoryProvider$HDF5Factory.openForReading(HDF5FactoryProvider.java:54)
	at ch.systemsx.cisd.hdf5.HDF5Factory.openForReading(HDF5Factory.java:64)
	at org.ilastik.ilastik4ij.hdf5.Hdf5DataSetReader.read(Hdf5DataSetReader.java:51)
	at org.ilastik.ilastik4ij.executors.AbstractIlastikExecutor.executeIlastik(AbstractIlastikExecutor.java:64)
	at org.ilastik.ilastik4ij.executors.PixelClassification.classifyPixels(PixelClassification.java:24)
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.runClassification(IlastikPixelClassificationCommand.java:72)
	at org.ilastik.ilastik4ij.ui.IlastikPixelClassificationCommand.run(IlastikPixelClassificationCommand.java:60)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Maybe @k-dominik can help (during office hours)?

Cheers,
Robert

Thanks, it is indeed super anoying. Let´s see if someone can help with that.
Best,
Miguel

Just give them some time. It’s Sunday :wink: I’m sure it’s not a big deal :slight_smile: