Ilastik headless syntax for wildcarding multiple files (inside multiple directories)

I’m trying to batch process images using ilastik-1.4.0b1 in headless mode from Windows (server 2012). I can analyze (pixel classification) multiple images at the same time if I individually list them like this:

"C:\Program Files\ilastik-1.4.0b1\run-ilastik.bat" --headless --readonly --project="20191220_stingscreen_dapi_pixel_Miner.ilp" foo1.tif foo2.tif

However, for actual production use, I’ll be doing multiple directories of multiple files per directory. I’ve tried wildcarding “foo1.tif foo2.tif” into variations of “foo*.tif”, with and without single or double quotes, etc, but ilastik always errors out:

ilastik.applets.dataSelection.opDataSelection.UnsuitedAxistagsException: Axistags ['y', 'x'] don't fit data shape (2, 1024, 1024) 

When I try to go headless on my Mac laptop, I can wildcard with no problems:

./ilastik_dapi_batch.sh "/Users/nieder/Documents/UTSW/8KTestSubset/images/D11S21900{3,9}??/*DAPI*.tif"

(ilastik_dapi_batch.sh is a small sh script I wrote to simplify testing and takes the first parameter in quotes " " above as $1). The actual ilastik command is otherwise identical (except run_ilastik.sh vs run-ilastik.bat).

But unfortunately, the mac is not good enough to run all the images I must do production analysis on Windows.

How can I pass wildcards on my image list on Windows?

Hi @nieder,

I don’t think windows cmd supports wildcard expansion. So consequently ilastik will interpret the input to be a stack, hence the axis mismatch :frowning:

So ideally you need a “scripty” way to call ilastik. Would doing this in Python be an option for you?

Cheers
Dominik

Yes, scripting the ilastik call would be an option. I would be generating ~30 folders of images every couple days, so being able to batch subsets of folders sort of like so "./ilastik-script.py folder0[11-20]" and the script magically converting that folder list to folder013/image1_dapi.tif folder013/image2_dapi.tif ... when calling ilastik would be great.

Hi @nieder,

this should totally be doable. if you need any hints just post your current progress here and I’m happy to take a look.

Hijacking my own thread:

Is ilastik serial, OpenMP, or MPI based? I now have test access to a large linux cluster and the slurm launcher works differently depending on how the app is built.

Hi @nieder, I just ran into the same problem. How did you end up reconciling it? Mind sharing your script?

@pdd2110
Unfortunately, I ended up not running the ilastik analysis for this project myself. A collaborator ran it on a linux cluster, with some python scripts driving the command launching. I need to ask him for the scripts as it is for my own records, but because they’re for linux it probably won’t help much if you’re also trying to do this on Windows like I was trying originally.

Fun fact, same thing here. collaborator figured it out. I haven’t tested running the script from my windows system, but since the cluster is linux the same script should work. I’ll test it today and post the solution if it works…

Windows users who want wildcards: install Git for Windows. It comes with the bash shell, so you can right-click on a directory, select Git Bash Here, and enter your headless ilastik command as if you are on Linux.