Ilastik headless mode for tracking unable to execute

Dear @ilastik_team @k-dominik,

I am writing this post as I haven’t been able to find my issue anywhere else on image.sc nor on the github page.

I am exploring and trialling ilastik (so far it’s proven itself a worthy tool!) prior to writing a Python code. The idea would be to run ilastik (for segmentation & tracking data) headless from Python and do some further processing in my own script.

Using the Terminal, I’ve been testing the headless mode and I’ve come to a bottleneck: running tracking projects.

Initially, it seemed as though the input was going through just fine but nothing was happening. For details: terminal_log.txt (2.2 KB)

Right, so then I found this issue on image.sc, and so, according to the discussion there, I slightly amended my input arguments (i.e. no ‘=’ symbol between --rawdata and image file). It does seem like it’s trying to run the project but an error comes up (error details: terminal_log2.txt (8.6 KB)). The software doesn’t seem to find that the ‘Classifier is ready’.

I have done the exact same process but within the ilastik GUI and it doesn’t spit out any error. Running the project on a raw data+segmentation file as well as export work just fine in GUI.

So I don’t really know what else to do. I double-checked the ‘Object Count Classifier’ was working properly by clicking on the ‘live preview’ button - prediction seemed normal/as intended. I have also re-run tracking as a way of saving the training on the ‘Object Count Classifier’ as I can’t find a way to tell the software that it has been set/trained.

Do let me know if you need any further information from me and I’d highly appreciate if you could provide some input/insight as to what I could do to fix this.

Looking forward to receiving your reply and thank you in advance.

Cheers,

Pablo

Hi @pablooriol2,

cool that ilastik is doing something useful for you! Let’s see if it can properly be included in your pipeline.

So the first thing I notice is that the command line option for supplying the raw data has a typo: it should be --raw_data (not --rawdata). The command line interface of ilastik is very finicky (unfortunately). So this might already resolve it. Would you mind checking this?

If not I’ll have a closer look!
Cheers
Dominik

Hi @k-dominik,

On the one side, I was hoping that it would be a very simple fix - and it is! On the other side it’s a bit embarrassing that I didn’t figure it out myself xD Apologies for not paying enough attention to the --raw_data command. May I point out that I had the other input file command wrong as well: --segmentation_image is the correct command (and not --binary_image). Might be worth updating this in the Headless Operation documentation.

Despite getting many ‘warning errors’ I have managed to run the project successfully and have confirmed the output is valid by opening the tracking-result.h5 file in ImageJ via the ilastik-ImageJ plugin.

For further inspection (if desired) I’m attaching the terminal output with all the ‘warning messages’ here: terminal_log_success.txt (1.1 MB)

For any future readers checking this post; I’d like to point out that it is crucial that both your input files (raw data & segmentation/probability data) have the same shape and dimensions - otherwise the project won’t be run. In the GUI you’ll have an error message pop up though.

I’ll proceed to use/test ilastik from Python now. Hopefully, it works just as well with --probability_maps and the Tracking [Inputs: Raw data, Pixel Prediciton Map] workflow too. Thanks a lot for your help @k-dominik (such a fast response as well - way to go!)

As a final note, I’d like to comment that although ilastik is an amazing tool in terms of enabling intuitive machine-learning image analysis methods (i.e. segmentation and tracking in my case) it is missing a couple of features that I think would highly increase its value as a software package.

  1. The ability to track objects that are ‘lost’. My understanding is that it currently does not have the ability to pick up objects with tracking gaps (even in the case of 1 frame). In such a scenario, ilastik simply labels the ‘re-appearing’ object with a new ID (correct me if I’m wrong). Now that I think of it, it’s a bit surprising this feature is not supported since there is the applet for tracking dividing cells & lineage tracing…
  2. The second feature that I think is really missing is that of curation. Being able to preview the results (when training a project) is absolutely crucial to get an idea of how good (or bad) segmentation/tracking performance is. So I’m grateful it is available. However, even with well-trained projects, there are still instances where some cells/objects are simply not well segmented/labelled/tracked. And not having the option to correct single-cell tracks leaves the user with unsuitable options. By this I mean that the user either 1) accepts that some objects are not well tracked and decides to include it in the final analysis (although I want to think no scientist would ever take this approach) 2) excludes the badly tracked objects as a quality control step in a script, and thus reduces the available sample size or 3) builds a tool/script that allows the user to ‘correct’ object tracks (although this is the best approach, it’s the most time-consuming one).

I realised I’ve written a bit too extensively, but I do this to give feedback to the @ilastik_team @k-dominik in the hope of improving what already is a great tool.

Oh, one final question/point: is it possible to pass through several datasets within the same .h5 file? Say I have an experiment.h5 file with the following structure:

/Data/FOV01/channel0 (tyx)
/Data/FOV01/channel1
/Data/FOV02/channel0
/Data/FOV02/channel1
...etc

Is it possible to run ilastik projects with --raw_data experiment.h5/Data/FOV01/channel0 /Data/FOV01/channel1 /Data...etc as input commands to batch process these? It would save the user from having to export each time-series as a single file…

Once again, thank you very much @k-dominik for all your hard work - not only responding to this post but also all other posts about ilastik issues raised by other users. It is very helpful.

Cheers,

Pablo

Hi @pablooriol2

good that it is working for you now! And also thank you very much for your detailed questions/suggestions. This is very valuable to us!!

thank you very much for pointing that out! Added to the todo list

The warnings are, well, just warnings. We’re actually working on reducing those. I always find it a bit odd that the output in Fiji classifiers them as Errors and highlights them in red…

Also thank you for your further notes on possible improvements. A few comments:

You are right, there is no such thing as a linage with a gap in ilastik. So if an object disappears it’s gone. In general allowing for this drives up the complexity of the problem (in the graph you’d need connections to more than just the next frame). Also from a more practical point of view, it might be super data dependent whether you can rely on an object disappearing and re-appearing in the span of a couple of frames in a “similar” position are actually the same…

For this we usually point users to use Fiji MaMuT/Mastodon. There is a plugin export for MaMuT in ilastik tracking (video demonstration here).

As for your last question:

Specifying the raw data like this (note the undocumented --stack_along parameter) should work:

--raw_data "/data_folder/data.h5//Data/FOV00/channel*" "/data_folder/data.h5/Data/FOV01/channel*" --stack_along=c

If you plan to integrate this in the pipeline (first pixel classification then tracking) then there currently is one not so elegant thing: you can not write multiple datasets into one .h5 file from ilastik. This is not a big thing for us to change, and we will do it, but currently you’ll have to live with your raw data files being organized differently than your predictions/segmentations produced in ilastik.

If something is unclear/not working please let us know!

Cheers
Dominik

Hi @k-dominik,

That’s a neat little trick there - this is (almost) awesome! I say almost because I’ve run into problems again, unfortunately:

  1. ImageJ/BigDataViewer doesn’t let me export a .h5 raw data file to XML/HDF5 - it uses more than 10GB of RAM (for a <250MB file, really?) and never seems to output anything. It eventually crashes/freezes and I have to force quit. It works with the same file but in .tiff format though - so I went with that.
  2. Maybe because I exported from a .tiff file my MaMut Viewer is messed up? Has anyone had this happen to them? See the screenshot below

  1. This is probably more relevant for ImageJ/MaMut/TrackScheme developers but I’ll put it here anyway. Below is a screenshot of the tracks in the TrackScheme ‘applet’ from MaMut. I tried ordering them by clicking on the style button or changing some settings to ‘track duration’ but I’ve only managed to change the colour of the tracks (as opposed to reordering them from longest to shortest). Because the TrackScheme GUI doesn’t seem to be working that well, I was just wondering if there’s any possibility of amending the tracks prior to importing the MaMut file exported by ilastik?

Noice - I’ll give that a try! Thanks!

Sure thing, I guessed that might be the case. No worries, I’ll merge the exported files with Python.

Once again thank you for your help.

Cheers,

Pablo

Good news! I have solved these two issues (they were indeed related). See below quote for solution.

So initially I was importing the original .h5 file (the one I used for training I think…) with the ilastik-ImageJ plugin (Plugins → ilastik → import HDF5). This turned to be (somehow!) too problematic for BigDataViewer. I imported the same file with Plugins → HDF5 → Load HDF5 File…/Load as… 'individual hyperstacks (custom layout) - data set layout: xyt (in my case). Select the right dataset (within the HDF5 file) and finally click on Load. Then export file with BigDataViewer and export should now be successful. At least it made it work for me. Just re-tried and works - good to know xD

With the newly exported xml file (from .h5 file) I imported the MaMut file (exported from ilastik) and now have a correct view of raw image & tracks. See below

I guess I’d also have to export the tracking-result from ilastik as masks and then use the MaMut xml output file (once data is curated) to amend the masks in Python? My first thought would be to use the X & Y positions from the spots to ensure it has the ‘right’ objectID frame by frame… As you can see I am figuring things out on the fly, but I don’t know how people would otherwise go on about amending the masks needed for measurements on the raw images.

Cheers,

Pablo

hi @k-dominik,

I make a reappearance unfortunately :-/

I am able to carry out batch processing (from python) for segmentation/pixel classification purposes. However, I haven’t been successful with the tracking workflow.

I must stress that I am unable to make it work from python, as I when I type in the right commands in the terminal it works just fine (see here successful_run2.txt (88.2 KB)). Using the exact same commands but from python though, I get an error message (see here log_fail.txt (12.3 KB) )

And here’s my Python code that I’m trying to run:

def run_ilastik():
    import os
    import subprocess
    import h5py

    h5fpath = '/Users/pablooriolvalls/PycharmProjects/SC_PyFRET/Images/2020-05-01_ANALYSIS/T_B_XP_Master.h5'
    inputpath = os.path.dirname(h5fpath)
    h5name = os.path.basename(os.path.splitext(h5fpath)[0])

    with h5py.File(h5fpath, 'r') as f:
        inputlist = list(f['RAW'].keys())

    for inp in inputlist:
        currinp = f'{h5fpath}/RAW/{inp}/YFP'
        fpredict = f'{inputpath}/{h5name}-RAW-{inp}-YFP_Probabilities.h5'
        chck = subprocess.call(["/Applications/ilastik-1.4.0b1-OSX.app/Contents/ilastik-release/run_ilastik.sh",
                                "--headless",
                                "--project=/Users/pablooriolvalls/My_trackingNprediction.ilp",
                                f"--prediction_maps {fpredict}",
                                f"--raw_data {currinp}"])

        if chck==0:
            print(f'\nTracked {inp}\n')
        else:
            print('\nilastik was UNABLE to be instantiated\n')

            print(f"\nOUTPUT {inp}: /Applications/ilastik-1.4.0b1-OSX.app/Contents/ilastik-release/run_ilastik.sh",
                   "--headless",
                   "--project=/Users/pablooriolvalls/My_trackingNprediction.ilp",
                   f"--prediction_maps {fpredict}",
                   f"--raw_data {currinp}\n\n")

run_ilastik()

It is odd as the same structure has worked to automate segmentation batch processing. I’ve even tried using ilastik’s own python interpreter, but with the same result.

My current workaround I have in mind is to ‘print’ a fat piece of text containing all the right commands and input files that then I’d need to copy and paste in a new terminal window. Dirty, but functional (I hope).

I was wondering if you could point out to what needs fixing? I haven’t tried this with binary images (indeed, my workflow generates & uses probability maps), so it may be a workflow-specific issue?

I look forward to your reply and thank you in advance :slight_smile:

Hi @pablooriol2,

sorry I’m a bit short on time. But could you try separating the --prediction_maps, and f"{fpredict} into to separate list entries?

chck = subprocess.call(["/Applications/ilastik-1.4.0b1-OSX.app/Contents/ilastik-release/run_ilastik.sh",
                                "--headless",
                                "--project=/Users/pablooriolvalls/My_trackingNprediction.ilp",
                                "--prediction_maps",
                                f"{fpredict}",
                                "--raw_data",
                                f"{currinp}"])

just a hunch…

1 Like

Hi @k-dominik,

LE-GEND.

Made my day. Currently running tracking 24 FOVs from running a python script.

I didn’t come about that ‘formula’ during my detective/trial&error session. It felt very frustrating as I did have a feeling that it would be something simple, and indeed it is.

So thank you so much for your input - you nailed it!

ilastik may not be flawless in headless mode (yet) but it sure has some flawless support.

Again thanks a lot, and I hope I don’t have to bother you again xD

Cheers,

Pablo

1 Like

thank you very much @pablooriol2, I really appreciate you nice words :slight_smile: