Tracking with Learning proper usage


I am currently using ilastik’s Tracking with Learning project to try and track cell trajectories. Things are not quite working as expected, so I was hoping I could check my workflow with someone:

  1. I use the Training applet to manually label a bunch of cell trajectories
  2. Then pressed Track! in the Tracking applet and exported the tracking data
  3. I scroll through the resultant tracked lineages (in the same ilastik GUI) to look over the results, notice that tracking can be improved, thus return to the Training applet to train more cells and repeat.

After doing this a few times (training more cells, pressing Track again, visually going through results, training more cells/etc) I realized I might not be doing things correctly because the tracking was not improving. This was when I realized that my Calculate Tracking Weights button is grayed out, so I got an academic license and downloaded gurobi910 and installed it per ilastik’s instructions, but the Calculate Tracking Weights button is still greyed out.

Am I using the wrong gurobi release? Is my iterative training/tracking workflow as described above the wrong way to use ilastik Tracking with Learning altogether?

Thank you!

Edit; i just noticed while running this tracking project headless that one of the WARNINGS says “structuredTrackingWorkflow 22271 140284772034368 Could not find any learning solver. Tracking will use flow-based solver (DPCT). Learning for tracking will be disabled!”, I’m guessing this means I did not install gurobi correctly, or it’s not compatible with gurobi910?

Hi @kyc37,

Tracking is really the most involved, and probably the most user-unfriendly workflow in ilastik. But you have figured everything out! I think the last part is setting up Gurobi correctly. Our documentation on this is not the most verbose I am afraid (especially for windows). I’ll use this opportunity to update it after we can resolve your issue. (I apologize in advance if it turns out to be lengthy…)

Depending on the version of ilastik you have installed, you might need different versions of gurobi:

  • ilastik 1.3.3post3: gurobi 8.1.1
  • ilastik 1.4.0b0 1.4.0b5: gurobi 8.1.1
  • starting from ilastik 1.4.0b6: gurobi 9.0.3

So make sure to download the correct one.

I’m assuming 8.1.1 from here, since it’s compatible with our stable version 1.3.3post3.

After the installation you need to restart.
Then, the gurobi installation needs to be activated:
On the gurobi website you can review your licenses under Downloads & Licenses -> Your Gurobi Licenses. If there is nothing, then request a new one (also on the Downloads & Licenses page). Once you have a table entry there, click on it and follow the instructions:
paste the grbgetkey XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXXX command that is mentioned under installation to your search bar and click on Run command. This will open a terminal and ask you to confirm the location for the license file.

Okay, from here it will get a little dodgy, if you’re still on versions 1.3.3... (honestly I’d recommend using the latest beta)…


there is a bug in this version - which will make it not find the gurobi solver (or the cplex one). The solution to this is copying the relevant .dll file into the ilastik installation (you need administrative privileges for this).:

  1. I have installed gurobi in the default location: C:\gurobi811
  2. I have installed ilastik in the default location: C:\Program Files\ilastik-1.3.3post3

No you need to copy the gurobi811.dll from C:\gurobi811\win64\bin to C:\Program Files\ilastik-1.3.3post3\Library\bin


(tested with 1.4.4b8 and gurobi 9.0.3)
just works :grimacing:

Please let us know if you can get it running! Would also be interested if you see an improvement of the tracking result!


@k-dominik Thank you so much! I decided to try it out with ilastik1.4.4b8 and gurobi 903 and it worked! I have the option to ‘Calculate Track Weights’ now. But I haven’t been able to get that to work quite yet though because I am getting this error that says 'ilastik.applets.base.applet.DatasetConstraintError'>): Constraint of 'Structured Learning' applet was violated: Your track count for object 13 in time frame 127 equals 2=|{25, 2}|, which is greater than the maximum object number 1 defined by object count classifier! Either remove track(s) from this object or train the object count classifier with more labels!
I went back and labeled more cells in the object count classifier, but the same error gets thrown each time (same object, time frame, etc). Do you have any tips on how to address the error? It seems like the algorithm is detecting a merger even though I have not labeled any cells as multiple objects in the object count classifier, is that right? In my dataset, there are basically never any mergers so my object count classifiers applet is just ‘1 object’. There are however a lot of instances of objects disappearing or new objects appearing, I wonder if that is causing the algorithm to misidentify a cell track that disappears as one that merged with another cell?

Thanks so much for your help and for the awesome software!

Hi @kyc37,

could you maybe check that the number of maximum objects per mergers in the tracking applet matches the maximum number of objects in your object count classifier?


@k-dominik thanks for the response! Just checked and it does match. Max.Object per Merge is 1 and object count classifier applet includes just ‘False Detection’ and ‘1 object’ . Also just checking, if attempting to use the gurobi optimizer, the ‘Solver’ should be set to ‘ILP’ and not ‘Flow-based’ right?

@k-dominik had to put this on hold for a while and just realized while coming back to it that my object 13 in frame 127 did indeed have two tracks assigned exactly as the error said, don’t know how I missed that originally! My new error now is that ‘constraint of ‘structured learning’ applet was violated/contradicts model assumptions, all tracks must be continuous’. Thinking back to how I was training before, I think I probably left some tracks unfinished (i.e. started tracking and then didn’t go to end/mark disappearance), is this the likely reason for the error?

Hmm, it sounds more like you got into an inconsistent state (which probably should not be allowed) when you deleted the track of that one object? The algorithm here expects tracks not to jump over frames. You could use the “remove track XX from here to end” function that is available via right click on that object to get you up and running again.

@k-dominik that worked, thanks! Calculate track weights w/gurobi is working now!

I recently was thinking of trying to improve my tracking by using the Crop functionality (because my objects tend to start getting clumped beyond a certain x,y limit). Looking at the docs I thought the way to do this in headless mode would be to use the --cutout_subregion option, but it’s not clear to me if this option will run the tracking on only a selected portion of the images (like Crop does in the GUI & is my intention), or if it will just export data only for the cropped region (or are these the same thing?). Could you clarify this option or point me to a resource?

Thanks so much for your help!