Ilastik thresholding technique for segmentation

Hi all,
I have been using Ilastik as part of a macro I have written in Fiji. I originally used the Pixel Classification Prediction plugin to produce a ‘Probabilities’ of all my different labels and then used Otsu thresholding to try and best segment the image. I’ve realised now that changing the output type to ‘Segmentation’ is much better but I was wondering what thresholding technique is used to produce it. I cannot find it in the documentation and I’ve tried a few other automated thresholds to no avail. I was wondering if instead hard limits are used based on the ‘Probabilities’ output type but cannot discern the parameters for this. Would be really helpful to understand for later tinkering with the macro. Thanks in advance!

I did this in Fiji to convert Ilastik output into a mask I could use somewhere else. If there is a better way I’m interested as well:

IJ.run(“Import HDF5”, “select=[”+fullPath+"] datasetname=[/exported_data: (2500, 2500, 1) float32] axisorder=yxc")
imp = IJ.getImage()
IJ.run(imp, “8-bit”, “”)
IJ.setAutoThreshold(imp, “Default dark no-reset”)
IJ.run(imp, “Make Binary”, “”)
##attempt to break up clumps
IJ.run(imp, “Options…”, “iterations=1 count=1 black do=Nothing”)
IJ.run(imp, “Open”, “”)
IJ.run(imp, “Watershed”, “”)

Hi @sebg,

the simple segmentation output (if you are referring to this by “segmentation”) can be reproduced easily from the (multi-channel) probability image by selecting for each pixel the channel (=label) with the maximum probability value. Does that make sense?

Thank you for your reply, @k-dominik . Yes, I think I understand. If you are saying that it considers pixels to be part of that label if they are the highest value in the channel, what are the specifics? Does it take the pixels above 90% probability to be part of the segmented output (pre-set threshold limit) or does it vary depending on the range of pixel values in a channel (variable thresholding)? If the latter, then by what technique?

the values across the channels should sum up to 1.0 for each pixel. So the “simple segmentation” does not operate on a fixed threshold level, it just takes the highest. So for a 3 label example with one pixel having [0.3, 0.3, 0.4] in the three channels, the third label would be assigned to this pixel.

2 Likes

Hello, and thank you for your reply to this post @k-dominik
I had a follow up question. What is ilastik’s segmentation algorithm when probabilities across two (or more, if you have >2 classes) are the same and maximum?
e.g. for a 5 class pixel classification with probability values of [0.3 0.3 0.2 0.1 0.1], how does it decide pixel is assigned to class 1 or 2? or if prob=[0.3 0.3 0.3 0.08 0.02], which class does this pixel belong to 1,2, or 3?
(I see this situation in not so few pixels in my analysis. I am exporting probability maps from ilastik and generating the segmentation based on max prob across channels per pixel in Matlab).
Thank you.

Hello, and thank you for your reply to this post.
I had a follow up question. What is ilastik’s segmentation algorithm when probabilities across two (or more, if you have >2 classes) are the same and maximum?
e.g. for a 5 class pixel classification with probability values of [0.3 0.3 0.2 0.1 0.1], how does it decide pixel is assigned to class 1 or 2? or if prob=[0.3 0.3 0.3 0.08 0.02], which class does this pixel belong to 1,2, or 3?
(I see this situation in not so few pixels in my analysis. I am exporting probability maps from ilastik and generating the segmentation based on max prob across channels per pixel in Matlab).
Thank you.

Hi @MinaG,

if you go for simple segmentation it’s max prob, as you do in matplab.

You might be able to mitigate the problem with the probabilities having equal probs accross many classes maybe a bit by smoothing the probability maps (given that this is not a systematic feat of your probability map, say that all pixels of a certain type (some structure you’re trying to segment) have this kind of uncertain probabilities). It could also be that your structures just look too similar and you will not be able to separate the first three classes. But maybe adding more training data can help. Have you ever looked at the uncertainty layer in ilastik? It hints at such regions.

Thank you very much for your reply and suggestions @k-dominik. The uncertainty maps suggest that most of the uncertain regions are at the contours where one class switches to another (and there are lots of such transitions in a given image). Hopefully providing more training examples from such border regions would help reduce uncertainty, but I doubt it will solve the problem as there might be implicit ambiguity in pixel information in those transition regions at such level (e.g. colors might be a mixture of the tow class for example).

I was also wondering whether not using features extracted from larger smoothing sigmas (e.g. 5, and 10) would also help reduce uncertainty around the edges??

Thanks for your suggestion for dealing with equal max probabilities. I will try smoothing the prediction map (at such pixel locations); we were thinking along the lines of taking the majority vote in the (first) neighborhood, but I would assume they work pretty similarly.

In general (if it is not too slow) I’d suggest to use all pixel features and let the random forest figure out which ones to use for the decisions. In pixel classification you should really quickly have enough training data for this approach to be safe.

yes, this is a good idea.