Paint ground-truth mask on image (without training a classifier)

Image Analysis
, , , ,
#1

Hi all,
In order to evaluate the quality of an automated segmentation, I am looking for a way to generate a manual segmentation by painting the ground-truth mask on an image (similarly to Ilastik, Weka… but without training a classifier).

With Ilastik for instance it is not possible to export the mask without training the classifier.

In Fiji, the paint brush tool can do the job but then I had no idea how to recover the painting since it seems to be burned into the image, it it possible to use some overlay ?

With Knime there is the interactive annotator but it is currently limited to ROI annotation.

Currently the best I found was to use an Image editor like Paint.Net (see screencast below). It is quite convenient as we can add a layer for each class, adjust transparency… After merging the layer and saving it as a tiff, it can be open in Fiji as RGB and converted to grayscale such that each classes has its own pixel/label value.
The only derawback is that if the painting of different classes overlap for some pixels then the result is a combination of the values (not sure it is a sum as it is RGB).

I am open to other suggestions of course !

Painting mask in Paint.Net
image

#2

Double-click on the paint brush tool to get the option to paint on an overlay which you can add and remove (optionally to the ROI manager) and save with the image in *.tiff format.

#3

I just found out that the Add To ROI Manager action results in a default black background with the brush painting.

You can only change the opacity in the properties dialog of the ROI Manager (e.g., 10%) to overlay your paintings with the original image.

#4

There are various tricks in QuPath to help with this task - at least for 2D.

The annotations themselves are based on vertices (not pixels), and may overlap - however you can adjust the export scripts to make sure that one annotation is ‘always on top’.

1 Like
#5

You can also use the ROI Manager with added selections to get the required results.

Draw a freehand selection, add the selection to the ROI Manager, Select Properties in the ROI Manager and insert a color with alpha value in hex notation (e.g.: #440000ff - blue with alpha 0.27). For each class add a selection to the ROI Manager.

I think this solution is more handy because you can save and restore your ground truth (selections) along with the image and you can select different alpha values for the selections.

1 Like
#6

About the overlay,
Interesting, I dont know much about this functionality so that’s always good to take !

I realised that the Overlay to Roi Manager turns every single continuous drawing into separate ROI though, independently of color or overlap.
Moreover, from the ROI set if I use Draw on a black image of same size of my initial image (to generate a mask image from the overlay/roi set) they are all drawn with the same gray level.

If someone has a suggestion to save the overlay as an image “as it is” namely same painting color = same pixel value in the mask image that would be great !
I can save the overlay with the image but it does not really help as I would need an image of the mask exclusively.

Probably a script could do the job, but I hope there would be a more straightforward way…

About the brush tools, I realised that I have 3 different brush tools in my fiji ! The last one does not make the black background bug.
It’s a bit confusing to have 3 brush tools to be honest ! maybe the first and second could be merged ?
Or are they both respectively ImageJ and Fiji tools ? so that merging them might be an issue for backward compatibility…

1 - The simplest one only has the thickness parameters
image

2 - The second one which has the Paint on overlay option

image
image492×537 53.2 KB

3 - The Overlay Brush
image

@petebankhead, I will definitly take a look at QuPath thanks !

#7

Do you know the Selection Brush Tool already?

image
image828×234 23.4 KB

  • start drawing from inside an existing selection to expand it
  • start drawing from outside to remove area from the selection
  • use Alt to remove parts inside an active selection
  • use Shift to add area outside of the selection
  • double-click the tool icon to adjust brush size.

Then use the ROI Manager – or the overlay – as suggested, or draw/fill the selections in a new black image/stack to create a binary mask.

1 Like
#8

Oh, yes, and then there’s Labkit, a work in progress by @maarzt and @frauzufall:

3 Likes
#9

Nope, nice one ! I would still need to take care to fill the different regions with different pixel values though…

LabKit offers a nice Label>Show labelling in ImageJ that generates a tiff mask, I think this is the closest to what I was looking for thanks !

image

1 Like
#10

The latest ImageJ daily build (1.52h1) fixes a bug that caused images to turn black when assigned an ImageRoi created by the Paintbrush tool, the one with the “Paint on overlay” option. View the source code changes at

#11

Hi all,

cc @constantinpape @akreshuk

For reference, I also wanted to share my experience using LabKit (btw: thanks a lot to @maarzt for developing it). We need to create a label-mask ground truth for many overlapping cells, like this (this is just a very small crop, the actual images are much bigger):

image
image1024×778 244 KB

Since these cells are overlapping I think we need one class per cell, isn’t it? Just have foreground and background would not do the job, I think.

LabKit seems to work very nice to do this work (and, as @LThomas pointed out), it has a convenient way to export the drawing as a label mask to imageJ).

image
image1678×1170 396 KB

1 Like