Define a "class/category" property for ROI?

Dear all,

Motivation

Detection/classification algorithm are popular now, but generating the ground truth annotation (bouding boxes or polygonal shapes) can be time-consuming, especially since every package expect a different formatting (xml…) for ROI coordinates. And there is no generic annotation tool.

ImageJ is pretty good with ROI, it´s widely used and it´s very easy to make a small script to generate xml or text output from a ROI list.
So to my mind ImageJ could be used as the perfect annotation tool, without needing to make a specific plugin for that.
The only missing thing as far as I know is the possibility to define a category for ROIs.

Example

If that’s unclear, let say I have 2 types of cells, and I want to be able to define categories (as strings). “Class1” and “Class2”. It would be great to be able to switch between categories while drawing consecutive ROI, and that the category be associated to the ROI object.

Possibilities

There is already the possibility to define arbitrary properties thanks to the setProperty method.
So one can set a property “class” by coding, but it is not visible in the GUI. And if someone uses a different property name like “category” you are screwed…
So basically my proposal is to define a new “official” category property for ROI such that one could have Roi.setCategory, Roi.getCategory.
The category could be defined by the user by a double click on the ROI tool icons.
image
This means quite some work I guess, to keep things backward compatible, for ROI import/export…
But sounds reasonable doable.

Looking forward to some feedback :wink:
@wayne @ctrueden @imagejan

2 Likes

I don’t feel strongly. The proposal would only affect ImageJ1, so I defer to @wayne.

1 Like

I already use the ImageJ ROI Manager in combination with R to perform supervised pixel classifications.

I didn’t write my own GUI because the ROI Manager already includes most of the required functionality and people who use ImageJ already know how to use the Manager or learn how to
use the ImageJ ROI Manager which is an essential tool in ImageJ.

My solution or workaround (one option for the next release) is to make it possible to extract the class category from the ROI’s name suffix which works quite well (something like classxxx_1, classxxx_2) in combination with my ROI transfer GUI.
So you simply have to rename the ROI’s accordingly.

However to enable a class property in the GUI would be a nice addition.

In addition I also would like to see a possibility to define or link an attribute table in combination with the ROI (e.g., to set an object property) which can be used to define covariates, e.g., for spatial analysis or point pattern and spatial analysis (like it is common in GIS systems).

Of course I could extend the ROI class myself but I would like to stay as close as possible to the default ImageJ class.

1 Like

@LThomas you might want to consider QuPath for that… which is widely used in histology/pathology for precisely this purpose, but is designed also for microscopy. I’ve spend a long time refining the annotation tools to make annotating less painful, and continue to improve them.

Features include*:

  • Support for ROI classifications (and shortcuts to set these quickly)
  • Various drawing tools, including wand and brush tools that
    • double as eraser tools if you press Alt
    • are magnification-dependent (i.e. zoom in to annotation small regions, zoom out to annotate large ones)
    • support pressure-sensitive graphics tablets via JPen (I’ve only tested with Wacom tablets)
    • support ‘locking’ around existing annotations (by pressing Ctrl + Shift)
  • Ability to merge / subtract / dilate / erode / invert ROIs
  • Ability to view multiple channels - both as composites and simultaneously side-by-side in the channel viewer while annotating; tested with over 40 channels so far
  • Ability to convert to & from other forms, including
    • ImageJ1 ROIs - passing the QuPath class via the ROI name and color
    • Java Topology Suite Geometry objects
    • GeoJSON
  • Export annotations as labelled images to train classifiers elsewhere (currently via scripts, will be built in soon)

There is a lot of other stuff as well, including interactive object and pixel classification - where the pixel classification can be used to generate initial annotations that may be manually refined if needed. This is usable now, but will be improved in the next milestone release, coming very soon.

Probably the fastest way to get an overview is through the YouTube channel: https://www.youtube.com/c/qupath

Other relevant links:

*-Some features have been there from the start, but others are only available in the latest milestone releases.

3 Likes

I started to experiment some stuff with the Roi class on my fork https://github.com/LauLauThom/ImageJA/blob/master/src/main/java/ij/gui/Roi.java

I added :

  • an integer group attribute to instances of the Roi class
  • setting the group attribute in the constructors to the currentGroup value
  • the setGroup and getGroup method

The idea is that the currentGroup value can be modified from outside the class via the GUI, and every newly created ROI have their group attribute assigned to this currentGroup value (previously created ROI are not changed obiously).

However I am not really used to Java, so I would be happy to have some review about those changes before I go further @Wayne, @ctrueden @imagejan, @haesleinhuepf

In particular, I dont know what qualifiers should have currentGroup :sweat_smile:
I think it would be static (for the all ROI class) and maybe public so that one can change it from outside ?

Thanks !

3 Likes

Hello Laurent -

I would suggest that you use a initial (default) value of 0 for
currentGroup (rather than 1 as in your current code). It
doesn’t make much difference one way or the other, but java
(like most modern language) is a zero-based language.

Would you anticipate currentGroup resetting to its default value
every time Fiji / ImageJ is launched, or would you imagine hooking
it into ImageJ’s persisent preferences system?

I don’t hold myself out as being a java expert, but I would make
the following comments:

Yes, currentGroup should be static – that is, a class variable
rather than an instance variable.

Since you want to change it, it can’t be final.

As a practical matter, currentGroup could be public – then you
can just change it – but the java-esque way of doing this would
be to make it private and introduce a public getter and setter
(just as you have done with the group instance variable). The
getter and setter would be static – that is, class methods
rather than instance methods.

Last, in addition to being able to modify currentGroup from the
gui, I think you would want to be able to easily modify the group
of an existing ROI from the gui. Perhaps the Properties...
function (button on the right-hand side) of the ROI Manager would
be a natural place to put this.

Thanks, mm

1 Like

Thanks a lot @mountain_man for the detailed answer !
I took into account most of your suggestion, only for currentGroup I set it to protected like lineWidth and Roicolor.

And… it works :partying_face: !!
Via the script editor I could get the group from drawn Roi, setGroup and currentGroup.
To be honest, I would not expect to have a working prototype from those few additions, really cool !!

Let’s work on the GUI now !
Also probably the RoiEncoder/Decoder to make sure the group is saved.

Definitly !

1 Like

Hello Laurent -

I wouldn’t model your Roi.currentGroup after Roi.lineWidth.
The lineWidth code is somewhat garbled, and looks to be
largely superseded by Roi.stroke (an instance variable).

Roi.ROIColor (a class variable) seems to be fully analogous
to what you want to do with currentGroup.

Note, if you decide it makes sense to hook currentGroup to
ImageJ’s Prefs facility, the initialization of ROIColor illustrates
part of how to do this:

protected static Color ROIColor = Prefs.getColor(Prefs.ROICOLOR,Color.yellow);

As a side note about private vs. protected, let me give you
two arguments about preferring private:

Remember private gives access only to the class itself, while
protected also gives access to subclasses.

Pro: The only thing you do with currentGroup is set and
retrieve its value (using its public setter and getter). (It could
have been directly exposed as a public class variable, but
you chose to be java-esque, and use a setter and getter.) Any
subclass has full access to currentGroup through its setter
and getter, so the subclass does not need the “direct” access to
currentGroup that would be granted by making it protected.
As a goofy hypothetical (the kind of java-esque hypothetical
that would be used to justify not just making currentGroup
public in the first place), if you were to change the implementation
of currentGroup – say by making it a character string – any
code that tried to access it directly, including code in a subclass,
would break. So, be consistent and make it private, and make
all “consumers” of currentGroup, including subclasses, use
the setter and getter.

Con: Almost all of Roi.java (and much of ImageJ) uses
protected static for this kind of use case. So be
consistent with ImageJ’s style and use protected.

(Pro-tip: Consistency with the existing code base wins.)

Thanks, mm

1 Like