BDV LabelEditor / Objects Counter (IJ2) [DEVELOPMENT]

Hi devs,

if you are dealing with ImgLabelings labeling in general and / or BigDataViewer or related viewers, have a look at the newest baby of the JugLab, the LabelEditor, and let us know what you think!

Resources

<dependency>
	<groupId>sc.fiji</groupId>
	<artifactId>labeleditor</artifactId>
	<version>0.5.1</version>
</dependency>

Technical details

Here are some technical details about the content of the repo:

sc/fiji/labeleditor/core

The heart of this project is a mechanism to tag any label of an ImgLabeling instance with tags. A tag is of type Object so it can be anything. One can assign colors to a tag. Renderers implementing LabelEditorRenderer will be discovered via the SciJava Plugin mechanism and translate an ImgLabeling into a RandomAccessibleInterval<ARGBType> with reference to the assigned tags and tag colors. Viewers are represented as LabelEditorInterfaces, so BDV is not baked into it.

sc/fiji/labeleditor/plugins

This includes a face and a border renderer, the BDV interface bridge, the right click menu, a bunch of behaviours to deal with labels and tags, two different selection modes (the default one and a conflict mode where only labels which are in conflict with the focused one get deselected), and a model where each time slice is handled individually (crucial for bigger data).

See the usage post for more details and images.

There is also a BVV branch including the plugins to create a LabelEditorBvvPanel which works rudimentarily.

sc/fiji/labeleditor/applications

Here lives the Objects Counter (see usage post).

a/fiji/labeleditor/plugin/imagej

This namespace contains the displays to be able to show an ImgLabeling or a LabelEditorModel with the LabelEditor via UIService:

ImgLabeling labeling = ij.op().labeling().cca( binary, ConnectedComponents.StructuringElement.EIGHT_CONNECTED);
ij.ui().show(labeling);

I had to change the namespace so that the DefaultImageDisplay would not be chosen first and fail to display it. I will look into how to improve this (I don’t think the plugin priorities are used here).

net.imagej.workflow

The Objects Counter is calling two commands in a row - to not have multiple windows opening in between the workflow asking for input parameters, I made an ImageWorkflowCommand which runs the commands first on a small part of the input image, collects the panels for harvesting the inputs and displays them in a joint window. It keeps the parameter choices the user selected in the first run and then executes the whole workflow again on the whole image without asking for parameters. My only test case so far was the Objects Counter class.

imagecommandworkflow

It currently looks quite ugly, but I think it is useful maybe also for others. Super interested in others opinions here!

Conclusion

Thank you for reading so far! Let us know how to make this useful to as many people as possible. Also happy to tear the repo apart.

6 Likes

Hey @frauzufall,

I love it! This is so cool. I just tried if I can incorporate it in my BDV tools. It took me half an hour - a sign that you made a very easy to use API and great examples! Thanks for this! :star_struck:

However, two little questions came up:

  • In my example, the labels are not clickable. If you find some time for taking a look at my code, I would love to learn how the interaction with labels can be activated. Select all/none works but individual labels cannot be clicked. I assume because clicking activates the rotate-view action in BigDataViewer. But I couldn’t figure out which key to press to toggle this. You can get my view from the screenshot above by executing this macro.
  • I noticed that I need to depend on sc.fiji-bigdataviewer-vistools-1.0.0-beta-18 in order to make the code compile, which is not rolled out by Fiji yet. Is there a time-plan when this version will ship with Fiji?

Thanks again for making this happen! Many of us dreamed for long time of a good 3D segmentation / label editor in Fiji :heart:

Cheers,
Robert

4 Likes

Hi @haesleinhuepf,

thanks for the positive feedback :hugs:

I have to debug why the interaction does not work in your case, but your guess of conflicting key actions might be correct… All the LabelEditor actions are behaviours so they are configurable and as soon as I get behind how to do this the best way, I’ll add an example.

Just one note on your code - I don’t think you need these two lines, you are not using this panel anywhere, right?

And about the specific bigdataviewer-vistools version - I got a runtime exception in IntelliJ without it, but in Fiji when I setup the update site I did not upload any JAR but the LabelEditor and it still worked :woman_shrugging: So you might be able to ignore it. I should figure that out and clean up the POM…

Ha, cool! Indeed! Thanks for spotting this!

If you point me to code where the label editor sets up its behavior/actions I might be able fix the label clicking issues in 3D myself :upside_down_face:

Hi @kephale,

I was hoping we could add the LabelEditor as a layer in SciView! But I had no time to investigate this yet. I can help figure out things on gitter or maybe also meet @skalarproduktraum and have a look?

All BDV-specific code in the LabelEditor is in this directory, the only crucial class there is the BdvInterface. If you create an interface like this for SciView, you can try the Add to existing BDV example to see how the interface is used to add the LabelEditor layers to an existing viewer.

Maybe also helpful: the BVV branch containing the this directory with BVV specifics. The important difference is that there is not just one coordinate at the mouse position, but a whole trace that has to be followed until an object is reached. I guess it would be similar to SciView. The code for BVV is a proof of concept, works only partially and might contain errors.

Let me know what you think!

1 Like

Do you mean these lines? I think you can use the names of the behaviours I install there to override the triggers, right?

1 Like

Basically, the consumer of LabelEditor should be able to pass in a InputTriggerConfig that he loaded from a file or created programatically.

(See this example for loading a InputTriggerConfig from a file (or rather from a String in this case). For creating manually you would just make a bunch of InputTriggerDescriptions and create a InputTriggerConfig with those.)

2 Likes

I clearly missed this thread and @Christian_Tischer’s awesome work on interactive tables which has quite some similarities and was posted earlier this year… I hope we can join forces during the December hackathon, come up with a useful set of modules and benefit from the different perspectives :bowing_woman:

4 Likes

I am really happy :slight_smile: you bring it up yourself, because honestly, I felt a bit ignored :frowning: when I read your announcement. However, I of course do understand that is easy to forget people that are not in the same office.

And yes, of course, I would love :heart: to work on consolidating this with you during the hackathon!

Maybe an idea for the future (cc @fjug) : I think it would be really awesome if we could do something to improve the communication around developments in the bigdataviewer ecosystem. For example, what @bogovicj did here Planned dev - nonlinear elastix imglib2 transforms (feedback requested) is, imho, absolutely awesome: Pinging possibly interested people ahead of time to coordinate early on, i.e. probably even before starting to write code.

And just to emphasise again: @frauzufall I love what you developed and it is absolutely great and useful for the community!

4 Likes

:hugs: Sorry again (also wrote you on gitter, don’t think you read it). I did not forget you but I thought for BDV it is sufficient to ask in the house if we were overlapping with someone else’s work and discuss it publicly with a prototype. It started with the need of a segmentation conflict handling tool a month ago and then somehow blew up. Next time I know better.

3 Likes