Exporting annotations - discussion about format

Sounds good!
Until then, I might come up with a few questions on how to set up a dev env with QuPath, but I will have a look at the documentation first.

I think this topic is super important! And in fact I would agree with both @constantinpape and @petebankhead arguments:

  • Having a intuitive (non-scripting) GUI based label export option would be extremely valuable, especially in the context of routine facility workflows.

  • As every downstream method might have different requirements the burden of standardisation, implementation, and maintenance of a general label export should not rest with the main QuPath developers.

So maybe having a dedicated plugin/extension that implements a specific export format is a good compromise satisfying both requirements. This way, it could ideally be tailored to the downstream task by their respective developers and yet still be installed easily via simple distribution of a jar file. Don’t know though how easy it would be to create an label export extension in QuPath for dense label mask output e.g. for stardist (@oburri and @romainGuiet maybe know more).

3 Likes

Hi @thread!

My opinion on this would be only based on our (@oburri and I ) little experience with small annotation campaigns (10-30 images) of 2D dataset for subsequent training of StarDist. For these projects, our users took care of the drawing of the annotations, and we took care of exporting the labels and training StarDist model.

We found QuPath convenient for the task because, our users were already familiar with the software and its interface as well as the use of script for QuPath.

Furthermore, with the QuPath project architecture it is simple to have the dataset and the annotations all together. We realized that getting annotations is a simple task BUT getting GOOD annotations is often an iterative process. Indeed, we often have to ask our users to refine their annotations to make them more “consistent”. Again, using QuPath this have been easy enough so our users were not discouraged.(and having pre-existing results to prove the increase of accuracy with some good annotations was a motivation boost for them)

About exporting label images, as mentioned before it’s usually a task we take care of. So having a script to do it was not a limiting step so far.

About creating a dedicated extension to do the job, we didn’t feel the need as it was mainly “exploratory” so far (few users). Maybe we need more projects, to secure the whole workflow, but I’m not entirely sure it’s necessary at this stage. (Plus I’ve no experience ,yet, with creating QuPath extension). Maybe one could think to have an ImageJ plugin and call it from QuPath?

Best,

Romain

4 Likes

I like the idea of a vanilla IJ1 plugin. This way it can work in QuPath if people need it, and people already annotating things using ImageJ (I recall @superresolusian using ImageJ for annotating) could use it directly.

Another <2 cent contribution. I stand with @petebankhead regarding the flexibility of scripts in this domain that is in constant movement and of which we are on the far upper branches.

Another idea is to perhaps see if the people hosting this kind of data already (such as https://www.kaggle.com/, or perhaps https://github.com/wkentaro/labelme/ ) are not already working on or considering a standardization effort much more downstream (down-branch?) than us, and we can be happy to be early adopters of a new format rather than rattle our own brains?

6 Likes

Another one that seems to be trying pretty hard. Does too much for us but has all that we need seems to be COCO http://cocodataset.org/#format-data.

3 Likes

Yes looking at the COCO format is a good idea.

1 Like

We’ll hopefully get a chance to discuss BD5 (https://www.biorxiv.org/content/10.1101/2020.04.26.062976v1.full) with @sonami et al at the OME meeting. ~J.

2 Likes

@joshmoore Nikon now supports BDML on their NIS Element software. https://twitter.com/sonamix/status/1290225983568535554?s=20

1 Like