Key-value pairs for annotation?

Usage & Issues
#1

Hello everyone,
I wonder if something like the following is possible using the tools Qupath provides:

  • Several regions within an image are marked with an annotation (e.g. a circle).
  • Each of these regions can have several/many features (all binary, e.g. Apoptosis - yes/no, Fibrosis - yes/no, …).
  • The features are assigned manually.
  • The features are independent of each other.
  • I imaging something like: I do a right click on the annotation and the (predefined) feature list pops up where I can select yes or no for each of the features.
  • The feature lists for all annotations can be exported/saved as a table.

Can anyone envision a workflow that would provide something along these lines?

#2

Maybe with an extension you could get access to the context menu, but I am not sure if that is possible simply through scripting. What has been done is cycling through all available TMA positions (which you could swap out for Annotation positions), and select characteristics within a dialog.

Here is a script by Thomas that iterates through TMA cores with a GUI that allows qualitative analysis to be done though a GUI: https://groups.google.com/d/msg/qupath-users/HYRSCf_ctyE/fOfYVnXHBQAJ

I have several scripts that involve building dialog boxes through javafx here that demonstrate interfacing with QuPath:https://gist.github.com/Svidro/6171d6d24a85539d3af5d417bc928d50
Specifically “Tissue detection” and “Classification with GUI” might help with further examples of interacting with measurement lists and such.

Feature lists for annotations can always be exported as a table using saveAnnotationMeasurements(“fileLocation”, 5,6,8) to get the 5th, 6th, and 8th columns only, for example. Pete has two scripts here: https://petebankhead.github.io/qupath/scripting/2018/03/04/script-annotation-export.html
and the next post that can summarize annotations for a project.

Depending on the staining, you might be able to use the pixel classifier to do that all automatically in 0.2.0. Haven’t seen the sample staining variability though, so YMMV.

#3

The user interface to enable it would need to be coded (but this could be done in a script). Then the info just needs to be stored somewhere…

Since everything is binary, one option is to use the classification mechanism. You’d add a (derived) class with the name of something that’s positive. Then later on you can extract the different pieces of the final classification to see if the name you’re looking for occurs.

Alternatively, you could add a new measurement to each annotation. The keys are strings and the values must be numbers (doubles), but you could use a 1 or 0. This is probably the easiest way to do it, and the measurements will then appear in any annotation table by default.

How flexible/customizable would you need this to be? Do you have a single list of keys that you’d want to use, or is this likely to change regularly? And roughly how many would you expect to have?

My guess is that if each annotation might have multiple ‘positives’ needing to be set, a separate panel that registers a hierarchy selection listener and offers a list of checkboxes for the currently-selected annotation might be a suitable way to do this.

But if you can say a bit more about the ultimate purpose maybe there are other better/easier ways to approach it.

#4

Thanks @Research_Associate for the links to the scripts. I will see if I can figure out how to use them for my approach (and will ask for help here if I fail).

@petebankhead, although I did not yet fully understand how to implement you “alternativly” suggestion, I feel that this might be the way to go.
Here are some more details about what we are trying to accomplish:

  • We manually mark/annotate glomeruli in kidney sections (circular structures, about 90 µm diameter).
  • There are about 5 to 25 glomeruli marked per section (=image).
  • Each glom should then be scored by a pathologist.
  • Scoring means that the pathologist should decide which features from a list of about 50 possible features apply to this particular glom.
  • Most gloms will have one or very few features, but the list of 50 is meant to contain every feature that can be envisioned for a glom. (At least one feature because there will be a “this looks normal” feature to ensure that no glom is forgotten).
  • The list of 50 features will not change.

The above mentioned idea (previous post) of a right-click pop-up might probably also be not the best solution given the length of the feature list… Our goal is to have the feature list somehow linked to the image, and more specifically, to the individual glom itself (as an annotation or whatever) and avoid messing around with Excel sheets that have to refer to the image and the particular glom within the image.

#5

You might be able to detect the glomeruli with the pixel classifier. I was able to get pretty good results with just SLICs.

You should also be able to name the annotations either in order (001, 002, etc), or by a set of features from your list, which might make finding them easier.

Color code them as well.

The Classifier script is really messy (I am not much of a coder), but that might be your best option to use as a comparison. It has both fields for values (if you want to type in a 1-10 scale) and dropdowns that reference lists. Try running it to see if it could work (assuming you made the dialog box big enough for 50 entries!).

For the alternative suggestion, you may want to check out the collection of scripts here: https://gist.github.com/Svidro/68dd668af64ad91b2f76022015dd8a45
they deal with adding measurements to things. I have tried to roughly group a lot of different script types, if you look up one level. You may find other things that are useful as starting points. Most scripts are not originally mine.

#6

So you were able to automatically detect the glom in your slides? How did you accomplish this?

#7

So this was a looong time ago now, but mostly fairly small SLICs with added features. Once M5 comes out (soon, hopefully), I think the pixel classifier would be a much better way to handle that kind of task. If you want to use SLICs, though, I recommend both adding the basic features to the SLICs, and then using a circular tile that is larger than the SLIC to provide context. More information on creating feature measurements here.

If you can host a full sized image somewhere I can take a look and see what I remember. To be fair, I was looking at fairly regular glomeruli, with an expression marker in another channel (a channel that I ignored for the purposes of finding the glomeruli). Disease states can make things much more difficult, and might require a deep learning approach with a solid set of examples for training.

#8

That would be great. Which hosting services do you recommend?

#9

I have normally used Google drive since it is free and I have several gmail accounts, but now that it is blocked at my work… shrug. It still might be your best bet.

#10

Ok how can I share the image with you? Sorry for the flurry of questions, I just stumbled onto qupath and am realtively new to this project. To give you some background, I want to extract the glomeruli as their own images but dont want to segment them manually.

#11

Right now, you pretty much can’t as I think most file sharing services are disabled here. If you do host it on Google Drive and share a link to it, I can take a look later.

I am not quite as experienced with image exporting, but you can definitely export bounding box images at full zoom based on annotations. There are… links in various places, as well as posts in Pete’s blog.


Note that with the rapidly changing code, the scripts may require some work. If you get an immediate error, it is most likely that the script is not compatible with whatever version of QuPath you are using, or it is intended for brightfield only and needs modification for multichannel images.

#12

I see, but isnt there a way to use the detection algorithms to detect structures and not just cells? I could annotate 20 or so glom for example, and then have the machine detect the similar regions (not cells). If I cant do it in qupath I will likely have to write my own algorithm to detect the images in python, which I would like to avoid for obvious reasons.

#13

At the moment you would want to use something that isn’t QuPath for that, I think. Or wait until M5 when you could try training the pixel classifier and applying it across images. I have definitely used workflows like that for Weka segmentation and in VisioPharm, but I don’t think what you want exists in QuPath.

Other software like Ilastik, Icy, or CellProfiler might handle that sort of thing better, though I am not sure how well they handle whole slide images, if that is what you are working with.

@petebankhead might have some other thoughts on this, and if you are not in a rush, I would recommend trying the pixel classifier in 0.2.0m4 to see if it will work once m5 is complete and can be saved (in m4, pixel classifiers cannot be applied except in the image where they are generated).

1 Like
#14

Thank you very much. I will take a look and see what I can find. I will certainly try the M5 function once it is released.

1 Like