Import png annotation into QuPath

Hi all!

First of all, thanks for this incredible and astonishing open-source image analysis tool for digital pathology, Pete! Life would have been wayy harder without!

Right now, I am working on a (supervised) semantic segmentation task on H&E stained WSIs, and generate image patch-label pairs by manually annotating randomly chosen patches of size 512x512 within the WSIs and applying your awesome export image tiling script from https://petebankhead.github.io/qupath/scripting/2018/03/14/script-export-labelled-images.html.
Since manual annotations are very time-consuming, I would like to use the accurate patch predictions of my current, preliminary network as initial annotations in QuPath. So my question is how to import multi-class annotations from a .png file into QuPath (while the corresponding png image patch being displayed)? Any simple way to do so?

Thanks a lot in advance and keep up the great work!

1 Like

Still kind of stumbling around as I wake up, but just to clarify, you want to take annotations in a png file and import them pixel-wise (since pngs have no metadata that I am aware of) into the exact same location as where the image was originally extracted from? Which would likely need to be done through ImageJ…

I think this would be an awesome feature to simply import patch-based png annotations + pixel coordinates (location where the patch got extracted from the WSI) into QuPath whilst the WSI being displayed (intuitively assuming equal spacings). But instead, I rather asked to import pixel-wise png annotations into its corresponding image patch (.png) of the same size (to be more precise: whilst the corresponding 512x512 image patch is displayed in QuPath)

1 Like

You may be better off approximating polygons of your annotations from each patch. Presumably, you have the x,y offset of each patch (the coordinate of the top left corner of the patch in the WSI). From here, you could approximate polygons then add the x,y offset from the patch’s location to the estimated polygon x,y coordinates and then there are scripts out there for importing these polygons as qupath annotations / detections.

If you anticipate your detections to span multiple patches, you will likely need to stitch the polygons together based on intersection of the polygons and you should definitely using a ‘sliding window’ patching approach so there is overlap between adjacent patches.

I recommend python libraries below simple because the time to get something functional with these libraries is probably much shorter than what you could do with the corresponding java libraries but that’s my opinion.

For polygon estimation-
https://docs.opencv.org/3.4/dd/d49/tutorial_py_contour_features.html

For shape based work:

Best of luck.

Hi @NBout glad you’re finding QuPath useful!

This blog post shows a way to generate annotations from an existing binary image: https://petebankhead.github.io/qupath/scripting/2018/03/13/script-export-import-binary-masks.html

And this recent discussion shows how to extend it to labeled images corresponding to a cropped/exported region: Importing binary masks in QuPath

I think the second may be closer to what you want, although if you do know the coordinates of the bounding box for your region then some combination of the scripts could get the annotations onto the original image at the proper location.

My scripts ultimately use ImageJ1 to trace boundaries. You could also use OpenCV as @nheathpatterson describes, although the results will not be identical because of a subtle difference in how contours are handled in both: https://petebankhead.github.io/qupath/technical/2018/03/13/note-on-contours.html

Neither way is right or wrong, but one might be preferable depending on what you want.

@nheathpatterson I see that Shapely is ported from Java Topology Suite, which I’m finding extremely useful in the Java world since I started using it recently. Potentially you might use Shapely on the Python side and JTS on the Java side to exchange annotations without a need for passing through a rasterized image

I’m thinking of adding import/export functionality to QuPath to handle contours in one or more of the Shapely/JTS-compatible formats… does that sound like it would be useful? And, if so, is there a preferred format (e.g. GeoJSON)…?

1 Like

@petebankhead Sounds very useful for some of the work I am doing… I ended up customizing a a very simple .csv file to deal with coordinate imports. For general formats, I think whatever format is easily exported across different topology libraries is ideal in terms of practicality. GeoJSON looks to be well established and human readable, so count that as my vote. :slight_smile:

1 Like