Macro Image displaying annotations

Hi All,

I’m trying to save a whole slide image using the Image> Macro selections. It’s giving me a great image of the whole slide but it’s not displaying the annotations completed on QuPath, just the HE prior to annotation.

I’ve tried every other export option, and file saving option. Has anyone a solution to this?

Thanks in advance!
A

Can you specify more exactly what steps you’re using, and include any extra code/script/macro that is involved?

Also, does this need to work in batch or is it feasible to export the images one at a time interactively?

Finally, do you need ImageJ ROIs that can be further manipulated or do you only require a RGB image that looks the same (i.e. what you can get by copying a snapshot)?

Since I never really use the macro functionality myself the code in QuPath for this purpose is a bit neglected. But if it doesn’t do the job you need here should be another way if it can be precisely defined.

Ok I’m going to reply to each point:

Can you specify more exactly what steps you’re using, and include any extra code/script/macro that is involved?
(A)Honestly, I wouldn’t trust myself enough to attempt any extra code/script , so we are just using the basic functionality of the app.

Also, does this need to work in batch or is it feasible to export the images one at a time interactively?
(A)Exporting images one at a time would be preferable at this moment in time, to allow for QC. I wouldn’t rule our batching but that’s a problem for another day.

Finally, do you need ImageJ ROIs that can be further manipulated or do you only require a RGB image that looks the same (i.e. what you can get by copying a snapshot)?
(A)Yes, I only need an RGB image that looks like the snapshot,. But I need it superimposed onto the original slide scan.

Since I never really use the macro functionality myself the code in QuPath for this purpose is a bit neglected. But if it doesn’t do the job you need here should be another way if it can be precisely defined.
(A)I’m new to the software so any help would be appreciated, neglected or otherwise! J

Ah ok, I thought you were running an ImageJ macro.

You could try Edit → Copy to clipboard… → Current viewer or its counterpart under File → Export snapshot. This will however export exactly what is shown in the viewer, and so you might need to crop it a bit.

In v0.2.0-m5 I’ve revised the image export to make it possible to export images with/without an overlay, which I think is more what you may want. But it’s not released yet… perhaps next week, but not certain yet.

1 Like

Hi Pete,

Thanks for that, I still don’t think that’s what we need. We need the annotations on the original slide image scanned so that we can print it to scale to allow for macrodissection. Hence the reason we are using the Image Tab> macro function. Does that make more sense?

Sorry!

Ah ok, that’s the macro (overview) image stored within the file. There’s also an ImageJ macro runner that I’d initially thought you were using.

The macro images aren’t at all standardized across scanners and I don’t know of any reliable / straightforward way to figure out how to transfer between the coordinate system of the whole slide image and whatever macro image your scanner can create.

Two options are:

  • Open the macro image in QuPath and annotate it (but you’ll only see the low-resolution information while annotating… so don’t gain a great deal from using QuPath compared to any other software that doesn’t handle whole slide images)
  • Use image registration to figure out how to transform annotations from the whole slide image to the macro image

The first can be achieved by saving the macro image as a PNG (that option should already exist in QuPath when you open it).

The second one is rather more complicated… I once wrote a script to go the opposite way (from a low-resolution image with a macrodissection boundary to the whole slide image) but it wasn’t exactly user-friendly and didn’t work in all cases.

Although it’s still not totally clear to me why you need the macro image as opposed to a scaled version of the whole slide image… which is certainly easier, and gives more control over the export resolution.

(Admittedly I say that as someone who doesn’t get involved in macrodissection so… I don’t know what exactly is needed)

Yes, that’s what I mean. So, we’ve tried the first way in our initial attempts but it’s not accurate enough. Option 2 seems like it might be what we need. The scaled version doesn’t give us the “real life” size which is what we need to use as a comparison to an actual slide.

How would I implement option 2?

You’d need a script that

  • Extracts the macro image, and the full resolution at a similar scale
  • Calculates a rigid transform to go from one to the other (e.g. using OpenCV’s findTransformECC)
  • Copies any annotations from the full resolution and applies the transform to draw them on top of the macro image

It is a fairly involved task that requires getting rather deep into scripting. If that part is ok, then how difficult the task remains depends somewhat on how easy it is to do the registration; the macro image might contain a lot of other bits that make automated alignment challenging.

An alternative ‘manual’ approach might be to use the script I posted to export the scaled full resolution image, save the macro image as a PNG, and then put one on top of the other (e.g. in PowerPoint) and adjust them until they match… less elegant, but arguably easier.

1 Like

Haha, exactly what I was thinking. Though I was already confused once, a while back, that there was an image type called “macro.”

Hi Pete,

We’re currently working our way through this problem, and are definitely getting somewhere! Ideally, however, we would like to find a way to capture the HE image, including annotations, with a specific desired line thickness, and to do this programmatically (but manually will do the job). Is there a way to save the current viewer content while preserving the annotation line thickness in the saved .png image? At the moment it always shows the lines as the default thin line, while we need to somehow have it output thicker and any way to control this in a script or otherwise seems to be eluding us. Any light you can shed on this would be welcome!

Edit: I meant to say we are trying to preserve the annotation line thickness in the exported region image, not the viewer snapshot! Sorry for any confusion.

Thanks,

Ben

@Benjamin_Smylie the issue with the export is that the annotation line thickness depends on the magnification at which the image is being displayed in the viewer - and so if the image is being exported in a script, and can’t be assumed to be the image currently in the viewer, the appropriate line thickness can’t be determined.

There are so many possible ways to export things from QuPath and I don’t know what the right starting point is in your case. Can you specify more exactly how you would like the final output to look?

For example, the annotation could be drawn like it is in the viewer (same display settings, color, transparency…) or just a big black line with a fixed thickness. The second option is a bit easier.

Also, the image could be exported at a fixed resolution (in microns per pixel) or a fixed downsample factor.

1 Like

Hi Pete, thanks so much for your reply.

I didn’t realise that there was a link between the current viewer magnification and the exported line thickness, and I can’t replicate it. BUT I have since discovered that the export downsample factor can be used to this effect, and this both simplifies the process for us and provides the control we needed!

Now all that we really need in order to automate the process is to have a way to capture the RGB image that would be exported via the ‘export image region’ command, and to do this via scripting. Unfortunately I’m having no luck finding this in the API. This is a small aspect of the overall process and I think we have it pretty much running optimally now apart from a few manual steps. At the moment we’re exporting the file manually and reading it back in to be used in the detection/alignment portion of the script.

If you know of a way to access this functionality that we have missed, that would really help immensely.

Hi @Benjamin_Smylie

I meant that the line thickness in the viewer depends upon the downsample in the viewer - and that’s why it isn’t the same as the line thickness during export generally. Although I don’t know exactly what export script you’re using…

Something like this might do what you want (in v0.2.0-m6).

double downsample = 100.0

def server = getCurrentServer()
def request = RegionRequest.createInstance(server, downsample)
def img = server.readBufferedImage(request)

float thickness = 2

def g2d = img.createGraphics()
g2d.setColor(java.awt.Color.BLACK)
g2d.scale(1.0/downsample, 1.0/downsample)
g2d.setStroke(new java.awt.BasicStroke((float)(thickness * downsample)))
getAnnotationObjects().each { g2d.draw(it.getROI().getShape()) } 
g2d.dispose()

def path = buildFilePath(PROJECT_BASE_DIR, 'something.png')
writeImage(img, path)
2 Likes

Ah, that seems to be exactly what was needed! I didn’t think of looking into the methods of the BufferedImage object. Thanks so much again for this and for your hard work on QuPath!

2 Likes