Using QuPath to convert (very large) tif to ome.tif and add metadata

Usage & Issues
#1

Hi all,

I wanted to check if I am missing something obvious.
Goal: Use CLI to take a non-pyramidal, no metadata, tif and convert to ome-tif (pyramidal) and add in metadata.

The CLI gives a very clean way to perform the conversion from Python, but the resulting OME-TIF has no pixel size metadata, as expected, and I did not see a way to add metadata in during the conversion process.

Is there any way to modify an image’s metadata through a script, such that it actually modifies the original file (not the way QuPath reads it)? I am assuming not, but wanted to check just in case.

The original .tif runs into a Java array size error if I try to open it on the CLI and edit it + use a script to perform the OME-TIF export. However, I can probably open the pyramidal OME-TIF, add metadata, and then re-export that file. It requires exporting the data twice, though. I considered calling ImageJ to apply the metadata through Image->Properties…, but the image will run into the same Java array size error.
There are apparently issues using bfconvert from Python that I do not fully understand, something about multiple javabridges (I am not handling the Python side of things, though).

It also might be possible to edit the metadata of the ome-tif directly through Python, I just wanted to explore QuPathy options first.

Cheers,
Mike
PS I can probably provide the .tif file through Google Drive if desired.

#2

Hmmm, I don’t think you’re missing anything obvious. I’ll let you know if I come up with something, but it sounds like you’ve already tried most things.

You may want to check out this for a libvips-based method: Converting an existing image into an Openslide compatible format - Andrew Janowczyk

If that doesn’t work, can you elaborate on this?

It sounds like the QuPath-based method that has the most potential… although quite how to apply it successfully I don’t know for sure.

2 Likes
#3

Once I am home again I can try to get the exact error, but it looks like the same Java array size error you see in ImageJ when opening a too-large image (18000x22000 pixels in this case, error seen when running the console version of the QuPath exe). I was not too surprised by this since the default server when dragging and dropping this particular image into QuPath -no project- is ImageJ (which does work, though it actually takes longer to load into QuPath than to perform the conversion and load the OME-TIF in!).

I am guessing when dragging and dropping the image into QuPath, QuPath makes some decisions about how to handle files like these - which may not take place trying to run a script through the CLI.

#4

Maybe adding it to a project first, specifying the server there, and passing the project?

Alternatively you can run the script without a project, and manually create the server with the URI of interest. This shows the general idea:

Since the server you create won’t be the ‘current’ image, the usual scripting helper methods won’t quite do the trick for setting the metadata in a single line. But here’s the source that may help:

You should be able to call if it you wrap your ImageServer in an ImageData – i.e. new ImageData<>(server) – and pass it as the first argument to setPixelSizeMicrons.

But libvips may be preferable to all of this.

1 Like
#5

I suppose if I can figure out a script for creating projects (I vaguely recall that coming up before? will need to look). The overall idea is that this image doesn’t really matter. Pull in a basic TIF (an overview of a slide), annotate on it, then the tiles representing that annotation get sent off in a CSV file back to the scope, ideally as quickly as possible.

Maybe having a dummy project that images can be loaded into and out of would be easier? Not sure. The images will all be of the same “type” (bit depth etc), but the hope is as smooth of a “use it and then lose it” pathway as possible.

#6

Ah, I was wrong, yet again! It was not “any” script that was causing the Java array size error, it was specifically the one I was calling to try to export the image as an OMETIFF, which, surprise surprise, tries to convert the image into a ImageJ img. The image also caused the array error when I tried to drag and drop it into QuPath but did not select Yes for the “do you want to build a pyramid” option.

I am running into the same trouble getting the name of the file or image data while running from the CLI, so I have not been able to reproduce the java heap space error running from the command line.

Source of the script I was trying to use.

I still need to figure out how to get the server and image name without using the getCurrentServer or currentImageData, but I will try and work it out.
For anyone else who finds this, I was making the mistake of putting the image before the script in my test case, resulting in getCurrentServer returning null for my CLI tests (but not my Python script).

Other than that, the getCurrentServer().getFile() does not seem to work for large TIFF files. However, you can use:
path= getCurrentServer().getBuilder().getURIs()[0].toString()
To get mostly the file name, though it looks a bit weird with the “file:\” in front of it, so that can be stripped off with:
def pathInput = path.substring(6,)

#7

For anyone else who wants to do something similar, the code to take the tif, save it as a pyramidal ome.tif, and add in fixed metadata ended up working out as (although not necessarily the best way):

// source https://forum.image.sc/t/issue-with-bfconvert-and-big-jpg-images/43276/2
// and https://forum.image.sc/t/using-qupath-to-convert-very-large-tif-to-ome-tif-and-add-metadata/48636/4
// With grand struggles from Mike Nelson
// QuPath 0.2.3

// file type to convert
extension = "tif"
server = getCurrentServer()

// I found that server.getFile() fails for this type of image, and I needed to use
// For other file types, this might not all be necessary.
path= server.getBuilder().getURIs()[0].toString()
imageData = getCurrentImageData()
//Strips out the leading file:// from the URI and the file extension
baseFilePath = path.substring(6, path.lastIndexOf(".")+1)
def pathInput = baseFilePath + extension
def pathOutput = baseFilePath + "ome.tif"

println 'Reading image...'
//def img = ij.IJ.openImage(pathInput).getBufferedImage()
//def server = new WrappedBufferedImageServer("Anything", img)
def metadataNew = new ImageServerMetadata.Builder(server.getMetadata())
    .pixelSizeMicrons(1.16, 1.16)
    .zSpacingMicrons(1)
    .build();
    
imageData.updateServerMetadata(metadataNew);
println 'Writing OME-TIFF'
new OMEPyramidWriter.Builder(server)
        .parallelize()
        .tileSize(512)
        .scaledDownsampling(1, 4)
        .build()
        .writePyramid(pathOutput)
        
println 'Done!'


import qupath.lib.images.writers.ome.OMEPyramidWriter
import qupath.lib.images.servers.*
import javax.imageio.*
import qupath.lib.images.servers.ImageServerMetadata;