QuPath not loading pyramid TIFFs properly

I’m trying to load large files (at least 3GB in size) in QuPath.
Information about the images: (i) Single channel (ii) unsigned 16 bit integers (iii) saved using libtiff (iv) at least 3GB in size
Issues with QuPath: (i) QuPath either loads them as 8-bit RGB images or (ii) warns me that they are not pyramid tiffs and asks me if I want to generate a pyramid tiff from the input image.
I have verified that I can read these images using libtiff. HALO (www.indicalab.com) can load these images properly too. I just have trouble loading them in QuPath.
Can you please help me? If it helps, I can upload a couple images to a cloud server and share the location.
Thanks in advance

As I mentioned in the duplicate issue QuPath cannot load large pyramid TIFFs · Issue #702 · qupath/qupath · GitHub we’d need an example image or two to be able to help.

Since you mention libtiff, I assume you’re writing the image at quite a low level. The best solution is perhaps to write the image so that it is compatible with Bio-Formats, since then it will be compatible with QuPath immediately – and with a lot of other software too. This discussion may be relevant: Writing QuPath/Bio-Formats compatible pyramidal image with libvips

I’ve also added the bio-formats tag to this post.

1 Like

Hi @celldiveUser, would it be possible to share one of the sample files with us to test?
Also, it might be worth using the Bio-Formats command line tools (available from Bio-Formats Downloads | Open Microscopy Environment (OME)) and run the below command to check the metadata is correct:
showinf -nopix path/to/my/file.tiff

1 Like

Thanks @dgault and @petebankhead for the immediate responses. I’m working with my IT to share large images. I will post the link here as soon as I hear back from my IT department.

1 Like

@dgault @petebankhead: Please check out https://danaherlifesciences.box.com/s/fvjfigsl32ivrjsm30mniw6evrrjs2gx

I have uploaded one image (~3GB) that was saved as a pyramid tiff using libtiff. When I try to upload it in QuPath, it prompts that the file is not a pyramid tiff and asks me if I want to create a pyramid tiff first. In the above link, I have also included two text files: (i) bftools.txt - header printed using Bio Formats tool that @dgault suggested (ii) vipsheader.txt - header printed using libvips. In the header printed using libvips, I can see that the file contains multiple pages (7) but QuPath does not seem to recognize it.

Thanks in advance for your help. Also, can you please let me know after you download this file? I currently have a limit on how many files I can upload to the above shared site. So, please let me know after you download the above file. I will delete it and upload another file that QuPath recognizes as a pyramid tiff but loads it incorrectly as a 16-bit RGB image.

@celldiveUser I’ve downloaded the file, although I’m afraid I haven’t found a way to open it or convert it into a Bio-formats friendly form.

Hi @celldiveUser ,

Looking at your tiff file I think this is the same having earlier, and ended up being resolved with help from the forum by utilizing both tiffsplit to break up the tiff initially, and then one of the bioformat conversion utilities (bioformats2raw plus raw2ome or btformats) to convert to ome. I believe the issue is that the tiff files we both start with do not use a subIFD tag to demarcate the different resolution levels but instead write them as Subfiles.

I did a quick check with your tiff to verify that tiffsplit will properly split it in to the various resolution levels, which you should then be able to convert.

Take a look at the thread below for more detail:

https://forum.image.sc/t/bfconvert-not-always-creating-ome-tiff-sometimes-just-plain-tiff/42907?u=adcorwin

1 Like

As mentioned above this appears to be a known compatibility issue between Bio-Formats and libvips pyramids. There is an open issue for this at Bioformats unable to read image pyramid generated by VIPS 8.7.0 · Issue #3300 · ome/bioformats · GitHub.

There is also a similar discussion in this thread Writing QuPath/Bio-Formats compatible pyramidal image with libvips - #6 by jcupitt, with the potential for using libvips version 8.10 which has made improvements for pyramids in Bio-Formats.

Are you generating your TIFF files in a C++ environment?

To get bioformats to read a pyramid TIFF, it must be saved with pyramid levels in subifds, and it must have a minimum amount of metadata in the IMAGEDESCRIPTION tag.

I was able to convert your TIFF with this script.

#!/usr/bin/python3
  
import sys
import pyvips

im = pyvips.Image.new_from_file(sys.argv[1])

# set minimal OME metadata
# before we can modify an image (set metadata in this case), we must take a 
# private copy
im = im.copy()
im.set_type(pyvips.GValue.gstr_type, "image-description",
f"""<?xml version="1.0" encoding="UTF-8"?>
<OME xmlns="http://www.openmicroscopy.org/Schemas/OME/2016-06"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.openmicroscopy.org/Schemas/OME/2016-06 http://www.openmicroscopy.org/Schemas/OME/2016-06/ome.xsd">
    <Image ID="Image:0">
        <!-- Minimum required fields about image dimensions -->
        <Pixels DimensionOrder="XYCZT"
                ID="Pixels:0"
                SizeC="{im.bands}"
                SizeT="1"
                SizeX="{im.width}"
                SizeY="{im.height}"
                SizeZ="1"
                Type="uint8">
        </Pixels>
    </Image>
</OME>""")

im.tiffsave(sys.argv[2], compression="lzw", tile=True,
            tile_width=512, tile_height=512,
            pyramid=True, subifd=True)

I run the script like this:

$ ./convert2.py ~/pics/SLIDE_069_1.0.4_R000_DAPI__FINAL_F.tif  x.tif
$ ~/packages/bioformats/QuPath-0.2.3/bin/QuPath-0.2.3 x.tif
... snip

The conversion script takes about a minute and 400mb of memory to run (lzw is slow), and x.tif then loads into QuPath instantly to display:

As David says, check that other thread if you need many-channel images, they are slightly different.

2 Likes

@dgault I’m generating TIFFs in a C++ environment.

@jcupitt - Thanks but now I cannot open the files in HALO because HALO supports only 8-bit OME TIFFs.

It looks like I have to use the solution suggested by @adcorwin.

Thanks again for all your replies

1 Like

You can convert your image to 8-bit mono by adding .colourspace("b-w") to that script, eg.:

im \
    .colourspace("b-w") \
    .tiffsave(sys.argv[2] ....
2 Likes

Thanks, @jcupitt. And, sorry for not being clear - I do not want to convert my images to 8-bit OME TIFFs just so they can load in HALO. I’m working with HALO to figure out a workaround. I will update this post if I find something.

1 Like