File import issues and the pyramidal tif format

Hi Image.sc Qupath community,

I’m a graduate student who learned about using qupath for digital pathology from my university’s microscopy core, and I have been investing some time into learning the software during this CoV stay-at-home period.
As a new user, I have done my best to read up on some beginner friendly material for QuPath (including Pete’s youtube page, github documentation, and the very helpful “choose your own adventure” post on this forum).
The issue I am currently running into is that I while I am able to load some TIFF files of whole slide images, for some reason QuPath fails to load specific TIFF files at all. The following error message appears: “Import Images: Failed to load # images”. This occurs even if other images from the same scan/data-set load perfectly fine.
For other images, QuPath ends up not recognizing them as pyrimadal TIFF files and also fails to import them with an “Image null” error.
My question is whether these errors could be a result of the export format or other property of the file (like tile size or total file size).
I am very novice at computer coding languages, so I’m a bit lost as to how to generate scripts to re-format my files into proper open-slide/aperio image formats that QuPath seems to better natively support.

Any help with trouble shooting this error with file loading would be greatly appreciated. Also, if anyone can point to a good primer on using openslide or bioformat API to batch convert files, that would also be very helpful.

I am happy to provide any information that would help tease these issues apart.

1 Like

Could you explain how you are generating the TIFFs that are either working or problematic (since it sounds like whatever set of steps you are taking generates both)? Are these conversions from another file type, or is some software outputting large pyramidal tiffs?

Have you tried forcing both BioFormats and OpenSlide when importing the images?

Have you tried opening the files in FIJI, either normally or through the BioFormats Importer?

Are you able to share any copies of the working and non-working files via GoogleDrive, OneDrive, Dropbox, etc?

Hi @kevin.m.gao,

It sounds like you might be encountering this problem: Several images are "Image null" when loading in QuPath
It will be fixed in the next version.

However, this won’t deal images not being recognized as pyramidal. I’d need to know more about the source of these images, and ideally an example file. It might be a Bio-Formats question rather than a QuPath one, but I’m not sure.

Hi, thank you for your reply!
Q1: Could you explain how you are generating the TIFFs that are either working or problematic (since it sounds like whatever set of steps you are taking generates both)? Are these conversions from another file type, or is some software outputting large pyramidal tiffs

For one set of files, I receive them from an outside vendor who provides whole slide scans a a TIFF Format. I’ve emailed them asking for more information regarding the file format/export process and am still waiting to hear back.

For other sets of files, I received TIFF files from the microscopy core at my university. The slide scanner that was used was a tissuegnostics tissuefax (https://www.tissuegnostics.com/products/hardware/tissuefaxs-plus). The files were exported as tiled TIFF files using either strataquest or the tissueFAX viewer software.

Q2: Have you tried forcing both BioFormats and OpenSlide when importing the images?

I have installed the open slide python library on my computer (https://pypi.org/project/openslide-python/), but haven’t had much success in using it to open files yet.
I have also downloaded the jar file for bioformats, but am not very familiar with how to use it to convert files.
I have tried clicking the box upon file import to automatically generate pyrimadal tifs, but this only works to open a small subset of files.

Q3: Have you tried opening the files in FIJI, either normally or through the BioFormats Importer?
I have not yet tried opening the files through fiji/the bioformats importer. I have fiji installed and can try installing the bioformats plugin. Would doing so allow me to open the files in qupath?

Q4: Are you able to share any copies of the working and non-working files via GoogleDrive, OneDrive, Dropbox, etc?
Yes. I have curated a few types of files that work/don’t work below.
Lung Whole slide image that loads provided by external vendor
Lung Whole slide image that does not load provided by external vendor
Lung Whole slide image provided by microscopy core that loads slowly ; server type is listed as Generated pyramid (openslide)
Lung Whole slide image provided by microscopy core that loads well; server type is listed as Generated pyramid (Bio-Formats)
Lung whole slide image that does not load provided by microscopy core that was exported by TissueFAXViewer as a tiled tif
Lung whole slide image provided by microscopy core that does not load with image null error
Log output for this last “image null” error:
WARN: Openslide: Property ‘openslide.mpp-x’ not available, will return default value NaN
WARN: Openslide: Property ‘openslide.mpp-y’ not available, will return default value NaN
WARN: Openslide: Property ‘openslide.objective-power’ not available, will return default value NaN
WARN: Openslide: Property ‘openslide.mpp-x’ not available, will return default value NaN
WARN: Openslide: Property ‘openslide.mpp-y’ not available, will return default value NaN
WARN: Openslide: Property ‘openslide.objective-power’ not available, will return default value NaN
INFO: Writing object hierarchy with 0 object(s)…
INFO: Image data written in 0.08 seconds
INFO:
INFO:

2 Likes

Hello thank you for your reply!

Good to know that this error has been encountered before and will be fixed soon.

I provided more information to @Research_Associate above as well as some example files.

Thank you both for your input into this matter. Thank you also for working on this very useful software!

1 Like

Hi Kevin,
Is it practical for you to try the workflow described here? https://www.glencoesoftware.com/blog/2019/12/09/converting-whole-slide-images-to-OME-TIFF.html
This works well for me to convert oddball tiff files into properly structured pyramided OME.tiff Files that QuPath likes.
Cheers,
Damir

5 Likes

" Caused by Array size too large: 52224 x 57344 x 3 x 1 at loci.common.DataTools.safeMultiply32(DataTools.java:1286)"

It looks like your TIFFs are too large and not pyramidal, at least in the case of the external vendor problematic slide. This is the same sort of size issue that would prevent it from being opened in FIJI. Converting it into some sort of pyramidal format should fix that particular issue.

I am not familiar with the scanner mentioned. It looks like their software has whole slide image export options, but doesn’t list in the manual what formats are available.

1 Like

@kevin.m.gao unfortunately most of the errors you report are unlikely to be fixed by the recent change I made.

I’ve added the Bio-Formats tag to this post. See also the info I wrote on the QuPath documentation here.

No specific file whole slide image readers are implemented directly in QuPath – so it depends entirely on whether Bio-Formats or OpenSlide can handle the files. As far as I’m aware OpenSlide is no longer actively maintained, so that means that to open any new, non-OpenSlide-friendly pyramidal image you will need one of the following:

  • Bio-Formats to support the image format
  • the vendors writing pyramidal images in a standard, open form that can already be read by Bio-Formats
  • the file to be converted, e.g. using the method @dsudar linked to

Any time you see ‘Generate pyramid’ the associated reader mentioned in parenthesis has only been able to read the TIFF as a single 2D plane – the entire thing needs to be read in memory, which is why it opens slowly. This is impossible for images above a certain size because of the need to store the pixels in a single Java array, which is limited in length to about 2^31 elements.

3 Likes

Hi @kevin.m.gao, looking at the provided sample files and there are a few different issues. Looking at the site for https://www.tissuegnostics.com/products/hardware/tissuefaxs-plus it appear that OME-TIFF is listed as a supported format, so it may be possible to export directly as pyramidal OME-TIFF which might solve the issue.

For the single plane images such as the 2nd file from the external vendor, as suggested above converting to a multi resolution pyramid should allow you to resolve the issue. The above mentioned conversion using https://www.glencoesoftware.com/blog/2019/12/09/converting-whole-slide-images-to-OME-TIFF.html will work but for these samples a more straightforward approach may be using the Bio-Formats command line tools (can be downloaded from https://www.openmicroscopy.org/bio-formats/downloads/).

Using the command line tools you can run the below command to convert to a multi resolution pyramid (in this case 4 resolutions each downsampled by a factor of 2):

bfconvert -noflat -pyramid-resolutions 4 -pyramid-scale 2 path/to/inputFile.tiff path/to/outputFile.ome.tiff 
4 Likes

hi dgault,

I was able to convert one of the tiff files from my external vendor and successfully load it into qupath (see screenshot below)


However, when I try to export the file from my microscopy core’s tiff files, it leads to a bfconvert java heap space error. I was in phone discussions with some of the developers of the tissuefaxs platform, and it seems that their compatability with ome.tiffs is not including pyramidal tiffs.

Any tips on how to alter the tile size to something that won’t break bfconvert?
Thank you all (@dgault, @petebankhead, @dsudar, @Research_Associate) I’m still working through trying each of your suggestions, but this discussion has been very educational so far!

Have you taken a look at these options?

1 Like

I took a look at the thread and tried setting up global system variables to increase my java heap size, but it didn’t seem to take…
Attached a screen shot of my system variables to see if I am doing the right thing…

This is rather out of my depth, but I don’t think the environment variable they are talking about is a system variable.
https://lists.openmicroscopy.org.uk/pipermail/ome-users/2018-August/007172.html
https://itk.org/pipermail/community/2014-June/002617.html

I would defer to literally anyone else about how to correctly use these though, but it might be worth a shot if you are in a hurry.

Hi Kevin,

I find that the bioformats2raw and raw2ometiff tools described in the Glencoe blog tend to be more forgiving than bfconvert and tend to run much faster as well. That may be a better route than trying to figure out weird Java heap size issues. But if you prefer using bfconvert, try to issue your conversion command like this (modify as needed for the Windows command line):

BF_MAX_MEM=5G bfconvert -noflat … etc.

See: https://docs.openmicroscopy.org/bio-formats/6.5.0/users/comlinetools/index.html
and play with the amount of memory as needed/possible in your environment.
Cheers,
Damir

2 Likes

As mentioned above, the BF_MAX_MEM option should allow you to increase the memory and hopefully overcome the heap space issue. If that still does not resolve the issue then the Glencoe tools mentioned by Damir might prove more efficient.

So I’ve decided to set up an ubuntu virtual box to run my the glencoe method inside of.
I have successfully unpacked and installed bioformats2raw, but am having some trouble getting the command to run on my input tif file.
The process spontaneously gets killed and does not give an error. The output folder has a data.n5 folder inside of it which is empty. (See below)

Does anyone here have a sense of what went wrong?

kevin@kevin-VirtualBox:~/bioformats2raw/build/distributions/bioformats2raw-0.2.2-SNAPSHOT$ export _JAVA_OPTIONS=-Xmx5G
kevin@kevin-VirtualBox:~/bioformats2raw/build/distributions/bioformats2raw-0.2.2-SNAPSHOT$ bin/bioformats2raw ‘/media/sf_Microscopy/20190921_SAVI IFNgRKO_WSI_SCOPE/Savi x INFgR KO lung-MONA Lung 15.tif’ ‘/media/sf_Microscopy/20190921_SAVI IFNgRKO_WSI_SCOPE/output’ --resolutions 6
Picked up _JAVA_OPTIONS: -Xmx5G
2020-05-18 20:06:45,909 [main] INFO loci.formats.ImageReader - TiffDelegateReader initializing /media/sf_Microscopy/20190921_SAVI IFNgRKO_WSI_SCOPE/Savi x INFgR KO lung-MONA Lung 15.tif
2020-05-18 20:06:45,932 [main] INFO loci.formats.in.MinimalTiffReader - Reading IFDs
2020-05-18 20:06:45,993 [main] INFO loci.formats.in.MinimalTiffReader - Populating metadata
2020-05-18 20:06:46,163 [main] INFO loci.formats.in.TiffReader - Checking comment style
2020-05-18 20:06:46,195 [main] INFO loci.formats.in.BaseTiffReader - Populating OME metadata
2020-05-18 20:06:46,364 [main] INFO loci.formats.in.MinimalTiffReader - Reading IFDs
2020-05-18 20:06:46,372 [main] INFO loci.formats.in.MinimalTiffReader - Populating metadata
2020-05-18 20:06:46,465 [main] INFO loci.formats.in.TiffReader - Checking comment style
2020-05-18 20:06:46,479 [main] INFO loci.formats.in.BaseTiffReader - Populating OME metadata
2020-05-18 20:06:47,061 [main] INFO c.g.bioformats2raw.Converter - Using 6 pyramid resolutions
2020-05-18 20:06:47,061 [main] INFO c.g.bioformats2raw.Converter - Preparing to write pyramid sizeX 21769 (tileWidth: 1024) sizeY 21907 (tileWidth: 1024) imageCount 3
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.janelia.saalfeldlab.n5.ReflectionUtils (file:/home/kevin/bioformats2raw/build/distributions/bioformats2raw-0.2.2-SNAPSHOT/lib/n5-2.2.0.jar) to field java.lang.reflect.Field.modifiers
WARNING: Please consider reporting this to the maintainers of org.janelia.saalfeldlab.n5.ReflectionUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2020-05-18 20:06:49,226 [pool-1-thread-1] INFO c.g.bioformats2raw.Converter - requesting tile to write at [0, 0, 0, 0, 0] to /0/0
Killed

Hi Kevin,

That bites. Would you be able to share an example file for me to try? Or even better than me, @melissa who is one of the architects of these tools, may have a much better insight what’s going on.

Btw. you can safely ignore the WARNINGS about illegal reflective access. That’s a Java 11 anomaly that apparently has been fixed for newer Java’s per @chris-allan

Cheers,
Damir

Hi,
I just realized that you had provided example files already. I first grabbed the “Lung Whole Slide image that does not load provided by external vendor” and was able to convert that fine with the glencoe tools. Let me know if you want the converted file.

Then I tried the “Lung whole slide image provided by microscopy core that does not load with image null error” and indeed it errors with:

2020-05-18 20:38:57,675 [pool-1-thread-20] INFO  c.g.bioformats2raw.Converter - tile read complete 1/1794
2020-05-18 20:38:57,676 [pool-1-thread-20] INFO  org.perf4j.TimingLogger - start[1589859534806] time[2869] tag[getTile]
Exception in thread "pool-1-thread-20" java.lang.OutOfMemoryError: Java heap space
        at loci.formats.tiff.TiffParser.getSamples(TiffParser.java:1082)2020-05-18 20:38:58,106 [pool-1-thread-2] INFO  c.g.bioformats2raw.Converter - tile read complete 2/1794
2020-05-18 20:38:58,107 [pool-1-thread-28] INFO  c.g.bioformats2raw.Converter - requesting tile to write at [9, 0, 0, 0, 0] to /0/0

it continues but generates loads of errors along the lines of:
Exception in thread “pool-1-thread-5” java.lang.OutOfMemoryError: Java heap space
and after a while it just hangs.

Hopefully the Glencoe folks can be of more help with those tissuegnostics images.

Cheers,
Damir

1 Like

I strongly suspect the Killed is the Linux kernel OOM killer killing your process due to its memory usage. If you run dmesg you will likely see messages to that effect.

The metadata for Savi x INFgR KO lung-MONA Lung 14.tif for the purposes of further discussion:

TIFF Directory at offset 0x2706dac4 (654760644)
  Subfile Type: (0 = 0x0)
  Image Width: 22728 Image Length: 25717
  Resolution: 300, 300 pixels/inch
  Bits/Sample: 8
  Compression Scheme: LZW
  Photometric Interpretation: RGB color
  Samples/Pixel: 3
  Rows/Strip: 25717
  Planar Configuration: single image plane
  Predictor: horizontal differencing 2 (0x2)

The data is LZW compressed, has X-Y dimensions of 22728 x 25717 and is 24-bit colour. As constructed, Bio-Formats, and by extension bioformats2raw will require at least 1.8GB of Java heap space just to decompress this data. It is likely to require several times that to process the data for conversion into the raw format. The fact that it works at all and does not just immediately exit in a form similar to:

is only due to the data being small enough to entirely fit into the maximum Java array size of 2^31 - 1.

Furthermore, there is limited plane caching in bioformats2raw causing this operation to need to be repeated over and over again. Making conversion of this data likely to take many 10s of minutes if not hours.

Fundamentally, the bioformats2raw pipeline is currently not designed to deal with data that is this poorly formed.

We’ve had a similar discussion on what the OME and Glencoe teams are prepared to do with similar data on this Bio-Formats issue:

You can certainly try our tiff2raw (https://github.com/glencoesoftware/tiff2raw) but there is limited support we can offer and you will need many, many GBs of RAM to perform conversions of data that is this poorly formed.

3 Likes

The null error should be resolved in v0.2.0-m12 (at least it opened for me… albeit not terribly quickly, since it does all need to be read into RAM).

2 Likes