File size limit or pixels limit in QuPath?

Hi,
I am using QuPath 0.2.0.m9.
I am using DeltaVision Systems to image the whole slide with 20xOil lens. Tissues were stained with four markers (including DAPI). After acquiring images in all the four channels, I deconvolve, stitch and max intensity project.
I have noticed one thing regarding importing big images in QuPath, if the file (max intensity projected) size is bigger than 2GB, it gives an error. Also, in the images tab, image name appears as “Null image”.

Here is the link for one of the sample image with a size of 3.29GB. I able to open this image with DeltaVision software and Fiji. (https://drive.google.com/file/d/1x2jCD_fs7mXJt4dz8NVfzYtMJkUcvCbk/view?usp=sharing)

Snapshot from Fiji


Thanks.

The file opens for me in the 0.1.2, after a minute or two.


BioFormats I was using for this install is 6.0.1 (0.0.7)
Upgrading to BF 6.4.0 and the image still opens correctly in 0.1.2.
Checked and confirmed this is true in 1.2 as well.

I get the same error message as above in 0.2.0M9.

Though, when I try to open the brightness and contrast window in the old version, I get a Java heap space error, so I’m not really suggesting that as a work-around :slight_smile:

@shoaib.arif there are really two problems:

  1. the image appears not to be stored in a tiled multiresolution, pyramidal format - which is necessary for good performance of large images in QuPath
  2. because of 1., when v0.2.0-m9 tried to estimate the image size in bytes, the result overflows the maximum range supported by a Java integer

QuPath can be fixed to handle 2 more elegantly, but this still won’t change the fact that performance won’t be very good because of 1.

I’d suggest trying to convert the image into a pyramidal format supported by Bio-Formats, e.g. a pyramidal OME-TIFF.

Please disregard this - it doesn’t solve the problem, and it refers to some incomplete changes I made to v0.1.2 before starting on v0.2.0. There was never a v0.1.3 release, and so referring to it is unnecessarily confusing :confused:

1 Like

Thanks @petebankhead and @Research_Associate for your quick replies.
I used bfconvert to convert the image to .ome.tiff but still I failed to upload the image. I got the following error;

INFO: Bio-Formats version 6.3.1
INFO: Loaded extension Bio-Formats server options (Bio-Formats 6.3.1) (17 ms)
INFO: Loaded extension Experimental commands (350 ms)
INFO: Loaded extension ImageJ extension (55 ms)
INFO: Loaded extension JPen extension (20 ms)
INFO: Loaded extension OpenCV extensions (4 ms)
INFO: Loaded extension Rich script editor extension (334 ms)
INFO: OpenSlide version 3.4.1
INFO: Selected style: null
INFO: Starting QuPath with parameters: []
INFO: Project set to Project: Multiplex-project
WARN: Unable to obtain full image format info for file:/Users/shoaib/Documents/Fiji_focus/Sample2.tiff (class java.util.NoSuchElementException)
WARN: Temp memoization directory created at /var/folders/_r/wjrzlnh16270snnr0jfy6wc00000gn/T/qupath-memo-12184149987040879481
WARN: If you want to avoid this warning, either disable Bio-Formats memoization in the preferences or specify a directory to use
WARN: Exception adding Image null
ERROR: Import images: Failed to load one image.
INFO: 

When you created it, did you make sure it was pyramidal?
https://docs.openmicroscopy.org/bio-formats/6.0.1/users/comlinetools/conversion.html

@Research_Associate
I used the following command for the conversion.
./bfconvert -bigtiff -noflat -pyramid-resolutions 1 ~/Documents/Fiji_focus/Sample2.dv ~/Documents/Fiji_focus/Sample2.ome.tiff

Sorry, I have no idea of pyramid resolutions so I just put 1. If I put 4, than file size is increased four times.

Not exactly sure about the format of the text line, but you should expect the file size to increase at least somewhat.

Pete can correct me if I’m off base here, but as I understand it… The performance increase and QuPath’s ability to handle the file quickly is based on being able to load individual tiles when you zoom in, and load lower resolution tiles when you are zoomed out. So it requires both the pyramid and the tiles. The actual file size doesn’t matter to QuPath (I think), just the part that you are loading.

I realize the file size might matter to you, depending on your storage requirements! But that’s not what will really cause any problems once the file is pyramidal.

You can look up plenty of other posts on bfconvert to look at the way people have written their commands.

Yes, the pyramid resolutions is to essential. It should increase the file size, but by much less than 4 times.

The link in my last post contains an illustration showing what this means.

Quick follow-up, provided you have enough RAM it should be possible to work with this image in v0.2.0-m10: https://github.com/qupath/qupath/pull/418
Performance seems to me ok using QuPath’s auto-generated pyramid.

v0.2.0-m10 should be available in the next 1-2 weeks (or you could try building the current snapshot from the dev branch yourself using JDK 14).

2 Likes

Thanks @petebankhead… I will check the next update.

Hi @petebankhead,

Regarding this

How does it work? Is there some documentation about this?

I tried importing a non-pyramidal image, expecting something like the pyramids to be written into the QuPath folder, but nothing gets created. Does it mean that the pseudo-pyramid is just in the RAM while the image is open?

Thank you for your insight

1 Like

See https://qupath.readthedocs.io/en/latest/docs/tutorials/projects.html#add-images

Yes, it’s all in RAM. It addresses the fact that previously QuPath was worse than other software for images that were only moderately large. But it is no replacement for full file conversion if the image is too big for RAM.

You can also now try full conversion to pyramidal OME-TIFF via the QuPath command line. I haven’t tried it yet with very large, non-pyramidal images – if you do try it out I’d be curious if it works. Speed will be limited by Bio-Formats’ pyramid writing, but it might be a bit faster than bfconvert because of how tiles are provided (I don’t know the details of the bfconvert implementation)…

Help given at the command line. On Mac, you’d type

./QuPath-0.2.0-m11.app/Contents/MacOS/QuPath-0.2.0-m11 --help

or

./QuPath-0.2.0-m11.app/Contents/MacOS/QuPath-0.2.0-m11 convert-ome --help

for the specific convert-ome sub-command.

3 Likes

Ah perfect! That clears my misconception thank you. I will try the conversion directly in QuPath. Right now we had a small tool developed by @romainGuiet and @NicoKiaru

Glad to know that there is a tool within QuPath!

Best

Oli