Issue with opening big planar 2D images with Bioformats

I have some code for opening CZI into Imglib2 CellImg:

This code is part of the Labkit FIJI plugin. You can download it from the Labkit update site. But this code doesn’t display the CellImg in a normal image window. This would fail. It instead loads the image into the BigDataViewer based Labkit plugin.

Based on that code one could write a BioFormats import plugin capable for large images. But I’m not experienced with image import plugins so far.

1 Like

@MichaelC Just to clarify, the Bio-Formats library can open and process these images and we have spent a lot of development time adding API support for tiled and multi resolution images over the past couple of years. This is how other applications such as OMERO and QuPath are able to open such images.

The problem is opening them as a single plane which is what happens when using it via the Bio-Formats Importer FIJI plugin at present. I fully understand how this can be frustrating for those who use the plugin as part of their workflow at present.

2 Likes

@MichaelC Echoing what @dgault said… very large 2D images pose rather specific problems, which more RAM won’t solve. But there are now at least three open source applications on this forum designed to handle these (QuPath, Orbit, Cytomine) - all of which (to the best of my knowledge) depend upon the hard work of the BioFormats team for much of their file format support. These applications have all been around for more than two years.

This is not to mention the important work being done with ImageJ2 and BigDataViewer, and the fact that Zeiss have made a CZI reader open source (here).

I don’t know if these would work immediately with your images, and there might be a need to convert them to image pyramids. But there Bio-Formats are leading the work to make this possible in open software.

1 Like

Hey all,

great hint @maarzt!

Is this piece of code part of the current labkit on the update site? I’m asking, because I just tried opening the file with labkit, and it throws this error message:

java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: 2097152
	at net.imglib2.labkit.utils.CheckedExceptionUtils.run(CheckedExceptionUtils.java:12)
	at net.imglib2.labkit.utils.ParallelUtils.executeInParallel(ParallelUtils.java:83)
	at net.imglib2.labkit.plugin.CziOpener.openImage(CziOpener.java:110)
	at net.imglib2.labkit.plugin.CziOpener.openInputImage(CziOpener.java:94)
	at net.imglib2.labkit.plugin.CziOpener.openWithDialog(CziOpener.java:68)
	at net.imglib2.labkit.plugin.LabkitImportPlugin.openImage(LabkitImportPlugin.java:48)
	at net.imglib2.labkit.plugin.LabkitImportPlugin.run(LabkitImportPlugin.java:39)
	at net.imglib2.labkit.plugin.LabkitImportPlugin.run(LabkitImportPlugin.java:33)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: 2097152
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at net.imglib2.labkit.utils.ParallelUtils.lambda$executeInParallel$7(ParallelUtils.java:85)
	at net.imglib2.labkit.utils.CheckedExceptionUtils.run(CheckedExceptionUtils.java:8)
	... 16 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 2097152
	at net.imglib2.labkit.plugin.CziOpener$MyReader.readToInterval(CziOpener.java:253)
	at net.imglib2.labkit.plugin.CziOpener$MyReader.access$400(CziOpener.java:212)
	at net.imglib2.labkit.plugin.CziOpener.lambda$openImage$1(CziOpener.java:109)
	at net.imglib2.labkit.utils.ParallelUtils.lambda$null$0(ParallelUtils.java:38)
	at net.imglib2.labkit.utils.ParallelUtils.lambda$null$5(ParallelUtils.java:69)
	... 4 more

Indeed! So much great work has been done, by so many great software engineers! But we can’t exploit it, if we can’t open the files. Let’s fix this bug in ImageJ or Zen or BioFormats or SCIFIO or wherever. I don’t care who is responsible.

I’m trying to get that file shared asap. Stay tuned.

In the meantime: Have a nice weekend everyone!

Cheers,
Robert

The correct place to fix the bug is SCIFIO.

The person currently responsible for maintaining SCIFIO is me. Unfortunately, I have too many other things on my plate right now to investigate this issue (or really any SCIFIO issue) in the short term. The good news is that the funding situation for ImageJ2 is improving, and the team at LOCI is in the process of growing, with at least one dedicated bug-fixer currently being trained. So SCIFIO development may move forward again soon.

In the meantime, if you have sufficient time and motivation, you could investigate the issue(s) yourself and file PRs for any fixes you develop.

@maarzt Thank you for posting working code—much appreciated. That said, I wish you were able to use (and improve if necessary) the intended common infrastructure of SCIFIO, rather than feeling the need to develop a format-specific, Bio-Formats-specific class for this special case.

4 Likes

Hi, I am sorry to hear that you are not happy with your CZI files.

But as mentioned above the limitations or issues are not caused by the CZI but are rather are fundamental limit.

Zen can handle those images and internally we use a Tiling Client allowing us to apply image analysis and Machine Learning algorithms to large tile images like this one.
Addionally we published an open source library that allows to read such files.

A python wrapper is still in testing and a integration in BioFormats (unfortunately) does not exist. Maybe the community wants to jump in and help?

But I am happy to file a bug entry for the CZI internally, if you let me know what the CZI issues is.

1 Like

Hey @sebi06,

again, I’m not so much interested in who is responsible. I just would like to process that file - or files of that kind. I’m in the unfortunate situation, I have an ImageJ/Jython script which can analyse that kind of image, but I cannot open the file.

The imglib2/ImageJ2 community worked very hard on overcoming this fundamental limit. We’re almost there. Even processing big 2D images on graphics cards using ImageJ macro appears feasible :wink:

:heart: Thanks for the link! And for making this available! I will check if I can knit a Java wrapper around.

Oh yes, please. Maybe it is an easy thing. Btw, the meta header of the file contains Information|Application|Name #1,ZEN 3.1 (blue edition). Maybe that helps as well. As soon as I can share a file, I let you know.

That sounds fantastic! Congrats! Looking forward to see things evolving.

Thanks so much everyone for your support!

Cheers,
Robert

I’m a bit confused by this thread… as @haesleinhuepf mentions here, QuPath can open the image. So too (presumably) could Orbit and Cytomine. If you want to work with the data, you can already - including in Fiji (albeit not yet directly).

Therefore there doesn’t appear to be any problem whatsoever with Bio-Formats or the CZI file itself.

I get the same error with a completely different AxioScan brightfield image, which supports this suspicion.

@haesleinhuepf Not sure if I’m missing something, but I think the conclusion is that you want to open the image directly with imglib2/ImageJ2, not anything else - which means SCIFIO, as Curtis has said.

1 Like

Hi Robert,

so what is the actual bug regarding the CZI, e.g. how should I describe the problem that one of our developers understands the actual issue. If the is a fundamental limit of the “TIFF format” and the used libraries, what should ZEISS do about it?

From what I read here it is still no clear to me (sorry if I overlooked something), what needs to be changed on the side of the CZI to make this work. Or do you mean to have another look into exporting large tile images as single BigTIFFs? This could be rather a feature request than a bug, but I can check.

1 Like

Guys, think high level: I’m a user who has a CZI file and cannot open it in Fiji. Please don’t think about who is responsible and try to think about how to solve the problem.

Of course, I do have advanced coding skills and could hack something to make it work for my particular case. However, is this how one is supposed to work with such a file? If yes, I have to recommend my collaborator to not produce such files. I’m very sorry.

Honestly, we talking about an array of 44000 elements which each contain an array of 39000 pixels. The 3.7 GB file can also fit the RAM of virtually all recent computers. Over the past years so much great work has been done for big data image processing, but it’s all useless if we can’t open such “big” files.

@haesleinhuepf It’s pretty depressing that you consider the multiple alternative ways to handle such images “all useless” if you can’t open them in Fiji in exactly the way you would like to.

1 Like

Which way opening it in Fiji do you refer to? I tried all suggested solutions and posted the error messages.

Links here. Unanswered questions here.

See also this.

3 Likes

The image is saved as pyramind:

I can open and process series 2. A nice workaround, but no solution to the general problem.

Sorry for not answering earlier. I’m processing the image on the GPU: Spot detection, segmentation and generating a Voronoi-like map to analyse neighborhood relationships between many cells. The last step utilizes custom OpenCL kernels of operations which don’t exist in ImageJ; implementing them in Java for ImageJ or QuPath might be some effort. Furthermore, the image fits in the GPU memory and can be processed there. No issue. Afterwards, I use ImageJ-ROIs to visualize analysis results and to quantitatively analyse pixels in sub-regions of the big image. I’m afraid introducing tiling in this workflow makes the whole thing quite complicated. Merging segmentations between and lines across tiles would be the challenge.

Cool, thanks for the link. I didn’t know that this library can convert files. I just tried it and unfortunately, the error message is:

java.lang.UnsupportedOperationException: Unhandled 16 bits RGB images
	at ch.epfl.biop.bdv.bioformats.bioformatssource.BioFormatsBdvSource.getBioformatsBdvSourceType(BioFormatsBdvSource.java:359)
	at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.lambda$null$2(BioFormatsImageLoader.java:102)
	at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
	at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
	at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.lambda$new$3(BioFormatsImageLoader.java:85)
	at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
	at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
	at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.<init>(BioFormatsImageLoader.java:58)
	at ch.epfl.biop.bdv.bioformats.export.spimdata.BioFormatsConvertFilesToSpimData.getSpimDataInstance(BioFormatsConvertFilesToSpimData.java:210)
	at ch.epfl.biop.bdv.bioformats.export.spimdata.BioFormatsConvertFilesToSpimData.getSpimData(BioFormatsConvertFilesToSpimData.java:222)
	at ch.epfl.biop.bdv.bioformats.command.OpenFilesWithBigdataviewerBioformatsBridgeCommand.run(OpenFilesWithBigdataviewerBioformatsBridgeCommand.java:52)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
java.lang.UnsupportedOperationException: Unhandled 16 bits RGB images
	at ch.epfl.biop.bdv.bioformats.bioformatssource.BioFormatsBdvSource.getBioformatsBdvSourceClass(BioFormatsBdvSource.java:378)
	at ch.epfl.biop.bdv.bioformats.bioformatssource.BioFormatsBdvOpener.getConcreteSource(BioFormatsBdvOpener.java:338)
	at ch.epfl.biop.bdv.bioformats.bioformatssource.BioFormatsBdvOpener.getConcreteAndVolatileSources(BioFormatsBdvOpener.java:415)
	at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsSetupLoader.<init>(BioFormatsSetupLoader.java:59)
	at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.getSetupImgLoader(BioFormatsImageLoader.java:132)
	at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.getSetupImgLoader(BioFormatsImageLoader.java:24)
	at bdv.BigDataViewer.initSetupNumericType(BigDataViewer.java:260)
	at bdv.BigDataViewer.initSetups(BigDataViewer.java:292)
	at bdv.util.BdvFunctions.show(BdvFunctions.java:223)
	at bdv.util.BdvFunctions.show(BdvFunctions.java:203)
	at ch.epfl.biop.bdv.bioformats.command.OpenFilesWithBigdataviewerBioformatsBridgeCommand.run(OpenFilesWithBigdataviewerBioformatsBridgeCommand.java:53)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
[ERROR] Module threw exception
java.lang.NullPointerException
	at ch.epfl.biop.bdv.bioformats.bioformatssource.BioFormatsBdvOpener.getConcreteAndVolatileSources(BioFormatsBdvOpener.java:417)
	at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsSetupLoader.<init>(BioFormatsSetupLoader.java:59)
	at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.getSetupImgLoader(BioFormatsImageLoader.java:132)
	at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.getSetupImgLoader(BioFormatsImageLoader.java:24)
	at bdv.BigDataViewer.initSetupNumericType(BigDataViewer.java:260)
	at bdv.BigDataViewer.initSetups(BigDataViewer.java:292)
	at bdv.util.BdvFunctions.show(BdvFunctions.java:223)
	at bdv.util.BdvFunctions.show(BdvFunctions.java:203)
	at ch.epfl.biop.bdv.bioformats.command.OpenFilesWithBigdataviewerBioformatsBridgeCommand.run(OpenFilesWithBigdataviewerBioformatsBridgeCommand.java:53)
	at org.scijava.command.CommandModule.run(CommandModule.java:199)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:127)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:66)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:228)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

I hope I can share the file soon.

Thanks again for your ideas and support!

Cheers,
Robert

True, but nor is opening series 1 given the potential for that to require huge memory and computational resources - recalling that a lot of image processing requires duplicating images / often increasing bit-depths.

Where you can’t downsample, you can crop. Sometimes that is also enough.

Sidenote: If you used QuPath, you could export at an intermediate resolution… not necessarily downsampling by the full factor of three, and optionally cropping out any whitespace to give yourself the largest single area at the highest manageable resolution - constrained by the (current) Java/Fiji limitations.


Handling tiles is certainly a challenge, but then often so is defining a manageable workflow without them. Without tiling, you’ll place stronger limits on your processing complexity, required hardware and absolute image size… possibly only postponing the problem for another day.

Not sure if you’re generating your Voronoi-like map as an image, but with OpenCV/Java Topology Suite you could do this using vertices instead. At least, this is how I would prefer to do things with large images.

2 Likes

Yes. That’s it! And one can do this even from a macro.

filename = //...
pyramid_factor = 3;

run("Bio-Formats", "open=" + filename + " color_mode=Default rois_import=[ROI manager] view=Hyperstack stack_order=XYCZT series_2");

setTool("rectangle");
waitForUser("Please outline the region to analyse!\n\nClick ok afterwards.");

Roi.getBounds(x, y, width, height)
x = x * pyramid_factor;
y = y * pyramid_factor;
width = width * pyramid_factor;
height= height * pyramid_factor;

run("Bio-Formats", "open=" + filename + " color_mode=Default crop rois_import=[ROI manager] view=Hyperstack stack_order=XYCZT series_1 x_coordinate_1=" + x + " y_coordinate_1=" + y + " width_1=" + width + " height_1=" + height);

That’s a wonderful workaround. And it works on my file.

Thanks a lot for your hints!

Cheers,
Robert

Dear Robert,

As alternative solution, I do the following with huge 2D images:

Open .czi in Python and save as pyramid-tiff as describe here:

The resulting file can be opened in QuPath.

There I do all drawing, viewing & user interaction.

I export all I need with a groovy script (can nicely be done per project).

I import all drawings in Python and do all further analysis on the pyramid tiff file.

If need, I call Fiji headless from Python.

Kind regards

Tobias

4 Likes

My complaint about a problem opening large fused multichannel images seems to have hit a nerve. The CZI images open fine in Zen, but Zen will not export them as TIF. Our correspondence with Zeiss regarding this matter dates to at least
early 2018. There have been new releases of Zen Blue 2.6 since we reported the issue. BioFormats, which is a fantastic resource that we rely on heavily, does not open this particular version of CZI. We uploaded example files to BioFormats in mid-2018 and
as of version 6.0.2, they do not open properly. This includes CZI saved with Zen 2.5 without pyramid and files saved with Zen 2.6 with pyramid. I have not pursued this because, although it was a high priority back in 2018, it no longer is a priority for
us.

But I think the responses here require further discussion.

Most biologists need solutions that are simple. This means turnkey solutions in the software package they know. The power of ImageJ is that is does everything in in one place. The power of the Fiji version is that it is supposed to have
additional features automatically available. To do a simple task, opening a file, having to hunt down a Python wrapper for someone’s custom code on Github is not a solution; it is more of a problem. Another open source library is not a solution for biologists.
A seamless transparent method for opening images is the necessary solution.

Also, this statement is incorrect: “above the limitations or issues are not caused by the CZI but are rather are fundamental limit.”

If Zeiss made fixing this problem a priority, then Zen 2.6 would successfully export the fused stitched images as TIF. They could even add a modicum of metadata, such as spatial scale. There is nothing “fundamental” that cannot be fixed
here. We last spoke to Zeiss about this last month. As we pay Zeiss a lot of money for service contracts and new instruments, they really have a responsibility to 1.) fix this problem of exporting files and 2.) make CZI available so that the data are useable
in other programs.

1 Like

My complaint about a problem opening large fused multichannel images seems to have hit a nerve. The CZI images open fine in Zen, but Zen will not export them as TIF. Our correspondence with Zeiss regarding this matter dates to at least early 2018. There have been new releases of Zen Blue 2.6 since we reported the issue. BioFormats, which is a fantastic resource that we rely on heavily, does not open this particular version of CZI. We uploaded example files to BioFormats in mid-2018 and as of version 6.0.2, they do not open properly. This includes CZI saved with Zen 2.5 without pyramid and files saved with Zen 2.6 with pyramid. I have not pursued this because, although it was a high priority back in 2018, it no longer is a priority for us.
But I think the responses here require further discussion.
Most biologists need solutions that are simple. This means turnkey solutions in the software package they know. The power of ImageJ is that is does everything in in one place. The power of the Fiji version is that it is supposed to have additional features automatically available. To do a simple task, opening a file, having to hunt down a Python wrapper for someone’s custom code on Github is not a solution; it is more of a problem. Another open source library is not a solution for biologists. A seamless transparent method for opening images is the necessary solution.
Also, this statement is incorrect: “above the limitations or issues are not caused by the CZI but are rather are fundamental limit.”
If Zeiss made fixing this problem a priority, then Zen 2.6 would successfully export the fused stitched images as TIF. They could even add a modicum of metadata, such as spatial scale. There is nothing “fundamental” that cannot be fixed here. We last spoke to Zeiss about this last month. As we pay Zeiss a lot of money for service contracts and new instruments, they really have a responsibility to 1.) fix this problem of exporting files and 2.) make CZI available so that the data are useable in other programs.

1 Like

I completely agree on the lack of metadata for the Zeiss export.

I haven’t seen any Zen 2.6 files just yet, so not sure what is going on there. I haven’t had problems with BioFormats opening large (multigigabyte) 2.5 CZI files though.

While I do wish that Zeiss would allow the export of pyramidal ome.tiff files, QuPath can both open most CZI files and can export pyramidal ome.tiff files in a variety of ways, for both brightfield and fluorescent images. If the file is not already pyramidal, it can be slow/maybe memory intensive, but QuPath should build the pyramid itself.
image
image

If the file is big enough, though, neither the original CZI nor the ome.tiff, or any other image type will open in FIJI at the moment due to the java array size limits. QuPath and Napari (with the right import settings) can open them, but they do not open the entire image at once.

Once FIJI has the ability to load lazily (I think the term is), it should be able to open large files as well. For the moment though, the file type doesn’t matter as much as the array size. @eliceiri mentioned that this should be possible in the next release. Fingers crossed, and hope this helps.

2 Likes