Can I open a large jpeg 2000 file with BioFormats and ImgLib2 from Java API?

I have a large jpeg 2000 image (42848 x 32960 x 3 pixels) that is made from lots of tile images of 2048x2048 images with 200 pixels overlap at every side (20 rows x 26 columns x 1648 pixels x 1648 pixels).

This jpeg 2000 file is MBF’s customized format from Stereo Investigator. The file size is 848 MB and is compressed with a ratio of 10.

Fiji can open the images saved in the same format but much smaller size with BioFormats Importer successfully.

However, the same approach does not work for the large image. It issues the following error instantly.

(Fiji Is Just) ImageJ 2.0.0-rc-69/1.52p; Java 1.8.0_172 [64-bit]; Linux 4.9.6-200.fc25.x86_64; 208MB of 145252MB (<1%)
 
java.lang.IllegalArgumentException: Array size too large: 42848 x 32960 x 2
	at loci.common.DataTools.safeMultiply32(DataTools.java:1286)
	at loci.common.DataTools.allocate(DataTools.java:1259)
	at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:160)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
	at loci.formats.DimensionSwapper.openBytes(DimensionSwapper.java:233)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
	at loci.plugins.util.ImageProcessorReader.openProcessors(ImageProcessorReader.java:186)
	at loci.plugins.in.ImagePlusReader.readProcessors(ImagePlusReader.java:422)
	at loci.plugins.in.ImagePlusReader.readPlanes(ImagePlusReader.java:387)
	at loci.plugins.in.ImagePlusReader.readImage(ImagePlusReader.java:282)
	at loci.plugins.in.ImagePlusReader.readImages(ImagePlusReader.java:243)
	at loci.plugins.in.ImagePlusReader.readImages(ImagePlusReader.java:221)
	at loci.plugins.in.ImagePlusReader.openImagePlus(ImagePlusReader.java:116)
	at loci.plugins.in.Importer.readPixels(Importer.java:149)
	at loci.plugins.in.Importer.run(Importer.java:86)
	at loci.plugins.LociImporter.run(LociImporter.java:78)
	at ij.IJ.runUserPlugIn(IJ.java:230)
	at ij.IJ.runPlugIn(IJ.java:193)
	at ij.IJ.runPlugIn(IJ.java:182)
	at HandleExtraFileTypes.openImage(HandleExtraFileTypes.java:499)
	at HandleExtraFileTypes.run(HandleExtraFileTypes.java:72)
	at ij.IJ.runUserPlugIn(IJ.java:230)
	at ij.IJ.runPlugIn(IJ.java:193)
	at ij.IJ.runPlugIn(IJ.java:182)
	at ij.io.Opener.openWithHandleExtraFileTypes(Opener.java:516)
	at ij.io.Opener.openImage(Opener.java:370)
	at ij.io.Opener.openImage(Opener.java:242)
	at ij.io.Opener.open(Opener.java:109)
	at ij.io.Opener.open(Opener.java:72)
	at ij.plugin.Commands.run(Commands.java:27)
	at ij.IJ.runPlugIn(IJ.java:199)
	at ij.Executer.runCommand(Executer.java:137)
	at ij.Executer.run(Executer.java:66)
	at java.lang.Thread.run(Thread.java:748)

This says the actual limit for image size is total of 2 gigapixels (2,147,488,281 pixels) for a plane. The error message says the array size was 42848 x 32960 < 2 gigapixels , but I noticed that the error message says there were two channels although in reality there were three.

If the limit of 2 gigapixels counts different channels separately, then 42848 x 32960 x 2 > 2 gigapixels, so maybe this is when the error was issued during the creation of the large array.

Further reading the documenation, ImgLib2 API can surpass this limit. But I don’t know how to use BioFormats with ImgLib2 using Java API. Is it even possible?

At the moment, I’m not interested in displaying the image in Fiji, so hidden operation with API is great if it works.

Though it complains about array size, the memory, Edit > Option > Memory & Threads… shows 145252 MB of maximum memory. This should be enough.


Then, I tried “Crop on import” option of the BioFormats Importer (200 x 200 pixels). However, it failed with a different error after a few seconds of pause.

(Fiji Is Just) ImageJ 2.0.0-rc-69/1.52p; Java 1.8.0_172 [64-bit]; Linux 4.9.6-200.fc25.x86_64; 1706MB of 145252MB (1%)
 
java.lang.IllegalArgumentException: Invalid scanline stride
	at java.awt.image.ComponentSampleModel.getBufferSize(ComponentSampleModel.java:274)
	at java.awt.image.ComponentSampleModel.verify(ComponentSampleModel.java:252)
	at java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:170)
	at java.awt.image.PixelInterleavedSampleModel.<init>(PixelInterleavedSampleModel.java:87)
	at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.getSampleModel(J2KReadState.java:1056)
	at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.readAsRaster(J2KReadState.java:427)
	at com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReader.readRaster(J2KImageReader.java:556)
	at ome.codecs.services.JAIIIOServiceImpl.readRaster(JAIIIOServiceImpl.java:177)
	at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:296)
	at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:267)
	at loci.formats.codec.WrappedCodec.decompress(WrappedCodec.java:132)
	at loci.formats.codec.JPEG2000Codec.decompress(JPEG2000Codec.java:63)
	at loci.formats.in.JPEG2000Reader.openBytes(JPEG2000Reader.java:182)
	at loci.formats.ChannelFiller.openBytes(ChannelFiller.java:156)
	at loci.formats.ChannelFiller.openBytes(ChannelFiller.java:148)
	at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:200)
	at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:161)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
	at loci.formats.DimensionSwapper.openBytes(DimensionSwapper.java:233)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
	at loci.plugins.util.ImageProcessorReader.openProcessors(ImageProcessorReader.java:186)
	at loci.plugins.in.ImagePlusReader.readProcessors(ImagePlusReader.java:422)
	at loci.plugins.in.ImagePlusReader.readPlanes(ImagePlusReader.java:387)
	at loci.plugins.in.ImagePlusReader.readImage(ImagePlusReader.java:282)
	at loci.plugins.in.ImagePlusReader.readImages(ImagePlusReader.java:243)
	at loci.plugins.in.ImagePlusReader.readImages(ImagePlusReader.java:221)
	at loci.plugins.in.ImagePlusReader.openImagePlus(ImagePlusReader.java:116)
	at loci.plugins.in.Importer.readPixels(Importer.java:149)
	at loci.plugins.in.Importer.run(Importer.java:86)
	at loci.plugins.LociImporter.run(LociImporter.java:78)
	at ij.IJ.runUserPlugIn(IJ.java:230)
	at ij.IJ.runPlugIn(IJ.java:193)
	at ij.IJ.runPlugIn(IJ.java:182)
	at HandleExtraFileTypes.openImage(HandleExtraFileTypes.java:499)
	at HandleExtraFileTypes.run(HandleExtraFileTypes.java:72)
	at ij.IJ.runUserPlugIn(IJ.java:230)
	at ij.IJ.runPlugIn(IJ.java:193)
	at ij.IJ.runPlugIn(IJ.java:182)
	at ij.io.Opener.openWithHandleExtraFileTypes(Opener.java:516)
	at ij.io.Opener.openImage(Opener.java:370)
	at ij.io.Opener.openImage(Opener.java:242)
	at ij.io.Opener.open(Opener.java:109)
	at ij.io.Opener.open(Opener.java:72)
	at ij.plugin.Commands.run(Commands.java:27)
	at ij.IJ.runPlugIn(IJ.java:199)
	at ij.Executer.runCommand(Executer.java:137)
	at ij.Executer.run(Executer.java:66)
	at java.lang.Thread.run(Thread.java:748)

Crop on import worked well for an image of the same jpeg 2000 format but with much smaller file size.

Can anyone tell the cause of this second error?

We’d liked to open and edit the image data using ImageJ.

Thanks in advance,
Kouichi

PS. This could be related since it’s custom jpeg 2000 format from the same company MBF.

Also see this:
http://imagej.1557.x6.nabble.com/Large-image-td5015380.html

1 Like

The SCIFIO project provides an ImgOpener utility class for reading data into ImgLib2 data structures using Bio-Formats.

This could be a solution:

Example:

@kouichi-c-nakamura Have you tried code like the following:

#@ ImageJ ij

import java.io.File;
import net.imagej.Dataset;

File file = new File("/path/to/myImage.j2k");
Dataset dataset = ij.scifio().datasetIO().open(file.getAbsolutePath());
System.out.println("Dataset = " + dataset);
System.out.println("Image type = " + dataset.getImgPlus().getImg().getClass().getName());

The above will run as a Groovy script in the Script Editor, but you can also use it from Java code without the #@ ImageJ ij line if you have an ImageJ gateway handy already (or create one first with import net.imagej.ImageJ; ImageJ ij = new ImageJ()).

The code uses SCIFIO (which calls the ImgOpener class you found under the hood), which will invoke Bio-Formats as needed depending on the file format. For large images, it opens them as ImgLib2 CachedCellImg objects, which read and cache tiles on demand. This should allow you to break the 2 gigapixel planar limit, and give you ImgLib2-style data structures upon which you can operate programmatically.

My only concern here is the error message you received when attempting to use Bio-Formats’s “Crop in import” option. That option is what SCIFIO uses internally to read tiles on demand via Bio-Formats, so if it is not working, you may encounter problems. On the other hand, IIRC, SCIFIO tries to respect Bio-Formats’s preference regarding tile size, which might avoid the issue. Give it a shot with your image data and let us know how it goes!

1 Like

The first error is indeed is indeed because the size is > 2GB, however it is not due to 2 channels, but rather each pixel appears to be using 2 bytes, so a single plane is 42848 x 32960 x 2.

The second exception looks more problomatic as it looks as though the Java library used to read the file is throwing an exception due to the size before Bio-Formast has a chance to crop the image. This might be something that can be fixed at a Bio-Formats level though.

Would you be able to upload the sample file to https://www.openmicroscopy.org/qa2/qa/upload/ for testing?

1 Like

Oh, I see. Indeed these files store data as unsigned 16bit integers!

I had to do trials and errors to see how the file paths are handled in Windows, but the following works for normal size images.

#@ ImageJ ij

import java.io.File;
import net.imagej.Dataset;

File file = new File("\\\\xxxx\\xxxx\\xxxx\\000000_000001.jp2"); // Need to escape \ as \\
Dataset dataset = ij.scifio().datasetIO().open(file.getAbsolutePath());
print("Dataset = " + dataset + "\n");
print("Image type = " + dataset.getImgPlus().getImg().getClass().getName() + "\n");
Started open2.groovy at Thu Jul 11 20:22:03 BST 2019
Dataset = 000000_000001.jp2
Image type = net.imglib2.img.planar.PlanarImg

But for the big jp2 file, I got the “Array size too large” again (I used the same kind, but different file, so please ignore the difference in image size).

Started open2.groovy at Thu Jul 11 14:59:30 BST 2019
java.lang.IllegalArgumentException: Array size too large: 41200 x 32960 x 2
...

Would you be able to upload the sample file to https://www.openmicroscopy.org/qa2/qa/upload/ for testing?

Sure, I will upload a large stitched image and smaller tile image of the same file type.

1 Like

The file uploader doesn’t like the image format, and complains “Cannot upload Zero Byte files.” irrespective of the file sizes.

Instead you can get the two files from here.
https://unioxfordnexus-my.sharepoint.com/:f:/g/personal/phar0528_ox_ac_uk/EkOayDiDXIhFp5a13aVAAYIBkitKKldKpR6YQRMC2WuNLA?e=FuScGQ

1 Like

In case that’s helpful I made a bigdataviewer-bioformats repository to open Multiresolution big 2d files using Bio Formats and ImgLib2.
Notably it can open vsi slide scanner files (RGB or fluorescence):

However this fails with your data. For the small image, it can open it, but I fear it’s a non standard RGB image (16 bits per channel ?), so the image looks bad, because pixels are not well read, since I assume 8 bits per channel in my repo in RGB images. This can be solved quite easily I hope.

But anyway for the big image, I get a java heap space error, but I have to say I encountered this error even for quite small RGB jpeg images. I wonder if there a memory leak with jpeg decompression in bioformats (or it’s my repo, or something in bdv, but when I do not have compressed data, there’s no issue, even for very big files). Here’s the full error:


Jul 11, 2019 9:47:13 PM ch.epfl.biop.bdv.bioformats.BioFormatsOpenPlugInSingleSourceSciJava run
INFO: name=null
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Invalid scanline stride
	at net.imglib2.cache.ref.GuardedStrongRefLoaderRemoverCache.get(GuardedStrongRefLoaderRemoverCache.java:194)
	at net.imglib2.cache.util.LoaderRemoverCacheAsLoaderCacheAdapter.get(LoaderRemoverCacheAsLoaderCacheAdapter.java:37)
	at net.imglib2.cache.util.LoaderCacheAsCacheAdapter.get(LoaderCacheAsCacheAdapter.java:30)
	at net.imglib2.cache.ref.WeakRefVolatileCache.getBlocking(WeakRefVolatileCache.java:362)
	at net.imglib2.cache.ref.WeakRefVolatileCache.access$000(WeakRefVolatileCache.java:19)
	at net.imglib2.cache.ref.WeakRefVolatileCache$FetchEntry.call(WeakRefVolatileCache.java:409)
	at net.imglib2.cache.ref.WeakRefVolatileCache$FetchEntry.call(WeakRefVolatileCache.java:386)
	at net.imglib2.cache.queue.FetcherThreads$Fetcher.run(FetcherThreads.java:180)
Caused by: java.lang.IllegalArgumentException: Invalid scanline stride
	at java.awt.image.ComponentSampleModel.getBufferSize(ComponentSampleModel.java:274)
	at java.awt.image.ComponentSampleModel.verify(ComponentSampleModel.java:252)
	at java.awt.image.ComponentSampleModel.<init>(ComponentSampleModel.java:170)
	at java.awt.image.PixelInterleavedSampleModel.<init>(PixelInterleavedSampleModel.java:87)
	at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.getSampleModel(J2KReadState.java:1056)
	at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.readAsRaster(J2KReadState.java:427)
	at com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReader.readRaster(J2KImageReader.java:556)
	at ome.codecs.services.JAIIIOServiceImpl.readRaster(JAIIIOServiceImpl.java:177)
	at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:296)
	at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:267)
	at loci.formats.codec.WrappedCodec.decompress(WrappedCodec.java:132)
	at loci.formats.codec.JPEG2000Codec.decompress(JPEG2000Codec.java:63)
	at loci.formats.in.JPEG2000Reader.openBytes(JPEG2000Reader.java:182)
	at loci.formats.FormatReader.openBytes(FormatReader.java:878)
	at loci.formats.ImageReader.openBytes(ImageReader.java:445)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
	at ch.epfl.biop.bdv.bioformats.BioFormatsBdvRGBSource.lambda$createSource$0(BioFormatsBdvRGBSource.java:83)
	at net.imglib2.cache.img.DiskCachedCellImgFactory.lambda$create$0(DiskCachedCellImgFactory.java:215)
	at net.imglib2.cache.img.LoadedCellCacheLoader.get(LoadedCellCacheLoader.java:91)
	at net.imglib2.cache.img.LoadedCellCacheLoader.get(LoadedCellCacheLoader.java:51)
	at net.imglib2.cache.img.DiskCellCache.get(DiskCellCache.java:104)
	at net.imglib2.cache.img.DiskCellCache.get(DiskCellCache.java:43)
	at net.imglib2.cache.IoSync.get(IoSync.java:174)
	at net.imglib2.cache.ref.GuardedStrongRefLoaderRemoverCache.get(GuardedStrongRefLoaderRemoverCache.java:183)
	... 7 more
Exception in thread "Fetcher-0" java.lang.OutOfMemoryError: Java heap space
	at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
	at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
	at loci.common.NIOByteBufferProvider.allocateDirect(NIOByteBufferProvider.java:126)
	at loci.common.NIOByteBufferProvider.allocate(NIOByteBufferProvider.java:112)
	at loci.common.NIOFileHandle.buffer(NIOFileHandle.java:647)
	at loci.common.NIOFileHandle.read(NIOFileHandle.java:310)
	at loci.common.NIOFileHandle.read(NIOFileHandle.java:298)
	at loci.common.NIOFileHandle.read(NIOFileHandle.java:286)
	at loci.common.RandomAccessInputStream.read(RandomAccessInputStream.java:681)
	at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:266)
	at loci.formats.codec.WrappedCodec.decompress(WrappedCodec.java:132)
	at loci.formats.codec.JPEG2000Codec.decompress(JPEG2000Codec.java:63)
	at loci.formats.in.JPEG2000Reader.openBytes(JPEG2000Reader.java:182)
	at loci.formats.FormatReader.openBytes(FormatReader.java:878)
	at loci.formats.ImageReader.openBytes(ImageReader.java:445)
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
	at ch.epfl.biop.bdv.bioformats.BioFormatsBdvRGBSource.lambda$createSource$0(BioFormatsBdvRGBSource.java:83)
	at ch.epfl.biop.bdv.bioformats.BioFormatsBdvRGBSource$$Lambda$103/1731127504.load(Unknown Source)
	at net.imglib2.cache.img.DiskCachedCellImgFactory.lambda$create$0(DiskCachedCellImgFactory.java:215)
	at net.imglib2.cache.img.DiskCachedCellImgFactory$$Lambda$104/1379150661.load(Unknown Source)
	at net.imglib2.cache.img.LoadedCellCacheLoader.get(LoadedCellCacheLoader.java:91)
	at net.imglib2.cache.img.LoadedCellCacheLoader.get(LoadedCellCacheLoader.java:51)
	at net.imglib2.cache.img.DiskCellCache.get(DiskCellCache.java:104)
	at net.imglib2.cache.img.DiskCellCache.get(DiskCellCache.java:43)
	at net.imglib2.cache.IoSync.get(IoSync.java:174)
	at net.imglib2.cache.ref.GuardedStrongRefLoaderRemoverCache.get(GuardedStrongRefLoaderRemoverCache.java:183)
	at net.imglib2.cache.util.LoaderRemoverCacheAsLoaderCacheAdapter.get(LoaderRemoverCacheAsLoaderCacheAdapter.java:37)
	at net.imglib2.cache.util.LoaderCacheAsCacheAdapter.get(LoaderCacheAsCacheAdapter.java:30)
	at net.imglib2.cache.ref.WeakRefVolatileCache.getBlocking(WeakRefVolatileCache.java:362)
	at net.imglib2.cache.ref.WeakRefVolatileCache.access$000(WeakRefVolatileCache.java:19)
	at net.imglib2.cache.ref.WeakRefVolatileCache$FetchEntry.call(WeakRefVolatileCache.java:409)
	at net.imglib2.cache.ref.WeakRefVolatileCache$FetchEntry.call(WeakRefVolatileCache.java:386)

PS : the bigdataviewer-bioformats repo opens the image in bigdataviewer, not in standard FIJI, so it may not be ideal for you anyway @kouichi-c-nakamura

3 Likes

It’s summer so the people who best know the answer are away, but we’ll get back to you as soon as possible.

Yes, it is. Thank you.

Basically, we have loads of these big images taken with software Stereo Investigator, and at the moment we can’t do anything at all outside of Stereo Investigator, let alone uploading to an OMERO server for sharing.

Thanks for uploading the sample files @kouichi-c-nakamura, I have been able to reproduce the issue this evening. I will continue testing it tomorrow morning to see if I can find a solution for you.

1 Like

Thanks for the report. Much appreciated.

Hi @kouichi-c-nakamura, apologies for the delayed follow up. Unfortunately the issue you are seeing with the scanline appears to caused due to the codec attempting to decompress the entire plane before cropping the requested region. The scaline error is essentially due to a similar issue with the overall size being larger than it can handle.

i have opened a Trello card for this issue but it is likely to require a significant code change: https://trello.com/c/JcpPpQQj/376-jpeg2k-invalid-scanline-stride

1 Like