Hello @melissa and OME Team,
I’m trying to run bioformats2raw on Windows 10 but it throws EOFException and terminates the thread. It prints that it has succesfully processed the final tile, but then appears to hang indefinitely unless terminated using Ctrl + C to return to the command line.
The files are MIRAX Z-stacks (approx. 120 gb; 4-channel; 38000 x 32000 px) and use lossless JPEG2000 compression. Is this supported? Or is there a known bug with JPEG2k?
I can succesfully convert the same images to “intermediate” N5 structures if they are in 8-bit lossy JPEG compression at 90% quality, with no EOFExceptions.
I’m using Anaconda + OpenJDK11.
This should be the beginning of the stack trace (although it is truncated, so I will try to grab the rest):
2020-01-29 17:52:54,787 [pool-1-thread-39] INFO c.g.bioformats2raw.Converter - tile read complete 1731/4588
2020-01-29 17:52:54,787 [pool-1-thread-39] INFO org.perf4j.TimingLogger - start[1580316772490] time[2297] tag[getTile]
2020-01-29 17:52:54,803 [pool-1-thread-39] INFO c.g.bioformats2raw.Converter - successfully wrote at [36, 11, 0] to /0
2020-01-29 17:52:54,803 [pool-1-thread-39] INFO c.g.bioformats2raw.Converter - Successfully processed tile; resolution=0 plane=0 xx=36864 yy=11264 width=18 height=1024
2020-01-29 17:52:54,803 [pool-1-thread-39] INFO c.g.bioformats2raw.Converter - requesting tile to write at [0, 12, 2] to /0
2020-01-29 17:52:54,928 [pool-1-thread-44] INFO c.g.bioformats2raw.Converter - tile read complete 1732/4588
2020-01-29 17:52:54,943 [pool-1-thread-44] INFO org.perf4j.TimingLogger - start[1580316761703] time[13239] tag[getTile]
java.io.EOFException
at jj2000.j2k.JJ2KExceptionHandler.handleException(JJ2KExceptionHandler.java:109)
at jj2000.j2k.codestream.reader.FileBitstreamReaderAgent.getCodeBlock(FileBitstreamReaderAgent.java:2494)
at jj2000.j2k.entropy.decoder.StdEntropyDecoder.getCodeBlock(StdEntropyDecoder.java:745)
at jj2000.j2k.entropy.decoder.StdEntropyDecoder.getInternCodeBlock(StdEntropyDecoder.java:977)
at jj2000.j2k.roi.ROIDeScaler.getInternCodeBlock(ROIDeScaler.java:292)
at jj2000.j2k.roi.ROIDeScaler.getCodeBlock(ROIDeScaler.java:239)
at jj2000.j2k.quantization.dequantizer.StdDequantizer.getInternCodeBlock(StdDequantizer.java:312)
at jj2000.j2k.wavelet.synthesis.InvWTFull.waveletTreeReconstruction(InvWTFull.java:547)2020-01-29 17:52:54,975 [pool-1-thread-44] INFO c.g.bioformats2raw.Converter - successfully wrote at [22, 11, 0] to /0
2020-01-29 17:52:54,975 [pool-1-thread-44] INFO c.g.bioformats2raw.Converter - Successfully processed tile; resolution=0 plane=0 xx=22528 yy=11264 width=1024 height=1024
2020-01-29 17:52:54,975 [pool-1-thread-44] INFO c.g.bioformats2raw.Converter - requesting tile to write at [0, 12, 3] to /0
at jj2000.j2k.wavelet.synthesis.InvWTFull.waveletTreeReconstruction(InvWTFull.java:572)
at jj2000.j2k.wavelet.synthesis.InvWTFull.getInternCompData(InvWTFull.java:314)
at jj2000.j2k.image.ImgDataConverter.getData(ImgDataConverter.java:280)
at jj2000.j2k.image.ImgDataConverter.getInternCompData(ImgDataConverter.java:236)
at jj2000.j2k.image.invcomptransf.InvCompTransf.getInternCompData(InvCompTransf.java:391)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.readSubsampledRaster(J2KReadState.java:862)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.readAsRaster(J2KReadState.java:437)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReader.readRaster(J2KImageReader.java:556)
at ome.codecs.services.JAIIIOServiceImpl.readRaster(JAIIIOServiceImpl.java:177)
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:296)
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:267)
at loci.formats.codec.WrappedCodec.decompress(WrappedCodec.java:132)
at loci.formats.codec.JPEG2000Codec.decompress(JPEG2000Codec.java:63)
at com.glencoesoftware.bioformats2raw.MiraxReader.readTile(MiraxReader.java:1058)
at com.glencoesoftware.bioformats2raw.MiraxReader.openBytes(MiraxReader.java:266)
at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:229)
at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:161)
at com.glencoesoftware.bioformats2raw.Converter.getTile(Converter.java:391)
at com.glencoesoftware.bioformats2raw.Converter.processTile(Converter.java:424)
at com.glencoesoftware.bioformats2raw.Converter.lambda$saveResolutions$2(Converter.java:574)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
The Thread is being terminated bacause an Exception (shown above)
has been thrown and no special action was defined for this Thread.
2020-01-29 17:52:54,990 [pool-1-thread-42] INFO c.g.bioformats2raw.Converter - tile read complete 1733/4588
2020-01-29 17:52:54,990 [pool-1-thread-42] INFO org.perf4j.TimingLogger - start[1580316767818] time[7178] tag[getTile]
2020-01-29 17:52:54,990 [pool-1-thread-49] INFO c.g.bioformats2raw.Converter - requesting tile to write at [1, 12, 0] to /0
2020-01-29 17:52:55,037 [pool-1-thread-25] INFO c.g.bioformats2raw.Converter - tile read complete 1734/4588
2020-01-29 17:52:55,037 [pool-1-thread-25] INFO org.perf4j.TimingLogger - start[1580316767959] time[7083] tag[getTile]
2020-01-29 17:52:55,068 [pool-1-thread-25] INFO c.g.bioformats2raw.Converter - successfully wrote at [30, 11, 0] to /0
2020-01-29 17:52:55,068 [pool-1-thread-25] INFO c.g.bioformats2raw.Converter - Successfully processed tile; resolution=0 plane=0 xx=30720 yy=11264 width=1024 height=1024
2020-01-29 17:52:55,068 [pool-1-thread-25] INFO c.g.bioformats2raw.Converter - requesting tile to write at [1, 12, 1] to /0
2020-01-29 17:52:55,084 [pool-1-thread-15] INFO c.g.bioformats2raw.Converter - tile read complete 1735/4588
2020-01-29 17:52:55,084 [pool-1-thread-15] INFO org.perf4j.TimingLogger - start[1580316767959] time[7125] tag[getTile]
2020-01-29 17:52:55,100 [pool-1-thread-2] INFO c.g.bioformats2raw.Converter - tile read complete 1736/4588
2020-01-29 17:52:55,100 [pool-1-thread-2] INFO org.perf4j.TimingLogger - start[1580316774240] time[848] tag[getTile]
2020-01-29 17:52:55,100 [pool-1-thread-2] INFO c.g.bioformats2raw.Converter - successfully wrote at [36, 11, 3] to /0
2020-01-29 17:52:55,100 [pool-1-thread-2] INFO c.g.bioformats2raw.Converter - Successfully processed tile; resolution=0 plane=3 xx=36864 yy=11264 width=18 height=1024
2020-01-29 17:52:55,100 [pool-1-thread-2] INFO c.g.bioformats2raw.Converter - requesting tile to write at [1, 12, 2] to /0
2020-01-29 17:52:55,131 [pool-1-thread-15] INFO c.g.bioformats2raw.Converter - successfully wrote at [30, 11, 1] to /0
2020-01-29 17:52:55,131 [pool-1-thread-15] INFO c.g.bioformats2raw.Converter - Successfully processed tile; resolution=0 plane=1 xx=30720 yy=11264 width=1024 height=1024
2020-01-29 17:52:55,131 [pool-1-thread-15] INFO c.g.bioformats2raw.Converter - requesting tile to write at [1, 12, 3] to /0
2020-01-29 17:52:55,240 [pool-1-thread-3] INFO c.g.bioformats2raw.Converter - tile read complete 1737/4588
2020-01-29 17:52:55,256 [pool-1-thread-3] INFO org.perf4j.TimingLogger - start[1580316772787] time[2466] tag[getTile]
2020-01-29 17:52:55,271 [pool-1-thread-3] INFO c.g.bioformats2raw.Converter - successfully wrote at [36, 11, 1] to /0
2020-01-29 17:52:55,271 [pool-1-thread-3] INFO c.g.bioformats2raw.Converter - Successfully processed tile; resolution=0 plane=1 xx=36864 yy=11264 width=18 height=1024
2020-01-29 17:52:55,271 [pool-1-thread-3] INFO c.g.bioformats2raw.Converter - requesting tile to write at [2, 12, 0] to /0
2020-01-29 17:52:55,365 [pool-1-thread-23] INFO c.g.bioformats2raw.Converter - tile read complete 1738/4588
2020-01-29 17:52:55,381 [pool-1-thread-23] INFO org.perf4j.TimingLogger - start[1580316768146] time[7232] tag[getTile]
java.io.EOFException
at jj2000.j2k.JJ2KExceptionHandler.handleException(JJ2KExceptionHandler.java:109)
at jj2000.j2k.codestream.reader.FileBitstreamReaderAgent.getCodeBlock(FileBitstreamReaderAgent.java:2494)
at jj2000.j2k.entropy.decoder.StdEntropyDecoder.getCodeBlock(StdEntropyDecoder.java:745)
at jj2000.j2k.entropy.decoder.StdEntropyDecoder.getInternCodeBlock(StdEntropyDecoder.java:977)
at jj2000.j2k.roi.ROIDeScaler.getInternCodeBlock(ROIDeScaler.java:292)
at jj2000.j2k.roi.ROIDeScaler.getCodeBlock(ROIDeScaler.java:239)
at jj2000.j2k.quantization.dequantizer.StdDequantizer.getInternCodeBlock(StdDequantizer.java:312)
at jj2000.j2k.wavelet.synthesis.InvWTFull.waveletTreeReconstruction(InvWTFull.java:547)
at jj2000.j2k.wavelet.synthesis.InvWTFull.waveletTreeReconstruction(InvWTFull.java:572)
at jj2000.j2k.wavelet.synthesis.InvWTFull.getInternCompData(InvWTFull.java:314)
at jj2000.j2k.image.ImgDataConverter.getData(ImgDataConverter.java:280)
at jj2000.j2k.image.ImgDataConverter.getInternCompData(ImgDataConverter.java:236)
at jj2000.j2k.image.invcomptransf.InvCompTransf.getInternCompData(InvCompTransf.java:391)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.readSubsampledRaster(J2KReadState.java:862)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.readAsRaster(J2KReadState.java:437)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReader.readRaster(J2KImageReader.java:556)
at ome.codecs.services.JAIIIOServiceImpl.readRaster(JAIIIOServiceImpl.java:177)
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:296)
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:267)
at loci.formats.codec.WrappedCodec.decompress(WrappedCodec.java:132)
at loci.formats.codec.JPEG2000Codec.decompress(JPEG2000Codec.java:63)2020-01-29 17:52:55,396 [pool-1-thread-23] INFO c.g.bioformats2raw.Converter - successfully wrote at [30, 11, 2] to /0
2020-01-29 17:52:55,396 [pool-1-thread-23] INFO c.g.bioformats2raw.Converter - Successfully processed tile; resolution=0 plane=2 xx=30720 yy=11264 width=1024 height=1024
2020-01-29 17:52:55,412 [pool-1-thread-23] INFO c.g.bioformats2raw.Converter - requesting tile to write at [2, 12, 1] to /0
at com.glencoesoftware.bioformats2raw.MiraxReader.readTile(MiraxReader.java:1058)
at com.glencoesoftware.bioformats2raw.MiraxReader.openBytes(MiraxReader.java:266)
at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:229)
at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:161)
at com.glencoesoftware.bioformats2raw.Converter.getTile(Converter.java:391)
at com.glencoesoftware.bioformats2raw.Converter.processTile(Converter.java:424)
at com.glencoesoftware.bioformats2raw.Converter.lambda$saveResolutions$2(Converter.java:574)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
The Thread is being terminated bacause an Exception (shown above)
has been thrown and no special action was defined for this Thread.
2020-01-29 17:52:55,412 [pool-1-thread-48] INFO c.g.bioformats2raw.Converter - tile read complete 1739/4588
2020-01-29 17:52:55,412 [pool-1-thread-48] INFO org.perf4j.TimingLogger - start[1580316767709] time[7713] tag[getTile]
2020-01-29 17:52:55,428 [pool-1-thread-50] INFO c.g.bioformats2raw.Converter - requesting tile to write at [2, 12, 2] to /0
java.io.EOFException
at jj2000.j2k.JJ2KExceptionHandler.handleException(JJ2KExceptionHandler.java:109)
at jj2000.j2k.codestream.reader.FileBitstreamReaderAgent.getCodeBlock(FileBitstreamReaderAgent.java:2494)
at jj2000.j2k.entropy.decoder.StdEntropyDecoder.getCodeBlock(StdEntropyDecoder.java:745)
at jj2000.j2k.entropy.decoder.StdEntropyDecoder.getInternCodeBlock(StdEntropyDecoder.java:977)
at jj2000.j2k.roi.ROIDeScaler.getInternCodeBlock(ROIDeScaler.java:292)
at jj2000.j2k.roi.ROIDeScaler.getCodeBlock(ROIDeScaler.java:239)
at jj2000.j2k.quantization.dequantizer.StdDequantizer.getInternCodeBlock(StdDequantizer.java:312)
at jj2000.j2k.wavelet.synthesis.InvWTFull.waveletTreeReconstruction(InvWTFull.java:547)
at jj2000.j2k.wavelet.synthesis.InvWTFull.waveletTreeReconstruction(InvWTFull.java:572)
at jj2000.j2k.wavelet.synthesis.InvWTFull.getInternCompData(InvWTFull.java:314)
at jj2000.j2k.image.ImgDataConverter.getData(ImgDataConverter.java:280)
at jj2000.j2k.image.ImgDataConverter.getInternCompData(ImgDataConverter.java:236)
at jj2000.j2k.image.invcomptransf.InvCompTransf.getInternCompData(InvCompTransf.java:391)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.readSubsampledRaster(J2KReadState.java:862)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.readAsRaster(J2KReadState.java:437)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReader.readRaster(J2KImageReader.java:556)
at ome.codecs.services.JAIIIOServiceImpl.readRaster(JAIIIOServiceImpl.java:177)
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:296)
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:267)
at loci.formats.codec.WrappedCodec.decompress(WrappedCodec.java:132)
at loci.formats.codec.JPEG2000Codec.decompress(JPEG2000Codec.java:63)
at com.glencoesoftware.bioformats2raw.MiraxReader.readTile(MiraxReader.java:1058)
at com.glencoesoftware.bioformats2raw.MiraxReader.openBytes(MiraxReader.java:266)
at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:229)
at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:161)
at com.glencoesoftware.bioformats2raw.Converter.getTile(Converter.java:391)
at com.glencoesoftware.bioformats2raw.Converter.processTile(Converter.java:424)
at com.glencoesoftware.bioformats2raw.Converter.lambda$saveResolutions$2(Converter.java:574)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
The Thread is being terminated bacause an Exception (shown above)
has been thrown and no special action was defined for this Thread.
java.io.EOFException2020-01-29 17:52:55,584 [pool-1-thread-7] INFO c.g.bioformats2raw.Converter - tile read complete 1740/4588
2020-01-29 17:52:55,584 [pool-1-thread-7] INFO org.perf4j.TimingLogger - start[1580316767600] time[7986] tag[getTile]
at jj2000.j2k.JJ2KExceptionHandler.handleException(JJ2KExceptionHandler.java:109)
at jj2000.j2k.codestream.reader.FileBitstreamReaderAgent.getCodeBlock(FileBitstreamReaderAgent.java:2494)2020-01-29 17:52:55,584 [pool-1-thread-51] INFO c.g.bioformats2raw.Converter - requesting tile to write at [2, 12, 3] to /0
at jj2000.j2k.entropy.decoder.StdEntropyDecoder.getCodeBlock(StdEntropyDecoder.java:745)
at jj2000.j2k.entropy.decoder.StdEntropyDecoder.getInternCodeBlock(StdEntropyDecoder.java:977)
at jj2000.j2k.roi.ROIDeScaler.getInternCodeBlock(ROIDeScaler.java:292)
at jj2000.j2k.roi.ROIDeScaler.getCodeBlock(ROIDeScaler.java:239)
at jj2000.j2k.quantization.dequantizer.StdDequantizer.getInternCodeBlock(StdDequantizer.java:312)
at jj2000.j2k.wavelet.synthesis.InvWTFull.waveletTreeReconstruction(InvWTFull.java:547)
at jj2000.j2k.wavelet.synthesis.InvWTFull.waveletTreeReconstruction(InvWTFull.java:572)
at jj2000.j2k.wavelet.synthesis.InvWTFull.getInternCompData(InvWTFull.java:314)
at jj2000.j2k.image.ImgDataConverter.getData(ImgDataConverter.java:280)
at jj2000.j2k.image.ImgDataConverter.getInternCompData(ImgDataConverter.java:236)
at jj2000.j2k.image.invcomptransf.InvCompTransf.getInternCompData(InvCompTransf.java:391)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.readSubsampledRaster(J2KReadState.java:862)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.readAsRaster(J2KReadState.java:437)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReader.readRaster(J2KImageReader.java:556)
at ome.codecs.services.JAIIIOServiceImpl.readRaster(JAIIIOServiceImpl.java:177)
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:296)
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:267)
at loci.formats.codec.WrappedCodec.decompress(WrappedCodec.java:132)
at loci.formats.codec.JPEG2000Codec.decompress(JPEG2000Codec.java:63)
at com.glencoesoftware.bioformats2raw.MiraxReader.readTile(MiraxReader.java:1058)
at com.glencoesoftware.bioformats2raw.MiraxReader.openBytes(MiraxReader.java:266)
at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:229)
at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:161)
at com.glencoesoftware.bioformats2raw.Converter.getTile(Converter.java:391)
at com.glencoesoftware.bioformats2raw.Converter.processTile(Converter.java:424)
at com.glencoesoftware.bioformats2raw.Converter.lambda$saveResolutions$2(Converter.java:574)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
The Thread is being terminated bacause an Exception (shown above)
has been thrown and no special action was defined for this Thread.
2020-01-29 17:52:55,615 [pool-1-thread-29] INFO c.g.bioformats2raw.Converter - tile read complete 1741/4588
2020-01-29 17:52:55,615 [pool-1-thread-29] INFO org.perf4j.TimingLogger - start[1580316767818] time[7799] tag[getTile]
2020-01-29 17:52:55,615 [pool-1-thread-52] INFO c.g.bioformats2raw.Converter - requesting tile to write at [3, 12, 0] to /0
2020-01-29 17:52:55,693 [pool-1-thread-9] INFO c.g.bioformats2raw.Converter - tile read complete 1742/4588
2020-01-29 17:52:55,693 [pool-1-thread-9] INFO org.perf4j.TimingLogger - start[1580316772225] time[3461] tag[getTile]
2020-01-29 17:52:55,709 [pool-1-thread-9] INFO c.g.bioformats2raw.Converter - successfully wrote at [35, 11, 3] to /0
2020-01-29 17:52:55,709 [pool-1-thread-9] INFO c.g.bioformats2raw.Converter - Successfully processed tile; resolution=0 plane=3 xx=35840 yy=11264 width=1024 height=1024
2020-01-29 17:52:55,709 [pool-1-thread-9] INFO c.g.bioformats2raw.Converter - requesting tile to write at [3, 12, 1] to /0
2020-01-29 17:52:55,725 [pool-1-thread-37] INFO c.g.bioformats2raw.Converter - tile read complete 1743/4588
2020-01-29 17:52:55,725 [pool-1-thread-37] INFO org.perf4j.TimingLogger - start[1580316767475] time[8255] tag[getTile]
2020-01-29 17:52:55,756 [pool-1-thread-37] INFO c.g.bioformats2raw.Converter - successfully wrote at [27, 11, 0] to /0
2020-01-29 17:52:55,756 [pool-1-thread-37] INFO c.g.bioformats2raw.Converter - Successfully processed tile; resolution=0 plane=0 xx=27648 yy=11264 width=1024 height=1024
2020-01-29 17:52:55,756 [pool-1-thread-37] INFO c.g.bioformats2raw.Converter - requesting tile to write at [3, 12, 2] to /0
Hopefully I’m not just being impatient.
Thanks!