Saving as KLB support in ImageJ/SciFio

Hello everyone,

We’ve recently become aware of the Keller Lab Block Filetype and it looks quite interesting. With the help of the forum, I’ve been able to open KLB files. However, I was hoping to also be able to save large files in KLB format. Overall, the dearth of documentation leads me to think KLB is not properly supported, is that correct?

I did actuall try to save to KLB but it fails with a 4GB (2048x2048x500x16) file (but not a 50MB crop of the same file). If I try the NONE or ZLIB compression, Fiji just crashes. But with BZIP2 I get a Failed to write 2 error, which the source clarifies as Error during BZIP compression of one of the blocks.

Below is the python script I am using (I’ve converted it to a Java plugin and the error is the same). Can anyone spot anything clearly wrong?

Test Script

from org.janelia.simview.klb import KLB
from ij import IJ, WindowManager
from net.imglib2.img import ImagePlusAdapter as ipa

imp = WindowManager.getCurrentImage()
imp = ipa.wrapImgPlus(imp)
file_name = "D:\\Code tests\\klb_test.klb"
klb = KLB.newInstance();
klb.writeFull(imp, file_name, [16,16,16] , KLB.CompressionType.BZIP2, "this is a klb saving test")

print "Finished"

Stack Strace

 klb.writeFull(imp, file_name, [16,16,16] , KLB.CompressionType.BZIP2, "this is a klb saving test")
	at org.janelia.simview.klb.KLBJNI.writeFull(KLBJNI.java:289)
	at org.janelia.simview.klb.KLB.writeFull(KLB.java:588)
	at org.janelia.simview.klb.KLB.writeFull(KLB.java:549)
	at org.janelia.simview.klb.KLB.writeFull(KLB.java:691)
	at org.janelia.simview.klb.KLB.writeFull(KLB.java:547)
	at org.janelia.simview.klb.KLB.writeFull(KLB.java:573)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
java.io.IOException: java.io.IOException: Failed to write 2
2 Likes

Hi,

I think you are using it correctly. We don’t use Java to write KLB files, so it is possible that something is broken and we haven’t noticed yet. I’ll take a look.

In general, writing KLB files via Java (i.e. through the JNI) is supported as long as the image is an instance of imglib2’s ArrayImg. The implication is that the image buffer must be a single primitive array. As you may be aware, there is an unfortunate restriction to the size of a primitive array in Java, and I think your image already exceeds that, so currently you won’t be able to write it – or at least not using Java.

There are also some pretty nice alternative file formats you may want to investigate, in particular @axtimwalde 's n5

Cheers,
Burkhard

2 Likes

Hi @palmada, I hope the code treats you well.

Could I ask how you imported and implemented KLB libraries in python? I see in the above you import from org.janelia.simview.klb. How is that setup? I’ve had troubles trying to get the KLB library to work in java with Maven, and ended up pointing it to the local copy of the jar (see KLB files and Maven repository for org.janelia.simview).

I originally tried to use python and ran into troubles with the import stage (see https://github.com/bhoeckendorf/pyklb/issues/8).

Any pointers are much appreciated. We’re trying to write KLB files and your experience I’m sure would be invaluable.

Thanks in advance for your time.

Neil

Hi @nranthony. Sorry for the late reply. Unfortunately it’s been too long for me to remember. I believe I was using FIJI’s Jython interpreter in the script editor. I think I added the KLB jar manually to the jars folder and went from there.

That’s about all of my experience; in the end we chose not to use KLB (or HDF5) as there was too strong a hit to write speeds!

@bhoeckendorf Sorry for not acknowledging before your reply, but thanks for explaining the limitation. In the end we went with plain OME-TIFF!