4D image file-format for Java and ITK

Hello,

I need to use ITK based image analysis software (i.e. elastix) and need to find a file-format that

  1. supports 4D images
  2. can be written from Java
  3. is opened properly (including pixel size metadata) by ITK (elastix)

So far, for 3D data I have successfully used the MHD file format: https://itk.org/Wiki/ITK/MetaIO/Documentation#Reading_a_Brick-of-Bytes_.28an_N-Dimensional_volume_in_a_single_file.29

However, from ImageJ saving 4D (3D + time) hyperstacks into MHD (using [ File > Save As > MHD ] ) still results in 3D images.

I guess I have two questions:

  1. Does someone know of a Java implementation of MHD reader and writer that supports 4D? @ctrueden @s.besson
  2. Does someone know of another suitable 4D image file format that is understood by ITK/elastix? @bogovicj

Thanks a lot for your help!

2 Likes

TIF doesn’t work?..

I remember I had trouble getting ITK to read the pixel size…but one could try again.

Hi @Christian_Tischer,

  1. I don’t know of one.
  2. I’ve had success using elastix with nrrd and nifti, though the only 4d-data I’ve used regularly are displacement fields.
    a. I’m pretty sure Fiji can not yet write 4d nrrds (3d yes). Not sure about nifti. Also not sure about bioformats…

John

Edit 2:
I put some effort into writing 4d displacement fields as nrrds, but (in my view) its not general enough to be pushed back into fiji’s nrrd writer yet. If I ever find time…

1 Like

I could try to make 4D MHD working, because I have Java code already anyway.
You think it would be worth it? My issue is however that I could not find a dimension order specification of MHD.

Not off-hand, no.
~Josh

Why not try adding #ome-tiff support to ITK? To me that would seem the most useful way forward into the future. :slight_smile:

3 Likes

Ome-tiff (in the sense of parsing the xml) is maybe dependent on the current state of the ome-files-cpp and/or the ability to parse the xml metadata into a minimal representation of the XYZCT layout of the planes.

I have an ITK ImageIO class written up as an external module to handle reading and writing of an ImageJ Tiff header. I haven’t tried it for 4D but can post it somewhere if it would be useful. Essentially it’s a partial rewrite of the standard tiff ImageIO to change the metadata info tag. You just manually instantiate the IO, tell it the spacing dimensions (for writing), and pass the IO object to reader/writer.

1 Like

So the tiff IO in ITK is explicitly 2D or 3D only, which is likely why I didn’t go down the route of implementing 4/5D. There’s nothing explicitly preventing it I think (other than copying the parent functions and modifying as needed), though ImageJ typically stores in XYCZT format IIRC so a 3D/4D/5D reader would need to figure out how to extract the volume of interest. Spacing should work correctly, but other ImageJ-specific information (like the pixel unit) may need to live in the IO object since ITK doesn’t have a great way to keep track of that.

Would people want an always 5D IO, or 3D with the IO reporting what dimensions it thinks are present so you can split the time series by ITK functions?

1 Like

I have the same questions. I don’t know how ITK deals with dimension orders… I also never used ITK directly, just through the elastix binary.

To be honest, if one does not need chunking or pyramids, I was always a big fan of simple plain file formats like ics or mhd which consist of a concise text header file and a simple raw file with simply one stream of pixel data. Advantages:

  • Very easy to understand
  • Very easy to modify the header file
  • Very performant to read and write the raw data

Unfortunately I cannot say the same things about Tiff, with all its IFDs and metadata “hidden” in the binary. Thus, given that I know already that ITK properly reads mhd (even 4D, according to one of the main elastix developers) I would rather try to make mhd work in 4D (spacetime) (or even 5D) also on our (i.e. ImageJ) side.

It largely doesn’t (or more precisely, it would rather things be 2D/3D or vector-valued 4D). Other code would need to yank a 3D volume out of the 5D representation. There are multiple classes to do this in ITK, but Python lovers should keep this in mind about SimpleITK:

MHA/MHD are well supported in ITK and can handle very fast streamed writing, ref this post and next in thread:

https://itk.org/pipermail/community/2016-February/010849.html

2 Likes

Sounds great! Motivates me to use it even more :slight_smile:
Right now the metadata of a 4D image would look like this right?
Note: I manually added the 4th (time) dimension.

ObjectType = Image
NDims = 4
BinaryData = True
BinaryDataByteOrderMSB = True
DimSize = 200 200 100 2
ElementSize = 0.5 0.5 5.0 1.0
ElementType = MET_USHORT
ElementDataFile = test.raw

Do you think it would be an option to add a new key-value pair for the dimension order, like this:

DimensionOrder = X Y Z T
  1. Would ITK ignore this or crash? Or maybe one could even make it read it?
  2. Is it correct to also specify an ElementSize for the time dimension (as I did in above example) or should one only specify sizes for the spatial dimensions?
  3. Is there a simple way to test whether ITK would properly read a given MHD file. Maybe some binary ITK based image viewer that I could download? @PerrineGilloteaux
  4. I think it should be easy, even for me, to modify the Java MHD reader in ImageJ to read this new DimensionOrder field and construct an appropriate ImagePlus (hyperstack).
1 Like

Here is some information I got from an elastix developer:

In general, MHD is quite agnostic to the meaning of the dimension. So the raw data is in principle stored in the order:
dim0 dim1 dim2 dim3… etc
For grey scale data in Elastix we assume:
2D+time: XYT
3D+time: XYZT
For color data I’m not 100% sure how it works, since Elastix does not support color images (it may read them, but convert the 3-channel color vectors right away to a single magnitude image I believe). To some extent it is a matter of choice how you store a color (multi-channel) image. You could either a) treat it as an additional dimension, or b) treat the pixeltype as a vector:
a) it’s still a question which dimension is the color dimension then. again this is a matter of choice. XYZCT seems most logical in most applications.
b) In this case, the raw data would be stored in the format CXYZT I think.

Re:

Is there a simple way to test whether ITK would properly read a given MHD file. Maybe some binary ITK based image viewer that I could download?

3d slicer should support the MHD/MHA format, but that’s not to say elastix and 3dSlicer will interpret a file in the same way :-/

John

1 Like

I tried 3d Slicer. In principle good news:

  1. Adding a DimensionOrder field does not lead to a crash
  2. If given a 4D file it also does not crash, but opens only the first time-point, which, I guess is fair enough.

I also found something that one could also install an extension that would allow to also read and display 4D images, but I did not manage yet (https://www.slicer.org/wiki/Modules:FourDImage-Documentation-3.6). If someone knows how to do this, I’d be grateful to learn.

1 Like

Sorry for the late reply.

In general: if you do put effort into image I/O code and want to contribute it somewhere, I’d prefer it go into Bio-Formats or SCIFIO, rather than Fiji’s old kludgy ImageJ1-specific I/O plugin…

There is already the SCIFIO ImageIO plugin for ITK, which enables ITK to call Bio-Formats. Have you tried it?

2 Likes