IJM.show() in ImageJ-MATLAB does not support multi-dimentional or multi-channel images

IJM.show('I') or net.imagej.matlab.ImageJMATLABCommands.show('I') is a great way to transfer array data from MATLAB workspace to ImageJ, thanks to ImageJ-MATLAB.

However, I’ve noticed that this method does not really support multi-dimentional or multi-channel images well. Whatever the dimensions are, all the channels are treated as a stack of single channel image.

Surely, one can define the syntax based on the dimensions of array I. I think *.ome.tif is using a 5d format. The third, forth, and fifth dimensions can be one of Z, channel, or time.

In addition, whatever data type I can be, ie, uint8, uint16, etc, data type in ImageJ is 32bit per channel. This can also be improved by specifying a rule, I suppose.

Dear @kouichi-c-nakamura,

This sounds like setting up some convention on how to interpret additional dimensions/axes since MATLAB matrices don’t offer any additional information. One would have to somehow define what the “meaning” of the additional axes is so that a proper conversion to an ImageJ2 Dataset can be achieved. You can take a look at how the conversion is actually done here. I only had a quick look at that, but I assume that the legacy UI does not know how to handle the additional “Page x” axes and falls back to showing them in stack.

I assume you are using ImageJ with the legacy UI. In this case, the imagej-legacy component kicks in to convert the mentioned types to something ImageJ 1.x can display which in many cases is a 32-bit float image. I haven’t tried using ImageJ-MATLAB with the modern UI (you can go to Help > Switch to Modern in the legacy UI to do that), but this should skip the step of conversion and tell you a bit more about the “real” type of the images.

Best,
Stefan

2 Likes

Thanks for the reply.

Yes, I think one needs to define a rule, and that’s what I meant.

For example,

I = zeros(400,600,10,3,20)

1st Y
2nd X
3rd Z
4th channels
5th time

I can probably come up with revision of DefaultImageJMATLABService.java, but I don’t know how to actually develop JAVA code for ImageJ. Learning the environment might take up a lot more of time than actual coding … Does anyone know a good guidance for JAVA coding for ImageJ? How to make modification to exisiting component?

Yes, I never knew there is a new face to ImageJ, and it suprsingly look as old as legecy UI!!!

>> ij.IJ.run("Switch to Modern Mode", "");
>> I = uint8(randi(255,400,400,3));
IJM.show('I')
[INFO] Found 10 JHotDraw adapters.

New UI resulted in even more weird behavior.

It showed up a 32 bit stack image with 3 pages and 64 bit stack image with 3 pages. Why do we need two windows??

I’ve also realized that IJM.show() transposes X and Y.

See the example here.

In MATLAB, convention is that the first and second dimensions represent rows (Y) and columns (X). But it appears that IJM.show() does not handle this and show an image in a transposed way.

I just clarified this point in the documentation, but wouldn’t it be more practical if IJM.show() internally correct the issue? Is there a use case in which someone prefer the X-Y transposed data?

If I is a 5D image data, to prepare data for IJM.show('I2')

order = [{2, 1}, num2cell(3:ndims(I))];
I2 = permute(I,[order{:}]);
2 Likes

Yes, I think so, but it would have to be done carefully to avoid performance hits.

One possible reason to prefer the current state of affairs is that it avoids copying data around. Matlab stores arrays in column major scheme (like Fortran) and ImageJ stores them in a row major scheme.

Matlabs permute seems to make a new copy of the data.

So if your image barely fits into memory, and IJM made a copy by default, everything would crash. Even if it would fit, but is just very large, copying could take a noticeable amount of time.

So the way to go about it (in my opinion) would be an imglib2 /ImageJ2 -style permute (which re-indexes but does not copy). I’m not familiar with IJM, so don’t know how much effort it would be.

2 Likes

The snippet I wrote is for someone to use IJM.show() in a meaningful way. I feel like this information is need for anyone who wants to use IJM.show(), so I’ll add this to the documentation.

In addition, the same issue of X-Y transposition applies to IJM.getDataset() and IJM.geDatasetAs() as well.

If MATLAB’s permute takes memory, then I agree with you that it’s better dealt with in Java side. I’m not that familiar to Java, so I’m reluctant to this myself.

2 Likes

I am not aware of any Java tutorials specifically designed for ImageJ. We have some content on contributing and how to write your own ImageJ2 plugins.

I will look into this next week.

That sounds like a good plan! Would you, @kouichi-c-nakamura, mind opening an issue on imagej-matlab: something along the lines of improving axis / multi-channel handling? We’d really appreciate that!

1 Like

Thanks. Done.

3 Likes

Looks like XYCZT is the default for ImageJ. (see this)

I can suggest syntaxes like below for IJM.show()

IJM.show('A') % interpret as 'YXCZT' 
% flip X and Y by default
% what if A has more than 5 dimensions?

IJM.show('A','CTZ') % specify the 3rd to 5th dimensions
IJM.show('A','XYCTZ') % force XYCTZ order (X and Y are flipped)

When IJM.show(A) opens an image A with more than 2D data, the data in ImageJ is in 32bit and the size is [j,i,1,1,k] in XYCZT format in dimensions (C, channels; Z, Z slices; T, time frames).

The X and Y axes are flipped over.

The third or more dimensions are all piled in the 5th dimension, which is by default interpreted as Time frames (with 1 channel, 1 Z slice).

In case you’re handling single plane multi-channel images, executing the following Java command in MATLAB can correct the format.

ij.IJ.run("Re-order Hyperstack ...", "channels=[Frames (t)] slices=[Slices (z)] frames=[Channels (c)]");

From GUI, Image > Hyperstacks > Re-order Hyperstack …

copytoImgPlus.m can support up to 5D hyperstack data, without relying on the object IJM.