Wrap ImagePlus VirtualStack into imglib2

imglib2
imagej
bigdataviewer

#21

You can set the cache to only retain a fixed number of cells using DiskCachedCellImgOptions, e.g., this

final DiskCachedCellImgOptions options = options()
		.cellDimensions( cellDimensions )
		.cacheType( CacheType.BOUNDED )
		.maxCacheSize( 100 );

sets up a cache that only retains 100 cells (can grow beyond that if you have >100 accessors that are in different cells).
See, for example https://github.com/imglib/imglib2-cache-examples/blob/a20220cb0f34c01dccd30e662255f88b8701c206/src/main/java/net/imglib2/cache/example02/Example02.java#L36-L39

Yes, that is on purpose. ReadOnlyCachedCellImgFactory cannot create an Img without a CellLoader so it cannot satisfy the interface in a meaningful way


#22

@tpietzsch First of all, I would like to apologize for my very late answer, but back then, when you posted the answer, I was not good enough in java and imglib2 to actually use it. However, today, I tested it and got it working and it is great!

@ctrueden I also started testing what happens to input images that are given to an IJ2 plugin (where they are converted to a ‘dataset’) and found the following:

  1. ImagePlus.ImageStack => dataset with PlanarImg inside
  2. CachedCellImg => dataset with CellImg inside
  3. ImagePlus.VirtualStack => dataset with CellImg inside

Case 1

Everything fine

Case 2

What I do not understand is whether the CellImg that is provided by the IJ2 plugin actually is the original CachedCellImg that has been somehow cast to a CellImg, or whether it really has been changed. The fact that the cellDims changed makes me suspect that it actually is not the original CachedCellIImg (see screenshots).

image

image

Case 3

Depends on the answer to case 2, maybe it is already working.
Anyway, I guess the idea would be that the IJ2 plugin builds a CachedCellImg from the VirtualStack, using code such as Tobias provided above, right?

In case this is not implemented yet, I am happy to try to write code (like below) that builds a CachedCellImg from any VirtualStack, i.e. covering the 3 different types (unsigned byte, unsigned short, and float), and also dealing with the 5 dimensions that the VirtualStack (ImagePlus) could have.


#23

I wrote a wrapper for ImagePlus VirtualStack.
It works as ImageJFunction.wrap() but uses imglib2-caches to lazily load the image.
https://github.com/imglib/imglib2-ij/pull/12


#24

The package https://github.com/imagej/imagej-legacy does the automatic conversion between ImagePlus to Dataset. We would need to change this code, if we want the automatic wrapping of VirtualStack to Dataset to be done lazily.

But the code in imagej-legacy confuses me a lot, for the moment, I don’t understand it well enough to make the required changes. Why are the displays needed. Why do we need Harmonizers


#25

@maarzt ( @tpietzsch )
Thanks so much for pulling this off.
This is really great!

@ctrueden
I hope you have/had a good travel back home!
I am herewith doing the job you gave me, which is to bug you to implement the next step :slight_smile: Namely using ImageJFunction.wrap at the place of (or instead of?) the Harmonizer.


#26

I agree that it is convoluted, and would benefit from simplification.

Because in ImageJ1, an ImagePlus can have visualization settings such as position in ZCT, current LUT, current selection, and an overlay containing ROIs. Whereas in ImageJ2, a Dataset cannot have any of those things—it is a wrapper around an ImgLib2 accessible without any associated current position. A DatasetView wraps a Dataset and has visualization settings; an ImageDisplay has a list of DataViews which can include linked Dataset and Overlay (i.e. ROI) objects. So it came to be that ImageDisplay was the correct choice for synchronization with ij.ImagePlus.

Because not everything in the ImageJ1 API can be offered dynamically. ImageJ1 has some fields internally, and a radically different API than ImageJ2, so there are places where copying/translating/adapting data was needed, rather than only wrapping it. That said, we need to be wrapping things in more situations. But you cannot e.g. wrap ImageJ2 Overlay objects into ImageJ1 ROIs, because ImageJ1 ROIs are not interface-driven.

Thank you @Christian_Tischer. I am sorry to say I might be quite slow with progress on this front, although if @maarzt and I stay in frequent contact, perhaps we can move things ahead more quickly. If @fjug and @maarzt agree: perhaps @maarzt and I could have a weekly video chat to keep the improvements to ImageJ Legacy flowing? Otherwise, I fear my time is going to be saturated with the SciJava Common overhaul, ImageJ Ops, the ImageJ Launcher, continued efforts toward eliminating the ImageJ Jenkins, …


#27

Would it be an option, for the time being, to add below lines of code to IJ2 plugins?
Like this, the wrapping that @maarzt developed would be applied and one would just have to make sure that the image that is passed on to the actual computations is the img rather than the dataset.

  1. What do you think?
  2. Is is already predictable when the new ImageJFunctions.wrap will be available to the community?
@Plugin( initializer = "init" )

@Parameter
    private ImagePlus imagePlus;

Img img;

protected void init()
{
        if ( imagePlus != null )
        {
            img = ImageJFunctions.wrap( imagePlus );
        }
}

#28

For the meantime you can write a plugin for wrapping the VirtualStack into an Img/Dataset:

@Plugin( type = Command.class , menuPath = "Image > Wrap Virtual Stack")
public class WrapVirtualStack implements Command
{
	@Parameter
	ImagePlus in;

	@Parameter(type = ItemIO.OUTPUT)
	Img<?> out;

	@Override public void run()
	{
		out = VirtualStackAdapter.wrap(in);
	}
}

Currently you would need to use VirtualStackAdapter instead of ImageJFunctions.
The VirtualStackAdapter can be found here: https://github.com/maarzt/imglib2-ij/blob/virtualstack/src/main/java/net/imglib2/img/VirtualStackAdapter.java


#29

That’s also an option, but imho the advantage of my approach would be that the users would not have to do the extra step of converting themselves, but it would just happen without them noticing. What do you think?


#30

@maarzt @tpietzsch @ctrueden
Hi All,

I have another question, now regarding the visualisation of an Img.

I am using this code to show the image:

uiService.show( img );

In my case the img is a 3GB xyz image with ~3000 z-planes where each z-planes has a different RealViews.transform() attached to it (as a consequence of a registration). Until I call above line of code everything is super fast, because - afaik - nothing is actually computed. However, once I am showing the result it takes a really long time (indicating that all the transforms are actually applied to all pixels).

Is there some way to make the computation of the transforms lazy, such that they only occur once the user actually browses to a specific z-plane?


What does uiService.show() do?
#31

Using Bdv for showing the data is much faster:

1 second: BdvFunctions.show( img, "", Bdv.options().is2D() );
100 seconds: uiService.show( img ) 

Simple problem now is how to save the data when it is displayed in the BDV :slight_smile:


#32

Hi @Christian_Tischer,

Glad that you found that Bdv helps!

In Bigwarp, I have some code that copies a random accessible into another one - this might be helpful to look at / steal. It obviously has to transform every pixel, but its multi-threaded, so that helps somewhat.

I use this method where the target is a wrapped ImagePlus, then have ImageJ/Fiji save the result to a file.

Hope this will be useful,
John

Edit: Forgot to mention that copying means you will allocate a bunch more memory, but it usually ends up being faster to copy (in parallel), then write, and I’m often happy to trade memory for speed.


IJ2 save an image (with lots of frames) from Command
#33

I thought that was exactly what @maarzt has been working on with the new imglib2-ij wrapping?


#34

Yes, this is true, and that works fine! I think the issue right now is “kind of the other way around”: I am already having an Img and want to show it using uiService.show( img ). Could you say what is happening at this point? Is the Img copied/wrapped into an ImagePlus or does it stay an imglib2 datastructure?

[Edit]: I am thinking about things like this:

  • If the context is ImageJ1 and uiService.show( img ) needs to convert the img into an ImagePlus in order to show it; how does it do it exactly?
    • Does it do it in a VirtualStack style, where the getProcessor() function is only evaluated once the user browses to the respective plane?
    • Or does it copy everything into RAM?
  • The reason all of this is important (to me) is because I am often dealing 500GB to 2.5TB sized data sets; so it is very relevant at which stage something is copied to RAM :slight_smile:

#35

Could you also point me to the code where you generate the wrapped ImagePlus for this specific use case? I guess you initialise it as an empty image with the same dimensions as your input RA? And then for actual saving: do you wrap back to ImagePlus and use IJ1 methods?


#36

Sure, its kind of spread out and with “clutter”, so I’ll try putting the relevant parts directly here with links to where I use it.

// create the image plus image
RandomAccessibleInterval<T> rai;
final T t = rai.realRandomAccess().get();
final ImagePlusImgFactory< T > factory = new ImagePlusImgFactory< T >();
final ImagePlusImg< T, ? > target = factory.create( itvl, t );

/*
 * do stuff...
*/

ImagePlus ip = target.getImagePlus();

see here.

Yea… :confused:


#37

Yes, great. @ctrueden I’m fine with working on this, if we have the weekly video chat, as you suggested. @fjug agreed too. The overall goal would be to make this VirtualStack wrapping possible, and change what’s necessary in imagej-legacy? It would like to this is smaller steps, which we both agree on.


#38

Hi @tpietzsch, @maarzt

I am playing with this in the case of a large 2D image, with is composed of cells of smaller 2D images.
It does do the job, but the BigDataViewer basically freezes until all cells in the 2D plane are fully loaded.

Can one make it more interactive in a sense that one can already zoom into one of the loaded cells while the other ones are not fully loaded yet?

Any help would be great :slight_smile:


#39

Yes, https://github.com/imglib/imglib2-cache-examples/blob/a20220cb0f34c01dccd30e662255f88b8701c206/src/main/java/net/imglib2/cache/exampleclassifier/ExampleClassifyingCell.java#L105-L106
VolatileViews.wrapAsVolatile() is the trick


#40

Works great! Thanks!

(Using just one thread in the SharedQueue and also in the CellLoader seems to give me the most interactive experience for the case of simply loading data from Tiff files on disc. When the Cells are based on computations such as in the classification example that might be of course different…)