Get pixel values from Volatile RandomAccessibleInterval

@hanslovsky @bogovicj @axtimwalde @tibuch

I am still not mastering the Volatile pixel types of imglib2…

It would be amazing if someone could help me out here!

The application is to implement an auto-contrast button for BigDataViewer.
To this end, I am currently getting the central plane of the currently displayed volume and trying to, for each channel, compute the min and max values in this plane. The problem is that, since this is an image of Volatile pixel type, I sometimes simply do not get the values…

In practice, it hangs forever in the \\wait for pixel value line in below code.
I also posted how the centralPlane object looks like beneath the code.

Do you know how to to this properly?

Code

IntervalView< Volatile< R > > centralPlane = getCentralPlane( channel );

Cursor< Volatile< R > > cursor = Views.iterable(centralPlane).cursor();
min = Double.MAX_VALUE;
max = -Double.MAX_VALUE;

double value;
while ( cursor.hasNext() ) {
   cursor.fwd();

    while( ! cursor.get().isValid() ){
        // wait for pixel value
    }

    value = cursor.get().get().getRealDouble();
    if ( value < min ) min = value;
    if ( value > max ) max = value;
 }

Object

centralPlane = {IntervalView@5966} "IntervalView [(0, 0) -- (185, 225) = 186x226]"
 source = {MixedTransformView@6101} "MixedTransformView(net.imglib2.view.MixedTransformView@528640c3)"
  n = 2
  source = {VolatileRandomAccessibleIntervalView@5969} 
   viewData = {VolatileViewData@6108} 
    img = {VolatileCachedCellImg@6109} "VolatileCachedCellImg [186x226x27x2x3]"
     grid = {CellGrid@6116} "CellGrid( dims = (186, 226, 27, 2, 3), cellDims = (186, 9, 1, 1, 1) )"
     cells = {VolatileCachedCellImg$VolatileCachedCells@6117} "VolatileCachedCells [1x26x27x2x3]"
2 Likes

I do not think that Volatile was designed with a blocking get call in mind. I recommend you hold on to both RandomAccessibleInterval<T> (for data manipulation) and RandomAccessibleInterval<Volatile<T>> (for interactive viewer experience)

cc @tpietzsch

4 Likes

Yes, that is how it should be done.

2 Likes

Ok. I see. Let’s say I do, e.g., Views.translate( rai, translation ), do you mean I should do that on both of them? As I perform quite a lot of operations on the images that would mean I have to to this quite a lot. That would almost suggest to write a class

class RaiAndVolatileRai
{
RAI< R > rai;
RAI< Volatile< R > > volatileRAI;
}

…to keep track of both of them. Is that the recommended pattern?

Depends on your actual use case.
If possible, I would defer wrapping as Volatile as long as possible.
So you do all Views operations on the non-volatile type. And only wrap the final result with VolatileViews.wrapAsVolatile for displaying.

Make sense!

Which brings me back to my initial issue (that I posted in the imglib2 gitter), namely that currently VolatileViews.wrapAsVolatile (https://github.com/bigdataviewer/bigdataviewer-vistools/blob/master/src/main/java/bdv/util/volatiles/VolatileViews.java#L86) does not support

  • StackView
  • SubsampleIntervalView ( just realised that I also need this… )
  • ConvertedRandomAccessibleInterval (also this…)
  • RectangleShape$NeighborhoodsAccessible (and this…)

Maybe that’s getting too complex to add all of them them? I tried a bit myself, but the ConvertedRandomAccessibleInterval was already annoying with all the types and for the last one I have no clue how to do it :slight_smile:

So maybe in my case just keep track of both (voltatile and non-volatile) in parallel?

Yes, it would be great to add those to VolatileViews.wrapAsVolatile(). I have no bandwidth at the moment to do it, unfortunately.

That sounds like the best option for now, yes.

So you think conceptually it should work for all of them? Then I can maybe have a look at some point…

1 Like

Not sure about RectangleShape$NeighborhoodsAccessible, but I think it should be possible

is probably not possible, only for some standard converters.

with

I see no problem

2 Likes

EDIT: Removed non-working version

Looks like I got it working:

else if ( rai instanceof ConvertedRandomAccessibleInterval )
	{
		final ConvertedRandomAccessibleInterval< T, S > view
					= ( ConvertedRandomAccessibleInterval< T, S > ) rai;

		final VolatileViewData< T, V > sourceData =
					wrapAsVolatileViewData( view.getSource(), queue, hints );

		final S destinationType = view.getDestinationType();

		final NativeType< ? > volatileDestinationType =
					VolatileTypeMatcher.getVolatileTypeForType( destinationType );
			
		final ConvertedRandomAccessibleInterval converted
					= new ConvertedRandomAccessibleInterval(
						( RandomAccessibleInterval< V > ) sourceData.getImg(),
						view.getConverter(),
						volatileDestinationType );
			
		final VolatileViewData volatileViewData = new VolatileViewData(
					converted,
					sourceData.getCacheControl(),
					destinationType,
					( Volatile ) volatileDestinationType );

		return volatileViewData;
	}

Can you wrap the Converter<T, U> into a Converter<Volatile<T>, Volatile<U>>?
Something like this (pseudo-code):

Converter<T, U> converter =...
Converter<VT, VU> vconverter = (vt, vu) -> {
    boolean isValid = vt.isValid();
    vu.setValid(isValid);
    if (isValid) {
        converter.convert(vt.get(), vu.get());
    }
};
1 Like

Thanks for the answer! But, in fact looks like I got it working now :slight_smile:
I edited my previous post and only put the working version.
Sorry for the confusion!

Maybe I actually have to do something like this, because now I found a case where it is not working…still digging…

Ok, I added bunch of more cases.

The code currently lives here: https://github.com/tischi/fiji-plugin-bigDataProcessor2/blob/master/src/main/java/de/embl/cba/bdp2/volatiles/VolatileViews.java#L105

It works for me right now, but there are many TODO items . If anyone (@tpietzsch) has any more brilliant suggestions (like @hanslovsky above), you are more than welcome :slight_smile:

  • IntervalView:
    • was there already
  • MixedTransformView:
    • was there already
  • SubsampleIntervalView:
    • added, should be OK
  • SubsampleView:
    • added, should be OK, but not sure we actually need it
  • ConvertedRandomAccessibleInterval:
    • added, works for some cases, but needs heavy review
  • StackView:
    • added, works, but needs some review
  • RectangleShape2.NeighborhoodsAccessible:
    • added, works
    • needed to modify RectangleShape to return its span and factory
  • ExtendedRandomAccessibleInterval:
    • added, should be OK, but some raw casts…

Do you think this would be also possible it the input type to the converter is a Neighborhood< Volatile< R > >, rather than a simple pixel? It would be easy to check whether all pixels in the neighborhood are valid. But then, I guess, one would have to construct a Neighborhood< R >, that has all the values, the correct coordinates, and the correct shape. Not sure whether that is feasible?!

While it might be possible, I cannot think of a good way to do that. Maybe there is a better alternative to using a Converter<Neighborhood<T>, ?> in the first place?

Maybe! The application is to have a lazy way of applying an average filter to an image:

If you have an idea, it would be great!

That was from this thread, right?

I would just use a CachedCellImg with an appropriate CacheLoader or CellLoader.

Update: You could even just write your final result into a CachedCellImg, which then could be made volatile with wrapAsVolatile.