BigDataViewer Blending Modes

Hi @bogovicj @axtimwalde @NicoKiaru @tibuch ,

I think Tobias mentioned once that you (Saalfeld) guys have an implementation of different blending modes for BigDataViewer, is that so? I guess the default is that it just sums up the values from the different channels?

We are having multiple EM data sets that we want to show on top of each other and were wondering whether there would be also some sort of averaging mode for overlaying them? Moreover, we also would like to show fluorescence data on top of this, such that we would probably need a logic of “blending groups” that are handled differently.

I think I would know how to implement this myself by building a ARGBConvertedAverageBlendingMultiEMSource< R > :slight_smile:
but in case something exists already I would of course prefer to reuse that!

Best wishes, Christian & @schorb

3 Likes

Adding @hanslovsky to this thread.

1 Like

You can set the AccumulatorProjectorFactory to a custom factory with the ViewerOptions.accumulateProjectorFactory method. For inspiration, I recommend you have a look at the org.janelia.saalfeldlab.paintera.composition package, in particular the
CompositeProjectorPreMultiply class.
In Paintera we do not have blending groups but we add multi-channel data as sources that provide RAI<Composite<T>> with appropriate converters into ARGB space. Downside: Changing grouping is not possible but that is not a concern for us.

5 Likes

I would also be interested in any code you come up with. We are interested in something similar, where we would like to take two grayscale channels, I1, and I2, and nonlinearly map them to ARGB space with a few mathematical formulas.

@hanslovsky Thanks!

I got something working, where signals from EM images are averaged, while signals from FM images are summed up.

The way I currently implemented it is by relying on source.getName() to contain either _em or _fm.

@tpietzsch Would there be a way to add more information to a Source, such that one could, e.g. have some explicit fields containing, e.g. renderMode? Maybe one flexible approach would be if a Source object contained a Map< String, String > metadata, which would be filled while parsing the xml?

I also have another question: the Cursor< ? extends ARGBType >[] accesses seem to also contain pixels that are out-of-bounds for the respective source.

Is there a way to check at this point whether an access is within bounds or not? I am asking, because I think for the averaging blending mode one should exclude the black pixels that are not part of an image (for the summing mode this does not matter).

Currently I am solving it like this:

But I don’t think that’s a good solution, because sometimes there might be really black pixels in an image.

1 Like

Unfortunately not, and it is not an easy change to support this :frowning:
If your sources are ARGBType you could use the alpha channel. Extend the sources with alpha = 0.

2 Likes

How about making a global static WeakHashMap <Source, AdditionalInfo>?
This way you can attach whatever you want

4 Likes

Good idea, I will try it out! At first I was not sure how to do this, but then saw that there is a Views.extend() implementation for a Source. I guess that’s the one to use.

[EDIT]
@tpietzsch @hanslovsky Sorry, I was mislead but the naming F Source in Views.extend, but it looks like F extends RandomAccessibleInterval< T >, which seems not the case for a Source. Long story short: How to you extend a Source?

The renderer uses Source.getInterpolatedSource internally, which returns an interpolated and extended view of the data. When creating instances of your sources, make sure that you extend them with alpha = 0. Source is just an interface and a specific implementation may or may not allow to specifiy the extension. This works, of course, only if your Source is actually Source< ARGBType > or Source< VolatileARGBType >. The downside is: Interpolation happens in ARGB space, which may not be what you want.

3 Likes

Thanks! That solves it.

3 Likes

Hi Guys, @hanslovsky @tpietzsch
Is there a way to change the AccumulateProjectorFactory in a running BDV?
I had a quick look but could not find it…
If not possible I can make my own AdjustableAccumulateProjectorFactory and then change it like this :slight_smile:

2 Likes

At the moment not. AdjustableAccumulateProjectorFactory sounds like a good solution. :+1:

2 Likes