5D volume viewing in Python

Hi everyone,

I’m currently putting together a short presentation on doing 3D image processing with Python and Jupyter. Reviewing existing visualisation tools, I noticed the absence of a general tool handling most classical cases of interactive volume visualisation. Here’s a short summary of existing tools and their limitations (they all are great tools that can also do other things than volume viewing and I don’t mean to be negative here, just trying to find out where the field is heading to):

So it seems that there’s no single tool that handles at the same time multi-channel + time-lapse + colormaps.

Therefore my questions here are:

  • Does anybody know about another tool that would be more complete?
  • Does anybody know if someone is working on a new tool?
  • Maybe someone has a solution I don’t know to tweak one of the above tools?

Thanks for any infos !

Cheers,

Guillaume

4 Likes

There is napari but I do not know how mature it is or if it is more complete than any of the above. Pinging @jni for some information

Another option would be to use the Java-based BigDataViewer (BDV) through imglyb or PyImageJ. BDV supports 5D data but AFAIK does not support arbitrary color maps out of the box. You would have to implement your own Converter into ARGB color space. Maybe a color map converter is readily available in the BDV world (cc @tpietzsch) or somewhere in ImageJ (cc @ctrueden).

1 Like

We are building a python interface for SciView based on @hanslovsky’s imglyb. It is in testing, but we’re waiting on one feature (poke @skalarproduktraum). This will provide BigDataViewer style volumes for 5D data. You can test it in ImageJ with BigDataViewer files, but we’re generalizing that to numpy arrays for this upcoming release.

2 Likes

Thanks for the tips!

I had a look previously at Napari but it seemed still under heavy construction. I just tried it again now and the volume rendering seems to work nicely and has all the features I was looking for!

I of course already tried BigDataViewer through your imglyb package and it worked great. It’s nice to see that these two worlds are converging and I’ll definitely mention this solution in my presentation. Any hope to make it work on Mac OSX however ?

That sounds great, I’ll be happy to test it as soon as it is officially released!

@thewtex what is the latest with VTK python interfaces?? I know it is actually really easy to visualize volumes inn Python using the VTK interface, and I believe you’ve presented example notebooks in the past, however my understanding is there is not support for more than 4 channels, which is a potential road block for microscopy applications.

You would have to run it through the wrapper that is provided by imglyb.

Note that this wrapper is (probably) not necessary for JavaFX applications like, for example, Paintera. However, Paintera currently does not support time series and there is no straight forward way to start it from Python (unless you really know about both Python and JavaFX). The experimental Python wrapper payntera is outdated but may be a starting point.

1 Like

A Python interface for BDV/SciView would be awesome. Glad to hear you guys are working on it.

In the meantime, you can have a look at https://github.com/pyvista/pyvista, a Python interface to vtk. Development is very active.

2 Likes

Are you talking about this issue?

There is a workaround. I (or anyone!) just need to find time to package it up in a nice way so things “just work” out of the box.

It might be the same issue. One of the imglyb example notebooks has a note saying that one can only run it as a file through the wrapper:

Thanks @guiwitz! It’s great to hear that napari fits your use case, which indeed is right in our ballpark. We are indeed still under heavy development but basically everyone on the team uses it in day-to-day work and it’s working great! (All credit to @sofroniewn for the state it’s in.) Please chime in with an issue if you have any feedback at all. We love to hear from new users! :blush:

1 Like

https://github.com/InsightSoftwareConsortium/itkwidgets: no multi-channel visualization, no time-lapse-visualization

Multi-channel visualization is actively being implemented :construction_worker_man: – feedback as drafts come out would be much appreciated :pray:! Time-lapse-visualization is on the roadmap, but longer out.

@bnorthan Yes, at first there will support for up to 4 independently color-mapped channels. The limit is 4 because of technical and performance considerations working with GPU’s and Jupyter. But, we could create an additional rendering path that does some pre-processing CPU-side to support more channels.

@jni @sofroniewn @kephale @@skalarproduktraum @tpietzsch @ctrueden et al, what sequences of colormaps are used in napari, BDV, SciView, ImageJ?

Is there a best practice and standard we can provide throughout tools?

With many channels, what colormap sequence provides differentiation of channels without being “messy”?

What sequence of colormaps are perceptually meaningful when multiple channels are combined?

3 Likes

With more than 3 channels I find that no colormaps are perceptually meaningful, in particular for individual voxels that have significant contributions from all channels.
What does become important is the ability to quickly toggle the visibility and contrast/brightnes of each channel using a checkbox etc. (similar to Photoshop layers).

1 Like