Image visualisation in Jupyter using ij.py

imagej
python
jupyter
i2k
pyimagej

#1

This topic is following up on @ctrueden’s very cool workshop at #i2k (https://github.com/imagej/i2k-2018), which covered use of pyimagej in a Jupyter Notebook.

One issue arose with regard to image visualization using the ij.py.show() function. This uses matplotlib to display image data inline, but there are currently not any parameters to, e.g., set the size of the display image. In matplotlib, this is handled by setting some style parameters:

import matplotlib as mpl
mpl.rcParams['figure.figsize'] = [8.0, 6.0]
mpl.rcParams['figure.dpi'] = 80

so image resizing is easy enough to handle now, but the point was raised that setting these parameters though ij.py.show() should be agnostic matplotlib (allowing for other visualization libraries).

My bandwidth is fairly limited at the moment, or else I would take a crack at an API, but I am interested to hear other thoughts on this, any suggestions for implementation, etc.


#2

I think this brings up another key question: How good do we want ij.py.show() to be in the first place? Is it supposed to be a catch-all for any kind of plotting we want to do, something similar to an ImageJ window, or something else altogether?

In my opinion, the main goal if ij.py.show() would be to act like a typical ImageJ window for those who are not familiar with python. Ideally with sliders for channels/timepoints/Z stack and so-on. This would give them a familiar interface for looking at their data. More experienced users can just extract the numpy arrays and use whatever other visualization library they like.

I’m not sure trying to cover other libraries with IJ.py.show() is worth doing in that light, as it would be a lot of effort on the programmers end but not much help on the users end. A user who wants to do anything complex would have to learn the other libraries anyway, or an API that is presumably close in difficulty.

IJ.py.show() still needs some work to reach the goal of being like an ImageJ window, e.g. size control and sliders are big wants, but I don’t think it should try to be much more complex than that.

However, that’s just my opinion. Anyone else have thoughts on this?


#3

One can already add sliders and size adjustments using ipywidgets (not the most elegant solution, but it would certainly work well enough). But @ctrueden seemed to think making changes to ij.py.show() to do this by default was not the best idea–hopefully he will chime in.


#4

Sorry for the slience, @mellertd—been swamped with other things. In broad strokes I think I agree with @mpinkert here:

So that means making show do something nice with sliders, etc.

@mellertd As we discussed in person, making show support different visualization backends would be very cool. What I’d like to stay focused on is ensuring the API is somehow agnostic of the backend. Some kind of generic hints object being passed (it could be as simple as a string, perhaps) to indicate which backend should be used could work, as long as the notebook still does something reasonable as a fallback should that backend not be installed. I’d love to eliminate the explicit matplotlib dependency from pyimagej, in favor of optional deps on these pluggable backends. Case logic code like: “If matplotlib is installed, do this; otherwise, if pillow is installed, do that.”

Do you remember what exactly I said? At this point, I don’t recall that specific part of our conversation. The main things I want to avoid are: A) framework-specific dependencies; and B) breaking behavior over time. Happy to elaborate in more detail as needed.


#5

Sorry I think I was a bit sloppy with my language there. What I recall is you mentioning that you didn’t like the idea of any hard-coded dependency, which would include ipywidgets and matplotlib. So I think we are perhaps on the same page?

As a separate but related point–I am not well-versed in all of the different interactive visualization options available for Jupyter, just ipywidgets. I am hoping there are better options, as ipywidgets is a bit slow to be a reasonable alternative to a proper ImageJ window. But I confess that I might not be skillfully using the tool. I usually do something like this:

def f(z, color):
    matplotlib.pyplot.imshow(image[:,:,z,color], cmap='gray')

ipywidgets.interact(f, z=ipywidgets.IntSlider(max=len(bac_vol[0,0,:,0])-1),
                    color = ipywidgets.Dropdown(options={'red': 0, 'green': 1, 'combined':(0, 1, 2)}));

which is totally functional but not terribly responsive.


#6

I filed imagej/pyimagej#24 to track this mini-project. I have no time to work on it this month, but please feel free to write your ideas into the issue. We could probably bang out a better show implementation in an hour or two if we had a list of backends to support together with how their hints parameters should look!