Alpha release of pyclesperanto + graphical user interface for napari 🥳

[haesleinhuepf (Robert Haase)]

Hi clij, clEsperanto and OpenCL-based image processing geeks,

following a Christmas tradition, I’m releasing the first alpha-testing release of pyclesperanto. It is the upcoming sibling of CLIJ in python land. You can find its full API reference of about 200 GPU-accelerated image processing functions here. Examples and details are available on its repository:

clEsperanto/pyclesperanto_prototype

One major goal of the project is building a strong bridge between Fiji island and Napari island across the biological image data science ocean. Therefore, I’ve updated the earlier introduced python code generators available in Fiji. As a bonus, there is now a graphical user interface available for napari. Both can turn image data flow graphs designed on screen into Jython / Python code that is compatible to Python 3 and the Jython 2.7 dialect in Fiji. No need to change code when switching platforms. clEsperanto

Of course, also from the napari interface, you can export Python Jupyter notebooks:

Installation

Online installation instructions are available for the image processing library pyclesperanto_prototype and the user interface napari-pyclesperanto-assistant. In very short:

pip install napari-pyclesperanto-assistant

You can then start the new napari-based user interface like this:

python -m napari_pyclesperanto_assistant https://samples.fiji.sc/blobs.png

Call for feedback

I would be happy about feedback! As I just recently started distributing python packages, I would be happy to hear if installation works painless on end-users computers. Bug reports and suggestions for improvement are also very welcome: either here in the forum or as github issue.

Happy Holidays!

Cheers,
Robert

[Jules]

That’s awesome!

How is this different than the Napari plugin approach?

Happy holidays!
Jules

[haesleinhuepf (Robert Haase)]

Hey @Jules,

@Jules

How is this different than the Napari plugin approach?

Well, it is some kind of napari plugin. The napari developers haven’t revealed yet how image processing plugins in napari can hook into napari. That’s why you need to call it from the command line in the meantime

Cheers,
Robert

[bnorthan (Brian Northan)]

@haesleinhuepf

The napari developers haven’t revealed yet how image processing plugins in napari can hook into napari. That’s why you need to call it from the command line in the meantime

Has anything changed on this in the last couple of months? Just this morning I was trying to learn how to make a Napari plugin, but it seems the example is for a file io plugin not an image processing plugin.

Brian

[haesleinhuepf (Robert Haase)]

@bnorthan

Has anything changed on this in the last couple of months?

Yes, indeed! Check out GitHub - napari/cookiecutter-napari-plugin: Cookiecutter for napari plugins

[talley (Talley Lambert)]

@bnorthan

but it seems the example is for a file io plugin not an image processing plugin.

Hi @bnorthan we haven’t made it that much closer to an image processing plugin spec. The closest thing we have (added in the last couple months) is the napari_experimental_provide_function hook spec. However, exactly how napari will be using that internally is still unclear.

The only immediate concrete application for that hook, was to turn well-annotated python functions into widgets that can be added to the viewer using magicgui. But in the longer term, we dream of using that spec to coordinate passing data through a pipeline, where type-annotated inputs and outputs can be connected.

At this point though, for pure analysis, we don’t really have a clear plugin model.
We’re always curious to hear about use cases though! You presumably came to check up on this with something in mind that you’d like to do… so we’d love to hear it!

edit: see post below… this comment is mostly about composing image processing between plugins. for isolated plugin functionality, the dock_widget hook specification may be useful.

[bnorthan (Brian Northan)]

Hi @talley

I have 2 applications in mind right now

1. Virtual H&E - I’ve been evaluating some virtual H&E algorithms for a client. One example is false color It would nice to be able to wrap this up in a simple GUI. Input would be a 2 channel fluorescence image, output an RGB image. Perhaps this is already done in another tool. QuPath maybe?

2. Deconvolution and “ML Restoration” testing - I would want to make a plugin where the user could define a test shape (points, lines, circles) with features at varying resolutions, then run an image simulator (convolve with PSF, add different types of noise), then run a restoration algorithm, and test the resolution of the restored image.

Both these applications, in the long run, would need the concept of embedded plugins. Each virtual H&E algorithm and restoration algorithm would be a plugin itself, that could be added to the parent plugin.

[talley (Talley Lambert)]

Cool. for both of those applications, since you likely don’t need to interact with other plugins in the near term and since you’ll likely need to gather some input from the user, I would recommend checking out the GUI/widget plugin hook spec (called napari_experimental_provide_dock_widget). That’s a very flexible mechanism that lets a plugin add a widget to the main interface. Once there, you can accept input from the user (including layers that are currently added to the viewer), trigger arbitrary computations in the background (such as decon or virtual H&E), and then add the result back to the viewer.

for a barebones boilerplate of a napari widget plugin package, try the cookiecutter template that @haesleinhuepf mentioned above:

pip install cookiecutter
cookiecutter https://github.com/napari/cookiecutter-napari-plugin

(be sure to answer “y” when asked include_dock_widget_plugin?)

As for how to construct the widget itself… you can either build your own QWidget, if you have experience with that… or you can try using magicgui, which can construct a UI for you based on the input types that you declare. For more on that see the image arithmetic example, and the parameter sweep example… as well as the documentation on using magicgui with napari type annotations

side note: don’t miss the bit about magic_factory at the bottom, which will come in handy if you’re implementing a plugin, rather than directly adding the widget to the viewer yourself.

[talley (Talley Lambert)]

(@haesleinhuepf 's readme in napari_pyclesperanto_assistant gives a nice view of how this can all look)

[bnorthan (Brian Northan)]

Thanks for the examples @talley

Adding a GUI/widget with viewer.window.add_dock_widget was straight forward.

However I’m a bit confused about how to get napari_experimental_provide_dock_widget to work.
In ImageJ, to ‘install’ a plugin, I’d simply put it in the ‘jars’ or ‘plugin’ directory. I don’t see anything equivalent in the Napari documentation, so I assume I need to make a setup.py file, and run it to ‘install’ the plugin in my current environment, then Napari scans all modules installed in the environment to find ones that implement the Napari specific hooks. Is that correct or are there other steps to make a plugin discoverable ?

[haesleinhuepf (Robert Haase)]

Hey Brian @bnorthan ,

@bnorthan

so I assume I need to make a setup.py file, and run it to ‘install’ the plugin in my current environment, then Napari scans all modules installed in the environment to find ones that implement the Napari specific hooks. Is that correct or are there other steps to make a plugin discoverable ?

Correct. See the setup.py of my minimal napari plugin. The procedure is explained here in the napari docs.

If it still doesn’t work, feel free to point us at code and we take a look

Cheers,
Robert

[talley (Talley Lambert)]

yep. As @haesleinhuepf said, you’ll need to install your package into your local environment.

the cookiecutter mentioned above should get you most of the way

pip install cookiecutter
cookiecutter https://github.com/napari/cookiecutter-napari-plugin

then initialize your repo

cd myplugin  # whatever you named it
git init
git add .
git commit -m 'initial commit'

then install it locally (-e stands for editable mode, which means you can change stuff and it will be reflected when you next run):

pip install -e .

now you can run napari and your widget will appear in the menu. For fast iteration (and if you have a newer napari version), you can start napari with your dock widget already open:

# where "myplugin" is the name you used in your `napari.plugin` entrypoint
napari -w myplugin 

[bnorthan (Brian Northan)]

Thanks. Those instructions were very clear. It is working now.