Using napari as ImagePy's canvas

Napari is OK now. It is a wonderful interactive canvas. And ImagePy is a image-processing framework like imagej, which can be extended by numpy based function easily.

As a framework, imagepy need a powerful canvas, I wrote a simple 2d canvas here, and a 3d canvas here. I spend most time building the framework, making it easy to be extended, and writing many plugins. But I did not pay a lot of time to write the canvas, so the it is very simple.

Now I want to replace my simple canvas by napari.canvas. I had run the napari.example, some questions to make sure wether napari fit imagepy.

  1. does napari support multi layers:
    ImagePy has a label tools, can blend two image togeter. (can draw on the mark layer, and blend with the background, like ilastik)


    I think the answer is Yes :grinning:

  2. can napari show float-image, complex-image?
    imagepy need show multichannels-float image, set one channel as current, or set three channels and their low-up limmit, then show as r-g-b. And for the fft image, can napari do the pow transform and show as two-channel image?

  3. can napari give a draw demo with fps?
    there are many examples on github, but with no fps. It seems that napari’s canvas is based on vispy, I did not know how the image crop/scale, and how the memory swap from cpu and gpu. Can napari give a example, draw by mouse, on a large image, and update timely, and print or show on status bar the fps?

  4. support geojson, and a large vector demo with fps
    Napari support vector, In image-processing, roi is very important. But there is no standurd format for roi. I think geojson is ok! it is widely used in gis. And imagepy has some function for geo image. Can napari give a example, reading a esri large shapefile (such as the US map), dump as geojson (I am aware of geo data processing, I may help), and show on the canvas with fps.

  5. about the license
    pyqt is under gpl, and napari using pyqt, can napari be bsd? that is why I chose wx for imagepy.
    and I see vispy support wxpython too, is it a difficult work to support both qt and wx?

@jni Hi, may be you can give me the answer :grinning:

2 Likes

Hi @yxdragon, always great to hear from you! =)

It would be wonderful if ImagePy could be built on top of napari!

To answer your questions:

  1. Yep, as you suspected. This is a fundamental part of the design of napari.

  2. float image, yes, float multichannel, yes, both RGB (specific VisPy visual) or as separate layers blended additively, using add_image(..., channel_axis=-1), complex, not yet but it is something we have been interested in supporting from the very beginning. Soon!

  3. We haven’t done any work with fps, but I’ve seen some on VisPy if I remember correctly, so this should not be too hard to investigate. Currently drawing is quite slow but slicing, panning, zooming should be blazing fast.

  4. No support for geojson so far but geojson -> shapes should not be hard. See the add_shapes API and examples (in the examples folder) for the data format. Our primary use case is biological/biomedical data, but we aim to support geodata including transforms in the mid/long-term. We would totally welcome a PR adding a geotiff + geojson example!

  5. We use PySide2 by default because its license is LGPL. The PyQt licensing is problematic but we haven’t heard of any real issues with it. Matplotlib and IPython, for example, integrate with PyQt and are BSD-licensed projects. Ultimately, we found that more people were familiar with Qt than wx and decided to use that for better support. I’m not sure it was the right decision (I wasn’t aware that VisPy was also compatible with wx, that’s great to hear!), but I haven’t seen any evidence that it was the wrong one, either. Matplotlib has multiple front-ends, so in the future napari could too, but it would be a lot of work to port what’s there now. I’m not sure we have the time for it now. But if someone were to provide resources (ie mainly time) to do this, I am sure it is technically feasible… Thanks to @royerloic’s guidance, our models and UI elements are nicely separated, so swapping out a front-end should be feasible.

3 Likes

I care about the drawing large image’s fps, because it is slow in pyqtgraph (which I prefer to choose at imagepy’s starting). I think napari should show fps in status bar, and give a drawing demo in napare.example.

that is what I care about. the drawing happend in cpu memory, and if vispy show image as opengl’s texture, It need swap memory between cpu and gpu. If the image is large (such as geo tif), I think it would be quite slow. or even overflow the gpu memory.

that is because all the scale/crop are gpu operation.

pixel label tools need a high fps (at least 20+ for image of anysize), So I care the fps to draw large image very much. I did not know if there is some solution in the future.

Yes, it is easy, but what I care about is fps too! ImagePy has a slow fps when the shape is complex. Because the shape structure (point, line, polygon, multipolygon with holes) is based on python’s list/map. So we can not operate them by numpy (I use shapely in imagepy). may be napari is util enough for bio image, But I want to test if it can fit a geo app.

In fact, all my confuse is the fps :rofl:

We can look into adding that! Agreed it will be useful to keep track of in many of these big data / interactive applications

I have made an issue on our repo where we can discuss the fps measurements more https://github.com/napari/napari/issues/836 vispy has some tooling to make this easier, but we can also add our own QTimer stuff too if we want slightly different behavior