Using napari as ImagePy's canvas

Napari is OK now. It is a wonderful interactive canvas. And ImagePy is a image-processing framework like imagej, which can be extended by numpy based function easily.

As a framework, imagepy need a powerful canvas, I wrote a simple 2d canvas here, and a 3d canvas here. I spend most time building the framework, making it easy to be extended, and writing many plugins. But I did not pay a lot of time to write the canvas, so the it is very simple.

Now I want to replace my simple canvas by napari.canvas. I had run the napari.example, some questions to make sure wether napari fit imagepy.

  1. does napari support multi layers:
    ImagePy has a label tools, can blend two image togeter. (can draw on the mark layer, and blend with the background, like ilastik)


    I think the answer is Yes :grinning:

  2. can napari show float-image, complex-image?
    imagepy need show multichannels-float image, set one channel as current, or set three channels and their low-up limmit, then show as r-g-b. And for the fft image, can napari do the pow transform and show as two-channel image?

  3. can napari give a draw demo with fps?
    there are many examples on github, but with no fps. It seems that napari’s canvas is based on vispy, I did not know how the image crop/scale, and how the memory swap from cpu and gpu. Can napari give a example, draw by mouse, on a large image, and update timely, and print or show on status bar the fps?

  4. support geojson, and a large vector demo with fps
    Napari support vector, In image-processing, roi is very important. But there is no standurd format for roi. I think geojson is ok! it is widely used in gis. And imagepy has some function for geo image. Can napari give a example, reading a esri large shapefile (such as the US map), dump as geojson (I am aware of geo data processing, I may help), and show on the canvas with fps.

  5. about the license
    pyqt is under gpl, and napari using pyqt, can napari be bsd? that is why I chose wx for imagepy.
    and I see vispy support wxpython too, is it a difficult work to support both qt and wx?

@jni Hi, may be you can give me the answer :grinning:

2 Likes

Hi @yxdragon, always great to hear from you! =)

It would be wonderful if ImagePy could be built on top of napari!

To answer your questions:

  1. Yep, as you suspected. This is a fundamental part of the design of napari.

  2. float image, yes, float multichannel, yes, both RGB (specific VisPy visual) or as separate layers blended additively, using add_image(..., channel_axis=-1), complex, not yet but it is something we have been interested in supporting from the very beginning. Soon!

  3. We haven’t done any work with fps, but I’ve seen some on VisPy if I remember correctly, so this should not be too hard to investigate. Currently drawing is quite slow but slicing, panning, zooming should be blazing fast.

  4. No support for geojson so far but geojson -> shapes should not be hard. See the add_shapes API and examples (in the examples folder) for the data format. Our primary use case is biological/biomedical data, but we aim to support geodata including transforms in the mid/long-term. We would totally welcome a PR adding a geotiff + geojson example!

  5. We use PySide2 by default because its license is LGPL. The PyQt licensing is problematic but we haven’t heard of any real issues with it. Matplotlib and IPython, for example, integrate with PyQt and are BSD-licensed projects. Ultimately, we found that more people were familiar with Qt than wx and decided to use that for better support. I’m not sure it was the right decision (I wasn’t aware that VisPy was also compatible with wx, that’s great to hear!), but I haven’t seen any evidence that it was the wrong one, either. Matplotlib has multiple front-ends, so in the future napari could too, but it would be a lot of work to port what’s there now. I’m not sure we have the time for it now. But if someone were to provide resources (ie mainly time) to do this, I am sure it is technically feasible… Thanks to @royerloic’s guidance, our models and UI elements are nicely separated, so swapping out a front-end should be feasible.

3 Likes

I care about the drawing large image’s fps, because it is slow in pyqtgraph (which I prefer to choose at imagepy’s starting). I think napari should show fps in status bar, and give a drawing demo in napare.example.

that is what I care about. the drawing happend in cpu memory, and if vispy show image as opengl’s texture, It need swap memory between cpu and gpu. If the image is large (such as geo tif), I think it would be quite slow. or even overflow the gpu memory.

that is because all the scale/crop are gpu operation.

pixel label tools need a high fps (at least 20+ for image of anysize), So I care the fps to draw large image very much. I did not know if there is some solution in the future.

Yes, it is easy, but what I care about is fps too! ImagePy has a slow fps when the shape is complex. Because the shape structure (point, line, polygon, multipolygon with holes) is based on python’s list/map. So we can not operate them by numpy (I use shapely in imagepy). may be napari is util enough for bio image, But I want to test if it can fit a geo app.

In fact, all my confuse is the fps :rofl:

We can look into adding that! Agreed it will be useful to keep track of in many of these big data / interactive applications

I have made an issue on our repo where we can discuss the fps measurements more https://github.com/napari/napari/issues/836 vispy has some tooling to make this easier, but we can also add our own QTimer stuff too if we want slightly different behavior

I have similar question connected with licensing PyQt based code and I ask it to PyQt author and PyQt based code could be BSD. Here is question and response:

On 12/02/2020 11:42, Grzegorz Bokota wrote:
Dear Phil,
In thread “Intersphinx with PyQt” you write that BSD is compatible with
GPL. could You explain when PyQt base code should be available on GPL
and
when on BSD, when author do not buy license? I still talking about open
source code.

Non-commercial PyQt is always GPL. It can be used with another package
so long as that other package uses a license that is GPL compatible. BSD
is one such license.

If your BSD package is used with GPL PyQt then the two together are
effectively licensed under the GPL - it takes precedence.

If your BSD package is used with another BSD package then the two
together are licensed under the BSD license.

Another example…

If you wanted your package to be used with both the commercial and GPL
versions of PyQt then you either have to release two versions of it (one
using your own commercial license and the other using the GPL), or you
choose a license which is compatible with both the PyQt commercial
license and the GPL (eg. BSD).

Regards,
Phil

There are any instruction how embed napari as a widget in Qt application?

There are any instruction how embed napari as a widget in Qt application?

can you elaborate?
The main napari viewer is an instance of QMainWindow and when you create a viewer instance with viewer = napari.Viewer(), the QtMainWindow object is available at viewer.window._qt_window. If you’re looking to add the whole napari window to some QLayout in some external program, that’s the object you need to add.

But I do not want to use whole main window of napari, but embed viewer as a part of bigger program. I found class QtViewer, but it in in _qt which in python convene is private module with no api stability.

Also from python convention _qt_window is class private member without warranty that it will be present in next release.

More interesting looks viewer.window.qt_viewer but it still force to instance whole image and cherypick interesting widgets.

Or there is some better way?

We havn’t focused on making the QtViewer embeddable, but it is something we want to be possible. We’re still pretty early so there might be some breaking changes around this stuff (though we will try and post good release notes making it clear how to update)

For now you might want to try something like

from napari._qt.qt_viewer import QtViewer
from napari.components import ViewerModel as Viewer

viewer = Viewer()
qt_viewer = QtViewer(viewer)

If you don’t want the main window.

Thanks for information. I will try with this. But I found that napari force to install PySide2 which is not necessary. I create PR to remove this https://github.com/napari/napari/pull/1059

Are you interested with other PR that will reduce requiremnts (I would like to avoid matplotlib and scikit-image requires it)?

If I finish my Work it is possible to ask if Yo can take short look and check If I do not do something stupid?

1 Like

thanks @gbokota! just saw that PR and taking a look now. This is a discussion we’ve been having for a long time (see https://github.com/napari/napari/issues/704) and we agree that forcing installation of pyside2 is undesirable. will add further thoughts at your PR.

We are open to any and all PRs! though each will of course be subject to discussion :slight_smile:. regarding scikit-image, skimage.io.imread forms the basis of our file input function (napari.utils.io.magic_imread), so that might be a tricky one to extract elegantly. but we are always open to well-considered proposals! As a sidenote: it is very important to us that even novice users can simply pip install napari and be up and running, so solutions that require modified installation instructions (like pip install napari[skimage]) will probably meet with some resistance.

Lastly, we’re always happy to take a look at implementations. feel free to post your work here or on our github issue tracker.

1 Like

magic_read was simple. More time consume generating thumbnail.

I see that my PR is labeled by release 0.3.0. In which time I can expect this release?

Thanks for the PR @gbokota! The main question is whether the full team will be satisfied with the dependency swaps that were required in your implementation. We’ll comment at your PR (no need to continue the conversation in this thread, which we’ve hijacked at this point :joy:). I can’t give you an exact timeframe on the next release. Generally we release once or twice a month, you can review our releases to get a sense for release frequency.