Understanding how Napari handles Pyramid images and zarr files


I am trying to understand how Napari works with Pyramid images, as well as zarr files. I have the following understanding, is it correct?:

Pyramid images (stored as a .zarr file):

  1. When I execute viewer.add_image(‘example_pyramid.zarr’, multiscale=True), Napari first inspect the current size of the viewer, and then, based on the current size, it decides which level of the pyramid to load (for example, it could decide to load the lowest resolution of the pyramid first), and proceed to load all the “chunks” of the zarr file that corresponds to the selected pyramid level.

  2. Once it finishes loading and visualizing the selected level of the pyramid, it waits for some user interaction. If the user zooms-in, Napari computes the new view_port and determines which region of the image the user wants to see, and based on that result, it only loads/visualize the required “chunks” of the zarr file.

  3. If the user does a lot of zoom-in/out, does Napari cache the results?, or does it always load all the “chunks” according to the zoom level, even if the user already visited a previous zoom level?

  4. According to the documentation, Napari uses VisPy for rendering the images. Therefore, every time the user zoom-in/out in a particular region of the image, Napari load the corresponding “chunks” and send the data to the GPU, am I right?..does this mean there is a lot of communication back-and-forth between the CPU and GPU?.

Image stored as a zarr file

  1. Let’s say I have a big image store as a zarr file (only the image in high resolution, without any pyramid data). If I call viewer.add_image(‘example_high_resolution.zarr’, multiscale=False), Napari would see that the image is big, it would throw a warning saying “UserWarning: data shape (26752, 10240) exceeds GL_MAX_TEXTURE_SIZE 8192 in at least one axis and will be downsampled.
    Rendering is currently in 2D mode.”…Does this mean Napari will compute the pyramid images and it will visualize instead one of the lower levels of the pyramid?

Thank you very much for all your help, and please excuse me for asking so many question :blush: , I really like the project and I would like to understand its inner working so that I can start contributing to it in the near future :slight_smile:

1 Like

Hi @jmontoyam, all the questions are great. What you say above sounds like a pretty good description of how napari handles multiresolution data, though for the add_image call I don’t think you can pass a path anymore. I think of the pyramid data as being a list of arrays of decreasing resolution, which could be represented in zarr in a number of different ways, see this discussion for one spec Multiscale arrays v0.1.

Your comment about lots of CPU/ GPU transfer is correct, we’re going to be working on an octree data representation soon which will help make things more efficient.

For the case where the single image is too large we don’t compute a pyramid anymore, we just show something downsampled. We might want to go back to computing a pyramid when we do the octree representation, but it was causing some un-expected behavior before.

1 Like

Thank you very much @sofroniewn!..Regarding the octree, is it possible to participate in the conversations that the main developers will have about the implementation details?..I am a newbie, but I would like to learn the Napari internal implementation details, as well as to help this amazing project continue growing :slight_smile:

1 Like

@jmontoyam we’d love to help you contribute! We haven’t started on octree yet but places to follow are:

If you’re new to open source, you’re in luck, I love getting people started. :grinning: If you hop on the chat we can set up a time to walk you through a contribution. Be aware that it’s very addictive! :joy:

In a more structured sense and for future reference, many coding conferences such as PyCon and SciPy have one or two days of “sprints”, in which you can contribute to a whole bunch of open source projects together with the maintainers, including napari. SciPy Japan is coming up (Oct 30-Nov 2) and there will be napari sprints there. The sprints at SciPy US are how I got started in open source!

1 Like

Hi @jni,

thank you very much for offering me your help! ;). For sure I accept your kind invitation for the introduction :slight_smile: … Will you be available on Friday so that we can meet at the chat room and set up a time?

Once again thank you very much!