Visualisation of images and million of dots in napari

Hi

I am using @napari to visualise images together with RNA molecule. The RNA molecules are visualised by plotting the centroid of the molecules as dots.

Everything has been working fine until we have been scaling up the number of genes, the number of molecules/gene and the sample area.
Napari is able to handle the plotting of the milions of dots but it takes forever and even a computer with decent amount of RAm can only plot a subset of the data.

I was just wondering if you have any suggestion on how we should tackle the issue. Should we somehow use an image-pyramid like approach? Should we load only a subset of dots depending on the level of zoom in?

We have been playing with Datashader before but now napari in an important part of our workflow and we would like to stick to it.

Thanks for the help!

Simone

Hi @simocode!

The bad news is, we haven’t optimised the points layer much at all, so you are probably exercising the limits of what napari can do. The good news is, we haven’t optimised the points layer much at all, so there’s actually a lot of low-hanging fruit for optimisation!

Could you provide more details about where you are hitting performance issues? How big actually is your data, is it 2D, 3D, 3D+t? Are you displaying all the dots at once or are you slicing through the data? Is the bottleneck reading in the data, sending it to the GPU, perhaps we are doing some unnecessary array copies?

Having these sorts of details would be very useful to guide us as to which low-hanging fruit to pick. :blush:, or even better yet, guide you! :smiley: :grimacing: If you were able to share a complete script with sample data, that would be incredibly useful!

I’ll point you to two relevant issues on our repo:

2 Likes

Yes, I’m actually fairly optimistic we could improve performance here quite a lot, so would be great to have an example to play with and figure out where you’re running into problems. I have some ideas!!

Hi
Thanks a lot for the prompt answer @jni and @sofroniewn!
The data I have are 2D data. The total number of dots varies between 900000-1500000 and at the moment (temporary solution) all the data are organised in a large pandas.dataframe with three columns, one with a gene tag and the other 2 columns with x,ycoords of the centroid of the dot. Before running the plotting I do a groupby by gene.

Yesterday I run a bunch of tests and realised that the issue is not related to the number of dots. I was able to plot 1.5M dots on a single layer in no time (2.63 s of wall time). And that’s amazing!
The slow/memory issue happens when I want to plot each gene separately and I think that is caused by the number of layers that I am using to visualise the dataset. I am plotting each gene on a separate layer and I can end up with 500 layers when the entire experiment is processed. This multilayer plotting approach is very useful to eyeball if genes show known patterns and also to quickly visualise different gene combinations.
So I guess I will need to find a logic to reduce the number of layers. What do you think?
Thanks a lot for the help! really appreciated!

Ah, yes, indeed, napari layers don’t scale well. That issue is quite fundamental and is on our radar, but will take time to fix. My main question is why you used this pattern? Did you want to color different genes differently? I presume you are not making individual layers visible and invisible? If it’s only for coloring, you are able to do:

viewer.add_points(
        pts[['y', 'x']].to_numpy(),
        properties=pts[['gene tag']],
        face_color='gene tag',
        )

See the add_points_with_properties.py example for details.

If you need additional features from the points layer, we could definitely work on that. I generally envision many improvements based on the API above — filtering points by properties, etc. We are not there yet but community feedback helps us prioritise!

1 Like

Hi
correct, I color each gene differently but also play around by manually toggling layer visibility to quickly visualise expression patterns. I looked at the example you provided and I may be able to tweak the toggling function to label dots when are plotted all together. I will look a little bit deeper in the API and try to come up with a potential solution for our user case. Thanks a lot for the great work! Napari really improved our workflow.

1 Like