Finding a ROI within an image array

Hi!
First of all: thanks for developing Napari! I am a python novice and recently started using simple Python scripts for my data analysis (everything post image processing). The first major turning point for me was discovering Jupyter notebooks and how powerful they are for reproducible analysis. When I learnt about Napari I quickly realized this could be a second major turning point for me as I can now do everything in one environment in a reproducible manner. I just started using it and love how easy the viewer can be run from a Jupyter code cell and going back to Jupyter to do numpy.array manipulations. I also find the usage of layers way more intuitive than different windows as in ImageJ.

Most of the time I am interested in the intensity change of a dye or GECI over time.
Concerning this, I have a really basic question about finding the pixels of a ROI within a numpy array (I figured it is better to post this question here as suggested in the napari readme rather than stackoverflow or as a github issue):

I have trouble about what is the best way to identify the pixels within a ROI (drawn in the shapes layer) in the image array in order to calculate the mean of these pixels for background subtraction or intensity quantification.

Just as an example:

[in]:

image = np.random.randint(1,100,(10, 10))
viewer = napari.Viewer()
viewer.add_image(image,name="image")
image

[out]:

array([[ 5, 28, 21, 29,  8, 96, 37, 60, 94, 46],
       [23, 16,  9,  6, 34, 82, 75, 37, 14, 51],
       [84, 60, 35, 47, 65, 23, 70, 27,  1, 21],
       [26, 94, 34, 82,  5, 74, 21, 88, 67, 50],
       [86, 84, 22, 93, 71, 64, 77,  9, 36, 86],
       [88, 98, 67, 37, 24, 65, 39, 14, 46, 46],
       [ 6, 11, 23, 55, 67, 40, 99, 54, 25, 25],
       [16, 47, 73,  9, 53, 89,  6, 15, 14, 18],
       [36, 39, 48, 35, 28,  2, 27, 50, 91, 15],
       [73, 32,  6, 13, 93, 62, 40, 26, 10, 99]])

I know that creating a mask and selecting a subset of pixels can be done easily with the labels layer:

[in]:

labels = viewer.layers["Labels"].data
labels

[out]:

array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 1, 1, 0, 0, 2, 2, 0, 0],
       [0, 0, 1, 1, 0, 0, 2, 2, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])

and using this array for masking the original image array:
e.g. for label 1

[in]:

mask = labels == 1
mean = image[mask].mean()
mean

[out]:

57.75

However I actually find it more intuitive and faster using the ellipse or polygon tool of the
shapes layer to draw a ROI around a cell, rather than painting the whole cell of interest in the labels layer.

I know that for example you can get the position of the corner points of a rectangle in the shapes layer as floats by

[in]:

shapes = viewer.layers['Shapes'].data
shapes

[out]:

[array([[4.93525164, 0.89928057],
        [7.33573119, 0.89928057],
        [7.33573119, 4.1438848 ],
        [4.93525164, 4.1438848 ]])]

and subsequently round them to get the corner locations

[in]:

shapes_rounded = np.around(shapes)
shapes_rounded.astype(int)

[out]:

array([[[5, 1],
        [7, 1],
        [7, 4],
        [5, 4]]])

With a rectangle I could further get the distances between the points and then iterate through the array and create mask of all pixels between these corner points but I have trouble how I would do it when using ellipses or which I tend to use more: (irregular shaped) polygons.
The only possible solution I found was this post on stackoverflow dealing with a similar thing and using the np.ogrid() function (though I don’t see why one needs np.ogrid for that). But I still cannot wrap my head around it.

I appreciate any hints and help, as I really want to switch my analysis wherever possible to napari/Jupyter.
Thanks!
Lukas

2 Likes

Hi Lukas

That’s so great that you’re excited about napari and find the layers concept intuitive. Thanks for such a detailed post, I think I have a good idea of the functionality you are looking for, but let me know if this isn’t what you want.

We provide a to_labels function on a shapes layer to do the conversion from a bunch of shapes, say polygons, rectangles etc. to an array where each pixel has the ID of the top most shape that it is inside (in the case of pixels in multiple shapes) or 0 if it is in nothing. This function should allow you to draw shapes using the polygon functionality but then get your data back in a format needed for the labels layer or doing the downstream processing that you’re interested in.

We have an example of this in the repo - examples/shapes_to_labels.py - but we need to add it to our shapes layer tutorial too I’ll work on doing this soon.

As I said above, if this isn’t what you want, or if you have other feature requests / questions about using this functionality, just let us know

5 Likes

I also want to point you to this recently-added function in scikit-image:

https://scikit-image.org/docs/stable/api/skimage.draw.html#skimage.draw.polygon2mask

which is useful for polygons. I’m not sure how well the scikit-image draw functions map to ellipse shapes, but we do also have ellipses:

https://scikit-image.org/docs/stable/api/skimage.draw.html#skimage.draw.ellipse

Overall, though, @sofroniewn’s response most meets your needs I think. =) Let us know if you have more questions!

1 Like

Hi @sofroniewn and @jni!
Thanks so much for the quick reply! I wasn’t aware of the to_labels() function. I just tried it out and it works perfectly! I especially like that the ROIs can then be easily traced back because of the index ability of the label layer. I also tried out the skimage.draw.polygon2mask function, which gave me the same result.
Thanks again!

Lukas

1 Like