Labeling with paintbrush over n-dims

When I use the paintbrush in a labels layer over n-dims, why do my annotations erode/disappear over each layer? For example, I am working with a 20x1500x1500 image and when I add Labels and annotate on the first layer with n-dims on, the marking disappears over each layer.

Hi @shaojus right now the brush size is taken to be isotropic in all dimensions if n-dims is on, so you might see something for the first 10 layers if the brush size is 10, and then see nothing. I see though how a reasonable alternative default would have just been to broadcast in all dimensions. I was thinking of a use case at the time when someone was painting into a 3D volume and so broadcasting wasn’t desirable. It would be great to learn a little bit more about your use case and maybe we can think about how to improve the n-dim painting for you.

Ah I see, I am working on a cell segmentation pipeline and have been trying to see if I could use the segmentify plugin as a replacement for ilastik for my initial pixel classification. Ideally I would want my labels/annotations broadcasted over all dimensions/channels of my stack of cell images, have the classifier trained on all layers of my stack of images to output one final segmentation. I have been trying to make modifications to segmentify to fit this usage but have been having some problems, one of which was this one.

Cool! You should know that of course segmentify + napari is right now just a proof of concept compared to a mature tool such as Ilastik, but I hope we can guide you to something that fits your usage!

Can you specify a bit more what the shapes of each of your images/channels and labels layer are, and what you would expect to see? n-dim painting changes the square brush to a cube and the circle brush to a sphere. If you only want to paint in the current slice, you should simply leave n-dim unchecked. If you want to paint the exact same shape over all slices, then you should make your labels layer have shape (1500, 1500), and napari will broadcast it over all slices. We try to follow the NumPy broadcasting rules for this behaviour.

I am working with a tif image stack, (20, 1500, 1500) dimensions. When I add a new label in the napari gui, is it not automatically set to (1500,1500) here? When i do n-dim painting I expect to see the labels layer the same no matter what slice of the stack I go to, but the paint annotations erode as I scroll through the stack and eventually disappear if I don’t use a big enough brush size.

And this is kind of off topic but is segmentify intended for multichannel images? I see some of the functions have a multichannel parameter but it looks like it is not implemented yet?

No, it is set to be a (20, 1500, 1500) segmentation, ie, one labels image for each slice in the image stack. You will need to use a script to create a labels layer of a different size than the currently-selected image. (1500, 1500) != (20, 1500, 1500). :wink:

I responded to this on the segmentify repo itself, but agree with what @jni said that project is more of a proof-of-concept. Hopefully there will be more segmentation plugins within napari soon!

I’d like to know more about why you want to replace ilastik with segmentify @shaojus. Is ilastik giving you inaccurate results, or is it that you think segmentify is easier to use with napari?

No, ilastik works great, it’s just that if I could get segmentify to work it would make my segmentation pipeline much more streamlined and easier to use.

1 Like

I am running into this same issue. I have image stacks that look like (pos, time, X, Y) and would like to be able to label a cylinder in timeXY with a single click. This would be much easier to have an API for in the case where I have shape (time, X, Y) but perhaps the labels layer could have an argument on construction like broadcast_dims which could be None for broadcast over all, '3d' for the current behavior, and 1 for the time axis in my first example, or [0, 1...] for arbitrary axes?

Interesting, I’d like to hear more about that @shaojus
I don’t want to derail this conversation, so I’ve sent you a DM

@ianhi sorry I missed your message originally. To be clear, you should be able to viewer.add_labels(np.zeros((pos, 1, x, y), dtype=np.uint32)) to get the desired effect?

@jni no worries and thanks for the response.

Sorry I may have been unclear. I can add a layer with shape

#(S, T, Y, X)
(2, 15, 512, 512)

but what I’d like to be able to do is click on the center of the image and label this slice

[cur_pos, :, <whatever the brush selects in 2D>]

or perhaps
[cur_pos, cur_time:, <whatever the brush selects in 2D>]

This is my motivation for Allow adding custom modes to the Labels layer · Issue #2314 · napari/napari · GitHub