Stardist 3D segmentation results to napari surface

Dear @uschmidt83, @mweigert, @jni,

Is there already an established workflow to convert 3D StarDist segmentation results
to surface objects in napari?

Thanks a lot &
Kind regards

Tobias

Hi Tobias,

StarDist and napari developers might have a better answer but in the meantime, here’s something that works. You just have to convert StarDist’s rays/distance output into 3D surface coordinates and combine them with the rays’ faces as input for napari’s surface rendering. Here’s an example with StarDist’s demo example:

# import packages 
import napari
import skimage.io
import numpy as np
from stardist.models import StarDist3D
from stardist.geometry import dist_to_coord3D
from csbdeep.utils import normalize

# use for napari in Jupyter notebook, comment otherwise
%gui qt5

# import image
image = skimage.io.imread('demo3D/test/images/stack_0027.tif')

# import model
model = StarDist3D.from_pretrained('3D_demo')

# normalize image
img = normalize(image, 1,99.8, axis=(0,1,2))

# predict segmentation
labels, details = model.predict_instances(img)

# calculate 3d coordinates from rays and distances
coord = dist_to_coord3D(details['dist'], details['points'], details['rays_vertices'])

# create list of colors to use randomly use for objects
colormaps = ['yellow', 'green', 'red','blue','cyan','magenta']
colors = np.random.choice(colormaps, len(coord))

# create napari viewer
viewer = napari.Viewer(ndisplay=3)
viewer.add_image(image, blending = 'additive')
for i in range(len(coord)):
    # plot each object as a surface object
    vertices = coord[i,:,:]
    faces = details['rays_faces']
    values = np.linspace(0, 1, len(vertices))
    surface = (vertices, faces, values)
    viewer.add_surface(surface, colormap=colors[i], blending = 'translucent')

And here’s the result:

Good luck!
Guillaume

5 Likes

Dear Guillaume,

Wow! That is amazing!! Thank you so much!

Kind regards

Tobias

Hi @Tobias,

yes that is possible - @guiwitz beat me to it! :slight_smile:

Instead of adding each cell as a different surface, you could as well add a single surface:

from stardist.geometry import dist_to_coord3D 
#....
labels, polys = model.predict_instances(x)



def surface_from_polys(polys): 
    faces = polys["rays_faces"] 
    coord = dist_to_coord3D(polys["dist"], polys["points"], polys["rays_vertices"]) 
    faces = np.concatenate([faces+coord.shape[1]*i for i in np.arange(len(coord))]) 
    vertices = np.concatenate(coord, axis = 0) 
    values = np.concatenate([np.random.rand()*np.ones(len(c)) for c in coord]) 
    return (vertices,faces,values) 
    

surface = surface_from_polys(polys)

with napari.gui_qt(): 
   viewer = napari.view_image(img) 
   viewer.add_surface(surface) 

4 Likes

Dear @mweigert,

thanks a lot!
This solution is equally good.
It always depends on the use case.

It just seems ImageSc allows only to select ONE solution…

I like both though.
Thanks again!

Kind regards

Tobias