Cellpose parameter optimisation

Dear @cellpose,
(ping @oburri @simonfn)

I am getting below output in a 3D CellPose segmentation:

I wonder whether there may be a parameter in CellPose that would impose a certain smoothness of the label mask, in order to avoid this very pixelated response in the central cell.

In fact, now writing this question, I wonder whether you would recommend smoothing the gray values in the input image? Normally I don’t pre-process images that are the input to deep learning algorithms, assuming that the smoothing is part of the DCNNs…

Thank you for your help!


While the smoothing can be part of the DNNs, it’s not there automagically. Depending on what CellPose did to augment their data, the noise model might not be the same.

In the case of using pretrained models, it makes sense to feed input data similar to the data used for training. Looking at the images used in CellPose, they are rather high in SNR. I would try smoothing with a median filter, using noise2void or even deconvolution.
In case it all fails then I would retrain.

Maybe @romainGuiet can tell me if I missed something. It’s rather early in the morning for me and coffee is still brewing :grin:


Hey @Christian_Tischer are you using the nucleus model or the cyto model? The cyto model has been trained with images which have a bit more noise, but still not too noisy. If the 3D aspect is failing, but the 2D segmentation of slices looks good, I’d recommend trying to use the stitch_threshold parameter instead of setting do_3D=True. See the docs for more info on that: Settings — cellpose 0.7.2 documentation

We don’t train with noise augmentations, but that may be something to try in the future. I agree with @oburri you probably want to try some filtering first.


Thanks @oburri and @cellpose !

Smoothing the image certainly helped:

I am missing a couple of cells entirely, so I will try now reducing the cell threshold…

@cellpose If my cells have an ellipsoidal shape, should I use the short or the long axis for the diameter?

@cellpose @oburri
We have another question: We have 3D data where the intensity in the first slices is much brighter than in later slices. I seem to observe that the segmentation works best at an average intensity level in the central slices. I assume CellPose does a normalization of the intensity values? Is it thus maybe expected that the very bright or very dim cells are not segmented?

cellpose normalizes with the same factor across all slices, which may be a problem for that. It will likely ignore dim cells. You may want to normalize each slice separately before feeding into cellpose.

this looks a little too oversaturated to correctly split some cell boundaries FYI

1 Like

I’d use the average of the two, but the results can be pretty sensitive to this so I’d recommend playing around with this a bit

1 Like

Hello Carsen,

Thank you for developing this amazing tool. It works out of the box for a few of my images.!!!

I want to improve it further for my usecase. For this purpose, I am training a cellpose model locally on my system. I am getting almost the same value for loss during training.

Log during training:

Can you comment on the loss during training, are these values normal or I might be doing something wrong!?

Thank you in advance!