Cell segmentation for extremely dense tissue

Hi all,
In some of melanoma multiplex samples immune infiltrate is so dense that cell segmentation becomes rather challenging.
I found that in those dense samples original Qupath cell segmentation works better than StarDist. Still there is a room for improvement (please see image).

Are there any hints for further improvements?


For samples like this, there often is not a good answer since there is seldom enough information in the image to accurately segment the cells. Especially not in a single channel. At least one other group (unpublished) has created a deep learning algorithm specifically for segmenting cells in this sort of area on their own, since nothing off the shelf worked well. Specific algorithms like that tend to apply to specific staining patterns and imaging modalities, though.

DeepCell might be worth a shot if you have a second channel that consistently separates the cells (some universal cytoplasmic marker), but I have not had a great deal of luck with it.

The most success I had with lymph nodes (similar density) was taking the image on confocal at 40x or higher. Then there was enough resolution, and the depth of focus was thin enough, that segmentation became reasonable based on the information in the nuclear channel.

1 Like

@Alice_in_multiplex all my tips for QuPath’s cell detection are in the YouTube tutorial – although I suspect you’ve pushed that as far as it can go, and it just isn’t capable of anything better.

For StarDist, adjusting the input pixel size can be important (somewhere around 0.5 µm per pixel is a good starting point, but I find moving that up or down a little can make a difference). And don’t forget that you can train your own StarDist model using annotations from your specific images.

In that regard, see also ZeroCostDLC4Mic.

1 Like