Elongated and overlapping cell segmentation challenge


@constantinpape @fjug @akreshuk @Sebastien @iarganda @mweigert @bcimini

We would like to know if some of you are aware to solutions that would help to segment cells in images that look like the one below.

Example image

Gray: Transmission
Magenta: Nuclei
Arrow: highlights crossing of two cells

Raw data

im.zip (3.4 MB)


The main challenge is that the cells are crossing each other with their very long protrusions.
So we would need some logic of pixels belonging to more than one cell. And also there would need to be a logic that after a crossing the cell should continue kind of straight such that after the crossing it still has the correct label.

We are very thankful for any pointers!

Christian & Veronika


Hi @Christian_Tischer,

looks like an interesting segmentation challenge. One question:
Do you really need multiple ids per pixel or would it be ok, if objects are not spatially connected? I.e. in a crossing the pixel would (ideally) get the id of the cell on top, but the cell below would continue and only be “occluded” by the upper cell.

In terms of segmentation approaches, I can’t really think of anything “classical” that would work right now, would be very interested to hear if anyone has some non Deep Learning ideas.

In terms of Deep Learning, I can think of two approaches:

  • Predict pixel embeddings and use a clustering algorithm that does not enforce spatial connectivity.
    The basic approach comes from this paper and this paper uses it for a conceptually similar application. With this approach, you would obtain a single id per pixel and objects would be occluded at crossings.
  • Use a mask r-cnn like approach, which would allow multiple ids per pixel, because the segmentations derived from the masks can overlap.

This is not particularly helpful from an open source point of view, but it did remind me of an advertisement I had seen for MIPAR, which keeps showing up in my LinkedIn feeds.

Scroll to the bottom to see both a “traditional detection” and “deep learning detection” (which looks worse to me) for nanofibers.

If you are in a rush and want to try something like that, maybe they could give you a demo. I don’t know anything about the company, though.

Hi Christian,

It’s always a challenging problem!

A few years ago now I wrote a plugin called IdentifyLinearObjects back in CellProfiler2 (based heavily on CellProfiler’s IdentifyDeadWorms module); it handles overlapping just fine, but assumes the whole object is linear, not just part of it. You could perhaps use it to call processes, then relate the processes to the somas, but it’s a definitely a far-from-perfect solution.


Could you comment what you mean by this? Would the object ids before and after the crossing be the correct ones?

Very good point! I fact, now that you say it: Probably not! It would be ok if the pixels at the crossing point are randomly assigned to one of the cells.

Would the object ids before and after the crossing be the correct ones?

Yup! Rather than storing the segmentations in a traditional label matrix, where each pixel is definitively assigned to 1 object, it stores each object’s segmentations in a separate layer, so that the same pixel can be assigned to many objects. Because we enforce that it should try to make it look like a straight line, an x shaped object usually successfully breaks down correctly into the two corresponding lines- see Great Overlaid Rigid Rod Segmentation Challenge.


Judging from the one image I’d say it should be possible. Still, I think it would come with some major effort since the only solutions I can think of to automate this problem are not simple and require some serious ‘love’ to be parametrized usefully.

DISCLAIMER: below this line I just spell out what comes to my mind. Giving the problem more thought might make me change all uttered opinions…

I would try to:

  • Perform a foreground/background segmentation in one of 1000 known ways.
  • Then use the fluorescent nuclei (or whatever it is) as seed points, constraining the space of all possible solutions only to those that have precisely as many segmented objects as one detects nuclei.
  • Use one of many existing methods developed in the context of e.g. blood-vessel segmentation or connectomics. What comes to (my) mind are the methods by Björn Andres or Pascal Fua (et al., of course). These methods can in principle incorporate constraints that express that cells, when crossing, continue on straight.

Here one such paper: Türetken, E., Benmansour, F., Andres, B., Pfister, H., & Fua, P. (2013). Reconstructing Loopy Curvilinear Structures Using Integer Programming. Presented at the CVPR '13: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, IEEE Computer Society.

If anybody reads this and thinks: good, this Florian guy must be quite old… I’d be happy to hear about more modern approaches for such problems… please post your ideas! :slight_smile:



11 posts were split to a new topic: Issues running identifylinearobjects.py in CellProfiler

Thanks @bcimini

This is great!

I managed to get your solution working on a simple toy example:

cellprofiler2.2.0 pipeline, input data, and identifylinearobjects.py

pipeline and data:Archive.zip (17.8 KB)
identifylinearobjects.py: identifylinearobjects.py.zip (6.8 KB)

input image

output objects

The next challenges will be to combine this with the seeded watershed based object detection, using the nuclei. And also to see what happens when there are also curved objects in the same image. I will work on it and post any progress here.

cc @v_saharuka

1 Like


I tried with a more realistic example where the lines are not exactly straight, but I could not find parameter settings that would work.

image for download: lines002.tif (244.3 KB)

Do you think there could be settings that would work for such data?

Given this comment by @bcimini:

… I guess it wouldn’t work as is right now.


good point. I missed that.