Positive cell detection not picking up cells in large annotations

Hi Everyone,

I am having an issue with positive cell detection on TMA cores. On some of the cores, the cell detection is not picking up most of the cells. This appears to be dependent on the annotation size. If I make a smaller annotation within the TMA core, it will pick up all the cells with no problem. As soon as I make the annotation the size of the core (1.5mm), it suddenly can’t see about 70-80% of the cells anymore.

It is only doing this on some of the cores. It does not seem to be affected by the level of DAB staining as it happens on some cores with almost no DAB staining and other cores with lots of DAB staining.

I’m not sure if these images are going to upload or not so I may create a second post with the images.

Any suggestions would be greatly appreciated.


It will be helpful if you post the exact cell detection parameters you have used.


And/or try to turn off whatever you have done in terms of Background radius.


Hi Ajay, thanks for getting back to me.

I’m attaching an image of the parameters used.

I tried turning off the Background radius and that really helped for that one core but it caused problems with other cores. I have the background radius set high to deal with background staining in stromal areas with very few cells. If I turn off the background radius then all that really light background is suddenly being labelled as cells.

Generally, I pick 4-5 representative cores and work out the best parameters for positive cell detection, then I delete those annotations, apply the TMA de-arrayer to annotate everything and then finally apply the positive cell detection parameters I worked out before.

What I don’t understand is, why would it be missing so many cells in some cores but not other seemingly identical cores. And why would it find all the cells if I use a smaller annotation but lose them if I use a larger one on the same core?

Thanks again for your help, I appreciate it.

Hard to say too much without really testing things on an image, but that maxBackground setting is two orders of magnitude below the default, which is set at 2.0 (log scale for OD) to eliminate tissue folds and dark area.

There have been several other posts where people have set thresholds and background levels that have been problematic based on the particular tile they were processing at the time.
Pete gives some explanation here:

And this was likely an issue with the same.

I am forgetting the rest of the places this has come up, but I have seen it a few times. Not sure what the right answer is, but it probably involves increasing that maxBackground value and maybe playing with the radius.

When you change the background radius, you may also have to adjust the threshold to work for that background.

Did you try optimizing the staining vector to reduce amount of DAB and other contributions in to the Hematoxylin channel, and may be then you can set the background to zero and adjust threshold. It may help.

I remember having hard time setting the maxBackground value, without having any numbers in front I did not quite understand what might be a good value for the maxBackround. Do you have any tips on how to find appropriate maxBackground value?
I have been avoiding using background substation and I prefer to set high threshold values to mitigate issues with uneven backgrounds from one region to another.


I generally only use the maxBackground to remove cells in tissue folds. So when areas are pretty close to black in brightfield images, or you get realllly high intensity DAPI in IF images. I have never really been happy with the IF results as there are often very bright cells I still want to keep.

I briefly mention it here, but the values tend to depend on the image.

Usually 2 is a bit higher than I end up using, but the exact threshold will, I think, depend on your color deconvolution settings (if you have or have not set a background). Whether you can use it successfully will also depend on how dark your staining is… it is really only useful if your real staining is quite a bit lighter than your tissue fold OD.
*broad staining, I suppose I should say. So like- cytokeratin. Your radius can take care of dark but non-pervasive stains like CD8.

I think I usually end up with around 1.0, but again in cases where the OD Sum of the DAB (or other stains) is not over 1.0. Radius again kind of depends on results. How much real data are you willing to lose to exclude however many “bad” cells. I think I usually made it a little bit higher than the default 8… but it has been a while since I did much brightfield.

And to be perfectly honest, I would rather use the pixel classifier these days to exclude such areas, unless I really need to save time or I am doing something quick. And the time taken to troubleshoot the values… meh.

1 Like

@Research_Associate, thank you for the explanation.

For many images StarDist have been a blessing to identify nuclei, thanks to @petebankhead and original StarDist authors, @uschmidt83 and other contributors for making it possible.