Counting nuclei by Fiji or Qupath (RNAscope chromogenic duplex)


I am doing chromogenic dual ISH (RNAscope chromogenic duplex) analysis.
I’m trying to assess target dots per cell, So I want to count the nuclei.
The counting of blue and red dots was satisfactorily done through the Fiji (imageJ) program,
but I have a hard time counting the nuclei (because of similarity of color between blue dots and purple nuclei).
How can I count the nuclei using Image J or Qupath (cell analysis)?
(Other analysis program recommendations are also welcome.)

I would really appreciate your help. Thank you.

Image_8135.tif (8.1 MB)

Here are a couple of topics to check through. Generally, though, it really depends on the problem stains. If you are ending up with “black,” there isn’t much you can do. IF the stains are still separable, I would set the color vectors to as close to pure hematoxylin as possible, and the second to as close to the confounding color as possible. As usual, it is best if you have a single stained sample (no hematoxylin) to get an accurate color vector, but I’ve usually been able to work with what is on the slide.

Ouch. It looks like you basically have a blueish stain, a red stain, and… a purple stain. Your purple stain varies in intensity quite a bit, which causes some severe problems. I suspect you would need to create a semi-complicated detection in FIJI/QuPath(through FIJI) to detect those nuclei. I was able to come somewhat close using green chromaticity, but I have a feeling you would need some logical statements as well. I can’t find an easy way, otherwise, to include strong red spots on top of nuclei while excluding them external to the nuclei. You might have better luck bluing your hematoxylin more, or better yet, using a different counterstain.

Also, in general the nuclear stain looks a little fuzzy. Not sure if that is the camera or underfixation (or if this is a downsampled subsection of an image), but even if the ISH spots were not there, it would be fairly difficult to segment those nuclei.

Good luck!

Thank you for your quick and kind reply!!
I’ll check the topics you attached.
I agree that it is the best way to using color deconvolution as much as possible through the Fiji program, as you have advised.
And in the case of the file I attached, the nuclear staining seems to be too blurry.
Is it possible to classify colors for newly attached images?

(I am wondering how the expensive image analysis program provided by the manufacturer can automatically identify the nucleus in this case(dual chromogenic ISH… Red-Blue-Purple), The more I try to do that, the more I wonder… When it is difficult, counts the dot automatically, and it seems to be a way to count the nucleus manually.)

Thank you for your kind response again

(8.1 MB)

Even here it looks fairly difficult to segment the nuclei. Not only do they have many varying shapes, in many places the edges are not well defined. In something like Visiopharm I could probably do enough channel math to get something “kind of close” to accurate segmentation, but even that isn’t going to be fully accurate since the information isn’t all there. Your ISH markers do actually obscure the nuclear stain, it isn’t like a fluorescence image where you can actually look at another channel.
Above shows that the teal and red stains are effectively on “either side” of your hematoxylin stain vector, and so I can only use the second stain vector to eliminate one of them from consideration during cell detection.
After using Pete’s script to apply metadata to the tif, this was about the best I could do. Which wasn’t great, obviously.

Since I used the second vector to eliminate teal, it tends towards picking up red spots as part of the nuclear color.

def metadata = getCurrentImageData().getServer().getOriginalMetadata() 
metadata.magnification = 20 
metadata.pixelCalibration.pixelWidth.value = 0.5 
metadata.pixelCalibration.pixelHeight.value = 0.5 
// If you want to trigger the 'Image' tab on the left to update, try setting a property to something different (and perhaps back again) 
type = getCurrentImageData().getImageType() 
setColorDeconvolutionStains('{"Name" : "H-DAB default", "Stain 1" : "Hematoxylin", "Values 1" : "0.53345 0.78583 0.31289 ", "Stain 2" : "DAB", "Values 2" : "0.89818 0.41274 0.15138 ", "Background" : " 184 164 172 "}');
runPlugin('qupath.imagej.detect.cells.WatershedCellDetection', '{"detectionImageBrightfield": "Hematoxylin OD",  "requestedPixelSizeMicrons": 1.0,  "backgroundRadiusMicrons": 0.0,  "medianRadiusMicrons": 2.0,  "sigmaMicrons": 2.0,  "minAreaMicrons": 40.0,  "maxAreaMicrons": 1000.0,  "threshold": 0.15,  "maxBackground": 2.0,  "watershedPostProcess": true,  "excludeDAB": false,  "cellExpansionMicrons": 10.0,  "includeNuclei": true,  "smoothBoundaries": true,  "makeMeasurements": true}');

You would want very different color vectors to pick out the ISH spots. Plus I suspect my estimation of the pixel size was off, and you may want more smoothing and lower or higher nuclear thresholding…