Nuclear DAB disrupts Stardist detection of Haematoxylin in qupath

Sample image and/or code

The above shows an example of my data, where cells have been detected using stardist in qupath. As you can see it misses many of the cells that are positive for DAB. The DAB seems to be well seperated from the rest of the signal, but where there is strong DAB signal, the heamatoxylin is gone.
A quick question in between: Stardist doesn’t care about the staining vectors right? It needs rgb information for each pixel and that is send to the u-net.

Is there a way to improve the detection, maybe separate and remove the DAB signal prior to sending the RGB image to stardist. Or maybe “recolor” the DAB to have the color of haematoxylin?


If you are using the H&E model for detecting nuclei using StarDist, this is not terribly surprising, as the model is not designed to work on this type of image. Misapplying deep learning models is one of the bigger dangers in their proliferation, and you may want to look into training or transfer learning your own model. Alternatively, if you want to specifically use the hematoxylin channel, there are QuPath stardist posts using the color deconvolved channels (essentially the DAPI model, but pass the hematoxylin “channel”). Again, this is not really the most appropriate use of the model, but it “might work.”
Even in the “does more than one might reasonably expect” example, the DAB is not fully or even mostly obscuring the blue of the nuclei, as it is in your example.

DAB also happens to be a rather terrible dye for color deconvolution processing in that it frequently obscures whatever else is in the same pixels, being brown - I suspect this is part of why QuPath has it’s own built in OD-Sum option for cell detection.

1 Like

Hi @rharkes see my answer here: QuPath + Stardist? - #42 by petebankhead

That will only work off the color-deconvolved hematoxylin values. If you want to sum the hemaotyxlin and DAB values it’s a bit more involved. The following script might do the job (I’ve just written it now quickly, and haven’t tested it with the current QuPath release):

import qupath.opencv.ops.ImageOp
import org.bytedeco.opencv.opencv_core.Mat

import static qupath.lib.gui.scripting.QPEx.*

import qupath.tensorflow.stardist.StarDist2D
import qupath.lib.images.servers.*

// Specify the model directory (you will need to change this!)
def pathModel = '/path/to/dsb2018_heavy_augment'
double originalPixelSize = getCurrentImageData().getServer().getPixelCalibration().getAveragedPixelSizeMicrons();

def stardist = StarDist2D.builder(pathModel)
        .threshold(0.5) // Probability (detection) threshold
                ColorTransforms.createColorDeconvolvedChannel(getCurrentImageData().getColorDeconvolutionStains(), 1),
                ColorTransforms.createColorDeconvolvedChannel(getCurrentImageData().getColorDeconvolutionStains(), 2)
        ) // Select detection channel
        .preprocess(new AddChannelsOp())
        .normalizePercentiles(1, 99) // Percentile normalization
        .pixelSize(originalPixelSize) // Resolution for detection
        .cellExpansion(3.0) // Approximate cells based upon nucleus expansion
        .cellConstrainScale(1.5) // Constrain cell expansion using nucleus size
        .measureShape() // Add shape measurements
        .measureIntensity() // Add cell measurements (in all compartments)
        .includeProbability(true) // Add probability as a measurement (enables later filtering)

// Run detection for the selected objects
def imageData = getCurrentImageData()
def pathObjects = getSelectedObjects()
if (pathObjects.isEmpty()) {
    Dialogs.showErrorMessage("StarDist", "Please select a parent object!")
stardist.detectObjects(imageData, pathObjects)
println 'Done!'

class AddChannelsOp implements ImageOp {

    public Mat apply(Mat input) {
        def channels = OpenCVTools.splitChannels(input)
        if (channels.size() == 1)
            return input
        def sum = opencv_core.add(channels[0], channels[1])
        for (int i = 2; i < channels.size(); i++)
            sum = opencv_core.add(sum, channels[i])
        return sum.asMat()


Hi @Research_Associate ,
Yes, I am using the H&E model, and I know you are right. The model is not made for this type of data. I was pleasantly surprised it works as well as it does. Training a new model or transfer learning on this model myself would be great, but I don’t have annotated data. I have been given one day a week to work with qupath on this type of data.

The good thing in this case is that DAB is a nuclear stain, so even if it obscures what is underneath, I know it must be the haematoxylin. Therefore it seems valid to recolor the brown to blue.

Hi @petebankhead ,

Thank you for the script. I tried it and it runs without errors, but doesn’t capture most of the nuclei. I tried playing with the threshold, but before it detects all cells it starts to detect a lot of cells on membranes. Probably because the heamatoxylin, unlike dapi, also stains other structures than just the nucleus and the H&E model is trained to exclude those.

However I did learn something new about how to send the sum of different channels to stardist, which is something I can use for the analysis of CyTOF data (different project). So thanks again!

If you know some alchemy to turn brown into blue I’d love to hear it.