Stardist extension

Hi @petebankhead

First of all, I’ve been trying the Stardist extension, using the current version in github (posterior to M11). I downloaded the he_heavy_augment.zip model from Stardist’s github.

When I run the code, if the ROI (i.e. the annotation) is larger than the tissue, the software finds cells in the background of the image (attached image).

Checking the code, I think the problem is that tiles are independenly normalized. Therefore, modifying tileSize, the problem can be reduced or even, depending the ROI size, avoided.

Is there any way to normalize the whole image, in order to test my theory?

@erme yes, the tiles are independently normalized – since as far as I know that is how it is normally done with StarDist, although @mweigert and @uschmidt83 are the authorities on that :slight_smile:

The normalization is called in the script itself - so it can be changed/turned off. But there is no quick change to the script to support global normalization.

However, the expected workflow in QuPath involves defining your region of interest more closely around the tissue, since it will then exclude large areas of whitespace automatically. With that approach I haven’t personally seen significant problems with the per-tile normalization yet.

There is no requirement that the annotation is rectangular; QuPath should be able to mask out any shape.

Well, I tried without normalization, removing normalizePercentiles, and this is what I get (pixel Size 0.45, tile Size 512). The result is “better” because there are no detections in the background, buuuuuuuuuut… mmm… maybe they are a bit large.

I used a rectangular ROI, just to try, but I was thinking of another approach, that I used to use it. In short, first, to carry out a TMA dearrayer and, later, perform the cell detection. I’ve checked that simple tissue detection can be performed (now) in conjunction with TMA dearrayer. In my old version, it was not possible and there was a gap.

Well, now I understand how it works and how to use it

Thank U!

Yes, there will be other tricks and ways to approach normalization – but it will take some time to write, test and document them. StarDist support in QuPath is at a very early stage :slight_smile:

Indeed, image normalisation in StarDist (both in Python and the Fiji plugin) is currently done before tiling, i.e. globally based on the whole image.

Yes, that is expected if tiles are normalised independently. Maybe you could normalise your image before importing and then apply the script?

Ah, I was thinking (with no justification) it was per-tile…

Whole image normalization in QuPath would be rather a lot more challenging, given that the regions in which it might be applied are potentially huge.

However, I suppose I should admit there is another (as yet undocumented) trick to apply arbitrary preprocessing…

def stardist = StarDist2D.builder(pathModel)
        .threshold(0.5)     // Prediction threshold
        .preprocess(
                ImageOps.Core.subtract(100),
                ImageOps.Core.divide(100)
        )
//        .normalizePercentiles(1, 99) // Percentile normalization
        .pixelSize(0.5)              // Resolution for detection
        .includeProbability(true)  // Include prediction probability as measurement
        .build()

This avoids computations on a per-tile basis, but it’s up to the script user to figure out what sensible normalization values are.

Since ImageOps in QuPath (not to be confused with ImageJ Ops!) were largely written on the weekend before the NEUBIAS webinar and may well change, I was avoiding referring to them much lest it result in lots of questions/breaking scripts in the future – but they are rather useful.

For instance, they provide a way to apply the pretrained fluorescence model on a single color-deconvolved channel (here, with a median filter thrown in for fun):

// Get current image - assumed to have color deconvolution stains set
def imageData = getCurrentImageData()
def stains = imageData.getColorDeconvolutionStains()

// Set everything up with single-channel fluorescence model
def pathModel = '/path/to/dsb2018_heavy_augment'

def stardist = StarDist2D.builder(pathModel)
        .preprocess(
            ImageOps.Channels.deconvolve(stains),
            ImageOps.Channels.extract(0),
            ImageOps.Filters.median(2),
            ImageOps.Core.divide(1.5)
         ) // Optional preprocessing (can chain multiple ops)
        .pixelSize(0.5)              
        .includeProbability(true)    
        .threshold(0.5)             
        .build()
4 Likes

The “correct” approach to normalize the input to the StarDist model really depends on the data. What the model needs is that background pixels are close to 0 and the brightest pixel of actual objects are close 1.

If the image brightness and contrast is about the same everywhere, then it’s fine to compute the normalization percentiles from the whole image or a representative region.

On the other hand, it the image brightness and contrast is very heterogenous, it might make more sense to normalize based on some local region.

What does not work is to normalize an “empty” image region, because that will strongly amplify the background noise and make StarDist hallucinate objects.

I hope that makes sense.

Best,
Uwe

4 Likes

Hi @petebankhead,
Could you explain a little about how the tiling happens in the stardist extension? I’m working with a bunch of small annotations, and I’d like each of them to be processed as a single tile so that the normalization cannot change in the middle of an object. But, even if I choose a tile size that is 2x the width of my object in pixels, according to the log, it still breaks some of the objects into tiles occasionally. In fact, sometimes when I increase the stardist tile size, it divides the same object into more tiles (from 2 to 4). I can probably avoid the entire problem by coding the normalization like you show above, but I was hoping to avoid that.

It is a while since I wrote it, but as best I recall the tiling is really performed at an image level – not a ROI level.

That is to say, it acts as if the tiling is computed from the top left corner of the image, regardless of where your ROI begins. For that reason you can set a tile size bigger than your ROI and still see tiles joins within any ROI.

(This approach comes from the pixel classifier, where it enables features etc. to be cached in a way that wouldn’t be possible if tiles were generated some other way.)

1 Like

Oh, that makes sense! Thanks for the information! I haven’t seen any problems with normalization changing dramatically in the middle of objects, I was just trying to avoid the theoretical possibility that it might happen.

2 Likes