Color deconvolution implementations & best practice

The posts about colour deconvolution today have reminded me of a question I had some time ago…

Color deconvolution, as described by Ruifrok and Johnston, involves generating a 3x3 stain matrix using three stain vectors.

I understand that if two stains are available, then the remaining elements can be created by generating a third (pseudo)stain that is orthogonal to the first two.

As far as I can tell, this third stain is generated using the cross product in several places:

I wasn’t able to tell what approach CellProfiler takes (I figure this code is relevant, but I got lost :slight_smile: ).

However, I understand that this is not exactly how it is implemented in the @gabriel’s ImageJ/Fiji plugin. From a quick look at the code, this may be because of negative values being clipped to 0:

In any case, the following Groovy script written for Fiji shows that the third stain vector is not orthogonal using the Fiji plugin, i.e. the dot product is not zero:

import sc.fiji.colourDeconvolution.*
def mat = new StainMatrix()
mat.init("H&E", 0.644211000,0.716556000,0.266844000,0.09278900,0.95411100,0.28311100,0.00000000,0.00000000,0.0000000)
mat.compute(true, false, new ij.ImagePlus("Anything", new ij.process.ColorProcessor(10, 10)))

double[] stain1 = [mat.cosx[0], mat.cosy[0], mat.cosz[0]]
double[] stain2 = [mat.cosx[1], mat.cosy[1], mat.cosz[1]]
double[] stain3 = [mat.cosx[2], mat.cosy[2], mat.cosz[2]]
println 'Stain 1: ' + stain1
println 'Stain 2: ' + stain2
println 'Stain 3: ' + stain3
println 'Dot product stain 1 x stain 2: ' + dot(stain1, stain2)
println 'Dot product stain 1 x stain 3: ' + dot(stain1, stain3)
println 'Dot product stain 2 x stain 3: ' + dot(stain2, stain3)


double dot(double[] v1, double[] v2) {
	double s = 0
	for (int i = 0; i < v1.length; i++)
		s += v1[i] * v2[i]
	return s
}

It’s not completely clear to me that negative values must be avoided in the stain matrix, and that this is more important than orthogonality.

It’s also not clear to me if/how much this matters.

I think probably all of us benefited from @gabriel’s implementation – I know I did, and I’ve seen it referred to a lot of time in other people’s code. But even though it seems to be pretty much the standard reference for many (in the absence of the original macro), I’m not sure it’s widely recognized that other implementations seem to have deviated a bit in this detail.

In any case, I’d be really interested to understand if there is a ‘right’ way to do it.

I’m also very interested in whether @phaub might have any more best practice suggestions from this :slight_smile:

3 Likes

Yes, I am aware of that implementation detail in Ruifrok’s code about the 3rd residual colour. A couple of people have made that observation in the past. I will have a look at the other implementations.

1 Like

@petebankhead The Histomics TK link does not work.

@petebankhead
Have you seen the supplement to this paper, here.
We discussed this issue in S1.5_A.

In case of two stains and using the 3x3 matrix approach to solve the over determined system a third ‘pseudo vector’ should be perpendicular to the first two. This can lead to negative values in the pseudo vector. The resulting 3rd cannel value is a quasi measure of the deconvolution error.
The negative vector components have a meaning and should not be corrected.

The right way to do it …?
Since there are a lot of disturbances (homogeneity and stability of the illumination, background and gamma correction, quality, stability and reliability of the staining, the non-linarity of IHC preparation, ect ect ect. and last be not least the invalid assumption of a linear realionship between the polychromatic measured absorbance and the stain concentration) the color deconvolution is questionable in any case.
Solve the system of equations will not dramatically change this.

So, just do it in one of the correct ways and avoid a wrong one.

3 Likes

Thanks, fixed now (hopefully).

I hadn’t seen that - thanks!

Do I understand correctly that using the cross product would count as one of the correct/least wrong ways…? With all the caveats regarding the use of color deconvolution generally, of course.

I ask because as QuPath becomes more widely used (approaching 100k downloads now…) I’m keen that it should promote good practice – or at least not inadvertently makes bad practice widespread. I think this is partly a matter of implementation and partly of documentation.

@gabriel and @phaub since you are the people I have encountered who I think know most about this topic, I particularly value your suggestions :slight_smile: I already link people to your writings, but wondering if there is more could be done – perhaps my using the UI to offer guidance that helps encourage better analysis and appropriate interpretation.

2 Likes

Some quick tests to compute the 3rd vector via the cross product show that you one cannot create nice looking LUTs: the negative components represent impossible colours. I assume that the original implementation (which generated OD images with LUTs to represent the dyes) avoided this by generating an alternative LUT.
Perhaps the plugin should not attempt to generate LUTs in those situations (more of a cosmetic issue) and generate also 32bit images instead.

Yes.

To show the connection between a third orthogonal pseudo vector and the GaussTransformation see the following information

GaussTransformation

So you can either solve the over determined systems by

  • on of the projections into 2D (RG, RB, GB)
  • usage of a third orthogonal residual vector
  • GaussTransformation / Moore-Penrose-Inverse (MPI)
  • SVD
  • QR decomposition

On no account you will get a error free result because of the incorrect assumption underlaying the approach. Even under theoretical perfect conditions (stoichiometric and linear staining, perfect imaging …) the linear approach can not model the nonlinear signal formed by a spectral integration without errors.

The extent of this error can not be estimated easily. The type of and the combination of staining strongly influences this error.

Its like modelling a circle by its tangent and asking for the error of that approximation.
The answer is: It depends.
Simplification only works under particular conditions.

4 Likes

QuPath generates a LUT color as described here (basically clipping values that end up out of range). I’m not sure that’s the best approach, but as you say it’s really cosmetic.

I do however like the option of 32-bit output, since this preserves negative values – which can help as a sanity-check / preserve useful information. One can always clip it later.

However, I understand better now this is starting to move from the original NIH Macro, and a faithful interpretation of the macro is also desirable.

@phaub thank you for the explanation!

3 Likes

For this 3rd residual vector there is no ‘true’ color. The information of this 3rd channel is the amount of error. You can simply assign a FALSE color.

1 Like

Thanks Peter. Did you mean “not necessarily” a true colour (e.g. when there is a negative component)? If there isn’t a negative value, one can generate such 3rd colour as well.

@gabriel
A 3rd residual vector has in no case a color!
Its direction is only defined in a mathematical sense.
In case all components are possitive a derived color will lead to the believe that this color has a meaning. But it doesn’t have.
I personally would assign a unique color to this residual channel to indicate that this channel contains a kind of quality measure (e.g. LUT Phase or Fire).

3 Likes

Hi all,

Forgive my maybe naive question, but in case there is more than 3 colors on the image ; would it be possible to make a color deconvolution with 4, 5 or 6 components (wich is done in some commercial products like Akoya’s Inform). Maybe it is written in Ruifrok and Johnston 's paper, but I fear my mathematical level is not good enough to understand it.

Nico

1 Like

The linear approach discussed here is based on solving a system of linear equations.

To solve a SLE it needs at least as much or more equations than unknowns.

Each unknown is a stain.
Each equation comes from one color channel of your image.

With a color image - typically 3-channels RGB - you can ‘separate’ up to 3 stains.
If you want to apply the linear approach to more than 3 stains than you need more color channels. The images have to be captured as multi-channel images with more then 3 channels.

If you find a suitable camera it would be nice to post it here :slightly_smiling_face:

2 Likes

I have heard that the camera on the Vectra systems could be used to generate a multichannel brightfield image (7 channels?), in theory. Though I have never tested it.

1 Like

@Research_Associate, @phaub, That’s the camera I use. And it does a multi-chanel image (the whole spectra is split in 35 images). But as I mainly use it for fluorescence, I had never imagined that the brightfield acquisition was the same.
But the color deconvolution implemented works very well (if you have the right mono-staining sample in your library :slight_smile: ).

Nico

2 Likes

is based on a liquid crystal tunable filter (LCTF).

1 Like

I would guess that it merges all of those channels into 3, as RGB is used for display on the screen. I’m not sure how you would really look at a 35 channel brightfield image, but it might allow the deconvolution of more channels from the data side?

I bet that a 35 channel image merged into RGB would look like… an RGB image, because displays have only 3 colours and the eye has (most commonly) only 3 sensors.
Most applications I have seen on multi and hyperspectral imaging rely on some kind of data reduction (like PCA) to extract information. Really interesting subject.

1 Like

The conversion of an n-channel brightfield image to RGB is described

see section S1.4

2 Likes

For CellProfiler, the relevant function is here, which calls out to these two other functions - only those 60 lines are “functional”, everything else is display, image saving, or the estimation of custom stains.

1 Like