Thanks for this source.
With this I could test the alignment of fluorescence and brightfield images of different magnification and I learned that it is not easy to work with different image types. The contrast is very low in the alignment overlay. An improvement would be nice here.
And - as mentioned here -
a 100% alignment on cell level is hardly possible.
Anyway, here is an example showing the overlay of Hematoxylin channel (from P2A_B6_M10_HE_0001-Scene-1-ScanRegion0.czi) and DAPI channel (from 2020_04_25_GNI__0097-Scene-1.czi)
thanks for using the data Maybe a note on the sections: The stainings are from subsequent sections (5µm if I’m not misstaken), so individual cells will probably never align ideally. But I think if you can get the outlines and the brain regions to align, this is of great help!
You would have to adjust the evaluation process to take the remaining missmatch into account, for instance by counting cells or doing colocalization analysis in a tile-wise fashion.
What transformations does the ImageCombiner allow?
I have created a second version of the ImageCombiner which optionally applies the display settings (channel selection, Brightness&Contrast) of the overlay image if this image is opened in a separat viewer.
The principle is shown in the following example.
The H&E image is opened in a second viewer and the ‘optical density sum’ channel is selected.
This OD channel is now used for the overlay instead of the original H&E RGB colors.
During my effort to find a solution I got totally lost in the code. After many attempts, I finally found a way out of the labyrinth, a simple and very straightforward one. It is clear and unambiguous and already prepared in the origin.
Assuming that this modification is also interesting for ImageAlignmentPane.java, I will create a PR.
The idea was suggested to support channel selection in image concatenation:
There seem to be some use cases where this could be helpful.
Here I provide a 3rd version of ImageCombiner, which offers the possibility to combine only certain image channels into the concatenated image.
The approach uses the viewer’s Brightness&Contrast settings to derive the channel selection for all images displayed in a separate viewer.
The following examples show 4 images displayed in different viewers with different channel selection. For the base image, the channel selection is R, G, and B. For the overlay images, stain 1, 2, and 3 are displayed.
The approach to use the viewers’s channel selection is not intended to be ‘optimal’ overall, but it is the best I could achieve within the current QuPath code layout.
To understand my ‘workaround’, some background should be explained:
I) In QuPath brightfield images are handled in a special way. From the RGB channels - the only available channels in the brightfield image data - several additional channels are dynamically generated, i.e. stain channels, ODsum channel, normalized OD channel, hue/saturation/chromaticity channels. These additional channels are not present in the image data itself.
If such channels are to be inserted into the concatenated image, they must be ‘regenerated’ by ImageCombiner.
Currently, ImageCombiner only supports the ‘regeneration’ of stain channels.
(Supporting other channels as well would be possible in principle, but the complexity of the current approach would be overloaded and it would be better to look for an alternative design).
II) If stain channels are added to the concatenated image, only the relevant stain channels are added.
This means that for the 2-color brightfield types Brightfield (H-DAB) or Brightfield (H&E), the remaining stain channel is not added to the concatenated image.
For the Brightfield (other) image type, all 3 stains are added to the concatenated image.
III) Channel selection in brightfield images is limited to a single channel at a time or a combination of R, G, B channels. Therefore, a decision was required on how to ‘interpret’ brightfield channel selection in ImageCombiner.
The details can be found below.
ImageCombiner Functionality / important notes:
Any number of images can be combined.
Images of different types (fluorescence, brightfield), (RGB, nChannel, 8bit, 16bit …) can be combined.
The channels of brightfield images are added as separate channels, even if the original data format was packed RGB.
The image format of concatenated images is always ‘Fluorescence’, because all channels are treated as separate channels. (Do not change the image type to Brightfield!).
The base image is the image of the current viewer.
Overlay images may or may not be displayed in their own viewer.
When an overlay image is displayed in its own viewer, the viewer’s channel selection is used to display the overlay image and when the new concatenated image is created.
The channel selection of overlay images displayed in their own viewer can be changed between the alignment phase and image creation. The channel selection is updated in ImageCombiner by ‘re-choosing’ the overlay images (just press button ‘Choose images from project’ and confirm with ‘OK’).
If an overlay image is not displayed in its own viewer, all* image channels are added to the concatenated image. (all*: depending on the image type, see below).
The overlay images are aligned one after the other. Each overlay image has its own affine transformation.
The new concatenated image is added as a new persistent project entry and displayed directly in the current viewer.
The base and overlay images can be removed from the project after the concatenated image is created.
The channel names in the concatenated image are derived from the base and overlay image names and the channel names.
Channel selection in fluorescence images
In fluorescence images, each channel is already present in the image data. No channel is created dynamically. Multiple channel selection is possible in the viewer. The single or multiple channel selection of the viewer is used for overlay display and image concatenation.
If a fluorescence image is used as an overlay image and is not displayed in a separate viewer, all channels are added to the concatenated image.
Channel selection in brightfield images
In brightfield images, channel selection is more complex for the reason stated above.
The following logic is currently used in ImageCombiner to “interpret” channel selection:
If one or a combination of R, G, B channels is selected, only the selected R, G, B channels are added to the concatenated image.
If the ‘Original’ channel is selected, the R,G,B channels and all relevant stain channels are added to the concatenated image.
With any channel selection other than a combination of R, G, B or ‘Original’, only the relevant stain channels are added to the concatenated image.
(As mentioned above, ‘relevant’ means 2 stains or 3 stains, depending on the brightfield type.)
Currently there is no possibility to add only a single stain channel to the concatenated image.
If a brightfield image is used as an overlay image and is not displayed in a separate viewer, the R,G,B channels and all relevant stain channels are added to the concatenated image (similar to channel selection ‘Original’).
Despite intensive testing of this new ImageCombiner version, it cannot be ruled out that errors have crept in or that the behavior deviates from the above definition.
Please let me know if you notice any irregularity.