Aligning and combining multiple multichannel fluorescence images to create one hyperpexed image

Apologies if this question has been addressed before. I hunted around the forum, but wasn’t able to find answers to this specific problem.

I’m trying to combine 8 different 5-channel fluorescent images of the same tissue section to create one 40-channel image. I’m doing fairly standard cyclic-immunofluorescene where I stain, image, remove the fluorphores, and repeat. I’m imaging on a Zeiss Axioscan slide scanner, which gives me large tiled and stiched images with 5 fluorescent channels. The images are usually shifted in X-Y in each round of imaging , but there may be small changes in rotation as well (and I can’t rule out small deformations in the tissue between rounds of staining).

Ideally I’d like to get perfect cell-to-cell aligment across each round of staining for every channel.

The channels of each imaging round seem well aligned to each other, so what I’d like to do is use the DAPI channel of each 5-channel image to do the alignment (that way any changes are applied to the other 4-channels of each image).

I’ve tried combining low-res versions of the tiled images with a few of the included Fiji registration plugins, but the process either fails, doesn’t allow me to choose which channel to use for aligment, or doesn’t include all the channels in the final output image.

This might be a fiarly routine issue, but I just don’t know ImageJ well enough to solve it. Does anyone have any thoughts or advice on how to proceed?

Thanks!

2 Likes

@Doc_Watson

No worries! Let’s see if we can get you some help… Firstly, would be great to have a few more details from you:

  1. what tools/plugins have you tried?
  2. were there errors in running them? what specific problems for which tool?

It could have something to do with the fact that you are trying to process/align lower res versions. Have you tried the original image files? Too - can you post an example image here? Perhaps DAPI is not the best choice - depending on the density of your cells and the image-by-image overlap…

Thanks for the response. There’s one detail I left out, and that’s the Axioscan creates pyramid image magnifications, so each .czi file has 4X up to 20X magnification of the same section. The 20X mag composite is still too big for ImageJ to open, so I’ve been using the lower mag composites for now. Here’s a link to 3 of the mid-magnification images:

https://drive.google.com/drive/folders/1csqYsGbEtIIWIzVQ0zTytalSgA74RK8c?usp=sharing

Right now I’ve tried the FIJI tools MultiStackReg, TurboReg, bUnwarpJ, and TrakEM2. I’ve had the best success with Descriptor-Based Registration (2D) plugin, but this is a long way from being the mostly automated or scriptable process I’ll need.

The errors I’m getting with the other methods could be down to my inexperience with them, but typically they either don’t produce a new aligned image, align the channels in a completely wrong orientation, or only include some of the channels in the final output image. MultiStackReg seemed promising, but it can’t handle images of different dimensions (and the images from the slide scanner are always a little bit different size each round).

It’s very likely one of these tools is the right approach, but I’m just not using them correctly, or I need to change my workflow (such as aligning/registering tiles before stitching them). But I’m hoping the DAPI channel is still the best approach for alignment, because it’s the only marker that’s consistant between each round of staining. For the sake of our current workflow it would be great if I could align the large stitched composites, rather than starting with individual tiles (but I’m open to any appoach that actually works :slight_smile:) .

1 Like

Have you tried HyperStackReg plugin, which would allow you to choose a specific channel (DAPI in your case) for registration? Check it out:

Ved

1 Like

Hi, Doc_Watson

I introduce a house made plugin.
I tried to align your image, it was probably succeeded by the plugin.

CoordinateShift

Firs, I concatenate these images and convert to HyperStackImage.
Then, run CoordinateShift.
And then, press the Shift button.(selected channel will be used for shifting method)
*ch5 is the best
hwada

1 Like

Thanks so much Ved! I’ve been playing with HyperStackReg, and I think it’s really working great for this approach. I made a workflow that successfully generated hyperplexed images, and I wanted to run it by you to see if you had any tips or advice on how to improve it:

I start by loading my first tiled and stitched image as a hyperstack. I can’t use the highest magnification our scanner produces because it’s too large for FIJI to open, so I open a lower mag version (I’d love to know a way to bypass the image size restriction in imageJ)

I load the next stack and concatenate it with the first. I then repeat this for every stack until I have a single hyperstack with 8 unaligned stacks, each with 5 channels.

I load HyperStackReg, choose my DAPI channel for alignment, and pick Rigid Body alignment. (I was trying Affine, but it would frequently misalign one of the stacks, is there a way to improve the results? Like doing Rigid first and then Affine?)

This produces 8 aligned stacks in one hyperstack with the 5 channels merged as composites.

I then convert the hyperstack to a stack, and then back to a hyperstack, except now I make it one stack with 40 channels. Then I delete the extra DAPI channels.

Does this look right to you, or are there ways I could streamline it? I’d like to eventually make the process scriptable.

Thanks!

Oh, and one more question. I’m realizing that dealing with 30+ channel images where everything is named ‘Channel 1, 2, 3’, etc., is a little problematic to keep track of which channel is which antibody marker. Is there a way to give meaningful string names to hyperstack channels? Or otherwise output a tiff with the channel names as metadata? I’m having trouble finding a solution on my own.

Hi,

I have developed my own hdf5-based file format and viewer for this particular use case. However, my code is in Python. The imaging work is still unpublished.

See my github repo here:

Here is a slide that shows the structure of the HDF5 file in HDFView

Note that the file stores the registration information between channels.
I might see whether I can create a public repo with the code that generates these hdf5 files.

2 Likes

Hi @Doc_Watson

You workflow seems alright to me. I think, trying Affine after running rigid body is a good idea. See if it improves results.

Things like above can be easily scripped with an ImageJ macro.

Regarding keeping track of your channels, you could set the slice label in your final stack with:

setMetadata(“Label”, string)
Sets string as the label of the current image or stack slice. The first 60 characters, or up to the first newline, of the label are displayed as part of the image subtitle. The labels are saved as part of the TIFF header. See also: getMetadata.

Ved

@Doc_Watson

To streamline your process my suggestion would be:

  1. For each of your dataset, generate a single hyperstack with 5 channels and 8 unaligned frames. A simple macro could save you some time/clicks, in case you have a lots of images. Save these hyperstacks in a folder.

  2. Use the HyperStackReg_processFolder.ijm macro to run HyperStackReg on all the files in the above folder.

  3. Optional: run the above step again with the Affine transformation, but first, you will need to check if running rigid body and then Affine improves your results or not.

  4. Like I suggested in my previous message, you could use a simple ImageJ macro to convert all of your registered hyperstacks into single stacks with DAPI channels removed.

I don’t know what kind of analysis you are subsequently performing with the aligned images, but if you work with 8-bit images, you might be able to open your highest mag images in Fiji.

HTH,
Ved

@vedsharma
Hello,
I am setting up a similar workflow and your plugin works beautifully for me!
Is there any way of instructing the plugin to only keep the parts of the image that are “covered” by all channels, i.e. the intersection of all the channels?

Thanks a lot for your contribution!
Greeting,
Felix

Hi @felixk,

Good to hear that my plugin is useful to you!

HyperStackReg plugin processes the whole image area, it does not have in-built functionality to work only on part of an image. I think what you might need is to pre-process your image to get the parts of the image that are covered by all channels and then run the plugin.

An example image might help, if you could post it.

Ved

Thanks for the Quick reply!

I figured an illustration might be helpful, so here it is:

Say I’m trying to align 3 images (or stacks actually).
The plugin aligns the three image by shifting them along the xy-axis.
This means there will be areas covered by only one of the images, two, or all three of them.

I am looking to only keep the part supported by all three images (the intersect).

I am aware that I could simply manually inspect the final merged image and just crop it, I was just wondering if there’s a smarter way or an inbuilt functionality…

Best,
Felix

Hi @felixk,

There is no such in-built functionality in the HyperStackReg plugin. But it seems to me that a simple macro could be written for finding the parts of the image covered by all the three channels.

Ved

Hello, I was curious if this work has been published yet I am curious to read the final paper or pre-print.
Thank you.

Hi @ArronSullivan,
unfortunately not. I’ve had a few attempts at writing everything up but there have been some rough edges with the data analysis that prevented me from doing that (and then I changed jobs).
I haven’t given up on it completely though and hopefully I’ll find some time. For the time being the references at the Github link and the included VizBi 2016 poster are all the written material that is in the public domain.

(Note that the viewer was just a side-project. The main project was fully automating the liquid handling, imaging and data analysis. The viewer could now be implemented in napari very quickly as it also provides a layer system, and in fact I had a play with that a while back )

Hi @VolkerH, Thank you for your reply. I have also been playing with napari recently, it is very nice you were able to quickly convert your viewer over. Best of luck with the new position and thank you again for sharing your work.

1 Like