Automatically align and rotate images

Hi everyone,

I have some large stitched images of consecutive tissue sections, each stained with different dyes. I’d like to align them in order to analyse the stains together. To achieve this the images are going to need to be rotated a bit as well as moved in xy. I’d like to know if there is a way to do this automatically? I understand that some approaches achieve this by having the user mark points of reference, but I’d ideally like to avoid having to do so with large numbers of images.

Thanks!

3 Likes

I think this was the primary reason we ended up getting Visiopharm, despite the fact that it wouldn’t output the aligned images. QuPath can translate objects between images after alignment (so that you can generate new data in those objects based on the new image), but the whole process is semi-automated right now. And affine only, which usually doesn’t work well for consecutive sections except in a very broad tissue organization kind of way, you won’t likely get accurate cell by cell data.

Would be neat if anyone has anything effective on whole slide images, handles deformation, and is open source.

1 Like

@DStirling,
@bcimini

I am not sure if Cellprofiler has stitching module/application, but it might be worth exploring this link.
Nevertheless, I am sure Bio-formats can do image stitching. Again on complete automation I am not sure, you might have to write a small script or so.

Regards,
Lakshmi
Fujifilm Wako Automation (Consultant)
www.wakoautomation.com
For CellProfiler training or optimised pipeline write to,
lakshmi.balasubramanian.contractor@fujifilm.com

SLAS 2019 Advanced 3D Human Models and High-Content Analysis Symposium

Read more on our site.

Yokogawa CV8000 - The Ultimate in Confocal HCS
https://www.wakoautomation.com/products/yokogawa-high-content-imaging

Hi @lakshmi,

We’re looking to align whole images, rather than trying to stitch them. They could be imagined as z planes. Sorry if that wasn’t clear. To align properly there will need to be some vertical and horizontal movement, which isn’t too difficult, but they’ll also need a bit of rotation which CellProfiler doesn’t seem to be able to do automatically.

Thanks

1 Like

Hi @DStirling,

I recommend checking out:


by @Christian_Tischer

It’s a wrapper around elastix, a tool I use often and have had good experiences with.

John

3 Likes

@bogovicj - That seems to work perfectly, thanks!

3 Likes

If you have suggestions for improvements, please let me know. I am maintaining this wrapper and am happy to have input from users.

3 Likes

Hi @Christian_Tischer,

It’s working pretty well! It’d be nice to be able to specify where to save transformed images, rather than them having to be placed in the temporary working directory. Also, when working with composite images is there an option somewhere to have it rebuild the composite after transformation? At the moment it’ll open each transformed channel in ImageJ seperately. It’s not too much trouble to write a script to overlay the channels again and then save them with an appropriate location/filename, but I’m wondering if there’s a better solution.

Thanks!

2 Likes

Kind of curious about this as I might know someone who would benefit, but what size images are “large” for you? File size, pixel count?

1 Like

@Research_Associate

I suppose it does depend on a personal definition of ‘large’. I’d consider ‘large’ to be a single image which starts to challenge ImageJ’s default memory or dimension limits. In this case the images are >30,000 x 30,000px, although for testing I’m using scaled-down versions.

1 Like

Hmm, I just recently became interested in this again, and I’m guessing the 80-100k px per side is going to make things difficult for FIJI associated plugins. Maybe elastix can be used on whole slide images directly? Will be looking into it, didn’t see a size limitation, but I haven’t searched that extensively yet.

I think in Fiji this could be a problem. But you can also use Elastix directly on the command line or from python. I do not know what the limitations are there. You can ask here, I think: https://groups.google.com/forum/#!categories/elastix-imageregistration/elastix

2 Likes

You’re right, there’s a known issue in ImageJ where it simply can’t open images above ~27000x27000 pixels properly. I expect running elastix from the command line should work, but I’d anticipate that on such a large image it may take a few hours to process depending on the settings used.

If you’d like to scale it turns out that IrfanView has no trouble handling oversized images and scaling them down to something ImageJ can handle.

2 Likes

Thanks, that was about what I figured. Scaling down isn’t going to work, unfortunately, as there is a good chance we will need to do some deformation (so no snagging the low res affine) to get approximately cell to cell alignment. It may just be a slow, grinding process. But if elastix can get the job done, in the end it is just computing time. Potentially cloud computing time at scale. Maybe.

1 Like

Come to think of it, I don’t think the ImageJ wrapper actually needs to open the images within ImageJ itself. I think it essentially just sends the command to the external program and there’s an option to export the result rather than opening it in ImageJ, so you might be able to just use the plugin anyway on the full size images.

1 Like

@Research_Associate,

Apologies if you know everything below already.

The following is possible (and easy), as long as the metadata for your images are correct.

  1. Downsample an image (making sure the pixel spacing in the metadata is correct relative to the original)
  2. Register that lo-res image to some target (also possibly low-res), resulting in some transform (T)
  3. Apply exactly that transform (T) to the high res image. .

The result of (3) will be what you would want (when using elastix and other good registration libraries).

This will be a problem only if there is high spatial frequency deformation necessary. In every task I’ve come across, this kind of deformation is exactly what I want to avoid. The point being, I find it really useful to go through the above process since (in my experience) using the highest-res original images is computationally wasteful (in the best case), and sometimes gives worse results.

You’re probably right (@Christian_Tischer ?) , though it may not be in the case that the file is in a format that does not play nicely with ITK (see this thread)

John

1 Like

Thanks! And ideally what you described first is exactly how we would want to proceed, but I don’t think it will give accurate cell-cell information due to problems with the slide scanning (stitching artifacts), slight stretching and sliding of parts of the tissue, etc. If you have been able to get single cell accuracy on overlays using downsampled alignment, that is great news, and hopefully we will see similar!

With practice much of those pre-analysis problems will be mitigated, but who knows. The plan is to give downsampling a try first anyway and seeing how the results look due to there not being a quick and easy way of accomplishing the high res alignment that would be preferable.

The objective is multiplex cell by cell quantification of whole slide images using strip and restain methods. That will require very precise alignment of not so perfect images, though the overall tissue structure shouuuuuuuld be the same. Though it won’t be :slight_smile:

I have seen this work and accomplished it in Visiopharm, which I have access to, but Visiopharm doesn’t play nice with outputting the aligned images for analysis in other software which makes that usefulness… limited. Plus, figuring out an entirely open source pipeline has it’s own benefits.

1 Like

You know your data better than I do, but if its relatively low effort, then I’d say its worth a try.

:+1: :+1:

:laughing:

:muscle:
Would love to hear how things work out, please keep us posted!

1 Like