(Elastix-) Registration of images after downsampling with different factors

[EL_Pollo_Diablo (Johannes Müller)]

Hi Forum,

I’m having a (bit theoretical) question regarding transformations of images as determined with Elastix.

My scenario is as follows:
I am registering consecutive histological slices (~30k x 30k) of different stainings (HE & IF) with each other. This is the workflow:

1. Downsampling and masking the fixed and the moving images. Both images are downsampled by the same factor s.
2. Use Elastix to align the images (similarity)
3. Multiply the translation part of the transformation parameters with s and apply transformation to original resolution.

This works for most cases. In some cases, however, both input images differ in size a lot (25k x 25k vs 20k x 20k), which is troublesome for Elastix because the relative difference in size is maintained after downsampling the moving and fixed image. One workaround is setting the registration to very aggressive optimization as described in the Elastix manual. I tried to go another way, but couldn’t get my head around it - that’s where the actual question starts:

Experience shows that registration with Elastix works extremely well if the objects to be matched in both images have about the same size. So I downsampled the moving and fixed images (let them be called M and F) by factors s_1 and s_2 so that I have images M’ and F’ of equal size. The respective transformations are S1 and S2.

The similarity transform in Elastix is defined as:
T(F’) = sR(x’ - c) + t + c
with R being the rotational matrix, s being the scaling factor and t, c being translational and center of rotation components.

Now theoretically, in order to find the transformation T(F) → M for the high res image, I should just have to do the following substitution:
T(F) = S1^{-1}( T(S2(x) ) ) = \frac{1}{s_1} \cdot (s\textbf{R}(s_2 \cdot x - c) + t + c)
then rewrite this to match the similarity transform formulation in the transformation parameter file and let transformix to the job. The registration works well in low resolution, but I can not seem to find the correct parameters to write into the elastix transformation file for a correct output in high res. Is there a flaw in my argument? Has anyone tried something similar?

I do understand that running the registration on images of equal downsampling factors with aggressive settings is a legit workaround - but I would consider the chained transformations as described above a more elegant solution. The concept should also work for affine transformations. Thanks in advance!

Best, Johannes

[bogovicj (John Bogovic)]

Hi Johannes (@EL_Pollo_Diablo ),

The registration works well in low resolution, but I can not seem to find the correct parameters to write into the elastix transformation file for a correct output in high res.

I’m going to completely skip discussing the theory part relating to “translating” transformations applied to low-res images to transformations for the high res images. That’s because I’m confident you will not / should not need to worry about it because elastix can do this for you. (but if you want I can not-skip the theory and discuss, because I do like that kind of thing )

Here’s how elastix can do the work for you:

1. Downsample your images like before
2. Set the image metadata for the downsampled images correctly to reflect the downsampling (see below)
3. Run elastix as usual on the downsampled images with metadata from step (2)
4. You can apply the transformation from (3) directly on your high-res images.

The main idea is: Don’t think about transformations in pixel space, but rather in physical space. A downsampled image is in the same physical space, “just” a different pixel sampling. So a transformation applied in a physical can be straightforwardly applied to any discrete sampling of the same physical thing.

Hope that helps, and do follow up if anything needs clarifying,
John

Other assorted / related information below

Demo

I taught this idea as part of my course in i2k 2020.
Follow the steps here in the section on:
Use your image metadata (and speeding up registration)

Example metadata

Original image metadata

Image metadata after downsampling by 2

[Research_Associate (MicroscopyRA)]

I can’t wait to see that i2k2020 video then I took a quick run at Elastix a couple of years ago and was not able to get good results with some 80kx80k+ images, so it would be interesting to try to figure out where I went wrong.

[bogovicj (John Bogovic)]

I can’t wait to see that i2k2020 video

I’m sooo slowww.
(Thanks for the patience, I’ll get there)

[EL_Pollo_Diablo (Johannes Müller)]

Hey John @bogovicj ,

thanks for the help, this definitely eases my problems - I wasn’t aware that the transformation parameters would refer to world coordinates instead of pixel coordinates. Seems like I really should have attended that course!

However - I can’t quite get that to work and I suspect it’s a matter of image formats. I’m (almost exclusively) using tif images, registering two images with proper metadata settings

gives me the following Transformation Parameters:

(Transform "SimilarityTransform")
(NumberOfParameters 4)
(TransformParameters 1.020258 -0.156742 114.441305 -420.632466)
(InitialTransformParametersFileName "NoInitialTransform")
(UseBinaryFormatForTransformationParameters "false")
(HowToCombineTransforms "Compose")

// Image specific
(FixedImageDimension 2)
(MovingImageDimension 2)
(FixedInternalImagePixelType "float")
(MovingInternalImagePixelType "float")
(Size 5000 4590)
(Index 0 0)
(Spacing 1.0000000000 1.0000000000)
(Origin 0.0000000000 0.0000000000)
(Direction 1.0000000000 0.0000000000 0.0000000000 1.0000000000)
(UseDirectionCosines "true")

// SimilarityTransform specific
(CenterOfRotationPoint 2245.9805312096 2120.9890962839)

// ResampleInterpolator specific
(ResampleInterpolator "FinalNearestNeighborInterpolator")

// Resampler specific
(Resampler "DefaultResampler")
(DefaultPixelValue 0.000000)
(ResultImageFormat "tif")
(ResultImagePixelType "short")
(CompressResultImage "false")

I guess the Spacing Parameter is not read accurately - (Spacing 1.0000000000 1.0000000000) should probably refer to the correct pixel width? I tried using nrrd images instead of tifs for the downsampled images, with these metadata parameters:

but it seems that ImageJ implicitly assumes that nrrd are always 3d images: The exporter is writing the space direction vetor with three elements despite it being a 2D image, which causes trouble for Elastix

Can you suggest an image format that satisfies the requirements of Elastix regarding metadata tags and which would have an exporter implemented in Fiji, too? Thanks in advance!

[bogovicj (John Bogovic)]

ImageJ implicitly assumes that nrrd are always 3d images

Booo! I’ll see if I can investigate.

MHD/MHA is another one to try.
I was going to also suggest nifti (nii), but I’m not sure Fiji can write to it, ;I thought it used to be able to. More investigation still.

[EL_Pollo_Diablo (Johannes Müller)]

@bogovicj

MHD/MHA is another one to try.

Following up on that: It seems like Fiji doesn’t write scale parameters to MHD/MHA files at all - if I write a scale setting to Images in FIji, save it as MHD/MHA and reopen it, the settings are not preserved.
So I think for the time being I think I’ll stick to my previous workaround with very aggressive registration and upscaling the translation parameters in the transformation file by hand.

Plus, I would prefer to stick to tif files and Fiji - the images I’m working on are intended for a TB-sized data publication and I…hesitate a bit to change all datatypes to nrrd and protocols towards, for instance, python

It would be nice, though, if Elastix were able to infer the scale parameters from tif metadata (origin set to 0,0 by default and space direction vector from pixelwidth metadata field)!