Applying Affine Matrix Result from 2D/3D Registration to Images

I’m using the Plugins/Registration/Descriptor-based registration (2D/3D) to match 2 images. After running the plugin I get a transformation result in the log screen with:

[3,3](AffineTransform[[1.00006214871402, -0.00767959566528, 44.758543863305086], [0.007348764855077, 1.000522805737912, -10.47798079925235]]) 0.3267746111922529

Now I want to apply the transform - affine in this case - to another set of images, so I’m using the Plugins/Transform/TransformJ/TransformJ Affine plugin to transform my image. However, this workflow never gets me the same registration as the resulting stack from the registration plugin. It always seems like there is a translation component missing.

When definining the matrix for the TransformJ I use the following mapping from the original result

1.00006214871402, -0.00767959566528, 0.0, 44.758543863305086
0.007348764855077, 1.000522805737912, 0.0, -10.47798079925235
0.0, 0.0, 1.0, 0.0
0.0, 0.0, 0.0, 1.0

I believe the issue is some coordinate mismatch between the two methods. For example, the registration plugin pads the images, so the coordinate origin matters. I tried in various iterations to separate the translation element and played around with “adjust bounds to fit result” option in the TransformJ to no avail.

Has anybody gone through a similar issue and found a solution?

2 Likes

Hi @Bodo_Schmidt,

Welcome to the forum.

I can’t be sure this is the issue without seeing your result, but I’m pretty sure you need to give TransformJ the inverse of the matrix that the registration based plugin provides you. *See below

Why don’t you give that a try. If that is indeed the answer, post back and I can explain in more detail why it’s the case.

Good luck,
John

*If you have a transformation that maps points (coordinates) from the moving image to the fixed image, you need the inverse of that transformation to render the moving image in fixed image space. this stackoverflow answer has some resources to explain it.

1 Like

Thanks John.

I tried this earlier and that doesn’t seem to be the problem.

When I look at the result it only looks like there’s an offset shift (the scale of the images looks about right). I’m going to try a couple translations before and after to see if this can fix the problem and will report back.

Bodo

1 Like

@Bodo_Schmidt,

I tried it too, and ran into what are probably similar issues. Maybe TransformJ is doing something like using the center of the image as the origin. Whatever it is, its very strange.

I followed up and whipped up a script to apply the transform correctly.
Here’s a script I ran to get some “test data” and a registration result from that plugin.

run("Boats (356K)");
makeRectangle(99, 118, 251, 280);
run("Crop");
run("TransformJ Affine", "matrix=/Users/bogovicj/Desktop/myTransform interpolation=Linear background=0.0");
run("Descriptor-based registration (2d/3d)", "first_image=[boats.gif affined] second_image=boats.gif brightness_of=Medium approximate_size=[6 px] type_of_detections=[Minima & Maxima] subpixel_localization=[3-dimensional quadratic fit] transformation_model=[Similarity (2d)] regularize_model images_pre-alignemnt=[Not prealigned] number_of_neighbors=3 redundancy=2 significance=2 allowed_error_for_ransac=13 choose_registration_channel_for_image_1=1 choose_registration_channel_for_image_2=1 create_overlayed add_point_rois transformation_model=[Similarity (2d)] lambda=0.10");

This is the transform matrix:

0.9659258263    -0.2588190451   0   5   
0.2588190451    0.9659258263    0   -8  
0   0   1   0   
0   0   0   1

The descriptor based registration gave this output:

[3,3](AffineTransform[[0.965962927606217, 0.258811098531563, -34.60594710944924], [-0.258811098531563, 0.965962927606217, 46.11714189733297]]) 1.7976931348623157E308

Then I selected the affine transformed boats and ran this script. It asks for some text that describes the affine - just give it exactly the line above. Look here for help on running scripts.

Then it should transform the image and display it. For me it looks like this:![demo|623x500]

Where

  • top left is the original (Reference) image
  • top middle is the image we want to register to the reference (“moving”)
  • Far right is the result of the registration plugin
  • bottom left is the result of the script i made
  • bottom middle is an overlay of the reference and the bottom left.

Hope that does it for you,
John

2 Likes

Indeed, that’s what I had observed as well in the past. Mix and match between ImgLib2-based plugins (such as Descriptor-based registration) and ImageScience-based plugins (such as TransformJ) should be done with care, as they use different origins.

See e.g. how it’s done in the Rotate class of imagescience:

1 Like

@imagejan as usual you are a champion of finding exactly those lines of code with the answer
:clap: :clap:

<rant>
I’m strongly of the opinion that doing that (using image center as origin) is a bad idea, unless origin information in metadata is respected and consistently applied. ( i remember checking to see if TransformJ touched the “origin” image property, and it did not).
</rant>

Thanks again for tracking that down!
John

1 Like

Fully agreed :smiley:

1 Like

Thanks for the input! It indeed looks like the problem that the image offset for the 2D/3D registration is in the (0,0) corner and the TransformJ image offset is in the center.

I have a workaround that addresses the issue (and can be used in macros) for future reference…

After loading the image that needs to be registered, I use the

Image / Adjust / Canvas Size ... /

function to double the size of the image. It is important to select the position as “Bottom-Right”. Now I can run the TransformJ plugin with the matrix from the registration. Suppose I have the transformation result from the 2D/3D registration

[3,3](AffineTransform[[A, B, C], [D, E, F]]) G

I can generate the transformation matrix

[[A,B,0,C],[D,E,0,F],[0,0,1,0],[0,0,0,1]]

and apply this in the Affine Transformation plugin. After the transformation we need to reduce the image size to its original size again using

Image / Adjust / Canvas Size ... /

For the image size we need to convert back to the original image size and select the position as “Bottom-Right”.

Thanks John - tried your reply and script as well! Works nicely.

Hi @Bodo_Schmidt

Glad you figured out something. Please consider sharing your workaround (even if unpolished) so others might benefit from it in the future. :slight_smile:

TransformJ does have lots of useful stuff, but I’d strongly encourage you to think hard before doing/using anything that uses the image center as the origin. It’s a great way to give yourself a big headache. For instance, the increasing / decreasing of the canvas size that you describe sounds pretty annoying. Of course it’s your decision in the end.

Thanks for participating in the forum!
John

Hi,

I’ve run into this issue as well and would just like to draw people’s attention to another headache.

As pointed out about, the coordinate system in at the centre of image and is in calibrated pixel units!!!. Looking at the code, the ivx, ivy etc are pixel dimensions. This is an easy trap to fall into as 99% of the code works on a pixel basis.

imagescience/ImageScience/blob/3b1c9a4ca26f1e3a3509a113600af3593c8a2301/src/main/java/imagescience/transform/Rotate.java#L257-L266

Cheers,

Chris