Descriptor based registration: How to reuse the 3d-affine model

Hi all,

I am working with the descriptor based registration from @StephanPreibisch plugin and would like to reapply a model to a set of 3D positions using an external code but can’t figure out the right operation.

To be more specific, my user has already analyzed a large population of cells but has only realized later that there is a shrinking effect between his 2 rounds of imaging, unfortunately his final analysis requires a perfect overlap of the 2 rounds of images and the spots he extracted from them. He could redo the whole analysis after registering the images, but I wonder if we could still do the registration of the images and apply the model to the lists of spots he already extracted from the unregistered images.

To test this I have used the plugin to register pairs of images and kept the alignment log results, so I have the 3d-affine model 12 parameters provided in the log and a control of the expected result. Based on linear algebra, I understand that these parameters should be used in the form of a 4x4 matrix with this format:
M =
i00, i01, i02, i03
i04, i05, i06, i07
i08, i09, i10, i11
0, 0, 0, 1
and to register the spots the equation should be:
v’ = Mv with the original spot position v=(x,y,z,1) and v’ the registered spot position.
However, this doesn’t seem to work in the test sample I have (1 image analyzed before and after registration). I have coded a small script in MATLAB since this is the language they have been using so far in the lab and can only obtain this kind of result.

% Scaling parameters
scale = [ 0.177, 0.177, 0.030 ]';

% Reshape the spots 3D positions before reg
load('V:\Spot_Alignement\Align\Gaussian_fit_mRNA_beforealign.mat')
spots3DRaw = vertcat( Xfit, Yfit, Zfit );
spots3DRaw = spots3DRaw./scale; % rescale the spots in pixels
spots3DRaw = vertcat ( spots3DRaw, ones(1, size(spots3DRaw, 2) ) );
clear X* Y* Z* Intensity

% Reshape the spots 3D positions after reg
load('V:\Spot_Alignement\Align\Gaussian_fit_mRNA_afteralign_noCroop.mat')
spots3DTrueReg = vertcat( Xfit, Yfit, Zfit );
spots3DTrueReg = spots3DTrueReg./scale;
spots3DTrueReg = vertcat( spots3DTrueReg, ones(1, size(spots3DTrueReg, 2) ) );

clear X* Y* Z* Intensity

% Reshape the transformation matrix into 4x4 mat
transMat = ...
    [0.9521668012751675, 0.01942896215185741, -0.02937386488638527,...
    -128.34156434174827, -0.02918781664433573, 0.9670877572733926,...
    0.007758961922641361, 22.80922305787985, 0.00666616264027195,...
    0.006974539745061396, 0.9102420623462557, -9.704733991824398];
transMat = reshape( transMat, 4, 3 )';
transMat = vertcat( transMat, [ 0, 0, 0, 1 ] );

% Calculate new registered positions
spots3DReg = transMat*spots3DRaw;

% Display both before and after reg
close all
figure
box on
plot3( spots3DRaw(1,:), spots3DRaw(2,:), spots3DRaw(3,:), 'ro' );
hold on
plot3( spots3DReg(1,:), spots3DReg(2,:), spots3DReg(3,:), 'b.' );
plot3( spots3DTrueReg(1,:), spots3DTrueReg(2,:), spots3DTrueReg(3,:), 'go' );

legend({'unregistered', 'spots registered', 'image registered'});
xlabel('X position (px)');
ylabel('Y position (px)');
zlabel('Z position (px)');
grid on;

(script download)

I’m sure I’m missing something here but I’ve tried to play with the parameters order or working with the scaling in physical unit, but even the angle of the registered spots doesn’t match what I expected so I’m wondering what I am doing wrong. If someone has a clue or a solution that would be great!

Thanks!

Sebastien

1 Like

Hi @sebherbert,

does the image output of the plugin look correct when you run the plugin?

The values from your alignment log:

3d-affine: (0.9521668012751675, 0.01942896215185741, -0.02937386488638527, -128.34156434174827,
          -0.02918781664433573, 0.9670877572733926,   0.007758961922641361,  22.80922305787985,
           0.00666616264027195, 0.006974539745061396, 0.9102420623462557,    -9.704733991824398)

… (with the high absolute values for x,y,z shifts) remind me of this old discussion:

… where I first supposed a wrong calculation of the 3D Affine matrix, but in the end we concluded that the approach doesn’t work well on almost-coplanar data, such as beads on a microscope slide.
In my experience, the 3D affine matrix tends to apply some arbitrary large z scaling if the input is quasi-planar.

Usually, the best solution is apply either

  • a 3D Similarity model (that doesn’t allow shearing), or
  • a 2D Affine model followed by a 3D Translation, and then concatenate the two resulting matrices.

Absolutely, we have a couple of workflows in place here at FMI, you might want to have a look at this post:

and also at this KNIME workflow for some inspiration:

(Note that this last one is also part of a publication: https://doi.org/10.3389/fcomp.2020.00008)

1 Like

Thanks a lot!

Yes, the registration is perfect and looks exactly as expected. But now that you said this, translation elements indeed look very high. I’ll do it again from scratch to make sure I didn’t mess up something somewhere, but I suspect it may be due to the 2 very independent rounds of imaging. Also I’m registering nuclei of an embryo so I would assume the detected spots not to be in the quasi-planar situation and it seems to be the X shift that is very high in my log while in your case it was apparently the Z shift?
If there’s still a doubt, I’ll try again with the 3D similarity model! I have the feeling that the second solution shouldn’t be able to correct for z shrinking though no?

Thanks for Knime links, I’m not very experience with the software and haven’t been able to download the “Apply 3D affine transform to Points” node on my first try, but I’ll try again once I’ve made sure the inputs are all correct!

Sebastien

1 Like

You’re right, your case might require all the degrees of freedom of a 3D affine transformation indeed.

I should have noted that the ImageJ2 integration is currently broken in KNIME 4.2 (see this topic on the KNIME forum), so you’d have to stick to version 4.1.3 when trying.

That said, these custom KNIME nodes are in fact just ImageJ(2) plugins! Since they take double[] inputs, running them from the UI will not work, but you can call the Command via scripting in ImageJ. Or you can take the source code directly as inspiration for your own ImageJ scripts.

The Apply 3D Affine Transform to Points plugin can be found here:

1 Like

Awesome, thanks!
I’ll have a look as soon as I can!