Imglib2 local transformation

Hi @bogovicj
Could you help me with the following issue?
I have a list of pairs of 3D points { ( original, transformed ) }.
From this list I would like to construct a smooth transformation that transforms an original RAI into a transformed RAI. Pretty much like transformed = RealViews.transform( original, transform ); but then giving as input not an AffineTransform3D but a list of matching points which is used to construct the transformation. I assume code for this should be present in BigWarp? Something like this here: https://github.com/saalfeldlab/bigwarp/blob/master/src/main/java/bigwarp/source/GridSource.java, but I don’t understand how to use it.

1 Like

Hi @Christian_Tischer,

Good question. This turned into a sort of big answer, most of it is explained in the comments in the example code below.

Bigwarp uses the ThinPlateSplineTransform. There are other wrappers that are helpful for Bigwarp / bigdataviewer, but for now I’ll post the imglib2-only solution.

One tricky part here is that RealViews.transform that you point to needs the “inverse transform”. So we need some annoying gymnastics / warppers to use it, but fortunately, we can avoid it. I show both the “pretty” and “ugly” solutions below.

// fill these arrays somehow
double[][] srcPts = new double[ nDimensions ][ nLandmarks ];
double[][] tgtPts = new double[ nDimensions ][ nLandmarks ];

RealRandomAccessible realSrcImg = ...

/* 
 * make the thin plate spline transform
 *  target and source are switched, so tps maps points in target space to source space
 *  tps is not invertible 
 */
ThinplateSplineTransform tps = new ThinplateSplineTransform( tgtPts, srcPts );

/* 
 * Transform the source RealRandomAccessible.
 * This is basically what RealViews.transform does, but avoides some headaches, which we'll see below
 */
RealTransformRandomAccessible transformedSrcImg = new RealTransformRandomAccessible( realSrcImg, tps );


// Here are the headaches we avoid:

/* 
 * Wrap the tps so we can iteratively compute the "inverse"
 * Note: we won't use that capability in this example, but we need the transform to be Invertible.
 * Switch roles of fwd and inverse with "InverseRealTransform"
 * 
 * Now the inverse transform of invtps goes from target to source space - 
 *    this is what RealViews.transform needs
 */
InvertibleRealTransform invtps = new InverseRealTransform( 
     new WrappedIterativeInvertibleRealTransform( tps ));

// use RealViews
RealTransformRandomAccessible annoyingTransformedSrcImg = RealViews.transformReal( realSrcImg, invtps );
6 Likes

Wow!!
In fact, I did not expect this exist in such a convenient way.
Amaaaaazing!
And thanks so much for the swift and detailed answer!

2 Likes