Details on the TPS used by Apply_Bigwarp_Xfm_csvPts.groovy in BigWarp?

I am just wondering if there is anywhere I can find the details on the method used to calculate the TPS interpolation from the landmark files. I am trying to write a function in MATLAB that will apply the transform from the landmark file to any array of 3d points, but so far I have not been able to get my interpolation to match that of Apply_Bigwarp_Xfm_csvPts.groovy exactly (although it is close). Does this application use a smoothing parameter or something? Thanks! FYI I am using an adaptation of the code at this link for my function https://www.mathworks.com/matlabcentral/fileexchange/37576-3d-thin-plate-spline-warping-function

Hi @zmiller22,

The implementation bigwarp uses is based on this paper:

There’s not a smoothing parameter.

How close?

Maybe also try the built in matlab function?
https://www.mathworks.com/help/curvefit/tpaps.html
(or is it 2d only? I didn’t look closely)

John

btw, another possibility would be to call the java code directly from matlab.

Thank you very much! This will help a lot. I only eyeballed the closeness by plotting some point grids that had undergone the transformation from Apply_BigWarp and my function. It was clear that they were both working but just not quite the same. Thanks again!

Ok so it seems like the (objects? methods? Sorry I am not very familiar with groovy) that I would want to use would be buildTransform and fitTransform from the Apply_Bigwarp_Xfm_csvPts.groovy script, but maybe there is a different class or method you would recommend. Reading a tutorial on how to use groovy in Matlab, it says that you just compile the .groovy file using groovyc, then use jar to make it a .jar file, and then from there is the same as a java file in matlab. However, Apply_Bigwarp_Xfm_csvPts.groovy is throwing an exception when I try to compile it using groovyc, command and error below. Can you help me figure out what is causing this error below? It looks like it was expecting a sha-bang or something.

(base) tracingpc1@TracingPC1:/mnt/c/Users/TracingPC1/Documents/bigwarp-master/bigwarp-master/scripts$ groovyc Apply_Bigwarp_Xfm_csvPts.groovy
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.vmplugin.v7.Java7$1 (file:/usr/share/groovy/lib/groovy-2.4.16.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.vmplugin.v7.Java7$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Apply_Bigwarp_Xfm_csvPts.groovy: 1: expecting '!', found '@' @ line 1, column 2.
   #@ File (label="Landmark file") landmarksPath
    ^

1 error

The stuff at the top of that script are script parameters. That’s an imagej2 thing, not a groovy thing.

Most of the stuff in that groovy script is almost identical to the Java, in fact I’d recommend calling the Java code from matlab and forgetting about the groovy for your purposes.

I hope that this java code snippet below will get you close to what you need - just have to call it inside matlab.

File f = // your landmark point csv file from Bigwarp
double[] point = ... // the point you want to transform

// load the landmarks, and get the transform
LandmarkTableModel ltm = new LandmarkTableModel( 3 );
ltm.load( f );
ThinPlateR2LogRSplineKernelTransform tpsTransform = ltm.getTransform();

// apply the transform
double[] transformedPoint = tpsTransform.apply( point );

I call out some key parts of the groovy script below in case it ends up being helpful.

Let me know how it goes,
John

Key parts of the groovy script

Thank you so much this is very helpful. Just a couple follow up questions. When you say “identical to the java”, do you mean similar to another java file? If so which one? Also, do I need to build the project using maven? I am assuming yes since it structures all the import calls? Thanks again.

I meant: “the groovy code in that script is very similar to the Java code one would have to write to get the same behavior”

Take the following with a grain of salt, because it’s been a long time since I’ve called java code from Matlab:

You don’t. Matlab makes it fairly easy (or it did when I did this stuff).
Matlab just needs to be able to see the bigwarp jars (and the jars for all its dependencies), then you can write the code above inside matlab and it should work. Check out this (artificially simple) example.

Ok I think I see what to do. I will pick this up again tomorrow. As always, thank you for taking the time to help me out!

Ok so I am now trying to import all the necessary classes into matlab. The only problem is some of the classes I am importing import other classes that are not included in the bigwarp repo or the standard java libraries, which in turn import more classes and so on and so forth. For example, bigwarp.landmarks.LandmarkTableModel imports jitk.spline.ThinPlateR2LogRSplineKernalTransform, so I went ahead and downloaded that github repo and added jitk to the matlab javapath. Then matlab throws an error again this time sayign that net.imglib2.RealLocalizable can not be found so now I need to go download and compile that. So my question is, before I go on a wild goose chase to find all the dependencies, do you have a package or something that has all the dependencies in one place? Or is there a readme somewhere that lists them all? Thank you!

@zmiller22,

Best way is to let maven tell you.

If you have bigwarp git repo checked out, run this from inside it:

mvn dependency:build-classpath | grep 'Dependencies classpath' -A 1 | tail -n 1 

will build the list for you (if you’re on mac or linux).

John

Oh that’s perfect, thanks!

1 Like

Ok… I am so close I have everything working, except when I apply the transform to the points it results in the inverse of the transform it should (shifts left instead of right, shrinks instead of stretching, etc.) Any ideas? Current Java code below, inputPoints is an nx3 vector of x,y,z coordinates of points, same as a landmark file

import java.io.*;
import java.nio.file.*;
import java.util.*;
import bigwarp.landmarks.*;
import bigwarp.BigWarp.WrappedCoordinateTransform;
import net.imglib2.realtransform.*;
import jitk.spline.ThinPlateR2LogRSplineKernelTransform;

import bdv.gui.TransformTypeSelectDialog;

public class TransformPoints {
	
	public TransformPoints() {
		}

	public double[][] transform(String landmarksPath, double[][] inputPoints) {
		File f = new File(landmarksPath); // reads in the landmark csv file

		// loads in landmarks and gets the transform
		LandmarkTableModel ltm = new LandmarkTableModel( 3 );
		try
		{
			ltm.load( f );
		}
		catch ( IOException e )
		{
			e.printStackTrace();
		}

		ThinPlateR2LogRSplineKernelTransform tpsTransform = ltm.getTransform();

		// creates the point array to be outputted
		double[][] newPoints = new double[inputPoints.length][inputPoints[1].length];

		for (int row = 0; row < inputPoints.length; row++) {
			double[] point = inputPoints[row];
			double[] transformedPoint = tpsTransform.apply( point );
			newPoints[row] = transformedPoint;
		}

		return newPoints;


	}
}

Happy to hear!

Yea, that’s the sneaky thing about image transformations :upside_down_face:
The groovy script has logic to take care of this, but you ran into it since you need to call the code yourself.

The “default” transform out of bigwarp maps points from target to moving space, because that’s the direction we need to render the moving image in target space. (Think: if the forward transform applies to points, we need the inverse transform for images). You can read about why that is here.

The way to go is to steal this part of the groovy script, ie:

InvertibleRealTransform inverseTransform = new WrappedIterativeInvertibleRealTransform( 
    new ThinplateSplineTransform( tpsTransform ) ).inverse();

and use that instead of tpsTransform.

You may also need to allocate your own transformed point, since the apply method is different (iirc):

int numDimensions = ...
double[] transformedPoint = new double[ numDimensions ];
inverseTransform.apply( point, transformedPoint );

Hope that does the trick,
John

Ok finally got it working! I have to say that was more of a pain the butt than I had expected it to be as matlab really does not have a great way to import multiple java classes/jars in a portable way without significant work. Anyway, I really appreciate your help it definitely cut the time down significantly. One final question though: the transformed points in matlab are all within about 0.1 of the transformed points using the groovy script. I assume this has something to do with the groovy script inverse tolerance and max iterations settings. Does this sound reasonable to you? If so, I think I will try to add those parameters so I can get even closer to the groovy script output. Thanks again!

Ok I tried to set the inverse tolerance and max iterations like the groovy script using the script below, but I got an error saying getOptimzer() is not a method of the InvertableRealTransform interface. It is, however, a method of the WrappedIterativeInvertibleReadtransform class, which implements InvertibleRealTransform. Is there something obvious that I am doing wrong?

import java.io.*;
import java.nio.file.*;
import java.util.*;
import bigwarp.landmarks.*;
import bigwarp.*;
import net.imglib2.realtransform.*;
import net.imglib2.realtransform.inverse.*;
import jitk.spline.ThinPlateR2LogRSplineKernelTransform;

import bdv.gui.TransformTypeSelectDialog;

public class TransformPoints2 {
	
	public TransformPoints2() {
		}

	public double[][] transform(String landmarksPath, double[][] inputPoints, double invTolerance, int maxIters) {
		File f = new File(landmarksPath); // reads in the landmark csv file

		// loads in landmarks and gets the transform
		LandmarkTableModel ltm = new LandmarkTableModel( 3 );
		try
		{
			ltm.load( f );
		}
		catch ( IOException e )
		{
			e.printStackTrace();
		}

		ThinPlateR2LogRSplineKernelTransform tpsTransform = ltm.getTransform();
		InvertibleRealTransform invertableTransform = new WrappedIterativeInvertibleRealTransform( new ThinplateSplineTransform( tpsTransform ) );
		InverseRealTransformGradientDescent invopt = invertableTransform.getOptimzer();
		invopt.setTolerance (invTolerance);
		invopt.setMaxIters( maxIters );
		// creates the point array to be outputted
		double[][] newPoints = new double[inputPoints.length][inputPoints[1].length];

		for (int row = 0; row < inputPoints.length; row++) {
			double[] point = inputPoints[row];
			double[] transformedPoint = new double[ 3 ];
			invertableTransform.inverse().apply( point, transformedPoint );
			newPoints[row] = transformedPoint;
		}

		return newPoints;


	}
}

With error

(base) tracingpc1@TracingPC1:/mnt/c/Users/TracingPC1/Documents/bigwarp_java_files/classes$ javac TransformPoints2.java
TransformPoints2.java:33: error: cannot find symbol
                InverseRealTransformGradientDescent invopt = invertableTransform.getOptimzer();
                                                                                ^
  symbol:   method getOptimzer()
  location: variable invertableTransform of type InvertibleRealTransform
Note: TransformPoints2.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 error

Wait, never mind I fixed it! All is working perfectly now, thank you so much for your help!

1 Like

I will be cleaning and documenting the code for this matlab project on github over the next few days, and would like to include the java files that are needed to make this work. Is there some way I can credit you for the code/help?

1 Like

:tada::tada:!!
Happy to hear!

If you use this work in a publication, please mention bigwarp and cite the first paper where I used bigwarp. Beyond that, up to you. I’ve found it helpful to link to forum posts that helped me in creating code / scripts, for example.

Consider also sharing your Matlab code in case someone else might find it useful!

E.g., if you think you might use this and related code often, I’d be happy if you filed a pull request into the bigwarp repo with this.

John

That all sounds good, once the code is clean and documented, I will add this matlab part to a new branch and then send you a pull request. Thanks again!