Planned dev - nonlinear elastix imglib2 transforms (feedback requested)

@Christian_Tischer @NicoKiaru,

A heads up - I’m finally making a push to more completely support elastix nonlinear (bspline) transforms in imglib2.

The idea is to eventually have:

  • Parsing elastix BSpline TransformParameters and directly creating a RealTransform. This requires:
    • A deformable RealTransform using the bsplines. This requires:
      • A bspline implementation in imglib2

Motivation

At the moment I need to convert deformable elastix transforms to displacement fields before I can apply them. This will let me skip the conversion step. As well, loading the transform will be faster, since the bspline coefficients are more compact than the full displacement field. Applying the transform might be a bit slower… we’ll see.

Questions

  1. Have you worked toward this at all? i.e. is there a starting point?
  2. Would you use this when it’s done?
  3. Would you willing to test it out in the early (and possibly partially broken) stages?

John

4 Likes

Hi @bogovicj,

Sounds awesome!

Have you worked toward this at all? i.e. is there a starting point?

The tiny bit of work we’ve done with you and @Christian_Tischer included the parsing of elastix transform files into a Java ‘convenient’ structure:

Nothing else on my side.

Would you use this when it’s done?

Probably. The reason why I’m not 300% enthusiastic is because I’ve found a lot of limitation in my use case : brain slice registration to an atlas. In a word: if the ‘grid’ is precisely defined, the registration is bad because it stretched too much the area where there are issues in the acquisition (parts missing). If the grid too coarse, then the registration is not precise enough for some use cases. Sometimes there’s no middleground. In comparison, manual correction with BigWarp is doing an amazing job. The ideal would be an elastix registration of a BigWarp Transform :wink: .

Would you willing to test it out in the early (and possibly partially broken) stages?

Definitely!

4 Likes
  1. Have you worked toward this at all? i.e. is there a starting point?
    Nope

  2. Would you use this when it’s done?
    YES

  3. Would you willing to test it out in the early (and possibly partially broken) stages?
    YES

Great that you announce it like this! This makes the development process much more efficient!

3 Likes

So far so good.

To try it out:

prelims

  1. Set up jgo
  2. Make sure you have python + numpy available

build and set up

Then:

# checkout and build a new imglib2-realtransform branch
git clone git@github.com:bogovicj/imglib2-realtransform.git
cd imglib2-realtransform
git checkout bspline
mvn clean compile install

# checkout and build the branch of the itc repo
git clone git@github.com:image-transform-converters/image-transform-converters.git
cd image-transform-converters
git checkout bspline
mvn clean compile install

# Run the demo
./src/scripts/compareBSplineTransform

conclusions

The last line of the output (for me) is:

maximum distance: 1e-06

indicating that the imglib2 wrapper is very close to the results that transformix produces.

Check out the comparison script:

TODOs

  1. Support compositions of transforms (note this change I had to make) - should be straightforward
  2. Benchmark / optimize. Specifically, I’m confident lots of computation savings can be had when transforming images since there are shared computations among adjacent pixels/voxels.
  3. Generalize the bspline implementation - possibly moving it to imglib2-core. @axtimwalde, @tpietzsch, you can safely ignore this for now, if you’d like, but you may hear from me in the medium term.

working with other tools

Yes! I’m going to work toward this. I’ll solicit for feed back about this as well :wink:
Specifically, I think it would be great to be able to quickly both call @Christian_Tischer’s elastix wrappers
from the bigwarp result and to preview / fine tune elastix results in bigdataviewer / bigwarp.

More to come!

Thanks

@NicoKiaru @Christian_Tischer

I gotta say, the work we (you two) did at the hackathon last year made really did make my life a lot easier.

John

3 Likes

I will work on that very soon! My plan here is to compile SimpleElastix and then call elastix using proper code rather than the current command line call to the binary.

1 Like

@Christian_Tischer,

Great! Let’s make sure to keep in touch.

Sounds good! I’ll see what I can do to make setting that up less painful (and finally make the PR to the SimpleElastix documentation…)

John

Hi @bogovicj,

So that was a bit painful to make this run on windows, but it looks like it worked. Here’s the output of the script:

comparing transformix and itc implementation
/d/pyimglib2/image-transform-converters/src/test/resources/elastix/TransformParameters.BSpline3D.txt
apply with transformix

transformix is started at Wed Dec  4 15:33:24 2019.

which transformix:   C:\elastix-4.9.0-win64\elastix-4.9.0-win64\transformix.exe
transformix runs at: SV-01-099
  Windows  Professional (x64),  (Build 9200)
  with 65451 MB memory, and 8 cores @ 3599 MHz.
Running transformix with parameter file "D:/pyimglib2/image-transform-converters/src/test/resources/elastix/TransformParameters.BSpline3D.txt".

Reading the elastix parameters from file ...

Installing all components.
InstallingComponents was successful.

ELASTIX version: 4.900
Command line options from ElastixBase:
-in       unspecified, so no input image specified
-out      D:\pyimglib2\image-transform-converters\src\scripts\
-threads  unspecified, so all available threads are used
-tp       D:/pyimglib2/image-transform-converters/src/test/resources/elastix/TransformParameters.BSpline3D.txt
-def      bsplinePointSamples.txt
-jac      unspecified, so no det(dT/dx) computed
-jacmat   unspecified, so no dT/dx computed
Calling all ReadFromFile()'s ...
WARNING: The parameter "InitialTransformParametersFileName", requested at entry number 0, does not exist at all.
  The default value "NoInitialTransform" is used instead.
WARNING: The parameter "HowToCombineTransforms", requested at entry number 0, does not exist at all.
  The default value "Compose" is used instead.
  Calling all ReadFromFile()'s took 0.001765 s
Transforming points ...
  The transform is evaluated on some points, specified in the input point file.
  Reading input point file: bsplinePointSamples.txt
  Input points are specified in world coordinates.
  Number of specified input points: 2197
  The input points are transformed.
  The transformed points are saved in: D:\pyimglib2\image-transform-converters\src\scripts\/outputpoints.txt
  Transforming points done, it took 0.03s
Compute determinant of spatial Jacobian ...
  The command-line option "-jac" is not used, so no det(dT/dx) computed.
  Computing determinant of spatial Jacobian done, it took 0.00s
Compute spatial Jacobian (full matrix) ...
  The command-line option "-jacmat" is not used, so no dT/dx computed.
  Computing spatial Jacobian done, it took 0.00s

transformix has finished at Wed Dec  4 15:33:25 2019.
Total time elapsed: 0.5s.

apply with imglib2
computing distances
maximum distance: 1e-06

1 Like

@NicoKiaru

Sorry about that, :-/ but thanks for trying it, I really appreciate it.

In the future maybe I’ll make demos Fiji scripts instead of command line scripts. In this case I was trying to avoid requiring installing new jars into Fiji (and possibly breaking other things). but I guess we all have 10 different versions lying around, right… right? :upside_down_face:

John

1 Like

I learned a few things about windows virtualenv, so it was a good thing anyway.

And now that it’s working, no need to change :wink:

What was the change ? A converter from the elastix wrapping class to your realtransform class ?

How slow is it currently ? Not fast enough for live display ?

Thanks a lot for your work, and that was incredibly fast…

If you write a SciJava script you could use scijava-grab with #@repository and #@dependency to depend on things dynamically. I’d like to start shipping scijava-grab with ImageJ2 soon…

Or you could make an ImageJ update site, of course.

2 Likes

@ctrueden,

:heart: :heart: :heart:
Thanks for the pointer! Look forward to trying this :smiley:

I’ll help document (on the wiki) when it’s shipped!

1 Like