Image Registration Suite

I have to admit this makes lots of sense!
I feel this might be the moment for me to finally embrace imglib2 :slight_smile:

3 Likes

:star_struck::clap::clap::fireworks:

I’ll be suuuuper happy to help if you run into roadblocks - I’d love an elastix wrapper/interface :smile:

Yes. It sounds very nice.

Thank you!

Are there already also interfaces, not for applying, but for finding the best transformation?

If not, would something like this make sense?

public RealTransform findTransform( RandomAccessibleInterval fixed, RandomAccessibleInterval moving )

As I said, I have hardly any clue about imglib2, but my idea would be that the RAI could be good because this could be a way to specify on which parts (intervals) of the two images the user wants to find the best transformation.

There is not such an interface in imglib2.

There is something like that in mpicbg: the Model interface, but it doesn’t operate on images, but rather on point correspondences, so not the same thing.

Yup, something like that looks good to me.

John

Here’s a beanshell script that builds a simple RealTransform (2d affine) and transforms and shows an image
using that transform.

It makes heavy use of convenience functions in the RealViews and Views classes.

import ij.IJ;
import net.imglib2.img.display.imagej.ImageJFunctions;
import net.imglib2.interpolation.randomaccess.NLinearInterpolatorFactory;
import net.imglib2.realtransform.*;
import net.imglib2.view.*;

img = ImageJFunctions.wrap( IJ.getImage());

AffineTransform2D xfm = new AffineTransform2D();
// rotate by some amount 
// note that the rotation is about the origin (0,0) in the top left of the image
xfm.rotate( Math.PI / 8 ); 
xfm.scale( 0.8 );  // scale by some amout

// real random accessible:
// extend: what are the values outside the image
// interpolate
// transform with the affine above
rra = RealViews.affine( 
    Views.interpolate( Views.extendZero( img ), new NLinearInterpolatorFactory() ),
	xfm );

// random accessible make it a raster image ,
// the same size as the original interval( _, interval)
ra = Views.interval( Views.raster( rra ), img );
ImageJFunctions.show( ra );
4 Likes

Great! Works for me.
It looks like there still is this “bug” that the interpolator has trouble dealing with 255 in 8bit images:

        AffineTransform2D xfm = new AffineTransform2D();
		xfm.rotate(0.0 );
		xfm.translate(100.5, 100.0 );

		ExtendedRandomAccessibleInterval erai = Views.extendZero( img );
		RealRandomAccessible rra = Views.interpolate( erai, new NLinearInterpolatorFactory()  );
		AffineRandomAccessible ara = RealViews.affine( rra, xfm );

		RandomAccessibleInterval rai = Views.interval( Views.raster( ara ), img );
		ImageJFunctions.show( rai );

2 Likes

We have another question. In case we measure the drift sequentially, i.e. 1->2, 2->3, 3->4, a.s.o.; in order to apply the drift correction for time-point 4 we need to apply all the drifts to up this time-point (I hope this is understandable). I found below class, which does that, but I was a bit worried that this is maybe slow because, as I understand it, it has to for each pixel compute the combined transformation over and over again, isn’t it? If that’s true, is there some way to actually really “get” the combined transformation rather than recomputing it on the fly for every pixel?

I’ll look into that interpolation issue and report back.

You are correct that the RealTransformSequence will do what you describe. Whether you can “get” the whole transform depends on what it is. For Affine’s (and linear transforms) it’s easy: they all have concatenate and preConcatenate methods.

For (most, that I’m aware of) non-linear transforms, there’s not a straightforward way of concatenating one transform to another. It is possible to compose displacement fields, but to do it, you’d end up implementing RealTransformSequence, effectively. And, I don’t think there would be a speed improvement if you only visit each location once (unless I’m overlooking something).

On the plus side for speed: rendering transformed images is not hard to parallelize across space e.g. an undocumented example from BigWarp

@tpietzsch renders in parallel over several resolutions for bigdataviewer.

John

3 Likes

Yea, that nasty behaviour is due to overflows.
Its fixed by this ClampingNLinearInterpolator, but it lives in the bigdataviewer-core at the moment and hasn’t made it into imglib2-core yet.

(Edit: since Fiji has the bigdataviewer built in, you can just replace NLinearInterpolatorFactory with ClampingNLinearInterpolatorFactory in the script above and it’ll just work :slight_smile:)

1 Like

I think this project is a great idea! I use Correct 3D drift a lot and all of the things mentioned in this thread would be really useful for me.

1 Like

Hi All, @stelfrich @imagejan

During a very productive hackathon in Dresden I managed to create version 0.1.2 of the Image Registration Suite :slight_smile:

It obviously is super beta, but maybe you can give it a try and give me feedback, both regarding usability and code.

  • [Fiji > Update > Manage Update Sites > EMBL-CBA
  • Restart Fiji
  • [Fiji > Plugins > Registration > N-D Sequence Registration]

Feel free to post issues here:

6 Likes

@Christian_Tischer, sorry for being a noob, but it throws up an error (below) when trying to compute registration. What have I not got?

[INFO] ImageRegistrationPlugin started
[INFO] ## TransformationFinderRotationTranslationPhaseCorrelation
[INFO] ### TransformationFinderTranslationPhaseCorrelation
org.scijava.module.MethodCallException: Error executing method: de.embl.cba.registration.gui.ImageRegistrationPlugin#computeRegistration
	at org.scijava.module.MethodRef.execute(MethodRef.java:73)
	at org.scijava.module.AbstractModuleItem.callback(AbstractModuleItem.java:229)
	at org.scijava.widget.DefaultWidgetModel.callback(DefaultWidgetModel.java:187)
	at org.scijava.ui.swing.widget.SwingButtonWidget$1.actionPerformed(SwingButtonWidget.java:83)
	at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
	at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
	at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
	at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
	at java.awt.Component.processMouseEvent(Component.java:6535)
	at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
	at java.awt.Component.processEvent(Component.java:6300)
	at java.awt.Container.processEvent(Container.java:2236)
	at java.awt.Component.dispatchEventImpl(Component.java:4891)
	at java.awt.Container.dispatchEventImpl(Container.java:2294)
	at java.awt.Component.dispatchEvent(Component.java:4713)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4888)
	at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4525)
	at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4466)
	at java.awt.Container.dispatchEventImpl(Container.java:2280)
	at java.awt.Window.dispatchEventImpl(Window.java:2750)
	at java.awt.Component.dispatchEvent(Component.java:4713)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
	at java.awt.EventQueue$4.run(EventQueue.java:731)
	at java.awt.EventQueue$4.run(EventQueue.java:729)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.scijava.module.MethodRef.execute(MethodRef.java:69)
	... 39 more
Caused by: java.lang.NoClassDefFoundError: net/imglib2/algorithm/phasecorrelation/PhaseCorrelation2
	at de.embl.cba.registration.transformationfinders.TransformationFinderTranslationPhaseCorrelation.findTransform(TransformationFinderTranslationPhaseCorrelation.java:109)
	at de.embl.cba.registration.transformationfinders.TransformationFinderRotationTranslationPhaseCorrelation.computeCrossCorrelationAndTranslation(TransformationFinderRotationTranslationPhaseCorrelation.java:177)
	at de.embl.cba.registration.transformationfinders.TransformationFinderRotationTranslationPhaseCorrelation.computeCrossCorrelationAndTranslation(TransformationFinderRotationTranslationPhaseCorrelation.java:140)
	at de.embl.cba.registration.transformationfinders.TransformationFinderRotationTranslationPhaseCorrelation.findTransform(TransformationFinderRotationTranslationPhaseCorrelation.java:77)
	at de.embl.cba.registration.ImageRegistration.run(ImageRegistration.java:231)
	at de.embl.cba.registration.gui.ImageRegistrationPlugin.registrationThread(ImageRegistrationPlugin.java:247)
	at de.embl.cba.registration.gui.ImageRegistrationPlugin.access$000(ImageRegistrationPlugin.java:48)
	at de.embl.cba.registration.gui.ImageRegistrationPlugin$1.run(ImageRegistrationPlugin.java:281)
	at java.lang.Thread.run(Thread.java:745)
	at de.embl.cba.registration.gui.ImageRegistrationPlugin.computeRegistration(ImageRegistrationPlugin.java:285)
	... 44 more
Caused by: java.lang.ClassNotFoundException: net.imglib2.algorithm.phasecorrelation.PhaseCorrelation2
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 54 more
2 Likes

I don’t have that class in my up-to-date Fiji either.
A quick search on maven.imagej.net told me that it is contained in the net.preibisch::BigStitcher artifact.

@Christian_Tischer, your plugin apparently requires having the BigStitcher update site enabled as well, right?!

2 Likes

This is right! Do I have to change something in my pom.xml or do something different during the upload to the update site?

Hallo Tischi,
I do have problem to process a multichannel image with N-D Sequence registration. The processing typically ends with ArrayIndexOutOfBounds exception.

Exception in thread “Thread-2659” java.lang.ArrayIndexOutOfBoundsException: 3

With one single channel it works.
Greetings
Antonio

Can you send me a minimal data set that I could use to reproduce the issue?
And could you please also paste a screenshot here of your UI settings of the plugin.

Dear Christian,

Thank you for your great Correct 3D Drift script !
I wonder if you know a way to apply the transform text file you can produce with Correct 3D Drift to another stack ?

Best wishes
Aurélien

2 Likes

Good question! I am not sure. Probaby currently not. I will look into it…

I agree, I really have to look into it. I hope I find time soon…