Manual Registration of SPIM Data

Hello!

I’m planning on adding a way to perform manual registrations on SPIM acquisitions. Essentially what I want to be able to do is:

  • View multiple SPIM acquisitions simultaneously in 3D
  • Perform linear transformations on the SPIM acquisitions
  • Simultaneously view the data from the XY, XZ, YZ and a custom perspective
    • Custom perspective would just be a different angle on the coordinate space that the user could dictate
  • Transform the object in one perspective and see how that affects the other perspectives
    • i.e. if I select the object in the XY view and start transforming it, I can also see how this transformation is changing the object from the other perspectives at the same time

My thoughts for achieving this are to extend the 3D viewer as it already has most of this functionality. However, before I do that I just wanted to get some additional input.

So if you could please let me know your thoughts on this, I would greatly appreciate it. Also, please feel free to let me know if you have any questions/concerns.

Thank you very much,
Alison

1 Like

Hi Alison,

to a large extend you can already do all this with the @StephanPreibisch SPIM Registration plugins and BigDataViewer. If you have a dataset defined and opened in Plugins > Multiview Reconstruction > Multiview Reconstruction Application, you can select some views, right-click and Display in BigDataViewer (on/off).

In the BigDataViewer window you can do manual transformations. This is unfortunately only minimally documented in the BDV help window, and not yet documented on the wiki. Basically:
Press T to start manual transformation. Only the current source (respectively group) is transformed using the standard BDV mouse and keyboard actions. All other sources are kept fixed. (See http://imagej.net/BigDataViewer#Displaying_Multiple_Sources and http://imagej.net/BigDataViewer#Grouping_Sources for detailed definition of current source and current group.) Press T again to fix the manual transform relative to the other sources. Press ESC to abort. While in manual transform mode press R to reset the manual transform to identity.

Ok, so this can be used to manually register views. To apply it to the dataset and save it to the XML you need to select all views you want to apply the current transform setup to (e.g. also from different timepoints) in the Multiview Reconstruction Application. The right-click and Bake BDV manual transform. Save XML.

What is not there yet is the multi-window (XY, XZ, YZ and a custom perspective) setup. But that should be possible to add. In principle you just need to open multiple BDV windows, add the same sources to all of them, and set up the view transformations. If you want to prevent users from certain actions, e.g., rotating in the axis-aligned views, that’s also easily possible. But the whole axis alignment requires some thought. In a multiview setup it is not clear what the desired axes are. It may be the axes of some individual view (then we need UI to pick that view) or something user defined, e.g., aligned with the sample somehow (then we need UI to define that).

best regards,
Tobias

4 Likes

Thanks @tpietzsch for this helpful info. One strong desire our users at LOCI have is to be able to perform these manual registrations in 3D in real time. (The “manual” and “real time” bits are important so they don’t feel powerless if/when things go wrong in the automatic registration.) We will definitely show them the BDV capabilities, and see how that goes for them, but at the moment they are using a custom prototype developed in C++, built on OpenGL etc. (code is here but unfortunately it does not build easily on most platforms). As I’m sure surprises no one, I strongly want to provide such functionality via ImageJ/Fiji instead of a standalone application.

@awalter17 It would be great if you could validate Tobias’s described BDV workflow, then meet with Jayne & Julie to show them. We can then discuss whether it makes sense to continue pursuing a 3D-viz-centric solution using @kephale’s ThreeDViewer.

Lastly, I know that ClearVolume/scenery+BDV integration is planned, but my understanding is that this is several months out at best (@skalarproduktraum is still cranking on the needed scenery bits, right?). If there is anything we can do to help move this along on our end, @awalter17 has ~10 hours a week to spend on this—although we understand if doing this work is only tenable for a senior developer. I would much appreciate keeping this dialog open though as things progress.

2 Likes