Registration of multi-channel timelapse with Linear Stack Alignment with SIFT

Usage & Issues
, ,
#1

Hi all,

I have a hyperstack with 3 channels and over 200 timepoints and would like to register one of the channels with the “Linear Stack Alignment with SIFT” algorithm and then apply the transformations calculated in that channel to the two other ones. Does anyone know of a way I can do this? I could not find any threads in image.sc discussing this scenario.

I have tried using other registration algorithms that have the ability to port one channel’s transformation to another channel (like MultiStackReg, or descriptor-based series registration), but they perform poorly with my dataset.

I also cannot use the Linear Stack Alignment with SIFT on each channel independently and then merge them because one of the channels it too noisy for the registration to work properly. This is due to a fluorescent protein expressed at low levels, sparsely and only occasionally in the cell cycle. Hence, I would like to use my highest quality channel to do the registration and then apply the same transformations to the other, lower quality channels.

If anyone has any ideas, they would be most welcome.

2 Likes
#2

Hi @pablo_a ,

I think I can help with this, can you reply back later this week if you haven’t heard from me?

John

#3

Thanks John. I will ping you later this week if I don’t hear from you.
Best,
P.

#4

Hi @pablo_a,

I encountered exactly the same problem not so long ago. I tried out many alignment tools that can do rigid body registration, including Linear Stack Alignment with SIFT, MultiStackReg and Image Stabilizer (only translation and affine), but I found that for my data HyperStackReg worked the best (thanks @vedsharma!).
Strangely enough I can’t get MultiStackReg to properly align the timepoints in that dataset, even though it too uses StackReg.

HyperStackReg has a really simple interface. You just select the type of transformation and channels (one or multiple) to be used in the calculations, after which all channels of the stack are transformed with the same matrix.
Just try it out. If that doesn’t work for you, I guess it’s up to @bogovicj’s magic!

Best regards,
Bram

#5

Dear @pablo_a
I looked into the "“Linear Stack with Shift” plugin and one of the options when running to plugin is to output the Transformation Matrix. This Matrix describes the XY offset between neighboring frames which could then be applied to different channels. Here is an example output:

Processing SIFT ...
 took 371ms.
265 features extracted.
Processing SIFT ...
 took 313ms.
541 features extracted.
55 potentially corresponding features identified
Transformation Matrix: AffineTransform[[0.999999878318377, 4.93318590661E-4, -5.176883695752914], [-4.93318590661E-4, 0.999999878318377, -1.102071746546699]]

Looking at the Transformation Matrix, 2 arrays are listed with 3 numbers in each array. From what I can determine, the last number of the first array (-5.176) corresponds to the X offset between 2 timepoints and the last number of the second array corresponds to the Y offset (-1.102) between the 2 timepoints.

You could try parsing the XY offsets from the Log file generated after running the “Linear Stack with Shift” plugin and then applying these offsets to the different channels.

Let me know if this helped.

#6

@romainGuiet filed a PR to support multichannel registation in “Linear Stack Alignment with SIFT” (specifying the channel to use for the registration):

Do you think this could be an option worth investigating ? @bogovicj @axtimwalde

But maybe it’s a better option to add another plugin alongside the original one instead of overriding it.

1 Like
#7

Thanks for this idea. Unfortunately, HyperStackReg performed very poorly on my dataset.

It would be nice if there was a general tool that allowed one to “call” any registration algorithm to apply to one channel, and then apply the results to other channels in a multi-channel timelapse. My guess is that would probably take a lot of work as each algorithm would need to be updated to output the transformation matrix in a standard form that the general tool could read. But perhaps, it is easier than I think for people who know what they are doing.

Best,
Pablo

#8

Thanks for pointing this out, it is helpful to know what those numbers are.

I only need to do a translation transformation, so conceptually this is not that hard. The general shape of a macro would be something like:

  1. Split the channels in the original dataset
  2. Select one of the channels and apply transformation to it
  3. Parse the relevant XY shift values from the transformation matrix text output in the log
  4. Apply the proper transformation to each image in the other channels, generating a single channel, registered, timelapse for each one (maybe using something like TransformJ translate?)
  5. Merge the mutliple, individual channel timelapses into a single hyperstack
    Unfortunately for me, steps 3) and 4) are a bit beyond what I can do quickly with my scripting skills.

I will see if some of the more experienced folks here can come up with an easy solution before trying to tackle this. But at least now I know it is doable, in principle, with a macro.

Thanks!
P.

1 Like
#9

Dear @pablo_a
Here is an example macro function which can parse the XY coordinates. The XY positions are saved to two arrays, xShift and yShift.

var xShift = newArray();
var yShift = newArray();

ParseTransformationMatrix();

Array.print(xShift);
Array.print(yShift);

function ParseTransformationMatrix(){
	//Get Log Output
	logString = getInfo("log");
	//Subdivide into Rows
	rows=split(logString,"\n");

	for(i=0;i<rows.length;i++){
		if(rows[i].contains("Transformation")){
			ParseXY(rows[i]);
		}
	}
}

function ParseXY(row){
	split1 = split(row,"[");
	XSplit = split(split1[1],",");
	YSplit = split(split1[2],",");

	XPosTemp = XSplit[2];
	YPosTemp = YSplit[2];
	
	XPos = XPosTemp.substring(0,XPosTemp.length-1);
	YPos = YPosTemp.substring(0,YPosTemp.length-2);

	xShift = Array.concat(xShift,parseFloat(XPos));
	yShift = Array.concat(yShift,parseFloat(YPos));
}

When I use parseFloat to convert the string to float values, the coordinate offsets get rounded to the thousandth place, although I don’t think this is an issues for offsetting the images.

I don’t really know how to offset the images based off the XY coordinates, so someone else will need to figure that out. Maybe some component of the Linear Stack with Shift plugin could be reused.

#10

Thanks, this is very helpful!
Best,
P.

#11

Sorry for being ignorant on this PR. This plugin is the most outdated version of SIFT based alignment tools in the Fiji universe and should not be used. For very simple (selective multi-channel) workflows, I suggest the plugins Register Virtual Stack Slices - ImageJ and Transform Virtual Stack Slices - ImageJ that are easily macro-recordable and combinable. For more finnicky things including regularized elastic registration and global optimization, please use TrakEM2 - ImageJ which is also easily scriptable to automate workflows.

1 Like
#12

Thanks a lot for pointing this out! I tried your suggested workflow and it was able to register effectively and then use the transform in one channel and apply it to another.

The problem I found with the approach is that it is not very user friendly if you are starting with the data in hyperstack form, as it requires a lot of extra data handling and organization steps. None of them are hard, and they could all be automated with a macro, but for a beginner user it makes it much harder. The advantage of something like HyperStackReg is that it takes the hyperstack and outputs a hyperstack. In contrast, using the combination of Register Virtual Slices and Transform Virtual Slices required the following steps:

  1. Split channels in hyperstack
  2. Save images in each channel as an image sequence (individual files) in separate folders
  3. Create output folders and a transform folder
  4. Run the Register plugin on the reference channel
  5. Run the Transform plugin on the other channels
  6. Merge the resulting registered stacks into a hyperstack
    None of this is hard, and I can now fix the dataset which started this thread, so thanks!

That said, for a beginner, the extra steps make it harder to use. If these steps could be integrated into an interface more similar to HyperStackReg I think it would improve usability, so perhaps it is something that could be considered.

3 Likes
#13

Point taken and glad to hear that you got it to do what you needed to do. I agree that there could be more up-front user friendliness, but often that comes with some baked in limitations of what you can do with the tool, so I am usually in favor of these clunky modules that you can combine to any flow you need. Now that you macro-recorded what you did, you basically have a user friendly tool for your workflow, so may be you can post it here for the next one running into something similar?

1 Like
#14

I tried devising a macro, but ran into the following issues:

  • Major: Transform Virtual Stack Slices does not get recorded with the macro recorder
  • Minor: Register Virtual Stack Slices is recordable, but requires user interaction to set the folder for the transforms, and to click on the first image (or reference image) for the registration.

Because of these limitations, I ended up doing a lot of the workflow manually.

Perhaps I made a dumb mistake somewhere along the way but if so, I can’t figure out what it might be.

#15

I found a similar thread, so I’ll post it.

Also, I’d like to promote the plugin I’m referring to in this thread (CoordinateShift).
It doesn’t do rotation or transformation, just XY movement, but it will probably do what you want.
Please give it a try.

hwada

1 Like