How to align similar outdoor photos automatically and crop them to maximal photo size

I have some hand-held photos with a moving person in front of always the same background. So the camera does not move with the person but stays on the background - however the photos are hand-held, so there is some variation in the background area the photos show. I.e. the images do not align perfectly.

Here some low-res black&white versions of a typical group of photos I have:

Image2 Image3

(Click on the thumbnails for larger versions. Also note: The originals I am dealing with are high-resolution and in color.)

Now I want to transform the images so that they align as good as possible. For this I am trying in FIJI/ImageJ the “Register Virtual Stack Slices” plugin with the “translate + rotate” registration model and I get reasonable results. However the transform is certainly not perfect, probably because between images I must have twisted the camera a bit.

So my question here is: Does ImageJ offer another better plugin for this use case? Or are there other settings in the “Register Virtual Stack Slices” plugin worth tweaking?


PS: I have no idea why the forum system always breaks the first image link - the url is certainly is valid!

Hi @halloleo, welcome to the forum!

There are certainly many options, but it all depends on the specific case. It would be alot easier for members of the forum to give you advice if you could post some example photos.

Fair point @Sverre! I have added a group of sample images to the post.

Your camera position is moved and rotated in 3D during capturing the both images.

A automatic projective (perspective) transformation/correction is not available in FIJI/IJ afaik.
(Find some more info in https://forum.image.sc/search?q=perspective%20transformation)

The best solution I found testing several options (StackReg, TurboReg, ImageStabilizer …) is in FIJI>Plugins>Registration>bUnwarpJ

Registered Target Image_2

1 Like

Thanks @phaub! Much appreciated.

I am trying the plugin in out, but I cannoy see how I can give the plugin three input iamges: The settings dialog of the plugin asks for one source and one target image.

Do I miss anything?

Unfortunately bUnwarpJ only work with two images.

See more information here: https://imagej.net/BUnwarpJ

A trick is to transform pairs of images to one reference image and to create a stack with the reference and all the transformed images.

Here is a macro which can do that for grayscale images.
(Not sure how to work with RGB color images.)

Modify the path string, the name of the fixedImage and the list of images accordingly. (The list can be longer then 3.)

path = "C:\\Path_to_My\\Downloads\\";

fixedImg = "7XeasPt.jpg";

// Modify the list of images HERE:
images = newArray("7X6IRvr.jpg","7X6IRvr.jpg","7X6IRvr.jpg");

open(path + fixedImg);
open(path + images[0]);

// Command from the Macro Recorder (with modified image names)
run("bUnwarpJ", "source_image=" + fixedImg + " target_image=" + images[0] + " registration=Accurate image_subsample_factor=2 initial_deformation=[Very Coarse] final_deformation=Fine divergence_weight=0 curl_weight=0 landmark_weight=0 image_weight=1 consistency_weight=10 stop_threshold=0.01");

// Modify the first result stack
selectWindow("Registered Target Image");
setSlice(2);
run("Copy");
setSlice(3);
run("Paste");
setSlice(1);
run("Copy");
setSlice(2);
run("Paste");
setSlice(3);
run("Copy");
setSlice(1);
run("Paste");
setSlice(3);
run("Delete Slice");
rename("OUT");

selectWindow("Registered Source Image");
close();
selectWindow(images[0]);
close();


for (i=1; i<images.length; i++){
     open(path + images[i]);
	
     run("bUnwarpJ", "source_image=" + fixedImg + " target_image=" + images[i] + " registration=Accurate image_subsample_factor=2 initial_deformation=[Very Coarse] final_deformation=Fine divergence_weight=0 curl_weight=0 landmark_weight=0 image_weight=1 consistency_weight=10 stop_threshold=0.01");
     
     selectWindow("Registered Target Image");
     setSlice(1);
     run("Copy");
     
     selectWindow("OUT");
     run("Add Slice");
     run("Paste");

     selectWindow("Registered Target Image");
     close();
     selectWindow("Registered Source Image");
     close();
     selectWindow(images[i]);
     close();
}

selectWindow(fixedImg);
close();
selectWindow("OUT");
setSlice(1);
resetMinAndMax();

Have fun :grinning:

Thanks a lot @phaub! You give me lots of pointers, so I can play around in depth now. :slight_smile:

Hi @halloleo

if you are familiar with all the details and options of bUnwarpJ and your results are reasonable you are ready for the last steps:

#1 Instead of typing the file list manually you can process all images stored in a directory
#2 You can use the information generated by bUnwarpJ to determine the maximum insribed rectangle in the overlapping region of all the images and use this to crop the final image to a reasonable size.

To use the following macro make sure that the
FindMaxRect_0.1.jar is stored in your FIJI/ImageJ plugins folder.
(more information see here: Find maximal rectangle inscribed in a selection)

The Macro (version II):

path = "C:\\Test\\example1\\";

images = getFileList(path);
sourceImg = images[0];
targetImg = images[1]

open(path + sourceImg);
open(path + targetImg);

print("sourceImg: " + sourceImg);
print("targetImg: " + targetImg);

// Command from the Macro Recorder (with modified image names)
run("bUnwarpJ", "source_image=" + sourceImg + " target_image=" + targetImg + " registration=Accurate image_subsample_factor=2 initial_deformation=[Very Coarse] final_deformation=Fine divergence_weight=0 curl_weight=0 landmark_weight=0 image_weight=1 consistency_weight=10 stop_threshold=0.01");

// Create  coverage stack
selectWindow("Registered Target Image");
setSlice(3);
run("Copy");
run("Internal Clipboard");
rename("COV");

// Modify the first result stack
selectWindow("Registered Target Image");
setSlice(2);
run("Copy");
setSlice(3);
run("Paste");
setSlice(1);
run("Copy");
setSlice(2);
run("Paste");
setSlice(3);
run("Copy");
setSlice(1);
run("Paste");
setSlice(3);
run("Delete Slice");
rename("OUT");

selectWindow("Registered Source Image");
close();
selectWindow(targetImg);
close();

for (i=2; i<images.length; i++){
	 targetImg = images[i];
	 print("targetImg: " + targetImg);
	 
     open(path + targetImg);
	
     run("bUnwarpJ", "source_image=" + sourceImg + " target_image=" + targetImg + " registration=Accurate image_subsample_factor=2 initial_deformation=[Very Coarse] final_deformation=Fine divergence_weight=0 curl_weight=0 landmark_weight=0 image_weight=1 consistency_weight=10 stop_threshold=0.01");
     
     selectWindow("Registered Target Image");
     setSlice(1);
     run("Copy");
     
     selectWindow("OUT");
     run("Add Slice");
     run("Paste");

     selectWindow("Registered Target Image");
     setSlice(3);
     run("Copy");
     
     selectWindow("COV");
     run("Add Slice");
     run("Paste");

     selectWindow("Registered Target Image");
     close();
     selectWindow("Registered Source Image");
     close();
     selectWindow(targetImg);
     close();
}

selectWindow(sourceImg);
close();
selectWindow("OUT");
setSlice(1);
resetMinAndMax();

selectWindow("COV");
run("Z Project...", "projection=[Min Intensity]");
setOption("BlackBackground", true);
run("Make Binary");
run("Invert");
run("Select None");

selectWindow("MIN_COV");
run("Find MaxRectangle", "phi=0");

selectWindow("OUT");
run("Restore Selection");
run("Crop");

selectWindow("COV");
close();
selectWindow("MIN_COV");
close();

Note: bUnwarpJ should have the ability to work with RGB color images. I have corrected my last post.