Image Alignment by Mutual Information in Scikit-Image

Hi, I’ve been working with the register_translation method in scikit-image to align some images to each other. While it often works well, sometimes it does not - some of my datasets are a little ornery for cross-correlation alignment. I’ve had some other luck using normalized mutual information as a matching metric, but can’t find a routine for this in scikit-image. I was wondering if I’m somehow “missing” this in scikit-image? Is there already a routine somewhere for greyscale or landmark-based image registration / image alignment? I’ve written my own using sckit-learn, but my routine is very slow… so I’m hoping that somebody more skilled than me has maybe already solved this problem. :slight_smile:

Hi @jgelb!

You’re not going crazy, we don’t yet have generalised registration in scikit-image, but it’s coming! We have a GitHub Project to track progress:

I had some students make a start with affine registration based on grayscale NMI here:

As a stopgap, you can just grab the code from there. Not that it’s super-fast or anything! You can also experiment with swapping out the optimizer, or the affine transform. Long term, we are aiming to allow using gputools as a backend for some scikit-image operations, which would dramatically speed things up!

Finally, for marker-based registration, we have a tutorial:

All of this stuff will be in scikit-image at some point, but it’s taking time to get it all right…!

I hope that all helps! =)

2 Likes

Very cool, thanks for this! I will give your routine a try too… look forward to seeing this capability in scikit-image proper. :slight_smile: The landmark registration is a super cool idea! I tried that with OpenCV, but ran into a lot of logistical issues with OpenCV (since my images are X-ray micrographs and not regular photos/videos)… so I had more or less given up on landmarks. You are restoring my interest now!

If it’s helpful, I’m very happy to send over what I’ve been working on… but it’s nothing fancy, essentially shifts the image around in a grid and uses scikit-learn to calculate the nmi, finds the maximum value, and then assumes “aligned.” My routine as it is, of course, is both risky and time-intensive, but it does seem to work when it finally completes!