Nonlinear Whole Slide Alignment Advice

I’d be really curious what software/pipelines are being used by anyone out there who is doing -nonlinear- whole scanned slide alignment. I know Visiopharm and Halo should do this in one form or another but I wonder if anyone has an open source approach they are happy with. The topic came up here: Automatically align and rotate images but I thought applications requiring nonlinear alignment could use the visibility of its own thread. I outline my application and what I’ve been tinkering with below but would love to hear about any relevant experiences.

The goal is to generate a merged aligned tiff for visual inspection of multiple scans superimposed (of the same slide restained primarily, rather than adjacent sections) as well as co-localization quantitation at the single cell level across multiple restainings (done in say QuPath).
With the same tissue restained, things seem to be linear locally and you could export and analyze ROI this way but this is awkward and globally, with re staining and drying out, things seem to creep around a good bit at the scale of the whole tissue slice requiring a nonlinear alignment. Though there aren’t necessarily shared channels, there should always be at least similar nuclear information in each scan.
So far, I’ve computed a cruddy warping on a low res images in Elastix just for testing purposes. I did this trying to map DAPI to one channel of color deconvolved IHC and applied this transformation to a high resolution tiff of one channel of the whole slide. I think I could get the alignment right with deeper study of Elastix parameters and though I don’t have much experience with these big images I imagine I could build script to semi-automate this and assemble all the resulting .raw files into a QuPath readable tiff. It looks from the literature like various folks have used a process like this, but so far I can’t find a real protocol. I am really curious however if anyone has an approach that is working for them, rather than reinventing the wheel since getting even this far as a test has taken quite a bit of time tinkering.
Thanks!

2 Likes

As an update on this if anyone is interested after tinkering with Elastix for a while, I went back to what I know best and all in Matlab have a decent two step linear/nonlinear alignment on a half resolution slide, computed at 1/16th and upsampled. I’m now fighting with how to get the aligned 8 channel ~60gb blob of data out of matlab in a Qupath readable format, if anyone has any input on best practice for this specific subproblem I’d love to hear it.

2 Likes

Hi @Anthony_Santella,

I’m definitely working on this, and for this I’m using BigWarp by @bogovicj combined with elastix. Things are working now but are undocumented. My workflow relies on a few repositories that I hope to make stable soon ® :wink: . Briefly, like what you did, I perform a rough global affine registration using elastix on low res images. Then I perform local high resolution registrations at some sparse points of interests -> a spline using these points is then applied on the whole slide image, leading to a very correct cellular registration for most of the slide.

The good thing about all this workflow, is that nothing is fully computed, thanks to the abstraction layers provided by ImgLib2 and BigDataViewer. If, cherry on the cake, you work with file formats which are multiresolution and supported by Bio-Formats, there will be no need to resave your data for the registration process and visualisation. Now if you need to quantify the fully transformed image on QuPath, you will need to recompute the image before importing into QuPath ( for the moment ).

Best,

Nicolas

5 Likes

Definitely, be curious to see more as you have it tided up, from my perspective the goal is to have something relatively tidy that can be put in the hand’s of only moderately computationally sophisticated core users as a fairly futz free end product, which is an advantage of an aligned image for Qupath as the end point, but definitely would be interested in trying it out. :slight_smile:

2 Likes

So far have gotten them written out from matlab as an image per channel with blockproc and attempt to concatenate and pyramidalize (if that’s a word) them with bfconvert on the command line has been running for about a day… I suspect I need a new strategy…

For those following the drama, that bfconvert is still running, just because I’m curious to see if it generates a usable file, though a method that takes a week is obviously not going to work. For now I’ve gotten frustrated and just given up, cutting up and writing the aligned, concatenated set of slides out to multiple individual files each of which was a chunk small enough to write with the tiff class in matlab (and read in imagej). Its not the solution I wanted but it will do for now, I’ll go back to it if get any advice or have time to dive deeper into low level file format issues to get the result readable in QuPath…

1 Like

Don’t have any advice- I just wanted to say I appreciate the work you are doing for this! Please share the solution when you have it :slight_smile:

2 Likes

Do the single channel MatLab exports open in QuPath? Sorry, not terribly familiar.

1 Like

The giant single channel images written from matlab with blocproc will open though slowly since they are not pyramidal. I imagine I could pyramidalize them more efficiently with bfconvert than concatenating and pyramidalizing them, but I haven’t tried. I did try writing everything out to one tiff page with 8 samples/pixel with blocproc from matlab, can generate a pyramid much faster from that big file with bfconvert, but the result is broken in some way. If I try to import it into fiji, the biformats importer thinks the top level has 40 channels, while the bfconvert generated pyramid levels have 8 channels as expected. If I try to open that file in qupath it silently sucks in a couple of the intermediate pyramid levels as individual images, but not the original full size one. My problem I think is twofold 1. I can’t figure out how to write giant multipage tiffs from matlab at all, only single pages. 2. I haven’t dug enough into code/documentation to understand how to ‘manually’ format the tiff and metadata properly for bioformats/qupath to like it. In the short term I’ve a bit run out of time for this, but I do want to get back to it.

1 Like

Interesting. I hit a metadata editing dead end recently too :slight_smile: So much to learn, not enough time. If you have two or three of the single channel TIFFs and could host them on GDrive or similar, I could take a shot at concatenating them in QuPath. Kind of curious.

Alternatively, if each individual channel is pyramidalized on it’s own, I think you could dynamically present all of the channels at once in QuPath. Though that is probably less useful from an analysis standpoint (though you could shuffle objects back and forth between single channel images to collect data).

1 Like

I haven’t used this, so I can’t offer specific advice, but have you tried the ImageJ-Matlab bridge here:
MATLAB Scripting - ImageJ ?