RealTime-like 2D image stitching

Dear users,

I would like to make a mosaic (6x6) with an unmotorized microscope that can only take pictures one by one. I currently managed to manually take them and later stitch on Fiji; however it results extremely time-consuming (as I need around 150 mosaics of 6x6 and sometimes miss parts of the image). To be faster, I tried to record a video and stitch part of the photograms, but motion blur occurs. In both cases, my computer took a lot of time to stitch them and usually misplaced some images.

My question is, is there any way to preview the result as the images are being taken? As they are sequential, maybe telling Fiji in which folder the new images appear? So Fiji takes the last one and adds to the already stitched ones. Would it be possible a macro for this (that does it fast)? Or do you think there is a more simple way?

Thanks for the advice

1 Like

Hi Quique,

Did you already try out the grid/collection stitching plugin?
If you save the images with a correct name you can indicate where the images need to be placed in the mosaics!
If you then write a macro to loop through all the images in a folder, this might speed up your work considerably. If you have troubles writing a macro, let me know! (Although recording a macro is pretty easy) I need to do practically the same thing with some CT-stacks, so I can maybe write a more generic macro to allow us to stitch several mosaics in one go.

Regards,
Bonehead

2 Likes

Hi Bonehead,
I will try to write more specifically my question:

I have a folder that each 5-8 seconds gets a new 2D image partially overlapping the previous one. Their names have an increasing number (“A 001”,”A 002”,”A 003”, etc.).

I want Fiji to form a mosaic with them and continue adding pieces to the mosaic as soon as they appear on the folder, but avoiding stitching all from the first one on each cycle and having to test all possible combinations (to be faster) note that as manually displacing the slide some rows could have different number of overlapping pictures
My first idea was to run a macro considering:

  1. “A001.tif” appears on the folder
  2. “A002.tif” appears on the folder
  3. Macro

    3.1. Close “A000.tif” (To avoid accumulating the command 3.6)
    3.2. Mosaic of all images named “A{iii}.tif” (Considering they are sequential but overlapping an unknown border)
    3.3. Save mosaic as “BACKUP.tif” on the folder (overwriting)
    3.4. Rename all “A{iii}.tif” to “B{iii}.tif”)
    3.5. Save a copy of “BACKUP.tif” as “A000.tif” on the folder
    3.6. Open “A000.tif”
  4. “A003.tif” appears on the folder and the macro runs again

By doing this, I keep a copy of each single picture (“B…”) and temporary the previous mosaic (“BACKUP”) in case the stitching is wrong (I can see the result after each cycle). It also permits to add several new pictures to the mosaic per cycle.

So my issues here are:

  1. How can I include in the macro the renaming function?
  2. Is it possible to execute the macro automatically each 3 seconds avoiding errors if the new image still did not appear? (To avoid doing it manually)
  3. And most important, is there an easier, more simple or faster way to do this? The point is that Fiji will not be able to communicate with the image capture program, so it will only “see” the folder where the new images appear.
  • Note: I just started writing macros so I only have experience in their recording. I personally think that this problem is common and an open source would help a lot of people in the future! (Check the link below)

http://www.jpathinformatics.org/article.asp?issn=2153-3539;year=2013;volume=4;issue=2;spage=9;epage=9;aulast=Gherardi

Thanks for the help!

Quique

1 Like

I do not know of any built-in plugins for doing this. Fiji’s Stitching plugin assumes you have already acquired all the tiles, and want to stitch them all, once only.

You might be able to hack something up like this using TrakEM2 but it would likely be nontrivial to implement.

See also the MIST stitching plugin which is very fast but only works with a limited selection of file formats. (TIFF is one of them, though, so maybe it helps you…)

MIST uses the core ImageJ image reader so it will handle any image files that reader can (definitely a limited selection of file formats compared to BioFormats). However, it does not handle multi-plane TIFFs. The plugin assumes that each image on disk is a 2D grid of pixels.

More importantly, it relies on estimating several model parameters of the automatic microscope stage so it needs a grid of images acquired using a motorized stage with roughly constant horizontal overlaps between images and roughly constant vertical overlaps between images (horizontal and vertical overlaps can be different).

1 Like

What I was trying to do was to call pairwise stitching for two images, but somehow I don’t manage to succeed. I don’t have enough experience with programming in Jython. I created a script and got this far:

from ij import IJ
import os
import plugin.Stitching_Pairwise as pairwiseStitch
import mpicbg.stitching.StitchingParameters as stitchingParameters
import stitching.CommonFunctions as commonFunctions
params = stitchingParameters()
# Setting the stitching parameters
params().checkPeaks = 20stitchingParameters
params().subpixelAccuracy = True
params().computeOverlap = True
params().dimensionality = 3 # SET THE DIMENSIONALITY OF YOUR IMAGES
print stitchingParameters().computeOverlap
# AVG = 0, LIN_BLEND = 1, MAX = 2, MIN = 3, RED_CYAN = 4, NONE = 5;
stitchingParameters().fusionMethod = 1


# fusion_method=[Max. Intensity] fused_image=tile_x01_y01.tif<->tile_x02_y01.tif check_peaks=10
    # compute_overlap subpixel_accuracy registration_channel_image_1=[Average all channels]
    # registration_channel_image_2=[Average all channels]"


def main():
    # Define the folder where the script should look for image folders
    imageFolder = "C:\\20160309" #!# INSERT YOUR OWN MAIN FOLDER HERE!
    imageExtension = ".tif" # DEFINE THE TYPE OF IMAGES YOU WANT TO STITCH
    minCorrelation = 0.95 # MINIMAL CORRELATION REQUIRED FOR A GOOD STITCH
    dirList = getFolderList(imageFolder)
    for subDir in dirList:
        stitchDir = imageFolder + os.sep + subDir
        # print stitchDir
        fileLst = getFileList(stitchDir,imageExtension)
        stitchImages(stitchDir,fileLst,minCorrelation)
    
# Function to get a list of subfolder
#    This function only gets the folders from the current directory depth
def getFolderList(folder):
    folderLst = []
    for item in os.listdir(folder):
        if os.path.isdir(os.path.join(folder, item)):
            folderLst.append(item)
    return folderLst

def getFileList(folder, extension):
    fileLst = []
    for file in os.listdir(folder):
        if file.endswith(extension):
            fileLst.append(file)
    return fileLst

# Stitches all the .tif files in a selected folder
def stitchImages(folder,fileLst,minCorrelation):
    if len(fileLst) > 2:
        image1location = folder + os.sep + fileLst[0]
        imp1 = IJ.openImage(image1location)
        ## imp1.show()
        ## print image1location
        image2location = folder + os.sep + fileLst[1]
        imp2 = IJ.openImage(image2location)
        ## imp2.show()
        ## print image2location
        # Stitch the image files pairwise
        # results = pairwiseStitch().performPairWiseStitching(imp1, imp2, stitchingParameters)
    
if __name__ == "__main__":
    main()

But i don’t know how I can call the pairwise stitching plugin in the background with the desired parameters. Can someone give any direction on how to do that!!??
After I’ve calculated all the pairwise stitches, the goal was to stitch them all together using the coordinates from the pairwise stitching.

This code can than be run after acquiring all the microscope images, or (when altered a bit) can be kept running in the background.

Regards,
Bonehead

@Bonehead Nice idea to use pairwise stitching in a script.

@StephanPreibisch Any suggestions?

I also tried to pairwise stitching. However, after 6-7 cycles it was not working anymore and parts of the image were lost. Could it be that the bigger the moisaic gets the less % overlapping area it has with the next incorporating image? Sorry that I can’t help too much but my programming knowledge is null

Does the Pairwise stitching plugin handle images that are of different sizes? I have only ever used it with image pairs that are the same size and I think (unless I am misreading) that @Quique is stitching each new image to the current mosaic of previous images; which might lead to trouble aligning the images after a number of cycles.

It might work better to accumulate a set of pairwise translations and then the use the global optimization from the Grid/Collections stitching plugin (source) to assemble the pairwise translations?

Grid/Collection Stitching Github Page

1 Like

@Quique

I know that the pairwise stitching might seem like it will not get you the correct output, but I just want to calculate the coordinates with respect to the previous images. What i want the code to do in “pseudo code” is:

  1. Calculate the location of the current image with respect to the previous image in the row/column
  2. If the correlation between the two images is to low, calculate the location of the current image with respect to the first image of the previous row/column
  3. Transform the coordinates from the stitching process to the coordinate system of the first image (the reference image)
  4. Write the set of coordinates to a formatted text file
  5. Use the coordinate set of all the images to run the “Grid/Collection Stitching” with the option “Positions from file”

These steps are what I’m currently doing to stitch my microCT stacks by hand. I copy the coordinates to an Excel file :dizzy_face:. However, it is the only option that doesn’t crash on me due to memory problems. (The microCT stacks have a very high resolution of 20 um)
I know this works for me, but I only have one row of images without any distortion at the edges. I don’t know if microscope images are also suited for stitching in this fashion.

@mmajurski
With the previous code all you do is stitch two images of the same size. I believe it actually does the same as what you are suggesting with: [quote=“mmajurski, post:9, topic:1565”]
It might work better to accumulate a set of pairwise translations and then the use the global optimization from the Grid/Collections stitching plugin (source) to assemble the pairwise translations?
[/quote]

@StephanPreibisch

Dear Stephan,

It would be great if you can give me a hint on how to output the correlation R and the dimensional coordinates (x, y, z, t) from a Pairwise Stitch.
The source code of the pairwise stitching on GitHub feels like a mangrove forest to me, because of all the varying classes and function and because I’m not familiar with Java.
However, it seems like I might be able to copy the: public static < T extends RealType<T>, S extends RealType<S> > PairWiseStitchingResult computePhaseCorrelation( final Image<T> img1, final Image<S> img2, final int numPeaks, final boolean subpixelAccuracy ) function from this link.

If i would rewrite the above script to Java, do you feel like I might be able to get the script work? I.e. output the coordinates of the pairwise stitches, writing the coordinates to a TileConfiguration.txt and using that file as an input for the Grid/Collection Stitching plugin?

Kind regards,
Bonehead