Batch processing of Leica image format with FIJI is painfully slow

Hi there!

I am working on some image correction of data from a Leica microscope, which is therefore saved in their proprietary data format, “.lif”. Often, I have more than 1000 images in a file (300 stacks, each with 3 channels and everywhere from 3 to 8 planes in z). My issue is that I need to collect all stacks as time points (they are actually positions, but that’s not important) in a single hyperstack before I can do my corrections. So fr I’ve managed to modify one of the example macros from the bio-formats webpage, but it takes forever to load the images, even though each slice is only 512x512, because for each slice, bio-formats re-reads and analyzes the entire .lif-file before extracting that one slice. Is there any way for me to work around this issue and speed up the process?

I’ve pasted my macro code below:

// bfOpenAsHyperstack.txt
// Written by Wayne Rasband
// Last updated on 2008 May 9

// Uses the Bio-Formats macro extensions to open a file as a hyperstack.

requires("1.41c");

path = File.openDialog("Select a File");
name = File.getName(path);

run("Bio-Formats Macro Extensions");
Ext.setId(path);
Ext.getCurrentFile(file);
Ext.getSizeX(sizeX);
Ext.getSizeY(sizeY);
Ext.getSizeC(sizeC);
Ext.getSizeZ(sizeZ);
Ext.getSizeT(sizeT);
Ext.getSeriesCount(sC);
Ext.getImageCount(n);

Dialog.create("Select range");
Dialog.addMessage("The file you selected contains " + sC + " image series\nPlease specify the range you wish to access.");
Dialog.addNumber("Begin", 0);
Dialog.addNumber("End", sC);
Dialog.show();
begin = Dialog.getNumber();
end = Dialog.getNumber();

//print(file+":", sizeX, sizeY, sizeC, sizeZ, sizeT);
setBatchMode(true);
Ext.getPixelsPhysicalSizeX(physicalSizeX);
Ext.getPixelsPhysicalSizeY(physicalSizeY);
Ext.getPixelsPhysicalSizeZ(physicalSizeZ);
for (s=begin; s <= end; s++) {
	showProgress(s-begin, end-begin);
	Ext.setSeries(s);
	for (i=0; i<n; i++) {
		Ext.openImage("plane "+i, i);
		if (i==0&&s==0) {
			stack = getImageID;
			setVoxelSize(physicalSizeX, physicalSizeY, physicalSizeZ, "microns");
		} else {
			run("Copy");
			close;
			selectImage(stack);
			run("Add Slice");
			run("Paste");
		}
	}
}
	rename(name);
	if (nSlices>1) {
		Stack.setDimensions(sizeC, sizeZ, end-begin);
		if (sizeC>1) {
			if (sizeC==3&&sizeC*sizeZ*(end-begin)==nSlices)
				mode = "Composite";
			else
				mode = "Color";
			run("Make Composite", "display="+mode);
		}
		setOption("OpenAsHyperStack", true);
	}

setBatchMode(false);

Ext.close();

Unfortunately, I am nowhere near familiar enough with Bio-formats or LIF files to help. However, I noticed what I think might be a mistake in your macro. I believe what you want is

if (i==0&&s==begin) {

The reason being that begin is not guaranteed to be 0 since you let the user set it.

Yeah, I actually discovered that one myself after posting because I was running the script and hit the bug :slight_smile: But thanks for letting me know!

If I understand your code correctly, in this section:

			run("Copy");
			close;
			selectImage(stack);

you are closing the stack prematurely, even if it has more slices or channels. Instead, keep the stack open and keep copying till you reach the last slice of the final channel. Don’t re-open the file for every slice. Try this first:

getDimensions(width, height, channels, slices, frames); //get Stack dimensions
//to see the values, try printing them out
print(width, height, channels, slices, frames); 

to check how many slices and channels the stack has. For example; If its two channels and 5 frames or slices each, then multiply channels x frames to get total number of slices.
Also, make sure you use
Stack.setChannel(n) to set the channel,
and Stack.setSlice or setFrame depending on your data, so you are copying from the right channel and slice

1 Like

Bio-formats works in mysterious ways :wink:

As I understand it, the line Ext.openImage("plane "+i, i); opens exactly one plane of an image and that is why I need to close that plane before opening the next in order to avoid wasting memory. I am beginning to suspect that the only solution is to use java, but that gives me a whole other problem (Fiji won’t run any of my java macros because it cannot find javac.exe and it also wants to download jcommon-1.0.17, which is apparently not possible from this location due to some network issues).

Aaah, ok.
Well, with the solution I proposed, you won’t have to close it every time.
perhaps, don’t use Ext.openImage command.
For each series, you could use:
run("Bio-Formats Importer" "open=["+your file path+"] color_mode=Default rois_import=[ROI manager] view=Hyperstack stack_order=XYCZT series_"+s);

You won’t need to use: Ext.setSeries(s); unless you plan to use Ext. commands later on…
I would recommend giving it at a try. It will save you time.

Let us know how you go.

1 Like

Using your suggestions to get the following code did certainly improve the overall speed, albeit not as much as if I ask bio-formats to just open everything in one go. If that was a viable option, I would prefer that (for a few of my files, it actually is possible, and I see a clear speed difference)

path = File.openDialog("Select a File");
name = File.getName(path);

run("Bio-Formats Macro Extensions");
Ext.setId(path);
Ext.getCurrentFile(file);
Ext.getSizeX(sizeX);
Ext.getSizeY(sizeY);
Ext.getSizeC(sizeC);
Ext.getSizeZ(sizeZ);
Ext.getSizeT(sizeT);
Ext.getSeriesCount(sC);
Ext.getImageCount(n);

Dialog.create("Select range");
Dialog.addMessage("The file you selected contains " + sC + " image series\nPlease specify the range you wish to access.");
Dialog.addNumber("Begin", 0);
Dialog.addNumber("End", sC);
Dialog.show();
begin = Dialog.getNumber();
end = Dialog.getNumber();

//print(file+":", sizeX, sizeY, sizeC, sizeZ, sizeT);
setBatchMode(true);
Ext.getPixelsPhysicalSizeX(physicalSizeX);
Ext.getPixelsPhysicalSizeY(physicalSizeY);
Ext.getPixelsPhysicalSizeZ(physicalSizeZ);
run("New HyperStack...", "title=HyperStack type=8-bit width=" + sizeX + " height=" + sizeY + " channels=" + sizeC + " slices=" + sizeZ + " frames=" + end-begin+1);
rename("HyperStack");
selectImage("HyperStack");
setVoxelSize(physicalSizeX, physicalSizeY, physicalSizeZ, "microns");
for (s=begin, t=1; s <= end; s++, t++) {
	showProgress(s-begin, end-begin);
	//Ext.setSeries(s);
	run("Bio-Formats Importer", "open=[" + path + "] color_mode=Composite rois_import=[ROI manager] view=Hyperstack stack_order=XYCZT series_"+s);
	rename("import");
	for (i=1; i<=sizeZ; i++) {
		//Ext.openImage("plane "+i, i);
		for (c = 1; c <= sizeC; c++) {
			selectImage("HyperStack");
			Stack.setPosition(c, i, t);
			selectImage("import");
			Stack.setPosition(c, i, 1);
			run("Copy");
			//close;
			selectImage("HyperStack");
			run("Paste");
		}
	}
	close("import");
}

I figured out a way to do what I need using bio-formats! If there are too many images in a file, it will switch to list mode and that can of course be forced using macros, so the following code will do what I want in a matter of seconds rather than minutes or even hours!

path = File.openDialog("Select a File");
name = File.getName(path);

run("Bio-Formats Macro Extensions");
Ext.setId(path);
Ext.getSeriesCount(sC);

Dialog.create("Select range");
Dialog.addMessage("The file you selected contains " + sC + " image series\nPlease specify the range you wish to access.");
Dialog.addNumber("Begin", 1);
Dialog.addNumber("End", sC);
Dialog.show();
begin = Dialog.getNumber();
end = Dialog.getNumber();

run("Bio-Formats", 
	"open=" + path + " color_mode=Composite " +
	"concatenate_series rois_import=[ROI manager] " + 
	"view=Hyperstack stack_order=XYCZT "+ 
	"series_list=" + begin + "-" + end);
1 Like

Nice one.
Good to know about this!

1 Like