Speeding up ImageJ macro py parallel image loading and processing (Preloader)

Dear *,

I’m working on a procedure for speeding up ImageJ macros by parallel loading and processing of images.

The idea is loading image2 while image1 is processed. Afterwards, image3 will be loaded while image2 is processed and so on.

image

I just programmed a prototype which performs pretty well. I can spare 30% of overall processing time.
image

You can try it yourself by downloading the preloader.jar file, putting it in the jars subdirectory of your Fiji installation and restarting Fiji. Try this macro to reproduce my log window above:

Also: Find the source code of the plugin here.

I was wondering if something like this aleady exists somewhere in the ImageJ/Fiji universe. If not, might there be a place for the preloader in the Fiji distribution?

Cheers,
Robert

6 Likes

Hi @haesleinhuepf,

Maybe not fully related but with the same idea of Batch Processing, you can have a look to TAPAS. The idea is to separate the data to process (that can be eventually pre-loaded) and the processing itself.

Best,

Thomas

1 Like

Just a little update FYI: The Preloader and his little brother the RawPreloader now ship with the clij update site. I think speedup via preloading and speedup via GPU-acceleration go well hand in hand.

1 Like

hmm a beginner here trying to start with macros et al (by copy/pasting existing ones and modifying) but I have to process multiple SIM images (biiig) and was thinking of including your preloader in my humble macro…

Dialog.create("Batch Processing - SIM Thresholding");
	Dialog.addCheckbox("Use Bio-Formats Importer ", true);
Dialog.show();
		Bio=Dialog.getCheckbox();
		
input = getDirectory("location where images are stored");



//select location where images are stored
output = getDirectory("Location for results");

list = getFileList(input);

setBatchMode(true);

//loop to sequentially open images
for (im=0; im<list.length; im++){{
full = input + list[im];


if (Bio==true){
run("Bio-Formats Importer", "open=full autoscale color_mode=Composite view=Hyperstack stack_order=XYCZT");
} else {
open(full);
}

filename=getTitle();
//Get the name without the extension
ShortFileName=substring(filename, 0, lastIndexOf(filename,"."));
run("Threshold and 16-bit Conversion", "auto-scale");
}
rename(ShortFileName+" _THR ");
print(ShortFileName+" _THR " + " done");
out = output + ShortFileName + " _THR ";
saveAs("tiff", out);
run("Close All");
number=number+1;


setBatchMode(false);


print("All done - " + number + " images processed");

should I then just add “run(“Preloader”, “current=” + filename + " next=” + nextFilename);" on my line 23 within the for loop (instead of the bioformats importer)?

Cheers

Debbi

Hi @Debora_KellerOlivier,

I’m very sorry, but the preloader isn’t compatible with BioFormats yet. But you’re not the first one asking for this. I’ll try to have a look into this during spring and let you know if there is something to test.

Thanks for the suggestion btw. I love seeing users pushing developers in the right direction!

Cheers,
Robert

1 Like