Automate tasks using macros in images placed in folders and subfolders

imagej

#1

I have recently discovered macros and how they can surprisingly ease my life working on Fiji/ImageJ.

I created this macro:

    run("Image Sequence...", "open=/home/mario/Desktop/prueba/1/Image-000002.tif");
    selectWindow("1");
    //setTool("rectangle");
    makeRectangle(406, 346, 1132, 845);
    run("Z Project...", "projection=[Average Intensity]");
    saveAs("Tiff", "/home/mario/Desktop/prueba/1/AVG_1.tif");

What this macro does is to import an image sequence stored in the referenced folder, to align the stack of images using the Template matching plugin, to do focus stacking using the Z Project function (Image > Stacks > Z Project…) and to save the newly generated image in the same folder using the tiff extension.

However, I do have a general folder with plenty subfolders filled with tiff files, so applying one by one the previous macro in each folder could become a tedious task as well. I came across with this macro that deals with batch processing:

    // "BatchProcessFolders"
    //
    // This macro batch processes all the files in a folder and any
    // subfolders in that folder. In this example, it runs the Subtract 
    // Background command of TIFF files. For other kinds of processing,
    // edit the processFile() function at the end of this macro.

       requires("1.33s"); 
       dir = getDirectory("Choose a Directory ");
       setBatchMode(true);
       count = 0;
       countFiles(dir);
       n = 0;
       processFiles(dir);
       //print(count+" files processed");
       
       function countFiles(dir) {
          list = getFileList(dir);
          for (i=0; i<list.length; i++) {
              if (endsWith(list[i], "/"))
                  countFiles(""+dir+list[i]);
              else
                  count++;
          }
      }

       function processFiles(dir) {
          list = getFileList(dir);
          for (i=0; i<list.length; i++) {
              if (endsWith(list[i], "/"))
                  processFiles(""+dir+list[i]);
              else {
                 showProgress(n++, count);
                 path = dir+list[i];
                 processFile(path);
              }
          }
      }

      function processFile(path) {
           if (endsWith(path, ".tif")) {
               open(path);
               run("Subtract Background...", "rolling=50 white");
               save(path);
               close();
          }
      }

However, I do not know how to merge my automated tasks written in my macro with the latter, as I am not an expert in coding.
To sum up, I would like to run my macro in every folder and subfolder automatically from a root directory of my selection.
Can anybody edit and merge the previous macros in order to accomplish my requirements?


#2

Hey @antecessor

Great start! Especially if you are just starting out with scripting in ImageJ…

I will point you to this older Forum post that should contain the answers you need to solve this issue:

Check out that post and the examples/links therein. And let us know if you need more specific help after that - we are here to help you.

But for sure - this should get you on the right path.

eta :slight_smile:


#3

Hi @etarena,

Thanks for your quick answer. I will, no doubt, go through all the links.
I have read the older post. However, I still cannot completely understand the language. I still don’t know the differences between a macro and a script. As you can see, I am still wet behind the ears.

Could you please help me developing the code?

Thanks in advance,
Mario


#4

Sure - we can help you @antecessor… but first try to go through those links and using that code example in that forum post - insert your own macro code and give it a try. You have everything you need already to get started. The best way to learn is by doing.

Too - if you read through some of those links posted there… you’ll see that a ‘macro’ is really just a short script - i.e. a few lines of code that carry out basic functions. The ImageJ Macro language is specific to ImageJ/Fiji and is just a ‘simple’ language that is useful to those just starting to write code… but scripts can be written in many languages in ImageJ to carry out tasks.

Just give a first go and post again when you have specific implementation questions. Going through the Scripting Workshop would be the best place to start if you really feel overwhelmed.

eta


#5

For the record: this same question was asked as well on stackoverflow.com. @antecessor: Please always disclose when cross-posting questions to different locations, so that others can also benefit from the answers.


#6

Thanks,

I am gonna try and let you know.


#7

Sure! If the answer is posted there, I will link to it here!


#8

I’ve been trying this macro/script @etarena, which is a mix between the one posted in your link and my own macro:

    /*
     * Macro template to process multiple images in a folder
     */

    #@ File (label = "/home/mario/Desktop/prueba", style = "directory") input
    #@ File (label = "/home/mario/Desktop/prueba", style = "directory") output
    #@ String (label = "File suffix", value = ".tif") suffix

    // See also Process_Folder.py for a version of this code
    // in the Python scripting language.

    processFolder(input);

    // function to scan folders/subfolders/files to find files with correct suffix
    function processFolder(input) {
    	list = getFileList(input);
    	list = Array.sort(list);
    	for (i = 0; i < list.length; i++) {
    		if(File.isDirectory(input + File.separator + list[i]))
    			processFolder(input + File.separator + list[i]);
    		if(endsWith(list[i], suffix))
    			processFile(input, output, list[i]);
    	}
    }

    function processFile(input, output, file) {
    	run("Image Sequence...");
    	selectWindow("1");
    	//setTool("rectangle");
    	makeRectangle(406, 346, 1132, 845);
    	run("Z Project...", "projection=[Average Intensity]");
    	saveAs("Tiff", "/home/mario/Desktop/prueba/");
    	print("Processing: " + input + File.separator + file);
    	print("Saving to: " + output);
    }

However, I get this error:
image

And my problems are:

  1. Don’t know where to write that the files it must import each time is a sequence, not independent tif files.
  2. Don’t know where to specify the saving directory of the files.
  3. Don’t know where to specify the root directory (I tried in the line 6 and 7, where input and output are.

#9

Ok @antecessor

Let’s take a quick step back…

  1. What version of ImageJ/Fiji are you using? In general - we recommend new users actually download and use the Fiji distribution of ImageJ (note: Fiji: Fiji Is Just ImageJ) So try installing/updating Fiji - that should solve the issue of not-recognizing the Script Parameters - the error you are getting.

  2. You need to USE the variables input, output, file within your processFile() function in order to work on and save the appropriate images, etc. so AGAIN - take the time and go through the Scripting Workshop… MINIMALLY - check out the example on this slide using the sample images mentioned on this slide.

I know it’s frustrating… but if you don’t just take the time now - it won’t stick.

eta


#10

I watched your workshop @etarena. It is fantastic.

And I also checked this post.

I think I am closer in the code, but still some points require attention.

This is the macro I adapted from the examples to my own purpose:

#@File(label = "Input directory", style = "directory") input
#@File(label = "Output directory", style = "directory") output
#@String(label = "File suffix", value = ".tif") suffix
	
processFolder(input);
	
// function to scan folders/subfolders/files to find files with correct suffix
function processFolder(input) {
	list = getFileList(input);
	for (i = 0; i < list.length; i++) {
		if(File.isDirectory(input + File.separator + list[i]))
			processFolder("" + input + File.separator + list[i]);
		if(endsWith(list[i], suffix))
			processFile(input, output, list[i]);
	}
}

function processFile(input, output, file) {
    run("Image Sequence...", "open=[input] number=304 starting=1 increment=1");
   	//setTool("rectangle");
	run("8-bit");
	makeRectangle(540, 354, 878, 746);
	run("Align slices in stack...", "method=5 windowsizex=620 windowsizey=454 x0=237 y0=138 swindow=0 subpixel=false itpmethod=0 ref.slice=1 show=true");
	run("Z Project...", "projection=[Average Intensity]");
	// save current binary image
	save(output + "/Confocal_image_" + file);
}

I decided to remove the setBatchMode() function in order to see all the intermediate steps and evaluate how the macro works. But in the final macro I will incluide it as it needs to deal with more than 1000 pictures.

This macro loops relatively well, as it detects 3 tif files in Folder 1, and 4 tif files in Folder 2, which is correct
I load them as Image sequence, where I transform in 8 bits. Then, I align all the tif files per folder using this Template matching plugin.

Later on, I do focus stacking on the aligned sequence of images using the Z-project function. Finally, I want to save the resulting focus-stacked image in the output folder following the same structure as in the input folder.

With the current macro I see the focus stacking and aligning run so many times as single tif files it detects. This is wrong as I want this tasks to be done only once per folder and image sequence.

The other process I do not know how to control is to save the final image per image sequence in the output folder following the same structure of the input one.

Thanks for your help

Mario


#11

@antecessor

Great that you found the workshop helpful!

So… just a few quick things that might help…

After your processFile() function call… you can then use another conditional statement (in pseudo code here) like:

if (filenamecontains("Confocal_image_")) {
then do the processing you need to do to that singular image
}

Just check the Built-in Macros Functions page.

That will get you started on the first part…

eta


#12

Hi Mario,

Here is some code that you might find helpful:


#13

Perfect @7rebor, I could solve this issue by adapting the code in the link. Great!


#14

@etarena

I don’t really know what to do with that code. I successfully maintained folder structure in batch processing in the output folder thanks to the comments of @7rebor.

However, I am still not able to deal with saving only one image per image sequence instead of so many images (which are the same) as images are in every folder and subfolder of the input directory. Would you mind being a little bit more specific?


#15

You must read your code logically and identify the source of the problem yourself. You know the problem, so only a few things can be resulting in this particular problem (all files being processed, rather than just one operation per folder).

The problem is in your if statement:

if(endsWith(list[i], suffix))
     processFile(input, output, list[i]);

This will result in all files being processed, no matter if they’re from the same folder or not. You want the processFile function to be called only once per folder. So you must end the for loop once it has been run once within a folder. To do this, I would increase the for loop variable (i) to its maximum, so no more loops are run for this folder and the master for loop will continue to the next folder.

Also, I have added more { } to your code for your if statements, to make sure the statements are constrained within the if statement itself. Partial edit of your code below:

for (i = 0; i < list.length; i++) {
	if(File.isDirectory(input + File.separator + list[i])) {
		processFolder("" + input + File.separator + list[i]);
    }
	if(endsWith(list[i], suffix)) {
		processFile(input, output, list[i]);
        i = list.length;
	}
}

I haven’t tried this, but it might help.

Best,

Rob


#16

@antecessor

I cannot help you in too much detail at the moment - bit of a crunch here at work.

But what I told you above still holds true… you don’t want the code that processes your output image in the processFile() function - as that will be called on each .tif file in each folder. You can create another function to process only the output image - based on the conditional I mentioned above - using the file name. Then call that function right after your first processFile() call. That’s one way of many that you could do this. Just write something and test it. Then edit - then test - etc. That’s what I’d have to do to try to solve this issue too.

What I wrote above is just pseudocode - so you can’t use it… was just written to give you the logic/structure to use.

Try something… and post your code here if you have more questions.

eta


#17

That’s awesome! Thanks @7rebor and @etarena for your invaluable support. I have just arrived to Macro/Scripting in ImageJ and surely will work harder in the future on this. I will write several posts on this in my open source webpage in Spanish (www.vivaelsoftwarelibre.com).


#18

I am here again… and again with an error. This is the complete code run:

    #@File(label = "Input directory", style = "directory") input
    #@File(label = "Output directory", style = "directory") output
    #@String(label = "File suffix", value = ".tif") suffix


    var outputDir // global variable to be passed to function later
    var inputDir 
    outputDir = File.getName(output); // retrieves the folder name string of the master input folder selected by the user above
    inputDir = File.getName(input);	

    setBatchMode(true)
    processFolder(input);
    	
    // function to scan folders/subfolders/files to find files with correct suffix
    function processFolder(input) {
    	list = getFileList(input);
    	for (i = 0; i < list.length; i++) {
    		if(File.isDirectory(input + File.separator + list[i])) {
    			processFolder("" + input + File.separator + list[i]);
    		}
    		if(endsWith(list[i], suffix)) {
    			processFile(input, output, list[i]);
    			i = list.length;
    		}
    	}
    }

    function processFile(input, output, file) {
        run("Image Sequence...", "open=[input] number=304 starting=1 increment=1");
    	run("8-bit");
    	run("Sharpen", "stack");
    	run("Enhance Contrast...", "saturated=0.3");
    	makeRectangle(540, 354, 878, 746);
    	run("Align slices in stack...", "method=5 windowsizex=620 windowsizey=454 x0=237 y0=138 swindow=0 subpixel=false itpmethod=0 ref.slice=1 show=true");
    	run("Z Project...", "projection=[Average Intensity]");
    	
    	// save the images following the same input directory folders
    	saveDir = replace(input, inputDir, outputDir); // replaces the input folder name (string) with the output folder name (string)
    	File.makeDirectory(saveDir); // makes the above directory
    	saveAs("TIFF", saveDir + file);
    }

Everything seems to work fine. The code scans the input folder directory searching for other folders and so on. However, when saving the pictures, I get this error:
Error

I have run the code several times, and I am pretty sure this error comes from the saving function. And it always occurs when trying to save in a subfolder directory. For instance, if I have this structure:

  • Input folder
    ** Subfolder 1
    ** Subfolder 2
    ** Subfolder 3
    ***** Sub-subfolder 1

where every folder/subfolder does have images inside, when loading the Sub-subfolder 1, it seems that it cannot save the image.

I have gone through the code, but I do not pick up the error. Any idea @7rebor @etarena?


#19

@antecessor

To start debugging a bit… i would just start inserting some print statements… to be sure what your variables are at each step. So print out input, inputDir, outputDir, etc. to be sure those strings are correct. Seems that you are getting a file as opposed to a folder at the end - eh?

eta


#20

If you have no output directories, and then run the script, does it create those output directories correctly? Do you have the same folder structure being created?

Rob