ImageJ Headless mode, ROI manager and extended particle analyzer

I have a complex macro that has been developed over some time for some image analysis tasks including batch processing of images in a folder. I was now trying to run the macro on a computer cluster from a terminal in headless mode. In the process of trying to adapt it for headless operation, I run into problems with the code not finding certain intermediate images produced during the processing. Looking for some clues, I came across various comments of issues with the Particle Analyzer and ROI Manager in headless mode. I assume that the same issues also apply to the Extended Particle Analyzer.
So, in a nutshell is running a macro that relies quite heavily on the Extended Particle Analyzer and the ROI Manager in headless mode a non starter? Or are there any tricks/methods to make this work?
Thanks for any advice,
Volko

Hi @Volko,

to my knowledge, the ROI Manager issue is related to the ROI Manager being used in batch mode e.g. in a macro which I guess applies equally to headless mode.

The Extended Particle Analyzer might run into another problem, if a certain intermediate image is not visible. Just in case you face this issue, you might want to try it via scripting. Here you find some infos on how to achieve this here:

Unfortunately, that is not doable via a macro directly. But you can copy and adapt the script from the link above and then call the script in your macro. That would give you some more flexibility regarding the EPA.

Thanks, I will give that a go when I have a bit more time. For the time being, I think I just stick with what I have got and abandon my attempt to use the headless mode.
Cheers,
Volko

Not sure if this helps, but I have ImageJ on a linux system and use a virtual frame buffer to run scripts headless. This takes care of when ImageJ needs to display something in order to run. I use a command like this:

$xvfb-run -a ./ImageJ-linux64 --run /opt/someimagejscript.py

-John

Thanks John for the suggestion. I seem to get the first part of it to work (i.e. starting ImageJ in headless mode with xvfb), but I seem to get stuck with actually running the macro code.
I also saw in the Xvfb section on the headless section of ImageJ.net (https://imagej.net/Headless.html#Xvfb) an example of a shell script to run an ImageJ macro:
export DISPLAY=:1
Xvfb $DISPLAY -auth /dev/null &
(
# the '(' starts a new sub shell. In this sub shell we start the worker processes:
script=$scriptDir "lsmrotate2nrrd.ijm \"dir=" $1"&angle-x=$2&angle-y=
$3&angle-z=$4&reverse=$5\ " -batch"
$imagejBin -macro $script # running the actual ijm script
wait # waits until all 'program' processes are finished
# this wait sees only the 'program' processes, not the Xvfb process
)

However, I am unable to make it work for me because I don’t quite understand the synthax of the lines of code that appear to set the path for the macro directory:
script=$scriptDir "lsmrotate2nrrd.ijm \"dir=" $1"&angle-x=$2&angle-y=
$3&angle-z=$4&reverse=$5\ " -batch"

If I understand it correctly, the first part $scriptDir “lsmrotate2nrrd.ijm” sets the path to the location of the macro “lsmrotate2nrrd.ijm”, but what is the second part: “dir=” $1&…" -batch"? It appears to be a way to pass some parameters to the script/macro. So, if my macro doesn’t require any additional parameters, can I just omit that part?

Sorry, if these are rather obvious questions, but this is all rather new to me.
Thanks,
Volko

Hmm, I’m not sure about that either. My guess is the “$1… $2…” part is for passing parameters and the “-batch” is a flag to do batch processing https://imagej.net/Batch_Processing.

In the example you cite can you run your macro by the more simple example above the one you are talking about? Like my command?

Based on your example, I tried
xvfb-run -a ~/Documents/Fiji.app/ImageJ-linux64 --run ~/Documents/Headless/TestHeadless_.ijm

It appeared to attempt to start ImageJ. The following output was generated in the terminal window:
[vs64@spectre09 ~]$ xvfb-run -a ~/Documents/Fiji.app/ImageJ-linux64 --run ~/Documents/Headless/TestHeadless_.ijm
Java HotSpot™ 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot™ 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
[INFO] Overriding Sholl Analysis (Tracings)…; identifier: command:sc.fiji.snt.plugin.CallLegacyShollPlugin; jar: file:/lustre/ahome3/v/vs64/Documents/Fiji.app/jars/SNT-3.1.105.jar

But then it appeared to stop without the cursor in the terminal window being visible.
In order to get the cursor back, I had to use ctrl-C, which produced the following message:

^C/usr/bin/xvfb-run: line 186: kill: (27865) - No such process

When I tried to run the command again, I get the message:

[ERROR] Could not connect to existing ImageJ instance. If problem persists, please delete file: /tmp/ImageJ-vs64-_100-7.stub

This suggests to me that the original command started ImageJ, but somehow got stuck.
So, I am a bit stuck as well and don’t really know where to go with using ImageJ in headless mode from here.

I think you maybe close. Don’t give up yet!

I get an error about “Could not connect…” and some other stuff, but the script ran ok (I put in a print statement to show paths), then when it’s done, it just sits there until I do a control-C:

Hi John,

I think I managed to get it to work - at least with a very simple test macro.

The macro at the end of this post prints a welcome message, gets the path to the home directory, opens a file, prints the image title and image ID and saves a copy of the file.

I can execute the macro in headless mode using:
./ImageJ-linux64 --ij2 --headless --console -macro ~/Documents/Headless/TestHeadless_.ijm

In that case, the print outputs are directed to the terminal window.

I can also execute the macro using xvfb with the following instruction based on your suggestion:
xvfb-run -a ./ImageJ-linux64 --run ~/Documents/Headless/TestHeadless_.ijm

In that case, I don’t get any output to the terminal (didn’t expect it), but a copy of the file is saved at the end of the macro as expected.
The only thing is that I don’t know how to exit the instance of ImageJ created by xvfb. At the end of the macro, there is no cursor in the terminal window and I have to use ctrl-C to get the cursor back. That results in the error message:
^C/usr/bin/xvfb-run: line 186: kill: (7754) - No such process

But the ImageJ instance appears to be still active in the terminal as I get an error that says:
[ERROR] Could not connect to existing ImageJ instance. If problem persists, please delete file: /tmp/ImageJ-vs64-_100-7.stub

In order to run the macro again using vxfb I need to close the terminal and open a new terminal. Not the end of the world, just a bit clumsy and I would like to learn how to properly terminate ImageJ in vxfb.

Anyway, thanks for your help. I can now try to see if that will work with my more complex macros.
Thanks,
Volko

TestHeadless macro code

print(“Hello - Imagej started”);
path=getDir(“home”);
print(path);
open(path+"//Documents//Headless//TestTrain.tif");
print(getTitle(),getImageID());
saveAs(“Tiff”, path+"//Documents//Headless//Test-copy.tif");
print(“Finished”);

Agreed the xvfb method is not very elegant. It’s a good solution when running jobs remotely and I don’t want to interact with the ImageJ GUI. Good for you for getting it working! -John