Elastic alignment via TrakEM2 on a cluster

Hello everyone,

I am trying to do some elastic image alignment on a cluster. Therefore, I want to use the scripts provided by @axtimwalde on Github: https://github.com/saalfeldlab/parallel-elastic-alignment. As I do not have access to the tested version of Fiji, described in the documentation, I am using the latest. I had to do some modifications in the create-jobs, submit.jobs and import-and-align.bsh (due to file-paths and cluster architecture). Though if I am trying to run a single job, there are several errors in the terminal:

Picked up JAVA_TOOL_OPTIONS: -Xmx2048m
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
64 cores available for multi-threading
MAX_MEMORY: 10251526144
cache size: 4100610560
Crash detector file is /home/ml488799/Bildregistrierung/Skripte/Saalfeldsammlung/0-1/trakem2.1555421070433.1471712537.403928766/trakem2.mipmaps/.open.t2
Created crash detection system.
Red phone at port 29391
removing kl: javax.swing.plaf.basic.BasicTreeUI$Handler@37858383
removing kl: javax.swing.plaf.basic.BasicTreeUI$Handler@51f116b8
ERROR while parsing TrakEM2 plugins:
==================
ERROR:
java.lang.NullPointerException
	at ini.trakem2.utils.Utils.fixDir(Utils.java:1558)
	at ini.trakem2.Project$1.run(Project.java:134)
==================

removing kl: javax.swing.plaf.basic.BasicTreeUI$Handler@346d61be
Creating bucket 0,0,2048,2048
Scaling script path is null
Started processing: Importing images
TiffDelegateReader initializing /home/ml488799/Bildregistrierung/SIFT_Aligned_kleiner_Stack_002a_aus_001c/002.tif
TiffDelegateReader initializing /home/ml488799/Bildregistrierung/SIFT_Aligned_kleiner_Stack_002a_aus_001c/001.tif
Reading IFDs
Reading IFDs
Populating metadata
Populating metadata
Checking comment style
Checking comment style
Populating OME metadata
Populating OME metadata
WARNING fixed max larger than maximum max for type 3
WARNING fixed max larger than maximum max for type 3
WARNING fixed min larger than max for patch 002.tif #8
WARNING fixed min larger than max for patch 001.tif #9
SUBMITTING to regen 001.tif #9
Regenerating mipmaps (1 to go)
SUBMITTING to regen 002.tif #8
Regenerating mipmaps (2 to go)
Regenerating mipmaps (2 to go)
mipmaps for 001.tif z=0.0 #9
opening image /home/ml488799/Bildregistrierung/SIFT_Aligned_kleiner_Stack_002a_aus_001c/001.tif
Creating bucket 0,0,13741,13696
Creating bucket 0,0,13741,13696
Creating bucket 0,0,4096,4096
Creating bucket 0,0,4096,4096
Creating bucket 0,4096,4096,4096
Creating bucket 0,8192,4096,4096
Creating bucket 0,12288,4096,1408
Creating bucket 4096,0,4096,4096
Creating bucket 4096,4096,4096,4096
Creating bucket 4096,8192,4096,4096
Creating bucket 4096,12288,4096,1408
Creating bucket 8192,0,4096,4096
Creating bucket 8192,4096,4096,4096
Creating bucket 8192,8192,4096,4096
Creating bucket 8192,12288,4096,1408
Creating bucket 12288,0,1453,4096
Creating bucket 12288,4096,1453,4096
Creating bucket 12288,8192,1453,4096
Creating bucket 12288,12288,1453,1408
Creating bucket 0,4096,4096,4096
Creating bucket 0,8192,4096,4096
Creating bucket 0,12288,4096,1408
Creating bucket 4096,0,4096,4096
Creating bucket 4096,4096,4096,4096
Creating bucket 4096,8192,4096,4096
Creating bucket 4096,12288,4096,1408
Creating bucket 8192,0,4096,4096
Creating bucket 8192,4096,4096,4096
Creating bucket 8192,8192,4096,4096
Creating bucket 8192,12288,4096,1408
Creating bucket 12288,0,1453,4096
Creating bucket 12288,4096,1453,4096
Creating bucket 12288,8192,1453,4096
Creating bucket 12288,12288,1453,1408
Creating bucket 0,0,13741,13696
Creating bucket 0,0,13741,13696
Creating bucket 0,0,13741,13696
Processing... Importing images - 5 seconds
Unsupported format or not found
Processing... Importing images - 12 seconds
instance of: class [B
instance of: class [B
instance of: class [B
Processing... Importing images - 17 seconds
Processing... Importing images - 22 seconds
MipMaps with area downsampling: creation took 2086ms, saving took 8245ms, total: 10331ms


Regenerating mipmaps (1 to go)
mipmaps for 002.tif z=1.0 #8
opening image /home/ml488799/Bildregistrierung/SIFT_Aligned_kleiner_Stack_002a_aus_001c/002.tif
Processing... Importing images - 27 seconds
Opener: Unsupported format or not found
Processing... Importing images - 35 seconds
instance of: class [B
instance of: class [B
instance of: class [B
Processing... Importing images - 40 seconds
Processing... Importing images - 45 seconds
MipMaps with area downsampling: creation took 2202ms, saving took 8187ms, total: 10389ms


Processing... Importing images - 50 seconds
Done Imported 3/2 (50.85s approx.)
Done Imported 3/2 (50.85s approx.)
After saveAs, new xml path is: /home/ml488799/Bildregistrierung/Skripte/Saalfeldsammlung/0-1/project-1-2-3-4-5-6-7-8.xml
0 features extracted for layer z=1,000 `z=1.0'
0 features extracted for layer z=0,000 `z=0.0'
Extracted features in 24452ms
layer z=1,000 `z=1.0' -> layer z=0,000 `z=0.0': no correspondences found.
java.util.NoSuchElementException
	at java.util.HashMap$HashIterator.nextNode(HashMap.java:1444)
	at java.util.HashMap$KeyIterator.next(HashMap.java:1466)
	at mpicbg.models.TileConfiguration.preAlign(TileConfiguration.java:487)
	at mpicbg.trakem2.align.RegularizedAffineLayerAlignment.exec(RegularizedAffineLayerAlignment.java:410)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at bsh.Reflect.invokeMethod(Reflect.java:131)
	at bsh.Reflect.invokeObjectMethod(Reflect.java:77)
	at bsh.BSHPrimarySuffix.doName(BSHPrimarySuffix.java:170)
	at bsh.BSHPrimarySuffix.doSuffix(BSHPrimarySuffix.java:114)
	at bsh.BSHPrimaryExpression.eval(BSHPrimaryExpression.java:74)
	at bsh.BSHPrimaryExpression.eval(BSHPrimaryExpression.java:41)
	at bsh.BSHBlock.evalBlock(BSHBlock.java:125)
	at bsh.BSHBlock.eval(BSHBlock.java:75)
	at bsh.BSHBlock.eval(BSHBlock.java:41)
	at bsh.BSHTryStatement.eval(BSHTryStatement.java:80)
	at bsh.Interpreter.eval(Interpreter.java:659)
	at bsh.Interpreter.source(Interpreter.java:577)
	at bsh.Interpreter.main(Interpreter.java:367)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.imagej.launcher.ClassLauncher.launch(ClassLauncher.java:279)
	at net.imagej.launcher.ClassLauncher.run(ClassLauncher.java:186)
	at net.imagej.launcher.ClassLauncher.main(ClassLauncher.java:77)
Evaluation Error: Sourced file: /home/ml488799/Bildregistrierung/Skripte/Saalfeldsammlung/import-and-align.bsh : Attempt to resolve method: exit() on undefined variable or class name: runtime : at Line: 210 : in file: /home/ml488799/Bildregistrierung/Skripte/Saalfeldsammlung/import-and-align.bsh : runtime .exit ( 0 ) 

First, there is a error when initialising TrakEM2. Any ideas here?
Secondly, it tells me that there is an unsupported format or file not found. However, using the GUI-version, TrakEM2 is able to load the exact same image.
Then, there is a java error quite at the end and a problem with the exit code.
I would be really thankful for any ideas!
Max

Hi Max (@MaxAC),

I’ll try to have a look this week.

For the first error, you can try putting this into your Fiji’s (beanshesll) script interpreter:

import ij.Menus;
Menus.getPlugInsPath();

For the second error, are all your images tifs?

John

Hi,
thanks I will try this. yes all files are tifs.
Max

Hi John,
I have added your code lines in the beginning of the import-and-align.bsh script. Unfortunately It did not change anything, the first error still occurs.
Thanks for the idea, though!
Max

@MaxAC,

My fault, I wasn’t clear.

I intended to try to understand what caused the first error:

java.lang.NullPointerException
	at ini.trakem2.utils.Utils.fixDir(Utils.java:1558)
	at ini.trakem2.Project$1.run(Project.java:134)

but without trying to fix it yet.
My guess was that this line causes the problem. Specifically that

ij.Menus.getPlugInsPath()

might be returning null.

To test that, in your Fiji, do Plugins > Scripting > Script Interpreter, put in the lines above, and see what happens.

If your Fiji-gui setup might be different from what you’re using to run those scripts, then you can try putting those lines in the script, and explicitly print the results like so:

import ij.Menus;
System.out.println( "PLUGINS PATH: " +  Menus.getPlugInsPath());

just to see if that’s the cause.

Havn’t been able to look into this in any more detail yet, just hoping that may give us a clue for the first error,

John

Hi John,
Thanks for the clarification! So the script interpreter gives me back the path of the plugin folder (I replaced the user name):

>>> import ij.Menus;
Menus.getPlugInsPath();
/rwthfs/rz/cluster/home/my_user_name/packages/Fiji.app/plugins/

This seems right to me, isn’t it?

Max

1 Like

@MaxAC,

Yep, that’s what I’d expect. Hmm. :confused:

John

Has anyone any further ideas on this topic?
I already tried to email Mr. Saalfeld and asked for the custom built Fiji version that is suggested in the manual. So far, I did not get an answer though.