I’ve been working to automate a lot of my labs data collection and analysis in python, so I’ve started working recently with pyimagej. It has proven to be quite the challenge to get things running. For context, the end goal is to create many 2-frame hyperstacks and get the Advanced PIV plugin up and running without any memory issues (this certainly involves closing things as we go, but that’s likely a logistics discussion for another time). I’m not really familiar with java, so there’s a chance I’ve done something explicitly incorrect, but I figured I’d post my experiences regardless.
I’m working with python 2.7 in Spyder, so I’ve created a python3 virtual env in my Anaconda2 folder via the recommended method. pyimagej github
Using the Anaconda Command Prompt in Administrator mode:
C:\Users\msout\Anaconda2>conda config --add channels conda-forge C:\Users\msout\Anaconda2>conda config --set channel_priority strict C:\Users\msout\Anaconda2>conda create -n pyimagej pyimagej openjdk=8
Making sure that the environment has all the modules it needs: any that were willing to cooperate with conda were installed as such. The others used pip. If memory serves, most of the entries beyond cython came back as already satisfied.
C:\Users\msout\Anaconda2>activate pyimagej (pyimagej) C:\Users\msout\Anaconda2>conda install requests (pyimagej) C:\Users\msout\Anaconda2>conda install cython (pyimagej) C:\Users\msout\Anaconda2>pip install update imglyb (pyimagej) C:\Users\msout\Anaconda2>pip install update maven (pyimagej) C:\Users\msout\Anaconda2>pip install update pyjnius (pyimagej) C:\Users\msout\Anaconda2>pip install update scyjava (pyimagej) C:\Users\msout\Anaconda2>conda install scikit-image
Confirming that imagej can be imported successfully
(pyimagej) C:\Users\msout\Anaconda2\envs\pyimagej>activate pyimagej (pyimagej) C:\Users\msout\Anaconda2\envs\pyimagej>python >>> >>>import imagej >>>
So things seem to work ok. Spyder is set to use the virtual environment by changing its interpreter to
C:\Users\msout\Anaconda2\envs\pyimagej\python.exe. I can confirm that it’s running python3 with print statements.
For anyone looking at this in the future, keep in mind that I’ve manually set the JAVA_HOME and PYJNIUS_JAR environment variables on my system. The proper values for these variables can be seen when you call
activate pyimagej in the Anaconda Prompt. I’ve tried changing these variables inside of my python script but for whatever reason that was giving me problems.
Initializing ImageJ in Python
import imagej import jnius
File "C:/Users/msout/Documents/virtual env testing.py", line 125, in <module> import jnius File "C:\Users\msout\Anaconda2\envs\pyimagej\lib\site-packages\jnius\__init__.py", line 12, in <module> from .jnius import * # noqa ImportError: DLL load failed: The specified module could not be found.
Curious, but I saw online that we could format that differently:
import imagej ij = imagej.init(headless=False) import jnius
That works. Apparently jnius can only be imported after ImageJ is initialized? Curious, but not a big issue. (I believe the necessity of this ordering has been discussed on this forum before, but I don’t still have the link to the thread. Im sorry, I might go hunt for it later.)
from jnius import autoclass ImagePlus = autoclass('ij.ImagePlus')
File "C:/Users/msout/Documents/virtual env testing.py", line 144, in <module> ImagePlus = autoclass('ij.ImagePlus') File "C:\Users\msout\Anaconda2\envs\pyimagej\lib\site-packages\jnius\reflect.py", line 159, in autoclass c = find_javaclass(clsname) File "jnius\jnius_export_func.pxi", line 26, in jnius.find_javaclass JavaException: Class not found b'ij/ImagePlus'
Hmmm. Trying another initialization endpoint, my local installation of imagej.
ij = imagej.init('C:\\Users\\msout\\Desktop\\ImageJ',headless=False) from jnius import autoclass ImagePlus = autoclass('ij.ImagePlus')
Same result as above.
ij = imagej.init('sc.fiji:fiji',headless=False) from jnius import autoclass ImagePlus = autoclass('ij.ImagePlus')
That one works!! But now I don’t have access to my locally installed plugins. I made a hack to get access to local plugins and the ij classes. This is inside of
def init() in
scyjava_config.add_options('-Dplugins.dir=' + plugins_dir) scyjava_config.add_endpoints(ij_dir_or_version_or_endpoint)
was changed to
scyjava_config.add_options('-Dplugins.dir=' + plugins_dir) scyjava_config.add_endpoints('sc.fiji:fiji')
This allows me to get past
autoclass('ij.ImagePlus') and gives me access to my local plugins.
To the best of my knowledge, it should be able to simply use
init('sc.fiji:fiji',headless=False) and install the plugins I want via
run commands? If that’s seems like a viable alternative to my approach, I’m certainly willing to try it provided some assistance on formatting the arguments in python.
SIDE NOTE: the piv plugin benefits from some keyword arguments that dont have values, so I made a tiny edit to _format_argument() and _format_value() in imagej.py such that it accepts None entries. This can be discussed in another (shorter) thread if desired.
The Fiji Memory Issue
And here’s where I’m at now. Moving away from PIV for a moment, I was building my code to open up our tiff stacks, combine them into a single file (it saves in pieces unfortunately), and then open the combined tiff stack. The two stacks are 1.7 and 1.13 GB respectively.
IJ = autoclass('ij.IJ') Concatenate = autoclass('ij.plugin.Concatenator') WindowManager = autoclass("ij.WindowManager") print("1: ",IJ.freeMemory()) im1 = IJ.open(im_1_path) im2 = IJ.open(im_2_path) print("2: ",IJ.freeMemory()) imp1 = WindowManager.getImage(im_1_name) imp2 = WindowManager.getImage(im_2_name) print("3: ",IJ.freeMemory()) imp3 = Concatenate.run(imp1,imp2) print("4: ",IJ.freeMemory()) IJ.saveAsTiff(imp3,im_1_dir+new_name) print("5: ",IJ.freeMemory())
1: 137MB of 12091MB (1%) 2: 2805MB of 12091MB (23%) 3: 2805MB of 12091MB (23%) 4: 2814MB of 12091MB (23%) 5: 2821MB of 12091MB (23%)
This is acceptable, as by all accounts we should now be free to open up the resulting 2.68GB file.
IJ.open('C:\\Users\\msout\\Downloads\\camera 1 flow Data.tiff')
Annnnd we get an error. (sorry about the image size)
Ok then, well since my python script has resolved, I’ll just click Edit>Options>Memory & Threads to see what’s going on.
I’m not particularly intimate with java or imagej so I could be full of it, but it looks like my hack has created a scenario where imageJ is reporting its available memory based off of ImageJ.cfg, but is actually limiting memory usage based on the defaults created by
imagej.init('sc.fiji:fiji',headless=False). It looks like a discussion on this topic was started already here. This might also be a relevant discussion?
Soooooo, anyone got any ideas? I’m open to changing my approach to any of these steps. Getting this far has been multiple days of troubleshooting, from installation to
IJ.open and back.