Difficulty with imagej.init using pyimageJ -- some workarounds and shortcomings

Hi all,

I’ve been working to automate a lot of my labs data collection and analysis in python, so I’ve started working recently with pyimagej. It has proven to be quite the challenge to get things running. For context, the end goal is to create many 2-frame hyperstacks and get the Advanced PIV plugin up and running without any memory issues (this certainly involves closing things as we go, but that’s likely a logistics discussion for another time). I’m not really familiar with java, so there’s a chance I’ve done something explicitly incorrect, but I figured I’d post my experiences regardless.

Installation

I’m working with python 2.7 in Spyder, so I’ve created a python3 virtual env in my Anaconda2 folder via the recommended method. pyimagej github

Using the Anaconda Command Prompt in Administrator mode:

C:\Users\msout\Anaconda2>conda config --add channels conda-forge
C:\Users\msout\Anaconda2>conda config --set channel_priority strict
C:\Users\msout\Anaconda2>conda create -n pyimagej pyimagej openjdk=8

Making sure that the environment has all the modules it needs: any that were willing to cooperate with conda were installed as such. The others used pip. If memory serves, most of the entries beyond cython came back as already satisfied.

C:\Users\msout\Anaconda2>activate pyimagej
(pyimagej) C:\Users\msout\Anaconda2>conda install requests
(pyimagej) C:\Users\msout\Anaconda2>conda install cython
(pyimagej) C:\Users\msout\Anaconda2>pip install update imglyb
(pyimagej) C:\Users\msout\Anaconda2>pip install update maven
(pyimagej) C:\Users\msout\Anaconda2>pip install update pyjnius
(pyimagej) C:\Users\msout\Anaconda2>pip install update scyjava
(pyimagej) C:\Users\msout\Anaconda2>conda install scikit-image

Confirming that imagej can be imported successfully

(pyimagej) C:\Users\msout\Anaconda2\envs\pyimagej>activate pyimagej
(pyimagej) C:\Users\msout\Anaconda2\envs\pyimagej>python
>>>
>>>import imagej
>>>

So things seem to work ok. Spyder is set to use the virtual environment by changing its interpreter to
C:\Users\msout\Anaconda2\envs\pyimagej\python.exe. I can confirm that it’s running python3 with print statements.

For anyone looking at this in the future, keep in mind that I’ve manually set the JAVA_HOME and PYJNIUS_JAR environment variables on my system. The proper values for these variables can be seen when you call activate pyimagej in the Anaconda Prompt. I’ve tried changing these variables inside of my python script but for whatever reason that was giving me problems.

Initializing ImageJ in Python

import imagej
import jnius

  File "C:/Users/msout/Documents/virtual env testing.py", line 125, in <module>
    import jnius

  File "C:\Users\msout\Anaconda2\envs\pyimagej\lib\site-packages\jnius\__init__.py", line 12, in <module>
    from .jnius import *  # noqa

ImportError: DLL load failed: The specified module could not be found.

Curious, but I saw online that we could format that differently:

import imagej
ij = imagej.init(headless=False)
import jnius

That works. Apparently jnius can only be imported after ImageJ is initialized? Curious, but not a big issue. (I believe the necessity of this ordering has been discussed on this forum before, but I don’t still have the link to the thread. Im sorry, I might go hunt for it later.)

However:

from jnius import autoclass
ImagePlus = autoclass('ij.ImagePlus')

  File "C:/Users/msout/Documents/virtual env testing.py", line 144, in <module>
    ImagePlus = autoclass('ij.ImagePlus')

  File "C:\Users\msout\Anaconda2\envs\pyimagej\lib\site-packages\jnius\reflect.py", line 159, in autoclass
    c = find_javaclass(clsname)

  File "jnius\jnius_export_func.pxi", line 26, in jnius.find_javaclass

JavaException: Class not found b'ij/ImagePlus'

Hmmm. Trying another initialization endpoint, my local installation of imagej.

ij = imagej.init('C:\\Users\\msout\\Desktop\\ImageJ',headless=False)
from jnius import autoclass
ImagePlus = autoclass('ij.ImagePlus')

Same result as above. :disappointed:

Another try:

ij = imagej.init('sc.fiji:fiji',headless=False)
from jnius import autoclass
ImagePlus = autoclass('ij.ImagePlus')

That one works!! But now I don’t have access to my locally installed plugins. I made a hack to get access to local plugins and the ij classes. This is inside of def init() in imagej.py

scyjava_config.add_options('-Dplugins.dir=' + plugins_dir)
scyjava_config.add_endpoints(ij_dir_or_version_or_endpoint)

was changed to

scyjava_config.add_options('-Dplugins.dir=' + plugins_dir)
scyjava_config.add_endpoints('sc.fiji:fiji')

This allows me to get past autoclass('ij.ImagePlus') and gives me access to my local plugins.

To the best of my knowledge, it should be able to simply use init('sc.fiji:fiji',headless=False) and install the plugins I want via run commands? If that’s seems like a viable alternative to my approach, I’m certainly willing to try it provided some assistance on formatting the arguments in python.

SIDE NOTE: the piv plugin benefits from some keyword arguments that dont have values, so I made a tiny edit to _format_argument() and _format_value() in imagej.py such that it accepts None entries. This can be discussed in another (shorter) thread if desired.

The Fiji Memory Issue

And here’s where I’m at now. Moving away from PIV for a moment, I was building my code to open up our tiff stacks, combine them into a single file (it saves in pieces unfortunately), and then open the combined tiff stack. The two stacks are 1.7 and 1.13 GB respectively.

IJ = autoclass('ij.IJ')
Concatenate = autoclass('ij.plugin.Concatenator')
WindowManager = autoclass("ij.WindowManager")

print("1: ",IJ.freeMemory())

im1 = IJ.open(im_1_path)
im2 = IJ.open(im_2_path)

print("2: ",IJ.freeMemory())

imp1 = WindowManager.getImage(im_1_name)
imp2 = WindowManager.getImage(im_2_name)

print("3: ",IJ.freeMemory())

imp3 = Concatenate.run(imp1,imp2)

print("4: ",IJ.freeMemory())

IJ.saveAsTiff(imp3,im_1_dir+new_name)

print("5: ",IJ.freeMemory())

Returns

1:  137MB of 12091MB (1%)
2:  2805MB of 12091MB (23%)
3:  2805MB of 12091MB (23%)
4:  2814MB of 12091MB (23%)
5:  2821MB of 12091MB (23%)

This is acceptable, as by all accounts we should now be free to open up the resulting 2.68GB file.

IJ.open('C:\\Users\\msout\\Downloads\\camera 1 flow Data.tiff')

Annnnd we get an error. (sorry about the image size)

Imgur

Ok then, well since my python script has resolved, I’ll just click Edit>Options>Memory & Threads to see what’s going on.

Imgur

sigh

I’m not particularly intimate with java or imagej so I could be full of it, but it looks like my hack has created a scenario where imageJ is reporting its available memory based off of ImageJ.cfg, but is actually limiting memory usage based on the defaults created by imagej.init('sc.fiji:fiji',headless=False). It looks like a discussion on this topic was started already here. This might also be a relevant discussion?

Soooooo, anyone got any ideas? I’m open to changing my approach to any of these steps. Getting this far has been multiple days of troubleshooting, from installation to IJ.open and back.

2 Likes

Hi @msouth,

Nice thorough description of what you did :+1:

Regarding this part:

JavaException: Class not found b'ij/ImagePlus'

Quick idea, ij.ImagePlus is likely in imagej-legacy, so this may help

ij = imagej.init('net.imagej:imagej+net.imagej:imagej-legacy')

more information and examples here.

John

3 Likes

@bogovicj Thank you so much! That does in fact get me through my autoclass calls! It does still lead me down the road to plugin access and memory issues unfortunately, but getting that far certainly looks cleaner now. Any suggestions for taking it further? I suppose I could hack the plugin directory the same way I originally hacked .add_endpoints, but I’m starting to believe there’s probably a better way. The memory is correctly reporting now thankfully, even if it is still small.

Out of my own curiosity, is there an altogether better approach to using pyimageJ? It didn’t occur to me that I was calling on classes from legacy imageJ so I’m forced to wonder if I didn’t incidentally follow an outdated example when I first got a foothold on this project. I notice that the example linked at the bottom of the page you shared looks quite different from my own implementation.

Sorry for all the questions. I’m not a fan of having to bother more than once but I’m afraid that I’m already pushing the boundaries on my current understanding of how to best format this project.

1 Like

@msouth,
Nice!

I don’t have ideas, but I havn’t used pyimagej much. @hanslovsky has, so if I were in your shoes, I’d try stealing (taking inspiration) from his stuff. Checking out what he does , it looks like add_endpoints is the way to go. Maybe he can comment further.

I’m not sure here. What’s the example you followed? It could be that what I pointed you to is what’s outdated.

Not at all, it’s what the forum is all about.

John

1 Like

I’m posting further partly for posterity, partly to provide extra details to anyone who wishes.

Changing to
imagej.init('net.imagej:imagej+net.imagej:imagej-legacy')
got me through all of my autoclass calls, but the memory issue really is a stickler. Other discussions of this issue online involve changing the allocated memory manually via Imagej.cfg, and then restarting the program. Getting acess to the config via python doesn’t sound impossible, but I’m under the assumption that the default would be used again the next time I called imagej.init anyway. Instead, I’ve chosen to try getting the local instalation to work, which is undergoing problems. If I get it working, it solves the plugin and memory issues right off the bat, since the plugins and memory allocation should be loaded directly from my local configuration.

I’ve followed the error prompt from that post to the _init_jvm_options() function from the __init__.py found in the imglyb folder of my virtual environment. I will investigate further when I have a moment. Given the format of the created directories mentioned in the linked post, I feel as though there’s a formatting bug hiding somewhere (perhaps hiding within a loop). As with many things, that could be a misguided thought, but we’ll see.

Unfortunately, Spyder and other IDEs like PyCharm only use the interpreter location of a conda environment but do not update the environment variables. This is why you have to set those variables manually inside an IDE. Hopefully, JAVA_HOME and PYJNIUS_JAR will be auto-detectable in the future.

To get more logging information it may be helpful to set the DEBUG environment variable:

This probably fails because jnius does not know about the location of pyjnius.jar (specified by PYJNIUS_JAR). The location of that jar is added to the classpath when scyjava is imported inside imagej.init

I would expect this to work but I am not a pyimagej expert. It could be an issue with inconsistent path separator handling (/ vs \). Can you try replacing \\ with / in the first argument of imagej.init? Just a wild guess but maybe it will do the trick. Maybe @ctrueden or @hadim have some ideas. Side note: Python has raw strings that do not require to escape backslash (\) but a raw string cannot end with a backslash.

Regarding your memory issue: I am not sure how ImageJ sets the maximum memory but I would assume that the launcher reads ImageJ.cfg and then sets the maximum memory (heap space) parameter of the Java virtual machine (this has to be done at launch time of the JVM). pyimagej does not use the ImageJ launcher and I do not know if the ImageJ.cfg is considered.

To set this option from within your Python script, add these lines before any call to imagej.init or import jnius:

import scyjava_config
scyjava_config.add_options('-Xmx<size>{m,g}')

For example,

import scyjava_config
scyjava_config.add_options('-Xmx10g')

would set the maximum heap space to 10GB. There may be a way to set it directly through pyimagej but I am not aware of it.

2 Likes

Fair enough. There’s one of my mysteries solved!

And another! Thank you!

Unfortunately that takes me back to This Problem. I’ve run into a lot of errors while troubleshooting all this, but that one seems relatively recent. I have to wonder if I didn’t introduce it myself at one point with a minor or even accidental change to something in imgej.py or __init__.py. More investigation may be needed on my end here.

That does in fact fix the memory issue! Thank you so much! I’m not sure I ever would have found that line on my own.

I will continue to investigate the remaining problem of being unable to use a local installation as my endpoint. Technically that memory fix could allow me to continue my work, but I’d rather not run on a hacked imagej.py if there’s a better solution.

2 Likes

That is correct.

Right. imagej.init(headless=False) will include only ImageJ2, not ImageJ1. To include ImageJ1 also, you could do one of these things instead:

  1. imagej.init('net.imagej:imagej+net.imagej:imagej-legacy', headless=False) – ImageJ2+ImageJ1 (no Fiji plugins)
  2. imagej.init('sc.fiji:fiji') – all of Fiji (but IJ1-style plugins won’t work)
  3. imagej.init('/path/to/Fiji.app') – Wrap a local Fiji installation (IJ1-style plugins should work)

See Including ImageJ 1.x support from the README, as well as Ways to initialize in the example Jupyter notebook.

Should not be necessary. Because you installed pyimagej in a conda environment, they should be autodetected even from Spyder. (The logic @hanslovsky linked was already merged and released.)

I recently did some work to fix this problem and others on Windows:

What version of scyjava are you using (conda list)? You need at least 0.2.3. What version of Java are you using? Java 8 is recommended, currently.

If it still doesn’t work, the problem is likely that jvm.dll cannot be found on the PATH. You could amend your PATH environment variable to point to the folder containing the jvm.dll of the Java you want to use. But you should not need to set any environment variables—if you still do, I’d like to fix that.

See also:

I’m not sure what you mean by “install the plugins I want via run commands”. Using sc.fiji:fiji bootstraps a Fiji by downloading and caching all the Java libraries (JAR files). It will not be aware of your local Fiji. In fact, in this scenario, there is no local plugins directory into which ImageJ1-style plugins could be installed.

If you want to include additional ImageJ2-style plugins, you can use the + syntax during init, assuming those plugins are also published to a Maven repository.

For more discussion and comparison of ImageJ1 versus ImageJ2 plugins, see the ImageJ2 paper [published version; or arxiv PDF), Compatibility section of Results and Discussion, including Figure S.3.

Anyway, the PIV plugin is an ImageJ1 plugin, unfortunately. So as of the writing, you must wrap a local Fiji installation if you want to use it. See this issue:

Would be great. Please file a PR at imagej/pyimagej. Thanks!

Yep. Remember that ImageJ1 was only designed to be an end-user desktop application launched via its launcher. The fact that you are able to launch the ImageJ GUI from Python at all goes beyond the scope of its intended use and testing. If you want to fully leverage the ImageJ world from Python, I strongly encourage you to move toward the ImageJ2, ImgLib2 and SciJava libraries, since they are designed to be fully modular, extensible, etc.

Fortunately, there is a way to change how much memory Java has available when launching it from Python:

>>> import scyjava_config
>>> scyjava_config.add_options('-Xmx6g')
>>> import imagej
i>>> ij = imagej.init()
log4j:WARN No appenders could be found for logger (org.bushe.swing.event.EventService).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
>>> print(ij.getApp().getInfo(True))
ImageJ 2.0.0-rc-71; Java 1.8.0_192 [x86_64]; 43MB of 5461MB

It does seem like the Java runtime ends up with significantly less memory than requested. But increasing the value does increase Java’s max heap in my tests:

$ python -c "import scyjava_config; scyjava_config.add_options('-Xmx8g'); import imagej; ij = imagej.init(); print(ij.getApp().getInfo(True))"
log4j:WARN No appenders could be found for logger (org.bushe.swing.event.EventService).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
ImageJ 2.0.0-rc-71; Java 1.8.0_192 [x86_64]; 41MB of 7282MB

I apologize that this information was not previously documented. I have now done so in the relevant places (scijava/scyjava@63aced9b, imagej/pyimagej@887a09ec, imagej/tutorials@9e95af76).

You will not be able to use Edit › Options › Memory & Threads… because that options dialog is an ImageJ1 plugin that has hardcoded assumptions about how Java was started. Instead, use the scyjava_config mechanism I outlined above, and you should be able to increase the RAM.

Sorry your experience has not been good. I encourage you to post here sooner when stuck, so that we can help.

Has he? I thought @hanslovsky only used the lower layers scyjava and imglyb?

Your approach is good. In particular, using conda is much less painful than pure pip, because conda can ship Java and Maven as dependencies. The pyimagej docs are up-to-date—but using ImageJ1 functionality from Python has some limitations. I’d recommend trying to use “pure ImageJ2” whenever possible instead, although I recognize that in many circumstances it’s not feasible due to needed plugins etc.

@hanslovsky is doing lower-level things. I would not recommend basing your stuff off that, unless you really need to dig into the nitty gritty layers.

Again: shouldn’t be needed anymore, thanks to the merged-and-released environment auto-detection support.

And now I jump over to the other thread at:

3 Likes

@ctrueden Huge thanks for taking the time to sift through all this. It was originally written in bulk so that others could follow my process, but I suppose I created a boatload of seperate issues for people to answer all at once. The option to alter the memory allocation that you and @hanslovsky have mentioned works great. I am running into another issue of memory not clearing after the closure of ImagePlus images. That seems to be a topic that has been discussed more than once on this forum so I’ll dig on that topic further before I ask any questions.

I’ve actually been using the plugin with sc.fiji:fiji as my endpoint, but I’m assuming that only works because of my hack and it is still loading the IJ1 content necessary to run the plugin.

Just tested this. Sure enough, it works just fine. I swear that wasn’t working the other day. Whatever issue I was hitting at the time must have been caused by something else. Sorry for the false alarm!

conda 4.7.12
java 8
scyjava 0.2.3

Unfortunately all attempts to use a local installation now lead to the error from my other post. That leads me to believe even further that this is an error that I have somehow introduced. Frustrating. I may do a clean installation of imagej.py and my virtual env.

something to the flavor of of run_macro('Install Plugin...',args). I vaguely remember seeing something like it while I was perusing different examples. Was just a passing thought really. I suppose my initial wording of that thought was posed more strongly than intended.

I learned that one the hard way after some experimentation. I’m afraid I don’t know all the implications of my endpoint hack, but is it at all worthwhile to include the option to load local plugins while using a Maven initialization? Purely a curiosity.

It seems to me that I need to spend some time looking more closely at how to do things in ImageJ2. My brain had all but neglected the idea that there was going to be a more up to date way to handle my image processing here.

No worries! Im just glad a solution exists!

Id be happy to, though as someone that has not worked with github and has not done any software development (in the professional sense), I’m afraid I wouldn’t know how to properly submit this code.

Despite the difficulty, I’ve actually had a really good time troubleshooting all this, and I’ve learned a lot about how pyimagej is implemented. All of you that work on ImageJ this diligently are absolutely awesome!

2 Likes

Hmm. Does this work for you?

import scyjava_config
scyjava_config.add_options('-Dplugins.dir=C:/Users/msout/Desktop/Fiji.app')
import imagej
ij = imagej.init('sc.fiji:fiji')

That will download and cache Fiji and dependencies, while still allowing you to point the ImageJ1 plugins directory to a local directory. Make sure your local Fiji.app already exists and is functional before doing this.

Awesome! :+1:

Ah, yeah, that endpoint hack you did could have ramifications. Not sure without scrutinizing it more closely. I’d suggest reverting to a standard installation before debugging further.

Probably won’t work in this scenario, but you could try. You almost certainly would need to point plugins.dir somewhere before something like that would work. And even then, it might make (untrue here) assumptions about the application directory structure.

:+1:

I’m afraid that ImageJ2 has a big learning curve. In particular, the ImgLib2 foundational library is much more powerful, but also more complex, than the ImageJ1 image data model. But there is an increasingly large wealth of functionality implemented using it, e.g. BigDataViewer, SciView, and ImageJ Ops.

If you decide to proceed with learning ImageJ2 programming, I heartily recommend starting with the ImageJ tutorial notebooks here:

Here is a detailed walkthrough for people in your situation:

It is focused on contributing to Java code, but many of the steps would be the same for Python code.

Alternately, GitHub now supports directly editing in the browser, so if you have already tested the changes locally, you can just browse to the relevant file on GitHub, click Edit, and paste in your new version. This will enable to you create a pull request (PR) directly via the web.

Glad to hear! :tada:

2 Likes