Does ImageJ Utilize the GPU?

imagej
gpu

#1

Hi all,

Does ImageJ/Fiji or any macros use the GPU? I’m planning on buying the new Thinkpad X1 Carbon and hooking up an external GPU set up, but was wondering if this would improve the efficiency of ImageJ/Fiji.

Thanks!


#2

Hello James -

Let me give you my rather uninformed understanding. (I hope
any experts will correct me if I’m wrong.)

I believe that ImageJ does not use GPUs.

ImageJ is written in java, and standard java does not take
advantage of GPUs.

There would be two ways to do this. The ImageJ java code could
call through the “java native interface” (JNI) to non-java modules
that then use the GPUs for various processing tasks. I have not
seen any JNI code in the ImageJ code base, so I don’t think ImageJ
uses GPUs with this mechanism.

Java code runs on a “java virtual machine” (JVM). If the JVM
uses the GPU (without there being any special GPU directives
in the java code), then a java program (e.g., ImageJ) would
get the benefits of the GPU automatically. I am quite certain
that mainstream JVMs (e.g., openjdk’s JVM) don’t use GPUs.
(Fiji ships with openjdk.) I’m not aware of any full-featured
JVMs that use GPUs but there are experimental efforts along
these lines.

If you had a standards-compliant JVM that used GPUs, then
ImageJ ought to be able to run on it (since it is compliant), and
should then get whatever benefits of the GPU that particular
JVM supports. But I’m not aware of any such JVMs.

Thanks, mm


#3

Hi @mountain_man

That’s a pretty good summary. I’d also like to point out that in addition to JNI, JavaCpp is a promising technology. JavaCpp comes with many presets (existing wrappers) including one for cuda. DL4J is also based on JavaCpp. It seems multiple people are looking at using JavaCpp related technologies, for example see these forum posts on DL4J and OpenCV scripting.

GPU code is usually written with “native” code (ie c/c++), and it is a bit more involved to build, test and distribute. If you google “imagej gpu” you’ll find examples of existing plugins that use GPU.

In addition to distributing the plugin (and related native libraries) there is a second issue, which is that imagej and ImgLib2 structures (ImagePlus, Img, etc.) need to be “converted” to a structure (such as JavaCpp Pointers) that can be used by native/GPU code.

I have written some experimental code that converts IterableInterval to FloatPointer. This works for my own use cases, but eventually there may be formal converters written in the style of imagej-matlab or imagej-itk. The advantage of formal converters is that script writers can simply request a specific structure in their script, and the imagej framework will automatically convert it. For example see this little script which asks for a SimpleITK structure

# @org.itk.simple.Image image
# @OUTPUT Dataset output

from org.itk.simple import SmoothingRecursiveGaussianImageFilter

itkGauss = SmoothingRecursiveGaussianImageFilter();

# call itk rl using simple itk wrapper
output = itkGauss.execute(image, 3.0, False);

So to summarize

  1. There are currently many examples of ImageJ plugins that use GPU, however they require some expertise and work to setup, build, distribute and use.
  2. In the future this process will be getting easier. There are oppurtunities for anyone interested in GPU and/or native libraries to contribute to the underlying framework to support this.

Brian


#4

I will chime in to mention that with the MIST plugin we went with the JNI approach which unfortunately ties you to a specific version of the cuda libraries.

It also makes the plugin much harder to distribute and configure easily.

Soapbox moment: For anyone adding gpu compute support, always provide a fallback method for when the gpu cannot be detected. For MIST we have a GPU version, a native FFTW version (JNI again), and a plain Java version so that if either gpu or JNI FFTW fails the user can still use the plugin. The drawback is MIST is by far the most difficult of the plugins I have released to manage and distribute.

Michael


#5

Good points, and this is one of the things image-ops is designed to deal with. You can write multiple implementations of the same Op interface, and by carefully setting the priority and implementing the Contigent interface, you can have your op check for the GPU and/or existance of native library and fall back to a java implementation if needed.