Converting data type on GPU with Java methods (via pyimagej)?

I am starting to learn how to use CLIJ, and I am super excited about it! I am using python and connecting to CLIJx via pyimagej, so I am using the Java-specific methods from CLIJx.

I am able to get a (16-bit) image onto my GPU successfully and do some operations on it. However, I would like to convert this image to 32-bit, ideally on the GPU to avoid data transfer. I see the documentation on the ImageJ macro extensions for doing this (convertFloat, convertUInt8 etc), but I don’t think that there is a dedicated method accessible from the Java package to do this.

In general I have been struggling with interpolating between the reference instructions on the web, which are specific to the ImageJ macro language commands, and the actual methods available to me from the Java source code (which I think is what I am ultimately accessing via pyimagej). They are not the same, although there is some consistency in the nomenclature differences for the most part.

If I am using the java functions available in net.haesleinhuepf.clijx.CLIJx, how would I convert data types on the GPU?

Minimal example (I think the error is pretty unambiguous, but just so you have a concrete idea what I’m doing)

import skimage.io as io
import imagej
#I am using my local Fiji, I just updated from the clij2 update site today.
#Let me know if I should report any versioning for clij2, I'm not sure how!
ij = imagej.init('/Applications/Fiji.app')
from jnius import autoclass

im = io.imread(path_to_some_image)
CLIJx = autoclass('net.haesleinhuepf.clijx.CLIJx')
clijx = CLIJx.getInstance()

im_ij = ij.py.to_java(im)
im_on_gpu = clijx.push(im_ij)
im_float = clijx.create(im_on_gpu.getDimensions(), clijx.Float)
#This is where the error happens:
clijx.convertFloat(im_on_gpu, im_float)

AttributeError: 'net.haesleinhuepf.clijx.CLIJx' object has no attribute 'convertFloat'

I have also tried net.haesleinhuepf.clij2.CLIJ2 and even the CLIJ version of that as well, but none of them have an appropriate conversion method.

For now I can try to make it work doing the data type conversion prior to loading onto the GPU. Perhaps this is what I should be doing anyway?

Thanks very much!

1 Like

Hey @akennard,

glad to see clij in python in action.

A very good question! The convert methods don’t exit on Java side because every clij method has conversion built-in. Thus, you can call clijx.addImages(a,b,c) with three images of different types and clij will manage. I’ll add a comment to the documentation.

If you want to call “just” a conversion, in your code that should do it:

clijx.copy(im_on_gpu, im_float)

Let me know if it works!

Cheers,
Robert

1 Like

Thanks for the quick reply! This is good to know. I tried your suggestion, and it seemed to work!

It did yield behavior that was unexpected to me: just pushing a uint16 image to the GPU and pulling it back caused a datatype conversion to float:

im_ij = ij.py.to_java(im) # im.dtype --> uint16
im_in = clijx.push(im_ij)
im_out_ij = clijx.pull(im_in)
im_out_np = ij.py.rai_to_numpy(im_out_ij) #im_out_np.dtype --> float64

This seems reasonable enough, just something for me be aware of! In this case it seems the copy command might not even be necessary due to default behavior.

1 Like

Interesting! Can you please print out im_in? What type does it have? What about im_out_ij? Curious!

This got interesting!

>>> print(im_in)
<net.haesleinhuepf.clij.clearcl.ClearCLBuffer at 0x119a6d410 jclass=net/haesleinhuepf/clij/clearcl/ClearCLBuffer jself=<LocalRef obj=0x1040a8980 at 0x1121e55f0>>

but puzzlingly:

>>> im_in.getPixelSizeInBytes() 
2

which seems like it is still 16-bit on the GPU.

Even more puzzlingly,

print(im_out_ij) returns

>>> print(im_out_ij)
<ij.ImagePlus at 0x1196cb620 jclass=ij/ImagePlus jself=<LocalRef obj=0x1040a8938 at 0x11235cfb0>>

>>> im_out_ij.bytesPerPixel 
2

So even the Java object is 16-bit! So perhaps the ij.py.rai_to_numpy helper function is what is actually doing the conversion? I am not at all comfortable with the pyimagej package yet, so not sure how to diagnose or try an alternative…

(I also confirmed that if I do the clijx.copy command you suggested and pull it out into a Java object, it is a 32-bit float as expected!)

The rai_to_numpy metho lives here. It generates a numpy image and copies over content. The new-image generator does this:

I’m not soo familiar with python. I would conclcude that self.dtype fails but I’m also not sure how to deal with that.

Just a side-note: I had issues earlier with push/pull performance. I did some experiments in this notebook. At the end it appears not very reasonable to copy a numpy-array to the java virtual machine, from where it’s copied back to GPU memory. It’s a very expensive detour time-wise. We’re working on a direct implementation of clij in python called pyclesperanto. It doesn’t have full functionality yet, but I wanted to at least mention it. Stay tuned!

1 Like

Thanks! I will look forward to pyclesperanto :slight_smile: There were a few tasks— like getting an arbitrary quantile (rather than a max or median) of an images’ grayscale values—that I knew exactly how to do in python/MATLAB but not in IJ macro—and I don’t know Java! So I was excited to see an option for combining GPU performance of CLIJ with the many operations that I know how to do in numpy. It’s really cool to see so much development on this front in real time, and I’ll be sure to keep in mind that things are changing fast and the most optimal approach may not be out yet.

1 Like

Great idea! I’ve put in on the wishlist for the next release.

It’s always good to hear what people would like to do with clij :slightly_smiling_face: