Deploy & run CPython algorithms within ImageJ

imagej
python

#1

Hello beloved & ambitious ImageJ developers,

I want to call some CPython code from within my ImageJ plugin. (Labkit)
How could this be done? The python code uses numpy and PyTorch, and is therefore incompatible with Jython. The solution should allow to be deployed via ImageJ update site.

I know this task can be solved: I could bundle the python code as self containing, platform dependend, binary using pyinstaller. The binary can easilly be started from Java. And data could be transferred, using files, stdio, or networking.

There are many solutions to bridge between ImageJ & Python:
ImgLyb, ImageJ.py, PyJNIus, CellProfiler-javabrigde, SCIJAVA scripting-python, JPY, JEP, Py4J

Additionally one could use less java-python related techniques like:
ImageJ-Server, ZeroMQ, REST, RCP

What would be a nice solution to run CPython from within ImageJ?

@hanslovsky What do you think? Can ImgLyb be used to run python from java?

@ctrueden Are there already plans to solve this task? Can I contribute?


#2

imglyb code cannot be called from Java. This would only work if you also start your application from within the same Python process. Think about doing this only if shared memory between numpy arrays and imglib2 data structures is critical to your performance (my guess is that it is probably not).

Do I understand correctly, that your Python code does not really have to touch Java objects and vice versa? Then, I would probably go one of these ways:

  1. Spin up a python interpreter every time you need to run the python code, and have it execute your script with the appropriate parameters (e.g. path to relevant files). This approach is probably the easier of the two, and might be worth following for a quick first prototype.
  2. Start a Python server that waits for requests, e.g. ZeroMQ or HTTP. This has the advantage that you do not have to spin up a new python interpreter every time you want to run python code but comes at the cost of higher complexity.

#3

Imglyb would have the great benefit, that image transfer between Python & ImageJ is already implemented. But is it already stable? What are your experiences, where are the challanges? One could install an additional ImageJ+Python starter (based on ImageJ.py), which enables the CPython functionalities in ImageJ.


#4

Yes, it is pretty stable (and I also do not plan to add/change any featureus momentarily). The user API has not changed in a long time, and if you look at the history, most recent changes are bug fixes or changes to documentation.

There is two major challenges that I see:

  1. In order to be able to wrap imglib2 data structures as numpy arrays without copies, they have to be ArrayImg backed by an unsafe access. This is most likely not the case for your data, as I expect cached CellImg and ImagePlus wrapped as RandomAccessibleInterval to be your typical data sources. You can, of course, make a crop-out of the relevant data into an appropriate ArrayImg, and then wrap that as a numpy array, which would add one copy but circumvent file IO/inter-process communication.
  2. You will need to create an entry point in labkit with an interface that you then implement in PyJNIus on the python side. This is definitely possible but can be hard to debug. Assuming that you would like to classify pixels, the interface could be something like
interface PyTorchCallback {
    <T> RandomAccessibleInterval<UnsignedLongType> classify(RandomAccessible<T> data, Interval targetROI, aditional parameters...);
}

You would then implement that interface in Python and pass it to labkit at an appropriate entry-point.