ImageJ Jupyter notebooks updated; ImageJ + Python; retiring the SciJava Jupyter Kernel

cellprofiler
imagej
python
tutorials
jupyter
scikit-image

#1

The ImageJ tutorial notebooks have now fully switched over to the stock BeakerX Groovy kernel. Previously the notebooks used a custom BeakerX-based kernel called the SciJava Jupyter Kernel (SJJK).

Newly added is an initial “ImageJ with Python Kernel” notebook illustrating how to use ImageJ from Python, so that it can be fully combined with other tools available from the Python software ecosystem, including NumPy, SciPy, scikit-image, CellProfiler, OpenCV, ITK and more.

To facilitate this update, I released ImageJ 2.0.0-rc-71 (and Fiji 2.0.0-pre-10) that includes support for images-inside-tables in the vanilla BeakerX Groovy kernel. This is the same code we wrote for the SJJK, but now migrated to imagej-notebook.

The current plan is now to retire the SciJava Jupyter Kernel. If you use this kernel and would prefer it not be retired, please respond here letting us know why not. In my view, the only thing SJJK can do that the Groovy kernel cannot is switch languages per cell, but I personally do not use this feature. It is still possible to mix languages in the Groovy (or other regular BeakerX) kernel by using the script service to run scripts in any supported SciJava script language. Do you know of something else that the SJJK can do that is still missing from plain Groovy notebooks? If so, please speak up, and we can discuss what to do about it.

One huge advantage of plain Groovy notebooks is the %classpath magic syntax to load whatever artifacts you want upfront, including a specific version of ImageJ itself, so that notebooks are fully reproducible and extensible. SJJK always suffered from the fact that it shipped a particular version of ImageJ, which quickly became outdated, and it was not possible to override this version in the notebook.

One downside of plain Groovy notebooks is that BeakerX always wants to handle rendering of List and Map with its TableDisplay logic. Unfortunately, SciJava Table extends List (I regret this design decision, and am considering changing it, although it would be disruptive to do so), and BeakerX’s built-in table rendering does not support invoking its Displayer extensions inside table cells. So we cannot register a Displayer to show the SciJava Table objects directly as cell outputs. That is why the updated ImageJ tutorial notebooks use ij.notebook().display([[...]]) around the [[...]] list-of-maps/table expressions.

The good news is: I have added an extensibility mechanism to imagej-notebook, such that if a Converter plugin exists that can go from some type Foo to a net.imagej.notebook.mime.MIMEObject (e.g., an HTMLObject), then Foo is automatically registered via BeakerX’s Displayer mechanism. So whenever a cell outputs a Foo, it will magically render as HTML by calling the appropriate converter.

Happy to elaborate on any of these details with more clarity if anyone is interested; just ask.


Need help to bring web interface to Fiji/ImageJ with ImJoy
Additional jars for scijava jupyter kernel
JS library based for interactive image analysis?
#2

If you think it would be better to not extend List here, let’s change it rather sooner than later, before more projects are picking up the API… (since we anyway only recently changed the package of the table classes from net.imagej.table to org.scijava.table).


#3

I agree that changing it sooner would be better. However, I have some urgent deadlines right now that mean I cannot work on it this week. Also, I’d like to discuss here a bit before we commit to changing the API. Here is a brainstorm of reasons for and against changing it. Thoughts and opinions welcome.

Reasons to make Table no longer extend List

  • The List interface brings in a large API, all of which should be tested. Implementing Table directly is somewhat involved—although less now since I maximized the use of default methods in the interface and introduced the Tables utility class as a helper.
  • Tables are 2D structures; implementing List<Column> means we are imposing a column-centric paradigm, which is awkward/inappropriate for some backing implementations, particularly row-centric ones. We could instead have table.columns() and table.rows() that offer List<Column> and List<Row> objects respectively. (And Column and Row would probably both extend the same base interface, since they are the same concept transposed.)
  • The List interface limits us to 2^{31}-1 elements. Maybe we want to enable tables to get bigger than that? We’d have to change to long indices everywhere, though.
  • The List interface imposes boundedness and support for random access. Probably we want all Table implementations to be bounded and randomly accessible, though, so I mention these here mainly for completeness.

Reasons not to change it

  • Some existing code in the wild would break.
  • The current API is tested and working fine as-is. Costs developer time to reengineer it.
  • Images-inside-lists-of-maps style tables would still not render as desired out of the box with BeakerX; need to explicitly wrap the list-of-maps to a SciJava Table first.
  • For things larger than 2^{31}-1 elements, we already have ImgLib2.

#4

Thanks @ctrueden, this is looking great! I didn’t realize how extensive the plotting options were in the BeakerX Groovy kernel.

Plotting has been one of the major missing pieces for us in our workflows. We usually save final results into tables and then plot elsewhere, but now we could do everything in Groovy jupyter notebooks. I also noticed columns of scijava tables can be directly given as x and y inputs in plots and it just works, not sure if changing from List would complicate this.

I would think having tables with more than 2^31 elements is a rare use case and there are better data structures for that (ImgLib2 etc…). We have worked a lot with large tables (5 GBs etc) in the past and recently switched to a different data model that makes multithreading easier. But I understand it’s easier to change sooner rather than later. We use the scijava tables a lot in our code, but it sounds like the change wouldn’t be that disruptive. I guess it just depends on how you will implement it.

I had a look at the “ImageJ with Python Kernel” but I didn’t see any examples with Tables there. I am curious, is there a way to quickly convert from a scijava table to a pandas dataframe?

Also, if you initialize using your local Fiji instance can you use locally installed plugins in Python as well? Do you have any suggestions when setting this up?


#5

Sorry for the long delay in reply, @karlduderstadt.

Yeah, it is pretty great. :smile:

Good to know. I am leaning toward not changing the table API anymore, since it is increasingly used in production already and I don’t want to break everyone’s code (yet again).

Great question. No one has worked on this yet. We would need to extend pyimagej to support Pandas<->SJ tables. I filed imagej/pyimagej#27 to track that idea, although I do not have time to work on it at the moment. If you implement anything along those lines, PRs are very welcome!

Yes, wrapping a local installation gives you full access to the installed plugins, with the following caveats:

  1. If you start ImageJ in headless mode from Python, the plugins you want to use will need to work headless. Not all plugins do—especially many ImageJ 1.x plugins.

  2. I have observed mixed success running ImageJ 1.x plugins from a local installation in general. The Python code does try to set the plugins.dir system property needed by ImageJ1, but on some people’s systems it does not seem to be working. See imagej/pyimagej#22 for further details and findings on that.

But definitely give it a try, and report back here with any problems or questions!


#6

The BeakerX kernel also allows switching languages per cell using some magic commands (as highlighted by @volker in this recent topic):

[1] Groovy:

a = 0..10

beakerx.a = a

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

[2] Python

%%python
from beakerx.object import beakerx
b = (i**2 for i in beakerx.a if i % 2 == 0)

beakerx.b = list(b)

[3] Groovy

beakerx.b.each { println it / 4}
0
1
4
9
16
25

[0, 4, 16, 36, 64, 100]

#7

Yes, it’s very cool. However, important to note that BeakerX is marshaling/unmarshaling the data between languages using JSON. When you write %%python in a BeakerX cell, you’re using CPython, not Jython, for better and for worse. This differs from SJJK, where a “Python” cell is really Jython, so the variables really point to the same objects on the JVM regardless of a cell’s language.

In practice, this means there are limitations on the sorts of data that can be passed between Groovy and Python. Primitives and collections work OK (in my limited experimentation). But Java objects will cause strange errors. E.g., in Groovy, do:

beakerx.ij = ij

And then in the next cell, try:

%%python
from beakerx.object import beakerx
beakerx.ij

And you’ll see this illuminating stack trace:

---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
<ipython-input-4-3e2c306c9d55> in <module>
      1 from beakerx.object import beakerx
----> 2 beakerx.ij

/usr/local/miniconda3/envs/scijava/lib/python3.7/site-packages/beakerx/runtime.py in __getattr__(self, name)
    681         if '_server' == name:
    682             return self.__dict__['_server']
--> 683         return self.get(name)
    684 
    685     def __contains__(self, name):

/usr/local/miniconda3/envs/scijava/lib/python3.7/site-packages/beakerx/runtime.py in get(self, var)
    511         if result == 'undefined':
    512             raise NameError('name \'' + var + '\' is not defined on the beakerx object')
--> 513         return transformBack(json.loads(result))
    514 
    515     def set_session(self, id):

/usr/local/miniconda3/envs/scijava/lib/python3.7/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    346             parse_int is None and parse_float is None and
    347             parse_constant is None and object_pairs_hook is None and not kw):
--> 348         return _default_decoder.decode(s)
    349     if cls is None:
    350         cls = JSONDecoder

/usr/local/miniconda3/envs/scijava/lib/python3.7/json/decoder.py in decode(self, s, _w)
    335 
    336         """
--> 337         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338         end = _w(s, end).end()
    339         if end != len(s):

/usr/local/miniconda3/envs/scijava/lib/python3.7/json/decoder.py in raw_decode(self, s, idx)
    353             obj, end = self.scan_once(s, idx)
    354         except StopIteration as err:
--> 355             raise JSONDecodeError("Expecting value", s, err.value) from None
    356         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Fortunately, if what you want is to write Jython code, you can also still do that in a Groovy cell by writing ij.script().run('script.py', myJythonScript, args).get().


#8

Thanks @ctrueden for your feedback. I think the addition of conversion from SciJava tables to pandas will be really powerful and I am excited to see the progress made by @hadim on github. I am eager to test it out, but unfortunately I can’t get pyimagej running.

After several failed attempts I decided to start from scratch. I installed a fresh miniconda, and then created the scijava environment exactly as described in the readme (https://github.com/imagej/tutorials). Then I activated the scijava environment. (to get this to work I had to add tornado<6 to the environment.yml, otherwise I was getting the AttributeError: module ‘tornado.web’ has no attribute ‘asynchronous’ error reported before). This finishes with no errors.

All the groovy notebooks work perfectly. In fact, they have been a huge benefit for us and we use them extensively with local plugins from our Fiji.app. They have also worked with several different installations of mini and full conda.

However, when I start the ImageJ with Python Notebook in the scijava environment, it fails saying it requires Java SE 6.

I have java 8 installed and need to use that version. Also, I don’t get an error in the other notebooks, so this seems rather strange to me.

Any tips anyone might have to fix this problem are very welcome because I think this java to python bridge is a great project and want to benefit from it.


#9

This is a nasty issue on macOS that I do not yet know how to solve properly. The issue is documented here:

As a workaround, you can edit the Info.plist of any one of your Java installations to include <string>JNI</string> in the capabilities, as described at kivy/pyjnius#277 (comment):

  1. If you don’t already have a system-level Java installed, install OpenJDK 8 from AdoptOpenJDK.
  2. Edit /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Info.plist as administrator (e.g., you can use sudo nano /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Info.plist from the Terminal).
  3. Find the part that reads:
    	<key>JavaVM</key>
    	<dict>
    		<key>JVMCapabilities</key>
    		<array>
    			<string>CommandLine</string>
    		</array>
    
    And add <string>JNI</string> on the line after <string>CommandLine</string>.
  4. Save and quit.
  5. Try the Python program again (if in Jupyter, restart the kernel). It will still use your conda installation’s Java—not the Java you hacked—but this change will bypass macOS’s interception of attempts to link to libjvm.dylib and presenting this obnoxious dialog box.

See also:

Where this same issue is being discussed from the perspective of end-user Fiji installations.


#10

That worked! This is like magic going from java to python. Thanks @ctrueden.