Help wanted to understand the service discovery model in scijava

Hi @ctrueden, @imagejan, all,

I am unable to make head or tails of code like this:

	@Parameter
	private LanguageSupportService languageSupportService;

From here the Script Editor github repository:

Question 1: where is the actual languageSupportService declared?

Question 2: how does one find out what properties the languageSupportService has?

Question 3: how does one register an additional service for e.g. jython, so that the language suport service engine can find it?

Question 4: how and where does one package it?

I’ve pointed out in the past that this style of programming is opaque to developers other than who authored the code, because it is not discoverable via IDEs, and it is not discoverable, or not easily, at run time via reflection or code introspection.

Thanks,

Albert

2 Likes

At scijava/scripting-jython, the jython language is defined rather simply, and it points to the AdaptedScriptLanguage:

… using a rather simple constructor call:

public JythonScriptLanguage() {
		super("jython");
	}

And I’ve found the AdaptedScriptLanguage (the superclass) after some guessing here in scijiava/scijava-common:

… with the constructor then calling findFactory("jython"). That then relies on a ScriptEngineManager, so the can has been kicked further down the abstraction road. Now where is that. Definitely not here, surprisingly:

The fragmentation of scijava into a myriad of tiny repositories makes it nearly impossible to navigate, except, I guess, if one git checkout all scijava repositories into one’s workspace folder in e.g. eclipse. Or is there another way?

So the ScriptEngineManager is not in scijava-common. Bummer. And the obviously named scijava-scripts is not about scripting in scijava, but rather, bash scripts than run in the command line. If not that, then which of these scijava repositories?

By the look of it, none of them.

Of course, I didn’t realize that the ScriptEngineManager is javax.script.ScriptEngineManager. So I’ve reached a dead end, because that is just a JVM built-in. Still no hint as to where the jython ScriptEngine may be.

I want to extend the jython ScriptEngine. It really shouldn’t be this hard to find.

So I tried a brute search approach:

Fiji.app$ find -name "*jar" -exec grep -Hsli ScriptEngine {} \;
./jars/groovy-3.0.4.jar
./jars/scijava-common-2.83.3.jar
./jars/jruby-core-9.1.17.0.jar
./jars/scripting-clojure-0.1.6.jar
./jars/bsh-2.0b6.jar
./jars/scripting-java-0.4.1.jar
./jars/jython-slim-2.7.2.jar
./jars/languagesupport-3.1.0.jar
./jars/renjin-script-engine-0.8.1906.jar
./jars/scripting-groovy-0.3.0.jar
./jars/scripting-beanshell-0.3.4.jar
./jars/scala-compiler-2.12.1.jar
./jars/scripting-scala-0.2.2.jar
./java/linux-amd64/jdk1.8.0_172/jre/lib/ext/nashorn.jar
./java/linux-amd64/jdk1.8.0_172/jre/lib/ext/jfxrt.jar
./java/linux-amd64/jdk1.8.0_172/jre/lib/rt.jar
./plugins/jars/Jython.jar

… and it turns out that, perhaps, it’s provided by jython-slim-2.7.2.jar itself. Seems the most plausible.

So to update scijava’s ScriptEngine for jython, I have to do so … in the jython repository itself. Perhaps that makes sense, but this means I can’t provide additional features to that engine without commiting code to the upstream jython project. That’s too much to ask, given that the code I want to add is Fiji-specific.

I’m going to take this finding as a license to implement an ad-hoc solution for jython in the Script Editor.

Turns out, there may still be a way to provide custom autocompletion code for jython in the Script Editor: the RSyntaxArea language support accepts plugins (for documentation: the RSyntaxArea is an extended JEditorPane that does syntax highlighting and much more, and where text is written in the Script Editor). There are two such plugins in there, implemented for java and for javascript, in the Script Editor repository:

Let’s see how far I can go that way. Well, not too far: I’ve created a class for JythonLanguageSupportPlugin within the script editor repository, which looks like:

@Plugin(type = LanguageSupportPlugin.class)
public class JythonLanguageSupportPlugin extends  AbstractLanguageSupport implements LanguageSupportPlugin
{
	
	private JythonAutoCompletionProxy acp;
	private RSyntaxTextArea text_area;
	
	public JythonLanguageSupportPlugin() {
		setAutoCompleteEnabled(true);
		setShowDescWindow(true);
	}

	@Override
	public String getLanguageName() {
		return "jython";
	}

	@Override
	public void install(final RSyntaxTextArea textArea) {
		this.text_area = textArea;
		this.acp = new JythonAutoCompletionProxy(new JythonAutocompletionProvider(text_area, new JythonImportFormat()));
	}

	@Override
	public void uninstall(final RSyntaxTextArea textArea) {
		if (textArea == this.text_area)
			this.acp.uninstall();
	}

}

… so, in the EditorPane.setLanguage, I’d expect the support to not be null for “jython”. but it is.

Question: what else, beyond the @PlugIn parameter of the class, is needed for this to work?

@ctrueden @hinerm @imagejan any answers to the above would be very appreciated.

OK I found one answer: “python” is the language name, not “jython”. Now support isn’t null. Progress!

1 Like

After all these unnecessary hours, the LanguageSupportService has bent to my long search across undocumented features, undiscoverable injected magic and disparate repositories. So now there is autocompletion support for python (jython) in the Fiji Script Editor. Limited, though, to importing java classes–it’s a start. And it can, and hopefully will, get much better.

Have you read the “SciJava in Detail” notebook?

I acknowledge your opinion. But the entire design of SciJava and ImageJ2 is built around type-safe extensibility via plugins. Changing this design would eliminate ImageJ’s primary advantage.

org.scijava.ui.swing.script.DefaultLanguageSupportService

In general: In Eclipse, ctrl+T lists the available implementations of an interface. Which one is used will depend on the runtime environment.

The properties (methods, really) are those defined in the interface. You should not rely on any properties defined in specific implementation, as they are intended to be internal only.

The LanguageSupportService is the service that manages available LanguageSupportPlugins. You should not need to implement your own LanguageSupportService (although the SciJava design allows it, if you need to override the infrastructure of how LanguageSupportPlugins are managed). Rather, if you are looking to support a new language in the script editor, you’d make a class implementing LanguageSupportPlugin and annotated with @Plugin(type = LanguageSupportPlugin.class).

Here is an example for JavaScript. Note that it is just a thin extension of the RSyntaxTextArea project’s JavaScriptLanguageSupport class.

Assuming you imported a project of interest into Eclipse via the Maven support (File › Import › Existing Maven Project), Eclipse should allow you to browse into the source code of classes residing within dependencies, via downloaded-on-demand -sources JARs. Just use the usual F3 mechanism or whatnot. This works especially well if you import a toplevel project such as Fiji itself.

That ScriptEngine implementation is part of Jython itself:

Here is how I found it using Eclipse:

  1. Import Fiji via File › Import › Existing Maven Project:


Wait for the project to fully import.

  1. Ctrl+Shift+T for Open Type; enter ScriptEngine:

  1. Ctrl+T for Type Hierarchy; look for Jython/Python in the list:

I’m not sure how helpful it will be to dig into the Jython ScriptEngine implementation, though. Earlier you mentioned LanguageSupportPlugin… that is not the script engine, but rather for RSyntaxTextArea’s language support.

What are you trying to achieve? I missed your last couple of messages above—I understand now what you were trying to do. Congratulations on getting it working.

1 Like

Thanks for the detailed tutorial, it helps.

Yes, the entire advantage of scijava over ImageJ1 disappears. What was lost, though, is the ability of the non-expert to contribute, given the nearly insurmountable amount of a priori knowledge on tooling necessary to waddle through the interface forests and runtime service discovery systems.

Indeed, I got somewhere, so it wasn’t all wasted. And indeed I use F3 lots in eclipse, but interfaces have a way of getting one nowhere. Didn’t occur to me to use the type hierarchy (via control + T or else), as I was under the impression that the classes wouldn’t be found since they weren’t in my imported projects. Will try to remember this going forward, thanks for making me realize that mvn does a lot more than I give it credit for.

Now I have to finish the Jython autocompletion system, yet most of my hours were committed to even finding out where to add the code that I already had working. This too shall pass, until I forget it all and have to discover it again. At least now there is this detailed tutorial. Whoever implements the next autocompletion system for the Fiji Script Editor, I hope all the above works as the necessary introduction.

On the tutorial on scijava, as a general explanation of what’s going on it is informative. To develop plugins on top of the ImageJ2 software stack, it reads like it should help (but I haven’t, and probably won’t). To develop the stack itself, it falls very short.

From my perspective, SCIFIO continues to endure growing pains in the form of performance issues (I understand the ImageJ2 project is understaffed). Image Ops are a complication layer, only simplifying code for those that already know how it all works underneath; reminds me of the Dylan programming language, which was at its most useful if one already knew lisp and understood the transformations that it made to simplify some obtuse lisp constructs. And scijava I find often impenetrable, a fragmentation of what once was a monolithic project (fiji) into a myriad of libraries that facilitate independent development at the cost of preventing outside developers from contributing unless they internalize the overall structure as if it still was the monolithic architecture that it used to be. Perhaps it suffers from second-system symptoms, but the good part is that it works and services thousands around the world. It’s just so hard to do anything with it as an outsider.

In summary, I am happy it all works, and thankful that you maintain it. Let’s hope some day it will actually be the easy entry-level hacking platform that would attract those entering the project from tangential disciplines through sheer need to get an as yet non-existing feature built.

1 Like

Hey @albertcardona,

what often helps me in the SciJava jungle is the debugger of the IDE. Write a little main function that opens a script editor and set a break point for example here. You can then inspect what script languages exist during runtime. And if you know the package and classname of a specific language, you can search for it in Fijis search bar to figure out its jar filename. Hopefully you can make a good guess from the jar name to the github repository name.

Btw. I was also not happy that maven, compilation of Fiji plugins and writing them is a lot more complicated than compiling a Java plugin with ImageJ1. That’s why I made a Fiji plugin code generator for clij. As you are my target audience for this, I would love to hear your feedback. Any hint that makes it even easier for people to contribute new plugins is very welcome :slightly_smiling_face:

Cheers,
Robert

1 Like

That’s cool @haesleinhuepf! A modern version of Kai Uwe Barthel’s visual programming framework for ImageJ.

On breakpoints: thanks, indeed, I should use the eclipse debugger more often. But the point stands about lack of discoverability–if it needs introspection inside a debugger, an API isn’t developer-friendly.

1 Like

I 100%ly agree. :sun_with_face:

Note if you have seen the end of the video: From the visual graph you can generate maven-based Java code and compile it to Fiji plugins from within Fiji :wink: