How to promote scripting over macros

fiji
imagej
macro
scripting
#1

…commenting on ctrueden’s rant:

This is completely true - and similar to what Robert earlier commented. …Here’s some thoughts on what might be barriers to that happening IMHO. …strictly personal thoughts/opinions.

I’m a programmer - have been professionally for 25 years. …But (even!) I’m writing stuff using the macro language.

It is completely overstretching what should be done with it (2000 line macro’s!)

This is Bad. It’s bad for me (honestly the limitations/bugs/issues with the macro language are frustrating as hell), it’s bad for the apps I’m writing and anyone later trying to maintain this.

But I’m writing it in the macro language mainly because when coming into IJ as a newbie it seemed to have the best (only?) reasonable documentation/examples etc. I didn’t even realise for a month or two that I could write things in javascript or python.

If I’d know that from the start, and those interfaces had half decent documentation/examples etc I would have done that. For me now it’s too late.

So my thoughts on this is having some better/good documentation and visibility around the alternatives to writing in the macro language would go a long, long, long way to helping nudge people in that direction at the start, rather than using the macro language. I think that would do wonders then for really helping out the IJ community in general.

…and pick one language to be the default, not the current smorgasbord of so many. I think that is too hard to maintain, support and do a good job of, simply due to the dilution of resources.

6 Likes

Macro call function defined in another file?
#2

Why? Wouldn’t it actually be easier to migrate to other languages now (for new scripts), since you know the APIs and paradigm better than when you started?

Sure. One thing that would help with this is a new script recorder / history tool that generates blocks of script code from recently executed commands. This is planned, but we are prioritizing a new Updater first. (CC @frauzufall @turekg @tpietzsch)

Recently discussed here:

Prime candidates for that would be Groovy and Python. Do you have a preference?

1 Like

#3

It might be, but for me the issue is I don’t know. What I mean by I don’t know, is I haven’t see any info, docs or examples for how to write using another language. So I don’t know if the API is the same, or paradigm is the same.

So the issue is - while you could very well be right - I don’t know that. So for me it’s safer/faster to just go with what I already know, which is the macro language, because I’m doing this for work, so speed matters.

What I think matters here generally - not for me - is that I’m probably not in the same boat with that “I don’t know” feeling, because of that lack of documentation/guides etc.

It would obviously be really good to try and get a feel for the issues facing all script/plugin writers, not just me. Are my issues unique or representative?

I use the script recorder a lot, and didn’t realise until recently it would “record” in other languages, not just the macro language.

It is very good for helping with figuring out how to do something when you in the UI already, but isn’t much help when you’re a) don’t know if/how to do it in the UI or b) looking at a problem that doesn’t fit into an existing UI tool

For me it would be javascript then python. But that’s only me - it would be good to know what’s the experience and knowledge of the entire installed base.

Wrt to groovy, I don’t know it at all, but I’m wondering if it would be better to focus on languages that have good general community adoption and documentation (ie javascript, python, java etc) as this would increase the general level of pure language documentation/help/examples that people could draw on when writing things.

Above all of course my experiences and feelings only :smiley:

0 Likes

#4

Hey guys,

I think a good way for promoting scripting languages over macro is providing an API for scripting which is as simple or even simpler as macro. Macro-coders must be made greedy in trying out the new API, because it’s better than just macro :wink:

Just as an example: Macro programmers are used to concepts such as showing the user a window using the selectWindow(imageTitle) command. A single line; self-explanatory.
Now assume you are coding ImageJ2. You have the original image as Dataset (or Img) variable and you want to bring its corresponding window to the front. How is the ImageJ2 command for this?
Another example: Stack.setSlice(5); extremely simple, and again self-explanatory. How might that be done in ImageJ2? I’m not even sure where to look this up. Auto-completion might help though…

I’m afraid it will be hard to promote scripting languages in general as long as these simple macro-API calls have no counter part in the ImageJ2-API…

Cheers,
Robert

4 Likes

#5

Let’s not mix up concerns here. This discussion is about Macro language vs other scripting languages, not ImageJ 1.x vs ImageJ2.

In Groovy, you can just as well write:

image = IJ.getImage()

Isn’t that simple enough?

You can even write:

image1 = IJ.openImage("/path/to/image/One.tif")
image2 = IJ.openImage("/path/to/image/Two.tif")

results = []

[image1, image2].each { image ->
  statistics = image.getStatistics()
  results << statistics.mean
}

println results

(And concerning ImageJ2, the situations where it gets really complicated is mostly when you have to dive into #imglib2 usage and more. ImageJ Ops and the SciJava services are trying to achieve exactly that easy-to-read, concise syntax. But, as I said, ImageJ2 is a different topic…)

4 Likes

#6

Fully agree. Just one comment:

For somebody with many years experience in macro coding, such a code snippet might not be readable at all. Thus, switching language is hard. Allowing smooth transitions might be helpful as well when it comes to promoting scripting languages…

3 Likes

#7

Oh and there are curly brackets in jython😲 Didn’t know that they exist.

0 Likes

#8

Agreed. With power comes complexity. Think of list comprehensions in Python, or closures/lambdas. That is something that long-time macro coders will not know (and might even not need).

Yes, agreed as well. Any help (by anybody in the community) in improving the current state of documentation is welcome. (The new ImageJ tutorials using Groovy in BeakerX Jupyter notebooks are a step in the right direction, IMHO. And workshops like the scripting workshop at the last #i2k conference.)

2 Likes

#9

I guess this is a 6 of one, half a dozen of another problem.

It is hard to learn a new language. Then if you compound that by saying also learn an API and methodology as well (as you’re trying to learn the language to drive IJ to do something) …well then it’s 2-3x harder problem.

So one solution is to support being about to write “macros” (I’m using that term in a general sense) in a bunch of languages, so people can at least avoid the initial learn a new language part.

…the problem that I see then is that you’ve got to write the documentation/tutorials/examples etc in all those languages in order to avoid the problem now where there is a scarcity of documentation.

So tough call on what’s best to do.

and FYI for me at least (never used groovy at all) is that code snippet would looks like a weird offspring of C++ and javascript smooshed together that would take me 5 min to be exactly certain of what it’s doing lol).

1 Like

#10

Great discussion. Lately I keep ranting about macros to every programmer I meet, but I’m not sure what or how I could use otherwise.

Prime candidates for that would be Groovy and Python. Do you have a preference?

Python please! It seems to me that this is the language that one learns one way or another these days and a lot of scientific code is written in it.

0 Likes

#11

Right, that’s what I meant by “a convenience library…that makes all the built-in macro functions also available as global functions from all the other languages.” After I wrote that, I took at look again at ij.macro.Functions (where the bulk of the macro language functions are implemented), but they aren’t exposed in a direct way reusable from scripting languages. So careful thought and experimentation is needed to come up with the best approach to create a suite of public static methods that are fully aligned with the macro language, in a way that maximizes functional equivalence while minimizing future maintenance burden as the macro language continues to evolve.

Even if you are stuck with Python 2 for the foreseeable future? (That is increasingly a problem with Jython—as is the lack of native-backed Python packages.) I fear that all the ImageJ tutorials are in Groovy now, and it’s probably not in the cards to port them all to Python any time soon. But the languages are quite similar!

Do you feel uncomfortable browsing the ImageJ tutorial notebooks (https://imagej.github.io/tutorials)? We really tried to keep them simple and readable. If you find places that are confusing, please point them out!

0 Likes

#12

Hi,

Just a couple thoughts on the topic from a someone with background in scientific coding (ie is largely self taught and terrible software practices).

I started with ijm and have written the 1,000 line macros and asked the forum how to call macros from macros.

I made the shift to python scripts and here are the parts that would make life easier

  • Choosing a language can be stumbling block
  • Documentation is sparser and can be old
  • Modularity is difficult in python scripting
  • A good, native-like linear algebra library

The first two points can be fixed with more documentation and I’m happy to help where I can.

After a larger standard library, a major reason for shifting to scripting is for the code reuse and testing that modularity enables. At first I was doing a lot of hard coding absolute paths for import and manually deleting $py.class files.

The best solution is I’ve found is from here:

What I do is soft link the source directory to the Fiji plugin folder. The boilerplate code only needs to be setup once and it takes of nuking the $py.class files.

A linear algebra a long the lines of numpy would great. I know there’s a couple of java ones but again there’s choice paralysis issue and java interop issues. Maybe it’s worth having python library that’s wrapper around one of the java libraries to replicate parts of the NumPy API?

Cheers,

Chris

0 Likes

#13

Hi @ctrueden,

thanks for bringing this up. Let me bring some examples on that particular website:

The very first code example on that page looks like this:


This is a deterrend first code example. Can we hide this piece of code and start with something simpler?

Opening images

Opening images means for experienced macro programmers:

filename = "path/image.tif"
open(filename);

In the above mentioned tutorial it looks like this:
image
For experienced programmers it might be easy to understand what display([["a":a]]); means, but for others that has to be explained. Would it be possible to change the API in a way that there is a show(image) method?

Draw a histogram

Macro programmers draw a histogram of an image with this piece of code:

run("Histogram");

On the web page you mentioned, the example for this is very complicated. And that’s the first functional example code on that page:

Again, can we start with something simpler? Basically every line needs an explanation here (not just for macro programmers):

  • What does %classpath add mvn... do? Do I have to write it in every script? How Do I know that it has to be exactly like this? How do I know what’s the current version of xchart?
  • What does import do? How do I know what to import?
  • What does image.dimension(0) mean? Does it return pixel size or image width?
  • The syntax object.width(400).height(400).......setOverLapped(true); is highly unreadable and needs explanation.
  • is ij.op().crop identical to run("Split channels")? If I want to split channels, how do I know that I have to use crop()?

Just a suggestion: Can this code example be boiled down to

histogram = ij.op().histogram(image);
ij.ui().show(histogram);

That would be supercool!

Suggesting a simple API

Again: I think simplifying the API and simplifying the code exampels would help a lot. Most macro programmers I met didn’t study computer sciene. And they should not need to. Let’s think about the most common use cases. How would we love to have an API to implement these use cases? I’ll give it a start. Let’s segment nuclei in a DAPI image and measure their mean, standard deviation and area:

image = ij.io.open("dapi.tif");
ij.ui.show(image);

binaryImage = ij.segment.threshold(image, "Otsu");
labelMap = ij.label.connectedComponents(binaryImage);

table = ij.measure.extractFeatures(labelMap, image, ["mean", "stddev", "area"]);
ij.ui.show(table);

Note: I didn’t put comments and the code is still readable.

Actually, I think the ImageJ2 API is almost there. Half of that piece of code might work already. Now let’s make the other half happen! Let’s make the API development user-centric. I’m super happy to help. But we need to do this as a community, because it is quite some work ahead! Let’s do it together - with the users hand in hand. :slight_smile:

Cheers,
Robert

4 Likes

#14

Wow, I didn’t even know this existed. May I ask how you found it? Was it ever mentioned on this forum or the mailing list?

That’s exactly why I’d oppose making Jython the default scripting language. We even require this boilerplate code, or to delete $py.class files manually.

In addition, Jython (in ImageJ) is slow to start up within ImageJ, and has a few other issues that were mentioned before already.

Groovy is much closer to the JVM.

If you look at an ImageJ script (of whatever language except IJ1 macros), it consists probably ~90% of calling into the ImageJ Java API anyways. Probably ~10% or less are language specific constructs (such as if/else, loops, list comprehensions, closures etc.) and they still are very similar among different languages like Python or Groovy.

Snippets like my example above ([image1, image2].each ...) are rare, I’d say.

Compare the Jython version of my snippet above:

results = (image.getStatistics().mean for image in [imagej1, imagej2])

You can always resort to more verbose explicit for loops in both languages to make it more readable for people coming from different programming languages.


It’s a required initialization, you could hide the cell in a notebook, but I think it’s good to show how to set up the basics. And for people used to Jupyter notebooks (again, that’s not the average ImageJ user, I agree), the % magic looks familiar.

The advantage of declaring the classpath in the notebook itself, rather than in a conda environment that you have to setup beforehand: it makes the code (and the science done with it) more reproducible.

I agree the ways of showing/displaying things in a notebook have to be improved. I had to struggle with these as well when setting up notebooks for presentations recently. In particular for label images, color maps, etc. the situation can still be improved.
But these issues are specific to Jupyter notebooks; in the script editor it should be sufficient to declare the image as an output, and the framework would take care to display it.

Oh, yes, I stumbled upon this one as well. I have the impression the current code is some hack to get something displayed, but the code generating the histogram image should migrate into some utility method at some point.

When is the next three-weeks hackathon focusing on this? :smiley: (And how to get more developers on board, who can afford the time and budget that cannot be spent on other, more “prestigious” projects in the meantime??)

2 Likes

#15

Even if you are stuck with Python 2 for the foreseeable future? (That is increasingly a problem with Jython—as is the lack of native-backed Python packages.) I fear that all the ImageJ tutorials are in Groovy now, and it’s probably not in the cards to port them all to Python any time soon. But the languages are quite similar!

As a matter of fact, “stuck in Python 2” is also the case of most of the scientific code I use or try to use :confused: (eg Clearmap, Terastitcher). As for Groovy, I have exactly zero experience with it, but it’s not just about “similar”. I’d look at how easy it is to set up for a newbie (as a matter of fact, after this discussion I guess I’ll just try as soon as I have time) and what libraries are available.

1 Like

#16

For Jython, there’s currently no chance to get numpy etc.

You can use Java-based libraries from any of the scripting languages, so nd4j might be the option you’re looking for.

0 Likes

#17

Hi imagejan,

I think it from googling for ways to write a plugin. Users want to plugin they can drag-and-drop and it appears in the menu system. For a long time I giving them scripts to getting to open the script editor, open the script and hit the run button.

There are other approaches that use maven to compile the Jython down to a jar that might be better for performance, but I haven’t tried them out yet. I don’t find the boilerplate python that bad - take a look at what’s involved to write a plugin in java.

Linear algebra library

that looks good. Interfacing with java in Jython is awkward, and as you point out, maybe we really should just Groovy.

But for the python recalcitrants out there I think that thin-wrapper around a library like nd4j that replicates ~10% of the full NumPy API would make it easier for people coming from python to script in imageJ. Maybe it’s possible write the library so that array slicing with A[:,:::10,:] - python experts can some amazing things.

Role of scripting languages

I think depends on bit of the problem. Once you need to start traversing graphs like Region Adjacent Graphs or pruning skeleton with some non-trivial logic it’s nice to be using your language of choice with the data structures you’re used to.

I think that scripting should be halfway-house between macro glue-code (90% API calls) and low-level native code. For this reason they tend to be dynamically typed and sacrifice performance for rapid prototyping.

I think the main point is it would be nice if the transition from macro to script to java was easier. Having an nested set of API’s progressively allow more control and expose more of the core API would help. The imageJ2 and ops look exciting but I can see that the imglib2 library with it RandomAccessibleInterval<T> and AbstractFactories is going to be another steep learning curve.

Cheers,

Chris

4 Likes

#18

@haesleinhuepf In response to your suggestions, I have pushed a new version of the demo notebook. See it here:

I split the first cell into two halves: 1) installing ImageJ into the notebook; and 2) creating the ImageJ gateway. The link to more information follows.

Once ImageJ is published to Maven Central, the %classpath config resolver line will no longer be necessary. But the %classpath add mvn and ij = new net.imagej.ImageJ() lines are fundamentally necessary, and I see no way around it without violating reproducibility. But note that you do not need to do these things if you are already inside ImageJ—only for notebooks.

We could leave off the final line ("ImageJ v${ij.getVersion()} is ready to go."), but then the cell output is a scary Java address string. One idea would be to override the toString() method to print something nicer. What would you suggest? It needs to be a general-purpose stringification of a SciJava gateway, not only intended for notebooks.

There is ij.ui().show(...). But for technical reasons it does not work in a notebook. See imagej/imagej-notebook#3. Unfortunately, fixing it would break backwards binary compatibility of existing Java classes, because the show(...) method currently returns void. :cry:

For SciJava V3 (next major iteration of SciJava), the show method will return Object.

The comparison is unfair, because your IJ1-macro code does not do the same thing as the notebook cell. Nonetheless, I agree that that cell of the demo was unnecessarily complex. I have changed it to:

image = ij.io().open("https://imagej.net/images/lymp.tif")

I wanted to demonstrate the power and flexibility of the programming interface: that you can dynamically load other Java libraries and use them in a notebook to do cool things; and that you can slice up an image and compute histograms on the slices. But I guess a different notebook would be more appropriate for that. So I simplified the histogram example according to your suggestion:

histogram = ij.op().image().histogram(image)

The histogram is less colorful now, because the image is 2D grayscale.

I also split up the next few examples into separate sections that are each very short. There is still room for improvement and simplification for some of the sections. For example, transform.crop wants an Interval which is annoying to create. Similarly, many filters want a Shape object, and will not accept a plain numeric radius.

(Nearly) any Java library can be used regardless of the JVM scripting language chosen. Or if you are talking about convenience libraries written in a specific JVM language to ease ImageJ scripting with that language: we could provide more of those. Currently, if I recall correctly, Fiji only offers such a helper library for its (J)Ruby support—but a library like that for Jython might be nice.

I agree about the transition from macro to script needing to be easier. As for jumping to Java: I’d prefer if most scientific programmers never needed to do that. Scripts should be able to do everything you need, without needing to learn all of the tooling machinery surrounding Java.

2 Likes

#19

Hey @ctrueden,

wow, thanks for spending time on this. The new version is great. I really like it!

However, I think one or another step is necessary in order to convince macro-programmers of switching to scripting languages. Imagine you are macro-programmer for 5 years and you do have (little) python / Juptyer notebook experience. You would like to switch to scripting languages and find this notebook on that website. What’s important here IMHO is to differentiate what code snippets run from ImageJs script editor and which are only necessary for making the notebooks run. I believe the user community (former macro-programmers) doesn’t use notebooks. But if they wanted to, they should be enabled to. Thus, I wrote a little paragraph explaining how to get them run. It’s just a suggestion, actually I would love to see an introduction to ImageJ scripting without any notebook specific stuff… but I’m not sure if this is possible.

Loading an image

I tried the code example for loading an image:

image = ij.io().open("https://imagej.net/images/lymp.tif")

It doesn’t work - both from within ImageJ and from the notebook:

The file exists online, I’ve just checked. And when I run it with a local file, it works. Maybe it has something to do with my OS? I’m running on Windows 10, 64 bit.

Splitting channels

I’m coming back to this example, because I really think that it is too complicated:


For experienced macro-programmers this might be speaking against switching to scripting. Some aspects:

  • What does import do?
  • If you are doing something like “Split Channels” for the first time, how can you know, that you need to import a FinalInterval ?
  • How can one know that ij.op() has something like transform().crop()? I’m afraid we indeed need to improve documentation here.

For a first easy example, this might be too much.

Counting cells / nuclei

I’m suggesting an example like this:

import net.imglib2.algorithm.labeling.ConnectedComponents;
ij.op().labeling().cca(image, ConnectedComponents.StructuringElement.EIGHT_CONNECTED);

However, apparently notebooks cannot deal with this (yet):

Also ImageJ has issues:

I really would like to get a simple example running where macro-coders can see that something like counting nuclei is super easy in ImageJ scripting and in ImageJ Jupyter notebooks. Converting types and Fourier Transform can come later. Let’s keep it simple on the starting page. I uploaded my alternate notebook to github. Any help is appreciated - feel free to push to that branch. I really believe, such a simple example for beginners is worth to put on the welcome page.

https://github.com/imagej/tutorials/blob/simplify_tutorials/notebooks/ImageJ-Tutorials-and-Demo.ipynb

Thanks again for your support!

Cheers,
Robert

0 Likes