My question is voluntary broad, as I would have your different point of views on this question. I am currently in a situation where I wrote a rather long Groovy script (>500 lines) that i share to the user.
So far so good, it is working well, and easy to share (one single file send by mail).
The point is that I would continue to add more and more options to this script, increasing the length of the file, eventually to the point where I will be lost in my own script.
I would like to avoid that to still be efficient while working on it, and also to be able to share the script broadly with the objective to give something “readable” in a near future.
My question is : how ? how to make something clean, yet easy to share ? The classic ImageJ/Fiji plugin is an option, but would i have to rewrite everything in Java ? Or is it possible to do it in Groovy ? Do i have to split part of my code into packages ? Wouldn’t I lose the “easy-to-share” parameter ?
It may sound dumb, but I would like not to be stuck in the future for not asking questions before !
Thank you very much for your help
Whatever format you choose (Groovy scripts, plugin jar files, or multiples/combinations of those), I recommend sharing via an update site:
You can upload single scripts to your update site, or entire collections of scripts, plugins and their dependencies. That makes installation for users easy, even if you have a lot of third-party dependencies.
Groovy or Java (or Jython or …) ?
There’s no need to rewrite everything in Java, as you can access the full Java API from Groovy as well, and even code classes etc. i.e. make your code modular.
That said, there’s still a benefit to programming in Java (or, maybe, Kotlin), because if you use an integrated development environment (IDE) such as Eclipse or IntelliJ, a lot of work (such as checking syntax, ensuring consistency between modules/dependencies and their versions, and automated building into a jar file) is taken over by the IDE and the build tool (e.g. Maven).
Also, the possibility of re-using modular parts of Groovy code is not well-documented and, in part, still a work in progress:
Splitting your code in modular, re-usable units
Independent of the programming language, it makes sense to split your work into modules, units of defined functionality, as it simplifies maintenance a lot once you reach a certain complexity, and it allows other developers to contribute more easily to your codebase.
As long as you 1) package your scripts/plugins into a jar file, and/or 2) distribute via an update site, I don’t see why it should prevent “shareability” of your work.
For Java, building and packaging your plugin as a jar file is simplified by Maven.
For Groovy (or any other supported script language), you can still combine all scripts into a single jar file for easy distribution. See here for an example:
Reduce/trim your codebase
When your script gets longer and longer, there’s a possibility that you actually implemented something that someone else somewhere also needed as well, and there’s already a solution somewhere online.
By getting in touch with the community, sharing your source code openly (e.g. via GitHub), there’s a chance that others can help you reduce the length of your script (and therefore the maintenance footprint) by using functionality from libraries published within the ImageJ/SciJava ecosystem, and avoiding code duplication etc. (In the best case, it will reduce your maintenance from actually fixing bugs in your scripts to just updating the versions of a few dependencies )
There is many interesting elements there, I didn’t consider update sites as a possibility, but now it sounds obvious that it is a good solution.
I will stick to Groovy, it took me a Covid-lockdown to handle it properly, let’s not wish another one to let me master Java ! Joke aside, coming from Python, Groovy is more digest, yet powerful on lowl-level operation. I wasn’t aware of Kotlin, though, I will check it, by curiosity.
I plan to split my code in modules soon, and keep in mind the example you provided.
I think you have gotten some great tips from @imagejan I only want to add my 2cents on Groovy as a programming platform for research applications and imaging. I have been developing in Groovy, Java, Python, R, etc. And still Groovy is my main go-to language for scripting and gluing other languages and apps together. For distributing and making Groovy scripts of any kind web accessible and user friendly, you should consider Jenkins the integration server for most serious development operations. See here how we made HPC-CellProfiler and other applications accessible to end users via Jenkins and Groovy https://github.com/Novartis/Jenkins-LSCI-Jobs I hope this gives you some additional ideas.
Your 2cents are welcome, thank you very much for it ! I did a really quick research on Jenkins, as I didn’t know it, but, in the context of Fiji, I don’t see the benefit of it compare to an update site for an easy distribution.
If i understood well, it provides many features for the development of scripts and more, which should be really interesting for an expert of the field, but, at my low-level, maybe a proper github and an update site will do the job.
Knowing ImageJ moved from Jenkins to Travis last year, it could be maybe more confusing for me to mixe everything in the case of plugin development.
Any search on Jenkins will probably lead you to the DevOps applications of this integration server, which although powerful and widely used, are NOT how we have used Jenkins in life science applications in our publication. In our ‘think outside the box’ use of Jenkins the DevOps Plugins are used to implement ‘single page web applications’ accessible through the Jenkins server. The end users access the Jenkins server directly to process images, do analytics, capture metadata, retrieve previous analysis results for downstream processing etc. There is no reason you could not integrate fiji, or it’s Plugins into these applications. As such although I did not propose Jenkins for building and hosting the Plugins you are developing, Jenkins would be your application dashboard. Not to forget that the native scripting language in Jenkins is groovy.