Batch analysis using ImageJ Ops

Hi All,

I was wondering whether the Imagej-Ops functionality also specifically supports batch analysis of many files that are stored on disk?!

My impression is that this is not the case, but the Ops “only” make analysis and automated GUI creation for individual data sets easier (which is awesome!).

Is that impression correct?

Hi Christian,

we will support this by exposing most of the Ops as KNIME nodes in the future. This, however, is work in progress, as the Ops API is not yet 100% stable. @gab1one is currently in Madison to work on this.

Cheers,

Christian

Edit: This of course allows you to run ops on tons of images on your local machine, on a cluster or any other environment KNIME supports (now or in the future) and combine your analysis with any machine learning and data analysis tools you like (R, Python, Spark, KNIME, Weka etc).

1 Like

Were you thinking of Op implementations that are optimized for batches of files?

As things are now, you could pick the script language of your choice and loop over files, feeding them into the Ops framework (or potentially reusing a single Op instance), similar to ImageJ 1.x batch processing.

This is actually not specific to Ops, or even images! :smile: It’s just an integral part of the ImageJ2 plugin framework, that commands should be declarative and UI-agnostic.

1 Like

Thanks for the answers!

I am aware that ImageJ Ops will be used in KNIME and in fact, once this is stable and powerful enough, I am wondering whether I should rather use KNIME GUI than ImageJ GUI, at least for batch analysis…

The reason I am asking is because a colleague of mine is working on some jython/java code to make batch processing within ImageJ more convenient (a core part will be some interactive table for easy browsing of raw data, segmented data and numerical results, another part would simply be some jython code populating this table, hopefully generic enough that it could be re-used in many projects; the actual image analysis steps hopefully would mainly be Ops). I just wanted to check whether we reinvent the wheel (at least within the ImageJ universe, KNIME is of course a different story).

3 Likes

Definitely reinventing the wheel.

A colleague of mine (@jaywarrick) did something very much along those lines here in Madison, too:

I would suggest to just learn KNIME. It has a large development effort behind it, is robust, and is getting better all the time. You could also give JEX a spin; I’m sure Jay would be happy to answer any questions.

thanks for letting us know!
we will have a look at JEX.
good I asked :wink:

Had a look at the screenshots and seems like JEX comes with its own GUI.
I think one could argue that there are situations where the users would like to stay within the ImageJ GUI (which is signified by very direct and flexible interaction with the image data), but simply would like to have some more convenience for switching between different data set. JEX feels like entering a whole new universe, does it?

Yes, it diverges from ImageJ quite a bit that way.

You could certainly create yet another thing built on ImageJ intended to make batch processing easier. I personally still believe it is reinventing the wheel though:

  • Will you really achieve something better than KNIME?
  • Maybe it will have a smaller learning curve, but will it be as powerful?
  • Will its architecture survive when more layers are built on top of it?
  • Will you be able to maintain it for the next 10-20+ years?
  • How many bugs will it have?
  • Will it be interoperable with other community efforts?
  • Will you be creating a 15th standard?

I urge you and your colleagues to thoroughly learn about all the existing tools before embarking on creating your own.

3 Likes

@Christian_Tischer

It might be useful to learn everything you can about how KNIME does batch processing, then learn about the ImageJ2 architecture, particularly Services and how to create new Services

Then you can identify ways to improve batch processing with these tools.

I think there could ways to contribute new Image2 services, or enhance existing ones to improve batch processing. For example here is a bit of pseudo code that reflects how I’d like to do batch processing.

# @IOService io
# @BatchService batch
# @String dir

imageNames=batch.getImages(dir, ['tif', 'jpg'])

for imageName in images:
    process(io.open(imageName))
2 Likes

Hi Curtis,
I get your point. In fact that’s why I started this discussion here, to see what’s out there.

Another major tool I know about in in this realm is CellProfiler, which is specifically intended for batch automation of reproducible image analysis workflows in cell biology and related fields. The CellProfiler team is very talented, with its development backed by the mighty Python scientific software community. CP also has its own excellent image analysis forum. And it interoperates well with ImageJ, too.

And see also MiToBo, an extension for ImageJ which adds graphical workflow construction and many other features.

Both of these tools interoperate with ImageJ2 via the SciJava module framework.

I use CellProfiler for 100% of my 2D image analysis. I am looking for something similar in 3D. I guess at some point KNIME will be it.

2 Likes

Yeah, and/or CellProfiler will grow 3D capabilities. Note that all three of these tools are really poised to leverage the ImageJ Ops library of N-dimensional image analysis routines. So as things get added to Ops, the hope/plan is that all of these tools (ImageJ, KNIME, CP, etc.) will become stronger because of it. I guess you know that already though, given the topic of this thread. :wink:

2 Likes