Modern mode future

fiji
imagej

#1

Hi all,

I would like to get a better picture of the future of the modern mode.

What is exactly preventing us from using this mode - is there a list / issue somewhere? The IJ1 plugins seem to be accessible so I guess it differs in ROI / labeling usage?

From past discussions I got the impression that this change (using the modern mode) might never happen or might not be wanted. What would be alternatives towards a future user interface? What are your general opinions?


#2

Good day,

and thanks for posing this very essential question.

With “modern mode” you mean ImageJ-2?

For me it appears as if to explor and partly apply the fast developing resources in informatics is more interesting than to fill the considerable functional gaps that exist between imageJ-1 and Image-2.

One example is “Compile and Run…” which is extremely useful for creating ImageJ-1 style plugins that, BTW are much easier to code than their ImageJ-2 counterparts, at least for none-informatic professionals.

But there is much more …

Regards

Herbie


#3

As far as I remember, there were simply too many issues with the (similar but different to IJ1) “new” user interface, and development focused on the backend. See also:

@ctrueden should be able to explain in more detail the reasons/issues why development of an IJ2 UI was halted.


#4

Thanks for answering! Let me clarify how I understood the existing modes, might be wrong, please correct me:

  • legacy mode The interface you get when you download Fiji and run it. Additionally to the IJ2 universe it includes IJ1 and executes the IJ1 interface which is hacked to display IJ2 commands in the menu (?). It’s like running ImageJ from the IDE with the legacy maven repository.
  • modern mode The interface you get when you download Fiji, run it and call Help > Switch to Modern Mode. It executes the IJ2 main interface where all the IJ1 PlugIns / PlugInFilters etc. are also in the menu and executed as a LegacyCommand. I can only spot small differences to the legacy mode visually.
  • IJ2 mode The IJ2 interface you get when running ImageJ from IDE without legacy maven repo. It’s only showing IJ2 commands / Ops (?).

I can imagine why the pure IJ2 mode is hard to sell, but since the modern mode looks so similar to the legacy mode, maybe the remaining issues are mostly backend related?

Don’t get me wrong, I don’t want to push anyone into tackling these issues right now, I just want to understand the different modes and motivations and get an idea of what the community thinks about future directions of the interface.


#5

@frauzufall btw, did you ever see https://github.com/cmongis/imagejfx/?

imho, it is the closest thing to a modern designed imagej UI. unfortunately, it missed the mark on doing imagej2 integration the right way. however, it contains examples of arguably the majority of the javafx widgets needed to provide basic imagej functionality.


#6

I wasn’t aware of ImageJFX, thanks for sharing @kephale!


#7

@frauzufall There are a total of six ImageJ2 user interfaces I know about:

  • The so-called “legacy” mode - part of imagej-legacy - a SciJava UserInterface implementation wrapping the ImageJ1 user interface.
  • The so-called “modern” mode - a Swing-based UserInterface implementation, included with scijava-ui-swing with some elements in imagej-ui-swing.
  • scijava-ui-awt and imagej-ui-awt - A partial proof-of-concept of a pure-AWT (i.e. no javax.swing classes) UserInterface implementation. Note that these components are also foundational libraries used in the Swing implementation.
  • scijava-ui-pivot - A partial proof-of-concept of an Apache-Pivot-based UserInterface implementation.
  • scijava-ui-swt - A largely incomplete proof-of-concept of an Eclipse-SWT-based UserInterface implementation.
  • imagejfx - A quite nice, but no longer developed, JavaFX-based user interface for ImageJ.

What you call the “IJ2 mode” and “modern mode” in your post above are both the SwingUI. The only difference is whether the legacy ImageJ1 plugins are included in the menus—which only happens if imagej-legacy is on the classpath.

There are many reasons the SwingUI has not been developed in recent years:

  • The current/legacy ImageJ UI is the result of many cycles of iteration and the existing user-base generally prefers it. When we released the first few betas of ImageJ2, which shipped with the Swing UI active by default, the bug reports flooded in for things like the brightness/contrast dialog, because users had very particular opinions about exactly how this dialog should work, based on their experience with ImageJ1.
  • While the modern Swing UI has some technical advantages, it was never fully finished and it has lots of disadvantages as well. Polishing it to the point where it is competitive with the legacy UI is not something anyone has time to do right now; furthermore, it is not clear that the technical gains of doing so would be worthwhile enough to justify the amount of effort needed.

In short: the general thinking is that it makes more sense to design a new UI “from the ground up” a la ImageJFX, making a clean break from the old UI, rather than futilely attempting to match everything ImageJ1 does, exactly the way it does it, all while ImageJ1 continues to be a moving target.


#8

That’s an interesting discussion here :slight_smile: Before starting a seventh UI, it might make sense to determine the current state. Thanks for doing so @frauzufall :wink: IMHO, building another UI that looks like the old is not worthwhile. I believe that would be the wrong direction. However, I agree that building a new UI is overdue for some 15 years :wink:

Goal of this discussion might also rather be “What do we want?”, “What can we do?” and “How do we combine both?”

I’m just posting some nice pictures of user interfaces, I could think of for ImageJ. I’m absolutely sure we could achieve such things. I mean, we can do better than in the ninetees. There has been quite some development we should exploit. ImageJFX was a great start, but now lets dream a bit. We are talking about the future :wink:

image
Source: https://twitter.com/frauzufall/status/1051453419607810048


Source: Jupyter Notebook for ImageJ

image
Source: BigVolumeViewer Tech Demo


http://icy.bioimageanalysis.org/doc/icy-protocols.pdf


Source: https://twitter.com/haesleinhuepf/status/1055167735964098560


#9

Good day Curtis,

I remember well that in times during the—in my opinion—rather short funding period for the construction of a more modern ImageJ, one of the central goals was the separation of function and GUI. I faintly remember that this goal was even part of the project proposal.

If this goal had been reached, it should be straigtforward to put whatever GUI-architecture on top of the function layer.

My impression over the years however is, that one of the many problems is that the modern function layer is neither complete nor separated from whatever GUI.

Given this situation, a new GUI appears not being the central problem today but the state of a mixed software architecture. Call it it Fiji, ImageJ-1/2 or whatever.

In short, I don’t see a chance and perhaps not even a need for a modern GUI for ImageJ, given the present state of the function layer.

Best

Herbie


#10

Good day Robert!

We are talking about the future

Maybe, but I’d prefer to look first at those goals that haven’t been tackled in the past: See my reply to Curtis.

Best

Herbie


#11

Yes. If I were going to develop a new Java UI (which is not currently on LOCI’s development roadmap), I would definitely start from ImageJFX, rather than starting from scratch. But it might be more impactful to develop a more general cross-language UI (see below).

Completely agreed.

I am not sure. While one of the original ImageJ2 project goals was to build a new UI, the needs of the community are very heterogeneous. The current UI is very flexible and meets many of the needs. Other more technical needs can be met by Jupyter Notebooks, or by embedding ImageJ into other more specialized applications (both Java and non-Java). I am skeptical of the added utility of a new JavaFX-based UI for ImageJ. I think it would be nice if it existed, but the development and maintenance costs of creating it are high.

IMHO, a new ImageJ UI project should have dedicated funding and people. And based on the current technological landscape I would favor developing in JavaScript over JavaFX, since JavaScript has a much huger community behind it and will offer more powerful options for how the UI is used: embedded in a web site, standalone desktop client, etc. Whereas a JavaFX-based UI will always be Java-centric and desktop-oriented (can’t even run NPAPI plugins like Java in Chrome anymore).

To be clear, I am not proposing to abandon Java in favor of JavaScript in general. JavaScript applications can interface with Java, and we could mix and match JS-based and Java-based UI components as well. There is a proof of concept here.

In particular, I am not skeptical of the utility of new image viewers (written in Java or otherwise). Tools like the BigDataViewer and SciView are needed as we continue to move toward bigger and bigger data, and away from 2D slice analysis toward 3D and N-dimensional analysis.

Indeed.

I do not know where you are getting that idea, but ImageJ2 does indeed have a thorough separation of concerns, with functional layers completely separate from the aforementioned UIs. In SciJava, user interfaces are merely a type of plugin. That is why you can run ImageJ2 (but not ImageJ1) headless inside of a Jupyter notebook, for example. Or from a Python script. Or from a node.js project. Or as a RESTful image server.

Please develop a technical understanding of the ImageJ2 architecture before you speak about which goals have, or have not, been tackled.


#12

Curtis,

with regard to

I do not know where you are getting that idea, but ImageJ2 does indeed have a thorough separation of concerns, with functional layers completely separate from the aforementioned UIs.

I wrote:

If this goal had been reached, it should be straigtforward to put whatever GUI-architecture on top of the function layer.

So where is the problem with constructing arbitrary GUIs ontop of this function layer?
This was one of the thread’s questions.

Please develop a technical understanding of the ImageJ2 architecture before you speak about which goals have, or have not, been tackled.

I see quite some tasks that haven’t been tackled. In my first post in this thread I mention one:

One example is “Compile and Run…” which is extremely useful for creating ImageJ-1 style plugins that, BTW are much easier to code than their ImageJ-2 counterparts, at least for none-informatic professionals.

Best

Herbie


#13

It certainly can be done. But developing a user interface is its own very involved and time-consuming project. This thread is speaking about how such a UI could look, and technical choices that could be made in developing it, hopefully as a cost-benefit analysis to ensure any potential work proceeds in effective directions.

Most definitely. There are many features that would be good to add to ImageJ2, some of which are available in ImageJ1 but not ImageJ2, such as “Compile and Run…” as you point out (issue at imagej/imagej-ui-swing#48). However, a lack of separation of concerns in not the problem; ImageJ2’s architecture is well separated.


#14

Given this situation, a new GUI appears not being the central problem today but the state of a mixed software architecture. Call it it Fiji, ImageJ-1/2 or whatever.

(Emphasis by me.)

Maybe ImageJ-2 has a perfect separation but it is incomplete and therefore we need to fall back to functions that don’t have separated layers (mixed software architecture) …

In conclusion, we realize that the present whole functionality, as we know it from ImageJ-1, is not separated from the GUI. That’s what I meant and why I wrote that before we think of GUI-layers we need to tackle tasks that haven’t been tackled.

This thread is speaking about how such a UI could look, and technical choices that could be made in developing it, hopefully as a cost-benefit analysis to ensure any potential work proceeds in effective directions.

Perhaps, but shouldn’t the missing functions be added first?

Best

Herbie


#15

TL;DR:
ImageJ2 separates functionality and user interface, we can explore ideas in both directions at the same time.

Added to what? How will people interface with the newly added functionality? Using Modern mode, ImageJFX, or @frauzufall’s next masterpiece? :smiley:

Others can speak to this much more than me, but a lot (but not all by any means) of ImageJ1 functionality has been introduced to ImageJ2 over the past few years. Some of these improvements are visually documented through the notebooks. The notebooks demonstrate how much of the functionality of ImageJ1 has been translated into ImageJ2, where there is a separation of functionality and user interface. One reason for having the discussion about the next ImageJ user interface now, is that we need to know when your question is completely answered. An easy way to answer this is by creating the independent UI; however, modern mode isn’t modern enough, and <insert all of the great arguments and ideas about a new UI that already exist in this thread>. It is worth noting that the architecture of ImageJ2 was designed in a way that actually makes it quite easy to explore different user interface ideas. So much so that interfaces that are fully compatible with ImageJ2 will essentially automatically extend to accommodate newly implemented functionality.

~Kyle


#16

Added to what?

ImageJ-2

Greetings

Herbie


#17

@ctrueden @kephale I am currently far from thinking about a new interface design, as Curtis said, this needs dedicated planning and funding. It is still helpful to grasp opinions of the community about that topic from time to time. Also because it is relevant for deciding how much work we will spend in the future on writing Java Swing plugin interfaces.

I was raising this question mainly though because the modern mode has advantages for me so I was wondering why we can’t use it, I know we can’t, but if I ask people I always get “there are issues” as a reply and rarely details.

I wrote a quick prototype translating ModuleEvents of an EventHistory service into script code. That made me look deeper into the differences between Modules, Commands, PlugIns, PlugInFilters, … Then I realized that only in the modern mode I can record IJ1 PlugIn events (because they are called from a LegacyCommand). I can probably hack it into IJ1 and maybe will and then discuss this prototype with all its issues, but that’s a story for another day.


#18

Some legacy plugins assume/hardcode UI implementation details, such as there being a visible ij.ImageJ main window that is a java.awt.Frame, or the active ImagePlus having a corresponding active and visible ImageWindow that is AWT-based. Some legacy plugins try to do real-time updates—e.g. Plot Profile in live mode, or Brightness/Contrast. None of these real-time operations will work as intended in modern mode, because the visible image window is not what ImageJ1 thinks it is.

Could you elaborate a bit? I don’t understand the issue. There is probably a way to achieve what you want by improving the imagej-legacy code…


#19

Thank you for the additional details, highly appreciated!

About my issue: I am tracking ModuleEvents (like the EventWatcher command does). In the legacy mode IJ1 PlugIns will not trigger an event whereas doing the same thing in the modern mode I get an event from the LegacyCommand which I can log.


#20

Ah, I see what you mean. E.g., if you open Blobs from the search bar, it goes through the SciJava module execution mechanism, whereas if you open Blobs directly from the menu bar, it goes through ImageJ1’s execution mechanism which does not fire an event.

I filed an issue: