Color Deconvolution plugin IJ2

I find your suggestion extraordinary, given that I am the author of the original plugin, from which the derivative to make it work in IJ2 has been announced after I explained that there was a new version coming along.
I am sure that @sunsear can avoid that name.

1 Like

I am sorry if I have come across rude. I apologize.

I wasn’t aware of this. My impression was that the plugin by @sunsear was around for a little while already, whereas I obviously hadn’t heard/read your earlier announcement (can you point me to where that was? found it further up in this topic). I’ll stop chipping in here, sorry.

As we have seen from Color deconvolution implementations & best practice there are diverse implementations with some - more or less - important differences.
The CD plugin based on IJ2 is yet another one. And we have seen from this post here that in this new IJ2 version there are potentially some (new) differences compared to Gabriel’s original version (and his announced new version).

I think it would be important to choose a distinct naming to allow users to recognize that parallel implementations exist and to differentiate one from the other. (The bare existence of parallel version can mix up users who only want to use a feature and not want to compare different versions neither on code analysis nor on result differences.)

It would be a benefit for the whole ImageJ1/FIJI/ImageJ2 project to provide a clear ‘project picture’ to the user instead of an opaque puzzle.

The extension ‘_IJ2’ could help to reach such a state.

1 Like

Replying to @phaub s comment, but this also applies to the comments from @gabriel himself. I meant no disrespect in the naming and am certainly trying to avoid confusion here.

What we were trying to build, was a plugin for ImageJ2 that was algorithm wise completely compatible with the ImageJ1 version. We merely needed it in ImageJ2, as IJ1 is not that compatible with Knime, where we use it.

In order to do that, I started from the repository https://github.com/fiji/Colour_Deconvolution Where the ImageJ1 plugin resides. I merely added an implementation for ImageJ2 to that repository and refactored the code a little bit to remove some of the resulting duplication. Also I added some tests to prove both versions have the same result. My goal and the reason for my question to @ctrueden, was to bring all of my changes back to that repository so that we would have 1 repo where both ImageJ1 and ImageJ2 versions would live.

To be clear, we made this open source. I have no need nor desire to take any credits for the algorithm that @gabriel put into code first. I’d simply like the code to be part of ImageJ so everybody can build on that.

Hope someone can help me bring this code into the ImageJ fold.

2 Likes

Hi @sunsear,
I hope you don’t read an implication. In my personal view I tout for clarity.

One more note:

Gabriels original CD version as well as the current CD version (used by FIJI, located on GitHub) are supporting RGB image stacks.
As far as I can see the IJ2 CD version currently don’t support RGB stacks.
The CD result of a single image maybe identical in all versions but the IJ2 CD versions is currently not fully compatible with the two other versions.

1 Like

Hi @sunsear, your reply unfortunately does not address my request.

The changes you made to the CD plugin add extra classes that appear in the Fiji menu categories: “Color Deconvolution 2” and “Color Deconvolution Select”.
Check at the bottom of the “Color” menu entry in the screenshot posted above.

The problem is this: as you already know, the new classes called "ColorDeconvolutionIJ2.class and ColorDeconvolurionIJ2Select.class when show up in the menus lose the “IJ” and the “IJ2”. I can’t tell why those classes appear in the menus like that (they are not defined in the plugins.config file), but for some reason they do, and that interferes with my new version of the plugin I announced before and which has different functionality than the original.

I think everybody would agree that having an extra menu entry called “Color Deconvolution 2”, with the same functionality of the current “Colour Deconvolution” but which is different from the already announced new version “Colour Deconvolution2” is quite confusing and unnecessary.

What I am requesting is quite simple and reasonable:

  1. modify that class name so it does not show as “2” in the name of that menu entry. Call that something else: “Colour Deconvolution for ImageJ2” or whatever so it is clear that it does not add any new functionality.
    However, if it there is a change of behaviour or the way it is used then:
  2. you need to be explicit about what the modifications are before requesting to be included as part of the distribution. For example I do not follow why the need to have a “Color Deconvolution Select” command to input the vectors when the original plugin already does that with the “User Values” option. That breaks a lot of already working code. As @phaub noted, if your modification does not support stacks that is another quite problematic loss of functionality.

Since I am the author and maintainer of the CD plugin, please understand that I will not agree to have modifications that interfere with itself, or that are confusing to users or that make old macros/code not work anymore. I hope that clarifies it.

Ah, we have indeed not implemented stacks yet, that might be something to make the plugin more compatible with the 1.0 version. Will put that on the todo list if that gets us closer to bringing back the plugin into ImageJ repo. To be clear, we don’t need that, so will put that effort in, only if we find a way to get the code base back into Fiji repositories.

Aah, I have misunderstood how Fiji works. Apologies, that does not make things clearer. Apparently FIJI works differently from a base ImageJ1 install, which I tested with. If I start up a base ImageJ1 install v1.53c and install the plugin you can download from our repository, I only get the original ImageJ1 plugin that was authored by @Gabriel. I was therefore under the impression that Fiji and ImageJ1 only load ImageJ1 plugins and that we could put the IJ1 and IJ2 plugins into the same jar file. Then if a 2.0 platform such as Knime loads that same plugin, it will automatically pick up the 2.0 plugin. If at some point the community moves to 2.0, we’d have the plugin already working.

Apparently Fiji can also load 2.0 plugins and therefore picks up both IJ1 and IJ2 versions at the same time. That will provide a lot of confusion indeed. @ctrueden, or anyone else, how should plugins provide a 1.0 and 2.0 version? Should those be different distributions? My lord, code sharing between versions will become a nightmare in such a world…

Aside from that, it was never intended to imply that the version we wrote was a 2.0 version of the algorithm as you have told us you were working on @Gabriel. I merely added a 2 to the menu item option so that we could distinguish them should they ever appear side by side. The latter was never the intention anyway. I’d like to find an answer to the question I raised above regarding IJ1 and IJ2 code sharing, then we can decide on a good naming scheme for the IJ2 version. For now I will update the plugin so that the menu items say Colour Deconvolution for ImageJ2.

Regarding the menu options, in the ImageJ2 plugin infrastructure there is no longer anything like a plugins.config necessary. The menu items are taken from the @Plugin and @Parameter annotations that are in the Java class itself. Since this method is very easy to work with, but less powerful in User Interface, we created 2 menu options. We cannot do what the 1.0 version does with a select list that then has an option called User Values and make a next screen where you can apply those. An easy fix seemed to be to create 2 different menu options.

It now looks as follows:

I have not released this version yet, first need to resolve a small code issue, but my family demands we go to the beach :wink:

Well, no. That is why there are authors and maintainers who take responsibility that things are consistent and do not break other software based on that code. Not all contributions to open source software are guaranteed to be accepted, no matter how good the intentions are.

May I suggest that you make your own release for knime or IJ2 and maintain it, so it does not interfere with the current IJ/Fiji distribution I maintain. If you decide to go that route, I have these suggestions that I hope would be useful:

  1. make it clear it is a derivative of whatever version of CD, and not superseding it,
  2. specify who maintains it,
  3. state what are the differences with the original, to avoid surprises,
  4. make sure that the plugin name is obviously distinct from “Colour Deconvolution” and “Colour Deconvolution2”.

Thank you.

1 Like

@gabriel, fine suggestion, that is a road that we can walk. I do see a way that we could keep both IJ1 and IJ2 versions together. We can then reuse the parts that are common and have the IJ-version specific stuff in different folders. It does complicate things a bit, but improvements we make to the core stuff are at least guaranteed to work among IJ versions.

How it could work from a directory perspective:

Repository
*- Common
*- IJ1Plugin
*- IJ2Plugin

From a maven perspective we could then make the common project a parent of both of the other projects:

           Common
             |
    *------------------*
    |                  |
IJ1Plugin          IJ2Plugin   

Using this setup we could have code dependencies from IJ1 and IJ2 to the Common project. It would mean that we would have to make the IJ2 version feature complete, so add the Stack feature probably. Also I can do this Maven part as I have a lot of experience with it and it really isn’t all that complicated to add.

Please let me know if this is a way that would work for you and with which you would like to work in the future. If not, I will remove all IJ1 code from our repo and make that a completely separate thing from the one that you are maintaining.

1 Like

Yes, Fiji actually is just ImageJ2 (!). Note that the goal of ImageJ2 always was backwards-compatibility with ImageJ 1.x. That’s why the imagej-legacy layer – if you have it on the class path – will make sure that all ImageJ 1.x plugins will continue to work fine, while additionally giving you all the benefits of ImageJ2 (headless commands, full separation of functionality vs. user interface, etc.).

Yes, you can do that with ImageJ2 commands as well. It likely requires calling one command from another command, but you can make the available choices dependent on other input parameters in a dynamic way. Let’s discuss this in a separate topic if you need help implementing this.

In my humble opinion, you would ideally provide only the IJ2 version: that way, it works in Fiji as well as in other (headless) frameworks. Users who want to use it in ImageJ will use Fiji anyways. (At least I know nobody who uses plain IJ1 for any real work. [Edit: I’m sorry for that unqualified statement. What I menat to say: in my field – biological image analysis – everybody I know prefers to use Fiji over plain ImageJ1, because of its bundled functionality.] That might be biased, but is the truth in my institute/facility and essentially all facilities that I’ve got to know within the #neubias network.)
And users who want to use it in #knime will have it as well. So I’d suggest that the fiji/Colour_Deconvolution switches to the IJ2 version. of course only if and when that one provides full backwards compatibility with the IJ1 version.

But again, that’s just my opinion, your mileage may vary…

While this certainly is a feasible solution, I think it’ll increase maintenance burden (probably for the Fiji maintainers, at the moment mostly @ctrueden). But as long as we have dedicated maintainers for both plugins, alright.

1 Like

One possibility is that you know very few people. Could I suggest to avoid making unnecessary, inflammatory statements like the above? You are in a privileged Forum Team Member position and expected to act as an impartial moderator, not freely insulting for no reason. It is the 2nd time this happens in this thread. I fully understand that you want to publicise IJ2, but please back the statements with some evidence.
To put things in perspective, some simple searches reveal that you actually might be wrong:

Science Citation Index:
ImageJ in all fields: 3966 records
ImageJ2 in all fields: 9 records

Google search:
ImageJ About: 4,970,000 results
ImageJ2 About: 40,800 results

Pubmed:
ImageJ 2,674: results
ImageJ2 6: results

Amazon books
ImageJ: 21 books
ImageJ2: No results.

If I delete the ij-1.52p.jar file sitting in the Fiji.app/jars folder, Fiji does not work anymore. One could equally conclude that everybody using Fiji is also using IJ1 for their (serious or no so serious depending on your definition) work.

I think that making it a separate thing is the best option. Thank you for your understanding.

Ok, I am sorry about my unqualified statement that was led by emotions, instead of being objective.
I didn’t mean to insult anyone, and regret if I was becoming too emotional here, and not acting in the spirit of this fantastic open community.

I edited my post above and tried to explain. If you feel I should delete any of my posts here, or edit it in another way, please let me know and I’ll do it.

Let me state that publicising IJ2 was not my goal here. I was very happy to see the efforts by @sunsear to make this plugin available also within KNIME (a framework I’m regularly using), and hence felt the need here to weigh in for that exiting new work.


Unfortunately, it is very hard to collect evidence about usage of ImageJ1/Fiji/ImageJ2, since these projects are so open and none of them collects reliable usage statistics.

Your searches suggest such evidence, but the results are skewed because:

  • Fiji (Fijj Is Just ImageJ) bundles both ImageJ1 and ImageJ2 and has “ImageJ” in its name
  • Users using Fiji often only cite ImageJ, or only Fiji, but almost never ImageJ2 explicitly, although they might be using much of its functionality (i.e. the updater, the SciJava command framework, etc.)
  • There’s a lot of confusion in the user community regarding the exact identity of ImageJ(1) and ImageJ2, unfortunately.
  • Obviously, it is hard to get meaningful results when searching for “fiji” alone without the term “imagej”.

I by no way meant to imply that Fiji is not ImageJ1. On the contrary, Fiji and ImageJ2 strive to be fully backward-compatible by actually bundling ij.jar.
I also didn’t want to say Fiji should stop wrapping ij.jar.

I did want to say that for a specific plugin like the one discussed here, it would be desirable to switch to using ImageJ2 functionality, because it still works in Fiji and would be accessible for users of other frameworks, such as #knime and #pyimagej. For current users of the plugin within Fiji, this would break any functionality (as long as it’s done right).
Of course, users who used the plugin from plain ImageJ1 so far will not be able to use any IJ2 version of it.


In the spirit of openness and collaboration, I’d like to conclude with a suggestion:
Why not combine the reworked version by @gabriel (which I do not know yet, but have no doubt will be of excellent scientific quality) with the efforts by @sunsear towards an IJ2 version of it, and make a new version that benefits from both improvements?

2 Likes

@sunsear is at this moment on holiday we will get back to you when he is at work again.

Thanks for all suggestions @imagejan, @phaub and @Gabriel. I have compiled a small list of things that I took from these discussions in the issue list in GitHub:

When I get time I will work on these, I’ll start with simplifying the menu options.

If I missed something here, please let me know!

I am curious though, @ctrueden , what is required for my repository to be accepted into the Fiji fold?

@sunsear Have you seen this page?

Which covers the more concrete technical side of it. On the social/cultural side: as maintainer I lean much more toward “include” than “exclude” by default. Of course, each approach has its own problems—but if there are people in the community (users, developers, or both) who want a plugin in Fiji, then it will generally be included with Fiji. There would need to be some substantial downside to do otherwise (tons of new dependencies doubling the Fiji distribution size; code that encourages misuse or bad science; stuff like that).

As for the discussion above about developing with IJ1 versus IJ2 data structures: I like your idea to keep as much as possible agnostic, and then write a thin IJ1 plugin wrapper, and thin IJ2 plugin wrapper, over top, that adapts to the appropriate data structures. My biggest concern with that would be if you need to make compromises for the sake of IJ1 compatibility that harm performance or make the code much harder to maintain.

The intended vision of IJ2 is for new developments to be done using ImgLib2 and friends without any import ij.* anywhere. To do otherwise does make the developer’s job substantially more complex—and is honestly not a design pattern I feel too enthusiastic about facilitating. As you point out, the status quo is quite a burden for the community in many ways, and encouraging new developers to explicitly target both platforms only exacerbates the problem further into the future.

That said, I understand that because the ImageJ2 codebase is less mature than ImageJ1, with some still-missing features, it remains necessary to mix them in some scenarios. That’s why we continue to support it. Perhaps you could summarize the challenges you have faced using “pure IJ2” with this project, such that we can file and/or prioritize issues to reduce these challenges as we continue developing the tools?

2 Likes

Thanks @ctrueden, will check that page.

On second thought, I think it will be hard to put our plugin into Fiji as the plugin by @Gabriel is already in there. It doesn’t make sense to deliver another plugin together with Fiji that does the exact same thing. For now, I think I will make sure that we get a nice 1.0 version of the plugin and distribute that through GitHub.

I can list the difficulties with ImageJ development. Where would you like that to go?

1 Like

I suggest you create an ImageJ update site.

How about a new thread here on the forum? Tag with imagej of course. I’ll be away the next four weeks, but hopefully the community can have a discussion about it. Any action items that emerge can be filed as issues in the relevant repositories. CC @hinerm

1 Like