Can APEER become a partner website on Image.sc

apeer

#1

Dear image.sc community,

Many of you know that I am part of APEER, the digital microscopy platform for scientific image processing.
Full Disclosure: The platform was initiated by ZEISS (see here).

We already talked to the image.sc admin team and they suggested we bring the discussion to the whole community. So here it is:
We as the APEER team would like to become a partner site with its own top level navigation entry here in image.sc. We believe that this is the best place to focus the discussion about scientific image processing and data analysis, enhancing community knowledge, learning and education

I have laid out our intention, now here is the main question of this thread: Would you agree with us that becoming a partner site, discussing current and future functionalities and interacting with the scientific community in one place is a good thing?

Some FAQ for the discussion

APEER is sponsored by ZEISS so it is commercial!

Yes, ZEISS is behind the APEER efforts. The idea and vision behind APEER is make research easier by making it a little more seamless. We want to enable researchers like you.
The platform is currently free to use for everybody and will remain free at its core. We might add paid features at some point in the future enabling options such as, purchasing more online storage space, purchasing more CPU/GPU resources to run processing tasks or the option to add paid modules by external content creators.

Why is APEER a community platform?

  • APEER is designed in such a way that the smallest functional unit is a "Module". These Modules perform certain processing tasks and can be connected to other Modules to build full featured processing workflows.
  • Every user of the platform has the option to add their own Modules and extend the functionality. Initially these Modules are private to the respective user account but can be shared with everybody if the Module creator chooses to do so. In the near future features such as sharing results or collaborative data viewing will be added.
  • A Module in itself is typically a docker™ container which contains the required code and hosted with GIT. This makes it easy develop open-source tools because one has the freedom to choose what tool or language to use

Is/can APEER be open source?

  • Yes, one of the next features that are going to be released will enable Module creators to decide if they want to share the source code of their Module(s) with every user. As mentioned above the versioning system is GIT so if you choose to open source your module other user can directly clone the source code with GIT.
  • If you chose to share your Module(s) with everybody you retain full ownership! APEER only hosts the backend processes and the infrastructure that allow users to build, run and share meaningful workflows.

What does APEER add to the image processing community?

  • APEER allows you to build your own workflow from start to finish and later share it with everybody.
  • You can also add your own functionalities and extend the platform according to your needs. Adding new functions is very easy if you are already programming or scripting your own processing functions. You are not limited to a certain programming language but you can use nearly any language of your choice be it Python, R, Java, C, etc.
  • APEER is continuously developed, and we will listen to your needs. We want to shape the future of this tool together. In addition, we already working hard on adding more community features for easy collaboration, sharing, education and distribution.

Do we want free support?

  • No, for technical support we still have our own direct support channel and email to reach our support staff. However since APEER is supposed to enable the community we cannot take responsibility for shared modules from content creators. Therefore, a unified forum such as image.sc will be ideal to bring the community together.

We hope for a good discussion and that we can ultimately become a partnersite and we can grow together with all of you.

Regards,
Robert Kirmse
APEER TEAM
https://www.apeer.com/


#2

No, for technical support we still have our own direct support channel […]

A community partner is a software project or other community organization that uses this forum as a primary recommended discussion channel.
(Formal policy for project to be a software partner)

I’m sure you can’t separate questions regarding “technical support” from “discussions”. At least this holds for the present Forum.

I’m sceptic about bringing together platforms that technically have little in common.

Regards

Herbie


#3

Dear Herbie
thanks for the first reply. I added the support topic since in another thread the issue was raised that it is not well regarded if a company tried to outsource its support to expert users in open forums. Towards that topic I made the statement that we will for sure have direct support channels regarding the platfrom itself and everything around it.

However since our goal is to also build a community platfrom that will hopefully host a lot of community content we cannot support every open source module on the platfrom. Similar to e.g. ImageJ where a plugin creator would also support her/his own plugin.

In addition if we were to become a partner site here we would also be present here for discussion that borders in support.

Regarding

I’m sceptic about bringing together platforms that technically have little in common.

I do believe APEER has a lot in common with the the other software packages featured here. We enable all of you to host processing functions as modules on the platfrom for everybody to use and to interconnect easily.

These Modules are - as mentioned - Docker containers at their core and because of that APEER can essentially host nearly anything you can package into the container from plain python, using scikit, to ImageJ and apparently even CellProfiler (I saw a docker container but haven’t tried it yet)

So for the open source part of the platfrom - it’s modules - at least in my opinion it would make perfect sense to have a presence here since we would discuss common features, learn together etc.

It would be far better then fragment the community with yet another forum hosted somewhere else. And fragmentation is something we specifically want to reduce with APEER.

Best
Robert


#4

Robert,

be honest, APEER and other attempts make sense for ZEISS because free/open-source software becomes more and more powerful and versatile when compared to commercial software.

In fact you touch on a critical problem:

  1. No support/help-hotline for commercial software on the Forum
  2. No community partner of the Forum with other relevant communication/discussion channels

I like seeing some institutions being in a double-bind!

Regards

Herbie


#5

Hi. Is Apeer open source? I think that the OP is not clear about this as it refers to modules that one could develop, rather than the application itself.
I tried to find out, but I was asked to register. Why?


#6

Hi there and sorry for the late reply

let me comment on the last two posts. Completely true that such an approach makes sense, I never claimed it to be otherwise. Why not join forces and collaborate? The clear aim is to join forces with the community of imaging specialist out there and also enable easy to use, easy to reproduce workflows and also establishing standards for this. One but not the only building block could be an infrastructure like APEER.

The support topic I only mentioned to alleviate the fear that we want to mis-use the forum here as support channel. This is not the case, becoming a partner would make sense to join forces as mentioned above and not fragment forums further and force people into multiple channels. So a solution for the support should be separated because your point 1. and 2. are mutual exclusive if the forum comes first but it should not be polluted for standard support.

Anyway I hope there is a solution because also personally - coming from academia and being a heavy user of ImageJ and others back then - I strongly believe that APEER is a good fit and can become a really valuable tool here.

@gabriel: Towards the questions if are we open source and why do you need to register.

Registration is free and APEER in the current version is free to use and there always will be a free version.
You need to register because because we have a user management that will also still grow with future updates. You need your own account so you can build your own modules and also keep your own data to yourself. Everything you do is private to your account until you choose to share it with somebody.

So for example if you add functions (in the form of modules) to APEER they are only available to you in your account. Maybe you want to develop and test the module to a certain quality level or maybe you want to publish a paper with your algorithm first. Once that is done you can publish your module for everybody to use and only then it will show up in the public area so that everybody can use it for other workflows.

Your module always belongs to you! The open source aspect is that you can decide what to do with the module when you make it available to everybody outside your personal account. You can “just publish” it and share the source somewhere else (e.g. Github). You can publish it and also open source it so everybody can clone the module’s source directly. This feature is in development and will come soon.
Or you can decide to keep it closed source.

In general APEER is just starting so a lot of the functions will come as we continuously develop the platform further. So things like sharing results with everybody or only specific persons will come. These functions require user accounts to also support user and group management. But in general we are very interested in engaging into the discussion of where to take the platform.

Open sourcing modules will come as a platform feature but is still under development.

The platform infrastructure will not be open source at this point but how things work are not a secret we use docker for the modules and kubernetes as orchestration tool. The main point here is that the actual functionality - the modules and how they work - will be open source for every module that the creator chooses to do so.

The point in time where APEEP will add commercial options are for future features such as dynamic up-scaling of resources. But I am sure that everybody understand if you run a workflow on APEER and request the use of multiple GPUs or a large number of CPUs in parallel that these infrastructure costs are then transferred to the user. Same as for more storage space if you wish to keep more data in your account. The advantage being you can dynamically request more resource depending on your needs similar to other cloud offers.

Hope that clears some more things - but if I can shed more light please do let me know.

Best
Robert


#7

So a solution for the support should be separated because your point 1. and 2. are mutual exclusive if the forum comes first but it should not be polluted for standard support.

Yes this is exactly what I wanted to point out.

Anyway I hope there is a solution because also personally - coming from academia and being a heavy user of ImageJ and others back then - I strongly believe that APEER is a good fit and can become a really valuable tool here.

I think we need no solution …

Regards

Herbie


#8

Thanks for clarifying.

“The platform infrastructure will not be open source at this point but how things work are not a secret we use docker for the modules and kubernetes as orchestration tool.”

So it is not really correct to say “Yes” when you posted:

"Is/can APEER be open source?
* Yes, one of the next features that are going to be released will enable Module creators to decide if they want to share the source code of their Module(s) with every user. As mentioned above the versioning system is GIT so if you choose to open source your module other user can directly clone the source code with GIT."

Nothing wrong with developing a commercial/ closed source package but the above could be read as a misleading statement. The “Yes” is not about Apeer, but about the procedures that users generate with it?

What about other platforms? Linux, Mac,


#9

Hi Gabriel
That is correct the underlying foundation is not open source because we are also relying on other software packages. However this is only the website itself and the “engine” that allows us to run different modules together as one meaningful workflow for data processing.

On its own such a system can’t do anything the functionality and usefulness comes from the modules we have on the platform. The modules (basically docker containers) provide the actual programs and algorithms that do the work. These can be open source if the owner of such a module decides so. That is the reason why I APEER supports open source. The module level is also where you want to share functionality and source code of how things are done because that is what processes your data.

Regarding other OS, APEER runs in the cloud through a website you can use it from any computer with internet access and a supported browser.

Also APEER does not care about your data you can process anything given a module for this exists.

If there is no Module, add one yourself, anything you can put and run in a docker container we can run on APEER. So no lock-in to a certain OS, programming language, or image/data acquisition system.

Hope that helps

Best

Robert


#10

Hi all,

one additional reason (IMHO) why I think APEER embraces the idea of open and reproducible science is the following:

  • Do your research and develop your code wherever you want.
  • Once you are ready to publish, put your tools into APEER modules or a complete worklflow, which is still private at that point in time.
  • When entering the review process, the reviewer can get access to your modules or workflows and your code to try it on the platform (no need for installations etc.)
  • When published, your open-source modules and workflows can be used by everybody to reproduce what you did or even try to reproduce it with their own data.
  • And keep in mind that your are not limited in anyway to include any other data processing and analysis steps that have nothing to do with image processing, which might be required for a publication.

I honestly think that APEER can be very good way to publish open and reproducible science with the benefit of being able to run all that code easily (even by non-coders), which is a plus over having it on GitHUB etc. only.

So APEER is as open as you want it to be.


#11

I want to be able to reproduce APEER workflows on systems not under the control of a specific commercial entity, such as my university’s cluster computing infrastructure. If APEER itself were open source and installable onto self-hosted systems, that would be possible, no? I.e.: a similar business model to projects like ownCloud and GitLab and Discourse: open source projects that also offer commercially hosted solutions. APEER could be commercially successful on commercially hosted hardware in the same way, while preserving full reproducibility on self-hosted APEER systems as well. Without this capability, APEER is not truly open and not truly reproducible, even if all individual modules of the workflow are open source, since it would not be possible to run them outside of a closed infrastructure.


#12

Hi Curtis,

You touched upon an interesting point.

I understand your arguments, especially the one that you want to be able to run the whole system on your local infrastructure. This is technically possible and might come soon as a feature depending on your feedback.
So are you afraid of the possibility that for whatever reason the commercial entity will not exit anymore
But this has nothing with the reproducibility itself since Apeer is powered by DOCKER and does not control any algorithm itself.
And consequently to be fully reproducible every device, every firmware and every piece of software including operating systems must be fully open. But is that feasible? I do not really think so.

So your main concern is the availability of an on-premise solutions that run on a infrastructure controlled by yourself, right?

Having said this I still think APEER can be an important tool towards open and reproducible science, even when running on proven cloud infrastructures.


#13

That is great news.

The APEER servers could fail. Or they could become temporarily inaccessible. Even with perfect uptime, it would not be possible to reproduce analyses without network access.

The decision-making entity behind the APEER infrastructure could decide that APEER is not profitable enough and that therefore users will need to start paying an annual fee to retain access, or pay based on their CPU usage.

The above concerns are directly tied to the current and future reproducibility of the system. As long as a closed infrastructure is required to execute the workflow, that closed infrastructure is an impediment to open reproducibility. That said, the fact that APEER is built on Docker is great, and will hopefully make it technically feasible to enable self-hosted infrastructures more easily.

Another concern is privacy. Even if the entity running the infrastructure is fully committed to it being free forever, and fully able to deliver on that commitment, the entity is in a position to gather detailed statistics about exactly who runs exactly which modules under exactly which circumstances. Several years ago, the ImageJ2 team tried to add a core anonymous usage statistics plugin to gather information about which plugins are popular, as an aid to developer prioritization of work, and the community reacted violently negatively. Some of the feedback was so irrational and emotionally charged that we decided to immediately remove the feature, rather than engage the FUD being spread. The APEER system, by comparison, can collect statistics that are not anonymous, not optional (neither opt-in nor opt-out), and not aggregated, with every module execution of every workflow potentially uniquely logged and stored.

The ability to install APEER and execute workflows on self-hosted infrastructure would avoid this very thorny issue.

Please correct me if I am misunderstanding how APEER works.

I do think you can achieve an end-to-end reproducible system in the real world, for example built on Linux. And many scientists do. Regarding closed firmware, sure, but these hardwares operate on common standards—e.g. Intel and AMD both use x86 instruction set; various graphics vendors all support OpenCL.


#14

From my point of view, this thread is a clear manifestation of one of the concerns posted in the Formal policy thread, which is that allowing closed-source, commercial software in the navbar or even a privileged “drawer” amounts to an announcement that the forum is happy to provide free advertising spots.

Personally I don’t think we should provide free advertising. Whether the admins want to provide paid advertising spots is another story.

I tried to find out, but I was asked to register. Why?

My thoughts exactly. (Same problem with e.g. Intellesis.)


#15

Hi, as already explained Apeer has a user management system and some other feature. And it is a web service. Therefore you must register Ike you would need to do for many other things as well.

But what has this do with Intellesis, which is a normal module in a commercial SW, which offers a free trial license? Of course one needs to register to download and test it.

Yes, of course Apeer will store some data (securely) etc, but does this mean everything commercial is bad per se? And as explained by Robert Kirmse already, it is not about getting free advertising space.


#16

in a commercial SW, which offers a free trial license? Of course one needs to register to download and test it.

There are many, many commercial software products that allow a free trial without registration. Let alone let you see the price without registration.


#17

Dear all
thank you for the good discussion so far.
I want to make a general point before commenting on some of the questions above.

We as “Team APEER” did initiate this discussion because, even though we are a commercial entity. The project itself is committed to open science and is at its core free. A basic service will always be free and accessible. However I did mention that certain aspects will become paid services in the future. This will focus on parts of the infrastructure such as purchasing more storage space in the cloud, access to more CPUs or GPUs or eventually commercial modules provided by 3rd party partners. Again a base access with the ability to run every free module etc will remain free.
But I am sure everybody can understand that if we enable the feature to request more resources dynamically and you want to run e.g. a workflow with 100 CPUs this cannot be free anymore.

Since all modules created by - a hopefully growing community - will stay property of the respective individuals. The modules will be open source, with all the restriction of the type of software license chosen by the module creator. So we felt that discussion the topic to be part of the community here is valid. We can for sure host a separate discussion forum for all users of APEER, but believe this is counterproductive in a scientific world that tries to grow closer together, to educate new images processing users, establish standards, etc…

So even if the majority here feels that it might not be a good idea to feature APEER directly in the image.sc forum no worries we also understand that. However in this case it would be nice to link both of our forums so connecting dots for image processing problems will be easier.

@ctrueden, I am not sure how to conclude this discussion at some point. I hope to hear some more voices yet. Obviously I personally - coming from and still feeling very connected to the academic world - believe it would be very beneficial to have a direct section here for APEER but feelings are more mixed I guess.

Best
Robert


Regarding individual points raised in the previous posts.

Yes APEER in the back-end is closed source. Features like our “Worfklow Engine” - the part of APEER that lets us run workflows based on the individually connected modules is not open currently. However because of numerous requests during early interview we will offer different flavors of APEER in the future. That means you won’t be restricted to cloud only but can install the “Worfklow Engine” on your local cluster or use a hybrid solution where you can run parts of a workflow on your local hardware. This is an active work item and will take some time to fully develop and make available.

Towards APEER not being fully open. We think it satisfies the definition of open source because the modules can be open source and you can exactly see what is happening. If a module is open source no one prevents you from installing a docker system and run the module locally to see what it does. This is possible even now and in fact we use it to debugg modules locally during development before checking them into APEER. So you can check a module out for yourself no problem. You can even distribute your module by other channels, say dockerhub, in addition to APEER.
One thing to mention here you cannot conveniently download the module source right now, since we don’t have the feature live yet. In the next weeks we will add the feature that a creator of a module can also publish the git link directly on APEER. This feature is under development and will come soon.

The privacy issue - you are correct that APEER being a web-service will produce a lot of metadata about its users, same as every other webservice. All I can say here is that ZEISS being a German company will stick to one of the strictest data privacy laws out there to keep data safe and protected. However every service will generate metadata and basically even the admins here are in a position to learn a lot about the image processing community as a whole by being able to analyze who talks to whom and about what, which topics are “hot” etc. Especially now with the combined forum. I mention this not to belittle online privacy and protecting it just to set the perspective. Even ImageJ/Fiji now with its automated update service can collect a lot of info of its distribution etc. Again data privacy is important and APEER is honoring German data privacy laws which should be more than sufficient.

Advertisement - I saw the question raised and to emphasize again we initiated the discussion not to have free advertisement here but we want to be part of the discussion about processing algorithm, best practices etc. directly where it is happening and not opening yet another channel that you have to visit.


#18

Sorry for the delay in reply, @rkirmse.

Based on the conversation here, I think a good next step would be to reply back again—or start a new thread renewing the discussion—once APEER has a fully open-source release. From my perspective, that would fully address the concerns above.

In the meantime, I want to reiterate and emphasize that APEER is very welcome to use this forum as its main discussion channel, or as one of its primary recommended discussion channels. And the APEER documentation can link directly to https://forum.image.sc/tags/apeer if you wish. The only difference from the “partner” projects is the lack of an entry in the top navigation bar.


#19

No worries
thanks for the reply I will let you all know here when we have the open source option released. From the current plans it should coming with the next release or the one after that. And we will then start to enable this feature reversely on most of the modules we build based on open source libraries.

We will direct users to this forum and will encourage using the link you suggested.

Best
Rob


#20

Dear Forum and @ctrueden,
I wanted to revive the discussion somewhat. As promised we had a couple of smaller and a major update on the APEER platfrom that introduced new features.

  • as promised already we now have the option for module creators to open source their modules once they choose to publish their creations for everybody to use.

In addition we have the first sharing options available:

  • so users can share their private modules with selected accounts on APEER so that you can use modules together with friends and colleagues before they are public to everybody
  • also user can share their workflow results with anybody (no account needed) be creating a public link to a specific workflow results page.

So especially with the new option to have open source modules on the platform we hope to make steps in the right direction to rekindle the discussion of becoming a full memeber of the community.

Best
Robert
TEAM APEER