Which platform combination is best for digital pathology?

Hi!
I am helping a group of pathologists to establish a new digital workflow supporting their day-to-day clinical work. It is a pilot to help them gain sufficient experience with the new technology based on which a robust new digital workflow can be designed and implemented.
Their digital work would entail: WSI analysis and annotation supported by pragmatic ML algorithms (cell segmentation, classification, counting, etc.). They would need to store and share WSIs (raw and annotated) as well as handle patient and treatment context data (unstructured) in combination with the WSIs.

Key aspects for the platform selection: provide effective team collaboration, cloud-based storage and computation capability, best of WSI image analysis and annotation functionalities, broad set of ML plug-ins, scalability, case management (case allocation, status tracking, mapped folder structure).

I would like to hear the Community’s recommendation on the ideal open source platform combination to be used for this pilot. My shortlist currently consists of QuPath, Orbit and Cytomine (one of the 3), all supported by Omero as a data management platform. Potentially considering Ilastik as a plug-in for specific ML training tasks.

What platform combination would you suggest, and why?
I appreciate your inputs.

Thanks,
Adam

6 Likes

Dear @afpro,
thanks for your post, I guess all platforms you mentioned are very good.
As author of Orbit I don’t want to judge which one is best. And probably there is no “best” but with different focus. The beauty of open-source solutions is that you can also easly use several in parallel.
The tasks you mentiond can be solved by Orbit, here you find the corresponding tutorials:
http://www.orbit.bio/help/
Please let me know if you need further assistance on this.

Regards,
Manuel

2 Likes

Thanks Manuel.
Yes, I arrived at the same conclusion that those 3 platforms are very similar in capabilities (based on the documentations I saw). That is why I am not sure which one to pick. And as you also pointed out, they may have slightly different focuses or some nuances in their functionalities which may however be relevant for us.

Would you please share your perspective in how Orbit is different from the other two?
For example, I read in an external article comparing the platforms (https://medevel.com/10-os-whole-slide-image/ and https://medevel.com/orbit-digital-pathology/) that while Orbit is “100 steps forward for digital pathology”, they “would recommend other software for pathologists”. The other two are referenced as “lightweight”. It is hard for me to judge based on this what the pros and cons of Orbit for our situation may be.

Can you please elaborate on what you consider Orbit’s strenghts and potential weaknesses from our point of view?
I hope to get some similar insights from the developers/users of the other two platforms as well.

Many thanks,
Adam

1 Like

As QuPath’s creator, I also don’t want to judge - but I’d say QuPath and Orbit are quite comparable, whereas Cytomine is quite different in its aims and scope (and overlaps with OMERO?).

But because they are all freely available and open source, your pathologists could try them out to find their own preferences. Or, in QuPath’s case, sign up for the webinar next week through http://neubiasacademy.org :slight_smile:

6 Likes

I wanted to add two things:

  1. It’s worth looking into how many other people have done similar analyses to your own in the past (the subforums for each software might be a good place to start there).
  2. It might help to add a bit of information about what types of analyses you want to do. I think all software does some kind of cell segmentation, but certain software will do better with weird cell shapes like neurons, while others handle millions of objects better. Also, whether you have anyone on hand who can code (at least at an introductory level) might make you lean towards or away from more flexible platforms (usually the flexibility is in the coding).

Identification of superstructures (glomeruli etc) might require some sort of deep learning integration.

Multiplexing for 5+ markers can be difficult without an interface designed to automatically handle multiple class assignments (6 markers is 2^6 or 64 potential class combinations, if each marker could represent one base class).

Do you need data visualization to be built in or can you use something else for that?

And as you pointed out, how well does it handle automatically importing or classifying metadata?

I’m not sure you will find many people who have used all three platforms in depth, so it might help to have a specific list of questions for your project (you mentioned some things you generally wanted, but the devil is in the details–Visiopharm handles multiplex brightfield, but you have to define EVERY class combination manually), and ask users of each software to describe their experiences with each. I’ve found that there aren’t a huge number of users who check the forum regularly; it seems to be more of a “when I need help” basis, so you may be better served trying to hunt down people who have posted :slight_smile:

Best of luck

1 Like

Hi Adam

We have developed few guides describing how to integrate some analytical tools and OMERO, it includes Orbit, QuPath, ilastik amongst others [1]. We use them during our regular training sessions.
Click on any icon representing a tool, this will redirect you to the guide for the tool.

As @petebankhead pointed out, Cytomine is different. It has its own data management system/database etc… Data in OMERO will have to be re-imported into Cytomine in order to be analysed. There is currently no bridge between the two.

Cheers

Jmarie

https://omero-guides.readthedocs.io/en/latest/

2 Likes

Hi @afpro,

As I am Cytomine’s initiator, it’s also hard to give an unbiased opinion, however I tried to somehow distinguish open initiatives (including others you might be interested in) in a (too short) paper last year: https://www.frontiersin.org/articles/10.3389/fmed.2019.00255/full

Please note Cytomine is not relying on OMERO. We have our own databases with extended access rights management with extensive collaborative features through web services and user interface. Latest developments allow multiple users to access the same images within organized and secured projects, annotate them with metadata and semantic information (at the image or ROI level) and build large shareable annotation datasets, compare annotation agreements, execute algorithms directly from the web user interface in a reproducible way through container technology (so any external software can be used incl. imagej, ilastik, cellprofiler, any deep learning libraries,…) on HPC, and review their predictions online, synchronize viewing in real-time over the web, etc.
For slide format reading, Cytomine is using OpenSlide and Bioformats conversion can also be activated to maximize supported formats.
Cytomine is highly interoperable with a Restful API and I think it would be technically feasible to connect Orbit and QuPath for specific desktop analysis using specific algorithms not available in Cytomine (but nobody tried to develop such a bridge as far as I know). For example, the Icy team at Pasteur institute developed a bridge to Cytomine using HTTP requests, so Icy end-users can also analyze their data within Icy and send back results (e.g. annotation predictions) to Cytomine server for sharing.
However, I personally think individual desktop analysis do not fit all use cases, it’s very good to efficiently quantify images and publish papers but I think it might be limited in terms of collaboration.
FYI, Cytomine is also heavily used in education settings with thousands of users, your pathologists might be interested by a common platform for research/education.
To be honest, I think none of these platform will fulfill all your needs. A “day-to-day” solution for specific clinical routine tasks is something else than a software for research projects, therefore extensibility and support/development/maintenance dedicated services might also be something to consider.
As previously suggested, the best way is to test these tools, with the help of people who know how to use them, and analyze how to extend them.
Please feel free to contact us. We can provide a Cytomine online demo account if you are not able to install it locally following our documentation (https://doc.cytomine.org/).

2 Likes

Thanks Raphael (@maree) for your answer.

I have looked at the user guide and Cytomine indeed seems particularly strong in faciliating collaboration. The key question for me is how the built-in algorithms (softwares) in Cytomine compare with the key functionalities of automated image processing operations currently available in Qupath and Orbit. What are the trade-offs in giving up productivity-boosting annotation features for superior collaboration capabilities.

How can I best get a full list of the available software packages? I guess the easiest will be to get access to an online demo account and test those myself. (I will reach out to the team on your website to get one.)

Thanks for your response.
Adam

Thanks @Research_Associate! Very valid and useful points to consider.
It seems that there is no shortcut answer and I need to invest more time to test these platforms myself in order to be able to identify the nuances which differentiate them from each each other.
Thanks for your response!
Adam

1 Like

Thanks @petebankhead! I will definitely check out the webinar materials next week. I guess the recordings will be available following the event? (I live in a different time zone.)
Thanks
Adam