Software for Whole Slide Image (WSI) Analysis (with API to add Deep Learning models)

Hello,

I’ve been playing with the CAMELYON dataset of WSIs and I’ve trained a neural network to predict with degree of certainty if a tile from the given virtual slide has metastasized cells. Combining all predictions over the entire slide with a threshold cutoff produces a mask depicting ROIs that the model thinks deserve attention:

I would like to have an integrated view of the virtual slide, model predicted annotations, be able to edit or add new annotations and labels/metadata so I can assess the apparent quality of the model and also share it in simple UI with others, instead of just sharing the TensorFlow model/weights with Jupyter notebooks.

Essentially, I’m looking for a software/suite that has at minimum the following:

  • A WSI Viewer
  • Annotation tools
  • An API to allow programmatic access to image tiles, annotations, metadata, etc of the image plus the ability to add update them
  • Has good documentation and community support
  • User authentication and being Web-based tool is a plus

I’ve searched and come across the following packages that seem to fit my requirements but I’ve had issues with:

  • Cytomine - this one is quite impressive as it boasts a lot of cool features however, I’ve found the documentation for it to be quite sparse and I do not wish to pay for consultation services given that this is not a commercial endeavor and moreover Cytomine is supposed to be an Open Source project. (I was able to install the docker containers but got stuck when trying to upload an image; couldn’t find a solution to this issue online, yet). The devs have created a wonderful behemoth of a Software that overwhelms me due to my lack of programming experience. If there was more community support for it I would love to have used it (a dedicated moderator could perhaps help jumpstart one)
  • QuPath - I haven’t spent a lot of time with this package since it is not geared towards WSIs primarily and the API wasn’t very clear to me (I use Python and prefer if the API was language neutral like REST)
  • ASAPAnnotation - quite straightforward but lacks documentation and an API that would allow me to add and run my model
  • OMERO - still investigating it but at the moment unable to view a ~2GB image (something viewers like Openseadragon are able to do quite seamlessly - I’m probably missing something here or it may not be optimized for WSIs…?)

My hopes for creating this topic are to save time (for myself and others) in searching for the right Software and help move my project forward.

Thank you for your time.

You may also wish to check out Orbit (on this forum here).

2 Likes

Both Omero and QuPath are designed for whole slide images, it is in the description for both. Though most whole slide images are pyramidal, so maybe you are encountering performance issues if your images are not formatted well? Both upcoming versions of QuPath and current versions of BioFormats have pyramidal ome-tiff builders.

1 Like

…and some may not be pyramidal in the same way. For some of the CAMELYON16 images I can open them in in QuPath if OpenSlide is the reader, but not if Bio-Formats is the reader. Whereas for other pyramidal files it is often the other way around…

In assessing WSI support, you might wish to first try more ‘standard’ images, e.g. some .svs files (which are TIFF-based). There are some samples with both OpenSlide and Bio-Formats, e.g.

If these work, but not the CAMELYON tiffs, it may be the non-standardness of the latter that is the issue.

1 Like

Seconding @petebankhead and @Research_Associate’s answers in general about the WSI support in the Bio-Formats/OMERO stack. I also briefly looked at some of the CAMELYON sample files. Looking at the data layout, the dataset seems to be stored using Philips TIFF-based file format which is only natively supported by OpenSlide at the moment - see https://openslide.org/formats/philips/.

As mentioned before, to maximize interoperability and allow you to be able to audit and use various tools effectively, we strongly encourage using open exchangeable file formats like OME-TIFF rather than PFFs whenever possible.

4 Likes

Hi,

Regarding our Cytomine software: thanks for your comments, we put a lot of efforts developing it since 2010 and we have language neutral API (REST) and software architecture to integrate Deep Learning algorithms. Cytomine is indeed open-source, with a permissive license (minimum requirements on redistribution) so you can work with it on your own without paying anything, and we do our best to help users having issues.

We were not very active on image.sc forum till now (we get and reply to many requests by email) but we will do our best to improve that.

We have currently different documentation servers and will merge and reorganize them soon:
https://doc.uliege.cytomine.org/ (R&D documentation ULiège)
https://doc.cytomine.org/ (official base version)

Please also note I published a mini-review paper on Open Practices and Resources for Collaborative Digital Pathology that might contain other useful resources for digital pathology people: https://www.frontiersin.org/articles/10.3389/fmed.2019.00255/full

2 Likes

Actually, at Objective Pathology, we’ve supported Philips TIFF format and similar zoomable TIFFs natively since 2005, more than a decade before OpenSlide or Bio-Formats even existed.

1 Like

Is your support for the format open source so that it can be integrated into other software? I wasn’t aware of it until now.

No, our source is not open, but we do natively support Philips TIFF. It is essentially quite similar to other pyramidal tiled TIFF formats, including our own ZIF (open spec at zif.photo, no open source yet but that will be coming). OME-TIFF zoomable images appear to have been modelled heavily after ZIF, as two OME consultants queried us about the format extensively prior to publishing their OME-TIFF paper, for which they have received much credit.

That is awkward, but it happens sometimes :confused:

Hi @makatx,
Have you succeed now to have a fully functional instance of Cytomine ?
If you have technical issues, the definitive best way to have help is to post your issues on Github.
If you don’t know on which repo to post, please consider the Cytomine-core one.
Best regards,
Grégoire

Hi Kemp-

That’s a pretty serious and important issue that would be good to understand. Shall we try to address and hopefully resolve together privately and then report back here? Alternatively, we can review all the docs and threads, probably on a separate thread here on image.sc, and try to come to an agreement on what happened and how we go forward. Let us know what you think.

Cheers,

Jason

3 Likes

Thank you for the reach-out, Jason. Let’s have a chat offline; I’ll be in touch next week,

Kemp

2 Likes