Software for Whole Slide Image (WSI) Analysis (with API to add Deep Learning models)

Hello,

I’ve been playing with the CAMELYON dataset of WSIs and I’ve trained a neural network to predict with degree of certainty if a tile from the given virtual slide has metastasized cells. Combining all predictions over the entire slide with a threshold cutoff produces a mask depicting ROIs that the model thinks deserve attention:

I would like to have an integrated view of the virtual slide, model predicted annotations, be able to edit or add new annotations and labels/metadata so I can assess the apparent quality of the model and also share it in simple UI with others, instead of just sharing the TensorFlow model/weights with Jupyter notebooks.

Essentially, I’m looking for a software/suite that has at minimum the following:

  • A WSI Viewer
  • Annotation tools
  • An API to allow programmatic access to image tiles, annotations, metadata, etc of the image plus the ability to add update them
  • Has good documentation and community support
  • User authentication and being Web-based tool is a plus

I’ve searched and come across the following packages that seem to fit my requirements but I’ve had issues with:

  • Cytomine - this one is quite impressive as it boasts a lot of cool features however, I’ve found the documentation for it to be quite sparse and I do not wish to pay for consultation services given that this is not a commercial endeavor and moreover Cytomine is supposed to be an Open Source project. (I was able to install the docker containers but got stuck when trying to upload an image; couldn’t find a solution to this issue online, yet). The devs have created a wonderful behemoth of a Software that overwhelms me due to my lack of programming experience. If there was more community support for it I would love to have used it (a dedicated moderator could perhaps help jumpstart one)
  • QuPath - I haven’t spent a lot of time with this package since it is not geared towards WSIs primarily and the API wasn’t very clear to me (I use Python and prefer if the API was language neutral like REST)
  • ASAPAnnotation - quite straightforward but lacks documentation and an API that would allow me to add and run my model
  • OMERO - still investigating it but at the moment unable to view a ~2GB image (something viewers like Openseadragon are able to do quite seamlessly - I’m probably missing something here or it may not be optimized for WSIs…?)

My hopes for creating this topic are to save time (for myself and others) in searching for the right Software and help move my project forward.

Thank you for your time.

You may also wish to check out Orbit (on this forum here).

1 Like

Both Omero and QuPath are designed for whole slide images, it is in the description for both. Though most whole slide images are pyramidal, so maybe you are encountering performance issues if your images are not formatted well? Both upcoming versions of QuPath and current versions of BioFormats have pyramidal ome-tiff builders.

1 Like

…and some may not be pyramidal in the same way. For some of the CAMELYON16 images I can open them in in QuPath if OpenSlide is the reader, but not if Bio-Formats is the reader. Whereas for other pyramidal files it is often the other way around…

In assessing WSI support, you might wish to first try more ‘standard’ images, e.g. some .svs files (which are TIFF-based). There are some samples with both OpenSlide and Bio-Formats, e.g.

If these work, but not the CAMELYON tiffs, it may be the non-standardness of the latter that is the issue.

1 Like

Seconding @petebankhead and @Research_Associate’s answers in general about the WSI support in the Bio-Formats/OMERO stack. I also briefly looked at some of the CAMELYON sample files. Looking at the data layout, the dataset seems to be stored using Philips TIFF-based file format which is only natively supported by OpenSlide at the moment - see https://openslide.org/formats/philips/.

As mentioned before, to maximize interoperability and allow you to be able to audit and use various tools effectively, we strongly encourage using open exchangeable file formats like OME-TIFF rather than PFFs whenever possible.

3 Likes

Hi,

Regarding our Cytomine software: thanks for your comments, we put a lot of efforts developing it since 2010 and we have language neutral API (REST) and software architecture to integrate Deep Learning algorithms. Cytomine is indeed open-source, with a permissive license (minimum requirements on redistribution) so you can work with it on your own without paying anything, and we do our best to help users having issues.

We were not very active on image.sc forum till now (we get and reply to many requests by email) but we will do our best to improve that.

We have currently different documentation servers and will merge and reorganize them soon:
https://doc.uliege.cytomine.org/ (R&D documentation ULiège)
https://doc.cytomine.org/ (official base version)

Please also note I published a mini-review paper on Open Practices and Resources for Collaborative Digital Pathology that might contain other useful resources for digital pathology people: https://www.frontiersin.org/articles/10.3389/fmed.2019.00255/full