QuPath with Tensorflow: Add new user models

I am training some WSI prediction models using PyTorch/Tensorflow.
I would like to know how difficult would be to adapt the StarDist/Tensorflow extension to support different models. CPU support would be enough.

As an example, there is the PanNuke model that performs nuclei segmentation but classifies between several nuclei types.

I would be great to have this kind of tool for prototyping inference models!

We’re working on the QuPath infrastructure to make this possible / as straightforward as we can, but it will take some time.

In the meantime, I would be really interested in assembling a collection of publicly-available, pre-trained models suitable for histology/pathology that we may use for development and testing. There are plenty of publications and some datasets, but finding actual model files seems rather more difficult…

Do you know is the model for PanNuke available online? I couldn’t find it via the link (although there is the statement Please download the HoVer-Net weights trained on PanNuke here I couldn’t actually find where ‘here’ referred to…).

2 Likes

Yeap. I didn’t find the weights either.

There is another option for the model collection. Since the differences between architectures and metrics would be kind of difficult to code I think it would be easier to train the datasets with some “gold standards”.

I am able to do this on some of the datasets you think would be good to have.

For example, I have a model for PanNuke segmentation using Unet both for 20X and 40X augmentation. They are in PyTorch (actually fastai2 but essentially PyTorch) but I think I could convert them to Tensorflow if you think it could be useful for you.

Ah, yes the lack of pretrained models is troublesome… all the publications raise expectations, but in the end there aren’t implementations ready to use :frowning: Hopefully fuller support in QuPath will help address this and encourage more model sharing… and there is also this to look out for: https://bioimage.io

I like the idea of having gold standard implementations. If any of yours are online I’d be happy to have a look.

There might not be any need to convert them to TensorFlow – in QuPath, the early TensorFlow/StarDist example is really just a starting point for demonstration purposes, but fundamentally QuPath is designed to support alternative frameworks (at least whichever we can get to play nicely with Java) via the same interface. We plan to explore ONNX very soon, but right now we don’t have any terribly worthwhile models in this format to test. If your starting point is fastai2, then I think ONNX export should be easier.

1 Like

Ok, I build a github repo with the code I used for nuclei segmentation using fastai2 and PanNuke dataset. You can find it here.
I will upload the model weigths in the following hours. I used pixel accuracy as metric
I am also playing with fastai2_extension in order to get also the onnx model ready.
I will come back with updates as soon as I get in the repo.

3 Likes

I rebuilt the repo, you can find the new one here.
Also, I added a link to the model. I am still struggling to port it to ONNX

1 Like

Hi Guys,

Very interesting discussion. Is it possible to run Keras model in Qupath. I have a few that I would like to run.

Hi @Max_Qureshi, not currently – see Apply hdf5 in QuPath similar to StarDist possible?