PyPi package to convert a TensorFlow2 models into a czmodel for usage inside the ZEN ecosystem

Hi all,

As promised last week on the NEUBIAS meeting (which was amazing :+1: ) we just released the 1st public version of: PyPi package - czmodel

It allows any external party or to convert/package their trained TensorFlow2 models into a czmodel , in order to use it inside the ZEN ecosystem with support for GPU, Tiling and Integration with Image Analysis.

Motivation: ZEISS offers to create networks on demand as a paid service (custom solutions) Therefore we need an easy way to import trained models in ZEN.
Knowing that some or hopefully many of you create and train your own networks we think it makes sense to allow everybody to use their models in ZEN to benefit from support for GPU, Tiling, multi-dimensional data sets and the integration in our Image Analysis Framework (if you or your user like our software platform). If not, let us now why and happily use your model in your favorite environment … :slight_smile:

The first supported python-based ML service is using networks for Segmentation. Other ML services (Image-2-Image processing, Classification etc.) will follow (based on feedback and need).

!!! Important !!!: In order to import and run such a model, one needs to have ZEN blue 3.2 (to be released soon) or ZEN core 3.1 (to be released) and the Intellesis module. The reasoning to publish the package earlier is to get feedback and to allow people to start training their networks early on … :-). So stay tuned.

Feedback and Suggestions are welcome.

2 Likes

Hi,

This sounds great. Do you have any plans for similar faciliation of trained models from Pytorch/Torchvision?

Kind regards
Fredrik Olsson

Hi Fredrik,

Thanks fro your feedback. This is a good questions and the honest answer is that we did not decide on this yet. Our whole training backend so far build on TF1 and since recently TF2.

Obviously adding support for PyTorch for at least inference is possible. It is just a matter of the need vs. benefit approach (which is what I have to justify when asking for money … :slight_smile: from my boss).

So button line here is: If we see that the community really like this idea, but PyTorch is somewhat a must-have feature, that the chances are good. But it all depends on your feedback.

From your perspective, what is the mean reason to add (or even favor) PyTorch / TorchVision over TensorFlow 2 + Keras?

Hi,
From our perspective when it comes to Pytorch/Torchvision its just ease of use while training. However, using Intellisis directly in Zen instead of Pytorch/Torchvision, onnx export, and finally ML.net in the IronPython Zen environment would be a great step forward…

Cheers,
Fredrik

Hi Fredrik,

currently our ML services are using our ZeissPy (customized Anaconda distribution). ZEN send chunks of data via REST calls to this independent segmentation service. Every Zen install comes with this ZeissPy

In contrast inside ZEN we use IronPython to automate ZEN (which is c#) to access all the devices etc. Do you mean you would like to use trained networks directly in IronPython?

Our idea so far is rater to use the models or networks as they are. One can use a Trained TF2 model in a jupyter notebook or … And when needed this model will just be packaged with the required meta-information and can be used from within ZEN (even the the execution is actually done via ZeissPy) without converting it. Does this make sense or did I got your remark wrong?

Hi Sebi,

“In contrast inside ZEN we use IronPython to automate ZEN (which is c#) to access all the devices etc. Do you mean you would like to use trained networks directly in IronPython?”

Yes. We have trained models from Pytorch/Torchvision. We also have inference in IronPython based on ML.Net. We are about to see if it may be implemented also under ZEN/IronPython for real-time inference if the requirements on speed holds.

“Our idea so far is rater to use the models or networks as they are. One can use a Trained TF2 model in a jupyter notebook or … And when needed this model will just be packaged with the required meta-information and can be used from within ZEN (even the the execution is actually done via ZeissPy) without converting it. Does this make sense or did I got your remark wrong?”

Yes it does. However, to be honest, living under the IronPython environment in ZEN it has become more natural to make speed-critical parts in C# and avoid otherwise nice and user-friendly solutions based on Jupyter/Numpy etc.

Cheers,
Fredrik

1 Like

Hi Fredrik,

interesting. We also use IronPython inside ZEN for the Automation itself. At least so far when segmenting an image we call our SegmentationService (Python) via the REST-API and did not try ML.net from with ZEN. But I will have a closer look for sure.

Additionally ZEN will also offer to integrate Docker Containers (incl a UI) from the APEER platform in ZEN, which might be also interesting to explore.

But if I got you correctly, I want real-time inference aka Online-Image-Analysis . ZEN offers Online-Image-Analysis already but not yet using inference. If you want to discuss the details, please contact me via message.