Cytomine - how to access algorithms from GitHub

Hi

I have downloaded Cytomine v3.0. Currently it shows only 4 algorithms accessible from the web UI (I reckon these 4 are established through the Trusted Source setting feature).

As per the documentation (https://doc.uliege.cytomine.org/display/ALGODOC/Existing+algorithms+in+CYTOMINE):

"The Cytomine Data Mining Python package includes all algorithm except the image retrieval algorithm that is implemented in Java.

The Cytomine-DataMining Python package comes with the automated Docker Installation in the software_router container"

In the referenced datamining package (on GitHub), there are currently 10+ available algorithms. Yet, these are not accessible from the webUI (in contrast to the above statement which suggests that they should be automatically installed with the version.)

Can you please help me with a precise list of instructions of how I can link the available algorithms on GitHub to my webUI and start using them as algorithms to my projects?

It feels that the documentation is out-dated in some parts and may need to be reviewed and updated.

Thanks for your help.
Adam

Hi Adam,

If you refer to Cytomine v3.0, your refer to the main official version, developed and maintained by the Cytomine cooperative (https://cytomine.coop) and which documentation is available at https://doc.cytomine.org. The repo of this main official version is https://github.com/cytomine/

The documentation you cite ( https://doc.uliege.cytomine.org ) is the documentation of the ULiège fork, currently in v2.0.0-rc.5, developed by the Cytomine research team at the Montefiore Institute of the University of Liège (https://uliege.cytomine.org), which includes all the latest developments of this research team, including their AI algorithms, and which have to be considered as a Research & Development fork. Their repo is https://github.com/cytomine-uliege/

These two versions are two different forks, even if the two teams work together to try to be the more close as possible. Recently, the ULiège team have adopted and adapted the open source front end developed by the cooperative, and the cooperative have adopted and adapted the algorithm managing system developed previously by the ULiège team. But we have our own agenda, driven by research grants for ULiège, and driven by the market and our customers for the cooperative. So, for now, an algorithm available for one version may not work in the other.

In the official repo of the cooperative there is actually only few open source algorithms available, because as algorithm management have really changed between v3 and previous versions, they will be adapted progressively. The other reason is that the Cytomine cooperative is currently more focused on providing an open source AI development and running environment for those who want to develop their own algorithms, than to produce a collection of routinely usable open source algorithms. The open source algorithms in the cooperative repo are then mostly there as examples to develop your own algorithms than to be routinely used out-of-the-box.

The ULiège research team produce more open source AI algorithms than the cooperative, because doing open source AI algorithms and promote them publicly in scientific publications (https://uliege.cytomine.org/#publications) is their core business. But there also, their algorithms have mainly been developed to answer their own research questions, not to provide a collection of routinely usable algorithms independently of the scientific context. Their algorithms are then also to be considered as good open source code examples, and as basis to develop your own algorithms, fitted to your own needs.

In both case, cooperative and ULiège, it’s only algorithms source code. There is no trained model provided. So you will need to train these algorithms on your own learning dataset before being able to use them for quantification or diagnosis.

Note that to be integrated in your Cytomine using the new trusted source feature, algorithms must be published on Github AND Dockerhub, AND have a correct descriptor. That can explain why you may found more algorithms available on a Github repo than those which will be imported in your Cytomine when adding this trusted source.

I hope that these few information will help you to better understand how the Cytomine community is currently organized, and why Cytomine does not come with a larger collection of algorithms out-of-the-box.

Best regards,

Grégoire