CellProfiler 4.0.0 timeline and roadmap

Howdy,

I want to provide CellProfiler users a timeline (and roadmap) for CellProfiler 4.0.0.

My plan is to freeze (i.e. I’ll stop merging into the master branch of CellProfiler Git repository) in November and release sometime in January (or earlier).

  • Update Python 2.7 to Python 3.8
  • Add type annotations
  • Update wxPython 3 to wxPython 4
  • Update PyZMQ 15 to PyZMQ 18
  • Rewrite the ZeroMQ message queue to use asyncio (Python’s standard asynchronous input/output library)
  • Add (and document) a public API that’ll enable users to use CellProfiler modules and pipelines from their Python packages and applications (or Jupyter notebooks)
  • Add CellProfiler to PyPI
  • Use scikit-image exclusively (including merging missing functions upstream to scikit-image)

It’s possible some non-planned features will ship. I imagine I’ll update some more modules to process volumes and I’ll promote some contributed CellProfiler modules to the official release.

I’m happy to answer any questions and more than happy to help anyone interested in contributing!

13 Likes

This sounds great, I particularly like the move to Python 3 and the public API exposing the individual modules to Python. This will greatly facilitate longer workflows which are currently difficult to maintain with the GUI interface (version diffs, lots of similar modules, refactoring of pipelines).

A couple of comments:

  • A few weeks ago I dug around in the source code to see whether launching of workers for multiprocesses could be simply converted to dask-workers, which would allow for multi-processing on a single PC as well as workers on nodes of HPC-clusters using the same code-base. However, I quickly came to the conclusion that this may not be as trivial as I thought as you seem to have communication between workers using ZMQ. I didn’t look any further.
    I assumed that this may have something to do with keeping the GUI updated and perhaps synchronizing writes to the .csv table.
  • Would it be possible to make bioformats/ javabridge optional dependencies? It appears to me that including them is a major source of problems with packaging and installation.

Thanks!

1 Like

For reading image it is possible to use tifffile package. it generate a litle problem with installation on linux and macos, bu in many cases old version is enough (0.15.*)

What about Python-BioFormats? Any updates planned here as well?