CP 2.2.0 won't start

cellprofiler
ilastik

#1

Hi there,
I’ve freshly installed CP 2.2.0 in an Ubuntu 18.04.1 environment. Apart from the usual ilastik_main warning there is no erroe message, all the modules installed fine, but CP won’t start. I’ve attached the debug report below. I have the same issue on a 16.04 system

python2 CellProfiler.py --log-level=DEBUG
$HOME=/home/anja
matplotlib data path /home/anja/.local/lib/python2.7/site-packages/matplotlib/mpl-data
loaded rc file /home/anja/.local/lib/python2.7/site-packages/matplotlib/mpl-data/matplotlibrc
matplotlib version 2.2.3
interactive is False
platform is linux2
loaded modules: ['numpy.core.info', 'cellprofiler.sys', 'ctypes.os', 'gc', 'cellprofiler.time', 'logging.weakref', 'pprint', 'cellprofiler.preferences', 'unittest.sys', 'numpy.core.umath', 'h5py.version', 'zmq', 'string', 'SocketServer', 'h5py.numpy', 'numpy.lib.arraysetops', 'h5py._hl.sys', 'numpy.core._multiarray_tests', 'json.encoder', 'subprocess', 'numpy.core.machar', 'zmq.backend.cython.utils', 'h5py._conv', 'unittest.StringIO', 'numpy.ma.extras', 'numpy.fft.fftpack_lite', 'matplotlib.cbook', 'multiprocessing', 'dis', 'numpy.lib', 'logging.threading', 'cellprofiler.utilities.utf16encode', '_json', 'logging.io', 'cellprofiler.utilities.sys', 'matplotlib.cbook.warnings', 'numpy.testing._private.pytesttester', 'pyparsing', 'matplotlib.cbook.textwrap', 'abc', '_thread', 'zmq.utils.sys', 'zmq.backend.cython.error', 'numpy._globals', 'numpy.lib.npyio', 'matplotlib.sys', 'h5py.h5fd', 'numpy.fft.helper', 'cellprofiler.logging', 'ctypes.tempfile', 'optparse', 'unittest.suite', '_ctypes', 'exceptions', 'json.scanner', 'codecs', 'numpy.os', 'multiprocessing.multiprocessing', 'StringIO', 'weakref', 'numpy.core._internal', 'distutils.sys', 'numpy.lib.arraypad', 'base64', '_sre', 'h5py.sys', 'logging.sys', 'logging.re', 'select', 'ctypes._ctypes', '_heapq', 'zmq.sugar.constants', 'numpy.lib.financial', 'zmq.backend.cython.time', 'binascii', 'zmq.backend.cython._poll', 'h5py.functools', 'unittest.fnmatch', 'zmq.sugar.stopwatch', 'numpy.polynomial.chebyshev', 'logging.thread', 'cPickle', 'h5py._errors', 'numpy.polynomial.hermite_e', 'h5py._hl.datatype', 'zmq.sugar.threading', 'zmq.backend.cython.cPickle', 'h5py.h5py', 'numpy.core.fromnumeric', 'numpy.ctypeslib', 'matplotlib._version', '_ast', 'zmq.sugar.tracker', '_bisect', 'zmq.backend.cython.codecs', 'encodings.aliases', 'fnmatch', 'sre_parse', 'pickle', 'numpy.random.warnings', 'logging.cStringIO', 'numpy.lib.polynomial', 'numpy.compat', 'cellprofiler.optparse', 'numbers', 'numpy.core.records', 'strop', 'numpy.core.numeric', 'six', 'matplotlib.testing', 'cellprofiler.tempfile', 'ctypes.util', 'numpy.lib.utils', 'encodings.utf_8', 'cellprofiler.multiprocessing', 'multiprocessing.sys', 'numpy.lib.arrayterator', 'zmq.sugar.atexit', 'os.path', '_weakrefset', 'zmq.backend.select', 'unittest.traceback', 'unittest.os', 'h5py._hl.selections2', 'functools', 'sysconfig', 'zmq.sugar', 'numpy.core.numerictypes', 'numpy.polynomial.legendre', 'uuid', 'numpy.matrixlib.defmatrix', 'tempfile', 'numpy.polynomial.laguerre', 'multiprocessing.os', 'multiprocessing.itertools', 'numpy.core', 'numpy.linalg.info', 'unittest.functools', 'unittest.util', 'logging.logging', 'httplib', 'h5py.operator', 'decimal', 'numpy.lib._datasource', 'logging.handlers', 'token', 'h5py', 'numpy.linalg._umath_linalg', 'cStringIO', 'numpy.polynomial', 'numpy.add_newdocs', 'zmq.sugar.frame', 'multiprocessing.process', 'ctypes.errno', 'encodings', 'cellprofiler.zmq', 'zmq.utils.strtypes', 'zmq.backend.cython.struct', 'zmq.sugar.poll', 'distutils.string', 'rfc822', 'numpy.lib.numpy', 'numpy.random.threading', 're', 'math', 'ast', 'zmq.utils.constant_names', 'numpy.lib.ufunclike', 'ctypes.struct', '_sysconfigdata_nd', 'numpy.testing._private', 'matplotlib.json', 'zmq.backend.cython.warnings', '_locale', 'logging', 'thread', 'traceback', 'cellprofiler.h5py', 'zmq.backend.platform', 'multiprocessing.threading', 'multiprocessing.util', 'ctypes.re', 'cellprofiler.threading', '_collections', 'numpy.random', 'zmq.sugar.sys', 'numpy.lib.twodim_base', 'array', 'ctypes.sys', 'posixpath', 'numpy.core.arrayprint', 'types', 'zmq.backend.cython._device', 'numpy.lib.stride_tricks', 'numpy.lib.scimath', 'matplotlib.cbook.functools', 'json._json', '_codecs', 'numpy.__config__', 'encodings.ascii', 'h5py._proxy', 'zmq.sugar.socket', 'copy', 'hashlib', 'zmq.error', 'keyword', 'numpy.lib.nanfunctions', '_warnings', 'posix', 'h5py.h5ac', 'logging.socket', 'sre_compile', '_hashlib', 'zmq.utils.itertools', 'numpy.lib.shape_base', 'numpy._import_tools', 'logging.collections', '__main__', 'numpy.fft.info', 'numpy.sys', 'dateutil._version', 'matplotlib._color_data', 'unittest.result', 'bz2', 'h5py._hl.dataset', 'encodings.codecs', 'unittest.difflib', '_ssl', 'numpy.lib.index_tricks', 'warnings', 'zmq.sugar.cPickle', 'zmq.backend.cython.sys', 'cellprofiler.weakref', 'zmq.backend.cython.random', 'h5py.utils', 'h5py._hl.filters', '_io', 'linecache', 'h5py.collections', 'numpy.linalg.linalg', 'numpy.lib._iotools', 'imp', 'random', 'unittest.types', 'zmq.sugar.context', 'datetime', 'urllib', 'logging.os', 'ctypes._endian', 'encodings.encodings', 'unittest.pprint', 'numpy.random.mtrand', 'logging.stat', '_cython_0_28_3', '_cython_0_28_5', 'h5py.h5ds', 'six.moves.urllib.request', 'numpy.linalg', 'h5py._hl', 'numpy.lib._version', 'zmq.backend.cython.threading', 'ssl', 'numpy.version', 'distutils.re', 'h5py.h5g', 'numpy.lib.type_check', 'bisect', 'h5py.h5a', 'cellprofiler.uuid', 'unittest.re', 'threading', 'zmq.ctypes', 'h5py._hl.six', 'cycler', 'tokenize', 'locale', 'atexit', 'zmq.backend.cython.message', 'unittest.weakref', 'h5py.defs', 'dateutil', 'h5py.h5z', 'h5py.h5t', 'h5py.h5p', 'h5py.h5s', 'h5py.h5r', 'h5py.h5l', 'h5py.h5o', 'h5py.h5i', 'h5py.h5d', 'zmq.sys', 'h5py.h5f', 'ctypes.subprocess', 'fcntl', 'unittest.case', 'h5py.weakref', 'zmq.utils', 'numpy.lib.info', 'ctypes', 'matplotlib', 'json.re', 'unittest.signal', 'itertools', 'numpy.fft.fftpack', 'opcode', 'six.moves', 'numpy.testing._private.nosetester', 'zmq.errno', 'unittest', 'logging.errno', 'numpy.testing._private.utils', 'h5py._hl.selections', 'unittest.collections', 'pkgutil', 'platform', 'zmq.backend.cython', 'logging.struct', 'sre_constants', 'zmq.backend.os', 'numpy.core._methods', 'numpy.core.function_base', 'numpy.compat.py3k', 'numpy', 'subprocess32', 'numpy.ma', 'logging.atexit', 'logging.cPickle', 'matplotlib.cbook._backports', 'cellprofiler.numpy', 'cellprofiler.os', 'h5py._objects', 'zlib', 'multiprocessing.weakref', 'json.decoder', 'copy_reg', 'site', 'h5py._hl.compat', 'h5py._hl.os', 'io', 'multiprocessing.atexit', 'h5py._hl.attrs', 'shutil', 'h5py.gc', 'zmq.sugar.zmq', 'zmq.backend.sys', 'h5py.platform', 'cellprofiler.utilities', 'unittest.time', 'zmq.zmq', 'matplotlib.rcsetup', 'zmq.os', 'cellprofiler.traceback', 'numpy.polynomial.polyutils', 'json.json', 'sys', 'numpy.compat._inspect', 'multiprocessing.subprocess', 'matplotlib.fontconfig_pattern', '_weakref', 'difflib', 'h5py._hl.codecs', 'urlparse', 'unittest.warnings', 'gzip', 'cellprofiler.cellprofiler', 'heapq', 'distutils', 'h5py.warnings', 'numpy.core.einsumfunc', 'zmq.backend.cython.socket', 'zmq.sugar.attrsettr', 'matplotlib.cbook.deprecation', 'matplotlib.colors', 'zmq.backend.cython.constants', 'struct', 'numpy.random.info', 'numpy.testing', 'logging.config', 'collections', 'h5py._hl.files', 'unittest.main', 'distutils.types', 'zipimport', 'zmq.sugar.errno', 'h5py._hl.group', 'textwrap', 'zmq.platform', 'cellprofiler.utilities.itertools', 'h5py._hl.base', 'zmq.backend.cython._version', 'signal', 'numpy.random.operator', 'numpy.core.multiarray', 'zmq.backend.cython.context', 'distutils.version', 'numpy.ma.core', 'numpy.core.getlimits', 'matplotlib.compat.subprocess', 'zmq.backend.cython.copy', 'logging.traceback', 'numpy.matrixlib', '_multiprocessing', 'backports', 'numpy.lib.mixins', '_posixsubprocess32', 'glob', 'mpl_toolkits', 'UserDict', 'inspect', 'six.moves.urllib', 'zmq.sugar.random', 'h5py.h5py_warnings', 'unittest.runner', 'unittest.loader', '_functools', 'json.sys', 'socket', 'cellprofiler.cStringIO', 'numpy.core.memmap', 'zmq.sugar.version', 'cython_runtime', 'numpy.linalg.lapack_lite', 'os', 'marshal', 'h5py.h5', '__future__', 'numpy.core.shape_base', 'cellprofiler', 'matplotlib.compat', '__builtin__', 'operator', 'json.struct', 'errno', '_socket', 'json', 'numpy.lib.histograms', 'multiprocessing._multiprocessing', 'encodings.__builtin__', 'zmq.backend.cython.zmq', 'cellprofiler.random', 'numpy._distributor_init', 'pwd', 'future_builtins', 'zmq.backend', '_sysconfigdata', '_struct', 'numpy.fft', 'numpy.random.numpy', 'logging.time', 'logging.SocketServer', 'numpy.lib.function_base', 'logging.warnings', 'mimetools', 'multiprocessing.signal', 'logging.codecs', '_random', 'numpy.polynomial._polybase', 'zmq.utils.zmq', 'numpy.polynomial.hermite', 'contextlib', 'cellprofiler.__main__', 'numpy.polynomial.polynomial', 'logging.types', 'grp', 'zmq.sugar.warnings', 'numpy.core.defchararray', 'gettext', '_abcoll', 'cellprofiler.re', 'zmq.sugar.time', 'genericpath', 'stat', 'ruamel', 'zmq.utils.jsonapi', 'urllib2', 'unittest.signals', 'backports.functools_lru_cache', 'ctypes.ctypes', 'numpy.lib.format', 'sitecustomize', 'numpy.testing._private.decorators', 'time']
Using /home/anja/software/CellProfiler/plugins as imagej plugin directory
Adding /home/anja/software/CellProfiler/plugins to class path
Skipping /home/anja/software/CellProfiler/plugins/loadimagesfromomero.py
JVM will be started with AWT in headless mode
Creating JVM object
Signalling caller
Enabled Bio-formats directory cacheing
Traversing file system
Exception fetching new version information from http://cellprofiler.org/CPupdate.html: HTTP Error 404: Not Found
CACHEDIR=/home/anja/.cache/matplotlib
Using fontManager instance from /home/anja/.cache/matplotlib/fontList.json
backend WXAgg version unknown
ilastik import: failed to import the ilastik. Please follow the instructions on
    "http://www.ilastik.org" to install ilastik
Traceback (most recent call last):
  File "/home/anja/software/CellProfiler/cellprofiler/modules/classifypixels.py", line 71, in <module>
    from ilastik.core.dataMgr import DataMgr, DataItemImage
ImportError: No module named ilastik.core.dataMgr
ilastik import: failed to import the ilastik. Please follow the instructions on
                          "http://www.ilastik.org" to install ilastik
Traceback (most recent call last):
  File "/home/anja/software/CellProfiler/cellprofiler/modules/ilastik_pixel_classification.py", line 70, in <module>
    import ilastik_main
ImportError: No module named ilastik_main
HDF5Dict.__init__(): /tmp/CpmeasurementsidoTUu.hdf5, temporary=False, copy=None, mode=w
Failed to stop Ilastik

Any help would be appreciated, I need CP for several projects…
Cheers


#2

Hi,
sorry I have no idea how solve that, but maybe it’s an alternative for you to use Docker? This way you won’t be trapped in the dependency hell.
I don’t see a 2.2.0 Docker image on dockerhub though, but there is one for 2.3.1 and also for 3.0.0 (cellprofiler/cellprofiler:2.3.1, cellprofiler/cellprofiler:3.0.0 and many others).
Good luck,
Patrick


#3

Hi Patrick,
thank you for your suggestion. I was still running CP 2.2.0 because of some old pipelines, and because the last time I tried, I couldn’t get CP 3.0.0 running on my system (Java GC overhead issue, which I don’t know if it has been resolved yet…).
I will use the docker files on my work machine, just didn’t see a reason to have multiple versions on my private one. I don’t get any errors about dependency issues, which is weird. The ilastik_main error has been there for ages, but normally CP starts up. Till a few weeks ago that means…


#4

Ok, if you are not afraid of using Docker :grinning: and if you are able to build CP 2.2.0 under Ubuntu, it’s probably not that hard to create a Docker image for that as well. I would suggest to start with this Dockerfile. It took me a few hours to get a working image for 3.1.5. If you are facing the same problems as I did, my Dockerfile might help you to sort that out (see there which part solved which error message). It can be found here
Cheers,
Patrick


#5

Thanks for that. I actually ran into trouble with the cellprofiler:latest and cellprofiler:3.0.0. I’m not that familiar with docker yet, so I’m not sure if it has something to do with my installation/environment or how I try to start it…

docker run -i -t cellprofiler/cellprofiler:3.0.0 /bin/bash
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
/usr/local/lib/python2.7/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Traceback (most recent call last):
  File "/usr/local/bin/cellprofiler", line 11, in <module>
    load_entry_point('CellProfiler', 'console_scripts', 'cellprofiler')()
  File "/usr/local/src/CellProfiler/cellprofiler/__main__.py", line 113, in main
    raise ValueError("You must specify a pipeline filename to run")
ValueError: You must specify a pipeline filename to run
[cmd] cellprofiler exited 1
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] syncing disks.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

#6

First of all: Sorry looks like versions/tags “3.0.0” and “latest” are different, even though latest is 3.0.0. Version/tag “3.0.0” does not work for me either, while “latest” does.

I usually run CellProfiler in headless mode (-c). To run in graphical mode under Linux with X11:

xhost +local:`docker inspect --format='{{ .Config.Hostname }}' cellprofiler/cellprofiler`
docker run -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix:ro cellprofiler/cellprofiler:latest ""

That works for me. A slightly different version is described on dockerhub:
https://hub.docker.com/r/cellprofiler/cellprofiler/
but please see also:
http://wiki.ros.org/docker/Tutorials/GUI

Hope that helps,
Patrick


#7

Yes! CP 3.0.0 actually started. Thank you very much! Would still be nice to get some feedback why the CP 2.2.0 doesn’t work anymore… :frowning:
Cheers,
Anja


#8

My guess is it’s a dependency issue- if you want to try to make an env with the versions linked here (or at least if you diff it against your current install to see what seems majorly different), you’ll be able to get it to work, but since you have another version of CP working it’s obviously fine to just use that as well.


Which matplotlib version for stable branch?