CP headless module problems

Hi Guys,

A few problems I'm running into.
  1. In one instance (Ubuntu 11) I’m getting an error in importing wx. The error is “AttributeError: ‘module’ object has no attribute ‘Log’” on CellProfiler.py line 289 “wx.Log.EnableLogging(False)” The funny thing is that if I pop into a python shell I can ‘import wx’ without error. If I comment out wx.Log.— then it will fail trying to load icons for the gui I think. I’ve confirmed my PYTHONPATH is set for the version of python I’ve been using since I started compiling the deps and also that this is the pythonpath that has wx. This is (almost) I 100% clone of a working instance of CP from another box.

  2. On another instance (RHEL6) I’m getting an error in ‘external_dependencies’, no idea what this error corresponds to … ">Traceback (most recent call last):
    File “CellProfiler.py”, line 216, in
    import external_dependencies
    ImportError: No module named external_dependencies<

  3. Is there anything I could do to decrease deps for a purely headless instance of CP? (wx, gtk, etc)

  4. Are there guides that anyone knows of to integrate headless CP into Omero? We are playing around with Omero and would like to send jobs to CP. I’m familiar with writing scripts to send jobs to the command line but was wondering if there was something more elegant designed. Ultimately I’d like to choose a few images, run a pipeline on them, collect the resulting marked up output images back into omero as well as somehow making note of where the measurement files ended up. I can use the dropbox functionality and just make the output folder of the pipeline as a dropbox folder with set metadata but it doesn’t seem like the ‘best’ option.


Hi Verbunk,
We have a build engineer devoted to CP on Linux: Sam Markson - email: smarkson at broadinstitute.edu . He’s away for the week and can get back to you about some of the Linux-related issues in detail. CellProfiler can be run headless (the -c command-line switch) and should not access any part of WX in that mode. If it does, it’s a bug and I’d appreciate a report of it. I think it’s safest to build all of the dependencies - possibly you can leave out those associated with the Ilastik pixel classifier if you don’t use it (pyVigra and QT are the big ones).

The “external_dependencies” module should be in the same directory as CellProfiler. If CellProfiler can’t import it, then your python path doesn’t include the directory that CellProfiler itself is in. You should add that to your PYTHONPATH.

We do have an unsupported module for loading images for Omero - github.com/CellProfiler/CellPro … omomero.py
You can try it, but we’re taking a different strategy going forward.The next CellProfiler release will have much better integration with Omero and will treat Omero URLs (such as omero:iid=) as if they were image files. The code for this is in CellProfiler’s multiprocessing branch on GitHub - it’s progressing to stability and you could use it to evaluate Omero connectivity. Headless mode is a little problematic because we’ve tried to avoid storing the user’s Omero password internally - prompting for password works in a user-interface version, but not so well headless. If you have any suggestions on how we should be handling authentication, they’d be appreciated. The best way to evaluate the Omero/CellProfiler integration is by using CellProfiler’s LoadData module. LoadData takes a .csv file composed of one column per channel with the first line in the .csv defining the channels. I’ve attached a .csv that I’ve been using as an example.
load_data_omero.csv (147 Bytes)

Hey Lee,

Regarding user auth for running jobs. I've done something *similar* in a past life but I think it could apply here. Essentially we had a custom web front end to a job management (omero.web is my new choice) that allowed users to, in this case, get an autogenerated CPA .properties file that attached to finished data hosted on the server. In CP terms you could allow users to perfect their pipeline and add it to a processing queue. We are exploring entering the analysis request into the omero dbase as a tables entry and using a small python middleman to get the jobs and parse them out on a cluster. Minus all the custom work, because the users already authenticate to Omero and they can't directly generate the queue entry (which contain references to the image files they may not own) I'd consider that safe 'enough' so long as the headless instances run on business hardware rather than a users.

This brings up a good question I think. How difficult would it be to implement the interface to CP and CPa as a django view? That could potentially be very useful as a way to embed them into a website (omero). It sure would solve some issues with job and data management as well as easier access to data. We are already trying to get some generic views that generate plots off of high content screens automatically.


Just as an aside: You may want to check out the discussion on our GitHub page on a similar issue: github.com/CellProfiler/CellProfiler/issues/535.