Cell profiler overhead, custom scripting and multithreading

I’ve noticed from watching the text output while batch processing a large number of images that the total time to complete analysis per-image after adding up the time-consumption per-module is less than half of the per-image time displayed by the CellProfiler GUI. I’m wondering if the cause of this is known overhead and where this overhead comes from. I found instructions on the Developers’ wiki on how to run CellProfiler ‘headless’ but it looks like UNIX terminal syntax and I’m stuck with a Windows machine where I don’t have a lot of experience with command prompt- can anyone show me the syntax for running CellProfiler headless in windows? I saw some posts about running CP headless on the forums, but they all seem to deal with 1.0 rather than the python version.

Also, is there a tutorial or example floating around somewhere to run CP from a python script? In my particular case, I’m using micro-manager’s python interface to acquire images from a microscope and I’m wondering how to call CP directly on the acquired images after the acquisition script completes its job.

In the long run, I’d really like to set it up in a producer/consumer fashion with the image acquisition script running on one thread and dumping acquired images into some intermediary container which is read out in a FIFO fashion by CP which would be running on a second thread to maximize utilization of available processor cycles, but I foresee this being quite challenging. My idea thus far has been to try to write a CP module to listen to such a container, perhaps a queue, but this is still a very embryonic idea. If anyone has any thoughts / suggestions on the matter I’d love to hear them.

Thanks,
Pavak

Hi Pavak,

Sorry it’s taken so long to respond to this post. I can address one of your issues here:

The wiki shows this:

The syntax on Windows is virtually the same, except the paths must be specified differently:

This assumes that

  • You are running from the CellProfiler directory.
  • python is on your Windows path so the command prompt knows where it is. See here
    for more details.

Regards,
-Mark