Inordinate CPU usage after completion of anlysis

I’ve been using CP with no problems for roughly a month. However, very recently after my upgrade to version 2.0, I began having a problem with the pipeline I was running.

The analysis would finish and the “Analysis Complete” dialogue would popup, but CP continued to use 60%-70% CPU as well as large amount of RAM. Further, the module I was interested in seeing “IdentifyPrimaryObjects” was displayed, but the window was completely blank. Trying to manipulate the windows was difficult due to the extreme CPU usage, but if I managed to close the “IdentifyPrimaryObjects” window, CP would fix itself and return to the normal state before running the analysis. The problem was also alleviated by clicking the eye next to the modules in the pipeline such that no extraneous windows were displayed at all. This, however, was not sufficient as I still needed to see the “IdentifyPrimaryObjects” window.

I have attached a screenshot of this happening and a screenshot of my computer specs, as well as the pipeline and image I was using. Any help is much appreciated.





FungalArea-red.cp (5.42 KB)

We’ve just made a new release, r10415. Do you see the same behavior with this version?
-Mark

Hi Mark,

I am having a similar issue processing large images that are around 4000x3000 pixels. It happens with either brightfield or fluorescent images. It never seems to stop using the cpu and memory usage is over 1GB after the pipeline is completed. Also, the color unmixing window will not display the images after the pipeline completes. When I resize the image dimensions by 33% it all works fine. Is there an upper limit to the number of pixels per image that CellProfiler can process effectively without the associated memory leaks? Thanks for any help with this issue.

Derek
BFPIPE.cp (1.75 KB)






Hi Derek and Michael,

This is a problem that we are aware of on our end, and we believe it comes down to two things:

  • For really
    big images, try performing your pipeline development on a resized or cropped image in test mode if at all possible. Then, once the settings are optimized, do the full analysis run with windows hidden.
  • It turns out that the delay post-completion is due to the output file being written at the end. The more measurements there are (especially if the image is big), the longer it takes to write; you can probably see it being written if you watch it in Windows explorer during this delay. Unfortunately, we don’t have a workaround for this other than limiting the measurements made. We may need an option to disable output file creation for some cases.

Regards,
-Mark

Hi all,

We have recently remedied this issue, in that CP now has an added prompt at the bottom of the GUI asking whether you want to write an output file at all. If you uncheck this box, no output file is created and you will not have the long wait-time at the end of the analysis run for huge image sets. Please note that this setting does not affect the output of measurements in other CP modules, such as ExportToSpreadsheet or ExportToDatabase.

Until we release a new version on our website, you can try our newest “bleeding edge” version here, which will have the checkbox: cellprofiler.org/cgi-bin/trunk_build.cgi (please note the caveats mentioned).

Regards,
-Mark