Memory management

I have about 17,000 images of 640*480 pixels to run in a short pipeline composed of IdentifyingPrimaryObject, OverlayObjects, SaveImages, OverlayOutlines, SaveImages, ExportToSpreadsheet. I am now running it on my laptop with 8GB RAM and it’s linux. CP version is 3.1.5. But only after processing 100 images, my RAM runs out. I did some search, and found there was a module called ConserveMemory, but I couldn’t find it in CP v3.1.5.

I want to ask: is there any way to recycle the memory after each cycle now? Or what are the possible way for me to run this pipeline on my laptop in one shot?

Hi,

That’s very fast for your memory to be running out- just to be sure, do you have the display “eyes” closed on all your modules when you run your pipeline?

I’m afraid CellProfiler relies on python itself to do the memory management, so there’s not a lot we can currently do to change memory handling. You can try running the pipeline headless from the command line, though you’ll lose the benefits of multithreading in that case. I’ll try to think of other options too.

I didn’t turn that “eyes” off, and indeed probably that was one of the main contributors to memory consumption. Thank you.