OutOfMemoryError exception, but processing continues?

Thank you for the very useful program.

I am occasionally getting java.lang.OutOfMemoryError exceptions about the Java heap space. I would expect this to be a fatal error, but for some reason the pipeline processing continues and even completes. Why isn’t it a fatal error (somehow CellProfiler recovers?) and are the results to be trusted?

I am using r11710 on Windows 7 64 bit with 16 GB RAM. The relevant excerpt from log is below:

Wed May 15 12:17:38 2013: Image # 1, module LoadImages # 1: 4.37 sec (bg) Exception in thread "Thread-16" java.lang.OutOfMemoryError: Java heap space Traceback (most recent call last): File "cellprofiler\modules\loadimages.pyc", line 2759, in provide_image File "cellprofiler\modules\loadimages.pyc", line 2928, in load_using_bioformat s File "cellprofiler\utilities\jutil.pyc", line 439, in method File "cellprofiler\utilities\jutil.pyc", line 377, in call JavaException: Java heap space Wed May 15 12:17:42 2013: Image # 1, module ColorToGray # 2: 3.99 sec (bg)

Thank you.

I’m not sure whether the results can be trusted, but you can increase the Java virtual machine heap size by running CellProfiler from the command line with a special flag.

Use a terminal/command line prompt to go to the folder where CellProfiler is installed, and type the following: CellProfiler --helpLook for the –jvm-heap-size=JVM_HEAP_SIZE entry and set it accordingly to increase your heap size to something more appropriate (perhaps 1GB or so, as much as is needed to get things to run successfully).

Regards,
-Mark

Thank you for the suggestion. I increased the size of the Java heap, and that seems to have gotten rid of the exceptions.

Is there a way to have CellProfiler write the console log to an external file (in case the log output exceeds the console’s buffer), so that I can examine the log for any exceptions afterwards?

Yes, there is. If you look at the listing of optional input arguments again, there is also a -L argument. This allows you to specify the logging level that is used, or can be used to point to a logging configuration file. It would be in this file where you can specify a filename of where you want log information to be reported; see here for more details on how to do this.

Regards,
-Mark

Thank you. I found another post (Unexpected h5 output files) where you mention the -L option with a sample log config file. I saved that locally as logconfig.txt and then ran (from the Windows command line), but got the following error message about no such -L option:

[code]>“C:\Program Files\CellProfiler\CellProfiler.exe” -L logconfig.txt
Usage: CellProfiler.exe [options] ])
where is the optional filename for the output file of measure
ments when running headless

CellProfiler.exe: error: no such option: -L[/code]

What am I doing wrong?

Ahh, my mistake. The -L option is avilable only if runnning the source code, which has not yet been released as an installable program. You can either wait until then, or use one of the trunk builds here (pay attention to the caveats listed therein).

Regards,
-Mark