Windows XP CP2.0 endless loops on MemoryErrors

jubersax · November 5, 2015, 7:02am

First, thanks so much for porting over to Python. The UI is much snappier and improved. Great work and great software suite.

I recently installed the CellProfilerSetup_2.0_9336_alpha.exe on a 32-bit windows XP box and noticed that when I hit what I’m guessing is an out-of-memory error, CP goes into an endless loop. Thought I’d post it. It seems that CP2.0 uses more memory than the Matlab version as well. I hit out of memory errors a lot earlier in pipelines than with the equivalent Matlab version of CP.

Traceback (most recent call last): File "matplotlib\backends\backend_wx.pyc", line 1154, in _onPaint File "matplotlib\backends\backend_wxagg.pyc", line 59, in draw File "matplotlib\backends\backend_agg.pyc", line 314, in draw File "matplotlib\artist.pyc", line 46, in draw_wrapper File "matplotlib\figure.pyc", line 774, in draw File "matplotlib\artist.pyc", line 46, in draw_wrapper File "matplotlib\axes.pyc", line 1687, in draw File "matplotlib\artist.pyc", line 46, in draw_wrapper File "matplotlib\image.pyc", line 235, in draw File "matplotlib\image.pyc", line 182, in make_image File "matplotlib\cm.pyc", line 167, in to_rgba File "matplotlib\colors.pyc", line 523, in __call__ MemoryError

Unlikely to be related, but when CP launches, the console kicks back the following errors:

failed to import Cython: No module named Plex.Scanners Traceback (most recent call last): File "cellprofiler\modules\loadimages.pyc", line 42, in <module> File "bioformats\__init__.pyc", line 7, in <module> File "cellprofiler\utilities\jutil.pyc", line 35, in <module> File "cellprofiler\utilities\setup.pyc", line 99, in find_javahome WindowsError: [Error 2] The system cannot find the file specified Subversion revision: 9336

mbray · November 5, 2015, 7:02am

Hi,

The memory error that you’re reporting is one that is known to us. The 32-bit Windows release will run out of memory when (a) run on large images, (b) pipelines that use many images simultaneously or © pipelines that open many windows. Windows users should use the ConserveMemory module to delete intermediate images, should only keep the windows that they need open and should consider resizing very large images before processing with their pipeline.

With the 2nd error, that is one that we will have to look into. However, we have a beta release with some changes that may take care of it (available here).

The same caveat with the 1st error applies to this beta release as well. If you still have the 2nd error, post it on this thread and we’ll take a closer look.

Hope this helps!
-Mark

jubersax · November 5, 2015, 7:04am

Thanks for the pointer to the Beta. I do hit some errors upon launch:

Traceback (most recent call last): File "cellprofiler\modules\loadimages.pyc", line 58, in <module> File "bioformats\__init__.pyc", line 7, in <module> File "cellprofiler\utilities\jutil.pyc", line 35, in <module> File "cellprofiler\utilities\setup.pyc", line 100, in find_javahome WindowsError: [Error 2] The system cannot find the file specified Traceback (most recent call last): File "cellprofiler\modules\__init__.pyc", line 257, in add_module File "cellprofiler\modules\saveimages.pyc", line 37, in <module> File "bioformats\__init__.pyc", line 7, in <module> File "cellprofiler\utilities\jutil.pyc", line 35, in <module> File "cellprofiler\utilities\setup.pyc", line 100, in find_javahome WindowsError: [Error 2] The system cannot find the file specified None could not load these modules ('cellprofiler.modules.saveimages', WindowsError(2 , 'The system cannot find the file specified'))] Subversion revision: 9663

Memory has always been an issue for me since we regularly are analyzing very large images with very small features. I’ve certainly gotten good with changing orders of operation and SpeedUpCellProfiler in the 7522 release. I didn’t expect it to be any different in the 9663 release, but it seems to consume much more memory than I’d expect from my experience with the 7522 version. For many images that progress fine through a load, split-to-gray, illumination correction/apply, ID of objects, measure, save QC images in 7522, I hit a memory wall at the illumination apply step. I’m not suggesting there is anything to do about this from the code side, I was just surprised to see it happen.

Thanks again for your reply.

mbray · November 5, 2015, 7:04am

Re: error message - To fix this, first you should make sure that Java is installed on your computer.

If it is, and you still get this error, then the JAVA_HOME environment variable probably needs to be set correctly. To do this, follow these steps:

Open Control Panel > Performance and Maintenance > System (or right-click on My Computer and choose “Properties”)
In the box that opens, click the “Advanced” tab.
Click the button “Environment Variables”
Under “System variables”, check for a variable called JAVA_HOME, and click “Edit”. If it’s not there, click “New” and type “JAVA_HOME” under “Variable name” in the box that opens.
Find the location of your Java installation (e.g., C:\Program Files\Java\jre1.6.0_07) and put this path as the variable value (or make sure it’s correct if it already exists).
Click OK to close the dialog boxes.

Let us know if this doesn’t solve the problem.
-Mark

thouis · November 5, 2015, 7:10am

The latest release includes Java as part of the CellProfiler distribution, so this should no longer be an issue.