CP 3.1.8 - Memory error in Identify Secondary Objects with large images

I’m running CP 3.1.8 on an HP workstation with 144 GB of RAM and more than 1 TB of available disk space, Java memory = 64000 MB, 1 worker maximum. If I run the attached pipeline on a small portion of the image of interest, separated by color deconvolution into Colour_1 = hematoxylin (nuclei) and Colour_2 = DAB (Ki67 in this case), execution is fast and perfect.

CP intensity3.cpproj (639.9 KB)


But if I run the same pipeline on a larger portion of the image (~675,000 nuclei), I get the following error (even though according to Task Manager there is ample memory):


Worker 0: Mon Mar 18 19:09:56 2019: Image # 1, module IdentifyPrimaryObjects # 5: CPU_time = 166.14 secs, Wall_time = 216.61 secs
Worker 0: Error detected during run of module IdentifySecondaryObjects#6
Worker 0: Traceback (most recent call last):
Worker 0: File "CellProfiler\cellprofiler\pipeline.py", line 1924, in run_image_set
Worker 0: File "CellProfiler\cellprofiler\pipeline.py", line 2034, in run_module
Worker 0: File "CellProfiler\cellprofiler\modules\identifysecondaryobjects.py", line 686, in run
Worker 0: File "CellProfiler\cellprofiler\module.py", line 1214, in add_measurements
Worker 0: File "CellProfiler\cellprofiler\object.py", line 312, in relate_children
Worker 0: File "CellProfiler\cellprofiler\object.py", line 422, in histogram_from_ijv
Worker 0: File "site-packages\scipy\sparse\compressed.py", line 947, in toarray
Worker 0: File "site-packages\scipy\sparse\base.py", line 1184, in _process_toarray_args
Worker 0: MemoryError

Not displaying images (“eye”) makes no difference. Exactly the same thing occurs on a similarly large Lenovo workstation in the lab, so this is not hardware specific. I tried increasing the memory available to Java as suggested in an earlier post, but that technique does not seem to be available in CP 3.1.8 (which according to posts handles Java differently from other versions).

CP Errors file.txt (13.7 KB)

I’m also attaching a text file with a long list of errors beginning from initiating CP and the pipeline. Are any of these, other than those leading to failure of Identify Secondary Objects, significant?

I’d upload the larger images but they are too large and gave me an error message.

I could really use some help with this, as it is not clear how to move forward.


Hi mbarish
Im actually having same issues as you, while working on stem cross section image for vessel and fibers examinations. Have you get soome solution by the time. Any help is very appreciated.
Thank you.