Memory Error in Cell Profiler

Hi Cell Profiler Community,

I am attempting to analyze a tiff that is about 400MB however, I am running into a memory error before it can start my pipeline. The error says

“Error while processing NamesAndTypes:
(Worker) MemoryError: Unable to allocate 16.5 GiB for an array with shape (25088, 58880,3) and data type float32
Do you want to stop processing?”

This error doesn’t save any of the data and stops the pipeline for that image. I am only running that singular image in the process so no data was collected.

Has anyone had a similar problem and knows how to fix it? Any and all help is welcomed and appreciated, Thanks!

Do you know exactly how big this image is (dimensions)? Because 25k by 50k is huge. If it’s a tissue section then the original .tif might be heavily compressed, yet this will need to be uncompressed to load into CellProfiler and so you may not have enough memory.

@DStirling It is a very large image yes, it was originally a svs but I converted it to a tiff because I was receiving the same error due to that file being too large as well. Would upgrading the amount of RAM I have solve this issue?

More RAM could certainly help you load the image into CellProfiler, but it’d take even more memory to be able to do much with it. CellProfiler generally isn’t designed to work on images of that size, our usual approach would be to break it down into smaller tiles and work on those individually. In the future we hope to have a better solution for this problem.