Error using CorrectIlluminationCalculate

Hi there,

I’m trying to correct illumination on 481 image sets of 4 channels each, with each image being 696x520 pixels. The pipeline runs if I reduce to 2 image sets, but with 481 I get this error:

Traceback (most recent call last):
  File "/Applications/Lab/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/pipeline.py", line 2289, in prepare_group
  File "/Applications/Lab/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/modules/correctilluminationcalculate.py", line 497, in prepare_group
  File "/Applications/Lab/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/pipeline.py", line 2020, in run_group_with_yield
  File "wx/_windows.pyc", line 2943, in Update
PyAssertionError: C++ assertion "value <= m_maximum" failed at /BUILD/wxPython-src-2.8.12.1/src/generic/progdlgg.cpp(337) in Update(): invalid progress value

Is this a memory problem?

This issue appears to have come up before. Can you try the solution there and see if clears things up? You may want to delete the installation folder if it’s still there after uninstalling, and then re-install.
-Mark

Thanks Mark.

Seems there was an odd-sized image set in my folder. After removing, strangely the pipeline executes fine with Analyze Images, but Test mode still has the error.

Have reinstalled CellProfiler and the behaviour is the same…

Would you mind posting your project file plus some sample images, including the oddly-sized one you mentioned earlier?
-Mark

Hi Mark,

Sorry, I missed your reply. Even after the odd-sized image was removed, the pipeline would execute in Analysis Mode but throw an error in Test Mode.

I have replicated this with a separate pipeline and image set. I am attaching these here.

The error is:

Traceback (most recent call last): File "/Applications/Lab/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/pipeline.py", line 2289, in prepare_group File "/Applications/Lab/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/modules/correctilluminationcalculate.py", line 497, in prepare_group File "/Applications/Lab/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/pipeline.py", line 2020, in run_group_with_yield File "wx/_windows.pyc", line 2943, in Update PyAssertionError: C++ assertion "value <= m_maximum" failed at /BUILD/wxPython-src-2.8.12.1/src/generic/progdlgg.cpp(337) in Update(): invalid progress value
Archive.zip (3.64 MB)

If you then press “continue processing” after the error, you get another error:

Traceback (most recent call last): File "/Applications/Lab/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/gui/pipelinecontroller.py", line 2615, in do_step File "/Applications/Lab/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/modules/correctilluminationcalculate.py", line 509, in run File "/Applications/Lab/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/modules/correctilluminationcalculate.py", line 974, in deserialize KeyError: 'name'

It seems that analysis mode is now not processing this pipeline, just counts up time endlessly. Would be interested to see if you get the same problem on your end.

Cheers

Hi,

I just ran your Archive.zip files on my MacBook Pro (8GB RAM) and it completed in Test Mode without error. It look a long time to run the 3 CorrectIlluminationCalculate modules (5+ minutes?), but it did finish. Non-Test-Mode Analyze Images also completed ok. The error indicates a progress bar dialog problem, and it is true that my progress bar didn’t advance much but CP finished anyway. But maybe that is a red herring.

What version of CP are you using? The 2.1.0 release?

Interestingly, I just tried this in our 2.1.1 release candidate and I got what looks to be a different error, which I will report:

Traceback (most recent call last): File "/Applications/CellProfiler2.1.1_rc.app/Contents/Resources/lib/python2.7/cellprofiler/pipeline.py", line 2287, in prepare_group File "/Applications/CellProfiler2.1.1_rc.app/Contents/Resources/lib/python2.7/cellprofiler/modules/correctilluminationcalculate.py", line 497, in prepare_group File "/Applications/CellProfiler2.1.1_rc.app/Contents/Resources/lib/python2.7/cellprofiler/pipeline.py", line 2010, in run_group_with_yield File "/Applications/CellProfiler2.1.1_rc.app/Contents/Resources/lib/python2.7/cellprofiler/modules/namesandtypes.py", line 1291, in run File "/Applications/CellProfiler2.1.1_rc.app/Contents/Resources/lib/python2.7/cellprofiler/modules/namesandtypes.py", line 1328, in add_image_provider File "/Applications/CellProfiler2.1.1_rc.app/Contents/Resources/lib/python2.7/cellprofiler/modules/namesandtypes.py", line 1361, in add_simple_image File "/Applications/CellProfiler2.1.1_rc.app/Contents/Resources/lib/python2.7/cellprofiler/modules/namesandtypes.py", line 1388, in add_provider_measurements AttributeError: 'MonochromeImageProvider' object has no attribute 'scale'
Reported as Github issue #1148

David

Thanks David. I’m on a MacBook Pro 8GB RAM too, CellProfiler version is the latest release, 2.1.0.

I’ll rejig and try again, but see what you guys come up with in future following the bug report.

Hi,
Lee did fix something which we hope helps. To test, you would have to install the newest trunk build here: cellprofiler.org/cgi-bin/trunk_build.cgi
(Note: the 2.1.1 release candidate on that page does NOT have this change)

Hope that helps,
David

Thanks David,

With the new build, I’m getting Test mode to execute a simplified pipeline using CorrectIlluminationCalculate with a set of 20 images.

But with a set of 558 images, I’m getting the same error as before, after the “CorrectIlluminationCalculate is averaging 558 images”:

Traceback (most recent call last): File "/Applications/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/pipeline.py", line 2288, in prepare_group File "/Applications/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/modules/correctilluminationcalculate.py", line 507, in prepare_group File "/Applications/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/pipeline.py", line 2013, in run_group_with_yield File "wx/_windows.pyc", line 2943, in Update PyAssertionError: C++ assertion "value <= m_maximum" failed at /BUILD/wxPython-src-2.8.12.1/src/generic/progdlgg.cpp(337) in Update(): invalid progress value

And if I then click “Continue processing…” and try to step in Test mode:

Traceback (most recent call last): File "/Applications/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/gui/pipelinecontroller.py", line 2722, in do_step File "/Applications/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/modules/correctilluminationcalculate.py", line 520, in run File "/Applications/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/modules/correctilluminationcalculate.py", line 985, in deserialize KeyError: 'name'

However, it seems to execute fine using Analyze Images.

Hopefully this might narrow down for you where the error is occurring.

Hi,

Thanks for reporting back. As noted by Lee here, the code for pre-calculating using All:First Cycle in Test Mode is a bit hairy. Since there are workarounds (use All:Across Cycles and save the illumination functions for later use withCorrectIlluminationApply; or just use Analyze Mode), unless our software engineer really feels the need, we’ll likely not prioritize any further fix in the near future.

In fact, in our own work, we tend to find using a separate illumination correction calculate pipeline and saving illum functions to be more efficient than running them on the fly. Yes, there are two pipelines to deal with, but in case you need to re-run the main pipeline with the CorrectIlluminationApply then you don’t have the need to waste computing cycles to re-calculate the illumination functions. But that is just a suggestion of our experience!

Cheers,
David

OK, thanks again David.

These seem like reasonable workarounds, so I’ll give them a go.

Cheers
Drew

This issue has been addressed with the 2.1.1 release, by warning the user re: creating the illumination correction function in the 1st cycle.
-Mark