Error in the middle of the run

Hello,
I am not sure why I am getting this error in the middexample.cpproj (1.3 MB)
le of the run. This is the code I run:
./CellProfiler -c -r -o //Volumes/sblab-1/users/rojin/OUTPUT/ -p //Volumes/sblab-1/users/rojin/OUTPUT/cellprofiler.cpproj

and there are 24192 images ( three channels), which means there are 8,064 set of images

I attached the pipeline as well.

Thanks.

Jul 26 21:38:43 CellProfiler[17586] : Tue Jul 26 21:38:42 2016: Image # 875, module NamesAndTypes # 3: 9.79 sec
Jul 26 21:38:43 CellProfiler[17586] : Tue Jul 26 21:38:43 2016: Image # 875, module Groups # 4: 0.00 sec
Jul 26 21:38:46 CellProfiler[17586] : Tue Jul 26 21:38:43 2016: Image # 875, module MeasureImageQuality # 5: 2.89 sec
Jul 26 21:38:47 CellProfiler[17586] : Tue Jul 26 21:38:46 2016: Image # 875, module CorrectIlluminationCalculate # 6: 0.70 sec
Jul 26 21:38:47 CellProfiler[17586] : Tue Jul 26 21:38:47 2016: Image # 875, module CorrectIlluminationApply # 7: 0.01 sec
Jul 26 21:38:47 CellProfiler[17586] : Tue Jul 26 21:38:47 2016: Image # 875, module CorrectIlluminationCalculate # 8: 0.64 sec
Jul 26 21:38:48 CellProfiler[17586] : Tue Jul 26 21:38:48 2016: Image # 875, module CorrectIlluminationApply # 9: 0.02 sec
Jul 26 21:38:48 CellProfiler[17586] : Tue Jul 26 21:38:48 2016: Image # 875, module CorrectIlluminationCalculate # 10: 0.65 sec
Jul 26 21:38:48 CellProfiler[17586] : Tue Jul 26 21:38:48 2016: Image # 875, module CorrectIlluminationApply # 11: 0.02 sec
Jul 26 21:38:49 CellProfiler[17586] : Tue Jul 26 21:38:49 2016: Image # 875, module ApplyThreshold # 12: 0.28 sec
Jul 26 21:38:51 CellProfiler[17586] : Tue Jul 26 21:38:49 2016: Image # 875, module IdentifyPrimaryObjects # 13: 2.24 sec
Jul 26 21:38:55 CellProfiler[17586] : Tue Jul 26 21:38:51 2016: Image # 875, module MeasureImageIntensity # 14: 3.97 sec
Jul 26 21:38:56 CellProfiler[17586] : Tue Jul 26 21:38:55 2016: Image # 875, module ApplyThreshold # 15: 0.68 sec
Jul 26 21:38:57 CellProfiler[17586] : Tue Jul 26 21:38:56 2016: Image # 875, module IdentifySecondaryObjects # 16: 1.08 sec
Jul 26 21:38:58 CellProfiler[17586] : Tue Jul 26 21:38:57 2016: Image # 875, module IdentifyTertiaryObjects # 17: 0.21 sec
Jul 26 21:38:59 CellProfiler[17586] : Tue Jul 26 21:38:58 2016: Image # 875, module MeasureObjectIntensity # 18: 1.48 sec
Jul 26 21:39:05 CellProfiler[17586] : Tue Jul 26 21:38:59 2016: Image # 875, module MeasureObjectSizeShape # 19: 5.38 sec
Jul 26 21:39:10 CellProfiler[17586] : Error detected during run of module MeasureObjectIntensityDistribution
Jul 26 21:39:10 CellProfiler[17586] : Traceback (most recent call last):
Jul 26 21:39:10 CellProfiler[17586] : File “/Users/rojin/Desktop/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/pipeline.py”, line 1819, in run_with_yield
Jul 26 21:39:10 CellProfiler[17586] : File “/Users/rojin/Desktop/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/pipeline.py”, line 2067, in run_module
Jul 26 21:39:10 CellProfiler[17586] : File “/Users/rojin/Desktop/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/modules/measureobjectintensitydistribution.py”, line 438, in run
Jul 26 21:39:10 CellProfiler[17586] : File “/Users/rojin/Desktop/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/modules/measureobjectintensitydistribution.py”, line 806, in calculate_zernikes
Jul 26 21:39:10 CellProfiler[17586] : File “/Users/rojin/Desktop/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/measurements.py”, line 369, in setitem
Jul 26 21:39:10 CellProfiler[17586] : File “/Users/rojin/Desktop/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/measurements.py”, line 845, in add_measurement
Jul 26 21:39:10 CellProfiler[17586] : File “/Users/rojin/Desktop/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/utilities/hdf5_dict.py”, line 481, in setitem
Jul 26 21:39:10 CellProfiler[17586] : File “/Users/rojin/Desktop/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/utilities/hdf5_dict.py”, line 569, in setitem
Jul 26 21:39:10 CellProfiler[17586] : File “/Users/rojin/Desktop/CellProfiler.app/Contents/Resources/lib/python2.7/cellprofiler/utilities/hdf5_dict.py”, line 42, in new_setitem
Jul 26 21:39:10 CellProfiler[17586] : File “h5py/_objects.pyx”, line 54, in h5py._objects.with_phil.wrapper (/Users/travis/build/MacPython/h5py-wheels/h5py/h5py/_objects.c:2458)
Jul 26 21:39:10 CellProfiler[17586] : File “h5py/_objects.pyx”, line 55, in h5py._objects.with_phil.wrapper (/Users/travis/build/MacPython/h5py-wheels/h5py/h5py/_objects.c:2415)
Jul 26 21:39:10 CellProfiler[17586] : File “/Users/rojin/Desktop/CellProfiler.app/Contents/Resources/lib/python2.7/h5py/_hl/dataset.py”, line 586, in setitem
Jul 26 21:39:10 CellProfiler[17586] : File “h5py/_objects.pyx”, line 54, in h5py._objects.with_phil.wrapper (/Users/travis/build/MacPython/h5py-wheels/h5py/h5py/_objects.c:2458)
Jul 26 21:39:10 CellProfiler[17586] : File “h5py/_objects.pyx”, line 55, in h5py._objects.with_phil.wrapper (/Users/travis/build/MacPython/h5py-wheels/h5py/h5py/_objects.c:2415)
Jul 26 21:39:10 CellProfiler[17586] : File “h5py/h5d.pyx”, line 217, in h5py.h5d.DatasetID.write (/Users/travis/build/MacPython/h5py-wheels/h5py/h5py/h5d.c:3345)
Jul 26 21:39:10 CellProfiler[17586] : File “h5py/_proxy.pyx”, line 126, in h5py._proxy.dset_rw (/Users/travis/build/MacPython/h5py-wheels/h5py/h5py/_proxy.c:1627)
Jul 26 21:39:10 CellProfiler[17586] : RuntimeError: Unable to register datatype (Can’t insert duplicate key)
Jul 26 21:39:10 CellProfiler[17586] : Tue Jul 26 21:39:05 2016: Image # 875, module MeasureObjectIntensityDistribution # 20: 4.35 sec
Jul 26 21:39:10 CellProfiler[17586] : Failed to stop Ilastik
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in '_javabridge.JB_Object.__deal
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in '_javabridge.JB_Object.__dea
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: "'NoneType
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in '_javabridge.JB_Objec
Jul 26 21:39:11 CellProfiler[17586] : Exception TypeError: “‘NoneType’ object is not callable” in ‘_javabridge.JB_Object.dealloc’ ignored

This is a Mac specific error and known bug for large data sets- see here. In the meantime you can try running your pipeline on a non-Mac system or breaking your analysis up into smaller batches. I’ve pinged our software team again about it.

Is there a specific code that I can use in unix to divide the run into subset of runs? meaning that giving a range of image numbers so that each time it runs that specific range and add it to database?
Thanks!

There is a “CreateBatchFiles” option but my understanding is the .h5 output is a pain in the butt to parse, you’re welcome toe explore it though; alternately you could export your image set with metadata as a CSV with File->Export->ImageSetListing and then modify your pipeline to use LoadData rather than the standard input modules; you could easily script something to break the large CSV into a bunch of smaller ones then run CellProfiler on each with the --file-list flag.