CP2 fails analysing the well #101, not before

Hello,

I have installed the latest version of CP2. It can analyse small [test] batches, but it fails when we tried to analyse images from a real 384-well plate. More precisely, it failed at the well #101 while loading illumination background, although analysis for the previous 100 wells ran without problems. The error stack suggests to install JAI from jai.dev.java.net/, and that IMHO makes no sense at the well #101.

Could somebody suggest, if really this Java library is missing, or this is just an ordinary mis-leading error message ? What is the real problem, then?

Thanks,

|Dmitry A. Kuznetsov, PhD
|Swiss Institute of Bioinformatics
|UNIL Genopode, 1015 Lausanne Switzerland
|WWW: isb-sib.ch/
|Tel: +41-21-692-4062 Fax: +41-21-692-4065
|E-mail: Dmitry.Kuznetsov@isb-sib.ch

[size=80]***********************************************************************************
Exception in thread “main” loci.formats.MissingLibraryException: Java Advanced Imaging (JAI) is required to read some TIFF files. Please install JAI from jai.dev.java.net/
at loci.formats.in.TiffJAIReader.initFile(TiffJAIReader.java:103)
at loci.formats.FormatReader.setId(FormatReader.java:1072)
at loci.formats.DelegateReader.setId(DelegateReader.java:239)
Caused by: loci.common.ReflectException: No such class:
javax.media.jai.NullOpImage
at loci.common.ReflectedUniverse.exec(ReflectedUniverse.java:153)
at loci.formats.in.TiffJAIReader.initFile(TiffJAIReader.java:96)
… 2 more
Caused by: java.lang.ClassNotFoundException: javax.media.jai.NullOpImage
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at loci.common.ReflectedUniverse.exec(ReflectedUniverse.java:145)
… 3 more
ERROR:cellprofiler.utilities.jutil:Caught exception when executing closure Traceback (most recent call last):
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 299, in synchronous_closure
File
"/local/openbis/CP2/out/CellProfiler/bioformats/formatreader.py", line 377, in fn
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 502, in method
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 440, in call
JavaException: Java Advanced Imaging (JAI) is required to read some TIFF files. Please install JAI from jai.dev.java.net/ Caught exception when executing closure Traceback (most recent call last):
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 299, in synchronous_closure
File
"/local/openbis/CP2/out/CellProfiler/bioformats/formatreader.py", line 377, in fn
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 502, in method
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 440, in call
JavaException: Java Advanced Imaging (JAI) is required to read some TIFF files. Please install JAI from jai.dev.java.net/ WARNING:cellprofiler.modules.loadimages:Failed to load /scratch/local/tmp/omartin/metadata/component/Prestwick_05092011_A431/BSF006284_000/illumination.tif
with bioformats. Use PIL instead
Traceback (most recent call last):
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/loadimages.py", line 2939, in provide_image
File
"/local/openbis/CP2/out/CellProfiler/bioformats/formatreader.py", line 469, in load_using_bioformats
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 308, in run_in_main_thread
JavaException: Java Advanced Imaging (JAI) is required to read some TIFF files. Please install JAI from jai.dev.java.net/ Failed to load /scratch/local/tmp/omartin/metadata/component/Prestwick_05092011_A431/BSF006284_000/illumination.tif
with bioformats. Use PIL instead
Traceback (most recent call last):
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/loadimages.py", line 2939, in provide_image
File
"/local/openbis/CP2/out/CellProfiler/bioformats/formatreader.py", line 469, in load_using_bioformats
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 308, in run_in_main_thread
JavaException: Java Advanced Imaging (JAI) is required to read some TIFF files. Please install JAI from jai.dev.java.net/ ERROR:cellprofiler.pipeline:Error detected during run of module Smooth Traceback (most recent call last):
File “/local/openbis/CP2/out/CellProfiler/cellprofiler/pipeline.py”,
line 1281, in run_with_yield
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/smooth.py",
line 138, in run
File “/local/openbis/CP2/out/CellProfiler/cellprofiler/cpimage.py”,
line 549, in get_image
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/loadimages.py", line 2946, in provide_image
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/loadimages.py", line 2984, in load_using_PIL
ImportError: No module named tifffile
Error detected during run of module Smooth Traceback (most recent call last):
File “/local/openbis/CP2/out/CellProfiler/cellprofiler/pipeline.py”,
line 1281, in run_with_yield
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/smooth.py",
line 138, in run
File “/local/openbis/CP2/out/CellProfiler/cellprofiler/cpimage.py”,
line 549, in get_image
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/loadimages.py", line 2946, in provide_image
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/loadimages.py", line 2984, in load_using_PIL
ImportError: No module named tifffile
INFO:PipelineStatistics:Fri Sep 23 21:36:17 2011: Image # 101, module Smooth # 7: 0.08 sec (bg) Fri Sep 23 21:36:17 2011: Image # 101, module Smooth # 7: 0.08 sec (bg) WARNING:root:Caught exception while killing ijbridge.
Traceback (most recent call last):
File “CP2/out/CellProfiler/CellProfiler.py”, line 484, in
ImportError: No module named ijbridge
Caught exception while killing ijbridge.
Traceback (most recent call last):
File “CP2/out/CellProfiler/CellProfiler.py”, line 484, in
ImportError: No module named ijbridge
[/size]

Can you post your pipeline, and images for wells 100 and 101? It might be something about the particular image itself.

thouis,

below is the pipeline.
I had a look on these images #100 and #101, they are normal (i.e. not white or black). To exclude this question completely, we ran same pipeline on the same data on images #145-250. Of course, the same error appear after 100 images processed [on image #155], at the step of Gaussian blur filter.

Hence, the problem is somewhere around internal limits of CellProfiler… MAX_IMAGES=100? memory leaking? number of open files?

Thanks for your suggestions…
Dmitry.


[size=85]
CellProfiler Pipeline: cellprofiler.org
Version:2
SVNRevision:11495

LoadImages:[module_num:1|svn_version:‘11439’|variable_revision_number:11|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
File type to be loaded:individual images
File selection method:Text-Exact match
Number of images in each group?:3
Type the text that the excluded images have in common:Do not use
Analyze all subfolders within the selected folder?:All
Input image file location:Default Input Folder\x7C.
Check image sets for missing or duplicate files?:
Group images by metadata?:No
Exclude certain files?:No
Specify metadata fields to group by:
Select subfolders to analyze:
Image count:4
Text that these images have in common (case-sensitive):Propidium Iodide_A431 - n000000.tif
Position of this image in each group:Propidium Iodide_MZ - n000000.tif
Extract metadata from where?:None
Regular expression that finds metadata in the file name:None
Type the regular expression that finds metadata in the subfolder path:None
Channel count:1
Group the movie frames?:No
Grouping method:Interleaved
Number of channels per group:2
Load the input as images or objects?:Images
Name this loaded image:rawPI
Name this loaded object:Nuclei
Retain outlines of loaded objects?:No
Name the outline image:NucleiOutlines
Channel number:1
Rescale intensities?:Yes
Text that these images have in common (case-sensitive):Cell Tracker Green_A431 - n000000.tif
Position of this image in each group:Cell Tracker Green_MZ - n000000.tif
Extract metadata from where?:None
Regular expression that finds metadata in the file name:None
Type the regular expression that finds metadata in the subfolder path:None
Channel count:1
Group the movie frames?:No
Grouping method:Interleaved
Number of channels per group:2
Load the input as images or objects?:Images
Name this loaded image:rawCT
Name this loaded object:Nuclei
Retain outlines of loaded objects?:No
Name the outline image:NucleiOutlines
Channel number:1
Rescale intensities?:Yes
Text that these images have in common (case-sensitive):Filipin_A431 - n000000.tif
Position of this image in each group:Filipin_MZ - n000000.tif
Extract metadata from where?:None
Regular expression that finds metadata in the file name:None
Type the regular expression that finds metadata in the subfolder path:None
Channel count:1
Group the movie frames?:No
Grouping method:Interleaved
Number of channels per group:2
Load the input as images or objects?:Images
Name this loaded image:RawFil
Name this loaded object:Nuclei
Retain outlines of loaded objects?:No
Name the outline image:NucleiOutlines
Channel number:1
Rescale intensities?:Yes
Text that these images have in common (case-sensitive):CY5_A431 - n000000.tif
Position of this image in each group:CY5_MZ - n000000.tif
Extract metadata from where?:None
Regular expression that finds metadata in the file name:None
Type the regular expression that finds metadata in the subfolder path:None
Channel count:1
Group the movie frames?:No
Grouping method:Interleaved
Number of channels per group:2
Load the input as images or objects?:Images
Name this loaded image:RawCY5
Name this loaded object:Nuclei
Retain outlines of loaded objects?:No
Name the outline image:NucleiOutlines
Channel number:1
Rescale intensities?:Yes

RescaleIntensity:[module_num:2|svn_version:‘6746’|variable_revision_number:2|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:rawPI
Name the output image:RescaledPI
Select rescaling method:Choose specific values to be reset to the full intensity range
How do you want to calculate the minimum intensity?:Custom
How do you want to calculate the maximum intensity?:Custom
Enter the lower limit for the intensity range for the input image:0
Enter the upper limit for the intensity range for the input image:0.0625
Enter the intensity range for the input image:0,0.0625
Enter the desired intensity range for the final, rescaled image:0,1
Select method for rescaling pixels below the lower limit:Set to custom value
Enter custom value for pixels below lower limit:0
Select method for rescaling pixels above the upper limit:Set to custom value
Enter custom value for pixels below upper limit:1
Select image to match in maximum intensity:None
Enter the divisor:1
Select the measurement to use as a divisor:None

RescaleIntensity:[module_num:3|svn_version:‘6746’|variable_revision_number:2|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:rawCT
Name the output image:RescaledCT
Select rescaling method:Choose specific values to be reset to the full intensity range
How do you want to calculate the minimum intensity?:Custom
How do you want to calculate the maximum intensity?:Custom
Enter the lower limit for the intensity range for the input image:0
Enter the upper limit for the intensity range for the input image:0.0625
Enter the intensity range for the input image:0,0.0625
Enter the desired intensity range for the final, rescaled image:0,1
Select method for rescaling pixels below the lower limit:Set to custom value
Enter custom value for pixels below lower limit:0
Select method for rescaling pixels above the upper limit:Set to custom value
Enter custom value for pixels below upper limit:1
Select image to match in maximum intensity:None
Enter the divisor:1
Select the measurement to use as a divisor:None

RescaleIntensity:[module_num:4|svn_version:‘6746’|variable_revision_number:2|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:RawCY5
Name the output image:RescaledCY5
Select rescaling method:Choose specific values to be reset to the full intensity range
How do you want to calculate the minimum intensity?:Custom
How do you want to calculate the maximum intensity?:Custom
Enter the lower limit for the intensity range for the input image:0
Enter the upper limit for the intensity range for the input image:0.0625
Enter the intensity range for the input image:0,0.0625
Enter the desired intensity range for the final, rescaled image:0,1
Select method for rescaling pixels below the lower limit:Set to custom value
Enter custom value for pixels below lower limit:0
Select method for rescaling pixels above the upper limit:Set to custom value
Enter custom value for pixels below upper limit:1
Select image to match in maximum intensity:None
Enter the divisor:1
Select the measurement to use as a divisor:None

RescaleIntensity:[module_num:5|svn_version:‘6746’|variable_revision_number:2|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:RawFil
Name the output image:RescaledFil
Select rescaling method:Choose specific values to be reset to the full intensity range
How do you want to calculate the minimum intensity?:Custom
How do you want to calculate the maximum intensity?:Custom
Enter the lower limit for the intensity range for the input image:0
Enter the upper limit for the intensity range for the input image:0.0625
Enter the intensity range for the input image:0,0.0625
Enter the desired intensity range for the final, rescaled image:0,1
Select method for rescaling pixels below the lower limit:Set to custom value
Enter custom value for pixels below lower limit:0
Select method for rescaling pixels above the upper limit:Set to custom value
Enter custom value for pixels below upper limit:1
Select image to match in maximum intensity:None
Enter the divisor:1
Select the measurement to use as a divisor:None

LoadSingleImage:[module_num:6|svn_version:‘11340’|variable_revision_number:5|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Input image file location:Default Output Folder\x7CIllumination_correction
Filename of the image to load (Include the extension, e.g., .tif):illumination.tif
Load as images or objects?:Images
Name the image that will be loaded:Illum
Name this loaded object:Nuclei
Retain outlines of loaded objects?:No
Name the outlines:NucleiOutlines
Rescale intensities?:Yes

Smooth:[module_num:7|svn_version:‘11025’|variable_revision_number:1|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:Illum
Name the output image:FilteredIllum
Select smoothing method:Gaussian Filter
Calculate artifact diameter automatically?:No
Typical artifact diameter, in pixels:50.0
Edge intensity difference:0.1

RescaleIntensity:[module_num:8|svn_version:‘6746’|variable_revision_number:2|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:FilteredIllum
Name the output image:RescaledIllum
Select rescaling method:Choose specific values to be reset to the full intensity range
How do you want to calculate the minimum intensity?:Custom
How do you want to calculate the maximum intensity?:Custom
Enter the lower limit for the intensity range for the input image:0
Enter the upper limit for the intensity range for the input image:1
Enter the intensity range for the input image:0.0,0.0625
Enter the desired intensity range for the final, rescaled image:0.000000,1.000000
Select method for rescaling pixels below the lower limit:Set to custom value
Enter custom value for pixels below lower limit:0
Select method for rescaling pixels above the upper limit:Set to custom value
Enter custom value for pixels below upper limit:1.0
Select image to match in maximum intensity:None
Enter the divisor:1
Select the measurement to use as a divisor:None

CorrectIlluminationApply:[module_num:9|svn_version:‘11410’|variable_revision_number:3|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:RescaledPI
Name the output image:CorrPI
Select the illumination function:RescaledIllum
Select how the illumination function is applied:Divide

CorrectIlluminationApply:[module_num:10|svn_version:‘11410’|variable_revision_number:3|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:RescaledCT
Name the output image:CorrCT
Select the illumination function:RescaledIllum
Select how the illumination function is applied:Divide

CorrectIlluminationApply:[module_num:11|svn_version:‘11410’|variable_revision_number:3|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:RescaledFil
Name the output image:CorrFil
Select the illumination function:RescaledIllum
Select how the illumination function is applied:Divide

CorrectIlluminationApply:[module_num:12|svn_version:‘11410’|variable_revision_number:3|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:RescaledCY5
Name the output image:CorrCY5
Select the illumination function:RescaledIllum
Select how the illumination function is applied:Divide

IdentifyPrimaryObjects:[module_num:13|svn_version:‘11331’|variable_revision_number:9|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:CorrPI
Name the primary objects to be identified:Nuclei
Typical diameter of objects, in pixel units (Min,Max):14,55
Discard objects outside the diameter range?:Yes
Try to merge too small objects with nearby larger objects?:Yes
Discard objects touching the border of the image?:Yes
Select the thresholding method:Otsu Global
Threshold correction factor:1
Lower and upper bounds on threshold:0.002,1
Approximate fraction of image covered by objects?:0.2
Method to distinguish clumped objects:Intensity
Method to draw dividing lines between clumped objects:Intensity
Size of smoothing filter:6
Suppress local maxima that are closer than this minimum allowed distance:5
Speed up by using lower-resolution image to find local maxima?:Yes
Name the outline image:CellNucleus
Fill holes in identified objects?:Yes
Automatically calculate size of smoothing filter?:No
Automatically calculate minimum allowed distance between local maxima?:Yes
Manual threshold:0.0
Select binary image:Otsu Global
Retain outlines of the identified objects?:Yes
Automatically calculate the threshold using the Otsu method?:Yes
Enter Laplacian of Gaussian threshold:.5
Two-class or three-class thresholding?:Two classes
Minimize the weighted variance or the entropy?:Weighted variance
Assign pixels in the middle intensity class to the foreground or the background?:Foreground
Automatically calculate the size of objects for the Laplacian of Gaussian filter?:Yes
Enter LoG filter diameter:5
Handling of objects if excessive number of objects identified:Continue
Maximum number of objects:500
Select the measurement to threshold with:None
Method to calculate adaptive window size:Image size
Size of adaptive window:10

IdentifySecondaryObjects:[module_num:14|svn_version:‘11267’|variable_revision_number:8|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input objects:Nuclei
Name the objects to be identified:WholeCells
Select the method to identify the secondary objects:Propagation
Select the input image:CorrCT
Select the thresholding method:Otsu Global
Threshold correction factor:0.6
Lower and upper bounds on threshold:0.02,1
Approximate fraction of image covered by objects?:0.3
Number of pixels by which to expand the primary objects:15
Regularization factor:0.05
Name the outline image:CellOutlines
Manual threshold:0
Select binary image:Do not use
Retain outlines of the identified secondary objects?:Yes
Two-class or three-class thresholding?:Two classes
Minimize the weighted variance or the entropy?:Weighted variance
Assign pixels in the middle intensity class to the foreground or the background?:Foreground
Discard secondary objects that touch the edge of the image?:No
Discard the associated primary objects?:No
Name the new primary objects:FilteredNuclei
Retain outlines of the new primary objects?:No
Name the new primary object outlines:FilteredNucleiOutlines
Select the measurement to threshold with:None
Fill holes in identified objects?:Yes
Method to calculate adaptive window size:Image size
Size of adaptive window:10

IdentifyPrimaryObjects:[module_num:15|svn_version:‘11331’|variable_revision_number:9|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:CorrCY5
Name the primary objects to be identified:LBPA_object
Typical diameter of objects, in pixel units (Min,Max):1,50
Discard objects outside the diameter range?:Yes
Try to merge too small objects with nearby larger objects?:No
Discard objects touching the border of the image?:Yes
Select the thresholding method:Otsu Global
Threshold correction factor:1
Lower and upper bounds on threshold:0.1,1.0
Approximate fraction of image covered by objects?:0.01
Method to distinguish clumped objects:Intensity
Method to draw dividing lines between clumped objects:Intensity
Size of smoothing filter:10
Suppress local maxima that are closer than this minimum allowed distance:7
Speed up by using lower-resolution image to find local maxima?:Yes
Name the outline image:PrimaryOutlines
Fill holes in identified objects?:Yes
Automatically calculate size of smoothing filter?:Yes
Automatically calculate minimum allowed distance between local maxima?:Yes
Manual threshold:0.0
Select binary image:None
Retain outlines of the identified objects?:No
Automatically calculate the threshold using the Otsu method?:Yes
Enter Laplacian of Gaussian threshold:0.5
Two-class or three-class thresholding?:Two classes
Minimize the weighted variance or the entropy?:Weighted variance
Assign pixels in the middle intensity class to the foreground or the background?:Foreground
Automatically calculate the size of objects for the Laplacian of Gaussian filter?:Yes
Enter LoG filter diameter:5
Handling of objects if excessive number of objects identified:Continue
Maximum number of objects:500
Select the measurement to threshold with:None
Method to calculate adaptive window size:Image size
Size of adaptive window:10

IdentifyPrimaryObjects:[module_num:16|svn_version:‘11331’|variable_revision_number:9|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:CorrFil
Name the primary objects to be identified:Fil_object
Typical diameter of objects, in pixel units (Min,Max):1,50
Discard objects outside the diameter range?:Yes
Try to merge too small objects with nearby larger objects?:No
Discard objects touching the border of the image?:Yes
Select the thresholding method:Otsu Global
Threshold correction factor:1
Lower and upper bounds on threshold:0.1,1.0
Approximate fraction of image covered by objects?:0.01
Method to distinguish clumped objects:Intensity
Method to draw dividing lines between clumped objects:Intensity
Size of smoothing filter:10
Suppress local maxima that are closer than this minimum allowed distance:7
Speed up by using lower-resolution image to find local maxima?:Yes
Name the outline image:PrimaryOutlines
Fill holes in identified objects?:Yes
Automatically calculate size of smoothing filter?:Yes
Automatically calculate minimum allowed distance between local maxima?:Yes
Manual threshold:0.0
Select binary image:None
Retain outlines of the identified objects?:No
Automatically calculate the threshold using the Otsu method?:Yes
Enter Laplacian of Gaussian threshold:0.5
Two-class or three-class thresholding?:Two classes
Minimize the weighted variance or the entropy?:Weighted variance
Assign pixels in the middle intensity class to the foreground or the background?:Foreground
Automatically calculate the size of objects for the Laplacian of Gaussian filter?:Yes
Enter LoG filter diameter:5
Handling of objects if excessive number of objects identified:Continue
Maximum number of objects:500
Select the measurement to threshold with:None
Method to calculate adaptive window size:Image size
Size of adaptive window:10

OverlayOutlines:[module_num:17|svn_version:‘11025’|variable_revision_number:2|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Display outlines on a blank image?:No
Select image on which to display outlines:CorrCT
Name the output image:CellNucleusOverlay
Select outline display mode:Color
Select method to determine brightness of outlines:Max of image
Width of outlines:1
Select outlines to display:CellNucleus
Select outline color:White

OverlayOutlines:[module_num:18|svn_version:‘11025’|variable_revision_number:2|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Display outlines on a blank image?:No
Select image on which to display outlines:CellNucleusOverlay
Name the output image:CellOutlinesOverlay
Select outline display mode:Color
Select method to determine brightness of outlines:Max of image
Width of outlines:1
Select outlines to display:CellOutlines
Select outline color:White

RescaleIntensity:[module_num:19|svn_version:‘6746’|variable_revision_number:2|show_window:True|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the input image:CellOutlinesOverlay
Name the output image:RescaledCellOutlinesOverlay
Select rescaling method:Stretch each image to use the full intensity range
How do you want to calculate the minimum intensity?:Custom
How do you want to calculate the maximum intensity?:Custom
Enter the lower limit for the intensity range for the input image:0
Enter the upper limit for the intensity range for the input image:1
Enter the intensity range for the input image:0.000000,1.000000
Enter the desired intensity range for the final, rescaled image:0.000000,1.000000
Select method for rescaling pixels below the lower limit:Mask pixels
Enter custom value for pixels below lower limit:0
Select method for rescaling pixels above the upper limit:Mask pixels
Enter custom value for pixels below upper limit:0
Select image to match in maximum intensity:None
Enter the divisor:1
Select the measurement to use as a divisor:None

SaveImages:[module_num:20|svn_version:‘11424’|variable_revision_number:7|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select the type of image to save:Image
Select the image to save:RescaledCellOutlinesOverlay
Select the objects to save:None
Select the module display window to save:CellOutlinesOverlay
Select method for constructing file names:Sequential numbers
Select image name for file prefix:None
Enter file prefix:segmentation
Do you want to add a suffix to the image file name?:No
Text to append to the image name:Do not use
Select file format to use:png
Output file location:Default Output Folder\x7CNone
Image bit depth:8
Overwrite existing files without warning?:No
Select how often to save:Every cycle
Rescale the images? :No
Save as grayscale or color image?:Grayscale
Select colormap:gray
Store file and path information to the saved image?:No
Create subfolders in the output folder?:No

MeasureObjectIntensity:[module_num:21|svn_version:‘11135’|variable_revision_number:3|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Hidden:1
Select an image to measure:CorrPI
Select objects to measure:Nuclei

MeasureObjectIntensity:[module_num:22|svn_version:‘11135’|variable_revision_number:3|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Hidden:1
Select an image to measure:CorrCT
Select objects to measure:Nuclei

MeasureObjectIntensity:[module_num:23|svn_version:‘11135’|variable_revision_number:3|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Hidden:1
Select an image to measure:CorrFil
Select objects to measure:Nuclei

MeasureObjectIntensity:[module_num:24|svn_version:‘11135’|variable_revision_number:3|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Hidden:1
Select an image to measure:CorrCY5
Select objects to measure:Nuclei

MeasureObjectIntensity:[module_num:25|svn_version:‘11135’|variable_revision_number:3|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Hidden:1
Select an image to measure:CorrPI
Select objects to measure:WholeCells

MeasureObjectIntensity:[module_num:26|svn_version:‘11135’|variable_revision_number:3|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Hidden:1
Select an image to measure:CorrCT
Select objects to measure:WholeCells

MeasureObjectIntensity:[module_num:27|svn_version:‘11135’|variable_revision_number:3|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Hidden:1
Select an image to measure:CorrFil
Select objects to measure:WholeCells

MeasureObjectIntensity:[module_num:28|svn_version:‘11135’|variable_revision_number:3|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Hidden:1
Select an image to measure:CorrCY5
Select objects to measure:WholeCells

MeasureObjectSizeShape:[module_num:29|svn_version:‘11400’|variable_revision_number:1|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select objects to measure:Nuclei
Calculate the Zernike features?:Yes

MeasureObjectSizeShape:[module_num:30|svn_version:‘11400’|variable_revision_number:1|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select objects to measure:WholeCells
Calculate the Zernike features?:Yes

MeasureObjectSizeShape:[module_num:31|svn_version:‘11400’|variable_revision_number:1|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select objects to measure:Fil_object
Calculate the Zernike features?:Yes

MeasureObjectSizeShape:[module_num:32|svn_version:‘11400’|variable_revision_number:1|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select objects to measure:LBPA_object
Calculate the Zernike features?:Yes

MeasureGranularity:[module_num:33|svn_version:‘11025’|variable_revision_number:3|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Image count:1
Object count:0
Select an image to measure:CorrCY5
Subsampling factor for granularity measurements:0.25
Subsampling factor for background reduction:0.25
Radius of structuring element:10
Range of the granular spectrum:16

MeasureGranularity:[module_num:34|svn_version:‘11025’|variable_revision_number:3|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Image count:1
Object count:0
Select an image to measure:CorrFil
Subsampling factor for granularity measurements:0.25
Subsampling factor for background reduction:0.25
Radius of structuring element:10
Range of the granular spectrum:16

ExportToSpreadsheet:[module_num:35|svn_version:‘11368’|variable_revision_number:7|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)]
Select or enter the column delimiter:Tab
Prepend the output file name to the data file names?:Yes
Add image metadata columns to your object data file?:No
Limit output to a size that is allowed in Excel?:No
Select the columns of measurements to export?:No
Calculate the per-image mean values for object measurements?:No
Calculate the per-image median values for object measurements?:No
Calculate the per-image standard deviation values for object measurements?:No
Output file location:Default Output Folder\x7C.
Create a GenePattern GCT file?:No
Select source of sample row name:Metadata
Select the image to use as the identifier:None
Select the metadata to use as the identifier:None
Export all measurements, using default file names?:No
Press button to select measurements to export:None\x7CNone
Data to export:Nuclei
Combine these object measurements with those of the previous object?:No
File name:Nuclei.csv
Use the object name for the file name?:No
Data to export:WholeCells
Combine these object measurements with those of the previous object?:No
File name:WholeCells.csv
Use the object name for the file name?:No
Data to export:Image
Combine these object measurements with those of the previous object?:No
File name:Image.csv
Use the object name for the file name?:No
Data to export:Experiment
Combine these object measurements with those of the previous object?:No
File name:Experiment.csv
Use the object name for the file name?:No
[/size]

I would guess open files, first. Can you post the first three image sets and the illum correction image? I will generate a fake set of images by copying and investigate.

By the way, what OS are you using?

Sorry, not sure I understand what are

?

Here are OS details:

$ cat /etc/redhat-release
CentOS release 5.4 (Final)

$ uname -a
Linux lipidx.vital-it.ch 2.6.18-164.15.1.el5 #1 SMP Wed Mar 17 11:30:06 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 204800
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 204800
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

I would like the first three images (well, first three pairs) to try running the pipeline myself to investigate the crash. I will also need a copy of the illumination file.

Hi,

Please find the zip file containing 3 wells x 4 images + illumination file at:
ludwig-sun1.unil.ch/~dkuznets/3Wells.zip

Thanks,
Dmitry.

I may have found the bug, but it will take some time to be sure. In the meantime, you might try patching CellProfiler/bioformats/formatreader.py with the change below, and see if that fixes the problem.

[code]----------------------------------

diff --git a/bioformats/formatreader.py b/bioformats/formatreader.py
index d514b90…7f2061c 100644
— a/bioformats/formatreader.py
+++ b/bioformats/formatreader.py
@@ -456,6 +456,7 @@ def load_using_bioformats(path, c=None, z=0, t=0,
series=None, index=None,
image.shape = (height,width)

    rdr.close()
  •    jutil.call(stream, 'close', '()V')
      del rdr
      #
      # Run the Java garbage collector here.[/code]

Thanks for submitting this and, Ray, thanks for the patch. I’ve updated bioformats/formatreader.py (svn revision 11574) which you can pick up and see if it helps.

–Lee

Hi,

I applied the patch (well, I just added this single line into formatreader.py). Unfortunately, it crashed again with the error message:

[size=85]Exception in thread “main” loci.formats.MissingLibraryException: Java Advanced Imaging (JAI) is required to read some TIFF files. Please install JAI from jai.dev.java.net/
at loci.formats.in.TiffJAIReader.initFile(TiffJAIReader.java:103)
at loci.formats.FormatReader.setId(FormatReader.java:1072)
at loci.formats.DelegateReader.setId(DelegateReader.java:239)
Caused by: loci.common.ReflectException: No such class:
javax.media.jai.NullOpImage
at loci.common.ReflectedUniverse.exec(ReflectedUniverse.java:153)
at loci.formats.in.TiffJAIReader.initFile(TiffJAIReader.java:96)
… 2 more
Caused by: java.lang.ClassNotFoundException: javax.media.jai.NullOpImage
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at loci.common.ReflectedUniverse.exec(ReflectedUniverse.java:145)
… 3 more
ERROR:cellprofiler.utilities.jutil:Caught exception when executing closure Traceback (most recent call last):
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 299, in synchronous_closure
File
"/local/openbis/CP2/out/CellProfiler/bioformats/formatreader.py", line 377, in fn
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 502, in method
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 440, in call
JavaException: Java Advanced Imaging (JAI) is required to read some TIFF files. Please install JAI from jai.dev.java.net/ Caught exception when executing closure Traceback (most recent call last):
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 299, in synchronous_closure
File
"/local/openbis/CP2/out/CellProfiler/bioformats/formatreader.py", line 377, in fn
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 502, in method
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 440, in call
JavaException: Java Advanced Imaging (JAI) is required to read some TIFF files. Please install JAI from jai.dev.java.net/ WARNING:cellprofiler.modules.loadimages:Failed to load /scratch/local/tmp/omartin/metadata/component/Prestwick_05092011_MZ/BSF006280_000/illumination.tif
with bioformats. Use PIL instead
Traceback (most recent call last):
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/loadimages.py", line 2939, in provide_image
File
"/local/openbis/CP2/out/CellProfiler/bioformats/formatreader.py", line 469, in load_using_bioformats
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 308, in run_in_main_thread
JavaException: Java Advanced Imaging (JAI) is required to read some TIFF files. Please install JAI from jai.dev.java.net/ Failed to load /scratch/local/tmp/omartin/metadata/component/Prestwick_05092011_MZ/BSF006280_000/illumination.tif
with bioformats. Use PIL instead
Traceback (most recent call last):
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/loadimages.py", line 2939, in provide_image
File
"/local/openbis/CP2/out/CellProfiler/bioformats/formatreader.py", line 469, in load_using_bioformats
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/utilities/jutil.py",
line 308, in run_in_main_thread
JavaException: Java Advanced Imaging (JAI) is required to read some TIFF files. Please install JAI from jai.dev.java.net/ ERROR:cellprofiler.pipeline:Error detected during run of module Smooth Traceback (most recent call last):
File “/local/openbis/CP2/out/CellProfiler/cellprofiler/pipeline.py”,
line 1281, in run_with_yield
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/smooth.py",
line 138, in run
File “/local/openbis/CP2/out/CellProfiler/cellprofiler/cpimage.py”,
line 549, in get_image
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/loadimages.py", line 2946, in provide_image
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/loadimages.py", line 2984, in load_using_PIL
ImportError: No module named tifffile
Error detected during run of module Smooth Traceback (most recent call last):
File “/local/openbis/CP2/out/CellProfiler/cellprofiler/pipeline.py”,
line 1281, in run_with_yield
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/smooth.py",
line 138, in run
File “/local/openbis/CP2/out/CellProfiler/cellprofiler/cpimage.py”,
line 549, in get_image
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/loadimages.py", line 2946, in provide_image
File
"/local/openbis/CP2/out/CellProfiler/cellprofiler/modules/loadimages.py", line 2984, in load_using_PIL
ImportError: No module named tifffile
INFO:PipelineStatistics:Mon Oct 3 21:28:25 2011: Image # 101, module Smooth # 7: 0.08 sec (bg) Mon Oct 3 21:28:25 2011: Image # 101, module Smooth # 7: 0.08 sec (bg) WARNING:root:Caught exception while killing ijbridge.
Traceback (most recent call last):
File “CP2/out/CellProfiler/CellProfiler.py”, line 484, in
ImportError: No module named ijbridge
Caught exception while killing ijbridge.
Traceback (most recent call last):
File “CP2/out/CellProfiler/CellProfiler.py”, line 484, in
ImportError: No module named ijbridge[/size]

Was testing on your side successful? Any ideas?
Dmitry.

Hi Dmitry,

I didn’t test to the full 101 cycles. After seeing after a few cycles that images were still open (via “lsof”), I tracked down the bug causing that and posted the fix.

Could you try this:

  • run for about 10 cycles
  • run “lsof -p <process_ID_for_CP>”
  • post the results?

It’s possible it’s a memory usage problem. I’ll investigate that in the meantime.

Hi,

I must correct my message - after I have REALLY applied the patch (sorry for stupidity!), the problem with open image files is gone.
… waiting CP to pass all 384 wells.

Sorry for confusion.
Dmitry.

Hi,

The run went through without problem, hence the problem is solved.
Thanks for your assistance,

Dmitry.