Trouble identifying nuclear foci in 2.0

Hi all, I had a good pipeline working for identifying small nuclear foci, but when I switched to 2.0 it stopped working well. In the attached image it’s only detecting 2 foci, when there are obviously many more. I’m using a single image, identifying the nuclei first with one thresholding method (Otsu Global), then cropping the image using the nuclei objects, and finally identifying the foci using another thresholding method (Robust Background Per Object). All my settings are the same as they were with 1.0, but the output is different (~300 foci vs 2 foci). Any help would be great.


Anybody have thoughts as to why the two versions are giving different results?
If it’s helpful, here are my Identify Primary Objects Parameters:

Typical diameter of objects, in pixel units (Min,Max):1,15
Discard objects outside the diameter range?:Yes
Try to merge too small objects with nearby larger objects?:No
Discard objects touching the border of the image?:Yes
Select the thresholding method:RobustBackground Global
Threshold correction factor:1.1
Lower and upper bounds on threshold:0.01,1
Approximate fraction of image covered by objects?:0.01
Method to distinguish clumped objects:Intensity
Method to draw dividing lines between clumped objects:Intensity
Size of smoothing filter:1
Suppress local maxima that are closer than this minimum allowed distance:0
Speed up by using lower-resolution image to find local maxima?:Yes
Name the outline image:None
Fill holes in identified objects?:Yes
Automatically calculate size of smoothing filter?:No
Automatically calculate minimum allowed distance between local maxima?:No
Manual threshold:0.0
Select binary image:RobustBackground PerObject
Retain outlines of the identified objects?:No
Automatically calculate the threshold using the Otsu method?:Yes
Enter Laplacian of Gaussian threshold:.5
Two-class or three-class thresholding?:Two classes
Minimize the weighted variance or the entropy?:Weighted variance
Assign pixels in the middle intensity class to the foreground or the background?:Foreground
Automatically calculate the size of objects for the Laplacian of Gaussian filter?:Yes
Enter LoG filter diameter:5
Handling of objects if excessive number of objects identified:Continue
Maximum number of objects:500
Select the measurement to threshold with:None

Could you post your CP 1.0 and 2.0 pipelines so we can compare?
Thanks!
-Mark

I’ve pasted both pipelines in their entirety below. Just FYI, I compared the second Identify Primary Automatic/Objects Module (which is the one that’s failing) between the two versions and, so far as I can tell, all the parameters are identical, which makes sense considering I created the 2.0 pipeline (.cp) by opening the 1.0 pipeline (.mat) in CellProfiler 2.0, modifying the couple things that needed updated (i.e. the Crop module), then saving it as a 2.0 pipeline (.cp).

1.0 Pipeline:

Pixel Size: 1

Pipeline:
LoadImages
IdentifyPrimAutomatic
Crop
IdentifyPrimAutomatic
Relate
ExportToExcel

Module #1: LoadImages revision - 5
How do you want to load these files? Text-Exact match
Type the text that one type of image has in common (for TEXT options), or their position in each group (for ORDER option): 24hr XRA L3P 20x 1
What do you want to call these images within CellProfiler? RGB
Type the text that one type of image has in common (for TEXT options), or their position in each group (for ORDER option). Type “Do not use” to ignore: Do not use
What do you want to call these images within CellProfiler? (Type “Do not use” to ignore) Do not use
Type the text that one type of image has in common (for TEXT options), or their position in each group (for ORDER option): Do not use
What do you want to call these images within CellProfiler? Do not use
Type the text that one type of image has in common (for TEXT options), or their position in each group (for ORDER option): Do not use
What do you want to call these images within CellProfiler? Do not use
If using ORDER, how many images are there in each group (i.e. each field of view)? 3
If you want to exclude files, type the text that the excluded images have in common (for TEXT options). Type “Do not use” to ignore. Do not use
What type of files are you loading? individual images
Analyze all subfolders within the selected folder? No
Enter the path name to the folder where the images to be loaded are located. Type period (.) for default image folder or ampersand (&) for default output folder. .
If the images you are loading are binary (black/white only), in what format do you want to store them? grayscale
If Yes to “Analyze all subfolders”, do you want to select the subfolders to process? No
Do you want to check image sets for missing or duplicate files? Tokens must be defined for the unique parts of the string. (REGULAR EXPRESSIONS ONLY) No
Note - If the movies contain more than just one image type (e.g., brightfield, fluorescent, field-of-view), add the GroupMovieFrames module.

Module #2: IdentifyPrimAutomatic revision - 12
What did you call the images you want to process? RGB
What do you want to call the objects identified by this module? Nuclei
Typical diameter of objects, in pixel units (Min,Max): 30,140
Discard objects outside the diameter range? Yes
Try to merge too small objects with nearby larger objects? No
Discard objects touching the border of the image? Yes
Select an automatic thresholding method or enter an absolute threshold in the range [0,1]. To choose a binary image, select “Other” and type its name. Choosing ‘‘All’’ will use the Otsu Global method to calculate a single threshold for the entire image group. The other methods calculate a threshold for each image individually. “Set interactively” will allow you to manually adjust the threshold during the first cycle to determine what will work well. Otsu Global
Threshold correction factor 3
Lower and upper bounds on threshold, in the range [0,1] 0.07,1
For MoG thresholding, what is the approximate fraction of image covered by objects? 10 Method to distinguish clumped objects (see help for details): Shape
Method to draw dividing lines between clumped objects (see help for details): Distance
Size of smoothing filter, in pixel units (if you are distinguishing between clumped objects). Enter 0 for low resolution images with small objects (~< 5 pixel diameter) to prevent any image smoothing. Automatic
Suppress local maxima within this distance, (a positive integer, in pixel units) (if you are distinguishing between clumped objects) Automatic
Speed up by using lower-resolution image to find local maxima? (if you are distinguishing between clumped objects) Yes
Enter the following information, separated by commas, if you would like to use the Laplacian of Gaussian method for identifying objects instead of using the above settings: Size of neighborhood(height,width),Sigma,Minimum Area,Size for Wiener Filter(height,width),Threshold Do not use
What do you want to call the outlines of the identified objects (optional)? Do not use
Do you want to fill holes in identified objects? Yes
Do you want to run in test mode where each method for distinguishing clumped objects is compared? No

Module #3: Crop revision - 5
What did you call the image to be cropped? RGB
What do you want to call the cropped image? GrayRed
Into which shape would you like to crop? See the help for several other options. Nuclei
For RECTANGLE + ELLIPSE, would you like to crop by typing in pixel coordinates or clicking with the mouse? Coordinates
Should the cropping pattern in the first image cycle be applied to all subsequent image cycles (First option) or should each image cycle be cropped individually? Individually
For COORDINATES + FIRST + RECTANGLE, specify the (Left, Right) pixel positions (the word “end” can be substituted for right pixel if you do not want to crop the right edge) 1,100
For COORDINATES + FIRST + RECTANGLE, specify the (Top, Bottom) pixel positions (the word “end” can be substituted for bottom pixel if you do not want to crop the bottom edge) 1,100
For COORDINATES + FIRST + ELLIPSE, what is the center pixel position of the ellipse in form X,Y? 500,500
For COORDINATES + FIRST + ELLIPSE, what is the radius of the ellipse in the X direction? 400
For COORDINATES + FIRST + ELLIPSE, what is the radius of the ellipse in the Y direction? 200
Do you want to use Plate Fix? (see Help, only used when cropping based on previously identified objects) No
Do you want to remove rows and columns that lack objects? (see Help) No

Module #4: IdentifyPrimAutomatic revision - 12
What did you call the images you want to process? GrayRed
What do you want to call the objects identified by this module? foci
Typical diameter of objects, in pixel units (Min,Max): 1,15
Discard objects outside the diameter range? Yes
Try to merge too small objects with nearby larger objects? No
Discard objects touching the border of the image? Yes
Select an automatic thresholding method or enter an absolute threshold in the range [0,1]. To choose a binary image, select “Other” and type its name. Choosing ‘‘All’’ will use the Otsu Global method to calculate a single threshold for the entire image group. The other methods calculate a threshold for each image individually. “Set interactively” will allow you to manually adjust the threshold during the first cycle to determine what will work well. RobustBackground PerObject
Threshold correction factor 1.1
Lower and upper bounds on threshold, in the range [0,1] 0.01,1
For MoG thresholding, what is the approximate fraction of image covered by objects? 0.01
Method to distinguish clumped objects (see help for details): Intensity
Method to draw dividing lines between clumped objects (see help for details): Intensity
Size of smoothing filter, in pixel units (if you are distinguishing between clumped objects). Enter 0 for low resolution images with small objects (~< 5 pixel diameter) to prevent any image smoothing. 1
Suppress local maxima within this distance, (a positive integer, in pixel units) (if you are distinguishing between clumped objects) 0
Speed up by using lower-resolution image to find local maxima? (if you are distinguishing between clumped objects) Yes
Enter the following information, separated by commas, if you would like to use the Laplacian of Gaussian method for identifying objects instead of using the above settings: Size of neighborhood(height,width),Sigma,Minimum Area,Size for Wiener Filter(height,width),Threshold Do not use
What do you want to call the outlines of the identified objects (optional)? Do not use
Do you want to fill holes in identified objects? Yes
Do you want to run in test mode where each method for distinguishing clumped objects is compared? No

Module #5: Relate revision - 4
What objects are the children objects (subobjects)? foci
What are the parent objects? Nuclei
Do you want to calculate distances of each child to its parent, and if so, what kind? Do not use
(If ‘‘Yes’’ to above) What other object do you want to find distances to? (Must be one object per parent object, e.g. Nuclei) Do not use
Do you want to generate per-parent means for all child measurements? No

Module #6: ExportToExcel revision - 3
Which objects do you want to export? Nuclei
Do not use
Do not use
Do not use
Do not use
Do not use
Do not use
Do not use
Enter the directory where the Excel files are to be saved. Type period (.) to use the default output folder or ampersand (&) for the default input folder. If a FileNameMetadata module was used, metadata tokens may be used here. If this directory does not exist, it will be created automatically. .
What prefix should be used to name the Excel files? An underscore will be added to the end of the prefix automatically. Metadata tokens may be used here. Use “Do not use” to prepend the Output filename to the file. Do not use

2.0 Pipeline

ellProfiler Pipeline: cellprofiler.org
Version:1
SVNRevision:10211

LoadImages:[module_num:1|svn_version:‘10211’|variable_revision_number:7|show_window:True|notes:\x5B\x5D]
File type to be loaded:individual images
File selection method:Text-Exact match
Number of images in each group?:3
Type the text that the excluded images have in common:Do not use
Analyze all subfolders within the selected folder?:No
Input image file location:Default Input Folder\x7C.
Check image sets for missing or duplicate files?:
Group images by metadata?:No
Exclude certain files?:No
Specify metadata fields to group by:
Image count:1
Text that these images have in common (case-sensitive):24hr XRA L3P 20x 1
Position of this image in each group:HCA2
Extract metadata from where?:None
Regular expression that finds metadata in the file name:None
Type the regular expression that finds metadata in the subfolder path:None
Channel count:1
Group movie frames?:No
Interleaving\x3A:Interleaved
Channels per group\x3A:2
Name this loaded image:RGB
Channel number\x3A:1

IdentifyPrimaryObjects:[module_num:2|svn_version:‘10041’|variable_revision_number:7|show_window:True|notes:\x5B\x5D]
Select the input image:RGB
Name the primary objects to be identified:Nuclei
Typical diameter of objects, in pixel units (Min,Max):30,140
Discard objects outside the diameter range?:Yes
Try to merge too small objects with nearby larger objects?:No
Discard objects touching the border of the image?:Yes
Select the thresholding method:Otsu Global
Threshold correction factor:3
Lower and upper bounds on threshold:0.07,1
Approximate fraction of image covered by objects?:10%
Method to distinguish clumped objects:Shape
Method to draw dividing lines between clumped objects:Distance
Size of smoothing filter:10
Suppress local maxima that are closer than this minimum allowed distance:5
Speed up by using lower-resolution image to find local maxima?:Yes
Name the outline image:None
Fill holes in identified objects?:Yes
Automatically calculate size of smoothing filter?:Yes
Automatically calculate minimum allowed distance between local maxima?:Yes
Manual threshold:0.0
Select binary image:Otsu Global
Retain outlines of the identified objects?:No
Automatically calculate the threshold using the Otsu method?:Yes
Enter Laplacian of Gaussian threshold:.5
Two-class or three-class thresholding?:Two classes
Minimize the weighted variance or the entropy?:Weighted variance
Assign pixels in the middle intensity class to the foreground or the background?:Foreground
Automatically calculate the size of objects for the Laplacian of Gaussian filter?:Yes
Enter LoG filter diameter:5
Handling of objects if excessive number of objects identified:Continue
Maximum number of objects:500
Select the measurement to threshold with:None

Crop:[module_num:3|svn_version:‘10211’|variable_revision_number:2|show_window:True|notes:\x5B\x5D]
Select the input image:RGB
Name the output image:GrayRed
Select the cropping shape:Objects
Select the cropping method:Coordinates
Apply which cycle’s cropping pattern?:Every
Left and right rectangle positions:1,100
Top and bottom rectangle positions:1,100
Coordinates of ellipse center:500,500
Ellipse radius, X direction:400
Ellipse radius, Y direction:200
Use Plate Fix?:No
Remove empty rows and columns?:No
Select the masking image:Nuclei
Select the image with a cropping mask:None
Select the objects:Nuclei

IdentifyPrimaryObjects:[module_num:4|svn_version:‘10041’|variable_revision_number:7|show_window:True|notes:\x5B\x5D]
Select the input image:GrayRed
Name the primary objects to be identified:foci
Typical diameter of objects, in pixel units (Min,Max):1,15
Discard objects outside the diameter range?:Yes
Try to merge too small objects with nearby larger objects?:No
Discard objects touching the border of the image?:Yes
Select the thresholding method:RobustBackground PerObject
Threshold correction factor:1.1
Lower and upper bounds on threshold:0.01,1
Approximate fraction of image covered by objects?:0.01
Method to distinguish clumped objects:Intensity
Method to draw dividing lines between clumped objects:Intensity
Size of smoothing filter:1
Suppress local maxima that are closer than this minimum allowed distance:0
Speed up by using lower-resolution image to find local maxima?:Yes
Name the outline image:None
Fill holes in identified objects?:Yes
Automatically calculate size of smoothing filter?:No
Automatically calculate minimum allowed distance between local maxima?:No
Manual threshold:0.0
Select binary image:RobustBackground PerObject
Retain outlines of the identified objects?:No
Automatically calculate the threshold using the Otsu method?:Yes
Enter Laplacian of Gaussian threshold:.5
Two-class or three-class thresholding?:Two classes
Minimize the weighted variance or the entropy?:Weighted variance
Assign pixels in the middle intensity class to the foreground or the background?:Foreground
Automatically calculate the size of objects for the Laplacian of Gaussian filter?:Yes
Enter LoG filter diameter:5
Handling of objects if excessive number of objects identified:Continue
Maximum number of objects:500
Select the measurement to threshold with:None

RelateObjects:[module_num:5|svn_version:‘10039’|variable_revision_number:2|show_window:True|notes:\x5B\x5D]
Select the input child objects:foci
Select the input parent objects:Nuclei
Calculate distances?:None
Calculate per-parent means for all child measurements?:Yes
Calculate distances to other parents?:No
Parent name:Do not use

ExportToSpreadsheet:[module_num:6|svn_version:‘10036’|variable_revision_number:7|show_window:True|notes:\x5B\x5D]
Select or enter the column delimiter:Tab
Prepend the output file name to the data file names?:Yes
Add image metadata columns to your object data file?:No
Limit output to a size that is allowed in Excel?:No
Select the columns of measurements to export?:No
Calculate the per-image mean values for object measurements?:No
Calculate the per-image median values for object measurements?:No
Calculate the per-image standard deviation values for object measurements?:No
Output file location:Default Output Folder\x7C.
Create a GenePattern GCT file?:No
Select source of sample row name:Metadata
Select the image to use as the identifier:None
Select the metadata to use as the identifier:None
Export all measurements?:No
Press button to select measurements to export:None\x7CNone
Data to export:Nuclei
Combine these object measurements with those of the previous object?:No
File name:Nuclei.csv
Use the object name for the file name?:No

Sorry, I should have been more clear: Could you upload the CP pipelines (.mat, .cp files) in a follow-up post, not just post the text file version.
-Mark

oh, of course - my bad.
Red channel only PIPE.cp (6.9 KB)
Red channel only PIPE.mat (1.11 KB)

Hi,

This is due to a subtle difference in how the maxima are calculated between CP 1.0 and 2.0. It does seem like it’s bug, so we’ll work to fix it. In the meantime, a workaround is to increase the allowed distance slightly; setting it to 5 pixels seems to get results similar to what you had with your CP 1.0 pipeline.

Regards,
-Mark