CPA score all error

I am trying to use classifier in CPA to generate some rules to eliminate objects from my data set that are not objects I want. The training process is working great except the thumbnail images that CPA brings up does not show the entire object. This really isn’t a big problem though, I can tell rather quickly if the object is good or not. The real issue is when I go to “score all” the following error occurs,

An error occurred in the program:
AssertionError: Number of column labels does not match the number of columns in data.

Traceback (most recent call last):
File “classifier.pyc”, line 1290, in ScoreAll
File “tableviewer.pyc”, line 712, in table_from_array
File “tableviewer.pyc”, line 173, in init

Thanks for any help you can provide.

Toby

Hi Toby,

You can increase the thumbnail size by a setting in the properties file toward the bottom, called image_tile_size.

As for

Did you export to a a single object table with multiple objects? If so, that could be your problem. In ExportToDatabase, it is easiest/best to choose “Export to multiple object tables” (or some such text). Or you need to manually go into the database and edit out the other objects.

If you did export to multiple object tables, was there any other manipulation of the database? In your properties file, I would check first:
(1) Under Database Columns section, check that the object that you are intending to classify is named in “cell_x_loc” and “cell_y_loc”
(2) Under ==Image Path and Filename Columns== check that the number of parameters for all the entries is the same (which should equal the number of channels)

Feel free to post your properties file here and we can see if there is a configuration issue.

Thanks,
David

Hi David,
I had only one type of object identified in my images, so I used single object table. Should I change this to single object view or one table per object type?

Ptoperties file:

#Tue Nov 25 09:28:26 2014

==============================================

CellProfiler Analyst 2.0 properties file

==============================================

==== Database Info ====

db_type = mysql
db_port =
db_host =
db_name = DefaultDB
db_user =
db_passwd =

==== Database Tables ====

image_table = MyExpt_Per_Image
object_table = MyExpt_Per_Object

==== Database Columns ====

Specify the database column names that contain unique IDs for images and

objects (and optionally tables).

table_id (OPTIONAL): This field lets Classifier handle multiple tables if

you merge them into one and add a table_number column as a foreign

key to your per-image and per-object tables.

image_id: must be a foreign key column between your per-image and per-object

tables

object_id: the object key column from your per-object table

image_id = ImageNumber
object_id = ObjectNumber
plate_id =
well_id =

Also specify the column names that contain X and Y coordinates for each

object within an image.

cell_x_loc = SE_Location_Center_X
cell_y_loc = SE_Location_Center_Y

==== Image Path and File Name Columns ====

Classifier needs to know where to find the images from your experiment.

Specify the column names from your per-image table that contain the image

paths and file names here.

Individual image files are expected to be monochromatic and represent a single

channel. However, any number of images may be combined by adding a new channel

path and filename column to the per-image table of your database and then

adding those column names here.

NOTE: These lists must have equal length!

image_path_cols = Image_PathName_SE
image_file_cols = Image_FileName_SE

CPA will now read image thumbnails directly from the database, if chosen in ExportToDatabase.

image_thumbnail_cols =

Give short names for each of the channels (respectively)…

image_names = SE

Specify a default color for each of the channels (respectively)

Valid colors are: [red, green, blue, magenta, cyan, yellow, gray, none]

image_channel_colors = gray,

==== Image Accesss Info ====

image_url_prepend =

==== Dynamic Groups ====

Here you can define groupings to choose from when classifier scores your experiment. (eg: per-well)

This is OPTIONAL, you may leave "groups = ".

FORMAT:

group_XXX = MySQL select statement that returns image-keys and group-keys. This will be associated with the group name “XXX” from above.

EXAMPLE GROUPS:

groups = Well, Gene, Well+Gene,

group_SQL_Well = SELECT Per_Image_Table.TableNumber, Per_Image_Table.ImageNumber, Per_Image_Table.well FROM Per_Image_Table

group_SQL_Gene = SELECT Per_Image_Table.TableNumber, Per_Image_Table.ImageNumber, Well_ID_Table.gene FROM Per_Image_Table, Well_ID_Table WHERE Per_Image_Table.well=Well_ID_Table.well

group_SQL_Well+Gene = SELECT Per_Image_Table.TableNumber, Per_Image_Table.ImageNumber, Well_ID_Table.well, Well_ID_Table.gene FROM Per_Image_Table, Well_ID_Table WHERE Per_Image_Table.well=Well_ID_Table.well

==== Image Filters ====

Here you can define image filters to let you select objects from a subset of your experiment when training the classifier.

FORMAT:

filter_SQL_XXX = MySQL select statement that returns image keys you wish to filter out. This will be associated with the filter name “XXX” from above.

EXAMPLE FILTERS:

filters = EMPTY, CDKs,

filter_SQL_EMPTY = SELECT TableNumber, ImageNumber FROM CPA_per_image, Well_ID_Table WHERE CPA_per_image.well=Well_ID_Table.well AND Well_ID_Table.Gene=“EMPTY”

filter_SQL_CDKs = SELECT TableNumber, ImageNumber FROM CPA_per_image, Well_ID_Table WHERE CPA_per_image.well=Well_ID_Table.well AND Well_ID_Table.Gene REGEXP ‘CDK.*’

==== Meta data ====

What are your objects called?

FORMAT:

object_name = singular object name, plural object name,

object_name = cell, cells,

What size plates were used? 96, 384 or 5600? This is for use in the PlateViewer. Leave blank if none

plate_type =

==== Excluded Columns ====

OPTIONAL

Classifier uses columns in your per_object table to find rules. It will

automatically ignore ID columns defined in table_id, image_id, and object_id

as well as any columns that contain non-numeric data.

Here you may list other columns in your per_object table that you wish the

classifier to ignore when finding rules.

You may also use regular expressions here to match more general column names.

Example: classifier_ignore_columns = WellID, Meta_., ._Position

This will ignore any column named “WellID”, any columns that start with

“Meta_”, and any columns that end in “_Position”.

A more restrictive example:

classifier_ignore_columns = ImageNumber, ObjectNumber, .Parent., .Children., .Location_Center.,.Metadata.

classifier_ignore_columns = table_number_key_column, image_number_key_column, object_number_key_column

==== Other ====

Specify the approximate diameter of your objects in pixels here.

image_tile_size = 200

======== Auto Load Training Set ========

OPTIONAL

You may enter the full path to a training set that you would like Classifier

to automatically load when started.

training_set =

======== Area Based Scoring ========

OPTIONAL

You may specify a column in your per-object table which will be summed and

reported in place of object-counts when scoring. The typical use for this

is to report the areas of objects on a per-image or per-group basis.

area_scoring_column =

======== Output Per-Object Classes ========

OPTIONAL

Here you can specify a MySQL table in your Database where you would like

Classifier to write out class information for each object in the

object_table

class_table =

======== Check Tables ========

OPTIONAL

[yes/no] You can ask classifier to check your tables for anomalies such

as orphaned objects or missing column indices. Default is on.

This check is run when Classifier starts and may take up to a minute if

your object_table is extremely large.

check_tables = yes

Thanks,
Toby

One table per object type is preferred. But if in CellProfiler you really only had one type of object defined (e.g. “SE” in your case, and no other object defined like “Cells” or “Nuclei”, etc), then it shouldn’t matter. But I would suggest outputting to one table per object type to see if this simple change will fix it for you.

I don’t see anything else obviously incorrect in your properties file, so I am hoping it is just an issue with the single object table.

Cheers,
David

Got the same error :frowning: :cry:

Toby

Hmm, so you re-ran CellProfiler with ExportToDatabase and loaded the new properties file and got the same error?
At this point, I would inspect the database for anomalies, i.e. missing values, NaNs, missing or extra columns, etc. There are free database viewers out available if you haven’t looked already.

A couple other questions:

  • Did the CellProfiler analysis complete without incident? The error indicates that the number of columns in your Per_Object table is not the same as expected with the number of labels you gave it (looks like 1 for the SE object).
  • When you click ScoreAll, are you choosing any grouping parameters in the popup window? If so, try no grouping (i.e. pick ‘Images’, I believe)

If the above checks out ok, you can try our trunk build here which may have solved a bug that is not obvious: cellprofiler.org/cgi-bin/trunk_build_cpa.cgi
It can take a few minutes to load, be patient.

David