Table checking removed all rows, you may have an empty column in your database

I have a project where I want to be able to fetch objects based on the folder name in the classifier module in CellProfiler Analyst. I have extracted the folder name as metadata and also created groups in CellProfiler. When starting CellProfiler Analyst, I get the following error message: “Table checking removed all rows, you may have an empty column in your database. Disable check_tables in your properties file if this is expected.”. It works for CellProfiler Analyst 2.2.1, but not for 3.0.0.

Hi @PetterRanefall,

The check_tables function is designed to remove rows which have missing measurements, but this can sometimes result in all rows being removed if you have a results table where each object had at least one measurement missing (common when working with child objects).

In CPA2 this feature didn’t actually work on SQLite databases, and so was effectively ignored. In CPA3 it does work, so we’ve added this warning when these exclusion criteria would remove all data. If you edit your .properties file to disable the check_tables option that should resolve this.

I did actually try to disable the check_tables option, but that did not work either.

That’s odd, are you still seeing the “Disable check_tables in your properties file” prompt with it disabled?

Something else to check - if you use the Table Viewer tool can you verify that there’s actually data in the tables?

When I disable the check_tables option, I can open the database and fetch random cells from the experiment, but it does not work to fetch cells from Image_Metadata_FolderName. Then I get an error message:
An error occurred in the program:
AttributeError: ‘Choice’ object has no attribute ‘Value’

Traceback (most recent call last):
File “cpa\classifier.py”, line 1007, in OnFetch
File “cpa\classifier.py”, line 2131, in GetGroupKeyFromGroupSizer

In CPA 2.2.1 it works.

Thanks, I’ve fixed this for 3.0.2

1 Like

Great, thanks!

However, I still don’t understand why I get empty columns. It is just a standard pipeline where I Identify Primary, Secondary and Tertiary objects and measure ObjectSizeShape, ObjectIntensity, Texture, and ObjectIntensityDistribution.

If you look in the table viewer, do you see any columns without data? There’s not much I can suggest without seeing the data in question.

The table MyExpt_Per_Object has some columns with the value None, e.g. Nuclei_AreaShape_NormalizedMoment_0_0, but otherwise I cannot find any empty columns. The table MyExpt_Per_Object_checked is completely empty.

That might do it. Thanks for the info.

The sqlite table checking functionality is a newer feature which might still have some problems. If you’re comfortable doing so it’d be really helpful if you could send over a copy of your database and .properties file. No worries if not, disabling the checking should keep things running smoothly.

Here is a copy of the database and .properties file. Hope it helps.
DefaultDB__MyExpt.zip (2.7 MB)

1 Like

Thanks so much, based on this I’ve made some improvements to check_tables so that empty columns will now be dropped instead of destroying the table. That should be in the next version.

1 Like