Data normalization using cytominer-database

cellprofiler

#1

Hi,

Can anyone give me some help with the cytominer-databases?

I tried to create a sqlite file which would be further cleaned-up using the vignettes in cytominer.

However, I got nothing after running the code.

Many thanks!

My files are also attached: https://drive.google.com/drive/folders/1z__TeUqoWP6_Pfs70xnbFXtEqnj7QLU4

Cheers,
Peter Lu


#2

Also, has anyone met the problems of too many column when follow the steps for the data normalization? (https://github.com/carpenterlab/2016_bray_natprot)

When I tried to import the file of “Per-object view of cell, cytoplasm, and nuclei tables. [gzip SQL dump file, 74 KB]”, the error of Too many column appears.

Cheers,
Peter Lu


#3

See the docs http://cytominer-database.readthedocs.io/en/latest/reference/ingest.html

which documents the expected structure for the CSV files

I rearranged your files

plate_a
├── ingest_config.ini
└── set_1
    ├── analysis_Per_image_ER.csv
    └── analysis_Per_objects_ER.csv

and changed ingest_config.ini by deleting the line

object = analysis_Per_objects_ER.csv

because that is actually just the CSV file for the Cells compartment

and then ran cytominer-database

cytominer-database ingest --no-munge -c plate_a/ingest_config.ini plate_a/ sqlite:///plate_a.sqlite

This gives me a file plate_a.sqlite with an image table with 753 rows and an object table with 66295 rows.

Although this works, I’d recommend a few changes to the way you are creating the CSV files in ExportToSpreadsheet

First do this so that each object (e.g. Cell, Nuclei, etc.) has a separate CSV file (not sure whether this is already the case for you)

    Combine these object measurements with those of the previous object?:No

and this

    Add a prefix to file names?:No

so that the CSV files are named e.g. Cells.csv , image.csv, etc.


#4

That indeed is a problem, and we had needed to modify our MySQL config to make this work. I don’t recommend you do this. We plan to update the supporting data for that paper (or create a new vignette) in the near future, and that should solve this problem.


#5

That’s great. The cytominer-database works for me now, thanks!

Looking forward your updating for the supporting data using cytominer~

Cheers,
P L