Failed to complete post_run processing for module ExportToDatabase

cellprofiler
error
database
mysql
exporttodatabase
aws

#1

Hi,

I recently set up a test MySQL database (v5.6.41) on AWS so that in the future I can use multiple copies of CellProfiler on our cluster. I performed a run of 308 images, and when I check the database via MySQL workbench it looks fine. However, right after finishing processing the last image in this run, I receive this error:

Failed to complete post_run processing for module ExportToDatabase.
Traceback (most recent call last):
File “/home/nel56/.conda/envs/cellprofiler/lib/python2.7/site-packages/cellprofiler/pipeline.py”, line 2194, in post_run
module.post_run(workspace)
File “/home/nel56/.conda/envs/cellprofiler/lib/python2.7/site-packages/cellprofiler/modules/exporttodatabase.py”, line 2376, in post_run
self.write_properties_file(workspace)
File “/home/nel56/.conda/envs/cellprofiler/lib/python2.7/site-packages/cellprofiler/modules/exporttodatabase.py”, line 3635, in write_properties_file
all_properties = self.get_property_file_text(workspace)
File “/home/nel56/.conda/envs/cellprofiler/lib/python2.7/site-packages/cellprofiler/modules/exporttodatabase.py”, line 4081, in get_property_file_text
“”" % (locals())
UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xff in position 254: ordinal not in range(128)

This results in no workspace or properties files. When I tested it on my PC with a single image, it worked just fine, though I’ve not tried it with 308 images. However, my colleague had the same error when he tried to run a similar pipeline on his PC, so I’m wondering if it might be something to do with the way the MySQL is configured.

I’ve attached the pipeline in case that might be the issue. Does anyone have any ideas?

Cheers,

EdGeneralPipelinePDXMSCMYSQLTEST.cppipe (18.7 KB)


#2

Hi Edward

Could you try to rename the experiment name so it does not start with a number? I don’t know if this is the issue though.
I have no problem exporting the analysis of thousands of images to a MySQL database in CellProfiler 2.1.1, but I have not attempted in 3.0.0.

Regards,
Esben


#3

Hi Esben,

Thanks for your suggestion! Unfortunately it did not work and we still have the same issue, though I can of course create the properties file myself.

Thanks,

Ed


#4

I observe the exact same error in 3.1.8, using ExportToDatabase (running mysql locally).

properties = self.get_property_file_text(workspace)
  File "/home/username/anaconda3/envs/cp/lib/python2.7/site-packages/cellprofiler/modules/exporttodatabase.py", line 4080, in get_property_file_text
    """ % (locals())
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 254: ordinal not in range(128)

The tables appear to have been created perfectly fine in MySQL Workbench. (Ubuntu 16.04). I’ve tried multiple different latin, utf and ascii encodings. I’ve also tried to make it work with an empty database on a GCP instance with no luck.


#5

Does anyone have an answer to this? It’s still causing me problems.

Best,

Ed