Creation of propertie file gives 'weakref' error

Image Analysis
#1

Hi all,

I’ve been trying to import a .czi file created with a fluorescent microscope from Zeiss. I got photos of 5 wells with four data points per well taken in 5 channels. So 100 images. The CP recognises 20 groups, instead of 5 wells. When I’m trying to create a propertie file for CPA, I’m getting a ‘weakref’ object has no attribute ‘commit’ error during startup. Looking at the example data provided, I’m wondering if I have not enough metadata to group the samples and no primary key for the database? I’ve attached the pipeline.
Any suggestions would be greatly appreciated.

Cheers,
Anja
test4col.cpproj (1.8 MB)

#2

Hi,
Did you sanitize your database entry in ExportToDatabase, or is that exactly how it was when you ran it? If that’s exactly how you set it up when you ran it, that’s at least part of the issue- if you don’t have a MySQL database to export to (or don’t know whether you do or not), you should use the database type Sqlite (which will create a local little data file on your computer) vs MySQL (which requires you to have a networked database to connect to).

I’m also not sure your Plate metadata is actually Series or that your Well metadata is actually C - it’s hard to tell exactly what the correct settings would be without the image in hand, but I’d recommend going through each one of the settings in the property file creation section step-by-step (don’t forget the ? next to each line is a help box!) and determine the best way to set them up. If you have specific questions about some, we’re happy to help!

#3

Hi,
thank you very much for your reply. I have installed MySQL and created the db Cellprofiler. The connection to the db host is successful and the tables are being created, but not filled. I’ve seen in one of the forums that the module ClassifyObjects is needed in order to train CPA, so I’ve included it in the pipeline. Unfortunately, I can’t upload an image, as it’s too large (even in zipped format). I could attach the modified pipeline though.
I’m not sure about the metadata either, as cellprofiler doesn’t recognise the single plates and wells. I have 5 plates with 4 wells per plate, times 5 for the channels, so 100 images. But cellprofiler groups it into 20 images, so that is actually where my problems start…
Cheers,
Anja

#4

I’m also having some of the same questions as the IT guy in this thread: Setting up new CPA
I can’t find the MySQL CPA plugin either, and the answer about the configuration of the tables and columns would help me probably as well.
By the way, I’m running CPA and MySQL on Ubuntu 16.04

Cheers,
Anja

#5

You definitely do not need ClassifyObjects in your pipelines using current versions of CP/CPA, and I think the MySQL CPA plugin is a thing of the past as well (CPA should just work with MySQL).

I have 5 plates with 4 wells per plate, times 5 for the channels, so 100 images. But cellprofiler groups it into 20 images, so that is actually where my problems start

Can you explain why that’s a problem? CellProfiler is recognizing that you took pictures in 20 locations (4 wells per plate in 5 plates), but that each site has 5 channels. This is the expected behavior.

The connection to the db host is successful and the tables are being created, but not filled.

This is due to the error that you’re getting; if you turned the properties file creation off and run it again, do the tables get populated?

The next thing to try would be turning property file creation back on, but setting the Do you want to add {group/filter} fields? settings to ‘No’ and setting the plate type,plate metadata, and well metadata all to None. Does it then work, and are you able to use the non-plate related tools in CPA (plotting, image viewer, classifier, etc)?

If you absolutely need/want the plate viewing tools, can you either screenshot the Metadata window to show what CP is extracting from your images, and/or can you upload your image to Dropbox/GoogleDrive/etc and post a link?

#6

It looks like I made some progress over the day. The tables are being populated and I can see the 20 images in the CPA Image Gallery and Classifier. However, the Image Gallery shows the same image 20 times, so something is still not quite working, and I’m missing something fundamental. Plus I haven’t found a way yet to group the data in a way that I have a unique combination for plate+well+channel
The main goal was to train CPA to recognise the images where we have cells with free calcium (EGFP channel). I have uploaded an example image to GoogleDrive:
https://drive.google.com/open?id=0B843VDZE-ww2elNZSVZoVVAwa00
And attached the modified pipeline:
Cellprofiler_test.properties (9.3 KB)
I’m sure I have got a wrong setup in my pipeline somewhere…

Cheers,
Anja

#7

Plus I haven’t found a way yet to group the data in a way that I have a unique combination for plate+well+channel

Can you explain what you mean by this?

However, the Image Gallery shows the same image 20 times, so something is still not quite working, and I’m missing something fundamental.

I started digging into this for you, then realized- as it states in the official documentation, CPA expects only single channel images, so it’s not going to be able to use your CZI files. I’ve reached out to some of my colleagues and will report back if I hear something different, but as far as I can tell the only ways to make this work for you are going to be either pre-split your image into TIFs before importing them into CellProfiler (some microscopes will give you this option; if not you can do it in FIJI/ImageJ) OR to save individual channel images in your pipeline using SaveImages (make sure to turn the Record the file and path information to the saved image? option on, or CPA won’t know where to find them later).

Edited version of the pipeline is below- I did a ‘DAPI’ version of the SaveImages trick to demonstrate, if you choose to go that way you can just duplicate it for the rest of the channels you want to use in Analyst. I also added a ‘Series’ metadata tag in your SaveImages modules, otherwise you would just keep overwriting your images every cycle since they’d always have the same name.

test4col_edits.cppipe (16.9 KB)

#8

Thank you very much for pointing that out, I totally missed that part!
Splitting up the images in CP works just fine, I duplicated the SaveImages module for the other channels, and thank you for the metadata tags trick, I haven’t seen that before.

I’m not sure I understand how the metadata is set up. I have 20 images which, I suppose, are my 5 wells with 4 data points per well? Sorry, I just saw that I mislabeled that in the previous post. How would I group those to get the images for one well?
I can’t access single images in the CPA Classifier (as shown in the video tutorial), so I tried to create filters, but it says no images found. Also I’m still getting the ‘weakref’ error at the start of the analysis. What could be the reason?
Is there a way to link the loaded images in CPA back to the originals? I can’t see a file name or similar which would tell me where the image is coming from. Thank you very much for your assistance.
test4col.cpproj (1.8 MB)

#9

Hi bcimini,
have you turned off the image creation in the test mode/suppressed it? The test mode runs through without creating the images for primary, secondary objects and so on. I can’t seem to find a setting to switch it on/off?
Cheers,
Anja

#10

ignore the last question, I found the eye :slight_smile:

#11

I’m not sure I understand how the metadata is set up. I have 20 images which, I suppose, are my 5 wells with 4 data points per well? Sorry, I just saw that I mislabeled that in the previous post. How would I group those to get the images for one well?

The only metadata the czi passes into CP is the ‘Series’- essentially the order each data point was taken in. If you want to teach CP how that breaks down into wells and sites, for example, the best thing to do is to create a CSV (in Excel or somewhere else) that looks like this:

Series     Well     Site
0           A1        1
1           A1        2
Creating file names using measurements and adding leading 0's
#12

Good to know, thank you.
So is that what I would have to do to access the images separately in the Classifier?
And would you know why the filter is not working? I initially set it to WHERE Image_Metadata_ChannelName = ‘EGFP’, hoping that I would only get the images from this channel… or is that the wrong approach?

#13

Sorry, hit ‘Reply’ before I was ready, then had to run to a meeting in the middle of writing the second half of my reply!

(Note that the Series numbering starts at 0, so it’s 0-19 not 1-20). You can then load this CSV into the Metadata module via the ‘Load from file’ option, and that will teach CP how to associate your Well/Site metadata to the Series.

Is there a way to link the loaded images in CPA back to the originals? I can’t see a file name or similar which would tell me where the image is coming from.

The top of the window should give you some Metadata about the image (particularly once you start having CP read it in the Metadata module); you can always back-compare this to the ‘FileName’ column in your database.

And would you know why the filter is not working? I initially set it to WHERE Image_Metadata_ChannelName = ‘EGFP’, hoping that I would only get the images from this channel… or is that the wrong approach?

It’s a bit the wrong approach- CP and CPA view all the channels as linked, so it’s going to give some strange errors if you try to unlink them in this way. If you only want to see the EGFP channel (and you’re sure you’ll never want to look at the other ones alongside it), you can adjust the Include information for all images setting in ExportToDatabase; the way I’d personally recommend doing it (so that you have access to the other channels again if you ever decide you want to look at them) is to continue exporting everything to the database, but then in the classifier/image gallery/ etc tell it the only image you want to look at is your EGFP image under the ‘Images’ menu tab (see screenshot below).

blob.jpg784×592 77.5 KB

#14

I’ve created a table and it works perfectly, I can analyse an image at a time :slight_smile: Thank you!

This approach only gives me gray images, what I did instead is setting the colors of the other channels to ‘none’ and just specifying DAPI and EGFP.

Unfortunately, after all this time, the biologists in the group decided that they actually only care about the intensity of the object, and that the we don’t need the machine learning approach at the moment (which is a real pitty).
However, I was wondering what would be the reason to not include the ‘Watershed’ algorithm in the module ‘IdentifyPrimaryObjects’? I’m having trouble de-clumping the nuclei, and the Watershed algorithm is working quite well in the Zeiss software… (I know it’s a CellProfiler question, but I didn’t want to open an extra thread for it)

NucleiClump.png2358×1290 363 KB

#15

If you’re having trouble declumping, you can try changing the declumping methods (from Intensity to Shape for example), the size of the smoothing filter applied before declumping, or the amount of local maxima suppression- it’s usually not possible to find settings that work 100% perfectly on 100% of images, but it seems like there’s decided room for improvement there.

#16

Hi, I have found a series of method which seems to work reasonably well for most image sets. Thank you very much for all the help you provided.
Quick question, for the ObjectIntensity module, is there a way to get the intensity for each identified object (instead of the mean, median etc.)?

#17

If you look at your spreadsheet and/or database table for the object in question, the individual measurements should be there; the menu window and Image spreadsheet/table report the means.

1 Like