Input more than Red Green and Blue Channels

cellprofiler-analyst

#1

I have an image analysis problem that uses 4 colors - Red, Pink, Blue, and Green . I know that in the properties files one can specify the filenames and paths to the grayscales of Red, Green and Blue Channels, but is there a way to work with more than 3 channels?

Any suggestions welcome…


#2

Hi there,

Well, the answer depends on what sort of analysis you’re looking to do. If you’re interested in using Classifier only, the latest release on the website handles multiple channels gracefully, however, CPAnalyst (which also provides plotting capabilities) doesn’t have this capability.

If you’d like some help getting rolling with the new Classifier, let me know!


#3

By the new classifier are you referring to the Classifier 2.0 and that it would be able to integrate more than 3 channels unlike CPAnalyst . If so would I be able to make it work on a linux system. I believe the current release is for a Mac OS. Please correct me if I am wrong.


#4

Yes, Classifier 2.0 will handle any number of channels and let you choose which colors to map them to (r,g,b, c,m,y,k, gray, none).

Unfortunately, I believe the currently released version is only tested on a Mac, although a bleeding edge SVN checkout would be compatible with Windows, and possibly on Linux (although, as I look at our bug tracking system now, it appears there is a problem with the drag and drop functionality on Linux machines which is pretty critical :\ ).

If you are particularly computer savvy, I could see about taking a stab at fixing that bug for you midway through next week, but it would still require a lot of work on your part to download and install python 2.5 and the MANY packages we use for it (wx, numpy, scipy, PIL, MySQLdb, pysqlite). Otherwise, I’m HOPING to be able to smooth the current version out to a state where we can package it for release in a couple weeks.

Think about it and let me know if I can be helpful to you. I’ll be gone from tomorrow over the long weekend though, so I won’t likely be able to get back to you until tuesday.

Cheers


#5

i can give it a shot if you can provide a working version of it for Linux while you guys get more finalized version out in a couple of weeks.

Thanks.


#6

Okay, you can now consider the app functional under Linux. I tested most of the central functionality and everything worked out well except for one non-critical feature of being able to delete classification bins. Classifier can be checked out of our SVN repository here svnrepos.broad.mit.edu/CellProf … CPAnalyst/

More info about requirements for running our development version can be found here: viewtopic.php?f=13&t=624#p2590

Let me know how it goes. Meanwhile, we’re just beginning to work on packaging the thing up for release.


#7

Hi Adam,
Thanks for providing the code for linux version of the classifier 2.0. I have downloaded that and also the required python modules that you have recommended. However I am not sure how should set the path of the modules and where should I extract the tarballs of the modules to ?? do I just extract them to the src folder of the code ??
Could you help me in this please ?

Thanks
Arunava


#8

Hi again,

Unfortunately I can’t give you a detailed description of how to install each of the modules. You’ll have to check the respective websites for how to do that on Linux.

On MacOS at least, the typical process goes like this:
First: extract the files to some folder asdf/

cd asdf/ python setup.py build sudo python setup.py install

Installing wx, I believe was different, however, because it came with a mac installer.
MySQLdb was tricky as well because install was looking in the wrong place for some MySQL file. You’ll probably want to try to install this first since it will likely be the most difficult. Unfortunately, I haven’t documented the solution so you’ll have to google around if you run into trouble. Here’s a start: blog.awarelabs.com/?p=61

I know, it’s not easy, and we’re working on streamlining this process. We’ve already done so for the CellProfiler developer version. If you aren’t having any luck, but you have access to a Mac, I did just package up our most recent svn revision for someone else here: viewtopic.php?f=13&t=624#p2665

Good luck!
Adam


#9

ok !! I have been able to get to a point where when do a
cd src
python Classifier.GUI

a file open menu shows up asking for a properties file. and when i select one of the properties file from the properties folder that comes with the code it the thing breaks showing the following message :

PROPERTIES WARNING (plate_id): Field is required for plate map viewer.
PROPERTIES WARNING (well_id): Field is required for plate map viewer.
PROPERTIES WARNING (plate_type): Field is required for plate map viewer.
Traceback (most recent call last):
File “/users/arun/imaging/clasifier beta/src/ClassifierGUI.py”, line 1071, in
dm.PopulateModel()
File “/users/arun/imaging/clasifier beta/src/DataModel.py”, line 37, in PopulateModel
db.CheckTables()
File “/users/arun/imaging/clasifier beta/src/DBConnect.py”, line 637, in CheckTables
res = self.execute(‘SHOW INDEX FROM %s’%(p.image_table))
File “/users/arun/imaging/clasifier beta/src/DBConnect.py”, line 228, in execute
self.Connect(db_host=p.db_host, db_user=p.db_user, db_passwd=p.db_passwd, db_name=p.db_name)
File “/users/arun/imaging/clasifier beta/src/DBConnect.py”, line 140, in Connect
raise DBException, ‘Failed to connect to database: %s as %s@%s (connID = “%s”).\n %s’%(db_name, db_user, db_host, connID, e)
DBConnect.DBException: ERROR: Failed to connect to database: 2008_07_29_Giemsa as cpadmin@imgdb01 (connID = “MainThread”).
(2005, “Unknown MySQL server host ‘imgdb01’ (1)”)


I noticed that the last line says something like a unknown MySQL server host. Could it be that I need to change the MySQL setting somewhere… Please advise

-Arunava


#10

Sounds like we’re in pretty good shape now. Nice work!

Okay, so the first question is where your data resides. Classifier can read data from either a MySQL database or a pair of CSV files (one with per_image data, the other with per_object data). Take a look in the README, it’s very specific about the format you’ll need. Also, look at the database info section of the Properties_README:

db_type = mysql
db_port = 3306
db_host = <your_host_name>
db_name = <your_database_name>
db_user = <your_user_name>
db_passwd = <your_password>

-Adam


#11

so i set up the properties file as per the new specs :

db_type = mysql
db_host = localhost
db_port = 3306
db_passwd = *****
db_name = cellprofiler
db_user = arun
image_table = Per_Image
object_table = Per_Object
image_id = ImageNumber
cell_x_loc = Nuclei_Location_Center_X
cell_y_loc = Nuclei_Location_Center_Y
image_channel_paths = <Image_PathName_Origd3>, <Image_PathName_Origd2>, <Image_PathName_Origd0>
image_channel_files = <Image_FileName_Origd3>, <Image_FileName_Origd2>, <Image_FileName_Origd0>
image_channel_names = , ,
image_channel_colors = , ,
image_url_prepend =
group_SQL_YourGroupName =
filter_SQL_YourFilterName =
object_name = cell, cells
plate_type =
classifier_per_object_ignore_substr = ImageNumber,ObjectNumber,Location,parent
image_title_size = 50
training_set =
area_scoring_column =
image_buffer_size = 1
tile_buffer_size = 1

but when i try to use this properties file as an input this is the error I get

$ python ClassifierGUI.py
Traceback (most recent call last):
File “/users/arun/imaging/clasifier beta/src/ClassifierGUI.py”, line 1069, in
LoadProperties()
File “/users/arun/imaging/clasifier beta/src/ClassifierGUI.py”, line 1021, in LoadProperties
p.LoadFile(filename)
File “/users/arun/imaging/clasifier beta/src/Properties.py”, line 71, in LoadFile
(name, val) = line.split(’=’, 1) # split each side of the first eq sign
ValueError: need more than 1 value to unpack

any insights on what might be the problem ?

Thanks again

Arunava


#12

Hrm, the error you’re getting indicates that the reader has found a (non-comment) line in your file that doesn’t have an ‘=’ sign anywhere.

You also need to remove the "<>"s from the following lines. The top of the properties readme explains that these are just placeholders for your data. It looks like you may have filled your data inside the “<>”'s, but really you just want to replace them altogether.

image_channel_paths = <Image_PathName_Origd3>, <Image_PathName_Origd2>, <Image_PathName_Origd0> image_channel_files = <Image_FileName_Origd3>, <Image_FileName_Origd2>, <Image_FileName_Origd0> image_channel_names = <DNA>, <cyclin>, <pHH3> image_channel_colors = <blue>, <green>, <red>

Assuming the column names for the paths and filenames above are correct, your file should look like this:

db_type = mysql db_host = localhost db_port = 3306 db_passwd = ***** (<--- don't forget to set this) db_name = cellprofiler db_user = arun image_table = Per_Image object_table = Per_Object image_id = ImageNumber cell_x_loc = Nuclei_Location_Center_X cell_y_loc = Nuclei_Location_Center_Y image_channel_paths = Image_PathName_Origd3, Image_PathName_Origd2, Image_PathName_Origd0 image_channel_files = Image_FileName_Origd3, Image_FileName_Origd2, Image_FileName_Origd0 image_channel_names = DNA, cyclin, pHH3 image_channel_colors = blue, green, red image_url_prepend = ******* If the paths in your database aren't absolute, then you'll need to fill this in ****** group_SQL_YourGroupName = filter_SQL_YourFilterName = object_name = cell, cells classifier_per_object_ignore_substr = ImageNumber,ObjectNumber,Location,parent image_title_size = 50

Let me know if this helps any.


#13

Ok … so here’s what i have on my properties file :

db_type = mysql
db_host = localhost
db_port = 3306
db_passwd = *****    
db_name = cellprofiler
db_user = arun
image_table = Per_Image
object_table = Per_Object
image_id = ImageNumber
cell_x_loc = Nuclei_Location_Center_X
cell_y_loc = Nuclei_Location_Center_Y
image_channel_paths = Image_PathName_Origd3, Image_PathName_Origd2, Image_PathName_Origd0
image_channel_files = Image_FileName_Origd3, Image_FileName_Origd2, Image_FileName_Origd0
image_channel_names = DNA, CyclinB, pHH3
image_channel_colors = blue, green, red
image_url_prepend  =
group_SQL_YourGroupName =
filter_SQL_YourFilterName =
object_name  =  cell, cells
classifier_per_object_ignore_substr = ImageNumber,ObjectNumber,Location,parent
image_title_size = 50

but i still get the error :

PROPERTIES WARNING: Unrecognized field “db_type” in properties file
PROPERTIES WARNING (group_SQL_YourGroupName): Undefined group
PROPERTIES WARNING (filter_SQL_YourFilterName): Undefined filter
PROPERTIES WARNING: Unrecognized field “classifier_per_object_ignore_substr” in properties file
PROPERTIES WARNING: Unrecognized field “image_title_size” in properties file
Traceback (most recent call last):
File “/users/arun/imaging/clasifier beta/src/ClassifierGUI.py”, line 1069, in
LoadProperties()
File “/users/arun/imaging/clasifier beta/src/ClassifierGUI.py”, line 1021, in LoadProperties
p.LoadFile(filename)
File “/users/arun/imaging/clasifier beta/src/Properties.py”, line 120, in LoadFile
self.Validate()
File “/users/arun/imaging/clasifier beta/src/Properties.py”, line 191, in Validate
assert field_defined(name), ‘PROPERTIES ERROR (%s): Field is missing or empty.’%(name)
AssertionError: PROPERTIES ERROR (db_type): Field is missing or empty.

In fact the unrecognized “db_type” seems to have some non_ascii characters appended to it from left when I see the warnings on my screen. I am wondering why is that


#14

Well… this could be caused by using a text editor like Microsoft Word or something that is likely to leave behind odd markup. You’re on linux though right? I’d suggest opening your props file in a few editors and making sure there isn’t any magic going on. I’d trust emacs anyway. Just make sure you’re saving it as plain text.

Also, I’m not sure what SVN rev you are using. Update to 7649.

How are you running Classifier? There are two ways:

…the former will prompt you to navigate to your props file.

What version of Linux are you running? So far the only testing we’ve done is on Debian, but I would be surprised if this would make a difference.

Sorry… I’m not being very helpful, but I’m having a hard time seeing what’s going wrong.

Below I’ve made some other minor corrections to your properties file:

[code]db_type = mysql
db_host = localhost
db_port = 3306
db_passwd = *****
db_name = cellprofiler
db_user = arun

image_table = Per_Image
object_table = Per_Object

image_id = ImageNumber
object_id = ObjectNumber

cell_x_loc = Nuclei_Location_Center_X
cell_y_loc = Nuclei_Location_Center_Y

image_channel_paths = Image_PathName_Origd3, Image_PathName_Origd2, Image_PathName_Origd0
image_channel_files = Image_FileName_Origd3, Image_FileName_Origd2, Image_FileName_Origd0
image_channel_names = DNA, CyclinB, pHH3
image_channel_colors = blue, green, red

image_url_prepend =

object_name = cell, cells
classifier_ignore_substrings = ImageNumber, ObjectNumber, Location, parent
image_tile_size = 50[/code]

If you have jabber or pidgin or some chat program, I can be reached on gchat (adam [dot] n [dot] fraser [at] gmail.com) and maybe we can resolve this more quickly.


#15

ineteresting that you mentioned about the version of the SVN because the copy that i have , has a ClassifierGUI.py and not a Classifier.py. Do you think might be some older version? I will still go ahead and check if there are any version differences

Thanks

Arun


#16

also i checked that the svn code that i have is from version 7654


#17

Sorry, that was a typo, it is “ClassifierGUI.py


#18

yohhoo!! Finally … it worked :smile: there was some non-ascii character sitting there before “db_type” which i couldnot see on my vi editor … but got captured in nano.

Thanks a ton for being so helpful

-Arun


#19

That’s great! It’s unbelievable how often things like this pop up as the source of a problem. Where are all these ugly characters coming from anyway?

Anyway, I’m glad it’s working for you. Let me know if you have any further questions. And again, read the README! There’s good stuff in there. :mrgreen:


#20

Hi Adam,
The new classifier beta works great but crashes very often while dragging images to different classes , giving a “Segmentation Error”. Any insights on why it does that and is there some way to get around it.

Thanks

Arunava