BD Pathway file causing problem

Hi,
I’m using a BD Pathway to do my image acquisition. I tested a pipeline that does a merge of three dyes on many sets of images that I previously copied in a new folder (image only) without any problem.
When I tried this same pipeline on a folder created by the BD Pathway (with many folders, images and files containing experiment informations) I found out CellProfiler were always using the same image to do the merges. I tried File>Open Image with many images but CellProfiler were always opening the same image. I tried too look if some file made by the BD Pathway could be the cause of the problem and finally found a file, called Experiment.exp that is responsible for the problem. If I remove only this file, my pipeline works fine. But it seems there is more files affecting Cellprofiler because if I let only the Experiment.exp removing all other experiment information files I get an over-saturated image as output (cannot even identify what it is). Is this file is known to affect CellProfiler ?
I’m using version 9978 on Windows XP.

Thanks,
Daniel

Greetings,
I can confirm this bug as well on the OS X version 9978 of Cell Profiler. When run on a directory only containing the relevant image files from a BD Pathway microscope, the images are loaded correctly. However, if pointed at the original output folder, the 4 images files are listed in the LoadImages module as being loaded correctly, but later in the pipeline appears that only one of the images has been loaded multiple times for each channel. I have tried both Text-Exact Match and Text-Regular Expressions but the bug persists.

Thanks,

cscott

Hi cscott and Daniel,

An initial question: What are you guys using as the text-matching string in LoadImages, both for Exact Match and Regular Expressions?
-Mark

Hi Mark
I’m using Text-exact match with the whole images name created by the BD Pathway.
In this case:
GQ-Hoechst - n000000.tif
GQ-Propidium Iodide - n000000.tif
GQ-Alexa647 AnxV - n000000.tif

Daniel

Typically we have used Exact Match for four images:

Propidium Iodide G - n000000.tif
Cell Tracker G - n000000.tif
Fil G - n000000.tif
CY5 G - n000000.tif

For the regular expression, I tried:

Propidium.*tif
Cell.*tif
Fil.*tif
CY5.*tif

and got the same result - despite the LoadImages module window displaying the correct name of the files, all images look like CY5 G - n000000.tif in subsequent module windows.

Cheers,

cscott

Hi,

It turns out that this an issue with Bio-Formats support of BD Pathway; Bio-Formats is the library that we now use to read image files. Bio-formats tries to be smart and it treats the directory as one big file; our software engineer found the switch to make it “dumb”, which will be in the next release. The workaround appears to be to remove that experiment.exp file, as cscott found.

Hope this helps!
-Mark

Hi,

I had exactly the same problem and have found a solution. The problem arises with the files starting with the dye name in the folder where the well folder are contained. For instance, there are a GFP(384 Well).mac, GFP.drt, GFP.dye, GFP.ltp files in the folder. These are the culprits. The text handler does not seem to handle them correctly and gets stuck on the first real image file it encounters. If you remove those files, the pipeline runs fine. I hope this helps.
Marc

Our latest release, r10415, should solve this problem without requiring a workaround.
-Mark

Hi Mark,

Unfortunately in the latest build importing BD pathway data still doesn’t work as expected. When I point my pipeline at an output subdirectory containing 4 channels of images, CellProfiler 2.0 still loads the same image 4 times. Now, it appears to load a 3-channel merged colour image 4 times over. If I remove all the extraneous files, CellProfiler loads the 4 separate grayscale images just fine.

Cheers,

cscott
Well C002.zip (5.02 MB)

Dear CScott,
I’ve unzipped the directory you sent and have created a pipeline (attached) that reads the files and creates a color image. My pipeline does read four different images when I run it. Could you send me your pipeline?

Lee Kamentsky
pipe.cp (8.49 KB)

Hi Lee,

I downloaded and tried your pipeline and it worked fine the data I provided. So I did some more testing and found that the error loading the images only occurs when the directory was in context of the whole output with at least one other well of data. I am attaching the enclosing directory containing the data from well C003. If you place the data I provided previously (well C002) into the directory and run your pipeline (after turning on analyze subfolders) you should be able to reproduce the bug.

Cheers,

cscott
2010-09-01_000.zip (5.63 MB)

Hi CScott,
I’ve replicated the behavior you’ve seen and have made a fix in CellProfiler. I would really appreciate you trying this out - I have trunk builds for Windows 32/64 bits and Mac. These are not vetted to the extent of the formal releases, but they pass our unit tests:

Windows 32-bit: broadinstitute.org/~leek/win … r10580.exe
Windows 64-bit: broadinstitute.org/~leek/win … r10580.exe
Mac: broadinstitute.org/~leek/mac … r10580.dmg

Thanks,
Lee

Hi Lee,

Thanks for the links. I downloaded and tested the Mac build and it seems to work fine with our data. I will post here if something crops up, but I think your change has fixed the problem.

All the best,

cscott