Problem of running pipeline for BBBC022

Hi,

I am reading the paper published at http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3847047/#pone.0080999.s009.
And would like to try out cellprofiler using data and pipeline described in the paper.
I have downloaded the image set http://www.broadinstitute.org/bbbc/image_sets.html, and these two pipelines. When I load forPilotrerun9925.cp and images to cellprofiler, it couldn’t not run because the “allplates_PILOT.csv” is missing in LoadData step.
My question is: where can I download “allplates_PILOT.csv”? Or am I missing something?

Thanks a lot!

Hi @xiaorong -

I will look into this and upload the missing file as soon as I can.

Thanks for letting me know!

Alison

Hi Alison,

Actually, I created a project by removing the LoadData module from the pipeline and redefining name and type module. I also added an step to export extracted features to a spreadsheet. It works well on my Mac with small image sets, but crushes when I tried to profile all images of one experiments, about 3500 image sets.

Now, I am trying to figure out how the batch file is created and how to use it to run on my linux box. Can you point me some good instructions? I use CellProfiler 2.1.1.

Thanks,

Xiaorong

Passing this onto @David_Logan @daviddao or @allen_goodman

I realize you have worked past this, but the CSV should be the one linked in the BBBC022 page, here: http://www.broadinstitute.org/bbbc/BBBC022/BBBC022_v1_image.csv. Yes, the name was probably changed at some point, but it looks like a LoadData input file.

We’ll send some tips on running CP headless in a bit.

Hi @xiaorong, I’d like to help.

Could you please upload your pipeline to this thread? CellProfiler can become memory intensive when processing large datasets leading to crashes. There are two approaches you can take to circumvent this issue.

  1. Run CellProfiler in headless mode. This entails executing a pipeline from a command line. e.g.

cellprofiler -c -r -p /location/of/mypipeline.cppipe

  1. Run cellprofiler over a subset of your images with the -l and -f options. You can process the entire the dataset by scripting.

Check out this post for a more in depth conversation.

process20585_small.cppipe (22.6 KB)

Hi

I am attaching the pipeline. Basically it is a modifier version of the one you published. I installed cellprofiler 2.1.1 on linux follows the instruction: http://cellprofiler.org/linux.html

I ran the pipeline with the following command, there are 18 images, 2 sets.
cellprofiler -p process20585_small.cppipe -c -r -o 20585_small_batch_run_out -i 20585_small -d 20585_small_batch_run_out/done.txt -L 10 out.txt

I got the following error message:

Signalling caller
Java virtual machine started.
Pipeline saved with CellProfiler version 20140723173957
Could not load cellprofiler.modules.ilastik_pixel_classification
Traceback (most recent call last):
File “/usr/cellprofiler/src/CellProfiler/cellprofiler/modules/init.py”, line 301, in add_module
m = import(mod, globals(), locals(), [‘all’], 0)
ImportError: No module named ilastik_pixel_classification
could not load these modules: cellprofiler.modules.ilastik_pixel_classification
HDF5Dict.init(): out.txt, temporary=False, copy=None, mode=w
HDF5Dict.flush(): out.txt, temporary=False
Failed to prepare run for module ExportToSpreadsheet
Traceback (most recent call last):
File “/usr/cellprofiler/src/CellProfiler/cellprofiler/pipeline.py”, line 2094, in prepare_run
if ((not module.prepare_run(workspace)) or
File “/usr/cellprofiler/src/CellProfiler/cellprofiler/modules/exporttospreadsheet.py”, line 466, in prepare_run
return self.check_overwrite(workspace)
File “/usr/cellprofiler/src/CellProfiler/cellprofiler/modules/exporttospreadsheet.py”, line 703, in check_overwrite
image_number = metadata_group.image_numbers[0]
IndexError: index 0 is out of bounds for axis 0 with size 0
HDF5Dict.flush(): out.txt, temporary=False
HDF5Dict.flush(): out.txt, temporary=False
HDF5Dict.del(): out.txt, temporary=False
stopping worker thread 0
stopping worker thread 1
stopping worker thread 2
stopping worker thread 3
stopping worker thread 4
stopping worker thread 5
stopping worker thread 6
stopping worker thread 7
stopping worker thread 8
stopping worker thread 9
stopping worker thread 10
stopping worker thread 11
stopping worker thread 12
stopping worker thread 13
stopping worker thread 14
stopping worker thread 15
stopping worker thread 16
stopping worker thread 17
stopping worker thread 18
stopping worker thread 19
stopping worker thread 20
stopping worker thread 21
stopping worker thread 22
stopping worker thread 23
stopping worker thread 24
Exiting the JVM monitor thread