Help automatically processing images

Image Analysis
#1

Hello,

We have an image analysis automation program called JEX, which basically pulls in a bunch of ImageJ functions. We want to import microscope images (probably multicolor .tifs or .nd2) into our program, process them, then export them to a folder. From there, we want to automatically start a CellProfiler process to process these images without having to open the GUI and adjust any settings.

Let’s say we’re working in the folder C:\Users\Tom\ExampleHuman and we’ve just exported images to C:\Users\Tom\ExampleHuman\images. We’d like to save the output in C:\Users\Tom\ExampleHuman\output. Can I put together a batch file or something that just automatically runs these images through a a CellProfiler process? Do these images have to have certain file names for this to work?

#2

Hi,

Good question. I would first direct you to how to run CellProfiler headless and from the command line: github.com/CellProfiler/CellPro … mmand_line

From there, you are mostly on your own, I’m afraid. We support many different configurations (OSs, cluster types, etc) and so are a bit agnostic about how you run your processing scripts, etc. You can run setup and run automated tools like a cron job on linux to poll your output directory and then spawn cellprofiler instances using the command, like detailed on the page above:

So all you need to specify is the image directory with the ‘-i’ switch, the output directory and the pipeline (hmm, it should work with *.cppipe as well as *.mat). If you want to parallel process, then look at the command-line help and use the ‘-f’ and ‘-l’ switches to batch process. If you’ll be running this on a computing cluster, then you’d need to have python accessible from each node, as well as CellProfiler (often much easier said than done!)

Another automation tool we use is Jenkins (though not to run CellProfiler) and I could imagine it polling every few minutes the output directory. But any automated tool or process you want here is likely ok (e.g. Travis).

Finally, unless you are really high-throughput and/or very standardized in your input and output, it might be just as easy to do this manually when you receive a lot of images, and set off as batch processes to a cluster or powerful machine(s). That is how we run it here at Broad, in short.

Hope that helps!
David

#3

New to all this, searched the forums , only example I found was with docker running the cppipe. Hence the below post

I’m having an issue related to this , I’m passing cppipe file, and its complaining its not a valid mat file
python CellProfiler.py -c -r -i TData -o output -p tpipeline.cppipe

folder TData has all the images

I get the below error, I’ve checked out 2.1.1, do I’ve to convert cppipe to mat ?

Updating Java dependencies using Maven.
running build_ext
skipping ‘_cpmorphology2.c’ Cython extension (up-to-date)
skipping ‘_watershed.c’ Cython extension (up-to-date)
skipping ‘_propagate.c’ Cython extension (up-to-date)
skipping ‘_filter.c’ Cython extension (up-to-date)
skipping ‘_lapjv.c’ Cython extension (up-to-date)
skipping ‘_convex_hull.c’ Cython extension (up-to-date)
running build_ext
skipping ‘javabridge.c’ Cython extension (up-to-date)
Version: 2014-07-23T17:39:57 6c2d896 / 20140723173957
Tried to load corrupted .MAT file: tpipeline.cppipe
Traceback (most recent call last):
File “/home/ubuntu/CellProfiler/cellprofiler/pipeline.py”, line 879, in load
struct_as_record=True)
File “/usr/lib/python2.7/dist-packages/scipy/io/matlab/mio.py”, line 125, in loadmat
MR = mat_reader_factory(file_name, appendmat, **kwargs)
File “/usr/lib/python2.7/dist-packages/scipy/io/matlab/mio.py”, line 55, in mat_reader_factory
mjv, mnv = get_matfile_version(byte_stream)
File “/usr/lib/python2.7/dist-packages/scipy/io/matlab/miobase.py”, line 236, in get_matfile_version
% ret)
ValueError: Unknown mat file type, version 62, 10
Failed to stop Ilastik
Exiting the JVM monitor thread

#4

Hi,

It’s possibly related to this issue? github.com/CellProfiler/CellPro … ssues/1318
Have you sent this pipeline through email, and it perhaps got ‘scrubbed’?

David