Running a Pipeline Iteratively

Hi, I am looking to run a pipeline multiple times iteratively and automatically (i.e. so I don’t have to manually change the values each time I run the pipeline, but instead I run it and the pipeline runs multiple times and the input values increment or decrement by a certain amount each time the pipeline is run), changing some of the parameters used to filter objects at each iteration. I was thinking about doing this using a system command in Matlab, but are there any ways to do this within CellProfiler itself? Thanks!

Matthew Papke

Hi Matthew,

CellProfiler doesn’t allow for modification of inputs at run-time for the most part. I can think of a couple of ways around this, though:

  • You don’t describe precisely what you are filtering or how, but some modules rely on inputs coming from prior measurements produced upstream, e.g, FilterObjects, FlagImage, IdentifyPrimary using “Measurement” as the thresholding strategy. If you use the LoadData module to load your images via CSV, you can also load metadata values along with your image sets. These metadata values can be referred to within the aforementioned modules, and can be used to input user-specified values that you list in the CSV.
  • The pipeline is a text-readable file, so a third-part script can be used to re-write portions of the pipeline file (such as the desired setting and associated value) as needed, and then run.

In both cases, your script would be running CellProfiler without the UI (i.e, “headless”); type “cellprofiler --help” at a command prompt to see the listing of arguments than can be used in your script.


I understand that I can modify the pipeline via a third party, and then run it, but is there any way that I can actually modify the CellProfiler Python code to make a pipeline run iteratively? I only ask because I’m having trouble running it from Matlab (as seen in here Executing a CellProfiler pipeline from Terminal (Mac OS X) ). Thank you, and any recommendations would be appreciated.

Do you really mean the underlying code, or just the settings in a normal pipeline? I suppose you could File > Export > pipeline and then find and modify the settings in the pipeline programmatically, then iterate.

But a collaborator of ours has already done this! … aramorama/