Using LoadData with CreateBatchFiles

I’m having some difficulties with CreateBatchFiles and LoadData, specifically with file paths when switching from the local computer I create the pipeline with to the computing cluster.

What I’m doing so far:

  1. Create a .csv file with the correct file paths for the local computer, along with metadata and group indexes etc.
  2. At the end of the pipeline use CreateBatchFiles, setting the local root path and the cluster root path. Run this on the local computer to generate a Batch_data.h5
  3. Then on the cluster I run cellprofiler --get-batch-commands Batch_data.h5 to generate a number of commands.

When I then try to run any of these I get the error file:

INFO:PipelineStatistics:Pipeline saved with CellProfiler version 20140723174500
INFO:PipelineStatistics:Times reported are CPU times for each module, not wall-clock time
ERROR:cellprofiler.pipeline:Error detected during run of module LoadData
...
...
OSError: [Errno 2] No such file or directory: '/exports/eddie/scratch/s1027820/test_2/Z:/scott/CP_cluster_test/test_2/load_data_eddie_paths.csv'
INFO:PipelineStatistics:Thu Jul  7 19:06:04 2016: Image # 1, module LoadData # 1: 0.01 sec

Where Z:/scott/CP_cluster_test/test_2/ is my local path and /exports/eddie/scratch/s1027820/test_2/ is the cluster root path. So it looks like it’s pasting together my local and root file paths before the LoadData .csv file.

Any idea where I’m going wrong? The only clear instructions for I can find for LoadData and CreateBatch files are for CellProfiler1.0.

My guess is that most likely there’s something funny in how your path mapping is set up that’s causing a conflict, but I can’t know for sure without the .cpproj and the CSV. Are your image files local or on the cluster? What about your csv and your batch_data file?

And why is your Windows (right?) path with forward slashes instead of backslashes? Windows may be forgiving about the direction of the slashes but I suspect that CellProfiler is not and does a straightforward search-replace.

Oops, that’s the reason. Changed the path separators and everything worked fine. Thanks for the help.