I’ve been working on getting DCP running on aws. I opened a new account and configured everything up to the point of running the “startCluster” command. I recorded all the aws and other various steps with screenshots and text and compiled that into a .pdf. Unfortunately the forum doesn’t upload pdf’s. I submitted it to the CP admin folks. Hopefully they will post it somewhere.
My problem is this: after getting everything configured I tried running a test analysis. fab setup and submitJob seem to work. I see a ecs.config file appear in s3 and the jobs in SQS, but when I try to run the next command this is the output:
~/Distributed-CellProfiler$ python run.py startCluster files/fleet03.json
Traceback (most recent call last):
File “run.py”, line 338, in
File “run.py”, line 199, in startCluster
File “run.py”, line 78, in generateUserData
File “/usr/lib/python2.7/subprocess.py”, line 711, in init
File “/usr/lib/python2.7/subprocess.py”, line 1343, in _execute_child
OSError: [Errno 2] No such file or directory
Here is a zip with all the relevant files: project.zip (4.4 MB)
I obscured my numbers in a few places (e.g. xxxxx). I tried changing a bunch of things, but keep getting the same error “No such file or directory”.
In case there was something weird going on with permissions I did all my file manipulations on the ubuntu CP control node. I downloaded from cellprofiler:
and did all my editing with vi. I uploaded to s3 like this:
$ sudo aws s3 sync project/ s3://cp.project.bucket/project
The comment earlier about the CP Lab having set up their aws services a long time ago seems like a red flag or at least a clue to what seems like a permissions issue.
Again thanks for your time. -john