I have been trying to get Distributed CellProfiler up and running on AWS and I’ve reached an issue that I haven’t been able to resolve after a week of googling and tweaking.
I’ve gone through all of the steps on the wiki page on setup and configuration and can create a spot fleet, but it never runs. After starting the cluster, when I monitor the spot fleet “xxxxSpotFleetRequestId.json” it returns: “In process: 0 Pending 1”. and it will stay like this. Occasionally, it will change to “In process: 1 Pending 0” for about 30 seconds, but then goes back to “In process: 0 Pending 1”. It will do this for hours if I let it. And it never outputs any of the files that the pipeline should (it is very simple-small pipeline for the purposes of just getting CP up and running)
In the EC2 console, a number of instances are create then quickly terminated.
In the ECS console, a cluster is created, but under Services->Events I get the following message:
" service [TrialCPService] was unable to place a task because no container instance met all of its requirements. Reason: No Container Instances were found in your cluster. For more information, see the [Troubleshooting section]"
In my S3 bucket, and “ecsconfigs” file and folder are created but nothing else.
Obviously, I’ve tried to troubleshoot this but I can’t figure it out. I can’t find any other error messages on AWS and I do not get any errors in PUTTY terminal.
I have been using the settings recommended in the Distributed CellProfiler wiki in terms of the size and computing.