Errors starting Anaconda for Deeplabcut

So far I am on day three (and 6 hours tonight - it is now 01:15 in the morning) of trying to get this working on (repeated) fresh installations of Ubuntu 18.0.4

Nvidia Drivers are in and Cuda 9.0 is OK

Anaconda does launch although there appear to be errors:

user@machine:~/anaconda3$ anaconda-navigator
/home/dlc/anaconda3/lib/python3.7/site-packages/anaconda_navigator/api/conda_api.py:1364: YAMLLoadWarning: calling yaml.load() without Loader=… is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
data = yaml.load(f)

However when I try to use deeplabcut in python I get:

(DLC2-36) user@machine:~$ ipython
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31)
Type ‘copyright’, ‘credits’ or ‘license’ for more information
IPython 7.9.0 – An enhanced Interactive Python. Type ‘?’ for help.

In [1]: import deeplabcut
Illegal instruction (core dumped)
(DLC2-36) dlc@DLC-CUDA:~$

That’s all it does. Any ideas before I throw the Linux box out of the window and try Windoze?

What happens when you run “nvidia-smi” ?

Can you tell me how you built the dlc2-36 conda environment?

Can you try this one? Deeplabcut.org ?

I’m typing this in the office as the PC built for this work is at home in my study. So from memory; Nvidia-Ami shows driver 318 and CUDA 10.1.

As this is Ubuntu 18.04 only Cuda 10 will install. I can easily rebuild to Ubuntu 17.94 (which is supported by Cuda 9.0) if Cuda 10 is likely to be the issue?

I did try the dlc-ubuntu-GPU.yaml but
<coda env create -f dlc-ubuntu-GPU.yaml>
would not run successfully.

Seems your anaconda installation is incorrect

also be sure you have 10 or lower, and the driver then needs to match; i.e.: https://github.com/AlexEMG/DeepLabCut/blob/master/docs/installation.md#troubleshooting

maybe you mean 418…

i would highly recommend docker as well, it’s very easy to install and easy to use, and a lot “safer” than conda, as it’s a true virtual env on your system.

Thanks, Docker looks much better. One question, following the instructions through, I have reached:

GPU=1 bash ./dlc-docker run -d -p 2351:8888 -e USER_HOME=$HOME/DeepLabCut --name containername dlc_username/dlcdocker

Can I use multiple GPUs? (both my Nvidia Tesla GPUs available in nvidia-smi as ‘0’ and ‘1’) If so, what is the syntax for that?

Cheers,
Steve

So close (maybe)

dlc@DLC-CUDA:~/Docker4DeepLabCut2.0$ docker run hello-world
Hello from Docker!
This message shows that your installation appears to be working correctly

So docker seems OK.

I had an educated guess at the syntax for the GPUs (although it makes no difference to the outcome)

GPU=0,1 bash ./dlc-docker run -d -p 2351:8888 -e USER_HOME=$HOME/Docker4DeepLabCut2.0 --name DLC_Docker dlc_username/dlcdocker

Initially all looks good:

1st CMD run

* Setting user name to:             dlc
* Setting user ID to:               1000
* Setting user groups to:           sudo
* Setting password to:              pw
* Setting Notebook port binding to: 2351 (to set manually add -p 2351:8888 as flag)

You can now open the notebook on the host machine by directing your browser to

    http://localhost:2351

then I get an error:

0,1 nvidia-docker run -v /home/dlc:/home/dlc -p 38041:22 -p 2351:8888 -e USER_GROUPS=sudo -e USER=dlc -e USER_ID=1000 -e USER_ENCRYPTED_PASSWORD=############## -e USER_HOME=/home/dlc -e GPU=0,1 -e DISPLAY=:1 -d -e USER_HOME=/home/dlc/Docker4DeepLabCut2.0 --name DLC_Docker dlc_username/dlcdocker
./dlc-docker: line 179: nvidia-docker: command not found
(base) dlc@DLC-CUDA:~/Docker4DeepLabCut2.0$ 

Unfortunately you need to bind to 1 GPU, but this also means you can run two things at the same time on each gpu. Meaning you can always have multiple Docker containers running :slight_smile: this is what we routinely do, as even the best GPUs are more than enough for running large batches, ie in our Titan RTXs (24 GB memory) we can run batch sizes of >100 for analysis, so it’s not necessary to bind multiple to one container