How to use ADAM optimizer

I am interested in using ADAM optimizer instead of SGD. How can I change the default SGD to ADAM?

[jalal@scc2 openfield-filtered]$ ls
total 131
drwxr-sr-x 3 jalal ivcgroup   512 Dec  6 00:51 ..
drwxr-sr-x 3 jalal ivcgroup   512 Dec  6 00:51 labeled-data
-rw-r--r-- 1 jalal ivcgroup  1220 Dec  6 00:57 config.yaml
-rw-r--r-- 1 jalal ivcgroup  1275 Dec  6 00:58 mouse.py
drwxr-sr-x 2 jalal ivcgroup   512 Dec  6 00:58 training-datasets
drwxr-sr-x 2 jalal ivcgroup   512 Dec  6 00:59 evaluation-results
drwxr-sr-x 2 jalal ivcgroup   512 Dec  6 00:59 dlc-models
drwxr-sr-x 2 jalal ivcgroup 32768 Dec  6 01:01 videos
-rw-r--r-- 1 jalal ivcgroup   335 Dec  6 01:04 mouse.qsub
drwxr-sr-x 7 jalal ivcgroup 32768 Dec  6 01:04 .
[jalal@scc2 openfield-filtered]$ tree .
.
|-- config.yaml
|-- dlc-models
|-- evaluation-results
|-- labeled-data
|   `-- m4s1
|       |-- CollectedData_Pranav.csv
|       |-- CollectedData_Pranav.h5
|       |-- img0000.png
|       |-- img0001.png
|       |-- img0002.png
|       |-- img0003.png
|       |-- img0004.png
|       |-- img0005.png
|       |-- img0006.png
|       |-- img0007.png
|       |-- img0008.png
|       |-- img0009.png
|       |-- img0010.png
|       |-- img0011.png
|       |-- img0012.png
|       |-- img0013.png
|       |-- img0014.png
|       |-- img0015.png
|       |-- img0016.png
|       |-- img0017.png
|       |-- img0018.png
|       |-- img0019.png
|       |-- img0020.png
|       |-- img0021.png
|       |-- img0022.png
|       |-- img0023.png
|       |-- img0024.png
|       |-- img0025.png
|       |-- img0026.png
|       |-- img0027.png
|       |-- img0028.png
|       |-- img0029.png
|       |-- img0030.png
|       |-- img0031.png
|       |-- img0032.png
|       |-- img0033.png
|       |-- img0034.png
|       |-- img0035.png
|       |-- img0036.png
|       |-- img0037.png
|       |-- img0038.png
|       |-- img0039.png
|       |-- img0040.png
|       |-- img0041.png
|       |-- img0042.png
|       |-- img0043.png
|       |-- img0044.png
|       |-- img0045.png
|       |-- img0046.png
|       |-- img0047.png
|       |-- img0048.png
|       |-- img0049.png
|       |-- img0050.png
|       |-- img0051.png
|       |-- img0052.png
|       |-- img0053.png
|       |-- img0054.png
|       |-- img0055.png
|       |-- img0056.png
|       |-- img0057.png
|       |-- img0058.png
|       |-- img0059.png
|       |-- img0060.png
|       |-- img0061.png
|       |-- img0062.png
|       |-- img0063.png
|       |-- img0064.png
|       |-- img0065.png
|       |-- img0066.png
|       |-- img0067.png
|       |-- img0068.png
|       |-- img0069.png
|       |-- img0070.png
|       |-- img0071.png
|       |-- img0072.png
|       |-- img0073.png
|       |-- img0074.png
|       |-- img0075.png
|       |-- img0076.png
|       |-- img0077.png
|       |-- img0078.png
|       |-- img0079.png
|       |-- img0080.png
|       |-- img0081.png
|       |-- img0082.png
|       |-- img0083.png
|       |-- img0084.png
|       |-- img0085.png
|       |-- img0086.png
|       |-- img0087.png
|       |-- img0088.png
|       |-- img0089.png
|       |-- img0090.png
|       |-- img0091.png
|       |-- img0092.png
|       |-- img0093.png
|       |-- img0094.png
|       |-- img0095.png
|       |-- img0096.png
|       |-- img0097.png
|       |-- img0098.png
|       |-- img0099.png
|       |-- img0100.png
|       |-- img0101.png
|       |-- img0102.png
|       |-- img0103.png
|       |-- img0104.png
|       |-- img0105.png
|       |-- img0106.png
|       |-- img0107.png
|       |-- img0108.png
|       |-- img0109.png
|       |-- img0110.png
|       |-- img0111.png
|       |-- img0112.png
|       |-- img0113.png
|       |-- img0114.png
|       `-- img0115.png
|-- mouse.py
|-- mouse.qsub
|-- training-datasets
`-- videos
    `-- m3v1mp4.mp4

6 directories, 122 files

Here is all the files I have before I start the training. As you see pose_cfg.yaml file which includes name of optimizer is not yet created. Do I pass it as an argument to training? How?

deeplabcut.train_network(path_config_file, shuffle=1, displayiters=10, saveiters=100, gputouse=gputouse, maxiters=1000000)

When I searched for ADAM it only showed this file: https://github.com/AlexEMG/DeepLabCut/blob/9a129d4946323799cf4328ec567f237ecde8c24d/deeplabcut/pose_estimation_tensorflow/train.py

You can set it in the pose_cfg.yaml file, by adding the variable:

optimizer: adam

Note that you might also want to change the multi_step & batch_size variables.

3 Likes