DeepLabCut: Step 1 Analyze Video "CalledProcessError"

Hi!

I recently have been having trouble with the step of “Analyzing Videos”. This time I had this error as code:

Analyzing ... 
Config:
{'Task': None,
 'TrainingFraction': None,
 'all_joints': [[0], [1], [2], [3], [4], [5], [6], [7], [8]],
 'all_joints_names': ['head',
                      'snout',
                      'forepawright',
                      'forepawleft',
                      'trunk',
                      'hindpawright',
                      'hindpawleft',
                      'tailbase',
                      'tail1'],
 'alphavalue': None,
 'batch_size': 8,
 'bodyparts': None,
 'bottomheight': 400,
 'colormap': None,
 'corner2move2': None,
 'crop': True,
 'crop_pad': 0,
 'croppedtraining': None,
 'cropping': None,
 'cropratio': 0.4,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_Project04Jun13/Project04_CorderLab95shuffle1.pickle',
 'dataset_type': 'multi-animal-imgaug',
 'date': None,
 'default_augmenter': None,
 'default_net_type': None,
 'deterministic': False,
 'display_iters': 500,
 'dotsize': None,
 'fg_fraction': 0.25,
 'global_scale': 0.8,
 'init_weights': '/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/resnet_v1_50.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'iteration': None,
 'leftwidth': 400,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 0.05,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'max_input_size': 1500,
 'mean_pixel': [123.68, 116.779, 103.939],
 'metadataset': 'training-datasets/iteration-0/UnaugmentedDataSet_Project04Jun13/Documentation_data-Project04_95shuffle1.pickle',
 'min_input_size': 64,
 'minconfidence': 0.01,
 'minsize': 100,
 'mirror': False,
 'move2corner': None,
 'multi_step': [[0.0001, 7500], [5e-05, 12000], [1e-05, 200000]],
 'multianimalproject': None,
 'net_type': 'resnet_50',
 'nmsradius': 5.0,
 'num_joints': 9,
 'num_limbs': 12,
 'num_outputs': 1,
 'numframes2pick': None,
 'optimizer': 'adam',
 'pafwidth': 20,
 'pairwise_huber_loss': False,
 'pairwise_loss_weight': 0.1,
 'pairwise_predict': False,
 'partaffinityfield_graph': [[1, 0],
                             [1, 2],
                             [1, 3],
                             [1, 4],
                             [1, 7],
                             [0, 4],
                             [4, 3],
                             [4, 2],
                             [4, 7],
                             [4, 6],
                             [4, 5],
                             [7, 8]],
 'partaffinityfield_predict': True,
 'pcutoff': None,
 'pos_dist_thresh': 17,
 'project_path': '/Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13',
 'regularize': False,
 'rightwidth': 400,
 'save_iters': 10000,
 'scale_jitter_lo': 0.5,
 'scale_jitter_up': 1.25,
 'scoremap_dir': 'test',
 'scorer': None,
 'shuffle': True,
 'skeleton': [],
 'skeleton_color': 'black',
 'snapshot_prefix': '/Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/dlc-models/iteration-0/Project04Jun13-trainset95shuffle1/test/snapshot',
 'snapshotindex': None,
 'start': None,
 'stop': None,
 'stride': 8.0,
 'topheight': 400,
 'video_sets': None,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001,
 'x1': None,
 'x2': None,
 'y1': None,
 'y2': None}
Using snapshot-100 for model /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/dlc-models/iteration-0/Project04Jun13-trainset95shuffle1
Initializing ResNet
Activating extracting of PAFs
Starting to analyze %  /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4
/Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos  already exists!
Loading  /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4
Traceback (most recent call last):
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/gui/analyze_videos.py", line 477, in analyze_videos
    robust_nframes=robust,
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/pose_estimation_tensorflow/predict_videos.py", line 300, in analyze_videos
    robust_nframes=robust_nframes,
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/pose_estimation_tensorflow/predict_multianimal.py", line 57, in AnalyzeMultiAnimalVideo
    nframes = auxfun_videos.get_nframes_robust(video)
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/utils/auxfun_videos.py", line 293, in get_nframes_robust
    output = subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'ffprobe -i /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4 -v error -count_frames -select_streams v:0 -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1' returned non-zero exit status 127.

When I submitted for this error before, I was told it was because the name of my video file or project file was too complex with its characters and whatnot, so I deleted it. I am not too sure why this is occurring but here is an image of my GUI also…

Please let me know how I can resolve this issue as I would greatly appreciate it!

1 Like

it could also be the meta data is somehow corrupted; try checking “use openCV” NO --> this tried an alternative mode with some difficult videos.

1 Like

Thank you for your speedy response! Unfortunately, trying that out did not work and responded back with code I never came across with DLC before… kind of stuck on knowing how to respond to it… :frowning:

Analyzing ... 
Using snapshot-100 for model /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/dlc-models/iteration-0/Project04Jun13-trainset95shuffle1
Initializing ResNet
Activating extracting of PAFs
2020-06-13 21:10:44.969882: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2020-06-13 21:10:44.971529: I tensorflow/core/common_runtime/process_util.cc:71] Creating new thread pool with default inter op setting: 8. Tune using inter_op_parallelism_threads for best performance.
Starting to analyze %  /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4
/Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos  already exists!
Loading  /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4
Traceback (most recent call last):
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/gui/analyze_videos.py", line 477, in analyze_videos
    robust_nframes=robust,
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/pose_estimation_tensorflow/predict_videos.py", line 300, in analyze_videos
    robust_nframes=robust_nframes,
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/pose_estimation_tensorflow/predict_multianimal.py", line 60, in AnalyzeMultiAnimalVideo
    duration = auxfun_videos.get_duration(video)
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/utils/auxfun_videos.py", line 287, in get_duration
    output = subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'ffprobe -i /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4 -show_entries format=duration -v quiet -of csv="p=0"' returned non-zero exit status 127.

Here is a shot of my GUI also:

Would love to hear your thoughts on this and thank you again for taking the time to help as always!

no problem, in the terminal, can you run and post the output?

import deeplabcut
deeplabcut.__version__

and also in the terminal (inside the conda env):

ffprobe /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4

Thank you for the speedy response once again.

Here is what I received as the output - let me know if I did anything wrong.

(DLC-CPU) justinjames@Justins-MacBook-Pro ~ % import deeplabcut
zsh: command not found: import
(DLC-CPU) justinjames@Justins-MacBook-Pro ~ % pythonw
Python 3.7.6 (default, Jan  8 2020, 13:42:34) 
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import deeplabcut
>>> deeplabcut.__version__
'2.2b5'
>>> ffprobe /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4
  File "<stdin>", line 1
    ffprobe /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4
                                                                            ^
SyntaxError: invalid token
>>> 


awe, can you then run

pip install deeplabcut==2.2b6

see if that will fix it :wink:

also just for kicks, sorry it should have said:

ffprobe "/Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4"

Unfortunately, the update to the 2.2b6 did not work. I ended up getting the same error at the end when running the analysis as before… I am really stuck on why this is happening.

Here is the code:

Analyzing ... 
Using snapshot-100 for model /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/dlc-models/iteration-0/Project04Jun13-trainset95shuffle1
Initializing ResNet
Activating extracting of PAFs
Starting to analyze %  /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4
/Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos  already exists!
Loading  /Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4
Traceback (most recent call last):
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/gui/analyze_videos.py", line 483, in analyze_videos
    robust_nframes=robust,
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/pose_estimation_tensorflow/predict_videos.py", line 302, in analyze_videos
    robust_nframes=robust_nframes,
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/pose_estimation_tensorflow/predict_multianimal.py", line 63, in AnalyzeMultiAnimalVideo
    duration = auxfun_videos.get_duration(video)
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/deeplabcut/utils/auxfun_videos.py", line 296, in get_duration
    output = subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "/Users/justinjames/Applications/anaconda3/envs/DLC-CPU/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'ffprobe -i "/Users/justinjames/Desktop/DeepLabCut/Project04-CorderLab-2020-06-13/videos/vid.mp4" -show_entries format=duration -v quiet -of csv="p=0"' returned non-zero exit status 127.

And here is an image of the GUI:

okay, let’s check the ffmpeg installation; in the “Video Editor” tab, can you try with a test video to crop or shorten or downsample something?

and if that doesn’t work, let’s reinstall ffmpeg: https://www.wikihow.com/Install-FFmpeg-on-Windows

I also just noticed it’s only trained for 100 iterations; you should def. train more… if you don’t have a GPU handy, just use this new notebook on COLAB: https://github.com/DeepLabCut/DeepLabCut/blob/c21931789df731af3f3fe987c65b08279fa1a367/examples/COLAB_maDLC_TrainNetwork_VideoAnalysis.ipynb

You were right when you said that I needed to check the ffmpeg installation btw! I saw that it was not installed, so I downloaded it and was able to crop, shorten, and downsample the test video. Would this allow me to complete an Analysis on the Project GUI I am running now?

Yeah I only did 100 iterations for the sake of time as I am trial and erring with DLC right now… I do not have a GPU, so I have been using the DLC-CPU environment…

Because I do not have a handy GPU, is it not possible to complete a project on the DLC GUI from the environment I created? Or do I have to use a new notebook on COLAB?

1 Like

Hi again!

I tried to run my project on the COLAB notebook through the link you provided, however, when getting to the step of “analyzing the video”, I once again came across a “CalledProcessError”.

Here is a code of the issue:

Config:
{'all_joints': [[0], [1], [2], [3], [4], [5], [6], [7], [8]],
 'all_joints_names': ['head',
                      'snout',
                      'forepawright',
                      'forepawleft',
                      'trunk',
                      'hindpawright',
                      'hindpawleft',
                      'tailbase',
                      'tail1'],
 'batch_size': 8,
 'bottomheight': 400,
 'crop': True,
 'crop_pad': 0,
 'cropratio': 0.4,
 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_Project05Jun15/Project05_JJ95shuffle1.pickle',
 'dataset_type': 'multi-animal-imgaug',
 'deterministic': False,
 'display_iters': 500,
 'fg_fraction': 0.25,
 'global_scale': 0.8,
 'init_weights': '/usr/local/lib/python3.6/dist-packages/deeplabcut/pose_estimation_tensorflow/models/pretrained/resnet_v1_50.ckpt',
 'intermediate_supervision': False,
 'intermediate_supervision_layer': 12,
 'leftwidth': 400,
 'location_refinement': True,
 'locref_huber_loss': True,
 'locref_loss_weight': 0.05,
 'locref_stdev': 7.2801,
 'log_dir': 'log',
 'max_input_size': 1500,
 'mean_pixel': [123.68, 116.779, 103.939],
 'metadataset': 'training-datasets/iteration-0/UnaugmentedDataSet_Project05Jun15/Documentation_data-Project05_95shuffle1.pickle',
 'min_input_size': 64,
 'minconfidence': 0.01,
 'minsize': 100,
 'mirror': False,
 'multi_step': [[0.0001, 7500], [5e-05, 12000], [1e-05, 200000]],
 'net_type': 'resnet_50',
 'nmsradius': 5.0,
 'num_joints': 9,
 'num_limbs': 12,
 'num_outputs': 1,
 'optimizer': 'adam',
 'pafwidth': 20,
 'pairwise_huber_loss': False,
 'pairwise_loss_weight': 0.1,
 'pairwise_predict': False,
 'partaffinityfield_graph': [[1, 0],
                             [1, 2],
                             [1, 3],
                             [1, 4],
                             [1, 7],
                             [0, 4],
                             [4, 3],
                             [4, 2],
                             [4, 7],
                             [4, 6],
                             [4, 5],
                             [7, 8]],
 'partaffinityfield_predict': True,
 'pos_dist_thresh': 17,
 'project_path': '/content/drive/My Drive/Project05-JJ-2020-06-15',
 'regularize': False,
 'rightwidth': 400,
 'save_iters': 10000,
 'scale_jitter_lo': 0.5,
 'scale_jitter_up': 1.25,
 'scoremap_dir': 'test',
 'shuffle': True,
 'snapshot_prefix': '/content/drive/My '
                    'Drive/Project05-JJ-2020-06-15/dlc-models/iteration-0/Project05Jun15-trainset95shuffle1/test/snapshot',
 'stride': 8.0,
 'topheight': 400,
 'weigh_negatives': False,
 'weigh_only_present_joints': False,
 'weigh_part_predictions': False,
 'weight_decay': 0.0001}
Start Analyzing my video(s)!
Using snapshot-5000 for model /content/drive/My Drive/Project05-JJ-2020-06-15/dlc-models/iteration-0/Project05Jun15-trainset95shuffle1
Initializing ResNet
Activating extracting of PAFs
Analyzing all the videos in the directory...
Starting to analyze %  CROPPED - Single - 10SecRVLisaM2.mp4
.  already exists!
Loading  CROPPED - Single - 10SecRVLisaM2.mp4
---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
<ipython-input-12-213d8938cf3d> in <module>()
      4                                        shuffle=shuffle,
      5                                        videotype=VideoType,
----> 6                                        c_engine=False)

4 frames
/usr/lib/python3.6/subprocess.py in run(input, timeout, check, *popenargs, **kwargs)
    436         if check and retcode:
    437             raise CalledProcessError(retcode, process.args,
--> 438                                      output=stdout, stderr=stderr)
    439     return CompletedProcess(process.args, retcode, stdout, stderr)
    440 

CalledProcessError: Command 'ffprobe -i CROPPED - Single - 10SecRVLisaM2.mp4 -show_entries format=duration -v quiet -of csv="p=0"' returned non-zero exit status 1.
SEARCH STACK OVERFLOW

And here is an image sharing where I am at:

Let me know your thoughts and thank you once again for the help throughout!

Hi! I just wanted to let you know that when I ran a new project, now in the updated DLC version and with the ffmpeg installed, I was able to successfully complete the step of “Analyze Videos”.

However, an issue I am now facing is the fact that the video analyzed/created in “Step 1: Analyze Videos” is not showing the tracklets or markers that I believe should be seen once it is done…

I went forward to the “Refine Tracklets” step to do “Step 1: Launch GUI” and there is no points for me to track and see the past and future trajectories…

I do not know what I did wrong here, but would appreciate whatever insight you can give me!

Hi did you run step 2, to create the tracklets with a specific tracker, i.e. either “box or skeleton”?

the colab image is that file name, can’t contain spaces :slight_smile:

Yeah, I did run “Step 2: Convert to Tracklets” on the Analyze Video page with the specific tracker being “box” — I tried to test it out with “skeleton” but it was saying that it had already created a video and what not…

And I am sorry but I do not understand what you mean when you say the file name cannot contain spaces in the collab images?

Can you make a screen recording video of this? There is no video created when you convert to tracklets, so I am confused. Running both box and skeleton is doable. Then on the next tab you need to load the video and the …bx…pickle file.

@MWMathis We are working on this together, so I took a screen recording to show you. As you can see, the GUI shows very few marker points throughout the video and the marker points are all very off. I had to upload the problem as two separate youtube videos: https://youtu.be/fm4gWrGXmbw https://youtu.be/L8Wbdr_ZONM

well this is a different issue that OG poster, i.e. he said he could not load the tracklets data. How they look is due to many factors, including how good your network is.

  • How does the video look that you need to create during analyze “full detections” ?
  • How did the cross-validation look? what is the RMSE? PCK? etc?

There are a LOT of parameters to look at/optimize, hence why MA tracking is much harder; but this looks like a single mouse that normal DLC would handle very well (albeit the lighting should be improved).

Hi Mackenzie,

I ran it as a single animal. These were the settings I included for the analyze videos step:

I then selected create videos with these settings: A new video appeared in my file, but it is unlabeled and looks just like the original video.

Hi @jwojick again I think there is some confusions here. the video you uploaded is for tracklets, for maDLC: https://www.youtube.com/watch?v=fm4gWrGXmbw&feature=youtu.be this is what I mean by a “multi-animal project.”

The image you now post are for different project, not the Test-JJ- etc etc…