Analyzing new videos (no data)

Hello,

I used a video to train the model and then when i try to add new video to analyze, i get “no data”.
The original video did fine. We labeled 200 frames from a 15 min video, and all went smoothly. It is just when we add new video to analyze that this happens.

I also followed the solutions in the posts below, but nothing worked.

If you have a solution outside of these 3, please let me know.

I am using Ubuntu 18/ DLC 2.2b8

Thanks in advance.

As outlined by the issues you documented, there are three possible failure points:

  • (trivial): video not found and thus nothing happens. Fix: check path
  • Video corrupt, which will not lead to “analyzed data”. However that would throw an error. Fix: check video integrity.

Perhaps, if you post the exact traceback for your situation we could be of more specific help!

1 Like

Thank you Alexander.

The video was found and the path was correct.
The video works fine, and no error “Fix: check video integrity.” was produced. The whole video plays smoothly.

Here is what i do:
1- I open the GUI, I click analyze video
2- I add the video (new video, not used before), assign all the options (video type mp4, shuffle 1, training index 0, tracker method -tried both, create video fro check detections yes, read video metadata tried both)
3- I get

Duration of video [s]: 1062.08 , recorded with 59.94 fps!
Overall # of frames: 63660 found with (before cropping) frame dimensions: 1920 1080
Starting to extract posture
0%| | 0/63660 [00:00<?, ?it/s]Detected frames: 44
1%|█▌ | 636/63660 [00:50<1:24:03, 12.49it/s]
Saving results in /k/path/to/dir/2020-12-01/videos…
The videos are analyzed. Time to assemble animals and track 'em…
Call ‘create_video_with_all_detections’ to check multi-animal detection quality before tracking.
If the tracking is not satisfactory for some videos, consider expanding the training set. You can use the function ‘extract_outlier_frames’ to extract a few representative outlier frames.
DLC_resnet50_topviewDec1shuffle1_200000
Creating labeled video for 15f18f_0.1mdma_91720_B1
0%| | 0/63660 [00:00<?, ?it/s]/home/k/anaconda3/envs/dlc-ubuntu-GPU/lib/python3.6/site-packages/deeplabcut/utils/make_labeled_video.py:846: FutureWarning: circle is deprecated in favor of disk.circle will be removed in version 0.19
rr, cc = circle(y, x, dotsize, shape=(ny, nx))
0%| | 39/63660 [00:00<15:43, 67.42it/s]Error: %s axis 2 is out of bounds for array of dimension 0
44 no data
44 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
45 no data
45 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
46 no data
46 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
47 no data
47 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
48 no data
48 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
49 no data
49 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
50 no data
50 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
51 no data
51 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
52 no data
52 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
53 no data
53 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
54 no data
54 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
55 no data
55 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
56 no data
56 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
57 no data
57 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
58 no data
58 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
59 no data
59 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
60 no data
60 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
61 no data
61 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
62 no data
62 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
63 no data
63 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
64 no data
64 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
65 no data
65 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
66 no data
66 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
67 no data
67 frame writing error.
0%|▏ | 68/63660 [00:00<12:06, 87.57it/s]
Error: %s axis 2 is out of bounds for array of dimension 0
68 no data
68 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
69 no data
69 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
70 no data
70 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
71 no data
71 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
72 no data
72 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
73 no data
73 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
74 no data
74 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
75 no data
75 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
76 no data
76 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
77 no data
77 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
78 no data
78 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
79 no data
79 frame writing error.
Error: %s axis 2 is out of bounds for array of dimension 0
80 no data
80 frame writing error.
etc…

@AlexanderMathis Just checking if you want more info about this?

It is failing to read video frames starting with frame 44. The no data is referring to video data, not track data.

You might try updating deeplabcut to 2.1.10.1. I can’t remember exactly, but I think there have been some updates to the video checking since 2.2b8.

However, no matter your version, that video file is probably corrupted. Usually, you can just re-encode such videos. You can use adobe premier or other video editors to open and export the video. I prefer to do it with ffmpeg - the process of re-encoding copies the data but fixes file headers. ffmpeg can ignore those errors, but openCV and moviepy (used in DLC) can’t ignore them.

Use the instructions below to re-encode, then try analyzing that new video in deeplabcut.

To re-encode:

On your command line (after activating your deeplabcut env):

ffmpeg -i "path/to/input.mp4" -c:v libx264 -crf 23 -c:a copy "/path/to/output.mp4"

  • set the full path to the corrupted video as the input
  • set the full path to your desired place and name of the output video. You can try to keep the extension the same as your input, but usually it’s best with this code to change it to mp4.
  • change -crf 23 as follows (its a compression algorithm that helps force the re-encoding, but can also save some disk space and analysis time):
  • 23 will shrink your file size, and be visually similar to the original, good enough for most deeplabcut analyses (see the 2018 deeplabcut paper)
  • Higher values = more compression, smaller files, faster analysis, but some possible loss of details (see papers). It maxes out at 51, but that is probably way too much.
  • Lower values = less compression: 18 is usually visually lossless, 0 is technically lossless but can result in needlessly large file sizes (larger than the original) and very slow DLC analysis
  • I’ve never used anything lower than 18 or higher than 28 for video analyses
1 Like

Thank you, that worked for me!

Just my piece of advice on video reencoding. With 1080p videos you can go for libx265, crf 18, veryfast preset. Although more cpu intensive and more time consuming (roughly 2x slower), with settings mentioned above libx265 will result in slightly better quality and most of all in way better compression.

For instance:

  • my input video size = 1.4GB
  • libx265 -preset veryfast -crf 18 = 62MB and I can’t tell the difference visually
  • libx264 -preset veryfast -crf 18 = 347MB and slightly worse quality, especially the saturation/contrast

It depends how much file size matters to you, I like to keep my videos as small as possible (since they are very long recording sessions) to be able to move them around easily if needed and as to the time of reencoding I leave it overnight for the whole directory of videos.

EDIT: Should’ve posted this earlier. For batch processing use ffmpeg in for loop when inside directory:
Windows command will be:

for %i in (*.avi) do ffmpeg -i "%i" -c:v libx265 -preset veryfast -crf 18 "%~ni.mp4"

Linux should be something like this i think:

for i in *.avi; do ffmpeg -i "$i" -c:v libx265 -preset veryfast -crf 18 "${i%.*}.mp4"; done
2 Likes

WOW, thanks so much! I wanted to ask but i felt that it would be too much! That was quite generous Konrad!!

1 Like