Extract_outlier_frames cannot find the new analyzed video

Dear DLC developers,

I have an issue with trying to extract outlier frames from a newly analyzed video with the purpose of using these frames in retraining. I am using multianimal DLC (2.2v8) for single animal tracking installed with pip inside the latest provided conda environment. The OS is Ubuntu 20 and I am running the code within a Jupyter Notebook.

Code summary

1.

After successfully training the network, I first cross-validate the parameters:

dlc.evaluate_multianimal_crossvalidate(config_path_uniformSampler,
Shuffles=[1, 2, 3],
pbounds = {‘pafthreshold’: (0, 0.9),
‘detectionthresholdsquare’: (0, 0.9),
‘minimalnumberofconnections’: (0, 21)},
edgewisecondition=True,
leastbpts=0,
init_points=20,
n_iter=50,
target=‘pck_train’)

2.

Next, I analyze a novel video:

novel_video_path = ‘/home/kejzarn/dlc/work/video_analysis/20200915_162014.mp4’
scorername = dlc.analyze_videos(config_path,
[novel_video_path],
videotype=‘mp4’,
gputouse=0)

dlc.create_video_with_all_detections(config_path,
[novel_video_path],
scorername)

This successfully creates the “novel_video_full” files, specifically, the .mp4, full.pickle and meta.pickle. Inspecting the novel_video_full.mp4, the labels are there and pretty accurate.

3.

After this, I convert the labeled video to tracklets and convert the .pickle file to required .h5 without refinement (I just want to extract outlier frames as the model is not good enough yet to refine tracklets):

dlc.convert_detections2tracklets(config_path,
[novel_video_path],
videotype=‘mp4’)

dlc.convert_raw_tracks_to_h5(config_path,
‘/home/kejzarn/dlc/work/video_analysis/20200915_162014DLC_resnet101_v1_uniformSamplerSep30shuffle1_100000_bx.pickle’)

4.

Finally, I attempt to extract outlier frames:

dlc.extract_outlier_frames(config_path,
[novel_video_path],
videotype=’.mp4’,
outlieralgorithm=‘uncertain’,
p_bound=0.1)

Error

In step 4 I obtain the following error:

No unfiltered data file found in /home/kejzarn/dlc/work/video_analysis for video 20200915_162014 and scorer DLC_resnet101_v1_uniformSamplerSep30shuffle1_100000.
It seems the video has not been analyzed yet, or the video is not found! You can only refine the labels after the a video is analyzed. Please run ‘analyze_video’ first. Or, please double check your video file path

Things I tried to remedy the problem

1.

Clearly, the video is there, since it was detected during all the previous steps (2-3). I checked, whether is perhaps a permission problem, but the folder has 777 permission.

2.

I tried the full video path (from /home folder) as well as the video path from where I executed my Jupyter notebook. Same error.

3.

I tried the whole process on a different video. Same error.

4.

I tried executing the manual selection of outlier frames. Same error.

5.

I tried extract outlier frames within the GUI (dlc.launch_dlc()). In manual mode, I could extract the frames, but the labels were not there. When trying automatic methods, I got the following error:

Command ‘ffprobe -i “/home/kejzarn/dlc/work/video_analysis/20200915_162014.mp4” -show_entries format=duration -v quiet -of csv=“p=0”’ returned non-zero exit status 127.

At this point, I ran out of ideas, so your help would be greatly appreciated. Please let me know, if you require any further info.

Many thanks,
Nejc

Hey @neitz, you need to specify the tracking method (‘box’ or ‘skeleton’) in extract_outlier_frames; deeplabcut will then find your tracks :slight_smile:

1 Like

Dear @jeylau, right you are! Thank you sincerely for such a swift reply!

I have noticed another potential issue: this seems to work only with box. skeleton produces the same error. I tried rerunning the convert_detections2tracklets as well as convert_raw_tracks_to_h5 with both track methods, but using skeleton during extraction always produces the same error.

Additionally, might I suggest adding track_method = 'box' as a default parameter value to the extract_outlier_frames function? Currently, it is not set (track_method=''), which is why I haven’t thought about requiring this parameter.

Changing the default is a valid point! It is not required for regular, single-animal projects, which is why it defaults to the empty string. I agree that making the argument non-optional is certainly wiser :slight_smile: You could submit a PR and we will review it!

1 Like