DeepLabCut - Increasing pixel error and p-cutoff with new videos

I have successfully trained a network with 5 videos that are all fairly similar in background/focus etc and had the train pixel error down to 3 pixels and test pixel error at 6. Each video had ~150 labeled frames and ~4,000 frames total. However, when I added a new video, still with the same background/focus and basic behavior, the labels weren’t very accurately. I went on to extract 50 outlier frames and label them, but when I returned the results, my p-cutoff increased from 0.1 to 0.4 which seems like a red flag to me.

Do people have experience with training on many videos? Is there a point at which you have been able to just add new videos without training and gotten accurate results? What could be the reasoning for the p-cutoff to increase so much?

p-cutoff you want to keep high… then the network only plots points that is it confident about. You can set this value in the config.yaml file. It has nothing to do with analysis, it is only for what is plotted, btw. You can look at the actual distribution of errors, etc when you use plot_trajectories

And, yes, the idea is to build robust networks so you only run analyze_videos I basically never re-train my networks. We run TBs of videos through this pipeline, and describe how to scale-up the analysis too (https://github.com/AlexEMG/DLCutils/blob/master/scale_analysis_oversubfolders.py)

The key to to first pick very diverse frames from across the behavior space you want to track. 150 frames that look the same doesn’t add anything to your network. Please see the discussions points in both papers, and on github about this: https://github.com/AlexEMG/DeepLabCut/blob/master/docs/functionDetails.md#c-data-selection