Cross-validation and network segmentation

Helllo all,

Thanks a lot for developing such great tool for analyzing animal behaviors!

I am trying to use “cross-validation” to train the datasets. Since I am new to Python and programming, could anyone please provide more details about how to perform cross-validation in DLC, especially for commends or codes?

Also, we also want to use the strategy that mentioned in the paper (Mathis et al, 2018) (“we systematically varied the size of the training set and trained 30 distinct networks (three splits for 50% and 80% training set size; six splits for 1%, 5%, 10% and 20% training set fraction.”). My question is should we create 30 new projects for each network or we could run these 30 networks within one project? If in one project, how to access each network? Any details for such operation?

Thanks so much for any comment or reply in advance!

Jinxin

Yes, you need to create 40 distinct networks. Just add the fractions you want to the config.yaml file:

TrainingFraction:
- 0.80
- 0.5
- 0.2
etc.

Then create 3 splits with standard “deeplabcut.create_training_set”. You can access the networks by passing the shuffle & trainingset index, see: https://github.com/AlexEMG/DeepLabCut/blob/1632f44d13502cfc8ca4300606b9811581562a65/deeplabcut/pose_estimation_tensorflow/training.py#L14

Hope that helps!

Cheers,
Alexander

Thank you so much for reply!