Non-overlap images when shuffle train set/ test set?

Hi @DeepLabCut,

I have a question regarding how DLC works during creating dataset.
After I labeled about 20% of the total frames, I splitted the dataset (which is the human labeled frames) into 80% training set and 20% testing set, then I used different shuffle numbers, I shuffled the dataset for 5 times and I trained the network for 5 times Then, each time would generate training error and testing error. Eventually, I computed the average training error and testing error of those 5 times. Would that be considered as cross-validation, testing performance? Because I was wondering whether DLC shuffle will generate non-overlap training images/ non-overlap testing images.

Huge thank you for your time,
Mai

Dear Mai,

Yes that’s fine! Of course there are multiple different cross validation schemes: https://en.wikipedia.org/wiki/Cross-validation_(statistics). The in DLC splits by default might have some overlap across the different test sets (it’s related to Repeated random sub-sampling validation). Some pros and cons are discussed there.

Alexander

1 Like

Hi Alex,

Thank you very much! I got the idea now.