DLC 2.2b6 Suggestions for improving training on multi animal different sizes datasets

Hi everyone

Our lab managed to train DLC 2.26b on a 50 labeled frame dataset (1024 x 1280 images). The frames contain 4 animals of slightly different sizes and each animal was uniquely labeled in each frame with unique identifiers as they had markers on their back. We only trained this NN for 2500 iterations, perhaps more would do better.

The Analyze Video step took about 13 minutes (on a Titan XP 12GB) to process a 15second recording (25FPS). That seemed very slow, and we're wondering if there are any suggestions for speeding this step up.

The results for a 15sec video are below. The results shows promise, and we're excited about the possibility of improving this. There are many dropouts (frames without labels) as well as animal ID switching even between 2 sequential frames. If there are any obvious errors we made, we would appreciate any input.

Thanks so much!

could you post the video that is created when you run:

Screen Shot 2020-06-13 at 9.05.04 PM

But 2500 is quite short; I would train to at least 20K in the meantime; with croppedandlabeled frames, this should only take ~4 hours at the most on a titan Xp

Hi

  1. Yes, we retrained using the 50 labeled frames for 50,000 iterations.

  2. The video you asked us to generate using the “Create video for checking detentions” is here:

  1. For comparison, I also reran the video creation with my own video generator that doesn’t threshold the likelihood, and it looks even better:
  1. It seems so much better than before, though there are still dropouts (i.e. periods where an animals is not labeled). I’m guessing that we need >>50 labeled frames as there are unique body angles that it may not have seen. Can you please confirm and let us know if there’s anything else we could do?

  2. The other issue is identity switching including label swapping between the smaller and larger animals. It seems once the identity has switched, it doesn’t go back that well. Any advice on this?

Thank you so much for the help, maDLC has a lot of potential for our projects.

Yes, we have a variable that can handle occlusions; you set what you want filled in, i.e. all gaps, only < 10 frame gaps, etc. See the Project Manager GUI where you can set “0” to be all frames filled in, or 1is none; >1 = # of frames to fill in. see: https://www.youtube.com/watch?v=YSRQT8N2vFE

and yes, I would label >50 frames for good generalization.