Multiple animals tracking and real-time tracking

two questions about further application of dlc
I’m wondering whether it could track two or more mice simultanously in dlc and whether it could real-timely track animal when it’s recording?

1 Like

The paper below describes a real-time dlc set-up/package

About the multiple animals, I believe the dlc authors have said it is not possible, yet, they are working on it.

For real time tracking, this might be useful: https://www.biorxiv.org/content/10.1101/482349v1. Depending on complexity of what you are live tracking, perhaps yolo might also be worth looking at? :slight_smile: https://pjreddie.com/darknet/yolo/

For multiple animals, it currently only works if animals are very different, e.g:


But I think they are working on it!

For tracking multiple animals there is also the idTracker: http://www.idtracker.es/home

Cheers,
Christopher

1 Like

As of 2.0.9 you can set num_parts=X when you run analyze videos, and this will extract multiple body parts; while this is not our full solution, that will be available soon. Stay tuned.

I just tried running deeplabcut.analyze_videos with num_parts=2 (for two animals) after installing 2.0.9, but I got an error message for an unexpected keyword argument “num_parts”. I also checked deeplabcut.analyze_videos? and no “num_parts” where shown to be a valid argument. Have I misunderstood something?

Check out the code: https://github.com/AlexEMG/DeepLabCut/blob/master/deeplabcut/pose_estimation_tensorflow/predict_videos.py#L168

You need to put an environment variable indlc-models/...yourproject.../test/pose_cfg.yaml that sets the number of outputs. I.e. add


num_outputs: 2
2 Likes

also please check out docs here: https://github.com/AlexEMG/DeepLabCut/blob/master/docs/functionDetails.md#i-novel-video-analysis

Thank you for the reply, playing around with these instructions now :slight_smile:

I did notice this was said in one of the docs “there is no support for plotting or further analysis at this time” as of 2.0.8+. Does this mean that the analyze_video extracts the coordinates for multiple body parts and saves them to the .h5 output file, but I am unable to use that information to plot trajectories or create a labeled video with two animals?

Sorry if this is a super basic question, still in the early days of learning programming!

you will get the .h5 and/or csv file if you like, but we haven’t edited the plotting functions in DLC to plot these, so you would have to do that yourself currently. As this is not really meant to be a solution to multi-animal tracking - there is much more involved in that codebase - that being said, multi-DLC will be released soon.

1 Like

Gotcha! I’ll play around with the coordinates I now have, but looking forward to the release of multi-DLC :slight_smile:

Thanks for the help and for all the work!

1 Like