Can I use an maDLC model with DLC-live?

Usage & Issues
#1

Hello,

I’ve exported a model I trained with classical DLC, that was configured to track multiple animals. I’m trying to get better processing speeds so have opted to use DLC-Live. This seems to have worked (far better processing FPS), but having dug a bit deeper, it’s only returning one prediction for a given feature, rather than multiple… is this correct?

Can I use an maDLC model with DLC-Live?

Thank you

Sotiris

#2

Further to this, I’ve tried changing the num_outputs parameter in the pose_cfg.yaml file in the associated exported model directory to 3, from 1, but it is still only outputting one instance of a prediction for a given feature-- any help would be greatly appreciated.

#3

Hmm. Still no joy with this–just wondering: is multi-pose inference possible with DLC-Live in the same way it’s done with normal, deeplabcut? There are hints that it might be as I browse the DLC-Live code but I haven’t gotten it working as yet… has anyone else had much luck?

#4

Hi @Sotiris – right now there is no maDLC within dlc-live. If you are looking for faster processing for maDLC, can you give me a sense of what you are at, and what you want to get to?

(Also, our new maDLC release in Dec. will be much faster, so it might be worth just waiting a few weeks)

#5

Hi @MWMathis, thanks so much for the reply.
We’re currently at around 5-10 FPS using either Google Colab or an RTX 2080 ti, the image size is 800x600.
The higher FPS we’re after is more just to hasten the inference tasks we have, we don’t have a target per se… the higher the better! we have a lot of video footage to get through. Excited for the new Dec release, thanks for the tip.
Sotiris

#6

sorry to hear that! if it’s running that slow, you have a bad skeleton; once our paper is up it will be more clear, but the order, the number, and the type of connections really matter. In general, an optimized skeleton runs at nearly the same speed as normal DLC!

(and of course, use batch inference! :wink: https://github.com/DeepLabCut/DeepLabCut/wiki/What-neural-network-should-I-use%3F#when-should-i-use-a-mobilenet