Some very basic questions

Hey you all awesome people,
New to DLC here. And trained my first model for tracing the snout, tail, and ears of the mouse and used it for video analysis. It was incredible. However, I have a couple of questions and was wondering if someone could answer them (warning: some of them might sound stupid):

  1. With Google Colab for deeplabcut.train_network(), I know that it saves the model once I hit the stop button. But if I hit the start button, does it automatically resume from where it had stopped? For example, let us say that I stop at 200k iterations. And then hit play again, does the model start from the 200k point or from 0? Is there a way to make it start from 200k?

  2. Where is the DLC predicted model stored? With Keras, one can export the model to disk with model.save(). and reuse it with load_model(). Is there a way to similarly export our DLC model? Can we directly use Keras to load this “saved” model to do predictions without having to use the DLC.analyze_videos() method?

  3. Is there a way to use my “trained” model on live stream data, e.g. a webcam? For example, with openCV if I capture frames from my webcam, can I individually feed each of them to DLC.analyze_videos()? If yes, can I make DLC.analyze_videos() return me the predicted frame instead of storing it somewhere on the disk as a video? If not, is there a text file that stores the x,y locations of the predictions?

Hey Zaigham

welcome to the forum!

  1. The latest snapshot is not automatically loaded, but you can pick it by setting the init_weights variable, see e.g. Stage VII: training the network in https://www.nature.com/articles/s41596-019-0176-0 and e.g. here: Restarting training from a particular snapshot in Colab
  2. The models are stored in the /dlc-models/yourproject/train folder. In fact the “snapshot” path you indicate is where the weights are. DLC is written in TensorFlow, so I suppose you cannot use it in Keras. Of course you can load it in TF without using DLC.analyze_videos() - it’s probably best to check out the code to see how to
  3. Yes there is, check e.g. check this out: https://devtalk.nvidia.com/default/topic/1067257/jetson-projects/realtime-pupil-detection-with-jetson-nano-and-deeplabcut/ and this DeepLabCut online tracking
    a native version will also be released soon!
1 Like