Implement automatically scaled ROIs based on corner tracking

[Sferland]

Hi everyone! I’m a first time user of deeplabcut and so far I’ve managed to make it through a first round of training using my images. I’d like to have your opinion on how to implement an idea I have about analysis. My goal is to track an animal across three ROIs starting from the time a door opens. The problem is that I have to analyze hundreds of short (around 1min) videos and that the distance between the camera and the box change between videos. I’ve managed to train the network to track the left and right corner of the box (need more training for the door) and this distance between them is known and constant. I thought I could use that distance to calculate a scaling factor so that all the tracked coordinates from the animal are on the same scale and then define ROIs in time_in_each_roi.py based on this scale. Then, if I can track the door, I’d like to only consider coordinates after the door has reached a given height. I’m a beginner python user, so I’m at a loss on how to do this. Should I modify the existing code for video analysis to do this or should I write separate code to use in post processing? Do you have any good resource on how to write/calculate from pandas dataframes?

About ime_in_each_roi.py, is there a way to get a list of visit durations in each ROIs?

Thank you so much for your help and for all of your hard work on this software! If I can get this to work it would help me a great deal.
Samuel

[backyardbiomech (Brandon E. Jackson)]

Hi, Samuel:
Your description of what to do is pretty spot on what I would do (scaling of coordinates, etc). That is all post-analysis processing, and it probably best written in custom python or MATLAB scripts, or even Excel (but it’s best to learn python or MATLAB for this).

There are a lot of free tutorials out there for pandas and numpy, but the deepest resource is probably this, which should help you understand (and then modify) what’s happening in the roi script.

[Sferland]

Good to know I’m in the right direction! So I’ve finally got my hands on some data from my own tracking. I’ve managed to scale my data and set the origin to the left corner using the bit of code in screenshot 1. Setting my ROIs based on the known length of the different chambers, I’ve plotted the centre point and it seems good. These are the unfiltered results, which could explain why it tracked a point outside the apparatus on the bottom right.

I’m looking into modifying time_in_each_roi to output a list of each visit duration to each chamber. When I test the code separately with my results, it outputs what I want (screenshot 2). However, when I run it from the time_in_roi_each_roi_modified code, I get NameError: name ‘visits’ is not defined. I’m not sure why since I create an empty dictionary right before. Any thoughts on this?

Thank you!