Deeplabcut 3D camera setup

Hello! I am trying to set up a Deeplabcut 3D workstation with two cameras. How do you all do this? How do I set things up properly, and set them up so that they both record (one from the front and one from the side) at the same time? Is there a guide somewhere, I cannot find one. I really appreciate all the help you all have given me in the past!
Once the videos are made, will it basically be the same process as the normal deeplabcut with one camera was?
Thank you all so much!

The recording of the video is separate from 3D DeepLabCut (just like with “normal” DLC).
How you set up your cameras, how they are synchronized, how they actual record, depend on your exact experimental set up and the cameras you have. There are a number of 3D kinematics papers and resources that can offer some general guidance.

As far as camera placement, though, you might want to rethink the orthongonal (front and side) placement. Such camera placement can be difficult to correctly calibrate with the grids, difficult to train a single DLC model for (since the views are so different) and can lead to tracking problems since both cameras would need to see any point to allow 3D reconstruction and they would have very different views of an animal. Stereo views, where the cameras are placed near each other provide similar views, good tracking, easy calibration, and single DLC model ability. As a rough guide, the distance between cameras should be about half the distance between the cameras and the filming target.

Also, if at all possible, adding a third cameras will GREATLY improve 3d accuracy and reduce occlusion errors.

Thank you!
Do you have the source of any paper or instructions manual for this type of thing?
I would really appreciate it!
I just dont know how to set everything up right!
Ill get two cameras, set them half as far apart as they are from the target.
How is the 3D grid assembled and how is this all calibrated?

Hello!

Thank you for the very useful information. I am currently working on a 3D DeepLabCut Project and I am having some trouble with the camera calibration, as the corners of my chessboard are not detected in any of my images. I would like to ask if you know if there are any other limitations concerning the camera placement, the angles… the setup in general.

If you have any documentation that you could share, it would be of great help! For example, where does the fact that the 2 cameras have to be placed about half the distance between the cameras and the filming target come from?

Thank you in advance.

Here are some useful links that could help you with the camera calibration:

  1. https://www.nature.com/articles/s41596-019-0176-0

  2. DeepLabCut/Overviewof3D.md at 87e18282c0ea354780cbf16ddfed85869a887664 · DeepLabCut/DeepLabCut · GitHub

  3. DeepLabCut/camera_calibration.py at 87e18282c0ea354780cbf16ddfed85869a887664 · DeepLabCut/DeepLabCut · GitHub

Hope it helps!

There is surprisingly little information in papers, that I know of, regarding suggestions for camera set up for 3D. I’ve long thought about putting together a best-practices guide (we started, but some of that is specific to one type of hardware and our software). This lack of guidance is most likely because each situation varies so much.

The 2:1 distance ratio comes from a paper I co-authored, which describes the software used in the original DLC cheetah 3D project for 3D reconstruction. I’m sure there is more info in some of the computer vision papers that describe the calibrations, but probably not much written for biologists.