Issues with 3D Deeplabcut triangulation

System- windows
Deeplabcut version = 2.1.10.2

Hi, I’ve had the following error when running the triangulation step of 3Deeplabcut, does anyone know what might be causing it? Thanks in advance!

In [1]: import deeplabcut

In [2]: config_path3d='D:\SinglePelletReach3D-Alex-2021-03-05-3d\config.yaml'

In [3]: deeplabcut.calibrate_cameras(config_path3d, cbrow=7, cbcol=5, calibrate=False, alpha=0.9)
Corners extracted! You may check for the extracted corners in the directory D:\SinglePelletReach3D-Alex-2021-03-05-3d\corners and remove the pair of images where the corners are incorrectly detected. If all the corners are detected correctly with right order, then re-run the same function and use the flag ``calibrate=True``, to calbrate the camera.

In [4]: deeplabcut.calibrate_cameras(config_path3d, cbrow=7, cbcol=5, calibrate=False, alpha=0.9)
Corners extracted! You may check for the extracted corners in the directory D:\SinglePelletReach3D-Alex-2021-03-05-3d\corners and remove the pair of images where the corners are incorrectly detected. If all the corners are detected correctly with right order, then re-run the same function and use the flag ``calibrate=True``, to calbrate the camera.

In [5]: deeplabcut.calibrate_cameras(config_path3d, cbrow=7, cbcol=5, calibrate=True, alpha=0.9)
Saving intrinsic camera calibration matrices for camera-1 as a pickle file in D:\SinglePelletReach3D-Alex-2021-03-05-3d\camera_matrix
Mean re-projection error for camera-1 images: 0.134 pixels
Saving intrinsic camera calibration matrices for camera-2 as a pickle file in D:\SinglePelletReach3D-Alex-2021-03-05-3d\camera_matrix
Mean re-projection error for camera-2 images: 0.095 pixels
Computing stereo calibration for
Saving the stereo parameters for every pair of cameras as a pickle file in D:\SinglePelletReach3D-Alex-2021-03-05-3d\camera_matrix
Camera calibration done! Use the function ``check_undistortion`` to check the check the calibration

In [6]: deeplabcut.check_undistortion(config_path3d, cbrow=7, cbcol=5)
All images are undistorted and stored in D:\SinglePelletReach3D-Alex-2021-03-05-3d\undistortion
Use the function ``triangulate`` to undistort the dataframes and compute the triangulation

In [7]: deeplabcut.triangulate(config_path3d, ['D:\DLC\FrontViewA1-Alex-2020-07-07\videos\YR29-A3-26-02-21-camera-1.avi','D:\DLC\SPR-SideView-Alex-2021-01-25\videos\YR29-A3-26-02-21-1-camera-2.avi'], filterpred
   ...: ictions=True)
List of pairs: ['D:\\DLC\\FrontViewA1-Alex-2020-07-07\x0bideos\\YR29-A3-26-02-21-camera-1.avi', 'D:\\DLC\\SPR-SideView-Alex-2021-01-25\x0bideos\\YR29-A3-26-02-21-1-camera-2.avi']
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-7-16c9fde66061> in <module>
----> 1 deeplabcut.triangulate(config_path3d, ['D:\DLC\FrontViewA1-Alex-2020-07-07\videos\YR29-A3-26-02-21-camera-1.avi','D:\DLC\SPR-SideView-Alex-2021-01-25\videos\YR29-A3-26-02-21-1-camera-2.avi'], filterpredictions=True)

D:\Programs\Anaconda\envs\DLC-GPU\lib\site-packages\deeplabcut\pose_estimation_3d\triangulation.py in triangulate(config, video_path, videotype, filterpredictions, filtertype, gputouse, destfolder, save_as_csv)
    133         dataname = []
    134         for j in range(len(video_list[i])):  # looping over cameras
--> 135             if cam_names[j] in video_list[i][j]:
    136                 print(
    137                     "Analyzing video %s using %s"

D:\Programs\Anaconda\envs\DLC-GPU\lib\site-packages\ruamel\yaml\compat.py in __getitem__(self, index)
    271         # type: (Any) -> Any
    272         if not isinstance(index, slice):
--> 273             return self.__getsingleitem__(index)
    274         return type(self)([self[i] for i in range(*index.indices(len(self)))])  # type: ignore
    275

D:\Programs\Anaconda\envs\DLC-GPU\lib\site-packages\ruamel\yaml\comments.py in __getsingleitem__(self, idx)
    386     def __getsingleitem__(self, idx):
    387         # type: (Any) -> Any
--> 388         return list.__getitem__(self, idx)
    389
    390     def __setsingleitem__(self, idx, value):

IndexError: list index out of range

for this, you just pass the folder, not the videos themselves, see here! DeepLabCut/Overviewof3D.md at df95fab35b7ad7ec090e833b613135091e000b16 · DeepLabCut/DeepLabCut · GitHub

deeplabcut.triangulate(config_path3d, '/yourcomputer/fullpath/videofolder', filterpredictions=True/False)

According to the docs, an alternative is to give a list of pairs of videos.
The following should work too:
deeplabcut.triangulate(config_path3d, [['D:\DLC\FrontViewA1-Alex-2020-07-07\videos\YR29-A3-26-02-21-camera-1.avi','D:\DLC\SPR-SideView-Alex-2021-01-25\videos\YR29-A3-26-02-21-1-camera-2.avi']], filterpredictions=True)

1 Like

Thank you, I’ve managed to get a little further this time but have now got the following error. I’ve seen it before when using DLC 2D but I’m not sure what it means:


In [31]: deeplabcut.triangulate(config_path3d, 'D:\SinglePelletReach3D-Alex-2021-03-05-3d\Videos', filt
    ...: erpredictions=True, videotype='avi')
List of pairs: [['D:\\SinglePelletReach3D-Alex-2021-03-05-3d\\Videos\\YR29camera-1.avi', 'D:\\SinglePelletReach3D-Alex-2021-03-05-3d\\Videos\\YR29camera-2.avi']]
Analyzing video D:\SinglePelletReach3D-Alex-2021-03-05-3d\Videos\YR29camera-1.avi using config_file_camera-1
Using snapshot-300000 for model D:\DLC\FrontViewA1-Alex-2020-07-07\dlc-models\iteration-1\FrontViewA1Jul7-trainset95shuffle1
Initializing ResNet
2021-03-26 14:30:59.465011: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0
2021-03-26 14:30:59.495358: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-03-26 14:30:59.495606: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990]      0
2021-03-26 14:30:59.496082: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0:   N
2021-03-26 14:30:59.527829: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2931 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1650, pci bus id: 0000:01:00.0, compute capability: 7.5)
Starting to analyze %  D:\SinglePelletReach3D-Alex-2021-03-05-3d\Videos\YR29camera-1.avi
D:\SinglePelletReach3D-Alex-2021-03-05-3d\Videos  already exists!
Loading  D:\SinglePelletReach3D-Alex-2021-03-05-3d\Videos\YR29camera-1.avi
Duration of video [s]:  185.04 , recorded with  200.0 fps!
Overall # of frames:  37008  found with (before cropping) frame dimensions:  720 480
Starting to extract posture
  0%|                                                                        | 0/37008 [00:00<?, ?it/s]2021-03-26 14:31:04.218917: E tensorflow/stream_executor/cuda/cuda_dnn.cc:334] Could not create cudnn handle: CUDNN_STATUS_ALLOC_FAILED
2021-03-26 14:31:04.219152: E tensorflow/stream_executor/cuda/cuda_dnn.cc:334] Could not create cudnn handle: CUDNN_STATUS_ALLOC_FAILED
The videos are analyzed. Now your research can truly start!
 You can create labeled videos with 'create_labeled_video'
If the tracking is not satisfactory for some videos, consider expanding the training set. You can use the function 'extract_outlier_frames' to extract a few representative outlier frames.
D:\SinglePelletReach3D-Alex-2021-03-05-3d\Videos YR29camera-1 DLC_resnet_50_FrontViewA1Jul7shuffle1_300000
  1%|▌                                                            | 370/37008 [00:01<02:06, 290.77it/s]
Filtering with median model D:\SinglePelletReach3D-Alex-2021-03-05-3d\Videos\YR29camera-1.avi
No unfiltered data file found in D:\SinglePelletReach3D-Alex-2021-03-05-3d\Videos for video YR29camera-1 and scorer DLC_resnet_50_FrontViewA1Jul7shuffle1_300000.
Analyzing video D:\SinglePelletReach3D-Alex-2021-03-05-3d\Videos\YR29camera-2.avi using config_file_camera-2
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-31-c55d90e3d385> in <module>
----> 1 deeplabcut.triangulate(config_path3d, 'D:\SinglePelletReach3D-Alex-2021-03-05-3d\Videos', filterpredictions=True, videotype='avi')

D:\Programs\Anaconda\envs\DLC-GPU\lib\site-packages\deeplabcut\pose_estimation_3d\triangulation.py in triangulate(config, video_path, videotype, filterpredictions, filtertype, gputouse, destfolder, save_as_csv)
    271                         trainingsetindex=trainingsetindex,
    272                         gputouse=gputouse,
--> 273                         destfolder=destfolder,
    274                     )
    275                     scorer_name[cam_names[j]] = DLCscorer

D:\Programs\Anaconda\envs\DLC-GPU\lib\site-packages\deeplabcut\pose_estimation_tensorflow\predict_videos.py in analyze_videos(config, videos, videotype, shuffle, trainingsetindex, gputouse, save_as_csv, destfolder, batchsize, cropping, get_nframesfrommetadata, TFGPUinference, dynamic, modelprefix, c_engine, robust_nframes)
    208     Snapshots = Snapshots[increasing_indices]
    209
--> 210     print("Using %s" % Snapshots[snapshotindex], "for model", modelfolder)
    211
    212     ##################################################

IndexError: index -1 is out of bounds for axis 0 with size 0

In [32]: