How to re-project triangulated tracked pts into 2D?


I triangulated the tracked pts from each camera, and now would like to re-project the 3D tracked pts into the image plane of one of the cameras.
I’ve looked both into opencv.projectPoints and your I also inspected the stereo_params.pickle and the intrinsic params of each camera.

I’m debating what are the DLC that I should feed into opencv.projectPoints;
I used their rvec=tvec=(0,0,0) option:

imgpoints_proj_EE, _ = cv.projectPoints(arr_EE, (0,0,0), (0,0,0), \
                                     stereo_dict['camera-1-camera-2']['cameraMatrix1'],  \

(also tried to use np.zeros((1,5)) as the distortion coeffs but it doesn’t help.
It seems that the tracked points are more or less in the right place, but slightly shifted.
In your I saw that for the re-projection error, you used the rvecs and tvecs computed from opencv.calibrateCamera, but that DLC doesn’t save these. Do I need the rotation and translation vectors to re-project? what would you suggest?

For more details see my question in the opencv forum: