Adding more label data to a pretrained model ( resulting in lower performance )

Hi , I am using the pre trained model ( horse left to right) to analyze the jumps for the horses, moving from left to right. My workflow was as follows

  1. Downloaded the pretrained model
  2. Opened the pretrained model with DLC GUI
  3. Followed instructions to pretrained config_path and used the label tabs to label additional frames and also saved it as .h5 after labelling
  4. Uploaded the modified pre trained model to google collab . ( with extra labelled frames)
  5. Merged the data sets from the labeled video with the command deeplabcut.merge_datasets(path_config_file)
  6. Created a new training set deeplabcut.create_training_dataset(path_config_file)
  7. Trained it again deeplabcut.train_network(path_config_file, shuffle=1, displayiters=1000,saveiters=10000)
  8. After evaluating the network ,it gave an error of about 4 pixels .
  9. Analyzing it on similar sequence , we saw that a lot of the body points were missing . ( compared to the video that was just ran on on the pre-trained model)

I was wondering if this is the correct way adding labels to enhance estimation ?