Plotting error in refine_labels(config_path) / evaluate_network(config_path)

deeplabcut

#1

After successfully training the network, I used the .evaluate_network(config_path) command and I got an error in the plotting of the images. Therefore, I could only see the evaluation numerically (Below is the error message in the plotting), which seem to have very good error values (0.7pixels). I tried to use .check_labels(config_path), but it couldn’t find the labels files in the GUI.

Done and results stored for snapshot:  snapshot-1030000
Results for 1030000  training iterations: 95 1 train error: 0.7 pixels. Test error: 1.36  pixels.
With pcutoff of 0.1  train error: 0.7 pixels. Test error: 1.36 pixels
Thereby, the errors are given by the average distances between the labels by DLC and the scorer.
Plotting...
Could not load matplotlib icon: can't use "pyimage901" as iconphoto: not a photo image
Could not load matplotlib icon: can't use "pyimage919" as iconphoto: not a photo image
.
. (list continues for another 50 pictures…)
.
Could not load matplotlib icon: can't use "pyimage1765" as iconphoto: not a photo image
The network is evaluated and the results are stored in the subdirectory 'evaluation_results'.

Since error values seemed fine, I then proceeded to analyse the video and create a labelled video with the recognized patterns. Here I realized that 1 of the 5 was placed in an entire different region than it should during the whole video. So I tried to redefine that wrong label using .refine_labels(config_path), but the same error of ‘could not load pyimage’ pupped up and couldn’t go further.

So now I can’t fix the wrong label to re-train the network.
Do you have any idea of what might the problem be?


#2

if you want the evaluation images, you must do deeplabcut.evaluate_network(configpath, plotting=True) The warning is fine; scroll through the demo notebooks that tell you what errors are ignorable warnings. i.e: https://github.com/AlexEMG/DeepLabCut/blob/master/examples/Demo_labeledexample_MouseReaching.ipynb

can you send a screenshot of the refine_labels? (also, I assume you extracted outlier frames first?) If you original labels are off, then you need to use the label_frames GUI to adjust those too. Also, perhaps your plotting just low confidence points? I.e. change the pcutoff to >0.4


#3

Hi Mathis,
Thanks for your response.
I wasn’t extracting outlier frames before the refining [Hence the reason I didn’t see the machinelabels-iter0.h5 file for the refining]. I thought I could use refine_frames alone to correct a misplaced label along the whole video (which was not an outlier), but now I realized this function can only be used to refine outliers.
Sorry for my misunderstanding and thanks for clarifying it.

But then, if a label remains in the same wrong position all the time, the only way to solve it is to increase the data set and train the network again hoping for the best, right?


#4

if the label was off, I would check your original images first and be sure there are no labeling errors, as even one can make a big difference!