Errors in Network Evaluation

I have finished training the DLC Model on 74,000 iterations with ResNet152. On network evaluation, the following error messages has been reported.

Please kindly advise what has gone wrong.



D:\EMX\EMX-David-2019-10-15/evaluation-results/ already exists!
D:\EMX\EMX-David-2019-10-15\evaluation-results\iteration-0\EMXOct15-trainset95shuffle1 already exists!
Running DLC_resnet152_EMXOct15shuffle1_74000 with # of trainingiterations: 74000
Initializing ResNet
INFO:tensorflow:Restoring parameters from D:\EMX\EMX-David-2019-10-15\dlc-models\iteration-0\EMXOct15-trainset95shuffle1\train\snapshot-74000
INFO:tensorflow:Restoring parameters from D:\EMX\EMX-David-2019-10-15\dlc-models\iteration-0\EMXOct15-trainset95shuffle1\train\snapshot-74000
Analyzing data…
180it [01:28, 2.04it/s]
Done and results stored for snapshot: snapshot-74000

KeyError Traceback (most recent call last)
C:\ProgramData\Anaconda3\envs\dlc-windowsGPU\lib\site-packages\deeplabcut\gui\evaluate_network.py in evaluate_network(self, event)
158 plotting = False
159 print(self.bodyparts)
–> 160 deeplabcut.evaluate_network(self.config,shuffle,trainingsetindex,plotting,show_errors=True,comparisonbodyparts=self.bodyparts,gputouse=None)
161
162 def cancel_evaluate_network(self,event):

C:\ProgramData\Anaconda3\envs\dlc-windowsGPU\lib\site-packages\deeplabcut\pose_estimation_tensorflow\evaluate.py in evaluate_network(config, Shuffles, trainingsetindex, plotting, show_errors, comparisonbodyparts, gputouse, rescale)
222 DataCombined = pd.concat([Data.T, DataMachine.T], axis=0).T
223
–> 224 RMSE,RMSEpcutoff = pairwisedistances(DataCombined, cfg[“scorer”], DLCscorer,cfg[“pcutoff”],comparisonbodyparts)
225 testerror = np.nanmean(RMSE.iloc[testIndices].values.flatten())
226 trainerror = np.nanmean(RMSE.iloc[trainIndices].values.flatten())

C:\ProgramData\Anaconda3\envs\dlc-windowsGPU\lib\site-packages\deeplabcut\pose_estimation_tensorflow\evaluate.py in pairwisedistances(DataCombined, scorer1, scorer2, pcutoff, bodyparts)
29 else:
30 Pointwisesquareddistance=(DataCombined[scorer1][bodyparts]-DataCombined[scorer2][bodyparts])**2
—> 31 RMSE=np.sqrt(Pointwisesquareddistance.xs(‘x’,level=1,axis=1)+Pointwisesquareddistance.xs(‘y’,level=1,axis=1)) #Euclidean distance (proportional to RMSE)
32 return RMSE,RMSE[mask]
33

C:\ProgramData\Anaconda3\envs\dlc-windowsGPU\lib\site-packages\pandas\core\generic.py in xs(self, key, axis, level, drop_level)
3715 labels = self._get_axis(axis)
3716 if level is not None:
-> 3717 loc, new_ax = labels.get_loc_level(key, level=level, drop_level=drop_level)
3718
3719 # create the tuple of the indexer

C:\ProgramData\Anaconda3\envs\dlc-windowsGPU\lib\site-packages\pandas\core\indexes\multi.py in get_loc_level(self, key, level, drop_level)
2854 return indexer, maybe_droplevels(indexer, ilevels, drop_level)
2855 else:
-> 2856 indexer = self._get_level_indexer(key, level=level)
2857 return indexer, maybe_droplevels(indexer, [level], drop_level)
2858

C:\ProgramData\Anaconda3\envs\dlc-windowsGPU\lib\site-packages\pandas\core\indexes\multi.py in _get_level_indexer(self, key, level, indexer)
2944 if not locs.any():
2945 # The label is present in self.levels[level] but unused:
-> 2946 raise KeyError(key)
2947 return locs
2948

KeyError: ‘x’


I have a similar error when evaluating network (DLC 2.1.0 with mobilenet). KeyError: ‘x’ appeared and no evaluation result, though analyzing new video with trained network worked fine.

I would think that this error is related to differences in# of body parts in your labeled data vs. the predicted data? Can you verify that those are the same, that prediction h5 files look correct?

@kjinn @szemingcheng - are you using terminal, or Project Manager GUI?

I am using Project Manager GUI.

Alright, I found an error that could potentially throw this error, so I will make an update. It would not affect your network performance at all, it was only an error in computing error-rates on all bodyparts. So, all is save ;). Thanks for your reply!

closed & fixed: https://github.com/AlexEMG/DeepLabCut/issues/454