Changing Scorer/Experimenter in h5 file

I have two projects (project 1 & 2) created with two different experimenters. I would like to have a new project (project 3) having the labeled data from both project to train a new network with the data from both projects. When I apply the check_labels function in project 3 it only works for the data copied from project 1 which has the same experimenter as project 3. For the data from project 2 it says that it does not appear to have labeled data. Changing the name of the h5 file to CollectedData_‘experimenter_as_in_project3’ gives the error:

Traceback (most recent call last):

File “C:\ProgramData\Anaconda3\envs\DLC-GPU\lib\site-packages\pandas\core\indexes\base.py”, line 2646, in get_loc
return self._engine.get_loc(key)

File “pandas_libs\index.pyx”, line 111, in pandas._libs.index.IndexEngine.get_loc

File “pandas_libs\index.pyx”, line 138, in pandas._libs.index.IndexEngine.get_loc

File “pandas_libs\hashtable_class_helper.pxi”, line 1618, in pandas._libs.hashtable.PyObjectHashTable.get_item

File “pandas_libs\hashtable_class_helper.pxi”, line 1626, in pandas._libs.hashtable.PyObjectHashTable.get_item

KeyError: ‘lha’

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File “”, line 1, in
dlc.check_labels(config_path)

File “C:\ProgramData\Anaconda3\envs\DLC-GPU\lib\site-packages\deeplabcut\generate_training_dataset\trainingsetmanipulation.py”, line 300, in check_labels
MakeLabeledPlots(folder,DataCombined,cfg,Labels,Colorscheme,cc,scale)

File “C:\ProgramData\Anaconda3\envs\DLC-GPU\lib\site-packages\deeplabcut\generate_training_dataset\trainingsetmanipulation.py”, line 329, in MakeLabeledPlots
DataCombined[cfg[‘scorer’]][bp][‘x’].values[index],

File “C:\ProgramData\Anaconda3\envs\DLC-GPU\lib\site-packages\pandas\core\frame.py”, line 2799, in getitem
return self._getitem_multilevel(key)

File “C:\ProgramData\Anaconda3\envs\DLC-GPU\lib\site-packages\pandas\core\frame.py”, line 2849, in _getitem_multilevel
loc = self.columns.get_loc(key)

File “C:\ProgramData\Anaconda3\envs\DLC-GPU\lib\site-packages\pandas\core\indexes\multi.py”, line 2662, in get_loc
loc = self._get_level_indexer(key, level=0)

File “C:\ProgramData\Anaconda3\envs\DLC-GPU\lib\site-packages\pandas\core\indexes\multi.py”, line 2929, in _get_level_indexer
code = self._get_loc_single_level_index(level_index, key)

File “C:\ProgramData\Anaconda3\envs\DLC-GPU\lib\site-packages\pandas\core\indexes\multi.py”, line 2598, in _get_loc_single_level_index
return level_index.get_loc(key)

File “C:\ProgramData\Anaconda3\envs\DLC-GPU\lib\site-packages\pandas\core\indexes\base.py”, line 2648, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))

File “pandas_libs\index.pyx”, line 111, in pandas._libs.index.IndexEngine.get_loc

File “pandas_libs\index.pyx”, line 138, in pandas._libs.index.IndexEngine.get_loc

File “pandas_libs\hashtable_class_helper.pxi”, line 1618, in pandas._libs.hashtable.PyObjectHashTable.get_item

File “pandas_libs\hashtable_class_helper.pxi”, line 1626, in pandas._libs.hashtable.PyObjectHashTable.get_item

KeyError: ‘lha’

lha is the name of the experimenter in project 1 & 3.

I assume the experimenter in the labeled data from project 2 must be set to the same name as in project 1 & 3. Doing so in the csv file was no problem. Is there a way to also do it in the h5 file? Or is there any other way to combine labeled data from projects with different experimenters

I would highly appreciate your help

HI,

You need to edit the inside of the h5 file to change the scorer name. You can edit the csv file, and use this function:

deeplabcut.convertcsv2h5

It is fully detailed here:

also, since I was doing this today anyhow, here is an example.

i.e. file was originally: CollectedData_user1.csv CollectedData_user1.h5.

First, manually rename files to be the user name of the project, here “mwm”

image

then run the following to edit the csv/h5 internally:

image

then you can run: deeplabcut.merge_datasets

Thank you. It worked perfectly!

1 Like