I’m looking for suggestions for speeding the refining portion for a training data set for a complex multi-animal project. I’m sure my questions are based on me knowing just enough about this to get myself into trouble, but not enough to solve my own problems.
Currently, I’m training the project on one video with one animal, 38 bodyparts. Some bodyparts are obscured frequently, others are always visible. I haven’t added a multi animal video (yet).
After analyzing the video, I can create a video with
dlc.create_video_with_all_detections(config, vids, scorername)
That video has all of the markers, though not necessarily in the correct place, as expected.
For single animal projects, I would extract outliers, refine, and merge. If there were points missing that I needed to add but that did not show in the refine GUI (or I deleted points by mistake), I would reload the image in label frames, and save. Then move on to retraining.
With 2.2b6, if I convert to tracklets and open the refine_tracklets GUI, there are problems:
- Some bodyparts are not marked (maybe off screen?) in some frames, so I can’t refine those frames properly. If I change various things in inference_cfg.yaml or max_gaps, then some points show up, but not necessarily all of them. So, again, I can’t properly refine without suggesting a negative image for specific points.
- Yet, some bodyparts are marked and shouldn’t be, or are in a different place that the detection video would suggest. And there is no way to delete them in the refine tracklets window (or is there? Some of the keyboard shortcuts don’t work on my windows workstation. I’ve changed a few and will submit a PR to provide some options that work).
So, I’ve tried the old style of extracting outlier frames. But that can’t find an unfiltered track file immediately after analyzing the video.
My first workaround was to convert to tracklets, refine the tracklets, save without changing anything, then extract outlier frames and refine.
That leaves a couple of questions, some of which might be answered in a future paper from you all?.
Are there specific parameters to change in the config file(s) that will let refine_tracklets show the raw detections for this kind of early-in-the-process training? In that way, the refine tracklets GUI would almost act like a manual extraction of outliers and refining?
Is there a function that would convert the output of analyze_videos (as used in create_video_with_all_detections) to a format usable by extract_outlier_frames? I have tried
convert_raw_tracks_to_h5(on a whim), and it doesn’t accept the full.pickle (KeyError: ‘header’, so obviously not the right format). Passing it a tracklet pickle works, and lets me extract outlier frames. But the labels have the same problem as the refine_tracklets (some are missing completely).
Is there a better way to refine such errors? Am I missing something completely obvious?