[Multi-animal] have markers, NO skeleton/box

Hi guys,

I am trying to draw skeleton for multi-animal project with deeplabcut v2.2b8 under Ubuntu 18.04.

I am kind of experienced with Deeplabcut, and everything goes perfect here:

deeplabcut.create_video_with_all_detections(.....)

I successfully generate video with all body-parts markers.

However, when I try to draw the skeleton as follows:

deeplabcut.convert_detections2tracklets(.....)
deeplabcut.convert_raw_tracks_to_h5(.....)
deeplabcut.filterpredictionsis(.....)
deeplabcut.create_labeled_video(.....)

I get the video totally same with the original video, but no skeleton, even no body-parts markers.

I checked the generated csv file after filterpredictionsis(), the pixel xy results are empty for each picture frame!

deeplabcut.refine_tracklets(...) also shows only same with original video, without any skeleton or marker to refine.

In summary, create_video_with_all_detections() works perfect, but seems that convert_detections2tracklets() fails, even if the printed output seems all right:

deeplabcut.convert_detections2tracklets(path_config_file, videofile_path, videotype=‘mp4’, track_method=tracktype, overwrite=True)
Using snapshot-8800 for model /home/…/dlc-models/iteration-0/maDLC-2020-11-12Nov13-trainset95shuffle1
Processing… /home/…/videos/video.mp4
/home/…/videos already exists!
Analyzing /home/…/videos/VID_xxxxshuffle1_8800.h5
681it [00:01, 638.33it/s]
The tracklets were created. Now you can ‘refine_tracklets’.

I found several unsolved issues with the similar problems as Missing tracking points, No labels in final step, No marker appear on Refine Tracklet.
If you need any further information to solve this problem, please don’t hesitate to let me know.

Thanks in advance.

Hi @stefen, the steps above are crucial and should be taken slowly. Did you run cross-validation after evaluation? This is important prior to converting detections to tracklets; you could try utils.make_labeled_video.create_video_from_pickled_tracks() after convert_detections2tracklets() to make sure animals are nicely assembled. Only then should you move to creating the final h5 file, whose default values of min_tracklet_len and max_gap may need tuning to your particular setup.

Thank you very much for the quick response! @jeylau

you could try utils.make_labeled_video.create_video_from_pickled_tracks() after convert_detections2tracklets()

utils.make_labeled_video.create_video_from_pickled_tracks() will generate a new folder with pictures of each frame and a video, but all of them are same with original video, without any body-parts marker or skeleton.

Then I did the pipeline again trying to draw the skeleton, but still failed as before.

Alright, then we now know that the assembly of the detections into animals is not functioning. Please use evaluate_multianimal_crossvalidate() before convert_detections2tracklets(). This will attempt to select optimal parameters for the stages coming afterwards. I think that post may clarify what those parameters do. If that still does not solve it, please get back to me :slight_smile:

Thank you @jeylau! Very clear declaration, it really helped my problem.

btw, just for making sure, could we randomly label an animal with different individual id in madlc?
e.g. I have many monkeys, one is labelled as individual 5 in last labelled frame, but new frame I label it as individual 7.
I guess it’s not a good idea?

Thank you!

If you can tell them apart, you should label consistently. If you cannot, it does not matter what ID you give them.

Could this have an impact for the tracking?
I have several animals in my training data and i selected frames at random.
Since the animals cannot really be distinguished i labelled them consistently per ID, but ignored the ID consistency.
The detection works very well, however when trying to track them, the skeleton maps body parts across animals. Thanks in advance!