Issues with tracking

Hi,

I’m having issues tracking my data. I want to track the end of an ant’s legs (three example frames uploaded). Sometimes it tracks something in the background but the biggest issue is that it tracks other legs instead. I’ve train the network with 50000 iterations and it plateaus. Training with double the iterations does not improve it, still having half the frames tracked incorrectly. I haven’t change any training parameters. My videos are around 70 fps but some movements are so fast the end of the legs is not completely visible (second frame uploaded). Sometimes they also get occluded (third frame uploaded). Could you help me figure which parameters I could change to improve tracking or is it just not possible to do the way is filmed now?

Thank you!

Hi @SofiaDF, I never tried DeepLabCut, unfortunately, but would be very interested in trying the analysis you want to achieve with existing tool in Aivia software.
Would it be possible for you to share one movie (or 50 frames for instance)?

Thanks,
Patrice
PS: your data would remain private, of course…

Hi @pmascalchi, thank you for your reply! I’ve never tried Aivia but would be great to know if it does a good job.
I’ve tried attaching here some frames in a zip folder but not sure it’s working. If not, is there another way I could send them to you?

Thanks for being willing to have a look!
Sofia

Hi Sofia, how many frames did you label for training? When you say it doesn’t track well, is that from evaluation images or videos? If videos, what pcutoff did you have in your config.yaml file? If you create a video with pcutoff .9 how does it look?

Hi @MWMathis, thank you for your reply! I have been training with around 100 frames labeled by me, covering as many behaviours as I could find. I’ve now tried with pcutoff of 0.9 and it works much much better. All frames for training were labeled correctly and the test videos also have fewer errors. Thank you for the tip!

1 Like

Hi @SofiaDF,

I’m glad you could find a way to improve your tracking. Again, I can’t comment on annotations for DLC, but what I noticed is that in the case of a standard U-net Deep Learning segmentation, increasing a bit the size of the annotated regions (leg tips) could potentially be good…
In my example below, I think I dilated my annotations a bit too much. An intermediate size would be better.
Training set: 27 frames (took 15 minutes)
Applied on 100 frames
Few errors with tracking, so edited with Aivia track editor…

Hi @pmascalchi - so, (1) very cool; (2) Deeplabcut isn’t a U-Net, it’s a much more robust pre-trained specialized version of resnet-50. (3) great to have you on the forum, but it’s for open source free code, and I see you aren’t that… so, if feels a bit wrong for you to come advertise your tool while I am spending hundreds of hours giving away my time and my code :wink:

Hi @MWMathis,
Thanks for your feedback.
The reason I used my main software here is that it’s saving me some time compared to other free software (because I ran into issues and didn’t invest more time in trying to fixing them).
What I show here is an alternative, as a protocol. People can use their preferred software for “standard U-net Deep Learning segmentation”. There is nothing specific about Aivia.

So:

  1. I come from people like you giving some time to create free tools for other people (and I still do). So I clearly understand why it is not a good place to advertise commercial tools here, so this is why I do not show the tool here, only the result that people could get. If you think I’m wrong somewhere, please let me know…
  2. For this application, there is a high chance that your implementation of ResNet-50 is more effective than U-net, and that your application delivers better output, isn’t it?

Thanks,
Patrice

1 Like

Hi @pmascalchi,
Thank you for sharing your results, they do look really great! It’s definitely worth comparing DLC’s results to these, I’m still working on acquiring a license for trying that. And your suggestions are valid for either software so thank you so much for that too.

1 Like

Thanks @SofiaDF. Again I think DLC is a wonderful software (published results on the website look really great) and I hope people will use its power as much as possible…

1 Like