Labeling frames with blurry animals

Usage & Issues
#1

Hi all,

the DLC documentation is very clear that you should always only label visible bodyparts in your frames and skip the occluded bodyparts. However, I was wondering what one should do, if the whole animal is blurry. Specifically, I encountered two scenarios here:

  1. You can clearly see where the animal is, but the animal is just a dark blob. In this case, I opted only to label the dark blob with the “bodycenter” label and skipped all other labels.
  2. You can see the shape of the animal, but the exact bodyparts are not visible. This happens when the animal is moving fast and is recorded with low fps. In this case, I opted to label all the bodyparts based on the outline of the animal; for example, I labeled the snout of the animal on the front of the blur and positioned ears behind it at an approximate distance, where they are usually located.

Are these good practices, or is there a better way to do it?

Many thanks for your great support!

P.S.: maybe an idea for the DLC cookbook - the labeling chapter could include these different scenarios and how to act upon encountering them :slight_smile:

#2

You can try to sharpen the videos a bit, but that probably won’t do much. When the animal is blurry you probably can’t see the ears (which would have distinct color when image is sharp) so I wouldn’t label them, snout is more obvious since it’s on the edge of the “blob” anyway (and this is how ResNets get a lot of information - based on edge detection). All in all, if going for bps on edge of the animal, you can try and label them, but the ones inside the “blob” should be ommited.

The blurrines of the video is mostly shutter speed dependent not fps dependent - with high shutter speed (each image is captured super fast) you get less blurrines since animal moved less during the time camera sensor was gathering light. So you can still go for 30fps (to keep file sizes reasonable) and have sharp videos - albeit you need way brighter scene so the image isn’t too dark. My go to shutter speed is 8ms for rats when using BFS-U3-16S2M camera.

2 Likes
#3

I have had reasonably good success labeling blurry animals under certain conditions.

Some of my wife’s data has darkly colored mice moving on a white background from relatively far overhead. The shutter is open for 100 ms (which is way too long) for an fps of 10. These cameras cost $20 each and see at 1080p from pitch black (with IR light) to daylight, and due to their relatively dirt-cheap cost, the power to control the device it pretty limited. When the mouse starts running fast, they blur pretty badly. I labeled nose, tail base and tail, even on blurry images. The system does a fairly good job differentiating between the nose and the tail base. I think this partly works because of the high contrast between the mouse and background.

For my own data, I had some videos where the rat pretty much disappeared as a diffuse grey cloud with the shutter open for around 67 ms per frame (15 fps) using the same cameras as for my wife’s data. I couldn’t use this camera system at all for my data, so I ended up trying a raspberry pi (which is relatively inexpensive and offers lots of control) and a proprietary closed DVR system built into the behavior chambers. It is easy with a raspberry pi to achieve a high frame rate (and correspondingly short exposure time), but the videos are very large. The DVR system doesn’t tell me enough about its settings to know how it works, but after dramatically improving the illumination with inexpensive IR LED strips, the DVR captures acceptably clear video for my purposes and is relatively easy to use after figuring out how to set everything up.

tl;dr – Labeling blurry frames might work, but it depends on the details of your system.