Cropping Video issues

Hi all,

I use both GUI and Google Colab for the analysis.
The issues came up to me when I crop the video in GUI on videoedit section.
I have 15 fps and 5 fps videos, and when I crop the video, they both seem to be cropping to 6 fps.
If anyone can teach me what is going on, that will be appreciated.
Also if possible, if I can change the crop setting so that I do not have to change frame number of the video.

Thank you!

1 Like

Hi @Jean11, I cannot reproduce that behavior. I’ve just cropped a .mp4 video using the GUI, and frame rate and number of frames were preserved. How did you read that value of 6 fps? Are your videos of a particular format? :slight_smile:

Hi, sorry for late reply.
I originally use .avi approximately 45 mins, video format with 5fps. I checked this value on windows and GUI as well. (Initial frame # was 13877)
When I opened the csv file, it gives me 16746 frames. (equivalent to 6fps)

I also pasted the command lines in anaconda shell. Hope this helps.

Starting GUI…
Created “C:\Users\Jean\Desktop\homebase\homebase-18017_OHDA-2020-07-18\videos”
Created “C:\Users\Jean\Desktop\homebase\homebase-18017_OHDA-2020-07-18\labeled-data”
Created “C:\Users\Jean\Desktop\homebase\homebase-18017_OHDA-2020-07-18\training-datasets”
Created “C:\Users\Jean\Desktop\homebase\homebase-18017_OHDA-2020-07-18\dlc-models”
Copying the videos
C:\Users\Jean\Desktop\homebase\homebase-18017_OHDA-2020-07-18\videos\18017.mp4
Generated “C:\Users\Jean\Desktop\homebase\homebase-18017_OHDA-2020-07-18\config.yaml”

A new project with name homebase-18017_OHDA-2020-07-18 is created at C:\Users\Jean\Desktop\homebase and a configurable file (config.yaml) is stored there. Change the parameters in this file to adapt to your project’s needs.
Once you have changed the configuration file, use the function ‘extract_frames’ to select frames for labeling.
. [OPTIONAL] Use the function ‘add_new_videos’ to add new videos to your project (at any stage).
Config file read successfully.
Extracting frames based on kmeans …
Kmeans-quantization based extracting of frames from 0.0 seconds to 2791.21 seconds.
Extracting and downsampling… 13877 frames from the video.
13877it [04:03, 57.07it/s]
Kmeans clustering … (this might take a while)
Frames were successfully extracted.

You can now label the frames using the function ‘label_frames’ (if you extracted enough frames for all videos).
Creating images with labels by 18017_OHDA.
They are stored in the following folder: C:\Users\Jean\Desktop\homebase\homebase-18017_OHDA-2020-07-18\labeled-data\18017_labeled.
If all the labels are ok, then use the function ‘create_training_dataset’ to create the training dataset!
Please, select your coordinates (draw from top left to bottom right …)
Cropping and saving to name C:\Users\Jean\Desktop\homebase\homebase-18017_OHDA-2020-07-18\videos\18017cropped.mp4
ffmpeg version 4.2.3 Copyright © 2000-2020 the FFmpeg developers
built with gcc 9.3.1 (GCC) 20200523
configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from ‘C:\Users\Jean\Desktop\homebase\homebase-18017_OHDA-2020-07-18\videos\18017.mp4’:
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
Duration: 00:46:31.21, start: 0.000000, bitrate: 100 kb/s
Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 100 kb/s, 4.97 fps, 14.92 tbr, 12288 tbn, 12 tbc (default)
Metadata:
handler_name : VideoHandler
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 000002012f857e80] using SAR=1/1
[libx264 @ 000002012f857e80] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 000002012f857e80] profile High, level 3.2, 4:2:0, 8-bit
[libx264 @ 000002012f857e80] 264 - core 160 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=6 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to ‘C:\Users\Jean\Desktop\homebase\homebase-18017_OHDA-2020-07-18\videos\18017cropped.mp4’:
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.29.100
Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1068x1044 [SAR 1:1 DAR 89:87], q=-1–1, 6 fps, 12288 tbn, 6 tbc (default)
Metadata:
handler_name : VideoHandler
encoder : Lavc58.54.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame= 1666 fps= 32 q=23.0 size= 5120kB time=00:04:28.83 bitrate= 156.0kbits/s dup=288 drop=0 speframe= 1683 fps= 32 q=23.0 size= 5120kB time=00:04:31.66 bitrate= 154.4kbits/s dup=291 drop=0 speframe= 1703 fps= 32 q=23.0 size= 5376kB time=00:04:35.00 bitrate= 160.1kbits/s dup=294 drop=0 speframe= 1725 fps= 33 q=23.0 size= 5376kB time=00:04:38.66 bitrate= 158.0kbits/s dup=298 drop=0 speframe= 1740 fps= 33 q=23.0 size= 5376kB time=00:04:41.16 bitrate= 156.6kbits/s dup=301 drop=0 speframe= 1755 fps= 32 q=23.0 size= 5376kB time=00:04:43.66 bitrate= 155.3kbits/s dup=303 drop=0 speframe= 1771 fps= 32 q=23.0 size= 5376kB time=00:04:46.33 bitrate= 153.8kbits/s dup=306 drop=0 speframe= 1794 fps= 33 q=23.0 size= 5376kB time=00:04:50.16 bitrate= 151.8kbits/s dup=310 drop=0 speframe= 1805 fps= 32 q=23.0 size= 5632kB time=00:04:52.00 bitrate= 158.0kbits/s dup=312 drop=0 speframe= 1814 fps= 32 q=23.0 size= 5632kB time=00:04:53.50 bitrate= 157.2kbits/s dup=313 drop=0 speframe= 1825 fps= 32 q=23.0 size= 5632kB time=00:04:55.33 bitrate= 156.2kbits/s dup=315 drop=0 speframe= 1847 fps= 32 q=23.0 size= 5632kB time=00:04:59.00 bitrate= 154.3kbits/s dup=319 drop=0 speframe= 1864 fps= 32 q=23.0 size= 5888kB time=00:05:01.83 bitrate= 159.8kbits/s dup=322 drop=0 speframe= 1887 fps= 32 q=23.0 size= 5888kB time=00:05:05.66 bitrate= 157.8kbits/s dup=326 drop=0 speframe= 1907 fps= 32 q=23.0 size= 5888kB time=00:05:09.00 bitrate= 156.1kbits/s dup=329 drop=0 speframe= 1924 fps= 32 q=23.0 size= 5888kB time=00:05:11.83 bitrate= 154.7kbits/s dup=332 drop=0 speframe= 1947 fps= 33 q=23.0 size= 5888kB time=00:05:15.66 bitrate= 152.8kbits/s dup=336 drop=0 speframe= 1969 fps= 33 q=23.0 size= 6144kB time=00:05:19.33 bitrate= 157.6kbits/s dup=340 drop=0 speframe= 1987 fps= 33 q=23.0 size= 6144kB time=00:05:22.33 bitrate= 156.1kbits/s dup=343 drop=0 speframe= 2004 fps= 33 q=23.0 size= 6144kB time=00:05:25.16 bitrate= 154.8kbits/s dup=346 drop=0 speframe= 2022 fps= 33 q=23.0 size= 6144kB time=00:05:28.16 bitrate= 153.4kbits/s dup=349 drop=0 speframe= 2039 fps= 33 q=23.0 size= 6144kB time=00:05:31.00 bitrate= 152.1kbits/s dup=352 drop=0 speframe= 2057 fps= 33 q=23.0 size= 6400kB time=00:05:34.00 bitrate= 157.0kbits/s dup=355 drop=0 speframe= 2065 fps= 32 q=23.0 size= 6400kB time=00:05:35.33 bitrate= 156.3kbits/s dup=356 drop=0 speframe= 2092 fps= 33 q=23.0 size= 6400kB time=00:05:39.83 bitrate= 154.3kbits/s dup=361 drop=0 speframe= 2112 fps= 33 q=23.0 size= 6400kB time=00:05:43.16 bitrate= 152.8kbits/s dup=364 drop=0 speframe= 2129 fps= 33 q=23.0 size= 6400kB time=00:05:46.00 bitrate= 151.5kbits/s dup=367 drop=0 speframe= 2150 fps= 33 q=23.0 size= 6400kB time=00:05:49.50 bitrate= 150.0kbits/s dup=371 drop=0 speframe= 2164 fps= 32 q=23.0 size= 6656kB time=00:05:51.83 bitrate= 155.0kbits/s dup=373 drop=0 speframe= 2180 fps= 32 q=23.0 size= 6656kB time=00:05:54.50 bitrate= 153.8kbits/s dup=376 drop=0 speframe= 2202 fps= 32 q=23.0 size= 6656kB time=00:05:58.16 bitrate= 152.2kbits/s dup=380 drop=0 speframe= 2226 fps= 32 q=23.0 size= 6656kB time=00:06:02.16 bitrate= 150.6kbits/s dup=384 drop=0 speframe= 2247 fps= 32 q=23.0 size= 6656kB time=00:06:05.66 bitrate= 149.1kbits/s dup=387 drop=0 speframe= 2273 fps= 32 q=23.0 size= 6912kB time=00:06:10.00 bitrate= 153.0kbits/s dup=392 drop=0 speframe= 2293 fps= 32 q=23.0 size= 6912kB time=00:06:13.33 bitrate= 151.7kbits/s dup=395 drop=0 speframe= 2316 fps= 33 q=23.0 size= 6912kB time=00:06:17.16 bitrate= 150.1kbits/s dup=399 drop=0 speframe= 2331 fps= 32 q=23.0 size= 6912kB time=00:06:19.66 bitrate= 149.1kbits/s dup=402 drop=0 speframe= 2340 fps= 32 q=23.0 size= 6912kB time=00:06:21.16 bitrate= 148.6kbits/s dup=403 drop=0 speframe= 2360 fps= 32 q=23.0 size= 7168kB time=00:06:24.50 bitrate= 152.7kbits/s dup=407 drop=0 speframe= 2380 fps= 32 q=23.0 size= 7168kB time=00:06:27.83 bitrate= 151.4kbits/s dup=410 drop=0 speframe= 2402 fps= 33 q=23.0 size= 7168kB time=00:06:31.50 bitrate= 150.0kbits/s dup=414 drop=0 speframe= 2421 fps= 33 q=23.0 size= 7168kB time=00:06:34.66 bitrate= 148.8kbits/s dup=417 drop=0 speframe= 2441 fps= 33 q=23.0 size= 7168kB time=00:06:38.00 bitrate= 147.5kbits/s dup=421 drop=0 speframe= 2466 fps= 33 q=23.0 size= 7424kB time=00:06:42.16 bitrate= 151.2kbits/s dup=425 drop=0 speframe= 2488 fps= 33 q=23.0 size= 7424kB time=00:06:45.83 bitrate= 149.9kbits/s dup=429 drop=0 speframe= 2508 fps= 33 q=23.0 size= 7424kB time=00:06:49.16 bitrate= 148.6kbits/s dup=432 drop=0 speframe= 2530 fps= 33 q=23.0 size= 7424kB time=00:06:52.83 bitrate= 147.3kbits/s dup=436 drop=0 speframe= 2547 fps= 33 q=23.0 size= 7424kB time=00:06:55.66 bitrate= 146.3kbits/s dup=439 drop=0 speframe= 2570 fps= 33 q=23.0 size= 7680kB time=00:06:59.50 bitrate= 150.0kbits/s dup=443 drop=0 speframe= 2596 fps= 33 q=23.0 size= 7680kB time=00:07:03.83 bitrate= 148.4kbits/s dup=447 drop=0 speframe= 2622 fps= 33 q=23.0 size= 7680kB time=00:07:08.16 bitrate= 146.9kbits/s dup=452 drop=0 speframe= 2647 fps= 33 q=23.0 size= 7680kB time=00:07:12.33 bitrate= 145.5kbits/s dup=456 drop=0 speframe= 2664 fps= 33 q=23.0 size= 7936kB time=00:07:15.16 bitrate= 149.4kbits/s dup=459 drop=0 speframe= 2678 fps= 33 q=23.0 size= 7936kB time=00:07:17.50 bitrate= 148.6kbits/s dup=461 drop=0 speframe= 2700 fps= 33 q=23.0 size= 7936kB time=00:07:21.16 bitrate= 147.4kbits/s dup=465 drop=0 speframe= 2727 fps= 33 q=23.0 size= 7936kB time=00:07:25.66 bitrate= 145.9kbits/s dup=470 drop=0 speframe= 2749 fps= 33 q=23.0 size= 7936kB time=00:07:29frame= 2775 fps= 33 q=23.0 size= 8192kB time=00:07:33.66 bitrate= 147.9kbits/s dup=478 drop=0 speed=5More than 1000 frames duplicated 15616kB time=00:15:58.16 bitrate= 133.5kbits/s dup=996 drop=0 speed= 6x
frame= 8517 fps= 38 q=23.0 size= 21504kB time=00:23:30.66 bitrate= 124.9kbits/s dup=1461 drop=0 speed=6.34x
(DLC-CPU) C:\Users\Jean>frame= 8518 fps= 17 q=23.0 size= 21504kB time=00:23:30.83 bitrate= 124.9kbits/s dup=1461 drop=frame=16747 fps= 25 q=-1.0 Lsize= 39678kB time=00:46:30.66 bitrate= 116.5kbits/s dup=2870 drop=0 speed=4.09x
video:39481kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.499792%
[libx264 @ 000002012f857e80] frame I:67 Avg QP: 5.66 size: 66254
[libx264 @ 000002012f857e80] frame P:4229 Avg QP:14.07 size: 6559
[libx264 @ 000002012f857e80] frame B:12451 Avg QP:20.33 size: 663
[libx264 @ 000002012f857e80] consecutive B-frames: 0.7% 0.3% 0.4% 98.5%
[libx264 @ 000002012f857e80] mb I I16…4: 17.6% 41.7% 40.7%
[libx264 @ 000002012f857e80] mb P I16…4: 0.4% 1.3% 0.5% P16…4: 8.4% 3.4% 2.5% 0.0% 0.0% skip:83.5%
[libx264 @ 000002012f857e80] mb B I16…4: 0.0% 0.1% 0.0% B16…8: 7.2% 0.5% 0.1% direct: 0.1% skip:92.0% L0:45.9% L1:51.6% BI: 2.5%
[libx264 @ 000002012f857e80] 8x8 transform intra:53.2% inter:47.0%
[libx264 @ 000002012f857e80] coded y,uvDC,uvAC intra: 61.8% 36.5% 26.1% inter: 1.7% 0.9% 0.1%
[libx264 @ 000002012f857e80] i16 v,h,dc,p: 22% 39% 10% 29%
[libx264 @ 000002012f857e80] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 32% 32% 12% 5% 4% 3% 4% 4% 4%
[libx264 @ 000002012f857e80] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 47% 28% 9% 3% 4% 3% 3% 2% 1%
[libx264 @ 000002012f857e80] i8c dc,h,v,p: 74% 11% 12% 3%
[libx264 @ 000002012f857e80] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 000002012f857e80] ref P L0: 72.2% 3.2% 18.8% 5.8%
[libx264 @ 000002012f857e80] ref B L0: 82.9% 15.5% 1.6%
[libx264 @ 000002012f857e80] ref B L1: 95.2% 4.8%
[libx264 @ 000002012f857e80] kb/s:115.87

That is weird. I wonder whether this is due to your original video having dropped frames (fps=4.97). I tried again on some .AVIs, and input’s and output’s frame rates are identical. What if you run x264 --version in the command prompt?

Variable frame rate was indeed part of the answer. FFmpeg by default will produce constant frame rate files (which is inevitable here since cropping requires re-encoding the whole video, in contrast to just cutting the video for example); hence the slightly different fps. Now I could at least reproduce the behavior, I’ll see if there is an elegant way around the issue! :slight_smile:

@Jean11, I’ve just updated the CropVideo()function in that branch https://github.com/DeepLabCut/DeepLabCut/tree/fix_crop_fps. It looks like this fixes the issue; let me know, and I’ll merge it later.

So, as a solution, I converted the video but keep the frame rate as same. (https://video.online-convert.com/convert-to-avi) And this does not give me frame changes during the cropping.
But I will also look into the function you updated and get back to you!