Hello, I hope this is the right place to post this.
I am currently working with a program which attempts to predict intermediate frames from a source movie to increase temporal resolution. This program works rather well and the inserted frames seem accurate to the human eye, but I am experiencing one major problem. These predictions are created by “warping and linearly fusing” two key frames. This means that all of the points which decrease in brightness between the key frames decrease at the same time instead of decreasing at different times over the course of the time between the Key frames. This leads to some bizarre measurements when I look at intensity slope, which I would like to fix. I know what the intensity slope should be at all points in the frames between the key frames, but I’m unsure of how I can apply this to my existing predictions.
I thought of trying to use a U-net image restoration model, such as CBSDeep’s package for a CARE model, but given that CARE models work primarily to denoise an image with poor spatial resolution, and I need a model which predicts the correct intensity for a spatially well defined object over the course of several frames, I’m unsure of how well this would work out.
Does anyone have any suggestions for programs which could help, models which I could train to deal with this problem, or Architectures which are best equipped to deal with this kind of issue if I have to create my own solution?
thanks and best regards