The researchers at Virginia Tech and Facebook AI have come with an improved technique that allows machine learning to edit videos like never before. With their paper titled, “Flow-edge Guided Video Completion”, they have presented a new flow-based video completion algorithm.
Video completion in this context refers to filling up a pre-recorded video with newly synthesised content. The use cases of a successful video completion algorithm are plenty. From automating VFX workflows to removing watermarks, they can be quite handy.
Previous methods on video completion tasks have used colours among local flow connections between adjacent frames. However, because the motion boundaries form impenetrable barriers, not all missing regions in a video can be reached in this way. So, the researchers in their method, try to address this problem by introducing non-local flow connections to temporally distant frames, which can propagate video content over motion boundaries. The whole experiment is validated on the DAVIS dataset.
So far, the ML techniques could not synthesise sharp flow edges, especially in complex situations. It is challenging to keep the output temporally coherent with respect to the dynamic motion of the camera. In this work, the researchers somehow seem to have managed to perform video completion seamlessly.
#developers corner #computer vision #object detection #vfx #machine-learning
