OpenCV comes with an advanced sample implementation which produces great results on still images, however, using this program on every single frame of video streams is unsurprisingly extremely slow. This is the solution which was used for 360° video using Raspberry Pi(s).

This article follows the one mentioned above, where video frames were transformed on the CPU with OpenCV, and introduce a full GPU pipeline.

It should be noted that OpenCV has GPU support for many operations, but enabling it still results in many inefficient copying of data back and forth between CPU and GPU.

Why OpenGL instead of OpenCV?

Aiming for low-latency and real-time video stitching, OpenCV 2D pixel transformation is replaced with a mini OpenGL 3D engine.

It has multiple benefits:

• Camera ISP or hardware video decoder can deliver video frames directly to an OpenGL texture in the GPU (avoiding a copy of the buffer through the CPU)
• GPU has specific hardware acceleration for pixel processing and texture sampling
• Stitched frame is already in the GPU and can be pushed to display with almost no latency
• OpenGL is an open standard and has good support on most embedded targets

#gpu #opencv #opengl