07-19-2021 04:03 AM
Dear Xilinx Community,
we would like to overlay two live video streams comming from two independant cameras with the Xilinx Video Mixer IP.
It is understood that the camera streams need to synchronized before they enter the Video Mixer IP.
Whould it be enough to place a Video Frame Buffer Write IP and a Video Frame Buffer Read IP on one of the camera streams?
Will the Video Frame Buffer Read IP automatically handle the backpressure from the Video Mixer IP?
Thanks in advance and Best Regards
07-19-2021 04:21 AM
Would you share the following to reply your question ?
- Block diagram
- What OS do you want to use ? Linux or baremetal ?
- What do you want to do after synchronized two camera stream ? Would you like to implement LiDAR to recognize 3D object ?
07-19-2021 04:51 AM
its a baremetal application using the TE0712-02-200-2I module from Trenz and a custom carrier board.
Already running system:
Parallel Video Source #1 -> Video to AXI4 -> VPSS (resolution change and yuv to rgb conversion) -> AXI4 to Video -> LCD
The aim is to duplicate the above structure and overlay the streams with the Mixer IP. As a final addon to the Mixer IP a chroma keying (greenscreen) function is planned.
07-19-2021 09:13 AM - edited 07-19-2021 09:15 AM
Both cameras should be synchronized to the output master stream. This can be done using either the VDMA or Video Frame Buffer Read/Write IP, or you could use a VDMA or Video Frame Buffer Write IP for the write side and use the memory interface of the Video Mixer IP instead of the AXI-stream interface to eliminate the need for a read-side VDMA or Video Frame Buffer Read IP.
You may be able to use one camera as the master stream and just synchronize the other camera to save resources, but if that camera output ever stops, you will lose all output and won't be able to cut to black or blue or display an overlay, so it is better to synchronize both cameras.
The VDMA or VFBR will automatically handle the backpressure from the Video Mixer IP. Beware the Video Mixer is a bit finicky about streaming inputs, they must be initialized and started before starting the Video Mixer IP, or the Video Mixer IP will stop and lock up. As long as you obey the rules, streaming interfaces work fine, but I prefer to use the memory interface instead of the streaming interfaces and control the reads and frame sync through software.
With either the Video Frame Buffer IP or the memory interface of the video mixer, you will have to manually control the frame synchronization in software to avoid reading a frame you are currently writing. The VDMA will automatically handle the frame synchronization for you, so that is the easiest implementation.
Some cameras have a frame sync or genlock input that will allow you to send a pulse every frame to synchronize the camera output to the frame pulse. If your camera has this, you can eliminate the need for the VDMA/VFBW/VFBR and DDR.
07-19-2021 11:00 AM
thank you very much for the indepth explanation! The theory seems now to be quite clear. Let's see if we get it implemented. I will share the progress in next few weeks or come back with more questions.