cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
sa_fpga
Participant
Participant
376 Views
Registered: ‎11-11-2019

Syncronize two cameras for the Video Mixer IP

Dear Xilinx Community,

we would like to overlay two live video streams comming from two independant cameras with the Xilinx Video Mixer IP. 

It is understood that the camera streams need to synchronized before they enter the Video Mixer IP. 

Whould it be enough to place a Video Frame Buffer Write IP and a Video Frame Buffer Read IP on one of the camera streams? 

Will the Video Frame Buffer Read IP automatically handle the backpressure from the Video Mixer IP?

 

Thanks in advance and Best Regards

0 Kudos
4 Replies
watari
Professor
Professor
364 Views
Registered: ‎06-16-2013

Hi @sa_fpga 

 

Would you share the following to reply your question ?

 

- Block diagram

- What OS do you want to use ? Linux or baremetal ?

- What do you want to do after synchronized two camera stream ? Would you like to implement LiDAR to recognize 3D object ?

 

Best regards,

0 Kudos
sa_fpga
Participant
Participant
353 Views
Registered: ‎11-11-2019

Dear Watari,

its a baremetal application using the TE0712-02-200-2I module from Trenz and a custom carrier board.

Already running system:

Parallel Video Source #1 -> Video to AXI4 -> VPSS (resolution change and yuv to rgb conversion) -> AXI4 to Video -> LCD

The aim is to duplicate the above structure and overlay the streams with the Mixer IP. As a final addon to the Mixer IP a chroma keying (greenscreen) function is planned. 

Best Regards

0 Kudos
reaiken
Explorer
Explorer
296 Views
Registered: ‎07-18-2011

@sa_fpga 

Both cameras should be synchronized to the output master stream.   This can be done using either the VDMA or Video Frame Buffer Read/Write IP, or you could use a VDMA or Video Frame Buffer Write IP for the write side and use the memory interface of the Video Mixer IP instead of the AXI-stream interface to eliminate the need for a read-side VDMA or Video Frame Buffer Read IP.   

You may be able to use one camera as the master stream and just synchronize the other camera to save resources, but if that camera output ever stops, you will lose all output and won't be able to cut to black or blue or display an overlay, so it is better to synchronize both cameras.

The VDMA or VFBR will automatically handle the backpressure from the Video Mixer IP.    Beware the Video Mixer is a bit finicky about streaming inputs, they must be initialized and started before starting the Video Mixer IP, or the Video Mixer IP will stop and lock up.   As long as you obey the rules, streaming interfaces work fine, but I prefer to use the memory interface instead of the streaming interfaces and control the reads and frame sync through software.

With either the Video Frame Buffer IP or the memory interface of the video mixer, you will have to manually control the frame synchronization in software to avoid reading a frame you are currently writing.   The VDMA will automatically handle the frame synchronization for you, so that is the easiest implementation.

Some cameras have a frame sync or genlock input that will allow you to send a pulse every frame to synchronize the camera output to the frame pulse.  If your camera has this, you can eliminate the need for the VDMA/VFBW/VFBR and DDR.

sa_fpga
Participant
Participant
272 Views
Registered: ‎11-11-2019

Dear reaiken,

thank you very much for the indepth explanation! The theory seems now to be quite clear. Let's see if we get it implemented. I will share the progress in next few weeks or come back with more questions.

 

Best Regards