cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Participant
Participant
1,064 Views
Registered: ‎01-27-2019

How to implement 4k video in(30Hz) --> vdma0 -> hls ip(10Hz) -> 4k video out(30Hz)?

I have a high speed camera that keeps feeding 4k video into the system. Then I need to do a very time-consuming video processing using a hls based ip. Afterward, I need to produce 4k video out

Because the hls ip is very slow, I need to reduce the frequency of updating to  10Hz.  My problem is how can I do it?

My configuration right now is :

4k video in(30Hz) --> vdma0 -> hls ip(10Hz) -> 4k video out(30Hz)

 

 

Tags (1)
0 Kudos
18 Replies
Highlighted
Teacher
Teacher
1,030 Views
Registered: ‎06-16-2013

Hi @dr.elichan 

 

Would you try the followings ?

 

1. Video streaming flow

4k video in(30Hz) --> vdma0 -> hls ip(10Hz) -> vdma1 -> 4k video out(30Hz)

2. Keep framerate between 4k video in and 4k video out via ex. genlock by yourself.

 

Best regards,

0 Kudos
Highlighted
Participant
Participant
1,017 Views
Registered: ‎01-27-2019

Dear Watari, Can you elaborate on how to do the external genlock?
0 Kudos
Highlighted
Teacher
Teacher
1,012 Views
Registered: ‎06-16-2013

0 Kudos
Highlighted
Participant
Participant
940 Views
Registered: ‎01-27-2019

@watari,

many thanks to your good reference.

I encounter another problem when adding one more VDMA.

I need two m_axi_dc  from Microblaze for each of these two VDMAs. One for video in and the other for video out.  Is it possible?

0 Kudos
Highlighted
Teacher
Teacher
908 Views
Registered: ‎06-16-2013

Hi @dr.elichan 

 

Would you share screen shot when you encountered another issue ?

 

Best regards,

0 Kudos
Highlighted
Participant
Participant
894 Views
Registered: ‎01-27-2019

@watari,

many thanks to your help. I will send you the screenshot ASAP. In the meantime, I would like to elaborate my problem. Zynq does support multiple hp ports but Microblaze (as far as I know) does not support multiple dc ports(equivalent to hp port in Zynq, I guess). Since Microblaze is a soft core, I speculate that a single FPGA board might support multiple Microblazes. But to my surprise, I find no examples on the application of multiple Microblazes.

 

 

0 Kudos
Highlighted
Teacher
Teacher
886 Views
Registered: ‎06-16-2013

Hi @dr.elichan 

 

I'm a little confusing.

Are you facing band width or performance issue via CCI-400 on ZYNQ MPSoC (or over UltraScale+ architecture devices) ?

If so, you might refer VCU DDR4 controller to improve performance issue...

 

Best regards,

0 Kudos
Highlighted
Participant
Participant
867 Views
Registered: ‎01-27-2019

@watari,

the board I am going to use does not have ZYNQ, it only supports Microblaze.

As per your suggestion, I need two vdma and thus I need either 

a) one Microblaze  with two high speed dc ports for these  two vdma respectively. But I can't add the second  dc port into a Microblaze

or 

b) two Microblazes

I am now looking for examples on using two Microblaze in a single fpga board. But I can't find any.

 

0 Kudos
Highlighted
Teacher
Teacher
822 Views
Registered: ‎06-16-2013

Hi @dr.elichan 

 

I'm a little confusing.

Microblaze only controls your IP (two VDMA, your hls ip and so on) via ex axi4 lite.

So, you don't need two MB.

You only implement relevant modules.

Would you try it ?

 

Best regards,

0 Kudos
Highlighted
Participant
Participant
766 Views
Registered: ‎01-27-2019

@watari 

How to use one Microblaze to connect two vdmas? Any examples?

 

0 Kudos
Highlighted
Participant
Participant
763 Views
Registered: ‎01-27-2019

@watari 

How to use one Microblaze to connect two vdmas? Any examples on how to do video passthrough?

0 Kudos
Highlighted
Teacher
Teacher
731 Views
Registered: ‎06-16-2013

Hi @dr.elichan 

 

Sorry for late reply.

 

You need to connect them via AXI4 lite to control their registers between MB and VDMAs.

If you'd like to know how to connect video stream, at least, you need to implement MIG and AXI interconnect IP.

 

Best regards,

 

Highlighted
Participant
Participant
693 Views
Registered: ‎01-27-2019

@watari,

 You hinted that I could use VDMA without miroblaze or zynq, didn't you?

 

By the way, I did find an example that might be able to solve my problem.

https://www.xilinx.com/support/documentation/application_notes/xapp521_XSVI_AXI4.pdf

However,  I could not find the source codes.

 

 

Screenshot_2020-07-29_15-39-07.png
0 Kudos
Highlighted
Explorer
Explorer
662 Views
Registered: ‎07-18-2011

@dr.elichan 

I think you have a bit of a misunderstanding of the role of the MicroBlaze in the VDMA operations.   As @watari  mentioned, you only need one MicroBlaze in your system.

The MicroBlaze is only used to configure the peripherals, such as the VDMA and the VTC, that use AXI4-Lite ports for configuration.   Video does not pass through the MicroBlaze.   VDMA is "Video Direct Memory Access", meaning it is autonomous, as it directly controls video into or out of the DDR memory without any intervention from the MicroBlaze, other than the initial setup of the IP for image size, frame buffer addresses, etc.  After the setup, the VDMA will move memory between the video source and DDR on its own, without consuming MCU overhead.

The way the MicroBlaze connects to multiple peripherals is through an AXI interconnect.   This is a central multiplexer that allows a single MicroBlaze to connect to all the AXI-controlled peripherals in the system.   It is automatically instantiated and connected when you use the "Run Connection Automation" feature in a Vivado block design.   Unique peripheral addresses are also automatically generated during the connection automation process, allowing the user to write code to configure each peripheral using those addresses.

The high-speed memory interface video path is connected between the peripheral and the memory interface generator, or MIG, via an AXI SmartConnect IP block.  This is also automatically instantiated and connected when you use the "Run Connection Automation" feature in a Vivado block design.   This SmartConnect is also a central multiplexer that allows multiple AXI sources to connect and interact with the MIG.    It is this IP that will allow you to have multiple VDMAs that move video to and from the DDR.

The easiest way to see how all this works is to make a small block design.   Start by adding a MIG and configure it for your board or DDR parameters.   Next, add a MicroBlaze and use the Block Automation to confgure it (I recommend connecting the clock to the ui_clock of the MIG prior to running the automation to make sure it uses the correct clock).   Then add a simple UART-lite for debug, and let the connection automation connect it for you.  Don't select the MIG, only the UART.  You will see that Vivado instantiates an AXI-interconnect block and connects it to the MicroBlaze and the UART.  

Next, add a VDMA and let run the connection automation for the S_AXI_LITE VDMA port only, don't select the M_AXI_MM2S or M_AXI_S2MM boxes or the MIG.   You will see that it is also connected to the same AXI-interconnect IP block.   Now you have two peripherals controlled by the same MicroBlaze.   Repeat the process with a second VDMA, and you'll see it is also connected to the AXI-interconnect, allowing a single MicroBlaze to configure all three peripherals.

Then run the connection automation again, and select only the MIG.  This will connect the IC and DC ports of the MicroBlaze to the MIG, and will instantiate an AXI Smart-Connect IP block, allowing a path between the MicroBlaze and the DDR.

Now, in order to connect the VDMA memory ports to the MIG, run the connection automation again, but this time, select the M_AXI_MM2S and M_AXI_S2MM ports for both VDMAs, and click OK.  This process will connect both VDMA IP instances and the MIG to the Smart-Connect.  It is this Smart-Connect multiplexer that provides a path for both VDMA instances to send and receive data from the DDR3 via the MIG IP.

Once you have all your peripherals connected, you can then compile and export the design and start writing MicroBlaze code to control the system.

In your case, you can use a VDMA to send 30Hz video to the DDR, then use the read side of that VDMA to read it out at a 10Hz rate, connect that 10Hz video path to your processing IP block, and send the 10Hz processed video back to the DDR via another VDMA write side, and out via the VDMA read side configured for 30Hz reads.   The VDMA will automatically drop or repeat frames to make up for the 10Hz/30Hz difference.  Obviously, this will result in choppy video, since you are essentially seeing only one out of every three video frames.

If your 30Hz video path is too fast for your VDMA/DDR bandwidth to allow processing of 30Hz frames both in and out as mentioned above, you may be able to improve it by writing only one of every three frames to the DDR, run all your processing at the 10Hz rate, and just do one 10Hz-30Hz frame rate conversion in the output VDMA read side.  

I hope this helps!

Highlighted
Participant
Participant
518 Views
Registered: ‎01-27-2019

@reaiken,

many thanks to your procedures. I did manage to implement 2 vdma with 1 microblaze but I have encountered another problem.

axi-lite's  clock must be slower than the hls ip's clock. It seems to me  that 55MHz is the minimum. If the axi-lite's clock is slower than 55MHz, there will be no video output.  However, I need to set it to as low as 20MHz.

 

Thank in advance for your help.

 

 

 

 

0 Kudos
Highlighted
Teacher
Teacher
509 Views
Registered: ‎06-16-2013

Hi @dr.elichan 

 

>axi-lite's clock must be slower than the hls ip's clock. It seems to me that 55MHz is the minimum. If the axi-lite's clock is slower than 55MHz, there will be no video output. However, I need to set it to as low as 20MHz.

 

It depends on your design and clock tree structure.

So, it's hard to reply this question without details.

If you share them, I will help you.

 

Best regards,

0 Kudos
Highlighted
Participant
Participant
448 Views
Registered: ‎01-27-2019

@watari 

I observed that I could not reduce  the clock over  axi-lite to below 55MHz in my design. But why?

Mere occasionally changing register for triple buffer shouldn't need a clock as high as 55MHz.

Anyway, my design is attached.

Thank again for your hep.

 

 

 

 

0 Kudos
Highlighted
Participant
Participant
440 Views
Registered: ‎01-27-2019

@watari,

The above desgin_1(1).tcl only contains one vdma and is indeed a very simple hdmi passthrough using microblaze. Even so, I still cannot reduce the axi-lite clock to below 55MHz.
0 Kudos