cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
jeffdaq
Explorer
Explorer
11,687 Views
Registered: ‎11-22-2015

streaming video via VDMA on linux

Hello, 

  I have a ARM based Zynq board with a camera running at 30fps and which provides video to the AXI-to-Stream block which is connected to my VDMA controller.   Controller is configured for 3 framestores and S2MM only and frame sync is tuser from the AXI-stream. I've written a front-end to the xilinx_vmda.c driver so that I can write an app in userspace that uses ioctls to initiate the DMA.  The data buffers are allocated in the kernel and memory mapped by the userspace app but ioctls are used to drive the process.

 

  What I want, is to endlessly stream video data getting a single interrupt for every frame.   Currently, I am using park mode so am only using one framestore and I do not enable frame count.  Surprisingly, I get a interrupt for every frame even though frame count is not enabled, but this is good because this is what I want.   However, when I look at the data is when the problem starts.

 

  The data in memory always has data from two different frames even when I transfer a single frame.  If I am using multiple dma buffers(actual memory buffers) the data is not from consecutive frames.  For example using 3 data buffers, in transfer 4, I get new data and data from transfer 1 in the buffer(and 2,5, 3,6 etc)

  Initially this looked like a cache issue but using the linux function to flush the dma buffer from cache has no effect.  However, I have discovered that putting a 10ms delay(yes 10ms) after the interrupt and before allowing userspace to touch the data, then the data is correct.  I've determined that delay allows the halt bit to be set in the status register which is supposed to mean the transaction is done.  I don't really understand how I can see the halt bit set when in theory there is a frame currently being transfered.

 

 Has anybody seen this behavior or have a theory on what is going on?   Or if you are streaming video like I described can you tell me what your configuration is?

 

thanks, 

jeff

  

Tags (3)
0 Kudos
Reply
7 Replies
bwiec
Xilinx Employee
Xilinx Employee
11,650 Views
Registered: ‎08-02-2011

Hi Jeff,

 

My hunch is that this has to do with the somewhat goofy behavior of the VDMA's interrupt. The s2mm fires its interrupt on fsync events which, for S2MM on tuser mode, is the first active pixel of the frame. Most folks I work with understandably expect that the S2MM interrupt instead fires once all data has been transferred to memory.

 

I'm guessing what's happening is something like:

1) SW enables interrupts and kicks off VDMA to park after 1 frame

2) When tuser asserts, VDMA starts to transfer data to memory and fires its interrupt

3) SW sees the interrupt and starts reading from the buffer before VDMA is done writing its frame

 

and by inserting the wait state, it looks more like:

1) SW enables interrupts and kicks off VDMA to park after 1 frame

2) When tuser asserts, VDMA starts to transfer data to memory and fires its interrupt

3) SW acknowledges the interrupt and sleeps for 10ms which gives VDMA enough time to finish writing its frame

4) After its wait state, SW reads complete frame from the buffer.

 

A quick ILA capture of s2mm_irq timing relative to AXI Lite kickoff and tuser could confirm this.

 

www.xilinx.com
0 Kudos
Reply
bwiec
Xilinx Employee
Xilinx Employee
11,647 Views
Registered: ‎08-02-2011

By the way, the zynq base TRD uses the VDMA in linux with simultaneous SW and HW access to framebuffers. You might find it helpful:
http://www.wiki.xilinx.com/Zynq+Base+TRD+2014.4
www.xilinx.com
0 Kudos
Reply
jeffdaq
Explorer
Explorer
11,618 Views
Registered: ‎11-22-2015

 

 

 Yes, it does seem like something like that is going on.   I will be working with the FPGA engineer to try to instrument the controller.   I did try changing the sync mode from tuser to none/free so that the interrupt comes at the end of the frame but so no real change.

 

 However, how am I supposed to use the interrupt if I have to stick in an abitrary delay to wait for the data?

 

 From the documentation, I should be able to set park to 1, frame_cnt_en = 1 and frame_cnt = 1 and the halt bit should be set at the end of every transaction.   This just seems to hang the xilinx_vdma.c driver.  So I don't see a way to get an interrupt for every frame and know when the transaction has completed without polling???

 

thanks,

jeff

 

0 Kudos
Reply
trimble_pome
Visitor
Visitor
11,111 Views
Registered: ‎11-24-2015

Jeff,

 

I am currently running a setup that matches yours, though I am running bare metal instead of Linux and after fighting cache issues we have this configuration working. What you are describing does sound very much like my system when I was having caching issues. A couple of quick questions

 

1. What Linux cache flush mechanism are you using ? Are you mmapping the video buffers with no-cache flags set or relying on a different mechanism ? In our bare metal setup we initailly tried one of the standalone cache flush calls and found this took way too long. If your linux configuration is executing this same functionality but returning before flush completion this could give you issues and the time delay for this would make it consistant with your problem. We ended up modifing the translation_table.S to control cache configuration for the memory space, but this route might not be so easy under Linux. 

 

2. If you can provide a direct dump of the AXI registers for the VMDA, I can compare with what we can have done and see if there are any major differences. We are running full speed now that cache issues are resolved, so it is possible in bare-metal at least and in this configuration there is not an interrupt latentcy issue.

 

Matt

0 Kudos
Reply
jeffdaq
Explorer
Explorer
11,098 Views
Registered: ‎11-22-2015

 

Hi Matt,

 

  My initial thought was that it was a caching issue but further testing showed that it was not.   I am using cached memory and calling sync_to_cpu but as I discovered we saw the same behaviour with and without flushing.

  After I was able to get chipscope hooked up we got a better understanding of what was going on.  In tuser mode, the interrupt does come when the first pixel is received.  Our calculated readout time was supposed to be 5ms but we actually measured 10ms on the trace.   This explained why the 10ms delay I put after each interrupt was working.

  Obviously this not a solid, working solution so I rebuilt the fpga to be in free run mode because the interrupt is supposed to come after VSIZE bytes have been "seen" .   The data was totally corrupted in this mode so the fpga guy is going to have a look at this.  It's unclear at this time when this interrupt actually occurs and if the transaction is actually complete.

 

 Which mode to you have VDMA configured in, tuser or free run?  Are you checking the halt bit after the interrupt?  The halt bit is what is supposed to indicate the transcation is complete but I can't get a halt bit set for every frame so I don't know when all the data has been flushed from the controllers fifo.

 

thanks,

jeff

0 Kudos
Reply
bwiec
Xilinx Employee
Xilinx Employee
10,943 Views
Registered: ‎08-02-2011

Hey guys,

Since this seems to come up fairly frequently, I dug into it a bit further. Here you go:
http://www.xilinx.com/support/answers/66493.html
www.xilinx.com
0 Kudos
Reply
jeffdaq
Explorer
Explorer
10,871 Views
Registered: ‎11-22-2015

 

Hi bwiec,

 

  Thanks for the link, it is useful.  However, I don't find this very satisfying becuase it will require custom logic to create an interrupt that SW can see and also it says this signal is asserted when the number of bytes is received at the streaming interface.    In other words, the SW still doesnt' know when it is really safe to touch the data in memory so that means we will need more custom logic to ensure all the data has been written.   Disappointing that such basic functionality is not present and in hindsight shows that VDMA is not the right choice for transferring video data in a latency sensative project.

 

thanks,

jeff

 

0 Kudos
Reply