UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Contributor
Contributor
355 Views
Registered: ‎06-05-2018

Image processing

hello,

i am designing an architecture which captures image from CMOS sensor (SONY MIPI) and store them in DDR.

I've done this part of storing frames in DDR (Data RAW10). and now i am working on how to use demosaic IP to get a colored image. i started by adding VDMA with both ports (MM2S and S2MM). i connected the MM2S to demosaic Slave port and the S2MM to demosaic master. Normally it might work correclty. (it is  not possible to use simple DMA instead of VDMA, right? because the sIMPLE dma doesn't provide yuser signal. am I right?)

After that, i need to display the image on Displayport PS. is there any special configuration for displaying the image using PS DisplayPort?

regards,

mohamad

Tags (1)
0 Kudos
12 Replies
Moderator
Moderator
332 Views
Registered: ‎10-04-2017

Re: Image processing

Hi @mohamadoussayran,

 

For the PS DisplayPort take a look at the TRD for the ZCU102, it implements the PS DisplayPort.

As for DMA vs. VDMA, VDMA is a DMA with added logic to make working with video streams easier. As long as the DMA you choose follows the requirements in UG934, you can use any DMA. However, it will likely be easier to use the VDMA or (Video Frame Buffer) as they will follow UG934 by default.

In the case of the Sensor Demosaic v1.0 (PG286), this is an AXI Stream IP. This means that it performs its image processing on streaming video. Because your image data is currently stored in memory this becomes more difficult. You will need to stream the video data from memory to the Demosaic and then back into memory. This will require a VDMA or Frame Buffer (pg278) to read from memory and then a VDMA or Frame Buffer to write back to memory. (Your VDMA configuration is correct)

Depending on the format of your stream from the CMOS sensor, you could save a step and stream into the Demosaic. The Demosaic's streaming interfaces are defined in UG934. This is what the VDMA and Frame Buffer follow as well.

 

Regards,

Sam

 


Don't forget to reply, kudo, and accept as solution.

Xilinx Video Design Hub
Contributor
Contributor
303 Views
Registered: ‎06-05-2018

Re: Image processing

Hello @samk ,

Thanks for your suggestions. 

About the Vdma and demosaic, i already connect the vdma to the demosaic to check the output data. I see correct data output from the vdma but there is not data output from demosaic. This is strange. 

 

And about the frame buffer it is not possible to use it in my xase because the data are not fomated to rgb or uvuy etc. In memory. 

Any idea? 

Thanks 

 

0 Kudos
Moderator
Moderator
281 Views
Registered: ‎11-09-2015

Re: Image processing

Hi @mohamadoussayran ,

You need to make sure you are starting the demosaic IP. If you want an example, I have used this IP in simulation in Video Beginner Series 11: From Sensor output to RGB image

Hope that helps,


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
Voyager
Voyager
273 Views
Registered: ‎03-28-2016

Re: Image processing

You are correct about the Video DMA and Xilinx Video IP.  The VDMA should properly handle both the Tuser and Tlast signals to conform to the Video over AXI4-Stream protocol.

As for the Displayport PS, it has it's own DMA that you will want to use.

Ted Booth - Tech. Lead FPGA Design Engineer
www.designlinxhs.com
0 Kudos
Contributor
Contributor
239 Views
Registered: ‎06-05-2018

Re: Image processing

Hello @tedbooth @florentw @samk

Thank you all for your help.

As i said i have raw data  in memory which refers for images. To store images in memory i have designed a block. Then i used Vdma and demosaic and i have data oitput from demosaic.

 

I want to do later is to use the displayport on the PS side to capture frames using yavta. In this case do i need the video pipeline (demosaic and video processing (scaler only or/and csc) and frame buffer). Am i right? And about the device tree what should be the input port of demosaic? In the xilinx trd the port is coming from MIPI CSI2 but in my case the vdma to directly connected to the demosaic, so it should work with my case normaly? Also i don't need any driver for the block that captures the frames because the data will be stored in memory before going to thr demosaic and the video processing subsystem.

 

Thank you in advance? 

Regards

0 Kudos
Moderator
Moderator
230 Views
Registered: ‎11-09-2015

Re: Image processing


@mohamadoussayran wrote:

Hello @tedbooth @florentw @samk

Thank you all for your help.

As i said i have raw data  in memory which refers for images. To store images in memory i have designed a block. Then i used Vdma and demosaic and i have data oitput from demosaic.

 

I want to do later is to use the displayport on the PS side to capture frames using yavta. In this case do i need the video pipeline (demosaic and video processing (scaler only or/and csc) and frame buffer). Am i right?

[Florent] - Yes correct. You might not need the csc if you send the data in RGB.

And about the device tree what should be the input port of demosaic? In the xilinx trd the port is coming from MIPI CSI2 but in my case the vdma to directly connected to the demosaic, so it should work with my case normaly?

[Florent] - Yes it should work. However, this use case VDMA or Framer buffer -> demosaic might not be covered by the device tree generator. In that case you will need to add the input port to the VDMA manually in the demosaic node.

Also i don't need any driver for the block that captures the frames because the data will be stored in memory before going to thr demosaic and the video processing subsystem.

 

Thank you in advance? 

Regards


 


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos
Contributor
Contributor
220 Views
Registered: ‎06-05-2018

Re: Image processing

thank you @florentw for your help.

As i said, my data are stored in memory so to create the video capture i need first a VDMA then demosaic to output RGB data, then video processing subsystem then framebuffer writer. in this case , i need to modify the device tree obviously. In the TRD, for each block they have input and output port, like demosaic (input port from MIPI and output port to video processing subsystem, the frame buffer its input port from video processing subsystem and output port to the video capture (VCAP), etc..), the MIPI has output port to demosaic and the input port from the I2C (IMX274 or ov5647 etc..). so in my use case what will be the input port to the demosaic. and if i connect the vdma and i create an output port from vdma to demosaic , what will be the input port for the VDMA. i mean from where should i start the port of the video capture??

Still confused about this case!

thank you in advance,

regards

0 Kudos
Moderator
Moderator
202 Views
Registered: ‎11-09-2015

Re: Image processing

Hi @mohamadoussayran 

I do not think you need a input port for the VDMA if you are using it a read only


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos
Contributor
Contributor
115 Views
Registered: ‎06-05-2018

Re: Image processing

hello @florentw ,

if i want to use the PS displayPort, don't i need to configure the Vdma, demosaic, VPSS and the frame buffer? or they will be probed with the test driver ? if yes, how the VDMA WHICH IS USED ONLY as READ mode will set the correct address of the buffer to read the images ?

and to display the image/video i need to use the the run_video.sh provided  bu xilinx base trd?

regards

0 Kudos
Moderator
Moderator
106 Views
Registered: ‎11-09-2015

Re: Image processing

Hi @mohamadoussayran 

It depends if you are using linux or baremetal.

In linux, the os should care of allocating the correct memory.


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos
Contributor
Contributor
103 Views
Registered: ‎06-05-2018

Re: Image processing

@florentw

I am using linux. I have been images that are stored in ddr PS address 0x810000000 and the size of each buffer is 2073600bytes. So i am confused how the vdma will know all of these infos automatically? The Vdma is just in read mode not in both read and write mode

Or i need to change them in the driver provided by xilinx?

Thank you 

0 Kudos
Moderator
Moderator
64 Views
Registered: ‎11-09-2015

Re: Image processing

Hi @mohamadoussayran 

I am not exactly sure what the run_video.sh is doing. You might want ot check. I guess it is setting a Gstreamer pipe which is done for the TRD. You will probably need to modify this command.

Regards,


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos