UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Observer shrtique
Observer
3,335 Views
Registered: ‎08-22-2016

Petalinux VDMA s2mm mm2s + Python1300

Hello, guys!

I have a project which includes Image Sensor OnSemi PYTHON1300 which is connected to ZYNQ7020.

As a start example I used MicroZed.org project FMC-HDMI-CAM + PYTHON-1300-C Vivado HLS Reference Design (http://microzed.org/support/design/6251/51 ). From this project I’ve learned how to work with PYTHON1300 and S2MM VDMA channel under V4L2-framework. And now I need to launch little bit more difficult system:

 

system.png

 

 

 

So, the architecture of the system is very similar to FMC-HDMI-CAM + PYTHON-1300-C Vivado HLS Reference Design. The difference is that my Image Processing module doesn’t have Linux Driver, cuz now it doesn’t need to be configure form OS.

 

 

How to use VideoCAP input when you don’t have sub-dev in Linux?

 

I tried to deceive Linux and put “fake” AXI-Switch IP block (it has V4L2 Driver). I just put it without physical connection to any of the blocks, and made the following DevTree:

For Python1300:

 

&amba {

	axi_vdma_4: axivdma@43000000 {
		compatible = "xlnx,axi-vdma-1.00.a";
		reg = <0x43000000 0x10000>;
		xlnx,flush-fsync = <1>;
		xlnx,num-fstores = <1>;
		#dma-cells = <1>;
		
		dma-s2mmchannel@43000030 {
			compatible = "xlnx,axi-vdma-s2mm-channel";
			interrupt-parent = <&intc>;
			interrupts = <0 52 4>;
			xlnx,datawidth = <0x40>;
		};
	};

	python_spi0: spi@43c10000 {
		compatible = "xlnx,python-spi-3.1";
		reg = <0x43c10000 0x1000>;
		status = "okay";
		clocks = <&clkc 15>;
		clock-names = "ref_clk";
		num-cs = <1>;
		#address-cells = <1>;
		#size-cells = <0>;
		
		python1300_sensor_0: python1300_sensor@0 {
			compatible = "onsemi,python1300-1.00.a";
			reg = <0>;
			spi-max-frequency = <100000>;

			ports {
				#address-cells = <1>;
				#size-cells = <0>;

				port@0 {
					reg = <0>;
					xlnx,video-format = <XVIP_VF_MONO_SENSOR>;
					xlnx,cfa-pattern = "mono";
					xlnx,video-width = <8>;
					python_sensor_source: endpoint {
						remote-endpoint = <&python_rxif_sink>;
					};
				};
			};
		};
	};

	python1300_rxif_0: python1300_rxif@43c20000 {
		compatible = "xlnx,v-python1300-rxif-3.1";
		reg = <0x43c20000 0x10000>;
		clocks = <&clkc 16>;
		
		regv18-gpios = <&gpio0 57 0>;
		regv33-gpios = <&gpio0 55 0>;
		regvpix-gpios = <&gpio0 56 0>;
		resetn-gpios = <&gpio0 59 1>; /* specify unused GPIO */

		ports {
			#address-cells = <1>;
			#size-cells = <0>;

			port@0 {
				reg = <0>;
				xlnx,video-format = <XVIP_VF_MONO_SENSOR>;
				xlnx,cfa-pattern = "mono";
				xlnx,video-width = <8>;
				python_rxif_sink: endpoint {
					remote-endpoint = <&python_sensor_source>;
				};
			};
			
			port@1 {
				reg = <1>;
				xlnx,video-format = <XVIP_VF_MONO_SENSOR>;
				xlnx,cfa-pattern = "mono";
				xlnx,video-width = <8>;
				python_rxif_source: endpoint {
					remote-endpoint = <&vcap_python_in>;
				};
			};
		};
	};


	
	vcap_python {
		compatible = "xlnx,video";
		dmas = <&axi_vdma_4 1>;
		dma-names = "port0";

		ports {
			#address-cells = <1>;
			#size-cells = <0>;

			port@0 {
				reg = <0>;
				direction = "input";
				vpipe-names = "python1300";
				vcap_python_in: endpoint {
					remote-endpoint = <&python_rxif_source>;
				};
			};
		};
	};
	
	
};

And for ImgProc w/ fake AXI-Switch:

 

 

&amba {

	axi_vdma_0: axivdma@43010000 {
		compatible = "xlnx,axi-vdma-1.00.a";
		reg = <0x43010000 0x10000>;

		xlnx,flush-fsync = <1>;
		xlnx,num-fstores = <1>;

		#dma-cells = <1>;
		dma-channel@0x43010000 {
			compatible = "xlnx,axi-vdma-mm2s-channel";
			interrupt-parent = <&intc>;
			interrupts = <0 35 4>;
			clocks = <&clkc 15>;
			clock-names = "axis";
			xlnx,datawidth = <0x40>;
		};
		dma-channel@0x43010030 {
			compatible = "xlnx,axi-vdma-s2mm-channel";
			interrupt-parent = <&intc>;
			interrupts = <0 36 4>;
			clocks = <&clkc 15>;
			clock-names = "axis";
			xlnx,datawidth = <0x40>;
		};
	};


	axis_switch_0: switch@0x43c00000 {
		compatible = "xlnx,v-switch-1.0";
		reg = <0x43c00000 0x10000>;
		clocks = <&clkc 15>;

		#xlnx,inputs = <2>;
		#xlnx,outputs = <1>;

		ports {
			#address-cells = <1>;
			#size-cells = <0>;

			port@0 {
				reg = <0>;
				switch_in0: endpoint {
					remote-endpoint = <&vcap_gamma_out>;
				};
			};
			port@1 {
				reg = <1>;
				
			};

			port@2 {
				reg = <2>;
				switch_out0: endpoint {
					remote-endpoint = <&vcap_gamma_in>;
				};
			};
			
			
		};
	};

	vcap_gamma {
		compatible = "xlnx,video";
		dmas = <&axi_vdma_0 1>, <&axi_vdma_0 0>;
		dma-names = "port0", "port1";

		ports {
			#address-cells = <1>;
			#size-cells = <0>;

			port@0 {
				reg = <0>;
				direction = "input";
				vcap_gamma_in: endpoint {
					remote-endpoint = <&switch_out0>;
				};
			};

			port@1 {
				reg = <1>;
				direction = "output";
				vcap_gamma_out: endpoint {
					remote-endpoint = <&switch_in0>;
				};
			};
		};
	};
};

 

 

  • The second problem is connected with initialization of these two VDMAs under V4L2-framework.

Firstly I’ve just used code example from FMC-HDMI-CAM + PYTHON-1300-C Vivado HLS Reference Design. In this example Python1300 S2MM channel is initialized as DMA, Image Processing MM2S channel as MMAP and ImgProc S2MM as DMA (but I suppose in my situation it could be MMAP too).

According to V4L2 API to share buffers between Python1300 S2MM DMA and Image Processing MM2S MMAP we should use ioctl VIDIOC_EXPBUF (https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/vidioc-expbuf.html ). In API’s example we get file descriptor from MMAP buffer, and assign it to DMA’s buffers, it seems OK. But if we look at the example’s code from Microzed.org, we could see, that they assign exported descriptor from MM2S MMAP buffers to MM2S MMAP buffers again (line 18)! And after that they queue only ImgProc MM2S and S2MM channels, so that I couldn’t see any mention of Python1300 S2MM DMA channel…

Snippet of the code:

 

#define BUFFER_CNT 3

int i;
for(i=0;i<BUFFER_CNT;i++)
{
	info->s2m_stream_handle.video_in.vid_buf[i].index =i ;
	info->s2m_stream_handle.video_post_process_in.vid_buf[i].index=i;
	info->s2m_stream_handle.video_post_process_out.vid_buf[i].index =i;

	/* export buffer for sharing buffer between two v4l2 devices*/
	memset(&eb, 0, sizeof(eb));
	eb.type = info->s2m_stream_handle.video_post_process_in.buf_type;
	eb.index = i;
	ret = ioctl(info->s2m_stream_handle.video_post_process_in.fd, VIDIOC_EXPBUF, &eb);
	ASSERT(ret< 0, "VIDIOC_EXPBUF failed: %s\n", ERRSTR);

	info->s2m_stream_handle.video_post_process_in.vid_buf[i].dbuf_fd =eb.fd;	

	/*Queue buffer for video_post_process_in pipeline*/
	v4l2_queue_buffer(&info->s2m_stream_handle.video_in,						
			& (info->s2m_stream_handle.video_post_process_in.vid_buf[i]));

	/* Queue buffer for video_post_process_out pipeline */
	v4l2_queue_buffer(&info->s2m_stream_handle.video_post_process_out,			
			& (info->s2m_stream_handle.video_post_process_out.vid_buf[i]));
}

void v4l2_queue_buffer(struct v4l2_dev *dev, const struct buffer *buffer)
{
	struct v4l2_buffer buf;
	int ret;

	memset(&buf, 0, sizeof buf);
	buf.type = dev->buf_type;
	buf.index = buffer->index;
	buf.memory = dev->mem_type;
	if(dev->mem_type == V4L2_MEMORY_DMABUF) {
	buf.m.fd = buffer->dbuf_fd;
	}
	ret = ioctl(dev->fd, VIDIOC_QBUF, &buf);
	if (ret < 0)
		ASSERT(ret, "VIDIOC_QBUF(index = %d) failed: %s\n", buffer->index, ERRSTR);
}



 

 

 

  • So, how this approach works? I can’t match it with the API’s description… But somehow… it works, but not all the stages!

There are some results of debugging my system with “strange” application code based on microzed’s example and previously mentioned “fake” dev-tree.

medias_v5.png

 

debug.png

 

As you could see on diagram, there is some data streaming from RAM to ImgProc module (M_AXIS_MM2S), ImgProc module works fine and it streams data to ImgProc S2MM channel (S_AXI_S2MM), but there is no DATA on the path to RAM Memory… (M_AXI_S2MM).

 

 I really hope, that someone have met this problem....

 

Thanks!

0 Kudos
2 Replies
Observer lrocher
Observer
376 Views
Registered: ‎09-28-2017

Re: Petalinux VDMA s2mm mm2s + Python1300

hello

when i try your solution with the vdma and signal processing i have a problem :

it don't want execute ret = ioctl(dev->fd, VIDIOC_REQBUFS, &rqbufs);

and i have a problem when i execute v4l2-compliance

 

Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
fail: ../../../v4l-utils-1.12.3/utils/v4l2-compliance/v4l2-test-buffers.cpp(571): q.has_expbuf(node)
test VIDIOC_EXPBUF: FAIL

0 Kudos
Observer shrtique
Observer
356 Views
Registered: ‎08-22-2016

Re: Petalinux VDMA s2mm mm2s + Python1300

@lrocher

Hi!

I'm off work now. Could you please wait till Monday 01.14? I will check our code structure and consult with my teammate.

And could you provide little bit more information about your problem? I want to understand with which part you have a problem, wether it's about S2MM or MM2S VDMA channel? Your Device tree and code of v4l2 stream initialization could also be useful.

 

Btw, I haven't solve the main problem. I've just put it off for some time. Now  I use only S2MM channel of VDMA  in my projects, all the image processing is being done "on the fly" w/o buffering the whole image. Just accumulate image data enough for "filtering window" and after that process everything in real time.

Something like this: Image 10x10 pixels, LINE_BUFFER SIZE = IMG_WIDTH - KERNEL_SIZE = 10 - 5  = 5

//    DATA         KERNEL_BUFFER 5x5      LINE_BUFFER
// 47, 46, 45 --> [ 44, 43, 42, 41, 40 ] --> [ 39, 38, 37, 36, 35 ] --> *
//                * --> [ 34, 33, 32, 31, 30 ] --> [ 29, 28, 27, 26, 25 ] --> *
//                * --> [ 24, 23, 22, 21, 20 ] --> [ 19, 18, 17, 16, 15 ] --> *
//                * --> [ 14, 13, 12, 11, 10 ] --> [ 09, 08, 07, 06, 05 ] --> *
//                * --> [ 04, 03, 02, 01, 00 ] 

0 Kudos