cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
mengxiang1326725
Contributor
Contributor
1,423 Views
Registered: ‎11-18-2015

XDMA linux driver(from git) C2H transfor 1MB fail/crash in Stream mode

Hello!

     I'm using the Vivado 2018.3.1 to design my appliticon which was tested on Ubuntu 18.04.1 (Kernel version: 4.15-88, x64OS). My block design diagram is the attachment, whose xdma(4.1) mode is AXI-Stream mode. I  git the master branch code from https://github.com/Xilinx/dma_ip_drivers/tree/master/XDMA/linux-kernel (at 2020-3-17)and follow the solution for issue 39 about signing key. I'm using KC705, and the PCIe is Gen2 x8.

     In my design, the data_generator_0 is controlled by AXI-Lite. The data generator can creat any amount of AXI-Stream data. The work flow is to enable the data generator first, then the host start a C2H transfer from KC705 to host.

Then I test the driver(C2H):

The cmd is './dma_from_device -d /dev/xdma0_c2h_0 -f data/output_file.bin -s transfersize -c 1'

If I transfer 4KB(transfersize ) one time, the driver work correctly.

When I transfer 1MB(transfersize ) one time, the driver return error.

I read the libxdma.h and found  that some parameters would be related to the transfersize: XDMA_TRANSFER_MAX_DESC and CYCLIC_RX_PAGES_MAX.

I changed XDMA_TRANSFER_MAX_DESC to 4096 and CYCLIC_RX_PAGES_MAX to 1024, then I test the transfer of 1MB but the error still exist:

 

/dev/xdma0_c2h_0, R off 0x0, 0xffffffffffffffff != 0x100000.
read file: Unknown error 512

 

The ILA on C2H_0 stream port show that the data was sent but unfinished(always crash when about 64KByte data was sent, the s_axis_c2h_tready0 was low.)

 

The debug info above is in /tools/dma_utils.c, but I'm not sure that it is in read_to_buffer or ethier in write_from_buffer. I think that the return value for rc is -1 make this problem.

 

ssize_t read_to_buffer(char *fname, int fd, char *buffer, uint64_t size,
			uint64_t base)
{
	ssize_t rc;
	uint64_t count = 0;
	char *buf = buffer;
	off_t offset = base;

	while (count < size) {
		uint64_t bytes = size - count;

		if (bytes > RW_MAX_SIZE)
			bytes = RW_MAX_SIZE;

		if (offset) {
			rc = lseek(fd, offset, SEEK_SET);
			if (rc != offset) {
				fprintf(stderr, "%s, seek off 0x%lx != 0x%lx.\n",
					fname, rc, offset);
				perror("seek file");
				return -EIO;
			}
		}

		/* read data from file into memory buffer */
		rc = read(fd, buf, bytes);
		if (rc != bytes) {
			fprintf(stderr, "%s, R off 0x%lx, 0x%lx != 0x%lx.\n",
				fname, count, rc, bytes);
				perror("read file");
			return -EIO;
		}

		count += bytes;
		buf += bytes;
		offset += bytes;
	}	 

	if (count != size) {
		fprintf(stderr, "%s, R failed 0x%lx != 0x%lx.\n",
				fname, count, size);
		return -EIO;
	}
	return count;
}

ssize_t write_from_buffer(char *fname, int fd, char *buffer, uint64_t size,
			uint64_t base)
{
	ssize_t rc;
	uint64_t count = 0;
	char *buf = buffer;
	off_t offset = base;

	while (count < size) {
		uint64_t bytes = size - count;

		if (bytes > RW_MAX_SIZE)
			bytes = RW_MAX_SIZE;

		if (offset) {
			rc = lseek(fd, offset, SEEK_SET);
			if (rc != offset) {
				fprintf(stderr, "%s, seek off 0x%lx != 0x%lx.\n",
					fname, rc, offset);
				perror("seek file");
				return -EIO;
			}
		}

		/* write data to file from memory buffer */
		rc = write(fd, buf, bytes);
		if (rc != bytes) {
			fprintf(stderr, "%s, W off 0x%lx, 0x%lx != 0x%lx.\n",
				fname, offset, rc, bytes);
				perror("write file");
			return -EIO;
		}

		count += bytes;
		buf += bytes;
		offset += bytes;
	}	 

	if (count != size) {
		fprintf(stderr, "%s, R failed 0x%lx != 0x%lx.\n",
				fname, count, size);
		return -EIO;
	}
	return count;
}

I read many questions about xdma driver in Community Forums and have tried some related solutions but it doen't work.

The offset 0x80000000(uint64_t base) added in read_to_buffer() and write_from_buffer() refer to another question (https://forums.xilinx.com/t5/PCIe-and-CPM/debug-the-driver-of-IP-PCIE-with-DMA/m-p/914480#M12599).

Can someone give me some help? 

Thanks.

 

KC705-block-design2020-03-21 220700.png
0 Kudos
6 Replies
mengxiang1326725
Contributor
Contributor
1,327 Views
Registered: ‎11-18-2015

@deepeshm @csattar
Would someone give this issue (https://github.com/Xilinx/dma_ip_drivers/issues/54) some attention?
0 Kudos
lesjokolat
Observer
Observer
1,318 Views
Registered: ‎03-20-2020

Hi found this relating to MSI-x at bottom

 

https://forums.xilinx.com/t5/PCIe-and-CPM/PCIe-XCZU4CG/td-p/1009734

 

Might it help?

0 Kudos
mengxiang1326725
Contributor
Contributor
1,299 Views
Registered: ‎11-18-2015

Thanks! I will have a try.

0 Kudos
mengxiang1326725
Contributor
Contributor
1,279 Views
Registered: ‎11-18-2015

I have tried that as follows:
1. Disable the MSI
2. Enable MSI-X
3. change load_driver.sh (insmod ../xdma/xdma.ko interrupt_mode=0, 0 is MSI-X)
but not work.

0 Kudos
lesjokolat
Observer
Observer
1,191 Views
Registered: ‎03-20-2020

There is a message of a race condition here:
https://forums.aws.amazon.com/thread.jspa?threadID=295643

ref to
timeout_ms

Maybe your review and dev knowledge may get some ideas...

0 Kudos
mengxiang1326725
Contributor
Contributor
1,062 Views
Registered: ‎11-18-2015

Thanks. I think that is about interrupt and the hardware dma operations.

0 Kudos