cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Observer
Observer
4,453 Views
Registered: ‎10-13-2017

Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

I want to use the PS PCIe as an Endpoint device on zynqmp Ultrascale+.

Vivado 2017.3 and SDK were used to generate the FSBL.

 

The configuration of PCIe in the PCW Advanced mode is  like this:屏幕快照 2018-05-15 下午12.06.34.png

The Device Port Type was set to Endpoint Device, and the Class Code was set to 0x60400.

I inserted the card into the slot of a PC and used "lspci" to check, I got this:

屏幕快照 2018-05-15 上午11.47.49.png

The Class Code seemed to be 0x000000, not as expected.

I checked the register of PCIE_ATTRIB_Module, the lowest 8bit of register attr_25 at address 0xfd480064 was 0x06,which means RP.

I also check the FSBL code in the SDK ,the lowest 8bit register was indeed set to 0x6 as a RP:

屏幕快照 2018-05-15 下午12.03.08.png

So the configuration in the PCW didn't work?

 

 

Can anyone help this?

Is there an example for the PS PCIe to work as an endpoint ?

Thanks.

   

0 Kudos
1 Solution

Accepted Solutions
Highlighted
Moderator
Moderator
4,731 Views
Registered: ‎02-16-2010

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

Please refer to the document at the link below.
http://www.wiki.xilinx.com/file/detail/PS_PCIe_DMA_v4.pdf

page 5 of the document provides details about "PCIe End Point DMA driver"

 

The link below gives the steps to configure a design to use PS PCIe as the endpoint

http://www.wiki.xilinx.com/PetaLinux+kernel+and+application+build+steps+in+configuring+ZU+MPSoC+board+as+PS+PCIe+End+Point+and+Root+Port

 

------------------------------------------------------------------------------
Don't forget to reply, give kudo and accept as solution
------------------------------------------------------------------------------

View solution in original post

0 Kudos
19 Replies
Highlighted
Moderator
Moderator
4,412 Views
Registered: ‎02-16-2010

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution
Can you provide full log with "lspci -vvv" command?
------------------------------------------------------------------------------
Don't forget to reply, give kudo and accept as solution
------------------------------------------------------------------------------
0 Kudos
Highlighted
Observer
Observer
4,400 Views
Registered: ‎10-13-2017

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

I changed the Class Code from default 0x60004 to 0x70006, then I can see the right class in the PC, the configuration seemed ok.

Here is the log of 'lspci -vvv' when the Class Code is 0x60004:

屏幕快照 2018-05-16 上午11.42.55.png

Here is the log after I set the Class Code to 0x70006:

屏幕快照 2018-05-16 上午11.43.07.png

 

 

Now the driver xilinx_ps_pcie_dma and  xilinx_ps_pcie_dma_client can work on the host PC, and I can find ps_pcie_dmachan*_0 in the dev directory.

So, if I run linux on the endpoint device,  which driver should I use on that? 

0 Kudos
Highlighted
Moderator
Moderator
4,732 Views
Registered: ‎02-16-2010

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

Please refer to the document at the link below.
http://www.wiki.xilinx.com/file/detail/PS_PCIe_DMA_v4.pdf

page 5 of the document provides details about "PCIe End Point DMA driver"

 

The link below gives the steps to configure a design to use PS PCIe as the endpoint

http://www.wiki.xilinx.com/PetaLinux+kernel+and+application+build+steps+in+configuring+ZU+MPSoC+board+as+PS+PCIe+End+Point+and+Root+Port

 

------------------------------------------------------------------------------
Don't forget to reply, give kudo and accept as solution
------------------------------------------------------------------------------

View solution in original post

0 Kudos
Highlighted
Observer
Observer
4,371 Views
Registered: ‎10-13-2017

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

I have read these two docs.

In my opinion, the driver mentioned in these files is the driver on the host for the endpoint device.

In both examples, ZCU102 work as a host, and ZCU112 work as an endpoint.

The driver is used to operate the DMA from pcie bus on the ZCU102. 

I have already port the driver to a PC that does not use Xilinx Linux kernel.

It works and I can find four dma channel nodes ( ps_pcie_dmachan*_0 ) and a pio node in the /dev directory. 

When a dma task is start on the host, I think the endpoint should also do something to coordinate. 

So my question is that, which program should I run on the endpoint device ? 

 

 

On page 5  of  PS_PCIe_DMA_v4.pdf, it says:

屏幕快照 2018-05-17 上午10.18.37.png

I think the program is for bare metal , not for a endpoint device which run linux.

What't more, the softwares mentioned in these two files are all unavailable, I can find nowhere to download.

Is there a way to get these code for me?

0 Kudos
Highlighted
Observer
Observer
4,309 Views
Registered: ‎10-13-2017

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

I have used the PCIe DMA of PS to transfer data from X86 to the zynqmp ultrascale+ device successfully.

The driver in the host is just xilinx_ps_pcie_dma and xilinx_ps_pcie_dma_client. 

The dma_transfer_mode should be MEMORY_MAPPED.

Use pwrite/pread to send and receive data, the parameter offset stands for the address of the destination in the  zynqmp+ DDR.

Observer
Observer
3,631 Views
Registered: ‎05-09-2018

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

This link

http://www.wiki.xilinx.com/file/detail/PS_PCIe_DMA_v3.pdf

does not work anymore.

 

Please can someone send me the pdf file?

 

Best Regards

marco

0 Kudos
Highlighted
Visitor
Visitor
3,607 Views
Registered: ‎09-07-2018

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

I am also really interested on this topic. It seems that much of the data related to PSEP was lost during the migration to Confluence on Xilinx Wiki.

There has been no luck for me on finding the sources for the bare metal application mentioned in here:
https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842008/PetaLinux+kernel+and+application+build+steps+in+configuring+ZU+MPSoC+board+as+PS+PCIe+End+Point+and+Root+Port

Is there any way to get the bare metal application example and/or the PS_PCIe_DMA_vX.pdf on my hands? My target is to configure a ZynqMP device as a PCIe PS Endpoint by running either Linux or bare metal software.

0 Kudos
Highlighted
Observer
Observer
2,641 Views
Registered: ‎02-19-2019

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

Where is this http://www.wiki.xilinx.com/file/detail/PS_PCIe_DMA_v4.pdf  document. I'ts a dead link and the document cannot be found using search.

0 Kudos
Highlighted
Observer
Observer
2,590 Views
Registered: ‎05-03-2010

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution
Hi Venkata,

The link to PS_PCIe_DMA_v4.pdf is not working. Can you send the correct link?

Thanks,
Dieter
0 Kudos
Highlighted
Moderator
Moderator
2,581 Views
Registered: ‎02-16-2010

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

Hi @steinbrueck,

I am checking for the document. I will reply here when I have the update.

Thanks,

Srinadh

------------------------------------------------------------------------------
Don't forget to reply, give kudo and accept as solution
------------------------------------------------------------------------------
0 Kudos
Highlighted
Moderator
Moderator
2,554 Views
Registered: ‎02-16-2010

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

Hi @steinbrueck,

Please check the AR#72076. This AR is an alternative for the document you referred.

https://www.xilinx.com/support/answers/72076.html

------------------------------------------------------------------------------
Don't forget to reply, give kudo and accept as solution
------------------------------------------------------------------------------
0 Kudos
Highlighted
Observer
Observer
2,541 Views
Registered: ‎02-19-2019

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

venkata, Thanks for tne link. Adds information that indicates I set the zynq up correctly in vivado. Though It still does not work as advertised in "https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842008/PetaLinux+kernel+and+application+build+steps+in+configuring+ZU+MPSoC+board+as+PS+PCIe+End+Point+and+Root+Port

  1.  Petalinux 2018.3 will not build if the linux kernel is remoted to master.
  2. I get the following error on the endpoint build when the 2018.3 kernel is configured as the document suggests. See the attached messages.endpoint for the kernel log.
  3. When the host scans the for the device the dma handler OOPses see messages.oops

Any thoughts as to how this is mis-configured?

The Host is a ZCU102, and the endpoint is using an ZynqUltra+ PS PCIe endpoint.

How are the PCIe Endpoint DMA regions defined and enumerated for the linux drivers?

 

Thanks

 

0 Kudos
Highlighted
Observer
Observer
2,532 Views
Registered: ‎02-19-2019

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

I mean, the endpoint DMA appears to be in the kernel code and this:

pci_rootdma: rootdma@fd0f0000 {
compatible = "xlnx,ps_pcie_dma-1.00.a";
reg = <0x0 0xfd0f0000 0x0 0x1000>;
reg-names = "ps_pcie_regbase";
interrupts = <0 117 4>;
interrupt-names = "ps_pcie_rootdma_intr";
interrupt-parent = <&gic>;
rootdma;
dma_vendorid = /bits/ 16 <0x10EE>;
dma_deviceid = /bits/ 16 <0xD021>;
numchannels = <0x4>;
#size-cells = <0x5>;
ps_pcie_channel0 = <0x1 0x7CF 0x4 0x0 0x3E8>;
ps_pcie_channel1 = <0x0 0x7CF 0x4 0x0 0x3E8>;
ps_pcie_channel2 = <0x1 0x7CF 0x4 0x0 0x3E8>;
ps_pcie_channel3 = <0x0 0x7CF 0x4 0x0 0x3E8>;

 };

Appears to be the mapping. Does the platform driver not handle the PCIe as an Endpoint under linux?

 

0 Kudos
Highlighted
Observer
Observer
1,253 Views
Registered: ‎10-01-2019

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

My configuration and deployment here are as follows:


1. ZCU102 acts as PCIE Root, KCU 105 acts as PCIE endpoint for data transmission

2. Using vivado 2018.3, petalinux 2018.3

3. Program KCU105 using the hdf file provided at https://www.xilinx.com/support/documentation/boards_and_kits/kcu105/2017_3/ug919-kcu105-pcie-aximm-data-plane-trd-ug.pdf

4. Petalinux config pcie dma is set as follows:
CONFIG_XILINX_PS_PCIE_DMA_TEST=y
CONFIG_XILINX_DMA=y
CONFIG_XILINX_PS_PCIE_DMA=y
CONFIG_AXI_DMAC=y
CONFIG_XILINX_DMA_ENGINES=y
CONFIG_XILINX_ZYNQMP_DMA=y
CONFIG_DMATEST=y

5. In the config process zynqmp.dtsi there is a list of 8*fpd+8*lpd dma channels and status="disable". But in zynqmp-zcu102-revA.dts will be overwritten as status="okay"

6. The above configuration has successfully established sd card image, the following dmesg appears during boot:
[ 2.039796] Error channel handle 0 board 0 channel
[ 2.039810] Unable to acquire dma channels 0
[ 2.039818] Error creating char devices for 0
The above 3 messages are from xilinx_ps_pcie_dma_client.c

In other words, the ps_pcie_dma_client driver was not successfully installed, and /dev/ps_pcie_dmachan * _0 was not created.
Hdf configuration as shown below

1.PNG

I would like to ask if there is any configuration or setting error, any suggestions or help is very grateful.

0 Kudos
Highlighted
Observer
Observer
1,223 Views
Registered: ‎02-19-2019

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

Not quite the same problem. Looks like you're using the Petalinux build. see ./project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi specifically to properly override.

0 Kudos
Highlighted
Observer
Observer
1,182 Views
Registered: ‎10-01-2019

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

 Hi @dtyree_nam  thanks for your reply
I checked the system-user.dtsi in my petalinux project and found that nothing was added (as shown below).
2.PNG

In addition, I will clearly describe my problem: 
1. dma channel will be defined in components/plnx_workspace/device-tree/device-tree/zynqmp.dtsi (fpd_dma_chan1~fpd_dma_chan8, lpd_dma_chan1~lpd_dma_chan8), 16 of which status="disable"

2. build/tmp/work-shared/plnx-zynqmp/kernel-source/arch/arm64/boot/dts/xilinx/zynqmp-zcu102-revA.dts also defines fpd_dma_chan1~fpd_dma_chan8, lpd_dma_chan1~lpd_dma_chan8 a total of 16 dma channels, Each of them status="okay"

3. In the process of executing petalinux-build, there will be script execution in zynqmp-zcu102-revA.dts "include zynqmp.dtsi", that is, the status="disable" of 16 channels will be changed to "okay".

4. After booting zcu102, I can see the 16 dma channels I set from /sys/firmware/devicetree/base/amba/ (as shown below)

5.PNG

Each channel has a set of files, each file corresponding to the property of the device tree node (including status), and the status file is written with "okay"

 

5. The driver "drivers/dma/xilinx/xilinx_ps_pcie_dma_client.c" has no way to get the dma channel and the following error message is displayed during the installation process.
[ 2.039796] Error channel handle 0 board 0 channel
[ 2.039810] Unable to acquire dma channels 0
[ 2.039818] Error creating char devices for 0

6. For zcu102 hdf configuration we refer to the Xilinx_Answer_71210 file

PCIE1.jpgPCIE2.jpgPCIE3.jpgPCIE4.jpg

7. Attachment I uploaded the device tree file and the zcu 102 hdf file. 

8. Using vivado 2018.3, petalinux 2018.3

I would like to ask if there is any configuration or setting error, any suggestions or help is very grateful.

 

0 Kudos
Highlighted
Observer
Observer
1,140 Views
Registered: ‎02-19-2019

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

I mean, you appear to be setting up a host port, not and endpoint. Device port type is set to Root Port. I had used the Xilinx board as host (Root port) for a Xilinx PS endpoint early in development. I activated the pci root port per the wiki and that was easy with the kernel module and system-user.dtt change. 

To get the PS to behave as an endpoint, that required some digging and realizing none of the xilinx linux code was written with that in mind. You have to configure the Bridge yourself, and write your own handler. The standard Xilinx drivers do not work for an endpoint PS driver (or didn't when I looked at it). 

If you are a Root port ... Did you do a lspci, I'll check above again...

0 Kudos
Highlighted
Observer
Observer
1,105 Views
Registered: ‎10-01-2019

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

Hi @dtyree_nam  Thank you for your reply

My goal is to use the ZCU102 board as the PCIE Root port, and the KCU105 as a PCIE endpoint for PCIE data reading and writing.

The last reply of my post is: hdf and petalinux image configuration & build of the ZCU102 board (PCIE Root port). The following figure shows the result of executing lspci on the ZCU102 board console.
6.PNG

7.PNG

8.PNG

I originally referred to the wiki "XAPP1289 PCIe Root DMA", my board configuration is the same as them. . They have a PCIe Root DMA driver source code and a bitstream file to program KCU105. but the actual use is not work, so I think: This wiki should not be for a newer version of petalinux, xilinx should have a built-in new version of PCIe Root DMA driver, so I no longer refer to this wiki.

I currently configure the petalinux image of my ZCU102 board (PCIE Root port) with reference to the wiki "PetaLinux kernel and application build steps in configuring ZU MPSoC board as PS PCIe End Point and Root Port". 

Maybe for my goal, my current implementation steps may be wrong...

0 Kudos
Highlighted
Observer
Observer
944 Views
Registered: ‎10-01-2019

Re: Problem when configure PS PCIe as Endpoint on zynqmp Ultrascale+ device

Jump to solution

Hi @dtyree_nam 

My configuration is similar to yours (host: ZCU102, endpoint: KCU105)

I also added "pci_rootdma:rootdma@fd0f0000" to system-user.dtsi, but it also can't work

Did you run it successfully later?

 

0 Kudos