cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
nzh
Contributor
Contributor
523 Views
Registered: ‎05-04-2020

XDMA AXI-Lite write works but read causes Ubuntu 16.04 to crash

I have a module that uses AXI-Lite and AXI interface(https://github.com/hsharma35/dnnweaver2/tree/master/hardware). I am trying to implement it on Alveo U200 . 

pic1.png

addr.jpg

I also believe the driver is installed correctly. Although I can see some suspicious lines in the dmesg.

[   13.383687] PKCS#7 signature not signed with a trusted key
[   13.383730] xdma: loading out-of-tree module taints kernel.
[   13.383888] xdma: module verification failed: signature and/or required key missing - tainting kernel

The full xdma part of dmesg is below:

[   12.674233] ipmi_si dmi-ipmi-si.0: IPMI kcs interface initialized
[   12.766206] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[   12.806237] IPMI SSIF Interface driver
[   12.821174] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0x900, revision 0
[   12.821565] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0x920
[   13.383687] PKCS#7 signature not signed with a trusted key
[   13.383730] xdma: loading out-of-tree module taints kernel.
[   13.383888] xdma: module verification failed: signature and/or required key missing - tainting kernel
[   13.386368] xdma:xdma_mod_init: Xilinx XDMA Reference Driver xdma v2019.2.51
[   13.386372] xdma:xdma_mod_init: desc_blen_max: 0xfffffff/268435455, sgdma_timeout: 10 sec.
[   13.386421] xdma:xdma_threads_create: xdma_threads_create
[   13.392825] xdma:xdma_device_open: xdma device 0000:23:00.0, 0x00000000eb11354b.
[   13.392847] xdma:alloc_dev_instance: xdev = 0x000000006c4ec11c
[   13.392864] xdma:xdev_list_add: dev 0000:23:00.0, xdev 0x000000006c4ec11c, xdma idx 0.
[   13.393289] xdma:request_regions: pci_request_regions()
[   13.393304] xdma:map_single_bar: BAR0: 4096 bytes to be mapped.
[   13.393391] xdma:map_single_bar: BAR0 at 0xe57ff000 mapped at 0x00000000e936cca7, length=4096(/4096)
[   13.393392] xdma:map_single_bar: BAR1: 65536 bytes to be mapped.
[   13.394158] xdma:map_single_bar: BAR1 at 0xe57e0000 mapped at 0x000000007b52b600, length=65536(/65536)
[   13.394163] xdma:is_config_bar: BAR 1 is the XDMA config BAR
[   13.394166] xdma:map_bars: config bar 1, pos 1.
[   13.394168] xdma:identify_bars: xdev 0x000000006c4ec11c, bars 2, config at 1.
[   13.394171] xdma:identify_bars: 2 BARs: config 1, user 0, bypass -1.
[   13.394174] xdma:set_dma_mask: sizeof(dma_addr_t) == 8
[   13.394176] xdma:set_dma_mask: pci_set_dma_mask()
[   13.394178] xdma:set_dma_mask: Using a 64-bit DMA mask.
[   13.394191] xdma:__write_register: channel_interrupts_disable: w reg 0x2000(0x0000000047bc216f), 0xffffffff.
[   13.394194] xdma:__write_register: user_interrupts_disable: w reg 0x2000(0x00000000e20ef1b4), 0xffffffff.
[   13.394200] xdma:read_interrupts: ioread32(0x0000000003c36767) returned 0x00000000 (user_int_request).
[   13.394205] xdma:read_interrupts: ioread32(0x00000000b9279ab4) returned 0x00000000 (channel_int_request)
[   13.394212] xdma:probe_for_engine: found AXI H2C 0 engine, reg. off 0x0, id 0x1fc0,0x0.
[   13.394215] xdma:engine_init: channel 0, offset 0x0, dir 1.
[   13.394219] xdma:engine_init: engine 00000000fb31050f name 0-H2C0-MM irq_bitmask=0x00000001
[   13.394262] xdma:__write_register: engine_init_regs: w reg 0x25d110d7dbb4(0x00000000d01edaf1), 0x2000000.
[   13.394266] xdma:engine_alignments: engine 00000000fb31050f name 0-H2C0-MM alignments=0x00010140
[   13.394269] xdma:engine_alignments: align_bytes = 1
[   13.394271] xdma:engine_alignments: granularity_bytes = 1
[   13.394273] xdma:engine_alignments: address_bits = 64
[   13.394277] xdma:__write_register: engine_init_regs: w reg 0x25d110d7dc38(0x0000000046f57aec), 0xf83e1e.
[   13.394280] xdma:probe_for_engine: found AXI H2C 1 engine, reg. off 0x100, id 0x1fc0,0x1.
[   13.394283] xdma:engine_init: channel 1, offset 0x100, dir 1.
[   13.394287] xdma:engine_init: engine 00000000e726edd6 name 0-H2C1-MM irq_bitmask=0x00000002
[   13.394321] xdma:__write_register: engine_init_regs: w reg 0x25d110d7db14(0x000000003373c8e0), 0x2000000.
[   13.394325] xdma:engine_alignments: engine 00000000e726edd6 name 0-H2C1-MM alignments=0x00010140
[   13.394328] xdma:engine_alignments: align_bytes = 1
[   13.394330] xdma:engine_alignments: granularity_bytes = 1
[   13.394332] xdma:engine_alignments: address_bits = 64
[   13.394335] xdma:__write_register: engine_init_regs: w reg 0x25d110d7db98(0x000000007541134a), 0xf83e1e.
[   13.394340] xdma:probe_for_engine: found AXI H2C 2 engine, reg. off 0x200, id 0x1fc0,0x2.
[   13.394345] xdma:engine_init: channel 2, offset 0x200, dir 1.
[   13.394349] xdma:engine_init: engine 0000000063371f2b name 0-H2C2-MM irq_bitmask=0x00000004
[   13.394387] xdma:__write_register: engine_init_regs: w reg 0x25d110d7da74(0x000000007d5490c5), 0x2000000.
[   13.394391] xdma:engine_alignments: engine 0000000063371f2b name 0-H2C2-MM alignments=0x00010140
[   13.394393] xdma:engine_alignments: align_bytes = 1
[   13.394396] xdma:engine_alignments: granularity_bytes = 1
[   13.394398] xdma:engine_alignments: address_bits = 64
[   13.394401] xdma:__write_register: engine_init_regs: w reg 0x25d110d7daf8(0x00000000cf10ae25), 0xf83e1e.
[   13.394405] xdma:probe_for_engine: found AXI H2C 3 engine, reg. off 0x300, id 0x1fc0,0x3.
[   13.394407] xdma:engine_init: channel 3, offset 0x300, dir 1.
[   13.394410] xdma:engine_init: engine 000000002a467a78 name 0-H2C3-MM irq_bitmask=0x00000008
[   13.394458] xdma:__write_register: engine_init_regs: w reg 0x25d110d7d9d4(0x0000000042c12f67), 0x2000000.
[   13.394462] xdma:engine_alignments: engine 000000002a467a78 name 0-H2C3-MM alignments=0x00010140
[   13.394463] xdma:engine_alignments: align_bytes = 1
[   13.394465] xdma:engine_alignments: granularity_bytes = 1
[   13.394468] xdma:engine_alignments: address_bits = 64
[   13.394472] xdma:__write_register: engine_init_regs: w reg 0x25d110d7da58(0x00000000dd8d4cef), 0xf83e1e.
[   13.394480] xdma:probe_for_engine: found AXI C2H 0 engine, reg. off 0x1000, id 0x1fc1,0x0.
[   13.394482] xdma:engine_init: channel 0, offset 0x1000, dir 2.
[   13.394486] xdma:engine_init: engine 000000009c228f65 name 0-C2H0-MM irq_bitmask=0x00000010
[   13.394525] xdma:__write_register: engine_init_regs: w reg 0x25d110d7e534(0x00000000c3c29aa0), 0x2000000.
[   13.394529] xdma:engine_alignments: engine 000000009c228f65 name 0-C2H0-MM alignments=0x00010140
[   13.394532] xdma:engine_alignments: align_bytes = 1
[   13.394533] xdma:engine_alignments: granularity_bytes = 1
[   13.394535] xdma:engine_alignments: address_bits = 64
[   13.394538] xdma:__write_register: engine_init_regs: w reg 0x25d110d7e5b8(0x00000000ce8b1e21), 0xf83e1e.
[   13.394543] xdma:probe_for_engine: found AXI C2H 1 engine, reg. off 0x1100, id 0x1fc1,0x1.
[   13.394546] xdma:engine_init: channel 1, offset 0x1100, dir 2.
[   13.394549] xdma:engine_init: engine 0000000049d0ced6 name 0-C2H1-MM irq_bitmask=0x00000020
[   13.394590] xdma:__write_register: engine_init_regs: w reg 0x25d110d7e494(0x00000000587119f7), 0x2000000.
[   13.394595] xdma:engine_alignments: engine 0000000049d0ced6 name 0-C2H1-MM alignments=0x00010140
[   13.394598] xdma:engine_alignments: align_bytes = 1
[   13.394601] xdma:engine_alignments: granularity_bytes = 1
[   13.394602] xdma:engine_alignments: address_bits = 64
[   13.394606] xdma:__write_register: engine_init_regs: w reg 0x25d110d7e518(0x000000000b3d0c81), 0xf83e1e.
[   13.394610] xdma:probe_for_engine: found AXI C2H 2 engine, reg. off 0x1200, id 0x1fc1,0x2.
[   13.394613] xdma:engine_init: channel 2, offset 0x1200, dir 2.
[   13.394616] xdma:engine_init: engine 00000000ce2a3fc8 name 0-C2H2-MM irq_bitmask=0x00000040
[   13.394654] xdma:__write_register: engine_init_regs: w reg 0x25d110d7e3f4(0x00000000ac24fcfe), 0x2000000.
[   13.394657] xdma:engine_alignments: engine 00000000ce2a3fc8 name 0-C2H2-MM alignments=0x00010140
[   13.394659] xdma:engine_alignments: align_bytes = 1
[   13.394660] xdma:engine_alignments: granularity_bytes = 1
[   13.394662] xdma:engine_alignments: address_bits = 64
[   13.394665] xdma:__write_register: engine_init_regs: w reg 0x25d110d7e478(0x0000000027ae7231), 0xf83e1e.
[   13.394670] xdma:probe_for_engine: found AXI C2H 3 engine, reg. off 0x1300, id 0x1fc1,0x3.
[   13.394672] xdma:engine_init: channel 3, offset 0x1300, dir 2.
[   13.394678] xdma:engine_init: engine 0000000047817943 name 0-C2H3-MM irq_bitmask=0x00000080
[   13.394722] xdma:__write_register: engine_init_regs: w reg 0x25d110d7e354(0x000000005b770175), 0x2000000.
[   13.394725] xdma:engine_alignments: engine 0000000047817943 name 0-C2H3-MM alignments=0x00010140
[   13.394727] xdma:engine_alignments: align_bytes = 1
[   13.394728] xdma:engine_alignments: granularity_bytes = 1
[   13.394730] xdma:engine_alignments: address_bits = 64
[   13.394732] xdma:__write_register: engine_init_regs: w reg 0x25d110d7e3d8(0x000000006235ab2a), 0xf83e1e.
[   13.394743] xdma:enable_msi_msix: MSI/MSI-X not detected - using legacy interrupts
[   13.394750] xdma:irq_legacy_setup: Legacy Interrupt register value = 1
[   13.394792] xdma:irq_legacy_setup: Using IRQ#42 with 0x000000006c4ec11c
[   13.394799] xdma:__write_register: channel_interrupts_enable: w reg 0x2000(0x0000000030105538), 0xffffffff.
[   13.394803] xdma:read_interrupts: ioread32(0x0000000003c36767) returned 0x00000000 (user_int_request).
[   13.394806] xdma:read_interrupts: ioread32(0x00000000b9279ab4) returned 0x00000000 (channel_int_request)
[   13.394813] xdma:__write_register: user_interrupts_enable: w reg 0x2000(0x000000000ac6f68c), 0x1ffff.
[   13.394816] xdma:read_interrupts: ioread32(0x0000000003c36767) returned 0x00000000 (user_int_request).
[   13.394819] xdma:read_interrupts: ioread32(0x00000000b9279ab4) returned 0x00000000 (channel_int_request)
[   13.394823] xdma:probe_one: 0000:23:00.0 xdma0, pdev 0x00000000eb11354b, xdev 0x0000000011a27151, 0x000000006c4ec11c, usr 16, ch 4,4.
[   13.394835] xdma:create_xcdev: xcdev 0x000000007767597c, 241:1, (null), type 0x1.
[   13.395048] xdma:create_xcdev: xcdev 0x0000000043ba10a9, 241:10, (null), type 0x3.
[   13.395130] xdma:create_xcdev: xcdev 0x000000003989c7fc, 241:11, (null), type 0x3.
[   13.395253] xdma:create_xcdev: xcdev 0x000000007eb75289, 241:12, (null), type 0x3.
[   13.395325] xdma:create_xcdev: xcdev 0x0000000048e66f59, 241:13, (null), type 0x3.
[   13.395390] xdma:create_xcdev: xcdev 0x000000007c67557f, 241:14, (null), type 0x3.
[   13.395437] xdma:create_xcdev: xcdev 0x0000000038a9fc2e, 241:15, (null), type 0x3.
[   13.395502] xdma:create_xcdev: xcdev 0x000000006184f3ba, 241:16, (null), type 0x3.
[   13.395563] xdma:create_xcdev: xcdev 0x000000002dadfc10, 241:17, (null), type 0x3.
[   13.395651] xdma:create_xcdev: xcdev 0x00000000313d8d62, 241:18, (null), type 0x3.
[   13.395725] xdma:create_xcdev: xcdev 0x000000003fc59a49, 241:19, (null), type 0x3.
[   13.395802] xdma:create_xcdev: xcdev 0x00000000d14d0bf9, 241:20, (null), type 0x3.
[   13.395885] xdma:create_xcdev: xcdev 0x00000000f3528222, 241:21, (null), type 0x3.
[   13.395945] xdma:create_xcdev: xcdev 0x000000006fdd9766, 241:22, (null), type 0x3.
[   13.396030] xdma:create_xcdev: xcdev 0x00000000cf3d30bc, 241:23, (null), type 0x3.
[   13.396159] xdma:create_xcdev: xcdev 0x000000009dcb79f9, 241:24, (null), type 0x3.
[   13.396224] xdma:create_xcdev: xcdev 0x0000000069c4eb53, 241:25, (null), type 0x3.
[   13.396323] xdma:create_xcdev: xcdev 0x000000000de4e9ac, 241:32, (null), type 0x4.
[   13.396375] xdma:create_xcdev: xcdev 0x000000002d4e289e, 241:33, (null), type 0x4.
[   13.396444] xdma:create_xcdev: xcdev 0x00000000c5a5dceb, 241:34, (null), type 0x4.
[   13.396540] xdma:create_xcdev: xcdev 0x00000000b5725c55, 241:35, (null), type 0x4.
[   13.396623] xdma:create_xcdev: xcdev 0x000000007bc7518f, 241:36, (null), type 0x5.
[   13.396732] xdma:create_xcdev: xcdev 0x00000000eec414d5, 241:37, (null), type 0x5.
[   13.396810] xdma:create_xcdev: xcdev 0x0000000017f4f232, 241:38, (null), type 0x5.
[   13.396865] xdma:create_xcdev: xcdev 0x00000000a6ed11fa, 241:39, (null), type 0x5.
[   13.396917] xdma:create_xcdev: xcdev 0x000000004e573ee6, 241:0, (null), type 0x0.
[   13.396968] xdma:cdev_xvc_init: xcdev 0x0000000041873310, bar 0, offset 0x40000.
[   13.396970] xdma:create_xcdev: xcdev 0x0000000041873310, 241:2, (null), type 0x2.
[   13.854807] AVX version of gcm_enc/dec engaged.
[   13.854810] AES CTR mode by8 optimization enabled
[   14.765916] kvm: Nested Virtualization enabled
[   14.765955] kvm: Nested Paging enabled
[   14.885336] MCE: In-kernel MCE decoding enabled.
[   14.944655] EDAC amd64: Node 0: DRAM ECC enabled.
[   14.944658] EDAC amd64: F15h detected (node 0).
[   14.944718] EDAC MC: DCT0 chip selects:
[   14.944719] EDAC amd64: MC: 0:     0MB 1:     0MB
[   14.944721] EDAC amd64: MC: 2:  8192MB 3:  8192MB
[   14.944722] EDAC amd64: MC: 4:     0MB 5:     0MB
[   14.944723] EDAC amd64: MC: 6:     0MB 7:     0MB
[   14.944723] EDAC MC: DCT1 chip selects:
[   14.944724] EDAC amd64: MC: 0:     0MB 1:     0MB
[   14.944725] EDAC amd64: MC: 2:  8192MB 3:  8192MB

 

When I write through axi-lite(4k mapped), it looks working, but when I read, it os freezes. testLite.c is exactly same code as reg_rw.c test but the size is 4kB. This is writing:

argus@argus-R815:~/Desktop/ML/dma_ip_drivers-master/XDMA/linux-kernel/tools$ ./testLite /dev/xdma0_user 0 b 1
argc = 5
device: /dev/xdma0_user
address: 0x00000000
access type: write
access width given.
access width: byte (8-bits)
character device /dev/xdma0_user opened.
Memory mapped at address 0x7fe30c6e8000.
Write 8-bits value 0x01 to 0x00000000 (0x0x7fe30c6e8000)

 

and dmesg for writing is :

[  464.244437] xdma:bridge_mmap: mmap(): xcdev = 0xffff9a5fbc5c7228
[  464.244443] xdma:bridge_mmap: mmap(): cdev->bar = 0
[  464.244447] xdma:bridge_mmap: mmap(): xdev = 0x000000006c4ec11c
[  464.244449] xdma:bridge_mmap: mmap(): pci_dev = 0xffff9a57bd539000
[  464.244450] xdma:bridge_mmap: off = 0x0
[  464.244452] xdma:bridge_mmap: start = 0xe57ff000
[  464.244453] xdma:bridge_mmap: phys = 0xe57ff000
[  464.244467] xdma:bridge_mmap: vma=0x000000008075c024, vma->vm_start=0x7fb9e6f48000, phys=0xe57ff, size=4096 = 0

 

But when I try to read, it freezes the computer and I have to restart the computer. 

freezw.jpg

Any idea what might be wrong here and cause this? 

 

0 Kudos
1 Reply
dgisselq
Scholar
Scholar
503 Views
Registered: ‎05-21-2015

@nzh,

Is this the AXI-lite slave core you are using?  If so, you should know that it has multiple bugs within it.  In particular, neither write nor read channels are responsive to back pressure.  As a result, the design will lock up on the second read (or write) in the presence of back pressure.

Here's a picture of what this might look like on the read side.

axi-read-fault.png

A quick formal verification check of the design would find this fault in a heart beat.

Dan