cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Visitor
Visitor
546 Views
Registered: ‎08-06-2020

Linux Kernel Debug - Vitis 2019.2

Is there a guide to setup and debug LINUX kernel crashes?

I'm using Vitis 2019.2 and during kernel boot it just stops (no crash prints). Below is the boot log. Can anyone help how i can hook debugger to see what is happening?

 

LOG

-------------------------------------------

Starting kernel ...

[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.19.0-xilinx-v2019.2 (oe-user@oe-host) (gcc version 8.2.0 (GCC)) #1 SMP PREEMPT Sat Aug 29 14:08:29 UTC 2020
[ 0.000000] CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=18c5387d
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[ 0.000000] OF: fdt: Machine model: Grace Series
[ 0.000000] Memory policy: Data cache writealloc
[ 0.000000] cma: Reserved 16 MiB at 0x3f000000
[ 0.000000] On node 0 totalpages: 262144
[ 0.000000] Normal zone: 1536 pages used for memmap
[ 0.000000] Normal zone: 0 pages reserved
[ 0.000000] Normal zone: 196608 pages, LIFO batch:63
[ 0.000000] HighMem zone: 65536 pages, LIFO batch:15
[ 0.000000] random: get_random_bytes called from start_kernel+0x80/0x3c4 with crng_init=0
[ 0.000000] percpu: Embedded 16 pages/cpu @(ptrval) s35916 r8192 d21428 u65536
[ 0.000000] pcpu-alloc: s35916 r8192 d21428 u65536 alloc=16*4096
[ 0.000000] pcpu-alloc: [0] 0 [0] 1
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 260608
[ 0.000000] Kernel command line:
[ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.000000] Memory: 1001176K/1048576K available (6144K kernel code, 205K rwdata, 1616K rodata, 1024K init, 132K bss, 31016K reserved, 16384K cma-reserved, 245760K highmem)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
[ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
[ 0.000000] vmalloc : 0xf0800000 - 0xff800000 ( 240 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xf0000000 ( 768 MB)
[ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
[ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
[ 0.000000] .text : 0x(ptrval) - 0x(ptrval) (7136 kB)
[ 0.000000] .init : 0x(ptrval) - 0x(ptrval) (1024 kB)
[ 0.000000] .data : 0x(ptrval) - 0x(ptrval) ( 206 kB)
[ 0.000000] .bss : 0x(ptrval) - 0x(ptrval) ( 133 kB)
[ 0.000000] rcu: Preemptible hierarchical RCU implementation.
[ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
[ 0.000000] Tasks RCU enabled.
[ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
[ 0.000000] efuse mapped to (ptrval)
[ 0.000000] slcr mapped to (ptrval)
[ 0.000000] L2C: platform modifies aux control register: 0x72360000 -> 0x72760000
[ 0.000000] L2C: DT/platform modifies aux control register: 0x72360000 -> 0x72760000
[ 0.000000] L2C-310 erratum 769419 enabled
[ 0.000000] L2C-310 enabling early BRESP for Cortex-A9
[ 0.000000] L2C-310 full line of zeros enabled for Cortex-A9
[ 0.000000] L2C-310 ID prefetch enabled, offset 1 lines
[ 0.000000] L2C-310 dynamic clock gating enabled, standby mode enabled
[ 0.000000] L2C-310 cache controller enabled, 8 ways, 512 kB
[ 0.000000] L2C-310: CACHE_ID 0x410000c8, AUX_CTRL 0x76760001
[ 0.000000] zynq_clock_init: clkc starts at (ptrval)
[ 0.000000] Zynq clock init
[ 0.000013] sched_clock: 64 bits at 337MHz, resolution 2ns, wraps every 4398046511103ns
[ 0.000037] clocksource: arm_global_timer: mask: 0xffffffffffffffff max_cycles: 0x4dd67c9137, max_idle_ns: 440795214692 ns
[ 0.000070] Switching to timer-based delay loop, resolution 2ns
[ 0.000180] clocksource: ttc_clocksource: mask: 0xffff max_cycles: 0xffff, max_idle_ns: 530903770 ns
[ 0.000224] timer #0 at (ptrval), irq=17
[ 0.000755] Console: colour dummy device 80x30
[ 0.001642] console [tty0] enabled
[ 0.001681] Calibrating delay loop (skipped), value calculated using timer frequency.. 675.00 BogoMIPS (lpj=3375000)
[ 0.001727] pid_max: default: 32768 minimum: 301
[ 0.001939] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.001975] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.002659] CPU: Testing write buffer coherency: ok
[ 0.002714] CPU0: Spectre v2: using BPIALL workaround
[ 0.002982] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
[ 0.060404] Setting up static identity map for 0x100000 - 0x100060
[ 0.080347] rcu: Hierarchical SRCU implementation.
[ 0.120346] smp: Bringing up secondary CPUs ...
[ 0.595665] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
[ 0.595674] CPU1: Spectre v2: using BPIALL workaround
[ 0.595847] smp: Brought up 1 node, 2 CPUs
[ 0.595880] SMP: Total of 2 processors activated (1350.00 BogoMIPS).
[ 0.595905] CPU: All CPU(s) started in SVC mode.
[ 0.597036] devtmpfs: initialized
[ 0.602019] VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 4
[ 0.602565] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[ 0.602624] futex hash table entries: 512 (order: 3, 32768 bytes)
[ 0.603826] pinctrl core: initialized pinctrl subsystem
[ 0.604931] NET: Registered protocol family 16
[ 0.607136] DMA: preallocated 256 KiB pool for atomic coherent allocations
[ 0.615830] cpuidle: using governor menu
[ 0.625635] hw-breakpoint: found 5 (+1 reserved) breakpoint and 1 watchpoint registers.
[ 0.625706] hw-breakpoint: maximum watchpoint size is 4 bytes.
[ 0.625895] zynq-ocm f800c000.ocmc: can't request region for resource [mem 0xfffc0000-0xffffffff]
[ 0.625950] zynq-ocm: probe of f800c000.ocmc failed with error -16
[ 0.626278] zynq-pinctrl 700.pinctrl: zynq pinctrl initialized
[ 0.627008] e0001000.serial: ttyPS0 at MMIO 0xe0001000 (irq = 27, base_baud = 3125000) is a xuartps
[ 1.139052] console [ttyPS0] enabled

Tags (3)
0 Kudos
Reply
7 Replies
Teacher
Teacher
519 Views
Registered: ‎06-16-2013

Hi @sathyahs 

 

It's kernel argument issue and/or rootfs issue.

 

>[ 0.000000] Kernel command line:

-> You need to set proper argument.

 

Would you make sure them ?

 

Best regards,

0 Kudos
Reply
Visitor
Visitor
446 Views
Registered: ‎08-06-2020

@watari 

Please find the working log attached, so i feel my config, kernel command line and rootfs are fine.

If i change my bit file, it is crashing. So i want to know which instruction is exactly causing the issue.

Is there a way i can debug kernel line by line hooking system debugger?

 

0 Kudos
Reply
Teacher
Teacher
418 Views
Registered: ‎06-16-2013

Hi @sathyahs 

 

I confirm "working_dump.txt" file. And it was fine.

 

>If i change my bit file, it is crashing.

I guess dtb file is wrong in your system when you change your bit file.

Because at least you need to update dtb file or do device-tree-overlay when you change dtb.

 

Would you make sure it, if possible ?

If impossible, would you share your log file when you encounter crashing issue ?

 

Best regards,

0 Kudos
Reply
Moderator
Moderator
387 Views
Registered: ‎02-07-2018

HI @sathyahs 

 

Can you try this below trails.

Trail 1: Remove pl device tree changes & build.

petalinux-config
DTG Settings --->
[*] Remove PL from devicetree

 

Trail 2 : Please clean complete project & try to rebuild once.

petalinux-build -x mrproper

petalinux-config

petalinux-build

 

Thanks & regards

Aravind

----------------------------------------------------------------------------------------------
Kindly note- Please mark the Answer as "Accept as solution" if information provided is helpful.
Give Kudos to a post which you think is helpful and reply oriented.
---------------------------------------------------------------------------------------------

 

0 Kudos
Reply
Visitor
Visitor
363 Views
Registered: ‎08-06-2020

@aravindb 

yes, if i remove pl device tree it works.

but i want to know which module in PL is creating the hang. 

Is there a simple way to debug kernel boot

0 Kudos
Reply
Visitor
Visitor
360 Views
Registered: ‎08-06-2020

i want to follow something like below, but that link is for older SDK, not for VITIS

https://www.xilinx.com/html_docs/xilinx2018_1/SDK_Doc/SDK_tasks/sdk_t_tcf_attach_debug_linux_kernel_tcf.html

 

Anyone has a link for kernel debug using VITIS?

0 Kudos
Reply
Explorer
Explorer
286 Views
Registered: ‎02-22-2012

Kernel crash will dump on on console and looks there is none from your console dump.
Note that "hang" you see, could also mean that your /etc/inittab has switched serial console to some other device (other than kernel boot has started with).
As @watari suggested I would check kernel argument and rootfs create.
0 Kudos
Reply