cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Visitor
Visitor
5,755 Views
Registered: ‎05-12-2010

V4FX60 Linux boot problem

Hi all,

    I'm am in the process of booting Linux on a V4Fx60.  The board itself is custom but it is known to work with Integrity and an older version of Linux (2.6.22).  That version use the "old way" with the ppc arch. 

The system inside the fpga is a basic system with ethernet, uartlite and a few other components.  The port of u-boot went like a charm but Linux still doesn't start.  Most often, the bootup don't even get to the point it can output on the serial port but I'm able to get the logs from the memory using a probe.

 

First log:

-------------------------------------------------------

<6>Using Xilinx Virtex machine description
<5>Linux version 2.6.33 (bmarchand@edouard) (gcc version 4.2.2) #2 PREEMPT Wed May 12 10:19:08 EDT 2010
<7>Top of RAM: 0x8000000, Total RAM: 0x8000000
<7>Memory hole size: 0MB
<4>Zone PFN ranges:
<4>  DMA      0x00000000 -> 0x00008000
<4>  Normal   0x00008000 -> 0x00008000
<4>Movable zone start PFN for each node
<4>early_node_map[1] active PFN ranges
<4>    0: 0x00000000 -> 0x00008000
<7>On node 0 totalpages: 32768
<7>free_area_init_node: node 0, pgdat c035cf64, node_mem_map c0388000
<7>  DMA zone: 256 pages used for memmap
<7>  DMA zone: 0 pages reserved
<7>  DMA zone: 32512 pages, LIFO batch:7
<6>MMU: Allocated 1088 bytes of context maps for 255 contexts
<4>Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32512
<5>Kernel command line: console=ttyUL0 ip=off root=/dev/xsysace/disc0/part2 rw
<6>PID hash table entries: 512 (order: -1, 2048 bytes)
<6>Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
<6>Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
<6>Memory: 126148k/131072k available (3360k kernel code, 4784k reserved, 124k data, 127k bss, 160k init)
<6>Kernel virtual memory layout:
<6>  * 0xfffdf000..0xfffff000  : fixmap
<6>  * 0xfde00000..0xfe000000  : consistent mem
<6>  * 0xfde00000..0xfde00000  : early ioremap
<6>  * 0xc9000000..0xfde00000  : vmalloc & ioremap
<6>Hierarchical RCU implementation.
<6>NR_IRQS:512 nr_irqs:512
<7>time_init: decrementer frequency = 300.000000 MHz
<7>time_init: processor frequency   = 300.000000 MHz
<6>clocksource: timebase mult[d55555] shift[22] registered
<7>clockevent: decrementer mult[4ccccccc] shift[32] cpu[0]
<1>Unable to handle kernel paging request for data at address 0x00000000
<1>Faulting instruction address: 0xc01b5a98
<4>Oops: Kernel access of bad area, sig: 11 [#1]
<4>PREEMPT Xilinx Virtex
<4>Modules linked in:
<4>**bleep**: c01b5a98 LR: c0336aec CTR: 000007d0
<4>REGS: c0361eb0 TRAP: 0300   Not tainted  (2.6.33)
<4>MSR: 00029030 <EE,ME,CE,IR,DR>  CR: 44000044  XER: 20000000
<4>DEAR: 00000000, ESR: 00800000
<4>TASK = c034b310[0] 'swapper' THREAD: c0360000
<4>GPR00: 00000720 c0361f60 c034b310 c7809400 00000002 00000000 00000019 00000050
<4>GPR08: 00000007 c037f998 00000000 00000001 c7809400 ffff8aa8 07fd3600 07fc6ec8
<4>GPR16: 07fc6eec 07fc6ef8 07ead984 07ead884 07ead7d7 00000000 07fccce4 011a241f
<4>GPR24: 00000000 00000000 07fcd4d4 c0355058 00000000 000007d0 00000000 c7809400
<4>**bleep** [c01b5a98] csi_J+0x6c/0x1d4
<4>LR [c0336aec] con_init+0x120/0x280
<4>Call Trace:
<4>[c0361f60] [c01b5d98] reset_terminal+0x198/0x228 (unreliable)
<4>[c0361f80] [c0336aec] con_init+0x120/0x280
<4>[c0361fb0] [c0335ff8] console_init+0x40/0x5c
<4>[c0361fc0] [c0320814] start_kernel+0x1e8/0x2e8
<4>[c0361ff0] [c000225c] start_here+0x44/0xb0
<4>Instruction dump:
<4>81430054 81690000 80030018 7f8b1800 7c0a0050 541df87e 7d5e5378 419e0114
<4>57a0007f 7c0903a6 a01f006c 41820010 <b01e0000> 3bde0002 4200fff8 801f00d4
<4>---[ end trace 31fd0ba7d8756001 ]---
<0>Kernel panic - not syncing: Attempted to kill the idle task!
<4>Call Trace:
<4>[c0361da0] [c0007078] show_stack+0x44/0x16c (unreliable)
<4>[c0361de0] [c0036138] panic+0x94/0x180
<4>[c0361e30] [c003a574] do_exit+0x614/0x678
<4>[c0361e70] [c000cf70] kernel_bad_stack+0x0/0x4c
<4>[c0361e90] [c00128b8] bad_page_fault+0x90/0xd8
<4>[c0361ea0] [c000fa20] handle_page_fault+0x7c/0x80
<4>[c0361f60] [c01b5d98] reset_terminal+0x198/0x228
<4>[c0361f80] [c0336aec] con_init+0x120/0x280
<4>[c0361fb0] [c0335ff8] console_init+0x40/0x5c
<4>[c0361fc0] [c0320814] start_kernel+0x1e8/0x2e8
<4>[c0361ff0] [c000225c] start_here+0x44/0xb0
<0>Rebooting in 180 seconds..

-------------------------------------------------------

Second log:

-------------------------------------------------------

<6>Using Xilinx Virtex machine description
<5>Linux version 2.6.31 (bmarchand@edouard) (gcc version 4.2.2) #5 PREEMPT Tue May 11 15:55:14 EDT 2010
<7>Top of RAM: 0x8000000, Total RAM: 0x8000000
<7>Memory hole size: 0MB
<4>Zone PFN ranges:
<4>  DMA      0x00000000 -> 0x00008000
<4>  Normal   0x00008000 -> 0x00008000
<4>Movable zone start PFN for each node
<4>early_node_map[1] active PFN ranges
<4>    0: 0x00000000 -> 0x00008000
<7>On node 0 totalpages: 32768
<7>free_area_init_node: node 0, pgdat c0352d34, node_mem_map c037c000
<7>  DMA zone: 256 pages used for memmap
<7>  DMA zone: 0 pages reserved
<7>  DMA zone: 32512 pages, LIFO batch:7
<6>MMU: Allocated 1088 bytes of context maps for 255 contexts
<4>Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32512
<5>Kernel command line: console=ttyUL0 root=/dev/ram
<4>PID hash table entries: 512 (order: 9, 2048 bytes)
<6>Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
<6>Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
<6>Memory: 126192k/131072k available (3280k kernel code, 4720k reserved, 128k data, 133k bss, 160k init)
<6>Kernel virtual memory layout:
<6>  * 0xffffe000..0xfffff000  : fixmap
<6>  * 0xfde00000..0xfe000000  : consistent mem
<6>  * 0xfde00000..0xfde00000  : early ioremap
<6>  * 0xc9000000..0xfde00000  : vmalloc & ioremap
<5>Callingsched_init()
<6>NR_IRQS:512
<7>time_init: decrementer frequency = 300.000000 MHz
<7>time_init: processor frequency   = 300.000000 MHz
<6>clocksource: timebase mult[d55555] shift[22] registered
<7>clockevent: decrementer mult[4ccccccc] shift[32] cpu[0]
<5>Callinglocal_irq_enable()
<4>Console: colour dummy device 80x25
<4>Mount-cache hash table entries: 512
<6>NET: Registered protocol family 16
<1>Unable to handle kernel paging request for data at address 0x07eab8ac
<1>Faulting instruction address: 0xc002bf48
<4>Oops: Kernel access of bad area, sig: 11 [#1]
<4>PREEMPT Xilinx Virtex
<4>Modules linked in:
<4>**bleep**: c002bf48 LR: c002bf48 CTR: c002f8c4
<4>REGS: c7829ea0 TRAP: 0300   Not tainted  (2.6.31)
<4>MSR: 00021030 <ME,CE,IR,DR>  CR: 24000084  XER: 00000000
<4>DEAR: 07eab8ac, ESR: 00000000
<4>TASK = c781d490[2] 'kthreadd' THREAD: c7828000
<4>GPR00: 00000000 c7829f50 c781d490 c781d520 c781d490 00000000 004c4b40 00000000
<4>GPR08: 004c4b40 00000000 00000000 00000000 00000000 ffff8aa8 07fd2d00 07fc5b3c
<4>GPR16: 07fc5b60 07fc5b6c 07eab984 c0287414 00000000 000f4240 c781d61c c0340000
<4>GPR24: c781d508 000f4240 c781d620 c0338a38 c7828000 00000001 c0338a38 07eab884
<4>**bleep** [c002bf48] dequeue_task+0xa4/0xdc
<4>LR [c002bf48] dequeue_task+0xa4/0xdc
<4>Call Trace:
<4>[c7829f70] [c002c018] deactivate_task+0x40/0x60
<4>[c7829f80] [c027e658] schedule+0x21c/0x350
<4>[c7829fc0] [c004b9f4] kthreadd+0x158/0x164
<4>[c7829ff0] [c000f288] kernel_thread+0x4c/0x68
<4>Instruction dump:
<4>38000000 901f004c 80010024 bba10014 7c0803a6 38210020 4e800020 3d20c035
<4>80c93bd8 38640090 38a00000 4bfffec9 <813f0028> 7fc3f378 81290008 7fa5eb78
<4>---[ end trace 31fd0ba7d8756001 ]---
<6>note: kthreadd[2] exited with preempt_count 2
<1>Unable to handle kernel paging request for data at address 0x00000027
<1>Faulting instruction address: 0xc002bf48
<4>Oops: Kernel access of bad area, sig: 11 [#2]
<4>PREEMPT Xilinx Virtex
<4>Modules linked in:
<4>**bleep**: c002bf48 LR: c002bf48 CTR: 00000000
<4>REGS: c781fcb0 TRAP: 0300   Tainted: G      D     (2.6.31)
<4>MSR: 00021030 <ME,CE,IR,DR>  CR: 24022024  XER: 00000000
<4>DEAR: 00000027, ESR: 00000000
<4>TASK = c781d8a0[1] 'swapper' THREAD: c781e000
<4>GPR00: 00000000 c781fd60 c781d8a0 c781d930 c781d8a0 00000000 004c4b40 00000000
<4>GPR08: 0045271c 00000000 0008294d 00000000 00010529 ffff8aa8 07fd2d00 07fc5b3c
<4>GPR16: 07fc5b60 07fc5b6c 07eab984 c0287414 00000000 000f4240 c781da2c c0340000
<4>GPR24: c781d918 000f4240 c781da30 c0338a38 c781e000 00000001 c0338a38 ffffffff
<4>**bleep** [c002bf48] dequeue_task+0xa4/0xdc
<4>LR [c002bf48] dequeue_task+0xa4/0xdc
<4>Call Trace:
<4>[c781fd60] [ffff8aa8] 0xffff8aa8 (unreliable)
<4>[c781fd80] [c002c018] deactivate_task+0x40/0x60
<4>[c781fd90] [c027e658] schedule+0x21c/0x350
<4>[c781fdd0] [c027ef04] schedule_timeout+0x164/0x1c0
<4>[c781fe10] [c027eb30] wait_for_common+0xd8/0x1c4
<4>[c781fe50] [c00467f0] call_usermodehelper_exec+0x94/0xb8
<4>[c781fe80] [c016cff0] kobject_uevent_env+0x424/0x44c
<4>[c781fed0] [c01b73f4] device_add+0x3f8/0x5e8
<4>[c781ff20] [c01b76d4] device_create_vargs+0xc0/0xd8
<4>[c781ff50] [c01b772c] device_create+0x40/0x50
<4>[c781ff80] [c0321b30] vtconsole_class_init+0x9c/0x108
<4>[c781ffa0] [c0002378] do_one_initcall+0x40/0x1d4
<4>[c781ffd0] [c030c1e0] kernel_init+0xc4/0x130
<4>[c781fff0] [c000f288] kernel_thread+0x4c/0x68
<4>Instruction dump:
<4>38000000 901f004c 80010024 bba10014 7c0803a6 38210020 4e800020 3d20c035
<4>80c93bd8 38640090 38a00000 4bfffec9 <813f0028> 7fc3f378 81290008 7fa5eb78
<4>---[ end trace 31fd0ba7d8756002 ]---
<6>note: swapper[1] exited with preempt_count 2
<0>Kernel panic - not syncing: Attempted to kill init!
<4>Call Trace:
<4>[c781fb90] [c0006f44] show_stack+0x44/0x16c (unreliable)
<4>[c781fbd0] [c0032e9c] panic+0x9c/0x170
<4>[c781fc20] [c0036bc8] do_exit+0x560/0x660
<4>[c781fc60] [c000ce54] kernel_bad_stack+0x0/0x4c
<4>[c781fc90] [c0012468] bad_page_fault+0x90/0xd8
<4>[c781fca0] [c000f954] handle_page_fault+0x7c/0x80
<4>[c781fd60] [ffff8aa8] 0xffff8aa8
<4>[c781fd80] [c002c018] deactivate_task+0x40/0x60
<4>[c781fd90] [c027e658] schedule+0x21c/0x350
<4>[c781fdd0] [c027ef04] schedule_timeout+0x164/0x1c0
<4>[c781fe10] [c027eb30] wait_for_common+0xd8/0x1c4
<4>[c781fe50] [c00467f0] call_usermodehelper_exec+0x94/0xb8
<4>[c781fe80] [c016cff0] kobject_uevent_env+0x424/0x44c
<4>[c781fed0] [c01b73f4] device_add+0x3f8/0x5e8
<4>[c781ff20] [c01b76d4] device_create_vargs+0xc0/0xd8
<4>[c781ff50] [c01b772c] device_create+0x40/0x50
<4>[c781ff80] [c0321b30] vtconsole_class_init+0x9c/0x108
<4>[c781ffa0] [c0002378] do_one_initcall+0x40/0x1d4
<4>[c781ffd0] [c030c1e0] kernel_init+0xc4/0x130
<4>[c781fff0] [c000f288] kernel_thread+0x4c/0x68
<0>Rebooting in 180 seconds..

-------------------------------------------------------

 

These are typical logs I get,  the kernel always fails either in "csi_J" or "dequeue_task", whatever the configuration I give to the kernel.

Do someone hava an idea what could cause this issue?

Thanks

0 Kudos
7 Replies
Highlighted
Xilinx Employee
Xilinx Employee
5,732 Views
Registered: ‎09-10-2008

Hi,

 

It sounds like this EDK system might be very old and we certainly don't test on real old stuff. I know that's a pain.  There could be an issue with your old system and the newer kernels.

 

It gives you addresses where the problem occurred in the dump so you could use objdump to trace down the problem, slow but effective usually.

 

I would start by comparing your device tree to the one in the tree for the ML405, arch/powerpc/boot/dts/virtex405-ml405.dts. 

 

Look at the deltas and then if you need to, starting changing your system to match. I realize that sounds painful, but they key is to get a simple system baselined well, then you can go back to you desired system.

 

I would simply by cutting out any devices in the device tree that you don't need. You don't have to take them out of the h/w, just the device tree.

 

The 2.6.33 console output looked like there might be console issues based on the crash.  I'll paste in the console output from a nitely test run and the system.mhs to match that system.

 

Thanks,

John

 

 

Using Xilinx Virtex machine description
Linux version 2.6.33 (abq_iplib@xaqiptest41) (gcc version 4.0.0 (DENX ELDK 4.1 4.0.0)) #1 PREEMPT Wed May 12 22:44:26 MDT 2010
Zone PFN ranges:
  DMA      0x00000000 -> 0x00008000
  Normal   0x00008000 -> 0x00008000
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0: 0x00000000 -> 0x00008000
MMU: Allocated 1088 bytes of context maps for 255 contexts
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32512
Kernel command line: console=ttyS0 ip=192.168.0.3 init=/test/ml50x_test.sh root=/dev/nfs rw nfsroot=192.168.0.2:/nfsroots/ppc/autotest,tcp
PID hash table entries: 512 (order: -1, 2048 bytes)
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
Memory: 126076k/131072k available (3408k kernel code, 4836k reserved, 140k data, 128k bss, 168k init)
Kernel virtual memory layout:
  * 0xfffdf000..0xfffff000  : fixmap
  * 0xfde00000..0xfe000000  : consistent mem
  * 0xfde00000..0xfde00000  : early ioremap
  * 0xc9000000..0xfde00000  : vmalloc & ioremap
Hierarchical RCU implementation.
RCU-based detection of stalled CPUs is enabled.
NR_IRQS:512 nr_irqs:512
clocksource: timebase mult[2800000] shift[22] registered
Console: colour dummy device 80x25
Mount-cache hash table entries: 512
NET: Registered protocol family 16
PCI: Probing PCI hardware
bio: create slab <bio-0> at 0
vgaarb: loaded
Switching to clocksource timebase
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 4096 (order: 3, 32768 bytes)
TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
TCP reno registered
UDP hash table entries: 256 (order: 0, 4096 bytes)
UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
NET: Registered protocol family 1
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
ROMFS MTD (C) 2007 Red Hat, Inc.
msgmni has been set to 246
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
83e00000.serial: ttyS0 at MMIO 0x83e01003 (irq = 16) is a 16550
console [ttyS0] enabled
brd: module loaded
loop: module loaded
Device Tree Probing 'ethernet'
xilinx_lltemac fff00000.ethernet: MAC address is now  0: a:35:37:ad: 0
xilinx_lltemac fff00000.ethernet: XLlTemac: using DMA mode.
XLlTemac: Dma base address: phy: 0x84600100, virt: 0xc9028100
XLlTemac: buffer descriptor size: 32768 (0x8000)
XLlTemac: Allocating DMA descriptors with kmalloc
XLlTemac: (buffer_descriptor_init) phy: 0x7950000, virt: 0xc7950000, size: 0x8000
XTemac: PHY detected at address 7.
eth0: Dropping NETIF_F_SG since no checksum feature.
xilinx_lltemac fff00000.ethernet: eth0: Xilinx TEMAC at 0xFFF00000 mapped to 0xC9024000, irq=17
mice: PS/2 mouse device common for all mice
i2c /dev entries driver
TCP cubic registered
NET: Registered protocol family 17
eth0: XLlTemac: Options: 0x3fa
eth0: XLlTemac: allocating interrupt 19 for dma mode tx.
eth0: XLlTemac: allocating interrupt 18 for dma mode rx.
eth0: XLlTemac: speed set to 100Mb/s
eth0: XLlTemac: Send Threshold = 24, Receive Threshold = 4
eth0: XLlTemac: Send Wait bound = 254, Receive Wait bound = 254
IP-Config: Guessing netmask 255.255.255.0
IP-Config: Complete:
     device=eth0, addr=192.168.0.3, mask=255.255.255.0, gw=255.255.255.255,
     host=192.168.0.3, domain=, nis-domain=(none),
     bootserver=255.255.255.255, rootserver=192.168.0.2, rootpath=
Looking up port of RPC 100003/2 on 192.168.0.2
Looking up port of RPC 100005/1 on 192.168.0.2
VFS: Mounted root (nfs filesystem) on device 0:12.
Freeing unused kernel memory: 168k init
nRunning ML50X Test
mounting proc file system
mounting sys file system
cpu  : Virtex-4 FX
Thu Jan  1 00:00:20 UTC 1970
powerpc 405, kernel boot test only
test complete
starting a shell
:/>

0 Kudos
Highlighted
Visitor
Visitor
5,725 Views
Registered: ‎05-12-2010

Hi John,

 

  Thank you  for your answer. I spent arround a week trying to figure out this issue and already tried several of the suggestions you proposed without success.  However, probably because I asked for help, It began to work this morning. :)

What I did is merging again a dts generated by the EDK and using the loader linked into the simpleImage.  I'm not sure why yet, but It looks like my u-boot leave the processor in a state Linux doesn't like.

Otherwise, I now get up to the prompt using the filesystem available on the wiki.  So for now, I use the simpleImage instead of u-boot to go further with the drivers integration.  Everything is working fine except for the ethernet.

The core which is available to the processor is a plb_ethernet.  Is it still supported?  It is available in the kernel tree but I had a hard time compiling it.  It seems it is able to transmit but I think it doesn't receive yet.  I'm still looking into it.  I'll also check what happenned with u-boot.

 

Any hint is very appreciated.

Thanks a lot,

Bruno

 

 

0 Kudos
Highlighted
Visitor
Visitor
5,702 Views
Registered: ‎05-12-2010

Hi again,

  I finally managed to get the ethernet working.  It needed some modifications to the driver to make it compiles.  Then my first issue was the interrupt number.  It always works better with the right interrupt line.  More importantly, I modified the phy detection algorithm in the driver.  It sometimes erronously detected the phy at address 0 instead of address 1.  I'll try to find a way to confgure it elsewhere rather than trying to detect.

Are you interested that I submit a patch with my modifications?  If yes, what is the best way to submit it?

 

Thanks,

Bruno

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
5,693 Views
Registered: ‎09-10-2008

Hi Bruno,

 

Sounds like you've made good progress.

 

We don't support the older ethernet anymore as that solution is pretty old.  We support the ethernet lite and the LL TEMAC cores as the LL TEMAC really replaced the older core.

 

The newer drivers are using the phy layer in the kernel as they work better than doing it in the driver and work better across different phys.

 

I'm interested to know if this is a commercial product using this old system and design as it's useful information for us in our strategy.

 

Thanks.

0 Kudos
Highlighted
Visitor
Visitor
5,660 Views
Registered: ‎05-12-2010

Hi John,

 

  I'm not sure if the architecture will remain or not.  I'm a software engineer and I'm not incharge of the VHDL for the fpga.  I just take what the firmware team gives me. :)  I admit the bitstream I use is pretty old by now and I also saw almost all of the component of this system in the "EDK Obsoleted Cores" list.  However, it is not sure yet if the system will be converted to the new architecture as it uses a special memory controller for the mobile DDR.  That controller seems to be a pain to connect to any processor bus and now it works with the  PLB.  But to answer the question you asked, yes it is part of a commercial product.

 

By thw way, The systems is now fully functionnal, even with u-boot.  The issue was about the cache and two bits to sets in CCR0.  Some deja vu form another OS port to this platform.  I'll submit the patchs for my modifications.

 

Thanks for your help,

Bruno

EDK Obsoleted Cores

0 Kudos
Highlighted
Visitor
Visitor
5,659 Views
Registered: ‎05-12-2010

Patch for u-boot against git revision 9abed00d387dfabebd4372bcd0083c45ed1ea117

0 Kudos
Highlighted
Visitor
Visitor
5,658 Views
Registered: ‎05-12-2010

Patch for linux against git revision 17431547113100a3ae0a622b9f76ad17fb76eb56

0 Kudos