UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Visitor hypertext
Visitor
9,757 Views
Registered: ‎11-11-2012

Zynq boot procedure - CPU0/1 WFE modes after BootROM

Jump to solution

Hi,

 

I'm trying to deeply understand the boot process of the Zynq family, with the ZedBoard as reference board.

In particular, UG585 says:

CPU 0 is in charge of starting code execution onCPU 1. The BootROM puts CPU 1 into the Wait for
Event mode.

Therefore, from what I understood, CPU 0 is always the core starting the First Stage Boot Loader, while CPU 1 is temporarily in the WFE mode (until CPU 0 wakes up CPU 1).

 

Reading through FSBL source code, I found the following boot procedure:

 

_boot:

#if XPAR_CPU_ID==0
/* only allow cpu0 through */
	mrc	p15,0,r1,c0,c0,5
	and	r1, r1, #0xf
	cmp	r1, #0
	beq	OKToRun
EndlessLoop0:
	wfe
	b	EndlessLoop0

#elif XPAR_CPU_ID==1
/* only allow cpu1 through */
	mrc	p15,0,r1,c0,c0,5
	and	r1, r1, #0xf
	cmp	r1, #1
	beq	OKToRun
EndlessLoop1:
	wfe
	b	EndlessLoop1
#endif

And I have some questions:

 

  1. Since XPAR_CPU_ID is defined as 0 (default value), it follows that CPU 0 branches to OKToRun and CPU 1 to EndlessLoop0 (basically WFE). However, why is this control needed since CPU 1 is already into the WFE mode (hence CPU 1 does never execute this code)?
  2. If I set XPAR_CPU_ID value to 1, CPU 1 branches to OKToRun and CPU 0 to EndlessLoop1. Again, since CPU 1 is in the WFE mode after the BootROM procedure has been carried out, why is there the possibility to change the value of XPAR_CPU_ID?

I guess I'm missing something important...

 

 

Thanks in advance,

Pierpaolo

0 Kudos
1 Solution

Accepted Solutions
Xilinx Employee
Xilinx Employee
18,407 Views
Registered: ‎02-01-2008

Re: Zynq boot procedure - CPU0/1 WFE modes after BootROM

Jump to solution

Your summary is correct. Good job at recognizing that the fsbl contains the _boot that is used by the cpu1 reset in the cpu0 app.

 

I'll see if I can answer your questions.

1. You could continue to use the fsbl vectors but you would need to modify cpu0 app bsp. In the startup code, the base address of the vector table is written into a register in the apu. You would have to prevent that from happening. The easiest solution was just to rely on cpu0 app bsp and only use the reset vector of the fsbl. There is no way to change the reset vector from 0x00000000 when cpu1 is reset.

 

2. It doesn't necessarily execute the same startup code because you could use a different bsp for cpu0 startup. For example changing the translation table.

 

3. When I originally released the xapp, I used sev. I believe I left it in the cpu0 app and commented it out. Other people wanted to learn how to reset/restart cpu1 so that's the only reason I used the reset mechanism.

 

4. You could comment out that remap. It is old code and was used in the early days of the example design with freertos running with petalinux. The non-cached virtual address space was used to comunicate between the two OSs. I don't use it but left it in so that bsp modifications were minimized.

0 Kudos
5 Replies
Highlighted
Xilinx Employee
Xilinx Employee
9,742 Views
Registered: ‎02-01-2008

Re: Zynq boot procedure - CPU0/1 WFE modes after BootROM

Jump to solution

You have a pretty good understanding. The bootrom first prepairs a wfe loop at the top of ocm memory around 0xffffff30. Then cpu1 is sent to that wfe loop. You can see this behaviour by connecting to a zynq device using sdk and attach to a running target. Then pause cpu1. You will then be able to single step through the wfe loop at the top of memory. You will notice that it jumps to the address stored at 0xfffffff0 and the content of that address is the beginning of the wfe loop.

 

The code that you are looking at is from the BSP for the fsbl. When a bsp is created for cpu0, then that 'catch' code in boot.S will allow cpu0 through but catch cpu1 and send cpu1 into a wfe loop found in boot.S. This is done because cpu1 should never be running a bsp that was compiled for cpu0.

 

If you create a bsp for cpu1, then the catch changes. Cpu0 will end up in the wfe loop within boot.S and cpu1 would be allowed to continue running the bsp and then finally jumping to main().

 

No one has access to the bootrom code (except for a select few) but I have always guessed that the bootrom might have simular 'catch' code as boot.S. But instead, it checks what cpu the code is running on, and if it is cpu1, then cpu1 will continue running code that writes the wfe loop at the top of ocm and then jumps to it. And if it is cpu0, then it reads the mode registers and starts the process of minor initialization and then loading a program from the boot device and jumping to it. This program is normally the fsbl. Now if cpu0 reads the mode registers and determines that jtag mode is set, then cpu0 will prepair its own wfe loop at the top of memory and jump to it. You can once again see this using sdk by configuring for jtag mode, use sdk to connect to a running target, then pause cpu0. Single stepping cpu0 will show you its wfe loop at high ocm memory but below the wfe loop being ran by cpu1.

 

The reason I make the above statement is because I don't think the bootrom actually has cpu0 taking control of cpu1. But either way, the outcome following the bootrom running is cpu1 running a wfe loop in high ocm.

0 Kudos
Xilinx Employee
Xilinx Employee
9,739 Views
Registered: ‎02-01-2008

Re: Zynq boot procedure - CPU0/1 WFE modes after BootROM

Jump to solution

BTW if you look at the latest custom version of the bsp for xapp1079 (on wiki.xilinx.com) you will notice that I changed this 'catch' code to give a little bit more control to send either cpu somewhere else. I have two address locations at the end of the vector table (because vector table is at a known location, look at file asm_vectors.S). Depending on if the bsp is created for cpu0 or cpu1, the two address locations hold the address of Endlessloop0 or OkToRun.

 

The modified 'catch' code looks like:

 

_boot:

/* Test which processor is running and jump to the catch address */
    mrc    p15,0,r1,c0,c0,5
    and    r1, r1, #0xf
    cmp    r1, #0
    bne    NotCpu0
    ldr    r0, =_cpu0_catch
    b cpuxCont
NotCpu0:
    cmp    r1, #1
    bne    EndlessLoop0
    ldr    r0, =_cpu1_catch
    b cpuxCont
EndlessLoop0:
    wfe
    b    EndlessLoop0

/* Jump to address pointed to by cpux_catch */
cpuxCont:
    ldr lr, [r0]
    bx    lr


OKToRun:

0 Kudos
Visitor hypertext
Visitor
9,728 Views
Registered: ‎11-11-2012

Re: Zynq boot procedure - CPU0/1 WFE modes after BootROM

Jump to solution

Thanks for the reply!

 

I have some other questions, following the example in XAPP1079.

If I understand it correctly, the boot procedure (after BootROM) is the following (I tried to summarize it a bit):

  1. The FSBL is loaded in memory (OCM) at address 0x0;
  2. The execution starts with the first instruction at 0x0, which corresponds to the first element of the vector table (reset handler), which is a branch to the _boot assembly routine (contained into the BSP);
  3. The assembly routine configures CPU0 and then branches to the main() procedure, which loads ELF files in RAM and the bitstream configuration.
  4. The FSBL branches to the entry point of the first partition (app_cpu0), which is 0x100000. However, 0x100000 is again a branch to the assembly _boot routine (this time of app_cpu0). After the boot routine, the execution goes to the main() procedure.
  5. The main procedure of app_cpu0 configures the entry point of app_cpu1 by writing its address (0x02000000) in the second entry following the FSBL vector table (address 0x00000024). Then, app_cpu0 resets CPU1 and continues with its execution;
  6. CPU1 wakes up, and starts from the first instruction located in 0x0, which is always a branch to the FSBL boot routine.
  7. Since this time CPU1 is executiing the FSBL _boot routine, and since _cpu1_catch value does not point to the EndlessLoop anymore, CPU1 branches to 0x02000000, which is again a branch to the app_cpu1 _boot routine. Since app_cpu1 has been configured with USE_AMP=1, its _boot routine skips some instruction. After the _boot routine, CPU1 executes the main() procedure.

My questions are the following:

  1. Looking inside app_cpu0.elf file, I see that app_cpu0 defines the entire vector table (and the exceptions handlers) before its _boot procedure. My question is: why the entry point of the app_cpu0 is not the first instruction of its _boot procedure? What is the purpose of RE-defining the vector table and the exceptions handlers, since they have been already defined (by the FSBL) starting at address 0x0? (The same question applies to app_cpu1, since its binary image is made up using the same scheme of app_cpu0)
  2. Based on the same doubt, why the _boot routine of app_cpu0 has to be executed, since it contains the same operations performed during the execution of the FSBL _boot routine?
  3. Why CPU1 is awakened by resetting it, instead of performing the SEV procedure? (I suppose it's because CPU1 has not been configured yet by the FSBL)
  4. Side question: in case of app_cpu1, its _boot routine maps virtual addresses 0x20000000 to 0x00000000. Why it does that?

 

Thanks again,

Pierpaolo

0 Kudos
Xilinx Employee
Xilinx Employee
18,408 Views
Registered: ‎02-01-2008

Re: Zynq boot procedure - CPU0/1 WFE modes after BootROM

Jump to solution

Your summary is correct. Good job at recognizing that the fsbl contains the _boot that is used by the cpu1 reset in the cpu0 app.

 

I'll see if I can answer your questions.

1. You could continue to use the fsbl vectors but you would need to modify cpu0 app bsp. In the startup code, the base address of the vector table is written into a register in the apu. You would have to prevent that from happening. The easiest solution was just to rely on cpu0 app bsp and only use the reset vector of the fsbl. There is no way to change the reset vector from 0x00000000 when cpu1 is reset.

 

2. It doesn't necessarily execute the same startup code because you could use a different bsp for cpu0 startup. For example changing the translation table.

 

3. When I originally released the xapp, I used sev. I believe I left it in the cpu0 app and commented it out. Other people wanted to learn how to reset/restart cpu1 so that's the only reason I used the reset mechanism.

 

4. You could comment out that remap. It is old code and was used in the early days of the example design with freertos running with petalinux. The non-cached virtual address space was used to comunicate between the two OSs. I don't use it but left it in so that bsp modifications were minimized.

0 Kudos
Visitor hypertext
Visitor
9,714 Views
Registered: ‎11-11-2012

Re: Zynq boot procedure - CPU0/1 WFE modes after BootROM

Jump to solution
Thanks a lot, all my doubts are now cleared :)

Have a good day,
Pierpaolo
0 Kudos