using both CPU cores on Zynq?

[pvranade]

Hi,

Has anyone tried to use both cpu cores of zynq?

I did not find any example from Xilinx, only related information

in "Zynq-7000 EPP Technical Reference Manual UG585 (v1.2)".

It says that cpu0 must do these two things:

1. Write the address of the application for CPU1 to 0xFFFFFFF0.
2. Execute the SEV instruction to cause CPU1 to wake up and

jump to the application.

Found a "dual cpu demo" example on ARM web-site:

http://www.arm.com/files/pdf/ZC702_DS5_2.pdf

However, the attached source code doesn't do any of the

above two actions. How can the second cpu start then?

Regards,

Pramod Ranade

[austin]

p,

The busybox linux that ships with the zc702 uses both cores.  Do a top, and see them both running.

If you execute two commands, and they have no dependencies, they will be each assigned to a core.

[barriet]

There is also some useful information (SMP vs AMP) here:

http://wiki.xilinx.com/zynq-linux

Cheers,

bt

[pvranade]

Thanks Austin and Barriet.

I will try to understand by looking into source code

of Linux...biggest problem is to locate the right c or asm file...

Maybe I willl grep for SEV in *.c and *.s.

I am really interested in using 2 CPUs with bare metal.

Or maybe one core with Linux and other bare metal.

Regards,

Pramod Ranade

[johnmcd]

There are a few things that you have to watch out for. If you are planning on running either linux/baremetal or baremetal/baremetal, you have to be carefull of any shared resources. A big one is L2 cache and if you plan on handling interrupts, theres also the ICD.

For linux/baremetal, it's easiest to configure linux as SMP with CPU_MAX=1.

If you plan on handling interrupts from the PL on only one cpu, you can bypass the ICD and just use that cpu's interrupt controller.

[pvranade]

Thank you j, for the suggestions.

I was basically trying with baremetal/baremetal.

First I used "hello world" example for CPU0, it

works fine. Then I added another "hello world",

but now for CPU1. Then I edited it's main() and

removed the initialization part and also the

printf("hello world"). Instead, I simply declare

a pointer to some memory location. And in an

infinite loop, I keep incrementing the value

pointed by that pointer.

The "hello world0" on the other hand has been

modified to write start address of the "hello world1"

to location 0xFFFFFFF0 and then a simple "SEV".

Then the program reads the same memory

location (which is being incremented by CPU1)

and prints it's value to UART. I was expecting that

the value will be seen increasing...but I see it

constant.

For the shared memory location, I tried 0x00700000

which is in DDR RAM. I also tried 0xFFFFE000 which

is in OCM. Neither worked...

I suspect, CPU0 needs to something more than just

writing CPU1 start address to 0xFFFFFFF0 and then

issuing a SEV. But I could not find any more information.

Nor can I find any example that uses both CPU cores.

Regards,

Pramod Ranade

[johnmcd]

Fire me a a private message with your email and I will forward you an example of standalone/standalone. Keep in mind that this example is not supported by Xilinx so you will be on your own.

It contains a modified BSP to bypass some code during boot on cpu1 (bypass L2 stuff). It also contains a modified FSBL that supports multiple elf file loading. cpu0 does the writing of 0xfffffff0. Also, cpu0 and cpu1 remap all ocm to high and disable cache on ocm.

[johnmcd]

BTW: are you using 14.1 or 14.2 ver of the tools?

[pvranade]

Thanks John.

I am using 14.2.

How do I "Fire you a private message" ?

Here is my email address:

pramod.ranade AT atdc.iitkgp.ernet.in

Regards,

Pramod Ranade

[pvranade]

John,

I have sent you private message.

Meanwhile, I somehow made it work - using the hints you passed me:

> It also contains a modified FSBL that supports multiple elf file loading.

I had already tweaked the FSBL for this. And I had confirmed that both

ELF files really get loaded. I loaded ELF for CPU1 at 0xFFFF0000.

Suitably changed CPU1 linker script settings.

> It contains a modified BSP to bypass some code during

> boot on cpu1 (bypass L2 stuff)

To do so, I copied boot.S from libsrc to src; and added this line in it:

#define    USE_AMP            1

But still it did not work. Finally, I wrote the address of main (of CPU1)

at address 0xFFFFFFF0 - thus entirely bypassed the startup code for CPU1.

Of course, this is not the right way, but a desperate step to start the second CPU.

And then it worked!

Nevertheless, please send me the example - hope it will show me the

"proper way" to start and use the second CPU (standalone/standalone).

Regards,

Pramod Ranade

[alex_sag]

to johnmcd

You wrote to pvrane:

10-02-2012 05:38 AM  

Fire me a a private message with your email and I will forward you an example of standalone/standalone. Keep in mind that this example is not supported by Xilinx so you will be on your own.

It contains a modified BSP to bypass some code during boot on cpu1 (bypass L2 stuff). It also contains a modified FSBL that supports multiple elf file loading. cpu0 does the writing of 0xfffffff0. Also, cpu0 and cpu1 remap all ocm to high and disable cache on ocm.

1. Can you send me this FSBL, BSP and example of programm?

    My e-mail is  alex_sag@live.ru

2. Wiil be Xilinx supported   "using both CPU cores on Zynq"?

Thanks,

Alex Skryglukov.

[alex_sag]

Fire me a a private message with your email and I will forward you an example of standalone/standalone. Keep in mind that this example is not supported by Xilinx so you will be on your own.

It contains a modified BSP to bypass some code during boot on cpu1 (bypass L2 stuff). It also contains a modified FSBL that supports multiple elf file loading. cpu0 does the writing of 0xfffffff0. Also, cpu0 and cpu1 remap all ocm to high and disable cache on ocm.

1. Can you send me this FSBL, BSP and example of programm?

    My e-mail is  alex_sag@live.ru

2. Wiil be Xilinx supported   "using both CPU cores on Zynq"?

Thanks,

Alex Skryglukov.

[hguerard]

Hi,

I am currently using a baremetal/baremetal application. The custom bootloader writes the PC of the CORE1 to execute at address 0xFFFFFFF0 and issues the SEV instruction.

When booting from the SD card, everything works as expected. But, when using the JTAG, somehow the CORE1 is waiting (waiting for SEV) at 0x6C instead of 0xFFFFFF00. When the custom bootloader issues the SEV instruction, the CORE1 starts executing code at 0x6C which will lead to an exception (since there is nothing there). Another problem I found using the JTAG, there is nothing at address 0xFFFFFF00. There should be a minimalistic bootstrap application (refer to http://www.xilinx.com/support/answers/46911.htm)

In order to run my application from the JTAG, the workaround I used is to connect to the CORE1 (via XMD) and configure his PC for the base address of my custom bootloader and then, start the CORE1. From there, everything works!

I am using EDK 14.2.

Thanks

[simpleart]

to johnmcd

Can you send me this FSBL, BSP and example of program?
My e-mail is jh.park@atusguru.com

Thanks

[johnmcd]

Look at boot.s in the standalone bsp. It sounds like you didn't compile the bsp for the second cpu. Verify your bsp settings.

[hguerard]

Hi johnmcd,

Your message is destined to who ?

Thanks

[johnmcd]

Sorry, the message was destined to you, hguerard regarding SEV at 0x6C.

The standalone bsp has a catch that halts the processor if the bsp is running on the wrong cpu. SEV at 0x6C sounds like that catch.

[aridai]

to johnmcd

I am troubled with this problem.
Could you send me example of program containing modified FSBL,BSP?

I am using EDK 14.2.

My e-mail address is arima@ibextech.jp

Thanks in Advance

[wietzjoubert]

To johnmcd

I'm having issues with starting up the second cpu in a standalone application.
Could you please send me the example source code.

I am using EDK 14.3

My e-mail address is:

wietz.joubert@cassidian-optronics.com

Thanks

Wietz

[wietzjoubert]

Hello, finally success. I implemented the following instructions:

http://www.xilinx.com/support/answers/51956.htm

There was however one additional thing. That was removing the 'SkipPartition = 0;'  in the main function of the FSBL. This would cause it to never load the second .elf file.

After which I loaded the address 0xfffffff0 with the executable address and then sending the SEV command. I implemented this towards the end of the main function in the FSBL.

I used these functions

Xil_Out32(0xfffffff0, 0x00200000);

__asm__ __volatile__("SEV\n");

Where 0x00200000 was the address I used in the linker script for the second CPU and compiling the .elf

[ajaitx]

Hi,

Can you please send the example FSBL and BSP code to use both CPU cores on zynq?

My mail ID: ajai.tx@mistralsolutions.com

Regards,

Ajai

[pvranade]

Ajai,

Link to download complete project folder is in this post.

It contains FSBL, BSP etc. for using Linux and FreeRTOS

on core0 and core1 respectively.

Regards,

Pramod Ranade

[ajaitx]

Hi Pramod,

Thanks for your reply.

But my requirement is to use 2 CPUs with 'freeRTOS + freeRTOS' or 'xilkernel + xilkernel' application. So i followed the modification of FSBL from this below link. http://www.xilinx.com/support/answers/51956.htm.

But its only loading one elf file in core0. core1 is not executing the second elf. 

In this forum, some posts mensioned the working code of both zynq cores using bare metal.

So kindly help me on this.

Regards,

Ajai

[pvranade]

Ajai,

My project already includes correct version of FSBL - which

loads both elf files.

Further, check latest from Xilinx - i.e. 14.4. It may already

have improved the FSBL to load both ELF files. I am not

sure about this, but seem likely - because the "correct"

FSBL in my project was given to me by a Xilinx employee.

Regards,

Pramod Ranade

[ajaitx]

Hi Pramod,

By using this latest FSBL, is it possible to load multiple elf files in JTAG boot mode?  like build multiple application in SDK and using 'Launch on hardware' method in SDK?

Or i need to copy the elf files to QSPI flash/ MMC and select mmc/flash boot mode?

Regards,

Ajai

[pvranade]

Sorry Ajai, I always used QSPI flash or SDCard.

Not sure if it works with JTAG.

Regards,

Pramod Ranade

[ajaitx]

Hi,

Can you please share your custom bootloader code?

Also please share,

1. how to connect to core1 via XMD? 

2. Loading an address to core1 PC via XMD?

Thanks,

Ajai

[johnmcd]

Sorry, I haven't been on the forum for a while.

An xapp1078 has been released that describes running linux on cpu0 and bare-metal on cpu1. It can be found here.

This xapp includes a modified FSBL for loading multiple elfs, info for connecting to cpu1 via xmd, starting cpu1 outside of xmd etc.

Another xapp is on the way for bare-metal on cpu0 and bare-metal on cpu1. It will use the same FSBL as xapp1078. The primary difference between linux/baremetal and baremetal/baremetal is that main() for cpu0 initializes the ICD in order for cpu1 to service interrupts from the PL and cpu0 starts cpu1. The FSBL will have already looked after downloading the elf for cpu1.

[pvranade]

Thanks John.

You have answered to all questions from ajaitx and other users.

Regards,

Pramod Ranade

[ajaitx]

Hi John,

Thanks for your support.

I tried the Linux/baremetal AMP solution, its working fine.

I am also looking for bare-metal in cpu0 and bare-metal in cpu1 AMP solution in JTAG boot mode. So for starting the CPU1 from CPU0 in JTAG boot mode, i did the following changes

In Jtag boot mode, after the reset the CPU1 pc is loading 0x6c instead of 0xfffffff0. (http://www.xilinx.com/support/answers/47567.htm).

To overcome this, i did the workaround as mensioned in http://www.origin.xilinx.com/support/answers/46911.htm

I connected to CPU1 via XMD console and written the below instructions in location 0xFFFFFF00 and cpu1 PC loaded with 0xFFFFFF00 and started the CPU1 with 'run' command from XMD console.

0xFFFFFF00: mvn r0, #15
0xFFFFFF04: mov r1
0xFFFFFF08: str r1, [r0]
0xFFFFFF0C: wfe
0xFFFFFF10: ldr r2, [r0]
0xFFFFFF14: cmp r2, r1
0xFFFFFF18: beq 0xc
0xFFFFFF1C: mov pc

So the CPU1 is waiting for an event. 

And then downloded the CPU1 helloworld application binary to DDR location 0x30000000.

Then added the below code in the main() function of CPU0 helloworld application.

Xil_Out32(0xfffffff0, 0x30000000);

__asm__ __volatile__("SEV\n");

Then started  the CPU0 helloworld application through JTAG. But the CPU1 is not executed the elf loaded in 0x30000000.

The SEV instruction from CPU0 doesn't started the CPU1 execution.

Can you please advise me on this?

Also please send me the xapp for bare-metal on cpu0 and bare-metal on cpu1, when its ready.

Regards,

Ajai

[johnmcd]

Instead of booting in JTAG mode, have you tried booting in another mode? For example, I always boot from SD but I don't plug the SD card in, so the bootrom will fallback to jtag mode. This way, you don't have the startup problem of CPU1 as outlined in the AR you pointed to.

How are you downloading the app for cpu1? In your description, it sounds like you are initializing 0xffffff00 then starting cpu1 via xmd. Then you mention you download the app for cpu1 at 0x30000000. You should do this the other way around. Download to 0x3.... then start cpu1 with 'con 0xffffff00'. From xmd, you can stop and do single stepping, so after running cpu1, try stopping it, and single stepping through the 0xfff.... code. During the single stepping, try writing 0x3.... to 0xfffffff0.

A few debug tips:

• use mrd_phy or mwr_phy to make sure cache isn't getting in the way of mem writes
• use chipscope to monitor the PS7 event bus or create events. The VIO core is great for this since you only need to know if there was activity and don't need to see how many clocks a signal is active