07-16-2020 04:14 PM
I have a baremetal AMP architecture (Zynq UltraScale+) that does not use OpenAMP. Here is a description of my processors:
- A53_0 (Core 0) to get interrupts from TTC for Ethernet
- A53_1 (Core 1) to get interrupts from Watchdog Timer, and a few other custom PL interrupts
- A53_2 (Core 2) to get interrupts from custom PL IP
For startup, Core 1 comes up first, sets up shared memory space, and then brings up Core 0 and Core 2.
My question is, how do I setup my GIC with this architecture? I have interrupts working for just Ethernet (Core 0), but I need to incorporate Cores 1 and 2. Currently, I only setup the GIC on Core 0, but obviously need to expand that. My current code for setting up interrupts on just one core is:
- XScuGic_LookupConfig
- XScuGic_CfgInitialize
- Xil_ExceptionRegisterHandler
- XScuGic_SetPriorityTriggerType
- XScuGic_InterruptMaptoCpu
- Xil_ExceptionEnable
- XScuGic_RegisterHandler
- XScuGic_EnableIntr
- XTtcPs_EnableInterrupts
Does each core need to have their own instance of the GIC? Do they each go through this entire process? I have been hearing that I cannot call CfgInitialize from each of the cores, because it will overwrite the configuration for the cores that have already setup the GIC.
I think that I can use my architecture to setup the GIC on Core 1 (first core up) and then just setup interrupts on Cores 0 & 2 - but there is not much out there for setting up baremetal AMP interrupts.
08-03-2020 05:16 AM
Update: Posting the solution that I came up with, since there is close to nothing available online for 2 (or more) cores all running baremetal.
The root of the problem is, to follow the programming sequence provided in documentation, you would think each core wants to setup the GIC and get a instance pointer using XScuGic_CfgInitialize. However, depending on the timing of the CPUs each calling this function, they will overwrite the settings in the distributor interface that were set by other CPUs (race conditions). Reading the Architecture Specification for the Arm Generic Interrupt Controller (v2 for Zynq Ultrascale+ MPSoC APU) each CPU has its own private "CPU Interface" registers, and shares the public distributor registers. So for the most part, the CPUs can each setup the GIC as if they were standalone (meaning not AMP architecture) except for the distributor interface.
The basic steps:
- One CPU (pick whichever one is up first, and you are considering "master core") runs the built in function XScuGic_CfgInitialize to setup it's CPU interface with the GIC, and the Distributor interface.
- Other CPU(s) run a modified version of XScuGic_CfgInitialize, where they setup the CPU interface, but do not touch the Distributor Interface
- Other CPU(s) setup the interrupts priority, trigger type, set the function handler (ISR callback function) and then enable the interrupt.
- The CPU that setup the Distributor Interface ("master CPU") calls XScuGic_InterruptMaptoCpu to setup the distributor registers such that the interrupt is sent to the correct CPU. There is alot of talk on the forums about how the XScuGic_InterruptMaptoCpu function doesn't work, but long story short that was fixed in one of the 2017 versions, I am using 2018.2 and it works properly.
The timing will look like this for 2 cores:
Core 1 (Master) Core 0 (Slave)
Xil_ExceptionEnable
XScuGic_SetPriorityTriggerType
XScuGic_RegisterHandler
XScuGic_EnableIntr
XTtcPs_EnableInterrupts
XScuGic_InterruptMaptoCpu
08-03-2020 05:16 AM
Update: Posting the solution that I came up with, since there is close to nothing available online for 2 (or more) cores all running baremetal.
The root of the problem is, to follow the programming sequence provided in documentation, you would think each core wants to setup the GIC and get a instance pointer using XScuGic_CfgInitialize. However, depending on the timing of the CPUs each calling this function, they will overwrite the settings in the distributor interface that were set by other CPUs (race conditions). Reading the Architecture Specification for the Arm Generic Interrupt Controller (v2 for Zynq Ultrascale+ MPSoC APU) each CPU has its own private "CPU Interface" registers, and shares the public distributor registers. So for the most part, the CPUs can each setup the GIC as if they were standalone (meaning not AMP architecture) except for the distributor interface.
The basic steps:
- One CPU (pick whichever one is up first, and you are considering "master core") runs the built in function XScuGic_CfgInitialize to setup it's CPU interface with the GIC, and the Distributor interface.
- Other CPU(s) run a modified version of XScuGic_CfgInitialize, where they setup the CPU interface, but do not touch the Distributor Interface
- Other CPU(s) setup the interrupts priority, trigger type, set the function handler (ISR callback function) and then enable the interrupt.
- The CPU that setup the Distributor Interface ("master CPU") calls XScuGic_InterruptMaptoCpu to setup the distributor registers such that the interrupt is sent to the correct CPU. There is alot of talk on the forums about how the XScuGic_InterruptMaptoCpu function doesn't work, but long story short that was fixed in one of the 2017 versions, I am using 2018.2 and it works properly.
The timing will look like this for 2 cores:
Core 1 (Master) Core 0 (Slave)
Xil_ExceptionEnable
XScuGic_SetPriorityTriggerType
XScuGic_RegisterHandler
XScuGic_EnableIntr
XTtcPs_EnableInterrupts
XScuGic_InterruptMaptoCpu
11-08-2020 11:08 PM
I found a bug in the function: s32 XScuGic_CfgInitialize in xscugic.c
The calculation of the Cpu_Id was wrong for CPUs > 1. So I changed it to the right calculation:
//u32 Cpu_Id = CpuId + (u32)1; // Here is the BUG!
u32 Cpu_Id = ((u32)0x1 << CpuId);
As the DistributorInit(InstancePtr, Cpu_Id); used this Cpu_Id the Distributor registers were configured for the wrong CPUs.
In newer versions (from v4.0 on) of xscugic.c the code is already changed a lot.
11-09-2020 12:59 PM
thank you so much for this. the xilinx documentation is vague and the online examples are old and/or useless.
turns out that commenting out the init dist call in Cpu1 and then adding a maptocpu for cpu1 in the cpu0 isr initialization did the trick.
very simple, but not at all clear from the Xilinx driver code or the trm.
thanks again.
ps using zynq 7020.