cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
maps-mpls
Mentor
Mentor
439 Views
Registered: ‎06-20-2017

Vitis debug of multiple processors not working

Is it possible to debug a Linux application in a Linux domain and an R5 baremetal application in an R5 domain all in same system project via Vitis 2020.1?

According to this, it should be:

For example, on a Zynq® UltraScale+™ MPSoC device, a Hello World standalone application on A53_0 and a Hello World 
application on R5_0 can be held in one system project if they are expected to run at the same time. A Hello World
standalone application on A53 and a Hello World application in Linux cannot be combined in one system project,
because these applications use the same A53 processors and cannot run simultaneously on them.

As an aside, and while the above quote doesn't say so, I have been able to simultaneously debug multiple linux applications in the same system project.

Additionally, this is from one of Xilinx's promotional slides about vitis

maps-mpls_0-1614825177579.png

 

And this is a direct quote from that slide notes:

The System Debugger is capable of debugging multiple processors, which is a requirement in today’s multiprocessor environments.

I agree.  It is a requirement.  And yet I am not having any luck.

If I have 3 Vitis system projects:

  • System project 1:  R5 domain only, requires mode pins to be JTAG. (Mode pins 0x0)  I can debug two applications (one per R5)
  • System project 2:  Linux domain only (mode pins 0xE).  USB->JTAG cable is a don't care.  I can simultaneously debug as many Linux applications as I want.
  • System project 3:  Linux domain and R5 domains.  <--- I am here, not having much luck after a few days troubleshooting, and not getting answers from my Xilinx FAEs who are a bit busy with another project.

In one experiment for system 3:  (modepins 0xE, jtag cable connected), I get:

Error while launching programs.  Cannot reset Cortex-R5 #0.  APB AP transaction error, DAP status 0x30000021

 More background:  The Embedded Linux is told it has 0x7000000 of DDR memory. 

 

maps-mpls_1-1614825947146.png

 

However, after I boot embedded Linux, and cat /proc/meminfo, I get this:

 

MemTotal:        2039208 kB
MemFree:         1979976 kB
MemAvailable:    1956456 kB
Buffers:            3512 kB
Cached:             9696 kB
SwapCached:            0 kB
Active:            15132 kB
Inactive:           4568 kB
Active(anon):       6552 kB
Inactive(anon):      136 kB
Active(file):       8580 kB
Inactive(file):     4432 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:          6548 kB
Mapped:             5960 kB
Shmem:               200 kB
KReclaimable:       7524 kB
Slab:              27468 kB
SReclaimable:       7524 kB

The 2039208 kB is a lot more than the 1835008 kB I expected.  Do I need to do something else in petaLinux-config to reserve memory for the R5 application (I did modify the R5 application's linker script, see below).

maps-mpls_2-1614826405824.png

 

The one thing that has me worried about the above is the memory region psu_r5_0_atcm_MEM_0.  This might be causing problems?

 

More background:

maps-mpls_3-1614826941498.png

maps-mpls_4-1614827031508.png

 

The Petalinux project works great on its own or via Vitis.  Using dynamic libraries, custom kernel module drivers, UIO Framework, XVC--the works. 

The R5 application works great on its own, as long as it is not in a system project with a linux domain.

I'm either missing something very fundamental, or what I want to do (debug an R5 application while debugging Linux applications) cannot be done.  If the later, please just be forthright about it.  If the former, please point me to an example that debugs in Vitis an R5 application with an embedded Linux application.

 

*** Destination: Rapid design and development cycles *** Unappreciated answers get deleted, unappreciative OPs get put on ignored list ***
0 Kudos
5 Replies
sadanan
Xilinx Employee
Xilinx Employee
390 Views
Registered: ‎10-21-2010

Hi @maps-mpls,

Do you cpu-idle feature turned on in kernel config. If so, can you turn it off. HW doesn't support powering of CPU cores when jtag debugger is connected

maps-mpls
Mentor
Mentor
365 Views
Registered: ‎06-20-2017

Thanks @sadanan .  I was not optimistic, but I just did a

petalinux-config -c kernel

  CPU Power Management ---> CPU Idle ---> [ ] CPU idle PM support

Rebuilt, tested with mode at 0xE (SD) and 0x0 (JTAG).  Neither worked.  The only way I can debug the R5 application is if it is in a stand alone domain in a system project that does not also have a Linux domain, and I used JTAG mode.  Then all is good.  But I cannot test any interactions between the standalone domain and the linux apps. 

Have you ever debugged a standalone application in Vitis that was in a system project that also had a linux domain?  Has anybody at XIlinx (I am sure somebody has, or it cannot be done).  If I knew this was not possible, I would develop some other custom/ad hoc debug techniques.  As it is now I am spending a lot of time.  And I am sure there is more than one problem (such as the location of the vectors, the Embedded Linux taking all of the memory even though I did a

petalinux-config

  Subsystem AUTO Hardware Settings --> Memory Settings -> System memory size (0x70000000)

when the XSA has it listed as 0x80000000.  The above reservation of memory seems to be ignored, or some additional modifications (perhaps to a system-user.dtsi) is also necessary, as well as the location of the R5's .vectors via the R5 application's linker script.  But to even get to the point of troubleshooting that I need the linux domain and the stand alone domain in the same system project in Vitis.

Hence, the

Error while launching programs.  Cannot reset Cortex-R5 #0.  APB AP transaction error, DAP status 0x30000021

seems to be the roadbloack, and turning off CPU Idle as you suggested unfortunately had no affect on this message.

 

Any other ideas?  (I really appreciate your attention to this issue).

 

*** Destination: Rapid design and development cycles *** Unappreciated answers get deleted, unappreciative OPs get put on ignored list ***
0 Kudos
maps-mpls
Mentor
Mentor
328 Views
Registered: ‎06-20-2017

@sadanan @nutang 

Just an update, I solved what would become a problem with the memory reserved for R50.  I had to edit a .dtsi file so that embedded linux did not claim it.

But still, with and without the CPU Idle enabled, it is still giving the DAP status 0x30000021 error message.  I think the CPU IDLE is a red herring.

What if any changes are necessary to boot.bif for the R50 application?  Do I need to create a dummy R5 standalone application (e.g., a main with while(1); ) and build it into the boot.bif so that it can be download it to the SD Card before I can start application debug in Vitis?  I also suspect some changes are necessary for interrupt routing in the application, and the location of the vector table for the standalone R50 application. 

Can you guys get in touch with somebody who has been through this before?  Do you have a way to look up DAP status 0x30000021?  All I found was this https://www.xilinx.com/support/answers/67623.html and that has to do with burst length.  Is there a problem with Vitis writing to the DDR that was reserved via .dtsi?

A pointer to a solution that describes the steps necessary for what every successful MPSoC/RFSoC project must get through to debug and develop standalone applications that works within a Vitis system project that has an embedded linux domain and a stand alone domain must  exist, or every such project must reinvent the wheel.

*** Destination: Rapid design and development cycles *** Unappreciated answers get deleted, unappreciative OPs get put on ignored list ***
0 Kudos
burleygreg
Visitor
Visitor
109 Views
Registered: ‎04-28-2021

I would like to say that I am having this exact same problem.  I have a system project with a linux(a53) domain, and a standalone (r5-0) domain.  I have made the same cpu_idle and memory adjustments in petalinux, and adjusted the linker script for the r5 memory.  I am using a simple hello_world app on both linux and standalone.

After booting linux from SD, I can launch the linux hello_world from vitis using linux-application-debug/tcf-agent (and it runs fine).  The standalone hello_world works fine on its own when launched from vitis using jtag.

But, I can't seem to figure out a way to launch and/or debug the r5 application from vitis, with a linux system booted from SD running on the a53 cores.  Is there a way to do this using Run Configuration or Debug Configuration?  Do I need to use xsdb?  Any pointers would be appreciated!

0 Kudos
stephenm
Moderator
Moderator
72 Views
Registered: ‎09-12-2007

You can add the R5 app to the BIF and connect to a running target in Vitis to debug your R5 app