AMP - Linux and bare-metal application

[ewong3]

I have configured Linux to release CPU1 and load a bare metal application.  

Linux is using the upper 256MB of DDR RAM:

ps7_ddr_0: memory@0 {
        device_type = "memory";
        reg = <0x10000000 0x10000000>;
    } ;

This is the device tree entry that loads the firmware for CPU1 via the zynq_remoteproc module.   

The firmware is using 32MB of the lower 256MB of DDR RAM available to it.

    test: remoteproc-test@0 {
        compatible = "xlnx,zynq_remoteproc";
        reg = < 0x100000 0x1F00000 >;
        interrupt-parent = <&ps7_scugic_0>;
        interrupts = < 0 37 4 0 38 4 >;
        firmware = "cpu1app.elf";
        ipino = <8>;
        vring0 = <2>;
        vring1 = <3>;
    } ;

On startup, this happens:

[    2.698709] udevd[569]: starting version 182
[    2.815569] CPU1: shutdown
[    2.819163] BUG: using smp_processor_id() in preemptible [00000000] code: udevd/572
[    2.826797] caller is set_ipi_handler+0x14/0x58
[    2.831252] CPU: 0 PID: 572 Comm: udevd Not tainted 3.14.2-xilinx #1
[    2.837639] [<800147b4>] (unwind_backtrace) from [<800116cc>](show_stack+0x10/0x14)
[    2.845344] [<800116cc>] (show_stack) from [<8040e040>](dump_stack+0x80/0xc0)
[    2.852527] [<8040e040>] (dump_stack) from [<8020a584>](debug_smp_processor_id+0xe8/0xec)
[    2.860785] [<8020a584>] (debug_smp_processor_id) from [<80012f30>](set_ipi_handler+0x14/0x58)
[    2.869470] [<80012f30>] (set_ipi_handler) from [<7f014408>](zynq_remoteproc_probe+0x1a8/0x3c4 [zynq_remoteproc])
[    2.879804] [<7f014408>] (zynq_remoteproc_probe [zynq_remoteproc])from [<80255de0>](platform_drv_probe+0x18/0x4c)
[    2.890206] [<80255de0>] (platform_drv_probe) from [<80254608>](driver_probe_device+0x7c/0x21c)
[    2.898973] [<80254608>] (driver_probe_device) from [<80254878>](__driver_attach+0x8c/0x90)
[    2.907396] [<80254878>] (__driver_attach) from [<80252c14>](bus_for_each_dev+0x6c/0xa0)
[    2.915563] [<80252c14>] (bus_for_each_dev) from [<80253e74>](bus_add_driver+0x148/0x1f0)
[    2.923783] [<80253e74>] (bus_add_driver) from [<80254e74>](driver_register+0x78/0xf8)
[    2.931782] [<80254e74>] (driver_register) from [<8000881c>](do_one_initcall+0xf8/0x154)
[    2.939945] [<8000881c>] (do_one_initcall) from [<8007c3ec>](load_module+0x1978/0x1ee0)
[    2.948012] [<8007c3ec>] (load_module) from [<8007caac>](SyS_finit_module+0x68/0x78)
[    2.955828] [<8007caac>] (SyS_finit_module) from [<8000e3e0>](ret_fast_syscall+0x0/0x30)
[    3.016176]  remoteproc0: 100000.remoteproc-test is available
[    3.021844]  remoteproc0: Note: remoteproc is still under development and considered experimental.
[    3.119806]  remoteproc0: THE BINARY FORMAT IS NOT YET FINALIZED,and backward compatibility isn't yet guaranteed.

What is "BUG: using smp_processor_id() in preemptible [00000000] code: udevd/572" error?  Do I have wrong ipino?

When I use other ipino numbers, in addition to the above error, I get errors like this:

CPU0: IPI handler 0x6 already registered to ipi_cpu_stop
zynq_remoteproc 1fe00000.remoteproc: IPI handler already registered

For different values of ipino, I get different functions that the ipino is registered to.  With ipino 8, I just get the first error.  

Is the ipino causing the first error too?  If so, how do I determine what is the proper ipino to use?  The bare metal ELF for cpu1 just toggles an LED.  

[ewong3]

Following this blog post: http://henryomd.blogspot.com/2015/02/zynq-amp-linux-on-cpu0-and-bare-metal.html, I got further:

[ 2.718247] udevd[572]: starting version 182
[ 2.845012] CPU1: shutdown
[ 2.848639] BUG: using smp_processor_id() in preemptible [00000000] code: udevd/575
[ 2.856266] caller is set_ipi_handler+0x14/0x58
[ 2.860727] CPU: 0 PID: 575 Comm: udevd Not tainted 3.14.2-xilinx #1
[ 2.867097] [<800147b4>] (unwind_backtrace) from [<800116cc>] (show_stack+0x10/0x14)
[ 2.874816] [<800116cc>] (show_stack) from [<8040e040>] (dump_stack+0x80/0xc0)
[ 2.882001] [<8040e040>] (dump_stack) from [<8020a584>] (debug_smp_processor_id+0xe8/0xec)
[ 2.890255] [<8020a584>] (debug_smp_processor_id) from [<80012f30>] (set_ipi_handler+0x14/0x58)
[ 2.898947] [<80012f30>] (set_ipi_handler) from [<7f014458>] (zynq_remoteproc_probe+0x1a8/0x460 [zynq_remoteproc])
[ 2.909275] [<7f014458>] (zynq_remoteproc_probe [zynq_remoteproc]) from [<80255de0>] (platform_drv_probe+0x18/0x4c)
[ 2.919683] [<80255de0>] (platform_drv_probe) from [<80254608>] (driver_probe_device+0x7c/0x21c)
[ 2.928444] [<80254608>] (driver_probe_device) from [<80254878>] (__driver_attach+0x8c/0x90)
[ 2.936874] [<80254878>] (__driver_attach) from [<80252c14>] (bus_for_each_dev+0x6c/0xa0)
[ 2.945025] [<80252c14>] (bus_for_each_dev) from [<80253e74>] (bus_add_driver+0x148/0x1f0)
[ 2.953257] [<80253e74>] (bus_add_driver) from [<80254e74>] (driver_register+0x78/0xf8)
[ 2.961257] [<80254e74>] (driver_register) from [<8000881c>] (do_one_initcall+0xf8/0x154)
[ 2.969415] [<8000881c>] (do_one_initcall) from [<8007c3ec>] (load_module+0x1978/0x1ee0)
[ 2.977492] [<8007c3ec>] (load_module) from [<8007caac>] (SyS_finit_module+0x68/0x78)
[ 2.985299] [<8007caac>] (SyS_finit_module) from [<8000e3e0>] (ret_fast_syscall+0x0/0x30)
[ 3.038436] remoteproc0: 100000.remoteproc-test is available
[ 3.044105] remoteproc0: Note: remoteproc is still under development and considered experimental.
[ 3.129696] remoteproc0: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
[ 3.253705] remoteproc0: firmware_loading_complete
[ 3.259125] remoteproc0: powering up 100000.remoteproc-test
[ 3.278338] remoteproc0: Booting fw image cpu1app.elf, size 66576
[ 3.314648] remoteproc0: remote processor 100000.remoteproc-test is now up 

I still get the error "BUG: using smp_processor_id() in preemptible [00000000] code" and stack trace though. And the LED that cpu1 app is suppose to toggle isn't doing so. Is this error one that will prevent cpu1 application from fully running?

To get this far, I had to modify the zynq_remoteproc module as indicated in the blog post and to add a resource table section in the cpu1app elf. I didn't modify boot.S to mark DDR sections as reserved/cached though. I may try that next, but it seems like a lot of off-the-cuff hacking to get Linux to load an elf to CPU1. Is there a more standard way to get this configured and working?

[korcek]

What is your progress? Is it already running? We have to implement AMP by our own finnaly. If you are interested, you can try it.

[oska874]

You  should disable preempt before set ipi handler and enable preempt after it.