I tried to restrict the memory size in for the Linux side in petalinux to 0x6fffffff. This did not work so well because there was lots of crashing and it turns out that petalinux is not creating a memory node in the DTS with the correct size so I added it manually.
I am not using openAMP, we have implented message queue between the APU<->RPU in the PL.
So I was just rebuilding the PL and happened to see this message from vivido go past:
INFO: [PSU-0] Address Range of DDR (0x7ff00000 to 0x7fffffff) is reserved by PMU for internal purpose.
OK, this seems like a problem since this is in the memory area I am using for RPU0. Since I fixed the device tree my system is totally stable so I'm not sure what to make of this message.