cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Contributor
Contributor
1,703 Views
Registered: ‎05-02-2019

Missing linux-cache-headers on UltraZed-EG-Platform

Jump to solution

Hello,

I am currently trying to build a kernel module, which contains functions like cache-flushing.

Some background Information as to how my System is generated:

I am using the Xilinx-V2019.1-Toolchain (Vivado & SDK, PetaLinux Tools) to generate all the required Embedded Linux components like bootloader, kernel image and device tree. Furthermore I have cloned the Xilinx-Linux-Repository, switched to the V2019.1 branch and configured PetaLinux Tools to use the mentioned Xilinx-Linux-Repo to generate the components using the repository. My platform is the Avnet-UltraZed-EG-IOCC Board and I am using a Debian-10 root filesystem on it. The Module I am trying to compile has worked on other Zynq-7000 Dev. Boards which have a different CPU-Architecture (32-Bit).

The issue:

The module won't compile, because I am trying to use kernel functions from a header which doesn't exist (outercache.h).

Regarding the outercache.h header, @primoz_beltram pointed out to me that this library can only be used on ARM-32-Bit-CPUs. As of right now I have worked through the arch/arm/include/asm/*.h and arch/arm64/include/asm/*.h within the Xilinx-Linux-Repository and found some other headers. But the problem persists, that the relevant headers (for example from the arm64-dir: cache.h, cacheflush.h) are not build. I have tried building the headers using PetaLinux Tools, as well as Cross Compiling using make xilinx_zynqmp_defconfig, make deb-pkg, make headers_install. Using the find command I have not figured out how to generate these cache headers, which could be the solution to my problem.

Another tip which I received from @shabbirk was to include the following line CONFIG_OUTER_CACHE=y in the  xilinx_zynqmp_defconfig, which seemed promising but also didn't help me generating the required headers.

The only cache-relevant header being built on my system is located under /usr/include/linux/bcache.h, but doesn't contain any helpful functions for my module.

So as of right now I am really desperate to find out, how the cache can be flushed on 64-Bit Architectures from the Xilinx Zynq UltraScale+ Series. Hopefully someone can help me out with this issue, it might also help future developers.

I am depending on this to work for my application which is why I am trying to get help from both Forums here as well as element14.

I would appreciate all the help I can get and please reply should you need more details, screenshots or other information,

best regards,

Bjørn

 

ERROR:

/<path-to-project>/memory.c: fatal error: asm/outercache.h: No such file or directory
#include <asm/outercache.h>
              ^~~~~~~~~~~~~~~~~~
compilation terminated.

 

0 Kudos
1 Solution

Accepted Solutions
Highlighted
Contributor
Contributor
1,333 Views
Registered: ‎04-02-2014

I don't know if it will answer your problem, but ...
It is not recommended to use an API that directly controls the cache in the Linux Kernel.

Generally, when you want to flush / invalidate the cache with Linux Kernel Module, use dma_sync _*() API.

Use dma_sync _*_ for_device() if you want to flush the cache before the PL accesses the memory.

Please refer to the slide at the following URL.

https://elinux.org/images/4/49/20140429-dma.pdf

 

I am doing PL-PS transfer with cache control in the same situation as you. I have published some examples on github, so please refer to them.

https://github.com/ikwzm/ZynqMP-FPGA-Linux

https://github.com/ikwzm/ZynqMP-FPGA-Linux-Example-2-UltraZed

 

In the above example, udmabuf is used for PS-PL transfer. udmabuf can actively flush / invalidate the cache. Please refer to this as well.

https://github.com/ikwzm/udmabuf

 

 

View solution in original post

0 Kudos
11 Replies
Highlighted
Moderator
Moderator
1,676 Views
Registered: ‎12-04-2016

Hi @bhoevding 

Can you try enabling this kernel option: 

CONFIG_OUTER_CACHE=y

 

0 Kudos
Highlighted
Contributor
Contributor
1,662 Views
Registered: ‎05-02-2019

Hello @shabbirk ,

I added the proposed line into the Kernel Config List (<Xilinx-Linux>/aarch/arm64/configs/xilinx_zynqmp_defconfig).

Also I have set the PetaLinux Tools Configuration to use the below mentioned external Kernel Source.

My current build steps are:

# 1) Using PetaLinux to build the Boot-Files, Kernel Image and Device Tree  
Host $ petalinux-config <hw-desc>  
Host $ petalinux-build  
Host $ petalinux-package ...  
  
# 2) cloning the Xilinx-Linux-Repo and building the Header Files  
Host $ git clone <xlnx-lnx-repo>; cd <xlnx-lnx-repo>  
Host $ make xilinx_zynqmp_defconfig  
Host $ make deb-pkg  
Host $ make headers_install  
Host $ ls usr/include/*  
Host $ find usr/include/ -name "outercache.h" # Header not found  
Host $ find <xlnx-lnx-repo>/ -name "outercache.h" # Header found under arch/arm/include/asm/outercache.h 

  
# 3) booting the System with Debian  
UZ3EG $ find /usr/include/ -name "outercache.h" # Header not found  
UZ3EG $ sudo dpkg -I linux-header* linux-image* linux-libc*  
UZ3EG $ cd /lib/modules/<uname -r>/build  
UZ3EG $ make headers_install  
UZ3EG $ make modules  
UZ3EG $ find usr/include/ -name "outercache.h" # Header not found  

I installed the generated linux-*.deb files but looking into usr/include/* the header is still missing. Is there another configuration preventing me to build this header?

0 Kudos
Highlighted
Explorer
Explorer
1,619 Views
Registered: ‎02-22-2012
arm and arm64 are different architectures with different cache-ing implementation. Kernel cache functions for arm will not work on arm64. The best hint I can give you is to look and compare arch/arm/ and arch/arm64/ files to see how they flush cache.
0 Kudos
Highlighted
Contributor
Contributor
1,611 Views
Registered: ‎05-02-2019

Hello @primoz_beltram,

thanks for pointing that out, it makes perfectly sense to me that the architectures differ and therefore other headers should be used.

I will take a look at the headers, but I have only found a bcache.h on the UltraZed-EG-IOCC. There must be some options to include the linux-xlnx/arch/arm64/

  • cache.h
  • cacheflush.h

But I am unsure as to which headers should be used for the Zynq UltraScale+ Platform to flush the cache, maybe someone can point me to architecture-independent headers?

And the other issue still persists, is the outercache.h only for ARM-32-Bit Processors?

0 Kudos
Highlighted
Adventurer
Adventurer
1,478 Views
Registered: ‎05-15-2018

Did you find an answer to this?

0 Kudos
Highlighted
Contributor
Contributor
1,466 Views
Registered: ‎05-02-2019

Hello @harish-anil,

No, unfortunately not.

I am still not sure whether there is a Cache-Kernel-API / Cache-Headers implemented in the Linux-Xilinx Kernel for the UltraZed.

And if it is implemented, which would be amazing, then I still have to figure out how to include this into the PetaLinux Tools Building Process.

Is someone struggling with the same issue?

Regards,

Bjørn

0 Kudos
Highlighted
Contributor
Contributor
1,394 Views
Registered: ‎05-02-2019

Furthermore a Kernel-API would be ideal, so no more Platform dependency exists. Maybe there is even a different header I have not yet considered.

fatal error: asm/cacheflush.h: No such file or directory
 #include <asm/cacheflush.h>
          ^~~~~~~~~~~~~~~~~~
fatal error: asm/cache.h: No such file or directory
 #include <asm/cache.h>
          ^~~~~~~~~~~~~~~~~~

ZU3EG (/usr/include) $ ls -l asm/
asm
├── auxvec.h
├── bitsperlong.h
├── bpf_perf_event.h
├── byteorder.h
├── errno.h
├── fcntl.h
├── hwcap.h
├── ioctl.h
├── ioctls.h
├── ipcbuf.h
├── kvm.h
├── kvm_para.h
├── mman.h
├── msgbuf.h
├── param.h
├── perf_regs.h
├── poll.h
├── posix_types.h
├── ptrace.h
├── resource.h
├── sembuf.h
├── setup.h
├── shmbuf.h
├── sigcontext.h
├── siginfo.h
├── signal.h
├── socket.h
├── sockios.h
├── statfs.h
├── stat.h
├── swab.h
├── termbits.h
├── termios.h
├── types.h
├── ucontext.h
└── unistd.h

ZU3EG (/usr/include) $ ls -l asm-generic/
same output..

I have also tried to build the module within PetaLinux Tools using:

petalinux-create -t modules --name linux-module --enable

unfortunately this does not resolve anything.

Hopefull an Embedded Linux expert or Xilinx employee can point me in the right direction, as this is essential for my application.

0 Kudos
Highlighted
Contributor
Contributor
1,363 Views
Registered: ‎05-02-2019

Please note I am trying to solve this issue using the ACP Port, which hopefully does not require me to flush the cache. But nonetheless I would still like to know how to enable arm64-cache-headers on embedded linux systems.

Another problem I have noticed is, that the following headers are missing as well:

  • <linux/slab.h>
  • <asm/uaccess.h>

I have also noticed others struggling with the same issue.

0 Kudos
Highlighted
Contributor
Contributor
1,334 Views
Registered: ‎04-02-2014

I don't know if it will answer your problem, but ...
It is not recommended to use an API that directly controls the cache in the Linux Kernel.

Generally, when you want to flush / invalidate the cache with Linux Kernel Module, use dma_sync _*() API.

Use dma_sync _*_ for_device() if you want to flush the cache before the PL accesses the memory.

Please refer to the slide at the following URL.

https://elinux.org/images/4/49/20140429-dma.pdf

 

I am doing PL-PS transfer with cache control in the same situation as you. I have published some examples on github, so please refer to them.

https://github.com/ikwzm/ZynqMP-FPGA-Linux

https://github.com/ikwzm/ZynqMP-FPGA-Linux-Example-2-UltraZed

 

In the above example, udmabuf is used for PS-PL transfer. udmabuf can actively flush / invalidate the cache. Please refer to this as well.

https://github.com/ikwzm/udmabuf

 

 

View solution in original post

0 Kudos
Highlighted
Contributor
Contributor
1,305 Views
Registered: ‎05-02-2019

Hello @kawazome,

Thanks for your tips regarding this matter. It brought me a little further, but I still have to figure out some configurations.

The dma-slides helped a lot and I will refrain from using the linux-arm-cache-api in the future. It will also require me to make some changes from the previous application, which successfully ran on a zynq soc. One issue which I have noticed is the Linux-Debian System I have generated doesnt contain the relevant APIs like #include <linux/dma-mapping.h>. I have also installed the headers which were build through my kernel but some dont seem to be deployed. The only dma relevant header on my system is: /usr/include/linux/dma-buf.h and it only contains defines. As of right now I am using the ACP Port for testing, which in my understanding doenst require me to flush or invaliate the cache. But the driver was also developed for HPC-Port Applications so for later appliances I will have to implement cache-flushing/invalidating functions.

I have looked at your gihub repository, which is an excellent project. I have also noticed that you dont build your system using PetaLinux Tools, is there a particular reason why?

Thanks though for the udmabuf example as well, I will have a closer look at it and maybe find a solution in there which I havent considered by now. Additionally I have seen in your udmabuf.c that you use headers like slab.h and uaccess.h, those are headers which are also present in my linux driver. But when I am trying to generate the devices, the complier fails due to those headers missing on the UltraZed. Why am I missing these headers, is there an issue with my building steps?

In any case I am thankful for your post, hopefully I can figure out a solution regarding the Kernel-Header-(APIs) and the share it with everyone

0 Kudos
Highlighted
Contributor
Contributor
1,291 Views
Registered: ‎04-02-2014

I don't use Petalinux because it's complicated, redundant and hard to understand.

I also use Debian. If you install the Linux Kernel header file package (linux-headers-xxxxx_xxxxx_arm64.deb) in Debian, /lib/modules/xxxxx/build/ is generated. This is a symbolic link to the directory /usr/src/linux-headers-xxxxx. (xxxxx is the LOCALVERSION (for example, "4.19.0-xlnx-v2019.1-zynqmp-fpga") that was used when building the Linux Kernel.)

There are ./include/linux/dma-mapping.h and ./include/linux/slab.h in this directory.

 

0 Kudos