cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Adventurer
Adventurer
11,473 Views
Registered: ‎09-28-2016

how disable cache in Linux

Jump to solution
hi,all

I have a zc706 board that run petalinux2015.4, now I want to disable the cache in Linux application, how can I do it?

thanks a lot


Tom
0 Kudos
1 Solution

Accepted Solutions
Highlighted
Scholar
Scholar
16,309 Views
Registered: ‎05-28-2013

Tom> you mean that the cache default enabled in linux kernel after linux system startup?

Yes. During boot up (before Linux) the caches are enabled and disabled, possibly several times. Once Linux is running, the caches are normally enabled. If you want to change that, you can change the kernel options (eg. rebuild the kernel).

Tom> is there any API function get the cache state in linux user space or bare metal application? enabled or disabled!!!

This is very system-specific and therefore there is not standard API for accessing it. However, on the ARM you can read it from the System Control Register, using assembly code like the following:

MRC p15, 0,<Rd>, c1, c0, 0; Read SCTLR

You will be interested in bit[2] and bit[12] which tell you about the state of the data and instruction caches, respectively. For details please see http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.100511_0401_10_en/ric1447333674419.html

 

Note that if you have caches disabled, Linux boot time will be significantly longer. It may run 2x or 10x slower than normal. The difference will be very obvious, so maybe you do not need any code to check it.

View solution in original post

0 Kudos
20 Replies
Highlighted
Moderator
Moderator
11,394 Views
Registered: ‎12-04-2016

Hi Tom

 

In your Application code, you can include "xil_cache.h" file, which has a set of API's related to Various Cache Functions.

 

 

To Disable the caches, you can make use of the following calls in your Application:

 

Xil_ICacheDisable();

Xil_L1ICacheDisable();

Xil_L2CacheDisable();

 

Please Refer the Below link, to know more about this:

https://www.xilinx.com/support/documentation/sw_manuals/xilinx2014_1/oslib_rm.pdf 

 

 

Best Regards

Shabbir

 

 

 

 

 

0 Kudos
Highlighted
Adventurer
Adventurer
11,372 Views
Registered: ‎09-28-2016

Hi,shabbirk

 

thanks for your response.

 

Xil_ICacheDisable();

Xil_L1ICacheDisable();

Xil_L2CacheDisable();

 

I think these three functions only work in standalone application, if I want to handle cache in linux application. how should I do?

 

Tom

0 Kudos
Highlighted
Moderator
Moderator
11,355 Views
Registered: ‎12-04-2016

Hi Tom

 

Yes I can Understand your Requirement for Disabling cache at Linux User Space. But as far as I know, cache disabling can only be done by first switching from user mode to SVC mode and then you can use asm volatile C equivalent Instructions to perform Cache disabling.

 

May be the below link gives you some insight into this:

https://www.community.arm.com/processors/b/blog/posts/caches-and-self-modifying-code

 

 

Thanks & Best Regards

Shabbir

0 Kudos
Highlighted
Scholar
Scholar
11,340 Views
Registered: ‎05-28-2013

Hi Tom,

 

Caches are normally enabled by the bootloader, and then disabled again just before starting the Linux kernel, where they get enabled again. So to switch them off, you might need to modify the bootloader, and you definitely need to modify the Linux kernel configuration.

 

Kernel configuration options you may need to toggle include: CONFIG_CPU_ICACHE_DISABLE and CONFIG_CPU_DCACHE_DISABLE, and possibly CONFIG_CACHE_L2X0. Keep in mind that this will have quite noticeable performance impact, your system will boot much more slowly.

 

 

0 Kudos
Highlighted
Adventurer
Adventurer
11,323 Views
Registered: ‎09-28-2016

Hi,

 

thanks for your clarification.

 

you mean that the cache default enabled in linux kernel after linux system startup?

 

is there any API function get the cache state in linux user space or bare metal application? enabled or disabled!!!

 

 

Tom

0 Kudos
Highlighted
Scholar
Scholar
16,310 Views
Registered: ‎05-28-2013

Tom> you mean that the cache default enabled in linux kernel after linux system startup?

Yes. During boot up (before Linux) the caches are enabled and disabled, possibly several times. Once Linux is running, the caches are normally enabled. If you want to change that, you can change the kernel options (eg. rebuild the kernel).

Tom> is there any API function get the cache state in linux user space or bare metal application? enabled or disabled!!!

This is very system-specific and therefore there is not standard API for accessing it. However, on the ARM you can read it from the System Control Register, using assembly code like the following:

MRC p15, 0,<Rd>, c1, c0, 0; Read SCTLR

You will be interested in bit[2] and bit[12] which tell you about the state of the data and instruction caches, respectively. For details please see http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.100511_0401_10_en/ric1447333674419.html

 

Note that if you have caches disabled, Linux boot time will be significantly longer. It may run 2x or 10x slower than normal. The difference will be very obvious, so maybe you do not need any code to check it.

View solution in original post

0 Kudos
Highlighted
Adventurer
Adventurer
11,280 Views
Registered: ‎09-28-2016

Hi,

 

thanks a lot for your clarification. I got it

 

 

Tom

0 Kudos
Highlighted
Explorer
Explorer
11,256 Views
Registered: ‎06-03-2015

Hi Tom,

 

In continuation of above discussion, Icache and Dcache are part of ARM Linux Boot Sequence,There are two stages in Booting Linux system one is Platfrom dependent Initilzation & Platform Independent Initilization.

In First part, Icache is definitely off and Dcache can be off or on. In Second part (Platform Independent Intilization), Icache and Dcache should be enable through Linux kernel using "start_kernel()" function.

 

Icache and Dcache are not supposed to be touched later on, as these are Initial Functions to make corrspending I/O peripherals work properly. So enabling/disabling of these functions are not good Idea.

 

Thanks & Regards
Satish.G

 

 

 

 

 

 

 

 

 

 

 

 

G Satish Kumar
0 Kudos
Highlighted
Explorer
Explorer
11,254 Views
Registered: ‎06-03-2015

Hi Tom,

 

In continuation of above discussion, Icache and Dcache are part of ARM Linux Boot Sequence,There are two stages in Booting Linux system one is Platfrom dependent Initilzation & Platform Independent Initilization.

In First part, Icache is definitely off and Dcache can be off or on. In Second part (Platform Independent Intilization), Icache and Dcache should be enable through Linux kernel using "start_kernel()" function.

 

Icache and Dcache are not supposed to be touched later on, as these are Initial Functions to make corrspending I/O peripherals work properly. So enabling/disabling of these functions are not good Idea.

 

Thanks & Regards
Satish.G 

G Satish Kumar
0 Kudos
Highlighted
Adventurer
Adventurer
8,412 Views
Registered: ‎09-28-2016

HI, Satish

 

Thanks for your clarification.

 

I want to consult you another two questions, please.

1.did the Dcache that you mentioned include L2 cache? or just only L1 Dcache?

2.how can I set one range of memory that reserved to uncache property through mmap function or other ways?

 

 

Tom

 

 

 

0 Kudos
Highlighted
Explorer
Explorer
8,398 Views
Registered: ‎06-03-2015

Hi Tom,

 

1.did the Dcache that you mentioned include L2 cache? or just only L1 Dcache?

 

[Satish]: In ARM Arch.. each CPU Core contains its own L1 cache (Icache & Dcache) and single L2 cache whicih is common to ALL CPU Cores.Example if Zc702 is having two ARM Cores , there are Two L1 caches and One L2 cache.

 

In the time of Booting, only Boot CPU Core (Probably CPU0) will enable the corrpsending L1Cache (Icache & Dcache) for Fast Excueation. Remaining Cores are get enabled through start_kernel () function where it calls other Cores corrspending L1 Cache  which will be enabled through SMP (Symmetric Multi Processing )functions.

 

L2 cache is taking care of simulation among/between these L1 caches. 

So Dcache mentioned in the earlier discussion is regards to L1 cache of Boot CPU Core. 

Where L1 cache is combination of Icache and Dcache.

 

 

2.how can I set one range of memory that reserved to uncache property through mmap function or other ways?

[Satish]:  I am not understanding your question properly, but based on initial Discussions, If CPU0 and CPU1 wants to share the memory , we use the shared memory region, It declares set of Memory areas for CPU0 and CPU1 when it wants to share the memory area , it will be declared in that area,(Part of DDR) instead of using mmap.

mmap is used to map device memory from user space specially when writng ioctl functionalities and when devolope user space device drivers.

 

But Internally to share the data between different CPUs shared memory, L2 cache, L1 Caches are internally taking care of the same.

 

Thanks & Regards
Satish.G
 

 

 

 

G Satish Kumar
0 Kudos
Highlighted
Adventurer
Adventurer
2,857 Views
Registered: ‎10-02-2018

Hi @gsatish10 , @atower , @rfs613 , @shabbirk 

Have you try disable cache in petalinux successfull?

I 'm use zc706 and need to disable cache to run cdma.

Thanks

0 Kudos
Highlighted
Scholar
Scholar
2,797 Views
Registered: ‎05-28-2013

@sonminh wrote:

 

Have you try disable cache in petalinux successfull?

I 'm use zc706 and need to disable cache to run cdma.


Under Linux, you can disable the cache using the following kernel configuration options:CONFIG_CPU_ICACHE_DISABLE and CONFIG_CPU_DCACHE_DISABLE, and possibly CONFIG_CACHE_L2X0. Note you will need to rebuild the kernel to change these settings.

 

However the effect is global and will significanly slow down everything, from boot onwards. So except for testing purposes, you really don't want to disble the caches. A better approach is to fix your driver...

0 Kudos
Highlighted
Adventurer
Adventurer
2,772 Views
Registered: ‎10-02-2018

Hi @rfs613 ,

I just try implement as you talk but no have any change once boot petalinux 2017.4 on zc706:

1. In file xilinx_zynq_defconfig of external linux  (components/ext_sources/linux-xlnx-2017.4_video_ea/arch/arm/configs), I add in to end of file:

CONFIG_CPU_ICACHE_DISABLE=y
CONFIG_CPU_DCACHE_DISABLE=y

2. petalinux-config -c kernel --defconfig xilinx_zynq_defconfig 

3. petalinux-build -c kernel

4. Modify file system-user.dtsi in /project-spec/meta-user/recipes-bsp/device-tree/files:

/include/ "system-conf.dtsi"
/ {
};
&amba {
cache-controller@f8f02000 {
status = "disabled";
};
};

5. Petalinux-build

This is my device-tree after disable cache and build:

cache-controller@f8f02000 {
compatible = "arm,pl310-cache";
reg = <0xf8f02000 0x1000>;
interrupts = <0x0 0x2 0x4>;
arm,data-latency = <0x3 0x2 0x2>;
arm,tag-latency = <0x2 0x2 0x2>;
cache-unified;
cache-level = <0x2>;
status = "disabled";
};

But once boot with BOOT.bin and image.ub just build, L2 cache still enable and I don't know veryfy that L1 is disabled. I see time to boot still fast. Here is console once boot linux, L2 cache still probe although disable.

Starting kernel ...

Uncompressing Linux... done, booting the kernel.
Booting Linux on physical CPU 0x0
Linux version 4.9.0-xilinx (thanhnt@ubuntu) (gcc version 6.2.1 20161016 (Linaro GCC 6.2-2016.11) ) #1 SMP PREEMPT Sat Nov 16 14:11:29 +07 2019
CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=18c5287d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
OF: fdt:Machine model: Zynq ZC706 Development Board
bootconsole [earlycon0] enabled
cma: Reserved 16 MiB at 0x3f000000
Memory policy: Data cache writealloc
percpu: Embedded 14 pages/cpu @ef7cb000 s25932 r8192 d23220 u57344
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 260608
Kernel command line: console=ttyPS0,115200 earlyprintk
PID hash table entries: 4096 (order: 2, 16384 bytes)
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Memory: 1007240K/1048576K available (6144K kernel code, 200K rwdata, 1464K rodata, 1024K init, 230K bss, 24952K reserved, 16384K cma-reserved, 245760K highmem)
Virtual kernel memory layout:
    vector  : 0xffff0000 - 0xffff1000   (   4 kB)
    fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
    vmalloc : 0xf0800000 - 0xff800000   ( 240 MB)
    lowmem  : 0xc0000000 - 0xf0000000   ( 768 MB)
    pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
    modules : 0xbf000000 - 0xbfe00000   (  14 MB)
      .text : 0xc0008000 - 0xc0700000   (7136 kB)
      .init : 0xc0900000 - 0xc0a00000   (1024 kB)
      .data : 0xc0a00000 - 0xc0a32180   ( 201 kB)
       .bss : 0xc0a32180 - 0xc0a6b998   ( 231 kB)
Preemptible hierarchical RCU implementation.
        Build-time adjustment of leaf fanout to 32.
        RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
RCU: Adjusting geometry for rcu_fanout_leaf=32, nr_cpu_ids=2
NR_IRQS:16 nr_irqs:16 16
efuse mapped to f0802000
slcr mapped to f0804000
L2C: platform modifies aux control register: 0x72360000 -> 0x72760000
L2C: DT/platform modifies aux control register: 0x72360000 -> 0x72760000
L2C-310 enabling early BRESP for Cortex-A9
L2C-310 full line of zeros enabled for Cortex-A9
L2C-310 ID prefetch enabled, offset 1 lines
L2C-310 dynamic clock gating enabled, standby mode enabled
L2C-310 cache controller enabled, 8 ways, 512 kB
L2C-310: CACHE_ID 0x410000c8, AUX_CTRL 0x76760001
zynq_clock_init: clkc starts at f0804100
Zynq clock init

Can you talk clearly? Thank you

 

0 Kudos
Highlighted
Scholar
Scholar
2,718 Views
Registered: ‎05-28-2013

I would suggest to confirm that the kernel and DTS changes actually got included in your build files:

  • zgrep CACHE /proc/config.gz  --- you can check your kernel CONFIG settings here
  • Somewhere in /proc/device-tree/... you should be able to find the DTS section for cache-controller@f8f02000. It will be a directory, containing a file called "status".  You can "cat" the status file to check that it is disabled.

If these no not match, then something in your build/boot process is not right, eg. your changes are not making it through.

 

For the outer pl310 cache, there are some additional kernel config settings. On my system for example, I have CONFIG_CACHE_L2X0=y (but this a different CPU than you are using).

 

Please check earlier message in this thread about how to check if the cache is disabled. But really, the difference is very visible on the serial console, booting takes considerably longer.

 

Let me re-emphasise that switching off caches under Linux is very uncommon. It is not a configuration that gets much testing (if any), and you won't find much information about it.

0 Kudos
Highlighted
Adventurer
Adventurer
2,655 Views
Registered: ‎10-02-2018

Hi @rfs613 

How can enable zgrep command in petalinux.

I just try zgrep but console notify:

-sh: zgrep: command not found

0 Kudos
Highlighted
Scholar
Scholar
2,639 Views
Registered: ‎05-28-2013

@sonminh wrote:

Hi @rfs613 

How can enable zgrep command in petalinux.

Instead of zgrep, you can use:

gzip -dc /proc/config.gz | grep CACHE

(To answer the question, zgrep command comes from the gzip package)

 

0 Kudos
Highlighted
Visitor
Visitor
2,232 Views
Registered: ‎12-15-2019

I have done it on armv8-a linux. I do it not to measure performance but to verify that xilinx zcu104 platform might have potential coherence error. As a result, the pynq image xilinx provided must have some coherence error during pl and ps communication. Here is my workaround:

  1. My platform is cortex-a53, ubuntu18 started at EL2 and switched to EL1 and supported SMP on four cpu cores. Thus, I need to turn off multi-core to ensure L2-cache coherency. Thanks to the feature of cpu-hot-plug, I just run :

    echo '0' > /sys/devices/system/cpu/cpu1/online,

    echo '0' > /sys/devices/system/cpu/cpu2/online,

    echo '0' > /sys/devices/system/cpu/cpu3/online

    then I run dmesg to verify that multi-core has been turn off.

  2. I build the kernel source tree, cause I cannot find it in my linux. You can run uname -r to see your kernel version. And find in /usr/src to see whether your linux already have it.

  3. I build the linux module. With gcc inline asm , I flush all cache and set sctlr_el1.c 0.

  4. I insmod the module. And I first run my program with the right result, although it takes 20 times slower than multi-core and D-cache on.

0 Kudos
Highlighted
Visitor
Visitor
2,091 Views
Registered: ‎01-31-2019

hi hht,

Could you tell me the details in step 3?  It will be helpful if you provide the source of module.  I want to disable cache on my zcu104 platform. 

Thanks.

0 Kudos
Highlighted
Visitor
Visitor
2,060 Views
Registered: ‎12-15-2019
https://github.com/i24361/DCacheDisable,you can directly use my module, if your linux version is xilinx 2018.3.
0 Kudos