UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Adventurer
Adventurer
16,170 Views
Registered: ‎09-19-2014

Booting from QSPI using U-Boot is very slow

Jump to solution

I've heard around the place that QSPI is supposed to maybe be the fastest boot medium (it's stated as being faster than SD card). I didn't have any problems with the SD card boot speed, so that's good.

 

But I did have problems with the QSPI boot speed.. which opposes that previous statement!

I have my booting sequence:

FSBL > bitstream > U-Boot > Linux...

 

My bitstream is 13.1MB, but that seems to get loaded by the FSBL in about ~300ms. That indicates how fast the QSPI reads could be at its full potential!

 

U-Boot uses the biggest chunk of time by far, because loading the Linux partitions takes so long.

It takes 16s from powering on to where the Linux kernel starts booting (I didn't include U-Boot's key timeout).

The total size of the partitions that U-Boot reads from is 10.3MB. This gives a read speed of 0.650MB/s.

 

This is very disappointing, as SD card takes ~1.5s for the same booting.

 

Has anyone else experienced slow U-Boot QSPI loading?

Does it require hacking the U-Boot sources to fix?

 

This message thread is similar:

http://forums.xilinx.com/t5/Embedded-Linux/QSPI-performance-under-Linux-is-very-disappointing/td-p/399901

But it's about Linux performance, and the fix seems to be with devicetrees, but as I understand it, U-Boot is before devicetrees and is unaffected by them (otherwise how do you read the devicetree off the QSPI device?). So this is a separate (but maybe related) issue.

0 Kudos
1 Solution

Accepted Solutions
Observer lemli
Observer
24,994 Views
Registered: ‎03-28-2010

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

We're loading about 12MB from a Spansion S25FL256 QSPI.

 

This used to take less than a second and was no source for concern.

 

However, after upgrading U-Boot from 2013.07-dirty (Jul 21 2014 - 11:31:31) to 2014.01-dirty (Sep 16 2014 - 04:32:29) we see the following:

 

Copying Linux from QSPI flash to RAM...
SF: Detected S25FL256S_64K with page size 256 Bytes, erase size 64 KiB, total 32 MiB

10.4 seconds pass...
SF: 4128768 bytes @ 0xc10000 Read: OK
SF: 65536 bytes @ 0xc00000 Read: OK

21.8 seconds pass...

SF: 8388608 bytes @ 0x1000000 Read: OK

## Booting kernel from Legacy Image at 03000000 ...

 

Something must have changed in U-Boot between 2013.7 and 2014.01 wrt QSPI.

 

While 400KB/s is a shame, the excessive 30 seconds can be tolerated atm so I'm not planning to go and search for the cause, nor am I going to try out newer U-boot versions unless the responsible fesses up and a proven solution is handed to me on a silver platter.

0 Kudos
27 Replies
Scholar milosoftware
Scholar
16,155 Views
Registered: ‎10-26-2012

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

Our 7030 and 7015 devices (only 5MB logic) boot from power-on to up and running a graphical Qt application in less than 7 seconds. So your numbers are indeed abominable.

 

First boot speed tip: Load the logic from Linux. This may require you to make kernel drivers that use logic to be compiled as modules so you can load them after programming. This allows the logic to be compressed and live in the rootfs, instead of occupying a large partition.

 

If you need to load so many megabytes, you are probably using a ramdisk or similar construction. Don't do that, it's a very bad way of booting your system. Just mount a UBI or Squash filesystem from flash directly.

 

Check your clocks. Most QSPI chips can run at ~100MHz, look at the datasheet ("fast read" timing is the one you need, this is what u-boot and the kernel use).

 

You should be able to get over 20MB/s from the QSPI chip.

 

The SD card is actually faster. Though it runs at lower clock speed (50MHz usually, which is half the QSPI clock rate), the SD/MMC controller supports DMA transfers which the QSPI controller does not, so QSPI will hog the CPU and slow down the system in addition to not being able to really transfer at top speed, while the SD transfers will run at full speed in background.

Adventurer
Adventurer
16,143 Views
Registered: ‎09-19-2014

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

Thanks for confirming my numbers are abominable.

 

As I understand, loading the PL bitstream from QSPI (in the FSBL stage) is not a problem, and it takes about 300ms (okay, maybe slightly more) for the blue light (on my Zynq 7045 MiniITX board) to turn on, which I believe is an indicator that the bitstream has completed loading. So if this is true, loading the bitstream is NOT the issue, and will in fact make it slower, since Linux can't read from QSPI as fast as the FSBL. The advantage there is compression I see (which won't make it any faster).

 

I can see while it is booting that U-Boot pauses for a long time for loading these three partitions from QSPI:

- Linux kernel

- devicetree (obviously very small)

- Rootfs

Together these are 10.3MB, as I said before.

The kernel tells me the rootfs is not an initramfs, but it looks like an initrd. I did slightly look at UBI or Squashfs, but I can't remember if I concluded it was feasible or not. I may look into that again. I don't know if that's the full solution though, because it's likely to just shift most of the QSPI loading time for the rootfs to when it loads user space applications. The Linux kernel will still take a good chunk (6 seconds maybe?) to load.

 

So the QSPI clock speed... I know the devicetree set it to 50MHz. So increasing that to 100MHz will help I/O be faster in Linux. But that won't change U-Boot. So the QSPI clock would otherwise be set in the Vivado design?

 

So I should be able to get over 20MB/s for QSPI reads... is that from U-Boot or from in Linux?

 

Thanks - I was aware of the lack of DMA on QSPI, so that doesn't help with multitasked situations (which U-Boot isn't!).

0 Kudos
Adventurer
Adventurer
16,140 Views
Registered: ‎09-19-2014

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

I meant to also add...

 

Your 7 second Linux/Qt boot time: is that from SD card or QSPI or other?

 

Also, I noticed in your other thread that you achieved just over 4MB/s QSPI read speed in Linux. I assume that's still about correct.

That would mean 20MB/s read is for U-Boot only. Please confirm.

0 Kudos
Scholar milosoftware
Scholar
16,127 Views
Registered: ‎10-26-2012

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

It boots in about 7 seconds from either SD or QSPI. SD is about 1s faster.

 

U-boot, FSBL and kernel should all get the same read speed, it is limited by the QSPI controller, not by CPU.

 

The kernel read speed has since long been fixed.

 

root@topic-miami-7015:~# dd if=/dev/mtd5 of=/dev/null bs=64k                    
440+0 records in                                                                
440+0 records out                                                               
28835840 bytes (27.5MB) copied, 1.060328 seconds, 25.9MB/s                      
root@topic-miami-7015:~#

 

If you use a compression like LZO, it will usually speed things up. Decompressing LZO data is very fast, and bitstreams compress very well (I get better then 1:5 ratios). If the bitstream compresses into 2MB, you'll only need to read 2MB which will take ~100ms at 20MB/s, and decompressing the data will probably take less than 200ms as well, so in total it will be faster to compress the bitstream. The only way to find out if this is true for your case is to just try and measure.

 

u-boot-xlnx also supports compressed bitstreams, by the way.

 

The clocking setup by Vivado is often borked. Most of these problems were fixed in 2014.2 though. So check the register contents in the ps7_init files produced by Vivado.

 

u-boot also has environments for setting the QSPI speed. By default it will be conservative (often 25MHz), so set it to whatever your board supports in your board's config.h.

0 Kudos
Scholar milosoftware
Scholar
16,124 Views
Registered: ‎10-26-2012

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

If your rootfs is squash or UBI, you won't have to load if from u-boot. This will save you a big read-to-ram during u-boot, which will take quite a lot of time. It will also free up the RAM.

 

The kernel will only read things it really needs from the flash (and cache them in RAM too). So it's likely to boot much faster. Fastest boot is with squashfs, experiment with compression options (e.g. XZ will make it smaller, LZO will make it faster, the net result may favor either for boot time, so try both).

UBI is slower to boot, but will give you excellent wear-levelling, and allows read/write access to flash.

0 Kudos
Observer lemli
Observer
24,995 Views
Registered: ‎03-28-2010

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

We're loading about 12MB from a Spansion S25FL256 QSPI.

 

This used to take less than a second and was no source for concern.

 

However, after upgrading U-Boot from 2013.07-dirty (Jul 21 2014 - 11:31:31) to 2014.01-dirty (Sep 16 2014 - 04:32:29) we see the following:

 

Copying Linux from QSPI flash to RAM...
SF: Detected S25FL256S_64K with page size 256 Bytes, erase size 64 KiB, total 32 MiB

10.4 seconds pass...
SF: 4128768 bytes @ 0xc10000 Read: OK
SF: 65536 bytes @ 0xc00000 Read: OK

21.8 seconds pass...

SF: 8388608 bytes @ 0x1000000 Read: OK

## Booting kernel from Legacy Image at 03000000 ...

 

Something must have changed in U-Boot between 2013.7 and 2014.01 wrt QSPI.

 

While 400KB/s is a shame, the excessive 30 seconds can be tolerated atm so I'm not planning to go and search for the cause, nor am I going to try out newer U-boot versions unless the responsible fesses up and a proven solution is handed to me on a silver platter.

0 Kudos
Adventurer
Adventurer
15,473 Views
Registered: ‎09-19-2014

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

Sorry, I had put this project aside for a little while there...

 

Thanks @lemli for confirming that the UBoot 2014.1 version was the cause of the slow QSPI access. I'm keen to wait until a newer bug fixed version is handed to me on a silver platter too.

 

@milosoftware, can you confirm that your fast QSPI booting was a version before 2014?

0 Kudos
Contributor
Contributor
15,441 Views
Registered: ‎09-18-2009

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

Hello,

We realized a difference between u-boot version 2013.7 and 2014.7 for QSPI access speed in Zynq based system.

QSPI frequency is the same with FSBL settings when you call "sf probe 0 0 0" with u-boot 2013.7 but QSPI frequency is the same with the x in "sf probe 0 x 0" with u-boot 2014.7.

 

It maybe works with u-boot 2014.1 too.


Ali

0 Kudos
Observer lancec
Observer
15,348 Views
Registered: ‎10-28-2013

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

We had the same problem. After reading here, I found the problem in zynq_qspi.c:

 

In struct spi_slave *spi_setup_slave(unsigned int bus, unsigned int cs,  unsigned int max_hz, unsigned int mode)

 

I removed:
     qspi->qspi.max_speed_hz = (max_hz < qspi->qspi.master.speed_hz) ?  max_hz : qspi->qspi.master.speed_hz;

 

and replaced with:
     qspi->qspi.max_speed_hz = qspi->qspi.master.speed_hz;

 

 

This is what was in older versions of U-boot.

0 Kudos
Adventurer
Adventurer
11,968 Views
Registered: ‎09-19-2014

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

Many thanks lancec! I will test that suggested fix in due time.

 

Is it because we're incorrectly setting the max hz? Is it user error (or just ignorance) at all?

0 Kudos
Scholar milosoftware
Scholar
11,961 Views
Registered: ‎10-26-2012

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

Maybe this patch helps:

 

https://github.com/topic-embedded-products/meta-topic/blob/master/recipes-bsp/u-boot/u-boot-zynq-git/0001-zynq_qspi.c-Set-correct-frequency-regardless-of-hard.patch

 

U-boot assumes that the QSPI frequency divider has already been programmed at boot, which isn't always the case.

0 Kudos
Observer lemli
Observer
11,953 Views
Registered: ‎03-28-2010

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

Thanks, @lancec for the workaround on a silver platter :-)

 

I can confirm that it works on U-Boot 2014.01-dirty (Jan 06 2015 - 02:47:06)

0 Kudos
Observer lancec
Observer
11,945 Views
Registered: ‎10-28-2013

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

That patch explains it. Works on U-boot 2014.07. Thanks milsoftware.

0 Kudos
Observer lancec
Observer
11,936 Views
Registered: ‎10-28-2013

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

I should note that with that patch I had to pass in a QSPI frequency in my sf probe call instead of just 0.

ie. sf probe 0 100000000 0

 

It should probably use the lqspi_frequency if the passed in max_hz is 0.

0 Kudos
Adventurer
Adventurer
11,916 Views
Registered: ‎09-19-2014

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

I agree with @lancec here - I modified the patch from @milosoftware slightly:

 

Changed:

+ qspi->qspi.max_speed_hz = (max_hz < lqspi_frequency) ?
+ max_hz : lqspi_frequency;

to:

+ qspi->qspi.max_speed_hz = (max_hz < lqspi_frequency) ?
+ (max_hz == 0 ? lqspi_frequency : max_hz) : lqspi_frequency;

 

This way I don't need to change the sf probe command at all from using 0.

I also changed the context around the patch slightly to make it fit with the latest meta-xilinx snapshot that I use.

But the bottom line is that it works brilliantly! Thank you!

0 Kudos
Adventurer
Adventurer
11,913 Views
Registered: ‎09-19-2014

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

I meant to say - 25s boot time down to 11s! :) (including the 4s countdown)

0 Kudos
Scholar milosoftware
Scholar
11,907 Views
Registered: ‎10-26-2012

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

In u-boot, these environments are supposed to control the (Q)SPI frequencies (unless you specify them as non-zero in sf probe):

 

#define CONFIG_SF_DEFAULT_SPEED        108000000
#define CONFIG_SF_DEFAULT_MODE SPI_MODE_0
#define CONFIG_ENV_SPI_MAX_HZ CONFIG_SF_DEFAULT_SPEED

 

To get rid of the countdown, set

 

#define CONFIG_ZERO_BOOTDELAY_CHECK
#define CONFIG_BOOTDELAY       0

 

This gives a 0 countdown, but still checks for serial input in the uart buffer, so you can get into u-boot basically by holding down a key while booting.

0 Kudos
Observer lancec
Observer
11,900 Views
Registered: ‎10-28-2013

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

@milosoftware I agree with those environment variables setting the frequencies. But they are overridden if that argument, including 0, is specified in sf probe. This is the case at least in Xilinx U-boot 2014.07.

0 Kudos
Adventurer
Adventurer
11,892 Views
Registered: ‎09-19-2014

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

@milosoftware, yes, that's one way of setting the boot countdown, or alternatively at the uboot prompt:

> setenv bootdelay 0

> saveenv

> reset

(Assuming you have allocated a place to store the environment.)

 

I can't get it to acknowledge any key when I set it to 0 delay though. :(

That's okay - I can reset the U-Boot environment to default and access the U-Boot prompt again by clearing the storage area for the environment settings.

0 Kudos
Scholar milosoftware
Scholar
10,247 Views
Registered: ‎10-26-2012

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

I can't get it to acknowledge any key when I set it to 0 delay though. :(

 

That is what CONFIG_ZERO_BOOTDELAY_CHECK is for :)

0 Kudos
Adventurer
Adventurer
10,214 Views
Registered: ‎09-19-2014

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

Ah, okay, thanks. :)

Currently using defaults for everything when building.

0 Kudos
Visitor baudouin
Visitor
10,009 Views
Registered: ‎04-28-2012

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

Change in zynq_common.h

 

"qspiboot=echo Copying Linux from QSPI flash to RAM... && " \
"sf probe 0 50000000 0 && " \
"sf read ${kernel_load_address} 0x100000 ${kernel_size} && " \
"sf read ${devicetree_load_address} 0x600000 ${devicetree_size} && " \
"echo Copying ramdisk... && " \
"sf read ${ramdisk_load_address} 0x620000 ${ramdisk_size} && " \
"bootm ${kernel_load_address} ${ramdisk_load_address} ${devicetree_load_address}\0" \

 

This forces the QSPI speed. Setting to zero gives the lowest rate possible.

Visitor f897772935
Visitor
9,940 Views
Registered: ‎03-09-2015

Re: Booting from QSPI using U-Boot is very slow

Jump to solution
I have the same trouble,I did it like what you write,but when exe"sf probe 0 0 0",it notice that
"zynq-uboot> sf probe 0 0 0
SF: Unsupported flash IDs: manuf 00, jedec 0000, ext_jedec 0000
Failed to initialize SPI flash at 0:0"
I feel uncertain;
0 Kudos
Visitor f897772935
Visitor
9,937 Views
Registered: ‎03-09-2015

Re: Booting from QSPI using U-Boot is very slow

Jump to solution
0 Kudos
Adventurer
Adventurer
9,870 Views
Registered: ‎09-19-2014

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

Sorry, exactly which change did you try?

Does it identify the flash device properly before you applied your code change?

0 Kudos
Visitor wmaxwell
Visitor
3,786 Views
Registered: ‎07-10-2017

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

Hello all! I know this post is a bit older, but I was having a very similar issue in U-Boot 2017.01 with my MicroZed board, and figured I would share a pretty solid workaround I found in case someone else stumbles across this post again (like I did). If you take a look at your u-boot environment variables, there is one describing the process of reading the kernel image (cp_kernel2ram), with an sf probe and sf read command. The second value of the sf probe command, detailing the clock speed of the QSPI, is left blank. I added in this value to the environment variable and set mine to the maximum speed allowed by the chip, which I also specified in the device tree ahead of time. Saved my environment and rebooted, went from about a 20+ sec read time to a 1 second read time. Hope this helps!

Visitor huojianyu
Visitor
139 Views
Registered: ‎09-08-2019

Re: Booting from QSPI using U-Boot is very slow

Jump to solution

Thanks wmaxwell! It works for me!!

0 Kudos