UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Scholar ronnywebers
Scholar
1,531 Views
Registered: ‎10-10-2014

Can I (re)configure the bitstream only after linux has booted on my Zynq ARM cores?

Some of our systems have the following architecture :

 

   microcontrollor chip  <-----> FPGA  <------> connector <------> IO Board

 

* The IO board has some 'ID' device (1-wire ID, or I2C eeprom), containing a descriptor that tells which board it is.

* The CPU reads the ID, and decides if it's safe to load the bitstream (i.e. a wrong IO board could be plugged in), or which version of a bitstream should be loaded (if there are multiple derivatives of the IO board). Some kind of plug & play system

 

In this case the microcontroller chip is master of the system, and programs the FPGA only after it has booted.

 

Now I'm wondering how to do the same with Zynq :

 

    [ ARM core  <->  PL ]  <---- connector ----> IO board

 

AFAIK, with Zynq running Linux, the bitstream is typically loaded by U-Boot (or FSBL can do this too I believe).

 

Q : what would be my architectural options to have the IO board identified before the (desired) bitstream is loaded? 

I think U-boot would be one option, but would this be possible from Linux? So let linux boot completely, then check the IO boards ID, and then load the bitstream? Or would that require 'partial reconfiguration', but then 'partial' means 100% reconfiguration. I'm not yet familiar with partial reconfiguration, but before diving in, I'd like to know what would be the best strategy?

 

 

** kudo if the answer was helpful. Accept as solution if your question is answered **
0 Kudos
9 Replies
1,498 Views
Registered: ‎01-08-2012

Re: Can I (re)configure the bitstream only after linux has booted on my Zynq ARM cores?

On our boards, we load the FPGA after the OS has booted.  The FSBL does not load the PL at all.  (BTW, this allows us to have a common FSBL across several boards.)

 

You do not need to use partial reconfiguration.

 

One (minor) advantage to doing it this way is that you can compress the FPGA image.  We get about 3:1 compression with gzip -9 on a moderately full FPGA.

This obviously saves filesystem space, but it also makes it load faster (as less data is read from the Flash, and decompression is fast).

We load the FPGA image (on our boards that run Linux) with this line of shell:

 

gunzip < fpga_file.bin.gz > /dev/xdevcfg

There's a bunch of other details (such as turning on the PS<->PL voltage level translation circuit), but you don't need to bother about that just yet.

 

About the only downside to loading the FPGA from the OS is that you can't use many of the security features that are available to the FSBL.  I believe this includes bitstream encryption and authentication.

 

Allan

Scholar ronnywebers
Scholar
1,157 Views
Registered: ‎10-10-2014

Re: Can I (re)configure the bitstream only after linux has booted on my Zynq ARM cores?

thanks @allanherriman, comming back on this one : 

 

you say you have a common FSBL for several boards

 

Now I thought FSBL does a lot of initialisation  of PS and preparation for the PL (ps7_init.c), like indeed enabling the PS-PL voltage level translation circuit, PS_FCLKx clocks, ... 

 

Now  if I'm correct, the contents of ps7_init.c is generated based on the .hdf file -> doesn't this mean that depending on what you put in the PL, and/or configure in the PS, that ps7_init.c gets updated?  So for example lets say I started with an FSBL at a time where I only used FS_FCLK0. But later on, I have a design that uses FS_FCLK1 too -> doesn't that mean that the FSBL needs to get updated to enable this new FS_FCLK1 ?

** kudo if the answer was helpful. Accept as solution if your question is answered **
0 Kudos
1,129 Views
Registered: ‎01-08-2012

Re: Can I (re)configure the bitstream only after linux has booted on my Zynq ARM cores?

Yes, it would need to get updated if you need that clock active by the time the FSBL has finished.

But the real requirement is that you won't need that clock until the (end of) the PL loading.  Since we're loading the PL bitstream from the OS, we can turn the clock on from the OS and we don't need to turn that clock on in the FSBL at all.

 

In general, if I'm using the same FSBL across a number of boards, that implies either:

- All those boards have the same PS <-> PL interface.

or

- All those boards start with the same PS <-> PL interface at boot, then I enable the extra parts later when the OS is running.

 

0 Kudos
Scholar ronnywebers
Scholar
755 Views
Registered: ‎10-10-2014

Re: Can I (re)configure the bitstream only after linux has booted on my Zynq ARM cores?

@allanherriman, it's been a while since you helped me in this thread, had some other stuff to finish.

Currently I'm still loading my bitstream from the boot.bin (by the FSBL), but I want to switch to a 'deferred' load under command of my Linux application. I'm not a Linux guru, so hope if you can help me out with some Linux stuff that is still puzzling me: 

* in the device tree there are items (like PS GEM0, PS UART1, ...) that are not connected through EMIO, and hence do not need the bitstream to be loaded. That way, we can boot all the way until my Linux application is running, and still communicate over ethernet and UART with Linux. Deferring the bitstream load would allow me to check the I2C eeprom contents on my FMC board, and make sure that the bitstream is compatible with that FMC board (otherwise an output of the Zynq might go in a fight with an output of the ADC ...). If the application finds the correct board, it tells Linux to load the bitstream, if not it flags an error. I think to have found that I need to call the command 'cat fpga.bin > /dev/xdevcfg'  from my application to load the bitstream. (hope I didn't tell anything wrong so far)

Now my questions(s):

* what happens when Linux starts, and it scans the device tree blob top-down, and comes across the PL and external devices that require a bitstream before the can be seen by the ARM cores? Does Linux simply load the driver for the - yet to be connected -  device? Does just it already add the (not yet 'connected') devices to the virtual file system (proc, sys, ...)? 

* what happens when the linux application finally gives the command to load the bitstream? Does Linux load the bitstream, and then automatically re-triggers a device tree scan to further add the newly connected PL and external devices? Like some kind of plug & play system gets triggered?

* does a linux driver, when it gets loaded by the kernel, in some cases does a 'peek for presence' to see if it's really there, or even 'initialize' a device (i.e. an ADC with SPI control port) ? Or is that 'init' not done by the kernel, and only on request of the application (which knows that it has loaded the bitstream and hence it's ok to poke the device?).

hope my questions are clear, and thanks in advance!

** kudo if the answer was helpful. Accept as solution if your question is answered **
0 Kudos
711 Views
Registered: ‎01-08-2012

Re: Can I (re)configure the bitstream only after linux has booted on my Zynq ARM cores?

@ronnywebers

I'm not a Linux guru (and I don't write most of the software for my boards) so take what I say with a grain of salt.

We load the optional parts of the device tree manually.  By that, I mean that they are not compiled in to the big device tree blob that the Linux kernel sees early on.  One of the programmers here created a simple module that allows the device trees to be loaded this way.  He speculated that there might be better support for this sort of thing in recent kernel versions.  Google for "dynamic device tree linux" for more info.

For example, our code would look like this if written in Bash:

echo "Loading FPGA with $FPGA_FILE"
gunzip < $FPGA_FILE > /dev/xdevcfg

modprobe magic_dtb_loader
echo "Loading device tree $DTB_FILE"
cat $DTB_FILE > /dev/magic_dtb_loader

 We have a different device tree file for each FPGA image file.

697 Views
Registered: ‎01-08-2012

Re: Can I (re)configure the bitstream only after linux has booted on my Zynq ARM cores?

Now to actually answer your questions(s):

* what happens when Linux starts, and it scans the device tree blob top-down, and comes across the PL and external devices that require a bitstream before the can be seen by the ARM cores? Does Linux simply load the driver for the - yet to be connected -  device? Does just it already add the (not yet 'connected') devices to the virtual file system (proc, sys, ...)? 

It's not that smart.  If you load the driver, it will almost certainly start poking at registers (i.e. AXI addresses) inside the PL ... and your system will hang when it doesn't get a bus completion.  (You did remember to enable the watchdog?)

* what happens when the linux application finally gives the command to load the bitstream? Does Linux load the bitstream, and then automatically re-triggers a device tree scan to further add the newly connected PL and external devices? Like some kind of plug & play system gets triggered?

It's plug & play only if you program it that way.  Otherwise, it's not that smart.

OTOH, I understand that a PCIe bus scan will load drivers if it finds any new device on the bus.  We aren't using PCIe though.  There is no AXI equivalent to PCIe scans and BARs (which contain the information needed for plug & play).

* does a linux driver, when it gets loaded by the kernel, in some cases does a 'peek for presence' to see if it's really there, or even 'initialize' a device (i.e. an ADC with SPI control port) ? Or is that 'init' not done by the kernel, and only on request of the application (which knows that it has loaded the bitstream and hence it's ok to poke the device?).

Typically you will only load a driver for some hardware if that hardware is there.  The driver will typically be responsible for the initialisation of that hardware.  The linux kernel and the drivers aren't that smart - the idea is to offload policy about which drivers to load to the device trees.

I think you want to be able to load device trees from your application, and your application can control when it loads the FPGA and when it loads the device trees.

On Zynq, some of the hardware init is in the FSBL.  You'll need to restrict that to common items such as MIO (or the subset of MIO pins and peripherals that don't change with your application).

You can also offload some of the sequencing and dependency management to systemd if it's supported in your Linux distro, although if you haven't used systemd before, you might want to avoid it.

Scholar ronnywebers
Scholar
680 Views
Registered: ‎10-10-2014

Re: Can I (re)configure the bitstream only after linux has booted on my Zynq ARM cores?

thanks @allanherriman, looks like you found a good way of working - still hoping that someone from Xilinx would shed a light on this too, so we know if they use the same method. 

I recently discovered this Xilinx wiki page (last edit 12 oct 2018, so very recent) : Solution ZynqMP PL Programming

might be interesting to have a look at ... I think it's more or less similar to what you guys do, they use something called 'device tree overlay' fragments:

"The Device Tree Overlay (DTO) is used to reprogram an FPGA while Linux is running. The DTO overlay will add the child node and the fragments from the .dtbo file to the base device tree,, The newly added device node/drivers will be probed after bitstream programming" 

That sounds a lot like what you described above.

looks like it's a work in progress, and not yet officially supported in 2018.2 ...

** kudo if the answer was helpful. Accept as solution if your question is answered **
0 Kudos
670 Views
Registered: ‎01-08-2012

Re: Can I (re)configure the bitstream only after linux has booted on my Zynq ARM cores?

... officially supported in 2018.2

It sounds like you are using Petalinux or some variant thereof.  We don't use any Xilinx distros here, and thus don't have to worry about whether Xilinx supports what we want to do.

 

P.S. Please fix the spelling mistake in your sig.

Scholar ronnywebers
Scholar
653 Views
Registered: ‎10-10-2014

Re: Can I (re)configure the bitstream only after linux has booted on my Zynq ARM cores?

thanks for the tip - I fixed the stupid spelling mistake (sorry, non-native speaker :-)

I'm still not sure about what way to follow : Petalinux - yocto (for Zynq US+) - or the 'OSL' linux flow (= 'manual build') . I started a thread on the subject here, still not clear to me what's the best way to follow .. but again, I'm relatively new to Linux, so sometimes 'I don't see the trees in the forest', as a saying in my country goes.

So far I used both Petalinux and the 'OSL' flow. For that last one we use the Xilinx linux kernel from their github and add our drivers/specifics to that. You say that you don't use any Xilinx distro - just to understand you correctly : do you mean you don't use the Xilinx Linux kernel? 

** kudo if the answer was helpful. Accept as solution if your question is answered **
0 Kudos