Spartan® Family FPGAs

[sjuliao]

Hi,

I have an Atlys (it is based in Spartan 6 XC6SLX45 with an Flash Numonyx N25Q12) and i need to implement some algorithms to process image adquired from VModCam.

One of these alggorithms is image segmentation where basically i have an LUT with 16.777.216 entries, addressable directly with each color component (8 bits Red, 8 bits Green, 8 bits Blue). Each entrie in LUT is 8 bit.

The LUT size is about 16MB so Block Ram is out of question and the DDR have poor latency (even when compared with flash, right?).

Apparently, the LUT can fit in Flash and i think it would be a good solution.

Since i don't want to use an MicroBlaze, i need to implement SPI to communicate with the Flash Memory.

I already tried with only 4 signals (SClk, CS, MOSI and MISO) but i can't read any value (apparently because SPI is configured in Quad SPI mode).

The UCF descrives the interface like 6 signals (SCK, CS and MemDq(4 downto 0)). When we use this kind of interface, we send and receive one bit in each signal, starting with de MSB, right? 

But now, i have one question that is boring me: since for each bit we send, i get an bit, is it possible to use the flash as i mentioned (24 bits address and only 8 bit data to be returned)?

Any help would be aprreciated.

Thanks for reading.

[eteam00]

One of these alggorithms is image segmentation where basically i have an LUT with 16.777.216 entries, addressable directly with each color component (8 bits Red, 8 bits Green, 8 bits Blue). Each entrie in LUT is 8 bit.

• Is your output format a 256-deep colour palette?  This is quite unusual, but this is what you seem to be describing.

• If you desire a 24-bit colour palette on output, then the output format is 8b/8b/8b (24b total) for R/G/B, not 8 bits total.  You would need a 16M x 24b array, not a 16M x 8b array.

• If you desire a per-channel LUT for each R/G/B channel, you need only three 256 x 8 LUTs (one for each colour channel).

• If you desire a grey-scale output format, there are simple arithmetic solutions for deriving Y (luminance) from RGB.  Or you can select the YCrCb format option from the VMODCAM, in which case the Y (luminance only) video data is provided directly.

Apparently, the LUT can fit in Flash and i think it would be a good solution.

Remember that the N25Q flash holds the FPGA configuration data for the S6LX45 device.  So you cannot use the entire 128Mbit flash memory for a lookup table.

You can use the DDR2 memory for a lookup table, but this memory is volatile.  From where are the table content data loaded or stored when the board is first powered on?

But now, i have one question that is boring me: since for each bit we send, i get an bit, is it possible to use the flash as i mentioned (24 bits address and only 8 bit data to be returned)?

Yes.  See Figure 15 in the N25Q128 device datasheet.  This plainly illustrates the quad-mode read access protocol.

-- Bob Elkind

[sjuliao]

Hi eteam00,

First of all, i appreciate your attention and answer.

• Is your output format a 256-deep colour palette?  This is quite unusual, but this is what you seem to be describing.

Yes, it is something like that. In fact, each of the 8 bit of output represents a class color (example: red, blue, green, orange, yellow, white, black, other). When i get a pixel from camera, i need to see which color(s) it represents and i put the desired bit to '1'. This way, for example, different combinations of a 24bit pixel can have the same class color. This is ok, since in the future i only need to know this kind of information.

Since this implies some runtime processing, i have created this LookUp Table (LUT) with all possible combinations of RGB (16.777.216 entries) from each, i then extract de byte representing the class color.

Remember that the N25Q flash holds the FPGA configuration data for the S6LX45 device.  So you cannot use the entire 128Mbit flash memory for a lookup table.

Yes, i noticed this in the Digilent documentation. But i think it is possible to load that configuration from BlockRam (if it is small enough - but i don't think will be my case) or from DDR (this needs something like an bootloader from other place, that can load the configuration to DDR and then runs from there), right?

Another option could be to reduce the LUT. I noticed that i can reduce the number of entries of the LUT (using only 5 bit R, 6 bit G and 5 bit B - this seem's to be acceptable to the human vision system). This way i can put the LUT in the Flash Memory, even with the configuration. But i need to confirm if this is acceptable for this project.

You can use the DDR2 memory for a lookup table, but this memory is volatile.  From where are the table content data loaded or stored when the board is first powered on?

 My doubt with the use of DDR for LUT is the associated latency. I need to use this LUT for every pixel in a frame adquired and i am afraid that this could cause an overhead in that port and a lag.

The Atlys has an HID USB port which lets you to attach a Pen with the .bit file and boot from that. I didn't search yet but it is possible that the LUT and another file can be loaded from there too (if i cannot use the Flash).

If you want, you can give your opinion or suggest a different solution. I am always open to new ideas.

Once more, thank you for your attention.

[eteam00]

Remember that the N25Q flash holds the FPGA configuration data for the S6LX45 device.  So you cannot use the entire 128Mbit flash memory for a lookup table.

Yes, i noticed this in the Digilent documentation. But i think it is possible to load that configuration from BlockRam (if it is small enough - but i don't think will be my case) or from DDR (this needs something like an bootloader from other place, that can load the configuration to DDR and then runs from there), right?

Where is the configuration data stored when the board is not powered?  When the board is first powered on, BlockRam and DDR2 contents are indeterminate.  BlockRAM is itself loaded from the configuration data, and hence it cannot store the configuration data.  Think about how to configure the FPGA on the ATLYS board, if not from the N25Q flash memory.

-- Bob Elkind

[joelby]

---

 The Atlys has an HID USB port which lets you to attach a Pen with the .bit file and boot from that. I didn't search yet but it is possible that the LUT and another file can be loaded from there too (if i cannot use the Flash).

---

The USB port emulates an SPI device of some sort (possibly SPI master - I'm a bit hazy on the details) and configures the FPGA that way. It's not really possible to load other data from it unless you were to rewrite the PIC microcontroller's firmware, and it'd be pretty slow.

Precisely how much latency are you actually willing to tolerate? Have you looked at the configuration flash memory's datasheet to see what its typical latency is? (I've never actually looked for the datasheet - I just did, and I couldn't find it. I couldn't even find any evidence that the N25Q12 actually exists. My board says "25Q2813840")

Unless you need latencies measured in nanoseconds, are you sure that DDR along with a bunch of BRAM caches wouldn't be faster and sufficient? If you do need low latencies, have you considered attaching an external SRAM? I'm not quite sure where you'd attach it if you already had a VMod board connected though.

How are you generating the LUT in the first place? Are the classes random, or is it algorithmically generated? Could the FPGA generate the values on the fly (perhaps with some run-time configurable knobs for tuning the algorithm)?

Normally if you were just classifying colours in to primary colours, you could convert RGB to the HSL colour space (for example). Below a certain threshold of lightness, it's black. Above a certain threshold of lightness, it's white. Otherwise, a hue of 320 to 20 degrees is red. And so on (these thresholds are the things you could adjust at run time).