The SSD Guy, Jim Handy, reports that Baidu—China’s leading search engine company—has created a radical SSD architecture called “Software-Defined Flash” (SDF) that maximally exploits the inherent performance of the SSD’s Flash memory chips. It does this by exposing the Flash chips’ individual channels to host software running on a server, allowing the server to organize its own data and better schedule data accesses. The experimenters report that their design approach extracts 95% of the raw bandwidth from the Flash chips while making 99% of the Flash memory capacity available for user data. These results appear in a paper titled “SDF: Software-Defined Flash for Web-Scale Internet Storage Systems,” presented at the ASPLOS conference held last year in Salt Lake City.
Baidu’s data centers store hundreds of petabytes of data with a daily data processing volume reaching dozens of petabytes. With an ever-increasing performance requirements for storage systems, Baidu is discovering that hard disks and conventional SSDs are becoming inadequate. The SDF represents Baidu’s latest effort to increase data-center performance.
A radically different Flash-based storage architecture like the SDF presented in this paper cannot use conventional SSD controller ASSPs, so the development team created a custom SDF architecture and built a PCIe SDF board based on Xilinx Virtex-5 and Spartan-6 FPGAs as shown in the diagram below:
Baidu SDF controller architecture
The Virtex-5 FPGA implements the SDF’s PCIe DMA and interface, performs chip-to-chip bridging among the FPGAs, and serves as the master controller for four Spartan-6 FPGAs. Each Spartan-6 FPGA implements eleven channels of independent Flash translation layer (FTL) control for a total of 44 channels per board. Each Flash channel controls two commodity Micron 8Gbyte MLC Flash chips, resulting in a total capacity of 704Gbytes per SDF board.
The design team expended significant effort to reduce board costs by including only required features as opposed to commercial SSDs that are designed for general-purpose storage and varying workloads. For example, the SDFs are mainly used as hard-disk caches and data that is rarely accessed is not expected to reside in the cache for a long period of time, so the SDF does not conduct static wear leveling.
In addition, the SDF hardware does not include a DRAM cache while conventional SSDs usually include a large DRAM cache to reduce access latency. Data consistency can be compromised by a power failure when the data resides in a DRAM cache, so a cached controller design must include a battery or capacitor to prevent data loss by making it possible to ride through power outages or by allowing time to shut down gracefully. However, batteries and capacitors add hardware cost. In Baidu’s storage infrastructure, server host memory caches recently accessed data, so the SDF design eliminates the DRAM cache and the associated costs of that cache memory.
The Baidu experimenters conducted a number of benchmark tests on the SDF design and the paper referenced above provides several pages of detailed results. The paper concludes:
“Our experimental measurements show that SDF can deliver about 95% of the raw Flash bandwidth and provide 99% of the Flash capacity for user data. SDF increases the I/O bandwidth by 3 times and reduces per-GByte hardware cost by 50% on average compared with Baidu’s commodity-SSD-based system.”