Accelerating RocksDB-based Databases with Pliops Extreme Data Processor
Cloud workloads and data continue to grow at accelerating rates, as does the demand for better performance, latency, and overall efficiency. Unfortunately, computing performance is not keeping pace, while solid-state drives (SSDs) are getting cheaper, denser, and faster—especially NVMe SSDs.
Today’s compute and storage architecture is unable to take full advantage of SSDs, making it more challenging to keep up with growing demands. Nowhere is this more evident with applications like MySQL, Ceph, and Spark using RocksDB, an open-source, key-value (KV) storage engine optimized for fast, low latency SSDs. CPU limitations due to higher computational loads and excessive storage I/O consumes valuable compute and storage resources. Without a better technology solution, enterprises have no choice but to deploy more servers and storage—ballooning capital and operating expenses. The Pliops Extreme Data Processor (XDP) is a breakthrough hardware-accelerated key-value storage engine that addresses the limitation of a software-only approach and delivers superior database performance and infrastructure utilization.
The primary performance bottleneck for RocksDB is the write and read amplification that stem from the Log Structure Merge-Tree (LSM) data structure. Each key update in RocksDB leads to many cascading writes in the LSM tree resulting in high write amplification. Sustaining writes to RocksDB requires dedicating a significant portion of I/O bandwidth to perform compactions concurrently. To read a key from the LSM tree, multiple levels need to be verified during read operations resulting in much higher read tail latencies. The compaction process reads several files into memory, sorts them and writes them back at the expense of storage and CPU resources.
