FlintKV: A Fast Durable Storage Engine for Modern Databases

2026-07-02

AI Summary

Existing non-volatile memory (NVM)-based key-value stores commonly lack support for modern database interfaces such as snapshots, consistent iterators, and atomic batch operations. To address this limitation, this work proposes and implements FlintKV, a skip-list-based NVM-optimized storage engine that natively supports a full production-grade transactional API while guaranteeing persistent linearizability. FlintKV achieves this through a novel concurrency protocol that integrates multi-version concurrency control with flat combining, leveraging byte-addressable NVM, custom persistence mechanisms, and highly efficient data structures. Experimental results demonstrate that FlintKV significantly enhances performance, achieving up to 75% higher end-to-end throughput compared to state-of-the-art alternatives. The system can be deployed standalone or integrated as a functional enhancement module into existing NVM-based storage systems.

Abstract

Byte-addressable non-volatile memory (NVM) offers an opportunity to rethink storage engine architectures. While recent NVM key-value stores achieve high throughput for ingestion and point lookups, they omit or under-specify the support for the richer interface guarantees required by modern databases. Production key-value engines (e.g., RocksDB) provide point-in-time snapshots, consistent iterators, and atomic batches-features essential for implementing transactions and concurrency control. We present FlintKV, an NVM-optimized skiplist-based storage engine that natively supports the full API of production key-value stores. FlintKV supports both atomic batch writes and snapshot-consistent iteration efficiently while guaranteeing durable linearizability. FlintKV can be deployed standalone or its durable skiplist can be integrated into existing NVM stores to enhance their capabilities. Central to FlintKV is a novel flat-combining based concurrency control algorithm that leverages multi-versioning and carefully co-designed persistence mechanisms to ensure high performance and scalability. Our empirical evaluation shows that FlintKV can achieve up to a 75% improvement in end-to-end throughput over prior work.

Problem

• NVM
• key-value store
• snapshot consistency
• atomic batch
• durable linearizability

Innovation

• NVM
• durable linearizability
• flat-combining
• snapshot-consistent iteration
• atomic batches