Leveraging I/O Stalls for Efficient Scheduling in ANNS

2026-05-19

AI Summary

This work addresses the challenge in disk-based graph indexing for approximate nearest neighbor search, where achieving both low query latency and high update throughput is difficult due to I/O-induced CPU idle cycles. The authors propose LIOS, a novel framework that leverages CPU cycles idle during I/O stalls of search threads to execute recoverable subtasks of index updates. By controlling the probability of subtask deadline violations and dynamically allocating idle time, LIOS enables cooperative scheduling between queries and updates. This approach overcomes the traditional limitation of thread schedulers being oblivious to I/O idle periods, thereby meeting strict user-specified latency targets while improving insertion and deletion throughput by up to 2.68× and 2.18×, respectively.

Abstract

Disk-based graph indexes for approximate nearest neighbor search (ANNS) must serve latency-sensitive queries and throughput-demanding updates concurrently. We observe that over 40% of search-thread CPU time is spent stalling on disk I/O; such idle cycles are invisible to thread-level scheduling yet available for other work. We present LIOS(Leverage I/O Stall), a framework that executes index updates inside search-side I/O stall windows. LIOS introduces three techniques: (i) splitting each update into resumable subtasks small enough to fit within a single stall window; (ii) bounding the expected overrun of update subtasks to a given threshold; and (iii) dynamically adjusting the fraction of idle time devoted to updates to drive end-to-end search latency degradation toward a user-specified target. We integrate LIOS into two update-optimized ANNS systems, FreshDiskANN and OdinANN. LIOS achieves speedups of up to 2.68$\times$ in insertion and 2.18$\times$ in deletion, with search latency degradation maintained near the user-specified target.

Problem

• approximate nearest neighbor search
• disk-based indexing
• I/O stalls
• concurrent queries and updates
• latency-throughput tradeoff

Innovation

• I/O stall
• ANNS
• disk-based indexing
• concurrent updates
• latency-aware scheduling