CAMEO: Autocorrelation-Preserving Line Simplification for Lossy Time Series Compression

2025-01-24

AI Summary

To address the problem that lossy compression of sensor and IoT time-series data distorts autocorrelation functions (ACF/PACF), leading to unpredictable degradation in downstream analytical tasks (e.g., forecasting), this paper proposes the first lossy compression method explicitly preserving statistical features. Our approach integrates an enhanced Douglas–Peucker polyline simplification, incremental ACF/PACF aggregation updates, and block-wise parallel computation—achieving, for the first time, theoretically bounded ACF/PACF approximation error and supporting multivariate extension. Experiments demonstrate an average 2× improvement in compression ratio, with peak gains up to 54×; remarkably, prediction accuracy remains stable or even improves under high compression, empirically validating the critical role of autocorrelation preservation for analytical utility. This work bridges statistical modeling and data compression, delivering both rigorous theoretical guarantees and practical scalability.

Technology Category

• Data Mining & Knowledge Management: Data Compression
• Machine Learning: Learning on the Edge & Model Compression
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

Time series data from a variety of sensors and IoT devices need effective compression to reduce storage and I/O bandwidth requirements. While most time series databases and systems rely on lossless compression, lossy techniques offer even greater space-saving with a small loss in precision. However, the unknown impact on downstream analytics applications requires a semi-manual trial-and-error exploration. We initiate work on lossy compression that provides guarantees on complex statistical features (which are strongly correlated with the accuracy of the downstream analytics). Specifically, we propose a new lossy compression method that provides guarantees on the autocorrelation and partial-autocorrelation functions (ACF/PACF) of a time series. Our method leverages line simplification techniques as well as incremental maintenance of aggregates, blocking, and parallelization strategies for effective and efficient compression. The results show that our method improves compression ratios by 2x on average and up to 54x on selected datasets, compared to previous lossy and lossless compression methods. Moreover, we maintain -- and sometimes even improve -- the forecasting accuracy by preserving the autocorrelation properties of the time series. Our framework is extensible to multivariate time series and other statistical features of the time series.

Problem

• Time Series Compression
• Data Accuracy
• Lossy Compression

Innovation

• CAMEO
• lossy compression
• time series data