Hybrid Attention Model Using Feature Decomposition and Knowledge Distillation for Glucose Forecasting

2024-11-16

arXiv.org

AI Summary

To address challenges in real-time diabetes management—including asynchronous multimodal data, irregular sampling, difficulty in long-term glucose prediction, and constraints on edge deployment—this paper proposes GlucoNet, a lightweight and efficient predictive model. Methodologically: (1) it introduces a novel feature-decomposed Transformer architecture that decouples sparse behavioral inputs from continuous physiological signals; (2) it proposes a mathematically continuous time-mapping method for irregularly sampled time-series data; and (3) it integrates high-/low-frequency signal decomposition with knowledge distillation to jointly optimize prediction accuracy and model compactness. Evaluated on clinical data from 12 individuals with type 1 diabetes, GlucoNet achieves a 60% reduction in RMSE and a 57% reduction in MAE compared to state-of-the-art methods, while reducing model parameters by 21%. It satisfies stringent edge-deployment requirements, enabling millisecond-level real-time inference on resource-constrained devices.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

The availability of continuous glucose monitors as over-the-counter commodities have created a unique opportunity to monitor a person's blood glucose levels, forecast blood glucose trajectories and provide automated interventions to prevent devastating chronic complications that arise from poor glucose control. However, forecasting blood glucose levels is challenging because blood glucose changes consistently in response to food intake, medication intake, physical activity, sleep, and stress. It is particularly difficult to accurately predict BGL from multimodal and irregularly sampled data and over long prediction horizons. Furthermore, these forecasting models must operate in real-time on edge devices to provide in-the-moment interventions. To address these challenges, we propose GlucoNet, an AI-powered sensor system for continuously monitoring behavioral and physiological health and robust forecasting of blood glucose patterns. GlucoNet devises a feature decomposition-based transformer model that incorporates patients' behavioral and physiological data and transforms sparse and irregular patient data (e.g., diet and medication intake data) into continuous features using a mathematical model, facilitating better integration with the BGL data. Given the non-linear and non-stationary nature of BG signals, we propose a decomposition method to extract both low and high-frequency components from the BGL signals, thus providing accurate forecasting. To reduce the computational complexity, we also propose to employ knowledge distillation to compress the transformer model. GlucoNet achieves a 60% improvement in RMSE and a 21% reduction in the number of parameters, improving RMSE and MAE by 51% and 57%, using data obtained involving 12 participants with T1-Diabetes. These results underscore GlucoNet's potential as a compact and reliable tool for real-world diabetes prevention and management.

Problem

• Forecasting blood glucose levels from multimodal, irregular data.
• Real-time operation on edge devices for immediate interventions.
• Improving accuracy and reducing computational complexity in glucose prediction.

Innovation

• Feature decomposition-based transformer model for glucose forecasting
• Knowledge distillation to compress transformer model complexity
• Integration of behavioral and physiological data for accurate predictions