NeuralPrefix: A Zero-shot Sensory Data Imputation Plugin

2025-02-09

AI Summary

Intermittent data loss—caused by sensor faults, communication failures, and power constraints—violates the continuity assumption underlying conventional time-series classification. Method: We propose the first zero-shot cross-modal data imputation framework, modeling imputation as a differentiable continuous dynamical system solved via neural ODEs to infer latent state evolution, integrated with a generative prefix architecture for temporal adaptation. No target-sensor fine-tuning or downstream task training is required. Contribution/Results: We introduce the first zero-shot sensory data imputation paradigm, eliminating modality- and device-specific dependencies. Experiments demonstrate robust performance under 50% missingness, achieving SSIM scores of 0.93–0.96 and strong cross-modal zero-shot generalization. The framework delivers a plug-and-play, resource-efficient solution for reliable data recovery in edge-constrained sensing environments.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Data transparency and provenance

Abstract

Real-world sensing challenges such as sensor failures, communication issues, and power constraints lead to data intermittency. An issue that is known to undermine the traditional classification task that assumes a continuous data stream. Previous works addressed this issue by designing bespoke solutions (i.e. task-specific and/or modality-specific imputation). These approaches, while effective for their intended purposes, had limitations in their applicability across different tasks and sensor modalities. This raises an important question: Can we build a task-agnostic imputation pipeline that is transferable to new sensors without requiring additional training? In this work, we formalise the concept of zero-shot imputation and propose a novel approach that enables the adaptation of pre-trained models to handle data intermittency. This framework, named NeuralPrefix, is a generative neural component that precedes a task model during inference, filling in gaps caused by data intermittency. NeuralPrefix is built as a continuous dynamical system, where its internal state can be estimated at any point in time by solving an Ordinary Differential Equation (ODE). This approach allows for a more versatile and adaptable imputation method, overcoming the limitations of task-specific and modality-specific solutions. We conduct a comprehensive evaluation of NeuralPrefix on multiple sensory datasets, demonstrating its effectiveness across various domains. When tested on intermittent data with a high 50% missing data rate, NeuralPreifx accurately recovers all the missing samples, achieving SSIM score between 0.93-0.96. Zero-shot evaluations show that NeuralPrefix generalises well to unseen datasets, even when the measurements come from a different modality.

Problem

• Develops zero-shot imputation for sensory data.
• Addresses data intermittency in various sensor modalities.
• Enables pre-trained models to handle missing data.

Innovation

• NeuralPrefix enables zero-shot sensory data imputation.
• It uses a generative neural component for data recovery.
• Built as a continuous dynamical system with ODEs.