Extracting Actionable Insights from Building Energy Data using Vision LLMs on Wavelet and 3D Recurrence Representations

2025-09-26

AI Summary

Building energy consumption time series exhibit strong nonlinearity and multiscale characteristics, leading to poor interpretability and challenges in anomaly detection. Method: This paper proposes a novel approach integrating wavelet–recurrence plot (RP) 3D visualization with vision-language models (VLMs). Specifically, it jointly applies continuous wavelet transform (CWT) and RP to construct an interpretable 3D time-series representation, then fine-tunes VLMs (e.g., Idefics-7B) end-to-end to directly generate energy-saving recommendations and anomaly diagnoses from the visualized representations. Contribution/Results: The method departs from conventional purely numerical modeling paradigms. Evaluated on real-world building datasets, the CWT-RP representation reduces validation loss to 0.0952—significantly lower than 0.1176 for raw sequences—while markedly improving pattern interpretability, anomaly detection accuracy, and recommendation quality. This work establishes a generalizable, vision-based paradigm for intelligent building energy analysis.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Representation Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

The analysis of complex building time-series for actionable insights and recommendations remains challenging due to the nonlinear and multi-scale characteristics of energy data. To address this, we propose a framework that fine-tunes visual language large models (VLLMs) on 3D graphical representations of the data. The approach converts 1D time-series into 3D representations using continuous wavelet transforms (CWTs) and recurrence plots (RPs), which capture temporal dynamics and localize frequency anomalies. These 3D encodings enable VLLMs to visually interpret energy-consumption patterns, detect anomalies, and provide recommendations for energy efficiency. We demonstrate the framework on real-world building-energy datasets, where fine-tuned VLLMs successfully monitor building states, identify recurring anomalies, and generate optimization recommendations. Quantitatively, the Idefics-7B VLLM achieves validation losses of 0.0952 with CWTs and 0.1064 with RPs on the University of Sharjah energy dataset, outperforming direct fine-tuning on raw time-series data (0.1176) for anomaly detection. This work bridges time-series analysis and visualization, providing a scalable and interpretable framework for energy analytics.

Problem

• Analyzing nonlinear multi-scale building energy time-series data
• Converting 1D energy data into 3D visual representations
• Detecting anomalies and providing energy efficiency recommendations

Innovation

• Fine-tunes VLLMs on 3D graphical data representations
• Converts time-series using wavelet transforms and recurrence plots
• Enables visual interpretation for anomaly detection and recommendations