Multimodal Data Integration for Sustainable Indoor Gardening: Tracking Anyplant with Time Series Foundation Model

2025-03-27

AI Summary

Urban food security and sustainable green building development necessitate automated, reliable monitoring and prediction of plant water stress in indoor horticulture. Method: We propose a multimodal fusion framework integrating high-resolution RGB visual features, dynamic size-to-surface-area phenotypic metrics, and microenvironmental sensor data. Crucially, we pioneer the adaptation of the Lag-Llama foundation model—a state-of-the-art time-series architecture—to plant health forecasting, enabling joint modeling across heterogeneous data sources to substantially reduce predictive uncertainty. Contribution/Results: Fine-tuned on real-world indoor cultivation data, our model achieves MSE = 0.4208 and MAE = 0.5954 on the test set, significantly outperforming unimodal baselines in both accuracy and robustness. This work establishes an interpretable, deployable multimodal time-series modeling paradigm for intelligent horticultural systems, directly supporting resource-efficient operation and building-scale smart facility management.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Indoor gardening within sustainable buildings offers a transformative solution to urban food security and environmental sustainability. By 2030, urban farming, including Controlled Environment Agriculture (CEA) and vertical farming, is expected to grow at a compound annual growth rate (CAGR) of 13.2% from 2024 to 2030, according to market reports. This growth is fueled by advancements in Internet of Things (IoT) technologies, sustainable innovations such as smart growing systems, and the rising interest in green interior design. This paper presents a novel framework that integrates computer vision, machine learning (ML), and environmental sensing for the automated monitoring of plant health and growth. Unlike previous approaches, this framework combines RGB imagery, plant phenotyping data, and environmental factors such as temperature and humidity, to predict plant water stress in a controlled growth environment. The system utilizes high-resolution cameras to extract phenotypic features, such as RGB, plant area, height, and width while employing the Lag-Llama time series model to analyze and predict water stress. Experimental results demonstrate that integrating RGB, size ratios, and environmental data significantly enhances predictive accuracy, with the Fine-tuned model achieving the lowest errors (MSE = 0.420777, MAE = 0.595428) and reduced uncertainty. These findings highlight the potential of multimodal data and intelligent systems to automate plant care, optimize resource consumption, and align indoor gardening with sustainable building management practices, paving the way for resilient, green urban spaces.

Problem

• Automated monitoring of plant health using multimodal data integration
• Predicting plant water stress with RGB, phenotyping, and environmental data
• Enhancing indoor gardening sustainability via AI and IoT technologies

Innovation

• Integrates computer vision and environmental sensing
• Uses Lag-Llama time series model
• Combines RGB, phenotyping, and environmental data