Improving Zero-shot ADL Recognition with Large Language Models through Event-based Context and Confidence

2026-01-13

AI Summary

This work addresses key limitations in existing zero-shot activities of daily living (ADL) recognition methods based on large language models (LLMs), which typically rely on temporal segmentation that poorly aligns with LLMs’ contextual reasoning capabilities and lack effective confidence estimation mechanisms. To overcome these issues, the authors propose an event-driven contextual segmentation strategy that replaces conventional fixed time windows, along with a novel confidence estimation algorithm capable of distinguishing between correct and incorrect predictions. Experimental results on complex real-world datasets demonstrate that the proposed approach not only significantly outperforms current zero-shot methods but also surpasses several supervised baselines. Moreover, the introduced confidence metric effectively reflects the reliability of model predictions, offering a practical tool for assessing prediction trustworthiness in deployment scenarios.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Segmentation
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Unobtrusive sensor-based recognition of Activities of Daily Living (ADLs) in smart homes by processing data collected from IoT sensing devices supports applications such as healthcare, safety, and energy management. Recent zero-shot methods based on Large Language Models (LLMs) have the advantage of removing the reliance on labeled ADL sensor data. However, existing approaches rely on time-based segmentation, which is poorly aligned with the contextual reasoning capabilities of LLMs. Moreover, existing approaches lack methods for estimating prediction confidence. This paper proposes to improve zero-shot ADL recognition with event-based segmentation and a novel method for estimating prediction confidence. Our experimental evaluation shows that event-based segmentation consistently outperforms time-based LLM approaches on complex, realistic datasets and surpasses supervised data-driven methods, even with relatively small LLMs (e.g., Gemma 3 27B). The proposed confidence measure effectively distinguishes correct from incorrect predictions.

Problem

• Zero-shot ADL recognition
• Large Language Models
• Event-based segmentation
• Prediction confidence
• Smart home sensing

Innovation

• zero-shot ADL recognition
• event-based segmentation
• Large Language Models
• prediction confidence estimation
• IoT sensor data