Large Language Models are Zero-Shot Recognizers for Activities of Daily Living

2024-07-01

arXiv.org

AI Summary

This study addresses the bottleneck in smart home activity recognition—its heavy reliance on large-scale labeled sensor data—for Activities of Daily Living (ADLs). We propose the first zero-shot sensor-data recognition framework leveraging Large Language Models (LLMs). Our method converts raw, heterogeneous sensor time-series into natural language descriptions and employs prompt engineering to guide the LLM in direct ADL classification, supporting both zero-shot and few-shot prompting while enabling cross-modal semantic alignment. Experiments on two public ADL datasets demonstrate that our approach achieves performance comparable to supervised deep learning models under zero-shot settings, drastically reducing annotation costs. Notably, this work provides the first empirical validation of LLMs’ effectiveness for purely sensor-driven, domain-unfine-tuned ADL recognition—establishing a novel paradigm for low-resource intelligent environment perception.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

The sensor-based recognition of Activities of Daily Living (ADLs) in smart home environments enables several applications in the areas of energy management, safety, well-being, and healthcare. ADLs recognition is typically based on deep learning methods requiring large datasets to be trained. Recently, several studies proved that Large Language Models (LLMs) effectively capture common-sense knowledge about human activities. However, the effectiveness of LLMs for ADLs recognition in smart home environments still deserves to be investigated. In this work, we propose ADL-LLM, a novel LLM-based ADLs recognition system. ADLLLM transforms raw sensor data into textual representations, that are processed by an LLM to perform zero-shot ADLs recognition. Moreover, in the scenario where a small labeled dataset is available, ADL-LLM can also be empowered with few-shot prompting. We evaluated ADL-LLM on two public datasets, showing its effectiveness in this domain.

Problem

• Investigates LLMs for zero-shot ADLs recognition in smart homes
• Transforms raw sensor data into textual representations for LLM processing
• Evaluates ADL-LLM effectiveness with few-shot prompting on public datasets

Innovation

• LLM-based zero-shot ADLs recognition system
• Transforms sensor data into textual representations
• Empowers few-shot prompting with small datasets