Search papers, labs, and topics across Lattice.
This paper introduces RAG-HAR+, an innovative approach to Human Activity Recognition (HAR) that leverages retrieval-augmented generation to minimize reliance on large language models (LLMs) while enhancing retrieval accuracy. By employing an offline Retrieval Designer Agent to create dataset-specific feature groups, RAG-HAR+ optimizes the comparison of sensor windows, allowing for efficient classification with reduced computational costs. The method demonstrates competitive performance across six HAR benchmarks, significantly lowering LLM usage and inference time, making it suitable for edge deployment in mobile sensing applications.
RAG-HAR+ achieves competitive HAR performance while slashing LLM usage and inference time, paving the way for efficient edge deployment.
Human Activity Recognition (HAR) from wearable sensors supports applications in healthcare, rehabilitation, fitness tracking, and smart environments. Yet, existing deep learning approaches require dataset-specific training, large labeled corpora, and repeated adaptation to new sensor settings or activity taxonomies. Retrieval-Augmented Generation for Human Activity Recognition (RAG-HAR) addresses this by framing HAR as a training-free, retrieval-augmented task, in which statistical descriptions of sensor windows are used to retrieve similar labeled examples that guide LLM-based classification. We introduce RAG-HAR+, a retrieval-first and cost-optimized extension that strengthens retrieval while reducing dependence on LLM-based inference. RAG-HAR+ uses an offline Retrieval Designer Agent to design dataset-specific feature groups from a diverse pool of motion descriptors, enabling sensor windows to be compared using features better aligned with dataset-specific activity patterns. During inference, RAG-HAR+ uses majority voting over retrieved neighbors for samples with strong retrieval evidence and defers only uncertain cases to an LLM-based Ambiguity Resolver Agent. Across six HAR benchmarks, RAG-HAR+ maintains competitive or improved performance while reducing LLM usage, token consumption, and inference time. We further extend the RAG-HAR mobile prototype to demonstrate the practical feasibility of retrieval-first, LLM-assisted HAR in mobile sensing scenarios.
