Search papers, labs, and topics across Lattice.
This paper explores the use of vision-language models (VLMs) to estimate horizontal (H) and vertical (V) hand distances from RGB video for ergonomic risk assessment, specifically for use in the Revised NIOSH Lifting Equation (RNLE). Two VLM pipelines were developed: a detection-only and a detection-plus-segmentation approach, both employing text-guided localization and transformer-based temporal regression. The segmentation-based pipeline, leveraging multi-view data, achieved mean absolute errors of 6-8 cm for H and 5-8 cm for V, demonstrating the feasibility of VLMs for non-invasive ergonomic assessment.
Skip the tape measure: VLMs can estimate hand distances in lifting tasks from video with ~6cm accuracy, enabling automated ergonomic risk assessment.
Manual lifting tasks are a major contributor to work-related musculoskeletal disorders, and effective ergonomic risk assessment is essential for quantifying physical exposure and informing ergonomic interventions. The Revised NIOSH Lifting Equation (RNLE) is a widely used ergonomic risk assessment tool for lifting tasks that relies on six task variables, including horizontal (H) and vertical (V) hand distances; such distances are typically obtained through manual measurement or specialized sensing systems and are difficult to use in real-world environments. We evaluated the feasibility of using innovative vision-language models (VLMs) to non-invasively estimate H and V from RGB video streams. Two multi-stage VLM-based pipelines were developed: a text-guided detection-only pipeline and a detection-plus-segmentation pipeline. Both pipelines used text-guided localization of task-relevant regions of interest, visual feature extraction from those regions, and transformer-based temporal regression to estimate H and V at the start and end of a lift. For a range of lifting tasks, estimation performance was evaluated using leave-one-subject-out validation across the two pipelines and seven camera view conditions. Results varied significantly across pipelines and camera view conditions, with the segmentation-based, multi-view pipeline consistently yielding the smallest errors, achieving mean absolute errors of approximately 6-8 cm when estimating H and 5-8 cm when estimating V. Across pipelines and camera view configurations, pixel-level segmentation reduced estimation error by approximately 20-30% for H and 35-40% for V relative to the detection-only pipeline. These findings support the feasibility of VLM-based pipelines for video-based estimation of RNLE distance parameters.
