Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study introduces ConsensusTAS, a self-supervised learning framework designed to segment long-horizon construction videos into distinct activity phases without the need for labeled data. By leveraging the internal consensus of candidate segmentations, the method effectively identifies fine-grained transitions in construction activities, outperforming existing state-of-the-art approaches on multiple datasets. Notably, it achieves competitive F1 scores while maintaining efficiency, allowing it to run on standard CPUs, making it suitable for real-world applications in human-robot collaboration.
Self-supervised learning can segment complex construction activities in real-time without the burden of extensive labeling, achieving impressive accuracy on challenging datasets.
Recognizing sequential construction activities is important for collaborative human-robot work; for example, robots are able to understand workers'current and upcoming actions and provide timely tool delivery or physical support. However, despite extensive research on construction worker activity recognition, existing studies have been limited to classifying activity categories, such as climbing, lifting, and walking, instead of recognizing fine-grained activity transitions from long-horizon sequences. Addressing this problem is challenging because annotating action temporal boundaries in long construction videos is time-consuming. In this study, we propose ConsensusTAS, a label-free, self-supervised learning approach to segment continuous video streams into distinct activity phases by exploiting the internal consensus of candidate segmentations. We evaluated our algorithm on three public datasets, where it outperformed state-of-the-art methods, achieving an F1@10 of 73.08 on GTEA, an F1@10 of 64.33 on Breakfast, and an F1@50 of 33.50 on static-camera videos from Assembly101. We also tested it on real-world construction videos, where post-hoc evaluation showed that the model successfully recognized and segmented actions within the composite activity of bricklaying, such as spreading mortar on a brick, placing the brick, pressing, and aligning. Compared with other temporal action segmentation models that require computationally intensive large vision-language models, our method can run on a CPU, which provides practical value for video surveillance and human-robot collaboration on mobile robotic platforms.