Search papers, labs, and topics across Lattice.
This study introduces an equipment-centric framework for continuous workpiece localization in hot forging environments, leveraging deep learning to analyze equipment operations from video streams. By shifting the focus from workpiece-centric to equipment-centric localization, the approach effectively addresses challenges posed by extreme temperatures and irregular routing, achieving 100% event detection accuracy and a mean localization error of 317.8 mm. The integration of Keypoint-Guided Attention (KPGA) enhances activity recognition performance, bridging vision-based perception with event-driven reasoning for improved traceability and quality control in forging operations.
Achieving 100% event detection accuracy in extreme conditions, this framework transforms how we localize workpieces in hot forging environments.
Continuous workpiece localization is critical for ensuring traceability, quality control, and process coordination in hot forging environments, where heavy workpieces are repeatedly transferred by specialized handling equipment such as overhead cranes among storage yards, furnaces, and forging stations. However, traditional workpiece-centric approaches are often unreliable under extreme temperatures, surface degradation, and irregular routing conditions, leaving a critical gap in continuous localization. To overcome these challenges, this study presents an equipment-centric framework that infers workpiece locations indirectly by analyzing equipment operations captured through video streams from multiple static 2D cameras. This paradigm shift from workpiece-centric to equipment-centric localization enables robust inference under harsh conditions by coupling equipment behavior with workpiece-handling events. The framework estimates floorplan-space 3D coordinates of handling equipment and recognizes workpiece-handling activities such as grasp and release. Event-driven finite state machines (FSMs) validate and detect discrete handling events based on equipment coordinates and recognized activity cues, enabling continuous inference and updating of workpiece states and floorplan-space coordinates. Experimental validation in an operational hot forging factory demonstrated the effectiveness and practicality of the proposed framework, achieving 100 % event detection accuracy within a 33-second tolerance window, a mean localization error of 317.8 mm, and a mean system latency of 21 seconds. Ablation studies further confirmed that integrating the Keypoint-Guided Attention (KPGA) mechanism improves activity recognition performance compared to baseline 3D convolutional neural networks (3D-CNNs) and transformer-based models. Beyond localization accuracy, the structured outputs of the framework bridge vision-based perception and event-driven reasoning by representing handling operations as interpretable state transitions. This capability enables data-driven visualization of workpiece transfers and quantitative evaluation of equipment handling states, contributing to more intelligent and traceable forging operations.
