Search papers, labs, and topics across Lattice.
This paper introduces TCG-AR, a real-time augmented reality system for trading card game streaming that utilizes standard RGB cameras to enhance live broadcasts without the need for specialized hardware or physical markers. The system employs a novel automatic procedure for generating annotated synthetic training data, allowing for effective card detection, orientation, and identification. Evaluation on a manually annotated dataset demonstrates the system's performance and usability, making augmented reality streaming more accessible for casual players and large-scale events.
Ordinary RGB cameras can now transform trading card game streams into immersive augmented reality experiences without costly hardware.
Trading card games are increasingly played and broadcast online, yet live streams remain mostly limited to flat top-down footage of the playing area. Augmenting such streams with virtual models of the played cards would improve the viewing experience, but most existing systems rely on instrumented playing surfaces and embedded chips, which are costly and impractical for casual players and large-scale events. In this work, we present TCG-AR, a novel real-time pipeline that augments trading card games using ordinary RGB cameras alone, without any physical markers or specialized hardware. Our pipeline detects, orients, and identifies the cards on the board, renders virtual content onto each card across all views, and can additionally compose a broadcaststyle view that summarizes the game state for spectators, streaming the augmented feeds to standard broadcasting software such as OBS. To train the detection, orientation, and identification models without manual labeling, we introduce an automatic procedure that generates annotated synthetic training data from a reference set of card images. Then, we evaluate several trained models on a new manually annotated dataset with real images, analyzing performance and runtime throughput that determine real-world usability. Overall, by relying only on commodity cameras and hardware, and by open-sourcing all code, models, and datasets, this work aims to serve as a reference for real-time trading card recognition and to make real-time augmented-reality streaming accessible to the broader community of players and streamers.
