TCG-AR: Real-Time Multi-View Augmented Reality for Trading Card Game Streaming

2026-07-02

AI Summary

This work addresses the limited immersion of top-down views in current card game livestreams and the impracticality of mainstream augmented reality (AR) solutions that rely on costly specialized hardware. The authors propose a real-time, multi-view AR system that operates with an ordinary RGB camera, requiring neither physical markers nor dedicated devices. The system automatically detects, orients, and recognizes tabletop cards, renders virtual content, and generates a consolidated viewer-oriented broadcast view compatible with standard streaming software such as OBS. To eliminate manual annotation, the method employs a fully automated synthetic data training strategy. Evaluated on real-world datasets, the approach achieves high accuracy and real-time performance, demonstrating practical viability on commodity hardware. The code, trained models, and dataset are publicly released.

Abstract

Trading card games are increasingly played and broadcast online, yet live streams remain mostly limited to flat top-down footage of the playing area. Augmenting such streams with virtual models of the played cards would improve the viewing experience, but most existing systems rely on instrumented playing surfaces and embedded chips, which are costly and impractical for casual players and large-scale events. In this work, we present TCG-AR, a novel real-time pipeline that augments trading card games using ordinary RGB cameras alone, without any physical markers or specialized hardware. Our pipeline detects, orients, and identifies the cards on the board, renders virtual content onto each card across all views, and can additionally compose a broadcaststyle view that summarizes the game state for spectators, streaming the augmented feeds to standard broadcasting software such as OBS. To train the detection, orientation, and identification models without manual labeling, we introduce an automatic procedure that generates annotated synthetic training data from a reference set of card images. Then, we evaluate several trained models on a new manually annotated dataset with real images, analyzing performance and runtime throughput that determine real-world usability. Overall, by relying only on commodity cameras and hardware, and by open-sourcing all code, models, and datasets, this work aims to serve as a reference for real-time trading card recognition and to make real-time augmented-reality streaming accessible to the broader community of players and streamers.

Problem

• Trading Card Game
• Augmented Reality
• Live Streaming
• Real-Time Recognition
• Markerless Tracking

Innovation

• real-time augmented reality
• trading card game
• markerless tracking
• synthetic data generation
• multi-view streaming