FERA: Foil Fencing Referee Assistant Using Pose-Based Multi-Label Move Recognition and Rule Reasoning

2025-09-22

AI Summary

Fencing referees face challenges including high subjectivity, substantial human error, inconsistent rule enforcement, and limited on-site resources. To address these issues, this paper proposes an automated, rule-aware adjudication assistance method integrating pose-based action recognition with formal fencing priority-rule reasoning. Specifically, we introduce the first framework that jointly leverages multi-label action recognition and knowledge-distilled language models encoding official fencing priority rules, enabling interpretable decision-making. Kinematic features (101-dimensional) are extracted from 2D pose estimations and fed into a Transformer architecture augmented with a rule-embedded logical reasoning module. Five-fold cross-validation demonstrates a macro-F1 score of 0.549—significantly outperforming TCN, BiLSTM, and vanilla Transformer baselines. Our core contributions are: (1) an interpretable adjudication modeling paradigm; and (2) domain-knowledge-driven, joint action–rule reasoning for fencing-specific decision support.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Knowledge Representation and Reasoning: Argumentation
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The sport of fencing, like many other sports, faces challenges in refereeing: subjective calls, human errors, bias, and limited availability in practice environments. We present FERA (Fencing Referee Assistant), a prototype AI referee for foil fencing which integrates pose-based multi-label action recognition and rule-based reasoning. FERA extracts 2D joint positions from video, normalizes them, computes a 101-dimensional kinematic feature set, and applies a Transformer for multi-label move and blade classification. To determine priority and scoring, FERA applies a distilled language model with encoded right-of-way rules, producing both a decision and an explanation for each exchange. With limited hand-labeled data, a 5-fold cross-validation achieves an average macro-F1 score of 0.549, outperforming multiple baselines, including a Temporal Convolutional Network (TCN), BiLSTM, and a vanilla Transformer. While not ready for deployment, these results demonstrate a promising path towards automated referee assistance in foil fencing and new opportunities for AI applications, such as coaching in the field of fencing.

Problem

• Automating foil fencing refereeing to reduce human errors and bias
• Developing AI system for multi-label move recognition from video poses
• Applying rule reasoning to determine fencing priority and scoring decisions

Innovation

• Pose-based multi-label action recognition using Transformer
• Rule reasoning with distilled language model for decisions
• Kinematic feature extraction from normalized 2D joint positions