Immersive feedback in fencing weapon practice using mixed reality

Abstract

Providing athletes, during sports training, with real-time feedback based on automatic analysis of motion is both useful and challenging. In this work, a novel system based on mixed reality is proposed and verified. The system allows for immersive, real-time, visual feedback in fencing weapon practice. Novel methods are introduced for 3D blade tracking from a single RGB camera, creating weapon action models by recording actions performed by the coach and evaluating fencers' performance against these models. Augmented reality glasses with see-through displays are employed and a method for coordinate mapping between virtual and real environments is proposed, which allows providing real-time visual cues and feedback by overlaying virtual trajectories on the real-world view. The system is verified experimentally in fencing bladework practice, with supervision of a fencing coach. Results indicate that the proposed system allows novice fencers to perform the exercises more correctly.