Audio Commentary System for Real-Time Racing Game Play

Tatsuya Ishigaki, Goran Topić, Yumi Hamazono, Ichiro Kobayashi, Yusuke Miyao, Hiroya Takamura


In Sessions:

GenChal Poster Session + demos, Simon Mille: (Thursday, 17:35 CEST, Foyer , Chat on Discord )

Abstract: Live commentaries are essential for enhancing spectators' enjoyment and understanding during sports events or e-sports streams. We introduce a live audio commentator system designed specifically for a racing game, driven by the high demand in the e-sports field. While a player is playing a racing game, our system tracks real-time user play data including speed and steer rotations, and generates commentary to accompany the live stream. Human evaluation suggested that generated commentary enhances enjoyment and understanding of races compared to streams without commentary. Incorporating additional modules to improve diversity and detect irregular events, such as course-outs and collisions, further increases the preference for the output commentaries.