GenerationMania: Learning to Semantically Choreograph

06/28/2018

∙

by   Zhiyu Lin, et al.

∙

0

∙

share

Beatmania is a rhythm action game where players play the role of a DJ that performs music by pressing specific controller buttons to mix "Keysounds" (audio samples) at the correct time, unlike other rhythm action games such as Dance Dance Revolution. It has an active amateur Chart (Game stage) creation community, though chart authoring is considered a difficult and time consuming task. We present a deep neural network based process for automatically generating Beatmania charts for arbitrary pieces of music. Given a raw audio track of a song, we identify notes according to instrument, and use a neural network to classify each note as playable or non-playable. The final chart is produced by mapping playable notes to controls. We achieve a high level of performance, beating LSTM baselines.