PreGLAM-MMM: Using MMM in video games

Multi-Track Music Machine (MMM) is a generative music creation system based on Transformer architecture, developed by Jeff Enns and Philippe Pasquier. We apply MMM to create an AI-generated score for a video game, which adapts to the gameplay based on modeled emotional perception. Generative music has many potential benefits for games, such as allowing music to more closely match the events and actions of gameplay than with current game music techniques, while also reducing the work needed to compose music for games. It is our hope that this application of technology will allow game composers to spend less time preparing their scores for use in a game, and more time composing music.

We manually compose a score using the IsoVAT Composition Guide, which we use as input for MMM. MMM provides AI-generated variations on our score, expanding our score from 27 composed clips to ~14 trillion unique musical arrangements. These variations are controlled during gameplay by the PreGLAM affect model, using the Elias audio middleware program. This creates an AI-generated score that responds in real-time to match the audience's emotional perception of the gameplay.

We implement our generative score in Galactic Defense (GalDef). GalDef can be downloaded from itch.io, and the source code is available on GitHub. GalDef also has a "Music Explorer" mode, which can be used to listen to three different musical scores outside of gameplay.