Adaptive music in video games
Video games, by nature, are a non-linear form of entertainment. A player can, at any point, do anything within the bounds of the game environment and mechanics. While this can cause problems in regards to timing music with set pieces, it can also give the sound team the ability to change arrangements and compositions based upon the player’s position in the game environment. This process is called adaptive, and allows the developers to change which music is playing and how it sounds “on-the-fly” using the games audio engine.
However, adaptive music in video games isn’t a new concept. While there are examples dating back to the 1980s, the more modern approach we see now can arguably be attributed to 1991’s Monkey Island 2: LeChuck’s Revenge. Developers Michael Land and Peter McConnell of LucasArts developed a system called iMUSE (Interactive Music Streaming Engine) after becoming frustrated with the limitations of audio engines. Prior to iMUSE, game music transitioned statically between tracks. For example, in Super Mario Bros on NES, music only transitioned between certain areas and levels. There’s themes for above ground, underground, underwater and Bowser’s castle, however they all abruptly change when the player reaches a pipe or changes level.
With iMUSE, Land and McConnell were able to make adjustments to arrangements and compositions at any point in the game. The engine could perform these feats by allowing seamless transitions between pieces of music. In Monkey Island 2, as the main character moves through the game’s environment, different variations of the main theme play, even in the same area, such as moving between shops and non-playable characters. Interacting with NPCs can also affect the arrangement, usually by adding another instrument, such as an accordion when talking to pirates.
Nobuo Uematsu used a similar technique in Final Fantasy VI for the games conclusive fight against Kefka. “Dancing Mad” incorporates several different musical styles and strings them together into a cohesive song, similar to an orchestral movement. This is because “Dancing Mad” tells the story of Final Fantasy VI in one song, by the clever use of leitmotifs. As this track is telling a story, it’s important that it doesn’t run ahead of itself in relation to where the gamer is during the boss fight. To ascertain that the right section of the score plays at the right time, the track is designed so that its movements loop until the player defeats each phase, where it’ll progress onto the next movement. It’s adaptive music in its most rudimentary form, but works to powerfully convey the stories themes in the game’s climax.
Mario Kart 8 uses adaptive music in a different way. Instead of transitioning the whole piece into different movements, Mario Kart 8 instead adds and subtracts instruments from its arrangement. This is most noticeable once you progress out of the title menu, as the music changes from the main theme into a more subtle piece. At the game type menu, there’s a synth lead, with a subtle bassline and quiet percussion. Progress to the character selection menu, and the bassline and percussion becomes more prominent. Progressing to the kart selection adds a new layer of effects before the player finally reaching the course selection, where the percussion arrives in full force. The music is designed in this way to build tension and excitement towards the upcoming race.
Mario Kart 8 takes this a step further in its track themes. For example, “Mount Wario” uses four different arrangements as you progress down the mountain. At the top of the mountain, there’s a mix of brass and strings that give a sense of wonderment, as you’re about to explore the peaks.
When you reach the cave, the arrangement takes a more jazz style, focusing on the brass section as the track takes a more swing tempo. Upon reaching the wooded area, the arrangement changes again, as the string section is highlighted and takes lead over the composition, before reaching the final slopes of the mountain, where the arrangement changes a final time. Here, as the player progresses towards the finish line, all the elements come together in the culmination of the score to deliver a sense of excitement as you come toward the end of the race.
So far we’ve looked at horizontal re-sequencing where the music changes into other music based upon player actions (Final Fantasy VI’s “Dancing Mad”) and vertical remixing where instruments, tempo and pitch are changed as events happen (Mario Kart 8, Monkey Island 2). These systems are
used in a wide range of video games in some shape or form. But what happens when the game that you’re composing randomly generates itself from a series of 0s and 1s?
This was a problem band 65daysofstatic and sound engineer Paul Weir had to overcome when creating the soundtrack to No Man’s Sky. Unlike most soundtracks, a regular score wouldn’t be necessary for the quintillion of planets the players could explore. Unlike other games, there’s no set
pieces or environment to create a score for. Instead, 65daysofstatic had to create an album of music based off the game’s themes and concepts.
The 10 tracks composed for No Man’s Sky were then taken by Paul Weir and broken up into individual instrumentations and sequences. Each instrument and section of each track on the score was assigned certain flags within the game’s code. This meant that Weir was able to pull any
element of the album together at any given time, including piecing parts of different songs together. This allowed the creation of soundscapes to be randomly generated dependent on the player’s placement in the environment, and then manipulate the audio dependent on the player’s actions.
The result is a surreal, constantly adapting score that seems to always have the perfect sound for each moment in the game, by combining both horizontal and vertical types of adaptive music.
As the player can do anything at any time, being able to change arrangements and compositions “on-the-fly” allows composers to provide the perfect piece of music for any given point, whether it be to provide tension, emotion or playfulness within the game environment. Adaptive music provides an important development in video games in terms of music, moving away from static soundtracks to providing a more dynamic listening experience.
Posted on January 28, 2018 by Mathew Gallagher. Last modified on January 28, 2018.
About the Author
Mathew Gallagher Mathew Gallagher is a Sound Design graduate from the UK. During his degree, Mathew studied the latest in game audio techniques, including VR and Dolby Atmos. When not gouging out on JRPGs or racing games, you can find him listening to all sorts of soundtracks or retweeting game news. He's currently learning how to play violin, but it's safe to say his guitar playing is the better of his musical talents.
Related Posts
Final Fantasy VII Remake Acoustic Arrangements →
Fleeting Symphony →
Final Symphony II – Listener’s Guide →
