DigiPen isn’t your typical college, and our course catalogue reflects that. In our Class Select series, we’ll explore a few of the many interesting and unusual courses we offer our students.
Lawrence Schwedler, director of DigiPen’s BA in Music and Sound Design program, compares composing a film score to being an “omniscient god.” “You know everything that’s going to happen at every moment in time,” Schwedler explains. “In a linear fashion, from beginning to end, you know when the jump scare is, when the romantic scene is about to unfold, and when the explosion happens.”
But when it comes to video games, the player is the all-knowing deity, literally controlling the action from moment to moment. “As a composer, it’s devilishly tricky!” Schwedler says. “Writing a musical score for a video game has to be, by its very definition, nonlinear, and adapt to changing circumstances driven by the player’s interaction.” Mastering the advanced techniques necessary to pull off such a feat is the core of DigiPen’s MUS 322 Adaptive Music for Video Games course, where students learn how to compose for the unpredictable nature of interactive media.
Even for experts, there’s always more to learn.
By the time music and sound design students arrive at the upper-level, third year course, the concept of adaptive music is far from new to them. “We talk about it their first week as freshmen, and they start to create simple adaptive music with their own original material as soon as their second semester at DigiPen,” Schwedler says. However, MUS 322 is the first time BA in Music and Sound Design students dive deep into the more sophisticated tools and techniques necessary to create an adaptive music score for an entire video game.
Students create adaptive music scores through Audiokinetic Wwise, the industry standard software for implementing video game music and sound effects. Known as “audio middleware,” Wwise integrates with game engines like Unity and Unreal, enabling pieces of music and sound effects to be triggered by specified events in the game. “It’s a massive, rich, deep tool. Even for experts, there’s always more to learn, but there’s a certain level of complexity that this class gets students to address with it,” Schwedler says.
Composing great music is only the first half of the equation when soundtracking a game. A lot of the work comes in what Schwedler calls music design — an intricate process students spend much of the class picking apart. “You can think of it as a kind of state machine,” Schwedler says. “Say you’re in a state of exploration in the game, so you have music invoking a neutral, emotional mood. Then maybe you spot an enemy in the distance and pass within a certain vicinity of it, triggering a pre-combat state where the music tempo picks up.”
By creating game flow diagrams, students map out these different potential player states and are challenged to not only write music for each, but sleuth out natural ways to transition between them all. The complexity can ramp up quickly, but Schwedler gives students some concrete adaptive music “schemes” to make things more manageable. In separate projects, students try composing using each of the two main schemes: vertical layering and horizontal resequencing.
Vertical layering schemes are made up of looping musical tracks of identical lengths, which composers can stack in Wwise and set to seamlessly mute or unmute based on the action in the game. Composers may or may not design the tracks to be played simultaneously depending on the desired effect. “You could have layers that are just percussion, or bass, or piano, and they all sound good when you play them together. Or each layer could be an entire mix of instruments, and only certain layers are compatible or in the same key as other layers,” Schwedler says.
Horizontal resequencing, on the other hand, is set up to move directly from one discrete track to another based on in-game actions, allowing for more dramatic changes. “You can put custom transitions between them to smooth it out, but with horizontal resequencing you can make these big jumps between sequences and get as far away from where you started as you want,” Schwedler says.
By the end of the course, students demonstrate their understanding of each scheme by combining the two into one adaptive music composition. “The challenge of adaptive music is actually listening and imagining what it sounds like in all its possible permutations,” Schwedler says. “For some of these more complex schemes, you might never actually hear them all because it’s so complicated!”
While students can compose their adaptive soundtracks as standalone assignments for the course, the junior year timing of MUS 322 is no accident. “They’re working on their junior year game team projects at the same time they are taking this course,” Schwedler says. “It’s not required, but students are encouraged to, ideally, use the music assignments from this class for their game team projects.”
Like their game team courses, MUS 322 is primarily project based, rather than lecture based. Schwedler gives students the tools and techniques, but the learning comes from crafting the scores themselves and comparing the results with peers. “The really fun part of the course is when we review everybody’s work together as a group,” Schwedler says. “It’s a great environment. A lot of the time my students give much better critiques of each other’s work than I do!”