Despite being a self-professed Dungeons & Dragons fanatic, DigiPen graduate Matt Rosen admits he has a hard time picking a character to role-play, but there’s a good reason for that. He’s always been more interested in crafting a memorable experience than merely participating in one.
“I’ve been a Dungeon Master since I was 13,” he says. “I’m always a Dungeon Master, so I never know what to do for a character.”
More than a decade after leading his first fantasy role-playing campaign, D&D has become more than just a hobby for Rosen. It’s a part of his career. That’s because in 2020, just months after his graduation from the BS in Computer Science and Digital Audio program, Rosen landed the opportunity to take his love for DM’ing in a bold new direction by joining the development team on Larian Studios’ RPG epic, Baldur’s Gate 3, a game famously set in the D&D universe.
“Ever since they announced Baldur’s Gate 3, I was like, ‘If I could get a job at this company, that would be sick,’” Rosen recalls. “Then they posted an audio programming position, and I got it. So it’s kind of a dream job.”
In fact, it wasn’t just any audio programming position that Larian Studios had posted. It was the studio’s first and only dedicated audio programming position in the company’s history. As such, Rosen’s role for the past three years has been a major one — namely to build and reconfigure new and improved audio systems and features for the entirety of the studio’s most recent flagship title, working simultaneously with sound designers and other Larian developers to assist with their audio tech needs.
It’s a very niche field to be in and something that DigiPen has a special degree for.
In other words, if you’ve played any portion of Baldur’s Gate 3’s sprawling 50-plus hour campaign, you’ll undoubtedly have come into contact with one of the many audio systems Rosen developed — from sound occlusion and automated mixing systems to a real-time armor sound effects generator for cutscene animations.
“Audio kind of sits in the middle of every single team. Everything needs a little bit of audio, whether people like it or not. If there’s an animation, chances are you need audio. If there’s VFX, you need audio,” Rosen says. “Basically, any system in the game needs audio to come with it because a game is an audio-visual experience. Audio is half of that.”
For most of his work, Rosen operates at the intersection of the Baldur’s Gate 3 core game engine, a modified version of the same in-house technology that previously powered Larian Studios’ popular Divinity: Original Sin series, and the audio middleware — in this case, the third-party software Audiokinetic Wwise.
“My job is usually writing code in the engine to facilitate that connection,” Rosen says. “It’s a very niche field to be in and something that DigiPen has a special degree for.”
For one of his larger tasks on Baldur’s Gate 3, Rosen’s knowledge on the subject was already expert-level, thanks to an independent student research project he completed during his senior year. That project, created in collaboration with Microsoft researchers, was an audio engine for simulating complex audio effects such as occlusion, which refers to the muffling that occurs when a sound wave interacts with objects or barriers before reaching the listener’s ear.
“My old acoustics project does occlusion in the most realistic way possible, but for most games that’s definitely not feasible,” Rosen says. “It wasn’t feasible for Baldur’s Gate 3, especially because we have to include the feature of split screen.”
Nonetheless, when it came time to propose and implement an occlusion system that could work, Rosen was sufficiently brushed-up on the subject to hit the ground running.
“In the previous games, we had done occlusion in a very old-school way, where basically a sound designer had to map out boxes and say, ‘If you’re inside this box, everything sounds fine, but if you’re outside of it, sounds coming from outside are occluded,” he says.
Not only did the old method require sound designers to manually map out the boxed parameters for every location in the game, it failed to take into account things like scaling volume based on a character’s distance from the actual audio source.
“One of the main things I did was redo that entire thing with a more modern occlusion system, which involved a series of ray casts,” Rosen says. In the new system, he says, environmental audio now takes into account a multitude of factors, “depending on how far away it is and how good your CPU is, making it scale with hardware, making it scale with the game, and also making it automated so sound designers don’t have to do any work related to that.”
It wasn’t the only system he designed with the goal of automating the sound design process. When the Larian Studios team decided to incorporate motion capture data for all of the voice actors’ cutscene dialogue, it showed their commitment to authentic storytelling.
“Every single cinematic in the game — whether it’s just talking to a trader or an end boss introduction scene — is fully motion-capture,” Rosen says.
While the decision to go full motion-capture made sense for animators, it presented a new challenge to the sound design team.
“An approach a lot of other games use is they just have a bank of animations for cinematics, and then sound designers are able to hand place things like cloth and armor sounds onto the characters’ micro movements so it sounds alive,” Rosen says.
Because the scenes in Baldur’s Gate 3 were fully motion-captured, that meant sound designers would need to apply cloth and armor sound effects to hundreds upon hundreds of unique scenes — something the team was not equipped to handle using traditional techniques. The answer, for Rosen, once again involved an automated and scalable technology solution, one that could simulate armor and cloth sounds based on real-time animation data.
“So I wrote a system that tracks hands, velocities, accelerations, rotations — I think head movements as well if you’re wearing a helmet, with your body twisting and all of that,” Rosen says. “And I configured it to play or not to play a sound, how loud that sound should be — if there should be a spike in the sound.”
Despite being in a class of his own as the game’s sole audio software engineer, Rosen was far from feeling like a solo party member. As part of a global studio with six offices on three continents, his past three years have been filled with opportunities to connect and collaborate with people from all backgrounds and disciplines — including some of Larian’s most senior programmers who built much of the code infrastructure he relies upon.
“Larian has a lot of talent, a lot of very skilled engineers,” Rosen says.
For his part, Rosen currently splits his time living in Quebec City, where he works on-site at the studio’s Canadian offices, and the United States, where he works remotely. Even though all Larian developers communicate in English for work, he’s been doing his best to learn French in his spare time, taking advantage of evening language courses offered free of charge by Larian. Most of his friends in Quebec, he says, have been fellow coworkers.
“Studio culture is — I would say — wonderful, because we’re all just tabletop nerds. We all like video games,” Rosen says. “We all play D&D, and we’re making the best D&D game of all time. So of course we’re stoked to be there.”
In a year stacked with critically acclaimed game releases, Baldur’s Gate 3’s arrival on the scene was among the most impactful of them all, racking up an impressive number of perfect and near-perfect review scores. More than that, it’s a game that continues to dominate much of the online conversation as people share their wild and memorable stories of adventure, heroism, friendship, and romance.
It’s enough to bring a smile to any veteran Dungeon Master’s face, but especially for Rosen, who knew from the get-go his team was onto something special when he played through the game’s Early Access version three years ago.
“If you like good stories, interesting characters, and fun narratives where your choice really matters, it’s a game for you,” he says. “It’s genuinely one of the best games I’ve ever played.”
Surprisingly enough, Rosen has so far decided to delay his own play-through of the full game, although he does plan to come back to it eventually.
“I’m at the point right now where, because I’ve spent so long developing it, I want to take a break before I actually play it,” he says.
That said, don’t worry about guarding him from any spoilers. At this point, he’s seen just about everything.
“I’ve had to debug the ending. I’ve had to debug parts of Act 2. Have you heard about the infamous Halsin bear scene?” Rosen asks, referring to a certain romance option involving a shapeshifting druid that’s become something of an internet meme. “There was a bug in that scene that I had to watch over and over again. So I knew about that long before anyone else did, even people inside the company.”
And if you had to twist his arm on choosing a character to roll for himself, he does — after giving it some thought — have an answer.
“I like playing clerics, because I like helping people,” he says. “I like playing dwarves too. So yeah, a dwarf cleric.”