After more than three decades of academic and private surgical practice, Dr. Evan Geller became convinced that the future of surgical education lay in video games. “He was inspired by flight sims for pilot training, which have long used video game technology,” says Mark Henne, director of DigiPen’s MFA in Digital Arts program. “But when he looked at existing surgical training systems, the fidelity of imagery and interactivity wasn’t very good.” Dr. Geller decided it was time to change that.
Since 2022, his company ParaSurgical Systems has partnered with a dozen DigiPen faculty, students, and alumni to build a cutting-edge surgery simulator prototype in Unreal Engine 5. The DigiPen team worked with support from an Epic Games MegaGrant, an annual award from the game software giant funding projects that showcase the Unreal Engine’s capabilities. After being kept under wraps for the last year and a half, the simulator prototype debuted this summer.
The DigiPen prototype for ParaSurgical Systems includes a photorealistic diorama of the upper abdomen, the typical environment for laparoscopic surgery.
As the project’s Principal Investigator at DigiPen, Henne calls it a perfect fit for Unreal Engine 5. “The whole goal of the project is to have something that looks as real to the surgical experience as possible,” he says. “Unreal Engine 5 has been pushing photorealism and naturalistic real-time rendering and interactivity, so now you really can have a full, 3D experience that feels like actual surgery in both appearance and behavior.”
Dr. Geller had the medical side of the project covered. As founder of Stony Brook University’s Trauma Research Lab, former chief of trauma surgery and critical care at the university hospital, and longtime president of North Suffolk Surgery Associates, he knew exactly what he wanted to see in his training simulator. What he lacked was the technical know-how to build it. For that, he turned to DigiPen’s expertise in real-time simulation, 3D graphics, and AAA game technology. He asked the team of Dragons to focus on laparoscopic surgery, a minimally invasive procedure where surgeons guide robotic arms and a camera to operate inside the abdomen or pelvis.
With DigiPen Adjunct Senior Lecturer John Hermanowski as art director, Henne assembled a team of nine students and recent graduates of the MFA in Digital Arts and BFA in Digital Art and Animation programs to join the project part-time.
“Evan [Geller] spent time teaching us anatomy and surgical procedures, and we had to study laparoscopy and human biology,” Henne says. “We purchased the best 3D anatomy models available for reference, which were more akin to medical illustrations than naturalistic anatomy, then rebuilt everything for proper appearance, detail, and scale.”
The challenge was unlike anything most of the artists had faced before. “Students gained a lot of experience in photorealistic presentation of organs, layered tissue structures, translucency, and anatomical accuracy, skills not typically required in traditional game projects,” Henne says. Even with 18 years at Pixar as a technical director specializing in hair, cloth, and particle simulation, Henne found himself learning new skills in volumetric simulation of soft tissues for the project.
The prototype also features a simulation of a gallbladder removal procedure, requiring advanced graphics effects and deformations.
The team built two proof-of-concept demos. The first recreated the laparoscopic surgery environment, where gas inflates the abdomen to create space for visibility and manipulation. The virtual diorama included lifelike depictions of the liver, stomach, gallbladder, ligaments, fat, and abdominal wall, with subtle movements to mimic blood flow and breathing. The second demo simulated a laparoscopic gallbladder removal procedure, complete with instruments like a grasper and cautery hook, which heats and seals tissue. “We developed techniques for rendering holes in membrane tissue, smoke effects, texture changes, and believable deformations,” Henne says. “As you manipulate the gallbladder, it should behave like a real one.”
The prototype was so realistic that social media platforms flagged ParaSurgical Systems’ demo videos as “graphic content” when they were posted in July. “It shows that the project can achieve a hyper-realistic look and organ behavior!” Henne says, interpreting the flag as an unintended endorsement. “The proof of concept videos and diorama clearly demonstrate the project’s potential. The next phase would be adding in the interactivity with joystick controllers.”
If further funding is secured, ParaSurgical Systems plans to continue developing the simulator with DigiPen, hiring select student and alumni contributors on full-time. “I know Evan [Geller] really values the collaboration, consulting with fresh talent and new perspectives in the game engine space,” Henne says. “Personally, the most rewarding aspect of this has been seeing students and graduates excel on a project with such real-world stakes.”
Project contributors from DigiPen included faculty Mark Henne (Principal Investigator, Animation and Simulation), John Hermanowski (Art Director), and Doug Zwick (Game Engine Consultant); MFA in Digital Arts alumni Miranda Penrod (Digital Artist, Materials & Lighting Lead), Nicola Dolci (Digital Artist, Producer), Angel Wang, Jaida Ho, Bingjun Lin, and Mingyuan Li (Digital Artists); and BFA in Digital Art and Animation alumni Doug Wu, Hwa Min Jung, and Jesus Barrera-Garcia (Digital Artists).