For most 3D developers, “texturing” means wrapping a gray 3D model with a colorful, detailed surface. Recently, it’s meant something much more physical to 2017 BS in Computer Science in Real-Time Interactive Simulation graduate Luc Kadletz. “In one of our simulations, you could run your hands over a virtual brick wall and feel the difference between the virtual brick and the virtual grout,” Kadletz says.
Welcome to the tactile world of Seattle hardware manufacturer HaptX, where Kadletz spent the last two-and-a-half years helping the company develop their groundbreaking HaptX Gloves G1. With features straight out of science fiction, the motion-tracking gloves and back-mounted “airpack” allow users to touch, feel, and manipulate 3D objects with their hands in virtual reality. The magic is made possible by hundreds of “microfluidic tactile actuators” in each glove that physically vibrate and push on the user’s skin to simulate surface textures. A force feedback mechanism — similar to a bicycle brake cable — adds up to 40 pounds of resistance to mimic object weight, volume, and shape.
Primarily designed for realistic workforce training simulations, remote robotics control, and virtual design, the gloves were a perfect fit for Kadletz. “It was almost like fate. It was like they made the job for me,” he says. That’s because the HaptX Gloves G1 actually weren’t Kadletz’s first experience merging the physical and virtual world.
Sand Garden, Kadletz’s senior game team project at DigiPen, was created out of a desire to do something new and interesting with Microsoft’s motion-sensing Kinect technology. “We lovingly called our project the ‘100-pound box of dirt,’” Kadletz laughs. With a large, physical box of kinetic sand manipulated by hand as the primary input, the game turned the Kinect’s sandbox depth readings into virtual valleys, rivers, and mountains on screen. Players were charged with sculpting the terrain to meet the needs of the game world’s growing villager settlements. The novel idea earned the project a finalist spot at the 2017 Alt.Ctrl.GDC competition, a showcase celebrating games with alternative interactions and controls.
“Looking at the wild stuff other people were doing at Alt.Ctrl.GDC that year, it was really cool to see that there’s a lot of room left to explore between the computer and the player,” Kadletz says. “That was something I was really excited about with HaptX.”
In 2022, after discovering Kadletz’s work on Sand Garden through LinkedIn, a recruiter from HaptX cold-called him asking for an interview. Having worked five years on Microsoft’s security team, Kadeltz was eager to return to the frontier of user-computer interaction. “I talked a lot about what I learned from Sand Garden in my HaptX interviews,” Kadletz says. “Having experience with a new thing people don’t know how to use yet helped a lot.”
Hired on as a “plug-in engineer,” Kadletz’s role at HaptX found him working on “everything between Unity, Unreal, and the USB port.” Practically, that meant diving into two main pillars of the Gloves G1. The first task was to improve and develop the gloves’ real-time hand animation algorithm that translates user finger movement into VR. The second involved extending game engine physics systems to integrate HaptX’s “vibrotactile” technology.
Hand tracking was particularly tricky, requiring Kadletz to digitally recreate the user’s finger movements using only wrist and fingertip location data. “There was a lot of work and math in trying to figure out how to position each finger joint in the hand animations,” Kadletz says. “Sometimes there would be a perfectly valid mathematical solution to getting fingertips to connect to the wrist that would result in a virtual finger doing something it really shouldn’t — like bending backwards — which really freaks people out!” To further complicate things, Kadletz discovered that the math governing fingers was completely different for thumbs. “There was a lot of, ‘If it’s not a thumb, do this,’” he says.
Kadletz also had to run lots of dexterity tests to ensure his system could handle both delicate and large-scale object interactions. “Something that worked well with fine manipulation, say, picking up a small nail or screw, wouldn’t work well with gross manipulation, like pulling a big lever on an assembly line,” Kadletz says.
Despite the challenges, Kadletz was thrilled when he saw all his “handiwork” pay off. “In my past experiences showing VR to people who aren’t tech users, you’d hand them the VR controllers and they’d have no idea what to do,” Kadletz says. “But with the gloves, newcomers just knew! They could pick something up, drop it, throw it — that barrier to entry was just gone.”
The other major pillar of Kadletz’s work, turning the texture of virtual objects into vibrotactile feedback, found him unexpectedly returning to lessons from a DigiPen elective he took on acoustics. “There’s so much overlap I wasn’t expecting from that class between how soundwaves work and how haptics work,” Kadletz says. “Because the gloves use pressure waves, it’s so much of the same math.” In fact, while exploring physics research papers on haptics, Kadletz discovered that scientists often use a device much like a record player needle to capture the haptic frequencies of physical surfaces. “It was a bit of a physics rabbit hole,” Kadletz says.
To achieve a realistic sense of touch, the HaptX Gloves G1 delivers pressure changes at 125 hertz (or cycles per second). Anything below that threshold, and users running their hands across a virtual surface wouldn’t experience a continuous texture as they would in real life. Most game engines, however, aren’t built to handle that much physics information, so Kadletz had to extend their capabilities. “It’s kind of analogous to sound again. You can’t just have one speaker update per frame of gameplay, or else you would only hear clicks!” Kadletz says. “I had to look at one sixtieth or one ninetieth of a second in the game engine’s physics system and come up with that much information for the touch simulation to make it feel real.”
There’s a lot of really interesting work in that intersection of gameplay programming and physics.
Despite its industrial focus, Kadletz says the gloves’ potential for pure fun was undeniable. One HaptX tech demo, a Jenga-style game called “HaptX Team Blocks,” was a crowd favorite at conventions. “We’d get these fun ideas, like we could do something where every time you pull out a Jenga block, your fingers get longer,” he laughs. “VR is fun and novel already, but with the gloves, even more so. It was even fun just doing normal stuff like screwing screws in.”
That joy in simple actions extended to development as well. Kadletz recalls gleefully summoning basic 3D objects out of thin air at the click of a button, testing their textured, volumetric forms with his hands, and tossing them when he was done. “One of the content guys came up with this tech where you could summon a keyboard and a display to enter console commands and edit things without having to take off the headset or gloves,” Kadletz says. “It felt really futuristic!”
After helping the Gloves G1 make their commercial debut in the summer of 2024, Kadletz parted ways with HaptX in October and is taking the moment to explore what’s next for his own future. “The Seattle area is a nice place to be, surrounded by lots of interesting opportunities. I really like being on the newer side of what’s happening,” Kadletz says. “There’s a lot of really interesting work in that intersection of gameplay programming and physics that I got into at DigiPen and HaptX — much more than I expected in that space!”