High-Tech Sign Language Could Replace the Mouse

John Underkoffler wants to build a better mouse.

Working in a downtown Los Angeles loft, the co-founder of G-speak is developing technology to replace the ubiquitous computer mouse with a more natural interface: human hands.

Riffs on Underkoffler's technology have featured prominently in movies and television shows. The 2002 sci-fi film "Minority Report," for example, included a dramatic scene in which Tom Cruise gestures with gloved hands to navigate and manipulate crime scene information on translucent screens.

That scene was the product of movie industry special effects. But it marked the birth of G-speak.

Underkoffler moved to Los Angeles to serve as a science and technology advisor on the Steven Spielberg movie. He came from the Massachusetts Institute of Technology Media Lab, where he was a doctoral candidate researching new ways for people to interact with computers.

"The film consultancy stuff in a way was a detour," Underkoffler said. "I'd been building stuff like G-speak before 'Minority Report,' and it was very natural to return to that after. Except that the movie also had the interesting effect of, in some sense, validating a specific form of those ideas. Audiences really responded to those sequences. So in a weird way we'd been able to use the film mechanism to prototype a technology."

In May 2005, Underkoffler and other former MIT colleagues pooled their own money and formed G-speak to find commercial applications for what's called gestural technology. The eight-person company has contracts with aerospace companies and federal agencies interested in the technology. It won't disclose revenue.

"It seems to us that manipulating things with our hands is a fundamental way to interact with the world," said Kwindla Hultman Kramer, G-speak's director of business development.

To demonstrate how the technology works, Underkoffler donned a pair of black gloves with reflective beads attached to the back of the hand, the thumb and two fingers. He drew a pair of heavy black curtains to block light from entering the loft and stepped onto the center of a makeshift stage, surrounded by a scaffolding holding eight infrared-sensitive cameras.

The cameras use light reflected off the BB-sized beads to calculate the position of Underkoffler's hands and interpret each of about two dozen core gestures as a computer command. He faced a 16-foot screen and held his hands like a kid holding imaginary twin six-shooters (index finger pointed, thumb raised) to tag objects on screen and depressed the thumb to grab the object -- in this case a letter of the alphabet.