Cold Constructs Blog

It appears that several companies are now developing multi-touch human-computer interfaces (MT HCIs).

Back in 2005 I made a post checking in on the effort over at NYU. They recently launched a spin-off company called Perceptive Pixel in an effort to refine the idea and get it to market. Of all the others, this one is the most advanced (most capable of MT input and most humane design implementation).

Now there’s Microsoft with an interesting idea–don’t simply deal with MT tech, but also have the thing interface with mobile devices. Props to MS for taking the idea to the logical next step, but a slam for all the painful marketing glitz. (Of course, leave it to some savvy users to uncover more info on the thing. Also, Ars Technica tried to get some deets on the thing.)

UPDATE: Rumor has it that that Perceptive Pixel is actually working with MS to develop Surface. I’ve sent an email to Jeff Han (PP CEO) about it. The plot thickens.
UPDATE 2: Didn’t get a response back, natch.

Going along with the idea of putting down other objects to interact with the MT HCI, a group over at the Audiovisual Institute at the Universitat Pompeu Fabra in Barcelona, Spain, has developed an audio synthesizer thingy called the reactable. Videos here. (And here’s another audio MT HCI.)

People have said Apple was working on one (and someone even pointed to the NYU effort), but the confusion appears to stem from the patents that Apple filed just for it’s iPhone interface. Apple’s implementation appears to prove that the MT idea scales rather nicely, but only when put into a different context (that of a task-based OS, rather than a ZUI). I’d like to experiment with ZUIs on small screens, touch-sensitive or not, so we’ll see.

There’s still TactaPad, a company I mentioned before. Most primitive implementation, but also (therefore?) market-ready, mainly because it’s just another input device rather than a whole new computing paradigm (ie a stand-alone system).

And at an earlier E3 (2006 I think?), Nintendo secretly threw down a floored version (Google video, the only evidence that it ever happened), where the system interacts with your foot movements instead of your hands. Interesting, but has very limited potential without handed input.

These things are cropping up all over the place it seems, a good indicator that the time has finally come for this tech. My guess is that we’ll have these MT HCIs combined with mobile AR devices to act as the next leap of computational evolution. New capabilities are unlocked with each leap–this one should break down the technical barriers that bar many people from fully embracing (ie taking advantage) of the digital revolution. We will see an incredible jump in cultural ingenuity, collaboration and networking, and a new flow of ideas coming to market as it becomes easier for non-technical people to bring their ideas to fruition. (It’s a combination of automating mundane technical tasks to get shit done on the computer, an increase in personal computational power that puts simulation and AI in the hands of everyone with a PC, and this new kind of HCI which nicely strings the previous 2 progressions together into a usable system.)

This is, I think, the leap just before a complete AR solution. After that we’ll probably get personal AI assistants, which would be fun. But right now we just need publicly-available live-feed GIS / audio/video recording nodes dotting the landscape for complete information awareness / ubiquious computing for our current PDAs.