Cold Constructs Blog

It appears that several companies are now developing multi-touch human-computer interfaces (MT HCIs).

Back in 2005 I made a post checking in on the effort over at NYU. They recently launched a spin-off company called Perceptive Pixel in an effort to refine the idea and get it to market. Of all the others, this one is the most advanced (most capable of MT input and most humane design implementation).

Now there’s Microsoft with an interesting idea–don’t simply deal with MT tech, but also have the thing interface with mobile devices. Props to MS for taking the idea to the logical next step, but a slam for all the painful marketing glitz. (Of course, leave it to some savvy users to uncover more info on the thing. Also, Ars Technica tried to get some deets on the thing.)

UPDATE: Rumor has it that that Perceptive Pixel is actually working with MS to develop Surface. I’ve sent an email to Jeff Han (PP CEO) about it. The plot thickens.
UPDATE 2: Didn’t get a response back, natch.

Going along with the idea of putting down other objects to interact with the MT HCI, a group over at the Audiovisual Institute at the Universitat Pompeu Fabra in Barcelona, Spain, has developed an audio synthesizer thingy called the reactable. Videos here. (And here’s another audio MT HCI.)

People have said Apple was working on one (and someone even pointed to the NYU effort), but the confusion appears to stem from the patents that Apple filed just for it’s iPhone interface. Apple’s implementation appears to prove that the MT idea scales rather nicely, but only when put into a different context (that of a task-based OS, rather than a ZUI). I’d like to experiment with ZUIs on small screens, touch-sensitive or not, so we’ll see.

There’s still TactaPad, a company I mentioned before. Most primitive implementation, but also (therefore?) market-ready, mainly because it’s just another input device rather than a whole new computing paradigm (ie a stand-alone system).

And at an earlier E3 (2006 I think?), Nintendo secretly threw down a floored version (Google video, the only evidence that it ever happened), where the system interacts with your foot movements instead of your hands. Interesting, but has very limited potential without handed input.

These things are cropping up all over the place it seems, a good indicator that the time has finally come for this tech. My guess is that we’ll have these MT HCIs combined with mobile AR devices to act as the next leap of computational evolution. New capabilities are unlocked with each leap–this one should break down the technical barriers that bar many people from fully embracing (ie taking advantage) of the digital revolution. We will see an incredible jump in cultural ingenuity, collaboration and networking, and a new flow of ideas coming to market as it becomes easier for non-technical people to bring their ideas to fruition. (It’s a combination of automating mundane technical tasks to get shit done on the computer, an increase in personal computational power that puts simulation and AI in the hands of everyone with a PC, and this new kind of HCI which nicely strings the previous 2 progressions together into a usable system.)

This is, I think, the leap just before a complete AR solution. After that we’ll probably get personal AI assistants, which would be fun. But right now we just need publicly-available live-feed GIS / audio/video recording nodes dotting the landscape for complete information awareness / ubiquious computing for our current PDAs.

An instant improvement for any software interface would be a new quasimode that allows structural manipulation of the application when activated.

In other words, by pressing say a Function key and holding it down, the application in focus would switch to a mode where the user can resize the window, rearrange buttons or even add new ones or delete others, and so on. When the user is done editing the interface with his mouse, he simply lets go of the Function key and normal operation continues. It should be made clear what can be edited (but un-editable items remain unchanged), such as new icons that tell a user the that the edges of the application can be resized. Those icons themselves could be used to grab and stretch the application whichever way they indicate is possible.

This is to keep the user from accidentally altering the face of a program.

Introducing the TactaPad. Use your computer in the most natural way possible - with your fingers. Check out these demos to see what I mean.

This should be the new standard input device for all desktops and (hopefully) computing devices. There are new display technologies available that are flexible and thin, so I’m sure one of these could be put under it and work just as well, but you might run into problems when you don’t know exactly where your finger will fall. Nevertheless, in its present form it is still exponentially better than a mouse or stylus.

We suck at mazes, but thats how we’re forced to navigate through computers, cameras, and a load of other electronic devices. Camera manufacturers need to bring back dials and knobs - we’re good at that shit. As for computers, I think the best we can do with the windowing environment is SymphonyOS.

Little keyboards on smartphones is a horrible idea. One letter at a time? FUCK that. I’ll draw words at a time with IBM’s SHARK thingy. Too bad it doesn’t work too well. It doesn’t watch what letter you begin with but rather the general area, and it does this through the whole gesture so you usually end up with a completely different word than what you intended. The worst part is that it has to have a database of any word you wish to draw. On the other hand, this is very early alpha software.

Obvious enough title. Wires suck, UWB is the new hottness (Bluetooth is the old and busted), and earphones don’t look as retarded as big headphones in public.

The earphones wouldn’t go in your ear but rather be aimed at your inner ear. Directional audio is nothing new (link to 2003 article). I can’t count the number of times I wanted to listen to music while wanting to be aware of my surroundings. Want a soundtrack to your life and not, as it is now, a soundtrack to your own little world?

Do it.