Cold Constructs Blog

Today is the day Google is supposed to release it’s mobile phone technology, Android (wikipedia entry), to the masses in the form of an SDK. Phones based on this tech should launch sometime next year. (Update: Right on schedule, it’s available for download. Also, they’re handing out a total of $10 million to developers who build something awesome for it.)

Thinking about the possible repercussions of this development floors me. Geeks like me have been waiting for this for a long time: an open, stable, fast software framework for whatever piece of mobile hardware we choose (ones participating in the Alliance, anyway). The thing that gets me most is that this is phase 1 of the augrecon concept I talked about a long time ago. Thanks to Google, it’s going to become reality.

I was expecting Google to pwn punk-bitches in the telecom business, but I had no idea they would do anything like this. Everyone wanted a Google-branded phone, but instead they did something exponentially better: they built a standard software framework for us to build on using any piece of hardware we choose. That last bit is incredibly important because it allows evolution by natural selection to do its thing. With a free OS that anyone can build on, we’ll see some fierce competition in hardware that supports it.

What we need is a single platform to build badass applications with that can be distributed to anyone running the same platform. The problem has been exactly that: there are as many software platforms as they are hardware platforms. We can’t link each other because everyone is floating on a different island, each without a ship to sail to other islands. But with Android, we’re all in the same place, and the only thing that will change is the hardware–which grants us more computing power to build better applications with, all on the same framework.

Anyone (you, me, that dude over there) will be able to create an application for this platform and be able to distribute it to millions, and in time perhaps billions of people all over the world. This is the platform I’ve been waiting for to build a truly useful ZoomWorld with. We could use it to circumvent the broken monetary system we’re running on and create the first mass-produced augmented reality device. It will connect people in ways we didn’t know were possible, just like what happened when the internet came about. This is the next giant leap forward in our struggle to tame the technologies we’ve created. We’re finally coming around to utilizing the full power of the net (this iteration of it anyway).

Hold tight, this is going to be one wild ride.

It appears that several companies are now developing multi-touch human-computer interfaces (MT HCIs).

Back in 2005 I made a post checking in on the effort over at NYU. They recently launched a spin-off company called Perceptive Pixel in an effort to refine the idea and get it to market. Of all the others, this one is the most advanced (most capable of MT input and most humane design implementation).

Now there’s Microsoft with an interesting idea–don’t simply deal with MT tech, but also have the thing interface with mobile devices. Props to MS for taking the idea to the logical next step, but a slam for all the painful marketing glitz. (Of course, leave it to some savvy users to uncover more info on the thing. Also, Ars Technica tried to get some deets on the thing.)

UPDATE: Rumor has it that that Perceptive Pixel is actually working with MS to develop Surface. I’ve sent an email to Jeff Han (PP CEO) about it. The plot thickens.
UPDATE 2: Didn’t get a response back, natch.

Going along with the idea of putting down other objects to interact with the MT HCI, a group over at the Audiovisual Institute at the Universitat Pompeu Fabra in Barcelona, Spain, has developed an audio synthesizer thingy called the reactable. Videos here. (And here’s another audio MT HCI.)

People have said Apple was working on one (and someone even pointed to the NYU effort), but the confusion appears to stem from the patents that Apple filed just for it’s iPhone interface. Apple’s implementation appears to prove that the MT idea scales rather nicely, but only when put into a different context (that of a task-based OS, rather than a ZUI). I’d like to experiment with ZUIs on small screens, touch-sensitive or not, so we’ll see.

There’s still TactaPad, a company I mentioned before. Most primitive implementation, but also (therefore?) market-ready, mainly because it’s just another input device rather than a whole new computing paradigm (ie a stand-alone system).

And at an earlier E3 (2006 I think?), Nintendo secretly threw down a floored version (Google video, the only evidence that it ever happened), where the system interacts with your foot movements instead of your hands. Interesting, but has very limited potential without handed input.

These things are cropping up all over the place it seems, a good indicator that the time has finally come for this tech. My guess is that we’ll have these MT HCIs combined with mobile AR devices to act as the next leap of computational evolution. New capabilities are unlocked with each leap–this one should break down the technical barriers that bar many people from fully embracing (ie taking advantage) of the digital revolution. We will see an incredible jump in cultural ingenuity, collaboration and networking, and a new flow of ideas coming to market as it becomes easier for non-technical people to bring their ideas to fruition. (It’s a combination of automating mundane technical tasks to get shit done on the computer, an increase in personal computational power that puts simulation and AI in the hands of everyone with a PC, and this new kind of HCI which nicely strings the previous 2 progressions together into a usable system.)

This is, I think, the leap just before a complete AR solution. After that we’ll probably get personal AI assistants, which would be fun. But right now we just need publicly-available live-feed GIS / audio/video recording nodes dotting the landscape for complete information awareness / ubiquious computing for our current PDAs.

An instant improvement for any software interface would be a new quasimode that allows structural manipulation of the application when activated.

In other words, by pressing say a Function key and holding it down, the application in focus would switch to a mode where the user can resize the window, rearrange buttons or even add new ones or delete others, and so on. When the user is done editing the interface with his mouse, he simply lets go of the Function key and normal operation continues. It should be made clear what can be edited (but un-editable items remain unchanged), such as new icons that tell a user the that the edges of the application can be resized. Those icons themselves could be used to grab and stretch the application whichever way they indicate is possible.

This is to keep the user from accidentally altering the face of a program.