John Underkoffler wants you to believe that, in five to ten years, we will all be able to interact with every screen we come across – from phone, to computer, to digital signage – without even touching them.
Underkoffler is the chief scientist at Oblong Industries, a company dedicated to ending the age of 'the tyranny of the mouse'. A decade ago, Underkoffler was working on obscure applications of gestural-based user interfaces for architects and urban planners. Now he and a number of others are developing systems that read the hand gestures and body language of people interacting with screens, and the results are head-turning.
Some firms are focusing on retrofitting existing digital signage systems with gesture sensitivity functions. One company developing gesture recognition with 3D camera sensors is Israel's Omek Interactive. Omek's trademarked Beckon Development Suite is a set of middleware and tools that enables signage application developers to add gesture control and body tracking capability to their interactive digital signage.
In Manchester Airport travellers navigate IBM's Smarter City campaign on Monster Wall screens using gestures and touch
"Our solution's camera-independence means you can take advantage of the latest technology in the rapidly expanding and evolving market of 3D sensors," says Omek Interactive president Jon Epstein.
This means that Omek is enabling pre-existing digital signage installations and networks to integrate gesture controlled games, touch-less information navigation and long-range engagement. Omek's tools can also accommodate social engagement where groups of customers manipulate digital signage content simultaneously.
Other companies, like developer of engagement-orientated campaigns for out-of-home spaces Monster Media, are designing installations from the ground up with gesture-based interactivity in mind. One such campaign, for IBM's Smarter City in Manchester Airport, lets travellers navigate their way through The Smarter City on large-format projections and digital screens called Monster Walls using a 'combination of gesture-activated, multitouch, audio and video calls to action'. Monster Media also manufactures interactive floor, mobile and kiosk systems for DOOH, retail and other applications, many of which include engagement compatibility with smartphones.
Another benefit of interactive advertising displays is the potential for acquisition of valuable consumer metrics and data. Monster Media's Monster Walls include a tracking system that 'measures direct user engagement down to each display and trigger, giving [clients] the ability to calculate a concrete ROI for the campaign'.
Gestural applications though are not limited to OOH advertising as traditionally conceived. Augmented reality developer Hidden's Gesture Controlled Window Display product is designed for installation on the outside of shops and works by projecting content directly onto shop windows. Potential customers can then control content with simple hand and body gestures. This can drive foot traffic and potentially build up high-streets' crowds.
Once inside shops, Hidden offers a Gesture Controlled Catalogue system that lets shoppers browse through inventory on 3D screens, again using simple gestures. Hidden technologies are also equipped with analytical collection tools to help track engagement figures and increase the potential to justify investment in the technologies.
This year developer of a 'patented technology for [real time] gesture control of interactive surfaces, signs, displays, devices and games' GestureTek brought to market a turnkey, plug-and-use solution for interactive floor projections that is finding applications in over 4,000 retail and OOH locations including in over 350 Old Navy outlets in Canada and the USA. The Cube is a small metal box which sits on the floor and plays any of 70 pre-installed applications designed to pick up on foot traffic. GestureTek has leased its patented technologies to Sony and more recently to Microsoft for its gesture-controlled Xbox Kinnect gaming product. But DOOH, retail and commercial applications for gestural technologies are only the tip of the iceberg.
One of GestureTek's goals, says company co-founder Vincent John Vincent, is to create immersive, gesture-based audio-visual systems that might 'actually be a positive tool to enhance full body development'. Since he began working on gesture-based user interfaces over a decade ago, Vincent's technologies have been used in physical therapy treatments for cerebral palsy sufferers and stroke victims.
Where else are we likely to see applications of gesture-based user interfaces for screens? Quite possibly everywhere.