PUT your smartphone down on a table and it will necessarily ignore you. But give it peripheral vision and it could interact with you and your surroundings like never before.
Phones can already record wrap-around imagery using omnidirectional lenses like the $49 Dot lens made by start-up firm Kogeto, based in New York City. Fixed to a smartphone's main camera, it lets you shoot immersive, 360-degree photos and video. On playback, you can pan around within the footage, or watch all 360 degrees at once. But until now, a Dot-equipped phone could not recognise or respond to anything in its environment.
It can now. Xing-Dong Yang at the University of Alberta in Edmonton and colleagues reasoned that, if placed on a phone's user-facing camera instead of the main, rear one – which mostly points at the ground – such a set-up could monitor activity and track objects in the user's surroundings. So the team fitted a Dot to the front camera on an HTC Butterfly phone and trained an Android app to recognise locations and gestures.
Users can use the app to control laptops, wireless speakers and other objects with pinch and swipe gestures in the air. The app can also ask the user if they have forgotten their phone when they walk away from it. If you don't want your phone, a sweeping hand gesture sets it to voicemail mode. Recognising multiple faces present in a gathering, the phone automatically mutes itself, and if it identifies the interior of your car and hears engine sounds, it blocks calls as a safety measure, Yang told New Scientist.
Called Surround-See, Yang's combination of lens and app gives just a glimpse of what smartphones could offer if fitted with depth-sensing cameras. That became a distinct possibility last week after it emerged that Apple is in talks to purchase PrimeSense, the Israel-based firm that pioneered the depth camera at the heart of Microsoft's Kinect sensor. The news is fuelling speculation that future Apple devices will be fitted with gesture-sensing technology.
"Surround-See is a nifty idea," says Chris Harrison of Carnegie Mellon University in Pittsburgh, Pennsylvania. "For smart devices to get smarter, they need more information, so having a sensor like this brings new capability."
- Subscribe to New Scientist and you'll get:
- New Scientist magazine delivered every week
- Unlimited access to all New Scientist online content -
a benefit only available to subscribers - Great savings from the normal price
- Subscribe now!
If you would like to reuse any content from New Scientist, either in print or online, please contact the syndication department first for permission. New Scientist does not own rights to photos, but there are a variety of licensing options available for use of articles and graphics we own the copyright to.
Have your say
Only subscribers may leave comments on this article. Please log in.
Only personal subscribers may leave comments on this article
All comments should respect the New Scientist House Rules. If you think a particular comment breaks these rules then please use the "Report" link in that comment to report it to us.
If you are having a technical problem posting a comment, please contact technical support.
