This time last year, computer vision company uSens introduced a stereo camera module capable of hand tracking. Now, uSens can achieve the same thing with just a smartphone’s camera.
On Thursday, at the Augmented World Expo in Santa Clara, uSens unveiled the beta version of its uSensAR Hand Tracking SDK, which enables developers to integrate hand tracking and 3D motion recognition for augmented and virtual reality mobile apps on iOS and Android devices. Using a smartphone’s RGB camera, uSensAR Hand Tracking uses computer vision and deep learning to track the full hand’s skeletal dynamic, rather than just a hand’s fingertips.
,
,
Image via uSens
According to a Wang, uSens is already working with Chinese social app Meitu to integrate the technology into its AR camera effects as well as an undisclosed game studio, though Wang hints that the company is publicly traded on Nasdaq and publishes “networked games in a social setting.”
Nevertheless, although mobile AR has accelerated the adoption of AR among consumers, the touchscreen still represents a barrier between the user and the content. Just browsing through the best mobile AR apps available today offers a few exciting ideas as to how hand tracking could improve already great experiences.
,
,
,
Even the best mobile AR apps, like PuzzlAR, would be even better with hand gestures vs. touchscreen interfaces.Images by Tommy Palladino/Next Reality
Games like PuzzlAR, which already employs one of the more unique mobile AR interfaces by integrating a gaze cursor with touchscreen gestures, could be even more immersive if players could manipulate puzzle pieces in space with their hands. Making lines in Just a Line would be more more intuitive with fingers as the drawing implement rather than the smartphone. And an educational app like Froggipediacould much more closely emulate the experience of dissecting frog without the mess.
Of course, with smartphones, one hand is essentially tied behind the user’s back, as something needs to hold the device up. As such, uSens technology could also facilitate swifter innovation in smartglasses, particularly Android-based wearables. A camera-based tracking solution would afford hardware makers the luxury of omitting a dedicated depth sensor.
Hopefully, the AR hardware industry is thinking along these very lines, which could mean better AR experiences even sooner than we think.
Source: Next Reality