Face filters on Instagram’s ‘Stories’ feature. Facebook is making a major bet on the future of augmented reality across its suite of apps.
Mark Zuckerberg got his first taste of the Oculus Rift, the pioneering virtual reality headset, in January 2014. Standing in one of the few Facebook FB -0.75% offices equipped with blinds, with the brick-like device strapped to his face, he was suddenly transported to the ruins of a medieval castle, thick snowflakes falling all around him as gargoyles sprouted lava from their beaks. The dazzling virtual escapade immediately convinced him that VR would one day become a major computing platform. Two months later, he backed that conviction with Facebook’s $2 billion purchase of Oculus.
But Zuckerberg left the demo with another, far less talked about hunch: a younger, related technology, called augmented reality (AR) had a shot at leapfrogging VR, with its potential to bring digital overlays of contextual information or special effects onto the physical world through a simple smartphone. While there was no headline-making billion-dollar deal, Zuckerberg ordered his engineers to begin building toward an AR and VR future at the same time. The dual push made sense given that the two mediums share so much of the same underlying technology, from hardware components to sophisticated computer vision software.
“Mark [Zuckerberg] was the one who really pushed us to invest in AR right around that time,” Facebook CTO Mike Schroepfer told Forbes. Together, AR and VR rank among Facebook’s top three tech priorities – along with connectivity and artificial intelligence – Schroepfer said. Inside Facebook, hundreds of engineers are working on underlying technologies like computer vision which will enable a phone to do everything from tracking facial movements in real time to identifying a coffee mug or recommending context-specific image effects. Artificial intelligence is so fundamental to powering AR that Facebookers often call their in-app camera an “AI camera.” The overall effort involves “significant capital outlays,” Schroepfer said.
Today, Facebook is fighting its fellow technology powerhouses,Apple AAPL +1.1% and Google — and still to some extent, Snap — in a high-stakes battle to rule as the platform of choice for AR developers. The technology itself, while still in its infancy, has exploded in popularity, confirming Zuckerberg’s more recent intuition that AR could sprint toward mass adoption even while VR remained an awkward technology whose appeal is largely limited to hardcore gamers. AR’s key advantage is that it doesn’t depend on a pricey, bulky headset that isolates its users. It works on a device already owned by more than one fourth of the world’s population.
“The big epiphany is that you can use your phone for AR, and we have about 1.3 billion people who use Facebook on the phone,” said Joaquin Candela, Facebook’s head of applied machine learning, the group building the AI backbone behind the company’s AR efforts. “One hundred times more people have phones than VR headsets. That makes AR really interesting and obvious to focus on.”
Obvious, too, because early evidence suggests AR has the power to draw consumers in, sometimes fundamentally changing how they interact with their phones. Witness the popularity of pioneering AR applications, like puppy masks on Snapchat and virtual treasure hunts on games like Pokémon Go. They make clear that AR, not VR, is the next major phase in our mixed reality future. The smartphone interactions that power AR — using a mask to turn into a singing rock star or pointing a phone to capture a Pokémon – are already acceptable social behavior.
But the reason tech powerhouses are investing so much in AR is that the applications go far beyond social media, games and goofy special effects. The technology could give rise to practical applications in areas ranging from navigation to e-commerce, where virtual dressing rooms could grease the wheels of online apparel purchases. An app from IKEA already helps shoppers virtually arrange furniture in their home. Pharmaceutical companies are playing with the idea of using AR to display real time information about drugs. Hyundai uses an AR app to walk consumers through the features of some cars. AR could also make its way into customer support tutorials that integrate with chat bots. “The smartphone can basically be a magic lens that you hold up to the world,” said Facebook Camera team engineering manager Tom Meyer.
Inside Facebook, engineers and executives acknowledge the stakes are high. Without getting its AR push right, the company risks seeing a decline in usage of its apps. Its battle with Snapchat for younger users shows loyalty to social apps can be fleeting. Users will quickly migrate to those that have the most alluring tools, and those that can turn communication, especially through images and videos, into engaging, constantly-evolving experiences.
Facebook didn’t launch its AR effects until several years after Snapchat, eons in Internet time. However, Facebook pulled off the gargantuan task of offsetting its late debut thanks to the social media giant’s powerful in-house AI, which supported more advanced effects at scale, and its strong product design. Together, these assets enabled Facebook to rapidly match Snapchat’s features and hamper its younger competitor’s growth, saving Facebook from the fate of other tech giants such as Google (with Google Plus), whose late product timing cost the search company its shot at becoming a social media player.
Success in AR could bring big rewards for Facebook. The company’s advertising business, fueled by activity and time spent across its apps, translated into $26.9 billion in revenue in 2016. AR effects generate a growing portion of overall time spent on the social network, encouraging users to message more frequently and spend more time viewing friends’ posts and making their own. The average Facebook user already spends about 50 minutes per day across its main app, Instagram and Messenger – and Facebook needs to continually roll out new product features that capture eyeballs and foster growth in image-heavy communication to keep this metric high and stave off competition, particularly as the company seeks to minimize other news feed thumb-stoppers like click bait and hoaxes.
,
,
Facebook CEO Mark Zuckerberg demos Oculus’ virtual reality headset at a conference in 2016.
The prevalence of masks and filters across Facebook’s apps marks the company’s first major step in transitioning from being a “past camera” for still images and albums to a “future camera,” as Meyer calls it — one that’s powered by AI for intriguing effects and attached to a network of friends and family. Masks might seem frivolous, but the implications of getting AR right for Facebook are existential. The social network’s ongoing shift toward image-heavy features is as critical to the company’s future as was the transition from desktop to mobile, according to Facebook’s chief product officer Chris Cox.
“As a broader story, Facebook has to get really good at AR if we want to be relevant in the next 10 or 20 years,” Schroepfer said.
From ‘Past’ To ‘Future’ Camera
On a morning last May, Zuckerberg appeared on the social network wearing taped glasses with math equations swirling above his head. The nerdy digital accessories were Zuckerberg’s attire of choice for announcing Instagram Stories’ first “face filters” (known as face “masks” on Snapchat). “This is my favorite one so far,” Zuckerberg said looking wide-eyed into the camera with a smile.
The effects, which include colorful confetti, a bubbly underwater scene, twitching koala noses and bunny ears, are inspiring 300 million people to experiment, everyone from celebrities like actress Reese Witherspoon and model Karlie Kloss to teenagers lounging at home use Instagram’s AR-heavy Stories feature each month. Buoyed by Instagram Stories (a series of photo and video clips that disappear after 24 hours) and its analogues on WhatsApp and Facebook’s main app, Facebook is now the largest social AR ecosystem in the world, just four years after Zuckerberg’s Oculus demo and six years after the company started picking away at the core AI technology.
,
Source: Forbes