I visited the Microsoft Reactor and all I got was this damned hologram
For what seems like forever, we’ve been consuming flat content on flat screens. We use traditional cameras to capture all of this, but those cameras aren’t suited for the new world of virtual and augmented realities. There needs to be a new way that can provide immersive experiences.
The solution is volumetric capture. The easy way to think about the technology is that it’s the complete opposite of another capture tech that’s proven popular in VR: 360 video recording. While 360 video cameras are placed in the center of something and capture everything around it, volumetric capture cameras are placed around something and capture every detail.
Microsoft recently invited me to check out its brand new capture studio in its newly renovated Microsoft Reactor in San Francisco. It turns out they were going to put me directly in the capture studio and turn me into a hologram.
Capture this
,
,
Microsoft has been working on volumetric capture for the past seven years, says Steve Sullivan, general manager of of Microsoft Mixed Reality Capture Studio. It started out as a project within Microsoft Research before moving to an incubator, where it was then graduated into part of the HoloLens project, which is now part of the larger Mixed Reality platform.
The newly opened Capture Studio in San Francisco is only the second version outside of the original on Microsoft’s Redmond campus. Sullivan says the company is already receiving a ton of demand from creatives to use the space, and is opening a third location called Dimension Studios in London in collaboration with Hammerhead and Digital Catapulti, giving European creators a chance to play with the tech without having to make the trek across the Atlantic and continental US.
Stepping into the Capture Studio is bizarre. It’s absurdly bright, with what feels like a thousand lights concentrated on you in every direction. All of those lights also mean that you can feel the temperature raise once you walk past the grey curtains that separate the recording space from the Capture Studios’ technical staff.
You stand in the center of the space, which Microsoft says is big enough to fit two sumo wrestlers into (they actually did do that). Around you, 106 cameras are trained on you and capturing every moment. Half of those cameras are, well, cameras that are capturing imaging data. The other half of those cameras are infrared cameras shooting dots at you. As one of the staff members told me, the tech is similar to the infrared cameras in the old Xbox Kinect.
,
,
The Capture Studio and all of those cameras creates a mesh version of yourself. It’s not exactly a video or photograph as you and I have come to understand it. It’s a little closer to a digital creation, like a character from a video game or a Pixar movie. Except, well, it looks absolutely incredible.
If someone told you this was a video you’d probably believe it. In some demos, Microsoft showed me – and the rest of the gaggle of press that were present – other captures it had done throughout the years. Some early captures, like that of Xbox head Phil Spencer’s golf coach, had some artifacts that kept it from looking perfect. However, the more recent ones were simply stunning.
Microsoft’s technology is advanced enough to where it can do materials and cloth really well, which is why brands and companies who want to use the system to push clothing are incredibly interested in it, according to Sullivan. It flows and bends and crumbles like real fabric.
Creators can adjust the mesh after the fact. For instance, in one demo a Microsoft employee simply moved a slider and turned a mesh of a Cirque du Soleil dancer a rainbow of colors. You could even change the color of someone’s clothes if you really wanted to. One of the more advanced things that was done for a mesh was for a Buzz Aldrin experience.
The developer wanted to create an immersive environment that users could look and move around in, but they also realized that that could break immersion. If the user is noodling around in one area, and then they turn around and see Aldrin has his back to the user, explaining something to no one, it breaks the immersion of actually being with Aldrin. So they adjusted the mesh so that it would always turn toward the user.
And really, that’s the big goal with the Capture Studio. Microsoft wants to enable creators of AR and VR experiences to have immersive, engaging performances.
“That’s our goal, to give people the ability to have human beings, real human beings, performing and engaging with them in these experiences,” Sullivan says. “So it’s much more connecting than working with a CG environment, for example.”
In the immediate future, Microsoft sees its capture studio – and volumetric capture on a whole – in the hands of creators making entertainment experiences. It’s definitely the future of VR filmmaking, for instance. Educational and instructional institutions have also been turning toward the Capture Studio to add authenticity to experiences. However, Microsoft is hoping that one day it has made the technology mainstream and prevalent enough that it can be for personal use.
“I’ve done this for the past four years with my kids and it’s really interesting to go back and see them four years ago,” Sullivan explains. “It’s a very different sense of who they were than just looking at a photo or video.”
A sense of scale
,
Source: Wareable