In recent times, Lytro, the company known mostly for developing light field cameras, has been moving into VR. It has also announced a specialized renderer called the Lytro Volume Tracer, for rendering CG VR experiences. The announcement is accompanied by a CG animated VR short called One Morning, directed by Trollhunters showrunner Rodrigo Blaas and made using the Nimble Collective platform.
What is a light field?
Before we get into Lytro’s volume tracer, let’s look first at light field technology. A light field is essentially all the light that passes through an area or volume. A normal camera just captures the rays of light coming through its lens and aperture which then hits the film or sensor to produce an image.
But a light field camera effectively has multiple lenses, or is made up of an array of cameras, each at slightly different positions. That means it can capture light from multiple vantage points and then work out different things about that light such as intensity and angular direction. Processing that information in a certain way means you can replicate the captured light field in a virtual space, and do things with it.
What can you do with it? Well, Lytro had initially been selling both pro and consumer light field cameras. One selling point with these cameras was that photographers could take a photo of a scene and then with special software re-focus onto different areas in the image. But in 2015 Lytro changed tack after realizing it could do more things with captured light fields. It jumped into the cinematic side of light fields, and in particular, into VR.
Recently, the company has been showcasing its Lytro Immerge system – a light field camera array, server, editing, and player set-up – to shoot and produce light field films. This involves shooting live action with a large light field camera array, but the system also facilitates the compositing of CG imagery into the live action. Lytro showcased the result in the VR film Hallelujah. This behind the scenes video, below, shows how Hallelujah was made.
,
,
The film was made via the cloud-based animation platform Nimble Collective. A team of artists from around the globe collaborated on it, something Blaas was keen to explore further. He was also keen to look into vr films, but his previous experience with vr had been that most projects he looked at were decidedly low resolution.
“I wasn’t interested in vr until now because I’m always trying to push the quality of the work that I do in animation,” Blaas said. “The thing that attracted me was when Lytro came back saying, ‘We don’t want you to change your process. We want you to keep all the detail – the really vibrant and high quality imagery. You create an animation and we’re going to be the ones to translate that into a vr experience.’”
Like a number of filmmakers who have worked in vr, Blaas had to adapt his style for a medium in which the viewer can look anywhere. The bird is animated to look directly at the viewer and also gaze elsewhere, which the viewer can follow along with.
“The tricky part is knowing how to direct the viewer to what you want to see and what you want to tell him in that environment,” said Blaas. “You have to be way more aware of what the sound around you is doing, and the visuals, and how to simplify the story you’re telling. You really concentrate the viewer on one thing.”
“One of the things that we started putting in originally was a lot of detail around us,” adds Blaas. “Then we started stripping it, in the way that we wanted to get the attention on one place so you can follow the story of this little bird.”
Lytro’s light field future
Other companies have been diving into this area, too, such as OTOY with its Octane renderer. But it’s clear that getting vr and light field content out there for audiences to see is going to be an important part of mainstream adoption.
Lytro acknowledges that it is still early days in terms of getting creators on board with its tools. Still, Berton Jr. says, many have been asking them about mixing more live-action light field footage with cg light field footage.
“That’s something that we’re very well aware of, and once you’ve got everything in light field space, then we’ve got a game to play,” the visual effects supervisor said. “We’re absolutely talking to those people, and we’re looking for great projects that we can continue to develop our pipeline with.”
One Morning was featured last month at both VR on the Lot and Animation is Film festival, both in Los Angeles. Lytro has a form on its website to request viewings, and tells Cartoon Brew there will be further screening locations announced soon.
Source: Cartoon Brew