The clapperboard has been a staple of Hollywood filmmaking for decades. But in an era of virtual reality (VR) 360-degree video, the simple tool may not be enough anymore, and new tools are needed.
One of these could be Adobe’s SonicScape project, which is set to be unveiled at the company’s Adobe Max conference in Las Vegas Thursday. “We’ve developed a new way of visualizing ambisonic audio,” explained Adobe Design lab lead senior experience developer Yaniv De Ridder during an exclusive preview of the project for Variety.
In many ways, SonicScape is a kind of 21st century extension of the clapperboard. First introduced when Hollywood started to add sound to its movies, the iconic tool is primarily used to synchronize recorded video with audio, allowing editors to precisely align the visual of the moment the board is closing with the resulting sharp clapping noise.
But with 360-degree video, time isn’t the only factor to consider. VR productions regularly combine 360-degree video cameras with special 3D microphones, with multiple individual capsules to capture sound from all directions. “Audio can come from anywhere,” said De Ridder.
This leaves video editors with the challenge to not only synchronize audio by time, but also by space. They have to make sure that what you see while wearing a VR headset actually matches what you hear, and that the sound for the action in front of you isn’t accidentally placed behind you.
Adobe’s approach with SonicScape is to visualize the sound by overlaying colorful spots over the video that also correspond to the sound’s frequency and intensity. With these visual cues, editors simply have to adjust the sound so those spots are where the action is. They can also use it to add new assets to an existing video, and for example include ambient sounds to improve the soundscape of a 360-degree scene. “We are trying to take the guesswork out of it,” said Adobe Design Lab senior experience designer Michael Cragg.
,
Source: Variety