Producer Maureen Towey and visual effects supervisor Orin Green look at Rachel Kolb through Lytro’s camera. Credit Marcelle Hopkins/The New York Times
By MAUREEN TOWEY
Seven years ago, when Rachel Kolb was 20, her friends pitched in to help her learn how to hear music. She was born profoundly deaf and had recently received a cochlear implant to give her partial hearing. “They were so gracious — they made a playlist and annotated it: At this time in the song, it’s this instrument coming in,” she recalled. “So I learned to recognize, oh, that’s a piano, because my friend wrote down, ‘At 35 seconds the piano starts to play.’ ”
Rachel is a former Rhodes scholar and current doctoral student whom I met through Peter Catapano, the editor of the Disability series, on the Opinion desk. I work as a senior producer in the VR department at The Times, creating both long- and short-form virtual reality videos. Part of my job is constantly looking for stories that will fit our uniquely experiential medium.
Peter introduced me to Rachel after she submitted an essay to him about her experiences of music both before and after receiving the implant. She described music as tactile and visual — not something that you just hear. We thought her story was a great match for the immersive treatment that virtual reality provides. We started to adapt Rachel’s article into a storyboard and quickly settled on a VR piece that would be a mix of animation and live-action, with narration from Rachel.
When we met Rachel in person, we were excited to see that she exhibited a natural on-camera presence — vibrantly intelligent and self-possessed. When we worked with her, she read our lips, since none of our team members knew American Sign Language (ASL). But lip-reading for more than two hours is tiring and it’s a less effective method for large groups, so Rachel often uses a sign language interpreter.
During our continuing conversations about the development of the piece, she suggested that we try to find another deaf collaborator for our team. This made a lot of sense, especially because we wanted to keep the piece deaf accessible and we knew that more deaf perspectives during the creative process would strengthen the final product. After some research, we found James Merry, an animator who works for the production company Squint/Opera in London.
,
Source: New York Times