I’m standing on a circular stage with neo-folk musician Beck. Surrounded by cheering fans, I blush, then turn and wave to thank them. They don’t see me, though, because I’m actually sitting in a dark, empty back room at Sundance Film Festival wearing an Oculus Rift.
A few minutes before, I was blasting spaceships through Oculus, but that exhilaration was familiar. Most people can imagine how gaming will evolve thanks to virtual reality. But bashful is an emotion no other piece of media has ever made me feel, which is why Oculus Cinema or something like it could unlock whole new dimensions of visual and aural performance artistry.
Famed music video director Chris Milk (Arcade Fire, Kanye West, U2) dreamed up this Lincoln-sponsored 3D Beck performance “Hello Again” back in late 2012. Beck would play “Sound And Vision”, a David Bowie cover, in a round room ringed with 160 musicians of every style. A brass line, a gospel choir, a string quartet. You can watch a behind-the-scenes video about the show here below:
I press a button, and suddenly I’m in the back of the audience, slowly revolving counterclockwise around this circuit. I’m free to turn in any direction so I can inspect the artists when I move past, or watch Beck on stage. As the boom-bap sound of a drum kit grows louder in my right ear, I know I’m approaching a rock band before I even see them. Sure enough, I turn around and find the drummer intensely keeping time. I listen as he eventually fades out of my left ear as I get farther away. The whole time, the look on his face says he knows this isn’t just any performance.
That might be because what’s recording what I saw, heard, and felt was a terrifying human head mannequin covered with extra ears. No face. Just eight ear-shaped microphones and a “360 Heros” rig with six GoPro cameras. Together, they simultaneously record 3D audio and video.
Music aficionados may have heard of binaural recordings, where a dummy head with microphones in each of its two silicone ears pick up sound the same way you’d hear it in person. A sound on the right will hit the dummy’s right ear a split second before its left. When listened to on headphones, binaural recordings make you feel like you’re standing still in a room with the musicians. But Milk took it a step further by inventing possibly the world’s first 360 binaural head microphone. When combined with Oculus’ head tracking sensor, this creepy face’s recordings maintain the binaural illusion no matter which direction you turn.
What’s surreal is how magnificently Hello Again felt on Oculus considering that wasn’t even what it was shot for. The show was originally designed to be viewed on a computer screen. A webcam would track the viewer’s head movement to adjust the video and sound, approximating virtual reality with no special hardware required. Experienced this way, it’s still painfully apparent that you are not there, though. It’s like looking out a window versus being outside. The facsimile just makes you long to have attended the concert in person. You probably don’t have a VR rig handy, though, so you can watch that version here.
But seen and heard through Oculus, I actually felt like I got a better deal than the live audience. They were stuck in their seats. I could instantly teleport from the center of the stage right beside Beck to the front of the crowd to the outer perimeter with close-ups of the backing bands. Not only did Oculus make me feel present, I felt omnipresent. And that’s where virtual reality’s opportunity to evolve performance art lies — in its ability to let us be both anywhere and anyone.
“My primary interest in technology is to tell stories that affect humans on an emotional level,” Milk tells me. “I’m not interested in tech demos. I’m not interested in how many hairs are in the paintbrush. I’m interested in the paintings you can make with the paintbrush.” He’s now founded a company called Omnia to build more hardware that could help artists explore the boundaries of recording a moment.
That aligns with Oculus’s plan to help developers build out an ecosystem of auxiliary gadgets. “The last thing we want to do is say to an artist, ‘Go figure out this 360 stereo camera technology binaural multi-access audio and then execute on your vision,’” Oculus product lead Joe Chen tells me. “The more folks like Chris that help us with the technology development the better, as then we can cut to the chase of creating art instead of just making tech for tech’s sake.” Thanks to its new parent company Facebook, Oculus has plenty of resources to invest into such a community of experience hackers.
As Gizmodo’s Jordan Kushins recently wrote, films like Gravity could become much more stimulating if made for Oculus Cinema. Instead of watching Sandra Bullock floating through outer space from the third-person perspective, you’d be there with her. Or you could be her, experiencing the space station tear apart all around you from whatever angle you decide to look. George Clooney would stare into your eyes as you figured out how to survive.
3D binaural recordings and 360 video could similarly transform the consumption of live performance art. It will be tough for virtual reality to match the invigoration of actually being there, but it could unlock much more vivid experiences for those who aren’t.
The top complaint from most people about seeing a concert or theater performance is that it’s hard to see. You’re either too short in a standing-room-only crowd or too far away in the stands. VR could give everyone a front-row spot, but with the autonomy of perspective missing from a video feed. A limitless number of people could occupy this same virtual space, so an intimate small-room experience could scale.
Even more exciting than being in the front row seeing a band would be being the band itself. Sleeker virtual-reality recording rigs like Milk’s can fit onstage, giving you the perspective of the artists like I got with Beck. Again, it’s not just the vantage point, but the ability to control where you’re looking that makes it feel like you’re the one who’s famous. Imagine getting to be a ballerina or the star of a play.
What’s most tantalizing is that performances could eventually be done with a virtual audience specifically in mind. Even Beck’s “Hello Again” could be watched cohesively in person. But what if Beck recorded alternate versions of the show where you got to call the shots? You could decide what backing musician he would give a solo to, whether he’d jam more or bring back the chorus, whether he’d cut the house lights and soulfully croon by himself or turn lights the up and invite the whole crowd to sing along. A virtual choose-your-own-adventure, shows like this would blur the lines between passive and interactive art
Sure, most people will still see music the old-fashioned way. But a few adventurous artists will explore the new virtual frontiers of performance art. And when you link arms with your favorite band and take a bow together, enveloped by thunderous applause, it won’t matter whether you call it a concert or a video game. All that will matter is how good it feels.
[Image Credit: Future Of Storytelling, 3diosound]
Comment