I'm sat in a deckchair relaxing on the beach, watching the sun go down. I would happily have stayed there for a good while, but my time is up and I have to return to the real world. When I remove the Oculus Rift headset and headphones, the noisy conference room comes as a huge shock.
I had been experiencing Perfect Beach , a virtual reality (VR) experience created by developer nDreams in Farnborough, UK. The firm was showing off its creation at the SouthWest VR conference in Bristol last week, where game developers, film-makers and visual-effects artists came together to discuss how VR experiences could improve films and games.
The rise of VR headsets in recent years has been led by game developers seeking a way to immerse players even deeper in virtual worlds – just yesterday top developer Valve announced a new headset in partnership with HTC. But adding VR to a video game can actually hamper play, because the goggles stop you seeing the controller in your hands.
Yet the ability to put someone in a space and give them limited but meaningful interaction provides a new experience – one that could shake up the film industry.
"The thing I find really engaging about VR is the sense of presence," says Dave Ranyard of Sony Computer Entertainment Europe, who is developing software for Sony's upcoming VR headset, Morpheus. "It feels like a step change in immersion."
Hollywood hits
One of the big hits at the Sundance film festival in Utah in January was Wild: The Experience, a 3-minute virtual reality version of the film Wild starring Reese Witherspoon, in which viewers can watch from all directions. VR firm Oculus has also set up a VR film division called Story Studio to produce short films for the latest version of its headset.
But VR film-making doesn't have to be a blow-out Hollywood extravaganza to work; even simple films, like my deckchair experience, can be surprisingly convincing. "In any other medium this would be boring, you wouldn't sit there for 10 minutes," says Nick Pittom of VR firm Fire Panda. But it's narrative experiences where VR will really shine, he says – if a static VR scene can be powerful, a story will be even more so.
As VR film is a new medium there are no set rules. Phil Harper of Alchemy VR, a UK firm working on immersive natural history films with veteran broadcaster David Attenborough, calls his nature documentaries 360° video, because they are made by arranging a number of cameras in a sphere and stitching the resulting videos together to create an all-encompassing film. Watching someone speak to you in VR is more intimate than on a TV screen, he says, because they appear to be the same size as a real human. "The brain begins to accept this as a social interaction, rather than something that's seen on a screen," Harper says.
Nowhere to hide
But this approach has its challenges. Because audiences can see all around them, it is impossible to hide equipment that would normally sit just out of shot. Even directors have to face their audience, or else hide. "Often I'm just stood in the background," says Harper. "You can't hide anything in 360° video."
In some situations that can free viewers of a framing camera's deception. "360° video is a truly honest format," says Harper, who thinks news broadcasts in particular could benefit. Earlier this year Vice News gave the first VR news report from a protest in New York.
For fictional stories, film-makers might want a different approach. One option used by Belgian VFX firm Nozon is to render high-quality 3D worlds on a computer and then "film" inside them, without having to worry about having equipment on show. This 360° CGI is convincing – Nozon's Matthieu Labeau showed me a short but incredibly lifelike clip in which a robot and a beaver hang out in an ornate ballroom.
Both 360° CGI and 360° video have the same problem, though: they can't react to the audience. Limited interaction, like selecting different scenes based on where you look, is possible, but that's about it. Viewers also tend to look straight ahead, so you have to cue them to look elsewhere with particular sounds or pointing characters. "I don't think a movie where things are happening all around you makes sense, because you will feel you missed half the movie," says Labeau.
That's where game engines, the software that powers gaming's 3D worlds, can help. Visual quality takes a hit because rendering happens in real time, but that also allows for a more interactive experience, heightening immersion. You can place a scene to the left of the viewer and encourage them to turn their head, but if they don't take the hint the characters can move directly into view, says Pittom.
If you would like to reuse any content from New Scientist, either in print or online, please contact the syndication department first for permission. New Scientist does not own rights to photos, but there are a variety of licensing options available for use of articles and graphics we own the copyright to.
