ButtonMasher is our column about video games and gaming culture – from the offbeat fringes to the cutting-edge innovations behind the latest blockbusters
The world is our playground – especially when it is scanned and replicated exactly in a video game.
A wave of new games use a technique called photogrammetry to produce digital replicas of the real world. The technique can be used to recreate rooms, objects, streets and even whole cities, says Krzysztof Plonka of special-effects studio Better Reality in Poland.
Better Reality started out creating effects for the film industry. Now for the first time its technology will be used in a video game. Get Even is being developed by Polish studio The Farm 51. Another games studio in Poland, The Astronauts, is using a similar approach in its latest game The Vanishing of Ethan Carter . (The images in this article are all taken from this game.)
Canadian studio Pixyul wants to take the technique to its limit – scanning the whole world using drones and reconstructing it inside a game.
The first section, ReROLL, features an exact reconstruction of downtown Montreal and will be released next year. The studio plans to release new areas one by one, starting with North America, though it will have to wait for aviation authority approval for some.
Playing in an environment you recognise can give an added thrill. "The ideal world for a player is one that faithfully recreates our world but lets you do things you cannot do in reality," says Plonka. That may be racing supercars through your home neighbourhood or surviving the apocalypse – as in ReROLL.
Photogrammetry works by gathering hundreds of photographs of an object or scene taken from multiple angles and combining them into a 3D model. Common points between photos are joined up to create a basic shell and this virtual object is then skinned with the overlapping images.
"You not only end up with the exact shape of the object but the exact look of the object as it appears in the photos," says Andrew Poznanski of The Astronauts. The results are so accurate that photogrammetry has been used to improve aerial mapping and forensic analysis of crime scenes. Its impact could be felt most strongly in gaming, however.
To get the photographs used in The Vanishing of Ethan Carter, Poznanski takes his camera into the mountains near his studio (see picture below). It's hard work, he says. Getting the photorealistic look he is after for the game requires a lot of photos. "It's insane, I've lost count," he says. "If you don't count the failed experiments, it's in the tens – if not hundreds – of thousands of photographs."
Such detailed graphics would have brought previous consoles to a standstill. But the latest machines have a lot more muscle. The games also feature algorithms that are very good at swapping graphics in and out as and when they're needed, freeing up space. Instead, the technical challenge now is how to squash all the extra data into a game that won't take hours and hours to download. Faithfully recreating the world isn't a problem, says Poznanski. "But would players be willing to download a terabyte for a few hours of entertainment?"
That depends. For the developers, it's worthwhile if it makes a game more immersive, helping players forget they're staring at pixels on a screen. Poznanski has worked in the games industry for 20 years and has long been frustrated by graphical glitches that make it harder for players to suspend their disbelief. Most game environments are created by teams of artists. Even when highly realistic, there are often details that make the look of the game fall short. The real-life appearance of wear and tear, for example, often fails to be captured correctly. Out of necessity, artists create sections of "texture" – a scuffed floor or brick wall – that are then repeated throughout the game.
Consciously or not, we pick up on the artifice, says Poznanski. Instead, he wants players to stop seeing graphical effects and start seeing the world. "We want players to walk through a forest and feel that it is a forest, not think 'Wow, these are beautiful graphics'," he says. "We want that part of the brain to switch off."
Add in virtual reality, which The Astronauts is also experimenting with, and immersion is taken to another level. In fact, the combination of photorealism and virtual reality could make games all too real. Violence, for example, might become a bigger problem when our actions feel realistic and look increasingly gruesome.
"By removing the frame of the game screen, we begin to delve into a realm of personal responsibility that goes beyond what one feels with current game experiences," says Mark Bolas at the University of Southern California. As a result, Bolas believes the type of games that players find enjoyable could change. "The market will sort this out," he says.
If you would like to reuse any content from New Scientist, either in print or online, please contact the syndication department first for permission. New Scientist does not own rights to photos, but there are a variety of licensing options available for use of articles and graphics we own the copyright to.
