Marc Whitten, president of Create Solutions at game software firm Unity notes that today’s most realistic content relies on highly-detailed 3D modelling of objects.
Last year, Unity showed off a computer generated clip of a lion and its cub, external featuring two million individually rendered strands of fur.
“If you don’t do that, it does not come across as photorealism,” argues Mr Whitten. The firm has also developed highly lifelike models of humans, where digital puppetry, external controls their subtle facial expressions.
There’s still room for improvement, he adds. There are many other difficult-to-simulate materials, such as clothing, which are still a long way from looking photorealistic in games.
One important emerging technology for game graphics is neural radiance fields, or NeRFs. California-based Luma AI specialises in this and says it already has customers using the tech to make games.
A NeRF is an artificial intelligence (AI) system that can represent objects or scenery captured in photographs or video footage in the real world.
“When you show it these images from different sides, the network learns how light is bouncing off of everything,” explains Luma AI co-founder Amit Jain. “It measures light and it learns from light.”
The way light reflects off a motorbike’s leather seat versus a headlamp, for example, is completely different and simulating that in a game is very challenging. Nerfs could help to automate the process.
Some of the best game graphics today use what’s known as ray-tracing – accurate simulations of the way light bounces off surfaces or creates glowing effects around neon signs and so on.
AI is making it possible to produce these effects in games despite only modest improvements in chip performance, says Bryan Catanzaro, vice president of applied deep learning research, Nvidia.
“We have to be smarter in how we construct the world and how we render it,” he explains.
A new mode for Cyberpunk 2077, an action-adventure game, called Ray Tracing: Overdrive, demonstrates the difference this can make.
Nvidia says its Deep Learning Super Sampling (DLSS) technology allows developers to create high resolution, high frame-rate graphics featuring ray-tracing with the help of AI.
“The model’s trained to know what things in the real-world look like,” explains Mr Catanzaro.
Games are increasingly difficult to tell apart from real life at times, says Nick Penwarden, vice president of engineering, Epic Games.
However he says it is still very hard to render certain materials convincingly – such as an iridescent layer of oil on a puddle of water.
“Those are aspects that we don’t yet have the power to simulate in real time,” he says.
And doing that on games consoles or home PCs is what matters. For movies featuring computer generated imagery, it’s possible to use huge computers and take many minutes or more to render individual frames.
