Remedy Entertainment (Alan Wake, Quantum Break) and GPU manufacturer Nvidia have teamed together in an effort to improve and streamline facial motion capture and animation for video game development.
Showcased at the annual SIGGRAPH conference on computer graphics, Remedy’s team fed videos of actors performing into a deep learning convolutional neural network running on Nvidia’s eight-GPU DGX-1 server. Using this information, the network generated eerily refined and realistic 3D facial animations, accurate down to the tiniest twitch of the actor’s facial muscles. According to Remedy and Nvidia, this process negates the need for hours of comprehensive data conversion and tweaks, decreasing the exorbitant price of traditional motion-capture animation.
As reported by Ars Technia, Remedy Entertainment lead character technical artist Antii Herva said, “Based on the Nvidia research work we’ve seen in AI-driven facial animation, we’re convinced AI will revolutionize content creation. Complex facial animation for digital doubles like that in Quantum Break can take several man-years to create. After working with Nvidia to build video-and audio-driven deep neural networks for facial animation, we can reduce that time by 80 percent in large scale projects and free our artists to focus on other tasks.”
In addition to lowering cost, the neural network technology has the potential to increase the production speed of new video games. As written in Nvidia’s published research, the network requires only five to ten minutes of captured footage to produce precise, accurate animations. In comparison to several other leading-edge techniques of facial motion capture, Nvidia’s new neural network shows great improvement, particularly in difficult areas such as eyes and lips. This process may assist in alleviating or eliminating the notorious ‘dead-eye’ issue which has long plagued modern video game animation.
To see the new motion capture animation technology in action, check out the video below.