Researchers have discovered a surprisingly simple way to detect deepfake video calls: ask the suspect to turn to the side.
The trick was shared this week by metaphysics.aia London-based startup behind the viral Tom Cruise deepfakes.
The company used DeepFaceLive, a popular video deepfaking app, to turn a volunteer into various celebrities.
And be the first in line for ticket offers, event news and more!
Most of the recreations were impressive when looking straight ahead. But when the faces turned a full 90 degrees, the images were distorted and the spell was broken.
The team believes that the defects arise because the software uses fewer reference points to estimate side views of faces. This doesn’t force the algorithm to guess what it would look like.
“Typical 2D alignment packs view a profile view as 50% hidden, hindering recognition as well as accurate training and subsequent facial synthesis,” explains Martin Anderson of Metaphysic.ai in a blog post.
“Often the generated profile landmarks will ‘pop out’ at any possible group of pixels that may represent a ‘missing eye’ or other facial detail hidden in a profile view.”
These weak spots can be strengthened, but it takes a lot of work.
YouTuber DesiFakes proved it was possible after adding a deepfake Jerry Seinfeld to a character in Pulp Fiction. But this required extensive post-processing. In addition, Seinfeld’s profile view was very similar to the original actor.
Yet this is difficult to replicate for the general public, as we are rarely filmed or photographed in profile – unless we are arrested.
As a result, deepfake models may have insufficient training data to generate realistic side views.
Metaphysic.ai’s research comes amid growing concerns about deepfake video calls.
In June, several European mayors were duped by a video call from Vitali Klitschko.
Days later, the FBI warned that scammers were using deepfakes in interviews for remote jobs that provide access to valuable information.
The side-on trick may not have saved all victims. Future 3D landmarks can deliver compelling profile views, while photo-realistic CGI models can replace entire headlines.
Nevertheless, the side view trick adds another chance to detect the fakers – and another reason not to get arrested.