Talk like a media forensics researcher

Artefacts or noise — irregularities or uncommon features in manipulated or synthesised media content that expose their unauthentic nature

Artificial intelligence (AI) — computer algorithms that can learn from data and experience to perform tasks which would usually require a human level of intelligence

Deepfake — digital media content created or manipulated using generative AI, most commonly used for images and videos

Disinformation — false information that is deliberately fabricated and spread to deceive or mislead people

Generative AI — a type of artificial intelligence capable of creating a wide variety of data, such as images, videos, audio, text and 3D models

Media forensics — the subfield of AI that aims to detect and mitigate manipulated or synthesised media using AI algorithms

How would you feel if you were scrolling through Instagram or TikTok and came across a video of yourself that you had not created or were even aware of? What if the video showed you doing something you would never do or saying something that you disagree with?

In March 2022, just a month after Russia launched its invasion of Ukraine, a video was released showing the Ukrainian president, Volodymir Zelensky, calling for his troops to surrender. Upon closer inspection, the video was quickly identified as a deepfake, a piece of media modified with generative AI models and designed to trick the viewer.

Luckily, the deepfake of Zelensky was fairly easy to spot as the image of his head and his voice were clearly inauthentic. However, just a year and a half after its release, advancements in generative AI have made deepfake videos much harder to identify. The improving quality of deepfake media is making many people nervous about the potential harm that the technology could cause.

Professor Siwei Lyu, a computer scientist at University at Buffalo, The State University of New York, is a leading expert in the field of generative AI and media forensics. Much of Siwei’s research focuses on the detection of deepfake media, although his expertise spans the whole field of generative AI.

What is generative AI?

In order to understand what generative AI is, we first need to clarify what we mean by the term ‘AI’. Artificial intelligence, or AI, refers to algorithms and systems that can learn, predict and, in some cases, create. Algorithms and tools that have this creative ability are known as generative AI, and they can be used to fabricate media such as images, videos and audio.

Recent advancements in the field mean that generative AI systems can now learn by analysing photos, audio and videos that are widely available on the Internet. As a result, it has become cheaper and easier for users to create convincing fake media, created to deceive the viewer, in large quantities. “Since 2015, generative AI technologies have experienced astronomical development, with new methods showing up on a monthly or even weekly basis,” says Siwei.

Although generative AI is often used to create deepfake media, researchers in the field are convinced that the technology can also be used for good. For example, a new generative AI algorithm is being used to help rehabilitate stroke patients who have lost their ability to speak. The algorithm translates the patient’s brain activity into simulated speech that resembles their old voice. Such technology will have a hugely positive impact on the patient’s everyday life.

What are the dangers of deepfakes?

Deepfakes can pose significant threats to society. “Personal security risks emerge as deepfakes can be used to fabricate convincing yet false representations of individuals,” explains Siwei. “This can lead to reputational damage and psychological distress for anyone whose personal images or videos can be accessed by people using AI to produce deepfakes.”

At a wider scale, deepfakes can impact our democratic processes by spreading disinformation. “By creating illusions of an individual’s presence and activities that did not occur in reality, deepfakes can influence our opinions or decisions.” This can become a particular issue around election time, when people are trying to decide who to vote for based on things they see and read online.

As the field of generative AI continues to develop, deepfakes will become more powerful but so will our methods of detecting them. Siwei compares the situation to developing vaccines to combat a virus. As vaccines improve, viruses evolve new ways of overcoming them, so scientists have to work even harder to develop new vaccines.

How can we detect deepfakes?

Although deepfakes are becoming more convincing, it is possible to detect them simply by looking closely to find artefacts. The generative AI models that create deepfakes are trained on enormous amounts of data, but they have no understanding of the laws of physics or how the human body works. This means that they often make mistakes, which attentive observers can pick up on.

For example, AI models are often trained on thousands of images of human faces downloaded from the Internet; however, almost all of the people in these images will have their eyes open. As a result, the simulated people in many deepfake videos do not blink, which is something that viewers can spot relatively easily. Other errors to look out for are missing teeth when someone is talking, hands that have the wrong number of fingers, and the reflections/light flecks in eyes pointing in different directions.

Unfortunately, as generative AI models become more sophisticated, deepfakes are becoming harder to spot, even if you know what to look for. To address this, Siwei has developed tools that can detect deepfakes that even a trained human would struggle to identify.

Whilst genuine videos come directly from the real world, deepfakes have been stitched together from a variety of sources. The process of stitching the deepfakes together is not perfect and often leaves minor inconsistencies or visual errors, known as artefacts or noise, that do not appear in the original media. Siwei likens his detection tools to X-ray scanners that can see inside deepfakes and uncover artifacts that might not be visible to the human eye.

How else can we stop deepfakes?

Detection methods can be effective at spotting deepfakes, but what if we could prevent deepfakes from being created in the first place? “Unlike detection methods, the pre-emptive approach directly obstructs the training or generation of deepfakes,” explains Siwei. “One idea is to ‘poison’ the would-be training data by adding specially designed patterns.”

These patterns disrupt the generative AI’s training process and cause the AI models to create low-quality deepfakes. For example, one of Siwei’s methods deliberately adds unnoticeable patterns into the original media, making it hard for generative AI models to detect faces. This could prevent one of the most common methods of creating deepfakes, which involves inserting someone’s face into a video that they did not actually appear in.

What does the future of generative AI hold?

“Looking into the future, we will continue to see accelerated development of deepfake technology,” says Siwei. He predicts that we will see new generative AI models that can create deepfakes that look more realistic and have fewer artifacts.

“In the not-too-distant future,” continues Siwei, “ordinary users may have access to more easy-to-use and ready-made tools to manipulate media in the same way that they use Photoshop to edit images today.”

Although deepfake technology will become more accessible, Siwei is not too worried about the consequences. “I don’t think it will necessarily lead to a future of dooming dystopia or an apocalypse!” he says. “Human brains are unbelievably flexible and as we are seeing more synthetic media created with generative AI, our ability to cope with them will evolve.” For example, in the early 2000s, spam emails were becoming a huge problem until we learnt how to spot them and developed technologies to help protect us.

“We must continue developing deepfake forensic methods that are more effective, efficient, robust and explainable,” says Siwei. In doing so, he, and other researchers like him, will help to halt the progress of deepfake technologies and ensure that generative AI is used for the good of society.