Picture this — you receive a video of a politician’s before-election speech from a friend. You see that the politician in question is saying something outrageous. The sheer ridiculousness of his speech makes you forward it in your own circle, only to realise much later that the video was manipulated. But by now, the video has been circulated wide and far, with almost no way to recall it back. Such hyper-realistic manipulated videos are called deepfakes. These videos leverage powerful techniques from machine learning and artificial intelligence to generate highly deceptive visual and audio content.
The repercussions of such deepfakes have been dangerous with compromised videos of public figures in circulation which threaten their reputation. Worse, it is anticipated that deepfakes may even play a large role in swaying elections of countries. Notably, Facebook, Twitter, and TikTok have already banned such deepfake content on their platform.
With such major consequences of deepfakes, there have been several efforts to create tools, albeit with varying degrees of success, which can help detect them. Here, we discuss a few of them:
Microsoft’s Video Authenticator Tool
Launched in September this year ahead of the US elections 2020, this video authenticator tool is created by Microsoft. It can analyse a still photo or video to provide a confidence score to detect whether or not the media is manipulated. It detects the blending boundary of the deepfake and subtle grayscale elements that are undetectable to the human eye. It also provides this confidence score in real-time.
The tool has been created with a public dataset from Face Forensics++ and has been tested using the Deepfake Detection Challenge Dataset. Both these datasets are leading technologies for training and testing deepfake detection technologies.
Additionally, the tech giant also introduced a new technology that can find doctored content and assure readers of its authenticity. It contains two parts – the first component is integrated into Microsoft Azure which allows the content creator to digital hashes and certificates which remains in the part of its metadata, and the second component is which helps the reader verifies and matches these certificates and hashes to check authenticity.
Deepfake Detection Using Biological Signals
The researchers from Binghamton University and Intel created a tool that goes beyond just deepfake detection and recognises the deepfake model behind the compromised video. This tool looks for unique biological and generative noise signals ‘deepfake heartbeats’ left by deepfake model videos. These signals are detected from 32 different spots in a person’s face, called the Photoplethysmography (PPG) cells.
The architecture of this model is based on convolutional neural networks with VGG blocks. It uses OpenFace library in Python for face detection, OpenCV for image processing, and Keras for neural network implementations. Like Video Authenticator tool from Microsoft, the learning setting of FaceCatcher is based on the FaceForensics++ (FF) dataset. As per the researchers, this tool has 97.29% accuracy for fake video detection.
Deepfake Detection Using Phoneme-Viseme Mismatches
This model is invented by researchers from Stanford University and the University of California. This technique exploits the fact that visemes, which denote the dynamics of the mouth shape, are sometimes different or inconsistent with the spoken phoneme. For example, for saying words such as mama, baba, and papa, there might be a phoneme-viseme mismatch, which can be used in detecting even spatially small and temporally localised manipulations in deepfake videos. The team used lipsync deep fakes created using three synthesis techniques – Audio-to-Video (A2V), Text-to-Video for short utterances (T2V-S), and Text-to-Video for longer utterances (T2V-L).
The technique was used for both manual video authentication and automatic approaches. For A2V, T2V-S, and T2V-L, this model showed an accuracy of 96.0%, 97.8%, and 97.4% for manual authentication, and 93.4%, 97.0%, and 92.8% for automatic authentication, respectively.
Forensic Technique Using Facial Movements
This model tracks facial expression and movements of a single video provided as input. It extracts the presence and strength of specific action units. This detection model has a one-class support vector model (SVM) which distinguishes an individual from others as well as comedic impersonators from deepfake impersonators.
This model utilises OpenFace, an open-source facial behaviour analysis toolkit to extract facial and head movements in a video. The library provides 2D and 3D facial landmark positions, head pose, and facial action units for each frame.
Recurrent Convolutional Strategy
The Recurrent Convolutional Strategy uses recurrent convolutional models (RCM) for detecting face manipulation in videos. RCM are a class of deep learning models which effectively exploit temporal information from image streams across domains. This technique can detect deepfake, Face2Face, and FaceSwap tampered faces in video streams. It was tested on the FaceForensics++ dataset and showed an accuracy of up to 97%, a 4.55% improvement from its preceding techniques.
Wrapping Up
The technique for creating deepfakes keeps developing and advancing. The tools and techniques discussed in this listicle do not present complete accuracy and effectiveness. However, they are making strides in the right direction for a battle that is much bigger.
Share
I am a technology journalist with AIM. I write stories focused on the AI landscape in India and around the world with a special interest in analysing its long term impact on individuals and societies. Reach out to me at shraddha.goled@analyticsindiamag.com.
Related Posts
Voice Slowly Catching Up on Multimodal AI Features
Vandana Nair02/03/2024
Unravelling Deep Model Artefacts for Deepfake Videos Detection
Gagan Tewari Tewari13/08/2021
Guide To Real-Time Face-To-Face Translation Using LipSync GANs
Aditya Singh06/03/2021
Interview With Olivier Grellier: Kaggle GrandMaster And Senior Data Scientist At H2O.ai
Ram Sagar13/01/2021
Sejuti Das07/01/2021
Don’t Have a Programming Background? This Course by AnalytixLabs Will Still Make You Data Science Job-ready
Srishti Deoras22/12/2020
Top 8 AI-Powered Features Released By Adobe In 2020
Ambika Choudhury17/12/2020
Top Data Science Service Providers In India 2020
Srishti Deoras03/12/2020
CORPORATE TRAINING PROGRAMS ON GENERATIVE AI
Generative AI Skilling for Enterprises
Our customized corporate training program on Generative AI provides a unique opportunity to empower, retain, and advance your talent.
Upcoming Large format Conference
Data Engineering Summit 2024
May 30 and 31, 2024 | 📍 Bangalore, India
Download the easiest way to
stay informed
Vidyashree Srinivas
It’s time to rewrite your resume.
Donna Eva
Mohit Pandey
Top Editorial Picks
Gopika Raj
Gopika Raj
Mohit Pandey
Mohit Pandey
Subscribe to The Belamy: Our Weekly Newsletter
Biggest AI stories, delivered to your inbox every week.
Also in News
Oracle Integrates GenAI to Enhance Supply Chain and Automate KPIs
T-Hub Incubated MATH is Launching AI Career Finder to Create AI Jobs
India Draws Inspiration from Census To Collect Data for AI
Zuckerberg Doesn’t Want AI to End Up Like Mobile Apps, Controlled by Apple and Google
Hugging Face Already has 1000s of Llama 3 Models – and Counting
‘Many Indian VCs Don’t Even have a Thesis on Deep-tech Investment’
Japan is the Next Big Hub for Indian Tech Talent
Will TypeScript Wipe Out JavaScript?
AI Courses &Careers
View All
India is a Goldmine for AI Talent
Donna Eva15/04/2024
Top 10 LMS Platforms for Enterprise AI Training and Development
Analytics India Magazine14/04/2024
AI Clock is Ticking: Wake Up Call for Education Institutions
Siddharth Jindal18/09/2023
Become a Certified Generative AI Engineer
Industry
Insights
View All
Genpact Launches AI Innovation Centre in Gurugram
Mohit Pandey23/04/2024
KissanAI Releases Dhenu Llama 3, an Indic LLM for Farmers
Siddharth Jindal19/04/2024
Enhancing AI Integration through Optimal Data Management in the Global Convenience Food and Beverage Sector
Devendra Desai19/04/2024
Check our Industry Research Reports
AI Forum for India
Our Discord Community for AI Ecosystem, In collaboration withNVIDIA.
AIM Videos
What is Computer Vision and How it Works?
Flagship Events
Rising 2024 | DE&I in Tech Summit
April 4 and 5, 2024 | 📍 Hilton Convention Center, Manyata Tech Park, Bangalore
MachineCon GCC Summit 2024
June 28 2024 | 📍Bangalore, India
MachineCon USA 2024
26 July 2024 | 583 Park Avenue, New York
Cypher India 2024
September 25-27, 2024 | 📍Bangalore, India
Cypher USA 2024
Nov 21-22 2024 | 📍Santa Clara Convention Center, California, USA
Data Engineering Summit 2024
May 30 and 31, 2024 | 📍 Bangalore, India
GenAI
Corner
View All
7 AI Startups that Featured on Shark Tank India Season 3
Siddharth Jindal15/04/2024
Top 9 Semiconductor GCCs in India
Shyam Nandan Upadhyay15/04/2024
Top 6 Devin Alternatives to Automate Your Coding Tasks
Siddharth Jindal08/04/2024
10 Free AI Courses by NVIDIA
Shritama Saha02/04/2024
Top 6 AI/ML Hackathons to Participate in 2024
Siddharth Jindal22/03/2024
What’s Devin Up to?
K L Krithika17/03/2024
10 Underrated Women in AI to Watchout For
K L Krithika11/03/2024
10 AI Startups Run by Incredible Women Entrepreneurs
K L Krithika08/03/2024
Data
Dialogues
View All
Automation Anywhere Wants to Augment Humans with AI, Not Replace Them
Shritama Saha18/04/2024
Father of Computational Theory Wins 2023 Turing Award
Shritama Saha13/04/2024
Building Open Source LLMs is Not for Everyone
Vandana Nair12/04/2024
This 20-year-old AI Researcher Created the much-needed Indic LLM Leaderboard
Mohit Pandey10/04/2024
NPCI is Exploring AI-Powered Futuristic Payment Frontiers: CTO
Pritam Bordoloi08/04/2024
Prisma AI Has an ‘Eye on You’ at Adani Airports
Vandana Nair06/04/2024
Salesforce Chief Ethicist Deems Doomsday AI Discussions a ‘Waste of Time’
Pritam Bordoloi01/04/2024
Zoho’s ManageEngine Invests $10 Mn in NVIDIA, Intel, and AMD GPUs
Vandana Nair30/03/2024
Future
Talks
View All
Quora’s Poe Eats Google’s Lunch
Gopika Raj17/04/2024
Gopika Raj08/04/2024
Rakuten Certified as Best Firm for Data Scientists for the 2nd Time
Analytics India Magazine08/04/2024
This Indian Logistics Company Developed an LLM to Enhance Last-Mile Delivery
Pritam Bordoloi02/04/2024
Perplexity AI Reviews with Pro Access
Vandana Nair02/04/2024
What to Expect at the ‘Absolutely Incredible’ Apple WWDC 2024
Vandana Nair31/03/2024
Will StarCoder 2 Win Over Enterprises?
Pritam Bordoloi20/03/2024
Missing NVIDIA GTC 2024 Would be a Foolish Sin
Vandana Nair11/03/2024
Developer’s Corner
Meta Forces Developers Cite ‘Llama 3’ in their AI Development
Sukriti Gupta19/04/2024
Why Developers Hate Jira
Mohit Pandey01/04/2024
Which is the Most Frustrating Programming Language?
Mohit Pandey18/03/2024
AI4Bharat Rolls Out IndicLLMSuite for Building LLMs in Indian Languages
Shritama Saha15/03/2024
In Case You Missed It
K L Krithika14/03/2024
Mohit Pandey13/03/2024
Anshika Mathews04/03/2024
Shritama Saha21/02/2024