Our Chapter’s Publications

Introduction to Deepfakes

Deepfakes, a blend of deep learning and fake, involve the manipulation of existing media files through the substitution of faces or voices, employing techniques like generative adversarial networks (GANs). In this process, a generator crafts synthetic content, mimicking the appearance, voice, or behavior of a targeted individual, while a discriminator evaluates authenticity, creating an iterative feedback loop for continuous improvement.

As AI services advance, offering improved capabilities, concerns mount over the proliferation of deepfake technology. With synthetic media tools becoming more accessible, the risk of misuse grows, presenting threats such as reputational damage, fabricated evidence, and erosion of trust. The evolving landscape of deep learning and AI further blurs the lines between reality and fabrication, necessitating a cautious approach to mitigate associated risks.

Despite the transformative potential of generative AI across sectors, challenges loom large. McKinsey’s report estimates that AI could contribute $13 trillion to the global economy by 2030, fostering economic growth and creating new opportunities. However, the benefits come with their own set of issues, particularly in the case of deepfakes, including job displacement, ethical considerations, and potential misuse. The graph underscores the substantial economic value AI brings, highlighting the need for a balanced approach to harness its potential while addressing associated challenges.

 

 

As artificial intelligence (AI) continues to advance, so do the tactics of identity fraudsters. Among the most concerning developments is the proliferation of synthetic identity fraud, colloquially termed ‘Frankenstein’ identity. This insidious practice involves the fusion of authentic and counterfeit elements to create entirely new identities, often for illicit purposes such as fraudulent banking transactions. With biometric artifacts like voice and video deepfakes becoming more accessible, the risk of sophisticated identity fraud has soared. Recent surveys indicate a growing apprehension among organizations, with a significant majority recognizing synthetic identity fraud as a looming threat. In this context, it’s imperative to delve deeper into the prevalence and impact of synthetic identity fraud, particularly within the banking sector, which remains a prime target for such nefarious activities.

The accessibility of AI has increased the risk of advanced identity fraud using biometric artifacts like voice and video deepfakes. Approximately 82% organizations consider biometric artifacts as a growing threat and 90% of them consider synthetic identity fraud to be of a greater concern in the future.

 

 

Taking into account the past experiences of these organizations, about 46% of the organizations globally have experienced synthetic identity fraud, also known as ‘Frankenstein’ identity where authentic and counterfeit identities are combined to curate completely new ones. It’s usually used to open bank accounts or make fraudulent purchases. For obvious reasons, the banking sector seems to be the most vulnerable when it comes to such identity fraud.

 

 

Economic Impact of Deepfakes

Positive Impacts:

1. Cost Savings in Entertainment Production: The entertainment industry can benefit from cost savings in special effects production. Deepfake technology may provide a more affordable alternative to traditional CGI, lowering production costs for movies, TV shows, and video games.
2. Increased Efficiency in Advertising: Deepfake technology can streamline the creation of personalized advertising content, making it more efficient for businesses to target specific consumer demographics. This efficiency may lead to more effective marketing campaigns and increased sales.
3. Efficiency in Content Creation: In sectors such as advertising, entertainment, and marketing, deepfake technology can streamline content creation processes, potentially reducing costs and improving efficiency.
4. Growth in AI and Tech Industries: Developing and applying deepfake technology could drive growth in the artificial intelligence (AI) and technology sectors. This growth may contribute to job creation and economic expansion.

 

Negative Impacts:

1. Increased Cybersecurity Spending: Businesses and governments may face higher cybersecurity expenses as they invest in technologies and strategies to combat the potential misuse of deepfakes, protecting economic assets and sensitive information.
2. Legal and Compliance Costs: The economic landscape may be affected by increased legal and compliance costs as regulations are developed to address the responsible use of deepfake technology. Businesses may need to invest in legal consultations and compliance measures.
3. Impact on Reputation and Brand Value: Deepfakes threaten the economy by manipulating stocks through fabricated events, using synthetic content to influence investor perception and spreading false rumors of bank weakness, triggering potential bank runs.
4. Training and Skill Development: The need for skilled professionals in deepfake technology can lead to investments in education and training programs, creating new avenues for employment and economic growth.
5. Job Displacement in Traditional Media Production: As deepfake technology automates certain aspects of content creation, there is a risk of job displacement in traditional media production, impacting roles such as video editing, animation, and special effects. 
6. Consumer Trust Issues Impacting E-commerce: The erosion of trust in digital content due to deepfakes could impact e-commerce. If consumers become hesitant to trust online information, it may affect their willingness to engage in digital transactions, potentially impacting online businesses.
7. Global Trade and Diplomatic Risks: Deepfake technology’s potential use in economic espionage or to manipulate international perceptions could pose risks to global trade relationships. Governments and businesses may need to navigate diplomatic challenges and uncertainties.
8. Trust in Governance: Employing voice cloning or face-swap videos to concoct a false sense of urgency, we witness the manipulation of financial markets through fabricated announcements of impending interest rate changes, policy shifts, or enforcement actions. This not only undermines trust in governance but also jeopardizes the stability of our economic systems.

 

Case Studies

Financial Scam in Baotou, China: Exploiting Deepfake for Personal Gain

In a recent deepfake-driven financial scam in Baotou, China, a victim lost over Rs 5 crore. The perpetrator employed sophisticated AI-powered face-swapping technology, successfully making a deepfake of one of the victim’s friends during a video call. The victim believed the friend’s urgent need for funds, and promptly transferred the demanded amount, which led to a substantial loss. However, swift intervention by local authorities resulted in the recovery of most stolen funds, emphasizing the crucial role of vigilance.

CEO Fraud with Deepfake Audio: Defrauding a U.K. Energy Company

A CEO fraud case featured a unique twist with the use of deepfake audio, an AI-generated voice, defrauding a U.K. energy company of $243,000. Fraudsters mimicked the voice of the German parent company’s CEO using AI. They coerced the U.K. CEO into urgent wire transfers, promising reimbursement. After the money was sent, it was funnelled through multiple locations, complicating detection.

Legal Challenges in Judicial Settings: Deepfake Threat to Evidence Integrity

The current judicial landscape is grappling with the inadequacy of legal frameworks in addressing the rising threat of deepfake technology in evidence manipulation. In the context of the UK’s Civil Evidence Act 1968, which permits the admission of video recordings deemed reliable, the challenge lies in the growing complexity of distinguishing authentic content from deepfakes. The existing measures for detecting deepfakes have proven fallible in certain cases, accentuating the urgent need for legal systems to adapt to this evolving technological landscape.

Geopolitical Impact: Deepfake Incident involving Australian Prime Minister

The repercussions of deepfake technology extend beyond individual impacts to influencing geopolitics and international relations. A notable case involves the Australian Prime Minister, Scott Morrison, who demanded an apology following the posting of a fabricated image on “Twitter” by Zhao Lijian, a spokesperson for China’s foreign ministry. The misleading image depicted an Australian soldier allegedly threatening an Afghan child with a knife, triggering widespread online outrage and fuelling a contentious dispute between the Chinese and Australian governments. This incident not only stoked diplomatic tensions but also highlights the imperative for implementing policies that govern the deceptive use of deepfakes for political purposes on social media platforms.

Entertainment Industry Embracing Deepfake Technology: Marketing Campaigns with Celebrities

Cadbury’s Raksha Bandhan campaign with Hrithik Roshan, utilizing DeepFake video messages, allows customers to customize greetings by scanning QR codes on limited edition chocolate boxes. Following suit, Zomato partners with Rephrase.ai, integrating Hrithik into ads where he names local food outlets and items, creating a surprising and engaging experience for customers.

MIT Virtual Reality Project Recreates Unheard Nixon Speech, Revealing Preparations for Moon Landing Contingencies

A similar project from the MIT Center for Advanced Virtuality recreated a speech that thankfully was never delivered by former President Richard Nixon. The speech was prepared in case the moon landing mission may have failed in 1969. These projects help us learn what our leaders were thinking and how they were preparing for important events like this.

 

The Complexity of Quantifying Losses Related to Deepfakes

Why is the process of quantifying the financial losses related to deepfakes so complex?

In the contemporary landscape, the task of quantifying financial losses attributable to deepfake incidents poses significant challenges. This complexity is rooted in three primary factors that merit comprehensive examination:

1. Growth of the underground economy: Transactions involving buying deepfakes from creators for malicious activity have led to an underground economy and are not formally recorded in the output of the economy.
2. Deepfakes go unrecorded or undetected: Many times deepfakes and related activities go undetected, which makes it impossible to understand the full performance of the technology.
3. Hidden costs: Elevated legal and cybersecurity spending, coupled with impacts on reputation and consumer trust, represent significant hidden costs associated with deepfake incidents.

 

Combating the Rising Tide of Deepfakes

As the capabilities of deepfake technology continue to evolve, the financial sector finds itself increasingly susceptible to the looming threats of fraud and misinformation. To effectively counter these risks, a comprehensive and proactive strategy becomes imperative. Lets delve into the multifaceted approach necessary to safeguard the economy from the potential pitfalls posed by advancing deepfake technologies.

1. Evaluation of Current Regulatory Landscape: While existing laws against fraud provide a basis for legal action, they may not be specific enough to address the unique challenges of deepfake fraud. Enforcement capabilities, including technological resources, are critical for regulations to have a significant impact.
2. Cybersecurity Measures: Integrate anti-deepfake measures into cybersecurity initiatives. Regular security audits, firewall updates, and intrusion detection systems are crucial components of a robust strategy to keep adversaries at bay.
3. Collaboration and Regulation: Encourage collaboration among industry stakeholders such as public private partnerships to develop standardized tools for detecting deepfakes. Regulators should work closely with technology experts to understand nuances, leading to effective and adaptable regulations. 
4. Early Detection Systems: Implement AI-powered early detection systems capable of spotting anomalies in media releases. By scrutinizing blinking patterns and speech cadences, financial institutions can prevent the spread of fraudulent information, averting potential market disruptions.
5. Education Initiatives: Raise awareness about deepfakes to empower individuals and businesses. Workshops, online courses, and informative campaigns can provide the tools needed to critically evaluate media, reducing the risk of deception and fostering a more resilient economic environment.
6. Areas for Regulatory Enhancement: Enhance detection tools through collaboration with tech companies and academic institutions. Promote international cooperation with shared databases and joint task forces to combat deepfake fraud.

 

Conclusion

In conclusion, the economic impact of deepfakes presents a multifaceted landscape that demands careful consideration. While deepfakes offer benefits such as cost savings and heightened efficiency across diverse sectors, they also introduce substantial challenges, encompassing job displacement, ethical dilemmas, and the potential for misuse.

The complexity of quantifying financial losses stemming from deepfake incidents is compounded by factors like the proliferation of underground economies, instances going unnoticed or unrecorded, and concealed expenses linked to deepfake occurrences. Addressing these challenges requires a proactive and comprehensive strategy, integrating early detection systems, robust cybersecurity measures, collaborative partnerships, regulatory enhancements, and educational initiatives.

Through the diligent implementation of these measures, we can effectively mitigate the adverse economic consequences of deepfakes and nurture a more resilient economic landscape poised for sustainable growth and innovation.

 

---

References

1. Deepfakes: Real Threat, KPMG. https://kpmg.com/kpmg-us/content/dam/kpmg/pdf/2023/deepfakes-real-threat.pdf
2. The Economic Potential of Generative AI: The Next Productivity Frontier, McKinsey & Company. https://www.mckinsey.de/~/media/mckinsey/locations/europe%20and%20middle%20east/deutschland/news/presse/2023/2023-06-14%20mgi%20genai%20report%2023/the-economic-potential-of-generative-ai-the-next-productivity-frontier-vf.pdf
3. Countermeasures, Bournemouth University, UK. https://arxiv.org/ftp/arxiv/papers/2012/2012.07989.pdf
4. Deepfakes and Synthetic Media in the Financial System: Assessing Threat Scenarios, Carnegie Endowment for International Peace. https://carnegieendowment.org/2020/07/08/deepfakes-and-synthetic-media-in-financial-system-assessing-threat-scenarios-pub-82237
5. How can we combat the worrying rise in the use of deepfakes in cybercrime? World Economic Forum. https://www.weforum.org/agenda/2023/05/how-can-we-combat-the-worrying-rise-in-deepfake-content/
6. One-Third of Businesses Hit by Voice and Video Deepfake Fraud. https://regulaforensics.com/news/one-third-of-global-businesses-already-hit-by-voice-and-video-deepfake-fraud/
7. Money Scams: Deepfakes, AI Will Drive $10 Trillion in Financial Fraud and Crime. https://www.bloomberg.com/news/articles/2023-08-21/money-scams-deepfakes-ai-will-drive-10-trillion-in-financial-fraud-and-crime
8. The Economics of Deepfake, CSDL: IEEE Computer Society. https://www.computer.org/csdl/magazine/co/2023/08/10206053/1PlcMAFEiNa
9. Deepfakes and Synthetic Media in the Financial System: Assessing Threat Scenarios. https://carnegieendowment.org/2020/07/08/deepfakes-and-synthetic-media-in-financial-system-assessing-threat-scenarios-pub-82237
10. Deepfake Stock Market Scams: How AI is being used to trick investors. https://www.businesstoday.in/technology/news/story/deepfake-stock-market-scams-how-ai-is-being-used-to-trick-investors-406972-2023-11-24  
11. A Voice Deepfake Was Used To Scam A CEO Out Of $243,000. https://www.forbes.com/sites/jessedamiani/2019/09/03/a-voice-deepfake-was-used-to-scam-a-ceo-out-of-243000/?sh=7b07d67b2241