Forum INCYBER 2026: The War on AI-Generated Fakes Is On

“We have truly entered an era in which deception has become industrialised,” says Jérôme Nevicato. The phrase, used by this reserve lieutenant-colonel assigned to COMCYBER-MI, the French Interior Ministry’s Cyber Command, neatly captures the change in scale brought about by the global success of generative AI.

So how can we now “tell what is real from what is fake”, to borrow the title of the panel discussion in which he took part at FIC 2026? Can collective trust still be built in an environment where a voice, an image, a video or even a piece of evidence can be fabricated, altered, amplified or challenged on an industrial scale?

ANSSI, France’s national cybersecurity agency, has also observed a “massification of the threat” in cyberspace. Generative AI does not merely make it possible to produce a convincing fake: it can generate thousands of pieces of content, adapt them, circulate them, test them, correct them and relaunch them.

During the same panel discussion, Adel Mebarki, CIO of Foresight Data Agency, pointed out that before generative AI, troll farms relied on “people paid to create content manually”. With AI, he says, “it costs very little”, is “fairly simple technically” and, above all, makes it possible to create “credible digital identities” — in other words, accounts that are “100% fake” but display online behaviour resembling that of a real person.

The result is “information manipulation on social networks around issues of foreign interference”, but also “a great deal of economic denigration”.

“It is not about using AI to fact-check”

Defenders are therefore no longer simply chasing lies. They are chasing a machine capable of manufacturing plausibility.

Adel Mebarki gives a telling example: audio. A year ago, he explains, it was difficult to inject subtle emotions into an artificial audio track. Today, emotions can be added “to every sentence, every word, making it difficult for defensive AI systems to identify the content as generated”.

Andrew Dudfield, Head of AI at Full Fact, a UK-based fact-checking organisation, and a representative of the European Fact-Checking Standards Network, highlights another challenge: scale.

“The Internet is very big, and that is a problem.”

This is precisely where AI can assist fact-checkers. “It is not about fact-checking with AI — I don’t necessarily think that is a good idea,” he stressed during the panel discussion Securing Facts in the Age of Generative AI. Instead, AI should be used to help fact-checkers “understand what is happening online” and prioritise what requires their attention.

According to Andrew Dudfield, fact-checking increasingly means sorting through a massive volume of content in which not everything is false, AI-generated or even harmful. Full Fact focuses on content capable of causing tangible harm: persuading someone to stop medical treatment, change their vote or abstain, buy a product or decide against purchasing it.

This is a pragmatic approach, but “harm” is not a neutral concept. While it may be relatively straightforward to define in healthcare, it becomes more difficult to handle in politics. Information may influence a vote because it is false — but it may also do so because it reveals an accurate and uncomfortable fact. Fact-checking therefore remains essential, provided it does not seek to neutralise the political consequences of a truth.

“The massification of social engineering”

The technique remains relatively labour-intensive, making the economic asymmetry between producing false content — which is becoming increasingly cheap — and correcting it all the more apparent.

AI can nevertheless help narrow that gap. According to Abhijnan Dasgupta, Practice Director for Trust and Safety at Everest Group, “if we look at the major platforms, on average, 85% of content moderation is now automated”.

Speaking during the panel discussion AI and Human Intelligence: A Fragile Balance for Digital Moderation, he nevertheless stressed that “we really need to distinguish between what is automated and what is not”.

Simple cases can reach “95 to 99%” automation, while medium- or high-complexity cases fall to 40%, or even below 20%. Human moderation, he added, would have cost “$10 billion” worldwide last year.

And this represents only one part of the cost of fake content. As well as being a media phenomenon, fake content is also becoming a cybersecurity tool.

Hugo Mania, AI Project Manager at ANSSI, points out that AI systems themselves are becoming attack surfaces: poisoning training data, extracting sensitive data or intellectual property, and carrying out evasion attacks designed to trigger undesirable model behaviour.

And Forum INCYBER 2026 took place before the release of Claude Mythos…

During the panel discussion Traceability and Authenticity of Generated Content: Telling the Real from the Fake, Hugo Mania also highlighted the “massification of social engineering”. AI makes it possible to identify and research targets more precisely using open-source intelligence and to amplify existing attack techniques.

This is where disinformation and cybersecurity converge. Fake LinkedIn profiles, voice messages, internal memos, videos of executives and fraudulent chatbots can all become intrusion vectors.

“Auditing model privacy”

ANSSI may not have a magic wand — or a magical AI system — but the agency wants, among other things, to be able to “rate” these attacks according to their difficulty, the technical expertise or equipment required and their cost, in order to develop “a framework […] for assessing the trustworthiness of your solutions”.

ANSSI is also working on PANAM (Privacy Auditing of High Models), a library designed to address “a number of extraction attacks in order to audit model privacy”.

Trust therefore has to be built from the lowest layers upwards: data, models, APIs, connectors, agents and non-human identities.

Joel-Oskar Raisanen, Director at DTCP Growth, points specifically to this last issue. “Non-human identities are important and will become even more important in the future.”

If an AI agent requests an action, organisations need to know that the agent is legitimate. “Existing AI systems, data pipelines, logs and the infrastructure surrounding them need to be protected.”

This technical observation foreshadows a profound shift. Tomorrow, the question will no longer simply be “who said this?” but rather: “which agent produced it, with what authorisation, in which system and with what audit trail?”

Within an organisation, trusted content will also increasingly be content tied to a reliable machine identity.

Then comes the temptation to rely on technology: mark, trace and detect.

Anthony Level, co-founder of Label4AI, defines watermarking as “an identifier deeply embedded in the content, which can only be extracted if you have access to a secret algorithm”.

This could help address the requirements of the AI Act, whose Article 50 establishes transparency obligations for certain AI systems and provides for the marking of synthetic content. Yet he does not present digital watermarking as a silver bullet.

Should every piece of content be examined like a crime scene?

Can a watermark be secure — meaning that “it cannot be removed or imitated”, otherwise it may become potentially dangerous — and robust, meaning that it “withstands compression, screenshots, cropping or resizing”, without becoming potentially useless?

There is no way to guarantee this 100%.

Jérôme Nevicato, for his part, points out that C2PA, which aims to certify the source and history of digital media, works well “in theory”. In practice, however, metadata can disappear when content is uploaded to a social network or compressed.

His conclusion is pragmatic: combine watermarking and C2PA, while remembering that “watermarking can be heavily attacked”.

The Achilles’ heel remains open source and locally deployed models.

Adel Mebarki estimates that “the majority of deepfake videos used in information operations come from open-source models”. In such cases, watermarking mechanisms can be circumvented “by design”, because actors retrain their own models and establish “their own rules, which drastically complicates traceability”.

Regulation can discipline visible actors. It has far less leverage over a clandestine actor using a modified local model.

Hence the importance of forensics, which Anthony Level describes as an investigation into the content itself, searching for traces, weak signals or signatures left by diffusion models.

Yet every piece of content cannot be examined as though it were a crime scene.

During the panel discussion on digital moderation, Lucile Bak, Chief Trust and Safety Officer at Dailymotion, brought the debate back to operational reality: “AI is now essential. Without it, moderation at scale is impossible.”

“A coin toss is more likely to identify a deepfake than the human eye”

Digital fingerprints and hash databases are effective for copyright infringement or terrorist content, Lucile Bak explains, but they work on media that has already been identified.

For new content, tools can detect certain signals but remain weak at understanding “the context and intent behind the content”, resulting in both false positives and false negatives.

“The difficulty is not in creating the tools, but in knowing when to trust them,” she says.

At Dailymotion, AI performs the initial sorting while humans make decisions on difficult cases, “because they understand the context, can identify intent and are capable of understanding cultural differences”.

“Is it satire or an insult? That matters. Human beings therefore have an important role to play,” confirms Akash Pugalia, Chief Digital Officer at TP.

Yet the human element is also the most fragile, because generative AI targets our instinctive tendency to trust what we see and hear.

Jérôme Nevicato says that “when faced with a high-quality deepfake video, the probability of identifying it as fake is around 25%”, meaning that “a coin toss is more likely to identify a deepfake than the human eye”.

This cognitive vulnerability applies to all forms of content.

One reason is the erosion of common reference points for determining what is true. “With dictionaries and encyclopaedias, we had reliable knowledge bases,” says Emilia Tantar, Director of AI and Head of Luxembourg’s Cybersecurity Factory.

That is no longer necessarily the case in a world where everyone can construct their own version of “truth” in isolation.

The liar’s dividend

Faced with this growing gap, Luxembourg’s House of Cybersecurity is attempting to create “an open cybersecurity database, a common space that tracks best practices”.

This collaborative approach makes it possible “to have at least three different reference points to check whether our perspective is valid in a broader context”.

Such a cybersecurity approach could usefully be transposed into the political sphere, where AI can profoundly distort the information environment, particularly through what is known as the liar’s dividend.

“Something that actually happened can be dismissed by the person involved as AI-generated, because we can no longer distinguish what is real from what is fake,” explains Jérôme Nevicato.

Fake content therefore does not merely make people believe that an invented event took place. It can also make them doubt that a real event ever happened.

Faced with this flood of false content, which is profoundly disrupting our reference points and weakening our infrastructures, there is no single answer.

Secure content when possible, provide context when appropriate, remove it when it is illegal, and preserve evidence when legal proceedings are involved.

“Trust cannot be taken for granted. It requires evidence: evidence of credibility, accountability and performance,” summarises Ysens de France, co-director of the French Gendarmerie Nationale’s AI Centre of Expertise.

With AI, Brandolini’s law is more relevant than ever: when producing convincing falsehoods becomes almost free and instantaneous, establishing proof still costs time and money.

That is the price of trust in a world where images, voices and text can no longer serve as proof on their own.