An awareness program adds a deepfake module and fills it with visual tells, such as hands with extra fingers, irregular blinking, or lips that never quite match the audio. Six months later the generator that produced those errors is obsolete and the module is unchanged. In that same half year two queries reach the security team. In one, someone approved a transfer after a video call with what appeared to be the CFO. In the other, someone refused to carry out a legitimate instruction because the audio sounded odd and they suspected a cloned voice.<\/p>\n
The first case is logged as an incident, the second as operational friction, if it gets logged at all. They come from the same place. The program placed the decision criterion in each person\u2019s perception. Training that perception did not make it more accurate, and at the same time it left the whole organisation more alert to the fact that images can lie. The second case comes out of that second effect.<\/p>\n
The two modes are worth separating before touching the content of the module. One is credulity towards audiovisual material, which lets a false request through because it comes with a face and a voice. The other is a distrust that spreads beyond its target and ends up reaching authentic evidence too. They share the symptom, because in both cases the decision is made with a criterion that cannot carry it, and the fix for one aggravates the other.<\/p>\n
Because the modality changes the status of what reaches us. Text arrives as a claim someone is making, and we can doubt it at no cost. An image or a voice arrives as a record of something that happened, and so it enters the decision as evidence rather than as a claim.<\/p>\n
That status held while fabricating a false record was expensive. In 2022, Sophie Nightingale and Hany Farid published a study in PNAS<\/a> measuring how much of that guarantee is left. Participants were correct 48.2% of the time when telling AI-synthesised faces from photographs of real people, a result indistinguishable from chance. In a second experiment, with prior training and feedback after each answer, accuracy rose to 59.0% and stayed there, without improving across the series.<\/p>\n A third experiment added a more uncomfortable result. When participants were asked to rate how trustworthy each face looked on a seven-point scale, the synthetic faces averaged 4.82 and the real ones 4.48. Faces of people who do not exist came out 7.7% more trustworthy than faces of people who do.<\/p>\n That way of deciding is known as the seeing-is-believing<\/em> heuristic. We take audiovisual evidence as truthful because of its form, before assessing its content.<\/p>\n The available evidence says it cannot be done reliably, and that the two levers an awareness program usually pulls do not move the result.<\/p>\n In November 2021, Nils K\u00f6bis, Barbora Dole\u017ealov\u00e1 and Ivan Soraperra published a preregistered experiment with 210 participants in iScience<\/a>. Each person watched sixteen short videos, half authentic and half manipulated, and flagged the ones they thought were fake. One group was given a warning about the harm deepfakes cause beforehand, another a financial incentive for accuracy, and neither condition improved accuracy compared with the control group.<\/p>\n Two further findings from the same work matter for program design. The error is not symmetrical, because people tend to take a manipulated video for an authentic one rather than the reverse. And people overestimate their own ability to detect a deepfake, with a confidence their accuracy does not support. The name seeing-is-believing<\/em> comes from that work.<\/p>\n A module built on visual tells therefore works against two limits. Accuracy barely moves, with neither motivation nor warning. Confidence does move, and someone who becomes convinced they know how to look verifies less than someone who doubts. What the module delivers is that conviction.<\/p>\n On top of that, the tell expires. The list of artefacts that gives a generator away today describes the errors of one software version, not a stable property of synthetic images. When the generator is updated those errors stop being produced, and they disappear without anyone having got better at spotting them. Each iteration renders the list useless.<\/p>\n When we argued that with deepfakes the teaching method matters more than the content<\/a>, the argument reached that same conclusion from training practice. The experimental evidence reaches the same place from perception.<\/p>\n The second failure mode appears, and it grows precisely when the campaign meets its objective. An organisation that has learned any record can be fabricated also gains a fresh reason to dismiss the records it would rather not accept.<\/p>\n Robert Chesney and Danielle Citron described this in 2019, in volume 107 of the California Law Review<\/a>, and called it the liar\u2019s dividend. That dividend flows, perversely, in proportion to success in educating the public about the dangers of deep fakes. The better an audience understands that video can be fabricated, the easier it becomes for someone to deny an authentic one.<\/p>\n Inside a company the phenomenon arrives through operations long before it arrives through public debate. A legitimate instruction goes unexecuted because the person receiving it distrusted the voice. A meeting recording loses its value as a record of what was agreed. And the security channel starts receiving reports that carry no attack, distrust with no object rather than an attack, and those reports consume the same analysis time.<\/p>\n The cost falls on the areas that need to believe someone in order to move, such as HR, finance, support and procurement. When the criterion for believing becomes each person\u2019s perceptual conviction, work gets slower without getting safer.<\/p>\n Neither mode is corrected by the intervention meant for the other. Whoever decides too readily has an excess of confidence in their perception. Whoever decides too rarely lacks a criterion that does not depend on it. Insisting that anything can be faked relieves the first picture and feeds the second.<\/p>\n By what the person does with the request, rather than by what they believe about the video.<\/p>\n The two failures separate cleanly by looking at two things: which channel brought the request, and what action followed from it. Three signals are enough to place a program.<\/p>\n The distinction resembles the one we use to separate error, negligence and intent<\/a>. A single label leaves a program applying the same correction to two pictures that do not match. As with security fatigue and habituation<\/a>, what blurs together on a dashboard comes apart when you observe behaviour. Which is why it is worth reviewing what the program is measuring<\/a> before adding a module to it.<\/p>\n The decision criterion moves from perception to procedure. Nobody can answer accurately whether the video in front of them is authentic. What can be answered is whether the channel that request arrived through authorises the action being asked, and that answer is verifiable.<\/p>\n The first translation is a short list of actions that are never authorised in the same channel where they are requested. Transfers, changes to a supplier\u2019s bank details, access provisioning and sending sensitive information all belong on that list. The rule is defined in advance and by type of action, independently of how convincing the request was, which takes the decision out of the territory where perception fails. Every organisation has its own version of that list, which is why it is worth teaching it through content tailored to each organisation<\/a> rather than through a generic procedure.<\/p>\n The second is to simulate the channel instead of the artefact. A simulation that trains the eye teaches people to recognise one version of the technology. One that reproduces the whole sequence, with an urgent request arriving through an unusual route and proposing to skip a step, trains behaviour that still works once the generator improves. SMARTFENSE simulation tools<\/a> operate at that level, the level of the decision process, which is why what they measure ages far more slowly than the attacker\u2019s software. It is the same criterion by which we already placed persuasion levers above the channel<\/a>.<\/p>\nCan a person be trained to detect a deepfake?<\/h2>\n
What happens when the warning works too well?<\/h2>\n
How do you tell one mode from the other in a program?<\/h2>\n
\n
What changes in the design of the program<\/h2>\n