By Austin VornhagenThe back of that mask looks like a face, too.
Look again at the last frame of the film. You’re staring into the inside of a hollow shell, and your brain insists a face is bulging out at you. Psychologists call it the hollow-face illusion. Your visual system has seen so many faces that it would rather flip the depth of the world than accept a face that points the wrong way.
Hold on to that. It’s the whole problem in one picture: we are built to recognize faces and voices, and we are not built to check where they came from.
In January 2024, a finance employee in the Hong Kong office of the engineering firm Arup joined a video call with the company’s chief financial officer and several colleagues. He recognized their faces. He recognized their voices. At their direction, he made 15 transfers totaling HK$200 million, about $25 million. Every other person on that call was a fake.
He didn’t skip the security check. He did the security check. He looked at the faces and listened to the voices, which is the check most of us have run on every phone call of our lives.
Hearing someone tells you what a thing sounds like. It doesn’t tell you who made it.
The obvious fix is a better deepfake detector: an AI that looks at the pixels and says “fake.” This essay argues that fix is a trap. The real one is stranger and much stronger: build systems where a perfect fake, indistinguishable from the real person, still fails.
You’ve been using your face as a password
A password works because of one property: you know it and the attacker doesn’t. A house key works because you have it and the burglar doesn’t. Security, at bottom, is a secret or a thing that only the right person holds.
Now apply that test to your voice. You don’t hide it. You broadcast it. Every voicemail greeting, every video you’ve posted, every podcast guest spot, every Zoom call someone recorded is a sample. Your face is the same, only more so.
So a voice has two properties that make it a terrible credential, and they get worse together:
You hand out copies all day. A password you shouted in every room you entered wouldn’t be much of a password.
If someone steals your password, you change it. If someone learns to synthesize your voice, you can’t get a new voice.
For most of history this didn’t matter, because copying a face or a voice convincingly was hard. We quietly merged two different ideas into one:
“That sounds like Mom.”
“Mom produced that sound.”
Those two used to travel together so reliably that nobody bothered to tell them apart. Generative AI is splitting them. And the interesting question isn’t when the split happened. It’s how much it cost to make it happen, at every step along the way.
The price of a puppet: what it took to fake a person, era by era
Identity meant presence. You knew someone because they were in front of you: their face, their voice, their handwriting, maybe a seal. None of it was cryptographically strong. It worked because copying a whole person was nearly impossible.
The phone made us decide things about people we could hear but not see. Banks, doctors, employers, and families slowly built habits around one assumption: a familiar voice plus a few personal details is good enough. Faking it took real talent, so the habit held.
Record someone saying their name, play it into a phone. Security researchers call this a presentation attack. It was manageable, because a recording is rigid. Ask it an unexpected question and it has nothing to say.
Computer speech is decades old, and for most of that time it sounded like a computer. Meanwhile, starting in 1996, NIST began running formal evaluations of how well machines could recognize speakers. The field split into two questions: is this Austin’s voice, and the nastier one, is Austin producing it right now?
Machine learning turned text-to-speech into speaker cloning: arbitrary new sentences, in your voice. A replay is a prerecorded puppet. A clone is an interactive one. In March 2023 the FTC warned consumers that a scammer needs only a short audio clip, like one posted online, and a voice-cloning program to fake a relative in trouble.
Video followed the same curve, all the way to the Arup call. In its 2025 report, the FBI’s Internet Crime Complaint Center broke out AI-related complaints for the first time: 22,364 of them, with more than $893 million in reported losses. That counts every scam where victims noticed AI, not just cloned voices, and it only counts what people recognized and reported.
AI didn’t invent this hole. It destroyed the economics that made the hole tolerable.
The line from “be you” to “a short clip” is the real story. The check never changed. We kept asking the voice to vouch for the person. What collapsed was the price of a voice that could lie.
So build a better detector?
It’s the instinct almost everyone has. If AI can make fakes, train AI to catch them. Feed a model thousands of real clips and thousands of generated ones, and let it learn the tells: odd phase patterns in the audio, a vocoder’s fingerprints, lighting that doesn’t quite agree with a face, a blink that comes on schedule.
That research is real and useful. It is also fighting uphill in a way that’s easy to miss, so let’s draw it.
How a deepfake detector learns, and why new generators slip past it
The detector draws a line between what it has seen.
Every dot is a clip. Real speech on the left, clips from four known generators, A through D, on the right. The detector finds a line between them, and on clips like these it can look superb.
Then generator E ships. It uses a technique the detector never saw, and its clips land somewhere new, on the real side of the line. The detector didn’t learn “what fake is.” It learned “what A through D look like.”
So you collect E’s clips and retrain. The line bends. You’re safe again, until the next release.
Generator F is built with detectors in mind, tuned until its output sits right beside real speech. Researchers take this so seriously that ASVspoof 5, a major benchmark for spoofed speech, uses 32 attack algorithms, adds adversarial attacks for the first time, and keeps the attacks used for testing separate from the ones used for training. The whole point is to test what a detector does with an attack it has never met.
And a smart attacker doesn’t hold a fake up to your camera at all. They inject generated video into the software pipeline, between the camera and the app. NIST’s identity guidelines, revised in July 2025, now say plainly that all types of remote identity proofing are “in some way vulnerable” to these attacks.
That’s an arms race with no finish line: better generator, better detector, a generator built to beat the detector, and around again. There’s no sign anyone has found a universal fingerprint of fake speech.
The standards people have noticed. Here is one sentence from NIST SP 800-63B-4, the federal government’s authentication rulebook, finalized in 2025. In standards language, “SHALL NOT” is as strong as it gets:
“Biometric comparison based on voice SHALL NOT be used.”
NIST SP 800-63B-4, section on biometric requirements
The same document says biometrics may only be used as part of multi-factor authentication with a physical authenticator, something you have. The face can help. It can’t carry the load alone.
Stop asking “is it fake?” Start asking “can it sign?”
Here’s the move that turns this from a losing race into a solvable problem. A detector asks: can I prove this is fake? The alternative asks: can the other side prove where this came from?
You already live with the second approach and never think about it. When you visit your bank’s website, your browser doesn’t squint at the page and decide it looks like your bank. It checks a cryptographic proof that the server controls the bank’s domain. A pixel-perfect clone of the site on a different domain fails, no matter how good it looks.
The same trick can work for people. It rests on one piece of math you can use without understanding: a key pair.
Your bank keeps a copy on file. It can check signatures but can’t make them. Stealing it gets an attacker nothing.
It lives in secure hardware on your phone, built so the key can be used but not copied out. It’s the only thing that can make your signature.
Your face and your voice are copyable. A private key sealed inside a chip isn’t something you broadcast on Instagram. That’s the whole trade: stop authenticating information that can be copied, and start authenticating the ability to do something that can’t be.
Let’s watch it work, and then watch it get attacked.
The handshake: how a signed challenge beats replay, relay, and a perfect deepfake
Your phone makes a key pair. The public half goes to your bank and gets filed under your account. The private half never leaves the chip. This is, roughly, what happens when you create a passkey.
Later, you want in. The bank doesn’t ask what your dog’s name is. It sends a fresh random number, one it has never sent before and will never accept again.
Your phone checks your face or PIN, on the phone. This is where biometrics belong. The face doesn’t go to the bank as proof that you’re you. It only tells the chip, “okay, you may use the key.”
The chip signs the challenge together with the name of who’s asking, bank.example. The bank checks the signature with the public key it has on file. It passes. Nothing about how you look or sound was involved.
An attacker recorded yesterday’s successful login and plays it back. It fails instantly. Yesterday’s signature answered yesterday’s random number. NIST calls this property replay resistance.
Sneakier: a fake bank calls you, gets a live challenge from the real bank, and passes it to you to sign. But your phone stamps in who it’s actually talking to. The signature says fakebank.example, and the real bank throws it out. NIST calls this verifier name binding; passkeys do a version of it by tying each credential to one website.
Now the nightmare: a real-time clone with your face, your voice, your memories, zero lag. It can answer any question about your childhood. It still can’t produce a signature, because the only thing that can is sealed in a chip in your pocket. Perfect you is not authenticated you.
The signature that breaks if you change one digit
Authentication proves who’s there. It doesn’t prove what they agreed to. If an attacker hijacks a session after you log in, “send $5,000” can quietly become “send $50,000.” So the strongest systems sign the action itself: the amount, the accounts, the time, all in one signed message.
Below is a real one. Your browser just made a key pair with the same kind of math many passkeys use (ECDSA on the P-256 curve), and it created the private key as non-extractable, so even this page’s own code can’t read it out. Sign the transfer, then play attacker.
Nothing signed yet.You can also edit the fields by hand after signing. Every check runs in your browser; nothing is sent anywhere.
Try it a few ways. Change a comma. Add a space to the account name. The signature was computed over the exact bytes you approved, so any change, however small, makes it fail to verify. The attacker can see the public key, the message, and the signature. None of it helps them sign a new message.
This is what shifts the whole problem. A deepfake can put words in your mouth. It can’t put a signature on your intent.
The six-layer trust stack
Stack the pieces up and you get an architecture where each layer answers one question, and the thing AI is getting frighteningly good at lives in exactly one of them. Tap a layer to see what it proves. Then send in the best deepfake ever made.
Did they approve exactly this action?
The amount, the account, and the time are signed together. Change one digit and the signature breaks.
Holds. Edits after approval are detected.
What am I seeing and hearing?
Voice, video, text, an AI assistant. The part that feels like the person.
The only layer a deepfake can reach.
Is this proof for this conversation, with this party, right now?
A new random challenge each time, signed together with the name of who's asking.
Holds. Replays and relays fail here.
Where does the private key live?
In a secure chip that is built not to hand the key out. Your face or PIN unlocks it, locally.
Holds, unless the phone itself is stolen and unlocked.
Which key speaks for this account?
Your public key is registered to your account. Anyone can hold a copy of it; that's the point.
Holds. Knowing the public key doesn't let you sign.
Who is this human, originally?
Documents, records, and sometimes a person checking you in the flesh, before any key is issued.
Hardest layer. A fake that fools enrollment gets a real key.
The fake starts at layer 5, the only place it can reach.
Make the voice infinitely good. Make the video flawless. Give the model superhuman charm and every memory you have. None of those improvements lift it out of layer 5. It can talk to you all day. It can’t sign for you.
The face becomes the interface. The key becomes the identity.
Your voice doesn’t stop mattering. It goes back to the job it was always good at, which is talking, and stops doing a job it was never qualified for.
What about the clip of you that goes viral?
Everything so far assumes a live conversation, where you can send a challenge and demand a signature. A video posted online can’t be challenged. It just sits there, looking like you.
The same flip still works: prove where it came from instead of proving it’s fake. That’s the idea behind C2PA Content Credentials, an open standard in which a camera, editing app, or publisher can attach signed records of what it did. Captured by this device. Cropped in this app. Published by this outlet. Each step signs the next. Version 2.4 of the specification, released in April 2026, extends the approach to live video streams.
- CaptureCamera signs the original
- EditEditor signs the crop and color
- PublishOutlet signs what it posted
- YouCheck the chain, not the pixels
The honest caveat: provenance proves where something came from if it carries credentials. Plenty of real footage never will, so a missing label doesn’t prove a fake. It’s a way to trust some media more, not a verdict on everything else.
A cousin of this idea protects the camera itself. NIST now says identity services should authenticate capture sensors or use device attestation during remote identity proofing, so the verifier can tell the frames actually came through a trusted camera and weren’t injected by software.
Cryptography faithfully protects whoever it was handed to
Look back at the stack. Layer 1 is marked unsolved, and it’s the load-bearing one.
Every signature traces back to a moment when someone decided, “this key belongs to this human.” If a criminal fakes your face, your ID, and your voice well enough to fool that moment, the bank will issue their device a perfectly valid credential in your name. From then on, the math works flawlessly. For them.
The same trap hides in recovery. Picture your phone at the bottom of a lake. You call and say, “I lost my phone, please trust this new one.” If the bank’s answer is a customer service rep who listens to your voice, the whole stack has a side door with the old lock on it.
Your security is only as strong as the way you replace a lost key.
That’s why the hardest remaining work isn’t in detection or signatures. It’s in enrollment and recovery: slowing down high-risk changes, notifying every device you already own, requiring a second trusted device or a hardware recovery key, and sometimes, for the highest stakes, asking you to show up in person. Detection and active challenges still earn their place here, as extra layers, not the foundation.
Active challenges are an especially clever layer. Instead of passively hunting for artifacts, the system asks the caller to do something unpredictable, live: turn your head, cover one eye, read a random number. A September 2026 paper calls its version Deep-Fake CAPTCHA and reports that this kind of challenge substantially beats passive detection on real-time voice and video fakes. It’s a strong speed bump. It’s also exactly the kind of bump that faster generators will keep working to flatten.
What to do this week
Your phone won’t show “identity verified” for your sister’s calls anytime soon. Until it does, you can run the same principles by hand.
- Origin check
Hang up and call back
If a voice you know asks for money or access, end the call and dial the number you already have for them. The FTC gives exactly this advice. You’re swapping “does it sound right” for “did it come from the right place,” which is the whole essay in one move.
- A secret you never broadcast
Agree on a family code word
The FBI recommends a secret word or phrase with your family. It’s a shared secret, the low-tech cousin of a private key, and it works for the same reason: unlike your voice, you have never posted it anywhere.
- Keys, not codes
Turn on passkeys where you can
A passkey is a key pair tied to one website, with the private half on your device. It’s the handshake from lesson 3, already in your pocket, and it doesn’t care how convincing the phishing page looks.
- Read the request, not the voice
Treat urgency as the tell
Scams pair a trusted likeness with pressure and an odd way to pay: wire transfers, cryptocurrency, gift cards. When a familiar voice demands speed and secrecy, the familiarity is the part to distrust.
Likeness was never proof. It was just expensive to fake.
Line up everything we’ve trusted to prove who someone is: a face, a voice, a handwritten signature, a photo of a driver’s license, a mother’s maiden name, a writing style. Every one of them is information. Information can be copied, and AI is making the copying cheap.
The systems that survive this will have one thing in common. They’ll stop asking “do you look and sound right?” and start asking “can you do the one thing only the right party can do, fresh, for this exact request, right now?”
That reframes the fight. We don’t need to win a permanent race between generators and detectors, which may be unwinnable. We need to make possession of the right key necessary for anything that matters, and then do the hard, unglamorous work of issuing and replacing keys carefully.
Remember the mask. From the front, a flawless face. From behind, still a face, because your brain can’t help it. Walk all the way around and there’s nobody inside.
The next time you think “I know that voice,” ask what it can prove.
This essay grew out of a long conversation with an AI that started with a Reddit comment about voice cloning, then got checked against primary sources. The NIST quotes are from SP 800-63B-4 and SP 800-63A-4 (finalized July 2025). The FTC’s voice-cloning alert is from March 2023; the FBI’s code-word advice is from its December 2024 public service announcement; the AI complaint figures are from the FBI’s 2025 Internet Crime Report. The Arup case is as reported by CNN and the South China Morning Post after Arup confirmed it in May 2024. The benchmark details are from the ASVspoof 5 paper, and the challenge-response results from the Deep-Fake CAPTCHA paper. The detector map, the effort meter, and the transfer are illustrations, though the signature in the transfer demo is real cryptography running in your browser.
If the stack made sense to you
Six Invisible Walls Stand Between You and a Plane Crash
The same defense-in-depth idea in a different world: why aviation stacks imperfect layers, and what happens when several crack at once.
Read the essay →Essay · AIAI Needs Nine Dominoes to Fall
Another fear about AI, broken into separate links so you can see which ones we actually hold.
Read the essay →Primary source · NISTSP 800-63B-4, the authentication rulebook
The federal guidance that bans voice biometrics, requires phishing resistance at the highest level, and explains replay resistance and name binding.
Read the guidelines →Primary source · FBIHow criminals use generative AI
The FBI’s public service announcement on AI-assisted fraud, with the family code-word advice.
Read the PSA →
Your customers can’t see your face either.
When someone finds a service business online, the website is the handshake: a real domain, a secure connection, and a booking and payment flow that works like it came from someone who’s actually there. I build those websites at Content Pilots. And if you think I got a layer of the stack wrong, tell me. I’ll fix it.