technical deep-dive: how Chat Control scanning actually works

this is the nerdy version. if you want the "what should i do" version, go to the protection guide. if you want to understand the actual technical mechanisms — how scanning works, why it's broken, and why security researchers are furious — keep reading.

the three detection methods

Chat Control mandates that platforms implement detection systems. there are three main approaches, each with different technical implications.

1. perceptual hash matching

how it works:

perceptual hashing creates a "fingerprint" of an image that's resistant to minor modifications (resizing, cropping, compression). unlike cryptographic hashes (SHA-256, etc.), perceptual hashes produce similar values for similar images.

the most well-known system is PhotoDNA, developed by Microsoft and donated to NCMEC. here's the process:

an image is uploaded to a platform
the platform computes a perceptual hash of the image
the hash is compared against a database of known CSAM hashes
if the hash matches (within a threshold), the image is flagged
the flag is sent to NCMEC (in the US) or the EU Centre (proposed)

the technical problems:

false positives. perceptual hashing isn't perfect. similar-looking but innocent images can produce similar hashes. a photo of a child at a beach could match a hash of CSAM if the perceptual features are similar enough. the threshold for matching is a trade-off: too strict and you miss real CSAM; too loose and you flag innocent images.

database opacity. the hash database is not public. nobody outside NCMEC and law enforcement knows exactly what's in it. researchers have shown that perceptual hash databases can be poisoned — adding hashes of non-CSAM content to flag innocent images.

manipulation attacks. in 2021, researchers at Stanford demonstrated that it's possible to modify an image to change its perceptual hash while keeping it visually identical. this means:

CSAM can be modified to avoid detection
innocent images can be modified to trigger false positives

version: Microsoft PhotoDNA

creates a 144-byte hash
resistant to cropping, resizing, compression
used by most major platforms
not open source (proprietary algorithm)

version: Apple NeuralHash

apple's own perceptual hashing algorithm
was going to be used for client-side scanning in 2021
reversed-engineered by researchers within weeks
collision attacks demonstrated (different images producing the same hash)
apple paused the program after backlash

2. AI behavioral analysis

how it works:

machine learning models analyze text conversations looking for patterns associated with "grooming" or "solicitation." the models are trained on datasets of conversations labeled as grooming or non-grooming.

the technical pipeline:

a message is sent
the message (and surrounding context) is fed to an ML model
the model outputs a probability score: "how likely is this conversation to involve grooming?"
if the score exceeds a threshold, the conversation is flagged
a human reviewer (or another AI) evaluates the flag

the technical problems:

false positive rate. this is the killer. the best-performing models in peer-reviewed research have false positive rates between 50% and 95%. a 2025 study by TU Munich tested the models proposed for Chat Control and found:

94.3% false positive rate on normal adult conversations
87.1% false positive rate on parent-child conversations
76.4% false positive rate on teacher-student conversations

that means if 100 conversations are flagged, only 6-24 actually involve grooming. the rest are innocent people getting flagged by an algorithm.

context blindness. AI models struggle with context. consider these scenarios:

a teacher discussing a student's behavior with a parent
a doctor discussing puberty with a patient
a novelist writing dialogue for a young character
a researcher studying grooming patterns
a teenager asking an AI about relationship advice

all of these could be flagged as "grooming" by an AI that doesn't understand context.

adversarial evasion. real groomers can evade AI detection by:

using coded language
switching to image-based communication
communicating through platforms that don't implement scanning
using steganography (hiding messages in images)

the AI behavioral analysis catches the stupid criminals and flags the innocent. the sophisticated ones evade it entirely.

bias. ML models inherit biases from their training data. if the training data overrepresents certain demographics (which it always does), those demographics get flagged more often. research has shown that these systems disproportionately flag conversations involving:

non-native english speakers
neurodiverse individuals
LGBTQ+ youth
minority ethnic groups

3. client-side scanning

how it works:

this is the most dangerous approach. the scanning happens on your device, before encryption.

you take a photo or write a message
before the message is encrypted and sent, your device scans it
the scan compares the content against a database (for images) or an AI model (for text)
if the content matches, a "safety voucher" is created — an encrypted flag that's sent along with the message
the platform can decrypt the safety voucher if the match confidence exceeds a threshold
the platform reports the match to authorities

why this breaks encryption:

client-side scanning doesn't technically break E2EE. the message is still encrypted in transit. but it makes E2EE irrelevant because the scanning happens before encryption. the surveillance is in the "plaintext moment" — on your screen, in your gallery, in your message draft.

as security researcher Matthew Green put it: "it's like having a security guard inside your house who reports everything he sees to the police, and then claiming your house is private because the walls are solid."

the backdoor problem:

the client-side scanning database is the backdoor. whoever controls the database controls what your device flags. in theory, it contains only CSAM hashes. in practice:

a government could add hashes of political protest photos
a government could add hashes of journalist sources
a government could add hashes of any content it wants to suppress
the database updates are opaque — you can't see what's being added

researchers at the University of Cambridge demonstrated in 2023 that the database can be manipulated to flag arbitrary content without the user knowing.

the infrastructure:

client-side scanning requires:

a database of "flagged" content stored on every device
a mechanism for updating the database (which is an attack vector)
a reporting system from device to platform to authorities
a threshold system for deciding when to report

each of these components is a potential vulnerability. the database can be poisoned. the update mechanism can be hijacked. the reporting system can be intercepted. the threshold can be adjusted by whoever controls it.

why security researchers oppose this

the consensus among security researchers is overwhelming. over 400 signed an open letter opposing client-side scanning. the arguments:

1. no backdoor is safe. any system that allows scanning can be expanded. there's no technical mechanism that limits scanning to CSAM only. the same infrastructure that detects CSAM can detect political speech, journalist sources, or anything else.

2. false positives are inevitable. at the scale of billions of messages per day, even a 1% false positive rate means millions of innocent people flagged daily.

3. it doesn't work. sophisticated criminals will evade client-side scanning using encryption tools that don't implement it, steganography, or simply using platforms outside the EU.

4. it creates a surveillance infrastructure. once the scanning infrastructure exists, it will be expanded. this is not a technical argument — it's a historical one. every surveillance tool ever created has been expanded beyond its original purpose.

the alternatives that actually work

if the goal is to protect children (which is what they claim), there are better approaches:

1. targeted surveillance. with a warrant, based on specific suspicion. this is how law enforcement has always worked and it's effective.

2. platform design. make it harder for adults to contact children on platforms. age verification, restricted DMs, parental controls. this addresses the problem without mass surveillance.

3. education. teach children about online safety. this is more effective than any algorithm.

4. hash sharing between platforms. platforms can share hashes of known CSAM without scanning all messages. this catches the worst offenders without mass surveillance.