Magnus Brunner, European Commissioner for Internal Affairs and Migration, at the trilogue on the Child Sexual Abuse Regulation in Brussels, 16 April 2026. Photo: European Parliament, © European Union 2026, Source: EP, Alain Rolland (cropped).
On 3 April 2026 the EU law that allowed companies to scan private messages for child abuse material ran out. Google, Meta, Microsoft and Snap said they would keep scanning, and Germany’s federal police kept receiving fifteen to twenty thousand reports a month. Whatever decides whether your messages are scanned, it is not the law.
This is the seventh piece in our Who Decides? series, and it is about the room the law never enters: the company.
Chat Control 1.0 is Regulation (EU) 2026/1881. It orders nobody to scan anything. It says that providers of email and messaging services may process your communications to detect child sexual abuse, “provided that” they meet a list of conditions. May, not must. In the previous part the Council’s own lawyers said the state may not order the scanning of everyone, and the Council answered by making the scanning voluntary. So the next question is: if it is voluntary, who exercises the choice, and what do they owe you?
The claim
The companies’ claim is simple: we scan voluntarily, to protect children, and the law merely allows it.
I am not taking child protection away from them. It would be cheap to say it is just a pretext. What bothers me is the word “voluntary”, because it is voluntary mainly from the company’s point of view. As a user I mostly cannot choose whether my communication passes through that system or not.
This spring tested the rest of the sentence. The exemption expired on 3 April after Parliament refused to extend it, and Germany’s Federal Criminal Police Office, the BKA, had warned of a “drastic decline” in reports. It did not come. According to the Interior Ministry’s written answer to MP Donata Vogtschmidt of 22 June 2026, the BKA received between 14,351 and 21,472 reports a month from the American clearing house NCMEC from January to May, the lowest figure in May. The BKA told netzpolitik.org that “a concrete connection with the lapse of the interim regulation cannot so far be established”. Google, Meta, Microsoft and Snap, netzpolitik.org reported, had publicly announced that they would carry on, citing other legal bases such as the Digital Services Act and their own terms of service. NCMEC itself notes that US law obliges companies to report abuse material once they become aware of it, “but there are no legal requirements for proactive efforts to detect this content”. American law commands the reporting. European law permits the searching. A company decides to search.
If a company scans even when the state does not force it to, then we have to stop asking only what the law allows and start asking what power we have in fact left to private platforms. A good intention can very easily turn into infrastructure that then runs by itself.
Who decides?
Follow one message through the machine, using the companies’ own descriptions to the European Commission.
Someone decides whether to scan at all, and with what: hash matching for known material, classifiers that guess whether a never-seen picture is abuse, text tools that score conversations for grooming. Each is a dial. The Commission puts it plainly: there is a trade-off between false positives and false negatives, and “the provider can tailor the accuracy settings to choose the appropriate balance”. Someone decides whether a flag is a hit; the Regulation requires human confirmation only for material “not previously identified” as abuse, so a match against a known hash can go out without a person looking. Someone decides what happens to you: the Regulation lets the provider keep your data for “blocking the account of, or suspending or terminating the provision of the service to, the user concerned”. Then the report leaves. All five providers that reported to the Commission, Google, LinkedIn, Meta, Microsoft and Yubo, send their reports to NCMEC, the National Center for Missing and Exploited Children, a private organisation in the United States, which in 2025 received 21.3 million reports, 77 percent of them about uploads from outside the US, and forwarded them to police in 170 countries, Europol and Interpol included. Five companies produced more than three quarters of those reports.
So the honest answer to “who decides” is a policy team, an engineer with a threshold, a reviewer in a queue, a private American organisation, and finally a police officer in your country who receives a file he did not ask for. Nobody in that chain needs a judge, a minister, or you.
I would not say that the state or a company is automatically better at this. I do not trust the state without oversight just because it is the state, and I do not trust a company just because it is private. What matters more to me is who is responsible for the decision, and whether I can realistically defend myself against it. What frightens me on private platforms is the shapelessness. A model may decide about my account, then some internal process, maybe a person on the other side of the world, and I do not actually know who said “this is a problem”. What faces me, more often than not, is a form.
I have no experience of being falsely flagged by such a scan, and I will not pretend that I do. But almost everyone has lived through a smaller version of that feeling, when a system refuses something and nobody properly explains why. That feeling of helplessness is the point.
By what rule?
On paper the rule is careful. Article 3 of the Regulation lets a provider scan only if the technology is the least privacy-intrusive available; if it has run a data protection impact assessment and consulted its data protection authority beforehand; if errors are limited “to the maximum extent possible”; if there is human oversight; if users are told that the provider scans and that their data may reach the police; if users whose content is removed or whose account is blocked are told how to complain; and if the provider publishes an annual report with “the numbers and ratios of errors (false positives)”.
Now the practice, as the Commission recorded it for 2023 and 2024 under the identical conditions of the previous regulation. Five companies reported, and they did not use the mandatory form. On the conditions that matter most, the Commission wrote that “no information was submitted by the providers on whether the technologies were deployed in line with the state of the art and in the least privacy-intrusive way, or on whether a prior data protection impact assessment … and a prior consultation procedure … had been conducted”. Meta’s users appealed the actions taken on around 254,500 items in 2023; Meta restored around 11,600. Google counted 297 appeals against account suspensions that year and reinstated 10 accounts. And across the whole Union in two years, “no instance of complaints to a judicial authority have been reported”. Nobody went to court. The new Regulation carries the same conditions forward, with a grace period: a provider already scanning before 31 July 2026 may carry on without a completed consultation until 1 April 2027.
What all this says is that paper will bear anything. We can write that the technology must be as unintrusive as possible, that results must be checked by a human, that errors must be tracked and that there must be oversight. It all sounds good. But if nobody then knows whether the companies actually do what they are supposed to, the rules are no protection. They are closer to a promise of protection.
So what interests me is less what the law demands than who ever said: show us that you really do it. Who saw the numbers. Who checked the error rate. Who was sanctioned for failing the conditions. Because a rule without oversight is a strange thing in the end. The citizen has to obey it, while a large system may merely claim that it does.
Who pays the price?
The cost of error is the most unfairly distributed part of the whole system. When the algorithm works, the company can say: we found a problem, the system works. When it errs, the algorithm feels nothing. The company usually does not lose its email, its photographs or ten years of digital life. One specific person carries that error.
How often? By the companies’ own numbers, rarely at the first step and often enough further down. Google reported that 10 of the 1,834 items its hash matching flagged in 2024 were not confirmed as abuse on human review. Meta restored 1,800 items on appeal in 2024, out of 1.5 million it had acted on. Yubo, which scans for grooming, reported that its moderators took no action on 13 percent of the conversations the tools flagged. Further down the chain: of the 91,242 NCMEC reports the BKA received between January and May 2026, about 60 percent were criminally relevant. The other four in ten, roughly 36,000 files about people in Germany in five months, were examined by federal police and found not to be crimes.
And sometimes the error lands on one person with full force. In February 2021 a father in San Francisco, identified in The New York Times only as Mark, photographed his toddler’s swollen groin at a nurse’s request, ahead of a video consultation during the pandemic. His phone backed the pictures up to Google, whose systems flagged them. Google disabled his account, his email, his photos and the phone number it provided, and reported him. The police concluded that no crime had occurred; the detective could not even reach Mark, because Google had cut off his phone and his email. Google did not restore the account.
“I have nothing to hide” seems to me a badly put question. I have nothing to hide from my neighbour either, and I still do not want him standing in my living room every evening. Privacy was never a licence to do something wrong. It is the space in which I do not have to keep proving that I am doing nothing wrong. And then there are the mistakes. Maybe you hide nothing. But a system can misunderstand something about you, and at that moment the question is no longer what you wanted to hide. The question is whether you can prove that the machine was wrong.
What does it mean for you?
I would avoid telling you that someone reads all your messages, because that would be misleading and would only manufacture fear. Here is the ordinary version: some of the services you use every day can automatically analyse certain content, and under certain circumstances a report can come out of that which does not stay inside the platform.
Which services? End-to-end encrypted ones are outside the Regulation by its own words: Signal, WhatsApp chats, iMessage, and personal chats on Messenger since Meta switched them to default encryption in December 2023. Outside the law does not mean outside reporting: WhatsApp’s head said in 2021 that the service had reported more than 400,000 cases to NCMEC in a year “all without breaking encryption”, from unencrypted parts such as profile photos and user reports. Unencrypted services are where the scanning runs: Gmail, Outlook, iCloud Mail, Instagram messages, Snapchat, Discord, TikTok, Xbox. The Czech police told the Commission that its 2024 reports came from 57 providers, led by Instagram with 11,857, then Facebook 4,461, Snapchat 3,610, Discord 1,510, Google 1,439, Microsoft 870, TikTok 825 and WhatsApp 620. Apple, whose mail is unencrypted, made 296 reports to NCMEC in all of 2025; the volume depends on the company’s choices. Cloud photo storage is a third category, outside the Regulation altogether, and it is where Mark’s pictures were caught.
After a flag, the Regulation obliges the provider to tell you in general terms that it scans and that your data may reach the police, and, if it removes your content or blocks your account, to tell you how to complain. It obliges nobody to tell you that a report about you was filed. You may learn that from the police, or never.
I want people to know that, so that they understand the environment they communicate in, without throwing the phone away. My advice is fairly boring: find out what the service you use actually protects, judged by the facts and not by the advertising. Whether it uses end-to-end encryption. What gets stored in the cloud. How reports work. For sensitive communication, think about the tool before you send anything. The word for that is literacy, and it has nothing to do with panic. Our tracker keeps a table of which apps are covered.
Is there another way?
I would start with one principle: the more private the communication, the higher the threshold before anyone intrudes into it. If there is a concrete suspicion and an independent body or a court says the intrusion is justified, that is a completely different situation from preventively combing through enormous amounts of communication of people who are suspected of nothing. That is, in short, the European Parliament’s line for the permanent law: detection only on a judicial order, against named suspects.
A European centre that would pass on and verify reports instead of an American organisation makes intuitively more sense to me, if only because of jurisdiction and accountability. The Council’s mandate of November 2025 keeps such a body, the EU Centre on Child Sexual Abuse, which would “assess and process the information supplied by the online providers” and pass it to Europol and national police. But a European logo on the door does not by itself guarantee a good system.
The very first step I would take is even less glamorous: actually enforce the rules we already have. Measure the errors. Publish the results, compulsorily. Check the procedures. Give people a working appeal. The dates are already in the law: consultations with data protection authorities must start by 1 September 2026, the Commission must publish by 1 October 2026 the list of organisations that receive Europeans’ reports, and the first public error-rate reports are due by 1 February 2027. We may find that we do not need another enormous mechanism, only to finally take seriously the protections we already wrote down once.
If Google or Meta blocked my account tomorrow because of a machine’s mistake, my first reaction would be anger. Then I would want to know the most elementary things. What did you find? What do you say I did? Did a machine decide, or a person? And to whom can I explain that it is a mistake? That is exactly what I would want guaranteed by law: an intelligible explanation, access to human review, a reasonable deadline, and a way to actually get the account back if the system was wrong. Our accounts today are almost small private archives of a life: photographs, contacts, work documents, conversations. A company should not have the right to cut a person off from all of that within a second and then offer an automated reply along the lines of “our decision is final”.
The fact that technology can detect things does not worry me. What worries me is the moment when it can make a mistake that turns a person’s life upside down, and nobody quite knows who answers for it. Until 29 September, when the negotiators meet again, the answer is a company. We keep the current status on the EU Chat Control Tracker, updated within 24 hours of every development.
Sources
- Regulation (EU) 2026/1881, Official Journal of the EU, 28 July 2026 (EUR-Lex)
- European Commission, Report on the implementation of Regulation (EU) 2021/1232, COM(2025) 740 final, 27 November 2025
- NCMEC, 2025 CyberTipline Report, and 2025 CyberTipline Reports by Electronic Service Providers (PDF)
- German Federal Ministry of the Interior, written answer to MP Donata Vogtschmidt, 22 June 2026, and BKA statement of 25 June 2026, both published in full by netzpolitik.org, 25 June 2026: Auch ohne Gesetz: Weiterhin massenhaft Hinweise auf Kindesmissbrauch
- Kashmir Hill, “A Dad Took Photos of His Naked Toddler for the Doctor. Google Flagged Him as a Criminal.”, The New York Times, 21 August 2022
- Meta Newsroom, “Launching Default End-to-End Encryption on Messenger”, 6 December 2023
- Will Cathcart (WhatsApp), post on X, 6 August 2021
- Council of the EU, press release “Child sexual abuse: Council reaches position on law protecting children from online abuse”, 26 November 2025
- Earlier parts of this series: When a No is not enough; Five trilogues, no deal; The Council’s lawyers said no. Chat Control went voluntary.
