UK Prime Minister Keir Starmer’s speech on Monday insisting that tech companies create device controls to somehow block children from viewing or creating sexually explicit imagery has raised alarms among CISOs, who worry that the same technology could undermine enterprise security. Starmer gave tech firms three months to create and implement such restrictions voluntarily, at which point he said he would push for legislation to make it mandatory.
Behind the technical and logistical hurdles for tech firms to clear, such as how a device would determine that an image was inappropriate, and how it could reliably determine the subject’s age, is the issue of whether this process would interfere with encryption protections for enterprises worldwide. And that comes down to whether the required data analysis happens on the device or in the cloud.
Starmer did not go into a lot of detail, preferring to let technology companies craft their own plans, but in this case the details matter. Analysts and consultants said that there has been a push for everything to happen on-device, which would avoid any encryption problems; if the inspected data never leaves the device, the encryption protection would stay intact.
But this plan for the process to stay on the device seems highly unlikely for multiple reasons. The first problem is device capabilities and hardware age. Although Apple and Google engineers would be working with the latest devices, much of the UK population is using much older and less capable hardware, analysts said.
Although a 2-, 3- or 4-year-old phone might still be able to handle the additional load, it would likely suffer a dramatic slowdown sufficient to make users decidedly unhappy. That would mean that even if the execution of the data analysis began on the device, it would likely have to be shifted to the cloud for performance reasons. And once it moved into the cloud, the encrypted data problem begins.
Trying to do this scanning on-device in the UK would fail, said Flavio Villanustre, CISO for the LexisNexis Risk Solutions Group. “It will make unusable the majority of devices used in the UK today. It just can’t work on-device.”
However, Villanustre observed that on-device analysis for this kind of effort, which would need to scan everything that gets downloaded to the phone in search of prohibited images, might be viable in a few years, once the typical device becomes much more powerful. But not today.
Creates new risks
Leading secure messaging app provider Signal also issued a strong statement opposing Starmer’s proposal.
“The UK governmentʼs demand that all content on all devices sold or used in the UK be scanned on the presumption of nudity, using a dystopian combination of age verification and content scanning, will not safeguard children. It endangers us all, whilst strengthening Apple, Google and Microsoft’s market dominance and their control over our most personal information,” Signal said. “Once created, [the program] will be expanded, forming a dangerous tool that will be wielded both in the UK and abroad to censor and surveil whatever they might consider ‘threats’ or ‘harmful content.’”
Signal has aggressively fought against such programs before. Similar privacy campaigns have also been launched in other parts of Europe.
The long held fear is that moving encrypted data to the cloud, regardless of whether it remains encrypted or is converted to clear text, creates opportunities for attackers to access the sensitive data.
“The mechanism that flags and reports a match to external authorities creates a new, built-in exfiltration path,” said Jeff Valdes, a director at consulting firm Acceligence.
Could do more harm than good
Sanchit Vir Gogia, chief analyst at Greyhound Research, argued that the UK proposal is likely to do far more damage than good. He pointed to the short three month timeframe as evidence of a lack of good faith.
“Legislation of this complexity cannot be drafted in a quarter. The deadline is a pressure instrument, not a delivery schedule. Child safety is the destination. Device-wide inspection is the wrong vehicle,” Gogia said. “Apple and Google already run on-device nudity detection in bounded contexts, and it works: a child can be warned, an image blurred, a sharing attempt interrupted.”
Gogia pointed to another logistical problem, which is that some devices such as tablets are often shared between family members, which makes reliable age determinations all but impossible.
“The deeper flaw is that the policy assumes a stable mapping between device, person, and age, and that mapping does not exist in real households,” Gogia said. “A device cannot know its holder has changed. The only architecture that survives this is default-child with recurring adult verification, which is surveillance arriving through the back door of household economics.”
In addition, he noted, “Children disproportionately inherit the old, out-of-support handsets the mandate cannot reach. Forcing churn manufactures electronic waste and punishes the families least able to buy new.”
Carmi Levy, an independent technology analyst, agreed that the computing overhead alone for such an effort could make this a deal-killer.
“The compute requirements, particularly in light of the need to execute this kind of filtering in real time, would be immense. It is futile to assume this capability can ever be rolled out at scale without running into massive concerns on several fronts,” Levy said. “Simply deciding how to tune the filters is an almost impossible task. Although the overall definition of nudity, namely not wearing clothing, is generally agreed upon, the line where it becomes inappropriate for minors is neither static nor universally established. So it’s wildly optimistic to assume that a single threshold would be workable at the scale proposed by Prime Minister Starmer.”
Nidhi Luthra, a director at Acceligence, added that the logistical and technological roadblocks are also a big problem.
“Technically, parts of this can work,” she said, but vendors would have to deal with age verifications, drifts in the models and false positives, and there is also the “lack of contextual information that truly would have let this work.”
Puts CISOs in ‘an impossible bind’
The UK proposal also puts enterprise CISOs and IT directors who need to protect sensitive data in an impossible bind, Gogia said.
They “can govern device management and conditional access. What they cannot govern is a mandatory inspection capability that updates according to political appetite rather than enterprise risk appetite,” he pointed out. “The proposal does not automatically create a breach inside Signal, WhatsApp, or Teams, but it creates the conditions for a new class of breach around them. The weakness need not live in the messaging protocol. It can live in the mandated inspection layer, the classifier update mechanism, the age-assurance workflow, or the logs that enforcement inevitably generates.”
Regime change could lead to abuse
Another common concern is that governments change hands, so limited capabilities granted today to one government might be used very differently by a future government.
Brian Jackson, principal research director at Info-Tech Research Group, noted, “the current government may only use it to detect nudes, but what is to stop a future authoritarian government from using it to detect unfavorable political commentary? Creating a back door means there is potential for third parties — hackers — to exploit that back door to gain access to the user’s communications. This is exactly what encryption and on-device security measures are supposed to prevent.”
He added, “Apple’s Communication Safety feature, Google’s Family Link, and a range of parental control tools already use on-device AI to detect and restrict explicit imagery on children’s devices. The government is not filling a gap the market failed to address. It is proposing to transfer control of an existing capability from the device owner to the state. Parents can deploy this protection right now, on their terms. That is where the decision should sit.”
Ryan O’Leary, research director for privacy and legal technology at IDC, said the current proposal only involves the UK, and there’s no way to determine whether other governments will try something similar. He noted that the EU’s GDPR was widely expected to go global when it launched in 2016, but in ten years, it hasn’t.
O’Leary said that if this proposal is enacted in the UK, he would advise IT and cybersecurity executives to be extra cautious when sending team members to the region.
“It would essentially be ‘China rules’” such as air gapping systems and traveling with disposable data-limited burner phones, O’Leary said. “It’s an exceptionally big deal if it goes through,” but, he added, the chance of it happening is very low. “It seems like the technology companies will call his bluff.”
This article originally appeared on CSOonline.
Login
