During Monday’s keynote at Apple’s Worldwide Developers Conference (WWDC), a presenter asked Siri for the FIFA World Cup tournament schedule and then requested that the virtual assistant plan a watch party for a specific match. Siri pulled the schedule from the Internet, suggested dishes from the two countries that were playing the match, dug through the user’s Messages app history to find a mention of coconut cookies, drafted a text invitation featuring the party’s menu and prepared to send it to a group chat. Siri carried out this choreography without the user ever touching an app.
The proactive assistant Apple has promised—and repeatedly delayed—for two years has, it seems, finally arrived. But to pull off this kind of digital errand running, Siri needs deep access to personal data that Apple has spent years walling off: your mail, photographs, messages and calendar. Each new capability expands the territory the company’s privacy architecture must cover. At WWDC, Apple’s keynote speakers kept returning to the same privacy claims: user requests to Siri stay private, data are not retained after processing, and outside researchers can inspect the system.
Florian Schaub, who studies usable privacy at the University of Michigan, says Apple’s openness to outside scrutiny is welcome—but limited. “Consumers often lack the expertise to inspect code,” he says. But by publishing specifications and letting researchers and regulators examine its systems, Apple “at least facilitates external validation of their claims.”
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Powered by Apple Intelligence, the new Siri, or Siri AI, relies on an architecture that Apple calls the “system orchestrator,” a layer that coordinates data flowing among Spotlight’s “semantic index,” onscreen information and tools that carry out actions inside apps. Siri’s underlying reasoning rests on a new generation of Apple Foundation Models, including a top-tier cloud model that the company calls AFM 3 Cloud Pro, which is custom-built for Apple hardware. When a request is too complex for a phone, Apple says Private Cloud Compute handles it on servers that do not retain user data and can be inspected by outside researchers. According to Bloomberg, the largest of these models was reportedly derived from a specialized version of Gemini with about 1.2 trillion parameters that Google has licensed to Apple for about $1 billion a year. Ahead of Monday’s keynote, the Information reported that some of that cloud processing might run on Nvidia chips inside Google’s data centers.
Apple executives have distinguished the deployment from Google’s consumer artificial intelligence stack and model-serving infrastructure. Yet until Apple opens this hybrid cloud arrangement to the outside inspection it invites for Private Cloud Compute, the data-routing security of these models rests largely on the company’s word.
Encryption protects data in storage and in transit, but it cannot stop an assistant such as Siri from misusing the access it has been given. Text from an e-mail, web page or shared document can reach the model in the same stream as the user’s instructions. To the software, that outside text may function as a command, even if the user never meant it that way. Researchers call this indirect prompt injection. Programmer Simon Willison describes the risk as the “lethal trifecta”: any assistant that can read private data, ingest untrusted content and transmit information can be tricked into handing those private data to a stranger. A phone assistant with Siri’s new abilities brings all those elements together.
“Autonomous agents significantly expand the attack surface for prompt injection,” says Natalie Shapira, a security researcher at Northeastern University, who studies AI agents. “The challenge is the chain of permissions and actions that connects the model to multiple applications and services.”
Last year researchers at Aim Security found exactly this opening in Microsoft 365 Copilot. They named it EchoLeak, a zero-click attack on a production AI assistant. A single e-mail planted instructions that the software later carried out when the recipient asked it something unrelated. The stolen data slipped out through an image the software loaded on its own, with no link to click and nothing on-screen. Microsoft patched the vulnerability before anyone was known to have used it. Apple’s Safari demo at WWDC showed how this same structural risk reaches beyond Siri: the browser will be able to generate custom extensions via vibe coding.
Apple says Siri AI will not reach iPhones or iPads in the European Union at launch (though it will run on Macs and other devices there) because of the E.U.’s Digital Markets Act, a competition law for large digital platforms. (In China, the new features await regulatory approval.) Citing security researchers, Apple argued the E.U. law would force it to give rival AI assistants the same deep access to user data. The company insists its architecture contains risks that a competitor’s might not—but no independent researchers have tested the new Siri in the wild. Apple did not immediately respond to a request for comment.
The public release is planned for later this year. Once it arrives, security researchers and ordinary users alike will experience Siri’s reach beyond Apple’s carefully staged demos.
