By Aaron Tilley
For those who saw them, the demonstrations inside Apple earlier this decade of a revamped Siri offered a showcase of the amazing capabilities a powerful AI voice assistant could have.
The famed assistant, one of the last projects Apple co-founder Steve Jobs worked on before his death, had been given a total overhaul. Capable of running on an iPhone and without an internet connection, the new Siri impressed people with its improved speed, conversational capabilities and the accuracy with which it understood user commands. Code-named Project Blackbird, the effort also imagined a Siri with capabilities built by third-party app developers, according to people familiar with the work.
Yet a competing project won out in an internal contest ahead of the 10-year anniversary of Siri’s launch. Known as Siri X, the more-modest upgrade involved moving more existing Siri software onto iPhones from remote servers to improve the voice assistant’s speed and privacy. The Siri X enhancement was unveiled in 2021.
Next week, at Apple’s annual Worldwide Developers Conference, the company is set to join an AI arms race that many think will define the future of technology. The iPhone maker is trying to catch up with Microsoft, Alphabet’s Google and other rivals that have begun to integrate generative AI into their core products.
Apple’s caution and characteristic secrecy, as well as the care it takes in upgrading devices — where hardware and software are seamlessly integrated — have hobbled its early efforts in the AI arena, the people said. It now finds itself in the unusual position of having to take risks.
The company is set to announce an array of generative-AI upgrades to its software products, including Siri, said people familiar with its plans. The AI features include assistance in message writing, photo editing and summarizing texts.
While Apple isn’t expected to catch up with leading AI innovators soon, the company is preparing to unveil AI features with impressive capabilities that also maximize privacy for users — a concern that will be central to unlocking the full potential of AI assistants. Apple is also expected to unveil one or more potential partnerships with major AI developers after holding talks with OpenAI, Google and Cohere, the people said.
Apple has long prided itself on perfection in its product rollouts, a near impossibility with emerging AI models. While OpenAI systems have dazzled more than 180 million users with their generation of writing, images and video, they are prone to occasional errors, often called hallucinations. Apple has had limited tolerance for such issues.
“There’s no such thing as 100% accuracy with AI, that’s the fundamental reality,” said Pedro Domingos, a professor emeritus of computer science and engineering at the University of Washington. “Apple is not compatible with that. They won’t release something until it’s perfect.”
Integrating AI
Apple has weighed whether to allow users to choose a third-party AI provider that could supplant or help power Siri, one of the people said. It is unclear in what way Siri could be powered, augmented or replaced by third-party AI providers, or whether Apple will move forward with a version of that idea.
Bloomberg earlier reported on an OpenAI partnership, and The Information reported on efforts to overhaul Siri.
Google, Microsoft and Samsung Electronics have begun rapidly integrating generative AI into their devices and services. While Apple finds itself behind in a generational shift taking place in the tech industry, many investors and AI experts have said the company will find a way to bring generative AI to the masses.
“Apple can do pretty much anything they set their minds to,” said Vineet Khosla, a former engineering manager on the Siri team who is currently chief technology officer at the Washington Post. “There is a consumer focus that exists at the company. The focus of their AI is to make it work in a very privacy-sensitive manner.”
Over the years, Apple has made improvements to Siri and incorporated smaller AI features throughout all of its products. In the recently released headset Vision Pro, AI is extensively used for tracking eyes and hand positions.
When Siri launched in 2011, Apple was ahead of rivals in seeking to establish the first AI assistant. Jobs, who spearheaded the acquisition in 2010 that led to Siri, encouraged the team to keep the assistant’s dry wit and sense of humor. The early launch demonstrated the company’s willingness to take risks.
“Siri was the last thing Apple was first on,” said Dag Kittlaus, co-founder of the Siri startup that Apple bought, who left Apple soon after the product’s launch.
Tensions with ex-Googlers
As Apple struggled to keep advancing Siri, the company hired one of Google’s top engineering executives to run its AI strategy: John Giannandrea. In 2018, he was elevated to the role of senior vice president, reporting directly to Apple Chief Executive Tim Cook.
In early team meetings, Giannandrea made it clear that improving Siri was a main focus. He was also tasked with centralizing much of Apple’s fragmented efforts regarding AI. He began building up his AI team by recruiting Google employees and through startup acquisitions, but his team had difficulty fitting in with the rest of Apple, according to people familiar with their work.
The new AI group operated much like parts of Google, where deadlines are more loosely defined. Teams inside Apple need to maintain rigorous deadlines to have products ready for major release events every fall. Efforts to collaborate between other parts of Apple that were building products and the AI team at times fell apart because they couldn’t agree on deadlines, the people said.
Instead of working with the AI team, other parts of Apple busy building software products maintained their own, separate AI capabilities. For example, the software group run by Senior Vice President Craig Federighi continued to invest in and build up the AI behind its image- and video-recognition capabilities, said former Apple employees.
Another big limiting factor for Giannandrea’s AI team was the lack of access to computing resources, said former executives and engineers. Compared with rivals, Apple in recent years has secured fewer chips known as graphical-processing units that are essential for training AI models, said people familiar with Apple’s internal infrastructure.
Much of the AI team had to rely on external cloud services — Google’s cloud was preferred by many of the former Google employees on Giannandrea’s team — to train their AI models, the people said.
The ChatGPT effect
When ChatGPT launched in late 2022, everything changed. Federighi, the software chief, became a convert that Christmas break after he began playing around with the Microsoft-owned GitHub AI coding tool called Copilot, which is powered by OpenAI’s technology, said people familiar with his experience.
After that moment, across Federighi’s software-engineering organization, employees were tasked with coming up with new ways of incorporating generative AI into products and given resources to pursue these projects, said former executives and engineers. At internal meetings, Federighi said that he had come to appreciate generative AI technology and that it would be incorporated into all aspects of Apple’s software.
Apple stepped up efforts to build its own internal generative AI. In February, the company canceled its decadeslong effort to build its own electric car. Some of the employees had been redeployed into these generative AI projects.
Some of the new features and updates Apple is expected to announce this year will be powered by Apple’s internally built generative AI models, but Apple has been looking at potential external partnerships for more advanced AI. Giannandrea and Federighi have met with Sam Altman, OpenAI’s chief executive officer, said people familiar with the discussions.
With a new urgency on AI technology, Kittlaus, the Siri co-founder, said this year could be an important one for Siri as the company plans to incorporate thoughtful AI features into the iPhone. “Siri has been stuck in the mud for years,” he said. “But I absolutely see a renaissance coming.”
Write to Aaron Tilley at aaron.tilley@wsj.com