You have to hand it to Apple Inc. After an embarrassing, tone-deaf ad last month that made the company look oblivious to AI’s impact on the world, its marketing department has now rebranded AI as “Apple Intelligence.” It’s a feat of superiority only the company could pull off.
Customers of Macs and the latest iPhones will use it to rewrite emails, transcribe and summarize calls, generate images and, most enticingly, cross-reference information from Apple apps. “Will I get to my daughter’s play performance on time?” Apple Software Chief Craig Federighi asked in one demo at the company’s annual World Wide Developers Conference on Monday. Apple Intelligence would talk to his iPhone’s proprietary Calendar, Maps, Mail and iMessage apps behind the scenes to answer the question.
After a string of mundane updates to its iOS and Mac OS software, these are the most exciting features in years, prompting praise from technologists on Twitter. But will Apple Intelligence really work as seamlessly as it did in the pre-recorded demos, when it rolls out this fall? I’m inclined to believe we’ll see glitches and latency issues that will make it a tough sell to consumers — at least initially.
The most sophisticated AI tools today process your queries on powerful cloud servers that require an Internet connection. Apple’s iPhone has a fraction of the power of those servers, but to make its AI service private and quick, it will run some AI queries via Siri “on device,” on a small language model Apple built itself to work on an iPhone. No internet connection needed.
Apple Intelligence will also decide, on the fly, if a query like “Will I get to my daughter’s play performance on time?” requires extra computing power. If it does, it’ll access a bigger AI model that Apple made, via something called “Private Cloud Compute,” which is essentially Apple’s own servers. Anything even more complex will request a query to ChatGPT, via a partnership with OpenAI. Apple, admirably, has gone to great lengths to keep this process private, with query requests being end-to-end encrypted and inaccessible to others.
The price for all of this could be speed.
When Apple answers a query using its smaller on-device AI, it’ll do so with a latency of 0.6 milliseconds per prompt token, according to Apple’s blog post announcing the features, or faster than the blink of an eye. But Apple didn’t offer corresponding latency times for when the phone has to access its Private Cloud Compute for more complex tasks, and that’s a noteworthy omission. It’ll likely be slower, but by how much? Apple doesn’t say.
As shallow as this sounds, consumers hate having to wait a few extra seconds for things they can do themselves, and if it’s simply quicker to look something up in their calendar or mapping apps, they might decide to avoid using Apple Intelligence altogether.
On the other hand, while the on-device AI will be faster, it will also be more prone to errors. According to Apple, the smaller AI model is about as capable as GPT-3.5 Turbo, which OpenAI launched more than a year ago, and which doesn’t have a stellar reputation for accuracy. The hallucination rate for GPT 3.5 has ranged from 3.5% to more than 15%, according to different estimates, and that points to another omission from Apple. It didn’t offer accuracy rates for either of its new AI models. Remember that even the most sophisticated models coming from leading players like Google — think about the new AI Overviews — are still laughably error-prone. Apple could have disclosed some decent hallucination rates. Instead, it offered vague stats about how human graders “preferred” its AI over others.
Little wonder then, that Bloomberg’s senior Apple reporter Mark Gurman has said the new AI features will be buggy and in “beta” when they launch this fall.
Apple’s role at the center of its customers’ lives and its access to reams of their personal information could end up being both a blessing and a curse. It is better positioned than most tech companies to make AI useful because is plugged so deeply into our daily existence. But that also means it can’t afford to make too many mistakes.
Federighi’s example of making it to his daughter’s play is a case in point. Were Apple Intelligence to inadvertently make its user 30 minutes late, it wouldn’t be easily forgiven. Much the same happened to Apple Maps, where a flurry of mistakes in its early days sent people to competitors like Waze and Google Maps for years afterward. It doesn’t help that Apple has been slow on the uptake with generative AI, thanks in part to a smaller AI team compared with other large tech firms, and a history of management troubles at its Siri division.
Apple’s more private approach is a welcome stance, and the company should be commended for sticking to its guns on using private, encrypted servers while others are playing fast and loose with AI in a battle for supremacy. But getting consumers to embrace its ambitious new features won’t be easy when the glitzy demos are suddenly made real.
A message from Advisor Perspectives and VettaFi: To learn more about this and other topics, check out our videos.
Bloomberg News provided this article. For more articles like this please visit
bloomberg.com.
Read more articles by Parmy Olson
