Apple LLM
While some other businesses are working on public chatbots, Apple appears to be more interested in on-device processing and useful apps when it comes to its own large language model (LLM) technology development.
Apple Ajax
Apple Insider reports that the company is concentrating on improving its current applications and services, and that it will reveal a number of artificial intelligence (AI) improvements in the next iOS 18 release. Apple’s massive language model dubbed Ajax, which strives to offer consumers useful capabilities while protecting their privacy through on-device processing, will power the new functionalities.
The ability of Siri to examine and summarise text messages within the Messages app is one of the major enhancements anticipated in iOS 18. With the help of this functionality, users won’t have to read through entire threads to rapidly understand the important points of lengthy chats.
AI improvements are on the way for Spotlight and Safari
Apple Insider claims that Safari will get a “Intelligent Search” feature that would allow users to create summaries of webpages in addition to the Messages app. Additionally, the business is working on enhancing Spotlight Search with AI so that it can respond to user queries with pertinent information and be more context-aware.
Apple Ajax LLM Siri OpenAI
According to reports, Apple’s Ajax LLM can generate straightforward replies wholly on-device, guaranteeing quicker processing and more privacy protection. The software might, however, have to rely on server-side processing for more complicated requests, which might entail collaborations with Google or OpenAI.
Apple Ajax LLM Siri messages OpenAI
Apple Insider claims that the business will give customers control over their data by displaying privacy warnings prior to Ajax accessing data from Messages or Safari.
Apple Music may soon allow AI-generated playlists.
Apple CEO Tim Cook said in a recent earnings call that the company is confident in its AI plans and has “advantages” that would set it apart. Apple’s “unwavering focus on privacy” sets it apart in the artificial intelligence field, according to Cook. Although there are currently little information available regarding the specific AI features in iOS 18, we anticipate learning more about Apple’s plans at the next Worldwide Developers Conference (WWDC) on June 10.
One would say Apple is late to AI. Since the ChatGPT went viral in late 2022, most Apple competitors have scrambled to catch up. Apple has talked about AI and introduced several AI-inspired products, but it seemed to be toeing the line.
However, rumours and stories have suggested that Apple has been waiting to act for months. Apple has been working on Ajax and talking to OpenAI and Google about powering some of its AI features, according to recent reports.
Apple’s public AI research paints a picture of its AI strategy. The path from research to shop shelves is windy and full of potholes, so making product assumptions based on research articles is inexact. At its annual developer conference, WWDC, in June, Apple will discuss its AI features and how they might work.
Smaller, more efficient models
Better Siri. It appears Better Siri is coming! Lots of Apple’s research (and the tech sector, the world, and elsewhere) assumes huge language models will instantly make virtual assistants smarter. Apple has to make those models quickly and everywhere to improve Siri.
Bloomberg stated that Apple will run all AI features on-device, offline with iOS 18. Even with a network of data centres and hundreds of cutting-edge GPUs, building a strong multifunctional model is difficult. It’s even tougher with your smartphone’s guts. So Apple must innovate.
In “Apple LLM in a flash: Efficient Large Language Model Inference with Limited Memory”, researchers developed a system to store model data on the SSD instead of RAM. The researchers noted, “We have demonstrated the ability to run Apple LLMs up to twice the size of available DRAM [on the SSD], achieving an acceleration in inference speed by 4-5x compared to traditional loading methods in CPU, and 20-25x in GPU.” They found that models operate faster and more efficiently by using the cheapest and most available storage on your device.
Apple researchers developed EELBERT to compress an Apple LLM without making it worse. Their reduced Google Bert model was 15 times smaller (1.2 gigabytes) and lost only 4% quality. However, latency tradeoffs existed.
In general, Apple is trying to resolve a model world tension: larger models are better and more helpful, but they are also bulky, power-hungry, and slow. The company, like many others, is striving to balance all those things and have it all.
Apple Siri
When we talk about AI products, we mostly mean virtual assistants that know things, remind us, answer questions, and do things for us. It’s not surprising that Apple’s AI research centres on one question: what if Siri was really, really good?
Instead of listening for “Hey Siri” or “Siri,” Apple engineers are working on a technique to use Siri without a wake word. The gadget may be able to distinguish between the two. This challenge is substantially more complex than speech trigger detection,” the researchers said, “since there could not be a leading trigger phrase that signals the commencement of a spoken command.” This may be why another set of researchers built a better wake word detection algorithm. Another paper taught a model to understand unusual words, which helpers struggle with.
Apple LLMs can process more information faster in both circumstances, which is appealing. In the wake-word report, the researchers found that providing all extraneous sound to the model and letting it process what matters worked better.
Apple works hard to improve Siri’s understanding and communication after hearing you. It developed STEER (Semantic Turn Extension-Expansion Recognition) in one paper to improve back-and-forth communication with an assistant by identifying follow-up questions and new ones. Another employs Apple LLMs to comprehend “ambiguous queries” to determine what you intend regardless of how you state it. In uncertain situations, “intelligent conversational agents may need to take the initiative to reduce their uncertainty by asking good questions proactively, thereby solving problems more effectively.” Another work employed Apple LLMs to make assistants’ answers less verbose and more intelligible.
Medical AI, image editors, Memojis
When Apple discusses AI publicly, it emphasises its everyday benefits rather than its raw technological might. While Siri gets a lot of attention as Apple competes with gadgets like the Humane AI Pin, the Rabbit R1, and Google’s ongoing integration of Gemini into Android, Apple sees AI’s other uses.
Apple may prioritise health via Apple LLMs, which could help you understand the seas of biometric data collected by your gadgets. Apple has been studying how to collect and analyse motion data, utilise gait recognition and headphones to identify you, and track and comprehend heart rate data. Apple revealed “the largest multi-device multi-location sensor-based human activity dataset” after collecting data from 50 people with several on-body sensors.
Apple sees AI as a creative tool. For one paper, researchers interviewed animators, designers, and engineers and created Keyframer, which “enable[s] users to iteratively construct and refine generated designs.” You start with a prompt and obtain a toolkit to alter and refine aspects of the image instead of typing another prompt to get another image. This kind of back-and-forth artistic process might appear in Memoji or Apple’s professional artistic tools.
Apple introduces MGIE, a tool that allows you change an image by specifying the edits, in another study. “Make the sky more blue,” “make my face less weird,” “add some rocks,” etc. Instead of brief but confusing guidance, MGIE extracts specific visual-aware purpose and leads to appropriate picture altering, researchers noted. Its early experiments were spectacular yet flawed.
Over time, Apple will likely lean on this, especially on iOS. Apple will integrate some of it into its apps and provide APIs to third-parties. The latest Journaling Suggestions function may help with that. Apple has always boasted its hardware, especially compared to Android devices; on-device, privacy-focused AI might be a huge differentiation.
Apple Ferret LLM
You must know Ferret to see Apple’s greatest, most ambitious AI project. Ferret is a multi-modal big language model that can follow instructions, focus on a targeted object, and understand its surroundings. It’s built for the now-standard AI use case of querying a device about the world, but it may also understand your screen. In the Ferret article, researchers show it may assist you navigate apps, answer App Store rating questions, describe what you’re looking at, and more. This has exciting accessibility implications that potentially revolutionise how you use your phone, Vision Pro, and smart glasses.
We’re ahead of ourselves, but this may work with other Apple projects. If Siri can grasp what you want and your gadget can see and understand everything on your display, your phone can utilise itself. Apple could automatically run apps and touch the relevant buttons without significant integrations.
Again, this is research, and for it to perform successfully this spring would be a technical feat. After trying chatbots, you know they’re bad. WWDC will have huge AI announcements. Tim Cook hinted to it in February and promised it on this week’s earnings call. Two things are clear: Apple is racing AI, and it may completely revamp the iPhone. Maybe you’ll use Siri gladly! That would be impressive.
