Apple Machine Learning Research

Y'all did such a good job with this. It captivated HN and was the top post for the entire day, and will probably last for much of tomorrow.

If you don't know already, you need to leverage this. HN is one of the biggest channels of engineers and venture capitalists on the internet. It's almost pure signal (minus some grumpy engineer grumblings - we're a grouchy lot sometimes).

Post your contract info here. You might get business inquiries. If you've got any special software or process in what you do, there might be "venture scale" business opportunities that come your way. Certainly clients, but potentially much more.

(I'd certainly like to get in touch!)

--

I believe volumetric is going to be huge for creative work in the coming years.

Gaussian splats are a huge improvement over point clouds and NeRFs in terms of accessibility and rendering, but the field has so many potential ways to evolve.

I was always in love with Intel's "volume", but it was impractical [1, 2] and got shut down.

During the pandemic, to get over my lack of in-person filmmaking, I wrote Unreal Engine shaders to combine the output of several Kinect point clouds [3] to build my own lightweight version. The VGA resolution was a pain and I was faced with fpga solutions for higher real time resolution, or going 100% offline.

Today, World Labs and Apple are doing exciting work with image-to-Gaussian models [4, 5], and World Labs created the fantastic Spark library [6] for viewing them.

Most recently,…

Conceptually, to the best of my understanding, nothing too serious; perhaps the inefficiency of processing a larger input than necessary?

Practically, a few things:

If you want to have your cake & eat it too, they recommend Enumerated Shapes[1] in their coremltools docs, where CoreML precompiles up to 128 (!) variants of input shapes, but again this is fairly limiting (1 tok, 2 tok, 3 tok... up to 128 token prompts.. maybe you enforce a minimum, say 80 tokens to account for a system prompt, so up to 200 tokens, but... still pretty short). But this is only compatible with CPU inference, so that reduces its appeal.

It seems like its current state was designed for text embedding models, where you normalize input length by chunking (often 128 or 256 tokens) and operate on the chunks — and indeed, that’s the only text-based CoreML model that Apple ships today, a Bert embedding model tuned for Q&A[2], not an LLM.

You could used a fixed input length that’s fairly large; I haven’t experimented with it once I grasped the memory requirements, but from what I gather from HuggingFace’s announcement blog post[3], it seems that is what they do with swift-transformers & their CoreML conversions, handling the details for you[4][5]. I haven’t carefully investigated the implementation, but I’m curious to learn more!

You can be sure that no one is more aware of all this than Apple — they published "Deploying Transformers on the Apple Neural Engine" in June 2022[6]. I look forward to…

> I'd also like to ask a different question: if there's no reason to ever worry about this feature, then why is there even an option to turn it off in the first place?

I mean for one, because of people like you that are concerned about it. Apple wants you to have the choice if you are against this feature. It's silly to try to use that as some sort of proof that the feature isn't safe.

My iPhone has a button to disable the flash in the camera app. Does that imply that somehow using the camera flash is dangerous and Apple is trying to hide the truth from us all? Obviously not, it simply means that sometimes you may not want to use the flash.

They likely chose to make it opt-out because their research shows that this is truly completely private, including being secure against future post-quantum attacks.

> If an adversary is able to intercept encrypted communications, they can store it in hopes of decrypting it in the future in the event that a feasible attack against the cryptosystem emerges. I don't know how likely this is to happen against homomorphic encryption schemes, but the answer is not zero.

Also, if you're going to wildly speculate like this it is at least (IMO) worth reading the research press release since it does answer many of the questions you've posed here[0].

> it's pretty upsetting that it's becoming incredibly hard to the point of being nearly impractical to get modern devices to behave this way and not just fling data around all over the place…

Most AI stories in 2025 still orbit the cloud: giant models, branded “copilots,” and oceans of user data flowing off your devices. On the Mac, the direction is more subtle — and arguably more interesting.

With macOS Sequoia and Apple Intelligence, Apple is turning the Mac into a *device-first AI machine*: intelligence built into the operating system, models that run increasingly on your own hardware, and developer tools that treat AI as part of normal computing, not a separate destination.

---

## macOS Sequoia + Apple Intelligence: AI as Part of the Interface

Apple’s latest desktop release, *macOS Sequoia*, looks like a classic productivity update — iPhone Mirroring, a smarter Safari, a dedicated Passwords app. But it’s also the main delivery vehicle for *Apple Intelligence*, Apple’s new system-wide AI layer.

Official overviews:

- Apple Intelligence: https://www.apple.com/apple-intelligence/ - macOS Sequoia announcement: https://www.apple.com/newsroom/2024/06/macos-sequoia-takes-productivity-and-intelligence-on-mac-to-new-heights/

On macOS, Apple Intelligence shows up as small, targeted upgrades:

Apple’s machine-learning hub for developers lays out that strategy:

- Machine Learning & AI on Apple platforms: https://developer.apple.com/machine-learning/

Key pieces that sit naturally on macOS:

- *Core ML* – runs optimized ML models on Apple silicon and Intel Macs, from image recognition to language models: …

Craig Federighi SVP of Software at Apple “The keyboard now leverages a transformer language model, which is state of the art for word prediction, making autocorrect more accurate than ever.”[1]

"New features available with macOS Sonoma" doc states "Autocorrect fixes mistakes for you more accurately than ever before by leveraging a new transformer language model in English, French, and Spanish keyboards." [2]

"Introducing Apple’s On-Device and Server Foundation Models" blog post states "Apple Intelligence is comprised of multiple highly-capable generative models that...have been fine-tuned for user experiences such as writing and refining text.. In the following overview, we will detail how two of these models — a ~3 billion parameter on-device language model, and a larger server-based language model."[3]

"A look at Apple’s new Transformer-powered predictive text model" blog post[4] is the kind of deep dive you'd have to do to find out what's going on under the hood in your spelling/grammar checker. TL;DR AppleSpell uses GPT-2 which proves my point.

[1] https://eu.usatoday.com/story/tech/2023/06/06/wwdc-2023-apple-ios17-autocorrect/70292365007/

[2] https://www.apple.com/mideast/macos/sonoma/pdf/en/macOS_All_New_Features_bh.pdf

[3] https://machinelearning.apple.com/research/introducing-apple-foundation-models

[4] https://jackcook.com/2023/09/08/predictive-text.html

"The apple study" is being overblown too, but here it is: https://machinelearning.apple.com/research/illusion-of-thinking

The crux is that beyond a bit of complexity the whole house of cards comes tumbling down. This is trivially obvious to any user of LLMs who has trained themselves to use LLMs (or LRMs in this case) to get better results ... the usual "But you're prompting it wrong" answer to any LLM skepticism. Well, that's definitely true! But it's also true that these aren't magical intelligent subservient omniscient creatures, because that would imply that they would learn how to work with you. And before you say "moving goalpost" remember, this is essentially what the world thinks they are being sold.

It can be both breathless hysteria and an amazing piece of revolutionary and useful technology at the same time.

The training set argument is just a fundamental misunderstanding, yes, but you should think about the contrapositive - can an LLM do well on things that are _inside_ its training set? This paper does use examples that are present all over the internet including solutions. Things children can learn to do well. Figure 5 is a good figure to show the collapse in the face of complexity. We've all seen that when tearing through a codebase or trying to "remember" old information.