Apple AI Research Reveals Key Model Limits

Apple researchers are set to unveil critical at the NeurIPS 2025 conference that current assumptions about AI capabilities and privacy. Their work spans reasoning models, privacy-preserving techniques, and generative AI, offering insights that could influence future AI development and deployment. This research highlights both the progress and pitfalls in creating systems that think and protect user data, with for everyday applications like virtual assistants and secure data analysis.

In one study, Apple researchers systematically tested AI reasoning models using controllable puzzle environments to see how they handle increasing complexity. They found that frontier Large Reasoning Models (LRMs) see their accuracy collapse beyond certain complexity thresholds, as shown in Figure 1. Interestingly, the models' reasoning effort rises with complexity up to a point, then declines even with sufficient token budgets, suggesting inherent limitations in current approaches.

When comparing Large Reasoning Models (LRMs) and Large Language Models (LLMs) with equal inference compute, the researchers discovered that LLMs outperform LRMs on low-complexity tasks, while LRMs have an advantage in medium-complexity scenarios. Both types fail on high-complexity tasks, indicating that today's AI struggles with advanced reasoning. This raises questions about the true capabilities of reasoning models and points to areas needing improvement for tasks like coding or robot navigation.

On the privacy front, Apple introduced new algorithms for estimating probability distributions accurately while ensuring differential privacy, as detailed in their Spotlight paper on instance-optimality for private KL distribution estimation. These s adapt to each dataset and perform nearly as well as the best possible approach for that case, all while mathematically guaranteeing that no individual's data can be inferred. This is crucial for applications like data compression and language modeling where diversity and smoothness in estimates matter.

Another privacy advancement comes from the analysis of random allocation sampling, where a user's data is used in k steps chosen randomly from a sequence of t steps. This scheme provides better privacy-utility tradeoffs for algorithms like differentially private SGD and secure aggregation, as explored in the PREAMBLE paper. By offering theoretical guarantees and numerical estimation algorithms, this work enables more efficient and private machine learning without sacrificing performance.

In generative AI, Apple's STARFlow paper presents a scalable approach to high-resolution image synthesis using normalizing flows and autoregressive transformers, rivaling diffusion and autoregressive models in quality while maintaining exact likelihood modeling and faster inference. As shown in Figure 2, this achieves resolutions previously thought unreachable for normalizing flow models, offering a less computationally expensive alternative for image generation. Additionally, the LinEAS technique allows precise control over model outputs, such as reducing toxicity in language models or adding new concepts in text-to-image generation, with minimal data and computational cost.

For training large foundation models, Apple researchers developed scaling laws to determine optimal data mixtures across domains, eliminating the need for costly trial-and-error s. As illustrated in Figure 4, these laws predict model performance for large language models, native multimodal models, and large vision models, allowing practitioners to optimize domain weights under specific training budgets. This principled approach could accelerate AI development by ensuring models are trained on the right data blends for better outcomes in real-world applications.