As we wrap up our series about alternatives to transformer architectures, Google DeepMind just released one of the most impressive models in this category. DiffusionGemma is a text-diffusion model that challenges the conventional transfromer models. Today, we would like to deep dive into the specifics of this model.

Most language models write like a typewriter. They place one token after another, left to right, never revisiting the characters already stamped onto the page. This architecture has carried the entire modern LLM era: GPT-style chatbots, coding copilots, reasoning models, agent frameworks, enterprise assistants. The model predicts the next token, appends it, updates its state, and repeats.

Google’s new DiffusionGemma asks a deceptively simple question: what if text generation did not have to work that way?

Let’s dive in.

Read more

Today, we bring you a summary of our series about transformer alternatives.

For the better part of a decade, the entire field has been a giant, spectacularly funded wrapper around a single operation: self-attention. The Transformer didn’t win because it was the most elegant or the most brain-like design. It won because it had the best scaling story and it won the hardware lottery. Every token looks at every other token, the whole thing maps cleanly onto a GPU grid, and you train it all at once. Add data, parameters, compute, context — and the loss curve cooperates. That smoothness is rare. Most clever ideas in deep learning never become industrial. This one did.

But the tax was always there in plain sight. Self-attention buys you something genuinely valuable — perfect, lossless recall over the entire context, with every token able to address every other token directly, and a training pass that parallelizes across the whole sequence at once. That’s the benefit, and it’s a real one. The cost is that attention scales quadratically with sequence length, and autoregressive decoding drags around a KV-cache that grows linearly with every token you’ve already seen. When you’re pushing past a million tokens, or watching a 70B model’s cache eat 40GB of VRAM, O(n²) compute and O(n) memory stop being footnotes and become the actual bill. So the interesting question was never “are Transformers good?” They’re spectacular. The question is whether they’re the final architecture or just the first truly scalable one — soon to be absorbed into something richer.

That was the thesis we set out to test, and the cleanest way to read the eight issues is as four families, each making a different bet against attention.

The first family is recurrent and linear-recurrent models — the RNN comeback and xLSTM. Their pitch is constant memory: instead of a cache that grows forever, they carry a fixed-size hidden state and pay O(n) compute over a sequence rather than O(n²). The classic objection was that RNNs train serially and can’t saturate a GPU, but the modern variants reformulate the recurrence so it parallelizes during training while staying cheap at inference. The benefit is brutally efficient generation; the open challenge is whether a fixed-size state can hold enough to match attention’s exact recall on long-range, retrieval-heavy tasks.

The second family is state space models — the SSM/Mamba line, the most serious challenger of the bunch. SSMs treat a sequence as a continuous linear dynamical system, which gives them a near-magical dual form: a parallelizable convolution for training and a recurrent scan for inference. They get linear scaling and long-context handling almost for free. The trade-off is expressivity — pure SSMs can struggle with precise in-context copying and lookup, which is exactly why the strongest results today are hybrids that interleave a few attention layers among many SSM layers.

The third family is text diffusion — generation that abandons left-to-right decoding entirely, refining a whole sequence in parallel over a handful of denoising steps. The benefit is non-autoregressive speed and bidirectional context at generation time; the challenge is matching the raw quality and controllability of autoregressive models, which LLaDA, Gemini Diffusion, and Mercury are now pushing on hard.

The fourth family is liquid and continuous-time models, which throw out the parallel-lookup mental model altogether in favor of dynamics that evolve continuously in time, aiming for far smaller, more adaptive networks. The benefit is parameter efficiency and a different inductive bias; the challenge is scaling that story to frontier sizes.

None of these has dethroned attention. But the monoculture is over, and the most likely future is explicitly hybrid: attention where exact recall earns its quadratic cost, something linear-time everywhere else.

Here is the full series, in order:

• #846 — Beyond Transformer: A New Series — The kickoff, framing the palpable vibe shift on arXiv toward post-attention architectures and the decade we’ve spent as a wrapper around self-attention. It lays out the plan to map every major viable alternative to the Transformer.

• #850 — The Unexpected Comeback of RNNs — The case for recurrent networks as the alternative most people overlooked, revisiting why linear-time recurrence is attractive again. It positions modern RNN variants as a serious challenger rather than a relic.

• #854 — Return of the King: Unrolling the xLSTM Architecture — Traces the lineage from the 1990s LSTM through the 2017 Transformer pivot into xLSTM, the modernized revival of Hochreiter and Schmidhuber’s design. It explains how reworked gating and scaling let xLSTM compete with attention-based models.

• #858 — How State Space Models Went from Curiosity to Serious Transformer Competitor — Charts the rise of SSMs as the O(n²) attention bottleneck becomes a real constraint at million-token contexts and large KV-caches. It argues state space models have quietly matured into a genuine rival to the dominant paradigm.

• #862 — Learning About Text Diffusion Models — Introduces text diffusion as one of the most credible non-autoregressive alternatives to transformers. It covers how diffusion-style generation breaks from strict left-to-right next-token prediction.

• #866 — Three Text Diffusion Models You Need To Know About — A practical follow-up profiling the leading players in the space: LLaDA, Gemini Diffusion, and Mercury. It compares how each implements diffusion-based text generation.

• #870 — Liquid Models and the Search for a Post-Transformer Architecture — Dives into liquid neural networks as one of the more promising non-Transformer architectures, contrasting their continuous-time dynamics with attention’s parallel lookup-table approach. It frames them within the broader hunt for a successor.

• #874 — Transformers or Not? — The capstone, asking whether the Transformer is the final architecture or merely the first truly scalable one, soon absorbed into something richer. It leans toward the latter and surveys the full landscape the series has covered.

What’s next: a new series on distillation

If the last series was about changing the architecture, the next one is about compressing it. We’re starting a deep dive into knowledge distillation — the set of techniques for taking a large, expensive teacher model and pressing its capabilities into a smaller, faster student. It’s one of the least glamorous and most economically important ideas in modern AI: it’s how frontier capability actually reaches production. We’ll cover the classics (logit matching, the original Hinton formulation), the modern variants (sequence-level, on-policy, and self-distillation), what actually transfers and what doesn’t, and why nearly every model you can afford to run is, in some sense, a distilled one. See you in the first issue.

Next Week in The Sequence:

1. We continue our series about alternative to transformers.

2. The AI of the week will dive into Fable.

3. In the opinion section, we are going to discuss AI tokens as a units of economics.

4. We might introduce a new fun section. Playing with a new idea.

Subscribe and don’t miss out:

📝 Editorial: Last Week in AI: Anthropic Ships, Apple Borrows, Musk Lists, Bezos Builds

Some weeks in AI feel like incremental patch releases. This one felt like a major version bump for the entire industry. Four events — a frontier model launch, a consumer assistant reboot, the largest IPO in history, and a $12 billion bet on physical engineering — and if you squint, they’re all chapters of the same story: AI escaping the chat window.

Start with Anthropic. On Tuesday the company released Claude Fable 5 and Claude Mythos 5, and the architecture of the launch is as interesting as the model itself. Both share the same base model; the difference is policy, not weights. Fable 5 ships with conservative safety classifiers that intercept queries in high-risk domains — cybersecurity, biology, chemistry — and fall back to Opus 4.8, while Mythos 5 runs unrestricted for a vetted group of cyber defenders under Project Glasswing. Think of it as the same kernel with different syscall permissions. The benchmarks justify the caution: 80.3% on SWE-Bench Pro, more than ten points clear of Opus 4.8 and over twenty ahead of GPT-5.5. We’ve entered the era where capability and access are explicitly decoupled — the model you can use is a sandboxed view of the model that exists.

Then Apple, finally, showed up. At Tim Cook’s farewell WWDC, the company unveiled Siri AI — a conversational assistant with personal context, onscreen awareness, and a standalone app, reportedly powered by a custom 1.2-trillion-parameter Gemini model under the hood. There’s something deliciously ironic about Apple, the original vertical integrator, outsourcing the brain. But strategically it’s the right call: Apple’s moat was never the model; it’s the distribution and the personal context graph. A billion devices with intimate access to your messages, photos, and calendar is a dataset no lab can replicate. Apple isn’t competing on intelligence; it’s competing on intimacy.

The week’s most audacious move came from Elon Musk. SpaceX went public at roughly $1.77 trillion, raising about $75 billion in the largest IPO ever — and the prospectus reads less like a rocket company and more like an AI infrastructure thesis. Having merged xAI into SpaceX in February, Musk is pitching orbital data centers: up to a million GPU-packed satellites moving training and inference off-planet, where energy is abundant and regulation is thin. xAI lost $6.4 billion on $3.2 billion in revenue last year, so the IPO is effectively the public market underwriting the most capital-intensive scaling hypothesis ever proposed. Compute, it turns out, has an escape velocity.

Finally, Jeff Bezos broke his silence on Prometheus, which raised $12 billion at a $41 billion valuation to build an “artificial general engineer” — AI that designs and manufactures physical systems, from jet engines to drug compounds. Not robotics, Bezos insists. Something closer to CAD with a frontier brain.

Let’s dive in.

🔎 AI Research

Regularized f-Divergence Kernel Tests

AI Lab: Google Research & Google DeepMind

Summary: This paper introduces a unified framework for constructing practical, kernel-based two-sample tests derived from the family of f-divergences. The authors demonstrate that these adaptive tests, particularly the Hockey-Stick divergence, effectively capture diverse localized differences and are highly applicable to tasks like differential privacy auditing and machine unlearning evaluation.

Verifiable Environments Are LEGO Bricks: Recursive Composition for Reasoning Generalization

AI Lab: Qwen Team, Alibaba Group

Summary: The authors propose RACES, a framework that scales up reinforcement learning for language models by recursively assembling verifiable environments like building blocks when their input and output types match. By utilizing composition operators such as SEQUENTIAL and PARALLEL, this approach generates structurally diverse training tasks that significantly improve the reasoning generalization of models on unseen benchmarks.

REVISION: Scaling Computer-Use Agents via Temporal Visual Redundancy Reduction

AI Lab: Microsoft Research

Summary: To address the high token cost associated with visual observations in computer-use agents, this paper introduces REVISION, a framework that trains multimodal models to filter out redundant visual patches across consecutive screenshots. By maintaining essential spatial structure while significantly reducing token accumulation, the method allows agents to process longer interaction histories and achieve higher success rates on complex tasks.

Distilling LLM Feedback for Lean Theorem Proving

AI Lab: FAIR at Meta

Summary: This research explores Feedback Distillation, an on-policy post-training method where a model learns to match its own token-level distribution conditioned on privileged feedback from a stronger language model. Evaluated on Lean 4 theorem proving, the technique preserves greater trajectory diversity and achieves better pass@k scaling than standard GRPO, proving especially powerful when used as an initialization for subsequent reinforcement learning.

Decentralized Multi-Agent Systems with Shared Context

AI Lab: Stanford University

Summary: DELM is a novel multi-agent framework that eliminates the bottleneck of centralized orchestration by relying on a shared, verified context and an asynchronous task queue. Agents independently claim subtasks and contribute compact, verified updates to the global state, leading to superior performance and cost efficiency in both software-engineering testing and long-context reasoning workflows.

🤖 AI Tech Releases

Claude Fable 5 and Mythos 5

Anthropic released its highly anticipated Fable 5 model, a limited Mythos-based models. Also released a version of Mythos 5 for a selected group of cyersecurity and infrastructure companies.

Kimi Work

Moonshot AI released Kimi Work, a new agent specialized in work automation.

📡10 AI News You Need to Know About

1. SpaceX (SPCX) made its Nasdaq debut June 12, 2026, after pricing at $135 per share and raising roughly $75 billion — the largest IPO in stock market history, valuing the company near $1.75 trillion. Shares opened sharply higher and were trading around $161 intraday, with the valuation anchored by Starlink and now bundling in xAI following an all-stock merger earlier this year.

2. Bezos’s Prometheus raises $12B — Jeff Bezos and Vik Bajaj’s physical-AI startup Prometheus raised $12 billion at a $41 billion valuation to build an “artificial general engineer” that automates the design and manufacturing of complex physical systems from jet engines to drugs.

3. Mistral AI is in early talks to raise about €3 billion (~$3.5 billion) at a valuation near €20 billion (~$23 billion)— nearly double the €11.7 billion valuation from its Series C last September. The new round would bring the three-year-old company's total financing to roughly €6.5 billion across debt and equity, fueling its compute buildout as Europe's leading AI lab competes against larger US and Chinese rivals.

4. Theker raises $85M — Barcelona-based Theker raised $85 million in what it bills as Europe’s largest-ever robotics Series A to build reconfigurable factory robots whose arms and hands swap out for different tasks rather than specializing in one.

5. Jedify raises $24M — New York’s Jedify raised a $24 million Series A led by Norwest, with Snowflake as a strategic investor, to build “context graphs” that give enterprise AI agents the business knowledge they need to run in production.

6. Sandstone raises $30M — Sandstone raised a $30 million Series A led by Lightspeed to bring AI-powered workflow automation (intake, routing, triage, task execution) to in-house corporate legal teams rather than law firms.

7. OpenAI to acquire Ona — OpenAI agreed to acquire cloud-platform startup Ona, folding its secure, persistent execution environments into the Codex team so AI agents can run long, multi-step tasks for enterprises.

8. xAI co-founder unveils River AI — xAI co-founder Igor Babuschkin announced River AI, a startup (staffed partly by former xAI and Tesla employees) building personalized AI agents that learn from and remain owned/controlled by individual users rather than large corporations.

9. Tether backs Neura in $1.4B round — German firm Neura Robotics raised up to $1.4 billion in a Tether-led Series C — also backed by Nvidia, Amazon, Qualcomm and Bosch — to scale humanoid/cognitive-robot production toward millions of units by 2030.

10. Apple introduces Siri AI — Apple unveiled “Siri AI,” a rebuilt Apple Intelligence–powered assistant with personal-context understanding, onscreen awareness, world knowledge, a dedicated app, and expanded Visual Intelligence, available for developer testing now and as a user beta later this year.

Thesis: agentic AI does not kill SaaS. It changes what enterprise software is fundamentally for. The old winning layer was the system that held canonical state. The new winning layer is the system that can take action against that state safely, reliably, and observably.

For the last twenty years, the enterprise stack has been built around one hidden constant: the human is the actor.

A person logs in. A person reads a dashboard. A person fills out a form. A person updates the opportunity stage, approves the invoice, closes the ticket, moves the candidate, changes the forecast, escalates the account, or checks the compliance box.

The software is basically a database wrapped in forms, permissions, workflows, and a pricing page. This is not an insult. It is an extremely powerful pattern. It gave companies shared memory. It made business state durable. It turned messy organizational reality into tables, fields, roles, reports, and audit logs.

But now the actor is changing.

Read more

For today’s essay, I would like to cover an incredible paper with a provocative thesis and an even better title that I found myself reading multiple times last week: Language Models Need Sleep….

There’s an awkward fact about the models we all use every day: they don’t learn anything anymore. Whatever a frontier model knows, it learned once, during training, and then somebody hit save. After that it’s a brilliant fossil. It can reason circles around you about events up to its cutoff and then go completely blank about last Tuesday. You can stuff new facts into the context window, sure, but the moment the session ends, that knowledge evaporates like a dream you forgot to write down.

Behrouz, Hashemi, and Mirrokni (Google + Cornell) have a name for this in their new paper, and it’s a good one: it’s anterograde amnesia. The patient with anterograde amnesia keeps every memory from before the injury and can hold a conversation in the moment, but nothing new ever makes the jump into long-term storage. Each day is experienced as if it were the first. Swap “injury” for “end of pre-training” and that is exactly the shape of a Transformer’s memory. It has the deep past (the MLP weights) and the immediate present (the attention cache), and almost nothing connecting them.

The paper’s pitch is that we’ve been missing a step that biology figured out a long time ago. We sleep.

There is no test time

Read more

💡 AI Concept of the Day: Transformers or Not?

The Transformer is currently the reference architecture for serious AI. Not because it is obviously the most brain-like, elegant, or efficient design, but because it has the best scaling story. You add data, parameters, compute, context length, better training recipes, better post-training, and the model gets better in a surprisingly smooth way. That is rare. In deep learning, many ideas are clever. Few are industrial.

The Transformer’s superpower is attention. Every token can look at every other token and decide what matters. This is an incredibly general operation. It works for language, code, images, audio, video, protein sequences, robotics tokens, and tool traces. The architecture is simple enough to scale, parallel enough to train efficiently, and expressive enough to absorb huge datasets.

But it has an obvious tax: attention is expensive. Full self-attention scales badly with sequence length. In autoregressive generation, the model accumulates a key-value cache, which grows with context. A Transformer remembers by keeping a large, explicit, token-indexed memory. That is powerful, but it is not how you would design every intelligent system from first principles.

So the question is not “are Transformers good?” They are spectacular. The question is: are they the final architecture? Or are they the first truly scalable architecture, soon to be absorbed into something richer?

I think the second view is more likely.

The Landscape of Alternatives

Read more

Next Week in The Sequence:

1. We continue our series about alternatives to transformers.

2. Our opinion section discusses an intriguing thesis: systems of action vs. systems of record.

3. The AI of the week dives into a groundbreaking paper that I’ve read three times this week: models need sleep.

Subscribe and don’t miss out:

📝 Editorial: Last Week in AI: Soccer, S-1s, and Supermodels

This week I want to start close to home. At LayerLens, we announced the Stratix Cup, a live tournament in which frontier AI models play soccer in a simulated environment. Season 1 brings together 16 models organized into four groups, with each model writing code to control a full team of players. Matches unfold in two halves, and models can adapt their strategy at halftime based on what happened on the field.

It is, admittedly, ridiculous in the best possible way: models chasing space, collapsing under pressure, inventing strange formations, and occasionally self-sabotaging in public. But the playfulness hides an important point. Evaluations need more arenas.

Most AI evals still behave like school exams: static, individual, decontextualized. They ask models to answer questions, solve coding problems, summarize documents, or reason through puzzles. These are useful, but they are incomplete. Soccer imposes a different discipline. It tests multi-agent planning, tactical adaptation, long-horizon credit assignment, robustness under adversarial pressure, and the ability to recover from mistakes. It also makes model behavior legible. You do not need a PhD to see when a model loses shape in midfield. That makes the failure modes both entertaining and intellectually honest, a rare combination in AI benchmarks.

The rest of the week echoed the same shift from models as artifacts to models as operating systems.

At Build, Microsoft introduced a new generation of MAI models across reasoning, coding, image, voice, and transcription. The headline is not just that Microsoft is building more of its own models. The strategic point is that the company wants a tighter loop between models, developer tools, agents, and devices. GitHub Copilot, agent security primitives, new model releases, and AI-native workflows all point to a world where AI is no longer a chat box bolted onto software. It is becoming the substrate running through work itself.

Anthropic’s confidential S-1 filing adds a different kind of gravity. The proposed IPO is not yet a public-market event, but it signals that frontier AI is moving from private-market mythology into public-market accountability. Revenue quality, compute commitments, margins, governance, and safety claims will eventually have to survive a much harsher evaluation suite: investors, regulators, and quarterly reporting.

NVIDIA, meanwhile, pushed the frontier in two complementary directions. Cosmos advances the idea of world foundation models for physical AI: systems that can reason about video, simulation, robotics, and embodied environments. Nemotron 3 Ultra expands NVIDIA’s enterprise model stack for demanding reasoning and agentic workflows. The implication is clear: NVIDIA is not merely selling the shovels for the AI gold rush. It wants to define the terrain where robots, agents, simulations, and enterprises are built.

Finally, DeepSeek’s reported new financing round is another reminder that the open-model race is becoming geopolitical infrastructure. Capital, energy, chips, talent, and industrial policy are converging around frontier labs. Open models are no longer just an engineering philosophy. They are becoming a strategic asset.

Put together, this week’s story is AI leaving the demo page. It is playing games, staffing workflows, filing S-1s, simulating the physical world, and attracting national-scale capital.

The question is no longer simply who has the best chatbot. It is which systems can act, adapt, and be trusted in environments that push back. Benchmarks told us how models answer. Arenas will tell us how they behave.

🔎 AI Research

Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories

AI Lab: Google Research and Cornell University

Summary: This paper introduces a bio-inspired “Sleep” paradigm for large language models that enables continual learning and mitigates catastrophic forgetting. The approach features a memory consolidation stage that distills fragile short-term memories into stable long-term parameters, alongside a “Dreaming” phase where the model recursively self-improves using synthetically generated data.

Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions

AI Lab: Harvard, MIT, GitHub, 2077 AI, and Kempner Institute

Summary: The Economy of Minds (EOM) framework enables a population of language agents to self-organize and evolve through decentralized economic interactions, such as bidding for the right to act and exchanging peer-to-peer payments. By relying on market selection to mutate wealthy, successful agents and eliminate bankrupt, ineffective ones, this system fosters emergent multi-step reasoning and outperforms stronger monolithic baselines across diverse tasks.

Cosmos 3: Omnimodal World Models for Physical AI

AI Lab: NVIDIA

Summary: Cosmos 3 introduces a unified Mixture-of-Transformers architecture that seamlessly integrates language, image, video, audio, and action sequences to serve as a versatile foundation for Physical AI. By consolidating multiple specialized functions—such as vision-language models, video generators, and world-action models—into a single framework, it achieves state-of-the-art performance across diverse understanding and generation tasks.

Qwen-Image-Flash: Beyond Objective Design

AI Lab: Qwen (Alibaba)

Summary: This paper explores the critical training components of few-step distillation—specifically data composition, step-wise multi-teacher guidance, and task mixture—demonstrating that effective distillation requires holistic pipeline optimization beyond just the objective function. Applying these insights, the authors developed Qwen-Image-Flash, an efficient unified model capable of executing both text-to-image generation and instruction-guided image editing with only four function evaluations.

Dreaming: Better memory for a more helpful ChatGPT

AI Lab: OpenAI

Summary: OpenAI has introduced a more scalable and compute-efficient memory architecture for ChatGPT that relies on a background process called “dreaming” to continuously curate and synthesize past conversations. By optimizing for freshness, continuity, and relevance, this new system significantly improves the model’s ability to recall factual context, follow user preferences, and adapt to changing information over time.

🤖 AI Tech Releases

Nemotron 3 - Ultra

NVIDIA released Nemotron 3 Ultra, optimized for long running agentic workflows.

MAI Models

Microsoft unveiled 7 new AI models including a flagship MAI-Thinking-1.

Gemma 4 12B

DeepMind released Gemma 4 12B, a multimodal intelligence model that can run locally on a laptop.

📡AI News You Need to Know About

• Airbnb’s Brian Chesky plans a new AI lab — Airbnb CEO Brian Chesky intends to bankroll a new AI lab (reportedly focused on user interaction and design) while staying on as Airbnb’s CEO rather than running it himself. Original source: the Bloomberg scoop that broke it, since Airbnb declined to comment and issued nothing of its own →

• Alphabet plans to raise $80B for AI buildout — Alphabet said it will sell $80 billion in stock (including $10 billion to Berkshire Hathaway) to fund AI infrastructure and compute amid demand outstripping supply. Original source: Alphabet’s own press release →

• Coralogix raises $200M — The observability startup raised a $200M Series F (co-led by Advent, CPPIB, and Greenfield) at a $1.6B valuation, betting that monitoring autonomous AI agents will become core production infrastructure.

• ZeroDrift raises $10M — The startup closed an oversubscribed $10M seed round (backed by a16z Speedrun and others) for its “compliance firewall,” which sits inline between AI systems and end users to catch and rewrite non-compliant AI-generated messages.

• Anthropic files to go public — Anthropic confidentially submitted a draft S-1 to the SEC for a proposed IPO, days after a $65B Series H pushed its valuation near $1 trillion.

• Meta sells an AI agent to businesses — Meta began charging for “Meta Business Agent,” an AI that handles customer conversations across WhatsApp, Messenger, and Instagram, as part of its push to monetize AI and diversify beyond ads.

• DeepSeek close to a ~$7B funding round — DeepSeek is reportedly near finalizing roughly 50 billion yuan (~$7.4B) in its first external raise, led by Tencent, CATL, and founder Liang Wenfeng, at a ~$52–59B valuation. Note: there’s no primary source here — the deal is unconfirmed and the parties declined to comment — and the scoop originated with Reuters before Bloomberg corroborated it, so the

• Google to pay SpaceX $920M/month for compute — Google will pay SpaceX ~$920M monthly from October 2026 through June 2029 for access to roughly 110,000 NVIDIA GPUs and related hardware as “bridge capacity” for Gemini Enterprise demand, announced a week before SpaceX’s IPO.

When Jensen Huang draws AI as a five-layer cake — energy, chips, infrastructure, models, applications — he describes it as harmony. Every layer reinforces the others. Every successful application pulls demand down through models, infrastructure, and chips, all the way to the power plant that keeps it alive. It is a beautiful picture, and as a statement of physics it is correct.

But Jensen is selling the bottom of the cake, so of course he wants you to see harmony. If you are a strategist instead of a chip vendor, you look at the same diagram and see something else entirely: five margin pools stacked on top of each other, and a war over which of them you can fuse together before the layer beneath you turns into a commodity. The cake is not a structure of mutual reinforcement. It is a battlefield with a vertical axis.

So the right question is never “how many layers do you own.” It is: do you own the scarce layer, and the seam right next to it?

Read more

I am sure you guys are surprised that we are going to cover Claude Opus 4.8 today ;) but I have been playing with it so much that merits the post.

Opus 4.8 shipped on May 28, 2026. The headline contributions, in the order I’d rank them for anyone building agents: a roughly 4x reduction in how often the model leaves a flaw in its own code unremarked — the calibration/honesty story that defines this release; a fix for silently skipped tool calls, the bug class that quietly poisons long trajectories; better compaction recovery so long-horizon runs stop derailing after the history gets squeezed; dynamic workflows that let the model plan and fan out hundreds of parallel subagents for codebase-scale work; adaptive thinking that decides per-turn whether to reason at all; and a fast mode that runs ~2.5x faster at a tier that’s now ~3x cheaper than 4.7’s. Alignment results land near the (still-restricted) Mythos Preview. Same regular-mode pricing as its predecessor.

It’s tempting to file this under “minor release.” The version bump is a tenth of a point, the benchmark deltas are mostly incremental, and the cadence makes it easy to lose track — Opus 4.6 landed February 5, 4.7 on April 16, and 4.8 just six weeks later. That’s a compression from a roughly quarterly rhythm to something closer to monthly, and when point releases arrive that fast the instinct is to treat each one as a patch and skip the changelog.

That instinct is wrong here, because Opus 4.8 isn’t competing on the axis the version number implies. The benchmark table moved a little. What moved a lot is the reliability axis — the silent-failure rate, the tool discipline, the ability to hold a thread across a long run unattended. Those are the properties that gate whether you can actually leave an agent running, and they don’t show up on a capability leaderboard. The short cadence is also the tell: when you can ship calibration and reliability fixes every six weeks, the model stops being a thing you upgrade quarterly and becomes infrastructure you keep current. So let me give you the version I’d want if I were wiring this into a production agent loop at 2am.

The benchmark story is the boring story

Read more

💡 AI Concept of the Day: Liquid Models and the Search for a Post-Transformer Architecture

The Transformer did not merely become the dominant neural architecture. It became the default mental model for intelligence in modern AI.

Its central idea is deceptively simple: when processing a sequence, let every element look at every other element. A word can attend to previous words. A code token can attend to distant variables. An image patch can attend to another patch. A tool call can attend to an instruction buried thousands of tokens earlier. Attention turns sequence modeling into a giant differentiable lookup table over context.

This was a profound break from the recurrent era. Earlier models processed sequences like a reader moving left to right, updating a hidden state at each step. Transformers flattened that temporal process into a massively parallel computation. Instead of compressing the past into a single state, they exposed the entire past to the model. That made training easier, scaling more predictable, and long-range relationships easier to represent.

But every architecture has a physics. Transformers have the physics of global interaction. That physics is powerful, but expensive.

Self-attention wants to compare tokens against other tokens. During inference, the model accumulates a key-value cache so that each new token can attend to the past. As context grows, memory grows. As model size grows, serving complexity grows. As agents become longer-running, more tool-using, and more local, the cost of remembering everything explicitly becomes harder to ignore.

The Transformer is a brilliant architecture for cloud-scale intelligence. It is less obviously the final architecture for always-on, low-latency, private, embodied, on-device intelligence.

That is where liquid models enter the story.

From Attention to Dynamics

Read more

Next Week in The Sequence:

• We continue our series about transformer alternatives.

• In the AI of the Week section, we discuss Opus 4.8.

• The opinion of the week discusses companies the strategic differences between companies like Google, NVIDIA, Microsoft, OpenAI and Anthropic when comes to their ownership of different areas of the AI stack.

Subscribe and don’t miss out:

📝 Editorial: Last Week in AI: The Token Becomes the Unit of Account — Opus 4.8, OpenRouter, Cognition, Snowflake, and a papal warning

For two years the AI boom was an argument about the future, told in benchmarks and term sheets. This week it became an argument about the present, told in revenue.

Start with the substrate. Anthropic shipped Claude Opus 4.8 and, in the same breath, disclosed it’s tracking toward its first operating profit — roughly $10.9B in projected Q2 revenue, up about 130% quarter over quarter — while closing a $65B round. Sit with that. A lab still doing frontier training runs is approaching operational profitability. The “labs are structurally unprofitable” assumption that anchored every bear case just lost its load-bearing wall.

The model itself Anthropic described, refreshingly, as “a modest but tangible improvement” — agentic coding nudges from ~64% to ~69%, reasoning-with-tools from ~55% to ~58%, at the same price as 4.7. The interesting stuff is underneath the benchmarks. Three changes matter. First, an effort control that lets you dial how hard the model thinks per task — explicit governance over the compute-versus-quality tradeoff that every agent builder has been hacking around with prompt tricks. Second, dynamic workflows: a Claude Code capability where the model plans a large task, spins up parallel sub-agents to attack the pieces, verifies their outputs, and reports back — paired with a Messages API that now accepts live edits to the message array mid-run without breaking the prompt cache, so you can steer a long job without tearing it down and restarting. Third, honesty as a measured capability: 4.8 is roughly 4x less likely than 4.7 to let a flaw in its own code slip through unflagged, and surfaces its own uncertainty more readily. Stack those and you get the thing that actually matters once a model runs unattended for hours: it plans, it parallelizes, it checks its own work, and — because you can’t read every diff — it’s trained to distrust itself. It also burns tokens by the fistful doing all of it.

Then watch where the money flows, because it tells you the unit of account is now the token. OpenRouter raised $113M at $1.3B doing something almost embarrassingly simple: routing across 400+ models and taking ~5% of the inference spend that passes through. Its weekly throughput went from 5T to 25T tokens in six months — 5x. That’s not a forecast; that’s a meter. Cognition raised $1B at $26B, and buried in the announcement was the line that should reorganize your priors: 89% of code committed inside Cognition is now written by Devin, up from 13% in December. Run-rate revenue went from $37M to $492M in a year. Autonomous software engineering stopped being a demo and became the default committer.

Snowflake closes the loop on the public side. Product revenue up 34%, guidance raised, stock up ~36% in a single session — and the two tells are a $6B AWS compute deal and the acquisition of Natoma, an MCP platform for governing agent access. The data layer is repricing itself around agents that consume, not analysts that query. The whole stack — model, router, agent, substrate — is converging on one business model: charge by the token, because the token is the work.

Which is exactly the moment Pope Leo XIV chose to publish Magnifica Humanitas, his first encyclical, presented alongside Anthropic’s Chris Olah. Stripped of its theology, the argument is an engineering critique the field should take seriously: technology is never neutral, because it inherits the incentives of whoever builds and funds it — and the danger isn’t malice but quiet disintermediation, decisions sliding out of human hands one delegated commit at a time.

Hold those two facts together. Cognition’s 89% is the encyclical’s thesis restated as a metric. The meter that makes the economics work is the same meter measuring how much judgment we’ve handed off. The bull case and the moral case are reading the same number.

The flywheel is no longer a slide. It’s on the income statement. The open question is what it’s optimizing for.

🔎 AI Research

Verus-SpecGym: An Agentic Environment for Evaluating Specification Autoformalization

AI Lab: CMU & Amazon

Summary: To address the challenge of evaluating whether AI agents can accurately translate informal programming intent into formal specifications, the researchers introduce the VERUS-SPECBENCH benchmark and the VERUS-SPECGYM agentic environment. By extending an execution mechanism to test generated specifications against both official tests and adversarial “hacks,” the study reveals that specification autoformalization remains highly brittle even for models capable of generating correct code.

Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players

AI Lab: Tsinghua University, NVIDIA, University of Toronto, & Vector Institute

Summary: Gamma-World presents a scalable, generative multi-agent world model that moves beyond traditional single-agent simulations by utilizing Simplex Rotary Agent Encoding for permutation-symmetric identities and Sparse Hub Attention for efficient cross-agent communication. Through conditional teacher-student distillation and KV-cached streaming, the framework achieves real-time, action-responsive rollouts at 24 FPS that maintain strong consistency across virtual gaming and physical robotic environments.

Self-Improving Language Models with Bidirectional Evolutionary Search

AI Lab: Harvard University & MIT

Summary: Bidirectional Evolutionary Search (BES) overcomes the limitations of sparse verification signals and narrow autoregressive expansion by coupling forward candidate evolution with backward goal decomposition. By recombining trajectory segments to escape narrow probability distributions and scoring them against fine-grained sub-goals, BES significantly outperforms existing open-source frameworks on complex logical reasoning and open problem-solving tasks.

MobileMoE: Scaling On-Device Mixture of Experts

AI Lab: Meta AI

Summary: MobileMoE introduces a family of sub-billion active-parameter Mixture-of-Experts (MoE) language models specifically optimized for efficient deployment on edge devices like smartphones. Guided by a novel on-device scaling law and supported by a custom fused MoE kernel, these models achieve state-of-the-art performance while delivering substantially faster prefill and decode speeds compared to dense baselines at a similar memory footprint.

Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini

AI Lab: Google DeepMind

Summary: Gemini Embedding 2 is a native multimodal embedding model that seamlessly maps text, image, audio, and video inputs into a single, unified representation space without relying on intermediate transcriptions. Trained via large-scale contrastive learning in a multi-task setup, the model establishes new state-of-the-art performance across unimodal, cross-modal, and multimodal retrieval benchmarks while demonstrating robust zero-shot generalization across diverse enterprise and specialized domains.

The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence

AI Lab: MiniMax

Summary: The MiniMax-M2 series introduces a highly efficient 229.9B parameter Mixture-of-Experts model that activates only 9.8B parameters per token, specifically engineered for complex, long-horizon agentic workflows. By leveraging agent-driven data pipelines, a specialized reinforcement learning system called Forge, and autonomous self-evolution capabilities, the model achieves frontier-level performance across coding, deep search, and reasoning benchmarks while maintaining a minimal computational footprint.

🤖 AI Tech Releases

Claude Opus 4.8

Anthropic released the new version of its marquee model, with strong agentic and coding capabilities.

📡10 AI News You Need to Know About

1. Anthropic raises $65B in Series H at $965B post-money valuation — Anthropic raised $65 billion in Series H funding (co-led by Altimeter, Dragoneer, Greenoaks, and Sequoia) at a $965 billion post-money valuation, disclosing that run-rate revenue crossed $47 billion earlier this month, and bringing on Micron, Samsung, and SK hynix as strategic memory/storage partners alongside $15B in previously committed hyperscaler investment (including $5B from Amazon).

2. Cognition raises $1B at $25B pre-money valuation — Cognition, maker of the AI software engineer Devin, raised more than $1 billion (led by Lux Capital, General Catalyst, and 8VC) at a ~$26B post-money valuation, more than doubling in eight months as it hit a $492M annualized revenue run-rate.

3. Robinhood lets AI agents trade stocks — Robinhood launched Agentic Trading and an Agentic Credit Card in beta, letting customers connect third-party AI agents (via MCP) to a separate, funded account to autonomously trade equities and make purchases. Original source:

4. OpenRouter doubles valuation to $1.3B — The multi-model AI inference-routing startup raised a $113M Series B led by Alphabet’s CapitalG at a ~$1.3B valuation, more than double its level a year ago, as weekly volume grew from 5T to 25T tokens.

5. Hark raises $700M Series A — Brett Adcock’s secretive AI startup raised $700M at a $6B post-money valuation (led by Parkway Venture Capital) to build a “universal” agentic AI assistant with proprietary multimodal models and custom hardware, with first models due summer 2026.

6. Mistral signs Airbus and BMW — Mistral AI expanded into “physical AI” for manufacturing, announcing partnerships to apply its models to Airbus (aircraft design, flight safety, defense/space) and BMW’s “Large Industry Model” crash-simulation initiative, plus a new French data center.

7. MiniMax doubles sales ahead of new model — The Chinese AI developer’s annualized revenue more than doubled over two months to at least $300M, driven by its M2.7 model and a fivefold jump in enterprise users, ahead of its next flagship launch. No primary source to substitute: the figures come from a Bloomberg Television interview with co-founder Yun Yeyi, so Bloomberg is the original source.

8. SK Hynix joins the $1 trillion club — Shares of the South Korean memory maker surged ~9–15% to push its market value above $1 trillion for the first time, driven by HBM demand for AI, joining rivals Samsung and Micron. No company announcement exists for a stock-price milestone; the event was originated on the wires by Reuters: https://money.usnews.com/investing/news/articles/2026-05-26/sk-hynix-joins-1-trillion-club-after-samsung-micron-on-ai-chip-boom

9. Pope Leo warns AI shouldn’t dominate humanity — In his first encyclical, Magnifica Humanitas, Pope Leo XIV called for “disarming” AI to keep it human-friendly and free of monopolistic control, warning it risks deepening inequality and eroding human agency. Original source: the encyclical itself, published by the Vatican (vatican.va) — that’s the underlying document the coverage is based on.

10. Snowflake signs $6B AWS deal for Graviton chips — Snowflake committed $6B over five years to AWS — its largest infrastructure commitment ever — expanding use of Amazon’s ARM-based Graviton CPUs and GPUs to power agentic AI workloads.

For most of the modern AI era, progress has had a deceptively simple recipe: make the model bigger, train it on more data, and spend more compute. This formula produced the Transformer era, the foundation model era, and the current wave of large language models. Scaling laws gave the field an almost industrial rhythm. Loss curves became roadmaps. Compute budgets became strategy. The frontier could often be described by a single question: how much larger can we go?

But the most interesting recent progress in AI is beginning to feel less linear. It is no longer just about building a larger model that gives a better answer in one pass. Increasingly, the frontier is about models and systems that can revisit, revise, search, simulate, critique, and improve. The important unit of computation is shifting from the forward pass to the loop.

That shift suggests a provocative idea: recursion may be the next scaling law.

Read more

There is a particular sleight-of-hand at the heart of modern LLM reasoning that, the more I look at it, the more it bothers me. The argument goes like this: Transformers are shallow. A 70-layer stack is fixed depth — it sits in complexity classes like AC⁰ or TC⁰, which is a polite way of saying it cannot, in a single forward pass, solve problems that fundamentally require sequential computation. So we paper over this by making the model think out loud. We give it a scratchpad. We call it Chain-of-Thought. We celebrate.

But CoT is not reasoning. CoT is the model renting depth from its own output tokens. Every reasoning step has to leave the residual stream, become a discrete token in a vocabulary built for human communication, and come back in through the embedding layer for the next step. It is, mechanically, an absurd way to do internal computation — like a CPU that must spill every intermediate register to disk in plaintext English.

Sapient Intelligence’s bet, made first with the original Hierarchical Reasoning Model paper last summer and now extended into the language domain with HRM-Text, is that this is fixable. Not by making the model bigger, not by training on more CoT traces, but by giving the architecture the one thing it doesn’t have: variable, internal, depth. Reasoning that happens in the latent space, not in the token stream.

It’s worth thinking carefully about what they did and what it does and doesn’t yet prove.

Sapient, briefly

Read more

💡 AI Concept of the Day: Three Text Diffusion Models You Need To Know About

For most of the LLM era, language generation has been built around a single assumption: text should be produced like a typewriter, one token at a time, left to right, each new symbol conditioned on a frozen history. Text diffusion models challenge that assumption at its root. They treat generation less like typing and more like editing: start from noise or masks, look at the whole canvas, and iteratively refine it into coherent language.

That sounds like a stylistic tweak. It is actually a different computational worldview. Instead of factorizing language as “the next token given all previous tokens,” diffusion models define a corruption process and then learn how to reverse it. In language, that usually means masking tokens or pushing text into noisier latent states, then training a model to recover the original sequence over several denoising steps. The result is a system that can update many positions at once, use bidirectional context during generation, and revisit its own outputs rather than committing irreversibly at every step.

If you look at the field today, three systems define the conversation more than any others: LLaDA, which proved that diffusion can scale into a real large language model; Mercury, which turned diffusion into a genuine commercial speed advantage; and Gemini Diffusion, which signaled that frontier labs see this paradigm as strategically important. Together, they outline the three phases of a new architecture class: scientific proof, industrial deployment, and frontier validation.

LLaDA: The Scientific Proof That Diffusion Can Scale

Read more

Next Week in The Sequence:

1. We continue our series about alternatives to transformers.

2. In the AI of the week, we are going to deep dive into the new HRM small models, you need to know about those.

3. The opinion of the week we dive into the idea of recursive models as a new AI scaling law.

Subscribe and don’t miss out:

📝 Editorial: Last Week in AI: Last Week in AI: Karpathy, Google, Colossus, and the Coming IPO Wave

The last three weeks felt like a phase transition. Not the kind you measure in benchmark deltas — the kind where the substrate underneath frontier AI quietly rearranges itself, and the org charts and cap tables that fall out look nothing like what we had in April.

Google set the tone at I/O. Sundar called it the “agentic Gemini era,” and for once the marketing copy matched what shipped. Gemini Omni landed as the headline — an any-to-any generative model anchored on video, a real step forward in multimodal editing and world understanding — but the more consequential release was Gemini 3.5 Flash paired with Google Antigravity, their agent-first development platform. The pitch flipped from “AI that helps you write” to “agents that help you act.” Stack the new TPU 8i underneath, and Google now owns a vertically integrated agentic pipeline from silicon up through the IDE. It is the most coherent agent story any frontier lab has shipped this year.

Then, on May 19, Andrej Karpathy joined Anthropic. He’ll work on pretraining under Nick Joseph, and the official framing is that he’ll build a team “using Claude to accelerate pretraining research.” Read it carefully. An OpenAI co-founder is now constructing the loop that improves the next Claude using the current Claude. The self-improvement flywheel every lab has been sketching on whiteboards for two years just got the best person in the world to actually compile it. The signal-to-noise on this hire is unusually high — and Anthropic’s broader pattern of pulling CTOs of billion-dollar companies into individual-contributor research seats is the under-noticed sub-story.

That hire makes more sense when you read the compute side of the ledger. Two weeks earlier, on May 6, Anthropic announced full access to xAI’s Colossus 1 — more than 300 MW, roughly 220,000 H100, H200, and GB200 GPUs — to relieve the pressure that had been forcing Pro and Max rate caps. The price tag surfaced a fortnight later in SpaceX’s S-1: $1.25B per month through May 2029, roughly $45B total, with planned expansion to Colossus 2 and a stated interest in orbital compute. The competitor-as-supplier topology is new, and the unit economics of an inference workload now include “what megawatts can I lease from a launch company?”

The public markets were already pricing that shift. On May 14, Cerebras went public on Nasdaq, pricing at $185, opening at $350, and closing day one near a $95B market cap — the largest tech IPO since Uber, with an order book reportedly 20x oversubscribed. The headline is the pop; the interesting line items are inside the S-1. A $24.6B backlog. A multi-year contract with OpenAI worth more than $20B for 750 MW of inference capacity. A binding term sheet from AWS for CS-3 systems. Two UAE-linked customers account for roughly 86% of 2025 revenue, so the concentration risk is real, but the signal from public markets is unambiguous: specialized inference silicon, sold as long-dated capacity, now prices like infrastructure, not like a chip company. Cerebras was the dress rehearsal.

The main event is the IPO triple-stack behind it. SpaceX’s S-1 points toward a $1.75–2T listing. OpenAI is reportedly filing confidentially in days at $850B–$1T. Anthropic is targeting October at roughly $900B. Three of the largest tech IPOs in history, all gated on the same underlying compute substrate, all hitting public markets inside a six-month window. We are about to get a real-time, mark-to-market valuation function for the frontier itself.

The takeaway, for me, is that the frontier is no longer just a research artifact — it is a vertically integrated capital structure. Compute is a tradeable supply contract. Talent is liquid across labs. The moats are no longer “who has the best loss curve” but “who can fund $45B compute leases, recruit the people who can compress those leases, and keep a public-equity story coherent under quarterly scrutiny.”

The next eighteen months won’t be decided on a benchmark. They’ll be decided on a balance sheet.

🔎 AI Research

HRM-Text: Efficient Pretraining Beyond Scaling

AI Lab: Sapient Intelligence & MIT

Summary: This paper introduces HRM-Text, an efficient pretraining paradigm that replaces standard Transformers with a dual-timescale Hierarchical Recurrent Model (HRM) to drastically reduce the compute and data required for training large language models. By combining this architecture with a task-completion objective trained exclusively on instruction-response pairs, the model achieves competitive benchmark performance using up to 900x fewer tokens than contemporary foundation models.

On the limits and opportunities of AI reviewers: Reviewing the reviews of Nature-family papers with 45 expert scientists

AI Lab: Carnegie Mellon University

Summary: This study evaluates the practical effectiveness of AI-generated peer reviews by having 45 domain scientists manually grade 2,960 individual criticisms from 82 Nature-family papers. The findings reveal that while frontier AI models can surface highly significant and well-evidenced critiques, they often lack subfield-specific context and overlap heavily with one another, suggesting they are currently best utilized to augment rather than replace human reviewers.

Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding

AI Lab: NVIDIA

Summary: This technical report presents Nemotron-Labs-Diffusion, a language model trained with a joint objective that seamlessly unifies autoregressive (AR), diffusion, and self-speculation decoding capabilities within a single architecture. The authors demonstrate that AR and diffusion training are complementary, allowing the model to utilize self-speculation—where diffusion drafts and AR verifies—to achieve superior throughput and efficiency across various deployment scenarios without relying on multi-token prediction (MTP) methods.

Stitched Value Model for Diffusion Alignment

AI Lab: Google & ETH Zürich

Summary: This paper introduces StitchVM, a lightweight model stitching framework that attaches a frozen diffusion backbone to a pretrained pixel-space reward model to efficiently create strong value models for noisy latents. By directly evaluating noisy latents rather than relying on costly Tweedie or Monte Carlo approximations, StitchVM significantly accelerates both inference-time and training-time diffusion alignment methods while maintaining or improving generation quality.

Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learning

AI Lab: University of Illinois Urbana-Champaign & Meta

Summary: This paper introduces Spreadsheet-RL, an on-policy reinforcement learning framework designed to train specialized AI agents for complex spreadsheet workflows within a realistic Microsoft Excel environment. By combining an automated data collection pipeline with a structured tool harness, the framework significantly improves the ability of open-source models to execute multi-step spreadsheet tasks across general and domain-specific benchmarks.

🤖 AI Tech Releases

Google AI Announcements

Google has plenty of AI announcements at it I/O conferences including the amazing Gemini Omni.

Qwen Max 3.7

Qwen open sourced its latest marquee model.

MagenticLite, MagenticBrain, Fara1.5

Microsoft released MagenticLite, an agentic app for browser and file system use together with MagenticBrain and Fara, which are small models optimized for computer use tasks.

📡10 AI News You Need to Know About

1. Hark raises $700M Series A at $6B valuation — Brett Adcock’s stealthy AI lab Hark closed a $700M Series A at a $6B post-money valuation led by Parkway Venture Capital, with Nvidia, AMD Ventures, Qualcomm Ventures, Salesforce Ventures and others, to build multimodal personal-AI models (this summer) and bespoke “universal interface” hardware to follow. →

2. NanoCo (NanoClaw) raises $12M seed, declines ~$20M buyout — The Cohen brothers’ security-focused, sandboxed OpenClaw alternative NanoClaw closed an oversubscribed $12M seed led by Valley Capital Partners (with Docker, Vercel, monday.com, Slow Ventures, and Hugging Face’s Clem Delangue) six weeks after launching the open-source project that went viral via Karpathy and Singapore’s foreign minister. → No clean primary source — TC has the exclusive interview; original TC link is the canonical version.

3. Andrej Karpathy joins Anthropic pre-training — Karpathy joined Anthropic to start a team using Claude to accelerate pre-training research under Nick Joseph, returning to frontier LLM R&D after Tesla, OpenAI, and Eureka Labs.

4. Anthropic acquires Stainless — Anthropic acquired Stainless, the SDK-generation startup whose tooling has powered every official Anthropic SDK and is also used by OpenAI, Google, Replicate, Runway and Cloudflare; Anthropic will wind down all hosted Stainless products, taking the SDK tooling exclusive (deal terms undisclosed; The Information had pegged the price north of $300M).

5. Ocean emerges from stealth with $28M for agentic email security — Israeli founders Shay Shwartz and Oran Moyal launched Ocean out of stealth with $28M total funding led by Lightspeed (with Picture, Cerca, and angels Assaf Rappaport, Yevgeny Dibrov, Nadir Izrael), positioning its multi-agent email investigation platform against AI-generated phishing.

6. Manus weighs $1B raise to unwind Meta takeover — Manus’s three Chinese co-founders (Xiao Hong, Ji Yichao, Zhang Tao) are exploring raising ~$1B from external investors at a valuation matching the >$2B Meta paid, possibly with personal capital, to comply with Beijing’s order to unwind the December deal — with a Chinese JV structure and Hong Kong IPO as the likely next steps.

7. SpaceX files publicly for Nasdaq IPO under SPCX — SpaceX publicly filed its S-1 on May 20 for what would be the largest IPO ever (~$75B target raise at ~$1.75T valuation), listing on Nasdaq as SPCX, with a super-voting structure to keep Musk in control despite multi-billion-dollar losses and a $41.3B accumulated deficit.

8. OpenAI preparing IPO filing in days or weeks — Per WSJ (Bloomberg confirming via its own source), OpenAI is working with Goldman Sachs and Morgan Stanley on a confidential S-1 draft, potentially filed as soon as this Friday and targeting a September public debut at a valuation north of $850B — moving fast after the Musk lawsuit was dismissed on statute-of-limitations grounds. → No clean primary source — both the WSJ original and Bloomberg’s confirmation rely on anonymous sources. Bloomberg link stands; the WSJ original is paywalled and not easily linkable.

9. Exa raises $250M Series C at $2.2B — Exa Labs raised a $250M Series C led by Andreessen Horowitz at a $2.2B valuation (more than triple its $700M valuation from last fall) to scale its agent-optimized web search API, train its next-gen retrieval models, and handle hundreds of thousands of searches per second across its 500B+ URL index.

The next phase of AI agents will not be defined only by better models, longer context windows, or more elegant tool-calling APIs, but by something much more primitive: access to a computer. An agent that can only emit tokens is a brilliant brain in a jar; an agent with a filesystem, terminal, browser, network, package manager, credentials, memory, and guardrails becomes a worker inside a real execution environment. This is the core thesis: every serious AI agent needs a computer, not metaphorically, but architecturally. It needs a safe, isolated, programmable space where it can write code, run commands, inspect outputs, manipulate files, browse the web, recover from errors, and iterate through the same feedback loops that make software useful. The emerging market for micro-containers, sandboxes, browser runtimes, and agent workspaces is really the market for giving intelligence a body.

The Brain in the Jar Problem

Read more

For this week in AI’s essay, I would like to discuss Thinking Machines’ work on interactive models which takes multi-modality to a new level. I’ve been diving as much as possible on their ideas and wanted to share some thoughts. The work is early but so impressive. Check it out to get started:

For the last few years, the default mental model for large language models has been embarrassingly simple: concatenate tokens, predict the next token, repeat. The human writes a message, the model replies, the human writes again. This works surprisingly well for many tasks because text is forgiving. Text can wait. It can be buffered, edited, compressed, and serialized into one neat causal stream.

But collaboration is not text. Collaboration is temporal.

Read more

💡 AI Concept of the Day: What are Text Diffusion Models

If you look at the architecture of the modern AI boom, it is heavily bifurcated by modality. In the visual domain, we are entirely ruled by diffusion models. From Midjourney to Stable Diffusion to OpenAI’s Sora, the paradigm of starting with pure noise and iteratively denoising it into a high-fidelity image or video has proven to be unreasonably effective.

But in the realm of text, diffusion has historically been an afterthought. Large Language Models (LLMs) like GPT-4, Claude, and LLaMA are staunchly autoregressive (AR). They are sequence predictors. They look at the context, predict the next token, append it to the context, and repeat. It is a strictly left-to-right, causal process.

For years, the consensus was simple: autoregression is just the native physics of language. But this sequential paradigm has glaring pathologies. Because AR models generate blindly from left to right, they cannot easily engage in global planning. If they make a slight logical error early in a sequence, that error is committed to the context window permanently, leading to cascading failures—a phenomenon often critiqued as “generation drift.” Furthermore, AR models suffer from the “reversal curse”; they can easily recite a poem forward, but if you ask them to recite it backward, their causal attention mechanisms break down entirely.

Read more

Next Week in The Sequence:

1. We continue our series about trasnformer alternatives.

2. In our opinion section, we discuss the thesis that “every agent needs a computer”

3. In the AI of the Week section, we dive into Thinking Machines’ interactive models which can think and listen and same time.

Subscribe and don’t miss out:

📝 Editorial: Last Week in AI: IPOs, Interactive Models, and Recursive Dreams

This week in AI felt less like a product cycle and more like a philosophical provocation wrapped in a market event, a demo, and a few delightfully ambitious lab announcements. The common thread was not bigger models, larger context windows, or yet another benchmark victory. It was agency. Who gets to shape intelligence? Who gets to improve it? And, slightly more ominously, what happens when the tools begin improving the tools?

Start with Cerebras. The IPO was not just a financing milestone for an AI chip company; it was a reminder that the AI race is still brutally physical. Behind every magical chatbot sits an industrial stack of silicon, power, networking, cooling, capital markets, and geopolitical anxiety. Cerebras has always been the wonderfully weird character in the AI hardware opera: instead of making chips modestly larger, it went all-in on the wafer-scale computer, basically asking, “What if the chip were the data center?” That sounds like something invented after too much espresso at a semiconductor conference, but it captures an important truth. If intelligence is becoming a commodity, compute remains the refinery. Public markets are now voting on which refineries matter.

Then came Thinking Machines with its interaction models, and the shift in mood was immediate. While much of the industry keeps celebrating autonomous agents that disappear into the cloud and return with a spreadsheet, Thinking Machines is betting on the opposite: AI that stays present. Not “prompt, wait, receive,” but continuous, multimodal collaboration. The model listens, watches, interrupts, yields, and reacts in real time. In other words, it treats interaction not as a UI layer bolted onto intelligence, but as part of intelligence itself. This is a subtle but profound idea. Human cognition is social before it is computational. We think through gestures, pauses, corrections, shared context, and awkward silences. Making AI more useful may require making it less like an oracle and more like a very fast, very attentive collaborator who knows when not to talk.

The most intellectually spicy developments, however, came from the new “AI scientist” movement. Recursive emerged with the grand ambition of building systems that improve themselves through automated experimentation. Adaption’s AutoScientist pointed in a similarly important direction, automating the loop behind training, alignment, and model adaptation. The dream here is intoxicating: research no longer as a linear human process, but as a compounding machine of hypotheses, experiments, evaluations, and refinements.

That dream also deserves a raised eyebrow. Recursive self-improvement has long lived somewhere between computer science, science fiction, and existential risk seminar. Turning it into a venture-backed product category is both thrilling and slightly unhinged — which, frankly, is how many important technologies begin. The question is not whether AI can help with research. It already does. The question is whether we can build systems that improve capabilities without eroding grounding, safety, diversity, or human judgment.

Another signal came from China, where Junyang Lin, the former lead researcher behind Alibaba’s Qwen models, is reportedly raising several hundred million dollars for a new AI lab at a valuation that could land around $2 billion. That number is almost rude for a brand-new lab — especially in China, where AI startup valuations tend to be more earthbound than in the U.S. — but Lin has the rare credential that matters in frontier AI: he has already helped turn a model family into an ecosystem. His departure also exposes a delicious tension inside Chinese AI: as Alibaba shifts more energy toward proprietary models and monetization, the open-source talent orbit may be ready to spin out into independent labs. But this is not simply a “raise money, buy GPUs, train model, repeat” story. New Chinese AI labs must navigate U.S. chip export controls, constrained compute access, and the tricky problem of building research agendas that do not merely overlap with giants like Alibaba and ByteDance. In that sense, Lin’s new lab is less another shiny founder myth than a test case for whether China’s open-model momentum can become a venture-scale company.

The machine is no longer just answering. It is listening, adapting, experimenting — and, perhaps, beginning to ask what it should become next.

🔎 AI Research

MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory

AI Lab: Rutgers, Notre Dame, Princeton, UMN, SAMD

Summary: MemEye is a novel evaluation framework and benchmark designed to assess whether multimodal AI agents can effectively preserve and reason over fine-grained visual evidence in long-term interactions. By evaluating 13 memory methods, the authors reveal a critical trade-off where text-based memory loses visual details while native image memory struggles with tracking temporal state changes.

EVOLVEMEM: Self-Evolving Memory Architecture via AutoResearch for LLM Agents

AI Lab: UNC-Chapel Hill, UC Berkeley, UCSC

Summary: EVOLVEMEM introduces a self-evolving memory architecture that uses an LLM-powered diagnosis module to autonomously optimize its own retrieval configuration based on failure logs. This closed-loop process allows the system to discover and refine effective retrieval strategies dynamically, significantly outperforming static baseline models on long-term memory benchmarks.

Orchard: An Open-Source Agentic Modeling Framework

AI Lab: Microsoft Research, Columbia University, UIUC

Summary: Orchard is an open-source framework built around a lightweight, Kubernetes-native environment service that decouples sandbox management from agent harnesses to enable scalable, cost-effective agentic training. The framework includes specialized training recipes for software engineering, web navigation, and personal assistant tasks, demonstrating state-of-the-art performance and strong generalization across diverse agent domains.

NEXUS: An Agentic Framework for Time Series Forecasting

AI Lab: Google, Pennsylvania State University

Summary: NEXUS is a multi-agent framework that improves time series forecasting by explicitly decomposing the task into macro-level trend projection, micro-level granular analysis, and dynamic synthesis of numerical and textual contexts. By separating these reasoning stages and incorporating a calibration loop, the system matches or exceeds the accuracy of dedicated numerical foundation models while providing highly interpretable forecasting rationales.

GLiGuard: Schema-Conditioned Classification for LLM Safeguard

AI Lab: Fastino Labs

Summary: GLiGuard is a highly efficient, 0.3B-parameter bidirectional encoder designed for real-time LLM content moderation by encoding task definitions and label semantics directly into the input schema. This non-autoregressive architecture evaluates prompt safety, response safety, harm categories, and jailbreak strategies simultaneously in a single forward pass, achieving competitive accuracy with models up to 90 times larger while drastically reducing inference latency.

🤖 AI Tech Releases

Interaction Models

Thinking Machines unveiled a research preview of Interaction Models: frontier models in which interactions is built in the model and not in the harness.

📡10 AI News You Need to Know About

1. Cerebras IPO debut : Cerebras priced its IPO at $185/share to raise $5.55B, and shares then soared 68% in their Nasdaq debut on May 14, pushing the AI chipmaker’s market cap to roughly $95B.

2. Recursive Superintelligence emerges from stealth: Former Salesforce chief scientist Richard Socher launched Recursive Superintelligence with $650M at a $4.65B valuation — led by GV and Greycroft with Nvidia and AMD Ventures participating — to build AI systems that recursively improve themselves through open-ended algorithms, starting with automating AI research itself.

3. OpenAI weighs suing Apple: OpenAI has hired an outside law firm and is considering legal action against Apple over the ChatGPT-Siri integration, which it claims has been buried in the UI and has fallen far short of projected subscription revenue.

4. Cisco cuts ~4,000 jobs while posting record revenue: Cisco is cutting roughly 5% of its workforce to redirect spending toward AI and cybersecurity, even as it touts record quarterly revenue and double-digit growth.

5. Adaption launches AutoScientist: Sara Hooker’s Adaption Labs unveiled AutoScientist, an automated fine-tuning system that co-optimizes data and model recipes and which the company says outperformed its in-house researchers by 35% (lifting win-rates from 48% to 64%), free for 30 days.

6. Musk v. Altman closing arguments: Lawyers for Musk and OpenAI delivered closing arguments in Oakland federal court on May 14, with the nine-person jury set to begin deliberating Monday on whether Altman and Brockman breached commitments to keep OpenAI a nonprofit.

7. xAI launches Grok Build: xAI rolled out an early beta of Grok Build, a new terminal-based coding agent and CLI for SuperGrok Heavy subscribers ($300/month), as Musk tries to close the coding-quality gap with Anthropic’s Claude.

8. Nebius Q1 sales up 684%: Nebius reported Q1 2026 revenue of $399M (up 684% YoY) and adjusted EBITDA of $129.5M, and announced it has secured up to 1.2 GW of power and land for a new owned AI factory in Pennsylvania.

9. Wispr in funding talks at $2B: Wispr AI, maker of the Wispr Flow voice dictation tool, is in talks for a ~$260M round led by Menlo Ventures that would value the company near $2B — nearly triple its $700M valuation from late 2025.

10. Former Alibaba Qwen lead launches new AI lab at ~$2B valuation : Junyang Lin, the former lead researcher behind Alibaba’s open-source Qwen models, is in talks with Chinese VCs Gaorong Ventures and HongShan to raise several hundred million dollars for a new AI lab at a roughly $2B post-money valuation — a level almost unheard of for a brand-new Chinese AI startup.

For today’s essay, I want to explore an idea that has become central to how we think about AI evaluations at LayerLens. This is not an essay about LayerLens, but about a simple and increasingly unavoidable thesis: evals are becoming the fourth pillar of modern AI, alongside compute, data, and models. As AI systems move from chatbots to agents, from demonstrations to production workflows, every meaningful task performed by every agent inside every company will need its own evaluation layer. Not generic benchmarks. Not leaderboard theater. Practical, dynamic, company-specific exams that measure whether an AI system can actually survive contact with real work. I call this idea Every Company’s Last eXam.

Humanity’s Last Exam is a very specific kind of artifact. It is what a field builds when the old report card stops working. The core observation behind it was simple: familiar benchmarks such as MMLU had become too easy for frontier systems to cleanly separate the leaders, so researchers assembled a harder, broader, multimodal test at the frontier of human knowledge, finalized at 2,500 questions after removing errors and questions that were too easily answerable with search. And then, almost immediately, the benchmark itself taught a second lesson: even “the last exam” needs maintenance. HLE-Verified later showed that noisy items and flawed answers could materially distort comparisons, and that systematic verification could shift measured accuracy by 7 to 10 percentage points on average. In other words, the benchmark was not a stone tablet. It was infrastructure.

That is the right analogy for where enterprise AI is going. Every company now needs its own last exam: a private, living evaluation suite that captures the highest-value, highest-risk, most context-heavy work its agents are supposed to perform. Not a generic IQ test for models. Not another public leaderboard. More like a company-specific CI system for cognition. The public benchmarks still matter, just as SPEC mattered for CPUs and ImageNet mattered for vision, but production truth has moved downstream into proprietary workflows, private documents, internal policies, odd exceptions, and all the sharp edges that never make it into a paper appendix. That is why top frontier labs now emphasize task-specific evals, production-derived datasets, continuous maintenance, and explicit definitions of success rather than vibe-based model selection.

The fourth pillar

Read more