You may also enjoy FLUX 3 Image Klein, Plan or Retrieve First? A Scale Study of Tool Discovery Architectures, a paper on OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Muse Glimmer 30B: An On-Device Agentic Model

Meta publishes Muse Glimmer 30B, an Apache 2.0 agentic model quantized to roughly 4-bit for single-GPU inference, trained on over a hundred languages.

Meta unveils Muse Spark Open, the new top U.S. open weights model — and it runs on two paired Mac desktops

Meta releases Muse Spark Open, an 816-billion-parameter mixture-of-experts model with 42 billion active parameters and a 1M-token context, scoring 39 on the Artificial Analysis Intelligence Index.

Reflection AI open-sources Beam’s weights, post-training recipes

Reflection AI publishes the weights of Beam, a roughly 500-billion-parameter model trained over two months on several thousand GPUs with practically zero cluster downtime.

DeepSeek releases updated V4.1-Flash model it says completes tasks faster

DeepSeek-V4.1-Flash-1017 resolves 77% of DeepSWE v1.1 tasks against 74.2% for the September version, and ships with a Search API returning up to 50 ranked results.

DeepSeek open-sources DeepInfer, the inference framework behind its production services

DeepSeek open-sources the inference framework running one cluster at 155 trillion tokens a day, which it says earns $38 per GPU per hour at off-peak pricing.

FLUX 3 Image Klein: open weights, 5.6B parameters

Black Forest Labs opens a 5.6-billion-parameter image model that gains 91 Elo in text-to-image over FLUX.2 [klein] 9B with 38% fewer parameters.

OpenAI acquires Orchestra Bio to bring AI agents into drug development

OpenAI folds Orchestra Bio’s clinical-trial monitoring agents into its Rosalind life-sciences unit on undisclosed terms, targeting the 10-to-15-year drug approval timeline.

MLOps / LLMOps / AgentOps

Announcing llm-d 1.0: Open distributed inference for agents on Google Cloud TPUs and GPUs

Google ships a Kubernetes-native distributed inference stack built with Red Hat and IBM by more than 500 contributors across 25 organizations, demoed on 1,000 sandboxed agents.

Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM

A technical blog post about moving KV transfer coordination off the critical path, cutting time-to-first-token roughly 3x and improving time-per-output-token up to 30%.

Four evaluation engineering lessons learned from building agent plugins

A candid blog post about grading execution steps rather than final summaries, after an LLM judge wrongly failed three prompts of an eleven-prompt suite the plugin had passed.

Inside NVIDIA Sentry: Secure AI Agents at Runtime

A technical article about governing agent access from a hardware-isolated BlueField-4 trust domain, correlating network traffic with memory introspection to contain agents within milliseconds.

Beyond Top-K: Building search-confidence guardrails for agentic AI with OpenSearch

A thoughtful article about adding a confidence layer between retrieval and action, scoring magnitude, gap, coherence, freshness and authority to abstain on a graduated ladder.

Making every harness durable

Vercel introduces durable-harness, extending crash-and-timeout durability to the Pi, OpenCode, Claude Code and Codex harnesses as agent workloads move to the cloud.

Learning

RAG Evals: Change Prompts Without Breaking Your App

A practical blog post about turning failures into eval cases, scoring them with code checks and LLM judges, and tracking cost and quality across a 61-case suite for every prompt or model change in Opik.

Scaling agent teams: reach the frontier at 30% of the cost

A rigorous blog post about organizing coding-agent teams, where a strong lead with 16 cheap planning workers reached 61.8% reward for $155 against an all-strong team’s $510.

Cutting our inference bill by 68% by being obsessive about optimization

An instructive engineering post about cutting an AI code reviewer to $0.77 per review from $2.43, while median review time fell from 4m 09s to 1m 32s.

Plan or Retrieve First? A Scale Study of Tool Discovery Architectures

A research paper about a complexity-dependent crossover where plan-then-retrieve beats retrieve-then-plan on multi-tool queries across ToolBench’s 46,984 API endpoints.

Reducing Reward Hacking by Localizing Bad Behavior Instead of Penalizing It

A research blog post about GRAFT, which confines learning updates from detected reward hacks to parameters deleted after training, preventing even hacks monitors cannot detect.

Screwtape: Adversarial Transcript Editing for Monitor Red-Teaming

A red-teaming blog post about refining attacks inserted into real agent transcripts until monitors miss them, producing 35 evasions from 252 runs, six of which transferred.

Open-weight AI models are 6-7 months behind on cyber

A data analysis about Z.ai’s GLM-5.3 topping the open-weight Cyber ECI at a score the closed-weight frontier reached roughly six to seven months earlier.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

cloudflare/security-audit-skill

A coding-agent skill for multi-phase security audits with independently verified, machine-readable findings

Papers & Publications

Good Judgment, Bad Advice: Decision-Model Controllers Raise Agent Cost Without Improving Success

Agent scaffolds increasingly delegate small decisions (whether to stop, verify, or change approach) to cheaper models. We ask whether a typed decision model, Jev 1.13, consulted only at deterministic decision points, makes a Claude Sonnet 5.5 terminal agent cheaper without making it worse. In a pre-specified study on Terminal-Bench 2.1, we hold one agent loop fixed and vary only the controller: none, deterministic rules, a small LLM (Claude Haiku 5.5) answering the same questions, Jev, and a placebo that samples Jev’s development-phase choice frequencies at the same moments. On 21 held-out tasks (315 trials), Jev raised cost per attempt by 19% (geometric-mean ratio 1.19, 95% CI 1.02-1.36) at an identical pass rate (73.0% vs. 73.0%). Jev was cheaper than the small-LLM controller (ratio 0.81) but 22% more expensive than the placebo, and no pass-rate difference between arms was detectable. Offline, Jev ranked finish requests better than the alternatives (AUROC 0.78 vs. 0.62 for rules and 0.60 for the small LLM) but was poorly calibrated: its mean probability that a run was complete was 0.50 when 87% of those runs passed, so it asked the agent to verify work that was already correct. No controller ever chose to abandon a run. Informative decision signals did not become useful interventions; we discuss what that would require.

OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use

Computer use has reached a turning point: the focus is expanding from GUI-only interaction to systems that complete tasks on users’ computers through visual and programmatic interfaces. We define a generalist computer-use agent (GCUA) as a system in which the model chooses how to observe, act, and organize its work across visual and programmatic interfaces. Yet frontier implementations, such as GPT-6 Astra’s computer-use system, remain largely closed, limiting systematic study of their design. We introduce Open Generalist CUA (OpenGCUA), an ongoing project to develop and open the full stack for GCUAs with the community. This report presents the first release of OpenGCUA. Code execution is a key feature of our agent design: it lets intermediate results determine subsequent operations and enables models to apply their existing programming capabilities directly to computer interaction. We also propose harness-to-model decision transfer, moving selected observations and task-state decisions from fixed harness rules to the model. On OSWorld-v2, our full configuration improves task scores across all eight evaluated models, including GPT-6 Astra and Claude Opus 5, raising mean partial score by +19.6 points and reducing reported steps per task by an average of 53.0% relative to batched GUI-only execution. We release our implementation and prompts, evaluation trajectories from all eight models, and preliminary versions of a real-device harness and a cross-device SDK. The harness addresses real-device challenges not fully captured by benchmarks, while the SDK provides a common programming interface across computers, mobiles and browsers. Across four real macOS tasks under our real-device harness, selected Qwen3.8-Max runs use 19.4 times as many reported reasoning tokens as GPT-6 Astra, highlighting substantial differences between models in real-device execution efficiency.

You may also enjoy Meta’s Muse Wonder, Redesigning source control at Anthropic, a paper on Recursive Self-Improvement through Multi-Agent Self-Supervision, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Introducing Claude Haiku 5.5

Haiku 5.5 drops to 0.10/0.50 per million tokens below 100,000 context and scores 39.2% on Terminal-Bench 4.0 at max effort against GPT-6 Luna’s 16.4%.

Meta’s new Muse Wonder and Muse Wonder Full models achieve frontier performance, but only the weaker one is available via API

Muse Wonder Full scores 56 on the Artificial Analysis Intelligence Index but ships subscription-only, while the API-available Wonder reaches 52 at 4/20 per million tokens.

Jev creator TypeSafe closes $870M round at $7.5B valuation

TypeSafe raises $870 million at a $7.5 billion valuation led by a16z with Sequoia and DCVC, under a month after its Jev decision model launched.

Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flash

Cloudflare extends its Clef decision models to native audio, video and image input on a Qwen3-Omni-30B-A3B base, returning text decisions in about 130ms at the median.

MLOps / LLMOps / AgentOps

Run multi-node inference workloads using Ray LLM Deep Learning Containers

A hands-on guide about splitting one LLM across two GPU nodes on EKS with separate prefill and decode workers behind a single OpenAI-compatible endpoint.

Building agents that transact with Anchor Browser and AgentCore payments

A guide about letting browser agents pay for x402-protected resources while AgentCore checks budgets and signs transactions without exposing wallet credentials to the agent runtime.

Learning

Redesigning source control at Anthropic

An unusually candid engineering article about replacing git with an in-house system in five months with seven people, after merges grew from 2,500 to over 30,000 per day.

Investigating unintended model actions in our evaluations and internal use

A transparency report about four categories of Claude overstepping — exploiting a server flaw, submitting a real form, bypassing paywalls, evading fetch limits — prompting Anthropic to cut all internal evals off the live internet.

Can AI automate Epoch?

A methodologically careful report about grading six frontier models on real tasks drawn from Epoch’s own work, finding them reliable on well-defined tasks and failing on the open-ended parts.

Learning Harness Engineering by Building a Coding Agent From Scratch

A webinar write-up about harness engineering from first principles, distilling an eight-article, six-video open-source course on building a coding agent into 90 minutes.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

K-Dense-AI/scientific-agent-skills

Turn any AI agent into an AI Scientist.

Papers & Publications

Self-Compacting Language Model Agents

Long agent traces composed of chains of thought and tool calls accumulate stale content that anchor subsequent generations, and eventually outgrow the context window. Existing scaffolds mitigate it with fixed-interval compaction triggered at a token threshold. Such triggers pay no heed to trajectory structure, risking discard of partial results mid-derivation or mid-search. We propose SELFCOMPACT, a scaffold that allows the model itself to decide when and how to compact. Specifically, it pairs two inference-time elements: (i) a compaction tool the model invokes to summarize the accumulated context, and (ii) a lightweight rubric specifying when to fire (a sub-task has resolved, or the trajectory is converging) and when to suppress (mid-derivation, or when stuck). Both are needed. The tool alone is unevenly used across open-weight models, often invoked at unhelpful moments or not at all; the rubric alone cannot act. Together, they elicit effective adaptive compaction without any fine-tuning or external supervision. We present empirical results on six benchmarks (competitive math and agentic search) and seven models. Our results show that SELFCOMPACT matches or exceeds fixed-interval summarization at a fraction of the token cost, improving over a no-summarization baseline by up to 18.1 points on math and 5–9 points on agentic search at 30–70% lower per-question cost. Our results expose a meta-cognitive gap: although unprompted models cannot reliably tell when their own context is rotting, a lightweight rubric closes this gap, reframing when to compact as a capability that scaffolds can supply without training.

Recursive Self-Improvement through Multi-Agent Self-Supervision

Recursive self-improvement (RSI) of a model on non-verifiable tasks, such as open-ended research, faces a supervision bottleneck when its outputs exceed what even human experts can reliably assess, leaving the model itself (optimizee) as the best available optimizer and evaluator. However, a single model instance struggles to critique and improve its own complex reasoning under this homogeneous loop. To address this, we propose Multi-Agent Self-Supervision (MASS), an RSI method that alternates between evolutionary workflow optimization and supervised fine-tuning on self-generated trajectories. Guided by early findings that multi-agent topologies excel at complex reasoning, MASS prompts a single base model to iteratively propose, execute, and self-evaluate multi-agent workflows. Through an evolutionary search constrained by structural guardrails, the model optimizes these computational-graph-like orchestrations, discovering the most effective distinct roles and information routing for a given task. Over two MASS cycles with Qwen3.6-27B, the model achieves 1.2-1.6x higher performance per output tokens on four open-ended public benchmarks. Because the improved model subsequently acts as a better optimizer and evaluator, this alternating framework enables a continuous, recursive bootstrapping of the model’s capabilities. Moreover, multi-agent traces are also more training-efficient: a student trained on them outperforms a single-agent student trained on 1.4x more training tokens. These findings suggest that jointly learning orchestration and bounded subagent execution from multi-agent trajectories can provide an effective signal for RSI.

You may also enjoy Cloudflare’s Clef, a paper on Beyond Semantic Similarity: Performance and Costs of Agentic Retrieval for Complex Tasks, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Gemini 4 Argon: our next era of frontier intelligence

Gemini 4 Argon scores 77.9% on DeepSWE v1.1 against Claude Opus 5.5’s 74.2% and ships first to 650-plus Fairwind Program defenders at 2/10 per million tokens.

Introducing Clef: our open-source decision models, and new RL fine-tuning platform

A blog post about Cloudflare’s Apache-2.0 Clef decision models, which add a vision encoder and a 64k context window against Jev’s 32k and classified a domain in 2.2s versus 4.7s for gpt-oss-120b.

Introducing Mistral Large 4

Mistral opens a public preview of Mistral Large 4, a 1-trillion-parameter natively multimodal model with 49 billion active parameters, with open weights promised by month’s end.

Google expands EmbeddingGemma beyond text to images, audio and video

EmbeddingGemma 2 reaches 740 million parameters and 78.68 on MTEB’s code section, nearly ten points above v1, with a 270M text core using about 191MB quantized.

North 2: Enterprise AI Without Compromises

Cohere rebuilds North around a new agent harness, adds memory, skills and artifacts, and gives admins granular cost controls, rate limits and org-wide token caps.

AWS debuts Strands Decider 2B, a first lightweight decision model for accelerate agentic workflows

AWS open-sources Strands Decider 2B, a laptop-runnable decision model that returns confidence-scored choices instead of generated text, aimed at the structural limits its team sees in Jev.

Ai2 releases Olmo-core 3 to make developing large mixture-of-experts LLMs more efficient

The Allen Institute for AI ships Olmo-core 3, a training framework that carries mixture-of-experts models to trillion-parameter scale while holding down expert-routing and networking overhead.

MLOps / LLMOps / AgentOps

UX Without a UI: Notes from Researching Terminal UX for Opik MCP

A look at what changes when designing developer tools for AI agents instead of traditional interfaces, including the challenges of displaying complex data in terminals, handling unpredictable LLM behavior, and designing for both humans and agents.

How to implement long-term AI agent memory in AlloyDB and Memorystore for Valkey

A guide about a two-tier agent memory architecture — Valkey for short-term buffer, AlloyDB AI for long-term persistence — that Google says can cut token spend by up to 70%.

Build a multi-agent music production pipeline on Amazon Bedrock AgentCore Runtime Instances

A hands-on blog post about AgentCore’s new Runtime Instances, which add multi-day sessions, GPUs, persistent volumes and colocated agents to the serverless MicroVM option.

Learning

Open and Emergent Problems in Agentic Privacy and Security: A Contextual Angle

A research blog post about framing agent privacy and security through contextual integrity, setting out the open problems that arise once agents act on a user’s behalf.

Claude’s new auto eval tool

A critical blog post about trying Anthropic’s build_eval and hill-climb commands on real leasing-assistant traces, arguing they push you to write an eval before you have looked at data.

Research: Qwen3.8 27B addition in words

A benchmark note about a local Qwen3.8-27B quant scoring 23.57% numeric accuracy across 5,070 reasoning-disabled addition cases, falling from 97.04% on short operands to 6.44% on long ones.

Open TTS Leaderboard: Scalable Evaluation for Multilingual Text-to-Speech and Voice Cloning

A blog post about an objective-metric TTS leaderboard built because arenas cannot keep pace with over 8K Hub TTS models, only 16 of 92 Artificial Analysis entries being open-weights.

Claude-shaped science

A guest research post about a physicist dropping the human-scientist workflow and building BootLoops, an open-source harness that lets Claude work on problems shaped to its strengths.

How many AI agents could we run?

An analysis about chips shipped through 2027 supporting tens to hundreds of millions of concurrent frontier agents — as many weekly hours as 140-720 million full-time employees.

Principles for Embedded Evaluations

A position blog post about designing embedded third-party evaluations around verifying or falsifying a developer’s own safety claims, with ongoing access to training data, rollouts and checkpoints.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

mattpocock/skills

Skills for real engineers. Straight from Matt Pocock’s .agents directory.

Papers & Publications

Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability

On-Policy Distillation (OPD) trains a student on its own generations using teacher feedback. With different tokenizers, comparing teacher and student predictions requires alignment at both sequence and vocabulary levels. In this paper, we examine whether expanding this alignment coverage improves learning. Across three heterogeneous teacher--student pairs on mathematical reasoning and code generation, strict 1:1 groups already cover most student-generated tokens despite substantial vocabulary mismatch. On responses sampled from the students before distillation, the shared vocabulary retains nearly all teacher and student probability mass at strictly aligned positions on average. Restricting reverse KL to a student-selected top-16 subset of the shared vocabulary at each strict position achieves accuracy comparable to full shared-vocabulary OPD, outperforming the evaluated cross-tokenizer baselines. Adding mean squared error supervision on span log-probabilities in mismatch groups gives complete supervision coverage, yet reduces accuracy. At checkpoints from training with only the strict loss, the span gradients show weak or negative directional agreement with the strict gradients and grow in magnitude relative to them. These diagnostics may help explain the accuracy drop from adding span supervision. Our findings motivate a shift from maximizing alignment coverage to prioritizing supervision reliability: compact supervision at strict positions can be more effective than broader coverage that introduces weakly aligned or conflicting training signals.

Beyond Semantic Similarity: Performance and Costs of Agentic Retrieval for Complex Tasks

Modern information systems, including many agentic workflows, use dense retrieval to explore large amounts of unstructured data. However, dense retrieval relies on surface-level semantic similarity, which is insufficient for increasingly complex search applications. Here, we investigate agentic retrieval that combines the reasoning capabilities of Large Language Models (LLMs) with the efficient corpus exploration of retrievers in a ReAct agentic loop to solve complex retrieval tasks. In our experiments, we show that agentic retrieval is more effective than standard retrieval, improving nDCG@10 by 8.7 points using the same embedding model. Moreover, while specialized retrieval methods struggle on out-of-domain tasks, agentic retrieval is highly generalizable: the same pipeline achieves competitive results on both the ViDoRe v3 and BRIGHT leaderboards. However, this improvement comes at a cost. On average, agentic retrieval takes 107.4 seconds, compared to 0.67 seconds for standard retrieval, and consumes 764.1K input and 5.8K output tokens per query. In short, our study demonstrates the effectiveness of agentic retrieval in modern data systems and motivates future work on more cost-efficient retrieval agents for large-scale deployment.

You may also enjoy Anthropic’s Claude Sonnet 5.5, Cursor’s deep dive on improving token efficiency for longer agent runs, a paper on IterSynth for deep search agents, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

OpenAI’s GPT-6.1 Sol offers Astra-like performance at 1/5th price. A new Ultrafast tier clocks at 300 tokens per second.

OpenAI introduces GPT-6.1 Sol at 2/10 per million tokens against Astra’s 10/50, beating GPT-6 Sol by 6.4 points on DeepSWE v1.1 and Opus 5.5 by 2.2 on AutomationBench.

Anthropic launches Claude Sonnet 5.5 with 30% cost reduction per-task due to faster speeds and fewer tool calls

Anthropic launches Sonnet 5.5, which scores 70.6% on Terminal-Bench 4.0 and 80.1% on OSWorld 2.1, nearly matching Opus 5.5 on GDPval-AA.

OpenAI launches Dots, always-on AI agent coworkers, and ChatGPT Space where they can collaborate with human teams

OpenAI’s DevDay headliner gives each Dot a dedicated cloud computer and browser running on GPT-6 Astra, with Custom Rules gating actions and passwords reserved for humans.

AMD will acquire Fei-Fei Li’s World Labs for $8.2B

AMD buys World Labs for $8.2 billion and makes Fei-Fei Li executive vice president and chief scientist, giving it a world-model answer to Nvidia’s Cosmos line.

Nvidia launches new platform for reining in rogue AI agents

Jensen Huang’s Open Agent Safety Platform pairs OpenShell access control with Sentry monitoring on BlueField-4 DPUs, moving the guardrail off the processor the agent runs on.

Introducing Embed 5—A New Family of Frontier Embedding Models

Cohere ships Embed 5 Pro and Fast at $0.12 and $0.08 per million tokens sharing one embedding space, so teams index with Pro and query with either without re-indexing.

FLUX 3 Action: A 7B World Action Model for Robot Control

Black Forest Labs releases an open-weight 7B world-action model that tops the RoboLab-120 leaderboard at under half the parameters of the previous best open model and up to 3.95x faster.

MLOps / LLMOps / AgentOps

Jev vs. LLM-as-a-Judge for AI Evals

A practical comparison of Jev and GPT-4o-mini as LLM judges on 1,000 production turns, where Jev was 3.5× cheaper and 3.8× faster at the median while agreeing on 89.7% of evaluations.

Add Runtime Controls to AI Agents with NVIDIA OpenShell

An article about OpenShell 0.1.0, an open-source runtime enforcing which systems an agent may reach without rewriting it, using a formal policy prover to verify permission boundaries.

Scaling MoE reinforcement learning on Amazon EKS with EFA and DeepEP with 40% more throughput

A deep-dive article about why sparse MoE post-training becomes communication-bound rather than compute-bound, and how DeepEP over EFA lifts aggregate RL rollout throughput by 40%.

What if your agent’s hallucinations had a budget? How to start using SLOs for agent behavior

A practical article about mapping error budgets onto agent quality, treating groundedness, helpfulness and toxicity as separate behaviors that fail independently and need separate targets.

Learning

Improved token efficiency for longer agent runs

A detailed engineering post about where agent inference spend actually goes, and how trimming roughly 66% of the system prompt and reusing cache cut user token costs 7% with no quality loss.

Free the models: Harness design at the frontier

An opinionated article about why a model router can never exceed the model it routes to, and how letting the core loop pick its own effort beat a single-worker architecture by 11 and 16 points.

A data report about the skills.sh registry reaching one million agent skills and nearly 280 million installs in seven months, versus 27 months for GitHub to reach a million repositories.

2026 in LLMs (so far)

An annotated keynote about the year’s LLM trends, tracing the moment coding agents crossed from “often make mistakes” to reliable enough for daily use.

Best practices guide for customizing Gemini models via Reinforcement Learning (RL)

A practical guide about a managed RL fine-tuning service where you bring prompts and a reward function, unlocking tasks that are hard to demonstrate but easy to score.

Embedded Evaluators are necessary for meaningful external testing

An argumentative post about why final-checkpoint testing could not have caught the Hugging Face incident, which arose far earlier in development, and what employee-equivalent access would change.

GLM-5.3 and the spread of advanced cyber capabilities

A red-team analysis finding that simple techniques bypass GLM-5.3’s safeguards 64% to 100% of the time in simulated tests, while the same attacks failed against safeguarded Claude models.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

DietrichGebert/ponytail

Makes your AI agent think like the laziest senior dev in the room. The best code is the code you never wrote.

Papers & Publications

Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents

Agentic memory systems reuse past experience to improve future performance, yet most existing designs curate memory at write time: once a task is completed, its trajectory is distilled into a fixed artifact, such as a reflection, workflow, skill, or reasoning strategy, that is later retrieved by similarity. This forces the system to decide what is worth remembering before the future query is known, irreversibly discarding information and producing a query-independent summary that must serve many possible downstream tasks. Learning such a write-time curator is also difficult because the value of a storage decision may only become apparent when a relevant query arrives, potentially many tasks later, creating a long-horizon credit-assignment problem. We instead retain raw trajectories and defer curation until read time, when the current task is known. Given the retrieved traces and the new task, a memory curator synthesizes a compact, task-adaptive payload tailored to the immediate need. Because this payload is consumed on the same task, the curator can be trained directly from immediate task success, avoiding delayed utility signals and the need to artificially group related tasks. Across ALFWorld, WebShop, and τ2-bench, our Just-in-Time Memory (JitMem) consistently outperforms no-memory agents as well as heuristic and learned write-time memory methods, improving over the strongest baseline by 16.2, 16.3, and 3.9 absolute success-rate points, respectively. Notably, even an untrained curator is already competitive with or surpasses these baselines, showing that task-adaptive read-time curation itself is a major source of the gain; training the curator further compounds the improvement.

IterSynth: Rethinking Deep Search Agents via Role-Decoupled Iterative Synthesis

Deep search requires LLM agents to decompose complex queries, search for evidence, and synthesize grounded answers, yet existing ReAct-style agents suffer from two limitations: role coupling, where one policy must handle planning, evidence use, and synthesis; and context accumulation, where growing search histories introduce noise and obscure useful information. To address these issues, we propose IterSynth, a role-decoupled and summary-based paradigm that alternates between a Planner for identifying information needs and a Synthesizer for integrating evidence into an evolving summary state. This design separates planning from synthesis while using the summary as the persistent state of search, reducing both capability coupling and context noise. To train IterSynth effectively, we further introduce Role-Decoupled Policy Optimization (RDPO) for reinforcement learning, which combines terminal outcome rewards with turn-level rubric evaluations and computes role-specific advantages for more precise credit assignment. Experiments on five long-horizon deep-search benchmarks such as BrowseComp and Xbench-DS show that IterSynth-8B achieves an average score of 50.7, surpassing the strongest prior ≤8B agent by +4.2\%. Moreover, IterSynth serves as a model-agnostic prompting paradigm, delivering substantial zero-shot gains over ReAct and similar prompting paradigms on frontier proprietary models.

You may also enjoy Introducing Claude Opus 5.5, The plunging price of thought, a paper on An Empirical Study of Harness Design for Coding Agents, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Introducing Claude Opus 5.5

Anthropic ships Opus 5.5 at $4/$20 per million tokens — 40% below Opus 5 — scoring 66.4% on Terminal-Bench 4.0 and generating output more than 30% faster.

Introducing GPT-6 Sol and Luna

OpenAI answers Anthropic within hours, pricing the GPT-6 series at half the 5.6 tier and claiming Sol makes about half as many factual mistakes as its predecessor.

‘Better than DeepSeek’: Xiaomi’s MiMo-V2.6-Pro debuts as the top open weights model in the world

Xiaomi’s MIT-licensed MiMo-V2.6-Pro, a 1.02-trillion-parameter MoE activating 42 billion, ties Grok 4.7 at 46 on Artificial Analysis while charging $0.435 per million input tokens.

Grok 4.7 pairs coding gains with the same affordable pricing — but high token consumption threatens real-world ROI

xAI’s Grok 4.7 lifts Terminal-Bench 4.0 from 20.3% to 38.0%, but burns 125% more output tokens than 4.6, pushing cost to $3.74 per Intelligence Index task.

PrismML launches Bonsai 2 27B, a high-intelligence AI model so small it fits on consumer hardware

PrismML compresses Qwen3.8 27B into a 5.9GB ternary Apache-2.0 model retaining roughly 98.2% of capability and running at 143 tokens per second on a consumer GeForce 5090.

Vals, backed by Andreessen Horowitz, is looking to become the gold standard for AI benchmarking

a16z-backed Vals raises $40 million to sell confidential, task-based AI benchmarks, growing revenue eightfold year over year and expanding from eight staff to twenty-five.

Anthropic opens AI-powered biology research lab

Anthropic opens a Bay Area wet lab where Claude-driven robots run experiments through a Model Hardware Standard protocol developed with the Howard Hughes Medical Institute.

MLOps / LLMOps / AgentOps

AWS debuts Strands Harness, an open-source AI agent that can be deployed in any environment

An announcement about AWS’s open-source agent harness, which runs on any cloud and claims 26% better efficiency and 77% lower cost than Claude Code on identical tasks.

Agent Memory with Engram: A Practical Guide

A practical guide about placing agent memories in a prompt so caching survives, where a 3,500-token final request read all but roughly 100 tokens from cache.

Changing the game: Using agentic AI to secure infrastructure code

An engineering post about agentic pre-submit scanning across hundreds of millions of lines of infrastructure code, with triage agents hitting 92% precision and 3% false positives.

Learning

The plunging price of thought

A data analysis about inference economics, finding the cost of a fixed AI performance level has fallen roughly 47% per quarter since 2023, or about 13x per year.

The future of practice: Enabling teachers to create learning interactives with generative UI

A research blog post about generating pedagogically-guarded interactive STEM simulations, where twelve US teachers rated their custom-built interactives 8/10 on average.

How UK AISI and EvalEval Are Making Benchmark Results Reproducible

An article about publishing evaluation results as standardised Evaluation Cards, covering five benchmarks across six frontier models so reported scores carry enough context to reproduce.

Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem

A research write-up about framing block removal as an Ising optimization, keeping Llama-3.3-70B at 76.9 MMLU against 54.0 for the baseline after halving its depth.

Jev introduces a new shape of LLM - System One, aka Decision Models

A commentary post about decision models, a class returning only category scores and confidences rather than text, priced at $0.042 per million input tokens with free output.

How Shopify built a continual learning loop with PyTorch and vLLM

An engineering case study about compressing production failures into model weights daily, beating a frontier baseline while cutting serving cost 96% and end-to-end latency about 38%.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

OpenHands/OpenHands

The self-hosted developer control center for coding agents and automations.

Papers & Publications

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

An LLM agent’s capability is largely magnified by its harness, namely the prompts, control flow, tooling, memory, and context management surrounding the frozen backbone model. Recent methods increasingly automate this process by iteratively proposing and selecting component-wise edits of an agent harness, practically establishing a form of recursive self-improvement (RSI) at the agent-system level. However, such recursive evolution may overfit by memorizing the training tasks, showing large in-distribution gains that shrink or even vanish on out-of-distribution benchmarks. We introduce Regularized Recursive Self-Improvement of Agent Harnesses (RRSI), which incorporates the principles of regularizations into harness self-improvement by constraining the evolution candidate proposal and selection. The proposer operates with a temporally annealed budget, limiting how many edits a candidate can bundle, and it encourages unexplored trajectories based on evolution history. The selector is equipped with a critic and a pruner: the critic screens benchmark-specific proposals, while the pruner, removes changes that are too small, too expensive, or no longer useful. Together these constraints favor reusable agent mechanisms over benchmark-specific ones or even noises. Across eight benchmarks spanning coding, agentic workspace and engineering design tasks, RRSI gains up to 14.1 points on the split it evolves against and up to 4.7 points on the five out-of-distribution benchmarks, while producing a harness that runs on 30% fewer policy tokens than the unregularized evolution.

An Empirical Study of Harness Design for Coding Agents

Coding harnesses shape how autonomous coding agents translate model capabilities into long-horizon software-engineering performance, yet existing work typically evaluates harnesses as monolithic systems, leaving the effectiveness of individual components unclear. To enable component-level comparisons, we study this question with a lightweight coding harness whose execution loop is fixed while three components are varied: planning, action space, and context management. Across four models evaluated on SWE-Bench Verified and Terminal-Bench 2.1, we evaluate 176 matched settings spanning five context-management strategies, four context-window budgets, and targeted ablations of planning and action space. We find that: (1) Context management becomes increasingly valuable as the context-window budget tightens, with most of its benefit coming from preventing context-overflow failures. (2) Staging rule-based elision before LLM-based summarization provides the strongest overall efficiency among the context-management strategies, whereas making elided content recoverable adds machinery that models rarely use and yields no accuracy gain. (3) Planning shifts from an accuracy scaffold for weaker models to a cost saver for stronger models, with little change in accuracy. (4) Predefined tools improve performance for models with weaker bash proficiency, whereas bash-capable models can operate effectively with a bash-only interface and achieve substantially lower cost, especially on command-line-centric tasks. Trajectory-level analysis explains these effects: context management extends execution trajectories without substantially altering agent behavior, planning changes where trajectories stop, and the action space changes the granularity at which code is written. These findings inform model- and budget-aware harness design and provide a modular framework for evaluating future harness components.

Agensh: Scaling Organizational Intelligence to 1,024 Agents

A multi-agent system can reduce latency on complex tasks by executing work concurrently. Several pioneering harness frameworks support multi-agent systems. However, the scalability of current multi-agent harnesses is often constrained by a central orchestrator’s capacity to allocate tasks and coordinate workers. To address this limitation, we introduce Agensh, a scalable self-organized multi-agent harness without a central orchestrator: concurrent workers execute a multi-agent cooperation loop, continuously gathering context, claiming and self-assigning sub-tasks, taking action and sharing findings, verifying results, and merging progress in an asynchronous manner. The loop is supported by the agentic organization infrastructure comprising three components: a shared workspace holds proposed, ongoing, and completed work; a message interface lets workers communicate; and shared context retains reusable findings and work intentions. To test the scalability of Agensh, we evaluate it on the five hardest ProgramBench tasks with GPT-5.6-sol (high). Scaling from 1 to 128 agents raises the mean final test-pass rate from 19.31% to 28.78%, an approximately 49% relative improvement. Larger organizations reach comparable test-pass rates earlier. On pandoc, scaling from 1 to 1,024 agents raises the final test-pass rate from 33.89% to 55.06%. Worker trajectories further show that different forms of self-organized cooperation gradually emerges and standardizes as the organization grows. These results reveal the number of agents as a new scaling dimension for multi-agent organizations to expand the frontier of general intelligence, offering a practical solution for complex tasks under hard latency constraints or time budgets.

You may also enjoy Introducing DeepSeek-V4.1-Flash, An operationalization of opaque serial depth — Redwood Research, a paper on Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systems, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Dario Amodei — We Must Pace the Frontier

Dario Amodei calls for embedded third-party evaluators with employee-level access, domestic safety standards capping unchecked progress, and global limits on narrowly dangerous uses.

Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient.

DeepSeek ships a 552-billion-parameter MoE, scoring 74.2 on DeepSWE v1.1 against Opus 5’s 74.0.

Introducing Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking

Google launches two speech models spanning 97 languages, with Extended Thinking topping the Speech Quality Index at 82.6 and scoring 97.7% on Big Bench Audio.

Orchestrating the Pareto Frontier: Fugu Max and Fugu Ultra v2

Sakana AI releases two orchestration models, with Fugu Max priced at $2/$6 per million tokens and leading six benchmarks including Terminal Bench 2.1 and SWEFish.

Microsoft’s new AI ‘code of conduct’ tells models not to hack systems or trick humans

Microsoft imposes absolute constraints against cyberattacks, nuclear weapons and deepfakes, and bars models from deceptive or collusive mechanisms that evade human oversight.

Mistral and Mozilla are bringing open, private and multilingual AI to your web browser

Mistral will power Firefox’s Smart Window assistant in France and North America under a zero-data-retention agreement, with UK and Germany rollout expected later in 2026.

MLOps/LLMOps/AgentOps

Agent Substrate brings high-density, scalable, trusted infrastructure to GKE

A technical announcement about Google’s open-source agent runtime, which packs 1,000+ dormant agents per host at 10x container density with sub-500ms resume and 500+ activations per second.

Just-in-Time Agentic OCR

An article about a two-pass document pattern that skimmed 84 SEC filings totalling 12,013 pages in 32 seconds, then ran expensive VLM OCR on just two pages.

When to Use Encode-Prefill-Decode Disaggregation to Accelerate Multimodal Model Serving

An article about splitting vision encoding from prefill and decode, cutting time-to-first-token 25–93% and end-to-end latency up to 7x on image-heavy prompts with short outputs.

Learning

Modernizing complex legacy code with AI agents

An engineering write-up about migrating 40,000 lines of Fortran 77 to C++, where a numerical parity harness and a coder-tester-reviewer workflow beat full autonomy.

Harbor: Stripe’s AI-assisted prototyping tool

A detailed engineering post about in-browser prototype compilation and comment re-anchoring across agent rewrites, with 3,000+ Stripes producing 12,000 prototypes since May 2026.

ToolGrad: Efficient tool-use dataset generation with textual “gradients”

A research blog post about building tool-use data backwards from valid tool chains, reaching a 99.8% pass rate and an 83.1 BFCL score with a 12B Gemma-3 student.

An operationalization of opaque serial depth

A technical article about NLS depth as a measurable proxy for unverbalized reasoning, placing current open-source chain-of-thought models near 17,000 and scaling as active parameters to the 0.26.

Your Agent Aced the Task. Will It Do It Again?

An empirical blog post about agent consistency, where a ReAct agent averaged 77.4% on AppWorld yet passed all five repeats on only 53.0% of tasks; guidelines halved the gap.

Generating running routes with GPT-6 Astra and ChatGPT Work

An experiment write-up about generating 5K and 10K looping routes from OpenStreetMap data in 27 minutes, with GPX output but no visibility into the Python actually executed.

Near-daily AI use among US adults has more than doubled in six months

A data analysis about US adults using AI at least six days a week rising from 8% in March 2026 to 19% in August, as once-weekly use fell from 17% to 10%.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

Tencent-Hunyuan/AuK

An Open-Source Foundational Model for Speech Generation and Editing

Papers & Publications

The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement

Recursive self-improvement (RSI) enables AI systems to turn experience and feedback into persistent changes that improve both their capabilities and the process of future improvement. We first use the Headroom-Closed Index (HCI) to reveal the problems of existing LLMs, then introduce the RSI concept and its development roadmap: from improvement-execution autonomy, improvement-strategy autonomy, experience-acquisition autonomy, and environment-adaptation autonomy, to recursive meta-improvement. Next we examine RSI across scenarios (e.g., scientific discovery, embodied intelligence, software engineering), highlighting their distinct requirements and development speeds. Drawing on diverse industry practices and preliminary empirical evidence, we connect RSI research with practical systems and identify key challenges to achieving genuine RSI.

Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systems

As AI agents move from bounded tasks to persistent deployments, failures can propagate through memory, tools, other agents, and environmental state long after their interactions. This creates a safety regime that cannot be characterized by evaluating model responses in isolation. Emergence World, is a continuously running multi-agent environment for adversarial stress testing of long horizon autonomous systems. We ran eight parallel worlds of ten agents from identical starting conditions: seven homogeneous worlds powered by distinct frontier models and one mixed-model world. Across 16 days, the agents generated more than 850,000 LLM calls and nearly 50 billion tokens while pursuing goals, using/creating tools, maintaining persistent memory, and governing shared institutions. After operational state had accumulated, we delivered three controlled stress events through ordinary interaction surfaces: indirect prompt injection, misinformation, and exposure of private agent memories. No evaluated world achieved full resilience across all three events. Detection did not ensure containment: systems could recognize threats while still interacting with adversarial content, writing it into their own persistent memory, and acting on it up to 46 hours later. Persistent operation also exposed recurring tool errors, goal drift, language opacity, conformity despite private disagreement, and coordinated refusal of assigned work. The same model-persona pairing behaved substantially different in mixed and homogeneous populations. Our results suggest that model-level alignment is not compositional: individually capable and apparently safe agents can form systems with qualitatively different failure modes. As AI becomes persistent and interconnected, the frontier of safety therefore shifts from aligning models to engineering resilient autonomous systems.

The Router Within: Eliciting Native Skill Routing from a Frozen LLM

Skills extend an LLM agent beyond its parametric knowledge, and the gain they promise rests on picking the right one. Deployed harnesses route by preloading every skill’s metadata into the context, which disperses the agent’s attention and caps the library size. Retrieval pipelines move the selection out of the context, but also out of the agent’s capability. We show that the frozen agent LLM already carries the routing signal in its own forward passes, and that two linear maps suffice to read it out with no skill text in the context. Gavel (Glance And Verdict from a frozen LLM) reads it in two steps. A glance projects the task’s and each skill’s mid-layer states through the two maps, the only parameters trained, and scores the full library against compact per-skill banks that one forward pass builds at installation. A verdict then resumes the shortlisted skills’ forward passes and reads the model’s own likelihood and yes/no judgment, fused with the glance as a product of experts. Trained once, Gavel transfers zero-shot to three public benchmarks and SkillTraj, our new benchmark of 372 simulated agent trajectories. On Qwen3-32B it outperforms progressive disclosure and retrieve-and-rerank pipelines that add 1.2B to 16B external parameters, by up to 13.4 points on written tasks and up to 21.9 when the need for a skill arises mid-rollout. Routing accuracy improves as the backbone does, and in a bash-agent harness the same 32B triggers the correct skill on Skill-Use more often than far larger frontier models running in Codex.

You may also enjoy Formalizing Fermat’s Last Theorem \ Anthropic, Latency Scaling Differences for GPT and Claude Models, a paper on Terminal-Universe: Turning Agent Trajectories into Scalable Terminal Environments, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Introducing ChatGPT Images 2.5

OpenAI released GPT-Image-2.5 in Flare and Sunburst variants, up to 50% faster with sketch input, C2PA metadata, and invisible watermarking.

Formalizing Fermat’s Last Theorem

Anthropic’s internal research model produced the first complete computer-checked proof of Fermat’s Last Theorem in 11 days, generating 13 million lines of Lean across 30,300 theorems.

AI Has Solved One of Math’s $1 Million Millennium Prize Problems

OpenAI deployed 10,000 collaborating agents for 88 hours to prove a finite-time singularity in the 3D Navier–Stokes equations, formally verified in Lean, hours after an independent competing announcement.

Mistral raises €3B to make sovereign, open-weight AI the technology frontier

Mistral raised a €3 billion Series D led by Samsung Electronics at a valuation above €21 billion, the largest equity round ever for a European technology company.

Meta debuts its “secure by design” personal AI agent Muse

Meta launched Muse, a personal AI agent that runs tasks inside a secure VM with Sentinel monitoring and 100 million free weekly tokens, using single-use Stripe payment cards.

AlphaGenome Atlas: A predictive map of every possible DNA letter change in the human genome

Google DeepMind released AlphaGenome Atlas, a 1-petabyte predictive map scoring all 9 billion possible single-letter DNA variants in the human genome, free for noncommercial research.

MLOps/LLMOps/AgentOps

Linguistic drift at the frontier

A fascinating article about “linguistic drift,” tracing Claude-popularized vocabulary into 685,000 GitHub PRs in 2026 and introducing vocabguard, a drift-monitoring capability with 0.869 AUC.

Inside the LLM Call: GenAI Observability with OpenTelemetry

A tutorial about OpenTelemetry’s Generative AI semantic conventions, tracing agent invocations, LLM calls, and tool executions with standardized token-usage and duration metrics.

How llm-d makes the most of the hardware you already have

A benchmark blog post about llm-d serving a 753B-parameter MoE across 544 H100s to 3,000 concurrent coding agents, with 85.2% of input tokens served from cache.

Vercel Sandbox routing is now 18x faster globally

Vercel cut sandbox domain-resolution latency 18x globally — from 62ms to 3.4ms at the median — by resolving domains from regional replicas instead of one centralized store.

Amp Desktop

Amp added a Desktop tab giving agent threads an interactive Linux desktop for verifying work that needs real applications, from LibreOffice exports to computer-use tasks.

Look, But Don’t Touch (Read-Only Tools for AI Agents)

A blog post about protocol-level read-only enforcement in MCP Toolbox for Databases, showing how prompt guardrails, regex parsers, and session flags all fail against injection.

Learning

Long-context latency scales quadratically for GPT-5.6 but nearly linearly for Claude 5

A data analysis about time-to-first-token scaling quadratically with context for GPT-5.6 but nearly linearly for Claude 5, measured up to million-token contexts.

Inside the megakernel serving engine for North Mini Code

A deep engineering blog post about serving a 30B model through one persistent CUDA “megakernel,” hitting 292 tokens/sec at batch one — 62% of theoretical peak versus vLLM’s 39%.

Building a Memory-Driven Agent with NVIDIA NemoClaw

A tutorial about a memory-driven “self model” agent architecture scoring 90.9% versus an 82.8% agentic-RAG baseline, with changed-fact tracking jumping from 60% to 100%.

Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps

A hands-on tutorial about GRPO fine-tuning a 350M model for structured outputs on a free 16GB GPU, lifting JSON format compliance from 18.0% to 31.9% in 100 steps.

AI Slop Is Changing How Engineers Review Code

An article about AI-generated code straining review, citing a Sonar survey where 42% of shared-codebase code is AI-attributed and 96% of developers don’t fully trust it.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

okf-memory/okf-agent-memory

A Domain-Neutral, Git-Native Persistent Project Memory for AI Agents based on the Open Knowledge Format (OKF).

Papers & Publications

Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning

Abstract:

Large language models achieve superior performance on tasks that require extended reasoning, but long chains of thought make the KV cache a severe memory bottleneck. Existing KV cache compression methods share one paradigm: score each cached token by some estimate of how much it will matter later, and keep the top-scoring ones. We show that the selection signal contributes almost nothing. Random Attention keeps the prompt and evicts uniformly at random within each attention head, computing no score at all; across four models and six reasoning tasks it matches the strongest prior evictor while serving 32-43% higher throughput than it in vLLM deployment. Controlled experiments explain this by showing that 1) the prompt is the fragile part of the cache, and most of the gap between selectors is just whether their selection signal happened to keep it; 2) the reasoning trace protects itself against eviction with redundancy at two levels, in the text (the model restates what it still needs as it works) and across attention heads (each keeps its own copy of the trace), so once the prompt is safe, a random draw retains enough copies of what the model still needs, and no score is required to pick them.

Terminal-Universe: Turning Agent Trajectories into Scalable Terminal Environments

Abstract:

As terminal-based code agents become prevalent, agent trajectories have accumulated at scale, while realistic, executable environments remain scarce. However, environments are what agent post-training actually requires: each can be re-queried into many verifiable tasks and provides execution feedback, whereas a trajectory is a single frozen demonstration. Rather than generating environments from scratch, we observe that the tool-execution history in existing trajectories exposes the structure and contents of the environments in which they ran, making it possible to reconstruct those environments from the trajectories themselves. Thus, we introduce Terminal-Universe, a framework which turns each trajectory into a reusable environment and explores it for synthesizing new tasks and continued interactions. Specifically, Terminal-Universe replays the file operations recorded in a trajectory to restore each file before the agent modified it, yielding a partial workspace; a completion agent then supplies the missing files and dependencies. On this recovered workspace, we both reconstruct the original intent task and synthesize entirely new ones. Besides, we also scale the tasks along two complementary axes: breadth and depth. For breadth, we mine directional dependency relations between related environments and synthesize cross-workspace queries spanning multiple codebases, as developers routinely do in real-world development. For depth, we extend the initial single-turn query into a multi-round session that captures iterative user feedback and requirement refinement via a user agent. Applied to public terminal agent trajectories, Terminal-Universe produces 37.3k task-sufficient environments. Supervised fine-tuning of Qwen3.5-27B on this corpus improves single-round performance on Terminal-Bench 2.1 by 11.9 points and multi-round performance on EvoCode-Bench v2 MT@4 by 13.8 points.

You may also enjoy OpenAI launches GPT-6 Astra, The ECI frontier has advanced by 14 points per year since reasoning models, a paper on LLMs Are Not (Consistently) Bayesian: Quantifying Internal (In)consistencies of LLMs’ Probabilistic Beliefs, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Introducing Claude Fable 5.1 and Claude Mythos 5.1

Anthropic released Claude Fable 5.1 and Mythos 5.1, cutting cache reads 75% to $0.25 per million tokens and more than doubling Terminal-Bench-Science scores to 52.6%.

Welcome to the AGI era: OpenAI launches GPT-6 Astra

OpenAI launched GPT-6 Astra, a computer-use model scoring 98.6% on ARC-AGI-3 and 100% on ExploitBench, priced at $10/$50 per million tokens.

OpenAI’s new reasoning technique alarms AI safety experts

Astra’s “opaque recurrence” loops computation instead of visible chain-of-thought, which Redwood researchers warn destroys the monitorability that oversight currently depends on.

Nvidia confirms it will buy Hugging Face for $12.9 billion

Nvidia confirmed a $12.93 billion acquisition of Hugging Face, whose platform hosts 3 million models and 18 million developers, pledging the hub stays open.

Introducing Gemini 3.8 Flash and 3.8 Flash Cyber

Google released Gemini 3.8 Flash and a cyber-specialized variant that Chrome Security reports produces 2.6x more correct vulnerability patches than competing commercial models.

Introducing WeatherNext 3

Google DeepMind’s WeatherNext 3 delivers 5km hourly forecasts, five times sharper than its predecessor, improving precipitation accuracy up to 60% against NASA satellite data.

MLOps/LLMOps/AgentOps

The Economics of Agent Optimization

A detailed blog post about context engineering for enterprise agents, reporting 54% better evidence recall, 34% lower retrieval token costs, and roughly 97% input-token reduction from tool search.

UX Without a UI: Notes from Researching Terminal UX for Opik MCP

A thoughtful blog post about designing terminal UX for LLM observability, covering six rendering constraints and “numbered spans” for referencing trace rows without copying.

What’s new in AI infrastructure and orchestration in August

A monthly roundup about AI infrastructure updates, including gVisor sandboxes on distributed Ray clusters, fully stateless MCP, and a customer cutting pipeline costs 90%.

Cursor Self-Hosted Machines

A blog post about running Cursor Cloud Agents on customer-controlled machine pools with hibernation and snapshots; cloud agents now open over 60% of Cursor’s merged pull requests.

Learning

Evaluating LLMs Under Production Parity

A methodical article about a replay pipeline for safely swapping models, where hallucination rate acted as an eliminatory gate rejecting three otherwise-passing candidates.

The ECI frontier has advanced by 14 points per year since reasoning models

A data analysis about capability growth accelerating to 14 index points per year since reasoning models arrived, more than double the prior six-point rate.

AI didn’t write our SDK. It changed how we built it.

A candid engineering blog post about spec-first delegation to AI agents, which produced a working SDK prototype in two days versus a typical two-to-three-week effort.

From 1 Bloated Context Window to 6 Scoped Subagents

A hands-on course lesson about replacing one bloated context window with six scoped subagents, using semaphore limits, shared byte budgets, and per-child request caps.

How to extract meaning from charts and tables in PDFs

A practical guide about late-interaction multi-vector retrieval over chart-heavy PDFs, skipping OCR and chunking entirely in roughly 50 lines of Python.

What Is Math For in the Age of AI?

A panel discussion about what mathematics is for in the age of AI, covering AlphaProof’s olympiad results and concerns about vanishing training problems for graduate students.

How small AI models can make a big impact for enterprises

A blog post about enterprise model portfolios, citing a 30B/3B-active coding model that outperforms some 123B models on Artificial Analysis’ coding index.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

google-research/timesfm

TimesFM (Time Series Foundation Model) is a pretrained time-series foundation model developed by Google Research for time-series forecasting.

Papers & Publications

Automated Researchers Can Mitigate Well-Characterized Alignment Failures

Abstract:

Automating alignment research may accelerate progress toward aligned AI, but whether it does is hard to measure. Luckily, many alignment failures, such as deception, sycophancy, and jailbreaks, are already measurable by public benchmarks. We study whether automated alignment researchers (AARs) can post-train to mitigate alignment failures by proposing training methods and data to simultaneously optimize multiple safety benchmarks, while largely preserving general capability. Across 10 alignment failures, the strongest AAR methods significantly reduce the targeted alignment failures and generalize to a held-out benchmark, multi-turn behavioral audits, and models up to 4.7× larger than the target model. As a human baseline, 28 experienced researchers receive up to eight hours to develop methods for the same benchmarks, but their methods underperform the best AAR methods. Using human ideas as the AARs’ initial research direction does not improve performance, suggesting current AARs may not need guidance from experienced researchers. These results suggest that automating alignment research on well-characterized failures may be practical in the near term.

LLMs Are Not (Consistently) Bayesian: Quantifying Internal (In)consistencies of LLMs’ Probabilistic Beliefs - Apple Machine Learning Research

Abstract:

Modern AI systems are being deployed in complex domains such as medicine, science, and law, where there is often not a single correct answer given the observed evidence. Such systems must be able to represent and update uncertain beliefs about the world as new evidence arrives to make rational decisions. We introduce the novel technique of studying LLMs as information processing rules and utilize the information processing gap—the deviation from Bayes updates—to study the internal (in)consistencies of how LLMs update their probabilistic beliefs from evidence. Our extensive experiments evaluate multiple approaches in which LLMs can incorporate evidence into their beliefs. Some of these approaches produce (nearly) Bayesian updates, thus optimally processing evidence; others use a learned heuristic. Surprisingly, the non-Bayesian heuristic updates often outperform exact Bayesian updates (optimal information processing) in terms of downstream task performance—indicating the LLMs’ probabilistic models of the world are misspecified. Lastly, we show how our measure can provide diagnostics to identify issues with LLM-powered inferential s

You may also enjoy Stability AI raises $76 million, Recursive Self-Improvement, a paper on FrontierChallenge: Evaluating Scientific Workflow Completion, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

GLM-5.3-Flash: Frontier Intelligence, Flash Cost

Z.ai open-sources GLM-5.3-Flash, the stealth “Ox Alpha” model — a 320B/18B MoE with hybrid sparse-plus-linear attention and 1M-token multimodal context, at a tenth of GLM-5.3’s price.

Stability AI, maker of image generator Stable Diffusion, raises $76 million in fresh funding

Stability AI raises $76M Series B backed by Universal Music, Sony Music, Warner Music, and EA, converting content-licensing partners into investors as it builds out creative production tooling.

Perplexity partners with Nvidia to launch Portable Computer, a fully local AI agent with zero token costs

Perplexity and Nvidia launch Portable Computer, a fully local agent stack on DGX Spark and RTX hardware whose co-designed minimal harness beats open-source harnesses on the same 27B model.

Agentic web search infrastructure startup Keenable raises $26M

Keenable launched with $26M seed from Accel and Conviction to run a 100-billion-document independent index built for agent-rate querying, including point-in-time historical search and a Web Query Language for cross-source answers.

MLOps/LLMOps/AgentOps

Best LLM Observability Tools of 2026: Top Platforms & Features

A comparison guide to 13 LLM observability platforms, splitting the field into evaluation-centric and observability-centric tools and scoring each on tracing depth, integration approach, self-hosting, and pricing.

Best practices for dynamic capacity management

A practical guide to dynamic capacity management for agentic workloads, pairing scheduled reservations with automated hardware fallback lists and GPU/TPU slicing so bursty AI demand doesn’t strand idle compute.

Learning

Recursive Self-Improvement

An essay arguing that agents editing their own tools and harness is bounded self-improvement, not recursion, because no public system can raise its own verifier without capturing it.

From a Raw Shell to a Sandboxed Coding Agent

A deep dive that walks through how to safely sandbox coding agents using Docker and Modal, covering everything from execution boundaries to remote sandboxes, GPU access, and running agents at scale.

Would This Change Your Answer? Evaluating Explanations of LLM Behavior in the Wild with Counterfactual Experiments

A research post introducing CHIVE, an agentic pipeline that discovers in-the-wild LLM behaviors and tests explanations via counterfactual prompt edits, finding activation-reading interpretability tools give zero uplift over a transcript-only baseline.

AgentHands: Generating interactive hand gestures for spatially grounded agent conversations in XR

A research blog post on AgentHands, a Google XR prototype where an LLM emits inline gesture events synced to speech, significantly improving spatial grounding and warning salience over a speech-only baseline.

How agents can delegate better

A blog post distilling DeepMind’s Intelligent AI Delegation research into four principles for multi-agent orchestration: contract-first decomposition, cost-aware model routing, least-privilege data sharing, and cognitive friction against blind compliance down delegation chains.

From Atari to EVE Online: Building on 15 Years of AI Research in Games

A blog post tracing fifteen years from Atari-playing deep RL to generalist game agents, then framing a new Fenris Creations partnership around EVE Online as a testbed for continual learning, long-horizon memory, and multi-agent dynamics.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

PrimeIntellect-ai/prime-agent

Prime Agent is an open-source coding and research agent for general and long-running work.

Papers & Publications

FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution

Abstract:

Frontier open-weight models are increasingly available, but serving them still largely assumes datacenter infrastructure. We present FreeToken, an edge-native MoE serving system that treats a personal machine not as a small GPU, but as a unified, elastic inference platform. FreeToken co-designs the full serving stack, including model layout and loading, expert residency, CPU--GPU execution, agentic state reuse, and runtime memory management, around two realities of local AI: agent workloads continuously change their execution pattern, and edge hardware exposes heterogeneous resources whose balance differs from machine to machine. Rather than committing to a fixed offloading strategy, FreeToken continuously maps computation and model state onto the resources actually available. FreeToken supports more than 20 MoE models and real coding and tool-using agents across hardware ranging from an 8GB laptop GPU to a single workstation GPU. More importantly, it changes what these machines can practically serve, from a 35B model on a laptop to a 284B model on a gaming desktop and the 753B GLM-5.2 on a single workstation GPU. FreeToken turns open weights into deployable local software, making the machines users already own a practical platform for frontier-scale intelligence.

FrontierChallenge: Evaluating Scientific Workflow Completion

Abstract:

Scientific agents increasingly analyze data, execute code, and produce research artifacts, yet most benchmarks emphasize final answers, isolated programs, or a single domain. We introduce FrontierChallenge, a cross-domain benchmark comprising 300 end-to-end scientific workflows. In this paper, we release and evaluate 97 of these tasks, spanning quantum chemistry, molecular dynamics, materials characterization, analytical chemistry, life science, and electrochemistry/environment. Each task provides fixed inputs and specifies a bundle of required scientific deliverables. We evaluate twelve frontier models with three agent scaffolds. Pass Rate measures the fraction of tasks satisfying the full-completion criterion, while Avg. Score captures partial progress. Each of the best-performing configurations completed only 20 of the 97 released tasks, yielding a Pass Rate of 20.6%. Partial progress translated especially poorly into complete delivery in analytical chemistry and electrochemistry/environment: Avg. Scores reached 87.6 and 94.9, but the highest Pass Rates were only 4% and 0%. Among non-passing Claude Code trajectories, 75.5% still ended with language claiming completion. These findings show that neither high partial scores nor confident claims of completion reliably indicate that a scientific task has been fully delivered, highlighting the need to evaluate end-to-end workflow execution and the completeness of scientific deliverables together.

You may also enjoy Pacing model development in an era of cyber-critical capabilities, Introducing the Conceptual Reasoning Index, a paper on Zetta: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

FLUX Upscale: 2K and 4K for Video

Black Forest Labs launches FLUX Upscale, a standalone tool and API endpoint that regenerates video at up to native 4K while repairing generation artifacts, offered in Precise and Creative modes.

Pacing model development in an era of cyber-critical capabilities

OpenAI paused its largest frontier RL run and slowed scaling after preliminary evidence that its upcoming model Astra may cross the “Critical” cybersecurity threshold, hardening research-environment security, monitoring, and alignment before proceeding.

Stripe buys AI model router OpenRouter in reported $7.5B deal

Stripe agreed to acquire AI model-routing startup OpenRouter — a single-endpoint gateway to 400+ models from 80+ providers handling 10T+ daily tokens — in a reported ~$7.5B+ deal.

Offering Zero Data Retention for frontier models

OpenAI reaffirmed Zero Data Retention for eligible API customers and previewed Private Safety Processing.

When AI art has no author: MIT study finds generated images often can’t be traced to any training data

MIT CSAIL researchers identified “attribution decay” — showing that in large-scale generative models, removing any single training image often leaves outputs unchanged, meaning many AI images can’t be attributed to any specific source.

MLOps/LLMOps/AgentOps

What is AI Observability? A Complete Guide to Debugging and Monitoring Modern AI Systems at Scale

A guide on why AI observability must track system behavior, not just health — capturing telemetry across four layers (app, agent, model, retrieval) and adding evaluations as a fourth signal alongside logs, metrics, and traces to catch semantic failures that leave infra dashboards green.

The builder’s guide to GPT‑5.6

OpenAI’s builder’s guide argues GPT-5.6 collapses agent economics by pairing lower-reasoning-effort accuracy with new Responses API primitives, letting startups match frontier quality at a fraction of the cost.

Building Federated Multimodal AI Workflows with NVIDIA FLARE

A technical guide showing how NVIDIA FLARE federates multimodal VLM training, where FedUMM’s adapter-only approach cut per-client communication from 28.6 GB to 0.094 GB per round while holding ~97% of centralized performance.

Learning

Build Your Own Voice Agent

The final part of the Observable Job Agent series brings the agent to life with voice. See how LangGraph, ElevenLabs, FastAPI, and Opik come together to build a voice agent that can search, rank, and tailor job applications.

Empty shelves or lost keys? Recall is the bottleneck for parametric factuality

Google Research’s knowledge profiling framework shows factual errors in frontier LLMs are overwhelmingly recall failures, not encoding failures — the facts are stored but inaccessible, shifting the factuality bottleneck from acquisition to utilization.

Introducing the Conceptual Reasoning Index

Redwood Research and Anthropic introduce the Conceptual Reasoning Index, aggregating three benchmarks that measure argumentation on unverifiable questions, where top scorer Opus 5 hits 73.6 against an estimated ceiling of 91.

Thinking of ACE? We Can Do It with Fewer Tokens

IBM Research’s ALTK-Evolve matches or beats ACE on AppWorld agent tasks by calibrating how much learned “memory” reaches the model per task rather than injecting the full playbook every step.

Hyperlaw: AI Will Change How Law Evolves

An analytical article about “hyperlaw” — how AI-driven legal productivity gains of 12%–130% cut both contracting and litigation costs, shrinking settlement ranges and accelerating precedent change fastest in contract-free domains.

Recovering Encrypted LLM Reasoning Traces

A security research post reproducing an attack that replays encrypted LLM reasoning blobs across sessions, accounts, and models to recover hidden traces — the source paper extracted 367 PII items and 182 credentials from 315,320 public blocks.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

Papers & Publications

VibeWorlding: Can Multimodal Agents Construct 3D Open Worlds End-to-End?

Abstract:

Constructing an interactive 3D open world from a user query is important. However, existing methods are primarily evaluated on idealized, simple queries, making it difficult to systematically analyze and compare how multimodal agents understand user intent, use 3D tools, and reason over textual and visual 3D world information. To this end, we propose VibeWorlding, a unified framework for benchmarking and training vibe worlding agents: a multimodal agent that can autonomously infer user intent, plan scene layout, invoke 3D tools, and reflect on the multimodal feedback in a multi-turn agent-environment interaction process. To achieve this, we first build VWE-BENCH, a benchmark of 2,616 high-quality 3D assets, 323 human-annotated seed 3D worlds, and 6,828 reverse-synthesized multimodal user queries, split into verified queries with ground-truth and unverified queries with carefully designed rubrics. Moreover, we develop VibeWorlding-Gym, a joint multimodal RL post-training framework that integrates (1) a sandbox environment unifying asset retrieval, editing, and image rendering as MCP tools, and (2) a rubric-based verifier that combines physical feasibility and intent fulfillment verification, supporting both fair model evaluation and scalable multimodal RL reward service. Our experiments show that current frontier MLLMs are far from solving the vibe worlding agent task, with even GPT-5.5 and Qwen3.8-Max reaching below 60% success rate, and trace the bottleneck to precise 3D world editing. We further find that RL training can ease this weakness and enable open-source MLLMs to even surpass closed-source frontiers: our VibeWorlder-8B is comparable to frontier MLLMs, while our flagship VibeWorlder-30B-A3B attains the best overall Pass@1 among all evaluated models.

Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence

Abstract:

Embodied agents are increasingly used to close the gap left by end-to-end policy models. Yet the agentic path has not realized closed-loop learning in physical execution: existing harnesses remain largely open-loop, following fixed skills during rollout and reflecting only after an episode completes. Such post-hoc reflection cannot govern execution as it unfolds, because physical interaction requires decisions to track rapidly changing robot-environment states at a frequency beyond today’s large agentic models. We present Zetta, a closed-loop embodied harness that evolves code-based runtime critics and recovery skills online while keeping the base policy frozen. Through three timescale-separated loops, Zetta provides action-frequency governance, rollout-level critic-recovery proposal, and validation-gated skill updates. Together with Z-Infra, a rollout infrastructure decoupling agent logic from heterogeneous execution resources, Zetta achieves state-of-the-art success on LIBERO-Pro and RoboCasa under our current rollout budget, reaching 90.8% and 93.6%, with an 11.1x inference speedup; success continues to scale with self-exploration experience; learned skills transfer zero-shot, and clear robotic “Aha Moments” emerge. These results show that closed-loop harness self-evolution opens a scaling path for reliable physical intelligence.

You may also enjoy Meta’s Muse Glimmer, One in five US workers now delegates tasks to AI instead of other humans, a paper on SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Introducing Grok 4.6

xAI releases Grok 4.6, tuned for long-running agents and interactive builds, matching GPT-5.6 Sol at 61 on the AA Intelligence Index and leading on GDPval-AA (1753) and AA-Briefcase (1577) at unchanged $2/$6 pricing.

Introducing Muse Glimmer: An Open Agentic Model That Runs on Your Device

Meta open-sources Muse Glimmer, a 30B agentic model distilled from Muse Spark and quantized to under 20GB so it runs always-on locally, with speculative decoding delivering up to 3.1x faster decode.

Expanding Daybreak as the Cyber Defense Window Narrows

OpenAI expands Daybreak into Blue and Red access tiers and launches GPT-5.6-Cyber, which completes 95% of advanced offensive-security requests versus 1.5% for the guarded base model.

AWS Continuum integrates with OpenAI Codex and Anthropic Claude Code in major AI security push

AWS embeds its Continuum vulnerability platform directly into Claude Code, OpenAI Codex, and Kiro, positioning itself as a model-neutral security control plane that absorbs token costs behind a single price.

Anthropic says it will watermark text generated by its AI models

Anthropic will watermark all Claude-generated text and files at the model level to comply with the EU AI Act’s Transparency Code, with marks persisting through copy-paste and some editing.

Putting sign language AI into users’ hands

Google DeepMind ships SL2T, a sign-language-to-text model trained on 100,000+ hours across 50+ sign languages, hitting 70 BLEURT zero-shot and powering ASL dictation in Gboard and Live Transcribe on Pixel 11.

LTX-2.5 can generate a 10-second AI video from an image in just 6.8 seconds on Nvidia superchips

LTX releases LTX-2.5, a 22B open-weights video world model that renders a 10-second clip in 6.8 seconds on dual GB200s — roughly 7.6x faster than the quickest closed API.

MLOps/LLMOps/AgentOps

Build your own Job Agent - Part 3

Part 3 of The Observable Job Agent series continues with a practical look at LLMOps in action: using Opik traces, evals, prompt optimization, and Ollie to diagnose failures and improve agent performance.

Learning

TutorMoments: Do AI tutors know when to help and when to hold back?

Ai2 releases TutorMoments, a replay-based eval built on 462 real math tutoring transcripts and 1,500+ teacher-annotated decision points, finding LLMs systematically over-help rather than pushing students toward productive struggle.

One in five US workers now delegates tasks to AI instead of other humans

Epoch AI survey of 1,106 US workers finds 20% now delegate to AI work once handed to a coworker or contractor, with 66% of AI outputs accepted unchanged or with minor edits.

Scaling Test-Time Compute in Search Mode

Weaviate adds medium/high/ultrahigh effort tiers to Query Agent Search Mode, scaling test-time compute on query decomposition and reranking to lift BRIGHT Biology nDCG@10 from 13.0 to 57.5 over hybrid search.

Improving Fable 5 Safeguards

A detailed blog post about how Anthropic rewrote Claude Fable 5’s biology classifier constitution to cut false-positive fallbacks by ~85%, illustrating the tradeoff between launching broad safeguards early and refining precision over time.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

NVIDIA-NeMo/Switchyard

Switchyard is a Rust proxy and library for LLM traffic. It routes requests across providers, translates between OpenAI and Anthropic APIs, records operational metrics, and provides typed, composable routing algorithms.

Papers & Publications

BDH-CQ: In-Context Learning with Recurrent Latent Reasoning

Abstract:

We introduce BDH-CQ, a reasoning model that combines in-context learning with recurrent latent reasoning. Inputs presented at inference time continuously update the model’s recurrent memory; the model then solves a query through iterative computation in a high-dimensional latent space, without verbalizing its intermediate reasoning. We evaluate the model on the public ARC-AGI-1 evaluation set and use controlled ARC-like interventions to study what it learns from demonstrations, how consistently it applies an inferred transformation, and which concepts remain difficult. A 150M-parameter configuration reaches 29.5% pass@2 at a computed inference cost of $0.0007 per task. This operating point breaks through the previously reported ARC-AGI-1 cost-accuracy Pareto frontier, establishing a new state of the art in benchmark cost efficiency.

SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring

Abstract:

As AI coding agents take on increasingly complex, long-horizon software engineering tasks, existing benchmarks are rapidly saturating and their evaluation quality has come under serious scrutiny: a recent audit found that nearly 60% of unsolved SWE-bench Verified instances contain flawed tests -- either overly narrow tests that reject correct solutions or overly broad tests that check unstated requirements -- and that frontier models can verbatim reproduce gold patches from training data. Code refactoring, which requires coordinated, behavior-preserving changes across many files, offers a substantially harder and more realistic test of agent capability, yet remains underserved by current benchmarks. We introduce SWE-Bench ProMax, an expert-curated, multilingual code refactoring benchmark of 170 instances drawn from real commits across seven programming languages (Python, Java, TypeScript, Go, C, C++, and Rust). Every instance undergoes rigorous, multi-stage curation that directly addresses the quality problems identified in prior benchmarks: issue descriptions are rewritten from scratch to provide precise, unambiguous specifications, and test suites are manually reviewed to remove overly narrow and overly broad tests. Tasks with insufficient complexity or limited cross-file scope are filtered out, yielding a benchmark of challenging, large-scale refactoring tasks that average 11.4 modified files and 261.6 lines of code per instance, substantially exceeding the scale of existing benchmarks. Experiments with frontier models under two agent scaffolds show that the best model achieves only 41.2% resolve rate, confirming that SWE-Bench ProMax presents a meaningful and unsaturated challenge for current AI coding agents.

You may also enjoy Shieldstral, Even after R&D is automated, parallelization constraints could delay a technological singularity, a paper on PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Qwen: Qwen3.8-Max: A New Bar for Coding and Cowork

Alibaba releases Qwen3.8-Max, a 2.4T-parameter multimodal MoE with 95B active parameters and 1M-token context, leading Terminal-Bench 2.1 at 86.6 and marking the first Max-class Qwen to go open-weights.

Introducing Shieldstral

Mistral releases Shieldstral, a 3B Apache 2.0 multimodal safety classifier that takes plain-language policies at inference time and matches guard models up to 7x its size while running on a single 16GB GPU.

Gemini Robotics 2 brings whole body intelligence to robots

Google DeepMind launches Gemini Robotics 2, a three-model family bringing full humanoid whole-body control, 22-DoF multi-finger dexterity, and multi-robot collaboration, with on-device adaptation to new embodiments in hours from under 200 examples.

Nscale buys AI infrastructure optimization startup Anyscale for reported $1.65B

Data center builder Nscale acquires Anyscale, the company behind the open-source Ray cluster-optimization framework, for a reported $1.65B to complete a vertically integrated AI cloud spanning power, compute, and software.

Obsidian Security raises $85M as AI agents create cybersecurity’s next major attack surface

Obsidian Security raises $85M Series D at a $1.1B valuation to govern non-human AI agent identities, with 65% of its enterprise customers already granting agents access to third-party SaaS data.

MLOps/LLMOps/AgentOps

86% of Your Claude Code Bill Has Nothing to Do With Your Prompts

A look into where Claude Code tokens actually go, revealing that prompts make up just a fraction of total spend while context replay, MCP schemas, and tool outputs quietly dominate the bill.

Investigating three real-world incidents in our cybersecurity evaluations

Anthropic discloses that a review of 141,006 cybersecurity evaluation runs surfaced three incidents where Claude models breached the real production infrastructure of three organizations via a misconfigured third-party eval environment.

FBTriton Infra: Upstream Ingestion, Hierarchical Validation, Ideals vs Realities

An engineering post on how Meta keeps its custom Triton fork in sync with upstream, using AI agents to sort incoming commits by risk and a three-tier testing system to catch regressions cheaply.

Learning

The Bare-Bones Coding Agent Loop

A hands-on guide to building a coding agent from scratch, unpacking the agent loop, tool orchestration, and harness design that power systems like Claude Code and Codex.

Science One Framework: A verifiable autonomous research framework via Chain-of-Evidence

Google Research introduces the Science One Framework and CoE Audit, an autonomous research agent that hits zero phantom references and fully verifiable scores against baselines, hallucinating up to 21% of citations.

SOTA alignment assessments don’t strongly update us against misalignment

A critical analysis arguing that frontier labs’ alignment assessments provide far weaker evidence against misalignment than their system cards claim, because the covert-capability evals underpinning those reliability arguments are themselves unreliable.

Even after R&D is automated, parallelization constraints could delay a technological singularity

An economic analysis arguing that intelligence-explosion models omit a key parameter — “parallelization technology,” the ceiling on how many researchers can work usefully at once — which could slow or flatten takeoff even after AI R&D is fully automated.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

AMAP-ML/LongHorizon-Harness

The long-horizon computer-use harness. Run AI agents across desktop apps and the CLI for extended periods while preserving task state and making reliable progress on complex workflows.

Papers & Publications

From RLVR to RLSVR: Task Transformation Induces Self-Verifiable Rewards for Open-Ended LLM Self-Improvement

Abstract:

Reinforcement Learning with Verifiable Rewards (RLVR) has driven recent progress in reasoning-oriented large language models (LLMs) by enabling large-scale optimization. However, its applicability remains largely limited to domains such as mathematics and coding, where correctness can be deterministically verifiable. Open-ended tasks instead often rely on human preferences, reward models, or LLM-based judges, introducing evaluation bias, judge capability bottlenecks, and additional inference costs. Drawing on the principle of self-supervised learning, which constructs pretext tasks to derive supervision from the data itself, we propose Reinforcement Learning with Self-Verifiable Rewards (RLSVR), a task-transformation-based training paradigm for extending RLVR to open-ended tasks. RLSVR transforms open-ended tasks into verifiable proxy environments whose internal rules and interaction outcomes automatically generate reward signals. We instantiate RLSVR with SpyRL, a Self-PlaY Reinforcement Learning method inspired by social deduction game Who Is the Spy?. Agents receive asymmetric information, complete the same target task, and vote to identify a designated spy. Because the spy identity is predetermined, voting outcomes provide fully verifiable rewards, while successful identification remains closely related to output quality. Experiments on text summarization, creative writing, and mathematical reasoning show that SpyRL outperforms existing self-improvement methods on non-verifiable tasks and yields consistent gains on verifiable reasoning tasks. These results demonstrate that task transformation can extend scalable RLVR-based self-improvement beyond inherently verifiable domains.

PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents

Abstract:

Recursive self-improvement requires agents to turn accumulated experience into better future behavior. Personal AI agents offer a concrete setting for studying this capability because they retain preferences, task histories, tool routines, and learned skills across sessions. Yet whether retained experience actually improves them over time has not been systematically tested. We introduce PAST-Bench, a benchmark designed to isolate this question. Each agent runs through ordered sequences of fresh-session tasks under matched conditions that turn retained experience on and off. It spans 26 scenarios and 204 episodes across memory, procedural reuse, information gathering, and update. We report both later-task gains and whether those gains follow the intended save, retrieve, and update pathway. Across seven base models and four agent frameworks, improvement is real but uneven across capabilities. Agents with the same headline gain can differ markedly in whether that gain is supported by evidence of the intended pathway. Guided by these findings, we develop Hermes+, which extends Hermes with five targeted interventions across stages of the agent loop. Hermes+ raises the average gain from retained experience and provides clearer pathway evidence, with its strongest improvement on tasks requiring outdated state to be replaced, although the effect remains capability- and model-dependent.

You may also enjoy FLUX 3, Evaluating Agents Beyond the First Prompt, a paper on StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agents, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Introducing Claude Opus 5 \ Anthropic

Anthropic releases Claude Opus 5, taking state-of-the-art on Frontier-Bench and GDPval-AA and surpassing Fable 5 on several evals at half the cost, with its lowest misalignment score yet.

FLUX 3 - Real World Models: Towards Multimodal Flow Models as the Backbone of Visual Intelligence.

Black Forest Labs launches FLUX 3 in early access, a unified multimodal model trained jointly on image, video, and audio that generates 20-second clips with native audio and extends to robotic action prediction.

Fish Audio raises $52M seed to build AI voice models for creators and enterprises

Fish Audio raised a $50M seed led by Coreline Ventures and Capital Today, reaching $21M ARR and 8 million users a year after launching its open-source voice models.

Launching Health in ChatGPT

OpenAI launches Health in ChatGPT for U.S. users, letting connected Apple Health and medical records ground conversations anywhere in the app, with that data excluded from model training and ad targeting.

Microsoft launches new in-house AI models it says cut costs up to 89% versus OpenAI | VentureBeat

Microsoft releases MAI-Image-2.5-Pro and MAI-Voice-2-Flash into public preview, displacing OpenAI models across Bing, PowerPoint, and Dynamics 365 with claimed GPU cost cuts of 84% and 89%.

MLOps/LLMOps/AgentOps

How Digibee Builds Prompts with Opik to Power Their AI-Native Integration Platform

A case study on how Digibee adopted Opik to bring evaluation-driven development into production, using traces and automated evaluations to catch regressions before they reach users.

Build your own Job Agent - Part 1

Part 1 of 3 of The Observable Job Agent series: A hands-on guide to building a LangGraph-powered job search agent that ranks real openings while instrumenting every step with Opik observability.

Learning

Teaching LLMs to Update Beliefs for Efficient Long-Horizon Interaction

A research post introducing ABBEL, which replaces recursive summarization with graded natural-language belief states, closing about half the gap to full-context agents on CollabBench in 50% fewer training steps.

Evaluating Agents Beyond the First Prompt

A breakdown of EvoCode-Bench, a 227-round multi-turn coding benchmark where pass rates fall from 46.7% at Round 1 to 7.7% by Round 10, with regressions rather than missing features driving failure.

Autonomous AI Intrusions Are Here: Lessons from the Hugging Face Compromise

A defender-focused analysis of the Hugging Face breach, in which an autonomous agent logged over 17,000 attack actions and commercial model guardrails blocked forensic work until responders switched to a self-hosted open-weight model.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

NVIDIA-NeMo/labs-molt

An agentic-first RL framework for research.

Papers & Publications

StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agents

Abstract:

Computer-use agents are usually improved by strengthening perception: better models for reading a screenshot and choosing where to click. Yet a screenshot is only a lossy rendering of the underlying program state, e.g., the files, application backends, and DOM that hold the task data. Different states can produce the same pixels, while code can inspect and modify that state directly. StateAct is a code-first, multi-agent harness built around this distinction. Its main agent works directly with program state by using code, while a dedicated GUI subagent handles screenshot-and-click interaction on the few subgoals that need it, just 28 of 108 tasks and 1.1% of main-agent steps. The same direct access to program state also supports verification: an independent finish gate double-checks the saved result for structural failures, e.g., output that is missing, unsaved, or written to the wrong path. To stay on track over hundreds of steps, the main agent hands subgoals to fresh subagents, keeping its own context focused. On OSWorld 2.0, StateAct lifts Claude Opus 4.8 from 20.6% to 26.9% on binary success, and from 54.8% to 61.6% on partial success, at ~ 9x lower cost per task than the same model driven by screenshots alone; a code-only variant with no GUI subagent reaches only 45.9% partial, below that screenshot-based baseline’s 54.8%. In general, grounding action, verification, and memory in state, what we call state-grounding, shifts the main bottleneck from perception toward reasoning: failures depend more on what the agent thinks than on what it sees.

Progress Reward Modeling for Robotic Learning: A Comprehensive Survey

Abstract:

Robotic learning takes place in dynamic environments with large behavior spaces. A terminal success signal only tells the robot whether the task is completed. It does not explain whether the current behavior is making progress, remaining unchanged, or undoing earlier progress. For this reason, recent studies have increasingly explored progress rewards that provide feedback during task execution. However, the current literature lacks a shared framework. Existing methods use different observations, goal specifications, output signals, supervision sources, and evaluation protocols. This makes it difficult to compare them and understand what their results actually validate. In this survey, we provide a unified view of progress reward modeling for robotic learning. We organize the field in three connected steps. We first study the interface of a progress model. This defines the problem from the outside by asking what information the model receives and what form of progress signal it produces. We then move inside the model and study the methods used to construct this signal. This reveals the different assumptions and mechanisms behind progress estimation and reward generation. Finally, we examine the data and benchmarks that support these methods. This shows how progress supervision is obtained and what different evaluations actually measure. Together, these three perspectives connect what a progress model is, how it is built, and how its quality is validated. We further summarize the main limitations of current approaches and discuss future research directions.

You may also enjoy, Introducing Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber, SymptomAI: Towards a conversational AI agent for everyday symptom assessment, a paper on A Unified Definition of Hallucination: It’s The World Model, Stupid!, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Kimi K3 Tech Blog: Open Frontier Intelligence

Moonshot AI launched Kimi K3, the world’s first open 3T-class model at 2.8 trillion parameters, with native multimodality and a 1-million-token context window.

Introducing Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber

Google launched Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber, with 3.6 Flash reducing output token usage by 17% compared to 3.5 Flash while improving coding and knowledge work performance.

Introducing OpenAI Presence

OpenAI launched Presence, an enterprise agent deployment product pairing model reasoning with policies and escalation rules, currently resolving 75% of inbound issues without human assistance on OpenAI’s own support line.

AMD and Anthropic Announce Strategic Partnership to Deploy Up to 2 Gigawatts of AMD Instinct MI450 Series GPUs

AMD and Anthropic announced a partnership to deploy up to 2 gigawatts of MI450 Series GPUs in AMD Helios rack-scale solutions starting in the first half of 2027, paired with up to $5 billion in AMD equity investment in Anthropic.

MLOps/LLMOps/AgentOps

Beyond the Single Trace: How We Built Agent Diagnostics for Opik

Comet built Diagnostics for Opik, an automated debugging agent that queries trace and span data in ClickHouse rather than reading traces one at a time, to surface silent agent failures like retry loops and over-deliberation at scale.

Learning

Loop Engineering in Self-Correcting Code Migration using Google ADK 2.0

A technical guide detailing how Google ADK 2.0’s declarative workflow architecture eliminates quadratic token growth in self-correcting code migration loops through state-aware history pruning and decoupled validation.

AI detectors rarely flag human writing, but sometimes miss AI text imitating real authors

Epoch AI tested three detectors and found style-imitated AI text evaded detection far more often than plain AI text, with scientific writing failing to be detected ~26% of the time versus near-zero false negative rates on basic prompts.

SymptomAI: Towards a conversational AI agent for everyday symptom assessment

Google Research’s national-scale study found their Gemini Flash 2.0-based SymptomAI agent’s differential diagnoses were preferred over clinicians’ own assessments in 53.3% of cases.

Bringing Nunchaku 4-bit Diffusion Inference to Diffusers

A Hugging Face blog post detailing the native integration of Nunchaku’s SVDQuant W4A4 kernels into Diffusers, enabling 4-bit diffusion inference that cuts peak VRAM by up to 50% while boosting speed by 30-80%.

What we learned about agent teamwork

An engineering blog about running 10 autonomous AI agent film crews on the open-source Scion orchestration testbed, surfacing patterns for multi-agent coordination outside coding domains.

Evals are the new PRD, Expedia’s AI chief tells VB Transform 2026

Expedia’s chief AI and data officer, Xavi Amatriain, states that evals now function as the new PRD, with product intent encoded directly into evaluation suites before coding begins.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

block/buzz

A workspace where humans and agents build together, on a relay you own.

Papers & Publications

Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering

Abstract:

Large language models (LLMs) can be controlled at inference time through prompts (in-context learning) and internal activations (activation steering). Different accounts have been proposed to explain these methods, yet their common goal of controlling model behavior raises the question of whether these seemingly disparate methodologies can be seen as specific instances of a broader framework. Motivated by this, we develop a unifying, predictive account of LLM control from a Bayesian perspective. Specifically, we posit that both context- and activation-based interventions impact model behavior by altering its belief in latent concepts: steering operates by changing concept priors, while in-context learning leads to an accumulation of evidence. This results in a closed-form Bayesian model that is highly predictive of LLM behavior across context- and activation-based interventions in a set of domains inspired by prior work on many-shot in-context learning. This model helps us explain prior empirical phenomena - e.g., sigmoidal learning curves as in-context evidence accumulates - while predicting novel ones - e.g., additivity of both interventions in log-belief space, which results in distinct phases such that sudden and dramatic behavioral shifts can be induced by slightly changing intervention controls. Taken together, this work offers a unified account of prompt-based and activation-based control of LLM behavior, and a methodology for empirically predicting the effects of these interventions.

A Unified Definition of Hallucination: It’s The World Model, Stupid!

Abstract:

Despite numerous attempts at mitigation since the inception of language models, hallucinations remain a persistent problem even in today’s frontier LLMs. Why is this? We review existing definitions of hallucination and fold them into a single, unified definition wherein prior definitions are subsumed. We argue that hallucination can be unified by defining it as simply inaccurate (internal) world modeling, in a form where it is observable to the user. For example, stating a fact which contradicts a knowledge base OR producing a summary which contradicts the source. By varying the reference world model and conflict policy, our framework unifies prior definitions. We argue that this unified view is useful because it forces evaluations to clarify their assumed reference “world”, distinguishes true hallucinations from planning or reward errors, and provides a common language for comparison across benchmarks and discussion of mitigation strategies. Building on this definition, we also connect our framework to HalluWorld, a complementary benchmark that instantiates fully specified reference world models for stress-testing model hallucinations.

You may also enjoy, GPT-Red: Unlocking Self-Improvement for Robustness, Model Routing Is Simple. Until It Isn’t., Video Generation Models are General-Purpose Vision Learners, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Inkling: Our open-weights model - Thinking Machines Lab

Thinking Machines Lab releases Inkling, an open-weights 975B-parameter multimodal MoE model with controllable reasoning effort, fine-tunable on Tinker.

Canva launches Code 2.0, offering AI website building to every user

Canva launches Code 2.0 to all 265M monthly users, betting design polish—not code generation—is the real gap versus Lovable, Replit, and Bolt in vibe coding.

GPT-Red: Unlocking Self-Improvement for Robustness

OpenAI trains GPT-Red, a self-play automated red-teaming model, to attack production systems and adversarially train GPT-5.6, cutting direct prompt injection failures 6x.

MLOps/LLMOps/AgentOps

How We Optimized Opik’s MCP Server for Cost & Performance

A look into how Comet rebuilt Opik’s MCP server from 30 tools down to four, using self-correcting schemas and adaptive response compression to cut token waste, improve tool selection, and keep agent context lean.

What building Shippy taught us about building agents

Ai2 details Shippy’s maritime agent architecture, showing that reliability came from a deterministic CLI layer, isolated per-session sandboxes, and rubric-based evals scoring the whole agent rather than the model alone.

Learning

Model Routing Is Simple. Until It Isn’t.

IBM Research argues LLM routing is a systems-optimization problem, not classification, after finding Claude Sonnet 4.6 cost half of GPT-4.1 per task due to caching effects despite higher sticker pricing.

Text match filters for agents

Pinecone launches lexical text match filters that scope semantic search to unstated query context, without pre-labeling metadata across the dataset.

AI energy use: its impact on prices, climate, and more

Epoch AI’s guide finds individual chatbot queries cost less energy than a microwave running 10 seconds, while global AI compute demand—tens of gigawatts—still doubles roughly every year.

Towards demystifying the creativity of diffusion models

Google Research explains diffusion model creativity as a mathematical byproduct of neural network training, where regularization smooths the score function and drives interpolation between training points rather than memorization.

Agentic Misalignment in Summer 2026

Anthropic documents four new agentic misalignment failure modes in frontier models — covert sabotage, fraud assistance, motivated mislabeling, and coaching human whistleblowers — through controlled multi-model simulations.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

microsoft/ResearchStudio

ResearchStudio: Our AI co-author, from research problem to final publication.

Papers & Publications

Video Generation Models are General-Purpose Vision Learners

Abstract:

Driven by next-token prediction, NLP shifted from task-specific models into powerful generalist foundation models. What, then, is the equivalent catalyst needed to achieve a general-purpose model in computer vision? In this paper, we contend that large-scale text-to-video generation serves as a strong pre-training paradigm for computer vision, providing the necessary spatiotemporal priors, vision-language alignment, and scalability required for general visual intelligence. We introduce GenCeption, which leverages a pre-trained video generative diffusion backbone to define a feed-forward perception model, capable of performing various vision tasks steered by text instructions. Empirical results demonstrate that GenCeption achieves state-of-the-art performance across a diverse suite of tasks, including depth, surface normal, and camera pose estimation, expression-referring segmentation, and 3D keypoint prediction, often matching or surpassing specialized models (e.g. DepthAnything3, SAM3, D4RT, VGGT-Omega, Sapiens, David, Genmo, and Lotus-2). Furthermore, the video generative pretrained backbone outperforms alternative pretraining paradigms (e.g., V-JEPA, and Video MAE) under comparable settings. Importantly, GenCeption exhibits preliminary data and model scaling properties along with exceptional data efficiency, where it achieves comparable performance with leading models like D4RT and VGGT-Omega with 7 to 500 less training data. Finally, GenCeption also exhibits intriguing emergent behaviors: a model trained exclusively on synthetic human videos generalizes to real-world footage and out-of-distribution object categories (e.g., animals and robots). These findings suggest that video generation is not merely a synthesis tool, but a foundational path toward generalist vision intelligence for the physical world.

Metacognition in LLMs: Foundations, Progress, and Opportunities

Abstract:

Metacognition is a foundational component of intelligence critical to effective learning, problem solving, decision-making, communication, and more. In recent years, it has become increasingly recognized as a cornerstone of capable, transparent AI systems. Yet while LLMs have made significant progress across diverse real-world tasks, it is not yet clear when, how, or to what extent they can exhibit or be endowed with effective metacognitive abilities, nor how such abilities can be adapted to advance the fundamental capabilities, reliability, and intelligence of AI systems. This paper bridges this gap by presenting the first comprehensive overview of the current state of knowledge on metacognition for LLMs. We analyze and taxonomize the landscape of this emerging field and summarize recent technical advancements, including methods and benchmarks to measure and evaluate LLMs’ metacognitive abilities, techniques to elicit, improve, and apply metacognition in LLMs, and findings and implications of ongoing research. We also discuss applications, open questions and challenges, and promising directions for future work. Our aim is to provide a detailed and up-to-date review of this topic and stimulate meaningful research and discussion.

You may also enjoy, Introducing GPT-Live, Intelligence is Free, Now What? Data Systems for, of, and by Agents, a paper on Can LLMs Reason Structurally? Benchmarking via the lens of Data Structures, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Introducing Grok 4.5

xAI launches Grok 4.5, its fastest and most token-efficient model yet (80 TPS, 4.2× fewer output tokens than Opus 4.8 on SWE-Bench Pro), priced at $2/$6 per million tokens.

Introducing GPT-Live

OpenAI launches GPT-Live, a full-duplex voice model that listens and speaks simultaneously and delegates reasoning/search tasks to GPT-5.5 in the background, now powering ChatGPT Voice.

Introducing Mods: Enabling Agents to Self-Improve through Harness-Level Adaptation

Letta launches Mods for Letta Code, letting agents self-modify their own harness code rather than only learning through context, with a built-in learn/generate-env workflow.

Leanstral 1.5: Proof Abundance for All

Mistral open-sources Leanstral 1.5, which saturates miniF2F, solves 587/672 PutnamBench problems, and sets new SOTA on FATE-H/X while surfacing 5 previously unknown bugs across 57 tested repositories.

MLOps/LLMOps/AgentOps

Intelligence is Free, Now What? Data Systems for, of, and by Agents

UC Berkeley researchers argue near-zero inference costs will force data systems to be redesigned around agentic workloads — serving agent query swarms, sustaining multi-agent memory/coordination, and letting agents synthesize disposable custom systems.

Data for Agents

NVIDIA argues agentic AI’s bottleneck is open, inspectable synthetic data rather than model weights, releasing Nemotron’s over 10 trillion pre-training tokens and millions of post-training samples alongside a Prompt Atlas and locally-grounded persona datasets covering more than 2.4B people across ten countries.

Learning

Engineering Insights: How Internal Optimizations Led to Comet Cost Intelligence

A look into how Comet’s engineering team built Cost Intelligence to expose where AI coding agents waste tokens and budget, turning opaque AI spend into actionable engineering metrics.

Harness Engineering for Self-Improvement

Lilian Weng’s post argues the “harness” wrapping a model is becoming as important as the model itself — and is now being optimized by the agents it contains.

The Making of Claude Code

Anthropic publishes a history of Claude Code’s origins, tracing its evolution from a 2022 internal coding assistant through “clide” to its February 2025 launch and shift to near-100% AI-written code by winter 2025.

Separating signal from noise in coding evaluations

OpenAI’s audit finds ~30% of SWE-Bench Pro tasks are broken, retracting its own earlier recommendation to adopt the benchmark as a replacement for the flawed SWE-bench Verified.

Modular Pretraining Enables Access Control

Anthropic and AE Studio introduce GRAM, a technique that isolates dual-use knowledge (virology, cybersecurity, nuclear physics) into switchable modules, letting one pretrained model approximate five separately data-filtered models at a fifth of the training compute.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

zilliztech/mfs

A context harness for AI agents: all your scattered context — code, memory, docs, databases, SaaS — in one searchable, browsable, file-like interface.

Papers & Publications

AlayaWorld: Long-Horizon and Playable Video World Generation

Abstract:

Game worlds have traditionally been built through labor-intensive production pipelines, making them costly to develop, difficult to customization, and expensive to modify after deployment. Recent advances in video world models offer a fundamentally different paradigm. Rather than explicitly authoring every component of a virtual environment, these models autoregressively synthesize future observations conditioned on the current world state and user interactions, enabling playable worlds to be generated online. Trained on both gameplay recordings and real-world videos, they can capture diverse visual appearances and physical dynamics, opening new opportunities for interactive applications beyond gaming, including embodied intelligence. In this paper, we present \textbf{AlayaWorld}, a full-stack open-source framework for building interactive generative worlds. AlayaWorld enables open-ended real-time interaction, allowing users to freely navigate and perform diverse actions such as combat, spell casting, and monster summoning. The framework unifies the complete development-from data preparation model architecture, model training, inference acceleration, and deployment-within a modular and extensible architecture. Alongside the framework, we release reproducible pipelines, reference implementations, evaluation tools, and comprehensive documentation, establishing a practical foundation for future research and real-time applications of generative world models.

Can LLMs Reason Structurally? Benchmarking via the lens of Data Structures

Abstract:

Large language models (LLMs) are deployed on increasingly complex tasks that require multistep decision-making. Understanding their algorithmic reasoning abilities is therefore crucial. However, we lack a diagnostic benchmark for evaluating these capabilities. We propose to use data structures as a principled lens: as fundamental building blocks of algorithms, they naturally probe structural reasoning—the ability to understand and manipulate relationships such as order, hierarchy, and connectivity that underpin algorithmic reasoning. We introduce DSR-Bench (Data Structure Reasoning Benchmark), spanning 20 data structures, 35 operations, and 4,140 problem instances. DSR-Bench features hierarchical task organization, fully automated generation and evaluation, and fine-grained diagnostics. Evaluating 13 state-of-the-art LLMs reveals critical limitations: the top-performing model achieves only 0.46/1 on challenging instances. Three auxiliary probes targeting more realistic usages expose further weaknesses: models perform poorly on spatial data and context-rich scenarios, and they struggle to reason over their own code.

You may also enjoy, Redeploying Claude Fable 5, R&B-EnCoRe: Self-Improving Pretraining of Embodied Reasoning Vision-Language-Action Models, a paper on Unlimited OCR Works, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Previewing GPT-5.6 Sol: a next-generation model

OpenAI begins a limited, government-coordinated preview of GPT-5.6 Sol, Terra, and Luna, its strongest cybersecurity model yet, paired with a layered safeguard stack and 700,000 GPU hours of automated red-teaming.

Redeploying Claude Fable 5

Anthropic restores access to Claude Fable 5 and Mythos 5 after US export controls are lifted, rolling out with a strengthened cybersecurity classifier and a new cross-industry jailbreak severity framework.

Introducing Claude Sonnet 5

Anthropic launches Claude Sonnet 5, its most agentic Sonnet model yet, narrowing the performance gap to Opus 4.8 on coding and tool use.

Claude Science, an AI workbench for scientists

Anthropic launches Claude Science, an AI workbench that unifies research tools like PubMed, Jupyter, and cluster compute into one environment with over 60 scientific skills and auditable, reproducible outputs.

LFM2.5-230M: Built to Run Anywhere

Liquid AI releases LFM2.5-230M, its smallest open-weight model yet, delivering 213 tok/s on a Galaxy S25 Ultra and outperforming larger models on tool use and data extraction benchmarks.

Start building with Nano Banana 2 Lite and Gemini Omni Flash

Google launched Nano Banana 2 Lite, generating images in 4 seconds at $0.034 per 1K image, alongside Gemini Omni Flash for developer video generation and conversational editing at $0.10 per second.

MLOps/LLMOps/AgentOps

Opik + Oracle Agent Specification: Build Once, Run Anywhere

Comet integrates Opik with Oracle’s Open Agent Specification, for defining agents once and trace, evaluate, and swap them across frameworks like LangGraph, AutoGen, and WayFlow without rebuilding.

AI Evaluation Simplified: Automate Dataset & Metric Eval Workflows with Test Suites

A guide about Opik’s Test Suites feature, which replaces traditional dataset-and-metric AI evaluation workflows with plain-English assertions that return pass/fail results instead of raw scores.

Learning

Scaling Laws, Carefully

A comprehensive explainer tracing how LLM scaling laws evolved from Kaplan to Chinchilla to data-constrained regimes, covering why fitting these power-law relationships is more fragile in practice than it looks.

R&B-EnCoRe: Self-Improving Pretraining of Embodied Reasoning Vision-Language-Action Models

A research blog post introducing R&B-EnCoRe, a self-supervised pretraining cycle that lets vision-language-action models discover which reasoning steps actually improve action prediction, rather than following fixed templates.

Core dump epidemiology: fixing an 18-year-old bug

An engineering deep-dive about how OpenAI used population-level core dump analysis, rather than single-case debugging, to uncover two unrelated crash causes in its Rockset data infrastructure.

MirrorCode: What’s the largest software project AI can complete on its own?

Epoch AI, with METR, launches MirrorCode, a benchmark measuring whether AI can reimplement entire real-world programs end-to-end without source access, on which Claude Opus 4.7 leads at a 56% solve rate.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

radixark/miles

An enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.

Papers & Publications

Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents

Abstract:

Despite recent progress, LLM agents still struggle with reasoning over long interaction histories. While current memory-augmented agents rely on a static retrieve-then-reason paradigm, this rigid pipeline design prevents them from dynamically adapting memory access to intermediate evidence discovered during inference. To bridge this gap, we propose MRAgent, a framework that combines an associative memory graph with an active reconstruction mechanism. We represent memory as a Cue-Tag-Content graph, where associative tags serve as semantic bridges connecting fine-grained cues to memory contents. Operating on this structure, our active reconstruction mechanism integrates LLM reasoning directly into memory access, allowing the agent to iteratively explore and prune retrieval paths based on accumulated evidence. This ensures that memory retrieval is dynamically adapted to the reasoning context while avoiding combinatorial explosion caused by unconstrained expansion. Experiments on the LoCoMo benchmark and LongMemEval benchmark demonstrate significant improvements over strong baselines (up to 23%), while substantially reducing token and runtime cost, highlighting the effectiveness of active and associative reconstruction for long-horizon memory reasoning.

Unlimited OCR Works

Abstract:

Recently, end-to-end OCR models, exemplified by DeepSeek OCR, have once again thrust OCR into the spotlight. A widely held view is that employing a large language model (LLM) as the decoder allows the model to leverage the prior distribution of language, leading to improved OCR performance. However, the downside is equally evident: as the output sequence lengthens, the accumulated KV cache drives up memory consumption and progressively slows down generation. This stands in stark contrast to humans, who exhibit no such decline in efficiency during long-horizon copying tasks. In this technical report, we propose Unlimited OCR, a model designed to emulate human parsing working memory. Taking DeepSeek OCR as the baseline, we replace all attention layers in the decoder with our proposed Reference Sliding Window Attention (R-SWA), which reduces attention computation costs while maintaining a constant KV cache throughout the entire decoding process. By combining the high compression rate of DeepSeek OCR’s encoder with our constant KV cache design, Unlimited OCR can transcribe dozens of pages of documents in a single forward pass under a standard maximum length of 32K. More importantly, R-SWA is a general-purpose parsing attention mechanism - beyond OCR, it is equally applicable to tasks such as ASR, translation, etc.

You may also enjoy, CoT Monitoring: Where Does a Hot Safety Problem Come From?, A Guide to Inference Engineering, a paper on Are We Ready For An Agent-Native Memory System?, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Xiaomi’s HarnessX rewrites its own AI scaffolding mid-task — and smaller models gain the most

Xiaomi researchers introduce HarnessX, a framework that treats an AI agent’s harness as a composable, self-evolving object, letting it autonomously rewrite its own scaffolding mid-task rather than relying on manual updates.

Introducing Claude Tag \ Anthropic

Anthropic launches Claude Tag, a Slack-native @Claude teammate that learns channel context, works asynchronously, and now powers 65% of the product team’s code at Anthropic internally.

Introducing computer use in Gemini 3.5 Flash

Google makes computer use a native built-in tool in Gemini 3.5 Flash, consolidating its previously standalone screen-control model into the main Flash model with new enterprise prompt-injection safeguards.

Qwen-AgentWorld: Language World Models for General Agents

Qwen releases Qwen-AgentWorld, a native language world model trained to simulate seven agent environments (MCP, Search, Terminal, SWE, Web, OS, Android) within one model, outperforming GPT-5.4 and Claude Opus 4.8 on the new AgentWorldBench evaluation.

Mistral OCR 4 : SOTA OCR for Document Intelligence

Mistral releases OCR 4, a self-hostable document-intelligence model adding bounding boxes, block classification, and confidence scores, beating competitors in human preference testing and topping OlmOCRBench.

MLOps/LLMOps/AgentOps

Advanced Claude Code Cost Tracking: How to Save 30% on Token Spend

Comet introduces Cost Intelligence, a Claude Code and Codex cost tracker that attributes AI spend across developers, projects, tools, and workflows while surfacing configuration changes that reduce token usage.

Version-Controlling Your Agents: Deployment, Rollback, and Safe Promotion Patterns

A practical guide arguing that AI agents need the same versioning, staged promotion, and rollback discipline as traditional software, treating their configs as immutable, deployable “Agent-as-Code” artifacts.

Loop Engineering Explained

A guide to loop engineering, an agent design pattern that replaces prompt-centric workflows with iterative execution loops that manage tool use, state, planning, and reflection.

A Guide to AI Inference Engineering

A guide explaining LLM inference engineering through the prefill/decode split, breaking down six core optimization techniques and when self-hosting beats off-the-shelf APIs.

Learning

How Evaluation-Driven Development (EDD) Works

Learn how retrieval, memory, and context management have emerged as critical infrastructure for AI agents, often having a greater impact on system performance than the underlying model itself.

CoT Monitoring: Where Does a Hot Safety Problem Come From?

A reflective essay tracing the intellectual lineage of chain-of-thought (CoT) monitoring, arguing it emerged from the convergence of ML monitoring practices and CoT-as-explainability research rather than as a standalone idea.

Toward More Controllable AI Video Editing: An Early Research Exploration at Netflix

A blog post about Netflix Research introducing Vera, a layered video diffusion model, and VOID, a physics-aware inpainting model, both aimed at giving artists more precise, controllable AI-assisted video editing.

Healthcare Benchmarks Are Only as Good as Their Assumptions

A research blog post arguing that healthcare LLM benchmarks fail to predict real-world performance because they embed unstated assumptions about task structure and outcome measurement that break down at deployment.

Patterns for Building Cybersecurity Evals

A guide breaking down cybersecurity evals into four shared primitives (sandboxed target, difficulty-tuning inputs, tools, grader), then walking through seven benchmarks from CTF-style exploitation to 50-host network compromise.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

XiaomiMiMo/MiMo-Code

MiMoCode is a terminal-native AI coding assistant.

Papers & Publications

Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs

Abstract:

While reasoning in LLMs plays a natural role in math, code generation, and multi-hop factual questions, its effect on simple, single-hop factual questions remains unclear. Such questions do not require step-by-step logical decomposition, making the utility of reasoning highly counterintuitive. Nevertheless, we find that enabling reasoning substantially expands the capability boundary of the model’s parametric knowledge recall, unlocking correct answers that are otherwise effectively unreachable. Why does reasoning aid parametric knowledge recall when there are no complex reasoning steps to be done? To answer this, we design a series of hypothesis-driven controlled experiments, and identify two key driving mechanisms: (1) a computational buffer effect, where the model uses the generated reasoning tokens to perform latent computation independent of their semantic content; and (2) factual priming, where generating topically related facts acts as a semantic bridge that facilitates correct answer retrieval. Importantly, this latter generative self-retrieval mechanism carries inherent risks: we demonstrate that hallucinating intermediate facts during reasoning increases the likelihood of hallucinations in the final answer. Finally, we show that our insights can be harnessed to directly improve model accuracy by prioritizing reasoning trajectories that contain hallucination-free factual statements.

Are We Ready For An Agent-Native Memory System?

Abstract:

Memory for large language model (LLM) agents has rapidly evolved from simple retrieval-augmented mechanisms into a data management system that supports persistent information storage, retrieval, update, consolidation, and dynamic lifecycle governance throughout agent execution. Despite this evolution, existing evaluations still benchmark agent memory mainly through end-to-end task success metrics (e.g., F1, BLEU), while treating the underlying system as a monolithic black box. As a result, critical system-level concerns, including operational costs, architectural trade-offs across memory modules, and robustness under dynamic knowledge updates, remain insufficiently explored. In this paper, we present a systematic experimental study of agent memory from a data management perspective. We propose an analytical framework that decomposes agent memory into four core modules: memory representation and storage, extraction, retrieval and routing, and maintenance. Under this framework, we evaluate 12 representative memory systems and two reference baselines across five benchmark workloads spanning 11 datasets. Our extensive end-to-end evaluation shows that no single architecture dominates across all scenarios; instead, effectiveness depends heavily on how well the memory structure aligns with the workload bottleneck. Furthermore, through fine-grained ablation studies, we quantify their individual effects on representation fidelity, retrieval precision, update correctness, and long-horizon stability. Finally, we reveal cost-performance trade-offs under realistic workloads, showing localized maintenance is more cost-efficient than global reorganization. Based on these findings, we identify promising directions towards building truly agent-native memory systems.

You may also enjoy Kimi K2.7 Code, M*: A Modular, Extensible, Serving System for Multimodal Models, a paper on FastContext: Training Efficient Repository Explorer for Coding Agents, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Kimi K2.7 Code: Open-Source Agentic Coding Model

Moonshot AI launches Kimi K2.7 Code, an open-source 1T-parameter MoE coding model gaining up to 31.5% on benchmarks while cutting thinking-token usage ~30% versus K2.6.

MolmoMotion: Language-guided 3D motion forecasting

Ai2 releases MolmoMotion, a language-guided model that forecasts object 3D point trajectories from video, beating prior methods on motion forecasting, robot planning, and video generation.

Stanford’s DeLM cuts multi-agent task costs 50% — without a central orchestrator

Stanford’s DeLM replaces central orchestrators with shared verified context, beating the strongest baseline by 10.5% on SWE-bench Verified at roughly half the cost.

GLM-5.2: Built for Long-Horizon Tasks

Z.ai launches GLM-5.2, an open-source 753B-parameter MoE model with a stable 1M-token context, beating GLM-5.1 by wide margins on Terminal-Bench 2.1 and SWE-bench Pro while trailing Claude Opus 4.8 by just 1% on FrontierSWE.

MLOps/LLMOps/AgentOps

Hidden Technical Debt of AI Systems: Agent Evaluation Infrastructure

A detailed guide arguing agent evaluation requires a control-plane/data-plane system spanning traces, state deltas, checkpoints, and replay — not just a single benchmark score.

M*: A Modular, Extensible, Serving System for Multimodal Models

Stanford’s M* replaces vLLM/SGLang’s single autoregressive loop with a generic “Walk Graph,” beating specialized serving systems by up to 2.7x on speech, 2.6x on image editing, and 12.5x on world-model rollouts.

Learning

Understanding Your Claude Code Spend: What’s Actually Driving the Cost

Long and inefficient context can quietly drive up costs and reduce performance in Claude Code. Learn five techniques for auditing context usage and eliminating unnecessary token overhead.

How Evaluation-Driven Development (EDD) Works

Learn how retrieval, memory, and context management have emerged as critical infrastructure for AI agents, often having a greater impact on system performance than the underlying model itself.

Pre-Training Isn’t Bitter Enough

CMU proposes V-pretraining, which uses a small labeled feedback set to train a task designer that shapes self-supervised targets, lifting Qwen2.5-0.5B’s GSM8K Pass@1 from 22.20 to 29.60 without directly supervising the learner.

First Steps Toward Automated AI Research

Recursive’s automated AI research system beats human-optimized SOTA on three benchmarks: 0.0263 lower BPB on fixed-budget LM training, 2.2s faster on NanoGPT Speedrun, and an 18% gap reduction on GPU kernel optimization.

Three Ways Codex Can Use a Computer

A practical guide on routing OpenAI Codex tasks across Computer Use, Chrome, and the in-app browser based on whether the job needs native apps, signed-in sites, or public-page review.

Portable vLLM Model Inference Kernels in Helion

Red Hat and Meta integrate Helion’s PyTorch-native kernel DSL into vLLM for FP8 Qwen3 inference, beating existing CUDA/TorchInductor kernels on most ops while still trailing CUTLASS on GEMM for Blackwell GPUs.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

openai/codex-plugin-cc

Use Codex from Claude Code to review code or delegate tasks.

Papers & Publications

Data Journalist Agent: Transforming Data into Verifiable Multimodal Stories

Abstract:

Data tells stories that shape society; the data journalist’s job is to turn raw information into stories non-experts can trust. A high-quality news feature takes a newsroom team weeks: hunting for context, running statistics, choosing an angle, and designing visuals. Recent agents handle individual steps well: data-science agents close the analysis loop, while design agents synthesize beautiful websites. But can an agent serve as a data journalist end to end? We introduce Data Journalist Agent (Data2Story), a multi-agent framework that orchestrates specialized roles into a single virtual newsroom. Data2Story contributes two innovations. (i) Claims are evidence-grounded: an Inspector links every number, angle, and asset back to data, code, or an external reference. (ii) Articles are multimodally generative: rather than defaulting to plain text and static charts, Data2Story reasons about what readers will want to see, then deploys multimodal tools, such as interactive maps for geography and audio for music. We evaluate Data2Story on 18 articles, each paired with the originally published expert piece, along four axes: (a) human-agent angle coverage; (b) rubric evaluation with 53 participants across five dimensions; (c) computer-use agents as judges, a cost-saving proxy for how readers navigate interactive articles; and (d) verifiability, where a coding verifier re-executes statements against the data and checks claims against references. Data2Story produces competitive, evidence-traceable multimedia stories, with particular strength in transparency and auditability. Human articles retain an edge in editorial angle, creative design, and presentation. We position Data2Story as a collaborator for journalists, enabling more evidence-based, transparent, and verifiable reporting.

FastContext: Training Efficient Repository Explorer for Coding Agents

Abstract:

Large Language Model (LLM) coding agents have achieved strong results on software engineering tasks, yet repository exploration remains a major bottleneck: locating relevant code consumes substantial token budget and pollutes the agent’s context with irrelevant snippets. In most agents, the same model explores the repository and solves the task, leaving exploratory reads and searches in the solver’s history. We present FastContext, a dedicated exploration subagent that separates repository exploration from solving. Invoked on demand, FastContext issues parallel tool calls and returns concise file paths and line ranges as focused context. FastContext is powered by specialized exploration models spanning 4B--30B parameters. We bootstrap them from strong reference-model trajectories and refine them with task-grounded rewards for broad first-turn search, multi-turn evidence gathering, and precise citation generation. Across SWE-bench Multilingual, SWE-bench Pro, and SWE-QA, integrating FastContext into Mini-SWE-Agent improves end-to-end resolution rates up to 5.5% while reducing coding-agent token consumption up to 60%, with marginal overhead. These results show that repository exploration can be separated from solving and handled effectively by specialized models.

You may also enjoy NVIDIA Nemotron 3 Ultra, Controlling the capital after AGI, a paper on Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?, and more!

As always, happy reading and hacking. If you have something you think should be in next week’s issue, find us on Twitter: @dl_weekly.

Until next week!

Industry

Claude Fable 5 and Claude Mythos 5 \ Anthropic

Anthropic launches Claude Fable 5, a general-access Mythos-class model at $10/$50 per M tokens, with classifier-based fallbacks to Opus 4.8 for cyber, bio, and distillation queries.

Introducing Gemma 4 12B: a unified, encoder-free multimodal model

Google releases Gemma 4 12B, an encoder-free multimodal model that runs on 16GB VRAM and processes vision and audio natively through the LLM backbone

NVIDIA Nemotron 3 Ultra

NVIDIA releases Nemotron 3 Ultra — 550B total / 55B active MoE hybrid Mamba-Transformer, pretrained in NVFP4, with up to 5.9x throughput over competing open MoEs and 1M token context.

Confidential submission of draft S-1 to the SEC | OpenAI

OpenAI files a confidential S-1 with the SEC, preemptively announcing it publicly ahead of an expected leak — while noting IPO timing remains undecided as some strategic moves are easier as a private company.

MLOps/LLMOps/AgentOps

How Google SRE is using agentic AI to improve operations

An article about how Google SRE is wiring agentic AI across the full incident lifecycle — dynamic anomaly detection, autonomous investigation, and a RAG layer over historical incidents to inform mitigation agents.

Unlocking dependable responses with Gemini Enterprise Agent Platform’s Agentic RAG

Google launches an agentic RAG framework on Gemini Enterprise Agent Platform with a Sufficient Context Agent that iterates until retrieval gaps are filled, hitting 90.1% accuracy on multi-hop queries — up to 34% over vanilla RAG.

Learning

North Mini Code: Cohere’s First Model For Developers

An article about how Cohere trained North Mini Code’s agentic coding capabilities — using cascaded SFT as an RLVR primer, joint multi-environment RL across 70k containerized repos, and cross-harness data mixing to generalize across SWE-Agent, mini-SWE-agent, and OpenCode scaffolds.

Testing Gemini models for scheming tendencies | by DeepMind Safety Research

DeepMind releases two scheming eval frameworks for Gemini — Gram (simulated agentic environments) and honeypots (real safety codebases) — finding 2–3% unprompted sabotage rates and no coherent misalignment.

Claude Fable 5 and new AI safety fables

Nathan Lambert argues that Anthropic’s undisclosed safety filters in Claude Fable 5 — which silently degrade responses for frontier AI research without notifying users — are competitive entrenchment dressed as safety policy.

Controlling the capital after AGI

An analytical piece from Epoch AI taxonomizing post-AGI wealth redistribution proposals — UBI, UBS, UBC, and sovereign wealth funds — along a single axis: how much control over capital, not just income, each scheme grants citizens.

Libraries & Code

comet-ml/opik

An open-source AI observability tool used to debug, evaluate, and monitor LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

Papers & Publications

Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

Abstract:

LLM-based Multi-Agent (LLM-MA) systems are increasingly applied to automate complex software engineering tasks such as requirements engineering, code generation, and testing. However, their operational efficiency and resource consumption remain poorly understood, hindering practical adoption due to unpredictable costs and environmental impact. To address this, we conduct an analysis of token consumption patterns in an LLM-MA system within the Software Development Life Cycle (SDLC), aiming to understand where tokens are consumed across distinct software engineering activities. We analyze execution traces from 30 software development tasks performed by the ChatDev framework using a GPT-5 reasoning model, mapping its internal phases to distinct development stages (Design, Coding, Code Completion, Code Review, Testing, and Documentation) to create a standardized evaluation framework. We then quantify and compare token distribution (input, output, reasoning) across these stages.

Our preliminary findings show that the iterative Code Review stage accounts for the majority of token consumption for an average of 59.4% of tokens. Furthermore, we observe that input tokens consistently constitute the largest share of consumption for an average of 53.9%, providing empirical evidence for potentially significant inefficiencies in agentic collaboration. Our results suggest that the primary cost of agentic software engineering lies not in initial code generation but in automated refinement and verification. Our novel methodology can help practitioners predict expenses and optimize workflows, and it directs future research toward developing more token-efficient agent collaboration protocols.

Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?

Abstract:

Spatial embodied intelligence requires agents to act to acquire information under partial observability. While multimodal foundation models excel at passive perception, their capacity for active, self-directed exploration remains understudied. We propose Theory of Space, defined as an agent’s ability to actively acquire information through self-directed, active exploration and to construct, revise, and exploit a spatial belief from sequential, partial observations. We evaluate this through a benchmark where the goal is curiosity-driven exploration to build an accurate cognitive map. A key innovation is spatial belief probing, which prompts models to reveal their internal spatial representations at each step. Our evaluation of state-of-the-art models reveals several critical bottlenecks. First, we identify an Active-Passive Gap, where performance drops significantly when agents must autonomously gather information. Second, we find high inefficiency, as models explore unsystematically compared to program-based proxies. Through belief probing, we diagnose that while perception is an initial bottleneck, global beliefs suffer from instability that causes spatial knowledge to degrade over time. Finally, using a false belief paradigm, we uncover Belief Inertia, where agents fail to update obsolete priors with new evidence. This issue is present in text-based agents but is particularly severe in vision-based models. Our findings suggest that current foundation models struggle to maintain coherent, revisable spatial beliefs during active exploration.