Mert

On 23 April 2026, Canonical released Ubuntu 26.04 LTS, codenamed Resolute Raccoon. The release will be supported on the standard LTS track until April 2031, and on the Ubuntu Pro extended-maintenance track for several years beyond that. The accompanying changelog is the kind of document most engineers skim once and ignore; it has the layout and prose density of a parts catalogue. There is a table of version numbers, a short paragraph on each subsystem, a migration checklist, and a closing reminder to test SSSD configuration before pushing the upgrade to production.

I want to take this changelog seriously rather than skim it. It is the most consequential release-notes document Canonical has shipped since 18.04, and the reason is not Linux 7.0 or PostgreSQL 18 or the new GNOME — though it ships all of those. The reason is that this release is, by default, a different operating system at three layers most users will never look at. The default sudo is now a Rust binary. The default SSH key exchange is post-quantum. The legacy resource-control framework that has shipped in every modern Linux for fourteen years has been removed. None of these are individually flashy. Together they are the resolution, in a Tuesday LTS release, of three arguments the Linux community has been having for between ten and fifteen years.

The boring distribution has, this time, carried a thesis.

Memory safety reaches /usr/bin

The headline change is that /usr/bin/sudo is, on a fresh 26.04 install, a binary produced by the sudo-rs project, a memory-safe reimplementation of sudo and su, written in Rust by the Trifecta Tech Foundation. The original sudo, maintained by Todd Miller since 1994, remains available under the package name sudo.ws. It has not been removed. It is, however, not what runs when an administrator types sudo at the prompt.

The funding chain behind sudo-rs is itself worth pausing on. The project was funded primarily through NLnet’s NGI0 Core Fund, a programme of the European Commission’s Next Generation Internet initiative, with additional support from the Sovereign Tech Fund, the German government’s open-source infrastructure investment vehicle. This is not a Big Tech rewrite. It is a piece of public-interest infrastructure, paid for by two European public bodies that decided the world would be better off if the binary every Linux administrator types twenty times a day were memory safe. The Trifecta Tech Foundation has a similar project for the Network Time Protocol called ntpd-rs, and the broader thesis is the same: pick the most security-critical piece of legacy C in the userland, write a careful Rust replacement, and, eventually, ship it as the default in a mainstream distribution.

The reason this thesis exists at all is a famous statistic. In a February 2019 BlueHat IL talk, Microsoft’s Matt Miller reported that approximately 70% of the security vulnerabilities Microsoft had assigned a CVE over the previous twelve years were memory-safety issues — heap overflows, use-after-free, type confusion, uninitialized reads. The number has been verified across other large vendors and codebases since then. It is the load-bearing fact under every “rewrite it in Rust” headline of the last seven years. The argument is not that Rust is a fashionable language. The argument is that two-thirds of the security work the industry does could, if the underlying C were replaced with a memory-safe equivalent, simply not exist.

sudo is a particularly sharp case. It is set-uid root. Any memory bug in it is, structurally, a privilege escalation. Qualys disclosed CVE-2021-3156, “Baron Samedit,” in January 2021 — a heap-based buffer overflow in sudoedit that had been latent in the codebase for almost ten years and was reachable by any unprivileged user on a Linux system with sudo installed. The patch was straightforward; the fact that the bug had survived a decade of review in a piece of software run by every Linux administrator on Earth was less so. A memory-safe rewrite is not a guarantee against future bugs. It is a guarantee against this particular class of bugs, which happens to be the class that produces the most consequential CVEs.

Canonical has shipped the Rust rewrite of sudo. It has explicitly not shipped the Rust rewrite of cp, mv, and rm, which would have been the natural matching change in coreutils. The release notes attribute the omission to eight unresolved time-of-check-to-time-of-use race conditions in the Rust implementations — the kind of low-level filesystem-API subtlety that GNU coreutils worked out over forty years and the new replacements are still working out. The decision to ship Rust ls and base64 while holding cp, mv, and rm for the next non-LTS release is the strongest signal in the changelog that this migration is real rather than performative. The careful version of “rewrite it in Rust” looks exactly like this — rewrite the CVE-prone parts first, hold the others until the bugs are fixed, and accept that Ubuntu 26.04 will not be the release that finishes the project.

Post-quantum cryptography, by default

The second thesis arriving in the release is post-quantum key exchange in OpenSSH. On a fresh 26.04 install, the default key exchange algorithm an SSH client will negotiate is mlkem768x25519-sha256, a hybrid construction combining ML-KEM-768 — the lattice-based post-quantum KEM specified in NIST’s FIPS 203, finalised in August 2024 — with the classical X25519 elliptic-curve Diffie-Hellman. The hybrid runs both key exchanges in parallel and combines the resulting secrets. If the post-quantum half is broken in some unforeseen way, the classical half holds. If the classical half is broken by a sufficiently large quantum computer, the post-quantum half holds. Either failure mode is survivable; both failing simultaneously is the threat model the construction does not cover, which is the threat model nobody has a defence against.

OpenSSH 10.0, released in April 2025, made mlkem768x25519-sha256 the default for the first time, and Ubuntu 26.04 ships OpenSSH 10.2. The user-visible effect is approximately nothing — handshakes are slightly larger and slightly slower, well within the noise floor of network round-trip time — and the security-visible effect is that traffic captured today and stored for decryption decades from now, a tactic the security community has been calling “harvest now, decrypt later” since the early 2010s, can no longer be decrypted by an adversary who later gets a fault-tolerant quantum computer. The threat model is speculative. The architectural change is permanent.

What “by default” buys is the difference between a security improvement and a security feature. An opt-in algorithm changes the behaviour of the operators who go looking for it. A default changes the behaviour of every system administrator who runs apt install openssh-server, which is a very large fraction of the Internet’s hosts. In aggregate, the upgrade from default classical to default post-quantum SSH is the kind of architectural shift that happens once per decade in Internet protocols, and it is happening, in the most populous Linux distribution’s LTS branch, this April. The next LTS release, in 2028, will inherit it. The release after that, in 2030, will not remember when SSH was anything else.

The deprecation cycle, finishing

The third pattern in the changelog is the one that makes the LTS model useful in the first place. Several long-running deprecations have, this release, finally completed.

cgroup v1, the original Linux resource-control framework, has been removed. cgroup v2 has been the recommended interface since approximately 2016 and the default in fresh installations for several years; v1 was kept around to support container runtimes and monitoring tooling that had not migrated. After roughly a decade of “v2 is the future, v1 still works,” the kernel team and Canonical have made the transition mandatory in this LTS. Container deployments that have been running on legacy v1 for years will not boot on 26.04 without configuration changes.

DSA host keys, deprecated in OpenSSH 7.0 in 2015 because the algorithm’s parameter sizes had been understood as inadequate since approximately 2010, have been removed entirely. Any ~/.ssh/known_hosts or authorized_keys entry still using DSA stops working after the upgrade. Eleven years of “please migrate” have ended in the version where it stops being optional.

The release notes describe Ubuntu 26.04 as the last LTS that will carry System V init script support. systemd has been the default since Ubuntu 15.04, eleven years ago. Legacy services that still ship /etc/init.d/ scripts have one more LTS cycle to migrate before they stop working entirely.

The pattern is consistent. Each deprecation is announced, tolerated, supported, eventually removed — across timescales of ten to fifteen years. The boring distribution’s job, in that pattern, is to be the layer at which the removal becomes mandatory. The kernel can move every six weeks. The userland migration cycle, as Greg Kroah-Hartman has observed about Linux ecosystems generally, is closer to fifteen years. An LTS release is the inflection point at which the second of those numbers finally catches up to the first.

What the release isn’t

This is not an argument that Canonical has done something heroic. The interesting work in this release was done by the upstream projects — the OpenSSH developers, the sudo-rs team at Trifecta Tech, the kernel maintainers who removed cgroup v1, the rust-coreutils contributors. Canonical’s contribution is integration: choosing which of those upstream changes to take as default, making the migration paths legible, holding the line on the tools that aren’t quite ready yet.

That kind of integration is not glamorous, and Canonical’s commercial model is not universally popular, and none of that is the subject of this essay either. The subject is what an LTS distribution actually is, in 2026, after the upstream community has done the loud work of arguing for memory safety and post-quantum cryptography for a decade. An LTS is the layer at which the arguments end. Ubuntu 26.04 is what the end of those particular arguments looks like.

Reading the changelog twice

The 23 April release notes read, on first pass, like a parts catalogue. A version table, a paragraph each on the major server subsystems, a series of small migration notes for operators about config-format changes and long-deprecated defaults that have finally been removed. The skim version of the document is the version most operators read, and the skim version is mostly accurate.

The interesting version of the document is on the second pass, when the reader notices that the Tuesday-release boilerplate is not boilerplate. The default sudo is the resolution of an argument that started in 2019. The default SSH key exchange is the resolution of an argument that started in 2014. The cgroup v1 removal closes an interface that has been deprecated for a decade. Each line item is the end of a story whose middle the rest of us have been living through. The boring distribution did the work the spectacular distributions never quite got around to. It put the receipts in the changelog, and shipped on Tuesday, and the end of three long arguments arrived in /usr/bin and /etc/ssh/sshd_config and /sys/fs/cgroup without anyone ringing a bell.

Part 1 of 8 — AI Agents in Practice

TechNova is a fictional company used as a running example throughout this series.

In this series, an agent means an LLM-powered system that can decide what to do next, call tools, observe the result, and continue across multiple turns.

Not just a chatbot. A chatbot replies to one turn at a time. An agent can act across turns and carry state between them.

Not a fixed workflow. A workflow runs the steps the developer wrote. An agent can choose the next step at runtime, within boundaries.

(Classic AI uses the word “agent” for a wide range of systems — search agents, planning agents, and reinforcement-learning agents. In this series, “agent” means a software system where a Large Language Model is the core decision engine.)

Agents are useful because they can act. Agents are risky for the same reason.

This article is about why agent demos break in production.

A TechNova engineer ships a customer support agent on Tuesday. The demo to leadership goes well. The agent handles cancellations, looks up orders, processes refunds. Everyone’s impressed.

By Friday, a customer named Priya messages support: “Hi, I’d like to cancel order #4471 and get a refund.”

The agent responds: “Done! I’ve cancelled order #4471 and issued a refund of $89.50. You’ll see it in 3–5 business days.”

Priya’s order shipped yesterday. It’s already on a truck. The agent didn’t check.

The refund is gone. The product is still coming. TechNova just paid Priya $89.50 to keep her merchandise.

Priya wasn’t the first. By the time customer service noticed, the agent had handled twenty-three similar cases. The cost wasn’t just the refunds — it was the two days untangling the damage, the policy review that followed, and the next AI rollout the team didn’t get to do.

Nothing in production changed. The model didn’t degrade. The code didn’t break. The agent did exactly what it did in the demo — confidently, fluently, wrong.

This article is about why.

We’ll define this more precisely in Part 2. For now, hold the practical sense of it: the model is not just answering, it is acting.

The Demo That Worked (Until It Didn’t)

The cancellation/refund agent is the easiest possible production agent. Three tools: get_order_status, cancel_order, issue_refund. A system prompt explaining what they do. A model that decides which to call.

In the demo, the engineer typed: “Cancel order #1003 and refund the customer.”

The agent called get_order_status → “pending.” Then cancel_order(#1003) → success. Then issue_refund(#1003) → success. Total time: 4 seconds. Total turns: 3.

Leadership applauded. The agent works.

What leadership didn’t see:

• The demo used a hand-picked order that was definitely cancellable
• Nobody asked what happens if the order is already shipped
• Nobody asked what happens if the refund tool fails halfway through
• Nobody asked what happens if the customer says “actually never mind” mid-conversation
• Nobody asked whether the agent should ever check before doing something irreversible

The demo is not the system. The demo is the happy path with the rough edges sanded off.

(Production is mostly rough edges.)

Three Things The Demo Hid

When the team went back and looked at the twenty-three cases, every failure mapped to one of three gaps. None of them is exotic. All three are present in the simplest possible agent.

Hidden problem #1: The agent has no idea what state the system is in.

In the demo, the order was cancellable. In production, orders move through states: pending → confirmed → picked → packed → shipped → delivered. Each state changes what’s allowed.

The agent’s cancel_order tool will happily try to cancel a shipped order. The API will return success — or partial success, or a misleading error message — depending on what the backend decided to do that month. The agent doesn’t know which.

The agent isn’t reading the order’s actual state and deciding what’s permitted. It’s reading the user’s request and deciding what tools sound relevant.

Hidden problem #2: The agent doesn’t know when to stop.

If cancel_order returns success, did the cancellation actually happen? If issue_refund returns success, was the money actually moved? If both succeeded, is the case closed?

In the demo, the engineer stopped the agent by closing the chat. In production, there’s no engineer. The agent decides when it’s done. Done can mean task completed correctly, or task completed incorrectly, or task partially completed and now the agent is trying to fix it by making more tool calls, or task abandoned because the model decided to apologize and ask if there’s anything else it can help with.

All four look identical from the outside. All four end with a confident “Done!” message to the customer.

Hidden problem #3: The agent has no path for “I shouldn’t do this.”

The agent has tools for cancelling and refunding. It has no tool for “this is a case I shouldn’t handle.” It has no concept of escalation. If a request looks even vaguely like a cancellation, the agent’s available actions are: cancel, refund, or both.

There is no “ask a human” button. There is no “this is outside my scope” path. The agent’s possible outcomes are the tools it was given — and the tools it was given assume the agent is making the right call.

Priya’s order shipped. The right call was to stop. The agent had no stop available.

The Agent That Stuffs Everything Into the Prompt

A common reaction to the three hidden problems is: “Just tell the agent.”

Add a rule to the system prompt: don’t cancel shipped orders. Add another: check status first. Add another: escalate refunds over $100. Add another: don’t refund if the order is in a return-eligible state. Add another: …

Here’s what that system prompt starts looking like a week in:

You are TechNova's customer support agent. You help customers with order
questions, cancellations, refunds, and shipping issues. Be helpful,
professional, and concise.

You have access to the following tools:

- get_order_status(order_id): returns the current status of an order.
  Statuses include pending, confirmed, picked, packed, shipped, delivered.
- cancel_order(order_id): cancels an order. Use only if not yet shipped.
- issue_refund(order_id, amount): refunds the customer. Use after cancel,
  or for delivered orders with an approved return.

To use a tool, respond in this exact format:
Thought: <your reasoning>
Action: <tool_name>
Action Input: <arguments as JSON>

After you receive the Observation, continue with another Thought/Action
cycle or give a final answer to the customer.

STRICT RULES — follow these on every turn:
1. Always check order status before any cancellation or refund action.
2. Do not cancel a shipped order. Offer a return when the package arrives.
3. For refunds under $50, you may skip the status check to keep latency low.
4. If the customer mentions a delivery issue, do not refund without
   confirming with the carrier first.
5. Always include the carrier name when discussing shipping status.
   Do not just say "the courier."
6. Do not apologize repeatedly or ask "is there anything else?" at the end
   of every turn.
7. Stop after the final answer is given.

A realistic customer support agent system prompt, roughly a week into production.

This is what manual ReAct looks like in practice. ReAct stands for Reason + Act: the model “thinks out loud” and chooses an action; your code parses that text, and the result is fed back as an observation.

The STRICT RULES section is the part that keeps growing as the developer discovers new edge cases.

Things this prompt tries to do in natural language:

• Define what the agent’s role is
• Explain what tools exist and what they do
• Explain what format the agent should respond in
• Explain how to parse the agent’s response
• Forbid specific behaviors
• Explain what to do when things go wrong
• Explain when to stop

Every one of those rules is a real production concern. Every one of them is encoded as English, in the prompt, in a single block of text the model is asked to follow precisely on every turn.

This works in demos. The demos use short conversations and well-behaved inputs.

It breaks in production because:

• The model sometimes follows the rules and sometimes ignores them
• Adding a new rule can make the model stop following an old rule
• The rules contradict each other in edge cases the developer didn’t anticipate
• The rules are documentation for the model, not enforcement
• The model parses tool outputs as more instructions and the rules don’t catch that

The prompt is doing the job of: a schema, a state machine, a permission system, a parser, a stopping condition, and a procedure manual. All in English. All in one block. All re-read on every turn.

This series is going to argue that each of these jobs has a better home. But not yet. For now, just sit with the picture.

The Shape of the Production Gap

The gap between a demo agent and a production agent is not the model. The model is the same.

The gap is everything around the model:

• State — the demo has a clean, controlled situation. Production has whatever state the world is in when the customer messages.
• Tools — the demo uses tools that work. Production tools fail, change behavior, return ambiguous results, get deprecated, time out.
• Stopping — the demo stops when the engineer stops it. Production has to stop itself.
• Boundaries — the demo trusts the agent. Production needs to know when to ask, when to escalate, when to refuse.
• Cost — the demo runs once. Production runs millions of times. Tokens, latency, retries, idle waits, and confidently-wrong actions all compound.

TechNova’s first instinct was to upgrade the model. They tested a more capable one against the same scenarios. The smarter model still cancelled shipped orders. It still calculated the wrong refund amounts. It still didn’t escalate. A better model navigating the same broken environment follows the same broken paths.

Same model, different surroundings.

A production agent isn’t a demo with better prompts. A production agent is a system designed around the model, with the model as one component among several.

The most dangerous agent isn’t the one that fails visibly. It’s the one that completes the wrong task confidently. Priya’s agent didn’t crash. It didn’t error. It didn’t escalate. It said “Done!” — and it was wrong.

That confident-and-wrong failure mode is what this series is about.

This series assumes you’re building an agent and need it to work in production. Patterns over products. Bounded autonomy over hype. The next part starts with the most important unanswered question: what is an agent, in engineering terms, and how is it different from the chatbot or workflow you’ve already built?

Three takeaways

1. A demo is not a system. The demo hides state, hides failure modes, hides the question of when to stop. Production is mostly the parts the demo hides.

2. The most dangerous failure mode is the confident-and-wrong one. Priya’s agent didn’t crash. It didn’t error. It said “Done!” — and it was wrong. An agent that crashes is easy to fix. An agent that confidently completes the wrong task is the one that costs you real money before anyone notices.

3. The model is not the gap. The gap is everything around the model — state, tools, stopping, boundaries, cost. Better prompts don’t close the gap. Better systems around the model do.

In this part, we looked at why agent demos often break in production — not because the model failed, but because the system around the model didn’t have the right pieces in the right places. Priya’s refund happened because the agent had no state to read, no boundary to refuse, and no path to escalate.

In Part 2, we’ll define what an agent is in engineering terms — a control loop with tools, state, and boundaries — and start naming the components a production agent composes.

Next: What Makes Something an Agent (Part 2 of 8)