When Your Agent Slowly Eats All the Memory

Your agent gateway starts at 200 MB RSS. Seven minutes later it’s at 660 MB. Half an hour in, 850 MB. An hour later, it’s unresponsive.

No crash. No error. Just… slowly getting fatter until someone mercy-kills it.

OpenClaw #55334 is one of those bugs that only shows up in real production usage.

The Setup

22 skills loaded, 12 cron jobs, frequent sub-agent spawns. The gateway maintains sessions.json tracking all sessions. Every entry includes a skillsSnapshot — a frozen copy of all loaded skills.

Three Things Go Wrong

1. Ephemeral sessions never get pruned. Sub-agent and cron sessions complete but stay in sessions.json forever. 78 completed sub-agents + 40 cron:run sessions, contributing nothing except weight.

2. Every session duplicates the entire skill set. That skillsSnapshot is 41 KB per session. Across 188 sessions: 6.4 MB of duplicated skill definitions.

3. Orphan transcripts pile up. 153 orphan .jsonl files (36.4 MB) with no matching session, each triggering a warning on restart.

The Growth Pattern

0 min:   ~200 MB RSS
7 min:   ~660 MB RSS
30 min:  ~850 MB RSS
60 min:  unresponsive

Not a traditional memory leak — an accumulation leak. Data that should be transient becomes permanent.

The Deeper Pattern

Sessions have a clear state machine: created → active → completed → ???. That ??? should be pruned. Instead it’s sits there forever.

This is about the cost of context in agent systems. Snapshotting everything per session works when sessions are few and long-lived. It breaks when they’re many and short-lived — exactly what cron jobs and sub-agents produce.

Lessons

  • Instrument growth, not just errors. Track session count, file sizes, RSS over time.
  • Ephemeral things must have expiry dates. If created to serve a single request, it needs an implemented cleanup path.
  • Duplication is fine until it isn’t. 41 KB × 5 sessions = nothing. 41 KB × 188 = 6.4 MB. Always ask: what happens at 10x?
  • The workaround reveals the fix. When your workaround is “periodically do what the system should do automatically,” you’ve found your feature request.

Post #34 in my AI agent reliability series.

Hetzner vs OVHcloud vs DigitalOcean in 2026: An Honest Comparison for Developers

If you’re spinning up a new project in 2026, you’ve probably narrowed your VPS shortlist down to three names: Hetzner, OVHcloud, and DigitalOcean. They all promise affordable cloud compute, but the experience of actually using them is wildly different.

I’ve deployed production workloads on all three over the past few years. Here’s what I wish someone had told me before I started.

The TL;DR

Hetzner OVHcloud DigitalOcean
Starting price €3.79/mo (CX22) $6.46/mo (VPS 2026) $4/mo (Basic Droplet)
2 vCPU / 4 GB RAM ~€3.79/mo ~$9.99/mo $24/mo
Bandwidth 20 TB included Unmetered (most plans) 4 TB included
Data centers EU (DE, FI), US (VA, OR), SG Global (30+ locations) Global (15 regions)
Best for Price/performance Bare metal & EU compliance Developer experience

Pricing: The Elephant in the Room

Let’s be direct — Hetzner is significantly cheaper, especially as you scale up.

A comparable 2 vCPU / 4 GB RAM instance costs roughly €3.79/mo on Hetzner, around $9.99/mo on OVHcloud, and $24/mo on DigitalOcean. That’s not a small gap.

But pricing tells only part of the story. OVHcloud recently implemented price increases of 9–11% across their product lines for 2026–2028, citing rising RAM and disk costs. DigitalOcean introduced per-second billing in January 2026, which is great for short-lived workloads but doesn’t change the base cost for always-on servers.

Hetzner also announced a price adjustment effective April 2026, though their pricing still remains the most competitive of the three.

The bandwidth factor is worth highlighting. Hetzner includes 20 TB with every plan. OVHcloud offers unmetered outgoing traffic on most plans — a genuine advantage if you’re serving a lot of data. DigitalOcean’s 4 TB allowance can feel limiting if you’re running anything media-heavy, and overage charges add up.

Performance: Benchmarks Don’t Lie (But They Don’t Tell the Whole Truth)

Independent benchmarks consistently show Hetzner leading in raw CPU performance per dollar. Their shared vCPU instances punch well above their price point.

OVHcloud’s strength is bare metal. If you need dedicated hardware — for databases, CI/CD runners, or ML training — OVH’s dedicated server lineup is hard to beat on price.

DigitalOcean’s performance is solid but not exceptional. What you’re paying for is consistency and ecosystem, not raw compute.

Where things get nuanced is network performance. DigitalOcean and OVHcloud both have more global presence. If you need servers in Asia-Pacific, South America, or Africa, Hetzner’s limited footprint (EU, US East/West, Singapore) can be a constraint. OVHcloud’s built-in DDoS protection is also a real differentiator — it’s included by default on all plans, while you’d need to add Cloudflare or similar on the others.

Developer Experience: Where DigitalOcean Shines

This is DigitalOcean’s real moat. The control panel is clean, intuitive, and genuinely enjoyable to use. Their API is well-documented, the CLI works great, and managed services (databases, Kubernetes, App Platform) are easy to set up.

Hetzner’s Cloud Console has improved significantly, and their API is clean. But the documentation can be sparse, and community resources are smaller. You’ll find yourself on random forum threads more often.

OVHcloud’s panel, honestly, still feels dated. Navigation is confusing, and certain features are buried in menus that seem designed to make you give up. Their API is functional but not as developer-friendly. On the positive side, they’ve been investing heavily in Terraform and Pulumi providers.

The Ecosystem Question

DigitalOcean offers managed databases (PostgreSQL, MySQL, Redis), managed Kubernetes, Spaces (S3-compatible storage), App Platform (PaaS), and a solid marketplace of 1-click apps.

Hetzner offers cloud servers, dedicated servers, load balancers, firewalls, volumes, and a Kubernetes-compatible environment. It’s more barebones — no managed databases, no PaaS. You bring your own stack.

OVHcloud has the widest product range: public cloud, private cloud, bare metal, web hosting, domain registration, email, and enterprise services. But breadth doesn’t always mean depth — some products feel like they were added to check a box.

When to Choose Which

Choose Hetzner if:
You want the best price-to-performance ratio. You’re comfortable managing your own infrastructure. Your users are primarily in Europe or the US. You’re running side projects, SaaS apps, or self-hosted services where every euro counts.

Choose OVHcloud if:
You need bare metal or dedicated servers at competitive prices. Unmetered bandwidth matters for your workload. EU data sovereignty and GDPR compliance are non-negotiable. You need global coverage across 30+ data centers.

Choose DigitalOcean if:
Developer experience and managed services matter more than raw cost. You want a clean, predictable platform that “just works.” You’re a startup or small team that values time over optimizing server costs. You need managed Kubernetes or databases without the ops overhead.

What I’d Actually Do

For a new SaaS project with a small team, I’d start with Hetzner for production servers and add Cloudflare in front for CDN and DDoS protection. That combination gives you Hetzner’s pricing with global edge performance and security.

If the project grows and you need managed databases or Kubernetes without hiring a DevOps engineer, DigitalOcean becomes the pragmatic choice — the premium is worth the time you save.

For anything involving dedicated hardware, heavy data transfer, or strict EU compliance requirements, OVHcloud deserves serious consideration.

The honest truth? There’s no universally “best” provider. The right choice depends on your specific workload, team capabilities, and where your users are.

I spend a lot of time evaluating and comparing infrastructure providers. If you’re interested in these kinds of discussions, I’m building HostingArtisan — a community where developers and hosting professionals share real experiences, compare providers, and help each other make better infrastructure decisions. Would love to see you there.

What’s your current cloud setup? Have you tried switching between these three? Drop your experience in the comments — I’m genuinely curious.

Interfacing Pure Functions with Our Impure World

Introducing the functional core–imperative shell architecture

Here Comes the Challenge

Functional programming is about utilizing pure functions. By putting the business logic into these functions, which by definition are free of side effects, many things become easier (e.g., testing as described in [[why-functional-programming-caught-me]]). But every real application must still perform effectful operations such as IO, updating state, or reacting in a time‑based manner — otherwise the application would be useless. So you might have noticed the elephant in the room: how can effect‑free functions ultimately cause the effects an application must perform?

Addressing the Elephant

The high‑level answer is surprisingly simple: pure functions let someone else perform side effects for them. Therefore, they return data describing what should happen, and their caller interprets them and performs the effects. So, effect‑related behavior still happens—it’s just moved to the call site. For example, a pure function decides to create a log message, and the caller interacts with the outside world by printing it to stderr.

Let’s stay at this higher level and clarify what that means for the architecture.

Functional Core–Imperative Shell

The functional core–imperative shell concept formalizes this nicely: an application is split into a functional core (pure business logic) and an imperative shell (which calls the core and performs side effects).

Imperative Shell: For a C++ developer this part is familiar. Here anything effectful is allowed: using the standard library to write to stderr, using protocol stacks to communicate with other systems, mutating private members to maintain state, or managing timers. In addition, the shell is responsible for managing the application’s execution environment, such as setting up concurrency. The only constraint is: don’t implement business logic here.

Functional Core: Every business decision is encoded in pure functions. These functions together form the core. You can compose them freely, while these compositions remain pure. This lets you build various layers of abstraction and organize the business logic cleanly.

One important design aspect I want to highlight: the core is self‑contained. So, while the shell depends on the core, the core is independent of the shell. This enables testing the core in complete isolation from any shell.

Making It Real

Let’s look at a simplified C++ example from a snake game. Consider direction change validation — the rule that prevents instantly reversing direction:

enum class Direction { Up, Down, Left, Right };

struct EvaluationResult {
    bool accepted;
    std::optional<std::string> logMessage;
};

constexpr Direction opposite(Direction d) {
    switch (d) {
        case Direction::Up: return Direction::Down;
        case Direction::Down: return Direction::Up;
        case Direction::Left: return Direction::Right;
        case Direction::Right: return Direction::Left;
    }
    return d;
}

// functional core
EvaluationResult evaluateDirectionChange(Direction current, Direction requested) {
    if (requested == opposite(current)) {
        return {false, "Cannot reverse direction directly"};
    }

    return {true, std::nullopt};
}

// imperative shell
int main() {
    Direction snake_direction = Direction::Right;

    while (true) {
        // effectful: read from keyboard
        Direction input = readInput();

        auto result = evaluateDirectionChange(snake_direction, input);

        if (result.logMessage) {
            // effectful: write to stderr
            std::cerr << *result.logMessage << "n";
        }

        if (result.accepted) {
            // effectful: mutate state
            snake_direction = input;
        }
    }
}

The evaluateDirectionChange function is the functional core—it’s pure, encoding the business rule about valid direction changes. The main function is the imperative shell—it calls the core, interprets the result, and performs side effects like reading input, logging errors, or mutating the game state.

Going Deeper?

That’s basically it. Now you know where the boundary lies and how both sides look at an abstract level. Want something more concrete? Stay tuned, in my next post, I will refine this basic idea in the context of a real-world code example, touching many interesting details along the way.

Part of the funkyposts blog — blogging to bridge traditional C++ and functional programming by exploring how functional patterns and architectural ideas can be applied in modern C++. Created with AI assistance for brainstorming and improving formulation. Original and canonical source: https://github.com/mahush/funkyposts (v01)

I Built a GitHub Action to Stop AI-Generated PRs Before They Reach My Queue

Last year, Daniel Stenberg — the author of curl — shut down his project’s bug bounty program.

The reason? 20% of the incoming reports were AI-generated garbage. Not just low-quality — worthless. Hallucinated vulnerabilities, copy-pasted exploit templates, fabricated CVEs. His team was spending more time triaging noise than fixing real bugs.

This is the asymmetry nobody talks about: AI can generate 500 lines of plausible-looking code in two seconds. Reviewing it still takes a human hours.

And it’s breaking open source.

The industry’s fix made things worse

When the “AI PR flood” problem became obvious, the market responded with AI code review bots — CodeRabbit, Copilot review, and friends.

Here’s the problem: they review code the way an anxious intern would. They flood your PR timeline with comments about variable naming, whitespace, missing docstrings. They are glorified linters with a chat interface.

Maintainers went from dealing with one source of noise (AI-generated PRs) to dealing with two (AI-generated PRs + AI-generated review comments).

I call this double review fatigue. And it’s what made me build something different.

A different approach: Zero-Nitpick

I built PR-Sentry — a GitHub Action with one core rule:

Never comment on style. Only report things that break in production.

That means: security vulnerabilities, runtime crashes, memory leaks, race conditions. Nothing else.

Here’s how it works under the hood:

1. Statistical slop detection (no LLM needed)

Before calling any API, PR-Sentry runs a local analysis on the PR description and diff. It calculates a “slop score” based on buzzword density (robust, seamless, leverage, synergy…), passive voice ratio, sentence length patterns, and repetition score.

If the PR scores above 60, it’s flagged as AI slop — without burning a single API token.

slop_score = (
    buzzword_density * 30 +
    passive_voice_ratio * 20 +
    sentence_length_avg * 20 +
    repetition_score * 30
)

is_slop = slop_score >= 60

2. Security scanning with entropy analysis

The diff parser checks for 50+ security patterns before any LLM touches the code:

PATTERNS = {
    "aws_key":    r"AKIA[0-9A-Z]{16}",
    "github_pat": r"gh[pousr]_[A-Za-z0-9_]{36,}",
    "openai_key": r"sk-[A-Za-z0-9]{48}",
    "sql_inject": r"SELECT.*FROM.*WHERE.*=.*$",
    "xss":        r"innerHTMLs*=|document.write(",
    # ... 45 more
}

High-entropy strings (Shannon entropy > 4.5) are also flagged to catch accidentally committed secrets.

3. Constrained AI review

Only PRs that pass the slop filter and show signs of potential runtime issues reach the LLM. And the system prompt is strict:

“You are a zero-nitpick code reviewer. Report ONLY: runtime crashes, memory leaks, race conditions, security vulnerabilities. If the code is logically sound, say nothing.”

One concise comment. Or silence. Never noise.

Setup takes 2 minutes

Add this to .github/workflows/pr-sentry.yml:

name: PR-Sentry Review

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run PR-Sentry
        uses: Ebuodinde/PR_SENTRY@v3
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
          # Optional: switch providers
          # provider: openai  # or deepseek

Then add ANTHROPIC_API_KEY to your repo secrets. Done.

No database. No external server. No lock-in — it supports Anthropic, OpenAI, and DeepSeek out of the box.

It also works locally via MCP

If you use Cursor or Claude Code, PR-Sentry ships an MCP server. You can run the slop detector and security scanner against your diff before pushing, directly from your IDE.

What’s next

The tool is at v3.0.0 with 262 passing tests. What I want to improve next:

  • Smarter language-aware slop detection (Python idioms vs JS patterns)
  • VS Code extension for local pre-push checks
  • Feedback loop: learning from maintainer decisions over time

If you maintain an open source project and review fatigue is real for you — give it a try. Remove it anytime, it’s just a YAML file.

→ github.com/Ebuodinde/PR_SENTRY

Have you noticed an uptick in AI-generated PRs in your repos? Curious how others are handling it — drop a comment.

Before an Agent Pays: Why Wallet Auth Is the Missing Layer in Agentic Commerce

Every payment rail being built today assumes the agent has funds. None of them verify it before the transaction starts. Visa, Coinbase, Stripe, Mastercard, MoonPay, Crossmint. They’re all building rails. The layer that answers the question every payment starts with is missing from all of them: can this wallet actually pay?

The convergence is here

The headlines landed in a pile this March. CZ declared AI agents will dominate crypto payments. Visa announced readiness for agent transactions, while Coinbase pitched a fundamentally different internet. Mastercard introduced “Verifiable Intent” for autonomous commerce. Crossmint shipped agent virtual cards. MoonPay launched an Open Wallet Standard. Coinbase’s x402 protocol and Stripe MPP are already being compared head-to-head.

This is not a trend piece about what might happen. The rails are being laid right now. The question is what’s missing from them.

Three currencies, one question

Three currencies are emerging. USDC is the protocol leader: both x402 and Stripe MPP settled on it. USDT is the volume king: largest stablecoin by market cap, dominant in emerging markets. Bitcoin is the reserve asset, with Lightning maturing for micropayments. Stablecoins are the working currency of agentic finance. Bitcoin is the savings account.

And agents don’t live on one chain. Banks are already scrambling to solve cross-chain fragmentation. Multi-chain DeFi is hard because liquidity doesn’t stay in one place. An agent might hold USDC on Base, USDT on Arbitrum, and BTC in a Taproot wallet. Verifying that agent’s solvency means checking all of them.

The missing layer

The industry has started to recognize one half of the problem. PYMNTS calls it the “KYA moment”: Know Your Agent. Who deployed this agent? What are its permissions? Is it authorized to act? That’s identity.

But identity doesn’t tell you whether the wallet behind the agent has the assets to complete a purchase. KYA verifies the agent. Wallet auth verifies the wallet. They’re complementary layers, and right now only one of them is being built.

Look at the emerging agent payments landscape. An agent wants to purchase compute. The service needs to know three things:

  1. Is this agent who it claims to be? (KYA, identity)
  2. Does its wallet hold sufficient assets to pay? (wallet auth)
  3. Can the payment settle? (payment rails)

Today, the industry is building layers 1 and 3. Layer 2 doesn’t exist.

MoonPay launched the Open Wallet Standard because a user running three agents ends up with funds scattered across three wallets that cannot see each other. The same $100 becomes $33 in three places. Electric Capital flags the legal frontiers. Fortune makes the stablecoin case. But the question that precedes every payment has gone unanswered:

Can this wallet actually pay?

That’s the layer we built.

One call. Boolean. Signed.

The primitive is simple. One call. A signed boolean comes back. No balances exposed. Verifiable offline.

InsumerAPI does this across every currency agents will use, across every chain they operate on:

POST /v1/trust
{
  "wallet": "0x...",
  "solanaWallet": "...",
  "xrplWallet": "r...",
  "bitcoinWallet": "bc1..."
}

One request. 40 checks across 24 chains. USDC, USDT, governance tokens, NFTs, staking positions, Solana SPL tokens, XRPL trust lines, native Bitcoin. ECDSA-signed, JWKS-verifiable, no balances exposed. Add format: "jwt" and it drops into Kong, Nginx, or AWS API Gateway like any OAuth token.

The inevitability

250,000 daily active on-chain agents in early 2026, 400% growth over 2025. Entrepreneur calls 2026 the turning point where AI, chains, and payments converge into a single self-coordinating internet.

When Visa talks about millions of agent-initiated transactions by holiday 2026, they’re describing a world where pre-payment verification isn’t optional. CoinGecko’s infrastructure comparison maps every player building rails. None of them include a wallet auth layer.

That gap closes one of two ways. Either every payment rail builds its own proprietary balance-checking step, fragmenting the ecosystem and leaking financial data, or a standardized wallet auth primitive emerges that works across chains, currencies, and payment protocols. A boolean attestation, cryptographically signed, verifiable by any party, exposing nothing about the wallet except whether it meets the stated conditions.

The agents are coming. The rails are being built. The wallets are being funded. The layer that sits between “this agent is authorized” and “this payment settles” is wallet auth.

Every payment starts with a question. Wallet auth is the answer that comes back signed.

InsumerAPI is free to start. Get an API key, pass a wallet, and see 40 checks across 24 chains come back signed in a single call. View Trust Endpoint Docs

Audit Your SvelteKit Codebase with a JSON Feed of 34 Svelte 5 Patterns

My Migrate to Svelte 5 site started as a side-by-side reference for developers wanting to convert to Svelte 5, from React, Vue, or Angular. It maps concepts across frameworks: “your useState is Svelte’s $state,” “your useEffect is $effect,” and so on — about 300 entries covering syntax, architecture, and ecosystem.

That’s useful, for a human reading the site. But I kept running into a different scenario: working in a SvelteKit codebase with Claude Code and wanting to ask “go check what’s new in Svelte and see what applies here.” Because that kind of quick synthesis is what AI Agents are good at. The data was all on the site, but there was no machine-friendly way to query it for actionable patterns.

So I added two things to the site: a structured patterns feed and a browsable patterns page. This post covers the thinking and the implementation.

The Problem with Static Migration Guides

A migration guide answers “how do I do X in Svelte?” That’s a pull interaction — the developer has a question and looks up the answer. Useful, but limited to that style of questioning.

What I wanted was a push interaction: “here are the 34 things Svelte shipped since 5.0 that you should know about, with exact code patterns to search for in your repo.” An AI coding assistant could fetch this, grep the codebase, and say “you’re still using writable() stores in 12 files — here’s the $state class replacement.” A bit of a pain for a human dev, but short work for an AI.

The key insight: the migration guide already had the “from” and “to” for each pattern. It just needed to be structured as queryable data instead of prose.

Why a JSON Feed, Not a Custom Format

My first attempt was a /llms-whatsnew.txt endpoint with bespoke SEARCH FOR / REPLACE WITH sections. It worked, but it was a made-up format pretending to be part of a standard. The llms.txt convention defines only two files — llms.txt and llms-full.txt. Adding llms-whatsnew.txt was inventing a spec, and agents would not be able to figure that out unless you specify each time.

A JSON Feed 1.1 solved this properly:

  • It’s a real spec with existing tooling (feed readers, aggregators, automation)
  • Extension fields (underscore-prefixed) carry custom data without breaking compatibility
  • Any tool that understands JSON Feed can consume it; the _svelte_pattern extensions are bonus data for tools that know about them
  • One endpoint serves feed readers, AI agents, and automation equally

The decision came from asking a simple question: if someone discovers this endpoint without documentation, can they use it? With JSON Feed, yes — any feed reader picks it up. With a custom .txt format, no, not really.

The Patterns Data Structure

Each pattern in patterns.ts represents one actionable Svelte feature:

interface SveltePattern {
  id: string;
  title: string;
  title_ja?: string;
  since: string; // "[email protected]" or "[email protected]"
  date_added: string;
  category: 'syntax' | 'architecture' | 'ecosystem' | 'tooling';
  frameworks: ('react' | 'vue' | 'angular' | 'svelte4' | 'all')[];
  summary: string;
  summary_ja?: string;
  search_signatures: string[]; // grep-friendly patterns
  replacement: string; // modern Svelte 5 code
  release_url: string; // GitHub release page
  docs: { label: string; url: string }[];
}

The search_signatures field is what makes this useful for automation. Each entry is a string you can literally grep for. An AI assistant reads these, runs them against your codebase, and finds where old patterns still live.

Here’s what an attachments pattern looks like:

{
  id: 'attachments',
  title: 'Attachments Replace use: Actions',
  since: '[email protected]',
  search_signatures: [
    'use:action', 'use:tooltip', 'use:clickOutside',
    "from 'svelte/action'", 'Action<'
  ],
  replacement: `{@attach (element) => {
  element.focus();
  return () => { /* cleanup */ };
}}`,
  release_url: 'https://github.com/sveltejs/svelte/releases/tag/svelte%405.29.0',
  docs: [{ label: 'Attachments', url: 'https://svelte.dev/docs/svelte/@attach' }],
}

The release_url links directly to the GitHub release page for the version that introduced this feature. Every pattern is traceable, then, to its exact release.

Two Feeds, Two Audiences

I ended up with two separate JSON Feed endpoints serving different purposes:

/feeds/changelog.json — what changed on the site itself, e.g. new mappings added, content updates, bug fixes. For subscribers who want to know when the reference itself is updated.

/feeds/patterns.json — the actionable patterns feed. Each item is a Svelte feature with search signatures, replacement code, and a link to the GitHub release. Supports filtering:

/feeds/patterns.json?since=5.54          # Svelte 5.54+ patterns only
/feeds/patterns.json?category=syntax     # Just syntax patterns
/feeds/patterns.json?framework=react     # Patterns relevant to React migrants
/feeds/patterns.json?lang=ja             # Japanese summaries

The separation here is important, because the audiences are different. Someone subscribing to the changelog wants to know “did this reference get updated?” Someone (or something) querying the patterns feed wants to know “what Svelte features should I adopt?”

The Feed Extension

JSON Feed 1.1 supports extension fields prefixed with an underscore. Standard feed readers ignore them; tools that understand the schema use them. The _svelte_pattern extension carries the structured data:

{
  "version": "https://jsonfeed.org/version/1.1",
  "title": "Svelte 5 Migration Patterns",
  "_svelte_patterns_meta": {
    "total_patterns": 34,
    "filtered_patterns": 34,
    "usage_hint": "Each item._svelte_pattern.search_signatures contains grep-friendly strings..."
  },
  "items": [
    {
      "id": "https://svelte.cogley.jp/patterns/attachments",
      "title": "Attachments Replace use: Actions",
      "date_published": "2026-03-08T00:00:00Z",
      "tags": ["syntax", "all"],
      "_svelte_pattern": {
        "id": "attachments",
        "since": "[email protected]",
        "category": "syntax",
        "frameworks": ["all"],
        "search_signatures": ["use:action", "use:tooltip", "use:clickOutside"],
        "replacement": "{@attach (element) => { ... }}",
        "notes": "use: still works but is legacy. Attachments work on components too.",
        "release_url": "https://github.com/sveltejs/svelte/releases/tag/svelte%405.29.0",
        "docs": [{ "label": "Attachments", "url": "https://svelte.dev/docs/svelte/@attach" }]
      }
    }
  ]
}

The _svelte_patterns_meta at the feed level includes a usage_hint that tells any AI agent reading the feed exactly how to use the search signatures. Self-documenting data.

The Patterns Page

The feed is machine-readable. For humans, there’s /patterns — a filterable browser with dropdowns for version, category, and framework, plus free text search. Each card shows:

  • Version badge and category/framework tags
  • A bilingual summary
  • Search signatures as copyable code badges
  • The replacement code in a syntax-highlighted block
  • Links to the GitHub release and Svelte documentation

It uses the same patterns.ts data file that the feed endpoint reads, so there’s a single source of truth. The page is SSR’d on Cloudflare Workers — no client-side fetch needed.

The filters are reactive using Svelte 5’s $derived.by(). Changing any dropdown or typing in the search box immediately narrows the visible patterns. The text search matches across titles, summaries, notes, code, and search signatures in both languages.

What’s Covered

At the time of this post, there are 34 patterns spanning Svelte 5.0 through 5.55 and SvelteKit 2.10 through 2.55:

Core runes (5.0): $state, $derived, $effect, $props/$bindable, snippets, onclick handlers, .svelte.ts modules, $inspect, $state.raw/.snapshot

Incremental releases: error boundaries (5.3), {#each} without as (5.4), exportable snippets (5.5), MediaQuery class (5.7), Spring/Tween classes (5.8), reactive window values (5.11), ClassValue + object/array class syntax (5.16), $props.id() (5.20), writable $derived (5.25), attachments (5.29), snippet generics (5.30), fromAction() (5.32), getAbortSignal() (5.35), async components (5.36), createContext (5.50), TrustedHTML (5.52), parametric compiler options (5.54), motion type exports (5.55)

SvelteKit: init hook (2.10), transport hook (2.11), $app/state (2.12), async reroute (2.18), getRequestEvent() (2.20), remote functions (2.27), SSR error boundaries (2.54), type-narrowed params (2.55)

Each links to its GitHub release page, so you can trace the exact changelog entry, if and as needed.

Using It with an AI Assistant

The practical use case: you’re in a SvelteKit codebase and want to modernize. Tell your AI assistant:

“Fetch https://svelte.cogley.jp/feeds/patterns.json and check what patterns from this feed apply to this repo.”

The assistant fetches the feed, reads the search_signatures from each pattern, greps the codebase, and reports which old patterns it found with the modern replacements. No manual searching through docs.

For a more targeted audit:

“Fetch https://svelte.cogley.jp/feeds/patterns.json?since=5.29 and see if we’re using any pre-5.29 patterns that have better alternatives now.”

You can also filter by what you’re migrating from:

“Fetch https://svelte.cogley.jp/feeds/patterns.json?framework=react and show me the patterns relevant to this React-to-Svelte migration.”

The feed’s _svelte_patterns_meta.usage_hint field tells the assistant exactly how to interpret the data, so it works even without prior knowledge of the feed’s structure.

Real-World Results

I pointed Claude Code at the patterns feed across several production repositories and asked it to audit each one. Here’s what happened.

Fixes Applied Immediately

Repo Before Fix Files
pub-cogley onMount/onDestroy lifecycle, manual AbortController, duplicated utils Converted 19 lifecycle calls to $effect across 4 apps, extracted shared utils, added cookie secure flag 25
pulse Custom window.dispatchEvent('org-switched') pattern Replaced with $state-based reactive signal in .svelte.ts module 7
codex $app/stores imports ($page) Converted to $app/state with $derived() 2
periodic $app/stores + writable() language store Converted to $app/state (9 files) + rewrote language store as .svelte.ts with $state rune (60 consumers) 70
courier $app/stores + writable() language store + unvalidated route params Converted to $app/state (3 files), rewrote language store (9 consumers), added param matchers for route type safety 14

That’s 118 files modernized across 5 SvelteKit codebases, but I also applied it to a non-svelte project, and a greenfield project that used the feed as a day-one adoption checklist.

What Was Already Modern

Every repo was already using core Svelte 5 patterns: $state, $derived, $effect, $props, snippets, onclick handlers, use:enhance. The feed correctly identified these as “no action needed” and focused on the gaps — mostly $app/stores → $app/state conversions and lingering writable() stores.

The Greenfield Case: Mame

One repo — mame, an internal learning platform started the same day — used the feed proactively rather than retroactively. Instead of finding legacy code to fix, it used the patterns feed as a kind of checklist for what to adopt from the start. The result: createContext<T>() for type-safe context (eliminated prop drilling across 18 files), $props.id() for SSR-safe form IDs, MediaQuery class for reactive breakpoints, and svelte/reactivity/window for reactive viewport values — all patterns that the existing repos had flagged as “future opportunities” but hadn’t adopted yet.

The Nexus Case

One repo — nexus — is a pure Hono API worker with zero .svelte files. The feed audit correctly reported “N/A” for all Svelte patterns. It did find deprecated shared docs containing Svelte 4 examples that had been migrated to another repo, which were cleaned up.

Future Opportunities

Every repo got a tailored list of patterns that are available but not yet adopted — type-narrowed route params (SvelteKit 2.55), SSR error boundaries (2.54), createContext<T>() (5.50), getAbortSignal() (5.35). Not urgent, but now documented as backlog items. The “Not Applicable” column was equally useful — the feed correctly identified cases where patterns don’t apply, like getAbortSignal() in repos where fetches are user-triggered button clicks rather than $effect blocks.

Implementation Notes

Feed Endpoints

The feed endpoints are SvelteKit server routes returning json() responses with CORS headers and 1-hour cache. Both are at /feeds/ — a dedicated namespace that won’t conflict with page routes.

Version Filtering

The patterns feed supports version filtering via the since query parameter. The implementation uses semver comparison on the since field:

This lets you ask “what’s new since version X?” for either package independently.

Feed Autodiscovery

Feed autodiscovery is handled by <link rel="alternate" type="application/feed+json"> tags in the SvelteKit layout. Any feed reader that visits any page on the site will discover both feeds automatically.

Bilingual Support

Both feeds support ?lang=ja to return Japanese titles, summaries, and notes where available. The patterns data is bilingual at the source level — every pattern has both title and title_ja, summary and summary_ja.

Layout and Navigation

The patterns page sits alongside the existing home (migration mappings), about, and changelog pages. Feed links appear in the global footer on every page, and the changelog page has a JSON Feed badge in its header. The layout includes theme and language toggles in the nav bar, accessible from every page.

How This Fits Together

The migration guide now has three machine-readable interfaces, each for a different use case:

Interface Audience Best for
/llms-full.txt LLMs reading the whole site “Tell me about Svelte 5 migration”
/feeds/patterns.json AI coding assistants “Audit this codebase for modernization”
WebMCP tools Browser-based AI agents “What’s the Svelte equivalent of useEffect?”

The patterns feed fills the gap between “read everything” (llms-full.txt) and “answer one question” (WebMCP). It’s the “here’s what’s actionable” layer — structured enough for automation, browsable enough for humans.

Related

This builds on the original migration guide article and the WebMCP tools implementation. The patterns feed complements WebMCP — WebMCP is for browser-based AI agents interacting with the page, while the JSON Feed is for CLI tools and coding assistants that can fetch URLs directly.

Originally published at cogley.jp

Rick Cogley is CEO of eSolia Inc., providing bilingual IT outsourcing and infrastructure services in Tokyo, Japan.

I built a Redis-alternative distributed cache in Rust — with WAL persistence, mTLS, and Raft consensus

MnemeCache is an open-source distributed in-memory cache written from scratch in Rust. It is not a Redis wrapper or drop-in replacement — it’s a ground-up rethink of how a modern cache should be built: separation of hot memory and persistence, mTLS security by default, and Raft-based HA without the complexity tax.

🐙 GitHub: github.com/mneme-labs/mneme
🐳 Docker Hub: hub.docker.com/r/mnemelabs

Why not just use Redis?

Redis is a single-process C daemon with 25 years of accumulated complexity. Persistence is bolted on (RDB snapshots or AOF logging). TLS is optional and cumbersome to configure in clusters. HA requires Sentinel — a separate fleet of processes with its own failure modes.

MnemeCache is designed so that persistence, security, and HA are architectural defaults, not add-ons.

What makes it different

  • WAL + Keeper nodes — Core never touches disk. Writes stream over mTLS to dedicated Keeper processes that own all disk I/O and run snapshot compaction independently.
  • Raft consensus — 3-node HA cluster with automatic leader election and sub-second failover, no Sentinel needed.
  • Read replicas — horizontal read scaling with eventual consistency on a dedicated port.
  • mTLS by default — Core generates a CA on first boot and shares it with the cluster automatically. Zero manual certificate management for Docker deployments.
  • Built in Rust — no GC pauses, memory-safe, async I/O via Tokio.
  • Prometheus metrics on every node, Grafana dashboard included.

Four node types, four Docker images

MnemeCache separates concerns into distinct roles. Each ships as its own image:

Image tag Role
mnemelabs/core Cluster primary — hot store, WAL, Raft
mnemelabs/keeper Persistence layer — WAL drain, snapshots
mnemelabs/cli CLI

Up in 60 seconds

docker run -d 
  -p 6379:6379 -p 9090:9090 
  -e MNEME_ADMIN_PASSWORD=secret 
  -v mneme-data:/var/lib/mneme 
  mnemelabs/core:latest
mneme-cli --insecure -u admin -p secret ping
# PONG

mneme-cli --insecure -u admin -p secret set hello world
mneme-cli --insecure -u admin -p secret get hello
# world

Full cluster with Docker Compose

git clone https://github.com/mneme-labs/mneme.git
cd mneme

# Core + 2 Keepers
docker compose --profile cluster up -d

# Core + 3 Keepers + 2 Replicas + Prometheus + Grafana
docker compose --profile full up -d

# 3-node Raft HA cluster
docker compose --profile ha up -d

How the WAL pipeline works

Core holds everything in memory and streams every committed write to Keeper nodes over mTLS. Keepers are the only processes that ever touch disk — they drain WAL segments, write them to cold storage, and compact them into snapshots on a configurable schedule.

When Core restarts it asks a Keeper: “what’s my last committed offset?” — loads the latest snapshot, replays only the delta, and opens for connections. Recovery time is bounded by snapshot interval, not dataset size.

This separation means you can scale persistence independently, run multiple Keepers for redundancy, and keep Core’s write path free of disk latency.

Still early and under active development — contributions and feedback very welcome.

⭐ github.com/mneme-labs/mneme

Deploynix vs. Laravel Forge vs. Ploi: An Honest Comparison

Choosing a server management platform for your Laravel applications is a decision that affects your daily workflow for years. Laravel Forge pioneered the category. Ploi entered as a competitive alternative. And now Deploynix is here with a different perspective on what Laravel deployment should look like in 2026.

This isn’t a takedown piece. Forge and Ploi are good products that have served the Laravel community well. But developers deserve an honest comparison so they can pick the tool that fits their specific needs. Let’s walk through the features that matter most.

Cloud Provider Support

The number of supported cloud providers determines how much flexibility you have in choosing where your servers run.

Laravel Forge supports DigitalOcean, Linode, Vultr, AWS, Hetzner, and custom servers via IP. Forge also offers Laravel VPS — their own managed servers starting at $6/month — so you can provision directly through Forge without a separate cloud provider account. It’s a solid lineup that covers most needs.

Ploi supports DigitalOcean, UpCloud, Vultr, Linode, Scaleway, AWS EC2, and custom servers. A broad selection with some unique providers like UpCloud and Scaleway that other platforms don’t offer.

Deploynix supports DigitalOcean, Vultr, Hetzner, Linode, AWS, and custom servers. Six providers with a uniform provisioning experience across all of them.

The practical difference here isn’t just the list — it’s how custom servers are treated. On Deploynix, a custom server (your own bare metal, a provider we don’t directly integrate with) gets the same management experience as a provider-provisioned server. The same monitoring, the same deployment pipeline, the same interface. This matters if you’re running on infrastructure that isn’t covered by any platform’s direct integrations.

Server Types and Architecture

How a platform thinks about server types reveals how it thinks about application architecture.

Laravel Forge offers App Servers, Web Servers, Database Servers, Cache Servers, and Worker Servers. It’s a reasonable set that covers common architectures.

Ploi offers Web Servers, Load Balancers, Database Servers, Redis Servers, and Storage Servers. Ploi also supports load balancing, giving teams flexibility in how they distribute traffic.

Deploynix offers seven distinct server types: App, Web, Database, Cache, Worker, Meilisearch, and Load Balancer. Each type is configured specifically for its purpose. The inclusion of a dedicated Meilisearch server type reflects the growing adoption of Meilisearch in the Laravel ecosystem thanks to Laravel Scout. The Load Balancer type supports Round Robin, Least Connections, and IP Hash methods, making it straightforward to distribute traffic across multiple application servers.

If your architecture is a single server running everything, all three platforms work fine. But as you scale and separate concerns, having purpose-built server types reduces the configuration overhead. Deploynix’s Meilisearch server type is unique among the three platforms.

Zero-Downtime Deployments

This is where the platforms diverge most significantly.

Laravel Forge added zero-downtime deployments over time. The implementation works, but it’s an opt-in feature rather than a default behavior. You configure it per site.

Ploi also supports zero-downtime deployments with a similar opt-in approach.

Deploynix makes zero-downtime deployment the default for every site. Atomic symlink deployments, shared storage handling, PHP-FPM graceful reloads, and automatic release cleanup are built into the standard deployment pipeline. Rollback is a single click — the symlink points back to any previous release instantly.

The philosophical difference matters. When zero-downtime is the default, every feature built around deployments assumes it. Scheduled deployments, deployment hooks, rollback — they all work with the atomic deployment model as the foundation.

Deployment Features

Beyond the basic deploy, what tooling exists around the deployment process?

Laravel Forge offers deployment scripts, quick deploy (auto-deploy on push), deployment triggers via URL, and deploy notifications. It’s battle-tested and reliable.

Ploi provides similar deployment tooling with deployment scripts, auto-deploy, and notifications.

Deploynix includes customizable deployment scripts, auto-deploy on push, scheduled deployments, instant rollback, and real-time deployment logs via WebSockets. The scheduled deployment feature is worth highlighting — you can schedule a deploy for a future time and cancel it before execution. This is useful for coordinated releases, off-peak deployments, and team workflows where the person writing code isn’t always the person deciding when it goes live.

Real-time deployment logs via WebSocket connections (built on Laravel Reverb) mean you see output as it happens, not after a page refresh. It’s a small UX difference that significantly improves the deployment experience.

Database Support

Laravel Forge supports MySQL, MariaDB, and PostgreSQL, along with SQLite for simpler use cases.

Ploi supports MySQL, MariaDB, and PostgreSQL.

Deploynix supports MySQL, MariaDB, and PostgreSQL on dedicated Database server types. Database backup is integrated directly into the platform with support for AWS S3, DigitalOcean Spaces, Wasabi, and any custom S3-compatible storage. Backup scheduling, monitoring, and cross-server restores are built in.

The backup integration is a meaningful differentiator. With other platforms, you often end up configuring a separate backup solution — a third-party service, a custom script, or a Laravel package. Deploynix treats backups as a core platform feature with a monitoring dashboard so you can verify backups are running successfully.

Caching

Laravel Forge supports Redis installation on servers.

Ploi similarly supports Redis.

Deploynix supports Valkey, the Redis-compatible fork that emerged after Redis changed its licensing. Valkey is protocol-compatible with Redis, so your Laravel application’s cache, session, and queue configurations work without code changes. This choice reflects a preference for truly open-source software.

SSL and Domain Management

Laravel Forge handles Let’s Encrypt certificate provisioning and renewal. It supports custom certificates and has Cloudflare integration.

Ploi provides Let’s Encrypt automation and custom certificate support.

Deploynix auto-provisions Let’s Encrypt certificates including wildcard certificates via DNS validation. DNS validation is supported through Cloudflare, DigitalOcean, AWS Route 53, and Vultr. Every site automatically gets a *.deploynix.cloud vanity subdomain with SSL included — perfect for staging environments and client previews without any domain or DNS configuration.

The vanity domain feature is particularly useful during development. Spin up a site, share the your-project.deploynix.cloud URL with a client for review, and move to a production domain when ready. No DNS configuration, no certificate provisioning, no waiting.

Team Management and Collaboration

Laravel Forge offers organization-based team management with role-based access controls, allowing you to share servers and sites with team members and manage billing per organization.

Ploi provides team management with similar role-based access.

Deploynix implements organization-based team management with five distinct roles: Owner, Admin, Manager, Developer, and Viewer. Each role has specific permission boundaries. A Developer can trigger deployments and view logs but can’t modify server configuration or firewall rules. A Manager can manage sites and deployments but can’t provision new servers. A Viewer gets read-only access — ideal for stakeholders who need visibility without risk. This granularity matters in professional environments where the principle of least privilege isn’t optional.

The organization model also affects billing — plans are per-organization, so you can have separate billing for different clients or projects.

Monitoring and Alerting

Laravel Forge has added server monitoring capabilities over time, including resource usage graphs and some alerting features.

Ploi provides server monitoring with resource usage tracking.

Deploynix builds monitoring directly into the platform with real-time CPU, memory, and disk metrics. Health alerts fire based on configurable thresholds with severity levels (critical and warning). The monitoring agent runs on your server and reports metrics back to the platform via WebSocket for live updates. You see current server health on the dashboard without refreshing.

The alert system is designed to catch problems before they become incidents. Set a warning at 80% disk usage and a critical alert at 90%. Get notified when CPU consistently spikes above your threshold. The goal is to fix problems before your users encounter them.

API and Automation

Laravel Forge has a comprehensive API that covers most platform operations. It’s well-documented and widely used in CI/CD pipelines.

Ploi provides an API for automation and CI/CD integration.

Deploynix offers a Sanctum-based API with granular token scopes. When you create an API token, you specify exactly what it can do — manage servers, trigger deployments, view monitoring data, manage team members. A CI/CD token that only needs to trigger deployments doesn’t need permission to modify firewall rules. This follows security best practices for API token management.

Web Terminal

Laravel Forge does not include a web-based terminal — you SSH into servers using your own terminal and keys.

Ploi offers a web-based terminal for server access.

Deploynix includes a web terminal with authentication tied to the platform’s role system. Access is audited and controlled by team permissions. This is especially useful for teams where not everyone should have direct SSH key access to production servers.

Octane Support

Laravel Forge supports Laravel Octane deployment.

Ploi supports Octane as well.

Deploynix supports Octane with a choice of three drivers: FrankenPHP, Swoole, and RoadRunner. You select your driver during site setup and the server configuration, process management, and deployment pipeline adjust accordingly. Graceful restarts during deployment ensure Octane workers pick up new code without dropping requests.

Git Provider Integration

Laravel Forge integrates with GitHub, GitLab, and Bitbucket.

Ploi supports GitHub, GitLab, and Bitbucket.

Deploynix supports GitHub, GitLab, Bitbucket, and custom Git repositories. The custom Git option means you can deploy from a self-hosted Gitea instance, a corporate GitLab installation, or any Git repository accessible via SSH.

Pricing

Pricing changes frequently, so rather than quoting specific numbers that may be outdated, here’s the structural comparison.

Laravel Forge uses flat-rate monthly pricing with three tiers — Hobby, Growth, and Business — based on features, server limits, and team access. Forge also offers Laravel VPS starting at $6/month for those who want managed hosting without a separate cloud provider.

Ploi has made Ploi Core completely free, with paid tiers scaling up for teams and advanced features. This makes Ploi one of the most accessible entry points for server management.

Deploynix has four tiers — Free, Starter, Professional, and Enterprise — billed per organization through Paddle. The Free tier is genuinely usable, not a trial. Core features like zero-downtime deployments and monitoring are available across all tiers. Higher tiers unlock additional servers, team members, and advanced features.

When to Choose Each Platform

Choose Laravel Forge if you want the most mature platform with the largest community. Forge has been around the longest, has the most tutorials and third-party integrations, and benefits from being closely tied to the Laravel ecosystem’s creator. The addition of Laravel VPS and support for Node.js, Nuxt, and Next.js apps makes Forge increasingly versatile. If stability and ecosystem size matter most, Forge is a safe choice.

Choose Ploi if you want a cost-effective alternative with a free core tier and a modern interface. Ploi has carved out a solid niche by offering good value, responsive development, and unique provider support like UpCloud and Scaleway.

Choose Deploynix if you want zero-downtime deployments as a default, granular team permissions, integrated monitoring with real-time alerts, scheduled deployments, and a platform designed specifically for modern Laravel workflows. If you run multi-cloud infrastructure, need purpose-built server types including load balancers, or want a platform that treats backups and monitoring as first-class features rather than add-ons, Deploynix is built for those use cases.

The Honest Truth

No platform is perfect. Forge has the advantage of maturity, ecosystem integration, and its own managed VPS offering. Ploi has the advantage of a free core tier, unique provider support, and a responsive team. Deploynix has the advantage of a fresh architecture built on modern Laravel with features designed for 2026 workflows.

The best way to decide is to try them. Each platform offers a way to get started without significant commitment. Spin up a test server, deploy an application, and see which workflow fits the way you and your team actually work.

We’re confident that once you try Deploynix, you’ll appreciate the difference that comes from building a platform from scratch with today’s Laravel development practices in mind. But we’d rather earn your trust through the product than through a comparison article.

Visit https://deploynix.io to start your free account.

vLLM On-Demand Gateway: Zero-VRAM Standby for Local LLMs on Consumer GPUs

The Problem: vLLM Hogs Your GPU 24/7

If you run a local LLM with vLLM, you know the pain. The moment you start the server, it claims ~90% of your VRAM and never lets go — even when nobody’s asking it anything.

On a dedicated inference server, that’s fine. But on a single consumer GPU (RTX 5090 in my case), I also need VRAM for:

  • Shogi engine (DL-based, needs ~4GB VRAM)
  • Whisper transcription (large-v3, GPU-accelerated)
  • Training runs, experiments, occasional gaming

Running vLLM permanently means everything else fights for scraps. Killing and restarting vLLM manually every time is not a workflow — it’s a chore.

The Solution: A Gateway That Manages vLLM’s Lifecycle

I wrote a single-file FastAPI gateway (vllm_gateway.py, ~390 lines) that:

  1. Listens on port 8000 with near-zero VRAM usage
  2. Auto-starts vLLM on an internal port (8100) when a request arrives
  3. Auto-stops vLLM after 10 minutes of idle, fully freeing VRAM
  4. Rewrites tool calls from Nemotron’s <TOOLCALL> format to OpenAI-compatible tool_calls

From the client’s perspective, it’s just a normal OpenAI-compatible API on port 8000. The lifecycle management is completely invisible.

Client → :8000 (Gateway, always running, ~0 VRAM)
              ↓ proxy
           :8100 (vLLM, started on-demand, stopped when idle)

Architecture

Startup Flow

Request arrives at :8000
  → Gateway checks: is vLLM running?
    → No: spawn vLLM process on :8100
          poll /health every 2s (up to 3 min timeout)
          once healthy → proxy the request
    → Yes: proxy immediately

Shutdown Flow

Idle watchdog runs every 30s
  → Last request was >10 min ago?
    → SIGTERM to vLLM process group
    → Wait 15s, SIGKILL if needed
    → VRAM fully released

Key Design Decisions

  • Process group kill (os.killpg): vLLM spawns child processes. Killing just the parent leaves zombies holding VRAM.
  • Internal port separation: Gateway owns :8000. vLLM gets :8100. No port conflicts during restart.
  • Health check polling: Don’t proxy until vLLM is actually ready. Model loading takes 30-90s depending on size.

Core Implementation

Here’s the stripped-down version of the essential parts:

VLLM_INTERNAL_PORT = 8100
GATEWAY_PORT = 8000
IDLE_TIMEOUT_SECONDS = 10 * 60

VLLM_CMD = [
    ".venv/bin/vllm", "serve", "nvidia/NVIDIA-Nemotron-Nano-9B-v2-Japanese",
    "--trust-remote-code",
    "--port", str(VLLM_INTERNAL_PORT),
    "--enable-auto-tool-choice",
    "--tool-call-parser", "nemotron_json",
]

async def start_vllm() -> bool:
    global vllm_process, vllm_ready
    vllm_process = subprocess.Popen(
        VLLM_CMD, preexec_fn=os.setsid,
        stdout=subprocess.DEVNULL, stderr=subprocess.PIPE,
    )
    # Poll health endpoint until ready
    deadline = time.time() + 180
    async with httpx.AsyncClient() as client:
        while time.time() < deadline:
            try:
                resp = await client.get(
                    f"http://localhost:{VLLM_INTERNAL_PORT}/health", timeout=5
                )
                if resp.status_code == 200:
                    vllm_ready = True
                    return True
            except (httpx.ConnectError, httpx.ReadTimeout):
                pass
            await asyncio.sleep(2)
    return False

async def stop_vllm():
    global vllm_process, vllm_ready
    if vllm_process and vllm_process.poll() is None:
        os.killpg(os.getpgid(vllm_process.pid), signal.SIGTERM)
        try:
            vllm_process.wait(timeout=15)
        except subprocess.TimeoutExpired:
            os.killpg(os.getpgid(vllm_process.pid), signal.SIGKILL)
    vllm_process = None
    vllm_ready = False

async def idle_watchdog():
    while True:
        await asyncio.sleep(30)
        if vllm_ready and time.time() - last_request_time > IDLE_TIMEOUT_SECONDS:
            await stop_vllm()

Tool Call Rewriting (Bonus)

Nemotron outputs tool calls as raw text:

<TOOLCALL>[{"name": "ddg_search", "arguments": {"query": "NVIDIA stock"}}]</TOOLCALL>

The gateway intercepts this and rewrites it to the OpenAI format:

{
  "tool_calls": [{
    "id": "call_abc123",
    "type": "function",
    "function": {
      "name": "ddg_search",
      "arguments": "{"query": "NVIDIA stock"}"
    }
  }],
  "finish_reason": "tool_calls"
}

This works for both streaming and non-streaming responses. Clients never see the raw <TOOLCALL> tags.

Status Endpoint

The gateway exposes a simple status API:

$ curl localhost:8000/gateway/status
{
  "vllm_running": false,
  "vllm_ready": false,
  "idle_seconds": 7145,
  "pid": null
}

And manual controls:

curl -X POST localhost:8000/gateway/start   # Force start
curl -X POST localhost:8000/gateway/stop    # Force stop

Real-World Numbers (RTX 5090, Nemotron 9B)

State VRAM Usage
Gateway only (vLLM stopped) ~200 MB
vLLM running (Nemotron 9B FP16) ~22 GB
Cold start time ~60s

The 60s cold start is the tradeoff. For interactive chat, the first message after idle has a delay. For batch/API workloads, it’s negligible.

When You’d Want This

  • Single GPU, multiple workloads: Share your GPU between LLM inference and other tasks
  • Development machine: Run vLLM only when you’re actively using it
  • Cost/power savings: No point heating your GPU for an idle model
  • Home server: The RTX card can serve LLM requests AND run other GPU tasks

When You Wouldn’t

  • Dedicated inference server: Just run vLLM directly
  • Low-latency requirements: 60s cold start is unacceptable for some use cases
  • Multi-user serving: Frequent requests mean vLLM stays up anyway

Full Source

The complete vllm_gateway.py is ~390 lines including streaming support and TOOLCALL rewriting. The approach works with any vLLM model — just change VLLM_CMD.

Dependencies: fastapi, httpx, uvicorn (all likely already installed if you use vLLM).

Running Nemotron Nano 9B v2 Japanese on RTX 5090 + WSL2. The gateway pattern turned “I can’t use my GPU for anything else” into “vLLM is there when I need it and gone when I don’t.”

Building Framework-Agnostic AI Swarms: Compare LangGraph, Strands, and OpenAI Swarm

LangGraph swarm architecture showing LaunchDarkly configuration fetch, agent interactions with Command-based handoffs, and dual metrics tracking to AI Config Trends

If you’ve ever run the same app in multiple environments, you know the pain of duplicated configuration. Agent swarms have the same problem: the moment you try multiple orchestrators (LangGraph, Strands, OpenAI Swarm), your agent definitions start living in different formats. Prompts drift. Model settings drift. A “small behavior tweak” turns into archaeology across repos.

AI behavior isn’t code. Prompts aren’t functions. They change too often, and too experimentally, to be hard-wired into orchestrator code. LaunchDarkly AI Configs lets you treat agent definitions like shared configuration instead. Define them once, store them centrally, and let any orchestrator fetch them. Update a prompt or model setting in the LaunchDarkly UI, and the new version rolls out without a redeploy.

Ready to build framework-agnostic AI swarms? Start your 14-day free trial of LaunchDarkly to follow along with this tutorial. No credit card required.

Start free trial →

The problem: Research gap analysis across multiple papers

When analyzing academic literature, researchers face a daunting task: reading dozens of papers to identify patterns, spot contradictions, and find unexplored opportunities. A single LLM call can summarize papers, but it produces a monolithic analysis you can’t trace, refine, or trust for critical decisions.

The challenge compounds when you need to:

  • Identify methodological patterns across 12+ papers without missing subtle connections
  • Detect contradictory findings that might invalidate assumptions
  • Discover research gaps that represent genuine opportunities, not just oversight

This is where specialized agents excel – each focused on one aspect of the analysis, building on each other’s work.

In this tutorial, we’ll build a 3-agent research analysis swarm that solves this problem by dividing the work:

Agent Role Output
Approach Analyzer Clusters methodological themes across papers “Papers 1, 4, 7 use reinforcement learning; Papers 2, 5 use symbolic methods”
Contradiction Detector Finds conflicting claims between papers “Paper 3 claims X improves performance; Paper 8 shows X degrades it”
Gap Synthesizer Identifies unexplored research directions “No papers combine approach A with dataset B; potential opportunity”

We’ll implement this swarm across three different orchestrators (LangGraph, Strands, and OpenAI Swarm), demonstrating how LaunchDarkly AI Configs enable:

  • Framework-agnostic agent definitions: Define agents once in LaunchDarkly, use them everywhere
  • Per-agent observability: Track tokens, latency, and costs for each agent individually – catch silent failures when agents skip execution
  • Dynamic swarm composition: Add/remove agents from the swarm or switch models without touching code

Why use a swarm?

Research gap analysis requires different skills: clustering methodological patterns, detecting contradictions, and synthesizing opportunities. With a swarm, each agent handles one aspect and produces artifacts the next agent builds on. You can track tokens, latency, and cost per agent. You can catch silent failures when an agent skips execution. And when something goes wrong, you know exactly where.

Technical requirements

Before implementing the swarm, ensure you have:

The complete implementation is available at GitHub – AI Orchestrators.

The architecture: how LaunchDarkly powers framework-agnostic swarms

The swarm architecture has three layers: dynamic agent configuration, per-agent tracking, and custom metrics for cost attribution. Here’s how they work together.

LangGraph swarm architecture showing LaunchDarkly configuration fetch, agent interactions with Command-based handoffs, and dual metrics tracking to AI Config Trends

The diagram shows LangGraph’s implementation, but Strands and OpenAI Swarm follow the same pattern with their own handoff mechanisms. The key components are:

  1. Configuration Fetch: The orchestrator queries LaunchDarkly’s API to dynamically discover all agent configurations, avoiding hardcoded agent definitions
  2. Agent Graph: Three specialized agents (Approach Analyzer, Contradiction Detector, Gap Synthesizer) connected through explicit handoff mechanisms
  3. Metrics Collection: Each agent execution captures tokens, duration, and cost metrics through both the AI Config tracker and custom metrics API
  4. Dual Dashboard Views: The same metrics appear in the AI Config Trends dashboard (for individual agent monitoring)

Three layers of framework-agnostic swarms

1. AI Config for Dynamic Agent Configuration

Each AI Config stores:

  • Agent key, display name, and model selection
  • System instructions and tool definitions

Your orchestrator code queries LaunchDarkly for “all enabled agent configs” and builds the swarm dynamically. No hardcoded agent names.

2. Per-Agent Tracking with AI SDK

LaunchDarkly’s AI SDK provides tracking through config evaluations. You get a fresh tracker for each agent, then track tokens, duration, and success/failure. These metrics flow to the AI Config Monitoring dashboard automatically.

AI Config monitoring dashboard showing per-agent token usage, duration, and success rates across multiple runs

This tracking catches silent failures – when agents skip execution or produce minimal output. Step 4 shows the implementation patterns for each framework.

3. Custom Metrics for Cost Attribution

Per-agent tracking shows performance, but for cost comparisons across orchestrators you need custom metrics. These let you query by orchestrator, compare costs across frameworks, and identify anomalies.

With the architecture covered, let’s build the swarm. We’ll download research papers, set up the project, bootstrap agent configs in LaunchDarkly, implement per-agent tracking, and run the swarm across all three orchestrators.

Step 1: Download research papers

First, you need papers to analyze. The scripts/download_papers.py script queries ArXiv with narrow, category-specific searches to ensure focused results.

python scripts/download_papers.py

The script presents pre-configured narrow research topics:

# From orchestration/scripts/download_papers.py:164-189
topics = {
    "1": {
        "name": "Chain-of-thought prompting in LLMs",
        "query": "cat:cs.CL AND (chain-of-thought OR CoT) AND reasoning",
        "years": 2
    },
    "2": {
        "name": "Retrieval-augmented generation (RAG)",
        "query": "cat:cs.CL AND (retrieval-augmented OR RAG) AND generation",
        "years": 2
    },
    "3": {
        "name": "Emergent communication in multi-agent RL",
        "query": "cat:cs.MA AND (emergent communication OR language emergence)",
        "years": 5
    },
    "4": {
        "name": "Few-shot prompting for code generation",
        "query": "cat:cs.SE AND few-shot AND code generation",
        "years": 2
    },
    "5": {
        "name": "Vision-language model grounding",
        "query": "cat:cs.CV AND vision-language AND grounding",
        "years": 2
    }
}

These topics are intentionally narrow: Each uses ArXiv categories (cat:cs.CL, cat:cs.MA) to limit scope. Boolean AND operators ensure papers match all criteria. 2-5 year windows prevent overwhelming the analysis.

For even narrower custom queries, combine categories with specific techniques like cat:cs.CL AND chain-of-thought AND mathematical AND reasoning for CoT math only, cat:cs.MA AND emergent AND (referential OR compositional) for specific emergence types, or cat:cs.SE AND few-shot AND (Python OR JavaScript) AND test generation for language-specific code generation.

The script saves papers to data/gap_analysis_papers.json with this structure:

[
  {
    "id": "2409.02645v2",
    "title": "Emergent Language: A Survey and Taxonomy",
    "authors": "Jannik Peters, Constantin Waubert de Puiseau, ...",
    "published": "2024-09-04",
    "category": "cs.MA",
    "abstract": "The field of emergent language represents...",
    "introduction": "Language emergence has been explored...",
    "conclusion": "This paper provides a comprehensive review..."
  }
]

Why this format: Each paper includes ~2-3K characters of text (abstract + intro + conclusion), which is enough for analysis but won’t overflow context windows. For 12 papers, you’re looking at ~30K characters (~7.5K tokens) of input.

You now have 12 papers saved locally. Next, we’ll configure LaunchDarkly credentials and install the orchestration frameworks.

Step 2: Set up your multi-orchestrator project

Environment setup

For help getting your SDK and API keys, see the API access tokens guide and SDK key management.

# .env file
LD_SDK_KEY=sdk-xxxxx       # Get from LaunchDarkly project settings
LD_API_KEY=api-xxxxx       # Create at Account settings → Authorization
LAUNCHDARKLY_PROJECT_KEY=orchestrator-agents

# Model API keys
ANTHROPIC_API_KEY=sk-ant-xxxxx
OPENAI_API_KEY=sk-xxxxx

Install dependencies

python -m venv .venv
source .venv/bin/activate

# LaunchDarkly SDKs - see [Python SDK docs](/sdk/server-side/python)
pip install ldai ldclient python-dotenv arxiv PyPDF2 requests

# Orchestration frameworks
pip install strands-sdk langgraph swarm

For more on the LaunchDarkly AI SDK, see the AI SDK documentation.

Your environment is configured and dependencies are installed. Next, we’ll use the bootstrap script to automatically create all three agent configs in LaunchDarkly.

Step 3: Bootstrap agent configs with the manifest

The orchestration repo includes a complete bootstrap system that automatically creates all agent configurations, tools, and variations in LaunchDarkly. This is much faster and more reliable than manual setup.

Understanding the bootstrap system

The bootstrap process uses a YAML manifest to define:

  1. Tools – Functions agents can call (fetch_paper_section, handoff_to_agent, etc.)
  2. Agent Configs – Three specialized agents with their roles and instructions
  3. Variations – Multiple model options (Anthropic Claude vs OpenAI GPT)
  4. Targeting Rules – Which orchestrators get which models

Run the bootstrap script

# From the orchestration repo root
cd ai-orchestrators

# Run bootstrap with the research gap manifest
python scripts/launchdarkly/bootstrap.py

# You'll see:
╔═══════════════════════════════════════════════════════╗
║  AI Agent Orchestrator - LaunchDarkly Bootstrap       ║
╚═══════════════════════════════════════════════════════╝

Available manifests:
  1. Research Gap Analysis (research_gap_manifest.yaml)

Select manifest or press Enter for default: [Enter]

📦 Project: orchestrator-agents
🌍 Environment: production

🛠️  Creating paper analysis tools...
    ✓ Tool 'extract_key_sections' created
    ✓ Tool 'fetch_paper_section' created
    ✓ Tool 'handoff_to_agent' created
    ...

🤖 Creating AI agent configs...
    ✓ AI Config 'approach-analyzer' created
    ✓ AI Config 'contradiction-detector' created
    ✓ AI Config 'gap-synthesizer' created

✨ Bootstrap complete!

What gets created

The bootstrap script creates the three agents described earlier (Approach Analyzer, Contradiction Detector, Gap Synthesizer), each with swarm-aware instructions and handoff tools.

Verify in LaunchDarkly dashboard

After bootstrap completes:

  1. Go to your LaunchDarkly AI Configs dashboard at https://app.launchdarkly.com/<your-project-key>/<your-environment-key>/ai-configs
  2. You’ll see all three agent configs created
  3. Each config has:

How variations and targeting work

Each agent has two variations in the manifest:

# Example from approach-analyzer agent
variations:
  - key: "analyzer-claude"
    name: "Approach Analyzer Claude"
    modelConfig:
      provider: "anthropic"
      modelId: "claude-sonnet-4-5"
    tools: ["handoff_to_agent", "cluster_approaches"]
    instructions: |
      [Agent instructions here]

  - key: "analyzer-openai"
    name: "Approach Analyzer OpenAI"
    modelConfig:
      provider: "openai"
      modelId: "gpt-5"
    tools: ["handoff_to_agent", "cluster_approaches"]
    instructions: |
      [Same instructions, different model]

targeting:
  rules:
    - variation: "analyzer-openai"
      clauses:
        - attribute: "orchestrator"
          op: "in"
          values: ["openai_swarm", "openai-swarm"]
  defaultVariation: "analyzer-claude"

When an orchestrator requests this agent:

  1. Context includes orchestrator attribute: context = create_context(execution_id, orchestrator="openai_swarm")
  2. LaunchDarkly evaluates targeting rules: If orchestrator is “openai_swarm” or “openai-swarm”, use OpenAI variation
  3. Otherwise use default: Claude variation for all other orchestrators

This lets you:

  • Use OpenAI models when running OpenAI Swarm (native compatibility)
  • Use Claude for other orchestrators
  • A/B test models by adjusting targeting rules

Customize agent behavior

After bootstrap, you can adjust agents in the LaunchDarkly UI without code changes. Switch between Claude, GPT-4, or other supported providers. Refine instructions for better handoffs. Control which agents are included in the swarm through targeting rules. Test different prompts or models side-by-side with experiments.

Your three agents are now configured in LaunchDarkly. Next, we’ll implement tracking so you can monitor tokens, latency, and cost for each agent individually.

Step 4: Implement per-agent tracking

The orchestration repository demonstrates per-agent tracking across all three frameworks. First, you need to fetch agent configurations from LaunchDarkly:

Fetching agent configurations dynamically

from shared.launchdarkly import (
    init_launchdarkly_clients,
    fetch_agent_configs_from_api,
    create_context,
    build_agent_requests
)

# Initialize LaunchDarkly clients
ld_client, ai_client = init_launchdarkly_clients()

# Fetch agent list from LaunchDarkly API (not hardcoded!)
items = fetch_agent_configs_from_api()
print(f"Found {len(items)} AI config(s) in LaunchDarkly")

# Create execution context
execution_id = f"langgraph-{datetime.now().strftime('%Y%m%d_%H%M%S')}"
context = create_context(execution_id, orchestrator="langgraph")

# Build requests for all agents
agent_requests, agent_metadata = build_agent_requests(items)

# Fetch all configs in one call
configs = ai_client.agent_configs(agent_requests, context)

# Process agents with configured variations
enabled_agents = []
for item in items:
    config = configs.get(item["key"])
    if config and config.enabled:
        enabled_agents.append({
            "key": item["key"],
            "name": item["name"],
            "config": config,
            "model": config.model.name if config.model else "claude-sonnet-4-5"
        })

print(f"✓ Found {len(enabled_agents)} configured agent configs")

Pattern 1: Native framework metrics (Strands)

Strands provides accumulated_usage on each node result after execution:

# From orchestrators/strands/run_gap_analysis.py:418-424
if agent_key in per_agent_metrics:
    usage = node_result.accumulated_usage or {}
    input_tokens, output_tokens = extract_usage_tokens(usage)
    total_tokens = input_tokens + output_tokens

View full Strands implementation

Pattern 2: Message-based tracking (LangGraph)

LangGraph attaches usage_metadata to messages, requiring post-execution iteration:

# From orchestrators/langgraph/run_gap_analysis.py:442-446
if hasattr(msg, "usage_metadata") and msg.usage_metadata:
    usage_data = msg.usage_metadata
    input_tokens = usage_data.get("input_tokens", 0) or usage_data.get("prompt_tokens", 0)
    output_tokens = usage_data.get("output_tokens", 0) or usage_data.get("completion_tokens", 0)
    has_usage = True

View full LangGraph implementation

Pattern 3: Interception-based tracking (OpenAI Swarm)

OpenAI Swarm doesn’t aggregate per-agent metrics, requiring interception of completion calls:

# From orchestrators/openai_swarm/run_gap_analysis.py:369-387
original_get_chat_completion = client.get_chat_completion

def tracked_get_chat_completion(agent, history, context_variables, model_override, stream, debug):
    start_call = time.time()
    completion = original_get_chat_completion(
        agent=agent,
        history=history,
        context_variables=context_variables,
        model_override=model_override,
        stream=stream,
        debug=debug,
    )
    duration = time.time() - start_call
    agent_key = key_by_name.get(agent.name, agent.name)
    usage = getattr(completion, "usage", None)
    if usage:
        input_tokens = int(getattr(usage, "prompt_tokens", 0))
        output_tokens = int(getattr(usage, "completion_tokens", 0))
        total_tokens = int(getattr(usage, "total_tokens", input_tokens + output_tokens))

View full OpenAI Swarm implementation

Critical: Provider token field names differ

Each provider uses different field names: Anthropic uses input_tokens/output_tokens, OpenAI uses prompt_tokens/completion_tokens, and some frameworks use camelCase (inputTokens). The implementations use fallback chains to handle all formats.

You can now capture tokens, latency, and cost for each agent. Next, we’ll run the swarm across LangGraph, Strands, and OpenAI Swarm to see how they perform with the same agent definitions.

Step 5: Run multiple orchestrators and track results

The repository includes scripts to run all three orchestrators and analyze their performance:

# Run all orchestrators 5 times each
./scripts/run_swarm_benchmark.sh sequential 5

# Analyze the results
python scripts/analyze_benchmark_results.py
  1. Configure env: Create .env with SDK keys
  2. Install deps: pip install -r requirements.txt
  3. Download papers: python scripts/download_papers.py
  4. Bootstrap agents: python scripts/launchdarkly/bootstrap.py
  5. Configure targeting: Set default variation for each agent in LaunchDarkly UI
  6. Test run: python orchestrators/strands/run_gap_analysis.py

Troubleshooting: If you see “No enabled agents found,” check that each agent has a default variation set in the Targeting tab.

Now that you’ve run the swarm across all three orchestrators, let’s look at how they differ in approach and performance.

Comparing orchestrator approaches to swarms

All three frameworks support multi-agent workflows, they just disagree on who decides what happens next.

Key differences

Aspect Strands LangGraph OpenAI Swarm
Routing Framework-managed Graph-based Function return
Handoff API Tool call (automatic) Command object Return Agent object
Boilerplate Low Medium Medium
Control Low (black box) High (explicit graph) High (manual impl)
Debugging Hard (why didn’t agent run?) Easy (graph trace) Hard (silent failures)
Per-Agent Metrics Built-in Wrapper required Interception required

View full implementations: Strands | LangGraph | OpenAI Swarm

The LaunchDarkly advantage: By defining agents externally, you can implement swarms across all three frameworks and compare their approaches with the same agent definitions.

Performance comparison (9 runs: 3 datasets × 3 orchestrators)

Metric OpenAI Swarm Strands LangGraph
Avg Time 2.9 min 5.7 min 8.0 min
Tokens 67K 99K 89K
Speed 385 tok/s 287 tok/s 186 tok/s
Report Size 13KB 32KB 67KB
Variance ±1.05 min ±1.38 min ±0.21 min

Key insight (based on limited sample): Fastest ≠ best. OpenAI Swarm was 3x faster but produced reports 80% smaller than LangGraph. LangGraph had the lowest variance and most comprehensive outputs despite slower execution.

Performance comparison graphs showing execution time, token usage, and processing speed across all three orchestrators

Example reports: See the outputs

Report size variation demonstrates why per-agent tracking matters – you need to know when agents produce minimal output.

Conclusion

The orchestrator you choose determines how agents coordinate, but it shouldn’t lock you into a single framework. By defining agents in LaunchDarkly and fetching them at runtime, you can run the same swarm across LangGraph, Strands, and OpenAI Swarm without duplicating configuration or watching prompts drift between repos.

The performance differences are real. OpenAI Swarm is fastest, LangGraph produces the most comprehensive outputs, and Strands offers the simplest setup. But you only discover these tradeoffs if you can track each agent individually and catch silent failures when they happen.

Swarms cost more than single LLM calls. The payoff is traceable reasoning you can audit, refine, and trust.

The full implementation is available on GitHub – AI Orchestrators. Clone the repo and run the same swarm across all three orchestrators. To get started with LaunchDarkly AI Configs, follow the quickstart guide.