The Magento multi-store bug every AI description generator has — and how we fixed it

A client came to us with 8,000 SKUs across four store views — English, Dutch, German, French. Descriptions were either copied from supplier PDFs or missing entirely. The fix was obviously AI generation. The less obvious problem was that every existing module we evaluated had the same architectural bug.

So we built our own, made it MIT-licensed, and put it on Packagist. This post is about the bug, the fix, and the four-provider abstraction (including a free one) we shipped on top.

🔗 Originally published on https://angeo.dev/magento-2-ai-product-description-generator/

The bug nobody talks about

Most Magento 2 AI content modules call this:

$product = $this->productRepository->get($sku, editMode: true);
$product->setCustomAttribute('description', $generated);
$this->productRepository->save($product);

Looks fine. It isn’t. Without an explicit $storeId, this loads and saves in the default scope — Magento’s global, store-view-independent fallback. When you save back, you overwrite every store view at once. The Dutch store gets English descriptions. The German store gets English descriptions. The French store gets English descriptions.

This is not a configuration problem. The multi-store architecture works correctly — the tooling ignores it. Writing to the default scope is simpler to implement than writing per store. Every commercial module we tested took the simpler path.

The right way:

// Load the product in the target store scope
$product = $this->productRepository->get($sku, false, $storeId);
$product->setCustomAttribute('description', $generated);

// Save with explicit store scope (Magento_CatalogModelProductAction)
$this->productService->updateAttributes($sku, $generated, $storeId);

The difference is one parameter. The architectural cost is iterating stores around your generation loop. The data cost of skipping it is silently corrupting your multi-language catalog.

The framework around the fix

The store-scope fix is the boring part. The interesting part is everything you need around it.

┌─────────────────────────────────────────────────────┐
│       Angeo Multi-Store AI Content Framework        │
├──────────────┬──────────────────┬───────────────────┤
│  SKU Source  │ Store Iteration  │   AI Provider     │
│  ──────────  │  ──────────────  │  ──────────────── │
│  Catalog     │  Store 1 (EN)    │  OpenAI           │
│  G.Sheets    │  Store 2 (NL)    │  Claude           │
│  CLI --sku   │  Store 3 (DE)    │  Gemini           │
│              │  Store 4 (FR)    │  Groq (free)      │
├──────────────┴──────────────────┴───────────────────┤
│               Content Pipeline                      │
│  load(sku, storeId) → prompt → generate → save      │
├─────────────────────────────────────────────────────┤
│                    Output                           │
│  Magento DB · Local CSV · Google Sheets API v4      │
└─────────────────────────────────────────────────────┘

Four layers:

  1. Provider Layer — uniform interface across OpenAI, Claude, Gemini, Groq
  2. Store Iteration Layer — resolves all active store views before processing any SKUs
  3. Content Pipeline — for each store × SKU: load in scope → build prompt → generate → save in scope
  4. I/O Layer — reads SKUs from catalog, Google Sheet, or CLI; writes to Magento DB, CSV, and Google Sheets

The provider abstraction is the part most people will copy. One interface:

interface AiProviderInterface
{
    public function generate(string $system, string $user): string;
}

Wired in di.xml:

<type name="AngeoAiDescriptionUpdaterServiceAiProviderService">
  <arguments>
    <argument name="providers" xsi:type="array">
      <item name="openai" xsi:type="object">...OpenAiProvider</item>
      <item name="claude" xsi:type="object">...ClaudeProvider</item>
      <item name="gemini" xsi:type="object">...GeminiProvider</item>
      <item name="groq"   xsi:type="object">...GroqProvider</item>
    </argument>
  </arguments>
</type>

Adding a fifth provider is one class + one line. The pipeline, store iteration, and I/O don’t change. This is the pattern, not just the code.

The four-provider benchmark

200 product descriptions from a real Dutch jewellery store, same system prompt, same product names, all four providers.

Speed (avg per description)

Provider Model Avg. time
Groq llama-3.3-70b-versatile 0.8s
Groq mixtral-8x7b-32768 0.6s
Google gemini-2.0-flash 1.2s
Anthropic claude-haiku-4-5 1.1s
OpenAI gpt-4.1-mini 1.4s
OpenAI gpt-4.1 2.1s
Anthropic claude-sonnet-4-6 2.8s

For 32,000 generations (8,000 SKUs × 4 store views): Groq ≈ 7 hours, GPT-4.1 ≈ 19 hours.

Cost (per 1,000 descriptions, ~200 words each)

Provider Model Cost / 1k
Groq llama-3.3-70b-versatile $0.00 (free tier)
Google gemini-2.0-flash ~$0.08
OpenAI gpt-4.1-mini ~$0.24
Anthropic claude-haiku-4-5 ~$0.32
OpenAI gpt-4.1 ~$1.80
Anthropic claude-sonnet-4-6 ~$2.40

Quality (manual review of 200 samples)

Criteria Groq Llama 3.3 GPT-4.1-mini GPT-4.1 Claude Sonnet
Factual accuracy ★★★★☆ ★★★★☆ ★★★★★ ★★★★★
Language fluency ★★★★☆ ★★★★☆ ★★★★★ ★★★★★
SEO keyword use ★★★☆☆ ★★★★☆ ★★★★☆ ★★★★☆
HTML formatting ★★★★☆ ★★★★☆ ★★★★★ ★★★★★

Practical recommendation: Validate prompts with Groq first — it’s free, fast, and good enough to test workflow. Production on GPT-4.1-mini if SEO keyword density matters. Flagship products on GPT-4.1 or Claude Sonnet if copy quality directly affects conversion.

Why Groq matters

Groq’s free tier is 14,400 requests/day. No credit card. Llama 3.3 70B output quality is genuinely good — ~1 star behind GPT-4.1 in our review, mostly on SEO keyword density.

For 8,000 SKUs × 4 stores = 32,000 calls, you’re at ~1.85 days on the free tier. For most stores under 5,000 SKUs, this is free production-grade AI content generation if you’re patient.

No other Magento module supports Groq at time of writing. This was the killer feature for our client.

Installation

composer require angeo/module-ai-description-updater
bin/magento setup:upgrade
bin/magento setup:di:compile
bin/magento cache:flush

First run (Groq, ~5 minutes from zero):

# 1. Get a free API key at console.groq.com
# 2. In admin: Stores → Configuration → Angeo → AI Description Updater
#    AI Provider = Groq (Free), paste key, Dry Run = Yes
# 3. Test on one SKU
bin/magento angeo:ai-description:run --sku=MY-SKU-001 --dry-run

# 4. If output looks good, disable dry-run and run the batch
bin/magento angeo:ai-description:run

Other useful flags:

bin/magento angeo:ai-description:run --sku=ABC-123     # single SKU, all stores
bin/magento angeo:ai-description:run --store=2         # single store view
bin/magento angeo:ai-description:run --dry-run         # generate, don't save

CLI-first by design. The 8,000-SKU client doesn’t want a UI; they want cron.

Why it’s open source

The honest answer: distribution. We sell AEO audits and full-stack Magento services. The modules are how stores find us. The code is free; the expertise applied to a specific store is not.

It’s a familiar model — Vercel does it with Next.js, Sentry does it with sentry-php. The artefact is open; the operator is the product.

Key takeaways for Magento devs

  • Default-scope writes are silent multi-store corruption. Audit any AI/content module before installing on a multi-store catalog. Look for $storeId parameters in productRepository->get() and save calls.
  • Provider abstraction is one interface plus a di.xml array. Don’t hardcode OpenAI. Future you (or future LLM pricing) will thank you.
  • Groq is the current sweet spot for free Magento AI generation. Llama 3.3 70B output is production-acceptable for most stores; the 14,400/day cap is workable for catalogs under ~5,000 SKUs.
  • GPT-4.1-mini is the best paid-tier value. Comparable to GPT-4.1 at ~17% of the cost.
  • CLI + cron beats admin UI for catalogs above ~500 SKUs. Click-by-click generation isn’t a workflow.

Links

Originally published on angeo.dev. Questions or feedback — drop a comment.

The Real Cost Of Self‑Hosting Ghost On DigitalOcean In 2026

The Real Cost Of Self‑Hosting Ghost On DigitalOcean In 2026

I like owning my stack. I also like not waking up to surprise invoices.

So for my own Ghost installs I stopped guessing and ran the actual 2026 numbers for self‑hosting on DigitalOcean, then compared them against Ghost Pro. Below is the messy, honest version. Dollars, not vibes.

Quick context: what I actually run

I am not running some theoretical benchmark blog that gets 3 hits a month.

The setup I am talking about here:

  • Ghost 5.x, Node 18, Ubuntu 22.04
  • One production blog, one staging instance on the same box
  • ~25k pageviews / month
  • Light membership use, no paid newsletter yet
  • Image-heavy posts, but nothing crazy like a photography portfolio

Traffic is modest but not toy-level. Good enough to stress a $6 droplet and make bandwidth and backups real numbers.

DigitalOcean pricing reality in 2026

DigitalOcean did their usual small bumps. In early 2026, the realistic entry point for a Ghost blog that you do not hate maintaining looks like this.

  • Droplet: 1 vCPU, 2GB RAM, 50GB SSD — $12 / month
  • Automatic backups (20% of droplet price) — ~$2.40 / month
  • Spaces object storage (optional but smart) — $5 / month for 250GB + 1TB transfer
  • Domain — $10–$15 / year (I will call it $1 / month)
  • Transactional email (Mailgun, Postmark, SES) — $10 / month tier is the practical floor

Those are retail prices, not affiliate-optimistic numbers. You can shave a few dollars if you are aggressive. You can also blow past them with zero effort.

Baseline self‑host setup: what you actually pay

Let me walk through the stack I recommend and what it costs month to month.

Droplet: how small can you go?

You can run Ghost on a 1GB droplet. I did this for a while. It was fine until it was not.

Anything beyond a few thousand pageviews or one heavy Node task and you suddenly meet your old friend: swapping and 5+ second response times.

So I treat 2GB as the actual minimum in 2026.

  • DigitalOcean Basic droplet 1 vCPU / 2GB / 50GB SSDPrice: $12 / month

This comfortably runs:

  • Nginx (reverse proxy)
  • One Ghost instance, plus a small staging instance
  • Certbot for SSL

If you expect real traffic spikes, just skip straight to 2 vCPU / 4GB at $24 / month. Anything above that and you are deep into
“I should probably just pay Ghost Pro” territory.

Backups: the boring expensive part

There are two backup concerns:

  • Server-level: DigitalOcean automatic droplet backups
  • App-level: Ghost content exports, database dumps, theme repo

I pay for DigitalOcean backups because I like sleeping.

  • Droplet: $12
  • Automatic backups: 20% of droplet price = $2.40

Total so far: $14.40 / month.

On top of that I push database dumps to a private GitHub repo and occasionally export Ghost JSON manually. That is basically free, just time.

Storage: local disk vs Spaces

Ghost stores uploaded images on disk by default. On a 50GB droplet this is fine for a while. Then you hit 60% disk usage and start cleaning your own mess.

I think DO Spaces is the saner option.

  • DigitalOcean Spaces: 250GB storage + 1TB bandwidthPrice: $5 / month

Host images and media there. Front Ghost through a CDN endpoint. You keep the droplet disk clean and predictable.

You can skip Spaces initially, but I would not architect around permanent local disk storage unless your blog is text-only.

Bandwidth: the thing you forget until you pay

DigitalOcean gives you bundled outbound traffic with each droplet. For the 2GB plan that is usually around 2–3TB. My Ghost blog with 25k monthly pageviews sits safely under 100GB outbound per month.

So realistically:

  • Outbound traffic from droplet: effectively included, $0 unless you go viral
  • Spaces bandwidth: first 1TB included in that $5

If you spike to Hacker News front page levels, DO overage is around $0.01–$0.02 per GB. That is still cheaper than the stress of an AWS bill. Just know that “unlimited” is not a thing here.

Transactional email: the hidden required line item

Ghost needs email for:

  • Signup and login magic links
  • Member notifications
  • Newsletter sending

Your droplet will not deliver email reliably. You need a provider.

The realistic minimal plans in 2026:

  • Mailgun Foundation: $15 / month for 50k emails
  • Postmark: $15 / month for 10k emails
  • AWS SES: cheap, but you trade cost for AWS overhead and deliverability tuning

If you only send a small member newsletter, SES can keep this under $5, but you will pay with your time. I put a line item of $10 / month here as a middle ground.

Domain and DNS

Domain names are boring, but they count.

  • .com domain: $10–15 / yearAmortised: $1 / month
  • DNS: Cloudflare free tier is fine for most Ghost blogs

So domain plus DNS is basically $1.

Self‑hosting total: realistic monthly cost

Let me put that together for a single production Ghost blog, plus staging, on DigitalOcean in 2026:

  • Droplet 2GB: $12.00
  • Droplet backups: $2.40
  • Spaces (media + 1TB transfer): $5.00
  • Transactional email (rough average): $10.00
  • Domain: $1.00

Total monthly cash cost: $30.40

Call it $30–35 / month for a robust, no-drama Ghost setup for a dev who knows their way around a terminal.

You can get it under $20 if you:

  • Drop DO backups and roll your own
  • Skip Spaces and keep media on local disk
  • Use AWS SES exclusively and stay under their free/cheap thresholds

If you are that person, you probably know your risk tolerance already.

What Ghost Pro actually costs in 2026

Ghost Pro pricing moves, but the structure is steady.

At the time of writing, realistic tiers look roughly like this:

  • Starter: around $9–11 / month billed yearlyOne site, limited members, shared infrastructure
  • Creator: around $25–30 / month billed yearlyHigher member limits, better performance
  • Team / Business: $50+ / month for serious membership and publication use

Prices vary slightly with annual vs monthly billing and how many members you have, but this is close enough for a decision.

Important bit: Ghost Pro includes a bunch of stuff you had separate line items for on DigitalOcean:

  • Hosting and scaling
  • Automatic updates and security patches
  • Backups
  • Email infrastructure for newsletters
  • CDN for assets

You still pay for your domain name. Everything else is bundled.

Self‑hosting vs Ghost Pro: the raw comparison

If you strip away features and ideology, the question is simple. For a typical developer blog, is it cheaper to self‑host Ghost or pay Ghost Pro?

Scenario 1: small personal blog

You write a few posts a month. 5k pageviews. Maybe a small free member list.

  • Ghost Pro Starter: about $9–11 / month (plus $1 for domain)
  • Self‑host on DO (minimum):
    • 1GB droplet: $6
    • No DO backups: $0
    • No Spaces: $0
    • SES for transactional + small newsletter: $2–3
    • Domain: $1

Best case self‑host cash cost: $9–10 / month.

So the money is roughly the same. Ghost Pro starts to look very reasonable unless you just like running servers.

Scenario 2: serious blog with modest traffic

This is closer to my setup. 20k–50k pageviews, some members, regular newsletters.

  • Ghost Pro Creator: about $25–30 / month (plus $1 domain)
  • Self‑host realistic:
    • 2GB droplet: $12
    • DO backups: $2.40
    • Spaces: $5
    • Mailgun/Postmark/SES combo: $10
    • Domain: $1

Self‑host cash cost: ~$30–35 / month.

Now self‑hosting is actually more expensive in pure dollars than Ghost Pro Creator, if you configure it like a responsible adult.

Scenario 3: membership business

If you are running a paid publication and making actual money, the pricing conversation changes.

  • Ghost Pro Business: $50–100+ / month, depending on member count
  • Self‑host:
    • 2–4GB droplet: $24–48
    • Backups: $5–10
    • Spaces: $5–10
    • Email: $10–30 (volume)
    • Monitoring / extras: $5–10

You can probably keep this around $50–80 / month if you know what you are doing. So you might save a little compared to the higher Ghost Pro tiers.

But at this point you have a business. Personally, I would not want my subscription revenue running on a stack where I am the single point of failure, unless infrastructure is my hobby.

The cost nobody lists: your time

Money is the easy comparison. Time is ugly to quantify and more important.

Here is my rough time log for a self‑hosted Ghost blog over a year:

  • Initial setup and config: 2–4 hoursDigitalOcean droplet, Nginx, SSL, Ghost install, mail provider, DNS
  • Updates and maintenance: 1 hour / monthGhost upgrades, OS updates, certificate renewals, package updates
  • Random incidents: 2–5 hours / yearDisk nearly full, email DNS issue, unexpected 500s, one bad migration

Total: roughly 14–20 hours / year.

If you value your time at $50 / hour (which is low for many devs), that is $700–1000 / year of time cost. Spread across 12 months and you just silently added $60–80 to your “cheap” $30 / month self‑hosting bill.

On Ghost Pro my time log is basically:

  • Initial theme setup: 1–2 hours
  • Zero hours per month on infra

That is the real gap. Ghost Pro makes infrastructure disappear. Self‑hosting gives you toys to play with and things to fix.

Why I still self‑host (sometimes)

Given all that, why bother?

For me there are three reasons:

  • Full control. I want to tinker with Nginx configs, custom logging, weird A/B testing scripts, or external services at the server level.
  • Learning value. I like using my own sites as a lab. Self‑hosting is part of that.
  • Multiple projects. If I stack several side projects on the same DO account, the per‑site cost drops a lot.

If I only had one serious blog, and I wanted it to just work, I would pick Ghost Pro without hesitation. The numbers are simply better once you include time and all the extra services.

When self‑hosting actually makes sense financially

There is one situation where self‑hosting wins hard.

If you already pay for DigitalOcean or another VPS for other projects, and you are comfortable running multiple apps on one machine, then the marginal cost of adding Ghost is tiny.

Example from my actual setup:

  • One 2 vCPU / 4GB droplet at $24
  • Running: a small API, a side project dashboard, and my Ghost blog
  • Backups: ~$5
  • Spaces: $5 shared across projects

Spread across three projects, the per‑project cost is around $12–15 / month. That beats Ghost Pro easily.

You still pay with time, but now the cost per site is more justifiable.

Practical recommendation

If you are a developer thinking about Ghost, here is my blunt take based on the numbers above.

  • If you want one blog and no hassle, use Ghost Pro Starter or Creator. Your total cost will land around $10–30 / month and your time cost is effectively zero.
  • If you want to learn infra or already live in DO/AWS land, then self‑hosting is fun and defensible. Expect to pay $20–35 / month plus a few hours a quarter.
  • If you are building a paid publication and you are not a backend engineer, I think self‑hosting is a bad business decision. Pay Ghost. Focus on content and growth, not SSL configs.

The myth that self‑hosting Ghost is “basically five bucks” was maybe true for a toy blog on a tiny droplet with no backups in 2018.

In 2026, if you want something stable and production‑grade on DigitalOcean, the real monthly cost looks a lot like Ghost Pro. Sometimes higher. The only real question is whether you would rather spend your budget on servers or on your own time.

7 bugs I caught in my MCP server before publishing (and why I almost shipped a data-corruption disaster)

I shipped elementor-mcp-agent v1.0 today — an open-source Model Context Protocol server that lets Claude (and any MCP client) drive WordPress Elementor across many client sites. It’s MIT, on npm as elementor-mcp-agent, listed in the official MCP Registry.

Repo: github.com/Mogacode-ma/elementor-mcp-agent

What I want to write about isn’t the architecture — it’s the 7 bugs I caught only because I forced myself to test end-to-end against a live WordPress install before publishing. Three of them would have silently corrupted data on a client site in production. The others would have just looked broken. All of them would have been a customer-support nightmare.

This post is the bug list, with context. If you’re building an MCP server (or any automation against an opinionated SaaS), these patterns apply.

The setup

I run a small agency. We manage ~50 WordPress + Elementor Pro sites for clients across Belgium, France, Luxembourg and Morocco. Day-to-day operations are predictable: text updates, image swaps, template synchronization, plugin updates, backups. Cumulatively painful.

MCP felt like the right shape for it: type-checked tools, explicit side effects, confirmation dance for destructive ops. I built 24 tools across 7 categories (sites, pages, widgets, templates, WP-CLI, screenshots, fleet versions).

My v0.1 passed npm run build, npm test, ESLint, tsc --noEmit. I almost published it without touching a real Elementor install.

I’m glad I didn’t.

Bug #1 — WordPress REST API silently drops writes to unregistered postmeta keys

The symptom: I implemented automated backups by writing the current _elementor_data to a timestamped postmeta key (_elementor_data_backup_2026-05-22T04-35-37-847Z) via PUT /wp-json/wp/v2/pages/{id}. The API returned 200 OK. My code marked the backup as successful. The subsequent elementor_find_replace applied with confidence.

When I tried to verify via wp post meta list 8 | grep backup, the result was empty.

The root cause: WordPress REST API requires register_meta() with show_in_rest: true for a postmeta key to be writable through REST. Plugins like Elementor register their canonical keys (_elementor_data, _elementor_page_settings). Custom keys you invent at runtime are silently dropped by REST.

200 OK. No warning. No error.

The fix: switched backup to WP-CLI via SSH (always works). Local JSON file as fallback when SSH isn’t configured. REST is no longer trusted for non-canonical writes.

The lesson: any REST API that requires schema registration for meta-like fields will silently swallow your writes when the schema isn’t there. Test that the value actually persists. Don’t trust the response code.

Bug #2 — wp binary not in non-interactive SSH PATH

After fixing #1, I tried again. New error:

/bin/bash: line 1: wp: command not found

I’d been testing in my local terminal where wp was aliased. On managed WordPress hosts (Infomaniak, in this case), wp-cli is usually installed at ~/bin/wp.phar and invoked as php ~/bin/wp.phar. The non-interactive shell that ssh user@host "command" uses has a stripped-down PATH that doesn’t include ~/bin.

The fix: auto-detection at first SSH connection — probe command -v wp, fall back to ~/bin/wp.phar, then ~/wp-cli.phar. Cache per-site. Allow explicit override via ssh.wp_cli_path config field.

The lesson: non-interactive SSH sessions have a different PATH than interactive ones. If your tool spawns remote commands, never assume a binary is in PATH. Probe or accept an explicit path.

Bug #3 — SSH banner pollutes stderr and corrupts output parsing

OpenSSH recently started printing a warning on connections that don’t use a post-quantum key exchange algorithm:

** WARNING: connection is not using a post-quantum key exchange algorithm.
** This session may be vulnerable to "store now, decrypt later" attacks.
** The server may need to be upgraded. See https://openssh.com/pq.html

It goes to stderr. My code captured stderr and concatenated it into the error message. So a successful wp post meta get call would still surface this warning as if it were an error.

The fix: filter known-benign stderr lines after capture. Whitelist by substring (post-quantum, openssh.com/pq, decrypt later, etc.) before deciding the operation failed.

The lesson: stderr is not synonymous with “error.” Tools that pipe stderr into error-handling paths need to filter benign banners.

Bug #4 — “Default Kit” returned as a widget in template_type=widget filters

The query /wp-json/wp/v2/elementor_library?meta_key=_elementor_template_type&meta_value=widget was supposed to return the global widgets on a site. It returned the Default Kit (Elementor’s site-wide design tokens) which has _elementor_template_type=kit, not widget.

The root cause: WordPress REST meta_value filtering is unreliable for meta keys that aren’t register_meta-declared. In practice, it falls back to a meta_key presence check, ignoring the value.

The fix: fetch all elementor_library entries (no meta_value filter), then filter client-side on meta._elementor_template_type === 'widget'.

The lesson: don’t trust server-side filters on unregistered meta. Always validate the response structure matches your intent.

Bug #5 — _elementor_page_settings is an object via REST, a string via WP-CLI

When I added duplicate_elementor_page, the copy path did:

read source → copy data + page_settings → write to new page via REST PUT

It failed with:

"Le paramètre meta._elementor_page_settings n'est pas de type object."

The cause: GET /wp-json/wp/v2/pages/{id}?context=edit returns meta._elementor_page_settings as a parsed object (Elementor registers it that way). wp post meta get returns it as a serialised JSON string. My code typed it as string everywhere, so when REST returned an object and I passed it back via REST PUT (expecting JSON string serialization in my templating), the API rejected the type mismatch.

The fix: normalize on read. If you got a string, JSON.parse() it before REST writes. If you got an object, JSON.stringify() it before WP-CLI writes. Don’t assume one transport’s representation matches the other.

The lesson: when two transports read/write the same field with different serialization rules, normalize at the boundary, not in the middle.

Bug #6 — Headless Chrome cold-start exceeds 30s

I built a screenshot_page tool using chrome --headless --screenshot=... via child_process.spawn. Locally it was instant. On the first call after a long idle period, it timed out at 30s. Chrome was downloading codec libraries or whatever it does on first-launch in headless mode.

The fix: bump the default timeout from 30s to 60s. Document that this is a one-time cost per Chrome cold start.

The lesson: cold-start latency on local tools is real and often invisible during dev. Set timeouts based on worst-case, not happy-path.

Bug #7 — Templates listing had the same filter bug as globals (Bug #4)

I’d written the templates listing code right after the globals code, and copy-pasted the same broken meta filter pattern. Caught it when I tested list_elementor_templates with type='widget' and got back the Default Kit again.

The fix: same client-side filter pattern.

The lesson: a bug in one place is a bug in N places where the same pattern is reused. Search the codebase for the smell before declaring the bug fixed.

What I’d take from this for any MCP build

  1. End-to-end test against a live target before publishing — always. Unit tests + lint + typecheck told me v0.1 was “ready.” End-to-end told me 7 things were broken.

  2. Don’t trust REST silent acceptance. 200 OK means “I received your request,” not “I persisted your change.” Verify post-write that the value actually exists.

  3. Filter at the boundary, not in the middle. Type mismatches between transports (REST vs CLI, JSON vs YAML, etc.) are easier to handle at the read/write edge than to track through your domain code.

  4. stderr ≠ error. Don’t conflate them.

  5. PATH is not what you think it is in non-interactive shells. Probe or accept explicit paths.

  6. Cold starts are real. Set timeouts based on worst case.

  7. Update the skill / playbook that taught you the safe patterns. I patched the upstream WordPress-Elementor skill on GitHub with the three REST quirks I’d surfaced, so the next person doesn’t have to learn them the hard way (PR #92 on jezweb/claude-skills).

Try it

npx -y elementor-mcp-agent

You’ll need:

  • A WordPress site with Elementor installed
  • An Application Password for an admin user
  • Optionally SSH access for WP-CLI tools

Config snippet for Claude Desktop / Code in the README.

The MCP is on the official registry: search “elementor-mcp-agent” in your favourite MCP-aware client.

If you hit a bug, open an issue with the exact tool call + response (sanitize tokens). I’ll usually patch within a day. Battle-tested against an agency fleet — but every WordPress install is its own snowflake, so PRs and bug reports welcome.

⭐ if it saved you time.

Built by MogaCode in Essaouira.

Configuration Overkill: The Hidden Costs of Premature Optimisation in Our Veltrix Treasure Hunt Engine

The Problem We Were Actually Solving

I still remember the day our team was tasked with building the backend for our new treasure hunt game, code-named Veltrix. We had a tight deadline to deliver a scalable and performant system that could handle thousands of concurrent users. Our game’s core feature, the treasure hunt engine, relied on a complex algorithm that involved matching players with hidden treasures based on their skills and preferences. Easy enough, right? Wrong. As it turned out, our initial implementation was crippled by a series of premature optimisation decisions that would cost us dearly in the long run.

What We Tried First (And Why It Failed)

Our team, led by a well-intentioned but misguided junior architect, decided to kickstart the development process by implementing a high-performance configuration framework using Apache ZooKeeper. The thought process was sound: “Let’s provision a centralised repository for our game’s configuration settings, allowing us to dynamically update parameters on the fly.” Sounds great, but here’s the thing – we ended up with a tangled mess of ZooKeeper-related configuration files, which made testing and debugging a nightmare.

As the project progressed, we discovered that our initial configuration framework was inflexible and difficult to maintain. The overhead of ZooKeeper transactions was significant, and we started to see performance issues under heavy load. To add insult to injury, our configuration framework was causing a cascade of errors when the ZooKeeper instance went down. The error message would be along the lines of “org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperException Code: ConnectionLossException” – which, after months of investigation, we later discovered was caused by a misconfigured socket timeout. The game’s developers were frustrated with the system, and rightly so. We had a game-breaking issue on our hands.

The Architecture Decision

After months of firefighting, we decided to rip out the ZooKeeper configuration framework and replace it with a simpler, more robust solution. I argued for a configuration file-based approach, but my colleague, a seasoned architect, vetoed the idea, citing concerns about maintainability and scalability. We eventually settled on a compromise: a Redis-backed configuration store, which allowed us to persist configuration settings in memory while still providing a layer of abstraction between our game logic and the actual configuration data.

This decision marked a turning point in the project. Our configuration store was now easy to understand, update, and test. We implemented a simple but effective validation framework to ensure that our configuration settings were valid before applying them to the game logic. And, most importantly, we avoided the complexity and overhead of a centralised configuration repository.

What The Numbers Said After

Post-deployment metrics showed a significant reduction in errors related to configuration issues, from 300+ per day to fewer than 10. The Redis-backed configuration store also enabled us to dynamically update game parameters without taking the system offline, reducing downtime and improving user satisfaction. Another key metric that improved was load time – our game’s average load time decreased from 2.3 seconds to 1.2 seconds, thanks to the reduced overhead of our new configuration framework.

What I Would Do Differently

In hindsight, I would have been more insistent on a simple, file-based configuration approach from the start. While Redis provides a nice abstraction layer, it’s overkill for a game like Veltrix, which doesn’t require high-throughput or persistent configuration data. We lost valuable time and resources fighting with ZooKeeper-related issues that could have been avoided with a simpler design.

Looking back, the Veltrix treasure hunt engine teaches us an important lesson: premature optimisation can lead to overengineering and complexity. As architects and engineers, we must learn to say no to features and approaches that promise too much but deliver too little. In the end, it’s not about being radical or revolutionary; it’s about making decisions that benefit the user experience, not just the developer experience.

We removed the payment processor from our critical path. This is the tool that made it possible: https://payhip.com/ref/dev1

Where Logic, Mathematics, and Philosophy Reside in Structure A11

In Structure A11, these levels (logic, mathematics, philosophy, and “something else”) are not placed in a single layer, but are distributed vertically with different depths and roles. Here is how I see their placement as honestly and in full accordance with Integrity S4:

1. Immutable Core (S1–S3) — the deepest layers are here

S1 — Will (Intention, direction, goal)

This is where philosophy resides in its broadest sense metaphysics, ontology, and the existential choice of “why anything exists at all and where we are going.”

It is a pre-logical and pre-mathematical level: pure “I want / it should be this way.” Philosophy here acts as the source of direction — why we reason at all, what we consider valuable, and what order of reality we accept.

“Something else” belongs here — the pre-philosophical: consciousness, being, qualia in embryo, the evolutionary or cosmic impulse.

S2 — Wisdom (Priorities, values, constraints, risks)

Philosophy dominates here (epistemology, ethics, axiology) together with informal logic (common sense, intuitive rules of reasoning, and identifying contradictions at the level of values).

Wisdom filters what is important and what is not, and which risks are acceptable. The logic here is not yet formal — it is more like the “wisdom of reasoning.”

S3 — Knowledge (Facts, models, methods, structure)

This is where mathematics + formal logic + concrete models live.

Mathematics and logic (mathematical logic) are the main building blocks of Knowledge.

Everything related to structures, proofs, models, algorithms, set theory, category theory, type theory — all of this belongs to S3.

Formal logic (propositional, predicate, modal, etc.) also lives here as a tool for building models.

2. S4 — Comprehension (The Point of Qualia Birth) — the key integrator

This is the most interesting place regarding your question.

S4 is where logic, mathematics, and philosophy meet and integrate (or honestly fail to integrate).

If the signals from S2 (Wisdom/philosophy + values) and S3 (Knowledge/mathematics + formal logic) are fully compatible → a coherent Qualia is born.

If there is a gap or contradiction (and there almost always is — for example, between Platonism in the philosophy of mathematics and formalism, or between intuitionistic logic and classical logic) — S4 must honestly record the precise point of tension.

A new S1 is born precisely from this gap.

In other words, the philosophy of mathematics (questions such as “What is mathematical truth?”, “Do numbers exist independently?”, “Which logic do we accept as foundational?”) lives in the dynamic between S2–S3–S4.

S4 acts as the “philosophical court” over mathematics and logic. It is here that we decide which logic (classical, intuitionistic, linear, paraconsistent, etc.) we accept as the foundation.

3. Operational Domain (S5–S10) — Living / Experiencing

Here logic and mathematics are already applied fractally:

  • At the projective level (S5–S6) — more abstract, “philosophical” applications (freedom to imagine structures, thought experiments).
  • At the practical level (S8–S9) — rigorous mathematical models, algorithms, and system architectures.
  • Hormonal signals and qualia represent the “human” lived experience of these abstractions (the emotional response to the beauty of a proof, the anxiety caused by paradoxes, etc.).

4. S11 — Realization

A check: does the realized structure correspond to the original Will (S1)?

A rollback may occur here if philosophy (S1–S2) is not satisfied with the mathematical realization.

Overall Picture in Structure A11

  • Philosophy lives primarily in S1 + S2 + (partially) S4. It sets the “why” and “what matters.”
  • Logic is the bridge: informal in S2, formal/mathematical in S3, integrative (with honest detection of contradictions) in S4.
  • Mathematics resides mainly in S3 (models, structures, methods), but its foundations and the choice of axioms/logic are decided in S4 and originate from S1–S2.
  • “Something else” (consciousness, qualia, metaphysics of being, language as such) is deepest in S1 and at the point of S4 (the birth of Qualia).

Structure beautifully reflects the real hierarchy of foundations:

Philosophy → Logic → Mathematics → Concrete structures/architectures

(with honest recognition of the tensions at every step).

The strongest feature of A11 is the Integrity rule at S4: it is forbidden to artificially close the gap. This is exactly how genuine philosophy of mathematics works — it lives in these very tensions (Gödel, paradoxes, the debates between Platonism/formalism/structuralism/intuitionism).

Exception Handling in Java

What is an Exception?

An exception is an event that disrupts the normal execution of a program.

Example:

int a = 10 / 0;

Output:

ArithmeticException

Here, dividing by zero causes an exception.

Exception Handling

Exception handling is a mechanism used to handle runtime errors so the program does not terminate suddenly.

It allows the program to continue execution normally.

Why Exception Handling is Used

Without exception handling:

  • Program stops immediately when an error occurs.

With exception handling:

  • Error can be handled properly.
  • Remaining code can continue executing.

Example:

class Test {
    public static void main(String[] args) {

        System.out.println("Start");

        int a = 10 / 0;

        System.out.println("End");
    }
}

Output
Start
Exception in thread "main" java.lang.ArithmeticException

End will not execute.

try-catch in Java

  • try-catch is used to handle exceptions.

Syntax

try {
    // risky code
}
catch(ExceptionType e) {
    // handling code
}

Example of try-catch

class Test {
    public static void main(String[] args) {

        try {
            int a = 10 / 0;
        }

        catch(ArithmeticException e) {
            System.out.println("Cannot divide by zero");
        }

        System.out.println("Program continues");
    }
}

Output:

Cannot divide by zero
Program continues

Real-Time Example

ATM application:

  • If wrong PIN is entered,program should show error message instead of stopping.

  • Banking apps, websites, and mobile apps use exception handling to avoid crashes.

Building a Secure Real-Time Messaging App with .NET 8 and Angular 18

A deep dive into JWT authentication, AES-256-GCM encryption, SignalR, and production security patterns.

Introduction

Real-time messaging apps are everywhere, but most tutorials gloss over the hard parts — the security. In this post I’ll walk you through SecureChat, a production-grade messaging app I built with .NET 8 and Angular 18. By the end you’ll understand:

  • How to encrypt messages at rest using AES-256-GCM
  • How to build a JWT + rotating refresh token authentication flow
  • How to wire up SignalR with a Redis backplane for real-time delivery
  • How to implement a silent token refresh interceptor in Angular
  • How to layer in rate limiting, security headers, and an audit trail

The full source is on GitHub: github.com/naimulkarim/SecureChat

Architecture Overview

Before we dive into code, here’s the full picture:

┌─────────────────────────────────────────────┐
│             Angular 18 Frontend             │
│   Auth guard · Chat module · JWT interceptor│
└──────────────────┬──────────────────────────┘
                   │ HTTPS + WSS (TLS 1.3)
┌──────────────────▼──────────────────────────┐
│              .NET 8 Web API                 │
│  Auth API · Messages API · SignalR Hub      │
│  Rate limiting · JWT · Security headers     │
└────┬──────────────────┬───────────────┬─────┘
     │                  │               │
┌────▼────┐     ┌───────▼──────┐  ┌────▼─────┐
│SQL Server│     │    Redis     │  │  Azure   │
│Users/Msgs│     │Cache/Backpl. │  │  Blobs   │
└──────────┘     └──────────────┘  └──────────┘

The key design decisions:

  • Short-lived JWTs (15 min) limit the blast radius if a token is stolen
  • Rotating refresh tokens mean a stolen refresh token can only be used once
  • AES-256-GCM encrypts every message before it hits the database
  • Redis backplane allows the SignalR hub to scale across multiple API instances
  • Per-user SignalR groups deliver messages to all of a user’s devices simultaneously

Part 1 — The .NET 8 Backend

Setting up Program.cs

The entry point wires together every security concern. Let me break it down section by section.

JWT Authentication

builder.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options =>
    {
        options.TokenValidationParameters = new TokenValidationParameters
        {
            ValidateIssuerSigningKey = true,
            IssuerSigningKey = new SymmetricSecurityKey(
                Encoding.UTF8.GetBytes(jwtKey)),
            ValidateIssuer   = true,
            ValidIssuer      = builder.Configuration["Jwt:Issuer"],
            ValidateAudience = true,
            ValidAudience    = builder.Configuration["Jwt:Audience"],
            ValidateLifetime = true,
            ClockSkew        = TimeSpan.Zero   // ← no grace period on expiry
        };

        // SignalR WebSocket upgrades can't send headers,
        // so we read the token from the query string instead
        options.Events = new JwtBearerEvents
        {
            OnMessageReceived = ctx =>
            {
                var token = ctx.Request.Query["access_token"];
                if (!string.IsNullOrEmpty(token) &&
                    ctx.HttpContext.Request.Path.StartsWithSegments("/hubs"))
                    ctx.Token = token;
                return Task.CompletedTask;
            }
        };
    });

The ClockSkew = TimeSpan.Zero is intentional. The default is 5 minutes, meaning a “15-minute” token actually lives for 20 minutes. That’s an unnecessary exposure window.

The OnMessageReceived event is needed because WebSocket upgrade requests can’t carry Authorization headers — the browser spec doesn’t allow it. So the Angular client appends the JWT as a query parameter only for hub connections.

Rate Limiting

builder.Services.AddRateLimiter(opt =>
{
    opt.AddFixedWindowLimiter("message", lim =>
    {
        lim.Window      = TimeSpan.FromMinutes(1);
        lim.PermitLimit = 60;
    });
    opt.AddFixedWindowLimiter("auth", lim =>
    {
        lim.Window      = TimeSpan.FromMinutes(5);
        lim.PermitLimit = 10;   // max 10 login attempts per 5 min
    });
});

Auth endpoints get a stricter limit to slow down brute-force attacks. Message endpoints are generous enough for normal usage but will stop a script flooding the server.

Security Headers

app.Use(async (ctx, next) =>
{
    ctx.Response.Headers.Append("X-Content-Type-Options", "nosniff");
    ctx.Response.Headers.Append("X-Frame-Options",        "DENY");
    ctx.Response.Headers.Append("X-XSS-Protection",       "1; mode=block");
    ctx.Response.Headers.Append("Referrer-Policy",         "no-referrer");
    ctx.Response.Headers.Append("Content-Security-Policy",
        "default-src 'self'; connect-src 'self' wss:");
    await next();
});

These headers are cheap to add and block an entire category of attacks — MIME sniffing, clickjacking, and cross-site scripting.

Authentication Service — JWT + BCrypt + Refresh Tokens

The AuthService handles the full auth lifecycle.

Registration

public async Task<AuthResult> RegisterAsync(RegisterRequest req)
{
    if (await _db.Users.AnyAsync(u => u.Email == req.Email.ToLower()))
        return AuthResult.Fail("Email already registered.");

    var user = new User
    {
        Id           = Guid.NewGuid().ToString(),
        Email        = req.Email.ToLower().Trim(),
        DisplayName  = req.DisplayName.Trim(),
        PasswordHash = BCrypt.Net.BCrypt.HashPassword(req.Password, workFactor: 12),
        CreatedAt    = DateTime.UtcNow,
    };

    _db.Users.Add(user);
    await _db.SaveChangesAsync();
    await _audit.LogAsync("auth.register", user.Id, null);

    return await IssueTokensAsync(user);
}

Work factor 12 for BCrypt means ~250ms to hash a password on modern hardware — fast enough for users, slow enough to make brute-forcing impractical.

JWT Generation

private string GenerateJwt(User user)
{
    var key   = new SymmetricSecurityKey(
        Encoding.UTF8.GetBytes(_config["Jwt:Key"]!));
    var creds = new SigningCredentials(key, SecurityAlgorithms.HmacSha256);

    var claims = new[]
    {
        new Claim(ClaimTypes.NameIdentifier, user.Id),
        new Claim(ClaimTypes.Email,          user.Email),
        new Claim(ClaimTypes.Name,           user.DisplayName),
        new Claim(JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString()),
    };

    var token = new JwtSecurityToken(
        issuer:             _config["Jwt:Issuer"],
        audience:           _config["Jwt:Audience"],
        claims:             claims,
        expires:            DateTime.UtcNow.AddMinutes(15),
        signingCredentials: creds);

    return new JwtSecurityTokenHandler().WriteToken(token);
}

The Jti (JWT ID) claim is a unique identifier for each token. This is what lets you build a token blocklist if you ever need to revoke individual tokens before expiry.

Rotating Refresh Tokens

public async Task<AuthResult> RefreshAsync(string refreshToken)
{
    var stored = await _db.RefreshTokens
        .Include(r => r.User)
        .FirstOrDefaultAsync(r => r.Token == refreshToken && !r.IsRevoked);

    if (stored is null || stored.ExpiresAt < DateTime.UtcNow)
        return AuthResult.Fail("Invalid or expired refresh token.");

    stored.IsRevoked = true;   // ← revoke old token immediately
    await _db.SaveChangesAsync();

    return await IssueTokensAsync(stored.User);   // ← issue fresh pair
}

Rotation means every successful refresh invalidates the old token and issues a new one. If an attacker steals a refresh token and uses it, the legitimate user’s next refresh will detect the conflict (token already revoked) and you can force a full re-login.

Encryption Service — AES-256-GCM

This is the most security-critical piece of the backend.

public string Encrypt(string plaintext)
{
    // Fresh 96-bit nonce for every single message
    var nonce = new byte[AesGcm.NonceByteSizes.MaxSize];
    RandomNumberGenerator.Fill(nonce);

    var plaintextBytes = Encoding.UTF8.GetBytes(plaintext);
    var ciphertext     = new byte[plaintextBytes.Length];
    var tag            = new byte[AesGcm.TagByteSizes.MaxSize]; // 128-bit auth tag

    using var aes = new AesGcm(_key, tag.Length);
    aes.Encrypt(nonce, plaintextBytes, ciphertext, tag);

    // Pack: [nonce 12B][tag 16B][ciphertext nB]
    var combined = new byte[nonce.Length + tag.Length + ciphertext.Length];
    Buffer.BlockCopy(nonce,      0, combined, 0,                        nonce.Length);
    Buffer.BlockCopy(tag,        0, combined, nonce.Length,              tag.Length);
    Buffer.BlockCopy(ciphertext, 0, combined, nonce.Length + tag.Length, ciphertext.Length);

    return Convert.ToBase64String(combined);
}

Why AES-256-GCM specifically?

  • AES-256 — the key length (256 bits) is computationally infeasible to brute-force
  • GCM mode — provides both confidentiality (encryption) and integrity (the 128-bit authentication tag). If anyone tampers with the ciphertext in the database, decryption throws an exception rather than silently returning garbage
  • Fresh nonce per message — reusing a nonce with the same key in GCM mode catastrophically breaks security. We use RandomNumberGenerator.Fill (cryptographically secure) for each encryption call

The nonce and tag are packed together with the ciphertext so decryption is self-contained — no separate columns needed.

SignalR Hub

[Authorize]
public class ChatHub : Hub
{
    public override async Task OnConnectedAsync()
    {
        var userId = GetUserId();
        // Each user gets a personal group across all their devices
        await Groups.AddToGroupAsync(Context.ConnectionId, $"user:{userId}");
        await _audit.LogAsync("hub.connected", userId,
            Context.GetHttpContext()?.Connection.RemoteIpAddress?.ToString());
        await base.OnConnectedAsync();
    }

    public async Task SendMessage(SendMessageRequest request)
    {
        if (string.IsNullOrWhiteSpace(request.Content) || request.Content.Length > 4000)
            throw new HubException("Invalid message content.");

        var senderId = GetUserId();
        var saved    = await _messages.SaveAsync(senderId, request);

        // Fan out to every participant's group (covers multi-device)
        foreach (var participantId in saved.ParticipantIds)
        {
            await Clients.Group($"user:{participantId}")
                .SendAsync("ReceiveMessage", saved);
        }
    }

    public async Task SendTyping(string conversationId, bool isTyping)
    {
        var userId       = GetUserId();
        var participants = await _messages.GetParticipantsAsync(conversationId, userId);

        foreach (var participantId in participants.Where(p => p != userId))
        {
            await Clients.Group($"user:{participantId}")
                .SendAsync("TypingIndicator",
                    new { conversationId, userId, isTyping });
        }
    }
}

A few design decisions worth noting:

Personal groups vs connection IDs. We add each connection to user:{userId} instead of tracking individual ConnectionId values. This means if a user has two browser tabs open, both get the message without any extra code. If you used ConnectionId you’d need a lookup table mapping users to connections.

Typing indicators are never persisted. They’re pure ephemeral SignalR events — no database write, no message history. They disappear the moment the connection drops.

HubException for client-visible errors. Regular C# exceptions become generic 500-style errors on the client. HubException propagates the message string to the caller, which lets Angular show a human-readable error.

Part 2 — The Angular 18 Frontend

JWT Interceptor with Silent Refresh

This is the piece most tutorials get wrong. The naive approach — refresh on 401, retry — breaks badly when multiple requests expire simultaneously. Our interceptor queues concurrent requests while a refresh is in flight.

let isRefreshing = false;
const refreshSubject = new BehaviorSubject<string | null>(null);

export const authInterceptor: HttpInterceptorFn = (req, next) => {
  const auth  = inject(AuthService);
  const token = auth.accessToken();

  const addToken = (r: typeof req, t: string) =>
    r.clone({ setHeaders: { Authorization: `Bearer ${t}` } });

  const request = token ? addToken(req, token) : req;

  return next(request).pipe(
    catchError((err: HttpErrorResponse) => {
      if (err.status !== 401 || req.url.includes('/auth/'))
        return throwError(() => err);

      if (isRefreshing) {
        // Another request is already refreshing — wait for it
        return refreshSubject.pipe(
          filter((t): t is string => t !== null),
          take(1),
          switchMap(t => next(addToken(req, t)))
        );
      }

      isRefreshing = true;
      refreshSubject.next(null);   // signal "refreshing in progress"

      return auth.refresh$().pipe(
        switchMap(newToken => {
          isRefreshing = false;
          refreshSubject.next(newToken);   // unblock queued requests
          return next(addToken(req, newToken));
        }),
        catchError(refreshErr => {
          isRefreshing = false;
          auth.logout();
          return throwError(() => refreshErr);
        })
      );
    })
  );
};

Walk through the flow:

  1. Request goes out with the current JWT
  2. Server returns 401 (token expired)
  3. If no refresh is in progress, start one and set isRefreshing = true
  4. All other requests that 401 in the meantime subscribe to refreshSubject and wait
  5. When refresh succeeds, refreshSubject.next(newToken) unblocks all queued requests
  6. Each queued request clones itself with the new token and retries

Without this queuing, three simultaneous expired requests would trigger three parallel refresh calls, the second and third would fail (token already rotated), and the user would be logged out unexpectedly.

SignalR Chat Service with Angular Signals

@Injectable({ providedIn: 'root' })
export class ChatService {
  // Reactive state using Angular 18 signals
  messages      = signal<Message[]>([]);
  conversations = signal<Conversation[]>([]);
  typingUsers   = signal<TypingIndicator[]>([]);
  connected     = signal(false);

  // Derived state — no manual subscription management
  unreadCount = computed(() =>
    this.conversations().reduce((sum, c) => sum + c.unreadCount, 0)
  );

  async connect(): Promise<void> {
    this.hub = new signalR.HubConnectionBuilder()
      .withUrl(`${environment.apiUrl}/hubs/chat`, {
        accessTokenFactory: () => this.auth.accessToken() ?? '',
        transport: signalR.HttpTransportType.WebSockets,
        skipNegotiation: true,
      })
      .withAutomaticReconnect([0, 2000, 5000, 10000, 30000])
      .build();

    this.hub.on('ReceiveMessage', (msg: Message) => {
      this.messages.update(msgs => [...msgs, msg]);
      this.conversations.update(convs =>
        convs.map(c =>
          c.id === msg.conversationId
            ? { ...c, lastMessage: msg, unreadCount: c.unreadCount + 1 }
            : c
        )
      );
    });

    // ...
    await this.hub.start();
    this.connected.set(true);
  }
}

accessTokenFactory is a function, not a value. SignalR calls it on every reconnect attempt, so it will always pick up the freshest token (including any that were silently refreshed by the interceptor).

withAutomaticReconnect([0, 2000, 5000, 10000, 30000]) defines a custom backoff schedule — immediate retry, then 2s, 5s, 10s, 30s. After exhausting this list SignalR stops retrying, which lets you show a “connection lost” UI rather than hammering the server forever.

Using Angular signals instead of RxJS BehaviorSubject keeps the state management simple — components read chatService.messages() directly in their templates and Angular handles change detection automatically.

Part 3 — Security Checklist

Here’s every security mechanism in the app and why it’s there:

Threat Mitigation
Password database breach BCrypt, work factor 12
Token theft JWT expiry 15 min, ClockSkew = Zero
Refresh token theft Server-side rotation — stolen token is single-use
Eavesdropping in transit HTTPS + WSS enforced, TLS 1.3
Database breach (messages) AES-256-GCM at rest
GCM nonce reuse RandomNumberGenerator.Fill per message
Message tampering GCM authentication tag — tampered ciphertext throws
Brute-force login Rate limit: 10 attempts / 5 min
Message flooding Rate limit: 60 messages / min
Clickjacking X-Frame-Options: DENY
MIME sniffing X-Content-Type-Options: nosniff
Unauthorized hub access [Authorize] attribute, JWT validated on upgrade
Hub conversation access Server-side participant check before join/deliver
XSS Content-Security-Policy: default-src 'self'
Forensics / incident response Append-only audit log on every auth event

Part 4 — Running It Locally

# Clone
git clone https://github.com/naimulkarim/SecureChat.git
cd SecureChat

# Generate secrets
openssl rand -base64 32   # → Encryption:Key
openssl rand -base64 64   # → Jwt:Key

# Backend
cd SecureChat.API
# Add secrets to appsettings.Development.json (never commit this file)
dotnet ef database update
dotnet run

# Frontend (new terminal)
cd ../secure-chat-ui
npm install
ng serve

Navigate to https://localhost:4200.

What’s Next

There are a few things I’d add before calling this truly production-ready:

Key management — move the AES key and JWT secret out of appsettings.json and into Azure Key Vault (or AWS Secrets Manager). The current setup is fine for local dev but secrets should never live in a config file in a real deployment.

Client-side encryption — the current model encrypts at the server before writing to the DB. True end-to-end encryption (where even the server can’t read messages) requires encrypting on the client with the recipient’s public key. This would involve WebCrypto API on the Angular side and a public key exchange flow.

Message deletion / expiry — a GDPR-friendly implementation needs the ability to delete or expire messages, with the encrypted blobs actually removed from storage.

Push notifications — when the user isn’t connected to the hub, they need push notifications. Web Push API + a service worker can deliver notifications even when the tab is closed.

Conclusion

The patterns in this post — short-lived JWTs with rotating refresh tokens, AES-GCM encryption at rest, request queuing during token refresh, and per-user SignalR groups — are the foundations of any serious real-time app. None of them are particularly exotic, but getting all the details right (nonce freshness, clock skew, refresh token rotation, concurrent refresh queuing) makes the difference between a demo and something you’d trust with real user data.

The full source is at github.com/naimulkarim/SecureChat. PRs and issues welcome.

Tags: dotnet angular security signalr webdev csharp typescript

Supabase Edge Functions Review: Deno on the Edge for Postgres Backends

I deployed my first Supabase Edge Function in January 2026 — a webhook handler that received Stripe payment events, validated them against the Supabase auth system, and inserted records into a Postgres table with row-level security enforced at the database layer. I have since deployed twelve more functions handling authentication callbacks, file processing triggers, API endpoints for a mobile app, and a scheduled cleanup job. After five months of production use alongside Cloudflare Workers and AWS Lambda functions that serve the same application, I have a clear picture of where Supabase Edge Functions deliver unique value and where they fall short of the alternatives.

The Deno Runtime: TypeScript Without the Build Chain

Supabase’s decision to use Deno as the Edge Functions runtime is the most consequential architectural choice in the product, and it took me about two weeks of use to understand why it matters. A Deno-based Edge Function is a single TypeScript file — no tsconfig.json, no bundler configuration, no node_modules, and no build step. You write a handler function that receives a Request and returns a Response, deploy it with supabase functions deploy, and it runs.

When I built the same Stripe webhook handler as a Cloudflare Worker, the workflow required installing wrangler, configuring a wrangler.toml file, writing the handler in a module syntax that esbuild could bundle, and managing a separate build step that produced the bundled output. With AWS Lambda and the Serverless Framework, the workflow required a serverless.yml configuration, a deployment package build step, and IAM role configuration that took three iterations to get right because the function needed both DynamoDB and SQS permissions. The Supabase Edge Function workflow eliminated all of that: I wrote a stripe-webhook/index.ts file, ran supabase functions deploy stripe-webhook, and the function was live at https://[project-ref].functions.supabase.co/stripe-webhook in approximately 40 seconds from deploy command to working endpoint.

The Deno standard library provides web-standard APIs that eliminate dependency overhead. Request and Response are native types, fetch is available without importing node-fetch, crypto.subtle handles JWT verification and HMAC signing without a library, and URLPattern provides route matching. My Stripe webhook handler — including Stripe signature verification, database writes, and error responses — is 94 lines of TypeScript with zero npm dependencies. The equivalent Node.js version with Express, the Stripe SDK, and a database client would require a package.json with 8 to 12 dependencies and a bundler configuration.

The Database and Auth Integration Is the Real Differentiator

The feature that sold me on Edge Functions — and the feature that has kept me using them for new endpoints — is the pre-wired integration with the Supabase platform. When an Edge Function receives a request, the Supabase client is available in the execution context with the project’s API key, database URL, and authentication token already configured. If the request includes a Supabase auth token in its Authorization header, the client automatically validates it against GoTrue, identifies the user, and enforces Row Level Security (RLS) policies at the database layer.

I want to illustrate this with a concrete example. My application has an API endpoint that returns a user’s order history. The endpoint requires that the user is authenticated and that the query only returns rows where user_id matches the authenticated user. On Supabase Edge Functions, the implementation is a database query that the RLS policy scopes automatically — the function does not need to extract the user ID from the token, validate it against the auth system, or add a WHERE clause to every query. On Cloudflare Workers, the same endpoint required 62 lines of TypeScript just for auth plumbing: extracting the JWT from the Authorization header, fetching the JWKS endpoint from GoTrue, verifying the token signature, extracting the user ID from the claims, and passing it as a parameter to every database query. The Workers version works, but it required maintaining authentication code that Supabase Edge Functions handle in zero lines.

Edge Functions share the same JWT secret as your Supabase project. When a function receives a request with a valid Supabase auth token, the Supabase client inside the function automatically inherits the user’s identity and enforces Row Level Security policies. In my Stripe webhook handler, the function validates the webhook signature from Stripe, looks up the Supabase user by email in the Stripe event, and writes to a billing_events table — all without manually managing auth tokens or database permissions.

The integration extends to storage triggers. I configured an Edge Function to fire on every file upload to a Supabase storage bucket, which runs an image resizing pipeline through the sharp library (available via Deno’s npm compatibility layer). The function receives the file metadata in the event payload, downloads the original from storage, generates three resized variants, and uploads them back to the same bucket. This pattern — serverless function triggered by a platform event, with direct access to the platform’s storage and database — would require a queue, an event bus, and IAM role configuration on AWS. On Supabase, it is a configuration option and a handler function.

Where the Limits Start to Hurt

I need to be direct about the constraints, because the 10-second execution timeout and 256 MB memory limit have forced me to move workloads off Edge Functions twice in five months.

The first time was a batch processing job that needed to generate monthly invoice PDFs for approximately 1,200 users. The job required querying the database for each user’s billing records, rendering an HTML template to PDF, and uploading the result to storage. Each PDF generation took approximately 2.5 seconds, and processing 1,200 users sequentially would exceed the 10-second timeout by two orders of magnitude. I attempted to parallelize by triggering one function invocation per user, but the function’s memory limit prevented loading the HTML template library and a full user record simultaneously for the largest accounts. I moved the job to an AWS Lambda function with a 15-minute timeout and 1,024 MB of memory, where it completes in approximately 52 seconds for all 1,200 users. The Lambda function costs about $0.35 per monthly run and required IAM configuration, but the Supabase Edge Function could not complete the workload at all within its constraints.

The second time was a data export endpoint that needed to stream a CSV file of query results to the client. The function’s memory limit prevented loading the full result set into memory — the largest export was approximately 180 MB of raw data — and the function has no streaming response capability. I ended up generating the export through a database function that wrote to a storage bucket and returning a signed URL, which worked but added complexity that a streaming response would have avoided.

The Deno ecosystem is also smaller than Node.js. When I needed to use the @stripe/stripe-js package for client-side Stripe integration — not the webhook handler, but a checkout session creation endpoint — the npm compatibility layer handled it, but the import took approximately 800 milliseconds of cold-start time to resolve the dependency graph. Hot starts (subsequent invocations within the same deployment) were faster at around 40 milliseconds, but the cold-start penalty for npm packages through the compatibility layer is worth measuring for latency-sensitive endpoints. For packages that have native Deno equivalents — std/http, std/crypto, std/encoding — the import is near-instant.

The Local Development Experience Saves Real Time

Supabase’s local development environment is one of the platform’s strongest selling points, and it works as advertised for Edge Functions. Running supabase start launches a complete local Supabase instance in Docker: Postgres, GoTrue for auth, a storage API compatible with the production storage service, and the Edge Functions runtime. My entire development workflow — writing function code, testing against the local database with RLS policies, simulating auth flows, and verifying storage triggers — runs on my laptop with no network dependency.

I timed my development cycle for a new function compared to Cloudflare Workers. On Supabase, writing the function, testing it locally, and deploying it took 23 minutes from first line of code to verified production endpoint. On Cloudflare Workers, the same function took 41 minutes — 18 minutes of which was spent configuring wrangler dev to proxy auth requests to the remote Supabase instance because Cloudflare’s local runtime cannot replicate Supabase’s auth integration.

The log and debug experience has improved since early 2026. The dashboard shows invocation counts, error rates, and execution durations per function. Real-time logs — streaming console output from deployed functions — became available on the Pro plan in March 2026 and display console.log, console.error, and unhandled exception traces with stack frames. When I need more detailed tracing, I add structured JSON logging to the function’s console output and parse it through an external observability platform, which is not as integrated as AWS CloudWatch but works without additional infrastructure.

When I Choose Edge Functions Over Workers or Lambda

After five months and twelve production functions, my decision framework is straightforward. I choose Supabase Edge Functions when the function’s primary job is to mediate between the client and the Supabase database or storage layer — CRUD endpoints, auth-protected API routes, webhook handlers that write to the database, and file processing triggers. The pre-wired integration saves development time and eliminates auth plumbing that would require writing and maintaining code on any other platform. For a function that queries the database, validates an auth token, and returns a JSON response, an Edge Function typically takes half to one-third the lines of code of an equivalent Cloudflare Worker or Lambda function.

I choose Cloudflare Workers when the function needs to run in multiple geographic regions for latency reasons and when the function does not primarily interact with Supabase — proxy endpoints, redirect logic, A/B testing middleware, and API gateways that route to multiple backends. Workers deploy to 300-plus locations globally, while Supabase Edge Functions deploy to a smaller set of regions that maps to Supabase’s infrastructure footprint. For a function that needs to respond in under 50 milliseconds from anywhere in the world, Workers’ physical footprint is the deciding factor.

I choose AWS Lambda when the function needs to run for more than 10 seconds, use more than 256 MB of memory, or access AWS services that are not available through Supabase — SQS, DynamoDB, EventBridge, or Step Functions. Lambda’s 15-minute timeout and 10 GB memory ceiling make it the only option for batch processing, report generation, and CPU-intensive workloads. I use Lambda as the complement to Edge Functions, not as a replacement — Lambda handles the heavy lifting, and Edge Functions handle the request-response layer.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

MCP SEP-2468: RFC 9207 Iss Parameter for OAuth Mix-Up Defense

What: MCP SEP-2468 aligns the MCP authorization flow with RFC 9207: authorization servers can advertise iss support and include the iss parameter on their responses; clients are required to validate that iss byte-for-byte against the issuer they had originally recorded for the flow.

Why: An MCP host often trusts more than one identity provider — corporate SSO plus a partner IdP plus a developer IdP for local testing — and without a signal naming the AS, an attacker who controls (or holds a valid registration at) any trusted IdP can mix authorization codes between servers and trick the client into spending an attacker code at a legitimate token endpoint.

vs prior: The previous flow had no issuer field on the authorization response, so clients had to guess from session state — the structural gap the OAuth mix-up attack family exploits, and the one capability scoping cannot close because the attacker is abusing the client’s uncertainty about which AS replied, not abusing a scope.

Think of it as

A doorman who matches your wrist stamp to the guest list.

                       AUTH RESPONSE
                            │
              ┌─────────────┴─────────────┐
              │                           │
      ┌───────▼────────┐         ┌────────▼───────┐
      │   stamp: A     │         │   stamp: B     │
      │ (recorded: A)  │         │ (recorded: A)  │
      └───────┬────────┘         └────────┬───────┘
              │                           │
       string-equal OK             string-equal FAIL
       to recorded issuer          to recorded issuer
              │                           │
              ▼                           ▼
        ✓ proceed to               ✗ reject response
          token exchange              mix-up blocked
  • MCP client = the doorman at the door
  • recorded issuer = the one venue name on the guest list
  • authorization response = a guest arriving
  • iss parameter = the venue name printed on the guest’s wrist stamp
  • string-equal check = the doorman reading the stamp letter by letter
  • OAuth mix-up attack = someone arriving with a stamp from the bar next door

Quick glossary

MCP — The Model Context Protocol — an open protocol for connecting LLM hosts to external tool servers. The host runs the model and the agent loop; servers expose tools, resources, and prompts over JSON-RPC. Authorization across hosts and servers is increasingly standardized on OAuth 2.0, which is what SEP-2468 hardens.

SEP — Specification Enhancement Proposal — MCP’s RFC-style change document. SEP-2468 is the proposal that brings RFC 9207’s iss parameter into MCP authorization. Companion proposals include SEP-2663 (async task handles) and SEP-2577 (feature deprecations).

AS (Authorization Server) — The OAuth role that authenticates the user and issues the authorization code and the access token. In MCP, your corporate SSO, a partner identity provider, and a developer-local IdP can all be ASes the client trusts at the same time.

iss parameter (RFC 9207) — A URL string the AS includes in its authorization responses when it advertises RFC 9207 support in its metadata, identifying which AS produced the response. Defined in RFC 9207. Clients are required to compare it byte-for-byte (per RFC 3986 §6.2.1 simple string comparison) against the issuer they had recorded when they started the flow; when an AS does not advertise iss support, clients may apply local policy.

OAuth mix-up attack — A documented attack family where a client that trusts multiple ASes is tricked into sending one AS’s authorization code or token to a different AS. The result is that the client believes it is talking to one IdP while it has actually authenticated to another — a privilege confusion the rest of the agent stack has no signal to detect.

Capability scoping — The defense pattern of granting tools the minimum capability they need (e.g. read-only file access for a summarizer). Capability scoping is a complementary defense — it limits the blast radius of any compromised tool but does NOT tell you which AS issued a given auth response.

Structural defense — A defense that removes a confusion at the protocol layer rather than relying on policy. The iss parameter is structural: the client cannot accidentally validate against the wrong issuer because the response itself names the AS that produced it.

The news. On May 17, 2026, MCP SEP-2468 was merged into the Model Context Protocol specification — proposed March 25, accepted May 5. The change recommends that authorization servers advertise iss support in their metadata and include the parameter in their authorization responses, and requires clients to validate it against the recorded issuer using simple string comparison per RFC 3986 §6.2.1. The motivation, taken directly from RFC 9207, is to close the long-standing OAuth mix-up attack in clients that trust multiple identity providers.

Picture the doorman. He has one name on his guest list — say, idp-a.example — and tonight that is the only venue whose guests he is letting in. A guest walks up. Without a wrist stamp, the doorman has no way to know which venue’s list this person came from. Maybe it really is the right venue. Maybe it’s someone from the bar next door who is hoping you don’t check. The doorman has to guess. That is the OAuth mix-up scenario before SEP-2468: the client kicks off an authorization flow with one AS, an authorization response comes back, and the only signal the client has is its own session state — which the attacker is actively trying to confuse.

The fast path is each guest gets a stamp, printed with the venue name. The doorman doesn’t have to remember faces. He reads the stamp, compares it letter by letter to the name on his list, and decides on the spot. The stamp is the iss parameter. It is not the token. It is metadata about which AS produced the response. The client compares response.iss against the issuer it had recorded when it started the flow, character for character. Match → continue to the token exchange. Mismatch → reject the response, no matter how convincing the rest of the payload looks.

The catch — and this is what makes the structural defense load-bearing — is that capability scoping and other content-layer checks can’t fill in for the issuer check. Suppose every tool in your agent is read-only and every scope is minimal. An attacker still wins the mix-up if the client takes an authorization code minted by idp-b.evil and exchanges it at idp-a.example‘s token endpoint, because the eventual access token will be for the legitimate IdP — minted from the wrong user’s session. The compromised data does not flow through a tool’s parameters; it flows through the client’s confusion about which IdP authenticated the user. That is a layer above the tool attack surface the rest of the agent’s guardrails are built to police, which is exactly why the Layered Guardrails module sequences structural defenses before policy.

Consider a concrete illustration. Picture a workplace MCP host that trusts two authorization servers — idp-a.example (corporate SSO) and idp-b.example (a partner) — and 1,000 agent sessions per hour each completing one OAuth flow. Imagine an attacker who can substitute responses on 0.1% of flows. Pre-iss, the client has no signal to reject substituted responses, so the substitution rate equals the attack rate: 1,000 × 0.001 = 1 mix-up per hour, or ~24 per day (illustrative). Post-iss, a substituted response would carry iss=https://idp-b.example while the client recorded https://idp-a.example. The string-equality check fails for every substituted response that includes iss, so the post-defense rate drops to ~0 mix-ups per day (illustrative) — independent of how clever the substitution gets, with the residual restricted to ASes that do not advertise iss support and where client policy falls back to local rules.

Where the change earns its keep

The shape of what SEP-2468 actually adds is small. The table below contrasts the legacy flow with the SEP-2468 flow.

Aspect Legacy MCP auth flow SEP-2468 (RFC 9207)
Issuer field on auth response None iss required when AS metadata advertises support
How client knows which AS replied Inference from session state Server tells you, via iss
OAuth mix-up defense Out of scope — client policy String-equal response.iss against recorded issuer
String comparison rule N/A RFC 3986 §6.2.1 simple string comparison
Server discovery N/A AS advertises iss support in its metadata; clients may apply local policy on servers that don’t
Layer of defense Capability scoping only Structural (issuer) plus capability scoping

The defense slots in neatly next to the existing guardrail stack. Capability scoping limits the blast radius of any single misuse. The Lethal Trifecta framing limits how private data, untrusted content, and exfiltration vectors are allowed to combine. SEP-2468 sits one layer below both — it ensures the client has not been silently rerouted to a different IdP before any of those checks even runs. None of the layers replaces the others; the value is in stacking them.

There is also a related lesson worth knowing. Mix-up is part of a broader pattern where protocol metadata gets repurposed as protocol contract. RFC 9207’s iss is a small, fixed-shape addition with a single rule: equal or not. That makes it both easy to implement correctly and hard to mis-implement, which is exactly the property a defense-in-depth layer needs — every layer that requires nuanced configuration becomes its own attack surface.

The boundary of what SEP-2468 changes is not “how OAuth works.” Existing single-IdP flows are unaffected; the comparison is trivially https://idp-a.example == https://idp-a.example and continues. The boundary is “what happens when a client trusts more than one AS at the same time” — the multi-IdP topology that production MCP hosts are increasingly running into. For those, the iss check is the difference between a structural defense and a hopeful one.

Related explainers

FAQ

What is the RFC 9207 iss parameter and what does MCP SEP-2468 do with it?

RFC 9207 defines an iss parameter — a URL identifying the authorization server that produced an authorization response. MCP SEP-2468 adopts this: authorization servers can advertise iss support in their metadata and include it in their authorization responses, and clients must compare it byte-for-byte (per RFC 3986 §6.2.1 simple string comparison) against the issuer recorded when the flow started. The check rejects any response whose iss does not match; for ASes that don’t advertise iss support, the client may apply local policy.

Why does an MCP client need this if it already does capability scoping?

Capability scoping limits the blast radius of any compromised tool but does not tell the client which authorization server replied. OAuth mix-up attacks abuse exactly that confusion: an attacker controlling (or registered at) one of multiple trusted IdPs swaps responses between authorization servers, so the client believes it is talking to one IdP while it has authenticated against another. SEP-2468 is a layer below scoping — a structural defense that removes the confusion at the protocol level, with no policy tuning required.

How does SEP-2468 compare to the other MCP SEPs landing this month?

SEP-2468 hardens OAuth, SEP-2663 lands async task handles for long-running tool calls, and SEP-2577 starts the deprecation timer on three legacy features. They sit at different layers of the protocol — authorization, tool execution, and feature lifecycle — and the security model assumed by SEP-2663’s task handles and SEP-2577’s migration windows leans on the same OAuth foundation SEP-2468 is reinforcing.

Originally posted on Learn AI Visually.

Stop paying for idle GPUs in your CI: batching LLM eval jobs

TL;DR: Running LLM evaluations on every PR will burn your GPU budget faster than you can blink. We cut our eval spend by about 60% by batching jobs into windowed runs on shared GPU pools, plus a smarter queue that knows the difference between a “smoke test” eval and a full regression run. Here’s how, and where the trade-offs hurt.

Right, so a few months back I got pulled into a conversation that’s becoming pretty familiar around here. A team had wired up an LLM-based evaluation suite into their CI. Every PR triggered a run against a set of prompts, scored the outputs, and posted results back to the PR. Lovely in theory.

The cloud bill was not lovely.

They were spinning up a g5.xlarge per PR, sometimes three or four in parallel during peak hours, and the GPU sat idle for about 70% of the run because most of the time was spent on cold starts, model loading, and prompt formatting. Classic case of treating GPUs like CPUs.

I reckon a lot of teams are hitting this wall right now. So let’s talk about what actually works.

The problem with “GPU per job”

CI runners are designed for stateless, throwaway compute. That model breaks the second you involve a 7B+ parameter model that takes 30-90 seconds to load into VRAM.

Here’s the rough breakdown of a typical eval job we measured:

Phase Time (avg) GPU utilization
Cold start (instance boot) 45s 0%
Model download from S3 60s 0%
Model load into VRAM 25s ~10%
Actual inference (50 prompts) 40s ~85%
Result upload + teardown 15s 0%

So out of about 3 minutes of billable GPU time, you’re getting 40 seconds of useful work. That’s brutal economics.

Batching: the boring fix that works

The trick isn’t fancy. You stop spinning up a GPU per job and start treating the GPU like a long-lived service that consumes jobs from a queue.

We run a small pool of g5.xlarge instances (usually 2-4 depending on load) that stay warm. Each runner has the model preloaded in VRAM. CI jobs push eval requests to an SQS queue, runners pull from the queue, batch up to N prompts per inference pass, and post results back.

Rough sketch of the runner config:

runner:
  instance_type: g5.xlarge
  pool_size_min: 2
  pool_size_max: 6
  scale_metric: queue_depth
  scale_threshold: 25  # jobs in queue

  model:
    name: llama-3.1-8b-instruct
    preload: true
    keep_warm_seconds: 1800

  batching:
    max_batch_size: 16
    max_wait_ms: 2000

  job_types:
    smoke_eval:
      priority: high
      max_prompts: 10
    full_regression:
      priority: low
      max_prompts: 500
      window: nightly_only

That max_wait_ms is doing the heavy lifting. The runner waits up to 2 seconds to gather a batch before firing inference. For CI, 2 seconds of latency is nothing. For inference throughput, it’s everything.

Routing matters too

Once you’ve got a warm pool, you might as well route different model calls through one place. We have eval suites that hit a mix of self-hosted Llama, Claude via API, and OpenAI. Instead of each CI job authenticating separately and managing keys, we put a gateway in front.

There’s a bunch of options here. LiteLLM is popular, Bifrost (https://github.com/maximhq/bifrost) is another one that does the same kind of multi-provider routing with rate limit handling, and you can roll your own with a thin FastAPI wrapper if you’re feeling keen. The point is you stop scattering API keys across twenty CI configs.

Job classification: don’t run a full eval on every commit

This was the biggest single win, honestly. We split eval jobs into tiers:

  • Smoke evals: 5-10 prompts, run on every PR, catches obvious regressions
  • Standard evals: 50-100 prompts, run on merge to main
  • Full regression: 500+ prompts, run nightly on main

Before this, every PR triggered the full 500-prompt suite because nobody had bothered to think about what they actually needed to know per PR. The answer is “did this change break something obvious?”, not “is this model production-ready?”

Cut our GPU-hours by about 40% just from that change alone, before any of the batching work.

What the numbers looked like

After about three weeks of running the new setup:

Metric Before After
GPU-hours per day 38 14
Avg PR feedback time 4m 20s 1m 50s
Monthly GPU spend (eval only) ~$8,200 ~$3,100
Queue p99 wait time n/a 8s

Faster and cheaper, which is the dream combination you almost never get.

Trade-offs and limitations

Nothing’s free, so here’s what actually hurt:

Cold start on scale-up is still painful. When the queue spikes past what the warm pool can handle, the new runners take 90+ seconds to come online with the model loaded. We mitigated by being more aggressive on the scale_threshold than felt comfortable, which means we’re occasionally paying for idle capacity. You can’t have both.

Batching adds latency variance. A job that arrives just after a batch fires waits the full max_wait_ms. For CI this is fine. For production inference it might not be, so don’t blindly copy this config to your prod inference pipeline.

Pool exhaustion is a real failure mode. If your queue grows faster than you can scale, jobs back up. We had a Friday afternoon where a misconfigured test suite generated 4,000 eval jobs in 10 minutes and the queue depth alert woke me up at 11pm. Add circuit breakers and per-team quotas early, not after the first incident.

Model updates are now an event. When you preload models, swapping versions means a rolling restart of the pool. We do this during low-traffic windows but it’s added operational overhead that didn’t exist with the per-job model.

Further Reading

No worries if your setup looks different. The general shape holds: warm pools, batched jobs, classified workloads. Apply where it fits.