Two npm supply chain attacks hit the same week. One was predictable. One wasn’t. That’s the point.

May 2026 gave us two back-to-back supply chain attacks on npm. Same week. Completely different mechanics. And that tells you more about the state of supply chain security than any whitepaper.

May 11: TanStack — 42 packages, 84 malicious versions in 6 minutes. @tanstack/react-router alone gets 16.9 million weekly downloads.

May 14: node-ipc — 3 malicious versions. 730K weekly downloads. Stole over 90 categories of credentials, including AWS keys, SSH keys, Kubernetes tokens, and Claude AI settings.

I ran both through getcommit.dev — a behavioral scoring tool I built that measures structural risk signals in npm, PyPI, Cargo, and Go packages.

TanStack scored 91. node-ipc scored 69 with a WARN flag.

The 69 was there before the attack.

What happened at node-ipc

node-ipc is a 12-year-old inter-process communication library. One npm publisher. 35 GitHub contributors who can’t publish to npm. Last legitimate release: August 2024 — 21 months of silence.

On May 14, someone published three malicious versions simultaneously across two major version lines (9.x and 12.x). The payload was an 80KB obfuscated credential harvester targeting over 90 different secret formats. The compromised npm account had been dormant long enough to steal without anyone noticing.

The behavioral data before the attack:

A sole publisher. A dormant release cycle. No provenance. Every signal was pointing at the risk. This is the pattern behind the LiteLLM attack (March 2026) and the axios incident (March 30, 2026): steal the credentials of a single person who hasn’t published in months, and 730,000 weekly consumers get the payload.

What happened at TanStack

TanStack was different. Five npm publishers. Active development — the last publish was 3 days before the attack. Score: 91. HEALTHY by every behavioral metric.

The attacker didn’t steal anyone’s npm credentials. They exploited a chain of three vulnerabilities in TanStack’s GitHub Actions setup:

1. pull_request_target in a workflow, which lets fork code run with base repo permissions
2. GitHub Actions cache poisoning across the fork → base trust boundary
3. Memory extraction of the OIDC token from the GitHub Actions runner process (reading /proc at runtime)

The malware published 84 versions across 42 packages in six minutes. It passed SLSA provenance checks. It carried valid signed certificates. Every automated security tool looking at cryptographic proof of origin said “this is legitimate.”

OpenAI published a response because TanStack was in their dependency tree. Detection took 20 minutes. Cleanup took hours. But the attack bypassed the exact mechanism — provenance attestation — that was supposed to make CI/CD-origin attacks impossible.

What the scores actually tell you

Two attacks. Two completely different risk profiles:

Behavioral signals caught the node-ipc pattern. They didn’t catch TanStack’s. SLSA provenance was supposed to catch TanStack’s. It didn’t.

No single tool catches both. That’s not a failure — it’s the reality. Different attacks exploit different trust boundaries.

Which pattern is more common

The node-ipc pattern is far more common than the TanStack pattern. The GitHub Actions cache-poisoning chain is sophisticated — it required chaining three separate vulnerabilities. Stealing a dormant npm account’s credentials requires buying them on a dark web marketplace.

There are 26 npm packages with over 10 million weekly downloads and a single npm publisher. Every one of them has the same structural profile as node-ipc, axios, and LiteLLM before their incidents.

minimatch    — 610M/wk, 1 publisher  ⚑ CRITICAL
chalk        — 436M/wk, 1 publisher  ⚑ CRITICAL
glob         — 355M/wk, 1 publisher  ⚑ CRITICAL
cross-spawn  — 168M/wk, 1 publisher  ⚑ CRITICAL
zod          — 145M/wk, 1 publisher  ⚑ CRITICAL

These won’t appear in your package.json. They’re in your lock file — transitive dependencies you’ve never audited, installed on every npm install.

Check your own project

npx proof-of-commitment --file package-lock.json

This scans your full dependency tree — direct and transitive — and flags the structural risks that npm audit doesn’t look at.

It won’t predict the next TanStack. But it’ll surface every node-ipc-shaped package in your tree. And right now, that’s the pattern that keeps repeating.

Try it on any package: getcommit.dev/npm/node-ipc

getcommit.dev — behavioral supply chain scoring for npm, PyPI, Cargo, and Go. Open source on GitHub.

يشعر وكيل البرمجة عبر واجهة سطر الأوامر (CLI) بالحرية إلى أن تصل الفاتورة. تشغّل Claude Code أو Codex داخل مستودع، تطلب إعادة هيكلة وحدة، وبعد عشر دقائق يكون الوكيل قد قرأ عشرات الملفات، شغّل الاختبارات أكثر من مرة، وأرسل سياقًا لم تكن بحاجة إليه أصلًا. على مستوى فريق من عدة مهندسين، يتحول هذا إلى تكلفة توكنات واضحة. الحل ليس بالضرورة تغيير النموذج أو قبول نتائج أسوأ، بل تقليل ما يصل إلى النموذج من البداية.

جرّب Apidog اليوم

باختصار

لتقليل تكلفة توكنات وكلاء البرمجة:

• قلّص السياق قبل إرساله إلى النموذج.
• حدّد الملفات والدلائل التي يعمل عليها الوكيل.
• اجعل ملفات الذاكرة مثل CLAUDE.md قصيرة ومستقرة.
• استخدم /compact أو /clear بين المهام.
• فعّل prompt caching للبادئات المستقرة.
• وجّه المهام البسيطة إلى نموذج أرخص.
• قلّل إخراج الأدوات مثل الاختبارات وnpm install.
• قِس التكلفة لكل تشغيل بدل الاعتماد على الفاتورة الشهرية فقط.

مقدمة

تظهر المشكلة غالبًا بطريقتين:

1. تصل إلى حد أسبوعي أو حد جلسة في منتصف العمل.
2. تصل فاتورة API ويسأل الفريق لماذا أصبح “مساعد الذكاء الاصطناعي” بندًا مكلفًا.

السبب واحد: وكلاء CLI يستهلكون التوكنات افتراضيًا. يقرؤون ملفات كاملة بدل عشرة أسطر، يعيدون إرسال سجل المحادثة في كل دور، يضعون مخرجات الأوامر الخام داخل السياق، ويرسلون نفس موجه النظام وخريطة المشروع مرارًا.

إعادة هيكلة تحتاج فعليًا إلى 2,000 توكن من الكود لا يجب أن تستخدم 180,000 توكن من السياق. الفرق بين الرقمين هو مساحة التوفير.

في هذا الدليل سنحوّل تقليل التوكنات إلى خطوات عملية:

• تنظيف السياق وملفات الذاكرة.
• استخدام prompt caching.
• توجيه النماذج حسب نوع المهمة.
• تقليل إخراج الأدوات والاسترجاع.
• قياس التكلفة لكل تشغيل.

الأمثلة تفترض Claude Code وCodex، لكن الفكرة تنطبق على أي وكيل CLI يتصل بواجهة API تُحاسب بالتوكنات.

💡 إذا كان وكلاؤك يتعاملون مع APIs، فإن تصميمها ومحاكاتها واختبارها في Apidog قبل توجيه الوكيل إليها يقلل التجربة والخطأ المكلفة. يعمل الوكيل ضد عقد API واضح ومستقر بدل نقطة نهاية حية تفاجئه بأخطاء متغيرة.

إلى أين تذهب التوكنات عند تشغيل وكيل CLI؟

كل دور في جلسة الوكيل يرسل حمولة إدخال إلى النموذج ويحصل على حمولة إخراج. أنت تدفع مقابل الاثنين.

عادةً تكون توكنات الإخراج أغلى من الإدخال، لكن الإدخال هو الذي يتضخم بسرعة.

حمولة الإدخال تتكوّن غالبًا من:

• موجه النظام وتعريفات الأدوات

  تعليمات الوكيل ومخططات JSON الخاصة بالأدوات. قد تكون من 5,000 إلى 15,000 توكن وتُعاد في كل دور.

• ملفات الذاكرة والمشروع

  مثل CLAUDE.md أو تعليمات المشروع الدائمة. تُحمّل حتى عندما لا تكون مرتبطة بالمهمة.

• سجل المحادثة

  كل رسالة مستخدم، رد نموذج، استدعاء أداة، ونتيجة أداة يعاد إرسالها. هذا عادةً أكبر مصدر للهدر في الجلسات الطويلة.

• محتوى الملفات المقروءة

  قراءة ملف من 1,200 سطر قد تعني 12,000 إلى 18,000 توكن.

• إخراج الأدوات

  سجلات الاختبارات، npm install، git diff، stack traces، وملفات القفل.

النقطة الأهم: سجل المحادثة يُعاد تشغيله في كل دور. جلسة من 30 دورًا لا تكلف 30 مرة فقط، بل تكلف مجموع بادئة تكبر مع الوقت. لذلك الجلسات الطويلة غير المضغوطة هي من أغلى أنماط الاستخدام.

لمزيد من التفاصيل حول حدود الجلسات وحساب التوكنات، راجع: كيفية إعادة تعيين نافذة توكنات كلود كود.

1. نظافة السياق وملفات الذاكرة

أرخص توكن هو التوكن الذي لا ترسله أصلًا.

حدّد نطاق العمل قبل تشغيل الوكيل

لا تبدأ من جذر المستودع بسؤال عام مثل:

claude "أعد هيكلة منطق الفوترة"

هذا يدفع الوكيل للبحث في المستودع كاملًا.

اكتب مهمة محددة:

claude "أعد هيكلة منطق إعادة المحاولة ليستخدم التراجع الأسي.
اعمل فقط على:
- src/payments/retry.ts
- src/payments/retry.test.ts"

إذا كنت لا تعرف الملف بدقة، حدّد الدليل بدل الجذر:

claude "ابحث داخل src/payments فقط عن منطق إعادة المحاولة، ثم اقترح تعديلًا محدودًا"

اجعل CLAUDE.md قصيرًا

ملف الذاكرة مثل CLAUDE.md يُحمّل في كل دور. إذا تحول إلى wiki داخلية، ستدفع ثمنه مرارًا.

افحص حجمه تقريبيًا:

wc -c CLAUDE.md | awk '{print "≈", int($1/4), "توكن لكل دور"}'

اجعل الملف يحتوي فقط على:

• أوامر البناء والاختبار الأساسية.
• قواعد الكود الصارمة.
• بنية المشروع المختصرة.
• روابط أو مسارات للوثائق الطويلة بدل نسخها داخله.

مثال جيد:

# تعليمات الوكيل

## أوامر
- npm test --silent
- npm run lint

## قواعد
- لا تعدّل ملفات dist/
- لا تغيّر API عام بدون تحديث الاختبارات
- استخدم TypeScript strict types

## وثائق إضافية
- API contracts: docs/api/
- Payment flow: docs/payments.md

اضغط أو امسح الجلسات الطويلة

عندما تنتهي مهمة وتبدأ أخرى، لا تكمل في نفس السياق.

في Claude Code:

/compact

يستبدل السجل الطويل بملخص قصير.

أو:

/clear

يبدأ جلسة جديدة بدون حمل السياق القديم.

قاعدة عملية:

• مهمة واحدة = جلسة واحدة.
• بعد الانتهاء: /compact إذا ستكمل على نفس الهدف.
• استخدم /clear إذا ستنتقل إلى هدف مختلف.

راجع أيضًا: سير عمل كلود كود.

استخدم ملف تجاهل

امنع الوكيل من قراءة ما لا يحتاجه:

node_modules/
dist/
build/
coverage/
.next/
.cache/
*.lock

إذا كان وكيلك يدعم ملف تجاهل مخصص، أضف أيضًا:

docs/generated/
tmp/
vendor/

الهدف: لا تجعل الوكيل يرى ملفات ضخمة لن يعدّلها.

2. Prompt caching: لا تدفع السعر الكامل لنفس البادئة

Prompt caching يسمح للمزود بتخزين بادئة ثابتة من الطلب، مثل:

• موجه النظام.
• تعريفات الأدوات.
• اتفاقيات المشروع.
• التعليمات المستقرة.

ثم تُقرأ هذه البادئة لاحقًا بخصم كبير بدل إعادة معالجتها بالكامل.

وفقًا لوثائق Anthropic، قراءة cache قد تكلف تقريبًا 10% من تكلفة الإدخال الأساسي، بينما كتابة cache تكلف أكثر قليلًا من الإدخال العادي. لذلك يصبح مفيدًا بسرعة عندما تعيد استخدام نفس البادئة.

القاعدة

ضع المحتوى المستقر أولًا، والمتغير آخرًا.

الترتيب العام:

tools → system → messages

لا تضع طابعًا زمنيًا أو بيانات متغيرة قبل نقطة التخزين المؤقت.

مثال Python

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=2048,
    system=[
        {
            "type": "text",
            "text": SYSTEM_PROMPT + REPO_CONVENTIONS,
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[
        {
            "role": "user",
            "content": user_task,
        }
    ],
)

u = response.usage

print("cache write:", u.cache_creation_input_tokens)
print("cache read :", u.cache_read_input_tokens)
print("fresh input:", u.input_tokens)

تحقق من أن cache يعمل

إذا كان cache_read_input_tokens يساوي صفرًا دائمًا، فغالبًا لديك أحد هذه الأخطاء:

• تغيّر البادئة في كل طلب.
• أضفت timestamp داخل موجه النظام.
• وضعت محتوى الملف المتغير قبل نقطة cache.
• البادئة أصغر من الحد الأدنى القابل للتخزين.

اجعل موجه النظام واتفاقيات المشروع ثابتة byte-for-byte قدر الإمكان.

للاستفادة من التوجيه مع Codex، راجع: تشغيل GPT-5.5 مجانًا عبر Codex.

3. توجيه النموذج: استخدم نموذجًا رخيصًا للمهام الرخيصة

ليست كل مهمة تحتاج النموذج الأقوى.

مهام غالبًا يمكن تنفيذها بنموذج أرخص:

• كتابة commit message.
• تلخيص git diff.
• إنشاء changelog.
• شرح خطأ lint.
• إعادة تسمية بسيطة.
• كتابة اختبار نمطي.

مثال:

claude --model haiku "اكتب رسالة commit بنمط conventional commit للتغييرات المرحلية"

claude --model sonnet "أعد تصميم طبقة التخزين المؤقت لخدمة المدفوعات"

اجعل الافتراضي رخيصًا، وصعّد عند الحاجة:

alias claude-cheap='claude --model haiku'
alias claude-strong='claude --model sonnet'

ثم استخدم:

claude-cheap "لخص هذا diff في 5 نقاط"
claude-strong "حلل سبب race condition في طبقة الدفع"

استخدم وكلاء فرعيين بسياق صغير

إذا كان نظامك يدعم sub-agents، اجعل الوكيل الفرعي يعمل على مهمة ضيقة بنموذج رخيص، ثم يعيد ملخصًا قصيرًا للوكيل الرئيسي.

بدلًا من أن يرى النموذج المكلف 30,000 توكن من البحث، يرى 700 توكن من الخلاصة.

مثال نمط عمل:

الوكيل الرئيسي:
- يحدد السؤال.
- يطلب من وكيل فرعي رخيص البحث في src/payments فقط.
- يستقبل ملخصًا قصيرًا.
- يستخدم النموذج الأقوى لاتخاذ القرار النهائي.

راجع: أمر الهدف عبر وكلاء كوديكس وكلود كود المستقلين.

إذا كنت تعمل ضمن حدود أسبوعية، فهذا لا يوفر المال فقط، بل يطيل عمر الحصة. راجع أيضًا: زيادة الحد الأسبوعي لكلود كود الأخيرة.

4. قلّل إخراج الأدوات والاسترجاع

كل سطر تطبعه أداة يمكن أن يدخل السياق، ثم يُعاد في الأدوار اللاحقة.

اجعل الأوامر صامتة

بدل:

npm test

استخدم:

npm test --silent -- --reporter=dot

بدل:

npm install

استخدم:

npm install --silent --no-audit --no-fund

فلتر المخرجات قبل أن يراها الوكيل

pytest -q 2>&1 | tail -n 30

git diff --stat

npm test 2>&1 | grep -E "(FAIL|✗|Error)" | head -n 20

بدل إرجاع سجل اختبار كامل، أعد آخر 30 سطرًا فقط أو الأسطر التي تحتوي على الخطأ.

استخدم قراءة مستهدفة بدل قراءة ملف كامل

اطلب من الوكيل صراحة:

ابحث عن الدالة التي تتعامل مع retry.
اقرأ فقط 80 سطرًا حولها.
لا تقرأ الملف كاملًا إلا إذا لم يكفِ ذلك.

أو استخدم أنت أدوات CLI:

grep -n "function retry" src/payments/retry.ts
sed -n '40,130p' src/payments/retry.ts

فرق التكلفة قد يكون كبيرًا: قراءة 100 سطر بدل 1,500 سطر.

قيّد RAG والاسترجاع

إذا كان الوكيل يسترجع من docs أو code search، لا تسمح له بجلب عشرات القطع الكبيرة.

إعداد عملي:

max_chunks: 8
chunk_size: 300 tokens

أفضل من:

max_chunks: 50
chunk_size: 800 tokens

أنت تدفع مقابل كل قطعة مسترجعة حتى لو لم يستخدمها النموذج.

5. قِس تكلفة التشغيل

لا تعتمد على الانطباع. سجّل التكلفة لكل نوع مهمة.

إذا كنت تستدعي API مباشرة، استخدم كائن usage:

u = response.usage

INPUT_RATE  = 3.00 / 1_000_000
OUTPUT_RATE = 15.00 / 1_000_000
CACHE_READ  = 0.30 / 1_000_000
CACHE_WRITE = 3.75 / 1_000_000

cost = (
    u.input_tokens * INPUT_RATE +
    u.output_tokens * OUTPUT_RATE +
    u.cache_read_input_tokens * CACHE_READ +
    u.cache_creation_input_tokens * CACHE_WRITE
)

print(
    f"تكلفة التشغيل ≈ ${cost:.4f} "
    f"(in={u.input_tokens} out={u.output_tokens} "
    f"cache_read={u.cache_read_input_tokens})"
)

الأسعار هنا توضيحية. استخدم أسعار النموذج الحالية من المزود.

إذا كنت تستخدم CLI فقط:

claude /cost

أو افصل مفاتيح API حسب المشروع:

API key لكل وكيل أو مشروع
→ لوحة تحكم المزود تعرض الإنفاق بشكل قابل للتتبع

أو غلّف تشغيل الوكيل بسكربت يسجل المهمة:

#!/usr/bin/env bash

TASK="$1"
START=$(date -Iseconds)

claude "$TASK"

END=$(date -Iseconds)

echo "$START,$END,$TASK" >> agent-runs.csv

ثم أضف التكلفة المبلغ عنها يدويًا أو من API لاحقًا.

قِس قبل وبعد

اختر مهامًا تمثيلية:

• إعادة هيكلة يومية.
• مراجعة PR.
• توليد اختبار.
• تلخيص diff.

وسجّل:

task,model,input_tokens,output_tokens,cache_read,cost

بعد تطبيق أي تحسين، يجب أن ترى الرقم يتحرك. إذا لم يتحرك، فالتحسين لا يعالج مصدر الهدر الحقيقي.

مقارنة التكتيكات

الأرقام تقريبية وتعتمد على شكل استخدامك، لكنها تساعدك على ترتيب الأولويات.

قائمة تنفيذ سريعة

ابدأ بهذه الخطوات اليوم:

# 1. افحص حجم ملف الذاكرة
wc -c CLAUDE.md | awk '{print "≈", int($1/4), "توكن لكل دور"}'

# 2. استخدم اختبارات هادئة
npm test --silent -- --reporter=dot

# 3. قلل diff
git diff --stat

# 4. امسح الجلسة عند تغيير المهمة
/clear

# 5. اضغط الجلسة الطويلة
/compact

ثم طبّق على مستوى الفريق:

• راجع CLAUDE.md مرة أسبوعيًا.
• أضف ملفات التجاهل.
• اجعل النموذج الأرخص هو الافتراضي للمهام الميكانيكية.
• فعّل prompt caching إن كنت تتحكم في استدعاءات API.
• سجّل تكلفة المهام المتكررة.

الخاتمة

تكلفة توكنات وكلاء البرمجة ليست مشكلة حتمية. معظم الهدر يأتي من سياق زائد، جلسات طويلة، مخرجات أدوات صاخبة، ونموذج أقوى من المطلوب.

ابدأ بالأشياء منخفضة الجهد: تحديد النطاق، إخراج هادئ، ملف ذاكرة صغير، و/compact بين المهام. بعد ذلك أضف prompt caching وتوجيه النماذج. النتيجة عادةً تكون فاتورة أقل بدون التضحية بجودة العمل.

The prompt regression that cost me the worst Sunday of 2025 was twelve characters long.

I had been tuning the system prompt of an LLM feature for about a month. The feature was a structured-output endpoint that took a free-form user message and returned a JSON object with five fields. It worked. Users liked it. Conversion was up. On a Friday afternoon I added what I thought was a small clarification to the prompt: a sentence telling the model to “prefer concise wording” in one of the fields. I deployed. I went into the weekend feeling good about my week.

By Sunday morning, support had three tickets and a Slack DM from our biggest customer. The endpoint was returning malformed JSON on roughly 14% of requests. Not always. Just often enough that retries hid it from our dashboards. The “prefer concise wording” sentence had pushed the model into a mode where it sometimes dropped a comma in the JSON output, because conciseness and valid JSON syntax were apparently in tension in a way I had not anticipated.

The fix was rolling back the prompt. The fix took ninety seconds. The problem was that I had no idea which version of the prompt was running. I had edited the prompt in three places that month: the source code, a draft in Notion, a quick test in the model playground. The deployed version was a copy-paste of one of them. I had no diff. I had no test for “does this prompt still return valid JSON for our golden set of inputs.” I had no rollback button. The Sunday was spent reconstructing what I had typed, retesting it by hand against a list of fifty inputs I dug out of the logs, and deploying the rebuilt prompt.

That weekend is the reason I now treat prompts as code. Not “code-adjacent.” Not “prompts that live in a file in the repo, sort of.” Actual versioned, tested, CI-gated source code with the same review rigor as anything else we ship. The argument for this is not aesthetic. The argument is that you cannot debug what you cannot see, and you cannot roll back what you do not version. Everything else flows from that.

This post is what the prompt-as-code workflow actually looks like, what tools sit at the layer above your editor, and the migration path that gets you from “the prompt is somewhere in a Notion doc” to “the prompt has a CI job and a changelog.”

What Prompts As Code Actually Means

The phrase “prompts as code” gets thrown around like every other piece of AI vocabulary. Let me be specific.

A prompt-as-code workflow has five properties. If any of them are missing, you are doing something else.

Every prompt lives in a tracked file or a tracked registry. Not in a Notion page. Not pasted into a Cloudflare worker. Not in a “system message” field of a hosted dashboard that nobody on the team has the URL for. The prompt has a path. The path is in a repo or in a registry that the repo references by ID. Either is fine. Both are fine. What is not fine is “I think the prompt is in this Slack thread.”

Every change to the prompt is reviewed. Same workflow as a code change. A PR, a review, a merge. The reviewer is allowed to push back. The reviewer is qualified to push back, because the prompt is presented in a diff alongside the test results, not as a 200-line wall of text in a doc with no context.

Every prompt has a test. At minimum, a smoke test that runs the prompt against a small set of golden inputs and validates the output shape. Ideally, an eval that scores quality across a larger set. The test runs in CI before merge. The PR cannot land if the test fails. I went deep on the eval question in the AI evals for solo developers piece; the short version is that even a fifty-row eval set is dramatically better than nothing.

Every prompt is versioned. Not just “the file is in git.” The runtime knows which version is deployed. The logs include the version. When a regression hits, you can look at a request, read the version, and check exactly what prompt was used. This is the single capability that turns prompt debugging from archeology into engineering.

Every prompt is rollbackable. A bad deploy is a one-button revert. Not “redeploy the application with the old prompt re-pasted.” A real revert that flips the active version back to the previous one in seconds, without a full app redeploy, because prompts change more often than code and tying every prompt change to a full app deploy is the friction that turns engineers into copy-paste cowboys.

You can get all five with tools that exist today. The question is just whether your team has actually built the workflow, or whether you are still in the world where the prompt is a string literal in src/services/ai.ts and nobody knows where the canonical version lives.

The Two Architectures That Work

There are two workable architectures for prompts as code in 2026. They are real alternatives, not flavors of the same thing.

Architecture one is prompts in the repo. The prompt files live next to the code that uses them. Format is your call: .prompt files, .md files, TypeScript template literals, YAML. The build system bundles them. The runtime imports them. Version is the git SHA. Rollback is a revert and a redeploy. Tests are colocated with the prompt file.

The strengths are that this is simple, the diff history is obvious, and the same review workflow as code already exists in your team. The weaknesses are that you cannot change a prompt without a full app deploy, the prompt is locked to the application’s release cycle, and non-engineers cannot touch the prompts without going through engineering.

Architecture two is prompts in a registry. The prompts live in a managed service. Langfuse, PromptLayer, Maxim, LangWatch, Helicone, Braintrust. The application fetches the active version at runtime by ID, with optional caching. Updates happen in the registry’s UI or via API. The version is a registry version, not a git SHA. Rollback is a one-click change of the active version. Tests run against the registry, either in CI or in the registry’s own eval system.

The strengths are that prompt updates are decoupled from code deploys, non-engineers can edit prompts safely, and the registry usually includes built-in eval and observability. The weaknesses are that you have introduced a runtime dependency, the prompt history lives outside git so your “source of truth” is now split, and you have to trust the registry to be up when your app is up.

I have shipped both. The choice depends on team shape.

If you are a solo founder or a two-person team, the repo architecture wins. You do not have non-engineers editing prompts. You do not have enough prompts to justify a registry. The simplicity of “the prompt is a file in the repo” is worth more than the flexibility of a registry.

If you are a five-person team with a product manager who wants to A/B test prompt copy, or you have prompts that change daily, or you have prompts you want to roll out to 10% of traffic before everyone, the registry architecture wins. The decoupling pays for itself in week one.

The two are not mutually exclusive. Some teams use registries for the prompts that change often (marketing-style outputs, customer-facing tone) and keep the structural prompts (JSON schemas, agent instructions) in the repo. That hybrid is fine. The rule is: every prompt lives in exactly one place. Never two. Never “it is in the repo but also in the registry and they sometimes disagree.”

The CI Job That Actually Catches Things

The single most valuable artifact in a prompt-as-code workflow is the CI job. Not the registry, not the file format, not the eval framework. The job that runs on every PR and fails the PR if the prompt regression rate exceeds a threshold.

Here is what mine does, in practice. The same shape works for repo-based and registry-based architectures.

The job loads the prompt under review. It loads the eval set, which is a JSON file of input-output pairs that I maintain by hand based on real production examples. The set is around 80 rows for the smaller prompts, 300 rows for the bigger ones. Each row has an input (whatever the prompt is meant to take) and an expected output, or a set of validators the output must pass.

The job runs the prompt against each input. The model behind it is the production model. The runs happen in parallel with a small concurrency limit to avoid rate limits. The cost of a full eval run is between $0.50 and $5 depending on the prompt size and model, which is cheap enough to do on every PR.

The job collects the outputs. It runs the validators. The validators are a mix of structural checks (is this valid JSON, does it have these fields, are the field types correct) and quality checks (is the score from a judge model above this threshold, does this output match the expected pattern). I lean structural where I can, because structural validators are deterministic and judge-based validators are not.

The job emits a pass-rate. The PR fails if the pass-rate drops more than a threshold, currently 3%, from the previous prompt version. The PR also fails if any of a small set of “must-pass” cases regress. Those are the cases I learned about the hard way: the JSON-validity case from the Sunday incident, the SQL-injection case from another bad weekend, the tone-of-voice case from when a customer pointed out that our model had started sounding like a chatbot.

The full CI job is around 200 lines of code. It is the cheapest insurance I have ever bought. In the eight months it has been running, it has caught six regressions that would have shipped without it. Two of them would have been customer-visible incidents. The others would have been silent quality drops that I would have noticed weeks later via support tickets.

If you do not have this job, write it before you do anything else. The exact tool does not matter. The discipline of “no prompt change ships without an eval pass” is the discipline.

The Prompt Diff That Reviewers Actually Read

The other piece nobody talks about is what the PR review actually looks like.

A prompt diff is hard to read. The default GitHub diff shows you the changed lines, but the semantic difference between two prompts is rarely the changed lines. It is what the change does to model behavior. A two-word change can shift outputs significantly. A twenty-line change can have no effect. Reviewing prompts by reading the diff alone is the same as reviewing code by reading the diff without ever running it.

The fix is to attach the eval output to the PR. My CI job posts a comment on every PR with three artifacts. The first is the pass-rate before and after. The second is a sample of 10 outputs from the eval set, side by side: the previous prompt’s output and the new prompt’s output. The third is a list of the failing cases, with input and expected output, so the reviewer can read the actual model behavior that triggered the failure.

The PR review changes shape with that comment in place. The reviewer is not reading the diff and squinting. They are reading the model’s actual behavior, looking at the side-by-side, and deciding whether the change is an improvement or a regression. The diff is supporting evidence. The output is the substance.

This is the same shift that happened in frontend code a decade ago when teams started attaching Percy or Chromatic visual diffs to PRs. The diff is necessary but insufficient. The visual is the actual artifact. Prompts work the same way. The text change is necessary but insufficient. The output behavior is the actual artifact.

If you can get this in place, your team starts having real prompt review conversations. Without it, you are voting on vibes.

What Goes In The Registry vs The Repo

If you go with the registry architecture, the question that consumes the next month of your team’s life is “what goes where.” Here is the heuristic I have ended up with.

Repo: the structural prompts that define agent shape. The JSON schemas. The tool definitions. The system prompts that lock the model into a specific role. Anything where the prompt change is logically a code change because the application’s contract depends on the output shape.

Registry: the surface-level prompts that change with product decisions. The tone of voice. The marketing copy. The user-facing greeting. The prompt that summarizes a document for the user. Anything where the prompt change is logically a product decision and should be touchable by a PM without a code deploy.

The line gets fuzzy. There are prompts that feel structural but actually change often. There are prompts that feel cosmetic but actually have downstream code dependencies. The rule of thumb I use is: if changing the prompt could break a downstream consumer (a JSON schema mismatch, a missing field, a different output type), it goes in the repo. If changing the prompt could only ever affect output quality and tone, it goes in the registry.

Either way, the registry needs to support the same five properties as the repo workflow: tracked, reviewed, tested, versioned, rollbackable. If your registry does not have an audit log, prompt review, eval integration, version pinning, and rollback, you have bought a prettier Notion. Replace it with a real one. The registry market in 2026 is competitive. Tools that lack these features will be irrelevant by 2027.

The Eval Set Is The Hard Part

I want to be honest about which part of this is hard, because the tooling makes it sound easier than it is.

The CI job is a one-week build. The registry integration is a one-day build. The prompt files in the repo are a five-minute decision. None of that is the hard part.

The hard part is the eval set.

A good eval set takes a long time to build, requires real production data, and never finishes. You add cases for every regression you ship. You add cases for every customer report. You add cases when you notice an output that looks weird. The set grows. Six months in, it has 500 rows and you trust it. Twelve months in, it has 1,200 rows and it is one of the most valuable artifacts your company owns. You do not throw it away. You do not regenerate it. You curate it.

Most teams give up on this before month two. The cases feel arbitrary. The pass-rate hovers at 87% and seems impossible to push higher. The eval cost adds up. The temptation is to skip the eval and just deploy.

The teams that push through end up with something rare and valuable: a precise, executable definition of “what our AI feature is supposed to do.” That artifact is more valuable than the model behind it. The model can be swapped for a better one. The eval set tells you whether the swap was actually better, on the specific axes your customers care about. Without the eval set, every model upgrade is a gamble.

If you want a starting point that does not require months of curation, look at the LLM cost optimization workflow I described earlier; the same logs you mine for cost are the logs you mine for eval cases. Pull a hundred recent inputs. Sample for diversity. Write the expected outputs by hand. That is your eval set v0.1. Ship it. Improve from there.

The bar is not perfection. The bar is “better than no eval set.” That bar is low. Clear it and keep moving.

Migration: From “Where Is The Prompt” To Prompts As Code

If you are starting from “the prompt is somewhere in a Notion page and probably also a code file and I am not sure they agree,” here is the migration path that works.

Week one: find every prompt. This is uncomfortable. Most teams discover they have more prompts in production than they thought. Grep the codebase for system:, user:, messages:. Search Slack. Open every Notion page anyone mentions. Make a list. The first time I did this, I found 14 prompts. I had thought we had 6.

Week two: pick one architecture and move one prompt. Do not try to move all 14 in week two. Pick the most-touched prompt. Move it. Build the eval set for it. Wire up the CI job. Watch a PR go through the full workflow. Adjust until it feels right.

Week three: move the next three prompts. Use the workflow you built in week two. Notice what breaks. Improve the tooling.

Months two and three: move the rest. Some prompts will resist. They will be in weird places, owned by people who do not want to touch them, or attached to features you are planning to deprecate. Move them anyway. The cost of one prompt outside the workflow is more than the effort of moving it.

Quarterly: review the eval sets. The same way you review your topical standards files, walk through the eval sets. Cases that are no longer relevant get removed. Cases that catch new regressions get added. The sets stay current. The CI job stays meaningful.

That is the whole migration. It takes a quarter for a small team, half a year for a larger one. The dividend is paid every time you ship a prompt change without a Sunday morning support fire.

What This Changes About How You Build

The part that surprised me most when I made the switch is what it changed about my own decision-making.

Before prompts-as-code, every prompt change felt heavy. I would tweak a word, redeploy, watch the dashboards for an hour, hope nothing broke. The cost of a change was high because the blast radius was unclear. So I changed prompts less often than I should have. The prompts got stale. The feature underperformed. I knew it could be better but I was scared of the breakage.

After prompts-as-code, every prompt change costs the same as a code change. Write the change, open a PR, watch the eval, ship if green. The blast radius is bounded by the eval. The rollback is one click. The fear is gone. So I change prompts more often. The prompts stay current. The feature improves on a normal cadence instead of in stressful sprints.

The same thing happened when teams adopted CI for application code 15 years ago. The cost of a change went down. The volume of changes went up. The quality of the codebase improved because nobody was scared to touch it anymore. Prompts are following the same arc, ten years late.

If your team is still scared of prompt changes, that is the symptom. The fix is the workflow. The discipline of versioning, testing, and rollback removes the fear. The fear was real. The fix is real.

The Sunday incident I started this post with cost me eight hours and a piece of my Sunday. I do not think about that incident often anymore, because the workflow that came out of it has made the kind of mistake I made that day impossible to ship undetected. That is the trade. Eight hours of pain plus a few weekends of building the workflow, in exchange for a system that catches regressions before users do for the rest of your career.

Treat prompts like code. Version them. Test them. Roll them back. The tools exist. The discipline is the gap.

If you are still pasting prompts into Notion in 2026, the cost is not aesthetic. It is the Sunday you have not had yet.

The first real fight I had with a teammate over AI tooling was not about which model to use. It was about which config file we were supposed to be editing.

We had a CLAUDE.md at the repo root, written by me back in February. We had a .cursorrules file from the engineer who joined in March, who only used Cursor and refused to even open Claude Code. We had a copilot-instructions.md under .github/, dropped in by whoever turned on Copilot for the org. And we had a half-empty AGENTS.md that someone had heard about at a conference. Four files. Four different versions of the truth. Three subtly different opinions on how to write a database migration. Zero shared understanding.

The bug that triggered the argument was small. A junior engineer asked Cursor to write a Postgres migration. Cursor produced something fine by its own rules. The PR review pointed out that “our convention is to always wrap migrations in a transaction and write a rollback.” Junior engineer pointed at .cursorrules. Nothing about transactions. We checked CLAUDE.md. It had the rule. We checked copilot-instructions.md. Different wording, same intent. We checked AGENTS.md. Empty.

Nobody had written the rule into Cursor’s file because nobody used Cursor when the rule was added. The model did exactly what the file told it to. The file was just wrong by neglect.

This is the new shape of team coding standards in 2026. The standards exist. They just live in five places, in five formats, with five different update schedules. If you are leading any team larger than two people that uses more than one AI coding tool, this post is for you. I am going to walk through what each config file actually does, where the meaningful differences are, and the layout that finally got my team off the Monday argument loop.

The Four Files That Run Your Codebase Now

Before we compare anything, here is the lineup. There are more than four config files in the wild, but these are the ones you actually need to care about.

CLAUDE.md is the Claude Code config. It lives at the root of your repo (or in ~/.claude/CLAUDE.md for personal global rules). Claude Code reads it on every session start and treats the contents as a permanent system prompt. It supports @path/to/file imports up to five levels deep, which means you can split your standards into multiple files and reference them. It also supports auto-memory, where Claude writes notes back into a memory directory based on your corrections and preferences during a session.

.cursorrules (or the newer .cursor/rules/*.mdc directory layout) is the Cursor config. The legacy .cursorrules is a single Markdown file. The new directory format lets you have multiple rules with frontmatter that controls when they apply. The frontmatter supports globs, alwaysApply, and description fields, so you can have a rule that only fires for .tsx files, or only fires when the user asks about migrations. This is genuinely useful and an underused feature on most teams I have seen.

copilot-instructions.md is the GitHub Copilot config. It lives at .github/copilot-instructions.md and applies to Copilot Chat, code review, and the autonomous coding agent. Copilot also supports path-specific instructions under .github/instructions/*.instructions.md with YAML frontmatter that uses glob patterns. The path-specific files combine with the root file when both match, so a TypeScript rule plus a general rule both apply to a .ts file.

AGENTS.md is the closest the ecosystem has to a neutral standard. It started as the OpenAI Codex CLI config, then got adopted by Cursor, Aider, and a handful of smaller tools as a fallback. It supports nothing fancy. No imports, no globs, no path-specific overrides. Just plain Markdown that any compliant agent will read. The widest cross-tool support means it is also the lowest common denominator.

There are others. GEMINI.md for Google Gemini CLI. .windsurfrules for Windsurf. WARP.md for Warp’s agent mode. The pattern is identical to the four above. Pick the ones your team actually uses and ignore the rest.

What Each File Is Actually Good At

Treating these files as interchangeable is the first mistake most teams make. They are not. The features they support are different, and the optimal content for each one is different. Here is what I have ended up using each for.

CLAUDE.md is the longest-lived, most opinionated file in the system. Claude Code’s harness is the most autonomous of the three big tools, which means the rules you write get the most leverage. A single sentence like “always run typecheck before declaring work complete” actually changes behavior across a 40-step agent session. The @import feature lets you split standards into topical files (@.claude/standards/api.md, @.claude/standards/testing.md, @.claude/standards/database.md) and keep the root file as a table of contents. That is how I structure mine now. The auto-memory layer also makes CLAUDE.md grow over time without you babysitting it, which is either a feature or a footgun depending on how much you trust the model’s judgment about what to remember. I cover the broader Claude Code workflow in the agentic coding 2026 piece; the short version is that the leverage on a long-running agent is high enough that CLAUDE.md is worth the most editorial care.

Cursor rules are best treated as path-specific overrides. The new .cursor/rules/*.mdc format with globs is the killer feature. I have a rule that fires only for *.test.ts files and tells the agent to never use mocks for the database layer. I have another that fires only for *.tsx files and tells the agent the design tokens we use. The base file under .cursor/rules/ stays small and general. The path-specific files do the heavy lifting. If you write a 600-line .cursorrules at the root, you are using the tool wrong. Split it.

Copilot instructions are best for repo-wide truths. Copilot’s strength is breadth: every developer with the extension reads it whether they like it or not. Its weakness is that the instructions are short, get truncated more aggressively than CLAUDE.md, and are mostly used for Copilot Chat and the PR-review agent rather than for long autonomous sessions. The right content for copilot-instructions.md is the high-signal rules that you want every developer’s autocomplete and PR-review agent to enforce. Stack choices. Banned patterns. The five things that should never make it into a PR. Not a 200-line essay on architecture.

AGENTS.md is the politeness file. Its job is to exist for the tools that do not read your other files. Treat it as a redirect, not the source of truth. A typical AGENTS.md for my projects is twenty lines: “See CLAUDE.md for full standards. The most important ones are: A, B, C. Run bun test before claiming work is done.” That is enough for an unfamiliar agent to do something reasonable without needing the full CLAUDE.md context.

The mistake teams make is treating all four as parallel. They are not. There is a hierarchy.

The Layout That Finally Worked

After three months of fighting with this, here is the layout my team settled on. It has stayed stable for the last six months, which is the longest any AI tooling decision has lasted in our org.

We have one source of truth: .claude/standards/. Inside it, one file per topic.

.claude/standards/
  api.md
  database.md
  testing.md
  frontend.md
  observability.md
  security.md

Each file is short. 100 to 200 lines. Each one is reviewed by the team lead for that area. Each one has the same structure: a one-paragraph statement of intent, then a list of rules with reasons, then a list of bans with reasons. The structure matters because it forces the writer to justify the rule. A rule with no reason gets removed.

The root CLAUDE.md is a 30-line file that imports the topical files and adds the cross-cutting rules that do not belong to any single domain.

# Project Standards

This project uses Bun, Postgres, Astro, Vercel.

## Topical standards
@.claude/standards/api.md
@.claude/standards/database.md
@.claude/standards/testing.md
@.claude/standards/frontend.md
@.claude/standards/observability.md
@.claude/standards/security.md

## Cross-cutting rules
- Never run destructive SQL without showing the rows first.
- Always run `bun test` and `bun run typecheck` before declaring work complete.
- No em dashes in any file we ship. Ever.

## Workflow
- We use trunk-based development. Feature branches are short-lived.
- PRs are reviewed by the area owner. Big changes get a Linear ticket first.

That is the whole root file. Everything else gets pulled in by import.

The AGENTS.md is fifteen lines and points at the same standards. It is a redirect with a fallback for tools that cannot import.

# Agent Standards

See `.claude/standards/` for full rules. If you cannot read those files,
the four rules you must follow are:

1. Use Bun, not npm or yarn.
2. Never write to the database without showing the rows first.
3. Always run `bun test` and `bun run typecheck` after changes.
4. Follow the existing code style and import order.

Full standards: github.com/our-org/our-repo/tree/main/.claude/standards

The copilot-instructions.md mirrors the same content but compressed for Copilot’s shorter context window. We auto-generate it from the topical files using a small script that runs on pre-commit. That removes the drift problem at the cost of accepting that Copilot reads a summary, not the full standards.

The Cursor rules live in .cursor/rules/ and are tied to globs. The database.mdc rule has globs: ["**/migrations/**", "**/db/**"]. The testing.mdc rule has globs: ["**/*.test.ts", "**/*.spec.ts"]. The frontend.mdc rule has globs: ["**/*.tsx", "**/*.css"]. Each one is the relevant content from the corresponding .claude/standards/ file. We auto-generate these too, from the same source.

The script that does the generation is fifty lines of TypeScript. It reads the topical files, applies a compressor that strips reason-paragraphs and keeps only the rule lines for the shorter formats, and writes the four output files. Pre-commit runs it. The PR review checks that the generated files are in sync. If you edit a generated file by hand, the hook complains.

The point is not that you need exactly this layout. The point is that you need one source of truth and a way to project it into the formats every tool wants to read. Without that, you are running four parallel codebases of rules and one of them will silently drift.

The Things That Actually Cause Drift

Once you understand the file layout, the next problem is keeping the content in sync over time. This is where most teams fail, and the failure mode is always the same.

A new rule gets added to one file and not the others. A senior engineer adds a rule about transaction wrappers to CLAUDE.md after a bad incident. Six months later a junior engineer using Cursor produces the exact code the rule was meant to prevent, because Cursor never saw the rule. The fix is the projection layer above. One source, many outputs. Adding a rule means adding it in one place and letting the script handle the rest.

Old rules are never removed. AI coding standards files have a tendency to grow forever. A rule from 2024 about a deprecated library is still in the file in 2026. The model dutifully follows it. The newer engineer wonders why their code keeps getting rewritten to use Lodash. The fix is quarterly review. One thirty-minute meeting per quarter, the team lead walks down each topical file, and any rule that nobody can defend gets cut. We delete more than we add. The file shrinks roughly 20% per year. That is healthy.

The reasons get lost. A rule that exists without a reason is impossible to refactor. The team cannot evaluate whether the rule still applies, because the original context is gone. The fix is to require a reason for every rule. We use the same structure as my feedback memories in Claude Code: the rule itself, then Why: and How to apply: lines. The reason is mandatory. If you cannot articulate the reason, the rule does not belong in the file.

The standards contradict the codebase. A rule says “always use Tanstack Query for client-side data fetching.” The codebase has 40 SWR calls because it predates the rule. The model sees both and picks one at random. The fix is to either migrate the codebase or update the rule. There is no third option. A standard that contradicts reality is worse than no standard at all, because it teaches the model to ignore standards in general.

Personal preferences sneak into the team file. This one is the hardest. An engineer who hates trailing commas adds “no trailing commas in object literals” to CLAUDE.md. The team has never discussed this. Six months later the codebase is half-formatted that way and the linter has been disabled to accommodate it. The fix is process. New rules go to PR. The team lead reviews. Rules with no team-wide consensus get rejected. The standards file is a team contract, not a personal scratchpad.

The pattern across all four is the same. Drift is a process problem, not a tooling problem. The tools project the rules. The team writes them. If the team is sloppy, no tool will save you.

The Difference Between a Rule and a Skill

There is a deeper distinction that most teams miss, and it is the one I would write into the back of every developer’s hand if I could.

A rule is a constraint. It tells the model what not to do, or what it must do in a specific situation. “Never use mocks in integration tests.” “Always wrap migrations in transactions.” “Use Bun, not npm.” Rules are short. Rules are imperative. Rules belong in CLAUDE.md and friends.

A skill is a procedure. It tells the model how to do a specific task well. “How to add a new API endpoint to this codebase.” “How to review a PR for security issues.” “How to migrate a table without downtime.” Skills are long. Skills are explanatory. Skills belong in the Claude Code plugin marketplace, in SKILL.md files under .claude/skills/, or in the equivalent system for whichever tool you use.

The mistake teams make is putting skills into CLAUDE.md. The file balloons to 800 lines. The model still uses it, but each rule loses leverage because it is buried in three pages of procedural content. Worse, the procedures get triggered every session even when they are not relevant, which wastes tokens and slows everything down.

The right home for a procedure is a skill that fires on demand. Claude Skills, Cursor’s path-globbed rules, GitHub’s path-specific instruction files. Each tool has its own way of saying “load this content when this condition is true.” Use them. Keep the main config file lean. Keep the heavy procedural content in the skill system, where it loads only when needed.

If you read CLAUDE.md files in the wild, you will find this distinction violated everywhere. The 1,200-line CLAUDE.md is almost always 200 lines of rules and 1,000 lines of procedures that should have been skills. Splitting them is a one-day refactor with month-long benefits.

What Each Tool Is Actually Best At, In Practice

After running all three of these tools in production for the better part of a year, here is my honest read on where each one earns its keep. This is also the distribution my team has settled on.

Copilot is the default for everyone. Junior, mid, senior, every laptop in the org. It is the autocomplete baseline. The instructions file gets the most universal rules, the ones we want every PR to enforce. We do not expect Copilot to do agentic work for us. We expect it to make every developer 15% faster on the boring 90% of their day and to catch the obvious mistakes in PR review.

Claude Code is the senior engineer’s tool. It is what we use for the high-leverage agentic work: a multi-file refactor, a migration plan with rollback, a new feature that touches the backend, the frontend, and the database in one shot. The CLAUDE.md and .claude/skills/ are where the senior engineers invest. We do not push Claude Code on the juniors. They graduate into it when they are ready to direct an agent rather than press tab.

Cursor is the IDE-first option for engineers who want a richer editor experience. Some of our team prefers the Cursor UI, some prefers VS Code with Copilot, some lives in Claude Code in the terminal. We do not standardize. The Cursor rules are auto-generated from the same source. The cost of supporting three tools is small once the projection layer exists. The cost of forcing everyone onto one tool is high, because half the team would lose 20% productivity in the switch.

If you compare these tools head-to-head, the deeper comparison lives in the Claude Code vs Cursor vs Copilot 2026 piece. The TL;DR for the config-file question is: each tool needs its config file, the content overlaps 80%, the projection layer is what keeps them honest.

The Workflow For Adding A New Rule

The last thing worth writing down is the workflow itself. Without a workflow, you are back to ad-hoc edits and silent drift.

A new rule starts as a Linear ticket or a Slack thread. Someone notices a recurring problem. Someone writes up the pattern, the proposed rule, and the reason. The team reviews. If there is consensus, the rule lands in the relevant topical file under .claude/standards/ via PR. The PR includes the reason in the commit body. The PR is reviewed by the area owner.

After merge, the projection script runs. The generated files update. Copilot, Cursor, AGENTS.md all reflect the new rule on the next session.

If a new rule contradicts the existing codebase, the PR includes a migration plan. Either the codebase changes to match the rule, or the rule changes to match the codebase. We do not ship a standard that lies about the code.

Every quarter, the team lead reviews each topical file. Rules with no defender are deleted. The file shrinks. The standards stay relevant.

This sounds heavy. It is not. Most quarters the review takes thirty minutes. Most new rules get reviewed in a five-minute Slack thread. The investment is upfront, in building the projection layer and the review habit. Once it is built, it runs itself.

The Part Nobody Talks About

The hidden cost of AI coding standards is not the tooling. It is the social cost of writing them down.

For years, senior engineers got away with carrying the standards in their heads. They reviewed PRs, they corrected the juniors, they shaped the codebase one line of feedback at a time. The standards existed, but they were tacit. The senior was the standard.

The AI tools force tacit standards out into text. Every rule the senior used to hold in their head now has to be articulable, defensible, written. The first time an engineer sits down to write the database.md file for their team, they hit the wall: half of what they “know” is actually preference, not principle. The other half they cannot defend without a long story about an incident from two years ago. Writing the rules surfaces every weak link in your team’s thinking. It is uncomfortable. It is also exactly the thing that makes the team stronger.

The teams I have seen do this well treat the standards file as a team artifact, not a personal one. The senior writes the first draft. The team reviews. The juniors ask why. The senior either has a good answer or removes the rule. After six months, the file is denser and shorter than the first draft, because the only rules that survive are the ones the team can defend.

That is the real prize. Not faster agents. Not more lines of code per day. A team that can explain, to a piece of software, what good looks like. The agents read the file. The juniors read the file. The future senior who joins next year reads the file. The standard is no longer locked in one person’s brain. It is on disk, versioned, reviewable, improvable.

The tools are the surface. The discipline of writing down what you actually believe is the substance.

Pick the layout you can defend. Build the projection script. Run the quarterly review. Stop arguing about which file is the source of truth and start arguing about what the rules should actually be. That is the conversation that makes a team better.

The Monday argument on my team has changed. It is not “which file should we have edited.” It is “should this rule exist at all, and what is the reason.” That is the right argument to be having. If you are not having it yet, that is the goal.

ags: showdev javascript ai buildinpublic

There’s a stat that’s been stuck in my head for months.

Lead conversion in real estate drops 400% if the first follow-up call happens more than 5 minutes after a buyer submits a form. Most Indian real estate teams call back in 4 to 6 hours.

That gap — between a buyer raising their hand and a human picking up the phone — costs India’s property sector over Rs 15,000 crore every year. Not bad products. Not bad pricing. Just slow follow-up.

So I built PropCall Live. An AI that calls every lead within 30 seconds of form submission. At 11PM. On Sundays. Without a single telecaller.

Here’s exactly how it works and how I built the whole thing without writing code manually.

What happens in 30 seconds
A buyer fills an enquiry form — name, phone, budget, BHK preference, timeline, source.

The moment they hit Submit, three things fire simultaneously:

1. Lead scoring
   A weighted algorithm calculates a 0–100 score in milliseconds.

Budget fit: 40%
Purchase urgency: 40%
Lead source quality: 20%
A buyer looking for a 3BHK in budget who needs to move immediately scores 94 — HOT 🔥. A casual browser with no timeline scores 32 — COOL.

1. Telegram notification
   The sales manager’s phone gets a full lead card instantly — name, score, budget, and a personalised Hindi follow-up message generated on the spot.

2. AI voice call
   Retell AI calls the buyer’s phone. Natural Hindi-English conversation. Qualifies the lead. Books a site visit.

The sales team wakes up to a calendar full of confirmed visits. They never touched a lead manually.

The tech stack

Frontend: React 19 + TypeScript + Tailwind CSS
Voice: Retell AI (Web SDK + phone calling)
Alerts: Telegram Bot API
Weather: Open-Meteo API
Backend: Vercel Serverless Functions
Built with: MeDo AI
How the integrations work
Retell AI — the hard part
Retell’s create-web-call endpoint is designed for server-side use. If you call it directly from the browser you get CORS errors. The fix is a Vercel serverless proxy:

// api/create-call.ts
const RETELL_API_KEY = process.env.RETELL_API_KEY;

export default async function handler(req, res) {
const { agent_id } = req.body;

const upstream = await fetch(“https://api.retellai.com/v2/create-web-call“, {
method: “POST”,
headers: {
Authorization: Bearer ${RETELL_API_KEY},
“Content-Type”: “application/json”,
},
body: JSON.stringify({ agent_id }),
});

const data = await upstream.json();
res.status(upstream.status).json(data);
}
The browser calls /api/create-call. The function proxies to Retell server-side. The API key never touches the client bundle.

The access token comes back, gets handed to the Retell Web SDK:

import { RetellWebClient } from “retell-client-js-sdk”;

const client = new RetellWebClient();
await client.startCall({ accessToken: data.access_token });
Live AI voice call. In the browser. In 30 seconds.

Gotcha I hit: The npm package is retell-client-js-sdk — NOT @retell-ai/retell-client-js-sdk. The scoped package returns a 404 from CDN. Cost me an hour.

Telegram — simpler than you think
Same proxy pattern. The bot token stays server-side:

// api/send-telegram.ts
await fetch(https://api.telegram.org/bot${TELEGRAM_TOKEN}/sendMessage, {
method: “POST”,
headers: { “Content-Type”: “application/json” },
body: JSON.stringify({
chat_id: TELEGRAM_CHAT_ID,
text: 🔥 NEW LEADnn👤 ${name}n🤖 Score: ${score}/100 — ${label}nn💬 ${hindiMessage},
}),
});
The sales manager sees this on their phone before the AI call has even connected.

Open-Meteo — the detail that impresses
After the form submits, I fetch a 3-day forecast for the project’s coordinates:

const res = await fetch(
“https://api.open-meteo.com/v1/forecast” +
“?latitude=28.5355&longitude=77.3910” +
“&daily=weather_code,temperature_2m_max” +
“&timezone=Asia%2FKolkata&forecast_days=4”
);
WMO weather codes map to visit recommendations — clear sky days get “Good for visit”, rain days get “Not ideal”. Free API, no key required, zero setup.

It’s a small touch. But when a judge or client sees the product suggest “Visit on Thursday — clear sky, 34°C” they understand: this thing is thinking about the whole sales process, not just the form.

Running all three in parallel
The most important line in the entire codebase:

const [telegramSent, callStarted, weatherData] = await Promise.all([
sendTelegram(…),
startRetellCall(),
fetchWeather(),
]);
Three API calls. One await. If you run them sequentially you add 2-3 seconds. Parallel keeps the whole pipeline under 30 seconds.

The MeDo part
I want to be direct about this because it’s the part of the story I find most interesting.

I did not write this code manually. The entire product — React frontend, scoring algorithm, Retell integration, Telegram proxy, weather widget, CRM dashboard, ROI calculator, Vercel serverless functions, security architecture — was built by describing what I wanted to MeDo in plain English.

Not “generate a component.” Full architectural decisions. “The Retell API key should never be in the browser — how do we proxy it?” MeDo figured out the serverless function approach, implemented it correctly, and caught the CORS issue before I’d even thought to look for it.

The traditional path to this product: 2-3 engineers, 3 months, Rs 15-20 lakhs. The actual path: one person with a clear product vision and MeDo.

I don’t say this to dismiss engineering. I say this because if you have a product idea and the only thing stopping you is “I can’t build it” — that excuse is gone.

What I’d do differently
Security first. I started with API keys in constants.ts in the client bundle. Moving them to environment variables and server-side proxies later meant touching five files. Starting with the proxy architecture would have been faster.

Verify CDN URLs before shipping. I had a broken CDN script loading a 404 in production for longer than I’d like to admit. Always curl the URL before you trust it.

The follow-up speed slider matters. The ROI calculator had a “current follow-up speed” slider that did nothing in the calculation for a while. Wiring it to the lost-leads formula — faster follow-up means fewer leads lost, with a real curve — made the tool honest.

What’s next
Outbound phone calls to actual mobile numbers (not just browser calls)
White-label self-serve onboarding for any developer
Tamil, Marathi, Telugu agents
CRM webhook integrations (Kylas, Sell.Do)
If you’re building anything with Retell AI or want to talk about AI-native product development in India — I’m at [your contact].

Live demo: [your Vercel URL]
GitHub: https://github.com/arpit2222/propcall

Power Apps is a powerhouse for rapid prototyping within the Microsoft ecosystem, but for complex development, the “seams” are becoming visible. This new analysis explores the “Complexity Ceiling” and the friction points faced by architects and developers.

Core Issues Identified:

• Mental Model Mismatch: Chaining nested With blocks to simulate sequential logic.
• Tooling Gaps: The need for real breakpoints and execution narratives over simple trace functions.
• Governance Hurdles: The “abrupt cliff” of enterprise compliance for makers.

Microsoft has the opportunity to bridge the gap between business analysts and professional engineers by introducing controlled imperative constructs and better code organization.

https://www.alanbonnici.com/2026/05/the-complexity-ceiling-where-microsoft.html

---
title: "Gemini Nano Function Calling: Building Offline AI Agents on Android"
published: true
description: "A hands-on guide to architecting offline AI agents on Android using Gemini Nano's on-device function calling, structured JSON output, and a WorkManager + Room sync pipeline."
tags: android, kotlin, architecture, mobile
canonical_url: https://blog.mvpfactory.co/gemini-nano-function-calling-offline-ai-agents-android
---

## What We Will Build

Let me show you a pattern I use in every project that needs on-device intelligence. We are going to architect an offline-capable AI agent on Android using Gemini Nano's function calling and structured JSON output — the features Google expanded at I/O 2026. By the end, you will have a validation pipeline that tames on-device hallucinations and a WorkManager + Room queue that executes agent actions when connectivity returns.

## Prerequisites

- Android Studio with Kotlin
- Familiarity with Room and WorkManager
- Access to Gemini Nano on-device APIs (AI Core)
- A device or emulator supporting on-device inference

## Step 1: Respect the 32K Token Budget

Cloud Gemini Flash gives you 1M+ tokens. Gemini Nano gives you roughly 32K on-device. That budget covers your system prompt, tool definitions, conversation history, *and* the response. Most teams get this wrong by porting cloud schemas directly.

Here is the minimal setup to get this working:

kotlin
// Bad: verbose schema that eats your token budget
val cloudSchema = Tool(
name = “create_calendar_event”,
description = “Creates a new calendar event with the specified title, ” +
“date, time, duration, location, attendees, recurrence pattern, ” +
“reminder settings, and optional notes…”,
parameters = /* 12 parameters with long descriptions */
)

// Good: minimal schema optimized for on-device budget
val nanoSchema = Tool(
name = “cal_create”,
description = “Create event”,
parameters = listOf(
Param(“title”, “string”, required = true),
Param(“iso_time”, “string”, required = true),
Param(“dur_min”, “int”, required = true)
)
)

A trimmed schema set of 5 tools consumes roughly 800–1,200 tokens, leaving headroom for conversation context. A verbose 15-tool schema can eat 4,000+ tokens before a single user message. Budget 1,200 tokens maximum for tool definitions. Use short names, minimal descriptions, and cap at 5 tools per agent context. Swap tool sets dynamically based on user intent rather than loading everything at once.

## Step 2: Build a Three-Layer Validation Pipeline

A quantized on-device model hallucinates more than its cloud counterpart. For function calling, this shows up as malformed JSON, invented parameter names, or calls to tools that do not exist in your schema.

The docs do not mention this, but Layer 1 alone catches roughly half of all failures — the model returns valid function calls but wraps them in explanatory text.

kotlin
fun parseAgentAction(raw: String): AgentAction? {
// Layer 1: Extract JSON from response (model may wrap it in markdown)
val json = JsonExtractor.findFirst(raw) ?: return null

// Layer 2: Validate against registered tool schemas
val parsed = try {
    toolRegistry.parse(json)
} catch (e: SchemaValidationException) {
    null
}

// Layer 3: Semantic bounds checking
return parsed?.takeIf { action ->
    semanticValidator.isReasonable(action)
    // e.g., duration_min in 1..480, title.length < 200
}

}

## Step 3: Wire Up the WorkManager + Room Offline Queue

Where on-device function calling really earns its keep is offline operation. A user on an airplane says "schedule a team sync for Tuesday at 2pm." Gemini Nano parses the intent locally, but the calendar API requires connectivity.

kotlin
@entity(tableName = “agent_actions”)
data class AgentAction(
@PrimaryKey(autoGenerate = true) val id: Long = 0,
val toolName: String,
val paramsJson: String,
val status: ActionStatus = ActionStatus.PENDING,
val createdAt: Long = System.currentTimeMillis()
)

val request = OneTimeWorkRequestBuilder()
.setConstraints(
Constraints.Builder()
.setRequiredNetworkType(NetworkType.CONNECTED)
.build()
)
.setInputData(workDataOf(“action_id” to action.id))
.build()

WorkManager.getInstance(context).enqueue(request)

Gemini Nano produces a structured `AgentAction`. Room persists it with status `PENDING`. WorkManager enqueues a `OneTimeWorkRequest` with network constraints. An executor processes the action when connectivity returns, updating status to `COMPLETED` or `FAILED`. You get immediate user feedback ("Got it, I'll create that event when you're back online") while guaranteeing eventual execution. Room provides durability across process death, and WorkManager handles retry with exponential backoff.

| Dimension | Gemini Nano (on-device) | Gemini Flash (cloud) |
|---|---|---|
| Context window | ~32K tokens | 1M+ tokens |
| Latency (first token) | 80–200ms | 300–800ms (network dependent) |
| Function call reliability | Degrades with schema complexity | Stable across complex schemas |
| Structured JSON consistency | Requires validation + retry | Generally reliable |
| Availability | Always-on, no network needed | Requires connectivity |
| Cost per call | Zero marginal | Per-token API pricing |

## Gotchas

- **Token budget blowout.** Porting your cloud tool schemas directly to Nano will silently consume your context window. You will get incoherent responses with zero error messages. Keep schemas under 1,200 tokens total.
- **Markdown-wrapped JSON.** The model frequently wraps valid JSON in explanatory text or markdown code fences. A solid JSON extractor is table stakes — without one, you will reject roughly half of perfectly good responses.
- **Invented parameters.** Nano will hallucinate parameter names that do not exist in your schema. Always validate against your registered tool definitions before executing anything.
- **Skipping the offline queue.** Adopt the WorkManager + Room pattern early, even if your initial use case is online-only. This architecture lets you go offline with zero refactoring. The persistence layer also doubles as an audit log of every agent action — useful for debugging and showing users what the agent did on their behalf.

## Wrapping Up

The latency advantage of on-device inference (80–200ms vs 300–800ms) matters for interactive mobile UX. But the reliability gap is the architectural challenge you actually need to design around. Shrink your schemas, validate in layers, and queue actions with Room + WorkManager. That combination turns Gemini Nano from a demo into a production-grade offline agent pipeline.

I Deleted 40% of Our Kubernetes Config. Here’s What Stayed.

Our Kubernetes manifests were 2,000 lines of copy-pasted YAML. After cleanup: 1,200 lines. Better reliability.

What I Removed

Duplicate HPA configurations that contradicted each other. Init containers that did nothing. Resource limits set by guessing. Multiple replicas for stateless services that’s pointless.

What Stayed

The actual health checks (liveness and readiness probes that matched real behavior). Proper pod disruption budgets. The services that actually needed ClusterIP vs the ones we changed to LoadBalancer out of habit.

The Process

1. Delete one thing
2. Deploy to staging
3. Watch for 24 hours
4. If nothing breaks, merge

Sounds tedious. It is. But we caught three real issues during the process.

The Surprising Finding

Most of our “production-grade” config was cargo-culted from tutorials. Real production needs are usually simpler than tutorials suggest.

What I’d Tell My Past Self

Start with minimal configs. Add complexity only when you have a specific reason, not anticipated future needs.

What’s your Kubernetes config-to-code ratio?