Mert

I had a system prompt that ended with: Reply with only a JSON object. Do not include code fences. Do not include explanation.

That should be enough. It is not.

Over a week of structured-output calls (12,400 of them, mostly Claude Sonnet 4.5 and 4.7), I logged how often the response was actually parseable as JSON on the first serde_json::from_str attempt. Result: 86.0%. The other 14% had at least one of these problems:

• json ... wrappers (most common, about 9.3% of total)
• Leading or trailing prose (“Here is the JSON you asked for:”)
• Trailing comma in the last array or object element
• Smart quotes pulled in from training data

So I wrote llm-json-repair. Three passes, applied in order, each one cheap, none of them re-invoking the LLM. If a downstream model fix is needed, that is your problem to glue on. This crate is the local cleanup before you spend another API call.

The three passes

use llm_json_repair::repair;

let raw = r#"```

json
{
  "intent": "book_flight",
  "slots": {
    "origin": "DAL",
    "destination": "JFK",
  },
}


```"#;

let cleaned = repair(raw)?;
let parsed: serde_json::Value = serde_json::from_str(&cleaned)?;

What repair actually does, in order:

Pass 1: strip fences

Look for the first ` and the last `. If both exist, take what is between them. The optional json language tag gets dropped along with the opening fence. If there is text after the closing fence, it is discarded.

This pass is a one-liner conceptually, but the edge case that bit me was a JSON string value that contained the literal substring ` (yes, a customer-support transcript got fed in once). So the matcher only strips fences at the outermost level and only if they are on their own line or at the start.

Pass 2: balanced extraction

Walk character by character. Find the first { or [. Count nesting. Stop when nesting hits zero. Return that substring.

This drops leading prose (“Here is the JSON:”) and trailing prose (“Let me know if you need anything else.”) without needing a model. It also handles the case where the model emitted two JSON objects in a row (rare but real); you get the first complete one.

`rust
let raw = "Sure thing! Here is the JSON:n{"intent": "refund", "reason": "late"}nLet me know if you have questions.";
let cleaned = repair(raw)?;
assert_eq!(cleaned, r#"{"intent": "refund", "reason": "late"}"#);
`

The cost of this pass is O(n) over the response. For a 4 KB response it runs in about 30 microseconds on my laptop.

Pass 3: trailing comma fix

Walk the candidate JSON. For each , followed only by whitespace and then } or ], delete the comma. Skip when inside a string literal (track quote state with escape handling).

This is the single most common syntactic failure I see from Claude when it is generating a long object: it puts a trailing comma after the last field. It is a small thing but serde_json rejects it. Fixing it locally saves you a round trip.

A real broken case from my logs

Raw response:

`json
`json
{
“summary”: “User wants to cancel order #4419”,
“actions”: [
{“name”: “cancel_order”, “args”: {“id”: 4419}},
{“name”: “notify_user”, “args”: {“channel”: “email”,}},
],
}
`
`json

Three problems: fences, two trailing commas, valid JSON otherwise. After repair:

`json
{
"summary": "User wants to cancel order #4419",
"actions": [
{"name": "cancel_order", "args": {"id": 4419}},
{"name": "notify_user", "args": {"channel": "email"}}
]
}
`

Parses cleanly. No second LLM call.

When to bail out

If all three passes run and the result still does not parse, repair returns an error. You should not silently swallow that. The crate exposes try_repair which returns Result<String, RepairError> and gives you the failing pass in the error variant.

`rust
match try_repair(raw) {
Ok(s) => use_json(s),
Err(RepairError::Pass3(_)) => {
// bracket structure is fine but content is wrong
// worth one model-side retry with the error attached
}
Err(e) => log::warn!("could not repair: {e}"),
}
`

What this does not solve

A few honest limits.

• It does not invoke the LLM. If the model emitted a hallucinated field name, this crate cannot fix it. Use agentcast-rs if you need a validate-and-retry loop.
• It does not enforce a schema. It only restores parseability. You still need serde derives or jsonschema on top.
• It does not handle every JSON5 extension. Single-quoted strings, unquoted keys, and comments are not supported. I have not seen Claude emit those, so I have not added the passes.
• It does nothing with NDJSON (newline-delimited JSON). One blob in, one blob out.

The whole crate is about 350 lines of safe Rust with no async. It works as a sync function or inside any runtime.

Repo: https://github.com/MukundaKatta/llm-json-repair
crates.io: llm-json-repair = "0.1"

Part of a small set of Rust crates I publish for the unglamorous LLM plumbing: parsing, cost, retry, budget, repair. Sibling crates: claude-cost, llm-retry, agentcast-rs.

Tuesday afternoon I kicked off a re-grading job. About 18,000 prompts against claude-opus-4-7, eight workers, each one looping messages.create as fast as it could.

Forty minutes in, every call started coming back with a 429 and a header that said anthropic-ratelimit-tokens-remaining: 0. Fine, I thought. Back off. I cut workers to four and waited. Still 429. Cut to two. Still 429.

Then I noticed the cap-clear timestamp was not minutes. It was rolling. I had pushed past the daily token budget for the whole org, and a daily window does not reset in five minutes.

I emailed support. They acknowledged Wednesday morning. They cleared the cap Friday afternoon. 72 hours.

I am not going to claim the engineering was elegant after that. I sat there refreshing the dashboard for three days. When the cap finally cleared, I built llmfleet so I would never sit there again.

What it does

llmfleet is a pooled dispatcher for messages.create. You hand it a list of message payloads and a concurrency cap, and it runs them with backpressure that respects two things at once: in-flight request count, and the most recent anthropic-ratelimit-tokens-remaining header.

The Sandler-inspired piece is the negotiation. Instead of a hard semaphore, the pool watches what the API tells it. If the remaining-tokens header drops under a threshold, in-flight slots get held until the window ticks. No frantic 429 retries.

import asyncio
from llmfleet import Fleet

fleet = Fleet(
    api_key=os.environ["ANTHROPIC_API_KEY"],
    max_in_flight=8,
    soft_token_floor=20_000,   # pause new dispatches under this
    hard_token_floor=2_000,    # full stop until next window
)

payloads = [
    {"model": "claude-opus-4-7", "max_tokens": 256,
     "messages": [{"role": "user", "content": prompt}]}
    for prompt in prompts
]

async def run():
    async for result in fleet.dispatch(payloads):
        store(result.payload_id, result.response, result.cost_usd)

asyncio.run(run())

dispatch is an async iterator that yields results in completion order, not submission order. Each result has the original payload id, the response, latency in ms, and a cost estimate.

Real numbers I cite when people ask

On a single Anthropic key with no special quotas:

• Messages/sec ceiling I see in practice for short prompts (about 400 input tokens, 200 output): around 6.2 req/s sustained before the soft floor kicks in.
• Time spent waiting at the soft floor over a 10-minute window: about 11% of wall clock.
• Time spent paused at the hard floor: zero, if you set soft_token_floor to about 10% of your tokens-per-minute quota. That is the whole point of the soft floor.

If you have higher tier quotas the numbers shift, but the shape is the same.

Queue depth math

The naive question is: how big should max_in_flight be?

Sandler’s answer is a Little’s Law calculation. If your average latency is L seconds and you want throughput R req/s, you need at least R*L concurrent calls in flight to saturate.

For Claude Opus with a 200-token output and typical 4-second responses at 6 req/s, that is 24 in-flight. But the Anthropic per-minute limit on most accounts will choke you before then. So the real max_in_flight is min(R*L, perminute_quota / 60 * L).

llmfleet does this math for you if you pass tier="default" or whatever your tier is. It logs the chosen ceiling at startup.

A small detail that mattered

The 429 retry that originally got me into this mess was not malicious. It was the SDK doing its default exponential backoff. Every worker was independently backing off and re-firing, which kept the cap pinned at zero for hours after the actual job was idle.

llmfleet disables the SDK’s internal retry. The pool owns the retry budget. One shared count. When a single request fails non-retriably, the pool can decide whether to surface or move on, and the dispatcher logs the cost of the failed attempt so it does not disappear from your budget tracking.

fleet = Fleet(api_key=...,
              retry_policy=dict(max_attempts=3, base_delay=2.0, max_delay=30.0),
              shared_retry_budget_per_min=20)

Cost guard

I also added a hard USD cap because I do not trust myself at 2 AM.

fleet = Fleet(api_key=..., max_spend_usd=15.00)

When the running total crosses the cap, no new dispatches go out. In-flight ones still complete. The iterator yields a final BudgetExceeded marker and stops.

What this does not solve

• It does not raise your account quota. Three days of waiting was a quota issue, not a code issue. llmfleet keeps you under the line, not over it.
• It only talks to Anthropic right now. The interface mirrors messages.create exactly. I could generalize to OpenAI, but I have not yet.
• It does not do prompt caching for you. If you want that, look at cachebench. The two compose: caching reduces the tokens you count against the floor.
• It does not implement priority lanes. Every payload is FIFO. If you want one job to jump the queue, run two fleets.

The whole library is about 700 lines. The interesting part is the floor logic, not the queue.

Repo: https://github.com/MukundaKatta/llmfleet
PyPI: pip install llmfleet

Part of a small stack of agent-plumbing libs I keep building from real incidents. The unglamorous ones.

Introduction

“~35% cheaper · ~70% fewer tool calls · 100% local”

This is the No.71 article in the “One Open Source Project a Day” series. Today we are exploring CodeGraph.

Start with a scenario: you ask Claude Code “How is AuthService being called?” Without any assistance, Claude’s approach is: glob-scan directories, run multiple greps, read several files — then finally answer. The whole process might trigger 10–15 tool calls and consume hundreds of thousands of tokens.

CodeGraph’s insight is to front-load this work: before you start, it has already parsed your codebase with tree-sitter into a semantic graph stored in a local SQLite database, then exposes 8 query tools to AI agents via MCP. When the agent needs to understand code, a single codegraph_context call returns entry points, related symbols, and code snippets — no file reading required.

9.6k Stars, 588 Forks. Benchmarks across 7 real open-source projects: average 35% cost savings, 70% fewer tool calls, 49% speed improvement. On VS Code’s large TypeScript repository, one architecture Q&A dropped from 1.4M tokens to 393k — cost from $0.64 to $0.42.

What You Will Learn

• CodeGraph’s four-stage pipeline: Extract → Store → Resolve → Auto-Sync
• The 8 MCP tools and when to use each
• A detailed breakdown of benchmark results across 7 projects: why do larger codebases benefit more?
• How 19-language support and 13-framework route recognition work
• Complete setup walkthrough from installation to Claude Code integration
• codegraph affected: using dependency tracing for smart CI test selection

Prerequisites

• Familiarity with Claude Code, Cursor, or similar AI coding tools
• Basic understanding of MCP (Model Context Protocol)
• Node.js experience

Project Background

Project Introduction

CodeGraph is a local semantic code knowledge graph tool designed specifically to improve AI coding agent efficiency. Its core insight:

AI agents spend a massive amount of tokens and time in the “discovery phase” — scanning directories, searching for symbols, reading files — rather than on the actual reasoning and generation.

CodeGraph’s solution is to outsource the discovery phase to a pre-built index: before you start working, the index is already ready, letting AI agents pull structured code knowledge directly instead of exploring the file system from scratch.

The technology choices are pragmatic: tree-sitter for AST parsing (mature, multi-language, high-performance), SQLite FTS5 for full-text search (zero external dependencies, fully local), and native OS file events for live sync (FSEvents/inotify/ReadDirectoryChangesW).

Author/Team

• Author: Colby McHenry (GitHub: colbymchenry)
• Repository: colbymchenry/codegraph
• Distribution: npm package @colbymchenry/codegraph

Project Stats

• ⭐ GitHub Stars: 9,600+
• 🍴 Forks: 588
• 📦 npm package: @colbymchenry/codegraph
• 🔧 Runtime: Node.js 20–24
• 💻 Platforms: Windows, macOS, Linux
• 📄 License: MIT
• 🌐 Repository: colbymchenry/codegraph

Main Features

Core Utility

CodeGraph inserts a pre-built index layer between AI agents and codebases:

Codebase (TypeScript / Python / Go / ...)
        ↓ tree-sitter parsing
  Semantic graph (symbols + relationships + call chains)
        ↓ stored in SQLite FTS5
  Local knowledge base
        ↓ exposed via MCP
  AI coding agents (Claude Code / Cursor / Codex CLI / OpenCode)

Without CodeGraph:

User: "How is AuthService being called?"
→ Agent: glob("src/**/*.ts")         # Tool call 1
→ Agent: grep("AuthService")         # Tool call 2
→ Agent: read("auth.service.ts")     # Tool call 3
→ Agent: grep("import.*Auth")        # Tool call 4
→ Agent: read("user.controller.ts")  # Tool call 5
→ Agent: read("app.module.ts")       # Tool call 6
... 10–15 total tool calls, massive token consumption

With CodeGraph:

User: "How is AuthService being called?"
→ Agent: codegraph_callers("AuthService")   # Tool call 1
→ Returns: full caller list + call sites + code snippets
→ Agent answers directly, no file reading needed

Quick Start

One-command install (recommended):

# Run the interactive installer — auto-detects installed AI agents and configures them
npx @colbymchenry/codegraph

# Initialize in your project (-i for interactive)
cd your-project
codegraph init -i

Non-interactive install (CI environments):

# Auto-detect all installed agents, global install
codegraph install --yes

# Target specific agents
codegraph install --target=cursor,claude --yes

# Project-local install
codegraph install --target=auto --location=local

Manual Claude Code configuration:

npm install -g @colbymchenry/codegraph

Add to ~/.claude.json (or project-level .claude.json):

{
  "mcpServers": {
    "codegraph": {
      "type": "stdio",
      "command": "codegraph",
      "args": ["serve", "--mcp"]
    }
  }
}

Verify installation:

codegraph status          # Check index status and stats
codegraph query "UserService"  # Test symbol search

The 8 MCP Tools

The complete toolset CodeGraph exposes to AI agents:

codegraph_context is the most important tool — it doesn’t just return search results; it intelligently assembles a comprehensive context package for a given task, including entry points, related symbols, and code snippets:

# Command-line equivalent
codegraph context "fix user login bug"
# → Automatically finds login-related functions, call chains, and relevant files
#   packaged into context Claude can consume directly

Project Advantages

Detailed Analysis

1. The Four-Stage Pipeline

Stage 1: Extraction

tree-sitter parses source files into ASTs, extracting:

• Symbols: functions, classes, methods, interfaces, variable definitions
• Relationships: function calls, module imports, class inheritance, interface implementations

tree-sitter’s key advantage: it is a fault-tolerant parser — it can extract partial structure even when code has syntax errors. This is critical for indexing files that are actively being edited.

Stage 2: Storage

All data lands in a local SQLite database using the FTS5 (Full-Text Search 5) extension:

-- Symbols table (simplified)
CREATE VIRTUAL TABLE symbols USING fts5(
  name,          -- Symbol name
  kind,          -- function/class/method/...
  file_path,     -- Source file
  line_start,    -- Starting line
  signature,     -- Function signature
  docstring,     -- Documentation comment
  code_snippet   -- Code excerpt
);

-- Relationships table
CREATE TABLE edges (
  from_id  INTEGER,  -- Caller symbol ID
  to_id    INTEGER,  -- Callee symbol ID
  kind     TEXT,     -- calls/imports/inherits/implements
  file     TEXT,
  line     INTEGER
);

Stage 3: Resolution

The critical step: resolving abstract “called something named X” into concrete “called the definition in file Y at line Z.”

Source code: import { AuthService } from './auth.service'
             ...
             this.authService.login(user)
            ↓ resolution
Graph edges: UserController.login → AuthService.login (calls)
             UserController → AuthService (imports)

Stage 4: Auto-Sync

Uses native OS file events (not polling!) to detect changes:

• macOS: FSEvents
• Linux: inotify
• Windows: ReadDirectoryChangesW

A 2-second debounce prevents triggering mass rebuilds when files change rapidly — it waits for changes to settle before doing incremental updates.

2. Benchmark Deep Dive

Test conditions: Claude Code (headless, Opus 4.7) answering architecture questions. Each result is the median of 4 runs on the same question, across 7 real open-source repositories.

Project        Language       Size            Cost ↓  Token ↓  Speed ↑  Tool Calls ↓
──────────────────────────────────────────────────────────────────────────────────────
VS Code        TypeScript     ~10k files      35%     73%      41%      72%
Excalidraw     TypeScript     ~600 files      47%     73%      60%      86%
Django         Python         ~2.7k files     34%     64%      59%      81%
Tokio          Rust           ~700 files      52%     81%      63%      89%
OkHttp         Java           ~640 files      17%     41%      36%      64%
Gin            Go             ~150 files      22%     23%      34%      19%
Alamofire      Swift          ~100 files      38%     59%      51%      77%
──────────────────────────────────────────────────────────────────────────────────────
Average                                       35%     59%      49%      70%

Patterns worth noting:

1. Tokio (Rust, 700 files) sees the biggest gains (81% token reduction, 89% fewer tool calls): Rust’s type system is complex — agents originally needed extensive file exploration to understand trait implementations and generic relationships. CodeGraph’s pre-built relationships make this dramatically cheaper.

2. Gin (Go, 150 files) sees the smallest gains (23% token reduction, 19% fewer tool calls): Small Go projects have simple file structures. Agents can already navigate them efficiently, so CodeGraph’s marginal value is lower.

3. VS Code’s absolute numbers are the most striking: the same question costs $0.64 (1.4M tokens) without CodeGraph, $0.42 (393k tokens) with it. A single task saves $0.22.

Takeaway: The larger the codebase, the more complex the dependencies, and the richer the language’s type system, the greater CodeGraph’s benefit. For developers using Claude Code heavily on large projects, the ROI is clear.

3. 19 Languages + 13 Framework Route Detection

Language support (via tree-sitter grammars):

TypeScript, JavaScript, Python, Go, Rust, Java, C#, PHP, Ruby, C, C++, Swift, Kotlin, Dart, Svelte, Vue, Liquid, Pascal/Delphi, Scala

Framework route detection is a differentiating feature — CodeGraph doesn’t just recognize symbols, it understands the mapping between URL routes and their handler functions:

# Django
urlpatterns = [
    path('users/<int:pk>/', UserDetailView.as_view()),
]
# → CodeGraph knows GET /users/{id}/ maps to UserDetailView

# FastAPI
@app.get("/items/{item_id}")
async def read_item(item_id: int):
    ...
# → CodeGraph knows GET /items/{id} maps to read_item()

The 13 supported frameworks: Django, Flask, FastAPI, Express, NestJS, Laravel, Rails, Spring, Gin/chi/gorilla/mux, Axum/actix/Rocket, ASP.NET, Vapor, React Router/SvelteKit.

This means AI agents can ask “Where is the handler for /api/users/:id?” and get a precise answer, without needing to scan routing config files.

4. codegraph affected — Smart CI Test Selection

An underappreciated feature: by tracing import dependencies, it identifies which test files are actually affected by changed source files.

# CI scenario: only run tests affected by this change
git diff --name-only | codegraph affected --stdin

# Manually specify changed files
codegraph affected src/auth.ts

# With filter (only e2e tests)
codegraph affected src/auth.ts --filter "e2e/*"

How it works:

Changed: src/auth.ts
  ↓ CodeGraph queries the dependency graph
  Direct importers: user.service.ts, auth.controller.ts
  Indirect importers: app.module.ts, integration.test.ts
  ↓ Filter to test files only
  Affected tests: auth.spec.ts, user.service.spec.ts, integration.test.ts
  ↓ Output
  [these files] ← run only these, not the full test suite

On large projects, this can compress CI test time from tens of minutes to a few minutes.

5. Configuration and Performance Notes

Project config file (.codegraph/config.json):

{
  "version": 1,
  "languages": ["typescript", "javascript"],
  "exclude": ["node_modules/**", "dist/**", "build/**", "*.min.js"],
  "maxFileSize": 1048576,
  "extractDocstrings": true,
  "trackCallSites": true
}

SQLite backend selection:

CodeGraph ships with two SQLite backends:

• Native better-sqlite3 (default, recommended): High performance, supports concurrent reads
• WASM fallback: Better compatibility, but 5–10x slower than native, and concurrent operations may produce database is locked errors

If you encounter performance issues or lock errors:

# Rebuild the native module
npm rebuild better-sqlite3

# Check which backend is active
codegraph status

CLI Reference

codegraph                        # Run interactive installer
codegraph init [path]            # Initialize in a project
codegraph uninit [path]          # Remove CodeGraph from a project
codegraph index [path]           # Full index (--force to rebuild)
codegraph sync [path]            # Incremental update
codegraph status [path]          # Show statistics
codegraph query <search>         # Search symbols
codegraph files [path]           # Show file structure
codegraph context <task>         # Build AI context for a task
codegraph affected [files]       # Find affected test files
codegraph serve --mcp            # Start MCP server

Library API (embed CodeGraph in your own tools):

import CodeGraph from '@colbymchenry/codegraph';

const cg = await CodeGraph.init('/path/to/project');

// Full index with progress callbacks
await cg.indexAll({
  onProgress: (p) => console.log(`${p.phase}: ${p.current}/${p.total}`)
});

// Search symbols
const results = cg.searchNodes('UserService');

// Get call chain
const callers = cg.getCallers(results[0].node.id);

// Build AI context
const context = await cg.buildContext('fix login bug', {
  maxNodes: 20,
  includeCode: true
});

// Impact radius analysis (depth 2)
const impact = cg.getImpactRadius(results[0].node.id, 2);

cg.watch();   // Start file watching for auto-sync
cg.close();   // Clean up resources

Project Links & Resources

Official Resources

• 🌟 GitHub: https://github.com/colbymchenry/codegraph
• 📦 npm: @colbymchenry/codegraph
• ⚡ Quick install: npx @colbymchenry/codegraph

Target Audience

• Heavy Claude Code / Cursor users: Working on large projects and looking to reduce cost and improve response speed
• Large TypeScript/Rust/Python project developers: Codebases large enough that AI agent file-scanning overhead is noticeable
• CI/CD engineers: Using codegraph affected for smart test selection to eliminate unnecessary full test runs
• Toolchain developers: Embedding code semantic analysis into their own tools via the Library API

Summary

Key Takeaways

1. Core value: Inserts a pre-built semantic index between AI agents and codebases — average 35% cost savings, 70% fewer tool calls, 49% speed improvement
2. Technology choices: tree-sitter (AST parsing) + SQLite FTS5 (full-text search) + native OS file events (live sync) — zero external service dependencies
3. 8 MCP tools: codegraph_context is the most critical — one call returns a complete context package for the task at hand
4. 19 languages + 13 framework route detection covering mainstream development stacks
5. codegraph affected: dependency-traced smart test selection, a CI acceleration tool
6. Gains scale with codebase size: Tokio (Rust, 700 files) reaches 89% fewer tool calls; small Go projects see ~19%

One-Line Review

CodeGraph does something deceptively simple yet extremely practical: it converts the code discovery work that AI agents redo on every task into a reusable local index — not a feature addition, but a workflow architecture optimization.

Find more useful knowledge and interesting products on my Homepage

A reply from Claude came back nonsense. The system prompt looked fine in my code. The messages looked fine in my logs. So I added a print(messages) right before client.messages.create(...). Still fine.

I was looking in the wrong place. The SDK was building the request body. What hit the wire was not what I was printing.

So I wrote a httpx transport that intercepts the outbound request, dumps the actual JSON, and lets me diff what I think I sent against what I actually sent. I called it agenttap.

The thing I missed

Here is the captured request for a call I thought was a clean two-turn conversation:

{
  "model": "claude-opus-4-7",
  "max_tokens": 1024,
  "system": "You are a careful code reviewer.",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Review this diff:nn```

diffn+ foon

```"}
      ]
    },
    {
      "role": "assistant",
      "content": "Looks fine to me."
    },
    {
      "role": "user",
      "content": "What about edge cases?"
    }
  ],
  "metadata": {"user_id": "u_8821"}
}

Three things I did not write:

1. The first user message got wrapped in a [{"type": "text", "text": ...}] block. My code passed a plain string. The SDK normalized it.
2. The triple-backtick code block in the diff was preserved literally, including the language tag. I had a helper that was supposed to strip those.
3. The metadata.user_id was leaking from a default I set in the client constructor six commits ago and forgot.

None of these would have shown up in logs of my own variables. They only show up at the wire.

What agenttap does

pip install agenttap. Then:

import httpx
from agenttap import TapTransport
from anthropic import Anthropic

tap = TapTransport(wrap=httpx.HTTPTransport())
client = Anthropic(http_client=httpx.Client(transport=tap))

client.messages.create(
    model="claude-opus-4-7",
    max_tokens=256,
    messages=[{"role": "user", "content": "hi"}],
)

for record in tap.records:
    print(record.request_json)
    print(record.response_status, record.duration_ms, "ms")

The transport sits underneath the SDK. It does not know or care that this is Anthropic. It works the same way with OpenAI’s Python SDK, with the Google client, with anything that ends up calling httpx.

It captures four things per call:

• The full request URL and headers (with authorization and x-api-key redacted to ***).
• The exact request body bytes (decoded as JSON if possible).
• The response status, duration in milliseconds, and a copy of the response body.
• A monotonic sequence number so you can sort across coroutines.

Replay and diff

The reason I built this was not just to look. I wanted to replay.

from agenttap import replay

# Pin a captured request as a fixture
tap.save("fixtures/review_call.json")

# Later, replay the exact bytes
resp = replay("fixtures/review_call.json", api_key=os.environ["ANTHROPIC_API_KEY"])

And diff two recordings:

from agenttap import diff_records

a = tap.records[0]
b = tap.records[1]
print(diff_records(a, b))
# - messages[0].content[0].text: "Review this diff:..."
# + messages[0].content[0].text: "Review this PR:..."
# - metadata.user_id: "u_8821"
# + metadata.user_id: "u_4410"

That diff caught a regression last week where a prompt template change added a stray newline. The output still looked plausible, but the deterministic eval drifted. The wire diff showed me the exact byte.

Numbers

Overhead per call when capturing in memory: about 0.4 ms on my laptop for a 2 KB body. The transport buffers the response body so streaming is slightly different. If you want to keep streaming responses streaming, pass capture_response_body=False and only the request side is recorded.

Default ring buffer holds the last 500 records. You can flush to disk with tap.save_all(dir="taps/") and rotate.

What this does not solve

A few honest limits.

• It only sees the wire. If your SDK retries internally and you only configured one logger, you get every retry. That can be noisy.
• It does not redact PII out of the message body. Only credentials in headers. If you are sending user data, you still need a redaction pass before persisting.
• Streaming responses are captured as the joined output, not the SSE event stream. If you need event-level traces, use claude-stream-rs or wire a separate handler.
• The redaction list is conservative. If your provider uses a custom auth header name, you need to add it: TapTransport(redact_headers=["x-my-auth"]).

If you are running agents in production and you have never looked at the literal JSON that left your process, do it once. You will find something.

Repo: https://github.com/MukundaKatta/agenttap
PyPI: pip install agenttap

This is one of a small set of focused libraries I publish for AI agent plumbing (snapshots, budgets, drift, repair). Built piece by piece from real incidents.

Spent the last day redesigning my github + portfolio until they finally started feeling like me instead of another developer template pretending to have personality.

dark interfaces.

tiny details nobody notices consciously.

machine learning mixed with interaction design.

and a slightly unhealthy obsession with atmosphere.

currently somewhere between:

• frontend engineering
• machine learning
• and “what if software felt haunted in a good way?”

portfolio:

:contentReference[oaicite:0]{index=0}

github:

:contentReference[oaicite:1]{index=1}

still evolving, but i think it finally feels alive now.

It was 11:47 PM on a Thursday when the Slack messages started rolling in.

“Hey, the checkout page looks broken.”

“Is the site down? I’m seeing a blank screen.”

“Tried three different browsers, same thing.”

I pulled up our uptime monitor. Green. Every check passing. Response times normal. HTTP 200 across the board. By every metric the monitor tracked, the site was healthy.

It was not healthy.

The checkout page – the single most important page in the entire application – was rendering a completely white screen for every user. Had been for about 40 minutes. Our monitoring had no idea.

That night cost us somewhere around four hours of investigation, a patch at 2 AM, and a post-mortem I’d rather forget. The root cause was a JavaScript runtime error triggered by a third-party A/B testing script that loaded asynchronously, after our monitor’s simple HTTP check had already returned 200 and moved on.

The monitor was doing exactly what it was designed to do. That was the problem.

What a ping monitor actually checks

Let’s be precise about what most uptime monitoring tools do, because I think a lot of developers have a fuzzy mental model here.

A basic uptime monitor makes an HTTP request to your URL. It checks:

1. Did the server respond?
2. Was the status code in the 2xx range?
3. Was the response time under some threshold?

That’s it. Some monitors add a string match – “check that the response body contains the word ‘homepage'” or whatever. That’s slightly better. But not by much.

The thing is, your users don’t care about any of that. They care whether the page works – whether the content they expect is visible, the buttons they need to click are there, and the thing they’re trying to do is actually possible.

An HTTP 200 response with a blank body is still an HTTP 200. A page that returns your shell HTML but fails to mount any React components is still an HTTP 200. A cached CDN response that’s three weeks stale is still an HTTP 200.

The failure modes nobody warns you about

Here are the actual production incidents I’ve seen (or personally caused) that a ping monitor will completely miss.

JavaScript crash on load

Your server renders the page and sends back valid HTML. But somewhere in the client-side bundle, there’s an unhandled exception – a null reference, a failed import, an API that returned an unexpected shape. The page stays blank, or partially rendered, or stuck in a loading state. The server was fine. The HTTP response was fine. The user experience was not fine.

These are especially nasty because they often only affect certain browsers, certain screen sizes, or users in certain states (logged in vs. logged out, items in cart vs. empty cart).

CDN serving stale content

You deployed a fix. Your origin server is running the new code. But your CDN edge nodes are still serving the old, broken version to 80% of your users because cache invalidation didn’t propagate correctly, or someone forgot to add a cache-busting header, or the CDN’s purge API returned 200 but didn’t actually do the thing.

Your monitor hits the origin. Clean. Users hit the CDN edge. Broken.

React/Next.js hydration failure

You’re server-side rendering a React app. The server sends down HTML, which looks great in your monitor’s response check. Then the client-side JavaScript tries to “hydrate” that HTML – attach event listeners, reconcile state – and something goes wrong. The page looks rendered, but nothing is interactive. Buttons don’t respond. Forms won’t submit. The hydration error is sitting in the browser console where nobody ever looks.

An A/B test or feature flag went sideways

Someone enabled a new variation in your experimentation platform. It works fine for 90% of users. For 10% – the control group, or a specific segment, or users with a certain cookie – it injects a script that breaks layout, hides a critical element, or throws an error. Your monitor isn’t in that segment. Your monitor sees the happy path.

The checkout button just… isn’t there anymore

A CSS change. A template logic error. A component that conditionally renders based on some state that’s slightly wrong. Your add-to-cart button, your checkout button, your submit form – just gone. The page looks fine at a glance. The hero image is there, the nav is there, the product photos are there. But the one element users actually need to convert? Missing.

A monitor checking for HTTP 200 and a response time under 2 seconds has no idea.

Why visual monitoring is conceptually different

The insight is simple: instead of asking “did the server respond?”, ask “does the page look right?”

Visual monitoring takes a screenshot of your page – using a real browser, running real JavaScript, waiting for render to complete – and compares it against a known-good baseline. If the current screenshot differs from the baseline beyond some threshold, something has changed and you get an alert.

This catches all the scenarios above because it’s evaluating the end result that users actually see, not an intermediate HTTP handshake that happens before any of the interesting rendering work occurs.

The diff approach is also useful beyond pure “is it broken” detection. It catches:

• Layout shifts you didn’t intentionally make
• Text that changed in a way it shouldn’t have (wrong price, wrong copy)
• Elements that disappeared
• New elements that appeared (a cookie banner blocking content, an error message you didn’t know was showing)
• Visual regressions from a deploy that seemed fine but quietly broke something

The key is the baseline. You take a screenshot when everything is known to be working, mark it as the baseline, and then every subsequent check compares against that. When the diff exceeds your threshold, alert.

The alert should include the diff image – not just “something changed,” but a visual showing exactly what changed, highlighted. That’s the thing that makes it immediately actionable instead of just alarming.

The practical gotchas

Visual monitoring isn’t magic, and there are real tradeoffs.

Dynamic content. If your page shows the current time, a live stock price, or a user-specific greeting, those will show up as diffs on every single check. You need to either mask those regions, or structure your pages so dynamic content is in predictable locations you can exclude.

Rendering variability. Fonts rendering slightly differently on different machines, antialiasing differences, animated elements caught mid-transition – all of these can cause false positives if your diff threshold is too sensitive. You need a threshold that’s high enough to ignore noise but low enough to catch real problems. Getting this calibrated takes some tuning.

Auth-gated pages. If you want to monitor pages behind a login, you need to handle authentication in your screenshot flow. This is doable – you can script login sequences – but it adds complexity.

Cost of headless Chrome. Running Chrome instances at scale is not free. It’s memory-hungry, and taking screenshots takes real time compared to a simple HTTP request. A 1-minute check interval for many URLs is meaningfully more expensive to operate than a ping monitor.

These are all solvable problems, but they’re real, and anyone selling you a visual monitoring solution that doesn’t mention them is glossing over the hard parts.

Where this leaves you

I’m not saying throw out your existing uptime monitor. Ping monitors are cheap, fast, and useful for catching the simplest failures – server down, TLS expired, DNS broken. Keep them.

But treat them for what they are: a necessary but not sufficient condition for “the site is working.” The gap between “the server responded with 200” and “users can actually use this page” is where a lot of production incidents live, and most teams don’t find out about those incidents until users tell them.

After the Thursday night incident I described at the start, I spent a weekend investigating visual monitoring options and ended up down a rabbit hole of how headless Chrome screenshot diffing actually works. None of the existing tools felt quite right – either too expensive, too complex to configure, or not quite targeted at the “is this page visually broken” question.

So I built GrabDiff. It takes screenshots of your URLs on a schedule using headless Chrome, diffs them against a stored baseline, and emails you with the diff image attached when something looks off. Free plan covers three monitors, no credit card required.

Your uptime monitor is probably lying to you right now. Hopefully about something minor.

Everybody in AI suddenly started talking about MCP like it’s obvious.

One week it was AI agents. Then workflow automation. Then Claude Desktop integrations. Then Cursor and Windsurf started pushing MCP support everywhere. And suddenly developers everywhere started saying:

“Just use MCP.”

But most people still don’t fully understand why this became such a big deal.

Honestly, I didn’t get it immediately either.

Took me forever to understand why MCP matters.

At first, I thought Model Context Protocol was just another fancy AI infrastructure buzzword that would disappear in a few months. But once you see the workflow problems it solves, the hype starts making sense very quickly.

So… What is MCP?

MCP stands for Model Context Protocol.

The simplest explanation?

It’s a standard way for AI systems to connect with tools, apps, files, APIs, databases, and workflows.

Think of it like USB‑C for AI applications.

Before USB‑C, every device needed weird adapters and different connectors. MCP tries to standardize how AI tools communicate with external systems so developers don’t need custom integrations for everything.

That’s why developers using Claude Desktop, Cursor, Windsurf, and VS Code are suddenly paying attention.

Because AI tools are no longer just chatbots.

Now they’re expected to actually do work.

The Real Problem MCP Solves

Here’s the thing.

Most AI workflows today are messy.

You ask an AI assistant to:

read GitHub repositories
check Slack discussions
search internal docs
update Jira tickets
summarize meetings
write deployment notes

…and somewhere in the middle the context breaks.

The AI forgets information.

Tools disconnect.

Workflows restart.

A founder showed me their AI workflow recently. Multiple disconnected tools. Constant context loss. That’s when MCP finally clicked for me.

The issue wasn’t model intelligence.

The issue was coordination.

That’s exactly where MCP enters the picture.

How MCP Actually Works

MCP usually has three parts:

the AI application
the MCP client
the MCP server

The AI application could be Claude Desktop or Cursor.

The MCP client handles communication.

Then MCP servers expose tools and capabilities the AI can use.

For example:

GitHub MCP server
Slack MCP server
database MCP server
filesystem MCP server

Instead of building separate integrations for every AI tool, developers can expose capabilities once through MCP.

Much cleaner.

MCP vs APIs

This is where people get confused.

My mistake was thinking MCP was just another API layer. Completely wrong.

APIs expose functionality.

MCP standardizes how AI systems interact with that functionality.

That distinction matters.

APIs are still important.

MCP sits above them and creates structure for AI workflows.

That’s why AI agents suddenly feel more practical when MCP enters the conversation.

Why Developers Suddenly Care

The timing matters.

AI tools became useful enough for real work.

But workflows became too complicated.

Developers now expect AI systems to:

maintain context
coordinate tools
execute workflows
access repositories
interact with documentation
stay consistent during long sessions

Without MCP, that experience becomes fragmented fast.

That’s why MCP exploded across the developer ecosystem this year.

My Hot Take

Here’s my hot take: most AI agents today are just wrappers until they properly use MCP.

A lot of “AI agents” still fall apart the second workflows become complicated.

Because context continuity is still one of the hardest problems in AI infrastructure.

MCP doesn’t magically solve everything.

But it solves a very real pain point developers were already struggling with.

And honestly, this probably becomes standard infrastructure faster than people expect.

Originally published on FutureDevTech

Read more AI infrastructure and developer workflow articles at: Future Dev Tech

for the past 2 weeks, life has mostly been:

wake up → laptop → confusion → somehow figure things out → repeat

and surprisingly… it’s been working.

i’ve been averaging around 8 actually productive hours almost every day

(except one break day because my brain started sounding like a dying laptop fan)

recently:

• prettified my portfolio + github
• finished my first ML project
• started learning graphs + logistic regression today
• restarted workouts because my wrist is probablyyyyy functional again

lowkey feels nice seeing things compound for once.

if things keep going like this, by next month i should be deep into:

• dynamic programming
• deep learning
• and whatever identity crisis comes with them

still feel lost half the time btw.

just slightly more organized now.