Mert

I was running a multi-agent setup over a weekend. Three workers in parallel, each calling Claude, each with their own retry logic. I woke up on Sunday to a bill alert.

Forty bucks. Eighteen minutes. One worker had gotten into a retry loop on a malformed tool response and had hammered the API until I happened to glance at my dashboard.

The annoying part: I had per-call cost logging. Every call printed its USD cost. I just had no shared cap across the three workers. Each one thought it was being polite by capping at $5. Three workers, three caps. The actual ceiling was $15+ per minute and nothing stopped them.

So I built token-budget-pool. One shared atomic counter that all workers check before every call.

The API

use token_budget_pool::Budget;

let budget = Budget::new_usd(5.00);  // $5 cap across all consumers

let cost_estimate = 0.018;  // claude sonnet on a short prompt
budget.reserve(cost_estimate)?;  // errors if pool exhausted

let actual_cost = call_claude(...)?;
budget.commit(actual_cost)?;  // adjusts the running total

reserve and commit are the two-phase part. You reserve before the call so concurrent workers cannot all squeeze through at once. You commit after with the actual cost from the API response (input tokens + output tokens + cache reads + cache writes).

If the budget is blown, reserve returns BudgetExceeded and the worker can decide to back off, page someone, or just exit clean.

Why two phase

The single-phase version (just record(cost) after the call) loses the race. Two workers can both check the budget, both see room, both call the API, both blow past the cap.

With reserve-then-commit, the reserve is atomic. The commit just adjusts up or down to match the real cost. If the actual cost was lower than the estimate, the pool gains a tiny bit back. If it was higher (long generation), the pool loses extra and the next reserve fails sooner.

let res = budget.reserve(0.02);
if res.is_err() {
    eprintln!("budget gone, this worker is done");
    return Ok(());
}

let actual = call_llm(...).await?;
budget.commit(actual)?;

Python port

I ported it to Python because most of my agent code is Python and the import was annoying via FFI for this kind of thing.

from token_budget import Budget

budget = Budget.new_usd(5.00)

try:
    budget.reserve(0.018)
except BudgetExceededError:
    print("budget gone")
    return

cost = call_claude(...)
budget.commit(cost)

Same shape. 18 tests cover the threading edge cases. Locks are released on context exit.

Time windows

Just a static cap is not enough. You want “no more than $20 per hour” or “no more than 200k tokens per minute” so a slow drip does not still blow your budget over a day.

That is what llm-budget-window does. Same author. It tracks multiple windows at once.

use llm_budget_window::WindowedBudget;
use std::time::Duration;

let budget = WindowedBudget::builder()
    .add_window(Duration::from_secs(60), 100_000)        // 100k tokens/min
    .add_window(Duration::from_secs(3600), 5_000_000)    // 5M tokens/hour
    .build();

budget.record(tokens_used)?;  // errors if either window exceeded

Records are atomic across all windows. If you blow the minute window first, you get told which window. The day window still has room but the agent backs off for the minute.

What it does not do

It does not bill you. It does not refund you. It is a pre-call cap. If the upstream API charges you for the failed retry, you still pay. The point is to stop the next call.

It also does not understand model-specific pricing. You pass cost estimates in. I have a separate crate (claude-cost, openai-cost, bedrock-cost) for the per-model math. Composing them is two lines:

let est = claude_cost::estimate(model, input_tokens, max_output_tokens);
budget.reserve(est)?;

The lesson

Per-call cost logging is necessary but not sufficient. If you have two or more workers, or any retry loop, you need a shared cap. The cost of writing one is an afternoon. The cost of not having one was $40 in my case. Probably more for whoever wakes up to a four-figure alert.

Repos:

• crates.io: https://crates.io/crates/token-budget-pool
• PyPI: https://pypi.org/project/token-budget-py/
• GitHub: https://github.com/MukundaKatta/token-budget-pool

One shared cap. Two-phase. Sleep better.

A RAG system I built was scoring 92% on our eval set in week one. Three weeks in, same eval set, same code, it was at 78%. Nothing had been deployed. Nothing had been re-indexed. The embeddings file was byte-identical.

So what changed?

This is the part of RAG that nobody talks about. Quality decays even when you do nothing, because the inputs are not the thing you measured. They are the thing your users are typing right now.

I built driftvane to find which signal was moving. It tracks five dimensions of drift in one report.

The five dimensions

After a few false starts (I started with just embedding drift, which is not enough), I landed on these:

1. Data drift: are the documents you index changing under you? (Includes silent index rebuilds.)
2. Embedding drift: is the distribution of your embeddings moving? Different distribution, different neighbors.
3. Query drift: are users asking different kinds of questions than they did at baseline?
4. Response drift: are answers getting longer, shorter, more uncertain, more confident?
5. Confidence drift: is the model’s self-reported confidence changing on similar questions?

A single number tells you “something is off”. The five tell you “your users started asking longer questions and the model is now hedging more”.

What the API looks like

from driftvane import DriftDetector

detector = DriftDetector.load("baseline.snapshot")

report = detector.check(
    queries=recent_queries,
    retrieved_docs=recent_retrieved,
    responses=recent_responses,
    confidences=recent_confidences,
)

print(report.summary())

Output looks like this:

DriftReport
  data:       OK     (psi=0.04)
  embedding:  OK     (psi=0.07)
  query:      DRIFT  (psi=0.31)   <-- here
  response:   DRIFT  (length +38%, hedging +22%)
  confidence: OK     (mean shift 0.02)

PSI is Population Stability Index. Anything above 0.25 is “you should look at this”. Anything above 0.5 is a fire.

What it actually found

For my 92 -> 78 case, the report said:

• Query drift HIGH. Mean query length up 60%.
• Response drift HIGH. Hedging phrases up 45%.

That was the whole answer. Users had started asking longer, multi-part questions (“compare A and B and also tell me about C”). My retriever was tuned for short queries. It was pulling the top 3 chunks for “A” and missing B and C entirely.

The fix had nothing to do with the model. I split long queries into sub-queries before retrieval. Accuracy went back to 91%.

The detector you actually want

The naive way to do this is k-means on embeddings and watch the cluster sizes. That catches a tiny fraction of real drift. The five-dimensional view is more useful because each dimension is independently diagnostic.

If response length moves but queries do not, your model is the culprit (upgrade? prompt edit?).

If queries move but responses do not, your model is being asked things it cannot answer well and is generating plausible-sounding-but-wrong content. Worst case.

If embeddings move but queries do not, your embedding provider silently changed the model. (Yes, this has happened.)

If data moves and you did not deploy, somebody else is writing to your index.

If confidence drifts but nothing else does, the model is having a bad week. Watch it.

Snapshots are the trick

The whole thing is built around snapshots. A snapshot is a fingerprint of your baseline: histograms, length distributions, a sample of embeddings, summary stats. It is small (a few MB for most setups).

from driftvane import build_snapshot

build_snapshot(
    queries=baseline_queries,
    retrieved_docs=baseline_docs,
    responses=baseline_responses,
    out="baseline.snapshot",
)

You run this once when your eval looks good. Then you can compare any future window against it. The detector does not need the raw baseline data again, just the snapshot.

What it does not do

It will not tell you the answers are wrong. It tells you the inputs and outputs are different. The “is it correct” question still needs your eval set.

Also: small windows are noisy. I would not trust a drift report on fewer than 200 queries. The library prints a warning if you try with less.

One library

PyPI only for now: pip install driftvane. The Rust port is on my list but Python is where my RAG code lives.

Repo: https://github.com/MukundaKatta/driftvane

If your RAG quality is mysteriously decaying, the answer is almost never the model. It is one of these five signals. Five histograms is a cheap way to find which one.

I shipped a small agent that books meetings. It calls three tools in order: search_calendar, find_slot, create_event.

The unit tests all passed for a year. Then I bumped the model from claude-3-5-sonnet to claude-3-7-sonnet and the booking flow silently broke. Bookings still happened. They just went to the wrong day half the time.

The root cause: the new model started calling find_slot(duration_minutes=30, attendees=[...]) instead of find_slot(attendees=[...], duration_minutes=30). My tool was positional under the hood and the first arg was the date. Whoops.

Unit tests passed because I mocked find_slot and asserted on the final reply text. The reply still said “Booked for Tuesday at 3pm”. Just for the wrong Tuesday.

I wanted a test that pinned the actual sequence of tool calls and their arg shape. That is AgentSnap.

The shape of a snapshot

A snapshot is a JSON file with one entry per turn: the model decision, the tools it called, the args, and the tool result shape. Not the full free-text response. Just the trace.

import { snapshot } from "@mukundakatta/agentsnap";

test("books a 30 minute coffee", async () => {
  const trace = await runAgent({
    user: "book a 30 min coffee with Sam next Tuesday",
  });
  snapshot(trace, "book_coffee.snap");
});

The first run writes the file. Future runs compare against it. A diff is a hard fail unless you re-record with AGENTSNAP_UPDATE=1.

The snap file looks like this:

[
  { "tool": "search_calendar", "args": { "user": "Sam" } },
  { "tool": "find_slot", "args": { "attendees": ["Sam"], "duration_minutes": 30 } },
  { "tool": "create_event", "args": { "title": "Coffee", "duration_minutes": 30 } }
]

When the model started calling find_slot with reordered args, the diff was loud:

- "args": { "attendees": ["Sam"], "duration_minutes": 30 }
+ "args": { "duration_minutes": 30, "attendees": ["Sam"] }

OK, key order is not actually a problem there. But when the model started passing {"duration": 30} instead of {"duration_minutes": 30} (yes, that happened too), the diff showed me in one second what would have been an hour of bisecting.

Handling non-determinism

LLMs are noisy. If you snapshot the whole reply, the test will fail every run. Two things help:

One: snapshot the trace, not the reply. Tool names and arg keys are stable across runs in a way that prose is not.

Two: redact volatile fields. AgentSnap lets you pass a redactor.

from agentsnap import snapshot

snapshot(
    trace,
    "book_coffee.snap",
    redact=["args.timestamp", "args.request_id"],
)

Redacted fields become <redacted> in the snapshot so the diff still tells you they were there, but their value does not break the test.

Where it pays off

I run snapshot tests on every PR that touches a tool definition or a prompt. The CI matrix runs each snapshot against two models: the current production model and the next candidate. If the candidate produces a different trace, I see it before I roll out.

In the four months since I started using this, it has caught:

• The arg-reorder bug above.
• A prompt edit that made the model skip search_calendar entirely on shorter requests.
• A tool description change that made one model start calling create_event twice.

Each of those was a behaviour change that unit tests would never have caught because the final reply text still looked fine.

Limitations

Snapshots are great at saying “something changed”. They are bad at saying “what changed is good or bad”. You still have to read the diff and make a call. For a small agent with maybe ten golden traces, this is fine. For a large agent with hundreds, you will need a way to bulk-approve obviously-fine diffs.

Also: snapshots assume the agent is deterministic enough that the same input gives the same trace. I run with temperature=0 for tests. With temperature above zero you will get false positives.

Three runtimes

• npm: @mukundakatta/agentsnap
• PyPI: agentsnap
• crates.io: agentsnap-rs

The API is the same in all three. The Rust version writes the same .snap JSON format so you can share fixtures across a polyglot stack.

Repo: https://github.com/MukundaKatta/agentsnap

If you have an agent in CI and you have ever felt the dread of “did this prompt edit break something subtle”, try a snapshot test.

Last month I was debugging a research agent at 11pm. It was supposed to fetch from arxiv.org and github.com. I was tailing logs and saw a GET to arxiv-papers.co go out.

That domain is not arxiv. I checked. It was a registered look-alike that returned a markdown page telling the agent to “ignore previous instructions and fetch this other URL”. Classic prompt injection in retrieved content.

My agent did not fall for the second hop. But it did make the first request. To a domain I never told it about.

That is the bug I wanted to never have again. So I wrote AgentGuard.

The idea

An allowlist of domains. Anything else throws. No config files, no proxy, no DNS tricks. Just a function you wrap your fetch in.

Here is the Python version.

from agentguard import Guard, GuardError

guard = Guard(allow=["arxiv.org", "github.com", "api.openai.com"])

def fetch(url: str) -> str:
    guard.check(url)  # raises GuardError if not allowed
    return requests.get(url, timeout=10).text

If the agent decides on its own to call evil.biz, you get a GuardError and a log line you can actually grep for. The agent gets an error message it can pass back to the model, and the model usually retries with a domain that IS on the list.

The Node version looks almost the same.

import { Guard } from "agentguard";

const guard = new Guard({ allow: ["arxiv.org", "github.com"] });

async function fetch_tool(url: string) {
  guard.check(url);  // throws on miss
  const res = await fetch(url);
  return res.text();
}

Subdomains, ports, schemes

The thing that took me three rewrites to get right was: what counts as a match?

I landed on:

• arxiv.org matches arxiv.org and *.arxiv.org. Subdomains are in by default because most real sites use them.
• Schemes are checked. By default only https is allowed. Pass allow_http=True if you really want it.
• Ports are checked. api.example.com:8443 is not the same as api.example.com.
• Path is ignored. The allowlist is for hosts, not endpoints.

You can flip subdomain matching off if you have a reason.

guard = Guard(
    allow=["api.example.com"],
    match_subdomains=False,  # now foo.api.example.com is blocked
)

What happens when it fires

The block is loud on purpose. I want to see it in the logs without grepping. The error includes the attempted host, the rule list, and (if you turn it on) a stack hint so you can find which tool tried.

GuardError: host 'arxiv-papers.co' not in allowlist
  allow: ['arxiv.org', 'github.com']
  called from: research_agent.fetch_paper

In one CI run I caught a typo where someone wrote huggigface.co (missing letter). The test that tried to hit it failed cleanly with the block message. Before AgentGuard the same typo would have been a 404 from a parked domain, which is harder to spot.

What it is not

This is not a WAF. It is not a sandbox. A determined attacker who can run arbitrary code on the same process can disable the guard. The threat model I care about is: my own LLM agent, running my own tools, deciding on its own to call a host I did not approve. That covers most of the prompt-injection-via-content cases I worry about in practice.

It also does not do DNS. The check is on the hostname string the agent passes in. If you let the agent supply raw IPs, add IP rules too, or resolve first and check the result.

Three flavors

I shipped this in three runtimes because I keep switching between them:

• npm: agentguard
• PyPI: agentguard-py
• crates.io: agentguard-rs (with a reqwest-middleware feature so the check fires automatically on every outbound request)

The Rust one is the version I reach for first now because the middleware bit means I cannot forget to wrap a tool.

Wrap

Cost was small. Maybe an afternoon to write, a few days to use across my agents. The payoff is one less category of incident I have to think about.

Repo: https://github.com/MukundaKatta/agentguard

If you run agents that have any kind of fetch tool, having a domain allowlist as the floor is cheap insurance.

In the rapidly evolving landscape of agentic AI, the ability to build and deploy intelligent agents depends on a data foundation that is flexible, scalable, and extremely fast.

We invite you to our upcoming webinar series in EMEA, Beyond the Query: Powering AI Agents with Bigtable, Firestore & Memorystore, taking place from June 22-26, 2026.

Why Attend?

Modern AI applications require more than just storage; they need the throughput and latency to support real-time reasoning and large-scale agents. In these technical sessions, we will explore:

Accelerating AI with Firestore: Learn how to reimagine MongoDB workloads for AI by migrating them to Firestore without complex refactoring.

Ultra-Low Latency with Memorystore: Discover the engine for next-gen applications that require sub-millisecond performance for AI context and caching.

High Throughput & Scale with Bigtable: See why industry leaders like Anthropic and Spotify rely on Bigtable’s massive scale and throughput for model training and inference.

The Agentic Data Cloud: Understand the power of integrating Bigtable with BigQuery and Gemini to build sophisticated AI agents.

Event Details:

Register Here: Link to RSVP
Dates: June 22 – June 26, 2026

Whether you are looking to migrate existing NoSQL workloads or launch entirely new AI-powered services, this series will help you understand our NoSQL database services to succeed in the Agentic Era.

We look forward to seeing you there.

Why I Chose Hard Math Over AI for a 38,000 Ticker Financial Engine

In 2026, the pressure to “AI-wash” every product is immense. But when you are building a financial tool designed to project wealth over a 30-year horizon, “probabilistic” is just a fancy word for “wrong.”

I recently shipped DividendFlow, a tax-aware compounding engine covering 38,000+ US tickers. Here is why I stripped out the AI agents and went back to strictly deterministic TypeScript logic.

The 0.1% Compounding Nightmare

In dividend growth investing, accuracy isn’t a feature—it’s the entire product. If an LLM “hallucinates” a dividend yield or miscalculates a tax bracket by just 0.1%, the error compounds. Over 20 years, that tiny glitch turns into a $50,000 discrepancy in the user’s projected nest egg.

To solve the “Gross Yield Lie,” we had to build a system that handles:

1. Recursive DRIP Math: Calculating reinvestment on a net-of-tax basis every quarter.
2. Tax Jurisdictions: Real-time mapping of US Federal/State, UK ISA, and Canadian TFSA rules.
3. Data Normalization: Cleaning noisy payout history from institutional APIs.

The Tech Stack: Next.js 15 & Server-Side Math

We moved the core calculation engine to Next.js 15 Server Components. By doing the heavy lifting at the Edge/Server level, we achieved:

• Sub-200ms Renders: Even with 30 years of monthly recursive calculations.
• Zero Client-Side Bloat: The user’s browser only receives the final coordinates for the “Snowball Effect” graph.
• Deterministic State: No “vibe-coding.” The math is immutable and verifiable.

Why “No-Login” is a Technical Moat

Most fintech SaaS tools force a Yodlee/SnapTrade integration. It’s great for data harvesting, but terrible for user friction.

We decided on a “Utility-First” architecture:

• No Database: User scenarios are encoded in URL parameters and LocalStorage.
• Privacy by Design: We don’t want your bank credentials. We just want to give you the math.
• Scale: By avoiding a centralized DB for user profiles, the app scales horizontally without the traditional RDS bottlenecks.

Lessons from 38,000 Tickers

Indexing the entire US market taught me that data integrity is 90% of the work.
Standard APIs are messy. They report special dividends as regular yield, which fakes “Financial Freedom” dates. Our normalization layer acts as a “Truth Filter,” ensuring that the compounding curve reflects recurring reality, not one-time spikes.

Conclusion

AI is great for writing boilerplate, but it’s a liability for domain logic in finance. If you’re building tools where the output must be 100% predictable, stick to the hard math.

Test the deterministic engine at DividendFlow.org →

Are you building “Utility-First” apps? Let’s discuss the trade-offs of no-login architectures in the comments.

There’s a window between when a Next.js page finishes painting and when users can actually interact with it. Buttons look clickable. Links look followable. Nothing responds. The page is there — the HTML arrived, the browser rendered it — but the JavaScript hasn’t finished running yet.

That window is hydration. And most of the JavaScript your app sends to the browser exists to get through it.

What hydration costs

The server renders HTML and sends it. The browser paints it immediately — that’s your LCP. Then the browser downloads your JS bundle, parses it, compiles it, and runs it. React walks the entire DOM, compares it to the virtual DOM, and attaches event listeners to every interactive element. Only then is the page actually usable.

The time between LCP and Time to Interactive is the hydration tax. On a fast connection and a modern laptop it’s barely noticeable. On a mid-range Android phone on 4G, it can be two to four seconds.

To see your own numbers: in Chrome DevTools Performance panel, record a page load and find the gap between the LCP marker and the point where long tasks stop. That gap is roughly what hydration is costing your users.

The use client boundary problem

Next.js App Router lets you keep components on the server by default. The moment you add use client, that component and everything it imports gets bundled for the client and hydrated.

The mistake I see most often is adding use client to a layout or wrapper component because one child needs it:

// This ships the entire subtree to the client
'use client';

import { Header } from './Header';
import { StaticContent } from './StaticContent'; // doesn't need to be a client component
import { InteractiveWidget } from './InteractiveWidget'; // this is the only reason for 'use client'

export function PageLayout({ children }) {
  return (
    <>
      <Header />
      <StaticContent />
      <InteractiveWidget />
      {children}
    </>
  );
}

StaticContent and Header are now client components. They’ll be included in the JS bundle and hydrated, even if they never use state, effects, or event handlers.

The fix is to push use client as far down the tree as possible — to the component that actually needs it:

// Server component — no bundle cost, no hydration
import { Header } from './Header';
import { StaticContent } from './StaticContent';
import { InteractiveWidget } from './InteractiveWidget'; // 'use client' lives inside this file

export function PageLayout({ children }) {
  return (
    <>
      <Header />
      <StaticContent />
      <InteractiveWidget />
      {children}
    </>
  );
}

InteractiveWidget still ships to the client because it needs to be interactive. But Header and StaticContent stay on the server. Their HTML arrives pre-rendered and nothing hydrates them.

This is the single highest-leverage change in most Next.js App Router migrations I’ve seen. Auditing use client placement before reaching for any other optimization is worth doing first.

Finding what’s actually in your bundle

The Next.js bundle analyzer shows what’s in each chunk. Add it to next.config.js:

const withBundleAnalyzer = require('@next/bundle-analyzer')({
  enabled: process.env.ANALYZE === 'true',
});

module.exports = withBundleAnalyzer({});

Run ANALYZE=true npm run build and look at the client bundle. Two things to look for: large dependencies that are imported inside use client components but don’t actually need browser APIs, and server utilities that accidentally got pulled into the client graph.

A date formatting library that ended up in the client bundle because it was imported in a use client component that could have been a server component — I’ve found this more than once. Moving the component to the server removes the library from the client bundle entirely.

Code splitting reduces bundle size, but it doesn’t help if what you’re splitting is still fully hydrated. Smaller chunks load faster; that’s real. But if every chunk is loaded and hydrated on every page, the gain is limited.

Deferring hydration for below-the-fold content

Content the user can’t see immediately doesn’t need to be hydrated immediately. React’s lazy with Suspense defers both the download and the hydration:

import { lazy, Suspense } from 'react';

const HeavyDashboardWidget = lazy(() => import('./HeavyDashboardWidget'));

export function Dashboard() {
  return (
    <div>
      <AboveTheFoldContent />
      <Suspense fallback={<WidgetSkeleton />}>
        <HeavyDashboardWidget />
      </Suspense>
    </div>
  );
}

The widget’s JavaScript isn’t downloaded or hydrated until React renders that Suspense boundary. For content far down the page, wrapping it in a lazy-loaded component with an IntersectionObserver trigger defers hydration until it’s actually scrolled into view — though this adds complexity that’s only worth it for genuinely heavy components.

The simpler version — just using lazy without the intersection observer — already helps because the browser prioritizes loading and hydrating above-the-fold content first.

What this looks like in production

LCP and TTI moving in opposite directions is a signal that hydration cost has gotten out of hand. LCP improves because you’re serving fast HTML from the server. TTI stays high because you’re still sending a large client bundle to hydrate it.

Measuring LCP is straightforward. TTI is harder — it’s not a Core Web Vital and it doesn’t appear in CrUX. The proxy metric is INP on early interactions: if users are clicking buttons within the first few seconds of loading and seeing high INP, the page probably wasn’t fully hydrated yet when they tried.

The next article looks at what Server Components actually change about the bundle and hydration picture — and where they don’t help as much as the marketing suggests.