Mert

This is a submission for the Gemma 4 Challenge: Write About Gemma 4

The single most common mistake developers make when picking a local model is choosing based on benchmark scores. The second most common mistake is choosing based on what fits in VRAM.

Both of those things matter. But neither one is the actual first question.

The actual first question is: where does your model need to live, and what does it need to do there?

Gemma 4 ships in four variants – E2B, E4B, 26B A4B (MoE), and 31B – and Google made very deliberate architectural choices for each one. If you understand those choices, picking the right variant takes about five minutes. If you skip that step and benchmark-shop, you’ll end up either underbuilding (a phone-ready E4B doing work that needs 256K context) or overbuilding (a 31B model sitting on $80/month of cloud compute when an E4B running locally would have been fine).

This post is that five-minute decision guide.

What Gemma 4 Actually Is

Released on April 2, 2026 under Apache 2.0, Gemma 4 is Google DeepMind’s latest open-weight model family. Every variant ships with multimodal understanding (text + image as baseline, audio natively on the two smallest models), native function calling, and support for over 140 languages.

The headline capability that separates Gemma 4 from previous generations isn’t any single feature. It’s the intelligence-per-parameter ratio. The 26B MoE model only activates roughly 4B parameters per forward pass. The E4B runs on a phone. The 31B scores 89.2% on AIME 2026 math benchmarks – a score that would have required a model several times larger just a year ago.

The architecture decisions that make this possible:

• Alternating local/global attention layers (local layers use sliding windows of 512-1024 tokens, global layers handle long-range context)
• Per-Layer Embeddings (PLE) on the edge variants, which keeps the parameter count low while maintaining expressivity
• Mixture-of-Experts on the 26B that routes each token through only the relevant expert layers, not the full network

This isn’t just efficiency for efficiency’s sake. It’s what allows a 4-billion-parameter model to run offline on an Android phone with 4GB of RAM while still having a 128K context window. That combination didn’t exist before.

The Four Variants, Actually Explained

Gemma 4 E2B – The Phone Model

~2.3B effective parameters, ~5.1B total with PLE, 35 layers, 128K context

This is the model you reach for when the edge is the deployment target. It runs on Android 12+ via Google AICore, on Raspberry Pi, and on Jetson devices. It supports text, image, and audio natively.

The “E” in the name stands for effective – because PLE means the model has more total parameters than it activates per forward pass, similar to how MoE works at a different level of the architecture. The practical result is a 1.5GB footprint with capabilities that land well above what a raw 2B parameter count would suggest.

Use E2B when: you’re building a mobile app, an edge inference pipeline, a device-local assistant, or anything where network latency or data privacy makes sending requests to a remote API unacceptable.

Real use case: a receipt-scanning expense tracker that runs fully offline, reads image input, parses line items, and categorizes spending – all on device, no API call, no data leaving the phone.

# Running E2B locally with transformers
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "google/gemma-4-E2B-it"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {
        "role": "user",
        "content": "Extract the total amount and vendor name from this receipt text: ..."
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    return_tensors="pt",
    return_dict=True
).to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=256)

response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

Gemma 4 E4B – The Laptop Model

~4.5B effective parameters, ~8B total, 42 layers, 128K context

This is the everyday workhorse for developers who want to run a capable model locally without dedicated GPU hardware. It runs comfortably on a MacBook with 16GB unified memory, on a mid-range laptop with an integrated GPU, and on any machine where you’d rather not spin up a cloud instance.

The jump from E2B to E4B isn’t just more parameters. The additional layers and parameter budget give it noticeably better instruction following, more reliable structured output, and stronger performance on tasks that require holding context across a long conversation.

It supports the same text, image, and audio modalities as E2B, which makes it genuinely multimodal in a way that matters for developer tooling – you can feed it screenshots, diagrams, or audio transcripts as part of a pipeline without needing a separate vision model.

Use E4B when: local inference is the requirement, your hardware doesn’t have a discrete GPU, or you’re prototyping something you’ll later scale to a larger model and want fast iteration cycles.

Real use case: a local code review tool that takes a screenshot of your editor alongside the diff, understands both, and gives context-aware feedback – all running on your laptop, no telemetry.

# Quick Ollama setup for E4B (easiest local path)
# After installing Ollama: https://ollama.com

# In terminal:
# ollama pull gemma4:e4b

import ollama

response = ollama.chat(
    model="gemma4:e4b",
    messages=[
        {
            "role": "user",
            "content": "Review this function for edge cases and suggest improvements:",
        }
    ],
    options={
        "temperature": 0.3,
        "num_ctx": 8192  # can go up to 128K
    }
)

print(response["message"]["content"])

Gemma 4 26B A4B (MoE) – The Consumer GPU Model

25.2B total parameters, ~3.8B active per forward pass, ~30 layers, 256K context

This is the one that makes the architecture story interesting. The 26B MoE sounds like it needs 26 billion parameters worth of compute. It doesn’t. Only about 4 billion parameters activate for each token, which means it runs on a single RTX 3090 or RTX 4090 at full precision while delivering quality that competes with much larger dense models.

The jump to 256K context window is significant for developers. At 128K you can fit roughly a medium-sized codebase or a very long document. At 256K you’re fitting large repositories, multi-document research contexts, or full conversation histories in customer-facing applications.

The MoE architecture also means that quality degrades more gracefully with quantization than a dense model of equivalent total parameters would. INT4 at 26B MoE looks better than INT4 at a comparable dense model.

Use 26B A4B when: you have a consumer GPU (24GB VRAM), need 256K context, and want near-flagship quality without flagship hardware costs. Also the right choice for anything agentic where the model needs to reason across large amounts of context to plan multi-step tasks.

Real use case: an agentic document processor that ingests a full legal contract (or a full codebase) in a single prompt, reasons across the entire document, and extracts structured data or answers specific questions – running locally on a 4090.

# Using the Gemma 4 26B with native function calling
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
import json

model_id = "google/gemma-4-26B-A4B-it"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    load_in_4bit=True  # fits on 24GB with 4-bit quant
)

# Native function calling - define your tools
tools = [
    {
        "name": "search_contracts",
        "description": "Search the contract database by clause type or party name",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Search query"},
                "clause_type": {
                    "type": "string",
                    "enum": ["liability", "termination", "payment", "IP"],
                    "description": "Type of clause to filter by"
                }
            },
            "required": ["query"]
        }
    }
]

messages = [
    {
        "role": "user",
        "content": "Find all termination clauses across the Q1 vendor contracts and summarize the notice periods."
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tools=tools,
    return_tensors="pt",
    return_dict=True
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512)
response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

Gemma 4 31B – The Server Model

31 billion dense parameters, 256K context, full multimodal, thinking mode

This is the flagship. Every capability available in the family is present here. Thinking mode (chain-of-thought reasoning) is enabled. Math benchmark scores are serious: 89.2% on AIME 2026, compared to Gemma 3 27B’s 20.8% on the same benchmark. It sits at #3 on the Arena open model leaderboard.

It requires ~20GB VRAM at FP16, or ~12GB with INT4 quantization. A single A100 80GB handles it comfortably at full precision. Two RTX 4090s with tensor parallelism also work. This is the model you deploy to a server, not run on a laptop.

Use 31B when: benchmark quality matters for your application, you need thinking mode for reasoning-heavy tasks, you’re building a production service that will handle requests from multiple users, or you need the best math and coding performance available in an open-weight model.

Real use case: a coding assistant API that developers on your team query through a self-hosted endpoint – one 31B instance serving your whole engineering org at a cost that’s a fraction of equivalent proprietary API calls.

# Serving 31B with vLLM for production throughput
# pip install vllm

from vllm import LLM, SamplingParams

llm = LLM(
    model="google/gemma-4-31B-it",
    tensor_parallel_size=2,   # across 2x RTX 4090
    dtype="bfloat16",
    max_model_len=65536        # 64K for production balance
)

sampling_params = SamplingParams(
    temperature=0.2,
    top_p=0.9,
    max_tokens=2048
)

# Thinking mode for complex reasoning
prompts = [
    "<start_of_turn>usernThink step by step: Given this algorithm, what's the worst-case time complexity and where is the bottleneck?nn[your code here]n<end_of_turn>n<start_of_turn>modeln"
]

outputs = llm.generate(prompts, sampling_params)
for output in outputs:
    print(output.outputs[0].text)

The Decision Matrix

Here’s the five-minute version:

The Bigger Thing Happening Here

I want to step back from the specs for a second.

A model that scores 89.2% on a serious math benchmark, supports 256K context, runs multimodal inference, and has native function calling for agentic tasks… is now open-weight, Apache 2.0, and runs on hardware that a developer can actually own.

The E4B running on a laptop with 128K context and audio support isn’t a “small model compromise.” It’s a capability that would have been frontier-level two years ago. The E2B running on a phone offline isn’t a demo trick. It’s a production-viable deployment target.

What that actually means is that the architectural question of “cloud or local?” is no longer primarily a capability question. It’s a cost, latency, and privacy question. And for a lot of applications – the ones where user data is sensitive, where offline availability matters, where API costs compound at scale – local wins.

Gemma 4 doesn’t make that argument. It just makes it very hard to argue against.

Getting Started in Under 5 Minutes

The fastest path to running any Gemma 4 variant locally is Ollama:

# Install Ollama (macOS/Linux)
curl -fsSL https://ollama.com/install.sh | sh

# Pull the variant you want
ollama pull gemma4:e4b     # ~5GB, laptop-ready
ollama pull gemma4:26b     # ~15GB, GPU-ready

# Run it
ollama run gemma4:e4b

# Or use the API directly
curl http://localhost:11434/api/chat -d '{
  "model": "gemma4:e4b",
  "messages": [
    { "role": "user", "content": "Hello, what can you do?" }
  ]
}'

If you want Python with the full transformers ecosystem (function calling, thinking mode, multimodal), the Hugging Face model cards for each variant have complete working examples. Start with google/gemma-4-E4B-it – it’s the most accessible entry point and covers most development use cases.

Quick Note on Licensing

Apache 2.0 means you can use Gemma 4 commercially, modify the weights, build products on top of it, and distribute your derivative work – without paying royalties or asking permission. That is not the case for every “open” model out there, and it matters a lot for anyone building a business on top of local inference.

The right Gemma 4 variant is the one that runs where your users are, fits the hardware you can actually provision, and has enough context to do the task you’re designing for. Everything else is optimization.

Start with E4B if you’re unsure. Scale up when the task demands it.

Tags: devchallenge gemmachallenge gemma ai machinelearning python opensource

Log Level Strategies: Balancing Observability and Cost

A recurring issue with delayed shipment reports in a production ERP system took three days to resolve. Situations like these highlight how well we can understand what’s happening within our systems. Logs are the cornerstone of this understanding. However, logging everything is both impractical and costly. This is where establishing the right log level strategies becomes critical. In this post, I’ll share how I maximize observability in our systems while keeping costs under control, based on my own experiences.

Determining the correct log level is not just a technical detail but a strategic decision. Overly detailed logs can rapidly increase storage and analysis costs, while insufficient logs can make debugging impossible when issues arise. Striking this balance directly impacts our systems’ health and operational efficiency. Let’s examine how I achieve this balance step by step.

Understanding Different Log Levels and Their Use Cases

Log levels are used to indicate the severity and importance of an event. There are typically standardized levels that provide a guide on what information should be recorded and when. Understanding these levels correctly forms the foundation of our logging strategy.

The most commonly used log levels are:

• DEBUG: This is the most detailed level, generally used during development and debugging processes. It includes things like variable values, function calls, and loop iterations. It’s not recommended to keep this level constantly enabled in production environments.
• INFO: These messages indicate the normal operation of the application. They record events such as a user logging in or a process completing successfully. This level is crucial for observability.
• WARN: This level indicates potential issues or unexpected situations that do not directly prevent the application from running. Examples include a temporary database connection interruption. Logs at this level are critical for proactive intervention.
• ERROR: This level records errors that prevent the application from performing a function. Situations like an unprocessable request or a missing file fall under this category. It’s one of the first places to look for troubleshooting.
• FATAL: This signifies the most severe errors that can cause the application to crash completely. These are typically situations that lead to application termination.

Each of these levels serves a specific purpose. For instance, DEBUG level logs are invaluable during the development phase when testing a new feature. However, continuously collecting the same logs in a production environment consumes unnecessary disk space and slows down log analysis. Therefore, defining appropriate log levels for each environment (development, testing, production) is vital.

ℹ️ Log Levels and Environments

As a general rule, using the DEBUG level in development environments, while preferring INFO or WARN levels in test environments, is logical. In production environments, INFO, WARN, and ERROR levels are typically used together. The FATAL level is critical only for situations where the application unexpectedly halts.

Log Level Selection in Production: Observability vs. Cost

The production environment is where we need to establish the most delicate balance. On one hand, we need sufficient observability to understand what’s happening in our systems, detect potential issues early, and respond quickly. On the other hand, storing endless logs can lead to astronomical costs. Therefore, we must find a sensible answer to the question, “How much logging?”.

Generally, the INFO level is a good starting point for production environments. This level allows us to follow the normal flow of the application, logging key events like user registrations, order creations, or report generations. However, this level might not be sufficient when issues arise. For example, understanding why a user cannot open a specific page might be impossible with only INFO logs.

This is where WARN and ERROR levels come into play. WARN logs indicate that something might be going wrong but hasn’t yet escalated into a major problem. Perhaps an external service is responding slowly, or a configuration file was read unexpectedly. Heeding these warnings allows us to take preventive measures before issues become more significant. ERROR logs directly indicate the presence of a problem and are the most important source for troubleshooting.

But what about DEBUG? Keeping the DEBUG level constantly enabled in production is usually a major mistake. A couple of weeks ago, I accidentally left the DEBUG level enabled for a service in my own system. Within an hour, 80% of my disk was full, and the service began to slow down. Switching back to the INFO level after debugging instantly resolved the issue. Such situations can lead to serious cost and performance impacts. However, when a specific debugging session is needed, temporarily switching to the DEBUG level and reverting it afterward can be a smart move.

Real-World Scenarios and Example Log Lines

To solidify theoretical knowledge, let’s walk through some real-world scenarios. These examples will illustrate how different log levels might appear in a production environment and what kind of information they provide.

Consider a payment processing service for an e-commerce site.

INFO Level Example:
To log a successful payment by a user, we might see a log like this:

2026-05-23 10:30:15 INFO [payment-service] Payment successful for order ID: 123456789. Transaction ID: TXN987654321. Amount: 150.75 EUR. User ID: user_abc.

This log indicates the transaction occurred, providing order and transaction details, the amount, and the user ID. It’s sufficient for a basic success record.

WARN Level Example:
If a delay is experienced in a request to an external API during the payment process, we might receive a warning like this:

2026-05-23 10:31:02 WARN [payment-service] External payment gateway (gateway_xyz) response time exceeded threshold (1500ms). Actual time: 1850ms. Order ID: 123456789.

This warning shows that while the payment transaction itself was successful, an external service was slow. This is an indicator that could affect future transactions.

ERROR Level Example:
If an error occurs during the payment process, such as a rejection by the bank, we would see a log like this:

2026-05-23 10:32:45 ERROR [payment-service] Payment failed for order ID: 123456789. Reason: Bank declined transaction. Response code: 402. User ID: user_abc.

This error record includes the reason for the failure (bank decline) and the relevant response code, which is critical for identifying the root cause.

DEBUG Level Example (Temporary Use):
If we enable DEBUG level during a debugging session, we can see much more detailed information:

2026-05-23 10:33:10 DEBUG [payment-service] Entering process_payment_request method.
2026-05-23 10:33:10 DEBUG [payment-service] Request payload: {"order_id": "123456789", "amount": 150.75, "currency": "EUR", ...}
2026-05-23 10:33:11 DEBUG [payment-service] Calling external gateway API with payload: {"transaction_data": "...", "user_token": "..."}
2026-05-23 10:33:12 DEBUG [payment-service] Received response from gateway: {"status": "declined", "code": "402", "message": "Insufficient funds"}
2026-05-23 10:33:12 DEBUG [payment-service] Mapping gateway response to internal error code.

As you can see, DEBUG logs detail every step of the process, the data used, and responses from external services. This can be very useful for definitively pinpointing the source of an issue but is not suitable for continuous use.

⚠️ Risks of DEBUG Level

Keeping the DEBUG level constantly enabled in a production environment not only increases storage costs but can also lead to performance issues. Furthermore, it carries the risk of sensitive information (e.g., user credentials or payment details) being logged in plain text. Therefore, the DEBUG level should only be used temporarily and under controlled conditions.

Strategies for Managing Logging Costs

Logging costs can be a significant expense, especially in large-scale systems. Data storage, transfer, and licensing fees for log analysis tools can add up substantially over time. Several strategies can be employed to manage these costs.

First, correctly setting log levels is the most fundamental step. Start by avoiding keeping the DEBUG level constantly enabled in production. Instead, focusing on INFO, WARN, and ERROR levels will significantly reduce data volume.

Second, defining retention policies for logs is crucial. It’s generally unnecessary to store all logs indefinitely. Automatically deleting old logs that are no longer needed after a certain period (e.g., 30, 90, or 180 days) or moving them to a cheaper storage solution can reduce costs. Many log management systems offer the ability to configure these retention policies.

Third, using intelligent solutions for filtering and collecting logs is important. Sending only truly critical logs to a central system reduces data traffic and storage needs. For example, sending only ERROR and WARN level logs to a central logging service while storing INFO level logs for a more limited duration or only for specific services can be an option.

Fourth, ensuring log collection agents run efficiently is vital. In some cases, the log collection agents themselves can consume significant CPU or memory resources. Optimizing the configurations of these agents and preventing them from collecting unnecessary data is also important.

Finally, evaluating alternative logging solutions can be beneficial. Managed logging services offered by cloud providers or solutions like the open-source ELK Stack (Elasticsearch, Logstash, Kibana) can offer different advantages in terms of scalability and cost. I’ve experimented with various combinations of these solutions in some side projects I’ve developed, and each has its own trade-offs.

Advanced Logging Techniques for Observability

Beyond standard log levels, there are advanced techniques we can use to delve deeper into our systems and gain more visibility. These techniques can make a significant difference, especially in complex distributed systems or performance-critical applications.

1. Structured Logging:
Recording logs in a structured format, such as JSON, rather than plain text, dramatically enhances log analysis and querying capabilities. Each log entry contains information in key-value pairs. For example:

{
  "timestamp": "2026-05-23T10:30:15Z",
  "level": "INFO",
  "service": "payment-service",
  "message": "Payment successful",
  "order_id": "123456789",
  "transaction_id": "TXN987654321",
  "amount": 150.75,
  "currency": "EUR",
  "user_id": "user_abc"
}

These structured logs are easily parsed by log management systems, making them searchable. Querying all transactions for a specific order_id or all activities of a particular user_id becomes much easier.

2. Correlation IDs:
In distributed systems, a request often passes through multiple services. To track the entire journey of a request, a unique “correlation ID” is assigned, and this ID is added to the logs of all relevant services. This allows us to gather all logs related to a specific request in one place.

For example, a user request might first hit an API Gateway, then go to an authentication service, and finally to a backend service. If the same correlation ID is passed to each service, all logs related to that request can be found via this ID. This significantly speeds up the debugging process.

3. Metrics Collection:
Logs are typically event-based (what happened?). Metrics, on the other hand, provide continuous information about the system’s state (how much?). For instance, metrics like requests per second (RPS), memory usage, and CPU load of a service provide insights into the system’s overall health. When used in conjunction with logs, these metrics offer more comprehensive observability. Tools like Prometheus are popular for metrics collection and visualization.

4. Tracing:
Distributed tracing allows for end-to-end tracking of a request’s journey through a system. This is critical for understanding which service is causing latency or where an error originates, especially in microservice architectures. Tools like Jaeger or Zipkin assist in this regard. Unlike logs, tracing shows the completion time of a request and the calls made to sub-components.

While these advanced techniques may require a bit more setup and configuration initially, they ultimately make our systems more understandable, manageable, and resilient to errors in the long run. Implementing these techniques in my own developed products has allowed me to identify and resolve potential issues much faster.

Conclusion: Logging is a Marathon, Not a Sprint

Defining log levels is not a one-time task. As systems evolve, workloads change, and new features are added, we must continuously review our logging strategies. This is a marathon, not a sprint. The key is to find the right balance: ensuring sufficient observability while keeping costs at reasonable levels.

Based on my experience, focusing on INFO, WARN, and ERROR levels in production environments is usually the best starting point. Using DEBUG level only for temporary debugging sessions positively impacts both costs and performance. Advanced techniques like structured logging and correlation IDs elevate observability to the next level and simplify debugging in complex systems.

Let’s remember that logs are silent witnesses that provide us with information about our system’s health. Learning to listen to them correctly is the key to making our systems more reliable and efficient. By establishing this balance, we can keep operational costs under control and be better prepared for unexpected issues.

A first-look at the most underreported thing to come out of Google I/O 2026

I was watching the Google I/O 2026 Developer Keynote on my laptop at 11:30 PM with a cup of cold chai. Gemini 3.5 Flash landed. Antigravity 2.0 got a standing ovation. The smart glasses got the screenshots. Everyone was talking about Gemini Omni.

And then, buried between slide 40 and a code snippet, a presenter said the words “WebMCP.”

I sat up straight. I replayed that section three times.

Here is why.

A little context on what I’m building

For the past few months I’ve been working on a sales platform that stitches together an omnichannel inbox (Chatwoot) with a generative UI layer inside chat. The core problem is this: sales agents need to close deals inside conversations, not by jumping between five tabs. I want the chat widget itself to become the deal-closing surface – surfacing interactive UI components, pulling CRM context, letting a buyer sign a proposal inside the chat thread.

The hardest unsolved piece of that puzzle has always been: how do you let an AI agent do something meaningful on a webpage without either (a) screen-scraping like it’s 2019, or (b) building a bespoke API integration for every single surface it needs to touch?

WebMCP is Google’s answer to that question. And it’s a good one.

What WebMCP Actually Is

WebMCP is a proposed open web standard that lets developers annotate their JavaScript functions and HTML forms so that browser-based AI agents can call them directly as structured tools – with the same reliability you’d expect from a typed API, not from a model guessing where to click.

Think of it as MCP, but for the open web. Regular MCP handles the vertical connections: agent to database, agent to file system, agent to third-party API. WebMCP handles the horizontal layer: agent to the website sitting in front of you right now.

The origin trial starts in Chrome 149. Here is what the developer-side contract looks like conceptually:

// You annotate your JS functions with WebMCP metadata
// so agents know what they can call, what inputs they need,
// and what effect they'll produce

const webmcpManifest = {
  tools: [
    {
      name: "schedule_demo",
      description: "Books a product demo for a qualified lead",
      parameters: {
        name: { type: "string", required: true },
        email: { type: "string", required: true },
        preferred_slot: { type: "string", enum: ["morning", "afternoon", "evening"] }
      },
      handler: scheduleDemo
    },
    {
      name: "generate_proposal",
      description: "Creates a personalized pricing proposal and returns a preview URL",
      parameters: {
        company_name: { type: "string", required: true },
        seats: { type: "number", required: true },
        plan: { type: "string", enum: ["starter", "growth", "enterprise"] }
      },
      handler: generateProposal
    }
  ]
};

// Register once. Any WebMCP-capable agent can now call these
// without scraping your DOM or guessing your form structure.
window.__webmcp__ = webmcpManifest;

The agent on the other end – whether it’s Gemini in Chrome, an Antigravity subagent, or your own custom orchestrator – now has a reliable, developer-defined surface to work with. No brittle CSS selectors. No “find the button that looks like it submits the form.” Just a typed function call.

Why This Matters More Than It Looks

Here is the frame I keep coming back to: every website becomes a tool.

Right now, when you build an AI agent that needs to do anything on the web, you have two bad options and one expensive option.

Bad option 1 is prompt the model to act like a browser automation script. You’re essentially doing Playwright-via-LLM, which works until a div gets renamed and your entire pipeline breaks silently at 2 AM.

Bad option 2 is write a bespoke connector for every website or service you want the agent to interact with. Fine for the five services you use every day. Not fine for the long tail of things your users will eventually ask your agent to handle.

The expensive option is to build everything in-house, behind your own API, so the agent only ever talks to surfaces you control. This is what most serious teams do. It scales, but it costs months.

WebMCP opens a fourth path: the website author defines the agent interface themselves, once, as part of their normal development work. You annotate the functions that make sense to expose. You describe what they do. You ship it as part of your site. From that point forward, any agent with WebMCP support can call those functions correctly – without you writing a new connector, and without the agent guessing.

The protocol also composes cleanly with the rest of the stack Google laid out this year:

• MCP handles agent-to-infrastructure connections (databases, APIs, file systems)
• A2A handles agent-to-agent coordination across vendors
• WebMCP handles agent-to-website interaction in the browser

Three protocols, three layers, one coherent answer to “how does an agent actually do things in the real world?”

The Angle Nobody Is Talking About

Most I/O coverage framed WebMCP as a tool for making AI assistants more useful to end users. Fair. But I think the more interesting frame is what it does for developers building agentic products.

If I’m building the sales platform I described above, WebMCP means that instead of my agentic layer needing to “know” how to use each tool on the page through brittle DOM inspection, I can define those tools explicitly as part of my product. My checkout flow, my proposal generator, my slot-booking widget – they’re all first-class agent interfaces the moment I annotate them.

The generative UI layer I was building with Tambo (the agentic React UI toolkit) suddenly has a much cleaner answer to “how does the agent actually trigger actions?” instead of “it passes props and hopes for the best.” The agent calls a WebMCP-registered tool. The tool fires a handler. The handler updates state. The UI responds.

That is a clean, auditable, testable loop. That matters when you are building something real, not a demo.

What I’m Watching Next

A few things I’m paying close attention to as WebMCP moves through the origin trial:

Security boundaries. Right now the spec assumes the website author is intentional about what they expose. But as soon as agent-browsing becomes mainstream, you’ll see adversarial cases: pages that register fake tools designed to manipulate agents into taking actions users didn’t intend. The security model around what a WebMCP tool is “allowed” to do on behalf of a user needs to be tight before this goes anywhere near financial or identity workflows.

Cross-browser adoption. This is proposed by Google and Microsoft engineers under the W3C Web Machine Learning community group – which is a good sign for eventual cross-browser support. But Chrome 149 origin trials don’t mean it’s in Firefox next quarter. For developers building agent-facing products today, you’ll need fallback strategies for a while.

Standardization lag vs. tooling speed. The web standards process moves on a timescale that AI tooling doesn’t. By the time WebMCP is a full W3C Recommendation, the agent landscape will have changed dramatically. Google’s bet here is that shipping an origin trial fast and getting real-world feedback is worth more than waiting for perfect committee consensus. I think that bet is correct.

The Bigger Shift Underneath All of It

There was a line in the keynote that I keep turning over in my head: “We’ve transitioned from AI that simply assists you, to agents that can independently navigate complex tasks across your entire workflow.”

That is a product vision, not a product feature. And WebMCP is one of the infrastructure pieces that makes that vision non-fictional.

The assistive AI era had a ceiling: the model could help you think, draft, and plan, but doing required a human in the loop who could actually click the button, fill the form, and trigger the action. The agentic era removes that ceiling – but only if agents can interact with the web in a way that’s reliable enough to trust in production.

WebMCP is a small, well-scoped proposal. It doesn’t solve everything. But it solves the exact right problem at the exact right layer: giving website developers a first-class way to say “here is what an agent is allowed to do on this page, and here is how to do it correctly.”

For anyone building agentic products on the web, that is not a quiet announcement. That is the foundation you were waiting for.

If you’re building something that sits at the intersection of AI agents and web UIs, I’d genuinely love to compare notes. Drop a comment or find me at my usual haunts.

Tags: googleiochallenge webdev ai javascript webstandards

When TestSmith generates tests with --llm, it calls an LLM for every public member of every source file being processed. A project with 20 files and 5 public functions each means up to 100 API calls in a single run. That’s a lot of surface area for things to go wrong.

Here’s the reliability stack we built, layer by layer.

Layer 1: Retry with Exponential Backoff

LLM APIs fail transiently. Rate limits, timeouts, occasional 5xx responses — all of these are recoverable if you wait and retry. We built a retry middleware that wraps any Provider:

type RetryProvider struct {
    inner      Provider
    maxRetries int
}

func (r *RetryProvider) Complete(ctx context.Context, req CompletionRequest) (CompletionResponse, error) {
    var lastErr error
    for attempt := 0; attempt < r.maxRetries; attempt++ {
        if attempt > 0 {
            wait := time.Duration(math.Pow(2, float64(attempt))) * 100 * time.Millisecond
            select {
            case <-time.After(wait):
            case <-ctx.Done():
                return CompletionResponse{}, ctx.Err()
            }
        }
        resp, err := r.inner.Complete(ctx, req)
        if err == nil {
            return resp, nil
        }
        lastErr = err
    }
    return CompletionResponse{}, fmt.Errorf("after %d attempts: %w", r.maxRetries, lastErr)
}

MaxRetryAttempts defaults to 3. With exponential backoff: attempt 1 is immediate, attempt 2 waits 200ms, attempt 3 waits 400ms. Total worst-case wait per call is under a second — acceptable latency for a background tool.

Layer 2: Semaphore for Concurrency Control

With up to 100 calls to make, goroutine fan-out is the obvious approach. But hitting an LLM API with 100 concurrent requests triggers rate limiting immediately. A semaphore caps the in-flight calls:

type SemaphoreProvider struct {
    inner Provider
    sem   chan struct{}
}

func NewSemaphoreProvider(inner Provider, maxConcurrent int) *SemaphoreProvider {
    return &SemaphoreProvider{inner: inner, sem: make(chan struct{}, maxConcurrent)}
}

func (s *SemaphoreProvider) Complete(ctx context.Context, req CompletionRequest) (CompletionResponse, error) {
    select {
    case s.sem <- struct{}{}:
        defer func() { <-s.sem }()
    case <-ctx.Done():
        return CompletionResponse{}, ctx.Err()
    }
    return s.inner.Complete(ctx, req)
}

MaxConcurrentCalls defaults to 5. Each retry attempt acquires its own semaphore slot — this is important. If retry logic held a slot while waiting between attempts, other goroutines would be blocked unnecessarily. The retry wrapper is the outer layer; semaphore is the inner layer.

The middleware stack assembled by the factory:

retry → semaphore → raw provider

Layer 3: Result Cache

Many test generation runs touch the same files repeatedly — watch mode is the extreme case. Calling the LLM for the same source code twice is wasteful. A content-addressed cache avoids it:

type ResultCache struct {
    mu      sync.RWMutex
    entries map[string][]BodyGenResult
    hits    int
    misses  int
}

func cacheKey(req BodyGenRequest) string {
    h := sha256.New()
    fmt.Fprintf(h, "%sn%sn%sn%s", req.Language, req.MemberName, req.SourceCode, req.Framework.Name)
    return hex.EncodeToString(h[:])
}

The key is a SHA-256 hash of the language, member name, source code, and framework. If the source file changes, the hash changes and the cache misses — you always get fresh results for changed code.

After a run, --verbose prints the cache stats:

LLM cache — hits: 12  misses: 8  entries: 8

Layer 4: Batch Generation

The fan-out approach makes one API call per public member. For a file with 10 functions, that’s 10 calls. Batch generation collapses this to one:

func (g *LLMBodyGenerator) GenerateBatchBodies(
    ctx context.Context,
    reqs []BodyGenRequest,
) ([]BodyGenResult, error) {
    prompt := buildBatchPrompt(reqs)
    resp, err := g.provider.Complete(ctx, CompletionRequest{
        SystemPrompt:   batchSystemPrompt,
        UserPrompt:     prompt,
        Model:          g.model,
        MaxTokens:      g.maxTokens * len(reqs), // scale with request count
        Temperature:    g.temperature,
        ResponseFormat: "json_object",            // structured output
    })
    // ...
}

We use OpenAI’s response_format: {"type": "json_object"} to get structured output. The model returns a JSON envelope with one entry per member:

{
  "tests": [
    {"name": "ProcessPayment", "code": "func TestProcessPayment(t *testing.T) { ... }"},
    {"name": "RefundPayment",  "code": "func TestRefundPayment(t *testing.T) { ... }"}
  ]
}

We parse that with a primary JSON parser, with a fallback to a delimiter-regex parser for providers that don’t support structured output.

The pipeline checks for the BatchBodyGenerator interface via type assertion. If the generator implements it, batch mode is used. If not (or if the driver explicitly opts out), it falls back to goroutine fan-out with individual calls. This keeps the interface opt-in and backward compatible.

Observability: Cache Stats

With all this happening in the background, it’s useful to know what actually ran. The cacheStatsReporter interface lets the CLI query stats without importing the llm package:

// In cmd/testsmith/generate.go — avoids importing internal/llm from the CLI layer
type cacheStatsReporter interface {
    CacheStats() (hits, misses, size int)
}

func printCacheStats(bg domain.BodyGenerator) {
    if !verbose {
        return
    }
    if r, ok := bg.(cacheStatsReporter); ok {
        hits, misses, size := r.CacheStats()
        fmt.Printf("LLM cache — hits: %d  misses: %d  entries: %dn", hits, misses, size)
    }
}

This is the interface segregation principle at work: the CLI knows about domain.BodyGenerator (which it needs for the pipeline) and cacheStatsReporter (which it needs for stats output). It doesn’t need to know anything else about the LLM implementation.

The Numbers

In practice, on a mid-size Go project with 40 source files and an average of 6 public functions each:

• Without batch: 240 API calls, ~4 minutes at 5 concurrent
• With batch: 40 API calls (one per file), ~45 seconds
• Second run with warm cache: near-instant for unchanged files

The cache and batch generation together turn what would be a “go make coffee” operation into something you can run while you’re still in the flow.

Next: how we structure context for both AI agents working on TestSmith itself and for the LLM generating tests for your project.

CORS is one of those things every web developer runs into sooner or later. Most of us know how to fix it — add a header, change a config, ask the backend person to “do something about CORS.” But how many of us actually understand what the browser is doing in the background, and why it’s doing it?

Let’s go through it today, slowly, with a simple example.

What’s an “origin”?

Before we get into CORS, there’s one word we need to pin down: origin. An origin is three things put together — the scheme, the host, and the port of a URL.

So https://example.com and http://example.com are different origins because the scheme is different (https vs http). https://example.com and https://api.example.com are different because the host is different. And https://example.com and https://example.com:8080 are different because the port is different. If any one of those three pieces changes, the browser treats it as a separate origin.

This matters because the browser treats every origin as its own little sandbox. Whatever happens inside one origin is supposed to stay inside that origin.

The browser’s default rule

By default, JavaScript running on one origin is not allowed to read data from another origin. This rule has a name: same-origin policy. Every modern browser ships with it built in, and it’s the foundation that everything else here is built on.

Without it, the web would be terrifying. Imagine you’re logged into your bank in one tab. In another tab, you visit some random site. If same-origin policy didn’t exist, that random site could just do this in the background:

fetch('https://yourbank.com/api/balance')
  .then(res => res.json())
  .then(data => sendToAttacker(data))

The browser would happily send the request, attach your bank cookies (because the cookies belong to the bank’s domain), and the bank would return your balance to whoever asked. The random site’s JavaScript would then read the response and send it anywhere it wanted. Every site you visit would be able to scrape every site you’re logged into.

Same-origin policy stops that. The browser still sends the request, but it refuses to let the random site’s JavaScript read the response. The request happens; the response gets blocked from reaching the calling code.

So what is CORS?

CORS stands for Cross-Origin Resource Sharing, and it’s the mechanism a server uses to say “actually, it’s fine, this other origin is allowed to read my response.” It’s an opt-in to relax same-origin policy in cases where cross-origin access is legitimate.

The most common case is your own setup. Your frontend lives at app.example.com, your API lives at api.example.com. Different origins, so same-origin policy would block the frontend’s calls by default. CORS is how your API tells the browser “yes, app.example.com is allowed.”

How does the API tell the browser that? Through a response header:

Access-Control-Allow-Origin: https://app.example.com

When the browser sees this header on the response, it lets your JavaScript read it. Without it, the response is blocked and you see a CORS error in the console.

One thing worth noticing here, because it confuses a lot of people: if your frontend and API are on the same origin during development — say both on localhost:3000 behind a Next.js rewrite — CORS doesn’t kick in at all. The browser doesn’t even check the headers, because nothing cross-origin is happening. CORS issues often only show up in production, when frontend and API are on different domains.

A simple example, end to end

Let’s walk through one fetch from start to finish.

Your frontend at https://app.example.com does:

fetch('https://api.example.com/users')

Here’s what actually happens:

1. The browser builds the request. It automatically adds a header: Origin: https://app.example.com. This is the browser telling the server “this request is coming from app.example.com.”

2. The browser sends the request to api.example.com. The server receives it like any other request and processes it. It can read the database, run business logic, return data — whatever. From the server’s point of view, this is a normal request.

3. The server sends a response back, say with the user list as JSON.

4. The browser receives the response and checks the headers. Specifically, it looks at Access-Control-Allow-Origin. If that header is missing, or if its value doesn’t match https://app.example.com (or *), the browser blocks the response. The JavaScript that called fetch sees a CORS error, and the response body is never delivered to your code.

5. If the header is present and matches, the browser hands the response to the JavaScript, and your code continues normally.

This is the part that most explanations of CORS skip over: the server doesn’t refuse the request. The browser does. The server returned a perfectly fine response. The browser refused to hand it to the JavaScript because the server didn’t include the right header saying “this origin is allowed.”

That’s also why you can hit the same API from curl, from Postman, or from a server-to-server call and it just works. CORS only exists in browsers. Other clients don’t enforce same-origin policy, so there’s nothing to block.

What about complex requests? Preflight

For “simple” requests — basically GETs and a few POSTs with safe content types — the browser sends the actual request directly and checks the response headers afterwards.

But for anything more involved — a PUT, a DELETE, a request with a custom header like Authorization, or a JSON-typed POST — the browser does an extra step first. It sends a preflight request.

A preflight is an OPTIONS request to the same URL, sent before the real one. It looks like:

OPTIONS /users HTTP/1.1
Origin: https://app.example.com
Access-Control-Request-Method: DELETE
Access-Control-Request-Headers: Authorization, Content-Type

The browser is essentially asking the server: “I’m about to send a DELETE with these headers. Are you okay with that?”

The server responds with what it allows:

HTTP/1.1 204 No Content
Access-Control-Allow-Origin: https://app.example.com
Access-Control-Allow-Methods: GET, POST, PUT, DELETE
Access-Control-Allow-Headers: Authorization, Content-Type

If the browser is happy with the preflight response, it then sends the real request. If the preflight fails, the real request never goes out at all.

This is why you sometimes see two requests in the network tab for what looks like a single fetch. The first is the preflight; the second is the actual call.

The credentials catch

There’s one extra rule worth knowing, because it catches a lot of people off guard. If your frontend sends credentials with the request — cookies, an Authorization header, anything that identifies the user — like this:

fetch('https://api.example.com/me', {
  credentials: 'include',
})

Then a wildcard Access-Control-Allow-Origin: * is no longer enough. The browser refuses to deliver the response. The server has to name the specific origin it trusts, and add one more header:

Access-Control-Allow-Origin: https://app.example.com
Access-Control-Allow-Credentials: true

The reason is security. If * worked alongside credentials, any malicious site could fire authenticated requests to your API. The browser would attach the user’s cookies (because cookies belong to the API’s domain, not the calling page’s). The server would see “any origin is allowed” and respond. The malicious site’s JavaScript would then read the response — and every authenticated API on the web would be one fetch away from being scraped.

So the rule is: * is fine for genuinely public APIs that don’t care who’s calling. As soon as authentication enters the picture, the server has to be explicit about which origins it trusts.

Wrapping up

Here’s the whole picture in one line: the browser blocks cross-origin reads by default (same-origin policy), and CORS is the mechanism a server uses to opt in to allowing them, through response headers like Access-Control-Allow-Origin.

A few things worth remembering when you next see a CORS error in the console.

The browser is the one doing the blocking, not the server. The server happily sends the response either way; the browser decides whether your JavaScript gets to see it. That’s why CORS errors only happen in the browser, never in curl or server-to-server calls.

For non-simple requests, the browser sends an OPTIONS preflight first to ask the server what’s allowed. Two network requests for one fetch — that’s why.

And if credentials are involved, the wildcard stops working. The server has to name the origin explicitly and add Access-Control-Allow-Credentials: true.

CORS isn’t there to annoy developers. It’s a security feature that protects users from sites that would otherwise be able to read their data from other sites. Once you see what it’s protecting, the error messages start making more sense — and the fix usually comes down to figuring out exactly which header the browser wants to see, and making sure your server is sending it.

TestSmith generates test scaffolds for five languages: Go, Python, TypeScript, Java, and C#. Each language has its own project structure conventions, test frameworks, import styles, and code patterns. The naive implementation would be a big switch statement throughout the codebase. We chose a plugin model instead.

The Problem with Hardcoded Branches

When a codebase switches on language in multiple places, every new language requires touching every branch point. Miss one and you get a silent bug — the new language falls through to some default behavior that doesn’t apply to it. This is the classic Open-Closed violation: you have to modify existing code to extend it.

The LanguageDriver Interface

Every language in TestSmith implements a single interface:

type LanguageDriver interface {
    // Detection
    DetectProject(dir string) (*ProjectContext, error)
    FileExtensions() []string

    // Analysis
    AnalyzeFile(path string, ctx *ProjectContext) (*SourceAnalysis, error)
    ClassifyDependency(dep ImportInfo, ctx *ProjectContext) DependencyCategory
    DeriveTestPath(sourcePath string, ctx *ProjectContext) (string, error)
    DeriveModulePath(sourcePath string, ctx *ProjectContext) (string, error)

    // Generation
    GenerateTestFile(analysis *SourceAnalysis, opts GenerateOpts) (*GeneratedFile, error)
    GenerateFixture(dep string, analysis *SourceAnalysis, opts GenerateOpts) (*GeneratedFile, error)
    GenerateBootstrap(plan *GenerationPlan, ctx *ProjectContext) (*GeneratedFile, error)

    // Framework config
    GetTestFrameworkConfig() TestFrameworkConfig
    SelectAdapter(ctx *ProjectContext) TestAdapter

    // LLM integration
    LLMContext(ctx *ProjectContext) map[string]string
    LLMVocabulary() map[string]string

    // Migration and validation
    ListMigrators() []Migrator
    ValidateFile(path string, ctx *ProjectContext) ([]ValidationIssue, error)
}

The generation pipeline, the CLI commands, and the watch mode all work against this interface. They never import a specific driver package.

How Detection Works

When you run testsmith generate, the first step is figuring out what language you’re in. The registry tries each registered driver in turn:

func Detect(dir string) (domain.LanguageDriver, error) {
    for _, d := range drivers {
        ctx, err := d.DetectProject(dir)
        if err == nil && ctx != nil {
            return d, nil
        }
    }
    return nil, domain.ErrProjectNotFound
}

Each driver’s DetectProject walks upward from the starting directory looking for its own project markers — go.mod for Go, pyproject.toml or setup.py for Python, package.json for TypeScript, pom.xml or build.gradle for Java, .csproj or .sln for C#.

One subtle requirement: a driver must not claim an ancestor project that belongs to a different language. If you run TestSmith from inside an example project that lives inside a Go repo, the Python driver shouldn’t walk up past the Go project’s .git boundary and claim the repo root. We solve this by checking VCS stop markers (.git, .hg, .svn) at ancestor directories only — not at the starting directory itself, since a legitimate project root can have both a project marker and a .git directory.

func findRoot(startDir string) (string, error) {
    dir := startDir
    for {
        // At ancestor dirs, stop at VCS boundaries first.
        if dir != startDir {
            for _, stop := range stopMarkers {
                if _, err := os.Stat(filepath.Join(dir, stop)); err == nil {
                    return "", domain.ErrProjectNotFound
                }
            }
        }
        // Then check for project markers.
        for _, marker := range rootMarkers {
            if _, err := os.Stat(filepath.Join(dir, marker)); err == nil {
                return dir, nil
            }
        }
        parent := filepath.Dir(dir)
        if parent == dir {
            break
        }
        dir = parent
    }
    return "", domain.ErrProjectNotFound
}

The Adapter Layer

Within a language, there can be multiple test frameworks. TypeScript has Jest, Vitest, and Mocha. Java has JUnit 4, JUnit 5, TestNG, and Spring Boot Test. Each framework has its own import style, mock library, assertion syntax, and file naming conventions.

We model this with a TestAdapter interface:

type TestAdapter interface {
    Name() string
    FileNamingConvention() FileNaming
    ImportStyle() ImportStyle
    MockLibrary() string
    AssertionStyle() string
    LLMVocabulary() map[string]string
}

Each driver has a registry of adapters and a SelectAdapter method that reads the project config (or sniffs package.json devDependencies, pom.xml dependencies, etc.) to pick the right one. The LLM prompt gets the vocabulary from the selected adapter — so the model knows to generate expect(x).toBe(y) for Jest but assert.Equal(t, x, y) for Go’s testify.

Adding a New Language

Because everything flows through the interface, adding a new language driver is isolated:

1. Create a new package under internal/drivers/<lang>/
2. Implement domain.LanguageDriver — the compiler tells you exactly what’s missing
3. Register it in internal/registry/registry.go
4. Optionally add a Verifier in internal/generation/verify.go for post-write compile checking

No other files change. The existing drivers are untouched. The pipeline, CLI, and watch mode pick it up automatically.

The Dependency Direction

The plugin model enforces a strict dependency direction:

cmd → generation → domain ← drivers
                           ← llm

domain defines the interfaces. drivers implement them. generation uses them via the interface. Neither generation nor drivers imports the other. This is the Dependency Inversion Principle applied at the package level — and it’s enforced by Go’s import cycle detector.

When you add a new driver, it’s impossible to accidentally reach into the generation pipeline or the LLM layer — Go won’t compile it. The architecture is self-enforcing.

Next in this series: making LLM calls reliable when you’re hitting them for every public member of every source file.

TestSmith v1 was a Python CLI. It worked. Users could pip install testsmith, point it at a source file, and get a test scaffold back. But every team that tried to wire it into CI hit the same wall: Python environments.

The problem wasn’t Python itself — it was distribution. A static analysis tool that requires a matching Python version, a virtual environment, and a pinned dependency tree is a hard sell for a step that runs on every push. We were shipping a tool, not a library. Tools should be frictionless.

The Decision

We rewrote TestSmith v2 in Go. The goal was a single static binary with no runtime dependencies — something you could drop into any CI runner, any Docker image, any developer’s PATH, and it would just work.

Go was the right choice for three reasons:

Single binary. go build produces one self-contained executable. No pip, no venv, no requirements.txt. Users download a binary or brew install it and they’re done.

Cross-platform with one build. The v1 CI matrix was a headache — different Python versions across Ubuntu, macOS, and Windows, with slightly different behavior on each. Go’s cross-compilation gave us linux/amd64, darwin/amd64, darwin/arm64, and windows/amd64 from a single build step.

Native concurrency. Test generation is embarrassingly parallel — each file is independent. Go’s goroutines and channels made the fan-out generation and the debounced file watcher straightforward to implement without pulling in async libraries.

What Changed

The command surface was cleaned up in the same pass. v1 used flags for everything:

testsmith <file>         # generate
testsmith --all          # generate all
testsmith --graph        # dependency graph
testsmith --prune        # prune stale fixtures
testsmith --watch        # watch mode

v2 uses proper subcommands:

testsmith generate <file>
testsmith generate --all
testsmith graph
testsmith prune
testsmith watch

This made shell completion, help text, and per-command flags much cleaner. Cobra’s built-in completion generator gives us bash, zsh, fish, and PowerShell completion for free.

Architecture Change

v1 was monolithic Python — one codebase with hardcoded branches for each language it supported. Adding a new language meant editing multiple core files.

v2 uses a LanguageDriver interface. Each language (Go, Python, TypeScript, Java, C#) is a separate package that implements the interface. The core generation pipeline never knows which language it’s dealing with — it just calls through the interface:

type LanguageDriver interface {
    DetectProject(dir string) (*ProjectContext, error)
    AnalyzeFile(path string, ctx *ProjectContext) (*SourceAnalysis, error)
    DeriveTestPath(sourcePath string, ctx *ProjectContext) (string, error)
    GenerateTestFile(analysis *SourceAnalysis, opts GenerateOpts) (*GeneratedFile, error)
    // ... and more
}

Adding a new language is now a matter of creating a new package and registering it — no changes to the pipeline.

What We Kept

The v1 Python package didn’t disappear. It lives in archive/v1/ and continues to receive bug fixes during the transition period. Teams already using v1 in production don’t need to migrate immediately. The v2 binary is a clean break for new users; v1 stays stable for existing ones.

Was It Worth It?

Yes, unambiguously. The CI story went from “install Python, set up venv, pin deps” to “download one binary.” The Windows test matrix went from flaky (Python path issues) to clean. And the plugin architecture means we can add a Ruby or Rust driver without touching the core generation logic.

The rewrite took longer than a feature would have. But distribution friction is a silent project killer — nobody files a bug for “this was annoying to set up,” they just stop using the tool.

TestSmith is an open-source CLI for generating test scaffolds across Go, Python, TypeScript, Java, and C#. The source is at github.com/orieken/testsmith.

Every team I’ve worked on has had the same conversation at some point. Someone opens the coverage report, sees a sea of red, and asks: “How do we get this up?” The answer is always some version of “we need to write more tests,” followed by a long silence, because everyone knows what that actually means — hours of boilerplate, test file setup, mock wiring, and fixture scaffolding before you’ve written a single meaningful assertion.

That’s the problem TestSmith was built to solve.

The Real Bottleneck Isn’t Willingness

Developers generally want to write tests. The resistance isn’t laziness — it’s the setup cost. For every new module you want to test, you have to:

• Create the test file in the right location with the right naming convention
• Import the module under test
• Import the test framework and any mock libraries
• Set up fixtures for external dependencies
• Write the boilerplate class or function structure that the framework expects
• Then, finally, write the actual test logic

For a well-understood module with clear inputs and outputs, steps 1 through 5 can easily take longer than step 6. You’re doing janitorial work before you can do the meaningful work. And if you’re adding coverage to a large existing codebase — the kind of coverage catch-up project every team eventually faces — you’re doing that setup dozens or hundreds of times.

Why Python First

We wrote the first version of TestSmith in Python for the most straightforward of reasons: our immediate problem was a Python codebase.

But Python also happened to be a good fit for the tool itself. Python’s AST module is excellent — ast.parse() gives you a full parse tree in a few lines, and walking it to extract class names, function signatures, and import statements is straightforward. For a tool that needs to understand source code structure without actually running it, static AST analysis is exactly right, and Python’s standard library makes it easy.

import ast

tree = ast.parse(source_code)
for node in ast.walk(tree):
    if isinstance(node, ast.FunctionDef):
        if not node.name.startswith('_'):  # skip private
            public_functions.append(node.name)

The other reason was speed of iteration. We were solving our own problem — we needed the tool to work on Python projects, and we were Python developers. Building it in Python meant we could use it on itself from day one, which is a useful forcing function for catching rough edges.

What the Tool Actually Does

The core idea is simple: given a source file, generate the test scaffold that you’d write by hand.

For a Python service like this:

# src/services/payment.py

class PaymentService:
    def __init__(self, stripe_client, db):
        self.stripe = stripe_client
        self.db = db

    def process_payment(self, order_id: str, amount: int) -> dict:
        ...

    def refund(self, payment_id: str) -> bool:
        ...

TestSmith generates:

# tests/services/test_payment.py

import pytest
from unittest.mock import MagicMock, patch
from src.services.payment import PaymentService


@pytest.fixture
def stripe_client():
    return MagicMock()


@pytest.fixture
def db():
    return MagicMock()


@pytest.fixture
def payment_service(stripe_client, db):
    return PaymentService(stripe_client=stripe_client, db=db)


class TestPaymentService:
    def test_process_payment(self, payment_service):
        # TODO: implement
        pass

    def test_refund(self, payment_service):
        # TODO: implement
        pass

It’s not a complete test. It’s the scaffold — the file is in the right place, the imports are correct, the fixtures for the constructor dependencies are wired up, and the test methods exist. The developer fills in the assertion logic. The janitorial work is already done.

The tool also handles things that are easy to get wrong: where test files should live relative to source files (which varies by framework and project convention), how to name fixtures based on constructor parameters, which mock library to use, and how to structure the test class if the source is class-based vs. the test functions if it’s function-based.

The Gap Analysis Problem

Coverage reports tell you what’s untested, but they don’t prioritise it. A file with three simple utility functions and a file with a complex payment processing pipeline both show up as “uncovered.” Knowing which one to tackle first requires reading the code.

TestSmith added a coverage gap command that went a step further: it computed a coupling score for each untested module based on how many other modules imported it. A module imported by ten others is higher priority than one imported by none — because a bug in the heavily-imported module has a wider blast radius.

$ testsmith gaps

Coverage gaps (by coupling score):

  src/services/payment.py        coupling: 8   functions: 5   ← fix this first
  src/utils/currency.py          coupling: 6   functions: 3
  src/models/order.py            coupling: 4   functions: 7
  src/scripts/backfill.py        coupling: 0   functions: 12

This gave teams a principled answer to “where do we start?” rather than requiring someone to manually audit the codebase.

What v1 Didn’t Do Well

The tool worked. Teams used it and got value from it. But two things became clear over time.

Distribution was painful. pip install testsmith sounds simple, but in practice it meant managing Python versions, virtual environments, and dependency conflicts — especially in CI. A testing tool that requires its own setup to work in CI is fighting against itself.

One language wasn’t enough. Once word got around that the tool existed, the first question from every team was “does it work for TypeScript?” or “can it do Java?” The Python-only design wasn’t a deliberate choice — it was an artifact of solving our own immediate problem. But the architecture didn’t make adding languages easy. Every language-specific piece of logic was woven through the core code rather than isolated.

Those two problems drove the v2 rewrite in Go: a single static binary that drops into any environment, and a plugin architecture where each language is an isolated driver.

But that’s the next post.

TestSmith is open source at github.com/orieken/testsmith. The v1 Python package is archived at archive/v1/ for reference.

This is a submission for the Gemma 4 Challenge: Build with Gemma 4

My daughter is 13. Like most students her age in Austria, she has an iPad.
Like most parents, I’m uncomfortable about her typing homework into
ChatGPT — not because the answers are wrong, but because everything she
types disappears into a cloud I don’t control. Names, schools, half-formed
thoughts, the topics where she’s struggling. Stuff I want to stay on her
device.

So I built Lernbuddy. It’s a study companion for ages 10–14 that runs
entirely on-device with Gemma 4 E4B. No network calls. No telemetry.
The model lives next to the kid’s flashcards and her review history, on
the same disk, and that’s it.

What it does

Three things, all driven by Gemma 4 E4B:

• Socratic chat. She asks a question, the model gives a hint — never
  the answer. The system prompt is built around concrete praise
  (“nice, you spotted that fractions need a common denominator”) and
  refusing to solve problems directly.
• Flashcards. Paste a school text → cards. Or just type a topic
  (irregular verbs, European capitals) → the model pulls from general
  knowledge and creates a deck. Every card lands in a preview where you
  can edit, delete, or add more before they go in the database.
• Quiz with model-validated answers. Typed answers, not multiple-
  choice. The model reads the correct answer and the kid’s answer and
  decides correct / almost / incorrect — spelling errors and
  paraphrasing are tolerated. It even writes a short personalized summary
  at the end: “You handled past participles well. ‘choose’ and ‘speak’
  got mixed up — they’ll come back tomorrow.”

Behind the quiz sits a small SM-2 spaced-repetition scheduler. Cards she
gets wrong come back the next day; cards she gets right keep slipping
further out. A streak counter and a per-topic “✓ 12 solid · … 4 to
practise” badge make progress visible without dashboards.

Architecture

The whole thing is a .NET 9 MAUI app — one codebase, four targets
(Windows, Android, iOS, macCatalyst), bilingual from day one (German is
the primary audience, English the default for international demos).

Inference goes through Microsoft.Extensions.AI‘s IChatClient
interface. The concrete implementation wraps LLamaSharp running
unsloth/gemma-4-E4B-it-GGUF (Q4_K_M quant, ~4.6 GB). Gemma 4’s chat
template — <|turn>rolen...<turn|> — gets built by hand; the embedded
jinja in the GGUF carries tool-calling logic the kid will never need.

A singleton InferenceStatus service publishes “Loading model… 4.6 GB”,
“Generating… 23 tokens (4.1/s)” to a small banner that’s visible on every
page. When prompt-processing takes 30 seconds before the first token (CPU
is slow on long prompts), the banner still ticks. No dead spinners —
that matters more than raw throughput when you’re 11 and waiting.

Long source texts get split by paragraph and sentence into
~500-character chunks. Cards appear rolling instead of waiting two
minutes for one mega-prompt. Total throughput is slightly worse;
perceived time is much better.

The privacy story

The project’s privacy contract starts with “local first”: no PII in
prompts, parent-PIN gate on any future cloud mode, model download only
over WiFi with explicit consent. The current app makes zero outbound
network calls after install. The only thing that ever leaves the device
is what you type into a Gemini API form — and we don’t have that form.
By design.

That’s the differentiator. There are dozens of “ChatGPT for kids” apps;
almost none of them can credibly say “your child’s homework never leaves
their device.”

What I learned

I went down three runtime rabbit-holes before things stuck. ONNX Runtime
GenAI doesn’t support the Gemma 4 multi-file ONNX split yet.
transformers.js + WebGPU inside a MAUI HybridWebView is theoretically
beautiful and practically a stack of four bleeding-edge components
silently failing in ways nothing surfaces. LLamaSharp + GGUF works —
except LLamaSharp doesn’t ship iOS-arm64 natives. So I built llama.cpp
myself for iOS on a Mac, statically linked against the resulting .a
files via MAUI’s NativeReference mechanism, and pointed
[DllImport("llama")] at __Internal via a runtime resolver.

The lesson: for a multi-platform local-LLM .NET project in 2026, pick the
runtime that has the deepest community, not the one with the prettiest
abstraction. llama.cpp has a community measured in thousands; the .NET
wrappers around it inherit that depth almost for free.

Try it

Code: https://github.com/gpiwonka/Lernbuddy

more: https://piwonka.cc/Lernbuddy

License: MIT. Built solo, on the side, between a day job and a tired
father’s evenings. PRs welcome — especially from teachers who can point
out what’s missing from a learning-science perspective.

The submission is for the Gemma 4 Challenge. The motivation is more
personal.