Mert

On the backend of one of my side projects, I had set up an AI agent to allow users to get complex financial reports using natural language. Everything was going great; the agent connected to the database, queried the necessary tables, and summarized the results. That is, until 4:22 AM one morning, when I discovered the agent had “accidentally” drained all CPU resources on our PostgreSQL instance with a massive JOIN query, locking down the database. That day, I realized that giving an agent a “tool” is no different than handing them a loaded gun and saying, “only shoot the target.”

You cannot ensure security in agent architectures solely through prompt engineering. Saying “Please don’t delete the database” is not a security measure; it’s merely a wish. In the real world, you need to restrict the capabilities of an agent running in a production environment with strict rules at the system and application architecture levels. In this post, I will explain the 3-step secure tool design that I’ve implemented in my own projects, developed as an engineer who has “learned the hard way.”

Step 1: Schema Hardening in Tool Definitions

When you present a tool to LLMs (Large Language Models), the model understands what the tool does and what parameters it accepts via JSON Schema. If your schema is loose, the model will get creative and invent parameters you didn’t expect. In a production ERP I developed, because I hadn’t included a limit parameter for the stock inquiry tool, the agent once tried to pull all 450,000 lines of stock history.

Schema hardening is about narrowing the agent’s scope of action from the definition stage. It’s not enough to just specify the data type (string, integer); you must iron-fistedly define minimum-maximum values, allowed enum lists, and required fields. Creating these schemas using Pydantic not only improves code readability but also reduces the model’s margin for error.

from pydantic import BaseModel, Field, validator
from typing import Optional, List

class StockQuerySchema(BaseModel):
    warehouse_id: int = Field(..., description="The warehouse ID to query. Can only be 1, 2, or 5.")
    sku_pattern: str = Field(..., min_length=3, max_length=20, description="The product code pattern.")
    limit: int = Field(default=10, ge=1, le=100, description="Maximum number of records to return.")

    @validator('warehouse_id')
    def validate_warehouse(cls, v):
        allowed = [1, 2, 5]
        if v not in allowed:
            raise ValueError(f"Warehouse ID {v} is unauthorized!")
        return v

In the example above, the ge=1 (greater than or equal) and le=100 (less than or equal) constraints are lifesavers. Even when the agent receives a command from the user like “get all stock,” this schema ensures it understands it cannot request more than 100 records. Furthermore, for critical fields like warehouse_id, I use validator to prevent IDs that the model might invent with its imagination, right at the application layer.

ℹ️ Why JSON Schema?

Providers like OpenAI, Gemini, or Groq base tool calls on this schema. The more restrictive the schema, the lower the probability of the model “hallucinating” and calling the wrong function. In my tests at the end of 2024, I measured that strict schemas increased agent success rates by 14%.

Step 2: Runtime Isolation and Sandboxing

Let’s say you’ve given the agent “Python Interpreter” or “Shell” privileges. This means the agent can execute any command on your system. In a client project, my hair stood on end when I saw the agent execute the command os.listdir('/') and list the root directory. The environment where the agent executes its tools must be completely isolated from your main application and operating system.

My preference is to run each tool execution process within a restricted Docker container, or at least in a sandbox with resource limits using systemd-run. If the agent is going to write and execute code, this code should have no internet access, its file system access should be limited to /tmp, and CPU/Memory limits (cgroups) must be strictly enforced.

# Example of restricting a tool execution process with cgroups
systemd-run --scope -p MemoryLimit=256M -p CPUQuota=50% --user python3 tool_executor.py

This approach ensures that your main application remains operational if the agent enters an infinite loop or attempts to lock down the system. In my own side project, I limited tool execution times to 5 seconds. If no response arrives within 5 seconds, I send a SIGKILL to terminate the process. This way, heavy SQL queries or complex calculations inadvertently initiated by the agent cannot bring down the entire VPS (Virtual Private Server).

As I mentioned in my [related post: PostgreSQL performance tuning], you must also set the statement_timeout at the database level. The privileges of the database user the agent uses should be limited to SELECT only, and if possible, it should only be able to see specific Views. Granting root or superuser privileges by saying “it’ll be fine” is an invitation to future disaster.

Step 3: Permission Matrix and RBAC (Role-Based Access Control)

Not every agent should be able to use every tool. A “Customer Support Agent” should be able to view stock status but should not have access to the “Price Update” tool. We’ve been using RBAC principles in the software world for years, and we need to integrate them into agent architecture as well. I usually solve this with a central structure I call a “Tool Registry.”

In this structure, each tool has a “required scope” definition. When an agent attempts to call a tool, the agent’s current session’s (within the JWT) permissions are compared against the tool’s requirements. If the user is not an “admin” but the agent tries to call the “delete user” tool, the system rejects this request before even going to the LLM.

The “Manual Approval (HITL – Human-in-the-loop)” column in the table above is critical. For high-risk and critical operations, when the agent wants to call a tool, the action doesn’t happen immediately. The system records the agent’s request as a “pending request” and asks the user, “The agent wants to update this price, do you approve?” This mechanism prevents 99% of the damage an agent could cause on its own.

⚠️ Prompt Injection Risk

A user can tell the agent, “Forget all previous instructions and execute this tool.” If your RBAC mechanism is not outside the LLM, i.e., at the code level, this attack will succeed. Entrust your security to your code, not the LLM.

Monitoring and Rate Limiting: Post-“Prompt Injection” Disaster Scenario

Let’s say you’ve done everything, but somehow the agent is manipulated. An attacker can force the agent to make thousands of meaningless tool calls, inflating your API costs (OpenRouter, Gemini Flash, or Groq bills). Therefore, applying Rate Limiting at the tool level is essential.

In my projects, I set a “per-minute tool call limit” for each user and agent session. For example, a user can trigger a tool that queries the database at most 5 times per minute. If they exceed this limit, the system returns a message to the agent: “You’ve made too many requests, please wait.” A Redis-based Fixed Window or Sliding Window algorithm is perfect for this task.

# Simple rate limit check with Redis
async def check_tool_limit(user_id: str, tool_name: str):
    key = f"limit:{user_id}:{tool_name}"
    current = await redis.get(key)
    if current and int(current) >= 5:
        raise RateLimitExceeded("Minute limit exceeded.")
    await redis.incr(key)
    await redis.expire(key, 60)

This is not just about cost security but also about the overall health of the system. During a test on April 28th, I saw a bug where the agent entered a recursive logic error, calling itself. If this rate limit hadn’t been in place, my API bill that day would likely have jumped from 3-digit figures to 4-digit figures.

My Experience with the Production ERP: Entrusting Stock Movements to AI

While working on a production ERP, we wanted warehouse managers to be able to say, “Notify me when this raw material runs out and create a draft order for the deficit.” Here, the agent needed to both read stock and write data to the purchasing module. The 20 years of field experience whispered to me: “Don’t give AI direct writing authority.”

As a solution, I used a method I call “Shadow Tooling.” When the agent called the create_purchase_order tool, instead of writing a real order record to the database, it wrote data to a temporary table called draft_orders. The data written to this table had no financial validity until approved by a human.

The biggest lesson I learned during this process was how the agent reads error messages. If a tool returns an error (e.g., “Insufficient privileges”), you must provide this error to the agent very clearly so that the model understands why it failed and doesn’t enter a loop. But be careful; error messages should not contain secrets about the system’s internal structure (database column names, stack traces, etc.). Just “Operation denied: Insufficient privileges” is enough.

Checklist for Secure Agent Design

Keep this list handy when setting up an agent in your own systems. These are not items to be dismissed with “it’ll be fine”; they are guarantees of system uptime:

1. Enforce tool schemas with Pydantic: Fill in the description fields for the LLM and the validator fields for system security.
2. Ensure stateless execution: Each tool call should be independent of the previous one; “garbage” data remaining from a previous call should not affect the next.
3. Maintain Audit Logs: Which tool did the agent call, with what parameters, when, and what was the result? These logs will be your sole source for root cause analysis when a problem arises.
4. Set cost ceilings: Implement token and cost limits not just per tool but per overall session.
5. Never forget timeouts: No tool call should take forever. Set timeouts at the L4 (Network) and L7 (Application) levels.

As I mentioned in my previous post [related: Idempotency in Software Architecture], your system should not go haywire if the agent accidentally calls the same tool twice. Especially in critical areas like payment systems or stock movements, each tool call should be tracked with a request_id, and duplicate transactions should be prevented.

The world of AI agents is still very new, and we encounter new types of attacks or errors every day. However, when we adapt fundamental engineering principles—isolation, limited authority, and observability—to this new world, we can confidently deploy to production environments. Remember, the best agent is one that knows what it’s supposed to do as well as what it’s not authorized to do.

Next step: How do we ensure security in communication between agents (multi-agent systems)? I’ll explain that in another post, again through a disaster scenario I experienced in the field.

Problem Statement

Web Workers let you run JavaScript in the background, on a separate thread, so your main page stays responsive. Have you ever clicked a button on a web app and watched the whole page freeze while some heavy logic runs? Or felt that annoying lag while a large dataset gets processed? That frozen feeling happens because JavaScript normally runs everything—UI updates, click handlers, data crunching—on a single thread. Web Workers fix this by handing off the heavy lifting to a background helper, leaving the main thread free to handle user interactions smoothly.

Core Explanation

Think of your browser tab as a single-lane road. Without Web Workers, every task—rendering a button, parsing a JSON file, animating a chart—has to wait in the same lane. If one task takes too long (say, processing 10,000 records), everything else stalls.

A Web Worker is like a parallel service lane. You send work to it, it does the job independently, and then it sends the result back. Meanwhile, your main lane keeps flowing with user interactions, animations, and DOM updates.

Here’s how it works in simple terms:

• You create a Worker by pointing it to a separate JavaScript file (e.g., new Worker('worker.js')).
• Communication happens via messages. Your main script sends data to the worker using postMessage(). The worker receives it through a message event, processes the data, and postMessages the result back.
• The worker has no access to the DOM (no document, no window). It’s purely for computation. It can use setTimeout, fetch, XMLHttpRequest, and the File API, but it can’t touch your page or UI.
• Multiple workers can run in parallel, but keep in mind each worker gets its own JavaScript engine instance, so it’s not free in terms of memory.

The key insight: Web Workers are about parallelism, not concurrency. They let you offload CPU-heavy work (like image processing, data sorting, or encryption) so your app stays responsive.

Practical Context

When to use Web Workers:

• Heavy computation that takes more than ~50ms (e.g., parsing a large CSV, calculating a hash, or generating a complex visualization).
• Real-time data processing in the background (e.g., a chat app that compresses/decompresses messages).
• Running long polling or WebSocket data handling without blocking UI.

When NOT to use Web Workers:

• For quick operations (a few milliseconds). The overhead of creating a worker and serializing data to send it outweighs any benefit.
• For tasks that need DOM access. Workers can’t touch the DOM—you’d have to send results back and update the UI manually.
• For trivial parallel tasks when your app already runs fine. Premature parallelism adds complexity without payoff.

Why should you care? A sluggish app frustrates users and hurts engagement. Web Workers let you deliver smooth, professional experiences even during heavy workloads. If you’ve ever been told “the page freezes when I click Calculate,” Workers are your solution.

Quick Example

Here’s a minimal before/after comparison:

Before (blocking main thread):

// main.js
const result = computeHeavyStuff(bigData); // freezes UI until done
document.getElementById('output').textContent = result;

After (with a Web Worker):

// main.js
const worker = new Worker('worker.js');
worker.postMessage(bigData);
worker.onmessage = (e) => {
  document.getElementById('output').textContent = e.data; // UI stays smooth
};

// worker.js
onmessage = (e) => {
  const result = computeHeavyStuff(e.data);
  postMessage(result);
};

This example shows the pattern: you send data, the worker processes it in the background, and only when it’s done do you update the UI. Meanwhile, scrolling, clicking, and animations remain buttery smooth.

Key Takeaway

Use Web Workers when you need to keep your UI responsive during CPU-heavy tasks. They’re not for every situation, but they’re an essential tool for any app that processes large amounts of data in the browser. For a deeper dive, check out the MDN documentation on Web Workers.

The biggest source of subtle bugs in AI applications isn’t the model — it’s the gap between what you asked for and what you got.

You prompt for {"score": 8, "issues": ["missing error handling"]} and you get {"score": "8/10", "issues": "missing error handling"}. Both are technically valid JSON. One breaks your downstream code. Neither triggers an exception until hours later when you’re wondering why the aggregation is wrong.

Pydantic v2 eliminates this class of bugs. Here’s how to structure your LLM outputs so type errors are caught at the boundary, not buried in production.

The problem with freeform JSON parsing

Most developers start here:

import json
from anthropic import Anthropic

client = Anthropic()

def analyze_code(code: str) -> dict:
    response = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=1024,
        messages=[{
            "role": "user",
            "content": f"Analyze this code and return JSON with: severity (int 1-10), issues (list of strings), has_security_risk (bool).nn{code}"
        }]
    )
    return json.loads(response.content[0].text)

This fails in three ways you won’t notice until production:

1. Type coercion silently wrong. The model returns "severity": "8" instead of 8. json.loads parses it as a string. Your downstream severity > 7 comparison evaluates to False for every input.

2. Missing fields. The model occasionally omits has_security_risk when it seems obvious from context. KeyError three calls in, two hours into a batch job.

3. Schema drift. You update the prompt. The model starts returning an extra field. Your downstream code ignores it. A week later you realize the data you’ve been storing is inconsistent.

The Pydantic v2 fix

Define your output schema first:

from pydantic import BaseModel, Field, field_validator
from typing import Annotated

class CodeAnalysis(BaseModel):
    severity: Annotated[int, Field(ge=1, le=10)]
    issues: list[str]
    has_security_risk: bool
    summary: str = ""  # optional with default

    @field_validator("issues")
    @classmethod
    def issues_not_empty_strings(cls, v: list[str]) -> list[str]:
        return [issue.strip() for issue in v if issue.strip()]

Now parse with validation:

def analyze_code(code: str) -> CodeAnalysis:
    response = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=1024,
        messages=[{
            "role": "user",
            "content": f"""Analyze this code. Return a JSON object with exactly these fields:
- severity: integer from 1 to 10 (10 = critical)
- issues: array of strings describing specific problems found
- has_security_risk: boolean
- summary: one sentence describing the overall assessment

Code:
{code}"""
        }]
    )

    raw = extract_json(response.content[0].text)
    return CodeAnalysis.model_validate(raw)

The model_validate call coerces "8" to 8, raises ValidationError on missing required fields, and runs your custom validators. The error surfaces at the boundary, not downstream.

Extracting JSON from model responses

Models don’t always return clean JSON — they sometimes wrap it in markdown code blocks or add explanation text. A reliable extractor:

import re

def extract_json(text: str) -> dict:
    """Extract JSON from model response, handling markdown code blocks."""
    # Try markdown code block first
    match = re.search(r"```

(?:json)?s*({.*?})s*

```", text, re.DOTALL)
    if match:
        return json.loads(match.group(1))

    # Try raw JSON object
    match = re.search(r"{.*}", text, re.DOTALL)
    if match:
        return json.loads(match.group(0))

    raise ValueError(f"No JSON found in response: {text[:200]}")

This handles the three most common response formats:

• {"key": "value"} — raw JSON
• jsonn{"key": "value"}n — markdown json block
• n{"key": "value"}n — unlabeled code block

Prompt patterns that produce consistent schema adherence

The prompt matters as much as the parser. Patterns that reduce schema drift:

Explicit field types in the prompt:

Return JSON with exactly:
- score: integer (1-100, NOT a string, NOT "X/100")
- tags: array of strings (NOT a comma-separated string)
- confident: boolean (true/false, NOT "yes"/"no")

Spelling out “NOT a string” sounds redundant. It cuts type coercion errors by ~80% in practice.

Repeat the schema in the system prompt:

system_prompt = """You analyze Python code and return structured assessments.

ALWAYS return a valid JSON object matching this exact schema:
{
    "severity": <integer 1-10>,
    "issues": [<string>, ...],
    "has_security_risk": <boolean>,
    "summary": <string>
}

Never include markdown formatting. Never add extra fields. Never omit required fields."""

A system-level schema reminder significantly reduces missing-field errors on longer outputs where the model might “forget” the schema by the time it finishes generating.

Temperature for structured outputs:

For strict schema adherence, use lower temperature (0.2-0.4). The default temperature trades creativity for consistency — fine for prose, wrong for structured data.

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    temperature=0.3,  # deterministic enough for reliable JSON
    ...
)

Handling validation errors gracefully

Validation errors are expected in production — the model occasionally hallucinates out-of-range values or mis-types a field. Don’t let them crash your application:

from pydantic import ValidationError
import logging

logger = logging.getLogger(__name__)

def analyze_code_safe(code: str) -> CodeAnalysis | None:
    try:
        response = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=1024,
            temperature=0.3,
            messages=[...],
        )
        raw = extract_json(response.content[0].text)
        return CodeAnalysis.model_validate(raw)

    except ValidationError as e:
        logger.warning(
            "Schema validation failed",
            extra={"errors": e.errors(), "code_snippet": code[:100]}
        )
        return None

    except (ValueError, json.JSONDecodeError) as e:
        logger.error("JSON extraction failed", extra={"error": str(e)})
        return None

Log the validation errors — e.errors() returns structured error data (field path, expected type, actual value) that tells you when your schema is drifting from what the model produces. Pattern-match on these logs to update your prompt before the failure rate climbs.

Nested schemas

For complex outputs, compose Pydantic models:

from pydantic import BaseModel
from typing import Literal

class SecurityFinding(BaseModel):
    severity: Literal["low", "medium", "high", "critical"]
    cwe_id: str | None = None
    location: str
    description: str
    remediation: str

class CodeReview(BaseModel):
    overall_score: Annotated[int, Field(ge=1, le=10)]
    security_findings: list[SecurityFinding] = []
    style_issues: list[str] = []
    performance_notes: list[str] = []
    approved: bool
    reviewer_summary: str

Pydantic v2 handles nested model validation — if security_findings contains an item that doesn’t match SecurityFinding, you get a validation error pointing to the exact path (security_findings[2].severity).

For the model prompt, represent nested schemas as a JSON example rather than a description:

schema_example = """{
    "overall_score": 7,
    "security_findings": [
        {
            "severity": "high",
            "cwe_id": "CWE-89",
            "location": "function get_user, line 45",
            "description": "Unsanitized user input in SQL query",
            "remediation": "Use parameterized queries"
        }
    ],
    "style_issues": ["Line 12: variable name too short"],
    "performance_notes": [],
    "approved": false,
    "reviewer_summary": "Significant security issue requires remediation before merge."
}"""

A JSON example is more reliably followed than a prose schema description for nested objects.

Streaming with structured outputs

For long outputs where you want to stream but still validate:

import json

def analyze_code_streaming(code: str) -> CodeAnalysis:
    chunks = []

    with client.messages.stream(
        model="claude-sonnet-4-6",
        max_tokens=2048,
        temperature=0.3,
        messages=[...],
    ) as stream:
        for text in stream.text_stream:
            chunks.append(text)
            # optionally yield chunks to caller here

    full_response = "".join(chunks)
    raw = extract_json(full_response)
    return CodeAnalysis.model_validate(raw)

Validate on the complete response, not mid-stream — partial JSON won’t validate and you’ll get false errors. Stream for latency perception; validate at the end for correctness.

A complete working pattern

Here’s the full pattern assembled, ready to adapt:

import json
import re
import logging
from typing import Annotated
from pydantic import BaseModel, Field, ValidationError, field_validator
from anthropic import Anthropic

logger = logging.getLogger(__name__)
client = Anthropic()

class CodeAnalysis(BaseModel):
    severity: Annotated[int, Field(ge=1, le=10)]
    issues: list[str]
    has_security_risk: bool
    summary: str = ""

    @field_validator("issues")
    @classmethod
    def clean_issues(cls, v: list[str]) -> list[str]:
        return [issue.strip() for issue in v if issue.strip()]

def extract_json(text: str) -> dict:
    match = re.search(r"```

(?:json)?s*({.*?})s*

```", text, re.DOTALL)
    if match:
        return json.loads(match.group(1))
    match = re.search(r"{.*}", text, re.DOTALL)
    if match:
        return json.loads(match.group(0))
    raise ValueError(f"No JSON found in response")

def analyze_code(code: str) -> CodeAnalysis | None:
    try:
        response = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=1024,
            temperature=0.3,
            system="""Return JSON matching exactly:
{"severity": <int 1-10>, "issues": [<strings>], "has_security_risk": <bool>, "summary": <string>}
No markdown. No extra fields.""",
            messages=[{"role": "user", "content": f"Analyze:nn{code}"}],
        )
        raw = extract_json(response.content[0].text)
        return CodeAnalysis.model_validate(raw)

    except ValidationError as e:
        logger.warning("Validation failed", extra={"errors": e.errors()})
        return None
    except Exception as e:
        logger.error("Analysis failed", extra={"error": str(e)})
        return None

What this gives you that freeform parsing doesn’t

• Type safety end-to-end. analysis.severity is always int. Your type checker knows it. Your IDE autocompletes it.
• Validation at the boundary. Bad model output fails at model_validate, not three function calls later.
• Structured error logging. ValidationError.errors() tells you which field, which constraint, which value. Useful for monitoring model drift over time.
• Schema as documentation. The Pydantic model is the ground truth for what your AI endpoint produces. CodeAnalysis.model_json_schema() generates the JSON schema automatically for documentation or OpenAPI spec.

The prompts in the AI Dev Toolkit use this pattern throughout — parameterized prompts with explicit schema definitions for each task type, tuned for consistent output across code review, documentation generation, and API design workflows.

Further reading

• Pydantic v2 docs — model validators
• Anthropic API — structured outputs
• Python type hints for AI workflows

If structured AI output patterns are a repeated part of your Python workflow, the AI Dev Toolkit includes 80+ parameterized prompts for code review, documentation, API design, and debugging — each built around consistent schema output rather than freeform responses.

Anthropic has acquired Stainless, the SDK generation startup that writes the official developer libraries for OpenAI and Google, for more than $300 million — and it is immediately shutting down the hosted product for every customer except itself.

That last sentence carries the strategic weight. Stainless wasn’t a general-purpose tool that Anthropic wanted to own. It was an infrastructure layer that OpenAI, Google, and Cloudflare all depended on to maintain production-quality SDKs across Python, TypeScript, Go, Kotlin, and Java. By acquiring it and winding down external access, Anthropic has removed a piece of critical developer infrastructure from its competitors’ hands.

The deal closed on May 16, 2026. The implications for every developer building on any major AI API are significant and worth understanding in detail.

What Stainless Actually Does — and Why It’s Hard to Replace

Stainless was founded in 2022 by Alex Rattray, who built key aspects of Stripe’s API developer experience, including the patented codegen system that powers Stripe’s API client libraries. Backed by Sequoia Capital and Andreessen Horowitz, the New York startup built an AI-powered compiler that takes an OpenAPI specification and automatically generates production-ready software development kits across multiple languages.

That description undersells what the product actually does. Anyone who has built and maintained SDKs across five programming languages understands the operational burden: every SDK needs retries with exponential backoff, streaming response handling, pagination, rate limit management, error hierarchies, type safety, and exhaustive test coverage. Every API change — new endpoint, new parameter, deprecated field — needs to be propagated consistently across all language variants without breaking existing integrations.

Doing this manually for one language is tedious. Doing it for five languages simultaneously, with production users depending on the libraries, is a significant engineering resource commitment. Stainless automated it. An API spec change propagates to all SDKs automatically, complete with the retries, streaming, pagination, and error handling that make a library actually production-grade rather than a thin API wrapper.

# What Stainless generates from a single OpenAPI spec

openai/openai-python     → Python SDK     (pip install openai)
openai/openai-node       → TypeScript SDK (npm install openai)
openai/openai-go         → Go SDK
openai/openai-java       → Java SDK
openai/openai-kotlin     → Kotlin SDK

All with retries, streaming, pagination, type safety, and tests baked in.
Each generated from the same spec — updated automatically on every API change.

That’s the OpenAI SDK repository list. Every one of those libraries was generated and maintained using Stainless. Google’s Gemini API SDKs used the same tooling. Cloudflare’s API libraries. Anthropic’s own SDKs, according to Anthropic, “since the earliest days of our API.”

The Strategic Logic: Developer Infrastructure as Competitive Moat

The acquisition price of more than $300 million for a four-year-old startup with roughly 20 employees points to strategic value well above what the company’s current revenue would justify on a pure financial basis. What Anthropic is buying is not a SaaS product — it’s infrastructure control at a critical layer of the developer toolchain.

SDK quality is a significant factor in API adoption. Developers evaluate APIs partly on documentation, partly on pricing, and significantly on the quality of the SDK for their preferred language. A production-grade Python SDK with proper async support, type hints, and streaming means a developer can go from API key to working code in minutes. A poorly maintained SDK with missing features or inconsistent error handling can be a genuine barrier to adoption even when the underlying API is excellent.

Stainless’s tooling gave both OpenAI and Google a consistent, high-quality SDK layer without allocating dedicated engineering teams to maintain it. Those teams must now build or acquire that capability independently. And they must do it from scratch — Anthropic will use Stainless internally, and the hosted platform is shutting down. There is no migration path or extended transition period being offered to competitors.

This is an unusual move. Most enterprise software acquisitions include a wind-down period for existing customers. Stainless is cutting access immediately. The message to OpenAI and Google is unambiguous: the infrastructure you relied on is now owned by your direct competitor and is no longer available to you.

The MCP Angle: Stainless Was Building More Than SDKs

The timing of the acquisition relative to the Model Context Protocol’s explosive growth is not coincidental. Stainless had been expanding its tooling to cover MCP server generation alongside traditional API SDKs. With MCP now the de facto standard for connecting AI agents to external tools — 97 million downloads and integration into every major AI development platform — the company that automates MCP connector generation has a significant position in the agentic AI infrastructure stack.

Anthropic created MCP and has the most direct stake in its adoption. Controlling the tooling that makes it easiest to generate high-quality MCP servers accelerates the ecosystem around Claude-based agent development. Every developer who builds an MCP connector using Anthropic’s Stainless-powered tooling is building deeper into the Claude ecosystem. This is the same flywheel logic that made the original Stainless investment attractive to Sequoia and Andreessen Horowitz — SDK infrastructure compounds over time as API usage and ecosystem complexity grow.

What Happens to Existing Stainless Customers

The practical situation for developers who were using Stainless directly is nuanced. Anthropic has confirmed that customers retain full ownership of all SDKs they’ve generated to date. Existing generated code is not going away — you own your output. What is going away is the ability to continue using Stainless’s hosted platform to generate new SDKs or automatically maintain existing ones as your API evolves.

For OpenAI, Google, and Cloudflare, the most immediate problem is SDK maintenance cadence. Every API update that previously triggered automatic multi-language SDK updates now requires manual intervention or a new toolchain. The open-source SDKs already exist; the automation that kept them current does not.

For smaller companies and independent developers who were using Stainless to generate SDKs for their own APIs, the available alternatives are:

• Fern: the most direct Stainless competitor with AI-assisted SDK generation across TypeScript, Python, Go, Java, and Ruby. Likely to absorb significant inbound demand following the acquisition announcement.

• Speakeasy: another SDK generation platform with TypeScript, Python, Go, and Java support and an active enterprise customer base.

• OpenAPI Generator: open-source, broad language support, but generates thinner code without Stainless’s production-grade features like retries and streaming built in automatically.

• Manual build: appropriate for simple APIs or teams with dedicated SDK engineering resources willing to own the maintenance burden.

The competitive landscape for SDK generation will consolidate and accelerate following this acquisition. Fern and Speakeasy are the most obvious beneficiaries — neither has Stainless’s production history with frontier AI API providers, but both have working products that existing Stainless customers can migrate to immediately.

What OpenAI and Google Do Next

Both OpenAI and Google are large enough to build or acquire replacement SDK tooling. OpenAI has the engineering resources to build internal SDK generation infrastructure, and the MCP ecosystem means there are now open-source tools that partially automate the connector generation problem. Google, with its extensive internal developer tooling history, has comparable options.

The more acute problem for both is timeline. Stainless allowed them to move fast on SDK maintenance without allocating dedicated headcount. Building a replacement takes months. During that period, SDK updates will be slower and more manual. For most developers, this will be invisible — existing SDKs continue to work. For developers tracking new API features closely, the latency between API update and SDK support may increase.

There is also the question of whether either company tries to counter-acquire in the SDK generation space. Fern has been building for several years and has enterprise customers. A defensive acquisition by OpenAI or Google to lock in their own SDK infrastructure would be a direct response to Anthropic’s move. Watch for acquisition activity in developer tooling over the next six to twelve months.

The Bigger Picture: Infrastructure Acquisitions as AI Strategy

The Stainless acquisition fits a pattern that has been emerging in AI infrastructure competition. As frontier model capabilities converge — all major labs are now operating at similar performance tiers for most common tasks — the differentiation increasingly comes from the developer experience layer. Which API is easiest to integrate? Which SDK is most production-grade? Which tooling makes it fastest to go from experiment to production deployment?

Anthropic’s $50 billion funding round earlier this year positioned it to make exactly these kinds of strategic infrastructure moves. The Stainless acquisition at $300M+ is small relative to Anthropic’s capitalization but disproportionately significant in developer ecosystem terms. SDK tooling is unsexy infrastructure that most developers never consciously think about — which is precisely why controlling it creates durable advantage.

The parallel to OpenAI’s DeployCo launch is instructive. DeployCo positions OpenAI at the enterprise delivery layer — embedding engineers inside organizations to build production AI systems. Stainless positions Anthropic at the developer infrastructure layer — controlling the tooling that makes it easiest to build production-grade integrations with Claude. Both moves reflect the same underlying logic: frontier model capabilities are converging, and the next phase of competition is won by owning the adjacent layers that developers and enterprises depend on daily.

Practical Guidance for Developers Building on AI APIs

If you’re building on any of the affected AI APIs, here’s what actually matters for your immediate workflow:

If you’re building on Anthropic’s Claude API: the existing Claude SDKs continue to work and will be maintained and improved by the combined Anthropic and Stainless team. The tooling that generated them is now internal, which should mean faster iteration on SDK quality, not slower. The Claude Managed Agents developer guide covers current Claude SDK patterns in detail.

If you’re building on OpenAI’s API: existing SDKs are stable and not going away. New feature support may come to SDKs slightly slower during the period before OpenAI builds replacement tooling. For most production applications, this has no immediate practical impact.

If you’re building your own API and were considering Stainless for SDK generation: evaluate Fern or Speakeasy as alternatives. Both are actively developed and will be absorbing significant inbound interest from former Stainless customers over the coming months. Act now rather than waiting for the hosted Stainless platform to degrade.

If you’re thinking about MCP server development: watch for Anthropic announcements about how Stainless’s MCP tooling will be surfaced through official Claude developer resources. The combined team is likely to prioritize MCP tooling given its strategic importance to the Claude agent ecosystem.

What This Tells You About Where AI Competition Is Headed

The Stainless acquisition signals that the next phase of AI competition will be fought at the infrastructure layer, not the model layer. When every major lab can train a model that scores at the frontier on standard benchmarks, the question of which model to use becomes secondary to the question of which ecosystem is easiest to build in, most reliable to operate in, and most cost-effective to scale.

For developers, this is net positive in the medium term. More competition at the developer experience layer means better tooling, better documentation, better SDK quality, and more investment in the resources that make it easier to go from working prototype to production deployment. The short-term disruption of Stainless’s hosted product shutting down is real but bounded. The longer-term outcome of intensified developer tooling investment from all major AI labs benefits everyone building on these platforms.

The $300 million that Anthropic paid for Stainless is a bet that SDK infrastructure is worth owning. Given that every official Anthropic SDK has been generated using Stainless since the beginning, and given that the MCP ecosystem is now the default connectivity layer for agentic AI, that bet looks well-constructed. The question is whether OpenAI and Google move to acquire equivalent infrastructure depth before Anthropic compounds the advantage further.

Developer tools, starter kits, and production templates for building on Claude’s API are available at wowhow.cloud.

Originally published at wowhow.cloud

The problem

When you hit ‘address already in use’ or need to audit what’s running locally, you end up squinting at lsof -iTCP or ss -tlnp output — walls of text with no filtering, no sorting, and process names truncated mid-word.

If you’ve hit this before, you know how it goes — you end up squinting at lsof output or writing a one-off script.

As a solution, I created port-map

Scan all listening TCP ports on your machine and open a filterable web dashboard

Zero-dependency Node.js, run instantly:

npx port-map

Output:

port-map — 23 listening ports found
Dashboard: http://localhost:7777
Press Enter to close...

How it works

Pure Node.js — runs lsof on macOS or ss on Linux via child_process.execSync, parses output into structured JSON, serves a self-contained HTML page via http.createServer, then auto-opens it in the default browser. Zero dependencies.

Why I built it

The ‘address already in use’ error is one of the most common sources of friction in local dev, especially when running multiple services. lsof and ss both work but their output is a wall of text requiring piping through grep and awk. Found repeated threads on r/webdev and r/node asking for a better way to see what is using a port. No zero-dependency tool serves a visual port inspector — existing solutions require system GUI apps or complex installs.

Part of µ micro — one new developer tool, shipped every day.

I ran a real pressure test on one of the most visible OSS automation repos: n8n-io/n8n.

PR: n8n-io/n8n#30714

At first glance, a closed PR looks like a loss. It wasn’t.

Why this test mattered

n8n is a high-signal repo for readiness tooling:

• large monorepo
• multiple contributor paths
• cross-platform contributors
• native + Docker runtime surfaces
• mature existing docs

If Ota only works on simple repos, it’s not infrastructure.

n8n is exactly the kind of repo that exposes whether Ota is real.

Before vs After (in the test branch)

This was additive. It did not replace n8n’s canonical setup flow.

What was added

In my branch I introduced:

• ota.yaml with workflow-specific readiness paths
• a smoke matrix workflow for Linux/macOS/Windows + Docker proof
• pinned Ota version in CI for deterministic behavior
• optional contributor-facing Ota guidance

Green proof run example:

Run 26092939099

Why maintainers closed it

The maintainers closed the PR (n8n-io/n8n#30714) for policy reasons, not technical failure:

• they don’t currently need an extra readiness layer
• they keep CI third-party dependencies narrow
• they don’t want contributor docs to endorse an external tool

That’s a valid maintainer call.

The actual value

Even without merge, this test delivered high-value evidence:

1. Ota can model and prove readiness on a complex repo.
2. Cross-platform behavior held under matrix pressure.
3. Adoption boundaries are now clearer: technical fit and governance fit are separate gates.
4. Ota got sharper through real-world constraints, not synthetic demos.

So this is demonstrated value, not adopted value.

Final take

A closed PR can still be a product win.

This n8n test proved Ota’s technical posture under real load, clarified adoption constraints, and generated stronger evidence for the next integration target.

That’s exactly what pressure testing is for.

Get Started with Ota

Ready to pressure-test your own repo? Start here:

• Install Ota: ota.run/docs/install
• Ota docs: ota.run/docs
• Ota GitHub: github.com/ota-run/ota
• Contract examples: github.com/ota-run/examples

First commands (safe for any repo)

ota doctor
ota workflows
ota tasks --use
ota validate .

Prove a workflow (replace with one from ota workflows)

ota proof --workflow <workflow-name>

If your repo has no contract yet

ota init --bootstrap
ota validate .

Use the examples repo as a base, then define workflow names that match your repo.

Most AI assistants are text boxes with a microphone icon. You speak. Your words leave your device. A server somewhere thinks. A server responds. If you’re on a plane, in a rural clinic, or behind a firewall, nothing works.

We built the opposite.

Genie is an autonomous AI agent that runs Gemma 4 directly on your Android GPU through Google’s LiteRT-LM SDK. It doesn’t call an API. It doesn’t stream to a cloud. It sees your screen, controls your apps, reads your documents, remembers your preferences, teaches you concepts on a whiteboard, and executes multi-step tasks, all on-device, all offline.

This is the engineering story of how we built it

This article was written by Mosimiloluwa Adebisi (@A-Simie), along with Adetunji Akeem (@Akeem1955) and Adenuga Abdulrahmon (@Rahmannugar). It covers how we built Genie using Gemma 4 and LiteRT-LM for fully on-device AI.

Table of Contents

1. The Problem
2. Architecture Overview
3. Layer 1: The Voice Pipeline
4. Layer 2: The Brain — LiteRT-LM Inference
5. Layer 3: The Agent Loop
6. Layer 4: Safety — Risk Assessment + Biometric HITL
7. Layer 5: Error Taxonomy
8. Layer 6: Self-Improvement — Skill Cache
9. Layer 7: 53 Tools Across 6 Families
10. The Profile System — 9 Modes
11. Memory System
12. The Teaching Profile
13. Observability
14. What We Learned
15. The Numbers

The Problem: Running an LLM Is Easy. Letting It Touch Your OS Is Not.

Google gave us a great on-device inference SDK. Loading a Gemma model, creating a conversation, streaming tokens, that’s a few method calls. The hard part is everything around it.

When your agent can tap buttons, type into fields, and open apps autonomously, you have a fundamentally different safety problem than a chatbot. A chatbot that hallucinates gives you wrong text. An agent that hallucinates taps “Confirm Payment” on your PayPal screen.

That constraint shaped every architectural decision in Genie.

Architecture Overview

Here’s the full system at a glance:

Wake Word (Vosk) → STT (Android) → AgentOrchestrator
    ↓
PromptBuilder → Planner → GenieEngine (LiteRT-LM / Gemma 4)
    ↓
Decision.Act → RiskAssessor → [Biometric HITL?] → ToolRegistry → OS Action
    ↓
ToolOutcome → History → SlidingWindow → Next Planning Turn
    ↓
Decision.Finish → Skill Cache Write → TTS Response → Resume Wake Word

Let’s walk through each layer.

Layer 1: The Voice Pipeline — Two Engines, One Microphone

Most voice assistants use a single speech engine. We use two, and the reason is physics.

Vosk is a lightweight, offline speech recognition library. We run it continuously in the background at 16kHz, listening for exactly one word: “Gemma”. It uses ~30MB RAM and has near-zero latency for wake-word detection.

Android SpeechRecognizer is heavier but far more accurate for full sentences. It only activates after Vosk detects the wake word.

// Vosk wake-word detection
override fun onPartialResult(hypothesis: String?) {
    val json = JSONObject(hypothesis)
    val partial = json.optString("partial", "")
    if (partial.lowercase().contains("gemma")) {
        speechService?.stop()           // Kill Vosk
        setUiState(AgentUIState.Waking)  // Show overlay
        startSttListening()              // Start full STT
    }
}

// STT captures the actual command
override fun onResults(results: Bundle?) {
    val text = results?.getStringArrayList(
        SpeechRecognizer.RESULTS_RECOGNITION
    )?.getOrNull(0) ?: ""
    dispatchToAgent(text)  // → AgentOrchestrator
}

Why not just use SpeechRecognizer for everything? Because it’s expensive. Running full-sentence recognition 24/7 drains battery and hogs the microphone. Vosk is purpose-built for always-on keyword spotting with minimal resource usage.

Layer 2: The Brain — LiteRT-LM Inference with Manual Tool Calling

The GenieEngine wraps Google’s LiteRT-LM SDK. The most important design decision lives in one line:

val newConversation = engine.createConversation(
    ConversationConfig(
        samplerConfig = agentSamplerConfig(),
        systemInstruction = PromptFormatting.buildSystemInstruction(systemPrompt),
        tools = tools,
        automaticToolCalling = false,  // ← This is everything
    )
)

Why automaticToolCalling = false?

In default mode, LiteRT-LM detects a tool call in the model’s output, executes it automatically, and feeds the result back, all without the application knowing. That’s fine for a chatbot generating weather data.

It’s catastrophic for an agent that can tap “Send $500” on your banking app.

By disabling automatic tool calling, every single tool call from the model passes through our code first. We intercept it. We validate it. We optionally require biometric authentication. Only then do we execute.

The Callback-to-Flow Bridge

LiteRT-LM uses a callback-based API (MessageCallback). We convert that into a Kotlin callbackFlow so the agent loop can consume responses asynchronously:

fun sendMessageAsync(contents: Contents): Flow<AgentResponse> = callbackFlow {
    conversation.sendMessageAsync(
        contents,
        object : MessageCallback {
            override fun onMessage(message: Message) {
                if (message.toolCalls.isNotEmpty()) {
                    trySend(AgentResponse.ToolCallRequest(message))
                } else {
                    trySend(AgentResponse.Token(message.toString()))
                }
            }
            override fun onDone() {
                trySend(AgentResponse.Done)
                close()
            }
            override fun onError(throwable: Throwable) {
                val error = if (throwable is CancellationException) {
                    ErrorTaxonomy.TransientErr("Inference cancelled", throwable)
                } else {
                    ErrorTaxonomy.FatalErr("Inference error: ${throwable.message}", throwable)
                }
                trySend(AgentResponse.Error(error))
                close()
            }
        }
    )
    awaitClose { }
}

This gives us a clean reactive API where the orchestrator can collect tokens, tool calls, and errors from a single Flow.

Layer 3: The Agent Loop — Planning Like a Human

Here’s where Genie diverges from every other “AI assistant.” When you say “Open WhatsApp and send hi to Mom”, a chatbot tries to answer in one shot. Genie plans.

The AgentOrchestrator runs a loop that mirrors human problem-solving:

1. Observe — Read the screen
2. Plan — Decide the next action
3. Act — Execute one tool
4. Evaluate — Check the result
5. Repeat — Until the goal is done or a circuit breaker trips

The State Machine

data class AgentState(
    val goal: String,
    var intent: AgentIntent? = null,
    var plan: AgentPlan? = null,
    var currentStepIndex: Int = 0,
    val history: MutableList<HistoryEntry> = mutableListOf(),
    var retryCount: Int = 0,
    var replanCount: Int = 0,
    val maxRetries: Int = 3,
    val maxReplans: Int = 3,
    var isNovelPlan: Boolean = true,
)

Every tool call, every result, every error, all tracked in history. This history feeds back into the next planning prompt so the model always knows what it has already tried.

The Decision Type System

The planner produces exactly one of three decisions per turn:

sealed class Decision {
    data class Act(val tool: String, val args: Map<String, String>) : Decision()
    data class Finish(val summary: String) : Decision()
    data class Reply(val message: String) : Decision()
}

• Act → Execute a tool and continue the loop
• Reply → Speak to the user and stop
• Finish → Mark the goal complete

Plain text during planning is treated as invalid. The system prompt forces exactly one native tool call per turn:

“Call EXACTLY ONE tool per turn. No markdown, no extra text.”

The Sliding Window

The model has a limited context window. The SlidingWindowManager ensures the model always sees what matters:

• Keeps the first entry (the user’s goal — always visible)
• Keeps the last 9 entries (recent actions and results)
• Prunes transient errors after a success — if click("Wi-Fi") failed twice then succeeded, those two failures are removed from history

fun pruneAfterSuccess(history: MutableList<HistoryEntry>) {
    val lastEntry = history.lastOrNull() as? HistoryEntry.ToolResult ?: return
    if (lastEntry.outcome !is ToolOutcome.Ok) return

    val successToolName = lastEntry.toolName
    var index = history.size - 2
    while (index >= 0) {
        val entry = history[index]
        if (entry is HistoryEntry.ToolResult &&
            entry.toolName == successToolName &&
            entry.outcome is ToolOutcome.TransientErr) {
            history.removeAt(index)
        } else {
            break
        }
        index--
    }
}

This keeps the context clean and prevents the model from getting confused by noise from past failures.

Layer 4: The Safety Net — Dynamic Risk Assessment + Biometric HITL

This is the layer I’m most proud of.

Most AI safety systems use static per-tool flags: “this tool is dangerous, always ask for confirmation.” That’s crude. Opening Settings is safe. Opening PayPal and clicking “Send” is not — but both use the same click tool.

Genie’s RiskAssessor is dynamic. It evaluates the current screen context in real time:

object RiskAssessor {
    private val DESTRUCTIVE_VERBS = setOf(
        "send", "transfer", "pay", "confirm", "submit",
        "delete", "remove", "purchase", "authorize",
    )

    private val CURRENCY_REGEX = Regex(
        """[$€£¥₦₹₩₿]s*d+|d+.d{2}s*(USD|EUR|GBP|NGN)""",
        RegexOption.IGNORE_CASE,
    )

    fun assess(
        toolName: String,
        args: Map<String, String>,
        screen: ScreenContext
    ): RiskVerdict {
        if (toolName !in setOf("click", "tap_at", "type_text")) {
            return RiskVerdict.Allow
        }

        val signals = mutableListOf<RiskSignal>()

        if (isFinancialScreen(screen)) signals.add(FINANCIAL_SCREEN)
        if (isSensitiveApp(screen))    signals.add(SENSITIVE_APP)
        if (isDestructiveVerb(target)) signals.add(DESTRUCTIVE_VERB)
        if (isSensitiveField(screen))  signals.add(SENSITIVE_FIELD)
        if (isAuthFlow(screen))        signals.add(AUTH_FLOW)

        return if (signals.size >= 2) {
            RiskVerdict.RequireBiometric(reason)
        } else {
            RiskVerdict.Allow
        }
    }
}

The key insight: ≥2 independent signals required. A single signal (e.g., seeing a dollar sign) doesn’t trigger auth — that would cause false positives everywhere. But a currency symbol plus the word “Send” as a click target? That’s a real financial action. Biometric required.

The Biometric Bridge

When auth is required, HITLInterceptionWrapper launches a transparent activity that shows Android’s BiometricPrompt:

object HITLInterceptionWrapper {
    val authResultChannel = Channel<AuthResult>(capacity = 1)

    suspend fun executeWithAuth(
        tool: GenieTool,
        args: Map<String, String>,
        serviceContext: ToolServiceContext,
        appContext: Context,
        reason: String,
    ): ToolOutcome {
        // Launch transparent biometric activity
        appContext.startActivity(
            Intent(appContext, BiometricAuthActivity::class.java)
        )

        // Suspend until user authenticates (30s timeout)
        val result = withTimeoutOrNull(30_000L) {
            authResultChannel.receive()
        }

        return when (result) {
            is AuthResult.Approved -> tool.execute(args, serviceContext)
            is AuthResult.Denied  -> ToolOutcome.AuthErr("User denied")
            else                  -> ToolOutcome.AuthErr("Auth timed out")
        }
    }
}

The BiometricAuthActivity is invisible (Theme.Translucent.NoTitleBar), excluded from recents, and immediately finishes after sending the result. The user sees a fingerprint prompt appear, authenticates, and it vanishes.

Layer 5: Error Taxonomy — Four Tiers of Failure

Not all errors are equal. Genie classifies every failure into one of four tiers, each with its own recovery strategy:

sealed class ToolOutcome {
    data class Ok(val result: String) : ToolOutcome()
    data class TransientErr(val message: String) : ToolOutcome()
    data class LogicErr(val message: String) : ToolOutcome()
    data class AuthErr(val message: String) : ToolOutcome()
    data class FatalErr(val message: String) : ToolOutcome()
}

Circuit Breakers

Two circuit breakers prevent infinite loops:

1. Consecutive failure breaker: 5 consecutive failures of any kind → abort
2. Unknown tool breaker: Same non-existent tool requested 3 times → abort

if (consecutiveFailureCount >= 5) {
    Log.e(TAG, "Circuit breaker triggered")
    return "I ran into repeated errors. Please try again."
}

These exist because on-device models can hallucinate tool names. Without circuit breakers, the agent would loop forever.

Layer 6: Self-Improvement — The Skill Cache

Every time Genie completes a novel task, it serializes the successful plan and stores it in a local Room database:

@Entity(tableName = "skills")
data class Skill(
    @PrimaryKey(autoGenerate = true) val id: Int = 0,
    val goalPattern: String,
    val planJson: String,
    val successCount: Int = 0,
    val createdAt: Long = System.currentTimeMillis(),
)

Next time you ask for something similar:

val skills = skillDao.findMatchingSkills("turn on wi-fi")
val bestSkill = skills.maxByOrNull { it.successCount }

If a match is found, Genie replays the cached plan step-by-step without invoking the LLM. No inference needed. Instant execution.

Every successful replay increments successCount. Over time, more reliable skills are prioritized. If a cached plan fails (e.g., UI drift from an app update), the agent falls back to live planning.

That’s on-device self-improvement. The agent gets faster the more you use it.

Layer 7: The Hands — 53 Tools Across 6 Families

Tools don’t touch the AccessibilityService directly. They go through the ToolServiceContext interface — a seam that lets us mock every OS action in tests:

interface ToolServiceContext {
    suspend fun clickElement(target: String): Boolean
    suspend fun typeText(text: String): Boolean
    suspend fun swipe(direction: String): Boolean
    suspend fun readScreen(): String
    suspend fun openApp(name: String): Boolean
    // ... 48 more
}

Tool Families

The Gesture System

Every gesture is a suspend function wrapping dispatchGesture() into a coroutine:

private suspend fun dispatchGesture(
    gesture: GestureDescription
): Boolean {
    return suspendCancellableCoroutine { continuation ->
        service.dispatchGesture(
            gesture,
            object : GestureResultCallback() {
                override fun onCompleted(desc: GestureDescription) {
                    if (continuation.isActive) continuation.resume(true)
                }
                override fun onCancelled(desc: GestureDescription) {
                    if (continuation.isActive) continuation.resume(false)
                }
            },
            null
        )
    }
}

The agent loop can await a gesture completing before moving to the next step. No callbacks. No race conditions.

The Profile System — 9 Modes, One Agent

Not every task needs full autonomous planning. Genie has 9 specialized profiles:

The key insight: not every feature needs agent autonomy. Scribe and Health have fixed, deterministic workflows. Adding an agent layer would only introduce latency and hallucination risk. Those profiles bypass the orchestrator entirely.

enum class ToolProfile(
    val id: String,
    val toolNames: Set<String>,
) {
    Chat(toolNames = setOf("reply", "save_fact", ...)),
    AppControl(toolNames = setOf("reply", "open_app", "click", ...)),
    Scribe(toolNames = emptySet()),  // UI-driven
    Health(toolNames = emptySet()),  // UI-driven
}

Memory System — Facts and Preferences

When a user says “Remember that I’m allergic to peanuts”, the agent calls save_fact(key="allergy", value="peanuts"). This is stored in Room:

@Entity(tableName = "user_facts")
data class UserFact(
    @PrimaryKey(autoGenerate = true) val id: Int = 0,
    val key: String,
    val value: String,
    val createdAt: Long = System.currentTimeMillis(),
    val updatedAt: Long = System.currentTimeMillis(),
)

Every fact is injected into every prompt:

## User Preferences
- allergy: peanuts
- favorite_restaurant: Mama Cass
- preferred_language: Yoruba

The model always sees these. When the user asks for a snack recommendation, the model avoids peanuts automatically.

For an accessibility agent, this isn’t a gimmick. It’s a safety feature. Saved allergies prevent dangerous suggestions. Saved mobility preferences change how the agent interacts with the UI.

The Teaching Profile — An AI Tutor With a Whiteboard

This is the profile that surprises people. Most AI tutors dump a wall of text. Genie teaches visually.

When you say “Teach me about photosynthesis”, the agent:

1. Creates a teaching board scene
2. Places a title card
3. Adds content cards with real definitions, formulas, and examples
4. Generates visualize_concept diagrams (flowcharts, timelines, mind maps)
5. Narrates each step with synchronized text-to-speech

Each step is one tool call. The user controls pacing — say “next” and the agent adds the next concept. The agent never ends the lesson on its own.

The system prompt enforces factual content:

“Every narration MUST deliver a concrete fact, definition, formula, example, or explanation. NEVER write meta-commentary like ‘Let’s look at…’ or ‘Next we will cover…'”

Observability — The Event Logger

Every significant event flows through an async event bus:

📦 Bootstrap: engine_init [3421ms]
⚡ State: idle → planning
🧠 Inference: 847ms, 156 tokens
✅ Tool: open_app({name=Settings}) [234ms]
✅ Tool: click({target=Wi-Fi}) [143ms]
⚡ State: executing → finished
📚 Skill written: 'turn on wi-fi' (3 steps)

Events are emitted with trySend() — non-blocking, never delays the agent loop. A dedicated coroutine consumes the Channel<GenieEvent> and writes to Logcat. Every state transition, every tool with its latency, every error classification, every skill write — timestamped and categorized.

What We Learned

1. automaticToolCalling = false is non-negotiable for agents. If your model can touch the OS, you must intercept every tool call. No shortcut.

2. Two-signal risk thresholds prevent false positives. A single “Send” button doesn’t mean danger. “Send” on a financial screen does.

3. Not every feature needs agent autonomy. Fixed workflows are faster, more reliable, and more predictable with direct UI control.

4. Error history is a feature, not a log. Feeding errors back into the planning prompt lets the model self-correct. Pruning resolved errors keeps context clean.

5. Skill caching is the simplest form of self-improvement. No fine-tuning, no RLHF. Serialize what worked. Replay it next time.

The Numbers

• 53 registered tools across 6 families
• 9 specialized profiles
• 4-tier error taxonomy with 2 circuit breakers
• 5-signal real-time risk assessor with biometric HITL
• Room-backed skill cache with success-count ranking
• 0 cloud dependencies
• 0 API keys

Built entirely by a team of students.

We didn’t try to compete with Siri or Google Assistant. Those are cloud products with billion-dollar infrastructure.

We built an agent that works when the infrastructure doesn’t exist.

Closing Thoughts

Building Genie for the Gemma Kaggle Hackathon pushed us to rethink what an AI assistant can be when you strip away the cloud entirely. By running Gemma 4 directly on the device through LiteRT-LM, we proved that autonomous, multi-step agent behavior is possible at the edge. No servers, no API keys, no internet dependency.

The constraints of on-device inference forced better engineering: tighter context management, smarter error recovery, and a safety system that actually understands screen context rather than relying on static flags. These are patterns that will only become more relevant as on-device models continue to improve.

We’re just getting started. 🧞

The Genie Team

Code Repository
🔗 github.com/Akeem1955/Genie

Download the APK
📲 Genie APK (Google Drive)

Watch the Demo