Most AI Tools Are Just LLM Wrappers. Here’s What Actually Matters.

In 2025, AI wrapper startups raised over $10 billion. The product? Take an LLM API. Add a text box. Maybe some prompt templates. Charge $30/month. Call it “AI-powered.”

Not mad at the hustle. But if your entire product disappears the moment ChatGPT adds your feature for free, you don’t have a product. You have a timing play.

The Wrapper Test

One question tells you everything:

Can you replicate the output by pasting the same input into ChatGPT or Claude?

If yes: it’s a wrapper. You’re paying for UI and convenience, not intelligence.

If no: because it’s pulling from multiple data sources, applying domain logic, or integrating with real systems, it might be something real.

Most fail the test.

Thin vs. Thick

Not all wrappers are equal. The market is splitting fast:

Thin Wrapper Thick Wrapper
What it does UI + API call + system prompt Real integrations, domain logic, data pipelines
Defensibility None. One platform update kills it High. Value is in the connectors
Example “AI email writer” (GPT call with a system prompt) Cursor (reads your codebase, understands project context)
Survival odds Low Decent

The graveyard of 2025–2026 is littered with thin wrappers that a platform update made irrelevant overnight.

What Actually Matters

Strip away the wrapper. Where does the real value live?

1. Connectors

The ability to talk to real systems: Salesforce, Jira, databases, email, file storage, APIs. This is where 80% of the actual work lives.

Getting an AI to generate text is trivial. Getting it to read your CRM records, cross-reference tickets, update a database, and notify Slack. That’s integration work. That’s hard. That’s valuable.

Most wrappers don’t touch this. They live in the text-in, text-out world.

2. Captured Domain Expertise

An AI that’s been learning your industry’s quirks for months is worth more than a fresh GPT-5 instance with a clever prompt.

Fresh AI + Great Prompt AI + 6 Months of Learnings
Platform quirks Discovers them painfully Already knows them
Common mistakes Makes them all Has guardrails for each
Your terminology Constant correction needed Uses it naturally
Edge cases Surprised every time Documented patterns

The knowledge compounds. Every session, every bug fix, every “oh, that’s how this actually works” gets captured and fed back.

No wrapper captures this. They start fresh every time.

3. Methodology

How you approach problems with AI matters more than which model you use.

The wrapper approach: open tool → type request → get output → hope it’s right.

The practitioner approach:

  1. Small test: constrained input, see what happens
  2. Evaluate: what worked? What broke?
  3. Capture: document the learning
  4. Adjust: update the approach
  5. Repeat

The tool is 10%. The methodology is 90%.

The “Just Build It” Case

Here’s the uncomfortable truth. Building your own system (even ugly, even scrappy) gives you something no wrapper provides: understanding.

You know why it works. Why it breaks. How to fix it. When the model changes (and it will), you swap the engine. The connectors, the learnings, the guardrails. Those persist. They’re yours.

Cost at scale:

Wrapper Stack Custom (Direct API)
Month 1 $150/seat, fast setup $500 dev time, slower start
Month 6 $150/seat, same capabilities $50/month API, growing capabilities
Year 1 (5 seats) $9,000 ~$3,100 + compound knowledge

Custom costs less AND gets smarter. The wrapper costs the same and stays the same.

The Philippines advantage: smaller teams with direct API access can outperform larger orgs paying for wrapper stacks. When you can’t afford $150/seat for 6 different AI tools, you build one system that does what you need. That constraint produces better architecture.

When Wrappers DO Make Sense

Fair is fair:

  • Speed to market: need something running tomorrow without engineering capacity? Wrapper gets you there.
  • Thick wrappers with real integrations: Cursor, Harvey, Perplexity add genuine value beyond the API call.
  • Exploration phase: trying 5 wrappers to understand the capability space before building your own is smart R&D.

The key question:

Are you buying a tool or renting a feature?

If the value prop is “we make it easy to talk to an LLM,” that feature is getting commoditized in real time. Every model provider is making their native interface better, faster, cheaper.

What to Build Instead

Ready to go beyond wrappers? Start here:

1. Map your connectors. What systems does your AI need to talk to? Build those integrations first. Hardest part. Most valuable.

2. Capture everything. Every platform quirk. Every failed approach. Every successful pattern. Your AI should learn from your organization’s experience, not start fresh every session.

3. Own your methodology. Document how you approach problems with AI. Small tests → captured learnings → iteration. More valuable than any tool you can buy.

4. Accept ugly. The most effective AI systems I’ve built are not pretty. Config files, markdown documents, scripts. They look like plumbing. They work like machines.

Bottom Line

The moat isn’t the model. It never was.

It’s the connectors that talk to your stack. The domain expertise captured over months. The methodology that turns every failure into a lesson.

None of that lives in a wrapper.

I’m Tom Tokita. I run Aether Global Technology out of Manila. We build production AI and Salesforce systems for enterprises that need real integrations, not another wrapper. Let’s talk.

Read next: Context Engineering: Why Your AI Strategy Needs Infrastructure, Not Better Prompts · Autonomous AI Agents Look Great in Demos. Here’s What They Cost in Production.

Environment Variables Done Right: The .env Guide Every Developer Needs (2026)

Environment Variables Done Right: The .env Guide Every Developer Needs (2026)

Stop hardcoding secrets. Stop losing config between environments. Here’s how to manage env vars like a pro.

Why Environment Variables Matter

// ❌ NEVER do this
const DB_PASSWORD = 'SuperSecret123!';
const API_KEY = 'sk-live-abc123456789';
const STRIPE_SECRET = 'sk_live_...';

// These end up in git, in logs, in your brain (worst of all)

// ✅ ALWAYS do this
const DB_PASSWORD = process.env.DB_PASSWORD;
const API_KEY = process.env.API_KEY;

The Basics

What Are Environment Variables?

Key-value pairs available to your process:

# Set in shell
export NODE_ENV=production
export PORT=3000

# Pass when running command
PORT=8080 node server.js

# From a file (.env)
DB_HOST=localhost DB_PORT=5432 node server.js

In Node.js

// All env vars live on process.env
console.log(process.env.HOME);        // /root
console.log(process.env.PATH);        // /usr/bin:...
console.log(process.env.NODE_ENV);    // undefined (unless set)
console.log(process.env.MY_VAR);      // whatever you set it to

The Standard: dotenv

npm install dotenv
// src/config/index.js — Load as early as possible
import dotenv from 'dotenv';
import { fileURLToPath } from 'url';
import path from 'path';

const __dirname = path.dirname(fileURLToPath(import.meta.url));

// Load .env from project root (goes up from src/)
dotenv.config({ path: path.resolve(__dirname, '../../.env') });

// Now all vars are on process.env!
console.log(process.env.DATABASE_URL);

.env File Format

# .env — NEVER commit this file!

# Server
NODE_ENV=development
PORT=3000
HOST=0.0.0.0

# Database
DATABASE_URL=postgres://user:pass@localhost:5432/myapp
DATABASE_POOL_SIZE=10

# Redis
REDIS_HOST=127.0.0.1
REDIS_PORT=6379

# JWT
JWT_SECRET=your-super-secret-key-at-least-32-chars
JWT_EXPIRES_IN=7d

# External APIs
OPENAI_API_KEY=sk-...
STRIPE_SECRET_KEY=sk_live_...

# Feature flags
ENABLE_CACHE=true
RATE_LIMIT=100

.env.example — For Your Team

# .env.example — COMMIT THIS FILE!

# Copy this to .env and fill in your values:
cp .env.example .env

# Server
NODE_ENV=development
PORT=3000

# Database
DATABASE_URL=postgres://user:pass@localhost:5432/myapp
DATABASE_POOL_SIZE=10

# Redis
REDIS_HOST=127.0.0.1
REDIS_PORT=6379

# JWT — Generate with: openssl rand -base64 32
JWT_SECRET=change-me-in-production

# External APIs
OPENAI_API_KEY=
STRIPE_SECRET_KEY=

# Feature flags
ENABLE_CACHE=true
RATE_LIMIT=100

Validation (Don’t Skip This!)

// src/config/index.js with validation
import dotenv from 'dotenv';
import path from 'path';

dotenv.config();

const requiredVars = [
  'NODE_ENV',
  'PORT',
  'DATABASE_URL',
  'JWT_SECRET',
];

const missingVars = requiredVars.filter(key => !process.env[key]);

if (missingVars.length > 0) {
  console.error('❌ Missing environment variables:');
  missingVars.forEach(v => console.error(`   - ${v}`));
  console.error('nCopy .env.example to .env and fill in the values.');
  process.exit(1);
}

// Type-safe config with defaults
const config = {
  nodeEnv: process.env.NODE_ENV || 'development',
  port: parseInt(process.env.PORT) || 3000,

  database: {
    url: process.env.DATABASE_URL,
    poolSize: parseInt(process.env.DATABASE_POOL_SIZE) || 10,
  },

  jwt: {
    secret: process.env.JWT_SECRET,
    expiresIn: process.env.JWT_EXPIRES_IN || '7d',
  },

  isDev: () => (process.env.NODE_ENV || 'development') === 'development',
  isProd: () => process.env.NODE_ENV === 'production',
  isTest: () => process.env.NODE_ENV === 'test',
};

// Freeze in production to prevent accidental mutation
if (config.isProd()) {
  Object.freeze(config);
}

export default config;

Multi-Environment Setup

project/
├── .env                # Local dev (gitignored)
├── .env.development   # Dev overrides (optional)
├── .env.production     # Production template (committed)
├── .env.test           # Test values (committed)
├── .env.example        # Template for new devs (committed)
└── .gitignore          # Includes .env, .env.local
// Choose which .env file based on NODE_ENV
import dotenv from 'dotenv';
import path from 'path';
import { fileURLToPath } from 'url';

const __dirname = path.dirname(fileURLToPath(import.meta.url));
const envFile = `.env.${process.env.NODE_ENV || 'development'}`;

dotenv.config({ 
  path: path.resolve(__dirname, `../../${envFile}`),
  override: true 
});

// Fallback to default .env
dotenv.config({ path: path.resolve(__dirname, '../../.env') });

Docker + Environment Variables

# Dockerfile
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
EXPOSE 3000
CMD ["node", "server.js"]
# docker-compose.yml
version: '3.8'

services:
  app:
    build: .
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
      - DATABASE_URL=postgres://app:secret@db:5432/appdb
      - REDIS_URL=redis://redis:6379
    env_file:
      - .env.production  # Load from file
    depends_on:
      - db
      - redis

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: appdb

  redis:
    image: redis:7-alpine

Best practice: Use environment for shared/secrets and env_file for non-sensitive defaults.

Security Rules

1. Never Commit .env

# .gitignore (add these lines)
.env
.env.local
.env.*.local
*.env
!.env.example
!.env.production.template
!.env.test

2. Use Different Keys Per Environment

// Development can use weaker keys
// Production MUST use strong keys
if (config.isProd() && config.jwt.secret.length < 32) {
  throw new Error('JWT_SECRET must be at least 32 characters in production');
}

3. Rotate Secrets Regularly

# Generate new secrets
openssl rand -base64 32  # JWT secret
openssl rand -hex 32     # API key format

# Update .env and restart:
# 1. Change value in .env
# 2. Restart application
# 3. Update any external services that used old key

4. Don’t Log Env Vars

// ❌ BAD
app.use((req, res, next) => {
  console.log('Config:', process.env); // Leaks ALL secrets!
  next();
});

// ✅ GOOD
app.use((req, res, next) => {
  console.log(`[${new Date().toISOString()}] ${req.method} ${req.path}`);
  next();
});

Debugging Common Issues

Problem Cause Fix
undefined when reading var .env not loaded Check dotenv.config() runs before imports
Var loads in dev but not prod .env not deployed Add to docker-compose or CI secrets
Special characters break Quotes missing Wrap values in quotes: MSG="Hello World"
Multiline value fails No n support Use JSON or base64 for complex values
Test uses production values Wrong .env file loaded Explicitly set NODE_ENV=test

Advanced: Vault Integration (When You Scale)

For teams, use a secret manager instead of .env files:

// Using HashiCorp Vault (simplified)
async function loadFromVault() {
  if (process.env.VAULT_ADDR) {
    const response = await fetch(`${process.env.VAULT_ADDR}/v1/secret/data/app`);
    const { data } = await response.json();

    Object.entries(data.data).forEach(([key, val]) => {
      if (!process.env[key]) {
        process.env[key] = val;
      }
    });
  }
}

// Falls back to .env if no vault configured
await loadFromVault();
require('dotenv').config();

Quick Reference

# CLI operations
export MY_VAR="hello"              # Set for current session
unset MY_VAR                       # Remove
env                                 # Show all vars
printenv | grep NODE                 # Filter vars
source .env                         # Load into shell (not recommended)

# Node.js
process.env.MY_VAR                  # Read
process.env.MY_VAR = 'value'         # Write (affects current process only)
delete process.env.MY_VAR           # Remove

How do you manage environment variables? Still using .env everywhere or moved to something else?

Follow @armorbreak for more practical Node.js guides.

Build MCP Servers that don’t suck…tokens.

First-generation MCP servers were great. They gave AI agents access to a ton of external apps and data — Jira, Confluence, GitHub, Linear, you name it. But most of them just wrapped REST APIs. And that causes a ton of context bloat, hallucinations, and token burning.

Combining a few strategies from the ultra-mcp-toolkit, you can reduce that bloat dramatically — and save money.

Generating a cost-efficient MCP server is easy. Just install the skill and off you go.

Here’s what “dramatically” looks like

Real benchmark, live Jira instance, reproducible:

Per-call response size

scenario naive with toolkit savings
fetch 1 simple ticket 20.3KB 1.2KB 17.5×
investigate rich ticket 270.7KB 15.5KB 17.5×
JQL search ~10 tickets 20.5KB 3.5KB 5.8×

That rich-ticket row is the one that hurts. 270 KB → 15.5 KB. ~67k tokens down to ~3.9k tokens. Same content; the full payload still lands on disk and the agent can fetch it via a ref: path only if it actually needs the detail.

Tool-list cost (paid every conversation)

approach bytes ~tokens savings
naive (one tool per op) 38.9KB 9,947 1×
consolidated tools 25.1KB 6,427 1.5×
consolidated + filtered ~6 KB ~1,600 5×
code-api mode 401B 100 99×

You read that right. Tool listings drop from ~10k tokens to ~100 tokens. On every. single. conversation.

Why MCP servers leak tokens

Four anti-patterns show up almost everywhere:

  1. Returning raw API JSON. A Jira issue carries iconUrls, nested self URLs, schema metadata, expand hints, three different shapes of the same status field. The agent needs none of it.
  2. One MCP tool per endpoint. A typical CRM has ~80 endpoints → 80 tool descriptions in the listing → ~10k tokens before the user types anything.
  3. Asking the LLM to filter or paginate. The model can’t reliably page through huge structures, and the chunking logic itself costs tokens. Filtering belongs server-side.
  4. No discipline on what gets kept. Denylist trimming (delete result.iconUrl) silently breaks the day the API adds a new noisy field. Allowlists keep the contract stable.

The fix, in three strategies

1. Allowlist-style trim projections

import { pick } from "ultra-mcp-toolkit/trim";

const issueSummary = (raw) => {
  const r = raw as { key: string; fields: Record<string, unknown> };
  return {
    key: r.key,
    ...pick(r.fields, ["summary", "status", "priority", "assignee"]),
  };
};

Register the trim once. Every response routes through it. New API fields default to dropped. The model sees what it needs; the full response lives on disk as a ref: the agent can dereference on demand.

2. Consolidated tools (action-discriminated)

Instead of 80 tools, expose ~15 — each taking an action arg:

{ action: "get", issueIdOrKey: "PROJ-1" }
{ action: "create", projectKey: "PROJ", summary: "..." }
{ action: "transition", issueIdOrKey: "PROJ-1", transition: "Done" }

Same operations, 1/5th the tool-list cost. The toolkit’s dispatcher handles per-action Zod validation, manifest routing, and a full: true escape hatch when the model genuinely needs the raw response.

3. Code-api mode (the 99× lever)

Expose a single MCP tool that hands the agent a path to a bundled CLI plus a socket address:

node <cli-path> issue.get --issueIdOrKey=PROJ-1
# stdout: trimmed summary as JSON
# final line: `ref: /path/to/full-response.json`

The agent drives the whole API from its shell. Tool list stays at one tool forever, no matter how many operations exist. For shell-capable agents (Claude Code, Cursor, anything with bash), it’s pure win.

Quick start

npm install ultra-mcp-toolkit

The toolkit ships a Claude Code skill that auto-loads when you work on an MCP server. Install it:

npm run install-skill

That’s it. The skill walks the agent through manifest design, trim projections, dispatcher wiring, and server boot — the patterns that produce the numbers above.

Working from a non-Claude agent (Codex CLI, Cursor, Aider, Continue, Zed)? Point it at the skill markdown directly — AGENTS.md shows you how.

What’s in the box

  • Operation manifest — declare endpoints as pure data; powers MCP tools, CLI, and code-api bridge from one source of truth.
  • Trim registry — type-safe allowlist projections.
  • Content-addressed sandbox — full responses land on disk; the model sees a ref: only.
  • Page cache — versioned-id disk cache for stable keys (PR diffs by SHA, Confluence pages by version).
  • Pooled retry-aware HTTP transport — undici + 429-aware retry honoring Retry-After.
  • Atomic streaming downloads — sha256-verified, path-traversal-safe.
  • Consolidated tool dispatcher — Zod-validated, action-discriminated.
  • CLI scaffolding — bridge mode + direct mode, free with createCli.
  • Bundled Claude Code skill — installs in one command.

Production proof

Used in ultra-jira-mcp and ultra-bitbucket-mcp. The benchmark numbers above come from the Jira server running against a real Jira Cloud instance — every byte measured is one a production agent would actually receive.

If you’re building an MCP server for any enterprise API — Jira, Confluence, GitHub, Linear, Notion, ServiceNow, Salesforce, whatever — and your token bill or context window is starting to bite, give it a try.

⭐ github.com/scottlepp/ultra-mcp-toolkit — issues, PRs, and benchmark contributions welcome.

What’s the most token-bloated MCP server you’ve shipped or seen? Drop it in the comments — I’m collecting horror stories.

EF Core Named query filters

Schema for Employee

Introduction

EF Core 10 introduces named query filters, an improvement to global query filters that makes common patterns like soft deletion and multitenancy easier to manage. In earlier versions, each entity type effectively had one combined filter, which meant disabling filters for a special query was all-or-nothing. With EF Core 10, filters can be given individual names, such as SoftDeletionFilter or TenantFilter, allowing developers to define multiple filters on the same entity and selectively disable only the ones needed for a specific LINQ query. This gives applications more precise control while keeping the default filtering behavior clean, consistent, and centralized. (learn.microsoft.com)

Topics covered

  • Creating filters
  • Ignoring filters
  • Using supplied language extensions for checking filter(s) exists for a model

Source code

  • The provided source code is fully documented.
  • SQL has been provided to create the database.

Source code samples
Source code for language extension

Table schema for code samples

Employee schema, IsDeleted column for soft delete, IsManager indicates if the record is a manager.

Schema for Employee

Soft delete DbContext setup

SaveChanges event needs to be overridden so that a record is not removed, instead State is set to modified and IsDeleted column/property is set to true.

public override async Task<int> SaveChangesAsync(CancellationToken cancellationToken = default)
{
    ChangeTracker.DetectChanges();

    foreach (var entry in ChangeTracker.Entries())
    {
        if (entry.State == EntityState.Deleted)
        {
            // Change state to modified and set delete flag
            entry.State = EntityState.Modified;
            entry.Property("IsDeleted").CurrentValue = true;
        }
    }

    return await base.SaveChangesAsync(cancellationToken);

}

public override int SaveChanges()
{

    ChangeTracker.DetectChanges();

    foreach (var entry in ChangeTracker.Entries())
    {
        if (entry.State == EntityState.Deleted)
        {
            // Change state to modified and set delete flag
            entry.State = EntityState.Modified;
            entry.Property("IsDeleted").CurrentValue = true;
        }
    }

    return base.SaveChanges();

}

Filter setup

For the provided code sample, there is a filter for soft delete and a filter for if a record is a manager.

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity<Employee>(entity =>
    {
        entity.Property(e => e.FirstName)
            .IsRequired()
            .HasMaxLength(50);
        entity.Property(e => e.LastName)
            .IsRequired()
            .HasMaxLength(50);

        entity.HasQueryFilter("SoftDelete", e => !e.IsDeleted);
        entity.HasQueryFilter("IsManager", e => e.IsManager);

    });

    OnModelCreatingPartial(modelBuilder);
}

Performing a soft delete

In this case the overridden SaveChangesAsync is used which changes the State from Deleted to Modified.

AnsiConsole.MarkupLine is from Spectre.Console

private static async Task PerformDelete()
{

    SpectreConsoleHelpers.PrintPink();

    int id = 2;

    await using var context = new Context();
    var employee = await context.Employees.FirstOrDefaultAsync(x => x.Id == id);

    if (employee is not null)
    {
        context.Employees.Remove(employee).State = EntityState.Deleted;
        var affected = await context.SaveChangesAsync();
        AnsiConsole.MarkupLine(affected > 0
            ? $"[green]Successfully deleted employee with ID {id}.[/]"
            : $"[red]Failed to delete employee with ID {id}. Affected rows: {affected}[/]");
    }
    else
    {
        AnsiConsole.MarkupLine($"[yellow]Employee with ID {id} not found.[/]");
    }
}

Ignore a filter

using var context = new Context();
var employees = context.Employees
    .IgnoreQueryFilters(["SoftDelete"])
    .ToList();

Get query filter language extensions

These can be helpful when a developer doesn’t have the DbContext source code.

public static class DbContextExtensions
{

    extension(DbContext context)
    {

        public bool HasQueryFilter<TEntity>()  where TEntity : class
        {
            var entityType = context.Model.FindEntityType(typeof(TEntity));
            return entityType?.GetDeclaredQueryFilters() != null;
        }

        public IReadOnlyCollection<IQueryFilter>? GetQueryFilters<TEntity>() where TEntity : class
        {
            var entityType = context.Model.FindEntityType(typeof(TEntity));
            return entityType?.GetDeclaredQueryFilters();
        }

        public IReadOnlyCollection<IQueryFilter> TryGetQueryFilters<TEntity>() where TEntity : class
        {
            var entityType = context.Model.FindEntityType(typeof(TEntity));
            var filters = entityType?.GetDeclaredQueryFilters();
            return filters ?? [];
        }

    }
}

One sample

output for code sample below

private static void DisplayEmployeeQueryFilters()
{
    using var context = new Context();
    if (context.HasQueryFilter<Employee>())
    {
        var filters = context.GetQueryFilters<Employee>();

        if (filters is null) return;

        foreach (var (index, filter) in filters.Index())
        {
            AnsiConsole.MarkupLine($"{index, -4}" +
                                    $"[cyan]Name[/] {filter.Key} " +
                                    $"[cyan]Expression[/] {filter.Expression}");
        }
    }
    else
    {
        AnsiConsole.MarkupLine("[red]No query filters found for Employee entity.[/]");
    }
}

Summary

The EF Core team has made it easy to set multiple query filters, and by following the instructions and sample code provided, developers can use named query filters.

No Internet? No Problem. How I Built a Crop Disease Detector for Rural India

How I Built an Offline AI Crop Doctor for Indian Farmers Using Gemma 4

Every year, Indian farmers lose 20–30% of their harvest to crop diseases they cannot identify in time. Agricultural experts are concentrated in cities. Internet is unreliable in villages. And most existing tools are in English – a language most farmers don’t read or speak.

I built KhetAI to change that.

What I Built

KhetAI is a fully offline AI crop diagnostic tool for Indian smallholder farmers. A farmer:

  1. Takes a photo of their affected crop
  2. Asks a question in their native language (Hindi, Kannada, Tamil, Telugu, and more)
  3. Gets an instant structured diagnosis with treatment steps, local remedies, and an escalation flag

No internet needed. No cloud. No data ever leaves the device.

Tech Stack:

Layer Technology
AI Model Gemma 4 (gemma4:e4b) via Ollama
Backend FastAPI (Python)
Frontend HTML + Tailwind CSS (single file, no build step)
Inference Ollama – fully on-device

The result the farmer sees includes:

  • Diagnosis with confidence level (High / Medium / Low)
  • Numbered treatment steps
  • Affordable local remedies like neem oil and copper sulfate
  • Prevention tips
  • A red escalation warning if they need to see a real expert

Demo

▶️ Demo Video: Watch on YouTube

Real output from KhetAI – tested with a flower photo asking “What is it’s name?”:

{
  "diagnosis": "Healthy crop - Black-eyed Susan (Rudbeckia family)",
  "confidence": "Medium",
  "treatment": [
    "Ensure soil remains consistently moist but not waterlogged",
    "Feed with balanced slow-release fertilizer at start of growing season"
  ],
  "local_remedies": [
    "Add compost around the base to improve soil nutrients",
    "Light mulch layer of wood chips conserves moisture"
  ],
  "prevention": [
    "Avoid overhead watering - water the base instead",
    "Deadhead spent flowers regularly to encourage new blooms"
  ],
  "escalate": false,
  "summary": "Your crop is healthy - maintain consistent watering and add compost for best yields"
}

Gemma 4 correctly identified the plant, gave practical advice, and suggested affordable local remedies – all in one response, in the farmer’s chosen language.

Code

🔗 GitHub Repository: https://github.com/Tech-Psycho95/KhetAI

Run it yourself:

# 1. Pull the model
ollama pull gemma4:e4b

# 2. Install dependencies
pip install fastapi uvicorn python-multipart ollama

# 3. Start the backend
uvicorn main:app --reload --port 8000

# 4. Open frontend
# Just double-click index.html in your browser

The backend /analyze endpoint accepts a crop photo + question, converts the image to base64, and sends both to Gemma 4 via Ollama:

response = ollama.chat(
    model="gemma4:e4b",
    messages=[
        {
            "role": "user",
            "content": f"[Language: {language}]nFarmer's question: {question}",
            "images": [image_b64],
        }
    ],
    system=SYSTEM_PROMPT,
    options={"temperature": 0.2},
)

The system prompt instructs Gemma 4 to always return a clean structured JSON object with diagnosis, treatment, remedies, prevention, and an escalation flag. Low temperature (0.2) ensures consistent, reliable output every time.

How I Used Gemma 4

I chose Gemma 4 E4B (gemma4:e4b) – here’s why it was the right fit:

Multimodal out of the box
Gemma 4 processes both a crop photo and a text question together in a single API call. No separate vision pipeline, no extra model. The farmer uploads an image and asks their question — Gemma 4 analyzes both simultaneously.

Runs fully offline via Ollama
gemma4:e4b runs locally on a consumer GPU with no API key, no internet, and no subscription. This is non-negotiable for rural India deployment where connectivity is absent.

Multilingual natively
Gemma 4 handles Hindi, Kannada, Tamil, Telugu, Marathi, Bengali, Gujarati, and Punjabi without any translation layer. I simply pass the farmer’s preferred language in the system prompt — the model responds in that language automatically.

Why E4B over E2B or 31B Dense?

  • E2B is too small for reliable multimodal diagnosis accuracy
  • 31B Dense requires expensive hardware beyond a typical village kiosk
  • E4B hits the sweet spot — strong diagnostic accuracy, runs on a mid-range GPU, 9.6GB model size

Before Gemma 4, this project would have required a separate vision model, a separate language model, a translation layer, and a cloud API — meaning internet dependency. Gemma 4 collapses all four into a single local model call.

What’s Next

KhetAI today runs on a single device for a single farmer. The roadmap:

  • Phase 2 – Shared village kiosk over local WiFi, no internet needed
  • Phase 3 – Fine-tune on Indian crop disease datasets using Unsloth + LoRA
  • Phase 4 – Voice input in local languages + government scheme alerts
  • Phase 5 – Federated learning across villages, models improve from anonymised farm data

The infrastructure cost to serve 1,000 farmers? Zero. Because everything runs locally.

Medical AI Doesn’t Just Need Bigger Models. It Needs an ImageNet for State Transitions

Whoever builds the “state–intervention–transition” dataset for biomedicine may define the next generation of medical AI infrastructure.

Author: Jianghui Xiong

Medical AI is moving beyond classification, risk prediction, and question answering.

The next frontier is not just:


sample → label

or:


question → answer

It is:


state + action → next state

In other words:


current biological state + intervention → future biological state

To build real biomedical world models, we need more than bigger models. We need something analogous to ImageNet — not for images, but for biological state transitions.

I will call this idea, for now:


Biomedical TransitionNet

A shared infrastructure for recording, standardizing, and evaluating:


baseline biological state

- intervention
- follow-up biological state
- mechanism evidence
- uncertainty

This article explains why such an infrastructure is needed, why it matters, and why it is scientifically difficult.

It does not claim that a complete biomedical world model already exists.

1. ImageNet was not just a dataset. It was infrastructure.

When people talk about the deep learning revolution in computer vision, they often mention AlexNet, VGG, ResNet, and other neural network architectures.

That is correct, but incomplete.

One of the most important enabling factors was ImageNet.

ImageNet was not merely a large collection of images. Its deeper value was that it gave computer vision a shared coordinate system:

  • a common task,
  • a common label hierarchy,
  • common training and test data,
  • common benchmarks,
  • and a way to compare progress across models and institutions.

Before ImageNet, many computer vision systems were difficult to compare because they were trained and evaluated on different datasets. ImageNet helped the field converge around shared evaluation.

That is why ImageNet became much more than a database. It became research infrastructure.

Medical AI may now need something similar.

But not an image dataset.

Medicine needs an ImageNet for state transitions.

2. Medical AI has many models, but not enough transition data

Today, we already have many types of medical AI systems:

  • medical large language models,
  • medical question-answering systems,
  • radiology models,
  • pathology models,
  • omics foundation models,
  • virtual cell models,
  • digital twin systems,
  • clinical decision support tools,
  • AI drug discovery platforms.

These are important.

But if we think the future of medical AI is only “a bigger medical chatbot”, we may miss the real challenge.

Medicine is not only about answering questions.

Medicine is about understanding and changing biological trajectories.

A clinician does not only ask:


What disease does this patient have?

They also ask:


Why is this biological state happening?

What is driving deterioration?

Which mechanisms are actionable?

Which intervention may shift the trajectory?

How should the response be measured?

What if the expected response does not happen?

What if an adverse response appears?

These are not just language problems.

They are state transition problems.

Most medical AI today is still closer to:


sample → label

or:


question → answer

But biomedical world models require something closer to:


state + action → next state

That is the key shift.

3. What is a biomedical world model?

In AI, a world model is usually understood as an internal model that helps an agent simulate how the environment changes after an action.

A simple abstraction is:


current state + action → future state

In robotics, this may mean:


robot pose + motor command → next scene state

In autonomous driving, it may mean:


traffic scene + driving action → future traffic scene

In biomedicine, the analogous formulation would be:


biological state + intervention → future biological state

This could apply at multiple scales:


cell state + perturbation → cellular response

tissue state + treatment → tissue response

patient state + intervention → follow-up state

A biomedical world model should therefore not be understood as a medical chatbot.

It is not merely:


medical text in → medical text out

A more meaningful biomedical world model would combine:


state representation

- intervention representation
- transition modeling
- mechanism evidence
- uncertainty estimation
- feedback correction

That is much harder than ordinary medical QA.

And it requires a different kind of data.

4. Why medicine needs its own ImageNet

In computer vision, a basic supervised learning unit can often be simplified as:


image + label

For biomedical world models, the basic unit should look more like:


baseline state + action + follow-up state

Or mathematically:


S(t) + A → S(t + Δt)

Where:


S(t)       = biological state before intervention

A          = action or intervention

S(t + Δt)  = biological state after intervention

Δt         = time interval

This is fundamentally different from a static medical database.

A biomedical world model does not only need:

  • medical images,
  • electronic health records,
  • omics profiles,
  • drug-target databases,
  • clinical notes,
  • literature graphs.

Those are useful, but insufficient.

It needs structured longitudinal data describing:


what the biological state was,

what action was taken,

what changed afterward,

over what time scale,

with what evidence,

and with what uncertainty.

This is why medicine needs something like a Biomedical TransitionNet.

Not a direct copy of ImageNet.

A new infrastructure designed for biological state transitions.

5. What should one data unit look like?

A conventional supervised learning sample may look like:


x → y

Examples:


image → diagnosis label

clinical note → ICD code

genomic variant → risk category

A biomedical world-model sample should look more like:


state_before

- intervention
- state_after
- time_interval
- evidence_chain
- uncertainty

A simplified schema might look like this:


{

"baseline_state": {

"molecular": "...",

"clinical": "...",

"phenotype": "...",

"lifestyle": "...",

"context": "..."

},

"action": {

"type": "...",

"dose": "...",

"frequency": "...",

"duration": "...",

"mechanism": "..."

},

"follow_up_state": {

"molecular": "...",

"clinical": "...",

"phenotype": "...",

"adverse_events": "..."

},

"transition": {

"direction": "...",

"magnitude": "...",

"time_scale": "...",

"confidence": "..."

},

"evidence_chain": {

"target": "...",

"pathway": "...",

"biomarker": "...",

"phenotype": "...",

"validation": "..."

}

}

This is obviously simplified.

But the principle matters:

A biomedical world model should learn not only:


what this sample is

but:


how this biological system changed after a defined intervention

6. Five layers of a biomedical ImageNet

If we want to build an ImageNet-like infrastructure for biomedical world models, it should include at least five layers.

6.1 State representation

The first question is:


What is the biological state?

A patient state is not just a diagnosis label.

Terms such as:


diabetes

hypertension

aging

inflammation

fatigue

frailty

are useful, but they are high-level descriptions.

A real biological state may include:

  • genome,
  • DNA methylation,
  • transcriptome,
  • proteome,
  • metabolome,
  • immune state,
  • inflammatory state,
  • organ function,
  • microbiome,
  • sleep,
  • activity,
  • diet,
  • medication history,
  • environmental exposure,
  • clinical background.

A simplified representation may be:


individual_state =

molecular_state

- pathway_state
- organ_state
- phenotype_state
- lifestyle_context
- clinical_context

Without a state representation, a biomedical world model does not know what it is simulating.

6.2 Action ontology

A world model needs actions.

In medicine, actions are complex.

They may include:

  • drugs,
  • supplements,
  • diet,
  • exercise,
  • sleep intervention,
  • stress management,
  • cell therapy,
  • gene therapy,
  • regenerative medicine,
  • combination therapy,
  • N-of-1 personalized intervention.

Even a drug intervention requires many parameters:


drug name

dose

frequency

route

duration

combination

adherence

contraindications

adverse events

Exercise intervention also requires:


type

intensity

frequency

duration

heart-rate zone

recovery condition

baseline fitness

If actions are not standardized, the model cannot learn meaningful transitions.

6.3 Transition record

The core of a biomedical world model is the transition:


before → after

Examples:


inflammatory state before intervention → inflammatory state after intervention

DNA methylation age before intervention → DNA methylation age after intervention

metabolic state before intervention → metabolic state after intervention

tumor state before treatment → tumor state after treatment

Without follow-up measurement, there is no transition.

Without transition, there is no world model.

Many medical datasets are still one-time measurements:


one-time measurement

Biomedical world models need:


longitudinal measurement

6.4 Evidence chain

A medical model should not only output a probability.

If a model says:


This intervention may help.

That is not enough.

It should also answer:


Which targets are involved?

Which pathways are affected?

Which abnormal state does this address?

Which biomarkers can validate the response?

Which evidence comes from experiments?

Which evidence comes from clinical data?

Which part is only model inference?

Which risks should be monitored?

In medicine, prediction alone is not sufficient.

A safer output should look more like:


prediction + mechanism + validation + uncertainty

This is especially important because medical AI should not become an uninspectable black box.

6.5 Benchmark task

ImageNet helped computer vision because different models could be compared on shared tasks.

Biomedical world models need benchmarks too.

Possible benchmark tasks include:

  • cellular perturbation response prediction,
  • gene expression response after drug perturbation,
  • tumor state simulation after treatment,
  • metabolic biomarker response prediction,
  • inflammatory state transition prediction,
  • aging-related biomarker transition prediction,
  • N-of-1 intervention response direction prediction.

But the metrics cannot be copied directly from image classification.

Useful metrics may include:


directional accuracy

mechanistic consistency

biomarker validation

uncertainty calibration

risk awareness

cross-context generalization

This is much harder than top-1 accuracy.

But medicine requires it.

7. Related progress: promising, but still early

To be scientifically careful, we should not pretend that complete biomedical world models already exist.

They do not.

But several related directions are emerging.

7.1 ImageNet as an infrastructure analogy

ImageNet and ILSVRC showed how large-scale, standardized datasets and benchmarks can accelerate a field.

However, ImageNet is a benchmark for image classification and detection.

It is not equivalent to what biomedicine needs.

Here, ImageNet is used only as an infrastructure analogy.

The biomedical version must be longitudinal, dynamic, intervention-aware, and mechanism-sensitive.

7.2 World Models in AI

Ha and Schmidhuber’s World Models is a representative work in AI world modeling.

Its key idea is that an agent can learn an internal model of the environment and use it to simulate future states.

Medicine cannot directly copy this setting.

A human body is not a game environment.

Clinical intervention cannot be freely explored by trial and error.

But the abstraction:


state + action → future state

is still useful for thinking about medical AI.

7.3 Virtual cells and perturbation response

Arc Institute’s State model is a recent example of virtual-cell modeling.

It aims to predict how cells respond to drugs, cytokines, or genetic perturbations. Public descriptions indicate that State was trained on large-scale observational and perturbational single-cell data.

This is important because it directly touches the pattern:


cell state + perturbation → cellular response

However, State is primarily a cellular-level model.

It should not be confused with a complete patient-level biomedical world model.

7.4 Medical World Model for tumor evolution

Recent work using the term Medical World Model, such as MeWM, explores generative simulation of tumor evolution under treatment conditions.

This is relevant because it moves medical AI from static recognition toward treatment-conditioned disease dynamics.

But this direction is still early.

It should not be interpreted as a general solution to biomedical world modeling.

7.5 Digital twins and virtual physiological systems

Long before today’s AI world-model terminology, fields such as computational physiology, systems biology, virtual physiological systems, and digital twins already tried to connect biological structure, mechanism, dynamics, and measurable outputs.

That tradition matters.

A good biomedical world model should not be just a black-box predictor.

It should connect:


state

mechanism

dynamic change

measurement

feedback

Today’s biomedical world models can be seen as an extension of this older systems-modeling tradition into the era of AI, multi-omics, real-world data, and large-scale computation.

8. Why steerability matters

A biomedical world model that only predicts is not enough.

A model may predict that a patient’s risk is increasing.

But medicine needs more than that.

It needs to ask:


Which state can be measured?

Which abnormality can be explained?

Which intervention can be described?

Which transition can be tested?

Which deviation can be traced?

Which failure can be corrected?

This is why I emphasize steerability.

Going forward, I will use the name:


SteeraMed: A Steerable Biomedical World Model

Website:


https://SteeraMed.com

The earlier preprint name was:


SEWO / Steerable Medicine World Model

or in Chinese:


可驾驭医学世界模型

Whenever I mention SEWO / 可驾驭医学世界模型, it should be understood together with the new unified naming:


SteeraMed: A Steerable Biomedical World Model

The idea behind SEWO / SteeraMed is that biomedical world models should not only pursue predictive accuracy. They should also support:

  • state definition,
  • intervention description,
  • transition hypothesis,
  • mechanism audit,
  • deviation tracing,
  • uncertainty inspection,
  • expert steering,
  • and iterative correction.

The related ideas were introduced in the preprint:


World Models for Biomedicine: A Steerability Framework

and are also presented at:


https://steerable.world

Important clarification:

SEWO / SteeraMed is not a clinically validated treatment system.

It is not a medical device.

It is better understood as a structural framework and evidence-chain design principle for future biomedical world models.

The key question is not only:


Can the model predict?

but:


Can researchers and clinicians inspect, question, correct, and steer the model within clearly defined boundaries?

9. Why longevity medicine may be one entry point

Biomedical world models could start from many areas:

  • oncology,
  • cardiovascular disease,
  • metabolic disease,
  • immunology,
  • neurodegeneration,
  • drug discovery,
  • virtual cells,
  • longevity medicine.

Longevity medicine is not the only entry point.

But it is an interesting one.

Why?

9.1 Aging is a continuous state

Aging is not a single disease label.

It is a continuous, multi-system biological process involving:

  • inflammation,
  • metabolism,
  • immunity,
  • epigenetics,
  • mitochondrial function,
  • proteostasis,
  • stem-cell exhaustion,
  • cellular senescence,
  • organ function decline.

That makes it naturally suitable for state modeling.

9.2 Longevity medicine requires repeated measurement

Longevity medicine is not a one-time diagnostic event.

It depends on repeated measurement over time.

A useful intervention must be evaluated through:


baseline state → intervention → follow-up state

This is exactly the structure needed for biomedical world modeling.

9.3 Interventions are diverse

Longevity-related interventions may include:

  • diet,
  • exercise,
  • sleep,
  • supplements,
  • drugs,
  • cell therapy,
  • regenerative medicine,
  • stress management,
  • environmental exposure management.

This provides a rich action space.

9.4 Individual responses vary

The same intervention may produce different responses in different people.

That means longevity medicine cannot rely only on average effects.

It needs N-of-1 style transition data:


individual state → intervention → individual transition

Each well-structured N-of-1 intervention can be seen as a small world-model experiment.

10. Engineering implications

From an engineering perspective, the biomedical ImageNet is not just a dataset.

It is a data infrastructure problem.

It requires:

  • data collection,
  • data standardization,
  • multimodal integration,
  • time-series modeling,
  • intervention encoding,
  • causal confounding control,
  • privacy protection,
  • benchmark design,
  • safety boundaries,
  • evidence-chain tracking.

A simplified loop may look like:


measure state

↓

standardize state representation

↓

record intervention

↓

measure follow-up state

↓

construct transition sample

↓

train / evaluate world model

↓

generate testable hypothesis

↓

repeat and correct

This is not a static dataset.

It is a data flywheel.

11. Main challenges

This is scientifically and technically difficult.

Some of the main challenges include:

11.1 Biological state is complex

A human state cannot be compressed into one label.

We need ways to represent multi-omics, clinical metrics, imaging, lifestyle, symptoms, environmental exposure, and medical history as computable state variables.

11.2 Interventions are hard to standardize

Drugs, exercise, diet, sleep, supplements, and cell therapies all have complex parameters.

Without action standardization, transition learning will be noisy.

11.3 Follow-up data is scarce

Most medical data is not collected as structured pre/post intervention transition data.

This requires new data collection workflows.

11.4 Causal confounding is serious

In the real world, people often change many things at once:


diet

exercise

sleep

medication

supplements

stress

Attributing a state change to one factor is difficult.

This requires careful study design and statistical methods.

11.5 Safety and ethics are central

A biomedical world model cannot freely experiment like a game-playing agent.

Any intervention-related model must clearly distinguish:


research hypothesis

health-management suggestion

clinical decision support

medical recommendation

validated therapy

Clinical use would require prospective validation, safety evaluation, ethical review, regulatory review where applicable, and professional oversight.

11.6 Open standards and business incentives may conflict

If everything is closed, the field cannot build shared benchmarks.

If everything is open, companies may lack incentives to invest.

A practical ecosystem will need a balance among:


open benchmarks

privacy protection

commercial incentives

scientific collaboration

12. A minimal viable direction

A biomedical ImageNet should not begin by trying to simulate the entire human body.

A more realistic path is to start with minimal viable tasks.

Examples:

  • cellular perturbation response prediction,
  • tumor state change after treatment,
  • metabolic biomarker response prediction,
  • inflammatory state transition prediction,
  • DNA methylation age transition,
  • N-of-1 longevity intervention tracking.

A minimal task should define:


1. state variables
2. intervention variables
3. follow-up time
4. transition metrics
5. benchmark task
6. safety boundary

Start narrow.

Make it measurable.

Make it repeatable.

Make it auditable.

Then scale.

13. Whoever defines state, action, and transition may define the field

Medical AI will still need better models.

But bigger models alone cannot solve the problem of biomedical state transition learning.

The scarce asset is the infrastructure that allows models to learn:


how life systems change after intervention

Future platform-level medical AI companies may not be the ones with the largest language models.

They may be the ones that can build the strongest data flywheel:


measure biological state

standardize interventions

record follow-up changes

construct mechanism evidence chains

evaluate transition models

repeat

Whoever defines state defines what medical AI can see.

Whoever defines action defines how medical AI understands intervention.

Whoever defines transition defines how medical AI learns biological change.

Whoever defines the benchmark defines how the field measures progress.

Conclusion

ImageNet helped machines learn to see the world.

A biomedical ImageNet should help AI learn how life responds to intervention.

That does not mean replacing clinicians.

It means building a scientific infrastructure where models can learn:


how states form

how interventions act

how systems transition

how evidence is validated

The next decade of medical AI may not be limited by model size alone.

It may be limited by the lack of a shared infrastructure for biological state transitions.

That is the real opportunity.

References

  1. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L.

    ImageNet: A Large-Scale Hierarchical Image Database. CVPR. 2009.

    https://ieeexplore.ieee.org/document/5206848

  2. Russakovsky O, Deng J, Su H, et al.

    ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision. 2015.

    https://arxiv.org/abs/1409.0575

  3. ImageNet official website.

    https://www.image-net.org/

  4. Ha D, Schmidhuber J.

    World Models. 2018.

    https://worldmodels.github.io/

  5. Arc Institute.

    Arc Institute’s first virtual cell model: State.

    https://arcinstitute.org/news/virtual-cell-model-state

  6. Predicting cellular responses to perturbation across diverse contexts with State. bioRxiv. 2025.

    https://www.biorxiv.org/content/10.1101/2025.06.26.661135v1

  7. Yang Y, Wang ZY, Liu Q, et al.

    Medical World Model: Generative Simulation of Tumor Evolution for Treatment Planning. arXiv.

    https://arxiv.org/abs/2506.02327

  8. IEEE Transactions on Biomedical Engineering.

    Digital Twins / AI World Models.

    https://www.embs.org/tbme/research-highlights/digital-twins-ai-world-models/

  9. Acosta JN, Falcone GJ, Rajpurkar P, Topol EJ.

    Multimodal biomedical AI. Nature Medicine. 2022.

    https://www.nature.com/articles/s41591-022-01981-2

  10. Xiong J.

    World Models for Biomedicine: A Steerability Framework. Preprints.org. 2026.

    https://www.preprints.org/manuscript/202605.0366

    DOI: https://doi.org/10.20944/preprints202605.0366.v1

  11. SteeraMed: A Steerable Biomedical World Model.

    https://steerable.world

Disclaimer

This article is for research, technical, and industry discussion only.

It is not medical advice, diagnostic advice, or treatment advice.

Any biomedical world model intended for clinical use would require prospective validation, safety evaluation, ethical review, regulatory review where applicable, and professional clinical oversight.


Building a Production-Grade Observability Platform for the Anvila API with LGTM, SLOs, DORA Metrics, and Game Day Testing

Introduction

For the HNG DevOps Stage 6 task, our team built a production-grade observability and reliability platform for the Anvila API.

The goal was not just to check whether a server was up or down. We needed to build a monitoring system that could help a team understand the health of an application from different angles:

  • Is the application available?
  • Is it fast enough for users?
  • Are requests failing?
  • Is the server under pressure?
  • Are deployments healthy?
  • Can engineers investigate logs, metrics, and traces from one place?
  • Can alerts reach the team with enough context to act quickly?

To achieve this, we used the LGTM observability stack:

  • Loki for logs
  • Grafana for dashboards
  • Tempo for traces
  • Prometheus for metrics

We also added Alertmanager for alert routing, Node Exporter for server metrics, Blackbox Exporter for uptime probing, OpenTelemetry Collector for logs and traces, and a custom GitHub Actions DORA exporter.

[Prometheus targets all up]

Architecture Overview

Our architecture uses a separate monitoring server. The Anvila API continues to run on its application server, while the monitoring stack runs on a dedicated AWS EC2 instance.

The monitoring server collects:

  • system metrics from Node Exporter
  • uptime and response time metrics from Blackbox Exporter
  • application metrics from the FastAPI /metrics endpoint
  • logs from the app server through OpenTelemetry Collector and Loki
  • traces from the instrumented FastAPI staging service through OpenTelemetry and Tempo
  • CI/CD metrics from GitHub Actions through a custom DORA exporter

Grafana connects all these data sources together so that we can view metrics, logs, and traces in one place.

[LGTM dashboards list]

At a high level, the data flow is:

Anvila API -> OpenTelemetry Collector -> Loki / Tempo
Anvila API /metrics -> Prometheus
Node Exporter -> Prometheus
Blackbox Exporter -> Prometheus
GitHub Actions exporter -> Prometheus
Prometheus / Loki / Tempo -> Grafana
Prometheus alerts -> Alertmanager -> Slack

Why We Chose LGTM Over Managed Alternatives

Managed observability platforms are useful because they reduce operational work. However, for this task, we chose LGTM because we needed to understand and control the full observability pipeline.

LGTM gave us:

  • full control over metrics, logs, and traces
  • version-controlled configuration
  • no dependency on a paid managed observability provider
  • a better learning experience for how observability systems actually work
  • the ability to provision dashboards, alert rules, and data sources as code

Prometheus is strong for metrics and alerting. Loki is useful for storing logs in a way that works well with Grafana. Tempo stores distributed traces and helps us understand request flow. Grafana brings all three together in one interface.

This made LGTM a good fit for a platform engineering task where the goal was to build the stack from the ground up.

Infrastructure as Code and Systemd Deployment

One important requirement was that the stack should be reproducible. We used Terraform to provision the monitoring EC2 server and upload the configuration files.

The services are installed and managed with systemd instead of Docker. This was important because the task update said we should not install Docker on the server.

The stack includes systemd services for:

  • Prometheus
  • Grafana
  • Loki
  • Tempo
  • Alertmanager
  • Node Exporter
  • Blackbox Exporter
  • OpenTelemetry Collector
  • DORA exporter

The repository contains:

  • Terraform files
  • Prometheus configuration
  • Alertmanager configuration
  • Grafana dashboard JSON files
  • Loki and Tempo configuration
  • OpenTelemetry Collector configuration
  • alert rules
  • runbooks
  • SLI/SLO documentation
  • Game Day documentation

[Alert rules YAML]

Metrics Collection

Metrics are numerical measurements that tell us what is happening in the system.

We collect several types of metrics:

Server Metrics

Node Exporter gives us system-level metrics such as:

  • CPU usage
  • memory usage
  • disk usage
  • disk I/O
  • network I/O
  • system load

These metrics help us understand saturation, which means how full or stressed the system is.

[Node Exporter dashboard]

Uptime and HTTP Probe Metrics

Blackbox Exporter probes both the staging and production API URLs.

It checks:

  • whether the endpoint is reachable
  • whether it returns a successful HTTP response
  • how long the response takes
  • SSL certificate expiry

This is important because it measures the service from the outside, closer to how a user or client would experience it.

[Blackbox dashboard]

Application Metrics

We also added app-level Prometheus metrics to the FastAPI staging application through a /metrics endpoint.

This gives Prometheus real application request metrics such as:

  • http_requests_total
  • http_request_duration_seconds
  • http_requests_in_progress

These metrics help us track request volume, request latency, and request status codes from inside the application itself.

[App metrics target up]

[App request metrics]

[App latency histogram]

Logs and Traces

Metrics tell us that something is wrong. Logs and traces help us understand why.

We used OpenTelemetry Collector to collect application logs and send them to Loki. In Grafana, we can search these logs using Loki queries.

[Loki logs]

We also instrumented the staging FastAPI service with OpenTelemetry so that it sends traces to Tempo.

A trace shows the path of a request through the application. This is very useful when investigating slow requests, because it helps identify which endpoint or operation caused the delay.

[Tempo traces]

The Unified Observability dashboard combines metrics, logs, and traces. The idea is that if latency or error rate increases, an engineer can look at the metric, check related logs in Loki, and then inspect traces in Tempo.

[Unified observability dashboard]

The Four Golden Signals

The Four Golden Signals are a simple way to define service reliability from a user-focused perspective.

They are:

  1. Latency
  2. Traffic
  3. Errors
  4. Saturation

Latency

Latency means how long it takes the service to respond.

For Anvila, we track latency using both Blackbox response time and application request duration metrics.

Example PromQL:

histogram_quantile(
  0.95,
  sum by (le, path) (
    rate(http_request_duration_seconds_bucket{job="anvila-api-staging",status!~"5.."}[5m])
  )
)

This tells us the 95th percentile request latency for successful requests.

Traffic

Traffic means how much demand the service is receiving.

Example PromQL:

sum(rate(http_requests_total{job="anvila-api-staging"}[5m]))

This tells us the number of requests per second.

Errors

Errors measure how many requests are failing.

Example PromQL:

sum(rate(http_requests_total{job="anvila-api-staging",status=~"5.."}[5m]))
/
clamp_min(sum(rate(http_requests_total{job="anvila-api-staging"}[5m])), 1)

This gives us the ratio of failed requests.

Saturation

Saturation shows how full the system is.

For example, CPU usage:

100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

This goes beyond simply checking if the server is alive. A server can be up but overloaded. The Four Golden Signals help us understand the actual quality of the service.

SLOs and Error Budgets

An SLI is a Service Level Indicator. It is a measurement.

An SLO is a Service Level Objective. It is the target we want to meet.

An error budget is the amount of unreliability we are allowed within a time window.

For Anvila, our main availability SLO is:

99.5% successful probes over 30 days

The error budget calculation is:

(1 - 0.995) * 30 days = 0.15 days = 3.6 hours

This means the service can be unavailable for about 3 hours and 36 minutes in a 30-day window before the error budget is fully consumed.

The SLO dashboard shows:

  • 30-day availability SLI
  • SLO target
  • error budget remaining
  • error budget time remaining
  • burn rate
  • 7-day and 30-day compliance history

[SLO and Error Budget dashboard]

Burn Rate Alerting

Burn rate tells us how quickly the service is consuming its error budget.

This is better than simple alerting because it reduces alert fatigue.

For example, a short small failure may not need to wake everyone up. But if the service is burning through its error budget very quickly, the team should respond immediately.

We configured:

  • Fast Burn alert: critical
  • Slow Burn alert: warning

The Fast Burn alert means the system is consuming the error budget too quickly and needs immediate action.

The Slow Burn alert means the system may become a serious incident if the trend continues.

DORA Metrics

DORA metrics help connect engineering work to business outcomes.

The four DORA metrics are:

  • Deployment Frequency
  • Lead Time for Changes
  • Change Failure Rate
  • Mean Time to Restore

These metrics help answer questions such as:

  • How often do we deploy?
  • How long does it take for code to reach production?
  • How often do deployments fail?
  • How quickly can we recover from incidents?

We built a GitHub Actions DORA exporter that exposes deployment workflow data to Prometheus.

The dashboard shows:

  • deployment frequency
  • lead time
  • change failure rate
  • MTTR
  • deployment frequency classification

[DORA dashboard]

Our current DORA implementation measures commit-to-workflow-completion lead time. A future improvement would be to break it into more detailed sub-intervals such as commit time, workflow trigger time, workflow completion time, and deployment confirmation time.

Alerting and Slack Notifications

Alert rules are stored in version-controlled YAML files, not manually created in the Grafana UI.

We configured alerts for:

  • host downtime
  • high CPU
  • high memory
  • disk usage
  • SLO burn rate
  • change failure rate
  • MTTR threshold

Alertmanager handles routing and sends alerts to Slack.

The Alertmanager configuration includes:

  • route grouping by alert name, service, and severity
  • warning and critical routes
  • inhibition rules
  • Slack receiver configuration

[Alertmanager config]

We also used a structured Slack template. The alert payload includes:

  • alert name
  • severity
  • affected host or target
  • current value
  • dashboard link
  • runbook link
  • firing or resolved status

[Slack template]

Here is an example of a firing and resolved Slack notification:

[Slack firing and resolved alert]

Runbooks and Incident Management

Every alert needs a runbook. A runbook explains what the alert means and what to do first.

Our runbooks include:

  • what the alert means
  • likely causes
  • first investigation steps
  • how to resolve it
  • when to roll back
  • when to escalate

Example runbook:

[SLO burn rate runbook]

We also documented a blameless post-incident review for the simulated latency incident.

The goal of a blameless PIR is not to blame a person. The goal is to understand what happened, what the impact was, what went well, what did not go well, and what should be improved.

Game Day Testing

Game Day is where we intentionally create controlled failures to test whether our monitoring and response system works.

We ran three scenarios.

Scenario 1: Deployment Failure

We created a temporary PR with an intentionally failing GitHub Actions workflow.

The goal was to prove that a deployment or pipeline failure can be detected without touching production.

The PR was closed without merging after screenshots were captured.

[Game Day deployment failure PR]

[Game Day deployment failure check]

Scenario 2: Latency Injection

During the latency Game Day, we temporarily added a 2-second delay to the staging root endpoint and generated test requests. The evidence below shows the resulting Tempo traces and the recovery after the delay was removed.

[Game Day latency traces]

[Game Day latency recovery]

Scenario 3: Resource Pressure

We created CPU pressure on the monitoring server using controlled background processes.

This triggered a CPU warning alert in Slack. After stopping the pressure, we confirmed the resolved alert was also sent.

[Game Day resource pressure trigger]

[Game Day resource pressure Slack alert](

[Game Day resource pressure recovery]

Toil Identified and Automation

Toil is repetitive manual work that can be automated.

We identified several examples of toil:

1. Manual Dashboard Creation

Creating dashboards manually in Grafana is slow and hard to reproduce.

Automation: All dashboards are stored as JSON and provisioned automatically.

2. Manual Monitoring Server Setup

Installing each service manually would be time-consuming and error prone.

Automation: Terraform creates the monitoring server, and systemd installation scripts configure the observability stack.

3. Vague Alert Investigation

If an alert only says, “CPU high”, the engineer still has to search for the server, dashboard, and runbook.

Automation: Alertmanager Slack messages include service, host, metric value, dashboard link, and runbook link.

4. Manual CI/CD Metrics Collection

Checking GitHub Actions manually does not scale.

Automation: The DORA exporter pulls GitHub Actions data and exposes it as Prometheus metrics.

What We Learned

This project helped us understand that observability is more than installing tools.

The tools are only useful when they are connected to reliability goals.

Prometheus helped us collect and alert on metrics. Loki helped us inspect logs. Tempo helped us understand request traces. Grafana helped us bring everything together. Alertmanager helped us route actionable alerts to Slack.

We also learned that good reliability engineering requires:

  • clear SLIs
  • realistic SLOs
  • error budgets
  • runbooks
  • incident reviews
  • controlled failure testing

Current Limitations and Next Steps

The platform is functional, but there are still improvements we can make.

Current limitations:

  • App-level /metrics is currently verified on staging.
  • Production is monitored through Blackbox probes and shared host-level metrics.
  • DORA lead time is currently simplified to commit-to-workflow-completion.
  • A follow-up PR was opened to normalize unmatched route labels in Prometheus metrics and reduce high-cardinality risk.

Next steps:

  • promote app-level metrics to production after staging validation
  • merge the metrics label normalization improvement
  • add more detailed DORA lead-time sub-intervals
  • add deployment annotations to Grafana dashboards
  • continue reviewing SLOs as the service matures

Conclusion

For this task, we built a complete observability platform around the Anvila API using the LGTM stack.

We deployed the monitoring stack with Terraform and systemd, collected metrics, logs, and traces, built dashboards, configured Slack alerts, wrote runbooks, and tested the system through Game Day simulations.

The biggest lesson is that observability is not just about knowing when something is broken. It is about giving the team enough context to understand the problem, respond quickly, and improve the system after every incident.

Repository:

https://github.com/Adeolu1024/anvila-observability-platform

Evidence screenshots:

https://drive.google.com/drive/folders/1v_DiT6XQtq0iJtn5JqxfZecAmxhCdd82?usp=sharing```


History repeats itself: Aerospace & AI-software-development

The Analogy:-

  1. When aircrafts were first being designed, they had to deal with the unpredictability that arose from working with fluids (that too, when fluid dynamics was not even a proper subject)
  2. 1920s-1930s: “Works most of the time“
  3. When it did fail, invariably it was unexpected and it happened at the worst possible situation; we then end up in a mini disaster, if not worse!
  4. Eventually, aircraft ENGINEERING, as a discipline, figured out how to systematically handle the chaotic nature of Air, ultimately delivering “deterministic-flights” even in the bad weather!!
  5. All Aerospace-engineers turned into Systems-Integrators and/or into Simulation/Modeling people.

What does it mean for those of us developers who’re struggling to survive this AI-armageddon?

The Challenges:

  1. STANDARD CONTROLS & METRICS: Engineering may likely reassert itself into AI-only software-development, once we have globally well-understood and well-standardized metrics + controls, to track progress and to track quality of the work done by AI. if everyone’s got their own standards, it’s not going to be enough.
  2. A BIT CLUELESS: If you listen to the famous names and what they’re publishing on the various media, as of April 2026, we still have insufficient control on what the models do. I believe, we must seek 100% control. (Dare I say this? Just frequently enough, we sometimes have no clue re: “how/what” the AI did! Forget about “control“).
  3. GOING NOWHERE FAST: As someone who started a career with 3-yrs-long projects, I now ask all end-users: “I CAN COOK SO FAST NOW. BUT CAN YOU EAST FAST ENOUGH?“. Let me answer on behalf of the end-users: “No!“

Survival Tips (part 1 of 2):-

  1. STOP: Stop wasting time hoarding/curating prompts/templates. My own recent experience was that GitHub’s CoPilot’s CodeReview was FAR-superior to my own curated set of “PR Review prompts” (run across multiple agents); FYI: I had copied these from famous people’s publications.
  2. NEW FOCUS: Your career now is NOT to decide which model/framework works best.
    Focus instead on how you can capture metrics & trends, the change-impact measurements, the cost-estimates, etc .. especially for already-deployed apps.. .. to help others make those decisions.
  3. Your career will ALSO be about GUARANTEEING / proving that the ROLLBACK in production, was 100% precise. What metrics are needed to provide such a guarantee, and how will you have them in place prior to starting the rollback?
  4. With AI, prepare for rollbacks to be routine. My gut says: This is the 1st true engineering within the AI-world. SRE may need to adapt.
  5. BEST TOOLS: Like a traditional engineer, your tools should be the best available: (1) best PR-review tool built & maintained by some corporation (2) best design-review built & maintained by some corporation (3) etc.. ..
  6. Open Source simply can NEVER keep up with this mind-blowing pace of progress. So, focus on commercial-tools (hopefully those that’ll “Not vendor-lock you forever”).
  7. MINDSET CHANGE: Start chanting “Can Not control what you do Not measure“. Drink that “kool-aid“. Believe in it.
  8. You must truly believe that it is NOT engineering to just break-apart a prompt into “Role, Context, Objectives, Constraints & Communication-style“.

Possibilities in the Future:-

  1. SIMULATION/MODELING: Following the trajectory of the Aeroplane-industry, its quite likely that Engineering will be more “Simulation & Modeling” (of the problems and of the future-solutions), and less about actual software-development (see more below). True Visionaries are so rare! That’s wonderful for our job-security! As, we get to have jobs to do modeling of endless scenarios for the clueless C-Suite.
  2. PRODUCT-MANAGEMENT:- With AI “listening in” into all calls/meetings and with access to all information, it’s certainly quite possible this role is no longer human! ☹️. Unless, .. .. like Steve Jobs and Henry Ford (as well-known examples), you know what the customers WILL want.
  3. REQUIREMENTS-ELICITATION:- This is the MAKE-or-BREAK MOST-IMPORTANT aspect of software-development, that .. .. if AI somehow nails it, .. .. it will destroy all prospects for humans. As one possibility, the day that the employees have a CONSTANTLY ATTACHED AI, that knows everything about that person’s role and a bit about their personal lives, .. that will be the day there will be no more IT-Dept. Google MAY soon achieve this in the context of the common-consumer. 99% of the time, end-users do NOT know what EXACTLY do they want. Hence my note above re: “never be able to eat fast enough“. It takes a lot of time & effort by us humans, to elicit updated-requirements from a end-user.

Survival Tips (Part 2 of 2):-

Enroll in a local college re: “Simulation and Modeling“. Become familiar with such terminology & tools

Sorry. I am Not able to predict whether “feature git-branches” with their own deployed tiers/environments (incl. freshly hydrated new databases/data-repos) counts as “Simulation and Modeling”. Feel free to debate about the future.

Common Misconceptions about the Wright-brothers:-

  1. For those who live out their liberal-arts degrees, and for those who don’t give a sh*t, and for the uneducated, the Wright Brothers invented the plane! That’s it.
  2. There were many many people who actually “lifted off” quite a bit before the famous success of the Wright Brothers. Their problem? No one could replicate their success ever again.
  3. From an engineering standpoint, the Wright Brothers spent years (perhaps close to a decade?), looking at it from a true engineering perspective. CONTROL. CONTROL. CONTROL. CONSISTENCY.
  4. What was the reason the Wright brothers were soooooo obsessed with control & consistency? The one brother who was designing it, was sooooo concerned that his brother would get hurt — in a TRADITIONAL sense of being brothers. It’s a relationship that’s hard to explain/understand in this world today. Perhaps it was due to the social / parental pressure back then. BTW, I’m old.

My view of the world today:-

  1. We just FINALLY got past the “It’s So Exciting! Another new hot-hot research paper was published. Can’t wait to read it and discuss/debate it endlessly.”
  2. We’re now in the phase:- “Throw it against the wall, see if it sticks” (Chaotic Experimentation)
  3. Software-Authoring -versus- Software-development: Claude is PUBLICLY leading in the latter. I’m quite sure Google is doing the same internally too.

But, But .. you say ..

  1. “AI is working so great !!”
    • You must be in sales/marketing/general-media/porn.
    • Words like “Quality” and “Guarantee/Warranty” never applied to your work.
    • And never will.
    • Appearances & Impressions do.
    • Good for you.
    • Enjoy!
  2. “We’re in the hockey stick part of the human evolution. It’s different this time.”
    • Decades may shrink down to years or lesser. But, human-history does rhyme, even if it does not repeat.
  3. “Nowadays. MVP and TTM have made the “brotherly relationships/objectives” irrelevant. Production is where testing is done / bugs are identified. So, we’re unlikely to see consistency, and even less likely determinism.”
    • Unfortunately, I need to admit that this is a realistic possibility.

If you are a PRACTICING software-ENGINEER (emphasizing the words “practicing” & “engineer”), then you barely skimmed through this article. Even so, you understood everything here and .. perhaps have better insights!!
For all such folks, “L.F.G.” !!

If you’re more of a thinker/intellectual-elite, then you’re more likely to debate this endlessly, perhaps even on some Dwarkesh podcast. Leave me alone.

I Built a Free AI Chatbot Template So You Don’t Have to Waste a Day


The setup problems are already solved. You just have to build.
I spent almost a full day building my first AI chatbot.
Most of that time wasn’t spent on AI. It was spent on one stupid error.

CORS.

If you’ve never hit a CORS error before, consider yourself lucky. It’s the kind of thing that makes you question whether you’re cut out for this. Your frontend can’t talk to your backend. Nothing works. The error message tells you nothing useful. You’re Googling the same Stack Overflow thread for the fourth time.

The actual AI part took a couple of hours. The setup ate the rest of my day.

So I built something to fix that.

What is AI Chat Starter?
A free template that gets you from zero to a working AI chatbot in 30 minutes. No complex frameworks. No boilerplate hell. No day lost to setup issues you didn’t see coming.

  1. Download the template
  2. Add your Claude API key
  3. Run two commands
  4. You have a working chatbot
    That’s it.

Why I built this
Every beginner hits the same walls when they try to build their first AI project:

→”Which framework should I use?”
→”How do I structure my API calls?”
→”Why won’t this deploy?”
→”Do I need a database for this?”
These questions have nothing to do with AI. But they stop people from ever shipping their first project. AI Chat Starter removes all of that friction so you can skip straight to the part that actually matters — building.

What you get
Backend —FastAPI server, Claude API integration, streaming responses, CORS pre-configured, environment variable setup
Frontend —Clean responsive chat interface, real-time streaming, mobile-friendly out of the box
Docs —Step-by-step setup guide, deployment instructions, troubleshooting for the most common issues
Who this is for
→You want to explore AI but don’t know where to start
→You’ve tried tutorials that assumed too much prior knowledge
→You need a working foundation you can actually customize
→You learn by building, not watching 10-hour courses
The problems I fixed so you don’t have to
CORS Errors —Pre-configured for local and production environments
Streaming Responses —Proper SSE handling that works across all browsers
API Key Management —.env setup with clear, step-by-step instructions
Error Handling —Try-catch blocks and user-friendly error messages built in
Deployment Confusion —Working configs and guides for 3 different platforms

What comes next
AI Chat Starter is just the beginning. Here’s where most ShipStack builders go next:

AI Chat Starter — start here
Free
Personality Pack — give it a real job instantly
$4.99
AI Memory Lite — make it remember conversations
$9.99
AI Memory Agent — full cloud-based memory
$24.99
AI Tool Agent — full business automation
$29.99
Start simple. Scale when you’re ready.

Why it’s free
I wasted a day on problems this template solves in 30 minutes. If this helps one person skip that pain, it’s worth it.
And honestly — if you try the free template and it works, you’ll trust the paid ones.

👉 Get AI Chat Starter (FREE)
[ https://getshipstack.gumroad.com/l/gijgjb]

Google Cloud & Claude Code workshop – 5/29 in SF, CA.

Hey developers!

Want to learn best practices for integrating Claude into your Google Cloud environment and deploying Claude Code?

Well, look no further! Anthropic and Google Cloud are teaming up for a hands-on, in-person workshop in SF on Friday, May 29, 2026 from 9:00 AM — 12:30 PM PT.

In this half-day event, you’ll:
🛠️ Set up Claude and Claude Code on Google Cloud
💻 Build working prototypes through guided labs.
💡 Get your questions answered in real-time by the experts.

Space is limited – apply for a spot today.