Google I/O 2026 Wasn’t About AI Models — It Was About Agent Execution Layers

title: Google I/O 2026 Wasn’t About AI Models — It Was About Agent Execution Layers
published: true
tags: ai, googleio, agents, architecture

This is a submission for the Google I/O Writing Challenge

Google I/O 2026 Wasn’t About AI Models — It Was About Agent Execution Layers

Most discussions around Google I/O 2026 focused on model capabilities.

Gemini got smarter.
AI Studio improved.
Agent workflows became easier.
On-device AI became more practical.

But I think the real shift happened somewhere deeper.

Google I/O 2026 was not just about better AI models.

It was about the emergence of an Agent Execution Layer.

And once you start building multi-agent systems in the real world, you quickly discover something uncomfortable:

The hardest problem is no longer intelligence.

It is state management.

⸻

The Problem Nobody Talks About

When developers first build AI systems, the architecture usually looks simple:

User -> LLM -> Response

But the moment you move into agent workflows, everything changes.

Now you suddenly have:

  • multiple agents
  • tool execution
  • memory systems
  • long context histories
  • role switching
  • state inheritance
  • retrieval pipelines
  • security boundaries
  • autonomous actions

And eventually, the architecture becomes something closer to:

User
↓
Coordinator Agent
↓
Execution Agents
↓
Memory Layer
↓
Tool Runtime
↓
External APIs / Environment

At this point, prompts stop being “messages.”

They become something closer to an operating system.

⸻

Context Is Becoming the New Bottleneck

Most people still think model performance is the primary scaling problem.

I don’t think that’s true anymore.

The bigger problem is this:

Context grows faster than reasoning quality.

The more capable agents become, the more memory, instructions, logs, and coordination data they accumulate.

This creates several failure modes:

  • context bloat
  • instruction conflicts
  • memory drift
  • role collapse
  • hidden prompt inheritance
  • prompt injection propagation
  • state contamination between agents

Ironically, smarter agents amplify orchestration problems.

This is where I think the next generation of AI infrastructure will emerge.

⸻

From “Prompt Engineering” to “State Engineering”

For the last two years, the industry focused heavily on prompt engineering.

But prompt engineering assumes something important:

That interaction is temporary.

Agent systems break this assumption.

Agents persist.
Agents inherit memory.
Agents maintain roles.
Agents accumulate behavioral state over time.

That means the problem changes from:

“What should the AI say?”

to:

“What state should the AI exist in?”

This is a fundamentally different design philosophy.

⸻

Building Around the Problem

Over the past year, I started building several experimental concepts around this issue while working on multi-agent workflows, memory systems, and autonomous orchestration experiments.

Some examples:

Context Pointer OS

Instead of continuously passing gigantic raw histories into models, agents should reference contextual structures through lightweight pointers.

In other words:

Don’t pass the entire world.
Pass references to the world.

This reduces token waste while making long-term coordination more stable.

Project:
https://github.com/kagioneko/context-pointer-os

⸻

AI Instruction Tape (AIT)

Human language is extremely expensive for agent-to-agent communication.

AIT experiments with compressed instruction transfer between AI systems.

Instead of repeatedly sending huge natural language prompts, agents exchange compact operational context.

Project:
https://github.com/kagioneko/ai-instruction-tape

⸻

Esoteric AI Protocol (EAP)

As multi-agent ecosystems grow, natural language alone becomes inefficient as an execution protocol.

EAP explores lightweight structured communication for agent coordination.

Project:
https://github.com/kagioneko/esoteric-ai-protocol

⸻

Google I/O 2026 Confirmed Something Important

What Google showed this year was not just AI tooling.

It was the beginning of infrastructure for persistent AI execution.

The moment agents become:

  • autonomous
  • stateful
  • collaborative
  • tool-connected
  • environment-aware

the industry stops being purely about model quality.

It becomes about:

  • orchestration
  • memory integrity
  • state synchronization
  • execution governance
  • agent operating systems

In other words:

The future of AI is not just model architecture.
It is runtime architecture.

⸻

The Security Side Is Going to Matter More Than People Think

One thing I learned from real-world VPS incidents and autonomous agent experiments:

The more authority agents gain, the more dangerous context corruption becomes.

A compromised context is effectively a compromised execution environment.

This means future AI systems will likely require:

  • memory validation
  • state auditing
  • execution boundaries
  • agent isolation
  • instruction provenance
  • behavioral monitoring

AI security may gradually evolve into something closer to operating system security.

And honestly, I think we are still very early.

⸻

Final Thoughts

Google I/O 2026 felt like a transition point.

Not because AI suddenly became intelligent.

But because the ecosystem started shifting from:

AI as conversation

to:

AI as infrastructure

And once that happens, developers will need new abstractions.

Not just better prompts.

But:

  • state layers
  • memory architectures
  • execution runtimes
  • agent protocols
  • orchestration operating systems

I think that’s where the next major wave of AI development is heading.

Monorepos in 2026: Turborepo vs Nx vs Bazel — What Actually Works

Monorepos in 2026: What Actually Works

The monorepo debate in 2026 has settled into something more mature. Turborepo became the default for most JS/TS teams, Nx found its niche in enterprises with complex dependency graphs, and Bazel still dominates at Google scale. Here’s what I’ve learned from running all three in production.

Why Monorepos Won

Let’s be clear about why monorepos became the default in 2026:

  1. Atomic commits across services — Change an API contract and update all consumers in one PR
  2. Shared tooling — One ESLint config, one Prettier config, one TypeScript config
  3. Easy refactoring — Rename a function used across 20 packages without coordination
  4. Unified CI/CD — One pipeline that understands dependency relationships
  5. Developer experience — One git clone, one npm install, everything works

The alternative — polyrepo hell where a simple rename requires 15 PRs across 15 repos — killed productivity. Teams migrated.

Turborepo: The Default Choice

Turborepo won because it has sensible defaults, a gentle learning curve, and just works for most JavaScript/TypeScript projects. It’s not the most powerful, but it’s the most practical.

Setting Up a Turborepo Project in 2026

// package.json (root)
{
  "name": "my-monorepo",
  "private": true,
  "workspaces": ["apps/*", "packages/*"],
  "scripts": {
    "build": "turbo build",
    "dev": "turbo dev",
    "lint": "turbo lint",
    "test": "turbo test"
  },
  "devDependencies": {
    "turbo": "^2.0.0",
    "typescript": "^5.4.0"
  }
}
// turbo.json
{
  "$schema": "https://turbo.build/schema.json",
  "tasks": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": [".next/**", "!.next/cache/**", "dist/**"]
    },
    "dev": {
      "cache": false,
      "persistent": true
    },
    "lint": {
      "dependsOn": ["^build"]
    },
    "test": {
      "dependsOn": ["build"],
      "outputs": ["coverage/**"],
      "inputs": ["src/**/*.tsx", "src/**/*.ts", "test/**/*.ts"]
    }
  }
}

Real Package Dependencies

apps/
  web/          → depends on @repo/ui, @repo/api-client
  admin/        → depends on @repo/ui, @repo/api-client
  docs/         → depends on @repo/ui, @repo/docs-utils
packages/
  ui/           → no internal deps
  api-client/   → no internal deps
  docs-utils/   → no internal deps
  config/       → no internal deps (ESLint, TypeScript configs)

The Cache That Actually Works

# First build (slow)
$ turbo build
• building web
• building admin
• building docs
• building ui
• building api-client
• building docs-utils

→ Tasks run by turbo: 6 (64ms each, 384ms total)

# Second build (instant - cache hit)
$ turbo build
• building web (cache hit)
• building admin (cache hit)
• building docs (cache hit)
• building ui (cache hit)
• building api-client (cache hit)
• building docs-utils (cache hit)

→ Tasks run by turbo: 0 (6 in cache, 12ms total)

Nx: When You Need the Power

Nx is Turborepo on steroids. It understands your project graph at a deeper level, has built-in code generators, affected commands, and powerful visualization tools. The tradeoff: more complexity, steeper learning curve.

Nx Project Setup

# Create Nx workspace
npx create-nx-workspace@latest myorg --preset=ts

# Add a new application
npx nx generate @nx/nextjs:app admin

# Add a library
npx nx generate @nx/js:library api-client --directory=packages/api-client

The Affected Command (Nx’s Killer Feature)

# Only build/test/lint what changed (and what depends on what changed)
npx nx affected --target=build --base=origin/main

# This is the game-changer for large monorepos
# Instead of rebuilding everything, Nx computes the dependency graph
# and only rebuilds the affected packages

Visualizing the Project Graph

# Opens browser with interactive dependency graph
npx nx graph

This generates a visual graph of your entire project. You can see exactly which packages depend on which, spot circular dependencies, and understand the impact of proposed changes.

Nx Computation Cache (Distributed)

// nx.json
{
  "namedInputs": {
    "default": ["{projectRoot}/**/*"],
    "production": ["!{projectRoot}/**/*.spec.ts"]
  },
  "targetDefaults": {
    "build": {
      "dependsOn": ["^build"],
      "cache": true,
      "inputs": ["production", "^production"]
    }
  },
  "nxCloudAccessToken": "your-nx-cloud-token"
}

Nx Cloud provides distributed caching — your CI server and local machine share the same build cache. Cold CI builds become warm builds instantly.

Bazel: At Google Scale

Bazel is the nuclear option. It’s infinitely scalable, hermetically sealed (reproducible builds), and can handle truly massive codebases. The tradeoff: significant operational overhead, Starlark learning curve, and ecosystem fragmentation outside the Google ecosystem.

When Bazel Makes Sense

  • 1,000+ engineers in a single codebase
  • Multiple languages (TypeScript + Python + Go + Rust + Java)
  • Need per-language optimized tooling
  • hermetic builds are a hard requirement
  • You’re okay with dedicated build infrastructure team

For most teams, Bazel is overkill. The rule of thumb: if you’re asking whether you need Bazel, you don’t.

The Migration Path

From Polyrepo to Turborepo

Step 1: Create the monorepo structure

mkdir my-monorepo && cd my-monorepo
git init
npm init -y
npm install -D turbo
mkdir apps packages

Step 2: Move packages one at a time

# Move one package
mv ../old-repo/packages/ui apps/ui
cd apps/ui
npm init -y
# Update package.json name to @myorg/ui
cd ../..
git add -A
git commit -m "feat: move ui package"

Step 3: Fix dependency issues

// Before (old package.json)
{
  "name": "ui",
  "dependencies": {
    "react": "^18.0.0"
  }
}

// After (in monorepo)
{
  "name": "@myorg/ui",
  "peerDependencies": {
    "react": "^18.0.0"
  }
}

Step 4: Add shared configs

# packages/config-eslint/index.js
module.exports = {
  extends: ['next/core-web-vitals', 'turbo'],
  rules: {
    '@typescript-eslint/no-unused-vars': 'error'
  }
};

# packages/config-typescript/base.json
{
  "compilerOptions": {
    "target": "ES2022",
    "lib": ["ES2022"],
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true
  }
}

The Hidden Costs Nobody Warns You About

Build Tool Lockstep

When you share a config package across 20 apps, updating TypeScript becomes a coordinated event. You can’t have App A on TypeScript 5.3 and App B on 5.4 while they’re in the same repo.

Solution: Use pnpm workspaces with version pinning:

# pnpm-workspace.yaml
packages:
  - 'apps/*'
  - 'packages/*'

# .npmrc
save-exact=true

The Big Git History Question

Do you keep git history when migrating?

# Option 1: Keep history (preserve all commit messages)
git subtree add --prefix=apps/web https://github.com/old/web.git main

# Option 2: Fresh start (cleaner, lose history)
mv old-repo/web apps/web

I recommend Option 2 for most migrations. Git history for individual packages in a monorepo is rarely useful, and the migration complexity isn’t worth it.

CI/CD Complexity

# GitHub Actions with Turborepo
name: CI

on: [push, pull_request]

jobs:
  affected:
    name: Check affected
    runs-on: ubuntu-latest
    outputs:
      affected: ${{ steps.turbo.outputs.affected }}
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: pnpm/action-setup@v3
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: pnpm
      - run: pnpm install --frozen-lockfile
      - id: turbo
        run: echo "affected=$(pnpm turbo run build --dry-run=json | jq -r '.packages | if length > 0 then "true" else "false" end')" >> $GITHUB_OUTPUT

  build:
    needs: affected
    if: needs.affected.outputs.affected == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v3
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: pnpm
      - run: pnpm install --frozen-lockfile
      - run: pnpm turbo build --filter=...affected packages...

2026 Tooling Ecosystem

Tool Language Agnostic Learning Curve Scale Best For
Turborepo JS/TS mostly Gentle < 100 packages Most teams
Nx Multiple Medium < 500 packages Enterprises
Bazel Any Steep Unlimited Google-scale
Lerna JS/TS Gentle < 50 packages Legacy (deprecated)

My Recommendation

For most teams in 2026: Start with Turborepo. It’s the right default. You get 80% of the benefit for 20% of the complexity.

If you’re in an enterprise with complex dependencies, code generators, and a dedicated platform team: Nx is worth the investment.

If you’re at Google scale: Bazel. But you already knew that.

The worst choice: using no monorepo tooling at all. Running a monorepo with just npm workspaces and manual dependency tracking is technical debt that compounds.

Running a monorepo? What’s your tool of choice in 2026? Let’s hear real-world experiences.

Further reading: Turborepo docs, Nx docs, Bazel migration guide

How I Built a Real-Time Robot Battle Simulator from Scratch — Logic Arena

A CS student’s journey from a blank repo to a production platform with a custom scripting language, 60fps physics engine, and 3,000+ lines of battle-tested TypeScript.

The Idea

I wanted to build something that made competitive programming fun. Not another LeetCode clone. Something where your algorithm doesn’t just pass a test case — it fights.

The concept: players write code to control robots. The robots battle in real-time. Your logic vs the world.

That’s Logic Arena.

Starting From Zero — The Engine

The first challenge was building a game engine from scratch. No Unity, no Phaser — pure TypeScript.

Version 0.2.0 was just two robots moving in a canvas. But getting there required solving the first real technical scar:

The Singleton Problem. Two NestJS services were sharing the game engine — or so I thought. They each had their own instance, so state was completely desynced. The fix: @Global() decorator to enforce a single shared engine across the entire module.

@Global()
@Module({
  providers: [GameService],
  exports: [GameService],
})
export class GameModule {}

One decorator. Three hours of debugging.

Building AliScript — A Custom Scripting Language

This was the craziest decision I made. Instead of using JavaScript or Lua for robot scripting, I built my own language.

Why? I wanted players to think algorithmically, not just copy-paste code. AliScript had to be simple enough for beginners but powerful enough to reward expert thinking.

The pipeline:

String Script → Lexer → Tokens → Parser → AST → Server Evaluator → Robot Actions

First version supported basic conditionals:

IF CAN_SEE_ENEMY
  FIRE
END
MOVE

By v2.4, it became Turing-complete-ish:

WHILE TRUE DO
  IF CAN_SEE_ENEMY AND MY_ENERGY > 30 DO
    BROADCAST(NEAREST_VISIBLE_X)
    BURST_FIRE
  ELSE IF IN_STASIS DO
    WAIT
  ELSE DO
    SCAN
    PATHFIND
  END
END

The hardest bug? Operator precedence.

2 + 3 * 4 was evaluating to 20 instead of 14. Every multiplication-heavy script was silently computing wrong values. The fix required splitting parseBinaryExpression() into two separate precedence levels — parseAddition() and parseMultiply().

The Performance Crisis

By v1.3.0, the arena was running at ~15fps. Profiling revealed the culprit:

3,862ms scripting bottleneck — the main thread was choking on useState updates 60 times per second.

The fix was a dual-state architecture:

// Before: useState causes re-render on every tick
const [gameState, setGameState] = useState(null);

// After: useRef for game logic, throttled state for UI only
const gameStateRef = useRef(null); // updates at 60fps, zero re-renders
const [uiState, setUiState] = useState(null); // updates at 10fps, DOM only

Scripting time dropped from 3,862ms to near zero.

Six Performance Fixes in One Release

By v2.5.0, I ran a full profiling audit and found six simultaneous bottlenecks:

1. Obstacles in every WebSocket payload — static data sent 10x/sec for no reason. Fix: initialize once, strip from all subsequent payloads.

2. 30 WebGL draw calls for obstacles — rewrote to THREE.InstancedMesh. 30 draw calls → 4 draw calls.

3. 10 useFrame JS callbacks for obstacle animations — moved all pulse math to GPU via fragment shaders. 0 JS callbacks per frame.

4. Server memory leak — replay snapshots were deep-cloning unboundedly. Added ring buffer capped at 300 objects.

5. The Ghost Match Massacre — when players closed their browser, the physics engine kept running at full speed indefinitely. Hundreds of ghost matches accumulating CPU in silence. Fix: wired disconnect lifecycle to stop the engine the moment the last player leaves.

The Deployment Nightmare

Going from localhost to production at logicarena.dev was a gauntlet.

The 2.57GB Docker Context Bomb — first build transferred 2.57GB to the Docker daemon because node_modules wasn’t excluded. Fixed .dockerignore, dropped build context to 15MB.

The Prisma Ghost Engine — Alpine Linux refused to execute the Windows-compiled Prisma binary. Added linux-musl to binaryTargets.

The WebSocket CORS Wall — every Socket.IO connection rejected because the gateway was hardcoded to localhost:3000. Extended CORS array, fixed Nginx WebSocket proxy headers.

The Silent Postman — DigitalOcean silently blocks outbound SMTP on new Droplets. Zero errors, zero delivery, just void.

The Architecture Today

logic-arena/
├── apps/
│   ├── client/     # Next.js 16, React Three Fiber, PWA
│   └── server/     # NestJS 11, Socket.io, JWT
└── packages/
    ├── engine/     # Custom TypeScript physics engine
    └── logic-parser/ # AliScript lexer, parser, evaluator

What I Learned

1. Architecture decisions compound. Every “quick fix” that bypassed the module system created three bugs later.

2. Profile before optimizing. The 3,862ms bottleneck was invisible until I actually measured it.

3. Custom languages are feasible. An AST evaluator is just a recursive tree walker — intimidating name, straightforward implementation.

4. Ship early, iterate fast. Logic Arena went from 2 robots on a canvas to a 60-level algorithmic campaign in 4 months of solo development.

Try It

Live: logicarena.dev — no account required, join as guest.

GitHub: Ali-Haggag7/logic-arena

Write your first script, watch your robot fight, and tell me what you think in the comments!

Rust in 2026: The Systems Language That Finally Became Approachable

Rust in 2026: The Systems Language That Finally Became Approachable

Rust crossed the chasm. In 2024 it was “the language everyone’s excited about but few use in production.” In 2026, it’s running in production at Microsoft, Google, Amazon, Cloudflare, and a generation of startups. The borrow checker that seemed impenetrable is now understood intuitively by hundreds of thousands of developers. Here’s what changed.

The Adoption Curve

The Rust user survey tells the story:

  • 2023: 28% of respondents used Rust in production
  • 2025: 47% of respondents used Rust in production
  • Most importantly: the “learning Rust” satisfaction rate went from 42% to 71%

The language stopped being “hard to learn” and started being “hard to learn but worth it.”

What Made Rust Approachable in 2026

1. Better Error Messages

The borrow checker error messages are legendary now. They don’t just say “no” — they explain why and suggest how to fix it.

fn main() {
    let s = String::from("hello");
    let r1 = &s;
    let r2 = &s;
    println!("{} and {}", r1, r2);
}

The compiler output:

error[E0502]: cannot borrow `s` as immutable because it is also
borrowed as mutable
  --> src/main.rs:4:13
   |
3  |     let r1 = &s;
   |              -- first borrow occurs here
4  |     let r2 = &s;
   |              ^^ second borrow occurs here
5  |     println!("{} and {}", r1, r2);
   |                            -- first borrow needs to be valid
   |                            for the duration of the borrow

help: consider using the reference count for shared ownership

help: consider using `Rc<String>` if you don't need exclusive ownership

Compare this to C++ segfaults that just say “Segmentation fault (core dumped).”

2. Async Rust Stabilized

The async/await syntax stabilized and the ecosystem matured significantly.

// Before (2024): Complex futures, Pin, Box, manual polling
// After (2026): Clean async/await like Go/Python

use tokio;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let data = fetch_data("https://api.example.com/data").await?;
    let processed = process(data).await;
    save_to_db(processed).await?;
    Ok(())
}

async fn fetch_data(url: &str) -> Result<String, reqwest::Error> {
    reqwest::get(url).await?.text().await
}

3. The crate ecosystem reached critical mass

The crates.io ecosystem now has everything you need for production:

# Cargo.toml — Production Rust is just dependency management
[dependencies]
# Web framework
axum = "0.7"
tower = "0.4"
tower-http = { version = "0.5", features = ["cors", "compression"] }

# Async runtime
tokio = { version = "1", features = ["full"] }

# Serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"

# Database
sqlx = { version = "0.8", features = ["runtime-tokio", "postgres"] }

# Error handling
anyhow = "1"
thiserror = "2"

# Logging
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }

# Testing
tokio-test = "0.4"

Real Production Example: A Web API in Axum

use axum::{
    extract::{Path, State},
    http::StatusCode,
    response::Json,
    routing::{get, post},
    Router,
};
use serde::{Deserialize, Serialize};
use sqlx::postgres::PgPoolOptions;
use std::sync::Arc;
use tower_http::cors::CorsLayer;

#[derive(Debug, Serialize, Deserialize, sqlx::FromRow)]
struct User {
    id: i32,
    name: String,
    email: String,
}

#[derive(Clone)]
struct AppState {
    db: sqlx::PgPool,
}

async fn create_user(
    State(state): State<Arc<AppState>>,
    Json(payload): Json<CreateUserPayload>,
) -> Result<(StatusCode, Json<User>), AppError> {
    let user = sqlx::query_as::<_, User>(
        r#"INSERT INTO users (name, email) VALUES ($1, $2)
           RETURNING id, name, email"#
    )
    .bind(&payload.name)
    .bind(&payload.email)
    .fetch_one(&state.db)
    .await?;

    Ok((StatusCode::CREATED, Json(user)))
}

async fn get_user(
    Path(user_id): Path<i32>,
    State(state): State<Arc<AppState>>,
) -> Result<Json<User>, AppError> {
    let user = sqlx::query_as::<_, User>(
        "SELECT id, name, email FROM users WHERE id = $1"
    )
    .bind(user_id)
    .fetch_optional(&state.db)
    .await?
    .ok_or(AppError::NotFound)?;

    Ok(Json(user))
}

#[derive(Debug, thiserror::Error)]
enum AppError {
    #[error("User not found")]
    NotFound,
    #[error("Database error: {0}")]
    Database(#[from] sqlx::Error),
}

impl IntoResponse for AppError {
    fn into_response(self) -> Response {
        let (status, message) = match self {
            AppError::NotFound => (StatusCode::NOT_FOUND, self.to_string()),
            AppError::Database(e) => (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()),
        };
        (status, Json(serde_json::json!({ "error": message }))).into_response()
    }
}

type Response = axum::response::Response;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    tracing_subscriber::fmt::init();

    let db = PgPoolOptions::new()
        .max_connections(5)
        .connect("postgres://user:pass@localhost/mydb")
        .await?;

    let state = Arc::new(AppState { db });

    let app = Router::new()
        .route("/users", post(create_user))
        .route("/users/:id", get(get_user))
        .layer(CorsLayer::permissive())
        .with_state(state);

    let listener = tokio::net::TcpListener::bind("0.0.0.0:3000")
        .await?;
    axum::serve(listener, app).await?;

    Ok(())
}

The Memory Safety Argument: No Longer Theoretical

The big shift in 2025-2026: real-world security vulnerabilities dropped in organizations that switched to Rust.

// C/C++ memory vulnerabilities (common in production):
// - Buffer overflow
// - Use after free
// - Double free
// - Race conditions

// The National Security Agency (NSA) and CISA now recommend:
// "Consider migrating to memory-safe languages like Rust"

Microsoft reported that 70% of their CVEs (security vulnerabilities) were memory safety issues. In C/C++, these are endemic. In Rust, the borrow checker makes them impossible by default.

Rust vs Go: The Honest Comparison

The Rust vs Go debate settled into a clearer consensus:

Use Case Winner Reason
Web APIs (simple) Go Faster to write, good enough performance
Web APIs (high performance) Rust 10x lower latency, predictable memory
Systems programming Rust Memory safety without GC
Networking infrastructure Rust Cloudflare, Fastly, AWS chose Rust
Microservices (throughput critical) Rust Wizer, Shuttle, Encore chose Rust
Rapid prototyping Go Smaller learning curve
CLI tools Either Both excellent
Embedded Rust No runtime, deterministic

The Compiler Performance Story

The biggest legitimate complaint about Rust was compile times. In 2026, it’s still slower than Go, but significant improvements arrived:

# Rust incremental compilation improved dramatically

# First build (cold cache)
$ cargo build --release
# Time: ~45s for a medium project

# Incremental build (one file changed)
$ cargo build --release
# Time: ~8s (was ~35s in 2024)

# rust-analyzer type checking
# VS Code + rust-analyzer gives instant feedback
# Before you even save, you see borrow errors

The trick: use cargo check during development (faster, no code generation), cargo build for final builds.

Rust’s Achilles Heel: Compile Times

# For faster iteration, use these settings in .cargo/config.toml

[build]
# Use all CPU cores for linking
rustflags = ["-C", "linker=clang", "-C", "codegen-units=1"]

# Or use mold linker (much faster than default lld)
[target.x86_64-unknown-linux-gnu]
linker = "clang"
rustflags = ["-C", "link-arg=-fuse-ld=mold"]

The WASM Story: Rust + WASM = Production Ready

Rust is the language of choice for WebAssembly:

// math_utils/src/lib.rs
use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub fn fibonacci(n: u32) -> u64 {
    match n {
        0 => 0,
        1 => 1,
        _ => {
            let mut a: u64 = 0;
            let mut b: u64 = 1;
            for _ in 2..=n {
                let temp = a + b;
                a = b;
                b = temp;
            }
            b
        }
    }
}
# Build for WASM
cargo install wasm-pack
wasm-pack build --target web

# 200x faster than JavaScript for Fibonacci
# Smaller binary than comparable C++ WASM
# Zero runtime overhead

What Rust Is NOT Good For

Let’s be honest:

# Data science / ML
# Python with NumPy/PyTorch is the standard
# Rust bindings exist but ecosystem is fragmented
# Not worth the learning curve for ML work

# Scripts and automation
# Python/Bash/Shell are faster to write
# Rust is overkill for one-off scripts

# Simple CRUD APIs where Go/Python is "good enough"
# Don't use Rust because it's cool
# Use it when you need the performance or safety

The Learning Curve Reality

# Realistic timeline to productivity in Rust:

Week 1: Fighting the borrow checker constantly
Week 2: Understanding ownership conceptually
Week 3: Writing simple programs without fighting
Month 2: Comfortable with lifetimes in structs
Month 3: Can navigate async Rust and the crate ecosystem
Month 6: Productive Rust developer

# The curve is real but the investment pays off
# Once you internalize ownership, you see memory bugs everywhere in C code

The Bottom Line

Rust in 2026 is production-ready for:

  • Systems where memory safety is non-negotiable
  • Networking infrastructure (proxies, CDNs, firewalls)
  • WebAssembly modules (performance-critical browser code)
  • High-throughput services where Go isn’t fast enough
  • Any codebase where you want to eliminate an entire class of bugs

It’s not ready for:

  • Rapid prototyping (use Python or Go)
  • Data science (use Python)
  • Teams without time to invest in the learning curve

The question isn’t “is Rust ready?” It’s “is Rust right for your use case?”

Learning or using Rust in 2026? What’s your experience been?

Observability in 2026: Distributed Tracing Replaced Logs, and OpenTelemetry Won

Observability in 2026: Distributed Tracing Replaced Logs, and OpenTelemetry Won

The observability landscape in 2026 looks nothing like 2020. Logs are now secondary. Traces are primary. And OpenTelemetry (OTel) won the instrumentation wars so decisively that the term “vendor-neutral observability” became a redundant phrase. Here’s what changed.

The Old Model: Logs as the Source of Truth

In 2020, debugging meant logs:

# The old way
logger.info(f"Processing order {order_id} for user {user_id}")
logger.info(f"Payment processing for ${amount}")
logger.error(f"Payment failed: {error_code}")

# Debugging a production issue:
# 1. Find the right log lines across 50 service logs
# 2. Correlate timestamps across machines (which may not be synced)
# 3. Reconstruct what happened from thousands of log lines
# 4. Hope the relevant lines weren't filtered out by your logging library

This model broke down with microservices. A single user request touches 20 services. Correlating logs across 20 services at different timestamps is archaeology, not engineering.

The New Model: Traces as Primary

# The new way: OpenTelemetry
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

# Set up the tracer
trace.set_tracer_provider(TracerProvider())
trace.get_tracer_provider().add_span_processor(
    BatchSpanProcessor(OTLPSpanExporter())
)
tracer = trace.get_tracer(__name__)

# Instrument your code
def process_order(order_id: str, user_id: str, amount: float):
    with tracer.start_as_current_span("process_order") as span:
        span.set_attribute("order.id", order_id)
        span.set_attribute("user.id", user_id)
        span.set_attribute("order.amount", amount)

        with tracer.start_as_current_span("validate_order"):
            # validation logic
            pass

        with tracer.start_as_current_span("process_payment") as payment_span:
            payment_span.set_attribute("payment.method", "stripe")
            result = stripe.charge(amount)
            payment_span.set_attribute("payment.status", result.status)

            with tracer.start_as_current_span("send_confirmation"):
                send_email(user_id, result)

Now when you look at your observability platform, you see:

process_order (2.3s)
├── validate_order (0.1s)
├── process_payment (2.1s)
│   ├── stripe.charge (1.8s)
│   └── send_confirmation (0.3s)

One trace. Every service. Complete latency breakdown. No log archaeology.

OpenTelemetry: The Standard That Won

OpenTelemetry is now the universal instrumentation standard. Every major observability platform supports it:

  • Datadog ✓
  • Honeycomb ✓
  • Grafana Tempo ✓
  • Jaeger ✓
  • New Relic ✓
  • AWS X-Ray ✓
  • Google Cloud Trace ✓
# otel-collector-config.yaml
receivers:
  otlp:
    protocols:
      grpc:
      http:

processors:
  batch:
    timeout: 5s
    send_batch_size: 1024

  memory_limiter:
    check_interval: 1s
    limit_mib: 4000

exporters:
  otlp/tempo:
    endpoint: tempo:4317
    tls:
      insecure: false

  datadog:
    api:
      key: ${DATADOG_API_KEY}

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp/tempo, datadog]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp/tempo, datadog]

Auto-Instrumentation: Zero-Code Observability

The biggest win in 2026: auto-instrumentation. You get distributed tracing without changing your code.

Python Auto-Instrumentation

# Install the agent
pip install opentelemetry-instrumentation-all

# Run your app with auto-instrumentation
opentelemetry-instrument python your_app.py

This automatically instruments:

  • HTTP requests (Flask, FastAPI, Django, aiohttp)
  • Database calls (psycopg2, SQLAlchemy, asyncpg)
  • Redis, Memcached, Kafka
  • gRPC, HTTPX

Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-service
spec:
  template:
    spec:
      containers:
      - name: my-service
        image: my-service:latest
        env:
        - name: OTEL_SERVICE_NAME
          value: "my-service"
        - name: OTEL_EXPORTER_OTLP_ENDPOINT
          value: "http://otel-collector:4317"
        - name: OTEL_RESOURCE_ATTRIBUTES
          value: "deployment.environment=production"
        - name: OTEL_PROPAGATORS
          value: "tracecontext,baggage"
        - name: OTEL_TRACES_SAMPLER
          value: "parentbased_traceidratio"
        - name: OTEL_TRACES_SAMPLER_ARG
          value: "0.1"  # Sample 10% of traces

The Three Pillars: Traces, Metrics, Logs

Metrics: SLOs and Alerts

from opentelemetry import metrics
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter

# Set up metrics
metric_reader = PeriodicExportingMetricReader(
    OTLPMetricExporter(), export_interval_millis=30000
)
metrics.set_meter_provider(MeterProvider(metric_readers=[metric_reader]))
meter = metrics.get_meter(__name__)

# Create metrics
order_counter = meter.create_counter(
    "orders_processed",
    description="Number of orders processed",
    unit="1"
)

payment_duration = meter.create_histogram(
    "payment_duration",
    description="Payment processing duration",
    unit="ms"
)

error_counter = meter.create_counter(
    "payment_errors",
    description="Number of payment errors"
)

# Use them
def process_payment(amount: float):
    with tracer.start_as_current_span("process_payment"):
        try:
            start = time.time()
            result = stripe.charge(amount)
            payment_duration.record((time.time() - start) * 1000)
            order_counter.add(1, {"status": "success"})
            return result
        except Exception as e:
            error_counter.add(1, {"error": type(e).__name__})
            raise

Structured Logs (Still Useful, But Secondary)

import structlog

structlog.configure(
    processors=[
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer()
    ]
)

log = structlog.get_logger()

# Structured logs now include trace context automatically
log.info("payment_processed",
    order_id="12345",
    amount=99.99,
    # These are automatically injected from the current trace:
    # trace_id, span_id, trace_flags
)

Sampling Strategies: The Key to Cost Control

Traces are verbose. You can’t afford to trace 100% of requests at scale. Sampling is essential.

Head-Based Sampling (At Trace Start)

from opentelemetry.sdk.trace.samplers import TraceIdRatioBased

# Sample 10% of all traces
sampler = TraceIdRatioBased(0.1)

provider = TracerProvider(sampler=sampler)

Tail-Based Sampling (After Trace Completes)

Tail-based sampling captures errors and slow requests while sampling most fast successful requests. This requires your observability platform to support it.

# Grafana Tempo tail-based sampling
overrides:
  "service.namespace:payments":
    processors:
      - type: latency
        threshold_ms: 1000  # Always keep traces > 1s
      - type: status_code
        status_codes:
          - ERROR  # Always keep errors
      - type: trace_state
        key: environment
        values: [production]  # Always keep production
      - type: probabilistic
        sampling_percentage: 5  # Sample 5% of the rest

Service Level Objectives (SLOs) in Your Observability Platform

# Grafana Tempo + SLO example
groups:
  - name: orders-slo
    rules:
    - alert: OrderLatencyHigh
      expr: |
        histogram_quantile(0.95,
          sum(rate(tracetest_s spans{ service="order-service" }[5m]))
          by (le)
        ) > 1000
      for: 5m
      labels:
        severity: critical
      annotations:
        summary: "Order processing P95 latency > 1s"
        runbook_url: "https://wiki.example.com/runbooks/order-latency"

    - alert: PaymentErrorRateHigh
      expr: |
        sum(rate(tracetest_spans{ 
          service="payment-service",
          span.kind="server",
          otel.status_code="ERROR"
        }[5m])) /
        sum(rate(tracetest_spans{ 
          service="payment-service",
          span.kind="server"
        }[5m])) > 0.01
      for: 2m
      labels:
        severity: critical

The Debugging Workflow in 2026

Before OTel

  1. Customer reports slow checkout
  2. Scrape logs from 20 services
  3. Reconstruct timeline from log timestamps
  4. Hope you can reproduce the issue
  5. Average time to resolution: 4+ hours

After OTel

  1. Customer reports slow checkout
  2. Open Grafana, search by user ID
  3. See the complete trace: 1.8s in Stripe, 0.5s in email
  4. Drill into the Stripe span: connection pool exhausted
  5. Average time to resolution: 15 minutes

The Observability Stack in 2026

Instrumentation Layer:
├── OpenTelemetry SDK (auto-instrumentation)
├── Language-specific agents (Python, Node, Go, Java, Rust)
└── Custom spans for business logic

Collection Layer:
├── OpenTelemetry Collector (otelcol)
├── Grafana Alloy (successor to Grafana Agent)
└── Vector (for logs and metrics)

Storage & Query Layer:
├── Grafana Tempo (traces) — S3/MinIO backend
├── Prometheus + Thanos (metrics)
├── Loki (logs)
└── Datadog/New Relic/Honeycomb (if you prefer managed)

Visualization:
└── Grafana (universal) or platform-native UIs

Alerting:
└── Grafana Alerting or platform-native

The Migration Path

Step 1: Deploy OTel Collector

# docker-compose.yml
services:
  otel-collector:
    image: otel/opentelemetry-collector:0.96.0
    volumes:
      - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
    ports:
      - "4317:4317"   # OTLP gRPC
      - "4318:4318"   # OTLP HTTP
      - "8888:8888"   # Prometheus metrics

Step 2: Instrument One Service

# Python
pip install opentelemetry-api 
            opentelemetry-sdk 
            opentelemetry-exporter-otlp 
            opentelemetry-instrumentation-flask

# Run with auto-instrumentation
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 
OTEL_SERVICE_NAME=my-service 
opentelemetry-instrument python app.py

Step 3: Verify in Grafana

Open Grafana → Explore → Select Tempo datasource → Search for your service name. If you see spans, instrumentation is working.

Step 4: Incremental Rollout

Add instrumentation service by service. Each service you add makes debugging easier across all previously-instrumented services.

The Bottom Line

OpenTelemetry won because it solved the real problem: instrument once, query anywhere, vendor-neutral forever. The cost is upfront instrumentation complexity, but the payoff is complete observability without vendor lock-in.

If you’re still running on logs alone, you’re debugging in 2020. Migrate to traces. Your future self (and your on-call rotations) will thank you.

Running OpenTelemetry in production? What’s your stack and biggest win?

Pause menus, asset deduplication, end-screen nodes (Devlog #2)

Hey again. I want to walk through where the canvas has gotten since then, because a lot of the work has been the unglamorous middle of a tool: the parts that don’t really sell anything but that you discover you can’t ship without. Pause menus. Asset cleanup. Editor performance. We added one big new node type in the last few days too, but most of the actual hours went into the boring half.

Here’s roughly where things landed.

The first thing we built was a pause menu. Until now ChoiceStory Studio had one editable surface, the main menu,and adding pause was the first time we had to decide what a sub-surface actually is. We landed on this: the pause menu is not its own node, it’s a second authoring face of the main menu, reached through a tab in the editor. Same right panel, same canvas tile, but at runtime it mounts as an overlay over the live scene when the player hits Escape. The design surface borrows the same drag and resize handles the main menu has, so the muscle memory carries over.

Then a feature we didn’t plan but turned out to be necessary: asset deduplication. When you import the same video twice, ChoiceStory now fingerprints both files and offers to share the existing entry instead of writing a duplicate.A test project we’d been kicking around had 106 files on disk; after dedup and orphan cleanup it sat at 4 referenced files. The fix had to go deeper than the dedup dialog. Every reference in a project (backgrounds, button images,fonts, the lot) now resolves through a registry that is the single source of truth, so “delete unused files” actually knows what’s used and what isn’t.

Around the same time we added a performance settings modal in the File menu. Three toggles: follow active node on selection, recompute canvas viewport on resize, node glow animations. Two presets: high-spec PC and low-spec PC. The reason is that our target audience is creators on consumer-grade machines, not workstations, and some of the canvas animations were burning more cycles than the canvas itself. The toggle group is a stopgap; before the alpha we want a real Performance tab. The internal rule that came out of this work: any new feature that adds visible CPU or GPU load gets a toggle on the way in, never as a retrofit.

Next came a side-quest we called the parity pass. The main menu had affordances the pause menu didn’t: per-element drag, per-element resize, draggable backgrounds with aspect-locked corner handles. So we pulled the drag and resize pipeline up out of main-menu/ (it never actually held any main-menu-specific knowledge in it) and pointed both surfaces at the same module. Future surfaces (choice overlays, options) inherit those affordances by reusing the same code, which was the whole point of the lift. We also closed a long-standing schema bug while we were in there. If you flipped a background between colour, image, and video schema keeps separate slots per variant and only the active one paints.

The newest addition is the end-screen node. It started life as another sub-surface (same shape as pause) and stayed that way for about ten minutes of planning before we changed our mind. End screens are different. A story can have several of them (good ending, bad ending, secret ending, joke ending) and each one needs its own configuration,so they had to be canvas-creatable nodes you can drop, configure, and connect to. Right-click on the canvas adds one.Right-click on the node copies it; right-click on empty canvas pastes a deep clone at the cursor. The config block for each instance is full: background, title, multi-line outro, music, font, a list of one to five buttons (per-button text, optional image, an appearance delay in milliseconds, and an action: return to main menu, restart, or quit). Optional auto-advance after a configurable delay. The buttons fade in one by one on a timeline the author writes. The runtime swap was the last piece. When a player choice routes to an end-screen node, the playback layer mounts the configured surface instead of trying to play it as a video.

Now the honest part.

A lot of the work above is what made it through. Plenty of stuff didn’t. The pause-menu post-closure pass shipped eleven follow-up commits across a single session, and three of them got reverted by commit ten, because every fix had downstream consequences we hadn’t mapped. Sub-state in React is the easy half; the hard half is the graph of “if I change this, what else re-renders, in what order, with which stale value.” That session ended with a rule we have actually been following since: for every bug fix, write down the side-effect surface before writing the patch, and a list of files you deliberately did not touch. The follow-up count has dropped sharply since.

The CSS side gave us a similar lesson. We spent a first iteration on container queries to scale the pause menu inside variable Player-slot sizes. They worked, mostly, until they didn’t on a small slot. We threw it out and used the recipe the main menu already used (viewport-relative clamps + flex with min-height: 0 + overflow: hidden) and it just behaved. The takeaway: if the older pattern already passes every test, the newer pattern needs a reason that survives ten minutes of “but does the existing one already do this?”

What’s next is the choice overlay editor, the panel where the buttons that appear over a video (or an image) get designed. Same surface contract, same shared drag and resize infrastructure. We expect it to come together faster than end-screen because most of the substrate is in place now.

If you’ve read this far, thanks. If you’ve built something with branching video, or want to, the same line as last time stands. Come tell us what you’d want from a tool like this. Reply here, find us on X as @ChoiceStoryDev, email,whatever’s easiest. The harsh stuff especially.

More soon…

From Models to Meaning: How Building NeuroSense AI with Gemma 4 Changed My View of Local AI

Introduction

As a computer science student, I have spent a lot of time experimenting with AI systems and reading about what they can do. But one thing kept bothering me.

Many AI systems today are powerful, but they often feel distant.

You send information to the cloud.

You wait for a response.

You get an answer.

And the cycle repeats.

That works for many applications, but I started asking myself a different question:

What happens when AI becomes more personal, more private, and closer to people?

That question became especially important while thinking about mental well-being applications.

People share deeply personal thoughts:

  • stress before examinations
  • anxiety
  • emotional struggles
  • moments of uncertainty

For systems handling sensitive conversations, privacy is not simply a feature.

It becomes part of the design itself.

While exploring this idea, I started working on a concept called NeuroSense AI, a privacy-focused stress insight assistant powered by Gemma 4.

And while building it, I realized I wasn’t only learning about a model.

I was learning about a different way to think about AI.

The Idea Behind NeuroSense AI

The purpose of NeuroSense AI is simple:

Allow users to express their thoughts naturally while receiving intelligent emotional insights and supportive guidance.

The system aims to:

  • understand conversational tone
  • identify emotional patterns
  • estimate stress indicators
  • provide helpful recommendations
  • preserve user privacy as much as possible

A user might type:

“I have exams tomorrow and I feel overwhelmed.”

Instead of generating only a generic answer, the system can attempt to understand emotional context and respond meaningfully.

That made me think about something important:

AI should not only process words.

Sometimes it should understand human context too.

Why I Chose Gemma 4

When building NeuroSense AI, choosing a model wasn’t only about selecting the largest model available.

I wanted the model choice to solve a specific problem.

Gemma 4 stood out because of several reasons.

Local Possibilities

Sensitive conversations are different from ordinary prompts.

Mental well-being applications often involve personal information.

Running AI closer to users can potentially improve:

  • privacy
  • accessibility
  • control

Multiple Model Sizes

Gemma 4 provides different model options depending on hardware requirements.

Smaller models can support:

  • mobile environments
  • edge systems
  • lower-resource devices

Larger variants can support:

  • reasoning-heavy workflows
  • larger conversations
  • advanced applications

This flexibility makes development more interesting.

Long Context Window

Gemma 4 introduces a 128K context window.

Initially I saw this as a technical specification.

Then I thought about practical use cases.

Long context can help with:

  • longer conversations
  • research assistance
  • large documents
  • session memory
  • understanding broader context

Context changes how AI feels.

Instead of isolated responses, interactions begin to feel more continuous.

What Building Taught Me

The most interesting lesson wasn’t technical.

It was human.

When people interact with AI systems, they are not always looking for perfect predictions.

Sometimes they want:

  • understanding
  • support
  • privacy
  • trust

As developers, we often focus on:

  • parameters
  • benchmarks
  • speed
  • performance

But building NeuroSense AI reminded me that behind every prompt is usually a person.

And that person matters more than the numbers.

Why Local AI Matters

I believe local AI changes several things:

Privacy

Sensitive information does not always need to leave the user’s environment.

Accessibility

Students and independent developers can build systems without requiring massive infrastructure.

Lower Latency

Less dependency on remote services can improve responsiveness.

Offline Intelligence

Useful AI experiences can exist even with limited internet access.

Final Thoughts

Before exploring Gemma 4, I mostly thought about AI in terms of capability.

Now I think more about responsibility.

Powerful models are important.

But meaningful applications are even more important.

The future of AI may not simply be larger models.

It may be smarter systems that work closer to people and solve real problems.

Building NeuroSense AI made me realize something:

The question is no longer:

“Can we build intelligent systems?”

The question is:

“How can we build systems that understand people better?”

I would love to hear what kinds of human-centered AI experiences others would build.

Docker Alternatives in 2026: Podman, Lima, containerd, and the End of the Docker Monopoly

Docker Alternatives in 2026: Podman, Lima, containerd, and the End of the Docker Monopoly

Docker is no longer the only game in town. Podman matured dramatically, Lima made macOS containers practical, and containerd became the standard for production. In 2026, choosing a container runtime requires actually understanding your options. Here’s the honest breakdown.

The Docker Monopoly Is Over

Docker’s dominance was always about convenience, not technical superiority. The Docker daemon (dockerd) that runs as root, the proprietary CLI, and the closed ecosystem were always compromises. In 2026, the alternatives are production-ready for most use cases.

Podman: Docker-Compatible Without the Daemon

Podman became the default for security-conscious teams. No daemon means no root privileges, no daemon crashes, and better systemd integration.

Installation and Setup

# macOS
brew install podman
podman machine init
podman machine start

# Linux (Fedora/RHEL already has it)
sudo dnf install podman

# Verify
podman run --rm docker.io/library/alpine echo "Podman works!"

The Daemon-Free Architecture

# Docker: daemon-based (root privilege required)
# dockerd runs as root, all containers are children of root process

# Podman: daemon-free (user privilege)
# Each container runs as a child of your user process
# No root daemon = no root vulnerabilities

# Podman 5.x (2026) features:
# - Rootless containers by default
# - Pods (like Kubernetes pods)
# - cgroups v2 fully supported
# - Kubernetes YAML support (podman generate kube)
# - Docker-compatible CLI (alias docker=podman works)

Pods: Kubernetes-Style Grouping

# Create a pod with multiple containers
podman pod create --name myapp-pod 
  -p 8080:80 
  -p 5432:5432

# Add containers to the pod
podman run -d --pod myapp-pod --name nginx nginx:alpine
podman run -d --pod myapp-pod --name postgres postgres:16

# All containers in the pod share the network namespace
# Access localhost:80 → nginx
# Access localhost:5432 → postgres

# Generate Kubernetes YAML from the pod
podman generate kube myapp-pod > myapp.yaml
# Now deploy to Kubernetes with zero changes

Rootless Containers

# Podman runs as your user, not root
$ podman run --rm alpine id
uid=0(root) gid=0(root)

# Wait, root? This is actually correct inside the container
# The container's root is mapped to an unprivileged user on the host

# Check on the host
podman unshare cat /proc/self/uid_map
# Shows: 0 1000 1 (container root = host user 1000)

# This means even if a container escapes, it has limited host access

Dockerfile Compatibility

# Podman uses Dockerfiles directly
# Just point to your existing Dockerfiles

podman build -t myapp:latest .
podman push myapp:latest docker://registry.example.com/myapp:latest
# or
podman push myapp:latest containers://registry.example.com/myapp:latest

# The container registries support both:
# docker:// (Docker registry protocol)
# containers:// (OCI registry protocol)

Lima: macOS Containers That Actually Work

Docker Desktop on macOS was always a compromise: a full Linux VM running Docker. Lima gives you the same result with less overhead.

The Problem with Docker Desktop on macOS

# Docker Desktop:
# - Runs a full Alpine Linux VM (2-4GB RAM)
# - Shares your file system via osxfs (slow)
# - Virtual USB/Network stack
# - $0-$21/month depending on company size

# Lima:
# - Uses macOS native virtualization (Hypervisor.framework)
# - Better performance
# - Native file sharing (virtiofs)
# - Free and open source

Setting Up Lima

# Install
brew install lima

# Create a template
limactl start

# It creates an Alpine Linux VM with:
# - containerd + nerdctl
# - BuildKit
# - BuildPull-Through caching
# - Rootful + Rootless support

# Use it like Docker
limactl shell default docker build -t myapp .
limactl shell default docker run -p 8080:80 myapp

Custom Lima Configuration

# lima.yaml (or any .yaml in ~/.lima/_config/)
images:
  - location: "https://deps.sh/lima/alpine/3.19.1/lima.yaml"
    arch: "x86_64"
  - location: "https://deps.sh/lima/alpine/3.19.1/lima.yaml"
    arch: "aarch64"

provision:
  - mode: system
    script: |
      # Install containerd and dependencies
      apk add --no-cache 
        containerd 
        docker 
        docker-cli-compose 
        buildkit

  - mode: user
    script: |
      # User-level setup
      systemctl --user enable containerd
      systemctl --user start containerd

provision_scripts:
  - mode: system
    script: |
      cat > /etc/docker/daemon.json <<'EOF'
      {
        "registry-mirrors": ["https://mirror.gcr.io"],
        "storage-driver": "overlay2"
      }
      EOF
      rc-service docker start

mounts:
  - location: "~"
    writable: true
  - location: "/tmp/lima"
    writable: true

networks:
  - lima: bridged

cpu: 4
memory: 8GB
disk: 100GB

containerd: The Standard for Production

containerd is what runs inside Docker and Kubernetes. You can use it directly for simpler, more secure deployments.

Why Use containerd Directly

# Docker stack (Docker Inc.'s product):
# docker CLI → dockerd (daemon) → containerd → runc → containers

# containerd directly:
# ctr CLI (or nerdctl) → containerd → runc → containers

# Benefits:
# - Smaller attack surface (no dockerd)
# - Direct access to OCI images
# - Better integration with Kubernetes
# - Simpler debugging

Using ctr (containerd CLI)

# Install
apt install containerd

# Pull images
ctr images pull docker.io/library/nginx:alpine

# List images
ctr images ls

# Run containers
ctr run -t --rm docker.io/library/alpine:latest test-container ash

# Manage namespaces (like docker ps)
ctr ns ls
ctr -n k8s.io containers ls

nerdctl: Docker-Compatible CLI for containerd

# Install nerdctl
brew install nerdctl

# nerdctl works like docker but uses containerd
nerdctl build -t myapp:latest .
nerdctl run -p 8080:80 myapp:latest
nerdctl compose up

# Extra features nerdctl adds:
# - Image encryption (--encrypt)
# - BuildKit with containerd snapshotter
# - Gzip compression for images
# - Lazy pulling (stargz)

BuildKit: Faster Builds with Cache Mounts

BuildKit is the modern builder for Docker/Podman/containerd. It handles concurrent builds, better caching, and more efficient layer management.

BuildKit.toml

# /etc/buildkit/buildkitd.toml
[registry."docker.io"]
  mirrors = ["registry.docker.io"]

[registry."gcr.io"]
  insecure = true  # For air-gapped environments

[worker.oci]
  max-parallelism = 4  # Limit concurrent builds

[driver]
  snapshotter = "overlayfs"  # Faster than native

Build Commands with Cache

# Build with inline cache (embed cache metadata in image)
docker build --build-arg BUILDKIT_INLINE_CACHE=1 -t myapp:latest .

# Build with cache mount (persist package manager caches)
docker build -t myapp:latest . <<'EOF'
# syntax=docker/dockerfile:1.7
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm 
    npm ci --only=production
COPY . .
EOF

# The npm cache persists across builds
# `npm ci` runs with the cached node_modules

Multi-Platform Builds

# Build for multiple architectures simultaneously
docker buildx create --use
docker buildx inspect --bootstrap

docker buildx build 
  --platform linux/amd64,linux/arm64 
  --tag myapp:latest 
  --push 
  .

# This builds simultaneously on:
# - amd64 (Intel/AMD)
# - arm64 (Apple Silicon, ARM servers)
# and pushes a manifest list to the registry

Kubernetes with containerd

# Kubernetes node configuration for containerd
# /etc/containerd/config.toml

version = 2

[plugins."io.containerd.grpc.v1.cri"]
  sandbox_image = "registry.k8s.io/pause:3.9"

  [plugins."io.containerd.grpc.v1.cri".containerd]
    default_runtime_name = "runc"
    snapshotter = "overlayfs"

    [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
      runtime_type = "io.containerd.runc.v2"
      privileged_without_host_devices = false

      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
        BinaryName = "/usr/bin/runc"
        SystemdCgroup = true

  [plugins."io.containerd.grpc.v1.cri".registry]
    config_path = "/etc/containerd/certs.d"

The Decision Framework

Tool Best For Installation Complexity
Docker Beginners, cross-platform dev Easy Low
Podman Security-conscious, Linux dev Easy Medium
Lima macOS users who want performance Easy Medium
containerd Production K8s nodes Manual High

The Cost of Docker Desktop

# Docker Desktop pricing (2026):
# - Individuals: Free
# - Small business (<250 employees, <$10M): Free
# - Medium business: $21/month/user
# - Large business: Commercial license required

# Alternatives:
# - Podman: Free
# - Lima: Free
# - Rancher Desktop: Free (macOS/Windows)
# - OrbStack: Free (macOS, faster than Lima)

The Bottom Line

Docker isn’t going away — it’s still the most compatible and well-documented option. But in 2026, you have real choices:

  • macOS users: Try OrbStack or Lima before Docker Desktop
  • Security-conscious teams: Podman is now production-ready
  • Kubernetes users: You already use containerd; consider using it directly
  • Everyone else: Docker still works fine

The days of “Docker is containers” are over. Containers are infrastructure, and infrastructure deserves thoughtful choices.

Using an alternative to Docker in 2026? What’s your setup?

TypeScript 5.5 — The Features That Actually Matter for Production Code

TypeScript 5.5: The Features That Actually Matter for Production Code

TypeScript 5.5 shipped with a mix of headline features and subtle improvements that compound significantly in large codebases. After running it in production for months, here are the features that actually moved the needle for our team.

The Big One: Inferred Type Predicates

This sounds small but it’s a massive DX improvement. TypeScript can now automatically infer type predicates from function implementations.

Before 5.5: The Verbose Manual Fix

// Before: TypeScript couldn't narrow the type
function isString(value: unknown): boolean {
  return typeof value === 'string';
}

const values: (string | number)[] = ['hello', 42, 'world', 100];

// You'd have to do this:
const strings = values.filter((v) => {
  if (isString(v)) {
    return v; // TypeScript still thought v could be number!
  }
  return false;
});

// Or this (more explicit but verbose):
const strings = values.filter((v): v is string => isString(v));

After 5.5: Automatic Inference

// Now TypeScript infers the type predicate automatically
function isString(value: unknown): boolean {
  return typeof value === 'string';
}

const values: (string | number)[] = ['hello', 42, 'world', 100];

// TypeScript now correctly narrows inside the filter callback
const strings = values.filter((v) => isString(v));
// strings is correctly typed as string[]

// Works with more complex predicates
function isNonNullable<T>(value: T): value is NonNullable<T> {
  return value !== null && value !== undefined;
}

const mixed: (string | null | undefined)[] = ['a', null, 'b', undefined, 'c'];
const defined = mixed.filter(isNonNullable);
// defined is string[]

Where This Compounds: Data Validation Chains

// Before 5.5 required explicit type predicates everywhere
interface User {
  id: string;
  email: string;
  role: 'admin' | 'user' | 'guest';
}

function isAdmin(user: User): boolean {
  return user.role === 'admin';
}

function isActive(user: User): boolean {
  return user.email.includes('@'); // simplified
}

const users: User[] = /* from API */;

// Old way: explicit type predicates
const admins = users.filter((u): u is User => isAdmin(u) && isActive(u));

// 5.5 way: inferred predicates chain naturally
const admins = users.filter((u) => isAdmin(u) && isActive(u));

Regular Expression Syntax Checking

TypeScript 5.5 adds type checking for regular expressions. This catches real bugs.

Caught at Compile Time

// This would have been a runtime error before
const emailRegex = new RegExp('[a-z+', 'i'); // Syntax error in regex!

// TypeScript 5.5: Error detected at compile time
// Error: Invalid regular expression: Range out of order in character class

// Another example
const dateRegex = new RegExp('(\d{4})-(\d{2})-(\d{2}', 'g'); // Missing closing paren
// Error: Invalid regular expression: Unterminated group

The Subtle Case It Catches

// Oops: accidentally escaped the wrong character
const phoneRegex = new RegExp('\d{3}[-*]\d{3}[-*]\d{4}');
// Valid! But what if you meant:

const zipRegex = new RegExp('d{5}'); // Missing backslash!
// TypeScript 5.5 catches this: 'd' should be '\d' in string
// Error: Invalid escape sequence in string literal

Object Types from Array.filter()

When you filter an array with a type predicate that narrows to a specific interface, TypeScript now properly infers the output type.

interface ApiResponse {
  status: 'success' | 'error';
  data?: object;
  error?: string;
}

const responses: ApiResponse[] = await fetchAll();

// Before 5.5: You had to manually type the filter result
const successful: ApiResponse[] = responses.filter((r) => r.status === 'success');
// This worked but was verbose for more complex scenarios

// 5.5 handles discriminated unions better
const successes = responses.filter((r) => r.status === 'success');
// successes is (ApiResponse & { status: 'success' })[] but crucially
// r.data is now properly typed as `object` (not `object | undefined`)

The import Attributes Fix

// Before 5.5: Weird edge cases with import assertions
import data from './data.json' assert { type: 'json' };

// 5.5: import attributes syntax (the new standard)
import data from './data.json' with { type: 'json' };

// This matters because `assert` is being deprecated in favor of `with`
// 5.5 ensures you're writing future-proof code

Performance: Faster Builds

TypeScript 5.5 brought meaningful build speed improvements. In our codebase (~800k lines, 300 packages):

TypeScript 5.4: 45.2s full build
TypeScript 5.5: 38.7s full build
Improvement: ~14%

With --incremental:
TypeScript 5.4: 12.1s incremental
TypeScript 5.5: 9.8s incremental
Improvement: ~19%

The speedup comes from:

  • More efficient type narrowing
  • Smarter stale file detection
  • Better caching of resolved imports

The new satisfies Behavior

// 5.5 refines what `satisfies` means for union types
type Color = 'red' | 'green' | 'blue';
type Theme = Record<string, Color | string>;

const theme = {
  primary: 'red',
  secondary: 'blue',
  custom: '#3498db', // This is valid (string, not Color)
} satisfies Theme;

// TypeScript now correctly narrows each property
const primary: Color = theme.primary; // 'red' (not Color | string)
const custom: string = theme.custom; // '#3498db' (not Color | string)

// Before 5.5, you sometimes had to cast even with satisfies

What Didn’t Change (And That’s Okay)

You Still Need Explicit Type Annotations for API Boundaries

// This still requires explicit types - TypeScript can't infer from runtime
async function fetchUser(id: string): Promise<User> {
  // You still need to tell TypeScript what User is
  const response = await fetch(`/api/users/${id}`);
  return response.json(); // still any, needs type assertion or schema validation
}

// Best practice: use a schema validator (zod, typebox)
import { z } from 'zod';

const UserSchema = z.object({
  id: z.string(),
  email: z.string().email(),
  role: z.enum(['admin', 'user', 'guest']),
});

const response = await fetch(`/api/users/${id}`);
const user = UserSchema.parse(await response.json()); // typed correctly

Complex Generics Still Need Help

// TypeScript still struggles with complex inference in some cases
function createReducer<S, A extends Action>(
  initialState: S,
  handlers: Record<string, (state: S, action: A) => S>
): Reducer<S, A> {
  return (state = initialState, action) => {
    const handler = handlers[action.type];
    return handler ? handler(state, action) : state;
  };
}

// The inference still requires explicit types in some scenarios
// This is fundamental to how TypeScript's type system works

Migration Guide: What to Change

Step 1: Update TypeScript

npm install [email protected] --save-dev
# or
pnpm add -D [email protected]

Step 2: Find Type Predicate Boilerplate to Remove

Search your codebase for value is patterns in filter callbacks:

// This pattern is now unnecessary in most cases:
array.filter((item): item is Type => predicate(item))

// Can become:
array.filter(predicate)

Step 3: Update import assertions to import attributes

// Replace:
import foo from './foo.json' assert { type: 'json' };

// With:
import foo from './foo.json' with { type: 'json' };

Step 4: Add RegExp Validation (Optional but Recommended)

// Add a lint rule to catch regex errors early
// ESLint rule: no-invalid-regexp

// Or add a custom rule in your CI:
const regex = new RegExp(pattern);
console.log('Regex valid:', regex.source);

The Bottom Line

TypeScript 5.5 isn’t a revolutionary release, but it’s a solid incremental improvement. The inferred type predicates alone save hours of boilerplate in large codebases. The performance improvements compound on every build.

The real value: TypeScript is getting better at reducing the gap between what you write and what TypeScript understands, without requiring increasingly complex type gymnastics.

Running TypeScript 5.5? What’s your experience been? Any surprises — good or bad?

Upgrade with confidence: npm install -D [email protected] && npx tsc --version

Database Migration Strategies That Actually Work in Production

Database Migration Strategies That Actually Work in Production

Database migrations are the thing that looks simple until you’re凌晨3点 trying to recover from a migration that locked your production table for 45 minutes. After running migrations on systems with billions of rows, here’s what I’ve learned about doing them safely.

The Fundamental Problem

Most migration tutorials show you this:

class AddIndexesToOrders < ActiveRecord::Migration[7.0]
  def change
    add_index :orders, :user_id
  end
end

This is fine in development. In production with 50 million orders, this will lock your table and bring down your application.

The Core Principle: Small Changes, Applied Incrementally

Every production-safe migration follows the same pattern: make the change in small, non-breaking steps that can be rolled back independently.

Expanding-Contract Pattern

Instead of one big migration, use three:

Migration 1: Add new column (nullable)

-- Migration 1: Safe - never locks
ALTER TABLE orders ADD COLUMN user_email VARCHAR(255);

-- Update application to write BOTH old and new columns
-- Deploy this first

Migration 2: Backfill data (in batches)

-- Safe batched backfill
DO $$
DECLARE
  batch_size INT := 10000;
  offset_val INT := 0;
  max_id INT;
  updated INT;
BEGIN
  SELECT MAX(id) INTO max_id FROM orders;

  LOOP
    UPDATE orders 
    SET user_email = (SELECT email FROM users WHERE users.id = orders.user_id)
    WHERE id IN (
      SELECT id FROM orders 
      WHERE user_email IS NULL 
      AND id <= max_id
      ORDER BY id 
      LIMIT batch_size
    );

    GET DIAGNOSTICS updated = ROW_COUNT;
    EXIT WHEN updated = 0;

    -- Prevent lock contention
    PERFORM pg_sleep(0.1);
  END LOOP;
END $$;

Migration 3: Add NOT NULL constraint

-- Now safe because all rows have values
ALTER TABLE orders ALTER COLUMN user_email SET NOT NULL;

Handling Long-Running Migrations

The Lock Timeout Strategy

-- Set a short lock timeout so migration fails fast instead of hanging
SET lock_timeout = '2s';

-- Migration that might need a lock
ALTER TABLE orders ADD COLUMN status VARCHAR(50);

-- If it can't get lock in 2s, it fails immediately
-- Instead of blocking for minutes

Concurrent Index Building

Never use CREATE INDEX in production. Always use CREATE INDEX CONCURRENTLY.

-- BAD: Locks table, blocks reads/writes
CREATE INDEX idx_orders_user_id ON orders(user_id);

-- GOOD: Runs without locking, takes longer but zero downtime
CREATE INDEX CONCURRENTLY idx_orders_user_id ON orders(user_id);

Critical note: CREATE INDEX CONCURRENTLY cannot run inside a transaction. Your migration framework needs to handle this.

# Rails: Tell it to run outside a transaction
class AddIndexesToOrders < ActiveRecord::Migration[7.0]
  disable_ddl_transaction!

  def change
    add_index :orders, :user_id, algorithm: :concurrently
  end
end

Schema Versioning: The Branching Model

For complex systems, treat database schema like code with proper branching:

main (production schema)
  └── staging-test (validate migrations)
        └── feature/user-email-migration (your change)
# Before starting a migration
git checkout main
git pull
git checkout -b migration/user-email-fix

# Run migrations locally against fresh production copy
# Once validated:
git checkout main
git merge migration/user-email-fix
# Deploy migration to production

The Pre-Migration Checklist

Before ANY production migration:

□ Tested on production-size dataset (at minimum on staging with production data snapshot)
□ Lock duration estimated (use EXPLAIN ANALYZE)
□ Rollback plan documented
□ Canary/deploy step prepared (migrate 1% of traffic, observe, then full deploy)
□ Alert thresholds set (if migration causes >X% error rate, auto-rollback)
□ Migrations scheduled during low-traffic window
□ On-call engineer aware and standing by
□ Database backup verified (point-in-time recovery tested)
□ Lock timeout set appropriately
□ Query plan examined for full table scans

Real-World Example: Renaming a Column Safely

Renaming a column is a four-migration process:

Migration 1: Add new column (double-write starts)

ALTER TABLE users ADD COLUMN display_name VARCHAR(100);
# Update application to WRITE to both columns
# User.where(name: 'John').update(display_name: 'John') 
# runs in background

Migration 2: Backfill

UPDATE users SET display_name = name WHERE display_name IS NULL;
-- In batches of 10,000 with 0.1s sleep

Migration 3: Stop reading from old column

# Deploy code that only reads from display_name
# Verify everything works

Migration 4: Drop old column

ALTER TABLE users DROP COLUMN name;
-- Must run outside transaction for PostgreSQL

PostgreSQL-Specific Tools

pg_repack: Remove Bloat Without Table Locks

# Install
CREATE EXTENSION pg_repack;

# Repack a bloated table without locking
pg_repack -d mydb -t orders --no-indexes

# Repack with specific index
pg_repack -d mydb -t orders -i idx_orders_user_id

pg_activity: Monitor Migration Progress

# Watch active queries during migration
pg_activity -h localhost -U postgres

# Or query directly
SELECT pid, state, query, query_start, now() - query_start AS duration
FROM pg_stat_activity
WHERE state != 'idle'
ORDER BY duration DESC;

The Migration Locking Hierarchy

Understanding lock modes prevents surprises:

Lock Mode Blocks
Access Share DROP TABLE, TRUNCATE
Row Share DELETE, UPDATE, SELECT FOR UPDATE
Row Exclusive INSERT, UPDATE, DELETE
Share Update Exclusive ANALYZE, CREATE INDEX CONCURRENTLY
Share CREATE INDEX (blocking)
Share Row Exclusive ALTER TABLE
Exclusive REFRESH MATERIALIZED VIEW CONCURRENTLY
Access Exclusive DROP TABLE, TRUNCATE, most ALTER TABLE

The Golden Rule

If your migration takes more than 100ms on a production table, it’s wrong. Go back and break it into smaller pieces.

The goal is always: zero downtime, zero data loss, instant rollback capability.

What migration horror stories do you have? What’s your go-to strategy for risky migrations?