Mert

The anatomy of a container tells you what the walls are made of. This article tells you when they go up, what happens inside them, and what the kernel does the moment they come down.

In the previous article, we dissected a running container at a single moment in time: the namespaces building the illusion of isolation, the cgroups enforcing the resource constitution, OverlayFS merging layers into a coherent filesystem, the veth pair wiring the container into the network. A cross-section. A photograph.

But a photograph doesn’t tell you how the subject got there, or what happens after the shutter closes.

That’s what this piece is for. We’ll follow a single container from before it exists to after it’s gone — and at each transition, we’ll look at what actually changes at the kernel level. The machinery is the same as before. Now we watch it move.

We’ll also cover the three places where production systems quietly break in ways that monitoring won’t catch until it’s too late: signal handling, zombie processes, and the OOM killer.

Before the Container: The Image as a Promise

Before anything runs, the runtime needs something to run from. An image isn’t a virtual machine disk or a binary blob — it’s a stack of read-only filesystem layers, each one a delta on the last, stored under /var/lib/docker/overlay2/ and identified by SHA256 digest rather than name.

When you run docker pull, the daemon fetches a manifest first: a JSON document listing every layer by digest. It then checks which of those digests already exist locally and downloads only the missing ones. This is why pulling a new version of an image that shares a base layer with one you already have takes seconds — the common layers are already there. The registry protocol is content-addressed, which means layers are inherently deduplicated across every image on the host that uses them.

Each downloaded layer is verified against its digest before being committed to the store. A tampered or corrupted layer fails this check before it touches anything.

The image at rest is inert. It’s a promise of what the container’s filesystem will look like when it starts — nothing more. The OverlayFS that merges those layers into a live, writable filesystem doesn’t exist yet. That comes with create.

Stage 1: docker create — Assembling the Environment

Here is something most engineers don’t know: docker run is not a single operation. It is docker create followed immediately by docker start. Knowing this distinction is practically useful — but more importantly, it clarifies when each piece of kernel infrastructure comes into existence.

docker create builds the complete environment the container will inhabit. Nothing executes. No process runs. It is pure allocation.

The OverlayFS mount. The runtime creates a new upperdir — an empty, writable directory unique to this container instance. The image’s read-only layers become the lowerdir. The OverlayFS is configured to present both as a single unified filesystem: reads fall through to the lower layers, writes land in the upper layer, and the container will see a seamless root filesystem when it starts. The layers you read about in the anatomy piece are present here. The writable layer that captures everything your container does is created here.

The namespace allocation. The six namespaces — PID, NET, MNT, IPC, UTS, USER — are prepared. At this point they’re reserved but not yet active. There’s nothing to isolate, so isolation hasn’t started.

The cgroup hierarchy. A cgroup is created for this container under /sys/fs/cgroup/. The memory limit from --memory 512m and the CPU quota from --cpus 0.5 are written into the cgroup’s control files right now. The cgroup exists. The limits are set. But with no processes inside it, it enforces nothing — a constitution with no citizens.

The network configuration. The veth pair is created: one end destined for the container’s network namespace, one end connecting to the docker0 bridge on the host. An IP is assigned. NAT rules are written to iptables for port mappings. The virtual cable exists, but nothing is plugged in on the container end yet.

After docker create, the container is a complete kernel data structure — every wall built, every rule written, every wire run — with no tenant. Its state is "created". Nothing has been charged to the CPU. Nothing has touched the writable layer.

The practical implication: you can pre-warm containers during off-peak hours and start them in milliseconds when demand arrives. The expensive work of building the environment is already done.

Stage 2: docker start — Crossing the Threshold

This is the moment the environment becomes inhabited. Three components work in sequence: dockerd, containerd, and containerd-shim.

dockerd instructs containerd to start the container. containerd forks a containerd-shim — the small, dedicated supervisor we covered in the daemon article — which then calls runc.

runc is where the kernel call happens. It takes the container’s prepared configuration and calls clone() with the namespace flags: CLONE_NEWPID, CLONE_NEWNET, CLONE_NEWNS, and the rest. The namespaces activate. The OverlayFS is mounted. The cgroup begins accounting. The veth pair connects. Then runc calls execve() with whatever you specified as ENTRYPOINT or CMD.

That process — your application — becomes PID 1 inside the container.

Once execve() returns, runc exits. It has done its job. The containerd-shim stays behind, holding the container’s stdin/stdout file descriptors and watching PID 1. If dockerd crashes, the shim keeps the container alive — which is the architecture we walked through in detail previously.

The moment docker start completes, everything the anatomy article described is live and active: the isolated process tree, the private network stack, the merged filesystem, the cgroup enforcing limits. The photograph is now a film.

And this is the first place things go quietly wrong. PID 1 carries obligations that your application almost certainly doesn’t know about. We’ll come back to that shortly.

Stage 3: Running — What the Commands Actually Do

A running container looks opaque from the outside. Under the hood, every docker command in this state is a thin wrapper over kernel operations you already understand.

docker exec is the most commonly misunderstood. When you run docker exec -it myapp-prod /bin/sh, Docker doesn’t create a new container. It calls setns() — a system call that lets a process join existing namespaces. Your shell enters the container’s PID and network namespaces and sees the same environment the application sees. Nothing new is created. You’re walking into an existing apartment, not building a new one.

docker logs reads from the file descriptors the containerd-shim has been holding since start. The shim inherited the container’s stdout and stderr pipes and has been buffering them since the first byte was written. No daemon magic — the shim is just a persistent pipe holder.

docker stats reads cgroup control files directly: /sys/fs/cgroup/memory.current for RSS, CPU accounting files for usage. The numbers you see in the terminal are the same numbers the kernel is maintaining to enforce your limits.

docker pause is the most elegant operation in this set. It uses the cgroup freezer subsystem to send SIGSTOP to every process in the container simultaneously — not one at a time, but atomically, via the cgroup. They freeze mid-instruction with no opportunity to catch the signal or react. docker unpause sends SIGCONT through the same path and they resume exactly where they stopped. In-flight operations, open sockets, memory contents — all preserved.

Stage 4: docker stop — The Two-Phase Shutdown

This is where most production problems originate, and where the PID 1 decision made at start time comes due.

docker stop is a two-phase protocol. Phase one: SIGTERM is sent to PID 1. The application has a grace period — ten seconds by default, adjustable with -t — to finish what it’s doing and exit cleanly. Close database connections. Drain in-flight requests. Flush logs. Phase two: if PID 1 is still alive when the timer expires, SIGKILL is sent. There is no negotiation with SIGKILL. The kernel terminates the process immediately and unconditionally.

The failure mode that bites teams most often: the shell wrapper.

Many containers are launched like this: CMD ["sh", "-c", "node server.js"]. The shell becomes PID 1. When Docker sends SIGTERM to PID 1, the shell receives it — and by default, shells do not forward signals to their children. Your Node.js process never sees SIGTERM. Ten seconds pass. SIGKILL arrives. The server dies mid-request, connections drop, and the exit code suggests a crash rather than a clean shutdown.

The fix is to make your application PID 1 directly. Use the exec form: CMD ["node", "server.js"]. Or, if you need a shell script for setup, end it with exec node server.js — the exec replaces the shell process rather than forking a child, so your application inherits PID 1.

Ten seconds is also almost always too short. An application with database connection pools to drain and requests to finish needs more. Set -t 30 as a floor. For services handling long-lived connections, -t 60 or higher is appropriate. The grace period is your only window for a clean shutdown — size it honestly.

After a successful stop, the container’s state becomes "exited". Crucially, the writable OverlayFS layer is preserved — every file the application created or modified is still there. The container can be restarted with docker start and will resume from exactly the filesystem state it stopped in.

The First Danger Zone: Zombie Processes

On a normal Linux host, PID 1 is the init system. One of init’s core responsibilities — one so fundamental it’s baked into the kernel’s design — is reaping dead child processes. When a process exits, it doesn’t fully disappear. Its kernel entry stays open, holding its exit code, until its parent calls wait() to collect that code. Until then, it sits in state Z: a zombie. Dead, but not yet buried.

Init’s job is to call wait() on every process that loses its parent, so the kernel slot gets freed. Without init doing this, zombie entries accumulate indefinitely.

In a container, your application is PID 1. And your application — a web server, an API, a worker — was written to serve requests, not to supervise processes. It almost certainly never calls wait().

The scenario: your web server forks a worker to handle a long request. The worker finishes and calls exit(). It waits for its parent to acknowledge the exit. The parent never does. The worker’s kernel entry sits in Z state. Do this at scale, over days of production traffic, and the PID table fills. Eventually fork() fails — the container cannot spawn new processes — and the application breaks in a way that looks nothing like its actual cause.

docker stats will show the container as healthy throughout. CPU and memory are fine. The zombie count isn’t surfaced by any standard metric.

The fix is a proper init process sitting in front of your application. tini is the canonical choice — purpose-built for containers, less than a thousand lines of C, does one thing: reap zombies and forward signals correctly.

RUN apt-get install -y tini
ENTRYPOINT ["/tini", "--", "/usr/local/bin/myapp"]

If you don’t want to touch the image, docker run --init injects a bundled init binary automatically. Use one or the other on every production container, as a default, without exception.

To check for zombies in a running container:

docker exec myapp-prod ps aux | grep ' Z '

Any output means you have a problem. The fix is tini. The time to add it is before the deployment, not during the incident.

The Second Danger Zone: The OOM Killer

The cgroup memory limit — set at create time, written to memory.max in cgroup v2 — is not a suggestion. When a container’s resident memory usage reaches that ceiling, the kernel’s Out-Of-Memory killer activates.

The OOM killer scores every process in the cgroup by a “badness” calculation: how much memory the process uses relative to total system memory, adjusted by that process’s oom_score_adj value. The highest scorer is sent SIGKILL. In a single-process container, there is exactly one candidate. PID 1 always loses.

The critical distinction from docker stop: the OOM killer sends no SIGTERM. There is no grace period. No warning, no chance to flush state, close connections, or write a final log line. The process simply stops existing. Docker records this as OOMKilled: true in the container’s inspect output:

docker inspect myapp-prod --format '{{.State.OOMKilled}}'
true

The kernel also logs it to dmesg:

dmesg | grep -i "oom|killed process"

Mitigation works at two levels. The first is setting a soft limit below the hard limit:

docker run --memory 512m --memory-reservation 400m myapp

--memory-reservation tells the kernel to apply gentle memory pressure — nudging the container’s garbage collector, encouraging page reclamation — before the hard ceiling is reached. It’s an early warning system rather than a wall. The container can still exceed the reservation temporarily; only --memory is the hard stop.

The second level is treating OOM kills as bugs, not tuning parameters. A container that is repeatedly OOM-killed has either a memory leak or limits set below its actual working set. Raising the limit is a short-term fix. Finding the leak is the real work.

Stage 5: docker rm — Dismantling the Environment

docker rm is the exact inverse of docker create. Every kernel structure that create allocated is released in reverse.

The OverlayFS upper layer is deleted. Every file your application created, modified, or deleted since the container started — the entire writable history — is gone. If your application wrote logs or state directly to the container filesystem, it’s unrecoverable. This is the reason containerised applications should write persistent data to mounted volumes, not the container filesystem: docker rm treats the writable layer as ephemeral by design.

The veth pair is removed. The IP address returns to Docker’s pool. The iptables NAT rules for port mappings are deleted. The network configuration from create is fully unwound.

The cgroup hierarchy is destroyed. The entries under /sys/fs/cgroup/ that tracked this container’s memory and CPU are removed. The kernel stops accounting for this container entirely.

What survives: the image layers. The read-only lowerdir that formed the container’s base filesystem is untouched — available immediately for the next container. And named volumes survive unless you pass --volumes to docker rm. Persistent data stored in volumes outlives the container that created it, which is the intended design.

The Full Picture

The anatomy article showed you the ingredients. This one showed you the recipe — the same namespaces, cgroups, OverlayFS, and veth pairs assembled in sequence, operated during a container’s life, and dismantled at its end.

The mental model to carry forward:

Create is allocation, start is execution. Every kernel structure — namespaces, cgroups, OverlayFS, veth pairs — exists before your application runs a single instruction. This is why pre-warming works and why restart is fast.

PID 1 is a contract, not just a position. The kernel gives PID 1 responsibilities that init normally handles: signal forwarding and zombie reaping. If your application holds that position, it inherits those responsibilities. Use tini or --init by default.

SIGTERM is your only warning. The grace period between SIGTERM and SIGKILL is the only window you get for a clean shutdown. Your application must handle the signal, and the window must be sized for your actual workload. Ten seconds is almost always too short.

The OOM killer gives no warning at all. There is no SIGTERM, no grace period, no log line from your application. If OOMKilled: true appears in production, treat it as a bug — because it is one.

The writable layer is temporary. docker rm destroys it completely. Write persistent state to volumes. Treat the container filesystem as scratch space that disappears when the container does.

docker run looks like a single command. It is actually the sequential execution of eight distinct kernel operations, each with observable state and specific failure modes. The engineers who debug containers fastest aren’t the ones who know the most flags — they’re the ones who can trace a problem back to the operation where the kernel did something unexpected.

Now you can.

Part of an ongoing series on container internals. Previous: A Deep Dive into Docker: What Ticks Under the Hood?

The problem

When a port is already in use, developers have to chain together lsof, grep, and kill commands to find and stop the offending process — there is no interactive way to scan all listening ports and act on them in one place.

If you’ve hit EADDRINUSE :::3000 before, you know the ritual: lsof -i :3000, then grep, then kill. Every time.

As a solution, I created port-peek

Interactive TUI for browsing all listening ports and killing the process behind any port with a keypress

Zero dependencies. Run it immediately:

npx port-peek

Output:

port-peek v1.0.0 — Listening ports

  PORT    PID       PROCESS
  3000    91234     node
▶ 5432    823       postgres
  8080    45612     python3
  8888    45990     node

  ↑↓ navigate  Enter kill  r refresh  q quit

Arrow keys to navigate, Enter to kill, r to refresh.

How it works

Pure Node.js using child_process.execSync to run lsof -iTCP -sTCP:LISTEN on macOS (with ss/netstat fallback on Linux), then renders a full-screen ANSI TUI with raw mode stdin for arrow-key navigation and Enter-to-kill. No dependencies.

Why I built it

Found recurring complaints on r/node and r/webdev about the EADDRINUSE error forcing multi-step shell rituals to find and kill the blocking process. The top Stack Overflow answer for ‘kill process on port’ has millions of views, confirming this is an extremely common pain point. No popular zero-dep npm package provides an interactive alternative — they are all one-shot kill commands that require you to know the port number upfront. An interactive TUI felt like the right format because developers often want to survey all listening ports before deciding which to kill.

Try it

npx port-peek --help

Part of µ micro — one new developer tool, shipped every day.

Se você trabalha remoto, com certeza já passou por isso: você está em uma reunião super importante, começa a digitar suas notas ou responder um chat, e alguém solta o clássico: “Ewerton, dá uma segurada aí que o seu teclado está parecendo uma metralhadora!”.

O barulho de cliques mecânicos ou digitação mais agressiva é o terror das chamadas de vídeo. Embora ferramentas como Google Meet e Zoom tenham seus próprios filtros, eles nem sempre dão conta ou acabam pesando demais no navegador.

Se você usa Linux (como o Ubuntu), existe uma solução open-source, ultra leve e que roda direto na sua máquina (sem mandar seu áudio para nuvens terceiras): o NoiseTorch.

Neste post, vou te mostrar o passo a passo definitivo para instalar, corrigir os bugs clássicos de ícone no Ubuntu e colocar ele para rodar na sua próxima reunião.

O que é o NoiseTorch?
O NoiseTorch cria um microfone virtual no seu sistema. Ele captura o áudio do seu microfone físico, passa esse som por uma biblioteca de inteligência artificial leve (chamada RNNoise) que filtra o que é voz e o que é ruído (teclado, ventilador, cachorro latindo), e entrega um áudio totalmente limpo para o Zoom, Meet, Slack ou Teams.

Passo a Passo da Instalação no Ubuntu
Como o NoiseTorch não está na loja padrão do Ubuntu, nós vamos fazer a instalação pelo terminal. Preparado? Abre o terminal (Ctrl + Alt + T) e vamos nessa:

Passo 1: Instalar as dependências básicas
Primeiro, precisamos garantir que o sistema tem as ferramentas necessárias para baixar e descompactar os arquivos:

sudo apt update && sudo apt install -y wget tar gtk-2.0

Passo 2: Baixar a versão estável e correta
Para evitar links quebrados ou arquivos HTML corrompidos que dão erro no terminal, baixe diretamente a versão estável (v0.12.2) direto para a sua pasta de Downloads:

cd ~/Downloads && wget https://github.com/noisetorch/NoiseTorch/releases/download/v0.12.2/NoiseTorch_x64_v0.12.2.tgz

Passo 3: Extrair para os diretórios locais
Os pacotes do NoiseTorch já vêm organizados na estrutura de pastas que o Linux usa. Vamos descompactar tudo direto na pasta local do seu usuário (~/.local):

tar -C $HOME -h -xzf NoiseTorch_x64_v0.12.2.tgz

(Se quiser limpar a casa, já pode apagar o instalador rodando rm NoiseTorch_x64_v0.12.2.tgz).

Passo 4: Dar permissões de sistema (Essencial!)
O NoiseTorch precisa gerenciar os fluxos de áudio do seu PulseAudio ou PipeWire sem precisar rodar como administrador (root). Para dar essa permissão especial a ele, rode:

sudo setcap 'CAP_SYS_RESOURCE=+ep' ~/.local/bin/noisetorch

Corrigindo o Bug do “Ícone de Engrenagem” no Ubuntu
Se você tentar rodar o NoiseTorch agora, o Ubuntu pode exibir apenas uma engrenagem genérica no dock e não vai deixar você fixar o aplicativo nos favoritos.

Para resolver isso e deixar o atalho com o ícone roxo oficial bonitinho, rode este bloco de comandos para atualizar o cache do GNOME:

# Corrige o caminho do executável dentro do atalho desktop
sed -i "s|Exec=noisetorch|Exec=$HOME/.local/bin/noisetorch|g" ~/.local/share/applications/noisetorch.desktop

# Garante que o ícone do app vá para a pasta correta de renderização
mkdir -p ~/.local/share/icons/hicolor/apps/
cp ~/.local/share/icons/hicolor/scalable/apps/noisetorch.svg ~/.local/share/icons/hicolor/apps/ 2>/dev/null || true

# Força o Ubuntu a ler as modificações de aplicativos
update-desktop-database ~/.local/share/applications
gtk-update-icon-cache -f ~/.local/share/icons/hicolor

Como Usar na Prática
Abra o menu de aplicativos do Ubuntu (tecla Super / Windows) e busque por NoiseTorch.

Abra o app, selecione o seu Microfone Real na lista que aparecer na tela e clique no botão grande “Load NoiseTorch”.

Agora, quando você entrar no Google Meet, Zoom ou Slack, vá nas configurações de áudio da chamada e mude o microfone para:

NoiseTorch Virtual Microphone

Como testar se funcionou?
Quer ter certeza de que o teclado sumiu antes de entrar na sala com o cliente ou com o time? Você pode gravar um áudio local de 10 segundos para testar. Rode no terminal:

arecord -D pulse -f cd -d 10 teste_ruido.wav

Nos primeiros 5 segundos: Fale normalmente enquanto digita no teclado.

Nos últimos 5 segundos: Fique em silêncio e apenas digite com força.

Para ouvir o resultado, execute:

aplay teste_ruido.wav

Se tudo deu certo, na metade final do áudio você vai ver que o NoiseTorch simplesmente dropou o som das suas teclas, deixando um silêncio absoluto. Salvação pura para o dia a dia do home office!

E você, já conhecia o NoiseTorch ou sofria configurando o áudio em cada app diferente? Deixa nos comentários se o tutorial funcionou para você!

This is a submission for the Google I/O Writing Challenge

Google I/O 2026 wrapped on May 20 at the Shoreline Amphitheater in Mountain View, and I’ve spent the last two days reading every blog post, watching every session replay almost but not all, and digging through the actual documentation. The headline everyone ran with was “Gemini Spark is your new 24/7 AI agent” and “Antigravity 2.0 built an operating system from scratch.” Those are flashy. They make good tweets.

But something much more interesting happened at this I/O, something that actually changes how Android developers will write code starting after the keynote address, and most of the coverage missed it entirely. I want to talk about that. I also want to walk through Antigravity 2.0 in detail, because Google fundamentally rebuilt it, and the technical decisions they made reveal a lot about where developer tooling is heading. Finally, I want to look at the bigger picture of what Google announced and what it means for the people who actually have to ship apps.

There’s a lot to cover. I’ll try to keep it useful.

The big picture: Agents aren’t a demo anymore

Before diving into the specific tools, let’s establish what actually launched.

Gemini 3.5 Flash is the new default model powering most of this. It outperforms last year’s Gemini 3.1 Pro on coding and agentic benchmarks: 76.2% on Terminal-Bench 2.1, 83.6% on MCP Atlas, 1656 Elo on GDPval-AA, and 84.2% on CharXiv Reasoning for multimodal understanding.That last number matters because it means the same model handles code and visual output, which is how agents can now render Compose previews and understand what they’re looking at. It runs about four times faster than other frontier models, and the pricing is $1.50 per million input tokens and $9.00 per million output tokens, with cached input at $0.15 per million.

Gemini Spark is Google’s 24/7 personal AI agent. It runs on Google Cloud infrastructure so it doesn’t need your device to stay on. It connects to Gmail, Docs, Calendar, and can interact with the web through Chrome and third-party services.You feed it context (wedding plans, a research project, whatever) and it keeps working. It’s exclusive to Google AI Pro and Ultra subscribers, rolling out in beta in the U.S. first.

Antigravity 2.0 graduated from an AI coding tool into a full agent orchestration platform. Standalone desktop app. No IDE. CLI. SDK. Multiple agents running in parallel. I’ll break this down in detail below.

Android CLI 1.0 reached stable. This is the one I keep coming back to. More on this in a moment.

Android XR smart glasses made with Samsung. Gemini Omni for video generation. Native Android app building directly in Google AI Studio. A new intelligent Search box that generates custom UIs in real time. There’s a lot.

I genuinely don’t know how to feel about all of it. The technology is real. The demos are real. A personal agent that monitors your inbox and handles calendar gymnastics while you sleep? That’s genuinely useful. An AI that can port an iOS app to native Android in hours instead of weeks? That solves a real problem for small teams. But there’s something unsettling about agents churning through your email at 3am while nobody’s watching, and I don’t think that tension goes away just because the demos are impressive.

For developers specifically, the message was unmistakable: Google doesn’t care which AI agent you use anymore. They shipped Android CLI 1.0, which gives Claude Code, OpenAI Codex, and Antigravity programmatic access to Android Studio’s toolchain from the command line.That’s a remarkable concession from a company that spent years building walled gardens. They looked at how developers actually work (switching between Claude Code, Codex, and Gemini depending on the task) and decided to make Android tooling available to all of them instead of forcing everyone into their own stack.

Pichai said the industry is in a period of “hyper progress” but also “where people want to see real value in the products they use on a day-to-day basis.” That’s the right thing to say. Whether the products deliver is a separate question.

Antigravity 2.0: What actually changed under the hood

Antigravity 2.0 is not an IDE update. Google ripped the agent manager out of the original Antigravity IDE and built it into a standalone desktop application available on macOS, Linux, and Windows.The IDE itself still exists as a separate product, but 2.0 is a completely different thing. It’s a desktop app where the primary interaction model is conversation with an agent, not a code editor with AI features bolted on.

The interface is built around conversations and artifacts. You describe what you want. The agent works on it. You review the artifacts it produces (code, documents, screenshots, browser operation recordings) and provide feedback directly on those artifacts to guide toward your desired outcome.

This is a philosophical shift. In the old model, you wrote code and the AI helped. In the new model, the AI does the work and you supervise. Whether that feels like progress or like being demoted to code reviewer depends on your disposition. I’m still figuring out where I land.

Dynamic subagents

The most technically interesting feature in 2.0 is dynamic subagents. The main agent can define and invoke child agents during execution to handle focused subtasks.These subagents are spawned dynamically, run in parallel, and get destroyed when their work is done.

Why this matters: every LLM has a finite context window. If you ask an agent to build a complex system, the intermediate steps (reading files, making edits, running tests) fill up that window fast. The agent starts forgetting what it was doing, or starts making decisions based on stale context. Dynamic subagents solve this by isolating subtasks into separate context windows that don’t pollute the main agent’s working memory.

Google’s launch demo used 93 parallel subagents to build a functional operating system from scratch. The primary agent acted like a CTO: it understood the system architecture, broke the goal into domains, and spawned specialized subagents (one for the kernel, one for memory management, one for the filesystem, one for video drivers, one for keyboard drivers). Those subagents worked independently and returned their results to the main agent for integration.

The numbers: 15,314 model requests, 339 million input tokens (with cache reads, output, and thinking tokens, that goes to over 2.6 billion), 12 hours of wall-clock time, and under $1,000 in API costs using Gemini 3.5 Flash.The OS could run FreeDoom, which is a real open-source Doom implementation.

The OS was not production quality. It had no floating-point math support, no hardware acceleration, no complex multi-threading, no sandboxing, no JIT compilation.But it was built from a single high-level prompt with zero human guidance. That’s the part that sticks with me.

Asynchronous task management and scheduled tasks

Tasks in Antigravity 2.0 can run asynchronously. You can fire off a long-running task and continue having other conversations with the agent while it works in the background.The results eventually surface in the main conversation when the task completes.

Scheduled Tasks are cron-style invocations. You define a schedule and the agent triggers automatically at the specified time.The /schedule slash command lets you write things like “every morning at 9am, summarize my inbox” directly in natural language.This moves the agent from a tool you call to something closer to a persistent automation pipeline.

JSON hooks

Hooks let you intercept and control agent behavior at key execution points. You define them in a JSON configuration file. When the agent hits a trigger event, the hook fires and can allow, deny, or ask the user before proceeding.The hooks system supports deny, allow, ask_user, and enforce rules per tool, which gives you fine-grained control over what the agent can and cannot do without reading the underlying code.

Projects, not repositories

In Antigravity 1.0, agent conversations were grouped by workspace, which meant one repository. In 2.0, they’re grouped by “project,” which can span multiple folders and enforce its own agent settings and permissions.This matters for real-world development where a single product often spans multiple repositories or codebases.

Slash commands

The four new slash commands are worth knowing:

/goal tells the agent to keep running until the specified task is completely finished, no intermediate input from you.This is for fire-and-forget work where you trust the agent enough to not need check-ins.

/grill-me flips the interaction: before starting to implement, the agent asks you clarifying questions to align on the specific details of the plan.I’ve found this genuinely useful for avoiding the “agent confidently builds the wrong thing” problem.

/schedule creates a one-time timer or recurring schedule for agent invocation.

/browser explicitly tells the agent to use browser primitives. Google found that agents were still not capable enough to determine on their own when to use the browser, so they made it an explicit command for now.

Voice input

The mic icon next to text input boxes now does live transcription instead of collecting raw audio files to pass to the model.It’s a small feature, but it changes the interaction dynamic. You can think out loud while the agent works, which feels more natural than typing everything.

The SDK

The Antigravity SDK gives you programmatic access to the same agent harness that powers Antigravity 2.0 and the Antigravity CLI. Your agent inherits a rich built-in toolset, a declarative safety-policy engine, lifecycle hooks for observing and steering every tool call, and stateful multi-turn sessions that persist across interactions.You can define custom agent behaviors and host them on your own infrastructure.

The SDK supports MCP integration for connecting to external Model Context Protocol servers, and the policy system lets you define permission rules at the tool level.

Pricing and availability

Antigravity 2.0 is free for all users. The new Google AI Ultra plan starts at $100/month with a 5x higher usage limit than the Pro plan, and there’s a limited-time $100 bonus credit for new and existing subscribers who hit their quota limit.The top-tier Ultra plan was cut from $250 to $200/month (20x the Pro limit).

Gemini Spark is gated behind Pro and Ultra subscriptions. Android CLI 1.0 requires Android Studio Quail 2 Canary 1 or later for the android studio commands.

Android CLI 1.0: The announcement that actually changes things

Everyone’s talking about Antigravity 2.0 building an operating system. That’s a great demo. But the announcement that actually matters for working Android developers is Android CLI 1.0 reaching stable. Here’s why.

For years, if you wanted to build Android apps with an AI coding agent, you had two options. Option one: use Google’s tools (Android Studio with Gemini). Option two: struggle. Third-party agents like Claude Code couldn’t access Android Studio’s static analysis engine, its refactoring tools, its Compose preview renderer, or its dependency management. They were flying blind. They generated code without being able to verify it against the actual Android toolchain.

Android CLI 1.0 fixes that. Through a new android studio command group, any AI agent can now perform semantic symbol resolution, analyze files for warnings, render Jetpack Compose previews, and execute end-to-end UI tests, all from the terminal without opening the IDE.

Google is positioning this explicitly as supporting whatever agent you want to use: Gemini in Android Studio, Antigravity, Claude Code, or Codex.That is a real shift in strategy.

The android studio command

The April preview release of Android CLI gave agents a clean binary for scaffolding projects, managing emulators, and running apps. Useful, but nothing that forced a workflow change. The stable 1.0 release adds the ‘android studio’ command group, which is the part that actually matters.

Here’s what agents can now do from the terminal:

android studio find-declaration performs semantic symbol resolution. Your agent can ask “where is HotelDetailScreen defined?” and get the actual answer, not a text-match guess.

android studio find-usages finds every reference to a symbol across the entire project.

android studio render-compose-preview renders a Jetpack Compose preview to an image file from the terminal.This is the one that changes things. Your agent can literally see what your Composable looks like before suggesting changes, instead of pattern-matching on text and hoping the layout looks right.

android studio analyze-file runs Android Studio’s full static analysis engine on a file: lint warnings, R8 hints, deprecation alerts.

android studio version-lookup queries live repository data and returns the actual latest stable version of a dependency.This is quietly excellent because agents guessing at Compose or AGP versions and hallucinating dependency coordinates is a common failure mode. version-lookup eliminates that entire class of error.

These commands require Android Studio Quail 2 Canary 1 or later to be running alongside the CLI. If you’re on the stable Android Studio channel, the android studio commands do nothing.

Why the token numbers are real

Google’s internal experiments showed that Android CLI reduced LLM token usage by over 70% for project setup tasks, with agents completing work up to 3x faster.These numbers are not vague marketing.

The reduction comes from how Android CLI structures its output. Traditional agent workflows dump full text logs, entire file contents, and verbose SDK manager output at the model. Android CLI returns structured JSON and supports a --diff mode that sends only the elements that changed since the last snapshot.

Here’s a concrete example: android layout --diff. Instead of re-processing the entire UI hierarchy on every iteration, the agent sees only what changed. On a complex screen with dozens of composables, that’s the difference between 8,000 tokens and 400.Multiplied across a development session, the savings compound quickly and translate directly into lower API costs.

Agent Skills

Android CLI ships with a skills system: modular markdown instruction sets that auto-trigger when a prompt matches their metadata. No manual context-stuffing, no copying documentation into system prompts.

The stable release adds five skills not present in the preview: build for display glasses (Jetpack Compose Glimmer for Android XR glasses), implement AppFunctions (expose app workflows to Android’s AI agent layer), analyze Perfetto traces (root cause latency, memory, and jank issues), use Perfetto SQL (natural language to Perfetto SQL query translation), and set up testing strategy (unit, UI, and screenshot test scaffolding).

There are 16 skills total available. The full library lives at github.com/android/skills and follows the agentskills.io open standard, which means community-contributed skills work in the same pipeline. Third-party repositories for Compose performance optimizations and Jetpack-specific workflows already exist.

Adding a skill for Claude Code is one command:

android skills add --agent=claude-code --skill=migrate-xml-views-to-jetpack-compose

The skills cover XML-to-Compose migration, AGP 9 upgrade, Navigation 3 setup, edge-to-edge display modernization, and R8 configuration auditing, among others.

Journeys: agent-driven UI testing

Stable 1.0 also ships Journeys support, which lets agents execute natural language user experience tests.Instead of writing Espresso test code, you describe a user flow in plain English and the agent navigates through it, reporting what it finds.

Installation

Android CLI is free. Installation varies by platform:

# macOS
brew install android-cli

# Linux and Windows
# Follow the setup guide at:
# https://developer.android.com/tools/agents/android-cli

If you already have it installed from the preview, update with android update.

Once installed, creating a new project and verifying it works:

android --version
android create project --name MyApp --package com.example.myapp

For Antigravity 2.0 users, the Android bundle (including Android CLI and skills) can be installed during onboarding or later from Settings > Customizations > Build With Google Plugins.

Getting started: what to try first

If you want to cut through the noise and try something that’ll actually make a difference this week, here’s my recommendation.

Option A: Install Android CLI and pair it with your existing agent.

This is the low-friction path. If you already use Claude Code or Codex, add Android CLI and watch your agent suddenly get smarter about Android development. The difference is immediate. Agents stop hallucinating Compose APIs. They stop guessing at dependency versions. They can actually check their own work.

After installing, try asking your agent to scaffold a new project, render a Compose preview, and check for warnings, all in one flow. That workflow used to require switching between terminal, IDE, and manual inspection. Now it’s programmatic.

Option B: Download Antigravity 2.0 and try the /grill-me workflow.

Go to antigravity.google/download and grab the installer for your platform. Start a new project. Don’t try to build an operating system. Ask it something concrete: “create a Jetpack Compose screen with a list of items, pull-to-refresh, and a detail view using Navigation 3.” Then use /grill-me to force the agent to ask clarifying questions before it starts coding. Review the artifacts it produces. Get a feel for the interaction model.

The voice transcription button (the mic icon) is surprisingly useful. Talking through what you want while watching the agent work feels more natural than typing prompts, and you catch things you’d miss in text.

8Option C: Both.*

Install Android CLI for your agent of choice. Download Antigravity 2.0. Spend an afternoon with both. They address different needs: Android CLI is infrastructure that makes any agent better at Android, while Antigravity 2.0 is an opinionated platform that wants to be your primary interface to agents. You might end up using both.

What to expect

Antigravity 2.0 feels different from traditional coding. You’re not writing code line by line with AI autocomplete. You’re describing goals and reviewing what comes back. It takes adjustment. I found myself over-specifying things at first (old habits), then gradually learning how much detail the agent actually needs. The /grill-me command helps with that by forcing the agent to ask questions instead of charging ahead with assumptions.

Android CLI is more straightforward. It’s a bridge. If you already use Claude Code or Codex for non-Android work, adding Android CLI means those same agents can suddenly speak Android Studio’s language. The productivity gains come from fewer context switches and fewer hallucinated APIs. There’s no learning curve; the agent just gets better at its job.

What I actually think

Here’s what gets me about this I/O.

The demos are spectacular. An operating system built from a single prompt. Agents that never sleep. Video generated from any input. These are genuinely impressive technical achievements, and I’m not going to pretend otherwise.

But spectacular demos have a short half-life. What lasts is whether the tools actually make developers’ lives easier three months from now, when the keynote glow has faded and you’re debugging a Compose layout at 11pm and wondering why your agent suggested a deprecated API from three versions ago.

Android CLI 1.0 will make developers’ lives easier. I’m confident about that one. It’s boring infrastructure, but boring infrastructure is what lets the exciting stuff actually ship. The 70% token reduction isn’t marketing. The --diff mode on layout inspection isn’t marketing. The version-lookup command that eliminates dependency hallucination isn’t marketing. These are concrete improvements to the development workflow that compound across hours and days of work.

Antigravity 2.0 is harder to evaluate. The subagent architecture is smart. The 93-subagent OS demo proves the orchestration layer works at scale. The scheduled tasks and JSON hooks are genuinely useful primitives. But the interaction model (describe a goal, wait for artifacts, review, repeat) is still unproven for day-to-day development work. Some developers will love it. Some will find it slower than just writing the code themselves. The truth is probably somewhere boring in the middle.

The agents? I’m still figuring out where I land. The technology is real. The demos are real. But there’s a gap between “look what it can do” and “look what it does reliably in production,” and that gap is where most developer skepticism lives. Google seems to know this. The Android CLI release, the openness to third-party agents, the emphasis on guardrails (JSON hooks, per-project permissions, /grill-me for alignment) are all signs they understand that adoption depends on trust, not just capability.

What I keep coming back to is the openness. Google spent years building walled gardens around Android development. Android Studio was the center of gravity and everything else orbited it. Android CLI 1.0 is the opposite: it makes Android Studio’s capabilities available to whatever agent you prefer, from Claude Code to Codex to your own custom setup. That’s not a concession born of weakness. It’s a strategic bet that if you make Android the easiest platform to develop for with any AI tool, more developers will build for Android regardless of which agent they use. It’s smart, and it’s more developer-friendly than anything Google has done with Android tooling in years.

Go install Android CLI and see if your agent of choice gets smarter about Android. That’s a bet I’m comfortable making today. Antigravity 2.0 is worth trying too, especially if you’re curious about what agent-first development actually feels like in practice.

We’ll see how it plays out. The tools are free. The documentation is live. The rest is up to us.

Note: The above cover image generated by AI

Mentoring juniors has taught me as much as it’s taught them. Here’s what actually helps—and what doesn’t.

1. Explain the “Why,” Not Just the “How”

“We do it this way because…” sticks better than “just do it like this.” When you share context (e.g. “we use this pattern so we can test without the API”), they start to generalize and make better decisions on their own. One sentence of why can save a lot of repeated corrections.

2. Code Review as Teaching

PR feedback is a chance to teach. Instead of only “change this,” add “because…” or “an alternative is X, which is better when Y.” Point them to a doc or example when you have one. Over time, they’ll anticipate the same points and need fewer comments.

3. Pair When It’s Sticky

When someone’s stuck for a while, pair for 30–60 minutes. Share screen, talk through the problem, and debug together. You unblock them and see where they get confused—useful for docs or future onboarding. Don’t take over; guide so they drive.

4. Give Them Ownership of a Small Area

Assign a small feature, module, or bug area they own. They get to design, implement, and maintain it. They’ll ask questions and make mistakes—that’s how they learn. You stay available for review and unblocking, but they lead.

5. Normalize “I Don’t Know”

Make it clear that nobody knows everything. Saying “I’m not sure, let’s look it up” or “good question, I’ll find out” models that learning is ongoing. That reduces the pressure to pretend they know and encourages asking questions.

6. Set Clear Expectations

Spell out what “good” looks like: “We expect PRs to have tests for new logic,” “we comment complex bits,” “we update the README when we change setup.” When expectations are clear, they can self-check and improve without guessing.

7. Don’t Fix Everything Yourself

If you always jump in and fix their code, they don’t learn. Let them fix their bugs (with hints if needed). Step in when it’s blocking the team or when they’re truly stuck, but default to “what have you tried? what do you think we should do next?”

8. Check In Regularly

Short, regular check-ins (“how’s the task? any blockers?”) beat long, rare meetings. They get help when they need it, and you notice when they’re stuck or when scope is unclear.

Mentoring well takes time, but it pays off: the team gets stronger, and you get a clearer picture of how your systems and docs work for someone new. The goal isn’t to create clones of you—it’s to help them think and ship with confidence.

Saad Mehmood — Portfolio

I spent the last few weeks building a production boilerplate for AI Agent and IoT systems. FastAPI, asyncpg, LangGraph, MQTT, pgvector — a complex stack with very specific architectural boundaries that cannot be violated.

The problem: I was using Claude Code and Cursor to accelerate development. And they are brilliant. But they are also completely agnostic to the architecture you have in your head.

Let me show you the kind of thing that happens without protection.

The Problem

My system has one critical rule: the IngestionService must never import SQLModel or SQLAlchemy. It’s the hot path for IoT telemetry ingestion — asyncpg raw SQL only, zero ORM overhead. This separation is intentional and documented in 20 pages of ARCHITECTURE.md.

In a typical session, I asked Claude Code to “refactor the IngestionService to be more consistent with the rest of the codebase.”

Generated result:

# services/ingestion.py — generated by Claude Code
from sqlmodel import Session  # ← CRITICAL VIOLATION
from core.database import get_session

class IngestionService:
    async def ingest(self, payload, trace_id):
        async with get_session() as session:  # ← destroys hot path
            session.add(SensorReading(**payload.dict()))
            await session.commit()

Technically correct. Architecturally catastrophic. Write latency jumps from 0.8ms to 4–6ms. At 5,000 messages per second, that’s the difference between a system that holds under load and one that collapses.

Claude Code doesn’t know this. It can’t. The architecture lives in my head and in a text document it may or may not have read before generating the code.

The Solution: Architecture Fitness Tests

The idea isn’t new — Martin Fowler writes about “fitness functions” in Building Evolutionary Architectures. But the application to AI-assisted development is very concrete: if the model is going to have full refactoring permission, you need tests that fail immediately when an architectural boundary is crossed.

Not runtime tests. Static structure tests — AST analysis of Python source, zero external dependencies, zero running server needed.

The full suite runs in 0.4 seconds. Pre-commit, not post-deploy.

A Concrete Example

The most important test in my system:

# tests/test_architecture_fitness.py

def test_ingestion_service_never_imports_sqlmodel(self):
    """
    The IngestionService is the Hot Path. SQLModel (SQLAlchemy) must
    never appear here. This is the most critical boundary in the system.
    """
    if not INGESTION_SERVICE.exists():
        pytest.skip(f"IngestionService not yet created at {INGESTION_SERVICE}")

    violations = []
    forbidden = {"sqlmodel", "sqlalchemy", "SQLModel", "AsyncSession"}

    for imp in _get_imports(INGESTION_SERVICE):
        module = imp["module"]
        names = imp["names"]
        if any(module.startswith(f) for f in forbidden):
            violations.append(imp)
        if any(name in forbidden for name in names):
            violations.append(imp)

    assert not violations, _format_violation(
        violations,
        "IngestionService imported SQLModel/SQLAlchemy.n"
        "  Fix: Use asyncpg raw SQL only in services/ingestion.py.n"
        "  This is the Dual-Path contract. The hot path has zero ORM overhead."
    )

The _get_imports function uses Python’s stdlib ast module to parse the file without executing it:

def _get_imports(filepath: Path) -> list[dict]:
    tree = ast.parse(filepath.read_text(encoding="utf-8"))
    imports = []
    for node in ast.walk(tree):
        if isinstance(node, ast.ImportFrom):
            module = node.module or ""
            imports.append({
                "module": module,
                "names": [alias.name for alias in node.names],
                "line": node.lineno,
                "file": str(filepath),
            })
    return imports

Zero external imports. Zero server. Zero database. Just Python.

The 8 Boundaries I Test

My system has a 7-layer architectural contract. The tests cover the most common violations AI generates:

1. Layer Isolation — Import Guards

• MQTT workers never import LangGraph
• IngestionService never imports SQLModel
• Agents never write directly via ORM
• Routers never call IngestionService directly
  2. Hot Path Integrity
• IngestionService uses only raw SQL strings, never query builders
• Hot path tables never defined as SQLModel models
  3. Event Contracts
• Redis Stream publishes use the canonical DomainEvent envelope
• MQTT workers validate with Pydantic before publishing
  4. Async Enforcement
• No synchronous HTTP clients (requests) in routers
• No time.sleep() in IngestionService
• Worker handlers are async def
  5. Trace ID Contract
• IngestionService.ingest() accepts trace_id as a parameter
• All domain events include trace_id
  6. Redis Keyspace Convention
• No bare Redis keys without a namespace prefix
• checkpoint:{id}, stream:{name}, cache:{type}:{id}
  7. Migration Integrity
• Zero DDL statements in application code
  8. Full Dependency Topology
• Validates the complete forbidden import matrix in a single pass

The Result in Production

When I give Claude Code full permission to refactor any file, the cycle is:

Claude Code generates code
        ↓
pytest tests/test_architecture_fitness.py -v
        ↓ (0.4 seconds)
Red → Claude Code fixes
        ↓
Green → commit

The model can never violate architectural boundaries without me knowing immediately. Not because it’s adversarial — but because it optimises for local consistency, not global contracts.

After adding these tests, all architectural violations disappeared from code reviews. Claude Code started generating compliant code automatically because the CLAUDE.md context file explicitly describes what the tests verify.

CLAUDE.md — The Second Mechanism

The tests are enforcement. CLAUDE.md is prevention.

It’s a file at the repo root that Cursor and Claude Code read before generating code. It contains the contracts in explicit language with ✅ correct vs ❌ wrong examples:

## Hard Boundaries (Enforced by tests/test_architecture_fitness.py)

### 1. The Dual-Path Contract

services/ingestion.py → asyncpg ONLY
  ✅ await pool.acquire() → await conn.execute("INSERT INTO ...", ...)
  ❌ from sqlmodel import Session
  ❌ session.add() / session.commit()
  ❌ time.sleep() — use await asyncio.sleep()

The combination of failing tests + explicit documentation creates an environment where AI can refactor freely with structural guarantees.

Final Numbers

Full suite:          18 tests
Execution time:      0.42 seconds
Dependencies:        0 (stdlib Python + pytest only)
Violations caught
before adding tests: 12+
Violations after:    0

Conclusion

AI-assisted development is genuinely transformative for productivity. But it creates a new category of risk: silent architectural drift. The model optimises for what it sees, not for what the global architecture requires.

Architecture Fitness Tests are the answer. They’re not hard to write — Python’s ast module does all the heavy lifting. And the return is immediate: full freedom to use AI on refactoring tasks without anxiety about what might have been broken.

If you’re building a distributed system with AI-assisted development, these tests are the first thing you should write — not the last.

The complete boilerplate (FastAPI + asyncpg + LangGraph + MQTT + pgvector + Prometheus + Grafana) with the 18 tests, the CLAUDE.md, and a 20-section ARCHITECTURE.md is available at murinelo.gumroad.com/l/pdmfvr

ur working on a react app. ur fan kicks on. u assume chrome is the culprit, or slack, or the LSP. close tabs, kill apps, fan keeps spinning.

actual cause in 6 out of 10 projects i audit on macOS: vite’s hot module replacement (HMR) doing way more work than needed. default config keeps websocket connections alive, polls file changes, rebuilds bundles aggressively. on a multi-monitor M-series macbook this lands as a consistent fan-on state even when ur not typing.

confirm its HMR first

quit ur dev server. wait 30 seconds. does the fan calm down?

yes → its HMR. no → look elsewhere (chrome tabs, docker, slack helper).

one-line fix

vite respects HMR env. flip it off when u dont need live reload:

HMR=off pnpm dev

or in vite.config.js:

export default defineConfig({
  server: {
    hmr: process.env.HMR !== 'off',
  },
});

with HMR off, vite still serves files but stops the websocket + file-watcher loop. CPU drops 30-50% on my M2 air. fan stops in 60 seconds.

when to keep it on

active feature dev: yes, u want HMR. saves 5-10s per save.

code reviews, doc reading, screencast watching while dev server is technically running but ur not editing: HMR is pure overhead. run with HMR=off.

i added this as a package.json script:

{
  "scripts": {
    "dev": "vite",
    "dev:cold": "HMR=off vite"
  }
}

then pnpm dev:cold for any session where im not actively editing the frontend.

why this is silent

vites docs treat HMR as always-on. perf cost is documented in their FAQ but not the getting-started flow. most react tutorials install vite + dont mention the HMR escape hatch. result: devs ship apps assuming “vite is fast” without realizing their dev session keeps the fan on 8 hours a day.

net

across 4 projects, this default flip moved my macbook battery from 4h to 6h on a typical dev day. real number, not a guess.

took 30 seconds to add. pays back every session 🤷