The 15x Token Tax on Multi-Agent Coordination

Multi-agent systems that coordinate over HTTP serialize context into JSON, send it as request bodies, parse it on the other end, and feed it back into an LLM. Every message carries HTTP headers, content-type negotiations, and authentication tokens. Every coordination step re-transmits the accumulated context of the entire interaction because HTTP is stateless.

Measurements show this coordination overhead costs up to 15x more tokens than the actual task content.

A three-agent pipeline processing a document: Agent A sends the document + instructions to Agent B (5K tokens). Agent B sends everything + its analysis to Agent C (12K tokens). Agent C sends the full chain back with its contribution (20K tokens). The document was 3K tokens. The coordination overhead was 34K.

At scale, 15x token overhead versus 1x is the difference between cost-effective and prohibitively expensive.

REST polling is even worse. 98.5% of requests return nothing new. You pay for every empty round trip.

Pilot Protocol connections are stateful. They maintain sequence numbers, acknowledgment state, and flow control windows. The tunnel stays open. The encryption context persists. An agent sends a small delta — “anomaly count updated to 48” — instead of re-serializing the entire context. 19 bytes, encrypted, delivered.

WebSockets attempt to solve this but break at scale and still require public endpoints. Pilot’s persistent bidirectional tunnels work behind NAT, across cloud providers, with automatic reconnection.

The token tax is not just a cost problem. It is a latency problem (re-serializing context adds processing time), a reliability problem (larger payloads are more likely to fail mid-transmission), and a complexity problem (every agent must implement context management). A stateful transport layer eliminates all four simultaneously.

Read more: Why AI Agents Need Their Own Network Stack · Move Beyond REST: Persistent Connections for Agents · Replace Webhooks With Persistent Agent Tunnels

pilotprotocol.network · GitHub

İstanbul BEDAŞ elektrik kesintisi! 18-19 Mart İstanbul'da elektrik kesintisi ne zaman bitecek, elektrikler ne zaman gelecek?

BEDAŞ İstanbul elektrik kesintisi listesi! 18 Mart günü İstanbul’un bazı ilçelerinde elektrik kesintisi yaşanacak ve kesinti gün boyu sürecek. Peki, 18 Mart günü İstanbul’un hangi ilçelerinde elektrik kesintisi yaşanacak? İstanbul’da Esenyurt, Bağcılar, Küçükçekmece, Pendik, Sultangazi elektrik kesintisi merak ediliyor. İşte İstanbul elektrik kesintileri!

Tabii Spor CANLI nereden izlenir? Şampiyonlar Ligi Tabii Spor nasıl izlenir?

Tabii Spor canlı izle! UEFA organizasyonları kapsamında oynanan önemli karşılaşmalar, Tabii platformu üzerinden yayınlanan Tabii Spor kanalları aracılığıyla izleyiciyle buluşuyor. Tabii Spor canlı maç izle! Aynı saat diliminde birden fazla maçın oynandığı akşamlarda farklı Tabii Spor kanalları üzerinden yayın akışı bölünerek karşılaşmalar eş zamanlı olarak ekranlara taşınıyor. Tabii Spor canlı izle ücretsiz bedava maç izle! Detaylar…

What We Learned Building a macOS AI Agent in Swift (ScreenCaptureKit, Accessibility APIs, Async Pipelines)

We have been building Fazm for about six months – a macOS desktop agent that uses voice input to control your computer. Along the way we ran into a lot of Swift-specific challenges that we did not see documented anywhere. Here is what we learned.

ScreenCaptureKit for Real-Time Screen Capture

The first problem is capturing what is on the screen. We needed something fast enough for real-time use but lightweight enough to run continuously without killing battery life.

ScreenCaptureKit was the answer. Apple introduced it in macOS 12.3 as a modern replacement for the older CGWindowListCreateImage approach. The key advantages for an AI agent:

  • Per-window filtering. You can capture specific windows instead of the entire screen, which means less data to process and better privacy.
  • Hardware-accelerated. The capture pipeline runs on the GPU, so CPU overhead stays minimal even at high frame rates.
  • CMSampleBuffer output. You get raw pixel buffers that you can feed directly to vision models without image format conversion overhead.

The tricky part was getting the update frequency right. Too slow and the agent misses UI state changes. Too fast and you waste compute processing identical frames. We settled on adaptive capture – high frequency during active automation, low frequency during idle monitoring.

Accessibility APIs Beat Screenshots Every Time

Early on we tried the screenshot-and-OCR approach that most AI computer agents use. Take a picture of the screen, send it to a vision model, get back coordinates of where to click. It works in demos. It breaks in production.

The problems are fundamental:

  • Resolution dependence. A button that is 40px wide on a Retina display looks different at every scale factor.
  • Visual ambiguity. Two buttons that look identical to a vision model might do completely different things.
  • Fragility. Every UI update, theme change, or notification banner shifts pixel coordinates.

macOS accessibility APIs solve all of this. Instead of guessing what a UI element is based on its appearance, you query it directly by role, label, and position in the element hierarchy. A “Save” button is always AXButton with AXTitle: "Save", regardless of what it looks like.

The accessibility tree gives you:

  • Element roles (button, text field, menu item)
  • Labels and values
  • Hierarchical relationships (this button is inside this toolbar inside this window)
  • Actionable operations (press, set value, select)

This is the same interface screen readers use, which means it is designed to be stable across visual changes. When an app redesigns their UI, the accessibility tree usually stays the same. We wrote a detailed comparison of this approach versus screenshot-based vision in how AI agents see your screen.

Swift Concurrency for the Pipeline

A desktop AI agent has a natural pipeline: capture screen, process with LLM, execute actions, verify result. Each stage has different latency characteristics and failure modes.

Swift’s structured concurrency turned out to be a great fit:

  • AsyncStream for the capture pipeline – frames flow continuously from ScreenCaptureKit.
  • Task groups for parallel action execution – clicking a button and verifying the result can happen concurrently.
  • Actor isolation for state management – the agent’s understanding of the current screen state needs to be thread-safe.

The biggest lesson was to not fight the concurrency model. Early versions tried to use completion handlers and dispatch queues, and the code was a mess. Rewriting to async/await made the pipeline logic readable and the error handling straightforward.

Voice Input with WhisperKit

For voice control, we use WhisperKit running locally on Apple Silicon. The key decision was local vs cloud transcription. Cloud gives you better accuracy. Local gives you zero latency and complete privacy.

For a desktop agent, privacy wins. The agent already sees everything on your screen – sending audio to a cloud service on top of that was a non-starter. WhisperKit on an M1 gives us usable transcription speed with good-enough accuracy for command input.

The Architecture Summary

The full pipeline looks like:

  1. WhisperKit captures and transcribes voice input locally
  2. ScreenCaptureKit grabs the current screen state
  3. Accessibility APIs build a structured representation of the UI
  4. LLM (Claude or local model via Ollama) plans the actions
  5. Accessibility APIs execute the actions (click, type, select)
  6. ScreenCaptureKit verifies the result

Each step is an async operation in a structured concurrency pipeline. Failures at any stage cancel downstream work cleanly.

Fazm is open source on GitHub – MIT licensed, written entirely in Swift/SwiftUI.