Mert

You connect three MCP servers to Claude Code on Monday morning. Before you type a single character, the agent has already burned through tokens — sometimes more than 50,000 of them. Tool definitions, schemas, descriptions, and parameter docs get injected into the system prompt every turn. A developer who ran the math measured 55,000 tokens of overhead per session, just from MCP server registration.

That number changes how you should think about MCP. It is not “free middleware.” Every server you add competes with your code, your conversation history, and your model’s working memory for the same context window.

The 55K Token Tax No One Talks About

When an MCP server connects to Claude Code, Cursor, or any MCP-compatible client, the client serializes every tool the server exposes — name, description, JSON schema for parameters, and often verbose example text — into the model’s system prompt. This payload re-enters context on every single turn.

The dev.to author who measured this found that a stack with GitHub, Linear, Slack, Postgres, and a few smaller servers passed 55,000 tokens before any user message. Claude Sonnet 4.6 ships with a 200K context window. That means roughly 27% of your usable context is gone before you ask a question.

On Opus 4.7 with 1M context the percentage shrinks, but the cost does not. Cached input tokens still bill at a fraction of regular input — and on every turn that fraction adds up.

Three concrete impacts:

• API cost per turn rises. At cached input rates around $0.30 per million tokens, 55K tokens of overhead bills you roughly $0.02 per turn just to remind the model which tools exist. Across a 100-turn session that is $2 of pure tooling tax.
• Response latency increases. Larger prompts take longer to process. First-token latency on a 60K-token prompt is measurably worse than on a 5K prompt, even with caching.
• Agent reliability drops. Models given more tools than they need confuse parameters, pick wrong tools, and hallucinate arguments. The “lost in the middle” effect is real for tool selection.

If your agent feels slower or less accurate after you “installed a few useful MCP servers,” the cause is almost always context dilution. The model is spending its attention budget reading tool docs instead of your code.

Where the Tokens Actually Go

Run a token accounting on a typical MCP server registration and you find three categories of bloat:

Tool descriptions written for humans. Most MCP servers ship descriptions like “Search for issues using a query string. Supports filters for status, assignee, priority, and project. Returns up to 50 results sorted by relevance. Use this when the user asks about specific issues or wants to find issues matching criteria.” That single description is ~200 tokens. Multiply by 30+ tools and you have 6,000 tokens before any schema.

JSON schemas with verbose property docs. Each parameter gets a description field, often duplicated across tools. A Linear MCP server exposing 20 tools with 5 parameters each, where each parameter carries a 30-token description, runs 3,000 tokens before nesting and enum values.

Example text and usage hints. Some servers append example calls like search_issues(query=’bug’, status=’open’) to every tool. Helpful for the model, but it triples the cost per tool.

The measurement broke down a real stack: GitHub MCP at ~12K tokens, Linear at ~9K, Slack at ~7K, a custom Postgres server at ~14K, plus filesystem and memory servers around ~5K each. Total: ~55K before the first user prompt.

How to Cut the Bloat

You do not have to disconnect every server. You have to be deliberate about which tools enter context.

Audit what you actually use. Most developers connect a server, use three of its tools regularly, and forget the rest exist. Claude Code’s /mcp command lists every tool; check it against your last week of sessions. If you have not called a tool, it should not be loaded.

Use scoped server profiles. Claude Code and Cursor both support per-project MCP config. Your blog repo does not need the Postgres server. Your data pipeline repo does not need the Notion server. Configure per-project rather than globally.

Prefer deferred tool loading where supported. Some clients now support lazy tool registration where only tool names load up front and full schemas fetch on demand. If your client supports it, enable it — token cost can drop 60–80%.

Write tighter tool descriptions if you author MCP servers. A description should tell the model when to use the tool in 1-2 sentences. Schema doc fields should be short noun phrases, not tutorials.

Measure before and after. Use your API provider’s token usage logs. Anthropic’s console shows cached vs uncached input per request. Set a budget — say, 10K tokens of overhead max — and trim until you hit it.

The deeper lesson: MCP is an abstraction with a token-shaped cost that nobody puts on the invoice. You pay for it in latency, accuracy, and dollars per turn. Treat each MCP connection like a dependency in package.json — useful when needed, dead weight when not.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

If you’ve built AI features in Java recently, you know the drill: choose an LLM SDK, wire up HTTP clients, handle SSE parsing, build a session store, figure out RAG, repeat for every provider you want to support.

Daimon takes a different approach. It’s a Go sidecar that runs next to your app and exposes a unified HTTP API for LLM inference, vector memory, graph queries, and session management — all wired from a single YAML file. Your application only talks to localhost.

Today the Daimon Java SDK (io.github.sonicboom15:daimon-client) lands on Maven Central. Here’s what you can do with it.

Installation

Gradle (build.gradle):

dependencies {
    implementation 'io.github.sonicboom15:daimon-client:0.4.1'
}

Maven (pom.xml):

<dependency>
    <groupId>io.github.sonicboom15</groupId>
    <artifactId>daimon-client</artifactId>
    <version>0.4.1</version>
</dependency>

Requires Java 17+. The only transitive dependency is Gson.

Step 1: Configure and start the sidecar

Create config.yaml:

components:
  - name: assistant         # your name — "gpt", "local", whatever
    type: anthropic
    metadata:
      api_key: ${ANTHROPIC_API_KEY}
      default_model: claude-opus-4-7

Start the sidecar (grab the binary from the GitHub releases page):

daimon serve --config config.yaml
# Listening on :3500

Or run it with Go installed:

go install github.com/sonicboom15/daimon/cmd/daimon@latest
daimon serve --config config.yaml

Step 2: Chat

Client client = new Client();

String reply = client.chat("assistant", "What is the capital of France?");
System.out.println(reply); // Paris

Three lines. No HTTP wiring. No SSE parsing. No JSON boilerplate.

"assistant" is the component name you chose in config.yaml — nothing more. Want to switch from Anthropic to GPT-4o or Gemini? Change two lines in the YAML. The Java code stays exactly as it is.

Step 3: Streaming

For long responses where you want to display tokens as they arrive:

LLMClient llm = client.llm("assistant");

for (String fragment : llm.stream("Write a haiku about distributed systems")) {
    System.out.print(fragment);
    System.out.flush();
}

stream() returns a lazy Iterable<String> backed by the SSE connection — no threads, no callbacks.

Step 4: Sessions (stateful conversations)

Attach a session_id to link requests into a conversation. The sidecar keeps the history server-side.

LLMClient llm = client.llm("assistant");

ChatOptions session = ChatOptions.builder()
        .sessionId("user-42")
        .build();

llm.chat("My name is Alice.", session);

String reply = llm.chat("What's my name?", session);
System.out.println(reply); // Alice

// Clear when done
llm.clearSession("user-42");

Sessions default to in-memory storage. Add a session/redis or session/postgres component to make them persistent across restarts.

Step 5: Vector memory (RAG)

This is where things get interesting. Daimon can query a vector store on every request and inject the top results as context — automatically, before the LLM ever sees the message.

Update config.yaml:

components:
  - name: docs
    type: inmemory          # BM25, no external service needed

  - name: assistant
    type: anthropic
    metadata:
      api_key: ${ANTHROPIC_API_KEY}
    memory_store: docs      # inject top-5 docs before every chat call

Java side:

MemoryStoreClient mem = client.memory("docs");

// Index some facts
mem.upsert("The Eiffel Tower is 330 metres tall and located in Paris.", "eiffel", null);
mem.upsert("The Colosseum is 48 metres tall and located in Rome.", "colosseum", null);
mem.upsert("The Burj Khalifa is 828 metres tall and located in Dubai.", "burj", null);

// Ask the LLM — relevant docs are injected automatically
String reply = client.chat("assistant", "Which famous landmark is tallest?");
System.out.println(reply); // mentions Burj Khalifa

You can also query the store directly:

List<MemoryResult> results = mem.query("tall structures", 3);
for (MemoryResult r : results) {
    System.out.printf("[%.2f] %s%n", r.score(), r.content());
}

Swap type: inmemory for type: chroma, type: qdrant, type: pgvector, or type: redis without touching a single line of Java.

Step 6: Graph queries

Daimon also exposes graph stores through the same thin client.

Add to config.yaml:

  - name: kg
    type: neo4j
    metadata:
      bolt_url: bolt://localhost:7687
      password: secret

Java side:

GraphStoreClient graph = client.graph("kg");

graph.addNode("alice", List.of("Person"), Map.of("name", "Alice", "role", "engineer"));
graph.addNode("daimon", List.of("Project"), Map.of("name", "Daimon"));
graph.addEdge("alice", "daimon", "MAINTAINS", null);

List<Map<String, Object>> rows = graph.cypher(
        "MATCH (p:Person)-[:MAINTAINS]->(proj) RETURN p.name, proj.name",
        null
);

rows.forEach(row -> System.out.println(row.get("p.name") + " maintains " + row.get("proj.name")));
// Alice maintains Daimon

For a deeper look at graph + LLM pipelines, see my earlier article: Build a Medical Chart Coding Pipeline with Daimon, Claude, and Neo4j.

Putting it together: a self-populating knowledge assistant

Here’s a complete example that combines all three — LLM, memory, and graph — in under 50 lines:

import io.github.sonicboom15.daimon.*;
import java.util.List;
import java.util.Map;

public class KnowledgeAssistant {

    public static void main(String[] args) {
        Client client = new Client();

        // ── Populate memory store ───────────────────────────────
        MemoryStoreClient mem = client.memory("docs");
        mem.upsert("Java 17 introduced sealed classes and pattern matching for instanceof.", "java17", null);
        mem.upsert("Java 21 introduced virtual threads (Project Loom) and record patterns.", "java21", null);
        mem.upsert("Java 23 introduced structured concurrency as a preview feature.", "java23", null);

        // ── Populate graph store ────────────────────────────────
        GraphStoreClient graph = client.graph("kg");
        graph.addNode("java17", List.of("Release"), Map.of("version", "17", "year", "2021"));
        graph.addNode("java21", List.of("Release"), Map.of("version", "21", "year", "2023"));
        graph.addEdge("java17", "java21", "FOLLOWED_BY", null);

        // ── Ask a question — docs are injected automatically ────
        // (memory_store: docs is set on the assistant component in config.yaml)
        LLMClient llm = client.llm("assistant");
        String answer = llm.chat("What major features came in Java 21?");
        System.out.println("Answer: " + answer);

        // ── Direct graph query ───────────────────────────────────
        var timeline = graph.cypher(
                "MATCH (a:Release)-[:FOLLOWED_BY]->(b:Release) RETURN a.version, b.version ORDER BY a.year",
                null
        );
        System.out.println("Timeline: " + timeline);
    }
}

The LLM will mention virtual threads and record patterns because the sidecar queried the memory store with “What major features came in Java 21?” and prepended the matching document as context.

What providers does Daimon support?

All configured in YAML — no code changes when you swap:

Links

• Example repo: github.com/sonicboom15/daimon-java-example
• GitHub: github.com/sonicboom15/daimon
• Maven Central: io.github.sonicboom15:daimon-client
• Docs: sonicboom15.github.io/daimon
• Previous article: Build a Medical Chart Coding Pipeline with Daimon, Claude, and Neo4j

Feedback, issues, and PRs are welcome.

So far, you’ve seen structured principles like SOLID and concepts like cohesion and coupling.

But in day-to-day development, you don’t always think in formal principles.

Instead, you rely on simple rules that guide decisions quickly.

Three of the most important ones are:

• DRY (Don’t Repeat Yourself)
• KISS (Keep It Simple)
• YAGNI (You Aren’t Gonna Need It)

These look simple, but they prevent a large number of real-world design issues.

DRY — Don’t Repeat Yourself

Every piece of knowledge should have a single, unambiguous source.

What goes wrong without DRY

You implement pricing logic in multiple places:

• checkout flow
• order summary
• invoice generation

Initially, everything works.

Later:

• tax calculation changes
• discount rules update

Now you must update logic in multiple places.

Miss one, and your system behaves inconsistently.

Applying DRY

Centralize logic:

PriceService:
- calculate_price(order)

Now:

• one source of truth
• consistent behavior across the system

Important nuance

DRY is not just about avoiding duplicate code.

It’s about:

Avoiding duplicate logic and knowledge

Common mistake

Over-applying DRY too early:

• creating generic abstractions
• forcing unrelated logic into one place

This leads to:

• complexity
• harder readability

KISS — Keep It Simple

Prefer simple solutions over complex ones.

What goes wrong without KISS

You design a simple feature using:

• multiple design patterns
• deep abstractions
• unnecessary layers

The system becomes:

• harder to understand
• slower to modify

Applying KISS

Choose the simplest approach that solves the problem.

Sometimes:

• a simple class is enough
• a straightforward condition is fine

Complexity should be introduced only when required.

Important nuance

KISS does not mean:

• writing messy or unstructured code

It means:

Avoiding unnecessary complexity

YAGNI — You Aren’t Gonna Need It

Don’t build functionality until it is actually needed.

What goes wrong without YAGNI

While building an MVP, you add:

• multi-region support
• advanced recommendation system
• dynamic pricing engine

None of these are required immediately.

Result:

• longer development time
• more bugs
• increased complexity

Applying YAGNI

Focus on current requirements:

• build what is needed now
• leave extension points for future

Do not implement features based on assumptions.

Important nuance

YAGNI does not mean:

• ignoring future scalability

It means:

Don’t implement future features without real need

How these three work together

• DRY → keeps logic consistent
• KISS → keeps design simple
• YAGNI → keeps scope controlled

Together, they help you:

• avoid over-engineering
• write maintainable code
• move faster without creating chaos

A practical way to use them

When designing, ask:

• Am I repeating logic? → DRY
• Am I overcomplicating this? → KISS
• Do I really need this right now? → YAGNI

Closing thought

Good design is not just about what you build.

It’s also about what you choose not to build.

Most AI coding assistants ship one model doing everything: parse your prompt, reason about the codebase, draft the response, format the output. That model is a generalist by necessity. DeepClaude takes a different approach — it splits the job between two specialists and routes them through a single agent loop.

The pattern: DeepSeek R1 handles the reasoning step, emitting an explicit chain-of-thought trace. Claude reads that trace, then synthesizes the final code or explanation. R1 thinks; Claude writes. Both models stay in their lane.

How the dual-model loop works

When you send a prompt to a DeepClaude-style agent, it doesn’t go to one endpoint. The orchestration layer does three passes:

1. Reasoning pass (DeepSeek R1). R1 is a reasoning-tuned model from DeepSeek that exposes its thinking as a structured <think> block before producing an answer. The agent intercepts the trace and discards R1’s final answer — only the reasoning is kept.

2. Synthesis pass (Claude). The R1 thinking trace becomes part of Claude’s context window. Claude is prompted to produce the actual response — code, edits, explanations — while treating R1’s reasoning as a planning document.

3. Loop, if needed. For agentic tasks (run a test, read a file, retry), the loop bounces between tool calls and the two-model cycle until the goal is satisfied.

The point isn’t that R1 is smarter than Claude or vice versa. It’s that R1’s training pushes hard toward exhaustive step-by-step reasoning, while Claude’s instruction-following and code generation are tuned for output quality. Stack them and you get both, at the cost of an extra API hop and roughly doubled latency on the reasoning step.

DeepSeek-R1 is open-weight, and the hosted API costs less per token than Claude. The bulk of your inference cost in a DeepClaude setup ends up on the Claude synthesis call, not the reasoning trace — even though R1 typically emits more tokens.

When it beats single-model assistants

Cursor, GitHub Copilot, and Claude Code all use a single model per turn. They’re fast, integrated with your editor, and good enough for autocomplete or small edits. The single-model approach starts breaking down on tasks that need two distinct cognitive modes:

• Multi-file refactors where you need to reason about call sites before touching code.
• Debugging unfamiliar code where the reasoning step is “what does this even do” before any fix.
• Architectural decisions where the model needs to weigh tradeoffs explicitly rather than pattern-match to a typical answer.

On these tasks, a single model often skips the reasoning and jumps to a plausible-looking edit. DeepClaude forces the separation: the reasoning model has to produce a chain-of-thought, and the synthesis model has to act on it. You see the plan before you see the diff.

The tradeoff is real. For autocomplete-style work, where you want a suggestion in under 300ms, DeepClaude is the wrong tool — you’ll wait for two sequential API calls. For non-trivial agent tasks where you’d otherwise spend ten minutes prompting Claude back into the right context, the dual-model loop is faster end-to-end.

Setting it up via API

There’s no managed DeepClaude service — it’s an architectural pattern, not a product. The reference implementation in the open-source community is a thin proxy that wraps two SDKs: DeepSeek’s chat-completions API for R1 and Anthropic’s Messages API for Claude.

The minimum loop, in pseudocode:

# 1. Get the reasoning trace from R1
r1_response = deepseek.chat.completions.create(
    model="deepseek-reasoner",
    messages=[{"role": "user", "content": user_prompt}],
)
reasoning = extract_think_block(r1_response.choices[0].message.content)

# 2. Hand the reasoning to Claude for synthesis
claude_response = anthropic.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=4096,
    system="Use the reasoning trace below as your plan. Produce the final response.",
    messages=[
        {"role": "user", "content": user_prompt},
        {"role": "assistant", "content": f"<reasoning>{reasoning}</reasoning>"},
        {"role": "user", "content": "Now produce the final answer."},
    ],
)

Two practical notes:

• Stream both. The reasoning trace can be hundreds of tokens. Streaming R1’s output gives you a progress signal so the UI doesn’t sit dead for ten seconds. Streaming Claude’s synthesis hides the second hop from the user.
• Cache the reasoning. If the user iterates (“apply the same plan to file B”), reuse the R1 trace and only re-run Claude. You cut latency in half and cost by more.

DeepSeek’s hosted API is operated from China. If your codebase or prompts contain regulated data — health records, payments, regulated PII — read DeepSeek’s terms and your own compliance posture before piping prompts through. Self-hosting R1 on your own GPUs (the weights are open) is the conservative path for sensitive workloads.

The pattern generalizes. You can swap R1 for any reasoning-tuned model (o1, QwQ, future open-weight reasoners) and Claude for any synthesis-strong model. The architecture is what wins, not the specific models.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

Quick Comparison

Quick Summary

Loom is the fastest path from “I need to show you something” to “here’s the link.” Screen Studio is the tool you use when the recording itself matters — polished demos, onboarding videos, and anything that represents your product to an audience.

We tested both for three developer workflows: code walkthroughs (PR reviews and architecture explanations), bug reports (repro steps with annotations), and team updates (async standups and sprint demos).

Winner: Screen Studio — for solo developers and small teams who want recordings that look polished without editing. But if you need team libraries, viewer analytics, or instant cloud sharing, Loom is the better fit.

Round 1: Recording Quality and Polish

info

Screen Studio wins polish. Automatic zoom, smooth motion, and cursor effects turn a raw screen recording into something that looks edited — without touching a timeline.

Screen Studio’s killer feature is invisible until you watch the output. Record your screen, stop, and the app automatically applies cursor zoom, smooth panning between windows, click highlights, and a background blur. A 5-minute raw recording comes out looking like someone spent 30 minutes editing it. The keystroke overlay is especially useful for coding videos — your viewers see exactly which shortcuts you’re pressing.

Loom’s recording quality is fine for internal use but falls flat for anything public-facing. Your cursor is a static dot. Window switches are abrupt cuts. There’s no motion smoothing or zoom. It looks like a screen recording — which is fine when you’re showing a teammate where the bug is, but not when you’re recording a feature demo for your landing page.

Screen Studio 9/10, Loom 5/10 for recording polish.

Round 2: Sharing and Collaboration

info

Loom wins sharing. Instant cloud upload + link generation means your video is shareable in seconds. Screen Studio requires manual export and upload.

Loom’s sharing workflow is the reason it became ubiquitous: stop recording, and a link is on your clipboard. Paste it into Slack, a PR comment, or a Notion page. Viewers can react with emoji and leave timestamped comments. You get analytics on who watched and for how long. The team library organizes recordings into searchable folders.

Screen Studio has none of this. Your recording saves as a local .mov file. To share it, you export (which takes 30-60 seconds for a high-quality render), then upload to YouTube, Dropbox, or wherever you host videos. There’s no team library, no viewer tracking, no comment system. It’s a creation tool, not a communication tool. For quick async updates (“here’s the bug, here’s the fix”), that friction matters.

Loom 9/10, Screen Studio 4/10 for sharing and collaboration.

Round 3: Use Case Fit

info

Different tools for different jobs. Loom is for internal communication; Screen Studio is for external content.

For a quick bug report — “the checkout button is broken, here’s a 30-second recording” — Loom is objectively faster. Record, stop, paste link. The viewer can comment directly on the video. Your PM doesn’t need to download a 200MB file.

For a product demo, changelog video, or onboarding walkthrough — Screen Studio is the clear winner. The polished output makes your product look professional. Customers watch a Screen Studio demo and assume your product is well-built. A Loom recording with a flat cursor and abrupt cuts doesn’t inspire the same confidence. And the one-time pricing means you’re not paying $12.50/month forever for a tool you might use twice a week.

Many teams use both: Loom for internal async updates, Screen Studio for anything customer-facing. If you can only pick one, choose based on your primary audience.

Screen Studio 9/10 for external content, Loom 8/10 for internal comms.

The Verdict

Screen Studio wins overall (8/10 → 9/10) for developers who create content that represents their work to the world. The automatic polish is a genuine time-saver — what would take 30 minutes of editing in another tool happens automatically. But keep Loom bookmarked. When you need to fire off a 15-second bug report to a teammate, nothing beats the Loom → clipboard → Slack flow.

Bottom Line

• Pick Screen Studio if: You record demos, changelogs, onboarding videos, or anything customer-facing. The one-time price and automatic polish make it the best value in screen recording.
• Pick Loom if: Your primary use case is internal async communication — PR reviews, bug reports, standup updates — and you value instant sharing over production quality.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

The MediaPipe Tasks documentation covers installation, initialization, and a working demo. It does not cover what happens when you try to ship a production app. Six months of building CareSpace AI — a real-time physiotherapy app that runs pose detection at 30fps on every frame — filled in those gaps.

Here’s what the docs leave out.

AVCaptureSession setup that actually works

The docs show a generic camera setup. This is the production-tuned configuration that keeps latency low and prevents dropped frames:

class CameraManager: NSObject {
    private let captureSession = AVCaptureSession()
    private let videoOutput = AVCaptureVideoDataOutput()
    private let processingQueue = DispatchQueue(label: "com.app.camera", qos: .userInitiated)

    func configure() throws {
        captureSession.beginConfiguration()

        // VGA is the MediaPipe sweet spot — higher resolution adds latency without accuracy gains
        captureSession.sessionPreset = .vga640x480

        guard let device = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .front),
              let input = try? AVCaptureDeviceInput(device: device) else {
            throw CameraError.deviceUnavailable
        }
        captureSession.addInput(input)

        // Lock frame rate — variable frame rate causes timestamp jitter in MediaPipe
        try device.lockForConfiguration()
        device.activeVideoMinFrameDuration = CMTime(value: 1, timescale: 30)
        device.activeVideoMaxFrameDuration = CMTime(value: 1, timescale: 30)
        device.unlockForConfiguration()

        // BGRA is what MediaPipe expects — do not use MJPEG or YUV without conversion
        videoOutput.videoSettings = [
            kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA
        ]
        videoOutput.alwaysDiscardsLateVideoFrames = true
        videoOutput.setSampleBufferDelegate(self, queue: processingQueue)
        captureSession.addOutput(videoOutput)

        captureSession.commitConfiguration()
    }
}

Two things that aren’t obvious: frame rate locking and pixel format. Without frame rate locking, AVFoundation’s automatic frame rate control kicks in when the CPU is under load — and sends MediaPipe frames with inconsistent timestamps, which breaks temporal smoothing. Without kCVPixelFormatType_32BGRA, you’re doing an implicit conversion on every frame.

GPU vs CPU delegate — it’s not what you think

The docs say: “Use GPU delegate for better performance.” That’s true for benchmarks. In production, it’s more complicated.

// GPU delegate — fast, but causes thermal issues in long sessions
let gpuOptions = PoseLandmarkerOptions()
gpuOptions.baseOptions.delegate = .GPU

// CPU delegate — slower per-frame, but thermally stable
let cpuOptions = PoseLandmarkerOptions()
cpuOptions.baseOptions.delegate = .CPU

The GPU delegate runs 20–30% faster per frame. But on an iPhone 12, after about 8 minutes of continuous use with the GPU delegate, the device throttles and frame rate drops from 30fps to 18fps. With the CPU delegate, the same session runs at a stable 28fps for 20+ minutes.

For a physiotherapy app where sessions can run 15–30 minutes, we use:

• CPU delegate by default
• GPU delegate only if the device is iPhone 14 Pro or newer (better thermal headroom)

let delegate: Delegate = {
    if ProcessInfo.processInfo.thermalState == .nominal &&
       UIDevice.current.modelIdentifier >= "iPhone14,2" {
        return .GPU
    }
    return .CPU
}()

Coordinate space mapping — the front camera trap

MediaPipe returns landmark coordinates in normalized space (0,0) to (1,1), relative to the camera frame. This seems straightforward until you try to overlay landmarks on a live preview with a front-facing camera.

Two transforms are required:

func convertToViewCoordinates(
    landmark: NormalizedLandmark,
    in viewBounds: CGRect,
    isFrontCamera: Bool
) -> CGPoint {
    var x = CGFloat(landmark.x)
    var y = CGFloat(landmark.y)

    // Front camera: mirror X coordinate
    // MediaPipe does NOT mirror automatically for the front camera
    if isFrontCamera {
        x = 1.0 - x
    }

    // Portrait mode: swap X and Y, then apply bounds
    // Camera captures in landscape; device is in portrait
    let rotatedX = y
    let rotatedY = 1.0 - x

    return CGPoint(
        x: rotatedX * viewBounds.width,
        y: rotatedY * viewBounds.height
    )
}

If you don’t mirror the X coordinate for the front camera, the skeleton is laterally flipped. If you don’t handle the portrait/landscape rotation, the skeleton is transposed to the wrong axis. Both problems are invisible in unit tests because they require a rendered UI to see.

The retain cycle that leaks your session

MediaPipe’s live stream detection mode uses a callback closure (poseLandmarkerLiveStreamDelegate). If you capture self strongly in that closure, you create a retain cycle that leaks the entire detection session.

// WRONG — retain cycle
class PoseDetectionService {
    var landmarker: PoseLandmarker?

    func setup() {
        let options = PoseLandmarkerOptions()
        // This captures self strongly inside MediaPipe's internal storage
        options.poseLandmarkerLiveStreamDelegate = self
        landmarker = try? PoseLandmarker(options: options)
    }

    deinit {
        // Never called — self is retained by landmarker's delegate reference
        print("PoseDetectionService deallocated")
    }
}

// CORRECT — use a wrapper to break the cycle
class WeakDelegateWrapper: PoseLandmarkerLiveStreamDelegate {
    weak var delegate: PoseLandmarkerLiveStreamDelegate?

    func poseLandmarker(
        _ poseLandmarker: PoseLandmarker,
        didFinishDetection result: PoseLandmarkerResult?,
        timestampInMilliseconds: Int,
        error: Error?
    ) {
        delegate?.poseLandmarker(poseLandmarker, didFinishDetection: result,
            timestampInMilliseconds: timestampInMilliseconds, error: error)
    }
}

// In setup:
let wrapper = WeakDelegateWrapper()
wrapper.delegate = self
options.poseLandmarkerLiveStreamDelegate = wrapper

This pattern is standard for any delegate-based API where you don’t control the delegate storage. The docs don’t mention it because the retention behaviour depends on how MediaPipe stores the delegate internally.

Model selection: Lite vs Full vs Heavy

Three models, three tradeoffs:

For production:

• Lite: Finger counting demos, casual fitness apps, situations where you only need rough body position
• Full: Most physiotherapy and fitness use cases — good accuracy at 30fps on iPhone 12+
• Heavy: Situations where visibility scores drive logic (like the gating in our noise-handling pipeline) — the scores are significantly more reliable

We ship Full by default and let users switch to Heavy in settings if they want higher accuracy at the cost of ~5fps.

Testing in the simulator

MediaPipe doesn’t work in the simulator — the GPU delegate isn’t available, and the camera isn’t present. Inject a video file instead:

class SimulatorVideoSource: NSObject, AVCaptureVideoDataOutputSampleBufferDelegate {
    private var displayLink: CADisplayLink?
    private var videoReader: AVAssetReader?
    private var trackOutput: AVAssetReaderTrackOutput?
    weak var delegate: AVCaptureVideoDataOutputSampleBufferDelegate?
    var queue = DispatchQueue(label: "simulator.video")

    func startPlayback(url: URL) {
        let asset = AVAsset(url: url)
        guard let track = asset.tracks(withMediaType: .video).first,
              let reader = try? AVAssetReader(asset: asset) else { return }

        let output = AVAssetReaderTrackOutput(
            track: track,
            outputSettings: [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA]
        )
        reader.add(output)
        reader.startReading()

        videoReader = reader
        trackOutput = output

        displayLink = CADisplayLink(target: self, selector: #selector(sendNextFrame))
        displayLink?.preferredFrameRateRange = CAFrameRateRange(minimum: 30, maximum: 30, preferred: 30)
        displayLink?.add(to: .main, forMode: .default)
    }

    @objc private func sendNextFrame() {
        guard let sampleBuffer = trackOutput?.copyNextSampleBuffer() else {
            displayLink?.invalidate()
            return
        }
        delegate?.captureOutput?(AVCaptureOutput(), didOutput: sampleBuffer, from: AVCaptureConnection())
    }
}

Add a test video (recorded from a device) to your asset catalogue. In #if targetEnvironment(simulator) blocks, use SimulatorVideoSource instead of AVCaptureSession. Your whole pipeline — MediaPipe, landmark smoothing, state machine — runs end-to-end in the simulator against a known video.