Why RAG Fails in Enterprise R&D (And What Actually Works)

RAG was a breakthrough. Embedding documents into vectors, retrieving the most similar chunks at query time, and feeding them to an LLM — it gave models access to external knowledge for the first time. For a customer support bot searching a knowledge base, it’s genuinely effective.

But when you deploy RAG into an enterprise R&D environment — with 1,000+ engineers, dozens of interconnected systems, and AI agents that need to take action, not just answer questions — it falls apart in predictable ways.

I’m Gilad Salinger, CEO of Naboo. We build the context layer that replaces RAG for enterprise AI agents. After deploying in production at companies like Global-E (NASDAQ: GLBE) and Melio, I want to share the five specific failure modes we kept seeing — and the architectural approach that fixes them.

The Setup: What Enterprise R&D Actually Looks Like

A typical enterprise engineering organization has context spread across:

• Code repositories (GitHub, GitLab, Bitbucket) — often 50+ repos
• Project management (Jira, Linear, Asana) — thousands of tickets
• Documentation (Confluence, Notion) — much of it outdated
• Communication (Slack, Teams) — where real decisions happen
• Monitoring (Datadog, Splunk, PagerDuty) — production state
• CI/CD (Jenkins, GitHub Actions) — deployment history

When a developer asks an AI coding agent “help me with this ticket,” the agent needs to pull context from all of these systems, understand the relationships between them, and filter by what the developer is allowed to see.

RAG can’t do this. Here’s why.

Failure Mode 1: Context Fragmentation

RAG indexes each data source independently. You get a vector store for your code, another for your Confluence docs, another for Slack messages. But enterprise context is relational. A Jira ticket is meaningless without the code it references, the PR that implements it, and the Slack thread where the team discussed why they chose that approach.

When an agent retrieves the 10 most similar chunks from each source, it gets 30 disconnected fragments. The LLM has to guess how they relate. In our benchmarks, this guessing is where most accuracy loss occurs.

What works instead: Build a cross-system understanding that maps dependencies, ownership, and decision trails across all sources. When the agent queries for context, it gets a coherent package — not scattered chunks.

Failure Mode 2: No Intent Understanding

RAG retrieves based on text similarity. “Fix the authentication bug” and “review the authentication module” would retrieve nearly identical chunks. But these tasks need completely different context.

Fixing a bug requires: the specific error, recent changes to the auth flow, the PR that introduced the regression, relevant test failures. Reviewing a module requires: architectural overview, code ownership, tech debt history, related design documents.

RAG treats both the same because it only understands text distance, not task semantics.

What works instead: Calculate what context is needed based on the task type, current system state, and the user’s role. Intent-aware retrieval, not similarity-based retrieval.

Failure Mode 3: Stale Context

Enterprise codebases change constantly. A PR merged 2 hours ago might change the correct approach to a task entirely. But most RAG systems re-index on a schedule — daily, sometimes weekly.

We had a case where a developer asked an AI agent for help refactoring a module. The agent suggested an approach based on the old architecture because the RAG index hadn’t caught the PR that changed the module’s interface the previous day. The developer spent 3 hours on a dead-end approach before realizing the context was stale.

What works instead: Continuous ingestion that updates the context model in real-time as commits, messages, tickets, and deployments happen. No batch re-indexing delays.

Failure Mode 4: Security as an Afterthought

This one is critical for enterprise. Most RAG implementations index everything into a single vector store, then try to filter results after retrieval. “Post-hoc RBAC.”

The problem: vector similarity search doesn’t natively support access controls. If a junior developer’s query is semantically similar to a document they shouldn’t see, the vector DB returns it. The filtering layer has to catch it. And filtering layers have gaps.

In defense, financial services, and healthcare organizations — where a single data access violation can mean regulatory penalties — this architecture is a non-starter.

What works instead: Native RBAC that enforces permissions at retrieval time, not post-retrieval. The context layer inherits permissions from your existing tools (GitHub org roles, Jira project permissions, Confluence space restrictions) and only surfaces context the user is authorized to see.

Failure Mode 5: Token Waste

RAG retrieves by similarity, which means many returned chunks are tangentially relevant at best. In our analysis of production RAG deployments, roughly 60-70% of retrieved chunks contributed nothing to the agent’s output. They just consumed tokens.

This matters for three reasons: cost (tokens aren’t free at enterprise scale), latency (more tokens = slower responses), and quality (noise in the context window degrades LLM output quality). The “lost in the middle” problem is well-documented — LLMs pay less attention to information in the middle of long contexts.

What works instead: Deliver only the context the agent needs for the specific task. In our benchmarks, this means 90% fewer tokens with 97% higher accuracy. Less is more when the “less” is precisely targeted.

The Architecture That Works: Context Layers

The pattern we landed on — and that we’re seeing the industry converge toward — is what we call a context layer. It sits between your data sources and your LLM/agent framework.

Instead of: Query → Embed → Vector search → Top-K chunks → LLM

It’s: Query → Intent calculation → Cross-system context assembly → RBAC filtering → Execution-ready context → LLM

The key differences:

1. Cross-system understanding instead of per-source indexing
2. Intent-aware retrieval instead of similarity-based retrieval
3. Continuous ingestion instead of batch re-indexing
4. Native RBAC instead of post-hoc filtering
5. Precise context packages instead of top-K similar chunks

Benchmarks

We ran benchmarks using LLM-as-a-judge evaluation across production enterprise environments:

The accuracy improvement comes primarily from the intent-aware retrieval and cross-system relationship mapping. The token reduction comes from delivering only relevant context instead of similarity-matched chunks.

When RAG Is Still Fine

To be clear: RAG isn’t wrong, it’s scoped. If you’re building:

• A customer support bot searching a knowledge base
• A research assistant querying a document corpus
• A chatbot for a small team with a single repo

RAG works. The failure modes above only manifest at enterprise scale — multiple systems, complex permissions, AI agents that need to execute (not just answer), and accuracy requirements where “approximately right” isn’t good enough.

What to Try

If you’re hitting these failure modes:

1. Audit your current RAG pipeline for the five failures above. Most enterprise teams are hitting at least 3.
2. Look at the context layer pattern — whether you build it yourself or use an existing implementation.
3. Start measuring accuracy with LLM-as-judge, not vibes. The gap between RAG and intent-aware context is only visible when you measure properly.

We’ve open-sourced our benchmark methodology. If you want to run it against your own data, reach out at naboo.ai or book a technical demo.

Gilad Salinger is CEO & Co-Founder of Naboo, the enterprise context layer for AI agents. Previously founded and scaled a developer tools company. Naboo is backed by Cardumen Capital and 91 Ventures, and is deployed in production at Global-E (NASDAQ: GLBE), Melio, and other enterprise R&D organizations.

I have a confession: I keep starting “automation projects” and abandoning them at the same point.

The pattern is always the same. I write a script that does One Useful Thing: fetch some news, summarise it, post a draft somewhere. Three weeks later I want to add a second destination, or a different source, or a new model provider. The original script has grown a // TODO: refactor comment, and the new thing requires touching six files I don’t remember (which is even worse since most of the code is written by Claude/Codex now).

So this time I tried a different rule:

Every feature is a plugin. The core knows nothing about my domain. If adding a capability requires me to edit the host, the host is wrong.

The result is BFrost (could you let me know what you think about this name in the comment?), a worker-first local AI operations platform (I’m publishing today as a public preview (v0.1.0)). It’s a small Node.js host that runs on your machine, schedules jobs, moves items between producers and consumers, and renders a live React dashboard. Everything else (news harvesting, publishing to X or WordPress, Telegram delivery, model providers, assistant tools) is a worker you drop into a folder.

Below: what makes this different from “yet another agent framework” what I had to give up to keep the rule honest, and how a bundled Claude Code skill scaffolds new workers without ever editing the core.

The one rule

Most extensible tools start extensible and decay. Plugins get “first-class” features the core knows about. Feature flags accumulate. The plugin API quietly becomes “the things our most-used plugin happens to need.”

BFrost’s hard rule, written into the repo’s contributing docs and a Claude Code skill:

src/ outside src/workers/    →  domain-free. Never names a worker.
web/src/ outside web/src/workers/   →  same.
workers/local/<id>/   →  where your stuff lives.

That single constraint forced every interesting decision in the project.

What “worker-first” looks like in practice

A worker is a folder with three things:

workers/local/my-mastodon-publisher/
├── worker.json          # the manifest - id, surfaces, settings, cron jobs
├── src/index.ts         # the backend module
└── dashboard.tsx        # optional - a runtime-loaded React tab

The manifest is the single source of truth. It declares the worker’s id, the cron jobs it contributes to the scheduler, the settings forms the dashboard should render for it, any credentials it needs, and any dashboard surfaces it owns. The host reads it, mounts the worker, and stays out of the way.

The backend module exports a BackendWorkerModule – TypeScript or JavaScript, your call. The host compiles TypeScript on first load with esbuild and caches the result. There’s no build step you run manually; you edit the file, save, and the next time the worker is enabled the host rebuilds it.

// workers/local/my-mastodon-publisher/src/index.ts
import {
  listItemsForConsumer,
  applyConsumerSuccess,
  applyConsumerFailure,
  openWorkerKv,
} from 'bfrost';

export const workerModule = {
  manifest: /* loaded from worker.json */,
  jobs: {
    'mastodon-publish': async ({ workerId }) => {
      const items = await listItemsForConsumer(workerId, {
        itemType: 'news.article',
        excludeAlreadyHandled: true,
      });
      for (const item of items) {
        try {
          const post = await publishToMastodon(item);
          await applyConsumerSuccess(workerId, item.id, {
            metadata: { postUrl: post.url, postedAt: new Date().toISOString() },
          });
        } catch (err) {
          await applyConsumerFailure(workerId, item.id, { reason: String(err) });
        }
      }
    },
  },
};

That’s it. No registration boilerplate, no DI container, no “register this with the scheduler.” The host already knows about this worker because its folder exists.

The Item Bus is just a typed queue

When the news worker scrapes an article and the X publisher posts it, what’s happening between them?

The temptation is to wire them directly. Worker A imports worker B, calls a method, done. That’s also exactly what I’m trying not to do – now A knows B exists, and adding a third consumer means editing A.

BFrost’s answer is the Item Bus. The shared queue is generic, owned by core, and looks like this:

{
  "id": "itm_abc",
  "producerWorkerId": "core.news",
  "itemType": "news.article",
  "tags": ["ai", "tooling"],
  "payload": {
    "article": { "title": "...", "url": "..." },
    "source":  { "host": "...", "label": "..." }
  },
  "metadata": {
    "local.publisher.wordpress": {
      "publishedUrl": "https://example.com/posts/xyz",
      "publishedAt": "2026-05-17T10:30:00Z"
    }
  }
}

A producer writes payload. Each consumer reads payload, does its work, and writes back into its own namespace under metadata. Consumers never mutate each other’s metadata. That single discipline lets you have five consumers of the same item type without any of them caring about the others.

Want to add Mastodon? Subscribe to news.article, write to metadata['local.mastodon-publisher'] when you post. Want to add BlueSky? Same shape, different folder. The news worker doesn’t know either of you exist.

Dashboards that workers own

The dashboard is a React SPA. Built-in workers ship their UI under web/src/workers/builtin/<id>/dashboard.tsx. Local workers can ship dashboard.tsx inside their own folder; the host bundles it with esbuild (browser target, IIFE, React resolved to window.bfrost.React so we don’t ship a duplicate copy), serves it from /api/workers/:id/dashboard.js, and the admin app loads it via a <script> tag at runtime.

The bundle calls window.bfrost.registerDashboardView({...}) with the surface ids it claims. The host cross-references those against the manifest, so a worker can’t accidentally take over a tab it didn’t declare. Toggle the worker off and its tab vanishes. Edit the TSX, refresh, the new bundle is served because the ETag is mtime-derived.

This means an end user can install a worker and immediately see its dashboard, with its own KPIs, its own forms, its own events – without me approving anything or merging any PR.

What I had to give up

This isn’t free. Several things became harder so that the contract could stay honest:

1. No worker can call another worker directly. Cross-worker coordination goes through the Item Bus (async) only. The synchronous “services” interface is on the roadmap but deliberately deferred until a real use case demands it.
2. The core never imports worker code. Even the bundled news, X publisher, and research workers live in src/workers/builtin/ and are loaded through the same registry path as any local worker. They prove the contract is expressive enough; they aren’t load-bearing for the host.
3. No worker marketplace. Workers load from local disk only. No remote fetching, no signed worker bundles from the internet. v1.0 will get a permission/sandbox model first.
4. No magic on either side. No decorators, no reflection, no class hierarchies. A worker is a folder with a manifest and an exported object. If you can read TypeScript, you can read every line of how a worker is wired.

The payoff: I can read BFrost end-to-end in an afternoon. So can you. So can a Claude Code agent.

Claude Code writes the workers

This is the part that surprised me.

I bundled a Claude Code skill – bfrost-worker-author – inside the repo at .claude/skills/. When you run claude in a BFrost checkout and ask “create a new BFrost worker,” the skill activates. It knows the worker contract, the file layout, the hard rules (“never edit the core”), and the order of operations (pick id → scaffold folder → declare surfaces → write tests).

A real session looks like:

> Add a BFrost worker that consumes news.article items and posts them
> to my Mastodon instance. I'll set the access token at runtime.

[Claude] I'll scaffold local.mastodon-publisher as a local worker under
workers/local/. Confirming a few things first:

  - Producer or consumer? → consumer of news.article
  - Credentials? → secret-reference field for the access token
  - Dashboard tab? → config-tab settings only (no custom tab)
  - Built-in or local? → local

Proceeding to scaffold...

Three minutes later there’s a working worker. It reads news.article items, posts them, writes back into its metadata namespace, exposes a settings form, and ships with a README.md describing what it produces and consumes. The skill physically refuses to edit the core – if a request would require it, Claude stops and surfaces “this looks like a contract gap, here’s what I think the core needs.”

The whole reason this works is that the contract is small and stable. The skill isn’t smart; the rules are simple enough that following them produces correct code.

What’s in v0.1.0

Worth being honest about state. Today, v0.1.0 ships:

• ✅ The full worker-first contract (manifest, lifecycle hooks, dashboard bundles, item bus, per-worker storage).
• ✅ Built-in reference workers: news harvester, X publisher, research notes, Telegram channel, LM Studio provider, plus assistant-tool workers (memory, Google search, article fetch).
• ✅ Local TypeScript workers with compile-on-load (esbuild).
• ✅ A typed bfrost SDK that workers import from – the host registers it as a synthetic module so a worker can’t accidentally bundle a duplicate copy.
• ✅ The Claude Code skill.
• ✅ A real-world example: workers/examples/wordpress-publisher/ consumes news items and publishes them to a self-hosted WordPress via the REST API. Copy it, configure it, run it.

What’s deferred to v1.0:

• 🔜 A permissioned action runtime – formal approval queue + per-worker filesystem/network/credential scopes. Until this lands, keep destructive workers narrow.
• 🔜 Frontend smoke tests, per-worker metrics in the dashboard, an accessibility pass.
• 🔜 A hosted docs site and a Worker Gallery in the dashboard.

The browsable docs already live at https://convertprivately.com/bfrost/ – getting started, architecture, example workers, and authoring with Claude Code, all four pages.

Try it

git clone https://github.com/ccascio/BFrost.git
cd BFrost
npm install
npm run dev          # host + admin API
npm run dev:web      # dashboard at http://localhost:5173

Then either copy workers/examples/wordpress-publisher into workers/local/, or run claude from the repo root and ask it to scaffold a worker for whatever you actually want to automate. Star the repo if you like the direction, open an issue if you don’t – the worker-proposal template is set up specifically for “I want to extend BFrost with X.”

Why I’m publishing this now

The honest answer: because I’ve shipped the project past its own bar twice and held it back both times for “one more polish pass.” Workstream 5 (the permission runtime) genuinely matters for v1.0 – workers that touch the real world need approval gates. But v0.1.0 is for people who want to read the code, copy an example, and tell me where the contract breaks.

If you build a worker, drop a link in the comments. If the contract doesn’t fit your case, I want to know – that’s a roadmap item, not a defect in your worker.

Repo: https://github.com/ccascio/BFrost
Docs: https://convertprivately.com/bfrost/
About me: https://convertprivately.com/about/

At one point, most of my thinking around software revolved around pages.

• landing pages
• dashboards
• admin panels
• APIs
• components
• layouts

And while those things still matter, over time I realized something important:

Most complexity in modern applications doesn’t come from screens.

It comes from flow.

Data flow.
Operational flow.
Deployment flow.
Communication flow.
Validation flow.
Infrastructure flow.

Eventually I stopped asking:

“What page are we building?”

And started asking:

“What system flow are we designing?”

That single shift completely changed how I approach architecture.

Applications Are Really Pipelines

Underneath almost every modern application is a series of transformations.

A request comes in.

Something validates it.

Something transforms it.

Something authorizes it.

Something enriches it.

Something stores it.

Something logs it.

Something renders it.

Something caches it.

Something distributes it.

Most systems are pipelines whether we intentionally design them that way or not.

The difference is whether the flow is explicit or hidden.

Hidden Flow Becomes Hidden Complexity

One of the biggest problems I kept running into in modern web development was invisible architecture.

Logic scattered everywhere:

• hooks
• middleware
• event systems
• decorators
• global state
• lifecycle callbacks
• framework magic
• plugin injections

At small scale, it feels convenient.

At larger scale, it becomes difficult to reason about.

You start asking questions like:

• “Where is this actually happening?”
• “What modified this request?”
• “Why did this state change?”
• “What order does this execute in?”
• “Why is debugging this so painful?”

Eventually I realized the issue wasn’t necessarily the tooling itself.

It was that the system flow wasn’t obvious anymore.

And once flow becomes invisible, architecture becomes harder to trust.

Explicit Pipelines Changed How I Think

The more systems I worked on, the more I started preferring explicit pipelines over hidden execution patterns.

Instead of:

“Something somewhere modifies this.”

I wanted:

“This enters here, moves through these stages, and exits here.”

That sounds simple.

But it changes everything.

Because now:

• debugging becomes easier
• observability improves
• onboarding improves
• testing improves
• scaling improves
• reasoning about the system improves

The architecture becomes understandable.

And understandable systems scale better organizationally.

Not just technically.

A Request Is Rarely “Just a Request”

Think about a modern application request.

Even something simple like:

POST /api/orders

Usually involves:

• authentication
• validation
• rate limiting
• business rules
• inventory checks
• database writes
• event creation
• notifications
• analytics
• caching
• auditing
• logging
• response transformation

That’s not a route.

That’s an operational flow.

And once I started viewing applications this way, the idea of “pages first” architecture started feeling incomplete.

Why I Started Designing Around Pipelines

This thinking heavily influenced how I approach systems now.

Especially in projects like:

• WebEngine
• Seltzer
• Nectarine
• KiwiPress

I became less interested in:

• massive controllers
• hidden middleware chains
• implicit execution
• deeply coupled framework logic

And more interested in:

• contracts
• stages
• transformations
• pipelines
• explicit runtime behavior

Because explicit systems are easier to maintain over time.

And maintainability becomes increasingly important as projects grow.

Flow Is Bigger Than Backend Architecture

This idea extends far beyond APIs.

Businesses themselves are pipelines.

Orders move through stages.

Content moves through stages.

Deployments move through stages.

Users move through stages.

Products move through stages.

Infrastructure moves through stages.

Even teams operate through pipelines:

• planning
• development
• testing
• deployment
• monitoring
• iteration

Once you start thinking in flows instead of isolated screens, you begin designing systems differently.

You stop building disconnected features.

You start building operational ecosystems.

The Industry Often Optimizes for Speed of Creation

A lot of modern tooling optimizes for:

• fast setup
• rapid prototyping
• shipping quickly

And again:
that matters.

But eventually systems reach a size where operational clarity becomes more valuable than short-term convenience.

Because complexity compounds.

Especially when:

• multiple developers touch the system
• infrastructure grows
• integrations increase
• business logic expands
• AI-generated code enters the stack
• scaling requirements appear

At that point, architecture quality matters a lot more than whether setup was fast in the beginning.

AI Makes This More Important, Not Less

I actually think AI increases the importance of explicit flows.

Because generated code can introduce even more hidden complexity if the surrounding architecture isn’t disciplined.

If systems are pipeline-driven and contract-driven:

• behavior becomes easier to validate
• execution becomes easier to trace
• generated logic has clearer boundaries
• architecture remains understandable

Without clear flow design, systems eventually become difficult to reason about regardless of whether humans or AI wrote the code.

What I Think We Need More Of

I think the industry needs more emphasis on:

• explicit runtime architecture
• operational clarity
• observable systems
• predictable pipelines
• modular transformations
• maintainable flow design

Not because it sounds architecturally “pure.”

But because software eventually becomes operational infrastructure.

And infrastructure needs clarity.

Final Thoughts

I still care about good UI.

I still care about developer experience.

I still care about elegant frontend systems.

But nowadays, when I look at an application, I don’t primarily see pages anymore.

I see movement.

I see:

• requests flowing
• systems communicating
• data transforming
• operations scaling
• workflows evolving

In other words:

I stopped designing websites.

And started designing flows.

Low-Level Design is not about writing classes or memorizing patterns.

It is about a fundamental shift in thinking:

Designing systems as a collection of responsibilities, not functions.

Most beginners start with code:

• functions
• APIs
• logic

But real system design starts before that. It starts with how you model the problem itself.

Why Object Thinking Matters

A system designed around functions tends to:

• scatter logic across multiple places
• lose control over state
• become hard to modify safely

A system designed around objects tends to:

• keep data and behavior together
• enforce rules at one place
• make behavior predictable

The difference is not syntactic—it is structural.

The Wrong Mental Model (Function-Centric Design)

deposit(account, amount)
withdraw(account, amount)
update_balance(account)

At first glance, this looks simple and readable.

But it raises deeper design issues:

• Who owns the account state?
• Where are validation rules enforced?
• What prevents invalid updates?
• What happens when business rules change?

The logic is distributed, not owned.

The Correct Mental Model (Object-Centric Design)

class BankAccount:
    def deposit(self, amount):
        ...

    def withdraw(self, amount):
        ...

Now the design changes fundamentally:

• state and behavior are encapsulated together
• rules are enforced inside the object
• external code interacts through controlled methods

The system becomes self-governing at the object level.

Core Idea: Responsibility Ownership

The most important question in object-oriented design is:

Which object owns this responsibility?

Not:

• where should this function go
• how should I structure files
• how many layers should I create

Instead:

• which entity is responsible for this behavior in the domain?

This shift determines the quality of the entire design.

Objects Represent Real-World Systems

A good mental model is to map systems into interacting entities:

Example: Food Delivery System

• User → initiates actions
• Restaurant → prepares food
• Order → maintains lifecycle state
• DeliveryPartner → fulfills delivery

Each object:

• owns state
• exposes behavior relevant to its responsibility
• does not manage unrelated logic

Class vs Object (Core Understanding)

Class

A blueprint that defines structure and behavior.

Object

A real instance with actual state.

class BankAccount:
    def __init__(self, balance):
        self.balance = balance

Objects created from this class:

• Account A → 5000
• Account B → 12000

Same structure, different state.

The Most Important Rule in OOD

A class should represent a single, meaningful responsibility.

When a class:

• handles too many responsibilities → it becomes fragile
• knows too much about unrelated domains → it becomes hard to maintain

Good design is not about size. It is about focus.

A Practical Design Flow

Before writing code, a structured approach helps:

1. Understand the problem clearly
2. Identify core entities
3. Assign responsibilities to each entity
4. Define interactions between entities
5. Only then move to implementation

Skipping these steps leads to incorrect abstractions and redesign later.

Real Insight

In object-oriented design, correctness is not about syntax or patterns.

It is about:

• clarity of responsibilities
• consistency of state management
• predictability of interactions

Good design is what remains stable when requirements change.

One-Line Takeaway

Object-oriented design begins when systems are modeled around responsibilities, not functions.

Everyone’s posting browser-agent demos this week. Click here, scroll there, fill that form. Most break by click seven.

Mine broke too. The submit button on a checkout form that the frontier vision model literally couldn’t see. Billed at $0.01-0.05 per call, called 20-50 times per agent run, the model was burning reasoning capacity on parsing pixel coordinates. A 2B specialist I trained for $4 hits that same button 2.5x more reliably on ScreenSpot-v2.

The architecture is the bug, not the model.

Two Jobs, One Forward Pass

Browser-agent stacks send a screenshot to a frontier vision model and ask it for both the next decision and the click coordinates in one call. Splitting that into two calls, a local 2B grounding model that emits JSON followed by a frontier model that reasons over the JSON, drops vision token spend and raises click accuracy.

browser-use (94k stars), Skyvern (22k stars), Claude Computer Use, OpenAI Operator. Same pattern. Same compound question every step:

Given this page, what should the agent do next, and where exactly does it click?

Two jobs welded together. Reasoning (“what next”) is a probabilistic problem worth a frontier model. Grounding (“where exactly”) is a structured-output problem with a tight schema: clickable elements, bounding boxes, accessible labels.

You’re paying frontier-tier rates for the second job. Per screenshot. Every step of the loop.

Grounding Is a Parser Problem

Once you name it as a parser problem, the right tool changes. You don’t need 200 billion parameters to emit a JSON list of clickable elements. You need a model that:

• Has seen enough UI screenshots to recognize buttons, inputs, links with sub-50-pixel precision
• Outputs strict JSON without hallucinating bounding boxes
• Runs locally so the per-step cost is zero

A 2B specialist trained on screen-parsing data. Not a frontier model.

I trained one. Total cost: $4 of RunPod compute on a single L40S GPU over three hours. The result, browserground, hits 45.3% on ScreenSpot-v2 against GPT-4o’s 18.3%. A 2.5x beat at the click-grounding job, by a model roughly 100x smaller. It runs in 2 GB of unified memory on a MacBook Air at 1.8 seconds per inference. A drop-in for any agent loop currently handing screenshots to a frontier API.

The Reasoning Model Gets Its Reasoning Capacity Back

When you split the call, the frontier model stops seeing pixels. It sees:

{
  "elements": [
    {"id": "e7", "label": "Submit order", "type": "button", "bbox": [344, 612, 478, 658]},
    {"id": "e8", "label": "Edit cart",    "type": "link",   "bbox": [...]}
  ]
}

Now the frontier model does the job it’s good at: deciding e7 vs e8 given the agent’s goal. A reasoning question over structured input. Cheap. Reliable. Auditable.

Three things change at once. Per-step token spend on vision collapses, because the grounding step runs locally. JSON validity hits 100% (the specialist learned the output convention with 17M LoRA parameters). Agent traces become debuggable. You read the structured grounding output before the reasoning step ever runs.

What Anthropic and OpenAI Ship Next

The frontier providers will absorb grounding into their own small models. Within twelve months, “fast vision” or “tool vision” tiers will appear in both Anthropic and OpenAI billing at a fraction of frontier rates. The economics demand it. Nobody can justify charging GPT-4 prices for a parser, and Hugging Face downloads already prove the demand: SeeClick, UI-TARS, and ShowUI pull ~300k category downloads a month between them.

When that ships, stack owners who already split grounding from reasoning have three things the wait-and-see crowd doesn’t. A local fallback if the provider has an outage. An auditable structured-grounding trace in every log line. An exit option to a different reasoning provider without re-validating click behavior, because the grounding step belongs to them.

Stack owners who didn’t split will find their grounding step has quietly become someone else’s API. Same vendor billing the reasoning calls. Same vendor setting the price. Same vendor’s deprecation calendar.

The Diagnostic

Pull up your last failed agent trace. Three numbers:

1. Total tokens spent on vision calls per agent step.
2. Fraction of those tokens spent on grounding (parsing pixel coordinates) vs reasoning (deciding actions).
3. Per-run vision cost at your current API rates.

If grounding dominates the first two numbers, and in most stacks it does, your stack has the split wrong.

Grounding is plumbing. Reasoning is cognition. Stop paying cognition rates for plumbing.

I build the split layer. browserground is the open-source reference. Model: huggingface.co/renezander030/browserground. CLI: github.com/renezander030/browserground. npm: browserground (npm install -g browserground, then browserground parse <img> --target "..."). Apache-2.0. Plugin entry points for Claude Code and Codex CLI. PRs welcome, especially eval cases where it fails.

I write field notes from real builds — AI integration, cron-driven automation, and the parts that break in production. New posts every two weeks at renezander.com.

The Problem

Doctors in India spend 2–3 hours daily on clinical documentation. Most EMR systems are glorified data entry forms that add to the burden rather than reduce it.

We built MedScribe — an AI-powered ambient clinical documentation tool that listens to doctor-patient conversations and generates structured SOAP notes, ICD-10 codes, and prescriptions in real time.

The Tech Stack

• Frontend: React JS
• Database: PostgreSQL 18 with a headless CMS layer
• AI Pipeline: Multilingual ASR → Medical NER → LLM structured output
• Deployment: Azure App Service (web) + on-premise GPU nodes (inference)

Key Technical Challenges

1. Multilingual Voice Capture

Indian doctors code-switch between English, Hindi, and regional languages mid-sentence. Off-the-shelf ASR models fail here. We fine-tuned Whisper on 500+ hours of Indian clinical audio with domain-specific vocabulary.

2. Structured Output from Messy Conversations

A 10-minute consultation produces unstructured dialogue. The LLM pipeline extracts:

• Chief complaint & history of present illness
• Differential diagnosis
• SOAP note sections
• ICD-10 codes
• Prescriptions with dosage

We use constrained decoding + JSON schema validation to ensure output is always machine-parseable.

3. Privacy-First Architecture

Healthcare data can’t leave the clinic network. Our inference runs on-premise with zero data retention — no patient audio or text hits external servers. The EMR syncs only anonymized metadata for analytics.

4. ABDM Compliance

India’s Ayushman Bharat Digital Mission requires specific health record formats (FHIR bundles), ABHA ID linking, and consent management. We built a middleware layer that translates our internal models to ABDM-compliant payloads.

Results

• Documentation time reduced from 15 min → 2 min per patient
• 95%+ accuracy on ICD-10 coding for common specialties
• Works offline — no internet dependency during consultations

What I Learned

1. Healthcare AI is a regulatory problem first, ML problem second. Get compliance right before optimizing accuracy.
2. Doctors won’t change workflows. Build around how they already work, not how you think they should.
3. On-premise isn’t dead. When data sensitivity is non-negotiable, cloud-only is a dealbreaker.

We’re building this at VivaLyn Labs. If you’re working on healthcare AI or building for the Indian market, I’d love to connect.

What’s the hardest compliance/privacy challenge you’ve faced while building AI products?

On May 19, 2026, we ran a simple test: ask 30 different LLM models “What is 2+3?” — 5 times each. 150 real API calls, zero simulation, zero fabrication.

The raw result? 86 succeeded, 64 failed. A 42.7% failure rate.

But that headline number is misleading. Here’s what really happened — and why it validates everything we’ve been building at NeuralBridge.

The Real Failure Rate Is ~4%

Strip out the deliberate fault injections and model deprecations, and the actual infrastructure failure rate is about 4% — all from rate limiting (HTTP 429).

This lines up almost perfectly with Datadog’s 2026 State of AI Engineering report, which found 5% of all LLM API calls fail in production, with 60% caused by rate limits and capacity issues.

Our test: 4%. Datadog (thousands of production customers): 5%. Same order of magnitude. Same root cause.

GitHub Models Are the Wild West

Out of 7 models on GitHub’s new AI inference endpoint:

• 3 returned 404 (model deprecated/removed): Mistral Large, Qwen 2.5-72B, Cohere Command-R+
• 1 (DeepSeek-R1) hit rate limits on 4 out of 5 calls
• Only 3 worked reliably

If you’re building on GitHub Models for production workloads, you need a fallback strategy. Models disappear without warning.

Speed Rankings

DeepSeek’s direct API is 12-16x faster than GitHub/Azure endpoints.

Self-Healing Works — 100% of the Time

In our fault injection group, two timeout→retry scenarios:

• C05: DeepSeek timeout → retry → 5/5 success ✅
• C07: Qwen timeout → retry → 5/5 success ✅

100% self-healing rate on recoverable failures.

The Energy Angle No One Talks About

• 5% of LLM API calls fail (Datadog 2026)
• 60% are infrastructure/capacity issues
• NeuralBridge self-heals 95.19% of those
• 2.86% of all AI compute recovered

At global scale: ~4.86 TWh/year saved ≈ half a nuclear power plant. ~146,000 tons CO₂ not emitted.

Every healed failure is energy saved.

No One Else Does LLM API Self-Healing

Datadog can tell you your LLM calls are failing. We can fix them.

Honest Limitations

1. Small sample: 150 calls, 4 rate-limit errors
2. Single node, not distributed production
3. Simple prompt, not real-world complexity

But the direction is clear: LLM APIs fail at measurable rates, and automatic self-healing works.

Try It

pip install neuralbridge-sdk
nb-doctor --quick

6.7μs diagnosis | 95.19% self-heal | 74.3KB | 1 dependency | Free: 100 calls/month

GitHub | PyPI | neuralbridge.cn

Test: 2026-05-19, Python 3.10.12, 150 real API calls. Datadog State of AI Engineering 2026 (CC BY-ND 4.0). IEA 2026.

Guigui Wang, Founder & CEO, NeuralBridge