Enterprise RAG — A practitioner’s build log | Post 4 of 6

RAG evaluation in most implementations stops at one question: did the system retrieve the document that was supposed to be retrieved?

That is the right question for a consumer search product. It is an incomplete question for an enterprise knowledge system where some documents are accessible to all employees and others are restricted by role. In that environment, retrieval correctness is necessary but not sufficient. You also need to know whether restricted documents leaked into answers they should not have influenced.

Enterprise RAG implements four metrics. Each measures a different failure mode. Together they constitute a validation standard that is appropriate for internal knowledge systems handling mixed-sensitivity documents.

The four metrics and what each catches

Pass rate — the percentage of evaluation cases where expected documents were retrieved and forbidden documents were not.

This is the composite metric. A case passes only when both conditions hold simultaneously: the expected documents appear in the citation set and no forbidden documents appear. A system that retrieves all expected documents but also leaks one forbidden document does not pass that case. Pass rate does not reward partial correctness.

Restricted leakage count — the number of evaluation cases that returned at least one forbidden document in the citation set.

This is the most operationally critical metric for an enterprise deployment. A restricted leakage count of zero means the role filter is working correctly across every test case in the evaluation set. Any non-zero value indicates a specific failure to investigate — which case, which document, which role, which query.

Citation coverage — the average number of citations returned per evaluation case.

Low citation coverage indicates that the retrieval system is returning answers without grounding them in source documents. In an enterprise context, an answer without citations cannot be verified, audited, or traced back to a source document. Citation coverage is a proxy for answer auditability.

Average latency (ms) — the mean query execution time across all evaluation cases.

Latency is not a quality metric. It is an operational baseline. If latency increases after a retrieval configuration change — switching from local lexical to Azure AI Search, adding a reranking step, increasing chunk count — the evaluation run captures it. Latency regression during evaluation is a signal worth investigating before deploying the change.

What the evaluation output looks like

Running POST /eval/run returns all four metrics in a single response alongside the per-case results:

{
  "pass_rate": 1.0,
  "restricted_leak_count": 0,
  "citation_coverage": 2.4,
  "average_latency_ms": 38.2,
  "cases": [...]
}

Each case in the result array includes the question, the role used, the retrieved document IDs, and a pass/fail indicator. A failed case shows exactly which expected document was missing or which forbidden document leaked.

The evaluation runner calls POST /query internally for each case, which means it exercises the entire pipeline: authentication, role filter, retrieval, generation, and citation assembly. The metrics reflect actual system behavior, not a mocked retrieval path.

Why the evaluation set structure matters as much as the metrics

The evaluation set (demo/evaluation_set.json) defines each case with three fields alongside the question and role:

• Expected document IDs — documents that must appear in the citation set for the case to pass
• Forbidden document IDs — documents that must not appear in the citation set

Both lists are required for every case. An evaluation case without forbidden document IDs cannot measure restricted leakage. An evaluation set without any cases involving restricted documents cannot validate access control at all.

Most RAG golden sets I have reviewed define expected documents only. They measure retrieval recall but provide no signal on access control correctness. Adding forbidden document IDs to every test case involving a restricted document is the minimum viable evaluation standard for an enterprise knowledge system.

What the evaluation set does not yet cover

The current evaluation set is optimized for access-control validation and citation tracing. It does not yet cover:

• Answer correctness — whether the generated answer is factually accurate relative to the cited documents. This requires human relevance labels or LLM-as-judge evaluation templates.
• Semantic retrieval quality — the lexical retriever handles the current evaluation set well. A semantic retrieval configuration may return different ranked results that pass the access-control checks but rank differently by relevance.
• Regression thresholds in CI — the evaluation runner is callable from CI. The current setup does not fail a CI run if pass rate drops below a threshold. Adding a threshold check (if restricted_leak_count > 0: fail) to the CI workflow is the practical next hardening step.

These are documented roadmap items, not silent gaps.

Current limits

• Evaluation set size is small — calibrated for repeatable local validation, not production-scale coverage.
• Answer correctness evaluation requires human labels or an LLM judge. Neither is implemented in the current evaluation runner.
• Latency benchmarks reflect the local SQLite retrieval path. Azure AI Search latency profiles will differ.
• The evaluation runner requires the ADMIN_TOKEN when management protection is enabled, which prevents accidental evaluation runs in shared environments.

Next engineering step

Review the evaluation cases in demo/evaluation_set.json and count how many include at least one forbidden document ID. If any cases have only expected documents, add a forbidden document ID for a restricted document that should not be retrievable by that role. Then re-run POST /eval/run and verify the leakage count remains zero.

One question for you

Does your current RAG evaluation set include forbidden document IDs alongside expected document IDs? If not, how do you validate that restricted documents are not influencing answers for unauthorized roles?

Next post: Security decisions in Enterprise RAG — how API keys, audit logs, and the order of role enforcement work together to make the system defensible.

Every AI Product Has the Same Hidden Privacy Problem

The more useful the assistant becomes, the more sensitive the user data becomes.

A user writes something like:

I need to move my appointment from Tuesday to Friday because I had surgery last month and I’m still on medical leave.

My policy number is INS-48291.

The task is simple: reschedule an appointment. But the prompt also includes sensitive medical, employment-related, and internal policy information that the model may not need to complete that task.

In many AI applications, that message is sent directly to a third-party LLM provider.

Why This Matters Now

In regulated environments, teams are expected to limit personal data processing, explain why data is needed, and apply privacy by design.

With LLMs, that becomes harder because users often paste sensitive information directly into prompts, even when that information is not required for the task.

The risk is not that every AI app will receive a massive fine.

The real risk is losing control over where sensitive data goes, how long it is stored, whether it appears in logs or traces, and whether the company can prove that the model actually needed that data in the first place.

Companies want to use LLMs in customer support, healthcare, finance, legal workflows, HR, internal tools, and enterprise automation.

But these are exactly the environments where sensitive data appears naturally in conversations.

So the question is not:

Can we remove all personal data?

The real question is:

Can we protect personal data while keeping the AI useful?

This is the problem I tried to solve with PII Firewall, an open-source privacy layer for AI applications.

PII Firewall behaves as a stateful proxy that sits between your application and any LLM provider. It detects sensitive information, replaces it with safe tokens before the model call, and restores the original values only inside your trusted environment.

It is model-agnostic by design, works across 55+ languages, and lets teams combine multiple PII detection techniques through a simple, unified framework.

The model still gets the context it needs. But it never needs to know who the user really is.

Redaction Is Not Enough

The most common approach to privacy is redaction.

Take this input:

My name is Maria Perez and my email is [email protected].

A simple redaction system might turn it into:

My name is [REDACTED] and my email is [REDACTED].

That protects the data, but it also destroys useful context.

If the model later responds with:

I have updated the record for [REDACTED].

The final answer is technically private, but not useful.The model does not know what was redacted. Was it a person, a company, an email address, or a case reference?

And in a multi-turn conversation, a stateless redaction layer cannot reliably preserve who or what each placeholder refers to across messages.

The Core Idea: Pseudonymize, Reason, Rehydrate

PII Firewall uses reversible pseudonymization.

Instead of deleting sensitive values, it replaces them with safe tokens:

John Doe         → PERSON_1
[email protected] → EMAIL_1

So the LLM receives:

Hi, I'm PERSON_1. My email is EMAIL_1.
Can you help me update my insurance claim?

The model can still understand that there is a person, an email address, and a task.

It can respond naturally:

Sure, PERSON_1. I can help you update the account linked to EMAIL_1.

Then PII Firewall rehydrates the response inside your trusted environment:

Sure, John Doe. I can help you update the account linked to [email protected].

The user gets a personalized answer, but the LLM provider never receives the raw personal data.

A Flexible Privacy Framework

PII Firewall is not the first attempt to protect LLM calls from sensitive data exposure.

Privacy proxies, PII redaction tools, anonymization layers, and AI security gateways are becoming part of the production LLM stack.

But in practice, privacy is rarely solved with a single detector, a single language, or a single rule.

Some applications need fast pattern matching for emails, phone numbers, credit cards, IBANs, or internal IDs. Others need language-aware models to detect names, locations, organizations, dates, or contextual references. Some teams need healthcare-specific detection, while others need finance, legal, HR, customer support, or internal enterprise rules.

PII Firewall lets teams:

• Choose the LLM provider
• Combine different PII detection techniques
• Adapt detection to multiple languages
• Define different actions per entity
• Keep reversible mappings in a scoped vault

In other words, the goal is not to provide one fixed privacy policy, but a framework for defining the right privacy behavior for each application.

Model-Agnostic by Design

PII Firewall can sit in front of OpenAI, Anthropic, or any provider that accepts text input and returns text output.

This matters because AI infrastructure changes quickly. A team may start with one provider, later route workloads to another, or eventually move some models in-house.

Your privacy layer should not have to be rewritten every time your model strategy changes.

PII Firewall is designed as middleware: a thin privacy boundary between your application and whichever model you choose.

Detector-Agnostic Too

PII detection is not a solved problem with a single perfect technique.

Some sensitive data is structured and easy to detect:

[email protected]
+34 600 123 456
4242 4242 4242 4242

Rule-based patterns work very well for these cases.

Other data is contextual:

John lives near the hospital and spoke with Dr. Martinez last Friday.

Here, names, roles, dates, and locations may need language understanding.

PII Firewall supports multiple detection engines. You can use:

• Simple rule-based patterns when you want speed and precision
• NER-based detection when you need context
• Transformer models for more specialized domains
• Hybrid mode to combine several approaches

Built for More Than English

A lot of privacy tooling works well in English and then quietly breaks down elsewhere.

Real users write in Spanish, French, Italian, Portuguese, German, Arabic, Japanese, and many other languages. They use different name structures, address formats, national identifiers, phone formats, bank account formats, and local conventions.

PII Firewall includes language-aware routing, so detection can adapt to the language and region of the text.

This allows the system to apply the right patterns and models for the context instead of relying on a single English-centric detector.

For example, a privacy layer used in Europe should understand IBANs, local phone numbers, national identifiers, and multilingual names.

A global AI product cannot treat privacy as an English-only problem.

Different Domains Need Different Privacy Behavior

Different industries have different privacy needs.

A healthcare assistant should preserve clinical utility while protecting patient identity.

A finance assistant should protect payment data while keeping enough context for analysis.

A legal assistant may need to preserve case references while anonymizing party names and addresses.

PII Firewall includes domain-specific presets for common scenarios such as:

• Healthcare
• Finance
• Legal
• Generic

These presets are not meant to be rigid. They are starting points.

Teams can override specific behaviors, add their own entity types, or define new profiles for their own internal data.

For example, a company may want to detect:

• Employee IDs
• Customer IDs
• Ticket numbers
• Contract references
• Internal project codes

Those identifiers may not be “standard PII”, but they can still be sensitive inside a business context.

The same applies to how different values are handled.

Not all sensitive data should be handled the same way:

• A person’s name might need to be pseudonymized so the model can continue referencing that person.
• A credit card number should usually be masked.
• A precise date of birth might be generalized.
• A value that should never be exposed may need to be fully redacted.

PII Firewall supports different actions depending on the type of data:

John Doe                   → PERSON_1
[email protected]            → EMAIL_1
4242 4242 4242 4242         → ************4242
17 March 1984               → 1980s
Highly sensitive value      → [REDACTED]

Privacy is not one-size-fits-all. The right behavior depends on the data type, the use case, and the risk profile.

Stateful Privacy

One of the most important parts of PII Firewall is that it is stateful.

A stateless filter can remove data, but it cannot easily remember that across a conversation:

PERSON_1 = John Doe
EMAIL_1  = [email protected]
CASE_1   = Insurance Claim 8821

PII Firewall keeps this mapping in a vault scoped to the right context.

That context can include things like:

• Organization
• User
• Case
• Thread

This makes it possible to track mappings safely and restore values only when appropriate.

It also makes compliance workflows easier.

For example, if a user invokes a right-to-forget request, the application can purge the mappings for that user, case, or session.

The goal is to manage sensitive data across its lifecycle, not just remove it from a single prompt.

Example

Here is the basic idea in code:

from privacy_firewall import PrivacyFirewallSDK

firewall = PrivacyFirewallSDK.create(
    domain="healthcare",
    detector_backend="hybrid",
)

context = {
    "tenant_id": "acme-corp",
    "case_id": "case-8821",
    "thread_id": "thread-001",
    "actor_id": "user-42",
}

sanitized = firewall.anonymize_text(
    text=user_prompt,
    context=context,
)

response = my_llm(
    sanitized.sanitized_text,
)

final = firewall.rehydrate_text(
    text=response,
    context=context,
)

Conclusion

As AI moves into production, privacy cannot remain an afterthought, especially as regulation becomes stricter in Europe.

The safest personal data to send to an LLM is the data it never receives.

PII Firewall is an open-source project. If you are building AI products in healthcare, finance, legal tech, customer support, internal automation, or enterprise SaaS, I would love feedback, issues, and contributions from people working on real-world systems.

This is a submission for the Google I/O Writing Challenge

Earlier this week, a software system built the core framework of a functioning operating system from scratch in exactly 12 hours. No human wrote a single line of code. The architecture wasn’t planned by a principal engineer; it was broken down, assigned, and compiled entirely by an autonomous ecosystem of AI agents.

The entire project cost less than $1000 in API compute credits.

If your current relationship with generative AI consists of manually treating a language model like a highly advanced autocomplete tool—typing a prompt, waiting for an isolated snippet of syntax, and copy-pasting it into your IDE—you are using a paradigm that Google just retired. At Google I/O 2026, the company shifted its focus entirely away from passive chatbot extensions. We are entering the era of independent, multi-agent orchestration engines.

As a Google Developer Expert (GDE) who designs and breaks these frameworks daily, I can tell you that this isn’t an incremental tooling upgrade. It changes the fundamental unit economics of what is financially and structurally viable to build.

1. Inside the 12-Hour Build: Parallel Orchestration in Action

During the developer keynote, Google DeepMind’s Varun Mohan introduced Antigravity 2.0, which has transitioned from an experimental IDE helper into a standalone, agent-first desktop application. To make the engineering capabilities of the platform clear, Mohan dropped the telemetry of their internal stress test: a 93-agent team assigned to write an OS.

• The Team Size: 93 autonomous sub-agents spun up on demand.
• The Compute Load: 15000 API requests processing 2.6 billion tokens.
• The Operational Cost: Under $1000 USD using the new Gemini 3.5 Flash engine.

Instead of stretching one massive context window to its absolute breaking point with a single giant prompt, the core Antigravity agent acted strictly as an engineering manager. It broke the master task into parallel micro-objectives, spinning off sub-agents to handle isolated modules independently. One group engineered the memory management sub-systems, another excelled at the process scheduler, and separate agents concurrently generated and executed unit tests within an isolated containerized sandbox.

The GDE Take: What fascinated me about this run wasn’t just that the code compiled without human syntax correction; it was the ecosystem’s capacity for autonomous problem-solving. When Mohan’s team first attempted to run the classic game DOOM on the newly compiled OS onstage, the game stalled due to missing video and keyboard drivers. I’m currently testing with some agents in current version in Gemini (only Agent Manager was updated so far).

Instead of a developer jumping in to write a wrapper, Mohan simply gave the agent harness a high-level correction command. The main agent automatically recognized the missing dependencies, deployed a specialized driver sub-agent, wrote the code, ran a quick compilation test, and launched the game live on stage. As an architect, seeing a multi-agent system patch its own environment mid-execution changes my entire reference point for what “agentic coding” means.

2. Fueling the Token Monster: Why Silicon Architecture Matters

If you manage application infrastructure or keep a tight eye on cloud spend, looking at a 15,000-request architecture probably gives you financial anxiety. Running nearly a hundred agents concurrently to process billions of tokens is a massive “token monster.” If you try to run this kind of dense, recurring automation pattern over traditional, generalized infrastructure models, your operational budgets will disappear before mid-year.

To understand why this is suddenly cost-effective, we have to look at how Google co-optimized the model layer with their hardware fabric:

• Gemini 3.5 Flash: This model is designed directly for the low-latency, rapid-fire iterations required by agent loops. Inside the Antigravity system harness, Flash is co-optimized to run up to 12x faster than standard frontier models, intentionally prioritizing execution throughput and cost efficiency without dropping the semantic reasoning necessary to maintain systemic code architecture.
• TPU v8i Inference Chips: Standard AI hardware is typically built for massive, heavy-duty training workloads—the computing equivalent of a cargo train. For Antigravity 2.0, Google split its 8th-generation silicon footprint, creating the TPU v8i variant optimized strictly for real-time inference. It handles massive, parallelized token streams without letting the agent loops stall while waiting for compute availability.
• Jackson Pathways: Imagine all 93 sub-agents blasting billions of tokens back and forth within a single local server rack; the localized bottleneck would completely freeze execution. Jackson Pathways acts as a dynamic traffic routing network, seamlessly distributing token loads across globally separate data center clusters in real time, keeping the entire pipeline completely fluid.

3. The Real Frontier: Autonomy Demands Absolute Responsibility

The system engineering behind Antigravity is a massive technical win, but the most important part of the keynote didn’t involve code telemetry. It came during Demis Hassabis’ final segment, where he explicitly noted that we are standing in the “foothills of the singularity.” For developers entering this space, we need a clear, unhyped understanding of what that actually implies. When an expert like Hassabis references the “singularity,” he isn’t talking about science fiction tropes or self-aware machines; he is describing a highly specific technological inflection point where AI systems become advanced enough to recursively design, iterate, and optimize software entirely independent of human engineers.

When code can propagate, scale, and compile itself across 93 parallel threads, human code review pipelines become a massive operational bottleneck. Our roles are fundamentally shifting from syntax writers to telemetry auditors, security gatekeepers, and boundary architects. If you let teams of autonomous sub-agents run wild in a repository without explicit constraints, security vulnerabilities and architectural debt will multiply faster than your team can track them. As architects, we have to master the defensive guardrails Google outlined:

Automated Pipeline Defense via Code Mender

We can no longer treat security as a post-build chore. Google highlighted Code Mender, an automated defense agent built directly into the integration layer. As your Antigravity sub-agents generate and push code to your sandbox, Code Mender runs alongside them asynchronously, scanning code blocks for critical vulnerabilities and committing automated security patches before the software ever touches a production branch.

Immutable Data Provenance (SynthID & C2PA)

In a development landscape flooded with autonomous agent systems, knowing the exact lineage of your code and media assets is a non-negotiable requirement for system compliance. Google announced widespread, unified backing for its SynthID watermarking engine and C2PA Content Credentials alongside industry majors like OpenAI, Microsoft, Nvidia, and ElevenLabs. This infrastructure creates a permanent, tamper-proof cryptographic paper trail. For a developer, this means you can cryptographically verify exactly which agent system generated a file, what data model it trained on, and whether a code block has been altered post-compilation.

4. Conclusion: Moving Up the Stack

The shift from single-turn prompts to autonomous orchestration frameworks marks the graduation of our industry. The core developer discipline of the next decade won’t be defined by how efficiently we look up documentation or format language blocks inside an IDE interface. The future belongs to the system architects—the engineers who know how to design multi-agent communication networks, configure secure containerized sandboxes, and orchestrate automated guardrails around independent code generation layers.

We aren’t being replaced; we are being promoted. The baseline engineering execution is becoming a commodity, forcing us to focus entirely on intent, boundaries, and overall system logic. The question is no longer about how to write a specific software utility—it’s about what system architecture you are ready to let a coordinated ecosystem of agents build for you.

Over to the community: If you had a $1000 token budget to spin up a 90-agent team inside Antigravity 2.0 right now, what manual deployment pipeline or complex architecture component are you letting them automate first? Let’s talk about system design in the comments below!