I took a screenshot of code with a SQL injection vulnerability, compressed it twice through WhatsApp, and fed it to Gemma 4 running entirely on my 8GB RAM laptop.

One minute and forty-seven seconds later, it pointed out the exact dangerous line, explained why it was vulnerable, and showed the correct way to fix it.

I’m a 19-year-old self-taught developer in Nigeria. I don’t have a high-end machine or a GPU. Just a consumer laptop, an internet connection, and four years of figuring things out alone.

When Google released Gemma 4, I skipped most of the benchmark discussions and tested it myself to see what it could actually do on limited hardware.

This is that report.

TL;DR for the skimmers:

• Gemma 4 E2B runs on 8GB RAM without a GPU
• It analyzed a WhatsApp-compressed screenshot and caught a real SQL injection vulnerability
• It handled Hausa naturally, while Yoruba and Igbo showed some limitations with diacritics
• Available RAM matters more than you think
• It’s free, private, offline, and surprisingly capable

What Gemma 4 Actually Is:

Before I get into what I found, here’s the context you need.

Gemma 4 is Google DeepMind’s latest family of open models. Open means you can download the weights and run them locally — no API costs, no data leaving your machine. For reference: E2B downloads at 7.2GB best for 8gb RAM device, E4B at 9.6GB best for 16gb RAM.

The family comes in three variants:

E2B and E4B — The Edge Models
Built for ultra-low resource deployment. Think mobile devices, Raspberry Pi, laptops without GPUs. E2B has around 2 billion effective parameters. E4B has around 4 billion. These are the models that run on hardware most developers in the world actually own. This is what I tested.

31B Dense — The Bridge Model

31 billion parameters in a dense architecture. Sits between consumer hardware and full server deployment. Bridges the gap between what you can run locally on a powerful machine and what requires a data center.

26B MoE — The Efficient Reasoner
26 billion parameters in a Mixture-of-Experts architecture. Not all parameters activate for every token; only the relevant experts fire. This makes it highly efficient for reasoning tasks at scale without burning through compute proportionally.

I tested E2B. Here’s why that matters for developers like me.

Test 1 — Vision: Low quality Image Test

This was not a clean lab test. This was real world conditions.

I had a screenshot of an Express.js route with a SQL injection vulnerability — the classic mistake where user input goes directly into a database query without sanitization. Instead of taking a clean screenshot and uploading it properly, I sent it through WhatsApp. Then I downloaded it and sent it through WhatsApp again. Anyone who has done this knows what happens; WhatsApp compresses images aggressively. By the time I fed it to Gemma 4, the image quality had degraded significantly.

I opened Google AI Studio, loaded Gemma 4, uploaded the image, and asked it to review the code for security issues.

What happened:

One minute and forty-seven seconds later; on a fresh boot with nothing else running Gemma 4 returned a structured response that:

• Identified the exact vulnerable line in the image
• Named the vulnerability correctly as SQL injection
• Explained how an attacker could exploit it
• Provided the corrected code snippet
• Gave step-by-step prevention advice

The output was specific. It referenced the actual code in the image, not generic advice. It did not say “make sure you validate your inputs.” It said here is the line, here is why it is dangerous, here is the fix.

Why this matters:

Most developers do not have perfect screenshots. They have photos of monitors taken in bad lighting, screenshots forwarded through three different messaging apps, images captured on a low-end phone. The documentation never tests for this. I did.

Gemma 4 processed a degraded, double-compressed image and returned accurate, actionable output. For a model running on consumer hardware, that is not nothing. That is the difference between a model that works in a lab and a model that works in the real world.

Test 2 — The Finding Nobody Else Will Write About

I asked Gemma 4 to explain JWT authentication JSON Web Tokens, a common auth mechanism in three Nigerian languages: Yoruba, Hausa, and Igbo.

This took approximately two minutes and fifty seconds. By this point I had more files open and my RAM was no longer as fresh as the first test. The model was noticeably slower.

But here is what it returned.

Hausa:
The response was accurate and natural. The model understood the request, switched languages correctly, and explained the concept in a way that read like genuine Hausa rather than a mechanical translation. For a locally running model with no internet access during inference, this was genuinely surprising.

Yoruba:
The response came through but with drift. Yoruba has tonal markers — accent marks that change the meaning of words entirely. Without those diacritics in my prompt, the output was approximate rather than precise. Writers targeting Yoruba-speaking audiences would need to verify carefully before publishing anything.

Igbo:
Similar story. Igbo has its own special characters and tonal markers. The model approximated and the nearest recognizable output came through; but it was not fully accurate Igbo. Close enough to understand, not close enough to trust without review.

What this means practically:

There are over 500 million people in West Africa. There are writers, developers right now building and writing for users who speak Hausa, Yoruba, Igbo, Twi, Amharic, Swahili. Those writers need to know exactly what these models can and cannot do in local languages before they ship something.

Here is my honest assessment:

Gemma 4 E2B handles Hausa better than I expected. Yoruba and Igbo have limitations tied directly to diacritics if your prompt does not include them, the output won’t either. For a model running entirely offline, the multilingual capability is remarkable. For production use in tonal African languages, test before you ship.

Test 3 — The 128K Context Window on 8GB RAM

The spec sheet says Gemma 4 supports a 128K context window. That number means nothing without knowing what it costs to use it on consumer hardware.

I fed it an entire README file — a long, detailed project documentation file — and asked for a structured summary.

It took five minutes to complete.

The output was accurate. It understood the document. It structured the summary well. It did not hallucinate content that was not there. It captured the main purpose, the architecture, the setup steps, and the key features correctly.

Five minutes is slow by cloud standards. By the standard of a free, private, offline model running on 8GB RAM with no GPU, five minutes to accurately process and summarize a long document is a different conversation entirely.

The 128K context window is not just a spec sheet number. It held an entire document in memory and reasoned about it correctly. For developers building tools that need to process long files — entire codebases, full documentation, lengthy configuration files — E2B can do this on hardware you already own. Just plan for the time it takes.

The RAM Reality Nobody Documents

Here is practical information that is not in the official documentation anywhere.

I noticed a clear performance pattern across my tests:

The pattern is obvious once you see it. As RAM fills with other processes, Gemma 4 E2B slows down significantly. This is not a flaw. The model needs memory to run and it competes with everything else on your machine.

Practical advice for 8GB RAM users:

• Close everything before running a local inference task
• Restart your machine for faster result — you want fresh RAM
• E2B is the realistic choice at 8GB, E4B will be tight
• Do your most demanding tasks first, before RAM fragments
• If you are building an app on top of Ollama, test your performance after extended use not just on first boot

I learned all while trying to build with it

Which Model Should You Actually Use

Stop reading benchmarks and use this decision guide instead.

You have an 8GB RAM laptop with no GPU → Gemma 4 E2B via Ollama. Nothing else is realistic.

Your project handles sensitive data and privacy is critical → Any Gemma 4 variant running locally via Ollama. Your data stays on your machine. Full stop.

You are building for multilingual users in Africa or South Asia → E2B has meaningful multilingual capability. Test your specific languages before shipping. Hausa works well. Tonal languages with special characters need careful prompting.

You need high performance for a server deployment → 31B Dense is your target.

You need efficient reasoning at high throughput → 26B MoE is built for this.

You are building for mobile or edge devices → E2B or E4B. These models were designed for exactly this hardware profile.

Your budget is zero and you need full capability → E2B via Ollama. Free to download, free to run, free forever. No API key. No subscription. No data leaving your machine.

What Running AI Locally Actually Means

Every conversation about AI accessibility focuses on API costs and internet connectivity. Those are real barriers. But there is a third barrier that nobody talks about: trust.

When a developer in Lagos pastes their production code into ChatGPT or any cloud AI tool, that code leaves their machine. If there are API keys in that code, database connection strings, auth secrets — they just went to a server somewhere. Most developers do not think about this. Most beginners definitely do not.

Running Gemma 4 locally via Ollama removes that problem entirely. Your code goes from your editor to your RAM and back to your screen. Nothing else happens. No network request. No logging. No third party.

For a self-taught developer building their first real project, that matters. For a developer in a region where cloud AI costs are prohibitive relative to local income, that matters. For anyone building tools that touch sensitive user data, that matters.

Gemma 4 E2B is not the most powerful model available. It is not trying to be. What it is — a capable, multimodal, multilingual model that runs on hardware most developers in the world actually own, for free, privately, offline — is something different from anything that existed before it.

There is a difference between a model that exists and a model that runs on hardware people actually own.

That difference is the whole thing.

How To Get Started Right Now

If you have not pulled Gemma 4 yet, here is everything you need.

Step 1 — Install Ollama

Go to ollama.com and download it for your operating system. Install it like any normal application.

Step 2 — Pull Gemma 4 E2B

ollama pull gemma4:e2b

This downloads the model to your machine. Approximately 2-3GB. You only do this once.

Step 3 — Start Ollama

ollama serve

This runs Ollama in the background on localhost port 11434. Leave this terminal open.

Step 4 — Test it immediately

ollama run gemma4:e2b "explain what a SQL injection attack is to a complete beginner"

If you get a response, everything is working. You are now running a capable multimodal AI model locally on your own machine at zero cost.

Step 5 — Try the vision capability

Head to aistudio.google.com, select Gemma 4, upload a screenshot of any code, and ask it to review for security issues. No setup required. See what it catches.

Final Thought

I started these tests expecting to be disappointed. Consumer hardware running open models has usually meant compromises — slow inference, shallow responses, limited context.

What I found instead was a model that analyzed a WhatsApp-compressed screenshot and caught a real security vulnerability. That explained JWT authentication in Hausa. That summarized long documents on 8GB RAM. All privately, offline, and free.

The compromises are still real. The speed is nowhere near cloud models. The tonal language limitations matter. The RAM constraints are physics.

But benchmark scores are measured in controlled environments on optimized hardware by people who are not your users.

I am the user.
8GB RAM. Nigeria. WhatsApp screenshots. Nigerian languages. Midnight deadlines.

And if Gemma 4 works in those conditions, then it works in the real world.

That is the benchmark that matters to me..

Pull it. Test it. Build with it.

ollama pull gemma4:e2b

Everything else is waiting on the other side of that command.

Tested on: 8GB RAM laptop, Windows, Ollama + Google AI Studio, May 2026
Models tested: Gemma 4 E2B
Location: Nigeria

Before we begin — if you’re interested in improving engineering quality and developer workflows, check out git-lrc on GitHub — a free micro AI code review tool that runs on every commit. If the idea resonates with you, consider giving the project a ⭐ on GitHub.

Python developers are used to expressive code.

You can automate workflows in minutes, prototype products quickly, manipulate huge datasets, and build entire backends with surprisingly little code.

But eventually many developers run into a frustrating problem:

“How do I know my program is actually correct?”

You can write tests.
You can add type hints.
You can run linters.

But tests only sample behavior.

They do not prove correctness.

That’s where Lean becomes fascinating.

Lean 4 is not just a programming language.

It’s also:

• a theorem prover
• a proof assistant
• a functional programming language
• and a system for mathematically verifying software

If you’re coming from Python, Lean can feel strange at first.

But once the mental model starts clicking, it genuinely changes how you think about programming.

This article is a beginner-friendly introduction to Lean 4 specifically for Python programmers.

The Biggest Difference: Lean Wants Precision

Python is flexible by design.

For example:

def add(a, b):
    return a + b

Python is happy with this.

But think carefully:

• What types are a and b?
• Integers?
• Strings?
• Lists?
• Custom objects?

Python delays those decisions until runtime.

Lean does the opposite.

In Lean:

def add (a : Nat) (b : Nat) : Nat :=
  a + b

This explicitly says:

• a is a natural number
• b is a natural number
• result is a natural number

At first this can feel verbose to Python developers.

But Lean’s philosophy is interesting:

Ambiguity is usually hidden complexity.

Lean tries to eliminate hidden assumptions.

Your First Lean Functions

Let’s compare Python and Lean side by side.

Python:

def square(x):
    return x * x

Lean:

def square (x : Int) : Int :=
  x * x

Very similar.

Now let’s try something slightly more interesting.

Python factorial:

def factorial(n):
    if n == 0:
        return 1
    return n * factorial(n - 1)

Lean factorial:

def factorial : Nat → Nat
| 0 => 1
| n + 1 => (n + 1) * factorial n

This looks unusual initially.

What’s happening here?

Lean uses pattern matching.

Explanation:

• if input is 0, return 1
• otherwise treat the number as n + 1
• recursively compute factorial

This style is extremely common in functional programming.

And importantly:

Lean can reason formally about recursive definitions.

Lean Cares Deeply About Correctness

Here’s a subtle Python issue:

def divide(a, b):
    return a / b

Looks harmless.

But:

divide(10, 0)

Runtime error.

Python lets dangerous states exist until execution.

Lean pushes you to think earlier.

For example, Lean separates different numeric systems carefully:

• natural numbers
• integers
• rational numbers
• finite values

That strictness is not accidental.

Lean assumes:

if something matters logically, it should be represented explicitly.

This is one reason theorem provers are powerful.

Immutable Thinking Feels Different

Python programmers often mutate state:

count = 0

def increment():
    global count
    count += 1

Lean strongly prefers immutable transformations:

def increment (n : Nat) : Nat :=
  n + 1

Instead of changing existing state:

• input goes in
• new value comes out

Why is this important?

Because mutable state becomes difficult to reason about formally.

Imagine trying to mathematically prove correctness of a large system where:

• variables change everywhere
• functions have hidden side effects
• execution order matters constantly

Functional programming reduces this chaos.

That’s one reason languages like Lean heavily embrace immutability.

Types in Lean Are Much More Powerful Than Python Type Hints

Python type hints help developers.

Lean types help prove things.

That’s a huge distinction.

Python:

def greet(name: str) -> str:
    return "Hello " + name

Helpful, but not enforced rigorously.

Lean types can encode logical guarantees.

For example, you can express:

• “this list cannot be empty”
• “this number is positive”
• “this function always terminates”

That’s where Lean starts feeling less like programming and more like formal reasoning.

A tiny theorem:

theorem add_zero (n : Nat) : n + 0 = n := by
  rfl

This proves:

adding zero changes nothing

Breaking it down:

• theorem declares a proposition
• (n : Nat) means for any natural number n
• : n + 0 = n is the claim
• by starts proof mode
• rfl means “this is true by direct simplification”

This may look trivial.

But Lean scales this exact mechanism to verify extremely sophisticated systems.

Lean Proofs Feel Like Interactive Debugging

One surprising thing about Lean:

Writing proofs often feels similar to debugging.

Suppose we define list length.

def length : List α → Nat
| [] => 0
| _ :: xs => 1 + length xs

Explanation:

• empty list has length 0

• otherwise:

  • ignore first element (_)
  • recursively compute remaining length

Now imagine proving something about lists.

Lean gives continuous feedback:

• current assumptions
• proof goals
• missing logical steps

This creates a very interactive workflow.

Python developers often enjoy REPL-driven development.

Lean has a similar “conversation with the system” feeling.

Except instead of debugging runtime behavior, you debug logical reasoning.

Tactics: Step-by-Step Proof Construction

Lean proofs are often built using tactics.

Example:

theorem reverse_reverse (xs : List Nat) :
  xs.reverse.reverse = xs := by
  simp

What does this mean?

It proves:

reversing a list twice returns the original list

And simp tells Lean:

simplify using known rules

Some useful beginner tactics:

• simp → simplify expressions
• rw → rewrite using known equalities
• intro → introduce assumptions
• exact → directly provide proof
• apply → use another theorem

Think of tactics like:

• debugging tools
• proof-building commands
• interactive reasoning helpers

Over time they become surprisingly intuitive.

Lean Quietly Improves Your Software Engineering

Even if you never become a theorem proving expert, Lean changes how you think.

You start asking:

• What assumptions exist here?
• What invariants should always hold?
• Can invalid states exist?
• Is this behavior guaranteed?

This mindset transfers back into ordinary engineering.

You often begin writing:

• cleaner APIs
• safer abstractions
• more predictable systems
• more explicit contracts

Ironically, learning Lean can improve your Python.

How Python Developers Should Learn Lean

A good beginner path:

1. Learn basic functional programming ideas

Focus on:

• recursion
• immutability
• pattern matching

2. Install Lean + VS Code

Using Visual Studio Code with the Lean extension gives excellent feedback.

3. Ignore advanced math initially

Many beginners get overwhelmed because online discussions quickly become abstract.

Start simple:

• functions
• lists
• recursion
• tiny proofs

4. Build tiny programs

Good practice ideas:

• reverse a list
• binary trees
• arithmetic functions
• list transformations

5. Expect gradual progress

Lean is closer to learning chess or mathematics than learning a typical framework.

The intuition develops slowly.

Conclusion

Lean 4 represents a very different philosophy of programming.

Python asks:

“Can we make development faster and more flexible?”

Lean asks:

“Can we make reasoning precise and verifiable?”

Both philosophies matter.

But as software systems become increasingly complex — especially in:

• AI
• infrastructure
• cryptography
• finance
• distributed systems

— the ability to formally reason about correctness may become far more important.

And perhaps that’s the most interesting thing about Lean:

It doesn’t just teach syntax.

It teaches disciplined thinking.

So here’s a question to leave you with:

If AI increasingly generates code automatically, do developers become more valuable not for writing code — but for verifying whether systems are actually correct?

That future may arrive sooner than we think.

And if it does, theorem provers like Lean may eventually move from research circles into mainstream software engineering.

And again — if you’re interested in developer tooling around engineering quality, check out git-lrc on GitHub — a free micro AI code review tool that runs on commits — and consider giving the project a ⭐ if you’d like to support it.

Originally published at mrnasdoggrowth.com by MrNasdog.

Most of what’s being written about AI right now is loud.

Loud about how AI will replace everyone. Loud about how AI changes everything. Loud about which AI tool to use this week. Loud about new model releases. Loud about productivity hacks.

The loud strategy is wrong. Not because AI doesn’t matter — it matters more than almost anything else right now. But because the people who’ll win in the next 20 years aren’t going to win by being louder. They’re going to win by being quiet, patient, and deep.

This article is about the quiet strategy. What it looks like, why it works, and how to live it.

Start with the question

The question I want to answer is simple: how do you build a life and career worth having in the next 10 to 20 years, when AI is reshaping everything?

This is a real question. People I talk to are worried about it. They should be. The world is changing fast.

But the answer isn’t to panic, and it isn’t to chase. The answer is to understand where AI is going, where AI structurally can’t go, and to build your life in the second category.

Where AI is going

AI is going to keep getting better at almost everything that involves processing information.

Writing standard documents. Coding common patterns. Synthesizing research. Analyzing data. Generating ideas. Translating. Summarizing. First-draft anything.

Most knowledge work is going to look very different in 10 years. Either AI will do it directly, or one person with AI will do what used to take a team. This isn’t a prediction — it’s already happening. Companies are quietly cutting roles. Solo operators are out-producing teams. The shift is well underway.

If your career is built on doing work AI can also do, you have a problem. Maybe not immediately, but within the planning horizon you care about.

Where AI structurally can’t go

Now the more interesting half.

There’s a class of work AI can’t do well, and won’t be able to do well for decades. This isn’t because AI is bad — it’s because the work itself has properties that don’t fit how AI works.

Think about how AlphaGo beat humans at Go. AlphaGo had three things going for it: it could generate millions of possible moves, it could test those moves quickly in self-play, and every game ended in a clean win or loss.

These three properties — generate, test, evaluate — are what made AI’s victory possible.

Now think about the questions in your actual life. What career to build. Who to spend your years with. What to work on. What kind of person to become.

You can generate possibilities. But you can’t test them in self-play — each possibility requires years of lived experience to evaluate. And there’s no clean win condition — what “winning at life” means is itself the question.

So for the deepest questions, AlphaGo’s approach doesn’t work. AI can list options for you. AI can analyze trade-offs. AI can summarize what worked for other people. But AI can’t tell you what’s right for you specifically, because the testing requires lived time that AI doesn’t have.

This is one structural limit. There are others.

AI’s training data is always behind the present. For fast-moving topics, AI is always slightly outdated. The gap between AI’s knowledge and current reality is real and ongoing.

AI’s best advice loses its edge when distributed. When AI gives you a great strategy, it gives the same strategy to everyone. Strategies that work because they’re rare stop working when they’re shared.

AI doesn’t have a body or a life. It can simulate possibilities but can’t experience consequences. The lived weight of having done something matters in ways AI can’t replicate.

Time-based outcomes can’t be compressed. Some things only become clear after decades of running. AI can model them but can’t accelerate them.

These four limits combined describe the space where humans still have structural advantages over AI. And they don’t move much in 10 or 20 years.

The shape of the quiet strategy

If you understand where AI is going and where AI structurally can’t go, the strategy follows.

You build in the space AI can’t easily enter.

What does that look like in practice?

Pick something specific and stay with it for years. Not weeks. Not months. Years. Deep expertise in a specific domain becomes more valuable as AI commodifies broad knowledge. The specialist beats the generalist as AI rises.

Build a public record over time. Whatever your work is, document it. Make it discoverable. Time-stamp it. The record itself becomes an asset that compounds. Someone with 10 years of public, traceable work in a specific field has something AI can’t replicate — because AI didn’t live those 10 years.

Use AI as a multiplier, not a replacement. Lean on AI for everything AI is good at. Don’t be precious. But don’t let AI make decisions only you can make. Strategy. Values. Direction. Trade-offs. These stay yours.

Take real positions with real stakes. Public predictions. Public investments. Public commitments. The track record matters because it can’t be faked. AI can hypothesize endlessly but can’t be publicly wrong with real consequences.

Pick fields where the present matters more than the past. AI’s training lag is a real disadvantage to AI in fast-moving areas. Stay current, and you stay ahead of what AI can know.

Accept the trade-offs. You have 24 hours a day. You can’t be everything. Pick what matters most to you and commit. The people who try to optimize everything lose to people who commit to specific things deeply.

What this requires

The quiet strategy is harder than the loud strategy in some ways and easier in others.

Harder because it requires patience. You won’t see fast results. You’ll spend years building things that don’t pay off until much later.

Harder because it requires commitment. You can’t keep switching strategies. The compounding only works if you stay with one thing long enough for it to actually compound.

Harder because it requires being unpopular sometimes. Public positions with real stakes mean being publicly wrong sometimes. The people who hedge to avoid being wrong end up with no track record at all.

Easier because it doesn’t require keeping up with every new AI tool release. You don’t need to be the most current. You need to be the most committed.

Easier because it doesn’t require maximum output. You don’t need to publish daily. You need to publish durably. One excellent piece of work that lasts 10 years beats 100 pieces of forgettable content.

Easier because the loud strategy is genuinely a trap. Most loud strategies in the AI era will produce nothing durable. The quiet strategy looks slow but actually wins.

The 20-year view

If you zoom out to 20 years, the picture clarifies.

The people who panicked about AI and tried to compete with it on AI’s terms will mostly lose. AI is too good at AI’s strengths.

The people who refused to use AI will also lose. The productivity advantage of AI-augmented work is too large to ignore.

The people who’ll do well are the ones who used AI heavily for what AI does well, while building patiently in the gaps where AI can’t go. They’ll look ordinary for years. They’ll look slow. They’ll look unimpressive next to people generating massive AI-assisted output.

But in year 15 or year 20, their lived expertise, their documented track record, their identity-anchored work, their patient specific compounding will be obvious. The fast people will have nothing durable to show. The quiet people will have something AI structurally cannot copy.

This is the strategy that actually works for the next 20 years.

It’s not flashy. It doesn’t go viral. It doesn’t lend itself to TikTok hot takes.

But it’s the real game.

Pick something specific. Commit to it for years. Use AI heavily. Build durable assets in the gaps. Take public positions with real stakes. Stay patient.

Do this for the next 10 to 20 years, and you’ll be one of the few people who actually built something while the rest were either replaced by AI or out-competed by people using AI better.

The quiet strategy is the answer. The hard part is having the patience to live it.

Enterprise RAG — A practitioner’s build log | Post 6 of 6

This series has documented a system built to a specific standard: one where access control is enforced before retrieval scoring, where every answer includes traceable citations, and where the evaluation set measures restricted document leakage rather than retrieval relevance alone.

This final post answers the question that matters most for teams considering this as a foundation: what works today, what needs to be in place before this handles real internal documents in a production environment, and what the gap between those two states actually looks like.

What is fully operational today

Every item below runs locally without external dependencies or provider credentials.

Document pipeline:

• Markdown document ingestion with front-matter role metadata (POST /ingest)
• SQLite metadata and chunk store with document and chunk count metrics
• Lexical retrieval with token cosine similarity scoring

Query and access control:

• Role-based candidate filter applied before retrieval scoring
• Citation-backed answer generation in deterministic mock mode
• RBAC_blocked_count logged per query — tracks how many chunks were filtered
• Role derivation from X-API-Key header, preventing request-body role elevation

API and authentication:

• FastAPI query API (POST /query) with health probes at GET /health
• Local user registration with role assignment (POST /auth/register)
• API key creation, listing, and revocation (POST /api-keys, POST /api-keys/{id}/revoke)
• SHA-256 key hash storage — raw key never persisted after creation
• Management endpoint protection via ADMIN_TOKEN

Evaluation:

• Evaluation runner via POST /eval/run — calls live query pipeline, not a mocked path
• Four metrics per run: pass rate, restricted leakage count, citation coverage, average latency
• Per-case results with expected vs. retrieved document IDs and pass/fail indicators

Operational controls:

• Audit log for all administrative actions (GET /audit-logs)
• Query log with citations, role, latency, and RBAC metrics
• Prometheus-style operational metrics endpoint
• Security headers enabled by default, CORS explicitly configured
• Structured JSON request logging (JSON_LOGS=true)
• In-memory rate limiting per client (RATE_LIMIT_PER_MINUTE)

Infrastructure:

• API-backed Streamlit dashboard — no direct database access from the UI
• Docker files for containerized runtime validation
• GitHub Actions CI workflow
• Azure AI Search retrieval adapter implemented and configuration-selectable
• OpenAI and Azure OpenAI generation adapters configuration-selectable

The Azure deployment path

The local runtime maps directly to an Azure deployment topology:

Employee → Microsoft Entra ID
↓
Azure Container Apps: API + Dashboard
↓
Azure AI Search (retrieval)
Azure OpenAI (answer generation)
Azure PostgreSQL or Cosmos DB (metadata + audit logs)
Azure Blob Storage (source documents)
↓
Azure Key Vault (secrets)
Application Insights (logs + metrics)

Switching from local to Azure requires environment variable changes only. No code changes. No schema migrations between SQLite and PostgreSQL — the SQLAlchemy layer handles both. Azure mode fails fast when required AZURE_* settings are missing rather than silently degrading to a local fallback.

What production deployment requires beyond the current implementation

Entra ID or OIDC role derivation from identity claims. The local implementation derives role from API key registration. Production deployment should derive role from authenticated identity token claims — not from request parameters or static key registration. The AUTH_PROVIDER=entra configuration path is implemented. End-to-end validation requires a live Azure tenant.

Semantic or hybrid retrieval. The local lexical retriever is deterministic and validates access control correctly. It does not match the retrieval quality of embedding-based semantic search for queries without token overlap with document chunks. Azure AI Search vector and hybrid query modes are the planned production retrieval path.

Distributed rate limiting. The in-memory rate limiter does not share state across multiple API instances. Horizontal scaling requires Redis-backed or API gateway rate limiting.

PII classification and retention policies. The reference document corpus is synthetic. Before ingesting real internal documents — HR records, finance reports, incident logs — the ingestion pipeline should classify content for PII, apply sensitivity labels, and enforce explicit data retention policies for stored queries and generated answers.

Tenant isolation. The current implementation is single-organization. A deployment serving multiple business units with strict data isolation between them requires a tenant isolation layer at the data model and query pipeline level.

Broader evaluation set. The current evaluation set is calibrated for access-control validation across a small synthetic corpus. A production evaluation set requires human relevance labels, answer correctness checks, and a regression threshold integrated into the CI workflow.

An honest assessment

Enterprise RAG demonstrates the architecture that matters for internal knowledge systems: pre-retrieval access control, citation-backed answers, and an evaluation standard that measures restricted document leakage. The local implementation is complete, testable, and fully reproducible without provider credentials.

The gap to production is real and specific. Entra ID integration, semantic retrieval, distributed rate limiting, PII handling, and tenant isolation are well-understood engineering problems with clear solutions. None of them require rethinking the core pipeline — the access control order, the citation model, and the evaluation structure remain intact.

For a team building an internal document Q&A system: the architecture here is worth adopting. The hardening list above is the production backlog, not a reason to start from scratch.

What I would implement next

The highest-impact single item is Entra ID role derivation in production. The entire value of pre-retrieval access control depends on the role being trustworthy. In a local environment with API key role binding, that trust is reasonable. In a production environment with hundreds of employees, role must come from an authenticated identity provider — not from a manually registered key that may become stale when someone changes teams or leaves the organization.

The concrete step: configure AUTH_PROVIDER=entra, map Entra group claims to retrieval roles, and validate that the role filter receives the correct role from the token rather than from the request body. That single change makes the access control guarantee durable against organizational changes.

One question for you

When an employee changes roles or leaves your organization, how quickly does your internal knowledge system stop serving them documents from their previous role? Is that enforced at the identity provider level or at the document system level?

This concludes the Enterprise RAG build log series.

Full series index

1. The Access Control Gap That Makes Most Enterprise RAG Systems Dangerous
2. How Enterprise RAG Is Structured: Why Access Control Comes Before Retrieval Scoring
3. Three Design Decisions That Shaped the Enterprise RAG Retrieval Pipeline
4. Four Metrics That Actually Tell You Whether Your Enterprise RAG Is Working
5. Security Controls in Enterprise RAG: Keys, Audit Logs, and the Hierarchy That Prevents Role Elevation
6. What Enterprise RAG Is Ready For Today and What Production Deployment Actually Requires (this post)