I gave Gemma 4 access to my Google Drive without sending a single file to the cloud

This is a submission for the Gemma 4 Challenge: Build with Gemma 4

What I Built

Gemma Drive is a local AI assistant for Google Drive that runs entirely on your laptop. Files extracted on-device, summaries generated on-device, questions answered on-device. The only data that crosses the network is the Drive API call that downloads the file you’ve explicitly chosen to analyze.

The problem it solves is straightforward: every “AI for your files” tool I’ve tried requires me to upload my data to someone else’s server. As a patent agent, that’s a non-starter. My (patent work) Drive is full of client work, invention disclosures, and prosecution files that aren’t going to a third-party LLM no matter how good the marketing copy is.

So I built one that doesn’t.

What it does:

  • Connects to Google Drive via OAuth with the narrow drive.readonly scope. Tokens stored locally in SQLite.
  • Lets you browse your Drive in a custom file browser, search by name, and add individual files or entire folder trees recursively.
  • Extracts text from PDFs, DOCX, XLSX, PPTX, Google Docs/Sheets/Slides, plain text, and markdown.
  • Generates per-file summaries with Gemma 4 E4B, then rolls those up into per-folder summaries.
  • Answers free-form questions across the entire workspace using the summaries as context — “what are the main themes across these folders,” “which folder is most relevant to patent prosecution,” that kind of thing.
  • Surfaces every unsupported file (images, audio, archives) in a dedicated transparency panel with its full metadata, so nothing gets silently dropped.

The stack: Ollama running Gemma 4 E4B with CUDA acceleration on an NVIDIA GeForce RTX 4070 Laptop GPU. Django + DRF for the backend (OAuth, Drive API, extraction pipeline, prompt orchestration). React + TypeScript + shadcn/ui for the frontend. SQLite for storage. No vector database, no Celery queue, no Postgres — the simplest stack that does the job, because for a single-user local tool, anything else is overkill.

Demo

The clip shows the full flow: adding a folder from Drive (which walks all subfolders recursively), clicking “Process workspace” to extract and summarize, then asking Gemma a free-form question about the contents. Total time on my hardware: about 14 seconds for this small example folder (featuring non-patent, non-client work related to conferences); a real workspace with 50+ files takes 1-3 minutes to fully process the first time, after which everything is cached.

Code

GitHub: https://github.com/KISSPatent/gemma-drive

Setup is one terminal session plus one Google Cloud project. The short version:

  1. Install Ollama and pull the model
  2. Clone the repo
  3. Create a Google OAuth client with drive.readonly scope (instructions in the README)
  4. Run Django (python manage.py runserver) and Vite (npm run dev)
  5. Open localhost:5173, connect your Drive, start adding folders

How I Used Gemma 4

I picked Gemma 4 E4B specifically, and the why matters because it represents a different kind of model-selection thinking than I’m used to.

When I’m building alpha versions of my RocketSmart platform, my default is to reach for the most powerful model available — get the capability right first, optimize later. For a local-first tool, you flip that. The first question is what can run at all on the target hardware, the second is what runs at conversational speed, and only then do you ask whether it’s smart enough.

Eighteen months ago, getting GPT-4-class “chat with my PDFs” quality meant renting a big cloud model or running a 30B–70B model on a workstation-class GPU. Today, a ~4B-parameter Gemma 4 E4B-class model can do local PDF QA at conversational speed on a consumer GPU using only single-digit GB of VRAM. My laptop has an NVIDIA GeForce RTX 4070 Laptop GPU — plenty capable, but a long way from the H100s and A100s people benchmark on. I needed a model that gave me real answers about real documents fast, not the smartest model that could grudgingly fit.

Why E4B was the sweet spot:

  • 4.5B effective parameters (8B with embeddings) in a Matformer architecture, quantized to ~6 GB at Q4_K_M. Fits comfortably in my GPU’s VRAM with room for the OS and browser.
  • 128K context window — large enough that I didn’t need a RAG layer for v1. Folder summaries fit in context for workspaces under a few hundred files.
  • Native multimodality in the weights (text + vision + audio + function calling). I’m only using text in v1, but the vision and audio paths are available one runtime upgrade away.
  • ~30 tokens/sec generation speed on my hardware — 11 seconds for a 300-token summary, 1-2 minutes to process a folder of 20 mixed-format files.

The summaries E4B produces are genuinely useful — not “good enough for a demo” but “good enough that I use the tool for actual work.” This is one of those moments in AI where the “best model for the job” stops being a synonym for “the biggest model you can afford” and starts being a real engineering tradeoff between speed, cost, capability, and where the workload runs.

How E4B powers each part of the pipeline:

  1. Per-file summarization. Each picked file’s extracted text gets sent to Gemma with a prompt asking for a 2-4 sentence summary focused on what the document is about, its purpose, and distinguishing names/dates/identifiers.

  2. Per-folder roll-up. The file summaries for each folder get fed back to Gemma as a single batch with a prompt asking for a paragraph describing the folder as a whole — common themes, key entities, what someone would use the folder for.

  3. Free-form Q&A. The user’s question gets prepended to all folder summaries plus all file summaries in a single prompt. The system message constrains Gemma to use only the provided context and to say so directly when the answer isn’t in the workspace.

The 128K context window matters here specifically — it means I can dump every summary in the workspace into a single prompt and let Gemma do the synthesis natively, without needing embeddings, vector search, or any retrieval layer. That keeps the architecture radically simple: extraction → summarization → ask. If folder counts grow past where context-stuffing works, I’ll add sqlite-vec and embeddings, but I’m not paying that complexity cost until I have to.

Three things I learned about running Gemma 4 in production (in the loose sense of “production”):

Ollama’s API doesn’t expose audio yet

E4B has native audio understanding in the weights. I tested it on Day 1, expecting to ship audio support in v1. Two failures:

{"role": "user", "content": "Transcribe this audio.", "audios": ["<base64 audio>"]}
{"type": "input_audio", "input_audio": {"data": "<base64 audio>", "format": "mp3"}}

That second error is interesting in a frustrating way — Ollama saw the non-text content block, routed it to the only multimodal pipeline it currently exposes (images), and tried to decode my MP3 as a PNG. The GitHub issue requesting audio support (ollama/ollama#11798) is still open. Audio works today through llama.cpp’s server directly with the mmproj file — I just didn’t want to add a second runtime for v1. The model has the capability; the runtime API doesn’t ship it yet.

Ollama silently truncates at 2,048 tokens by default

After my first round of summarization, I noticed the database had empty strings even though the endpoint reported success for every file. The culprit: num_ctx defaults to 2,048 tokens in Ollama unless you explicitly set it. That’s roughly 8,000 characters. My 11,000-character PDFs were getting silently truncated during inference, and the model returned empty content rather than partial answers.

The fix is one line:

"options": {"num_predict": 600, "num_ctx": 32768}

The default makes sense for chat-style interactions; it bites the moment you summarize anything longer than a short email.

Unsupported files belong in a transparency panel

Most AI-on-your-files tools quietly skip what they can’t handle. I built a dedicated panel that lists every file in the workspace whose MIME type isn’t in the extraction pipeline — with name, folder path, MIME type, modification date, and owner. It changes the implicit contract: instead of “this tool understands your Drive,” it’s “this tool tells you exactly what it understands and what it doesn’t.”

For my Drive specifically, the panel surfaces a hundred-plus JPGs of conference photos and a few audio recordings, as this Drive is for non-patent, non-client work. Once Ollama exposes audio and I add image handling for Gemma’s vision, those move into the supported list. Until then, they’re not hiding.

What’s next: audio and video, image-heavy PDFs (via Gemma 4’s vision once I add rasterization), RAG with sqlite-vec if context-stuffing stops working, streaming chat responses, and a “watch this folder” mode that auto-extracts on Drive changes.

The whole project is built on a specific claim: edge-class AI is now good enough that you don’t have to send your files to someone else’s cloud to get useful work out of an LLM. A week of building, four real engineering gotchas, and one working tool later — I think the claim holds up.

Long-running agents need more than memory

Anthropic’s managed-agent harness solves one hard problem: continuity. Progress logs, feature lists, git checkpoints, and startup scripts give each new session a map of what happened. But continuity is not governance. As agents work across more sessions, the question changes from “did the agent remember?” to “did the agent stay within its architectural constraints?”

In May 2026, Anthropic published a detailed look at how their internal engineering teams use Claude Code as a long-running managed agent. The infrastructure pattern they describe is worth reading carefully: initializer agents that prepare the workspace, feature lists that define remaining work, progress files that record what happened, git commits that preserve recoverable state, startup checks that orient each new session, and end-to-end tests that stop agents from declaring victory prematurely.

This is not prompt engineering. It is operational infrastructure for agents working across many sessions on the same codebase. The problems it solves are real, and the solutions are well-reasoned.

But the pattern solves continuity. It does not solve governance. Those are two different problems, and conflating them is the most expensive mistake a team can make when designing long-running agent workflows.

Agents as shift workers

The framing that makes the managed-agent pattern click is the relay team metaphor. A long-running agent workflow looks less like one developer with a prompt and more like a team of engineers handing work across shifts.

Each shift worker arrives, reads the handoff notes, picks up where the last person stopped, makes progress, and leaves a record for the next person. The work continues across interruptions. The codebase evolves across sessions. No single session owns the full context.

That framing makes the continuity infrastructure obvious. You need handoff notes that are authoritative (progress files), a work queue that persists across shifts (feature lists), recoverable state at every checkpoint (git commits), orientation scripts so each shift starts correctly (startup checks), and pass/fail criteria that the work must satisfy (E2E tests).

Anthropic’s harness provides all five. What it does not provide is the architectural contract that defines what kind of work each shift is allowed to do.

In a real engineering team, that contract exists in ADRs, architecture review boards, code review standards, and the accumulated institutional knowledge of senior engineers. In a long-running agent loop, none of that is automatically present. The harness tells the agent what happened. It does not tell the agent what must remain true.

What the harness gets right

Before addressing the gap, it is worth being precise about what the harness actually solves:

  • Initializer agent — prepares the workspace before the main agent session begins.
  • Feature list — a durable queue of remaining work, written as discrete, completable items.
  • Progress file — a running record of what each session changed, decided, and left incomplete.
  • Git commits as checkpoints — every meaningful unit of work lands as a recoverable commit.
  • E2E tests as the victory condition — agents cannot declare a feature complete until the tests pass.

The pattern is good engineering. Each piece of infrastructure corresponds to a real failure mode that long-running agents encounter in practice.

The remaining gap: continuity is not governance

A progress file can tell the next agent: “Here is what I changed.”

It cannot reliably tell the agent: “This architecture boundary must not be crossed. This dependency is forbidden. This ADR supersedes that older decision. This pattern is allowed only in this scope.”

That distinction matters in practice because the questions a progress file answers and the questions a governance layer answers are different in kind, not just degree:

Layer Question it answers
Progress log What happened?
Feature list What remains?
Git history What changed?
Test harness Does it work?
Governance layer Is this allowed?

The first four layers are all answered by the managed-agent harness. The fifth is not. A test suite can verify that the output is functionally correct. It cannot verify that the output is architecturally compliant. Those are different properties, and a codebase can be full of passing tests while being full of architectural violations.

Agent harnesses preserve continuity. Governance preserves intent.

Why this gets harder as agents run longer

Over many sessions, a long-running agent loop may:

  • Infer outdated patterns from old code. If earlier sessions used a deprecated pattern, the new session infers that pattern is correct and continues it.
  • Reintroduce forbidden dependencies. A dependency was removed for a documented architectural reason. A later session adds it back because it solves the immediate problem and the prohibition is not in any artifact the agent reads.
  • Bypass undocumented conventions. Architecture that exists in institutional memory but not in enforceable documents is invisible to the agent.
  • Optimize locally while violating system-level constraints. Each session makes a locally reasonable change. The cumulative effect crosses an architectural boundary that no single session was responsible for maintaining.

None of these failures show up in a progress file. None of them cause a test suite to fail. They accumulate silently across sessions and become visible only when the codebase is far enough from its architectural intent that the cost of correction is high.

The role of governance

Governance sits beside the harness. It does not replace progress logs, tests, or git. It gives the agent a deterministic way to check architectural compatibility at each session boundary and at each commit boundary.

The managed-agent startup sequence, extended with governance:

pwd
git log --oneline -20
cat claude-progress.txt
cat feature_list.json
mneme check --mode warn

Before commit or PR:

mneme check --mode strict

In CI, on every push:

mneme check --mode strict --ci

The framing is important: the harness tells the agent where it is. Governance tells it what boundaries it must respect. Both are necessary. Neither substitutes for the other.

ADRs as durable intent, not documentation

The governance layer requires a source of architectural authority. In well-run engineering teams, that source is the ADR corpus: Architecture Decision Records that capture not just what was decided, but why, what alternatives were rejected, and what constraints the decision implies.

For most teams, ADRs sit in /docs/adr and are read only when someone thinks to look. They are documentation, not enforcement. A long-running agent will not read them at session start. A commit hook will not check against them.

A governance layer changes this. Rather than reading the ADR folder as a documentation corpus, it compiles the ADR corpus into a decision graph with declared properties:

  • Which decisions are active, superseded, or deprecated?
  • Which decision applies to which file, service, or scope?
  • Which decision is newer and overrides an older one?
  • Which dependencies or patterns does each decision forbid or require?
  • When two decisions conflict on the same scope, which one wins?

A long-running agent operating under that system can answer: which decision applies to this change, and am I compliant with it? That is a different question from what did the progress file say? and it requires a different infrastructure to answer.

Where governance checkpoints belong

Governance is not a single check at a single moment. The right enforcement points correspond to the moments of highest leverage:

  1. Session start (warn mode) — before any code is written, load constraints and surface existing violations without blocking work.
  2. Pre-tool execution — block actions that are obviously forbidden before they happen.
  3. Pre-commit (strict mode) — the primary enforcement gate, catching architectural drift before it becomes branch history.
  4. Pre-PR — produces an explainable report of which rules applied, which passed, which failed, and why.
  5. CI — the backstop that enforces team-level architectural contracts on every push.

The harness ensures the agent knows where it is. Governance ensures the agent knows where it must not go. Both are infrastructure. Neither is a nice-to-have for long-running loops.

Conclusion: memory is not enough

Anthropic’s managed-agent harness is well-designed infrastructure for a real problem. Teams building on Claude Code or similar agent systems should study and adopt this pattern.

But a progress file is descriptive, not prescriptive. It records what happened. It does not enforce what must remain true. And as agent loops grow longer, the gap between those two things grows wider.

The next phase of agent infrastructure needs a governance layer — one that resolves competing ADRs deterministically, produces explainable audit traces, and enforces architectural contracts at the boundaries where agents make consequential changes.

Long-running agents need memory to continue work. They need governance to continue work safely. The next generation of agent infrastructure will not just preserve context. It will preserve intent.

That is the layer Mneme is built for.

Originally published at https://mnemehq.com/insights/long-running-agents-need-governance/

Why Code Review Cannot Scale With AI Output

AI coding assistants have made generation cheap. They haven’t made review cheap. The result is a compounding bottleneck that most engineering teams are only beginning to feel — and that no amount of hiring will resolve.

For most of software engineering history, writing code was the bottleneck. Senior engineers reviewed what juniors wrote, and the review burden was proportional to human writing speed. The ratio was manageable.

AI coding assistants break this assumption. When a single engineer can generate a thousand lines of plausible, compilable code in an hour, the bottleneck shifts. Generation is no longer scarce. Review is.

The volume problem is already here

Teams that have adopted AI coding assistants at scale — Claude Code, Cursor, Copilot, Devin — consistently report the same pattern: PR volume increases faster than review capacity. A single AI agent operating on a well-scoped task can produce a multi-file changeset in minutes that would take a human engineer half a day to write.

The arithmetic is straightforward. If your team generates 10x more code, someone still has to review 10x more code. You cannot hire 10x more reviewers to compensate — even if you could find them, reviewers themselves become AI-assisted and generate more output. The bottleneck tightens.

Reviewing AI output is harder than reviewing human output

The volume problem would be difficult enough. But AI-generated code is also harder to review than human-written code in several specific ways.

No institutional memory between sessions. A human engineer writing a service that interacts with the payments pipeline carries context from prior code reviews, architecture discussions, and incident postmortems. An AI agent starting a new session has none of this unless it’s explicitly provided. The result is code that is syntactically correct and passes tests but violates architectural invariants that aren’t written down anywhere the model can see.

Plausible violations are harder to catch than obvious ones. Human engineers tend to violate architectural rules either obviously or not at all. AI agents produce a third category: code that looks architecturally correct but violates a constraint in a subtle way — a service reaching across a boundary via a shared utility function, or a new table added to a database that was supposed to be read-only from this service. These violations pass automated tests and linters. They require a reviewer who understands the architectural intent.

Style and convention drift. AI coding assistants have their own implicit style conventions, drawn from training data rather than your codebase’s evolution. Without explicit constraint injection, they drift toward generic patterns — which means reviewers must also police style consistency that previously self-enforced via team osmosis.

The “just review more carefully” response doesn’t work

The intuitive organizational response is to tighten review requirements: require two reviewers, require an architect sign-off on structural changes, institute more thorough checklists. This approach fails for a predictable reason: it reduces velocity precisely when AI-assisted development is supposed to be increasing it.

The fundamental tension: tighter review processes reduce the speed advantage that AI coding provides. Looser review processes allow architectural violations to compound. There is no review-process solution to a generation-speed problem.

Teams that try to solve this with review tooling — AI-assisted code review, static analysis, architecture rule checkers — observe partial improvement. These tools can catch mechanical violations: undefined variables, type errors, obvious anti-patterns. They cannot catch violations that require understanding your team’s specific architectural decisions.

The shift-left argument

Security engineering confronted a structurally identical problem a decade ago. When application development accelerated and security testing was relegated to the end of the pipeline, the volume of vulnerabilities reaching production exceeded the security team’s capacity to address them. The response was the shift-left movement: move security checks earlier in the development process, so violations are caught before they accumulate.

The same logic applies to architectural governance. If you move constraint enforcement to before the AI agent writes the file — rather than after the PR is opened — you eliminate the violation before it needs to be reviewed. No review time consumed. No back-and-forth on the PR. No accumulated drift.

What pre-generation enforcement actually requires

Shifting enforcement left is the right direction. Implementing it correctly requires more than telling the AI agent “follow our rules” in the system prompt.

Effective pre-generation enforcement needs:

  • A structured decision corpus — architectural decisions captured in a machine-readable schema, not free-form documentation. Decisions must have explicit scope, status, and constraint fields.
  • Scope-aware retrieval — the ability to retrieve only the decisions relevant to the specific file or module being modified, not a semantic-similarity approximation of what might be relevant.
  • Hook-level integration — enforcement must happen at the tool-use layer, before the write completes, not in the prompt or post-hoc in review.
  • A precedence engine — when multiple decisions apply, the system must resolve conflicts deterministically rather than leaving the model to interpret contradictions.

None of these requirements are met by a system prompt containing your ADR documents, or by a RAG pipeline that retrieves them. They require a governance architecture that treats decisions as structured, executable constraints rather than advisory text.

The cost of not solving this

Teams that adopt AI coding assistants without addressing the review bottleneck converge on one of two failure modes:

Velocity collapse — review requirements tighten to the point that AI-generated PRs queue for days, negating the generation speed advantage.

Architectural debt accumulation — review is loosened or overwhelmed, violations merge, and the codebase drifts away from its intended architecture over months.

Both outcomes are predictable. Both are avoidable if the governance problem is addressed at the generation layer rather than the review layer.

The structural conclusion

Code review is a human-time-bounded process. AI code generation is not. You cannot solve a generation-speed problem with a review-speed solution. The governance layer must operate at generation time, enforcing architectural constraints before the code is written — not after it’s merged.

This is the architectural shift that the current generation of AI coding tools hasn’t yet made. It’s also the gap that Mneme is built to close.

Originally published at https://mnemehq.com/insights/why-code-review-cannot-scale-with-ai-output/

Harness Engineering Still Needs Governance

The industry has moved from prompt engineering to harness engineering: execution systems that coordinate models, tools, memory, and retries across long-running agent loops. Harnesses solve how agents act. They do not solve whether the actions stay within architectural boundaries. As autonomous workflows scale, the missing layer is governance infrastructure.

The shift: prompts, harnesses, execution systems

Three years ago, the prevailing question was how to phrase a prompt. The model was the product, and prompt engineering was the surface where teams competed.

That framing no longer matches what is being built. OpenAI’s recent harness work on Codex, Anthropic’s pattern for long-running managed agents, Cursor’s background-agent runtime, Claude Code’s session-aware workflow, and the open-source frameworks — LangGraph, CrewAI, AutoGen — all converge on the same architectural move: the model is wrapped inside an execution system that handles tools, memory, retries, planning, and continuity.

Single calls have given way to execution loops. Assistants have given way to autonomous workflows. The model is no longer the product boundary. The execution environment is.

That is the right move. Anyone building agentic systems in 2026 needs a harness. But it is also incomplete. As soon as agents are doing real work across many sessions, on real codebases, the harness reveals what it does not cover: architectural intent.

What harness engineering solves

Harness engineering handles context lifecycle management, tool orchestration, retries and error recovery, planning and execution loops, memory injection and continuity, and observability and execution coordination.

This is a major architectural advancement over prompt engineering. Treating the execution environment as the product boundary lets autonomous workflows survive contact with real systems: rate limits, partial failures, long horizons, multi-tool coordination, and cross-session continuity.

None of what follows is an argument against any of that. The argument is that this layer alone is not enough.

The governance gap

A harness can make an agent faster, more persistent, more autonomous, and more capable. It cannot, by construction, make the agent architecturally aligned. Continuity is not constraint. Orchestration is not enforcement. Memory of past decisions is not authority to refuse a future one.

The failures look like this:

  • An ADR is bypassed. The repo has a recorded decision — “do not introduce a runtime ORM” — that the agent does not read at session start. The agent introduces an ORM because it solves the immediate ticket.
  • A forbidden dependency reappears. A package was removed for a documented reason. A later session reintroduces it because the prohibition lives only in a stale doc, not in an enforcement hook.
  • A governed system is rewritten. The agent refactors a module that had a specific layering contract. The new version is functionally equivalent and passes tests, but violates the layering rule that was the entire point of the original design.
  • Layering boundaries are crossed. A controller starts calling into a data layer that the architecture forbids it from touching directly.
  • Naming conventions drift. Each session is internally consistent. Across sessions, the naming gradually changes.
  • Infrastructure patterns mutate. A standard for how services are exposed, configured, or deployed is silently replaced by a sensible-looking alternative that the rest of the system does not expect.

None of these failures are caused by the harness being bad. They are caused by the harness being the wrong place to enforce architectural decisions. The harness’s job is to keep work moving. Its incentives are continuity and throughput, not refusal.

Harnesses preserve execution continuity. They do not preserve architectural intent.

Why observability is insufficient

The most common response to the governance gap is to lean harder on observability. Trace every tool call. Log every diff. Pipe agent activity into a dashboard. If we can see what the agent did, we can correct it.

That argument confuses two different questions.

Observability answers what happened. Governance answers what should have been allowed. These are not the same problem.

  • Logs are not policy. A log records that a forbidden dependency was added. It does not refuse the add.
  • Traces are not invariants. A trace shows the call graph. It does not declare which call graphs are valid.
  • Visibility is not enforcement. A dashboard surfaces drift after it occurs. It does not block the change that produced the drift.

Observability is necessary — you cannot govern what you cannot see — but it sits on the wrong side of the action. By the time the trace reaches the dashboard, the commit has already happened. Governance has to sit in front of the action it constrains, with a deterministic rule about whether to allow it.

Governance propagation across execution surfaces

Long-running, autonomous agents do not only write source code. They write everywhere the workflow touches:

  • Branch names and PR titles — auto-generated by the harness, often outside the team’s branch and title taxonomy.
  • Commit messages and tags — workflow-generated commits accumulate in history.
  • CI metadata and pipeline config — written by agents the same way they write code, but with stricter governance constraints.
  • Deployment artifacts and release notes — manifests, container tags, generated changelogs.
  • Generated configuration — feature flags, routing rules, scaling policies.
  • Agent-produced documentation — READMEs, ADR drafts, runbooks that become the next agent’s training context.

Governance must propagate across every surface touched by autonomous execution. A governance layer that enforces ADR compliance in src/ but ignores commit messages, PR titles, CI config, and generated docs is governing a fraction of the agent’s output.

The next layer: governance infrastructure

The clean way to think about the emerging stack is to stop treating it as a model-plus-tooling problem and start treating it as a layered system:

Models           — produce candidate output
  ↓
Harnesses        — coordinate execution, retries, tools
  ↓
Execution        — long-running loops, sessions, memory
  ↓
Governance       — defines and enforces architectural constraints
  ↓
Verification     — tests, builds, deploy-time checks

Each layer answers a question the layer above it cannot:

  • Harnesses answer how does the agent act?
  • Execution systems answer how does the agent keep working across time?
  • Governance answers which actions are allowed, and according to which decisions?
  • Verification answers did the resulting system still pass its objective checks?

Governance is its own layer because the problem it solves is not solvable inside any of the others. Models produce text. Harnesses coordinate. Memory recalls. None of them can deterministically resolve which ADR governs a given change, or block output that violates the active decision graph.

This is what makes governance infrastructure a category, not a feature. It cannot be folded into the harness without giving the harness an enforcement responsibility that conflicts with its continuity responsibility. It cannot be folded into observability without losing its blocking authority.

Where Mneme fits

Mneme is a deliberately narrow layer in this stack. It does not orchestrate tools. It does not retry calls. It does not manage memory or context. It does one thing: it compiles the repository’s ADR corpus into a deterministic decision graph and enforces it at the boundaries where agents make consequential changes.

  • Repo-native. ADRs live in the repository. The decision graph is rebuilt from them on every check.
  • Deterministic enforcement. Given the same decision graph and the same change, the result is the same every time.
  • Governance before generation. mneme check --mode warn at session start tells the agent which decisions are active before it writes anything. mneme check --mode strict at pre-commit and CI is the enforcement gate.

Harnesses help agents act. Governance ensures they act within architectural boundaries. Verification confirms the result. All three layers are infrastructure. None of them substitute for the others.

Long-running agents need more than memory and orchestration. They need enforceable architectural boundaries. The next phase of agent infrastructure is the layer that provides them.

Originally published at https://mnemehq.com/insights/harness-engineering-still-needs-governance/

Review Is Not Governance

CodeRabbit helps review AI-generated code. Mneme helps govern what the AI generates in the first place. These are not competing tools. They are different layers of the same problem.

Most teams using AI coding assistants have invested in the review layer. Tools like CodeRabbit, PR comment bots, and AI-assisted linters sit at the end of the pipeline, inspecting output after it surfaces in a pull request. That is useful work.

But it is reactive by design. You are reading the result of a generation process that already happened. The architectural violation, the naming drift, the deprecated dependency — they are already in the diff. Now you are deciding whether to accept them or ask for a rewrite.

Governance operates differently. It runs before generation. It takes the architectural decisions your team has made — which services own which data, which dependencies are approved, which patterns are preferred — and makes them available to the AI assistant at the moment it is about to write code. Constraints are enforced at the source, not caught in review.

The distinction matters at scale. When AI multiplies code output 10-50x, catching violations after generation means reviewing the same class of problems over and over. Preventing them before generation means they stop appearing.

Two layers, one stack

The AI coding stack is evolving into distinct layers:

Generation runtimes like Cursor and Claude Code accelerate output. They are the engine.

Review platforms like CodeRabbit inspect what surfaces in pull requests. They are the quality gate.

Governance systems enforce architectural constraints before code is generated. They are the upstream layer that most teams are only beginning to build.

These layers are complementary. A team with strong generation tooling and strong review tooling but no governance layer will ship fast and review often — and still accumulate architectural drift at AI velocity. A team that adds the governance layer stops correcting and starts directing.

When intent only enters the workflow at PR review, the organization has already allowed the wrong work to be generated. That is the core failure mode of intent debt in AI-native engineering: decisions that exist but are not enforced at the moment agents produce code.

Why governance is infrastructure, not a feature

Security engineering went through this same evolution. Code security was once a review concern: scan the output, flag the vulnerabilities, ask for fixes. Shift-left security moved those checks earlier — into the IDE, into the CI pipeline, into the generation process itself. The industry now treats security tooling as infrastructure, not an optional review enhancement.

Architectural governance is at the same inflection point. As AI dramatically increases the volume of code entering a codebase, the constraints that govern that code must be embedded earlier in the process. Review catches what slips through. Governance determines what gets generated in the first place.

That is the layer Mneme is built for.

Originally published at https://mnemehq.com/insights/review-is-not-governance/

How I Built a Privacy-First Word Counter Chrome Extension (with CJK Support)

I’ve always been frustrated with online word counters.

You copy text, paste it into a random website, and hope it doesn’t store your content or track you. For writers, editors, and students who work with sensitive drafts, that’s a real problem.

So I built Instant Word Counter — a Chrome extension that counts words, characters, and reading time instantly, offline, and 100% locally. No text ever leaves your device.

Why another word counter?

Most existing extensions either:

  • require copy-pasting,
  • use clunky right-click menus,
  • or send your text to external servers for processing.

I wanted something faster and truly private.

How it works

  1. Select any text on a webpage.
  2. Click the extension icon (or press Alt+W).
  3. See word count, character count (with/without spaces), and estimated reading time in a clean popup.

If no text is selected, the extension gently guides you. No clutter, no confusion.

CJK character counting

One feature I’m especially proud of: the extension correctly handles Chinese, Japanese, and Korean (CJK) characters. When CJK characters are detected, it automatically switches from “Words” to “CJK Characters” — something most word counters get completely wrong.

It uses the Intl.Segmenter API for word segmentation and falls back to a regex-based counter for older browsers.

Tech stack

  • Vanilla JavaScript (no frameworks)
  • Chrome Manifest V3
  • Intl.Segmenter for accurate word counting
  • chrome.i18n for localization (10 languages so far)
  • All processing happens locally — no backend, no analytics, no tracking

Privacy by design

  • No data collection
  • No network requests
  • No cookies
  • The extension only requests activeTab and scripting permissions — nothing else

What I learned

  • Intl.Segmenter is powerful but still limited for mixed-language texts
  • Chrome Web Store localization is surprisingly tricky (detailed descriptions must be added manually per language)
  • Users genuinely care about privacy — several early reviews mentioned it as the main reason for installing

Try it out

Instant Word Counter is free and available on the Chrome Web Store.

Happy to answer questions or hear your feedback in the comments!



Deep Dive into OpenCode Agent Orchestration

The evolution of AI coding assistants has rapidly shifted from single-prompt chat interactions to autonomous, multi-agent systems. At the forefront of this movement is OpenCode, a terminal-native AI engine built to read, write, test, and debug code directly within your local environment.

While standard AI tools handle simple, isolated edits, solving complex software tickets—such as a multi-layered codebase refactor, writing matching integration tests, or updating complex deployment pipelines—requires orchestration. By dividing responsibilities into Agents, Sub-agents, Tools, and Skills, the OpenCode ecosystem provides an enterprise-ready blueprint for true software autonomy.

The Four Pillars of Agentic Autonomy

To build an automated workflow, you must understand how its core layers pass context and execute logic.

1. Primary Agents (The Project Leads)

Primary agents are the high-level controllers that you interface with during a terminal session. They maintain the overarching goal of the task and map out the step-by-step strategy.

  • Build Mode: The default primary agent. It operates with full tool write privileges (file operations, system terminal access) and is optimized for heavy implementation.
  • Plan Mode: A restricted, read-only primary agent. It is designed purely for architectural analysis, brainstorming, and code review. It defaults to “ask before writing,” ensuring it won’t alter your filesystem while plotting a migration strategy.

2. Sub-agents (The Specialized Contractors)

A primary agent’s main bottleneck is its context window; loading massive dependency files or heavy documentation can cause the model to lose track of the core objective. Sub-agents are temporary, highly isolated assistants spun up to execute highly focused micro-tasks.

  • @explore: A lightning-fast, read-only sub-agent built solely to navigate large codebases and locate files or specific structural patterns.
  • @scout: A dedicated research sub-agent that safely clones external repositories or pulls down upstream documentation into a managed cache, cross-referencing logic without cluttering your local environment.

3. Custom Tools (The Hands)

Tools are the deterministic functions and shell hooks that bridge an agent’s reasoning loop with your physical system. When an AI generates a structured command, the underlying harness converts it into a concrete action, such as executing localized scripts or custom database query checkers stored inside your project’s .opencode/tools/ folder.

4. Custom Skills (The Blueprints)

While tools are functional mechanisms, Skills represent specialized domain knowledge. Defined via the cross-platform Agent Skills Open Standard, these are structured SKILL.md markdown files containing YAML frontmatter that teach agents how to execute a workflow according to specific rules.

Real-World Architecture: The Automated Garage Platform

To see how these four pillars interact outside of theoretical abstractions, consider a real-world project: an Automated Garage & Maintenance Platform. This custom local stack is designed to track vehicle telemetry, manage analytics dashboards on Google Cloud, and orchestrate heavy vehicle detailing logs.

Instead of relying on a generic LLM that might mix up infrastructure code with chemical equations, the environment is orchestrated using custom OpenCode components.

       [User Prompt]
             │
             ▼
     @garage-lead (Agent)
             │
      ┌──────┴────────────────────────┐
      ▼                               ▼
@gcp-provisioner (Sub-agent)    @detailing-planner (Sub-agent)
      │                               │
      ├─► [cloud-native-standards]    ├─► [surface-prep-guidelines]
      │   (Skill)                     │   (Skill)
      │                               │
      └─► terraform_apply()           └─► query_inventory()
          (Tool)                          (Tool)

The Custom Setup

  • The Custom Agent (@garage-lead): The master coordinator. It is injected with a high-level system prompt via AGENTS.md to understand vehicle diagnostics and infrastructure boundaries, routing incoming tasks to specialized sub-agents.
  • The Custom Sub-agents: * @gcp-provisioner: A sub-agent restricted entirely to the /infrastructure directory, tasked with handling cloud deployments.

    • @detailing-planner: A domain-specific sub-agent engineered to sequence vehicle restoration, paint protection steps, and chemical ratios.
  • The Custom Tools: Bespoke local scripts exposed to the AI, including read_obd2_telemetry() (extracting temperature and exhaust errors from a local diagnostic database), query_inventory() (checking active stocks of parts and detailing products), and a restricted terraform_apply() hook.

  • The Custom Skills: Bound via localized markdown files to inject strict operational guardrails:

    • cloud-native-standards: Forces the infrastructure sub-agent to ensure any new cloud services are isolated to internal traffic and use proper service accounts.
    • surface-prep-guidelines: Hardcodes strict domain-specific physical rules. It instructs the agent that V-Floc is a neutral pH shampoo (never to be used as an all-purpose cleaner/APC) and explicitly dictates that V-04 and Sinergy are entirely different products, preventing chemical layering hallucinations.

The Execution Loop

When a user inputs: “Check the vehicle telemetry, provision a dashboard for the data, and build a detailing checklist for the weekend,” the orchestrator executes a multi-threaded autonomous loop:

  1. @garage-lead triggers read_obd2_telemetry() to assess the vehicle’s cooling and exhaust metrics.
  2. The agent dispatches @gcp-provisioner, which reads the cloud-native-standards skill and uses terraform_apply() to safely stand up an internal analytics dashboard on Google Cloud.
  3. Simultaneously, @detailing-planner wakes up, calls query_inventory() to check available products, pulls the surface-prep-guidelines skill, and generates a precise step-by-step cleaning log—ensuring the interior APC teardown and the neutral pH exterior wash happen in the exact sequence required to avoid material damage.

OpenCode vs. Claude Code: Quick Comparison

For engineering teams evaluating terminal-native automation, OpenCode and Anthropic’s Claude Code share similar design principles, but their execution philosophies differ:

Architectural Component OpenCode Framework Claude Code CLI
Model Integration Fully model-agnostic; supports 75+ cloud and local engines (Ollama, OpenRouter). Proprietary ecosystem; heavily optimized for Anthropic’s native Claude models.
The Skills Standard Natively implements the cross-platform Agent Skills Open Standard (SKILL.md). Developed the initial Agent Skills Open Standard for progressive token context reduction.
Tool Execution Executes localized scripts, binary hooks, and raw shell commands out of the box. Integrates heavily with the Model Context Protocol (MCP) to talk to local or remote servers.
Task Isolation Relies on community plugins or wrapper workspaces (e.g., Superset) for multi-branch tasks. Natively splits complex tasks into parallel executions using automated Git worktrees.

Summary

Modern software automation is shifting away from simple text completion toward structured agent networks. As demonstrated by the automated garage platform, separating high-level strategy (Primary Agents) from isolated research (Sub-agents), and pairing execution mechanics (Tools) with architectural rules (Skills) allows developers to step back from manual code writing and step into the role of a systems manager over a highly efficient AI factory.

Creating SM4-Encrypted Tables in GBase 8a and Managing Encryption Keys

GBase 8a MPP Cluster supports the SM4 national cipher for transparent data encryption at the storage level. To create an SM4-encrypted table, you follow a three-step process: pick the algorithm, create a certificate, and define the table. The encryption key is managed through a centralized certificate and distributed as files across all cluster nodes.

Step 1: Choose the SM4 Algorithm

Set the cluster-wide encryption algorithm to SM4. Once set, it cannot be changed.

SET GLOBAL gbase_encrypt_new_mode = 'SM4';

Step 2: Create an Encryption Certificate

Certificates can be plain or ciphered. Ciphered certificates require a password and offer stronger security.

CREATE ENCRYPTION CERTIFICATE my_sm4_cert
    TYPE = CIPHERED
    PASSWORD = 'YourStrongPassword123!';

The certificate file is automatically distributed to every node’s $GBASE_HOME/config/ directory.

Step 3: Create the Encrypted Table

Use the ENCRYPT keyword for table-level or column-level encryption.

-- Table-level: encrypts all columns
CREATE TABLE encrypted_whole_table (
    id INT,
    sensitive_data VARCHAR(200),
    create_time DATETIME
) ENCRYPT;

-- Column-level: encrypts only specific columns
CREATE TABLE customer_info (
    user_id INT,
    name VARCHAR(100),
    id_card_no VARCHAR(20) ENCRYPT,
    phone VARCHAR(20) ENCRYPT,
    email VARCHAR(50)
);

Important: Encryption must be defined at table creation time; it cannot be added or removed via ALTER TABLE.

Using Ciphered Certificates

Before running DML on encrypted columns, you must open the certificate:

OPEN ENCRYPTION CERTIFICATE my_sm4_cert PASSWORD = 'YourStrongPassword123!';

Closing the certificate will cause all subsequent DML on encrypted columns to fail.

Key Management and Storage

  • Key generation & storage: The key is generated when the certificate is created and stored inside the certificate file. A copy of that file lives on every cluster node.
  • Certificate types: Plain (no password) or Ciphered (password-protected). You can convert between them.
  • Management tasks: Backup the certificate file, change the password, query certificate status via information_schema.encryption_certificate_status, and open/close the certificate as needed.
  • Security model: One cluster‑wide certificate. For ciphered certificates, the password is the user’s responsibility — the system does not store or recover it. Protect the certificate file with OS permissions (e.g., chmod 600) and ensure the password meets the cluster’s password policy.

Important Restrictions

  • Once the encryption algorithm is set, it cannot be changed.
  • Encrypted tables do not support DBLink, and encrypted columns cannot be used as distribution keys.
  • The performance overhead is minimal — typically less than 5%.

This design gives you strong, transparent encryption for your gbase database while keeping key management clean and compliant with enterprise security requirements. If you’re storing sensitive data in GBASE’s MPP platform, SM4 encryption is a powerful tool to have in your security toolkit.

When to Choose On-Premises vs. Cloud for Vector Databases

Image 1: GDPR fines accumulated cost

For most of the last decade, the on-premises vs. cloud debate felt settled. Cloud computing was cheaper, faster, and easier to adopt. Enterprises moved workloads from on-premises infrastructure to public cloud services, relying on major cloud providers to handle scalability, maintenance, and security.

In 2026, that assumption is breaking, and cracks are showing up in legal reviews, financial projects, and SLA negotiations. Enterprises are facing an increasing pressure for data residency regulations, stricter enforcement, and scrutiny around cloud security models. Compliance constraints, data security requirements, cost predictability, and latency are forcing teams to reconsider on-premises solutions, private cloud computing, and hybrid cloud infrastructure.

At the same time, AI is moving closer to where data is generated. Manufacturing sites, retail stores, and healthcare environments increasingly require offline capability and sub-100ms latency. That shift helps explain why Oracle released AI Database 26ai for on-premises deployment and why Google is pushing Gemini onto Distributed Cloud for air-gapped environments. This shift signals that large-scale enterprise AI no longer fits neatly in cloud environments.

In this article, we’ll examine why on-premises infrastructure is resurging, what trade-offs you need to know, and how to make defensible deployment decisions.

What’s Driving The On-premises Resurgence

The renewed interest in on-premises infrastructure is not about going back to old systems. It is a response to clear changes in how AI systems are being built and used in 2025 and 2026. For many enterprises, cloud-only vector databases no longer fit their compliance, cost, and reliability needs.

A lot of factors drive this current on-premises resurgence, but in this article we will consider four key causes.

*Large vendors now support on-premises AI
*

On-premises AI is no longer treated as an edge case by major vendors. Oracle’s release of AI Database 26ai and Google’s decision to run Gemini on Distributed Cloud show a clear shift in how enterprise AI is being packaged and delivered.

These products are built for large enterprises, not early-stage experiments or research projects. That distinction matters. Large vendors do not invest in complex on-premises AI platforms unless there is strong and growing customer demand. These announcements confirm that many enterprises want to run AI systems inside their own environments, close to their data, and under their full operational control. Why is this?

Regulatory pressure is now a real blocker

Teams used to plan for regulatory risk as a future possibility. Now it’s a day-to-day reality. GDPR enforcement reached record levels in 2025, with insufficient legal basis for data processing driving the largest penalties. That year alone, regulators issued nearly 2,700 fines totaling billions of euros.

From a data security perspective, GDPR enforcement has fundamentally changed how enterprises evaluate cloud services. While cloud service providers offer compliance tooling, legal teams are increasingly wary of relying on third-party providers for sensitive data storage and processing.

Image 1: GDPR fines accumulated cost

Image 2: Total GDPR fines

HIPAA adds another layer of complexity. For example, in Florida, physicians must maintain medical records for five years after the last patient contact, whereas hospitals must maintain them for seven years under state record-retention requirements. This makes repeated data movement risky and expensive. Financial services and government contractors face similar data sovereignty requirements that limit where data can be stored and processed. In these situations, cloud deployments add legal review, audit work, and ongoing risk. Keeping data on-premises is often the most straightforward way to meet these obligations.

Edge AI requires local and offline operation

AI workloads are increasingly deployed close to where data is created. Manufacturing facilities may operate in air-gapped environments or remote locations with limited connectivity. Retail systems must continue working during network outages. Healthcare applications often require very low latency for real-time decision support.

In these environments, relying on a remote cloud service introduces risk. Network delays and outages directly affect system reliability. On-premises and edge deployments allow vector search and inference to run locally, without depending on constant network access. For many use cases, this local execution is not an optimization but a requirement.

Together, these shifts explain why on-premises vector databases are gaining traction again. The change is driven by the practical realities of deploying production AI systems under real regulatory, cost, and reliability constraints.

The Compliance Calculus

For many enterprises, compliance is the deciding factor in the on-premises versus cloud debate. While cloud providers offer compliance certifications, the real challenge is not whether a platform can be compliant in theory, but whether it can withstand legal review, audits, and long-term operational scrutiny in practice. Once vector databases move into production and begin storing sensitive or regulated data, these questions become unavoidable.

GDPR and the limits of cross-border transfers

The Schrems II ruling changed how European data can be processed outside the EU. Privacy Shield was invalidated, leaving Standard Contractual Clauses as the primary legal mechanism for cross-border data transfers. In highly regulated industries such as financial services and healthcare, many legal teams consider SCCs insufficient due to enforcement uncertainty and ongoing legal challenges.

For vector databases, this matters because embeddings often contain derived personal data. Even if raw records are masked or tokenized, embeddings can still be considered personal data under GDPR. If data must remain within the EEA, or within a specific country, cloud deployments that rely on global infrastructure introduce legal risk. In these cases, on-premises or in-region deployment becomes a requirement rather than a preference.

HIPAA retention and the real cost of data movement

HIPAA does not explicitly require data to stay on-premises, but it does require long retention periods and strict access controls. When vector embeddings are built on top of this data, they inherit the same retention requirements. HIPAA data governance must be enforced when considering on-premises or cloud vector databases.

The cost impact becomes clear when egress fees are included. Consider a system storing 100 TB of embeddings in a cloud environment. At a common egress rate of $0.09 per GB, moving that data out of the cloud over a seven-year retention period results in:

100 TB × $0.09 per GB × 84 months = over $750,000 in egress costs alone

This does not include compute, storage, or indexing costs. With this in mind, will cloud data warehouses really help you cut costs?

Financial services and data sovereignty rules

Financial institutions face additional constraints beyond GDPR. Regulations such as GLBA, APRA, and regional data sovereignty mandates often require strict control over where customer data is stored and processed. Regulators may demand clear evidence of geographic boundaries, access controls, and auditability.

Cloud services can meet some of these requirements, but they often introduce complex configurations, contractual dependencies, and ongoing compliance reviews. For many banks and insurers, on-premises deployment simplifies audits by keeping data within controlled infrastructure that regulators already understand.

Government and public sector constraints

Government contracts introduce some of the strictest infrastructure requirements. Standards such as FedRAMP often mandate US-only infrastructure, restricted access, and tightly controlled environments.

In these cases, public cloud services are frequently disallowed or require extensive approvals. On-premises deployment is often the only viable option for running vector databases in support of government workloads.

When compliance makes cloud untenable

If legal teams flag cross-border data transfers as unacceptable, cloud deployments quickly become impractical. Once data residency is mandatory, on-premises deployment is no longer a trade-off decision. It is a compliance requirement.

Image 3: Compliance framework

The Cost Breakdown Analysis

Cost is often the reason teams revisit the on-premises versus cloud decision. To make a defensible decision, teams need to understand where costs diverge and when self-hosting becomes economically rational.

Where self-hosting breaks even

Research from OpenMetal shows a consistent breakeven point for Pinecone vector databases at scale. Once workloads reach roughly 80 to 100 million queries per month, self-hosted deployments tend to be cheaper than managed cloud services. Below this range, cloud pricing is usually competitive. Above it, usage-based billing begins to dominate total cost.

This threshold matters because many enterprise RAG systems cross it quickly. Customer support, document search, fraud detection, and recommendation systems often serve tens or hundreds of millions of queries each month once deployed across business units or regions.

The hidden cost in cloud pricing

Cloud pricing is rarely just a per-query fee. Vector databases introduce several cost drivers that are easy to overlook during planning.

Egress fees are a major factor. Most cloud providers charge around $0.09 per GB for data leaving their network. Moving embeddings between regions, exporting data for analytics, or migrating to another system all incur these fees. Over time, they become a meaningful portion of total spend.

Finally, vector search does not scale linearly. As vector counts grow and dimensionality increases, query costs rise faster than expected. What looks affordable at 10 million vectors can become expensive at 500 million, even if query volume grows steadily.

On-premises costs are fixed and predictable

On-premises deployments have real costs, but they behave differently. Hardware is typically amortized over three to five years. Staffing requirements are stable once the system is running. Facilities and power costs are known in advance.

The key difference is predictability. Costs do not spike because of usage patterns or data movement. Once the system is sized correctly, monthly spend remains largely flat, even as query volume increases.

A real world example

Consider a production e-commerce application with the following scale:

  • 500M vectors
  • 200M queries every month
  • 1024 vector dimensions
  • 6M writes monthly

At this scale, a typical managed Pinecone vector database costs around $8,500 per month once compute, storage, and rebuild overhead are included.

Estimated monthly cost
Total Estimated Cost: $8,454 / month

  1. Storage
  • Usage: 845 GB
  • Cost: $279
  1. Query Costs
  • Configuration:
    • 24 b1 nodes
    • 4 shards × 6 replicas
  • Assumption: 1% filter selectivity
  • Estimated Cost: $8,074
    > Note: Actual query cost may vary. Benchmark your workload on DRN for more accurate estimates.

*3. Write Costs
*

  • Write Volume: 30 million Write Units (WU)
  • Assumption: Each write request consumes ≥ 5 WU
  • Cost: $101
    Image 4: Pinecone cost estimation

An equivalent on-premises deployment might cost approximately half of that after hardware amortization, assuming an 18-month payback period and one to two engineers supporting the system. After that payback period, costs drop further while capacity remains available.

A study by Enterprise Storage Forum shows the cost projection of on-premises and cloud workloads.

Image 5: Enterprise storage forum TCO

Cost alone does not decide every deployment, but once vector workloads reach scale, the economics become difficult to ignore. Understanding where your system sits on this curve is essential before locking in a long-term vector database strategy.

When Latency And Connectivity Matter

Latency and connectivity are often treated as secondary concerns in architecture decisions. For many AI workloads, they are decisive. Once vector databases support real-time systems, network round-trips and internet dependency can make cloud deployments impractical or unsafe.

Real-time response requirements
Some applications have strict response time limits. In healthcare, clinical decision support and diagnostic systems often require responses in under 50 milliseconds. This budget includes data retrieval, vector search, and model inference. Similarly, banks and financial institutions often require very low latency for maximum user experience.

Public cloud deployments add unavoidable network latency. Even within the same region, round-trip latency typically adds 20 to 80 milliseconds before any compute work begins. For applications with tight latency targets, this overhead alone can exceed the total allowed response time. On-premises deployments remove that network hop, allowing systems to meet real-time requirements consistently.

**Systems that must work offline
**Many environments cannot rely on constant connectivity. Retail point-of-sale systems must continue operating during network outages. Manufacturing facilities are often located in remote areas with unstable connections. Military and maritime deployments may operate in fully disconnected or classified environments.

In these scenarios, a cloud dependency is a single point of failure. If the network goes down, the AI system stops working. On-premises and edge deployments allow vector search and inference to run locally, ensuring the system continues to function even when external connectivity is unavailable.

**The cost of downtime
**It is no news that there has been an increase in downtime from cloud providers. On November 18, 2025, Cloudflare outage disrupted large portions of the internet, causing downtime across major platforms including X, Amazon Web Services, Spotify, and so on. The impact of connectivity failures is not theoretical. In manufacturing, average downtime costs are estimated at $260,000 per hour. When AI systems support quality control, predictive maintenance, or process automation, any outage directly affects production.

A cloud-only architecture introduces risk that is hard to justify in these environments. Even short network disruptions can lead to significant financial loss. On-premises deployments reduce this risk by removing external dependencies from critical execution paths.

For workloads with strict latency targets or limited connectivity, the choice is often clear. Cloud-based vector databases may work during development, but they fail to meet operational requirements in production.

The Operational Complexity Question

The strongest argument for cloud vector databases is operational simplicity. Managed services remove the need to provision hardware, manage clusters, apply patches, or handle failures. For small teams or early-stage projects, this advantage is real and often decisive. Cloud deployments allow engineers to focus on application logic rather than infrastructure.

It is also important to recognize that modern on-premises deployments look very different from those of a decade ago. This is not the world of manual server provisioning and fragile scripts. Kubernetes, infrastructure-as-code, and automated deployment pipelines have reduced operational overhead significantly. Rolling upgrades, automated scaling, and monitoring are now standard practices in on-premises environments as well as in the cloud.

Many enterprises adopt hybrid approaches to balance speed and control. Development and experimentation happen in the cloud, where teams can move quickly and iterate. Production systems run on-premises, where costs are predictable and compliance is easier to enforce. This pattern allows teams to get the best of both models without committing fully to either.

Decision Framework: Eight Questions

The fastest way to make a defensible deployment decision is to walk through a small set of yes or no questions with engineering, legal, finance, and operations.

**1. Does your data require geographic restrictions?
**Regulations such as GDPR, HIPAA, and financial services rules may limit where data can be stored or processed.
If yes, on-premises should be strongly considered because it provides full control over data location. If no, cloud deployment remains viable.

**2. Do you have predictable, high-volume query patterns?
**Cloud vector database costs scale with usage. A simple check is monthly queries multiplied by the unit cost.
If usage exceeds roughly 80 to 100 million queries per month, on-premises is often cheaper. Below that range, cloud pricing is usually more economical.

**3. Do you need offline capability?
**Some systems must continue working without network access, such as in manufacturing, retail, or edge environments.
If yes, on-premises is required. If no, cloud remains an option.

**4. Can you tolerate additional latency?
**Cloud deployments add network latency, often 50 to 100 milliseconds.
If your application cannot tolerate this, on-premises deployment is necessary. If it can, cloud performance may be acceptable.

**5. Do you have existing infrastructure teams?
**Operational capacity matters.
If you already run on-premises systems, the added burden is limited. If not, cloud-managed services provide a clear operational advantage.

**6. Is cost predictability important?
**Usage-based billing introduces cost variability.
If predictable costs matter, on-premises provides stability. If flexibility matters more, cloud pricing may be a better fit.

**7. Are you extending existing IT infrastructure?
**Deployment context affects the decision.
If you are extending existing systems, on-premises leverages current investments. If you are building something new, cloud may be faster to deploy.

**8. How large is your data footprint?
**Data volume and access frequency influence long-term cost.
If you manage more than 10 TB with frequent access, on-premises becomes attractive. If your data is smaller, cloud is often sufficient.

Image 6: Decision framework

When several answers point in the same direction, the decision becomes easy to explain and defend across engineering, legal, finance, and operations teams.

When Cloud Makes Sense

On-premises deployment is not always the right answer. In many situations, cloud-based vector databases remain the better choice. Being clear about these cases helps avoid over-engineering.

  • Unpredictable scaling: Startups and new products often face uncertain growth. Cloud platforms allow rapid scaling without long-term infrastructure commitments, which reduces risk when demand is unclear.
  • Small data volumes: When total data is under 10 TB and query volume stays below about 50 million queries per month, cloud pricing usually works well and is simpler than self-hosting.
  • Rapid experimentation: Proof-of-concepts, research projects, and early prototypes benefit from fast setup and easy teardown. Cloud services support quick iteration with minimal operational effort.
    -** No compliance constraints**: If data residency, sovereignty, and regulatory requirements are not an issue, cloud deployment avoids legal complexity and speeds up delivery.
  • Limited infrastructure expertise: Teams focused on application logic rather than operations can rely on managed services instead of maintaining databases, clusters, and hardware.

In these cases, cloud is the most effective and practical option.

Hybrid Deployment Strategies

Hybrid deployments act as the middle ground for enterprises that need both speed and control. Rather than treating cloud and on-premises as mutually exclusive, teams place each part of the system where it performs best.

**Cloud for iteration, on-prem for scale
**A common pattern is to develop and test in the cloud, where managed services and elastic infrastructure enable rapid iteration. Once models, indexes, or pipelines are stable, they are promoted into on-premises production environments to meet compliance, latency, and operational requirements. This preserves developer velocity without compromising production guarantees.

**Data segregation by risk and regulation
**Hybrid architectures also allow organizations to separate workloads by risk profile. Sensitive or regulated data stays on-premises, while analytics, training, or search over derived data runs in the cloud. The same logic applies regionally: EU data may remain on-premises or in sovereign environments, while US workloads run in public cloud regions, avoiding global systems being constrained by the strictest jurisdiction.

**Cost and migration flexibility
**Cost optimization is another driver. Frequently accessed vectors or low-latency services can be cheaper and more predictable on-premises, while cold storage and bursty workloads benefit from cloud pricing. Many teams start cloud-first, then selectively move components on-premises as scale or compliance pressures grow. Hybrid makes this a controlled evolution rather than a disruptive rewrite.

Industry research shows this is a stable operating model. Google Distributed Cloud and similar platforms explicitly frame hybrid as a long-term strategy, recognizing that modern systems are designed to span environments, not collapse them into one.

Actian’s Approach To On-premises Vector Databases

For teams that conclude that on-premises is the right deployment model, the next question is: which platform can actually meet these requirements? Actian’s approach is built specifically for this audience, without assuming the cloud is the default or the end state.

Actian delivers an enterprise-grade vector database that runs fully in your own data center or controlled environments. You retain full control over data placement, networking, and operations. There is no forced dependency on external cloud services, which simplifies audits and long-term system design.

Compliance requirements are treated as baseline constraints. By keeping data local and eliminating egress paths, Actian aligns with GDPR, HIPAA, FedRAMP, and similar regulatory frameworks. This reduces the need for compensating controls or complex legal workarounds.
Cost behavior is also predictable. Actian avoids usage-based pricing models that scale with queries or vector counts. This makes budgeting simpler and removes surprises as workloads grow.

Edge support is also taken into consideration. Actian’s architecture supports offline operation and local inference, making it suitable for manufacturing sites, retail locations, and other environments where connectivity is limited or unreliable. The system is designed to keep working even when the network does not.

Final Thoughts

Choosing between cloud and on-premises for vector databases is about understanding your priorities. Cloud works well for small workloads, rapid experimentation, and teams without deep infrastructure expertise. On-premises makes sense when compliance, latency, cost predictability, or scale are critical.

Many enterprises find a hybrid approach is the best balance, combining cloud flexibility with on-premises control. The key is making intentional decisions based on your data, workloads, and regulatory needs rather than following trends.

Actian empowers enterprises to confidently manage and govern data at scale. Organizations trust Actian data management and data intelligence solutions to streamline complex data environments and accelerate the delivery of AI-ready data. As the data and AI division of HCLSoftware, Actian helps enterprises manage and govern data at scale across on-premises, cloud, and hybrid environments. Learn more about Actian and how it fits into your on-premises AI strategy.