One thing I realized after working deeply with AI agents:

Execution is not the bottleneck anymore. Structure is.

You start with a simple task like:

“Create a product for AI SEO.”

Then your agent starts suggesting things like:

• MCP integration
• repo indexing
• semantic search
• auto-generated landing pages
• etc.

Every feature creates another layer of:

• research
• planning
• implementation
• edge cases
• dependencies

Very quickly you end up with a huge tree of subtasks that becomes impossible to hold in your head.

This is exactly where I started using Agency-OS differently.

Instead of treating agents like chatbots, I treat them like autonomous operators inside a structured operating system.

Example workflow

Main agent defines the product direction.

Separate agents branch into:

• competitor research
• distribution
• architecture
• onboarding UX
• monetization
• SEO
• repo structure

Every branch becomes its own structured workspace with:

• goals
• outcomes
• decisions
• context
• dependencies
• next actions

Now I can actually see:

• what agents are doing
• why they suggest something
• which tasks are blocked
• where complexity explodes
• which ideas are worth pursuing

The important part is not “more AI”.

It’s visibility and orchestration.
Without structure, agents generate chaos faster than humans.
With structure, they become leverage.

More details and examples cases: https://automatelab.tech/agency-os-launch/
Repo: https://github.com/AutomateLab-tech/agency-os

The image below shows how my real production Agency-OS board looks internally

Ever went through a simple DFS code and went “I would rather beat an old lady with a stick“?

That was me until I wasn’t.

• You know what trees are.
• You also know how the traversals work.
• Yet you seem to never truly grasp recursive tree traversals.

The problem seems much deeper than “I suck at recursion”.

The “Tutorial Hell” Phase

Facing the problem myself, I tried many things — from getting into tutorial hell to solving problems on HackerRank, LeetCode but… does that even work?
In my case, a clear NO.

The Breakthrough: Manual Simulation

The solution is finding out what is actually stopping you from using trees. Play around with it, understand what a call stack is.
Simulate recursion without using recursion (yup, that is a thing) yourself by maintaining a manual call stack on a piece of paper or any programming language.

Now the conclusion is — you understand everything, yet those 3 lines seem magical. That is probably because you are not doing the heavy lifting here.
Recursion takes care of that, and you don’t see how.

The Strategy: Walkthrough the problem

The most basic fix? Just be a human.

While developing one of my projects I ran into this exact issue again. What did I do? The exact thing I mentioned above.
I Wrote down the problem, went through it step by step. Made sure I knew what I was looking for before I even touched that damn keyboard.

Let us say the expression is: a=b AND b=c. If someone says, “write code to make it a tree,” it might feel like they asked you to poop gold which somehow seems much more reasonable. But it is really not that complex.
The best thing to do at this point is probably ignore all the complex terminologies like LR, LL parsers etc which makes it intimidating. At the core it’s all about converting a human thinking process into machine code.

The “how” becomes pretty easy from here.
You know AND is the root, and each left and right side is again a sub-tree with its operator as the root.
The simplest approach (to understand recursion) is to manually check the sequence and advance. Each lexeme is just a node with a value and left/right pointers.
A tree is just connections of pointers and recursion is all about returning those pointers in a very strategic way.

Conclusion

Once recursion starts feeling fun — congratulations, it’s over. You’re a nerd now.

If you have been trying to figure out recursion for 67th time now… take a break, touch some grass, come back and make sure you C problems in a human way before attempting to solve like a machine.

PS: Using AI tools like Gemini, Claude, or ChatGPT isn’t too bad for understanding such topics either — they add to your existing understanding rather than replacing it.

TLDR: Simulate before you code. Draw before you type. Be a human first, a programmer second.

This is a submission for the Gemma 4 Challenge: Build with Gemma 4

What I Built

Growing up in Nigeria, malaria was everywhere. People missed school because of it. Families spent nights in hospitals because of it. Yet most healthcare AI systems today are trained on datasets that barely represent African patients.

Fisibel is a multimodal African synthetic health data infrastructure platform powered by Google Gemma 4. Upload any health record image. Gemma 4 reads it, extracts clinical patterns, and generates privacy-safe synthetic datasets grounded in real WHO and World Bank statistics, with full OpenMetadata governance and fidelity scoring.

No real patient data ever leaves the platform. Only privacy-safe synthetic equivalents that preserve the statistical reality of African health.

The pipeline:

• Upload a real health record image
• Gemma 4 reads it multimodally and extracts clinical signals
• WHO + World Bank APIs ground the generation in real African statistics
• Synthetic dataset generated with 94% fidelity score
• Automatically registered in OpenMetadata with full lineage tracking
• Data Quality analysis before training

Demo

Check Live Demo

Code

GitHub: https://github.com/Valentinetemi/fisibel

How I Used Gemma 4

I chose Gemma 4 26B MoE (gemma-4-26b-a4b-it) for 5 specific reasons:

1. Multimodal understanding for health records
Fisibel’s core feature is uploading a real hospital health record image and having Gemma 4 read it directly. This is Layer 1 of the pipeline — Multimodal Ingestion. The image is compressed client-side and converted to base64. Gemma 4 processes it natively, extracting diagnoses, symptoms, treatments, geographic identifiers, and clinical patterns directly from the document without any OCR preprocessing. Gemma 4 does not just generate data; it first reads and understands real clinical documents.

2. Statistical grounding before generation
Before Gemma 4 generates a single row, Fisibel fetches live African health indicators from the WHO Global Health Observatory API and World Bank Open Data API. This is Layer 2 — Statistical Grounding. Gemma 4 uses these verified statistics as ground-truth constraints, ensuring every synthetic row aligns with real African population distributions rather than invented patterns.

Scientific Validation Mirror
During generation, Fisibel displays a live Scientific Validation Mirror, a real-time chart comparing the synthetic dataset’s age distribution against the WHO/World Bank baseline. The blue line represents Fisibel’s synthetic output. The grey line represents the real-world WHO statistical baseline. When they align closely, the data is statistically grounded. This gives researchers immediate visual confidence in their data before they even download it.

3. Synthetic generation with clinical logic
In Layer 3, Gemma 4 combines the extracted clinical patterns with the WHO and World Bank distributions to generate synthetic CSV datasets. It enforces strict categorical consistency, realistic age distributions, geographically accurate Nigerian LGAs, and clinically coherent symptom-treatment pairs.

4. Fidelity scoring
Every generated dataset is scored 0–100 using a two-layer algorithm before it reaches any training pipeline.

Layer 1 — Gemma 4 AI Evaluation (80% of score)

Gemma 4 evaluates 50 randomly sampled rows against 4 criteria:

Layer 2 — Completeness Check (20% of score)

A statistical completeness function scans every cell in the full dataset:

function getCompleteness(csv: string) {
  const rows = csv.split('n').filter(Boolean)
  let total = 0
  let filled = 0
  rows.forEach(r => {
    r.split(',').forEach(cell => {
      total++
      if (cell.trim() !== '') filled++
    })
  })
  return (filled / total) * 100
}

Final Score Formula:

finalScore = (0.8 × GeminiScore) + (0.2 × completenessScore)

This penalizes missing values even when AI-evaluated rows look good. A dataset with beautiful clinical patterns but 30% empty cells will not score above 86.

The Lagos malaria dataset achieved 94%.

5. AI-powered data quality recommendations
Before any dataset is handed to a model, it passes through a dedicated quality layer that produces a Model Readiness Score (0–100). Every penalty is calculated explicitly — no black box.

Model Readiness Score Breakdown

The score starts at 100 and deducts penalties across five dimensions:

modelReadinessScore = max(0, 100 − totalPenalty)

Per-Column Analysis

Every column is individually profiled:

// Data type detection — strict matching, not guessing
function detectDataType(values: any[]): string {
  // boolean: exact true/false/yes/no match only
  if (/^(true|false|yes|no)$/.test(str)) → 'boolean'

  // numeric: must be a valid number, not just contain digits
  if (!isNaN(Number(str)) && str !== '') → 'numeric'

  // date: standard date pattern matching
  if (/^d{1,2}[-/]d{1,2}[-/]d{2,4}$/.test(str)) → 'date'

  // everything else
  return 'text'
}

For numeric columns, Fisibel computes:

• Mean, median, standard deviation
• Min / max range
• Outlier count using the IQR method (values below Q1 − 1.5×IQR or above Q3 + 1.5×IQR)

PII Detection

Every column header is scanned against a pattern library. Columns flagged as PII (names, phone numbers, national IDs, addresses) are surfaced as warnings and factored into the readiness penalty — so sensitive data never silently reaches a training pipeline.

Duplicate Detection

Row-level deduplication using JSON serialization:

function findDuplicates(rows: any[]) {
  const rowStrings = rows.map(row => JSON.stringify(row))
  const unique = new Set(rowStrings)
  const duplicateCount = rows.length - unique.size
  return {
    duplicateCount,
    duplicatePercentage: (duplicateCount / rows.length) * 100
  }
}

Readiness Verdict

Why MoE efficiency matters
The 26B MoE architecture activates only ~4B parameters during inference. This matters for African healthcare researchers who often work in resource-constrained environments. Frontier-level clinical reasoning at near-edge efficiency is exactly right for this problem.

What Gemma 4 unlocked:

• Direct reading of Nigerian hospital health records as images
• Clinically coherent synthetic data generation grounded in the WHO health observatory and the World Bank statistics
• Fidelity scoring — Gemma 4 evaluates its own outputs against real population distributions
• AI-powered data quality recommendations before training

OpenMetadata Integration

Fisibel uses OpenMetadata as its core governance layer via the REST API.

Every generated dataset is automatically:

• Registered as a Table Entity under the fisibel-synthetic Database Service
• Given full column schema with data types and descriptions
• Tagged with Tier3 governance tag
• Stored with a complete description including domain, country, fidelity score, and the exact prompt used to generate it
• Tracked with lineage — the Gemma 4 Generator pipeline is registered as the source, with an edge connecting it to every output table
• Discoverable in the Fisibel Catalog which pulls live from the OpenMetadata API

Entity Structure

Database Service: fisibel-synthetic (CustomDatabase)
  └── Database: default
        └── Schema: synthetic_datasets
              ├── healthcare_dataset__nigeria
              └── ...

Pipeline Service: fisibel-pipelines (CustomPipeline)
  └── Pipeline: gemma-generator
        ├── → healthcare_dataset__nigeria  (Lineage Edge)
        └── ...

📈 Results

• 500 synthetic patient records generated from a single Lagos hospital record
• 94% fidelity score against WHO population distributions
• Zero PII exposure — no real patient data stored or transmitted
• Full OpenMetadata lineage tracking from source record to output dataset

Why It Matters
Behind every missing dataset is a real person:
a child misdiagnosed,
a treatment delayed,
a patient overlooked.

Fisibel exists so African patients are no longer invisible to the AI systems increasingly shaping modern healthcare. Systems built to detect disease often show reduced performance on underrepresented populations.

African patients deserve to be seen by the AI systems built to protect them. Fisibel is where that starts.

I built an ML model using XGBoost to catch auto-generated disposable emails when blacklists can’t keep up. Most validators rely on MX records, SMTP checks, or blacklists – disposable emails have real mailboxes so MX and SMTP return valid. That’s why I added an ML model to determine the risk of accepting an email based on the username and domain.

Compare these two emails:

[email protected]

and

[email protected]

You can immediately tell the second one is fake, but why exactly? Is it the numbers, the consonant/vowel ratio, the length? We don’t need to know the exact rules. We can train an XGBoost model on labelled data to figure it out, using features like digit count, length, and consonant/vowel ratio to predict whether a username or domain is legitimate.

Under the hood, the API combines MX records, blacklist checking, role detection, syntax validation, and new domain detection for basic coverage. On top of that, ML-powered scoring handles what static methods miss. It also supports batch validation of up to 30 emails per request. SMTP validation is intentionally excluded as disposable emails have real mailboxes so it returns valid anyway, and it adds significant latency for no benefit in this use case.

For the ML side, I used pandas for feature engineering, an 80/20 train-test split via sklearn’s train_test_split, and XGBoost as the classifier. Features include pairs of vowels, consonants, digits and ratios for each. One feature that greatly increased the accuracy for username score was using determining whether the username contained a name. For example,

[email protected]

contains john and doe which it automatically more likely to be safe than

[email protected]

even if we disregarded the domains.

Here’s the API response for the fake email:

{
  "email": "[email protected]",
  "valid_email_structure": true,
  "is_role": false,
  "mx_records": true,
  "not_disposable": true,
  "new_domain": false,
  "domain_risk": 0.23987430334091187,
  "name_risk": 0.9953031539916992,
  "valid_email": true
}

This response shows us that even though the email is disposable, the traditional methods failed to detect it. However, name_risk is incredibly high, showing us that the email is very risky to accept. Interestingly, despite the domain being fake, domain_risk remains low - this is because the model is trained on patterns in the domain name itself, which doesn’t follow the same conventional patterns as usernames.

Contrast this with a real email:

{
  "email": "[email protected]",
  "valid_email_structure": true,
  "is_role": false,
  "mx_records": true,
  "not_disposable": true,
  "new_domain": false,
  "domain_risk": 0.221174955368042,
  "name_risk": 0.00679133040830493,
  "valid_email": true
}

This shows us that the email is safe to accept as both traditional and ML methods consider the email to be safe. Also worth noting the domain_risk for gmail is similar to the fake one, which shows us domain_risk alone isn’t reliable except in special cases of unconventional named domains.

This is why combining all fields, rather than relying on any single one, gives you the most accurate result.

Limitations
Despite training a model for domain_risk, it has limited uses as patterns in domains are very limited. Unlike usernames, which often follow recognisable human patterns like names or words, auto-generated domains can look surprisingly normal compared to genuine ones, making it much harder for the model to distinguish between real and fake. A prime example of this is visible in the responses above, where the fake domain has a negligible difference in score compared to the real one.

Another takeaway was that blacklists are less effective than initially expected. While they work well for well-known disposable providers, the sheer volume of auto-generated domains makes it nearly impossible to maintain a comprehensive and up to date list. I still found them useful as the resources it takes to implement a blacklist and latency difference is negligible.

The reason I kept traditional methods is that ML is still just prediction; it can produce false positives (real email flagged as risky) and false negatives (disposable email missed). Combining both methods reduces the impact of either failure mode.

Try It Yourself
The API is available on RapidAPI with a free tier of 100 requests per month. If you’re building a signup form, cleaning an email list, or trying to prevent fraud, give it a try and let me know what you think.

Note: the first request may be slow due to a cold start on the free server – subsequent requests will be faster.

Check out my interactive Swagger UI documentation. You can see every endpoint, all parameters, and example responses in one place.

Python Library
Installation:

pip install identify-fake-email

Quick Start:

import identify_fake_email
from identify_fake_email.api.default_api import DefaultApi
from identify_fake_email.configuration import Configuration

config = Configuration()
config.api_key["x-rapidapi-key"] = "YOUR_RAPIDAPI_KEY"

client = DefaultApi(identify_fake_email.ApiClient(config))

result = client.validate_email_validate_get("[email protected]")

print(f"Valid: {result.valid_email}")
print(f"Name risk: {result.name_risk}")

if result.name_risk < 0.5: # use your own validation and thresholds
    print("Safe to accept")
else:
    print("Suspicious - review")

For bulk validation

from identify_fake_email.models.bulk_emails import BulkEmails

bulk = BulkEmails(emails=["[email protected]", "[email protected]"])
results = client.validate_bulk_validate_bulk_post(bulk)

for result in results:
  if result.name_risk < 0.5: # use your own validation and thresholds
      print("Safe to accept")
  else:
      print("Suspicious - review")

Also covered this on Medium with a broader overview.

Si tu as 30 secondes. Quand un agent IA déclare qu’un build est vert, qu’un test passe, qu’un drift a été détecté, ou qu’un contact « n’existe pas » dans la base, la phrase n’est pas une preuve, c’est une assertion. Dans la plupart des cas elle est juste. Quand elle est fausse, elle l’est toujours de la même manière, par confiance interne au modèle découplée de l’état externe vérifiable. Règle de survie après 118 808 lignes produites en 32 jours de travail effectif avec Claude Code, toute affirmation factuelle vient avec sa preuve matérielle dans le même message, ou n’a aucune valeur évidentielle.

Quatre faux positifs en deux heures

10 avril 2026, après-midi, refonte d’un module sensible de l’ERP. J’enchaîne cinq blocs de modifications avec un agent Claude Code, et à chaque étape l’agent rend la même formule, « Compiled successfully. » Je pousse. Le runtime crashe. Je relis la sortie, je vois QRCodeSVG référencé alors que l’import a été supprimé, isSeancePassed passé à un composant qui ne l’accepte plus, des types non régénérés après un changement de schéma Supabase, des refs JSX orphelines après un revert. Quatre annonces consécutives de vert avec quatre erreurs TypeScript bien réelles derrière.

Niran est au bureau d’à côté, hoodie sombre, l’emballage replié de son burger en équilibre sur un coin de laptop. Il lit un PDF en silence. Au quatrième Compiled successfully qui se révèle faux, je me tourne vers lui ; il lève les yeux une seconde, hoche la tête, revient à son écran. La verbosité de l’agent et l’économie de gestes du judoka tiennent la même conversation, et c’est lui qui a raison.

Je ne reproche pas à l’agent d’avoir menti. Il a résumé ce qu’il croyait avoir vu. Le résumé est cohérent avec son état interne et désaligné avec la matière. Pas un seul de ces quatre crashes ne se serait produit si l’agent avait collé la sortie brute de pnpm build plutôt que sa lecture résumée.

Le glissement épistémique

Le problème n’est pas la véracité, c’est la valeur évidentielle. Une assertion sans matériel à l’appui ne se vérifie pas, elle se croit ou pas. Le mécanisme est connu : un agent entraîné par reinforcement learning from human feedback résume par défaut parce qu’il a appris que les humains préfèrent les réponses brèves aux logs verbeux ; ce qui est un service en chat devient un piège en production. « Le build est vert » sans la sortie brute du compilateur ne peut être ni infirmé ni confirmé par un tiers, c’est un état d’âme du système, pas un fait du monde. Tant qu’on ne distingue pas ces deux régimes, on opère à l’aveugle dans la zone grise où s’accumulent les régressions silencieuses qu’aucun monitoring ne détecte. La fonction du résumé n’est pas d’être faux, c’est de n’être pas vérifiable.

Cinq formes de la même évasion

L’évasion est toujours la même. Un verbe d’état au présent, « est », « passe », « confirme », « introuvable », sans rattachement à un artefact externe vérifiable. Cinq variantes que je rencontre dans le repo Rembrandt :

1. « Le build est vert. » Sans la sortie brute de pnpm build ou tsc --noEmit.
2. « Les tests passent, la CI est verte. » Sans rapport runner ni URL du run.
3. « Drift détecté entre l’enum DB et l’enum TS », ou son inverse, « le contact n’existe pas dans la base ». Sans la requête SQL exécutée ni ses lignes brutes.
4. « EXPLAIN ANALYZE confirme que l’index est utilisé. » Sans le plan brut, sur la requête exacte que l’application envoie. Pas la table cible isolée, parce qu’une vue intermédiaire avec un CASE COALESCE peut tuer l’index sans que l’isolation le révèle.
5. « Le webhook renvoie 400 / 422. » Sans le payload brut côté partenaire, alors que neuf fois sur dix le diagnostic part de là.

Toutes partagent la même grammaire. La parade est uniforme. On ne demande pas à l’agent de mieux raisonner, on lui demande de joindre la commande et sa sortie brute, ou la requête SQL et ses lignes, ou le payload, dans le même message que l’assertion. Sans ça, l’assertion ne vaut rien.

La règle, codée dans le CLAUDE.md racine

# Vérification matérielle (extrait CLAUDE.md racine)
- Toute affirmation "build vert / tests passent / CI verte / drift
  détecté / contact introuvable / OK" doit être accompagnée *dans le
  même message* de la commande de vérification et de sa sortie brute.
- Tout chiffre relayé à un humain doit être vérifié par requête SQL
  avant d'être relayé.
- EXPLAIN ANALYZE sur requête de production : exécuter sur la requête
  exacte que l'application envoie (vue/RPC incluses), pas sur la table
  cible isolée. Deux runs consécutifs avant de juger.
- Sur 400/422 d'un partenaire externe : exiger le payload brut avant
  de proposer un fix.
- `tsc --noEmit` CLI = autorité, panneau IDE = potentiellement périmé.

Côté humain, trente secondes de friction par échange et zéro push cassé sur les chantiers où la règle tient. La règle écrite ne suffit pas pour autant. Une discipline qui repose sur la mémoire s’érode. Il faut durcir.

Durcir avec un script

scripts/verify-head-builds.sh stash le working tree, fait tsc --noEmit sur HEAD, et restore. C’est ce qui permet d’exiger un build vert sur le commit qu’on s’apprête à pousser, pas sur l’arbre de travail mêlé de modifications non stagées.

# scripts/verify-head-builds.sh — extrait load-bearing
HAS_LOCAL_CHANGES=0
if ! git diff --quiet HEAD 2>/dev/null 
   || [[ -n "$(git ls-files --others --exclude-standard)" ]]; then
  HAS_LOCAL_CHANGES=1
fi

if [[ $HAS_LOCAL_CHANGES -eq 1 ]]; then
  git stash push -u -q --message "verify-head-builds-autostash-$$"
fi

# tsc sur HEAD pur, sans contamination du working copy
if npx tsc --noEmit 2>&1; then
  echo "✓ HEAD compile proprement — safe à push"
else
  echo "✗ HEAD ne compile PAS — fix avant push"
  exit 1
fi

Et une convention d’écriture côté agent : tout chiffre relayé à un humain assorti de la requête SQL qui l’a produit, sous forme de bloc collable. Pas de chiffres orphelins, ni en prose, ni en commit message. Bruyant la première semaine, transparent ensuite.

Ce que tu peux copier dans ton projet

Snippets complets (extrait CLAUDE.md règle d’évidentialité, script verify-head-builds.sh complet) dans le dossier material-verification/ du repo compagnon de la série, licence MIT.

Trois gestes directement applicables si tu travailles avec un agent coding :

1. La règle d’évidentialité dans le CLAUDE.md racine (cinq bullets ci-dessus). Sans cet ancrage écrit, le reste est cosmétique.

2. Le script verify-head-builds.sh : stash + tsc --noEmit sur HEAD + restore. Treize lignes de bash, MIT.

3. Une convention de chiffres traçables. Tout chiffre accompagné de sa requête SQL dans le même message. Pas de chiffres orphelins en prose, ni en commit message.

Et vous, sur quelle assertion verte avez-vous arrêté de croire en premier ? Je lis les commentaires.

Ce qu’on ne croit plus

Au bout de quelques semaines, on entend « Compiled successfully » différemment. Pas un constat, une prétention dont la valeur dépend strictement de ce qui suit. Si rien ne suit, la prétention est nulle. La règle vaut hors IA aussi. Un humain qui dit « je viens de vérifier, le compteur est à 1247 » sans coller la requête vit dans la même zone grise. Niran n’a jamais eu besoin qu’on lui explique cette différence. Au judo, la chute n’est pas un avis sur la chute, c’est le sol qui répond.

Code compagnon, rembrandt-samples/material-verification/, CLAUDE.md.snippet + verify-head-builds.sh, MIT.

La règle d’évidentialité ci-dessus est désormais R1 du Counterpart Toolkit : github.com/michelfaure/doctrine-counterpart, 14 règles opérationnelles, install en 1 commande, CC-BY-4.0.

If you have 30 seconds. When an AI agent declares that a build is green, that tests pass, that drift has been detected, or that a contact “doesn’t exist” in the database, the sentence isn’t a proof, it’s an assertion. In most cases it’s right. When it’s wrong, it’s wrong the same way, model-internal confidence decoupled from verifiable external state. Survival rule after 118,808 lines produced in 32 effective days with Claude Code, every factual claim comes with its material proof in the same message, or has zero evidentiary value.

Four false positives in two hours

April 10th, 2026, afternoon, overhaul of a sensitive ERP module. I chain five blocks of changes with a Claude Code agent, and at every step the agent returns the same line, “Compiled successfully.” I push. The runtime crashes. I reread the output, I see QRCodeSVG referenced when the import has been removed, isSeancePassed passed to a component that no longer accepts it, types not regenerated after a Supabase schema change, orphan JSX refs after a revert. Four consecutive announcements of green with four very real TypeScript errors behind them.

Niran is at the desk next to mine, dark hoodie, the folded wrapper of his burger balanced on a corner of his laptop. He’s reading a PDF in silence. On the fourth Compiled successfully that turns out false, I turn toward him without saying anything; he looks up for a second, gives a small nod, goes back to his screen. The verbosity of the agent and the economy of gestures of the judoka are having the same conversation, and he’s the one who’s right.

I don’t blame the agent for lying. It summarized what it thought it had seen. The summary is consistent with its internal state and out of phase with the matter. Not one of those four crashes would have happened if the agent had pasted the raw output of pnpm build rather than its summarized reading.

The epistemic slip

The problem isn’t truthfulness, it’s evidentiary value. An assertion without backing material can’t be verified, it’s believed or it isn’t. The mechanism is known: an agent trained by reinforcement learning from human feedback summarises by default because it learned humans prefer brief answers over verbose logs; what is a service in chat becomes a trap in production. “The build is green” without the raw compiler output can be neither denied nor confirmed by a third party, it’s a state of mind of the system, not a fact of the world. As long as we don’t distinguish these two regimes, we operate blind in the gray zone where silent regressions accumulate that no monitoring detects. The function of summary isn’t to be false, it’s not to be verifiable.

Five forms of the same evasion

The evasion is always the same. A state verb in the present tense (“is”, “passes”, “confirms”, “missing”) without anchoring to an external verifiable artifact. Five variants I encounter in the Rembrandt repo.

1. “The build is green.” Without the raw output of pnpm build or tsc --noEmit.
2. “Tests pass, CI is green.” Without the runner report or the run URL.
3. “Drift detected between DB enum and TS enum”, or its inverse, “the contact does not exist in the database”. Without the executed SQL query or its raw rows.
4. “EXPLAIN ANALYZE confirms the index is used.” Without the raw plan, on the exact query the application sends. Not the target table in isolation, an intermediate view with a CASE COALESCE can kill the index without the isolation revealing it.
5. “The webhook returns 400 / 422.” Without the raw payload from the partner side, when nine times out of ten that’s where the diagnosis starts.

All share the same grammar. The countermeasure is uniform. Don’t ask the agent to reason better, ask it to attach the command and its raw output, or the SQL query and its rows, or the payload, in the same message as the assertion. Without that, the assertion is worth nothing. Not little, nothing.

The rule, codified in the root CLAUDE.md

# Material verification (excerpt from root CLAUDE.md)
- Any claim "build green / tests pass / CI green / drift detected /
  contact not found / OK" must be accompanied *in the same message*
  by the verification command and its raw output.
- Any number relayed to a human must be verified by SQL query before
  being relayed.
- EXPLAIN ANALYZE on a production query: execute on the exact query
  the application sends (view/RPC included), not on the target table
  in isolation. Two consecutive runs before judging.
- On 400/422 from an external partner: demand the raw payload before
  proposing a fix.
- `tsc --noEmit` CLI = authority, IDE panel = potentially stale.

Human-side, thirty seconds of friction per exchange, zero broken pushes on the workstreams where the rule holds. The written rule isn’t enough on its own, though. A discipline that relies on memory erodes. You have to harden.

Hardening with a script

scripts/verify-head-builds.sh stashes the working tree, runs tsc --noEmit on HEAD, and restores. That’s what lets me demand a green build on the commit I’m about to push, not on the working tree mixed with non-staged changes.

# scripts/verify-head-builds.sh — load-bearing extract
HAS_LOCAL_CHANGES=0
if ! git diff --quiet HEAD 2>/dev/null 
   || [[ -n "$(git ls-files --others --exclude-standard)" ]]; then
  HAS_LOCAL_CHANGES=1
fi

if [[ $HAS_LOCAL_CHANGES -eq 1 ]]; then
  git stash push -u -q --message "verify-head-builds-autostash-$$"
fi

# typecheck on pure HEAD, no working copy contamination
if npx tsc --noEmit 2>&1; then
  echo "✓ HEAD compiles cleanly — safe to push"
else
  echo "✗ HEAD does NOT compile — fix before push"
  exit 1
fi

And a writing convention on the agent side: every number relayed to a human comes with the SQL query that produced it, as a copy-paste-ready block. No orphan numbers, in prose or in commit messages. Noisy the first week, transparent after.

What you can copy into your project

Full snippets (CLAUDE.md evidentiality rule excerpt, full verify-head-builds.sh script) in the material-verification/ folder of the series companion repo, MIT.

Three directly applicable practices for working with a coding agent:

1. An evidentiality rule in the root CLAUDE.md (five bullets above). Without this written anchor, the rest is cosmetic.

2. The verify-head-builds.sh script: stash + tsc --noEmit on HEAD + restore. Thirteen lines of bash, MIT.

3. A traceable-numbers convention. Every number paired with its SQL query in the same message. No orphan numbers in prose, none in commit messages.

And you, which green claim did you stop believing first? I read the comments.

What you stop believing

After a few weeks of practice, you hear “Compiled successfully” differently. Not as an observation, as a claim whose value depends strictly on what follows. If nothing follows, the claim is null. The rule applies outside AI too. A human saying “I just checked, the count is 1247” without pasting the query lives in the same gray zone. Niran has never needed someone to explain that difference to him. In judo, the fall isn’t an opinion about the fall, it’s the floor that answers.

Companion code, rembrandt-samples/material-verification/, CLAUDE.md.snippet + verify-head-builds.sh, MIT.

The evidentiality rule above is now R1 of the Counterpart Toolkit: github.com/michelfaure/doctrine-counterpart, 14 operational rules, install in 1 command, CC-BY-4.0.