The Return of Recursion: How 5M-Parameter Models Are Outperforming Frontier LLMs on Reasoning in 2026

TL;DR Summary

• Tiny recursive models with 5-7 million parameters are achieving state-of-the-art on deterministic reasoning tasks that frontier LLMs score 0% on — including Sudoku-Extreme, ARC-AGI puzzles, and maze navigation
• The key innovation: reasoning in latent space instead of generating “thinking tokens” like Chain-of-Thought — delivering 100x speedups and 75% token reduction
• Probabilistic TRM (7M params) achieves 98.75% on Sudoku-Extreme using Gaussian noise to escape local optima, while DeepSeek-R1 scores 0.0%
• RecursiveMAS applies recursion to multi-agent systems — agents communicate via latent representations (“telepathically”), cutting tokens by 75.6% and improving accuracy by 8.3%
• Attractor Models (27M params) outperform 1.3B Transformers trained on twice as many tokens — by solving for fixed points instead of iterating

Direct Answer Block

Recursive models revive a pre-transformer AI concept — iterative reasoning — but with modern training methods that avoid the vanishing gradient problems that killed RNNs. Instead of generating Chain-of-Thought tokens (slow, expensive), they refine representations in hidden latent space through loops. A 5M-parameter TRM achieves 87.4% on Sudoku-Extreme where DeepSeek-R1 scores 0%, while probabilistic extensions push this to 98.75% — at less than 0.0001x the cost of frontier LLMs.

Introduction

The AI industry has spent three years scaling transformers — more parameters, more data, more compute. Chain-of-Thought reasoning made them smarter but also slower and more expensive: every reasoning step is a token, every token costs money, and long chains hit context limits. Meanwhile, a parallel research thread has been quietly reviving recursion — and the results are startling. Models with 5 million parameters are solving puzzles that billion-parameter systems fail completely, using 100x less compute and generating 75% fewer tokens. Here’s how recursive architectures work, why they’re making a comeback, and where they fit in production.

Why did the AI industry abandon recursion — and why are 5M-parameter models now beating frontier LLMs at reasoning?

The story of recursion in AI is a story of training instability. Recurrent Neural Networks (RNNs) were the dominant architecture before transformers. They processed sequences iteratively — refining a hidden state through repeated passes — which is, conceptually, exactly what recursive reasoning models do today.

The problem was vanishing and exploding gradients. When you backpropagate through a recursive loop, gradients either shrink to zero (vanishing) or blow up to infinity (exploding) as the number of iterations grows. Training became unstable. The transformer’s solution — process everything in parallel with attention, no recurrence — eliminated the gradient problem and enabled the scaling revolution of 2018-2025.

But attention has its own scaling problem: quadratic compute cost. Each token attends to every other token. Chain-of-Thought makes this worse — every reasoning step generates a new token that must attend to every previous token. Long reasoning chains become exponentially expensive.

“Autoregressive LLMs hit a reasoning wall — Chain-of-Thought forces models to externalize intermediate thoughts token by token, becoming slow and memory-intensive as sequences grow.” — AlphaSignal summary of the recursive architecture revival

The recursive models being published in 2026 solve the gradient problem that killed RNNs through modern training innovations:

• TRM and HRM use weight-sharing across recursion steps, keeping the parameter count tiny (5-27M) and making gradient flow manageable
• Attractor Models use implicit differentiation — solving for fixed points analytically rather than through backpropagation-through-time — making training memory constant in effective depth
• Probabilistic TRM injects Gaussian noise at each step and uses a learned Q-head for early stopping, avoiding convergence to suboptimal solutions

The result: recursion is back, and it works. The arXiv:2605.19943 paper on Probabilistic TRM demonstrates 91.2% accuracy on Pencil Puzzle Bench vs 55.1% for frontier LLMs — “at less than 0.0001x the cost, using only 7M parameters.”

How does recursive latent reasoning differ from Chain-of-Thought — and why does it deliver 100x speedups?

The fundamental difference is where reasoning happens:

Chain-of-Thought (autoregressive):

• Model generates reasoning steps as text tokens: “Step 1: Let me think about this… Step 2: If X then Y… Step 3: Therefore…”
• Each token must be generated, then fed back as input for the next token
• Token generation is sequential — cannot parallelize
• All intermediate tokens count toward context length and API costs
• Each token invokes the full forward pass of a massive model

Recursive Latent Reasoning:

• Model refines a hidden representation through a loop — no tokens emitted until the final answer
• The loop runs in the model’s latent space (hidden states, not text)
• Iteration count is determined by convergence or a fixed budget, not by token generation speed
• No intermediate tokens = no context bloat, no token costs for reasoning steps
• The loop uses a tiny model (5-27M params), not a massive transformer

The 100x speedup claim comes from this architectural difference: each Chain-of-Thought step requires a full forward pass through a billion-parameter model and generates a token. Each recursive latent step requires a forward pass through a million-parameter model and produces no token. The HRM paper (cited in the newsletter) demonstrated up to 100x speedup for deterministic reasoning tasks compared to autoregressive CoT approaches.

The token reduction is even more dramatic. RecursiveMAS — which applies recursive principles to multi-agent systems — achieved 75.6% token reduction by round 3 (arXiv:2604.25917). Agents pass continuous latent representations to each other instead of text messages. Only the final answer is converted to text.

What are HRM, TRM, Probabilistic TRM, RecursiveMAS, and Attractor Models — and how do they compare?

Five distinct recursive approaches have emerged. Here’s how they compare:

HRM (arXiv:2603.22871 — March 2026)

The oldest of the modern recursive models. Uses two modules: H (high-level) for slow abstract planning and L (low-level) for fast detailed computation, coupled in a recursive loop. Inspired by human cognition — the dual-process theory where System 2 (slow, deliberate) plans and System 1 (fast, automatic) executes. Achieved state-of-the-art on ARC-AGI puzzles with only 1,000 training examples.

TRM (published at ICLR 2026 Latent & Implicit Thinking Workshop)

Strips HRM to its essence: a single 2-layer weight-sharing network. The key insight: increase recursion steps, not layers. More recursion depth improves generalization more than more parameters. The 5M-parameter TRM hit 87.4% on Sudoku-Extreme — a task where DeepSeek-R1 scored 0.0%. The TRM+Mamba-2 hybrid from arXiv:2602.12078 improved pass@2 on ARC-AGI by +2.0% while maintaining parameter parity.

Probabilistic TRM (arXiv:2605.19943 — May 2026)

TRM’s deterministic recursion can converge to suboptimal solutions with no escape mechanism. PTRM solves this by injecting Gaussian noise at each recursion step, creating parallel trajectories that explore diverse solution basins. A learned Q-head (initially used for early stopping in TRM) selects the best trajectory. The improvement: Sudoku-Extreme from 87.4% to 98.75%, Pencil Puzzle Bench from 62.6% to 91.2% — nearly double frontier LLM accuracy.

RecursiveMAS (arXiv:2604.25917 — April 2026)

Applies recursion to the multi-agent paradigm. Instead of agents exchanging text messages (expensive, verbose), they pass continuous latent representations through a lightweight RecursiveLink module — described as “telepathic” communication. The system is trained with an inner-outer loop algorithm for whole-system co-optimization. Results: 8.3% accuracy gain across 9 benchmarks, 1.2x-2.4x inference speedup, 34.6-75.6% token reduction.

Attractor Models (arXiv:2605.12466 — May 2026)

The most mathematically novel approach. Instead of iterating a fixed number of times, Attractor Models solve for a fixed point using implicit differentiation. The model proposes output embeddings, then an attractor module refines them by solving for equilibrium — training memory stays constant regardless of effective depth. The most remarkable finding: equilibrium internalization — after training, the model’s initial output is already near equilibrium, allowing the solver to be removed at inference with little degradation. A 770M Attractor Model outperforms a 1.3B Transformer trained on twice as many tokens.

How does Probabilistic TRM use Gaussian noise to break out of local optima and achieve 98.75% on Sudoku-Extreme?

Deterministic recursion has a fundamental weakness: it follows the same path every time. If that path leads to a suboptimal solution, there’s no escape — the recursion converges to a local minimum and stays there.

Probabilistic TRM introduces stochastic exploration as a test-time compute scaling strategy:

1. Inject Gaussian noise at each deep recursion step — small perturbations that nudge the latent state into neighboring regions of the solution space
2. Run multiple parallel trajectories — each with different noise realizations, exploring different solution basins
3. Use the Q-head for selection — the same Q-head originally designed for early stopping in TRM now scores each trajectory’s quality
4. Select the best trajectory — highest Q-head score wins

The key insight: this requires no retraining. The original TRM’s Q-head — trained for early stopping — naturally generalizes to trajectory selection. The noise injection is applied at inference time only. The PTRM paper shows accuracy gains across multiple benchmarks without any task-specific augmentations.

“PTRM injects Gaussian noise at each deep recursion step, enabling parallel trajectories to explore diverse solution basins, and selects among them using the model’s existing Q head. Without requiring retraining or task-specific augmentations, PTRM enables substantial accuracy gains.” — arXiv:2605.19943 abstract

The practical implication: for deterministic reasoning tasks (puzzles, logic, math proofs), you can take an existing tiny recursive model and improve its accuracy by 10-30% simply by adding noise at inference time and running a few parallel trajectories. No model modification needed.

How does RecursiveMAS apply recursion to multi-agent systems — and why does “telepathic” latent communication reduce tokens by 75%?

Standard multi-agent systems work like a chat room: Agent A generates text, Agent B reads it and generates text, Agent C reads both and generates text. Every message consumes tokens, adds latency, and accumulates error as text summaries lose information.

RecursiveMAS changes the communication channel: agents pass continuous latent representations — floating-point vectors in the model’s hidden space — through a lightweight RecursiveLink module. The module is a small learned network that transforms one agent’s latent state into a format the next agent can process.

This is described as “telepathic” communication because:

• No information is lost to text compression (a latent vector preserves more information than a text summary)
• No tokens are consumed (the communication is continuous, not discrete)
• Communication is parallelizable (multiple agent pairs can exchange latent states simultaneously)
• The RecursiveLink module is optimized end-to-end with the agents, so the latent format evolves to be maximally useful

The results from arXiv:2604.25917:

• 75.6% token reduction by round 3 (vs text-based MAS)
• 2.4x end-to-end speedup (latent communication is faster than text generation)
• 8.3% accuracy improvement (latent states preserve more information than text summaries)

The framework was evaluated under 4 representative agent collaboration patterns across 9 benchmarks spanning mathematics, science, medicine, search, and code generation. The latent approach consistently outperformed text-based alternatives across all patterns.

The inner-outer loop training algorithm deserves attention: the outer loop optimizes the whole multi-agent system, while the inner loop handles per-agent recursion. Shared gradient-based credit assignment propagates across recursion rounds — meaning later agents can influence the training of earlier agents, and vice versa.

Where do recursive models fit in production — and when should you still use frontier LLMs instead?

Recursive models are specialized reasoning engines, not general-purpose language models. The deployment boundary is clear:

Use recursive models for:

• Deterministic logic tasks: Sudoku, constraint satisfaction, puzzle solving, theorem proving
• Pattern recognition: ARC-AGI puzzles, Raven’s matrices, abstract reasoning
• Latency-critical applications: Robotics, embodied AI, real-time systems where 100ms matters
• Cost-sensitive tasks: Running 7M-parameter models locally vs API calls to billion-parameter models
• Data-scarce domains: Scientific exploration where training examples are limited (HRM achieved SOTA with 1,000 examples)

Use frontier LLMs for:

• Language understanding and generation: Creative writing, summarization, translation, conversation
• General knowledge tasks: Question answering, fact retrieval, explanation
• Code generation: Real-world software engineering (note: RecursiveMAS showed gains on code benchmarks, but general SWE requires LLM capabilities recursive models don’t have)
• Multi-modal tasks: Images, audio, video understanding — recursive models are currently text/latent only

The hybrid future:

The newsletter source describes the optimal architecture as hybrid systems — recursive models as specialized reasoning engines inside LLM-powered applications. An LLM handles the interface (understanding user intent, generating explanations, formatting output), then delegates deterministic reasoning tasks to a recursive sub-component that returns results in milliseconds rather than seconds.

The Attractor Models paper suggests another direction: equilibrium internalization. If models can learn to internalize reasoning to the point where the solver can be removed at inference, then recursive training becomes a way to produce standard feed-forward models that have internalized deeper reasoning — no recursion needed at inference time.

Frequently Asked Questions

Q: Can I run a recursive model on my laptop?

Yes. These models are 5-27 million parameters — orders of magnitude smaller than even a “small” LLM (1B+). A 7M-parameter TRM or PTRM runs easily on consumer hardware. The challenge is that recursive inference loops may require multiple forward passes, but even 50 passes through a 7M model is computationally trivial compared to one pass through a 70B LLM.

Q: Are recursive models a replacement for transformers?

No. They’re complementary. Recursive models excel at deterministic reasoning and pattern recognition. LLMs excel at language, creativity, and general knowledge. The most promising direction is hybrid systems where recursive models serve as reasoning engines inside LLM-based applications.

Q: Why can’t I just use CoT with my existing LLM?

You can — for many tasks, CoT works well. But for specific classes of problems (Sudoku, mazes, ARC-AGI), CoT fails because the problem requires exploring a solution space iteratively, not generating a linear chain of reasoning. Frontier LLMs score 0% on these tasks. Recursive models are designed specifically for iterative solution-space exploration.

Q: How do recursive models handle tasks they weren’t trained on?

Generalization is where they shine. Because recursive models have so few parameters (5-27M), they can’t memorize — they must learn general reasoning strategies. TRM achieved 45% on ARC-AGI-1 with 5M parameters, while frontier LLMs with orders of magnitude more parameters struggle. The weight-sharing across recursion steps acts as a strong regularizer.

Q: What’s the difference between HRM and TRM?

HRM uses two separate modules (H for abstract planning, L for detailed computation) in a coupled loop. TRM simplifies this to a single weight-sharing network. TRM is smaller (5-7M vs 27M), simpler, and achieved competitive results. Probabilistic TRM builds on TRM. Attractor Models are a different approach — solving for fixed points rather than iterating.

Q: Is the recursive architecture revival connected to the “Titans” and “deep thinking” trends?

Yes — they’re parallel developments. Titans (Google, 2025) introduced neural memory modules for long context. Deep thinking approaches extend reasoning through iterative refinement. The recursive architecture revival is the most radical version: tiny models that replace autoregressive token generation with latent-space iteration entirely, rather than augmenting it.

Glossary

• Recursive latent reasoning: Iterative refinement of hidden representations in a model’s latent space without emitting intermediate tokens — the core mechanism behind TRM, HRM, and related architectures
• Chain-of-Thought (CoT): An autoregressive reasoning method where models generate intermediate reasoning steps as text tokens — effective but slow and token-expensive
• Weight-sharing: Using the same parameters across multiple recursion steps, keeping model size tiny while enabling deep computation through iteration count
• Probabilistic recursion: Injecting Gaussian noise at each recursion step to explore diverse solution basins, then selecting the best trajectory — improves accuracy without retraining
• Equilibrium internalization (Attractor Models): A phenomenon where fixed-point training causes the model’s initial output to already be near equilibrium, allowing the solver to be removed at inference
• Test-time compute scaling: Improving model accuracy by spending more computation at inference (more iterations, more trajectories) rather than during training

Author

Ramsis Hammadi — AI/ML engineer specializing in GenAI, LLM engineering, and automation. Full bio →

I finally published my first Chrome extension.

It is called DataSidekick, and I built it because I got tired of fighting DevTools every time I needed to inspect or edit localStorage and sessionStorage.

Also, yes, I paid the legendary $5 Chrome Web Store developer fee, so emotionally I had to ship something. Otherwise, it would have been the worst investment portfolio of my life.

What is DataSidekick?

DataSidekick is a Chrome extension that opens as a side panel and helps developers work with browser storage in a cleaner way.

Instead of digging through DevTools and editing raw strings, you get a focused interface for:

• viewing localStorage
• viewing sessionStorage
• searching by key or value
• editing simple values inline
• editing JSON visually
• importing data from JSON
• exporting data to JSON
• hiding noisy keys
• requesting access per site/origin

You can check it out here:

https://datasidekick.site

Why I built it

Browser storage is simple until it is not.

A lot of apps store useful state in localStorage, but the default developer experience is still pretty rough:

{"user":{"name":"Rodrigo","settings":{"theme":"dark"}}}

Technically readable? Sure.

Pleasant to edit? Absolutely not.

I wanted something closer to a small developer cockpit: open the panel, find the key, inspect the value, edit it safely, and move on.

The feature I cared about most: visual JSON editing

The main idea was simple:

If a value is valid JSON, don’t treat it like a random string.

So instead of this:

{"a":"b","settings":{"theme":"dark"}}

DataSidekick displays structured JSON in a more readable way, so it becomes easier to inspect and edit nested values.

That was the first “okay, this is actually useful” moment.

Chrome permissions were the real boss fight

The extension needs to read and edit storage from the current page. That means permissions matter.

At first, I had an unnecessary permission in the manifest and the Chrome Web Store rejected the extension.

Fair enough. My bad.

The final approach is more intentional:

• no unnecessary downloads permission
• site access is requested when needed
• the user sees when access is required
• data stays local in the browser

For a tool that touches browser storage, I think this transparency is important.

Exporting without the downloads permission

One small lesson: exporting a JSON file does not necessarily require the Chrome downloads permission.

This is enough for my use case:

const blob = new Blob([JSON.stringify(payload, null, 2)], {
  type: "application/json"
});

const url = URL.createObjectURL(blob);

const a = document.createElement("a");
a.href = url;
a.download = "datasidekick-export.json";
a.click();

URL.revokeObjectURL(url);

Since I was not using chrome.downloads.download(), the permission was unnecessary.

Chrome Web Store noticed. Chrome Web Store was right.

Painful, but useful.

Current features

The first public version includes:

• Chrome Side Panel UI
• localStorage and sessionStorage support
• visual JSON editor
• key/value search
• import/export JSON
• per-site access flow
• hidden keys
• noisy key filtering
• favorites
• dark/light mode
• font size controls
• playground on the website

The playground

I also added a playground to the site so people can understand the idea before installing anything.

That was important because browser storage tools can feel a bit sensitive. I wanted people to see what the extension does before giving it access to a page.

Try it here:

https://datasidekick.site

What I learned

A few things I took away from this first release:

1. Small dev tools are still worth building.
2. Chrome extension permissions deserve real attention.
3. A simple UX decision can matter more than a complex feature.
4. Shipping to a store is very different from “it works locally”.
5. Paying $5 makes you emotionally committed.

What’s next?

This is my first Chrome extension, but definitely not the last.

Some ideas for future versions:

• IndexedDB support
• better diff view
• undo/redo
• schema validation
• storage history
• better JSON editing interactions

For now, I am happy it is live.

If you work with localStorage or sessionStorage often, I would love your feedback:

https://datasidekick.site

Or plugin link: DataSideKick

Built by a developer who just wanted to stop editing JSON like a caveman.

This is a submission for the Gemma 4 Challenge: Build with Gemma 4

AI tools remember now, but they remember in separate silos. Claude has projects, ChatGPT has personalization, Cursor indexes your codebase, and somehow you still end up re-explaining the same decisions, constraints, preferences, and project state every time you move between tools.

That felt backwards to me.

If memory is becoming part of the AI operating system, then personal context should not be trapped inside one vendor’s product. It should be portable, permissioned, local-first, and owned by the user.

So I built ContextFabric: a local AI memory layer powered by Gemma 4.

What I Built

ContextFabric is a desktop app, local daemon, memory graph, and browser extension bridge that lets AI tools share approved context without sending your personal memory to a cloud memory server.

The idea is simple:

1. Import your real project context: repos, folders, markdown, PDFs, ChatGPT exports, Claude exports, notes, and documents.
2. Gemma 4 runs locally through Ollama and extracts structured memory nodes.
3. ContextFabric stores those nodes in a local SQLite graph.
4. External tools request access.
5. You approve the request.
6. The browser extension injects the right context into Claude, ChatGPT, Cursor, Gemini, Perplexity, and other AI tools.

The five core memory node types are:

• project: what you are building
• decision: choices already made and why
• preference: stable working preferences
• style: how you communicate, design, or code
• person: collaborators and relevant human context

This is not meant to replace Claude projects, ChatGPT memory, or Cursor indexing.

It solves a different problem: your context should be portable across them.

Demo

Browser extension injection:

The demo shows the full loop:

• paste messy project context
• Gemma 4 extracts structured memory nodes
• nodes are saved locally with confidence scores
• AI Query answers with sources
• a permission request controls external access
• the browser extension injects approved context into an AI chat tool

Code

GitHub:

https://github.com/Boweii22/ContextFabric

Live Site:

https://boweii22.github.io/ContextFabric/

The project is built with Electron, React, TypeScript, SQLite, Express, Ollama, and a Manifest V3 browser extension.

The local app exposes two loopback APIs:

• 127.0.0.1:47821 for the desktop app permission/token API
• 127.0.0.1:7749 for the simple demo daemon UI and compatibility endpoints

Both are bound to loopback, not 0.0.0.0.

That matters because the privacy claim is not just a paragraph in a README. The architecture does not expose a public server for your memory graph.

How I Built It

The architecture has six parts:

User-owned sources
repos, exports, docs, notes, PDFs
        |
        v
Local ingestion
chunking + metadata
        |
        v
Gemma 4 via Ollama
extract + reason
        |
        v
SQLite memory graph
nodes + embeddings
        |
        v
Permissioned daemon
localhost only
        |
        v
Browser extension
injects context

The first hard problem was extraction.

I did not want a generic summary. I wanted durable memory. That means the model has to decide whether a piece of text contains a project fact, a decision, a preference, a style signal, or a person.

Here is the actual extraction schema prompt from the project:

export const CONTEXT_NODE_TYPES = ['project', 'style', 'decision', 'preference', 'person'] as const

export const CONTEXT_EXTRACTION_SYSTEM_PROMPT = `You are ContextFabric's local Gemma 4 context extractor.

Your job is to read one piece of user-owned context and output ONLY valid JSON.
No markdown. No prose. No comments. No trailing commas.

Extract durable context nodes that another AI assistant should remember later.
Use only facts supported by the input. Do not invent people, projects, tools, or decisions.

Allowed node types:
- project: what the user is building, maintaining, researching, or planning.
- style: how the user writes, communicates, designs, codes, or prefers answers to be shaped.
- decision: a choice already made, including why, tradeoffs, rejected alternatives, or reversibility.
- preference: a stable working preference, constraint, tool choice, privacy preference, format preference, or habit.
- person: a collaborator, stakeholder, user, client, author, or named human with relevant relationship/role context.

Return this exact JSON shape:
{
  "nodes": [
    {
      "type": "project" | "style" | "decision" | "preference" | "person",
      "title": "short human-readable title",
      "summary": "one factual sentence, max 220 characters",
      "confidence": 0.0,
      "evidence": "short direct evidence phrase from the input, max 180 characters",
      "entities": ["important names, tools, projects, people"],
      "tags": ["lowercase-keywords"]
    }
  ]
}`

The parser is intentionally defensive. Gemma 4 is good at structured output, but production code still needs repair paths.

export function parseContextExtraction(raw: string): ContextExtractionParseResult {
  const errors: string[] = []
  const parsed = parseJsonObject(raw)

  if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {
    return { ok: false, result: { nodes: [] }, errors: ['Output is not a JSON object.'] }
  }

  const root = parsed as Record<string, unknown>
  if (!Array.isArray(root.nodes)) {
    return { ok: false, result: { nodes: [] }, errors: ['Missing nodes array.'] }
  }

  const nodes: ExtractedContextNode[] = []
  for (const [index, value] of root.nodes.entries()) {
    const node = normalizeNode(value, index, errors)
    if (node) nodes.push(node)
  }

  return { ok: errors.length === 0, result: { nodes: nodes.slice(0, 6) }, errors }
}

The second hard problem was assembling context for different tools.

Claude, ChatGPT, and Cursor do not want the same payload. Claude benefits from concise prose sections. ChatGPT works well with a compact bullet brief. Cursor needs engineering-focused context.

So ContextFabric asks Gemma 4 to assemble app-aware context briefs:

export const PAYLOAD_ASSEMBLY_SYSTEM_PROMPT = `You are ContextFabric's local Gemma 4 payload assembler.

Goal:
Turn user-approved local memory nodes into one coherent context brief for another AI tool.

Rules:
- Use ONLY the supplied memory nodes. Do not invent facts, names, features, dates, metrics, or claims.
- Prefer stable project, decision, style, preference, and person nodes over raw conversation/code snippets.
- Write a useful brief, not a JSON dump.
- Include source node ids inline as [node:id] after concrete claims.
- If the nodes do not support a requested claim, omit it.
- Respect the requested app format.
- Stay under the requested maximum word count.

App formats:
- claude: concise prose with sections "Context", "Decisions", "Working Style", "How to Use This".
- chatgpt: short bullet-oriented brief with "Known Context", "Preferences", "Relevant Sources".
- cursor: engineering-focused brief with "Project", "Architecture / Decisions", "Coding Preferences", "Files / Sources".
- generic: compact neutral brief with clear source ids.

Return JSON only:
{
  "payload": "the final context brief",
  "usedNodeIds": ["node-id"],
  "warnings": ["optional warning when data is thin or uncertain"]
}`

The third hard problem was making the local model usable on normal hardware.

I hit memory issues while testing Gemma locally, so ContextFabric creates a constrained Ollama profile called cf-gemma4.

const res = await fetch(`${this.baseUrl}/api/create`, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: this.constrainedModelName,
    from: sourceModel,
    parameters: {
      num_ctx: this.runtimeContext,
      num_predict: 64,
      num_batch: 4,
    },
    stream: false,
  }),
  signal: ctrl.signal,
})

This was not about making the model weaker.

It was about making the demo run on real laptops, not just on a perfect GPU workstation.

For the local HTTP daemon, I added a small API that judges can test without understanding the whole Electron app:

compat.post('/extract', async (req: Request, res: Response) => {
  const { text, title = 'HTTP Extract', inputType = 'api', save = false } = req.body
  if (!text?.trim()) {
    res.status(400).json({ error: 'text is required' })
    return
  }

  const result = await extractNodesFromText(db, ollama, text, title, inputType, Boolean(save))
  res.json({
    ok: true,
    saved: Boolean(save),
    savedCount: result.savedCount,
    nodes: result.nodes.map(nodeToPublicJson),
  })
})

compat.get('/context', async (req: Request, res: Response) => {
  const appId = String(req.query.app || req.query.appId || 'generic')
  const query = String(req.query.query || 'current project context, writing style, technical decisions, preferences')
  const nodes = selectTokenNodes(db.getNodes(800), query, 16)
  const assembly = await assembleTokenPayloadWithTimeout(ollama, { appId, query, nodes, maxWords: 800 })
  res.json({ ok: true, appFormat: assembly.appFormat, payload: assembly.payload })
})

That endpoint is what makes the browser extension bridge simple. The extension does not need to know how the graph works. It asks the local daemon for approved context and inserts it into the active AI chat box.

Why Gemma 4

Gemma 4 is not a decorative dependency here.

It is the part of the system that turns ContextFabric from a searchable note bucket into a memory protocol.

I chose Gemma 4 E2B as the target model profile because ContextFabric is supposed to run where personal context actually lives: on laptops, desktops, and eventually smaller edge devices.

A cloud model would have defeated the core privacy constraint. If your private context graph has to leave the machine for extraction, then the product becomes a privacy policy promise instead of a privacy-preserving architecture.

A much larger local model could produce stronger answers, but it would make the product less usable for the people who need it most. The challenge specifically highlights small Gemma 4 models for edge and local use, and that is exactly the design space ContextFabric lives in.

Gemma 4 plays three roles:

1. Context extraction

It reads messy user-owned text and converts it into typed, durable memory nodes.

This is different from summarization. A summary says “what was this text about?” Context extraction asks “what should another AI assistant remember later?”

2. Conflict detection

If a new memory contradicts an existing one, Gemma 4 can mark the conflict or uncertainty. That matters because memory should not silently rot.

For example, if an old preference says “prefer short answers” and a new note says “prefer detailed long answers”, ContextFabric should surface that conflict instead of pretending both are equally true forever.

3. Payload assembly

When Claude, ChatGPT, or Cursor asks for context, Gemma 4 turns relevant graph nodes into a coherent brief with citations and a word limit.

This is where the model’s reasoning is useful: not to invent project facts, but to decide how to package approved facts for another tool.

The architecture also keeps Gemma 4 on the correct side of the trust boundary.

The normal challenge path uses Ollama locally. The daemon binds to loopback. The database is local. The browser extension talks to localhost. There is no ContextFabric cloud memory service receiving your data.

That is the difference between “we care about privacy” and “the data path cannot reach our server because there is no server in the path.”

The Bigger Picture

I do not think the long-term version of this idea is just an app.

I think it is protocol infrastructure.

HTTP made documents portable across servers. SMTP made email portable across providers. ContextFabric is an early sketch of what a personal AI context protocol could look like.

Today, every AI company is building memory as a product feature. That makes sense. Memory improves retention.

But as developers, we should ask a harder question:

Should personal AI context belong to the tool, or to the user?

My answer is the user.

That is why ContextFabric has permission requests, scoped grants, source citations, local storage, and a browser extension bridge. The extension is the adoption wedge: it makes the protocol useful before any AI company agrees to support it natively.

That was the “I never thought of it that way” moment for me.

The future of AI memory should not be one giant memory per vendor. It should be a user-controlled context layer that tools can request access to.

The browser extension is the wedge.

The protocol is the point.

Challenges I Ran Into

The hardest challenge was not building a chat UI.

It was keeping the system honest.

Early versions returned raw code chunks when I asked project-level questions. That was technically “retrieval”, but it was bad memory. I had to improve ranking so durable nodes like project, decision, style, and preference win over random bundled JavaScript or CSS.

The second challenge was local model reliability. Gemma 4 needs enough free memory, and normal laptops are messy. People have Chrome, VS Code, Docker, Discord, and ten other things open.

That led to the constrained Ollama profile, shorter prompts, fallback parsing, and clearer error messages.

The third challenge was browser injection. Claude, ChatGPT, Cursor, and Perplexity do not share one DOM structure. The extension has to find active inputs, avoid stale text areas, handle single-page-app navigation, and never crash the page if the daemon is offline.

The fourth challenge was packaging. A project that only works on my machine is not a challenge submission. I added a one-command startup path, release assets, Chrome extension packaging, screenshots, and a GitHub Pages landing page.

What’s Next

The next version is about turning the prototype into a real protocol.

My roadmap:

• publish the Chrome Web Store listing after review
• add native macOS and Windows installers
• improve LAN sync between devices
• add richer conflict resolution workflows
• publish a formal context payload schema
• build SDKs so indie AI tools can request ContextFabric memory directly
• explore a standard token format for scoped context grants

The browser extension is useful now, but the bigger win is native integration.

I want AI tools to request context the way apps request OAuth scopes, except the resource is not your Google Drive or GitHub account. It is your personal working context.

Try It Yourself

Repo:

https://github.com/Boweii22/ContextFabric

Live Site:

https://boweii22.github.io/ContextFabric/

Install Ollama:

https://ollama.com

Then run:

git clone https://github.com/Boweii22/ContextFabric.git
cd ContextFabric
npm run start

On macOS, use Node 20 or 22:

nvm install 20
nvm use 20
npm run start

On Windows, use Node 20 via fnm or nvm-windows:

fnm use 20
npm run start

Open the local demo UI:

http://127.0.0.1:7749/ui

The fastest test:

1. Paste some project context into Extract Context.
2. Click Extract and save.
3. Watch Gemma 4 create typed memory nodes.
4. Open the Claude context preview.
5. Try the browser extension bridge.

Built by Bowei Tombri for the DEV Gemma 4 Challenge.

If you build with AI tools every day, I am curious: would you rather each tool keep its own memory, or would you prefer a local memory layer that every tool has to request permission from?

You know the moment. Everyone’s picked a card, the host hits reveal, and you find out whether your team is roughly aligned or about to spend ten minutes arguing about whether a ticket is a 3 or a 5.

I run Kollabe, so I had a lot of those reveals to look at: 5,055,407 votes across 165,510 planning poker sessions. Anonymised, counted, no story-point opinions of my own attached.

The headline finding is almost boring, and that’s what makes it interesting. The cards barely matter.

The most common estimate on the planet is 3. It shows up in about a quarter of all votes. Add 1, 2, 5 and 8 and you’ve covered roughly four out of five estimates. The giant Fibonacci cards, 34, 55, 89, almost never get touched. Teams think small.

Everyone defaults to Fibonacci. A third quietly rebuild it. Fibonacci is the deck you get when you open a room, so its 60% share is partly just inertia. The real tell is the runner-up: about 31% of sessions run a fully custom deck, more than pick any other preset. And the single most popular custom deck? 1, 2, 3, 5, 8, 13. Fibonacci with the zero and the big numbers chopped off. It’s the most engineer thing imaginable. Keep the scale, delete the cards you never use.

The coffee card gets played more than 13. The little “I need a break” cup shows up in 4.3% of all votes, ahead of 13 and every t-shirt size. Someone plays it in roughly one round in eight, and 3,634 rounds officially ended on it. I have cast that card mid-session more than once, so no judgement here.

Teams agree on the first vote about 1 in 6 times. Only 16.6% of rounds come up unanimous on the reveal. The average round has more than two different numbers on the table. That gap isn’t a failure of the process. It is the process.

The rest of the breakdown, with charts for decks, days of the week, integrations and consensus, is here: Planning Poker Statistics 2026.

A rule of thumb I’ve started repeating: if your reveals are always unanimous, you’re not estimating, you’re rubber-stamping. The disagreement is where the useful conversation lives. If it never happens, the cards are theatre.

Fair caveat: this is data from teams who chose Kollabe, so it leans toward people who like running quick, low-ceremony poker. But five million votes is enough that the shape holds. Small numbers dominate, and almost nobody wants a 55-point card.

If you want to run a round this week, Kollabe poker is free and needs no signup. Or open whatever you already use and check: how often does anyone actually pick a number above 13?

L.E.N.S. (Local Edge Native Studio) is a voice-guided photography coach that runs Gemma 4 E4B locally through Ollama — so a maker can verify and improve product photos before listing, without sending images to the cloud and without asking someone sighted to “just check this one.”

Gemma 4’s native multimodal vision is the engine: each coaching turn sends a real product photo (base64 in the Ollama chat) and gets back structured JSON the app validates before speaking.

🔗 Try it (no install): lens-app-gemma4.vercel.app

📹 Demo video: YouTube walkthrough

💻 Source: github.com/prasadt1/photography-coach-gemma4 (Apache 2.0)

What I Built

I built L.E.N.S. for someone like Mohan — a low-vision artisan who hand-knits sweaters to sell online. He can judge the knit by touch: tension, pattern, finish. What he cannot reliably judge is the photograph of the piece. Is it in focus? Is the light flat? Is the sweater cropped awkwardly or lost against the background? On a marketplace like Etsy, the photo is the product; a weak photo quietly costs the sale. Until now, that step has meant borrowing someone else’s eyes.

L.E.N.S. closes that gap.

1. The maker points their camera and takes a photo.
2. Gemma 4 E4B — on their own machine, via Ollama — assesses framing, lighting, focus, and composition from the image itself (multimodal input, not a text-only description).
3. L.E.N.S. speaks back one specific, actionable fix: not “this photo is bad,” but “move back about six inches” or “the light is behind the sweater — turn toward the window.”
4. They take a second photo; L.E.N.S. compares the two images out loud and says which is stronger and why.
5. It drafts copy-ready listing text — title, description, and alt-text — ready to paste into their store.

It is voice-first by design, not a visual UI with audio bolted on. I built and tested the flow with a screen reader on and the screen off, because that is how it will actually be used. Structured JSON is an accessibility choice too: the client validates a strict schema and surfaces discrete, ordered points, so coaching stays one fix at a time instead of a wall of feedback the user cannot skim.

I designed for the hardest case — a blind maker, fully offline — and by the curb-cut effect, the same coaching helps any maker without a photographer or a reliable connection.

Alt: infographic of five steps — artisan capture, on-device analysis, voice feedback loop, compare and iterate, then listing copy for Etsy or Shopify.

Demo

Full walkthrough: first photo, spoken coaching, stronger retake, comparison, generated listing.

Try it live

No account. No tracking. Copy-ready output only — L.E.N.S. does not auto-publish to Etsy or Shopify.

Code

Source, README, architecture notes, and spike write-ups:

Stack: React 19 + TypeScript PWA, optional Electron desktop build, Ollama for local multimodal inference, Web Speech API for coach voice output.

Repo highlights:

• Strict JSON contract — one schema drives description, colour check, single fix, alt-text, and listing copy.
• Three honestly labelled inference modes (see below).
• Spike docs: Spike 1 — E4B via Ollama, quantization study, LiteRT iOS spike.

This is original work I built for accessibility-first product photography coaching; the repo is not a repackaged template.

How I Used Gemma 4

Gemma 4 is the core of L.E.N.S.: multimodal photo assessment and coaching generation. Every model and runtime choice followed from local-first privacy and voice-loop latency.

Why Gemma 4 E4B (and what I ruled out)

The Gemma 4 family spans small edge models, 31B Dense, and 26B MoE. For this project:

E4B is the deliberate middle: the trade-off is the project.

What E4B unlocked for this project

• Multimodal vision on-device — real product photos in, structured coaching out (framing, light, focus, colour), not text-only guesses.
• Offline independence — the product path never requires sending photos to a remote API.
• Usable voice-loop latency — ~4B + Q4_K_M + streaming TTS ≈ ~20s warm (down from ~40s early on).
• Strict JSON coaching — one spatial fix, two-photo compare, listing copy — all from schemas Ollama enforces at generation time.
• Honest dual deploy — E4B for the real maker story; 31B only where judges need a zero-install upload path.

Multimodal + structured output (how it’s wired)

Each analyze call sends the image in Ollama’s messages[].images[] array and asks Gemma 4 E4B for JSON via Ollama’s format field (JSON Schema). The client validates before TTS speaks:

// services/ollamaService.ts — simplified
const messages = [
  { role: 'system', content: buildSystemPrompt(/* artisan coaching */) },
  { role: 'user', content: userPrompt, images: [base64ProductPhoto] },
];

await fetch(`${OLLAMA_BASE}/api/chat`, {
  method: 'POST',
  body: JSON.stringify({
    model: 'gemma4:e4b',
    messages,
    format: ARTISAN_V3_OUTPUT_SCHEMA,  // Ollama enforces JSON shape
    stream: true,                       // TTS starts before generation ends
    options: { num_predict: cappedTokens },
    keep_alive: '30m',
  }),
});

The artisan schema drives fields like scene description, one priorityFix, alt-text, and listing title/description — so VoiceOver/TalkBack and the coach voice never drown the maker in a paragraph of fixes.

Runtime: Ollama

I spiked Cactus and llama.cpp as well. Ollama won for the cleanest local multimodal serving and the simplest path to multiple inference modes without rebuilding the pipeline each time.

Quantization: Q4_K_M

On modest hardware, Q4_K_M keeps E4B runnable without meaningfully hurting visual assessment. Lighter quants started to cost coaching quality; heavier ones were not worth the memory for this use case.

Latency and voice

Early warm inference was ~40s — too long for a spoken coaching loop. Prompt tuning, a token cap, a warm-up call on startup, and streaming brought warm runs to roughly 20s.

Three honest inference modes

I also spiked LiteRT for true on-device iOS inference (~25 tok/s in Google’s reference app). That is Phase 2 — documented as roadmap, not claimed as shipped. Today, iOS is covered by the installable PWA talking to Ollama on the Mac (same Wi‑Fi or tunnel).

Why local Gemma matters

Privacy here is not a bullet point — it is the mechanism of independence. A cloud coach swaps one dependency for another: instead of a sighted helper, you need connectivity, an account, and a server that receives your product photos. A capable Gemma 4 model on the maker’s own hardware is what makes “I can list this myself” real.

Accessibility (why the UX matches the model story)

• Voice-first with an equivalent labelled control for every voice action.
• Screen reader: landmarks, live regions, managed focus; coach TTS works alongside VoiceOver/TalkBack, not instead of it.
• One fix at a time — same discipline in prompt design and UI.
• Anti-hallucination — states uncertainty when the image does not support a claim.
• Multilingual coaching paths in the prompt layer.

What’s next

• Native on-device iOS via LiteRT (spike done; integration is post-hackathon).
• More languages and tighter cold-start latency.
• Deeper maker workflows (batch listing prep) — still local-first.

Links

• This challenge: Gemma 4 Challenge on DEV
• Live demo: lens-app-gemma4.vercel.app
• Product / video deploy: photography-coach-gemma4.vercel.app
• GitHub: photography-coach-gemma4
• Demo video: youtu.be/qoDLKzzcYHM

You do not need to be a coder or developer to understand open-source communities. If you are curious about how developers actually work, collaborate, and contribute to open-source projects, then this article is for you. In this article, I will walk through the step-by-step process of how developers contribute their skills and participate in open-source community projects by using K-serve as an example.

This article is also helpful for user experience researchers who already work with developers but want to better understand how developers work on projects and use GitHub. It is also useful for UX designers who are interested in learning about the developer experience (DevEx) domain and becoming developer experience designers. Additionally, this article is intended for non-technical people who want to learn how developers and open-source communities work, and how GitHub repositories are used to manage, collaborate on, and operate projects.

Let’s learn what is Kserve? KServe is an open-source Kubernetes-based model serving platform that enables scalable and reliable deployment of machine learning models in production environments. I chose KServe because it sits at the intersection of machine learning, Kubernetes, platform engineering, and open-source collaboration, making it a rich example for understanding real-world engineering workflows.

It is important to understand that engineers are not simply “writing code”; instead, they are continuously discovering problems, discussing solutions, implementing changes, reviewing and refining decisions, validating outcomes, and maintaining the system over time.

This entire workflow—spanning collaboration, decision-making, and iteration—leaves behind valuable traces in GitHub through issues, pull requests, discussions, and commits, which together reveal how complex systems are actually built and evolved.

What is Github repository?

A GitHub repository is created by developers, project maintainers, or organizations as a central place to store, organize, and manage a project’s code, documentation, issues, and collaboration history. Once created, it is used by a wide range of people, including developers who write and maintain code, contributors who fix bugs or add new features, reviewers who evaluate and approve changes, and non-developers such as UX researchers, product managers, and technical writers who explore the project to understand it or contribute documentation and feedback.

GitHub repositories are important because they reveal not just the final software product but the entire development process behind it. Developers use them to collaborate, track issues, discuss and review changes, manage tasks, and maintain transparency in their work. Learning how repositories function helps in understanding real developer workflows, decision-making processes, and how complex software systems evolve over time in both open-source and professional environments.

I am going to explain how developers and the open-source community work with entire GitHub repository in 5 simple steps.

Step : 1. The Project Exists as a Living System

Imagine that KServe already exists and is being used by people around the world. Its users include ML engineers who deploy and manage machine learning models, platform engineers who build and maintain the underlying infrastructure, MLOps teams who ensure smooth model lifecycle operations, Kubernetes administrators who handle cluster configuration and deployment, and researchers who experiment with and evaluate machine learning systems.

Most people think software development means typing code all day. In reality, a typical improvement starts with a real person running into a problem, and ends with millions of users getting a better experience. Here’s how that journey unfolds, using KServe — a tool that helps AI models run in the cloud — as the example.
Let me show you the big picture first, then zoom into each phase.

Now let’s look at each phase in plain language, with a closer diagram for the parts that need more detail. For example: A user(Engineer) deploys an inference service.

Step :2 A User Encounters a Problem & Created the issue.

Imagine you’re an engineer who uses KServe to run an AI model. You follow all the instructions, but when you try to use it, nothing responds. You’re not sure if you did something wrong, or if there’s a bug in the software.

So you do what most people do: check the logs, search the documentation, ask the community — and eventually open a “bug report” (called an issue on GitHub). This is where the real work begins, even though no one has touched code yet.

When you file that issue, volunteer maintainers (people who help run the project) step in to classify it. Is this a bug? A misunderstanding? Something missing from the docs? They ask questions to narrow it down

Step: 3 Can I make it break on my machine?

Before anyone can fix a problem, they have to be able to see the problem. An engineer sets up the same software environment as the user and tries to make the bug happen intentionally. This is called reproducing the bug.

Once they can make it fail consistently, they start playing detective — tracing the path a request takes through the system, reading log files, and asking: where exactly does things go wrong?
This phase is mostly thinking, not typing. Engineers build a mental map of how the system is supposed to behave, then look for where reality diverges from that map. It looks like this:

Step: 4 Choosing the right fix (not just any fix)

Once the root cause is found, the next question isn’t can we fix it? — it’s what’s the safest way to fix it?

In the KServe example, the bug was that a missing network port would cause a silent, invisible failure. Engineers debated three possible solutions:

This kind of debate — often hundreds of comment threads for just a few lines of code — is what separates good software from brittle software.

Step : 5 From code to released fix

Only after all that thinking does actual coding begin. The engineer writes the fix in a separate “branch” (a safe copy of the code), then submits it as a “pull request” — essentially saying “here’s my proposed change, please check my work.”

Other engineers review it, suggest improvements, and run automated tests. When everyone agrees it’s safe, it gets merged into the main codebase and released.

The ending: and why it never really ends

After the update is released, what changed for the user? Before: “It just doesn’t work.” After: “Configuration invalid: missing port.” That’s a small sentence, but it means the user now understands what to fix instead of guessing blindly.

And here’s the final truth: the moment one problem is solved, a user somewhere will notice the next thing that could be better. The loop starts again. That’s not a flaw in open source — it’s the whole point. Software improves because real people keep using it, keep running into its rough edges, and keep caring enough to speak up.