Mert

7 Next.js 16 Caching Bugs That Compile Fine and Break Silently in Production

yazar

I lost hours debugging a Next.js 16 caching issue that had no error, no warning, and only showed up in production.

The Next.js 16 caching model is genuinely good. But it introduces a class of bugs that are harder to detect than anything in previous versions: bugs that look correct, compile without errors, deploy successfully, and then silently misbehave in production.

These are the most common ones I’ve seen across real projects. Every one comes from real production incidents.

(Assumes you have cacheComponents: true enabled in next.config.ts.)

This is a follow-up to my previous post where I built a dev-only debugger to surface these issues during development. That tool helps you detect them. This post breaks down the exact failure cases behind those warnings.

Bug 1: 'use cache' on the Wrapper Instead of Inside the Function

// This looks cached. It is not.
export const getProducts = someWrapper(async function() {
  'use cache'
  cacheLife('hours')
  return db.query('SELECT * FROM products')
})

The 'use cache' directive tells the Next.js compiler to treat that function as a cache boundary. When you wrap it, the compiler sees the wrapper as the entry point. The inner function may be cached, but the wrapper becomes the execution boundary, so you still end up running it on every request.

No error. No warning. Just a function running on every request when it should be cached.

Fix: 

async function _getProducts() {
  'use cache'
  cacheLife('hours')
  cacheTag('products')
  return db.query('SELECT * FROM products')
}

// Wrapper receives the already-cached function
export const getProducts = someWrapper(_getProducts)

The rule is simple: 'use cache' lives inside the data function, never on anything that wraps it.

Bug 2: Deprecated revalidateTag That Compiles and Uses Legacy Behavior

// Next.js 15: correct
// Next.js 16: TypeScript error, silently uses legacy behavior in loose tsconfig
revalidateTag('products')

In Next.js 16, revalidateTag without a second argument is deprecated and produces a TypeScript error. But if your tsconfig is not in strict mode (common in older projects), it compiles cleanly and falls back to legacy invalidation behavior instead of the new SWR-based system.

Pages stop reflecting mutations. No error anywhere.

Fix: 

revalidateTag('products', 'max')         // SWR, recommended for most content
revalidateTag('products', { expire: 0 }) // Immediate expiry for webhooks/payments

Run npx @next/codemod@canary upgrade latest during your migration, it handles this automatically. But check your tsconfig strictness regardless.

Bug 3: Tag String Mismatch Across Files

// lib/data.ts -- written by developer A
async function getProducts() {
  'use cache'
  cacheTag('product-list')  // Tag is 'product-list'
  return db.query('...')
}

// app/actions/products.ts -- written by developer B
export async function createProduct(data: ProductData) {
  await db.query('INSERT INTO products ...', [...])
  revalidateTag('products', 'max')  // Different string, invalidates nothing
}

Two different strings. Zero errors. The product list never refreshes after a new product is created. Users see stale data until cacheLife expires on its own.

Fix: 

// lib/tags.ts -- one source of truth
export const tags = {
  products: 'product-list',
  product: (id: string) => `product-${id}`,
  user: (id: string) => `user-${id}`,
} as const

// Both files import from tags
cacheTag(tags.products)
revalidateTag(tags.products, 'max')

Tag typos become TypeScript errors. The string mismatch bug becomes structurally impossible.

Bug 4: Server Action Mutation Where the Acting User Sees Stale Data

'use server'
export async function updateProductPrice(id: string, newPrice: number) {
  await db.query('UPDATE products SET price = $1 WHERE id = $2', [newPrice, id])
  revalidateTag(`product-${id}`, 'max')
}

revalidateTag with any named profile uses stale-while-revalidate. It marks the cache as stale. The next request still gets the cached version while fresh data loads in the background.

For the admin who just clicked save, that means they navigate to the product page and see the old price. Looks like the save failed. Causes confusion and duplicate mutations.

Fix: 

'use server'
import { revalidateTag, updateTag } from 'next/cache'

export async function updateProductPrice(id: string, newPrice: number) {
  await db.query('UPDATE products SET price = $1 WHERE id = $2', [newPrice, id])
  updateTag(`product-${id}`)            // Acting user sees change immediately
  revalidateTag(`product-${id}`, 'max') // Everyone else gets SWR
  revalidateTag('products', 'max')      // Product list also refreshes
}

updateTag expires the cache entry immediately. The next request waits for fresh data. The admin sees their change. Everyone else gets the fast SWR treatment.

Constraint: updateTag only works inside Server Actions. In Route Handlers, use revalidateTag(tag, { expire: 0 }) instead.

Bug 5: updateTag in a Route Handler

// app/api/webhooks/stripe/route.ts
import { updateTag } from 'next/cache'

export async function POST(req: Request) {
  const event = await parseStripeWebhook(req)
  if (event.type === 'price.updated') {
    updateTag('products')  // Throws at runtime
  }
  return new Response('ok', { status: 200 })
}

This compiles. It deploys. On the first real webhook call from Stripe, it throws at runtime. updateTag only works inside Server Actions. Calling it anywhere else throws.

Fix: 

import { revalidateTag } from 'next/cache'

export async function POST(req: Request) {
  const event = await parseStripeWebhook(req)
  if (event.type === 'price.updated') {
    revalidateTag('products', { expire: 0 })  // Immediate expiry in Route Handlers
  }
  return new Response('ok', { status: 200 })
}

Bug 6: Short cacheLife That Silently Affects PPR

async function LiveStockStatus({ productId }: { productId: string }) {
  'use cache'
  cacheLife('seconds')  // Seems right for live stock data
  cacheTag(`stock-${productId}`)
  return fetchStockLevel(productId)
}

cacheLife('seconds'), revalidate: 0, and expire under 5 minutes are automatically excluded from the PPR static shell. They become dynamic holes that run at request time.

One component with cacheLife('seconds') can push parts of the page out of the static shell and turn them into request-time work. No warning. The page still works. It just becomes fully dynamic without any obvious signal.

Fix, if the data can tolerate a short delay: 

cacheLife('minutes')  // Now included in the static shell

Fix, if it genuinely needs to be live: 

// Parent page
<Suspense fallback={<StockSkeleton />}>
  <LiveStockStatus productId={id} />  {/* Streams in after static shell */}
</Suspense>

The second approach is the correct PPR pattern for truly dynamic data. The static shell renders instantly and the live data streams in after.

Bug 7: Runtime API Inside a Cached Scope

async function UserHeader() {
  'use cache'
  const cookieStore = await cookies()  // Throws at build time
  const user = await getUser(cookieStore.get('user-id')?.value)
  return <div>{user.name}</div>
}

cookies(), headers(), and draftMode() are runtime APIs. They read request-specific data. They cannot live inside a 'use cache' scope because cached output is stored and replayed across requests.

This one at least throws at build time with “Uncached data was accessed outside of Suspense”. But the error gives you no component name, no file path, and no useful stack trace. You get to play binary search across your codebase to find it.

Fix: 

// Read runtime values OUTSIDE the cached scope
async function UserHeader() {
  const cookieStore = await cookies()
  const userId = cookieStore.get('user-id')?.value
  return <CachedUserProfile userId={userId} />
}

// Pass the VALUE as a serializable prop to the cached component
async function CachedUserProfile({ userId }: { userId?: string }) {
  'use cache'
  cacheLife('hours')
  cacheTag(`user-${userId}`)
  if (!userId) return <GuestGreeting />
  const user = await getUser(userId)
  return <div>{user.name}</div>
}

userId is a string so it becomes part of the cache key automatically. Different users produce different cache entries without any manual key construction.

The Common Thread

None of these have good error messages. Five of the seven compile and deploy without complaint. The other two throw, but either without enough information to find the cause quickly or only after the first real production request.

The pattern across all of them is the same: the new caching model requires explicit correctness. When you get something wrong, it does not always tell you.

If you are in the middle of a Next.js 16 migration and want to catch these during development rather than in production, I ended up building a free dev-only debugger that logs cache misses, dynamic holes, missing tags, and deprecated invalidation calls directly in your terminal. Zero production cost, one .tsx file: Next.js cache debugger.

And if you want these patterns enforced at the type level so the wrong call is a compile error rather than a runtime surprise, the production enforcement layer is Cache Pro Kit.

Have you run into any of these? Or something even stranger? I’m curious what the distribution looks like across different kinds of projects.

I got tired of writing READMEs so I built a tool that generates them from your GitHub URL

yazar

Every time I finish a project the last thing I want to do is write a README. So I built Readify. Paste a GitHub URL, it scans your repo (stack, dependencies, file structure) and generates a clean professional README in seconds. Badges, install steps, features, all of it. Just copy and paste into your repo.
Built it with Next.js, Supabase and the Anthropic API. Free to try, no signup needed for the first few generations.
Would love some feedback! What sections are missing, what could be better? Link: https://readify-virid.vercel.app

FrontGate: a Lightweight Package Proxy for Supply Chain Security

yazar

Supply chain attacks are starting to feel like part of the daily routine.

You grab your morning coffee, open the laptop, and check which package ecosystem is on fire today.

Malicious packages, compromised maintainers, typosquatting, dependency confusion, suspicious new releases — the public package ecosystem is powerful, but we also trust it a lot by default.

There are mature tools for this. JFrog, Sonatype, Synk, and similar platforms exist for a reason. But not every small team, startup is ready to buy and operate a full artifact-management platform.

So I started thinking about the gap between:

“we do nothing”

and

“we adopt a full enterprise solution.”

The idea was simple:

What if package installs had a lightweight policy checkpoint?

Kind of like Cloudflare, but for dependencies.

Your existing tools still work mostly the same: 

pip install ...
uv add ...
poetry add ...

But instead of talking directly to PyPI, they go through a small proxy first.

That proxy can decide whether to allow, block, warn, or log a package request based on policy. For example:

  • block packages released in the last 7 days;
  • block yanked versions;
  • block certain licenses;
  • block known-bad publishers or package names;
  • log every decision for audit/debugging.

That became FrontGate.

FrontGate is a small Go-based POC that sits between Python package clients and PyPI. It implements a PEP 503-compatible proxy surface, supports policy config via YAML, exposes Prometheus metrics, and has integration examples for pip, Poetry, uv, and CI.

It is intentionally not a private registry.

It does not try to store artifacts.

It is just a policy layer before dependencies reach your resolver.

The first use case I wanted to explore is release-age gating: blocking packages that are too new. It is not a silver bullet, but it can reduce exposure to freshly published malicious versions before the ecosystem has time to react.

But I think the idea is worth testing:

Can we make dependency installation a little less blind without introducing a heavy platform?

Repo: https://github.com/maxkrivich/frontgate

Curious if this is a problem other people have run into, especially in small teams or platform/security work where “buy the full thing” is not always the first option.

Why Your Expense Tracking Architecture Keeps Breaking

yazar
DigitXBooks Expense screenshot in English

Most developers treat expense tracking as a simple CRUD operation—a user uploads a receipt, adds a dollar amount, and hits save. Then, reality hits in the form of month-end reconciliation, missing audit trails, and the inevitable “why is this category wrong?” Slack message from the finance team.

After building and maintaining business-facing accounting tools, I’ve learned that the complexity isn’t in the database schema; it’s in the messy, high-friction handoff between the point of purchase and the general ledger. When we look at standard expense management workflows, we often ignore the operational debt that accumulates when a system doesn’t enforce context at the moment of capture.

Workflow screenshots

These screenshots are useful here because they hint at where Expense either stays grounded in the user’s real task or becomes another detached admin screen.

DigitXBooks Expense screenshot in English

The Anatomy of Expense Friction

When we build or integrate expense systems, we tend to design for the “happy path”: the user remembers the expense, categorizes it correctly, and attaches a high-resolution receipt. In production, that path is the exception. The reality is usually a mobile upload from a shaky train, a missing receipt, or a misclassified transaction that breaks the tax-readiness of the entire ledger.

1. The Context Gap

Software engineers often underestimate the importance of metadata. If an expense is just a dollar amount and a date, it’s useless to an accountant. To make an expense actionable, you need the “why.” Was it travel? Software subscriptions? Client meals? Without strict validation or intelligent suggestion at the point of entry, the system becomes a repository of “Miscellaneous” line items that eventually require hours of manual cleanup.

2. The Approval Bottleneck

If your architecture treats approvals as a simple boolean flag in a database, you’re missing the audit trail. A robust system needs to track state transitions—submitted, pending, flagged, approved, or rejected—alongside the user who initiated the change. When these states aren’t clearly defined in the backend, you end up with “zombie expenses” that sit in limbo because the workflow didn’t force a clear handoff between the requester and the approver.

Engineering Better Financial Workflows

If you are building or integrating accounting features, stop thinking about data storage and start thinking about data integrity. Here are three architectural principles that save teams from financial headaches:

  • Enforce Immutability for Audit Trails: Once an expense is submitted and verified, it should be locked. If an edit is required, generate a new version record rather than overwriting the original. This is non-negotiable for compliance.
  • Asynchronous Processing for AI Enrichment: Don’t make the user wait for OCR or category suggestions. Handle receipt scanning as a background job, updating the UI via WebSockets or polling once the enrichment is complete. This keeps the UX snappy while ensuring the data is actually useful.
  • The “Reconciliation First” Mindset: Build your database models to map directly to ledger accounts. If your expense module doesn’t speak the same language as your general ledger, you are just creating another silo that will eventually need to be reconciled by hand.

The Hidden Cost of “Easy” Features

There is a trend in modern SaaS to abstract away the “boring” parts of accounting to keep the UI clean. While this helps with user adoption, it often hides the operational reality that accounting software isn’t just a digital notebook—it’s a data integration pipeline.

When we look at platforms like DigitXBooks, the goal shouldn’t be to make expense entry “invisible,” but to make it “precise.” Providing the user with immediate feedback on why a receipt might be rejected or why a category selection is mandatory reduces the back-and-forth friction that kills productivity. Engineering for transparency—rather than just simplicity—is what separates a hobby project from a production-grade tool.

Practical Lessons for Builders

If you’re currently designing a financial workflow, do yourself a favor and map out the “negative paths” first. What happens if the receipt is blurry? What happens if the user exceeds their monthly budget? What happens when an expense is rejected three weeks after it was incurred?

If your code handles these edge cases gracefully, your users won’t just have a functional app; they’ll have a reliable system that actually saves them time during the month-end crunch. Remember, the goal of automation is to remove the manual toil, not to move the manual work from the user to the accountant.

Disclosure: This article was drafted with AI assistance from product screenshots, current trend cues, and strict human-written constraints for DEV Community style.

Closing thought

In products like DigitXBooks, the hard part of expense is rarely the screen itself. It is the product decision behind it: whether the workflow helps people act with confidence or pushes the real complexity into cleanup later. If you care about building calmer finance and operations software, follow along. I keep sharing the tradeoffs that only show up once real teams start using the product.

Question for builders

How are you designing expense in your own product so it stays useful in the moment without making the accounting side harder to trust?

Stop your AI trading agent from hallucinating technical analysis

yazar

Ask any LLM “is NVDA bullish here?” and it will answer with total confidence — a cup-and-handle here, a win rate there, an “expect a 4% move.” The problem: most of it is invented. Language models are fluent in the vocabulary of technical analysis but have no grounded source for the numbers. They’ll give you a win rate they made up.

If you’re building an agent that touches markets, that’s a real liability. The fix isn’t a better prompt — it’s a grounded tool the agent can call instead of guessing.

The idea: a base-rate engine, exposed over MCP

Chart Library is an MCP server that answers one question with real history: “given a setup that looks like this, what did statistically similar setups do next?” It returns the cohort of ~40–50 historical analogs across 10 years and 19K+ symbols, plus the forward-return distribution (median, win rate, percentile bands) — every number a verified historical fact, not a generation.

Because it speaks the Model Context Protocol, any MCP-aware client — Claude Desktop, Cursor, Cline, LangChain, the OpenAI Agents SDK — can use it in a couple of lines.

30 seconds to grounded answers

pip install chartlibrary-mcp

LangChain / LangGraph: 

from langchain_mcp_adapters.client import MultiServerMCPClient
from langgraph.prebuilt import create_react_agent
from langchain_anthropic import ChatAnthropic
import os, asyncio

async def main():
    client = MultiServerMCPClient({
        "chartlibrary": {
            "command": "chartlibrary-mcp",
            "transport": "stdio",
            "env": {"CHART_LIBRARY_API_KEY": os.environ["CHART_LIBRARY_API_KEY"]},
        }
    })
    agent = create_react_agent(ChatAnthropic(model="claude-sonnet-4-6"),
                               await client.get_tools())
    out = await agent.ainvoke({"messages": [{"role": "user",
        "content": "What did setups like NVDA on 2024-08-05 do next? Cite the 5-day distribution."}]})
    print(out["messages"][-1].content)

asyncio.run(main())

The agent calls the cohort tool, gets the real distribution, and answers from it. No hallucinated stats.

The honest part (which is the whole point)

Here’s something most “AI alpha” tools won’t tell you, validated on 300K+ historical observations: chart shape predicts the size of the next move far better than the direction. Conditioning on a “bullish-looking” cohort earns a real magnitude edge (bigger average moves) but no win-rate edge — direction stays a coin flip. Our backtest shows it cleanly: a “cohort bullish” rule prints +0.77% avg vs +0.34% baseline, but 52% wins vs 54% baseline.

That’s not a bug — it’s the truth, and it’s why grounding matters. A tool that returns calibrated base rates (and tells you when it doesn’t know) is more useful to an agent than one that confidently predicts a direction it can’t.

What the agent gets

Eight read-only tools (all annotated readOnlyHint): search, cohort (the distribution primitive), discover, analyze, context, narrative, explain, portfolio. Free Sandbox tier (200 calls/day, no auth) covers search + context + explain; the full cohort depth is a paid tier.

Try it

If you’re building anything that reasons about markets, give your agent a source of truth instead of a confident guess.

Breaking the Monorepo Barrier in a Crypto Store for Digital Products

yazar

The Problem We Were Actually Solving

As an engineer, my primary focus was ensuring our store could handle any data we might collect, considering our restrictions. Traditional e-commerce platforms were out, so I built our store directly on top of IPFS. The IPFS advantage of decentralized data storage allowed us to work around the usual payment issues. With a monorepo setup, I could handle all frontend and server-side logic in a single repository. The idea behind this decision was to simplify the development process and reduce the load on our servers, which are usually the bottleneck in complex applications.

What We Tried First (And Why It Failed)

Initially, I used a Next.js monorepo setup, with a clear separation of frontend and server logic. While this seemed like a good idea, it led to a significant increase in our build times. As we started to onboard new team members, it became apparent that managing the build process was almost as difficult as writing the application itself. The monorepo was meant to simplify things, but we quickly found ourselves wasting hours on merging code and debugging build-related issues.

The Architecture Decision

To get around these issues, I switched to a modular, multi-repo setup with a separate repository for our server logic. This allowed us to easily manage separate build processes for our frontend and backend code. We then used a third-party service for our IPFS data storage, which significantly reduced our development time and made it easier for the team to manage our codebase. In addition, we replaced the complex setup of our Next.js monorepo with a more straightforward React setup, which allowed our developers to focus on the logic of the application rather than dealing with an overly complex build process.

What The Numbers Said After

After implementing our new architecture, our build times decreased by 30%, and our code quality improved significantly. Our team spent less time on debugging and more time on actually writing code that mattered. The number of pull requests merged daily also went up, signaling that our team had made a crucial step forward in productivity.

What I Would Do Differently

If I had to do it all over again, I’d implement the modular setup from the start. It’s clear now that this approach would have saved us time and resources in the long run. I’d also consider using a more robust tooling setup to manage our monorepo, like Yarn Workspaces or Nx. In hindsight, making the switch to a multi-repo setup might have been overkill, but the benefits we achieved were worth the investment.

Imposter Syndrome Is Something We All Struggle With at Some Point in Our Careers

yazar
Developer

You got the job.

Take a second. Let that sink in. You applied, you survived the three-round interview that somehow included a whiteboard problem nobody would ever solve in real life, and they picked you.

So why does it feel like you’re about to get caught?

That feeling has a name. Imposter syndrome. And if you work in tech, it’s basically a rite of passage, like learning that tabs vs. spaces is apparently worth dying on a hill over, or discovering that “it works on my machine” is not, in fact, a solution.

The Offer Letter Doesn’t Fix Anything

Here’s what nobody tells you when you land the job, the promotion, or the senior title: “the anxiety doesn’t go away. It just upgrades itself.”

Junior developer you worried about writing bad code. Now you worry about reviewing someone else’s bad code and not noticing it’s bad. Congratulations. You’ve unlocked a new difficulty level.

The first week at a new job is a particular kind of psychological workout. You smile through onboarding. You nod confidently when someone explains the codebase, even though what they’re describing sounds like it was architected by someone who really loved spaghetti. You spend your lunch break reading internal documentation and feeling like you’re studying for an exam you didn’t know existed.

And somewhere in the middle of all that, the thought creeps in: everyone here seems to know exactly what they’re doing. What am I doing here?

The answer, which you won’t believe for at least six months, is that they don’t. They’re just further along in pretending they do.

The Meeting Where You Almost Said Something

Every developer has this story.

You’re in a technical discussion. There are diagrams on a whiteboard, or a Figma screen someone is sharing, or a Jira ticket that has somehow become a philosophical debate. And you have a thought. A real, legitimate, potentially useful thought.

But then the internal committee convenes.

What if I’m missing something obvious? What if I say this and someone just goes “yeah, we already tried that in 2019”? What if this is the moment they all realize I don’t belong here?

So you sit on it. You watch the meeting run in circles. You wait for someone else to say the thing you were thinking, and when they do, you feel equal parts relieved and annoyed at yourself.

Or, on a braver day, you say it. Quickly, casually, like it’s barely worth mentioning. And it lands. The tech lead nods. The discussion moves forward. The meeting ends.

And instead of feeling good about it, you think: lucky.

That’s imposter syndrome doing its best work. It takes your wins and files them under coincidence. It takes your struggles and files them under evidence. The accounting is completely rigged, and somehow you keep letting it audit you.

The Five Characters You’ll Recognize

Imposter syndrome doesn’t have one face. It shows up differently depending on who you are and where you are in your career. After enough years in this industry, you start to recognize the types.

There’s the Perfectionist, who ships the feature, gets good feedback, and immediately opens the PR to look at everything they’d do differently. The code works. Users are happy. Somewhere, a single variable name is bothering them deeply.

There’s the Natural Genius, who learned to code fast, got praised for it, and now treats any moment of difficulty as a personal betrayal. A new language that doesn’t click in three days isn’t a new language. It’s a verdict.

There’s the Soloist, who would rather lose an entire afternoon to a bug than spend five minutes asking for help. Because asking means admitting a gap. And a gap means exposure. Never mind that the person they’d ask has been in the same gap twice this week.

There’s the Expert-in-Waiting, who is one course, one certification, one side project away from feeling qualified. The job posting asks for five things. They have four. They do not apply. The job posting asks for three things. They have all three, but the wording of the third one feels slightly different from their experience. They do not apply.

And there’s the Superhero, who compensates by just doing more. More tickets, more late nights, more responsibilities taken on without being asked. As if eventually the output will be loud enough to drown out the voice that says it still isn’t enough.

Most developers aren’t just one of these. They’re a rotating cast depending on the week, the project, and how the last code review went.

Why This Industry Is Practically Designed to Make You Feel This Way

Imposter syndrome exists everywhere, but tech has really optimized for it.

The technology moves faster than anyone can keep up with. There is always a new framework getting hype, an old one getting deprecated, and a Medium article explaining why everything you currently know is about to be irrelevant. You cannot win. You can only stay in motion and try not to read the comments.

Then there’s the visibility. Your work lives in public in a way most professions don’t experience. Pull requests that can be picked apart line by line. GitHub histories that go back years. Stack Overflow answers aging in real time. When you’re wrong in this industry, there is often a timestamp on it.

And the comparison culture is relentless. Everyone’s profile is a highlight reel. The person who seems to know everything has simply been doing this longer and has quietly deleted their early commits. The conference speaker you admire spent a year not knowing how DNS worked. You just didn’t see that part.

A survey found that 58% of tech workers, including people at Google, Amazon, and Meta, feel like impostors. Which means at any given all-hands meeting, more than half the room has a private internal monologue running that sounds a lot like yours.

The Uncomfortable Truth About Getting Better

Here’s something that sounds like bad news but is actually good news: imposter syndrome tends to get worse as you get more experienced, not better.

Sit with that for a second.

When you’re a beginner, you don’t yet know the scope of what you don’t know. You have confident gaps. As you grow, you start to see the full map and realize just how much territory exists beyond where you’ve been. The feeling of not knowing everything intensifies not because you’re falling behind, but because you finally understand how much there is to know.

This is why senior developers, staff engineers, and people fifteen years into their careers still have imposter moments. It’s not a bug. It’s almost a feature. The self-awareness that comes with experience is the same self-awareness that occasionally makes you feel like a fraud.

So if the feeling has been getting louder as your career has grown, that’s not a warning sign. That might just be competence arriving with its very annoying sidekick.

Things That Actually Help (And One That Doesn’t)

The thing that doesn’t help: trying to logic your way out of it. Imposter syndrome doesn’t respond to evidence. You can recite your own resume at it and it will shrug. It operates on a different frequency than facts.

What does help:

Keep a record. A running document, a folder, a few notes somewhere private. Features shipped. Bugs squashed. Feedback that was kind. When your brain tells you that you’ve never done anything right, you need something concrete to push back with. The brain is dramatic. The document is not.

Ask questions out loud. In channels where people can see. In meetings where the answer might help someone else too. The developers who never ask anything aren’t the most competent people in the room. They’re just the loneliest. Every time you ask and the response is normal and helpful, you chip away at the idea that not knowing is shameful.

Let other people be real with you. The polished career narrative is everywhere. The actual one, with the failed projects and the code someone wrote at 2am that they’d rather forget, that one only comes out in honest conversations. Have those. The colleagues you most admire have a drawer full of mistakes. You just haven’t seen it yet.

Look at the room, not just yourself. Some imposter syndrome is internal. Some of it is a completely rational response to an environment that isn’t actually safe to fail in. Those are different problems. If you’ve done the internal work and the feeling is still relentless, it might be worth asking whether the issue is you, or the culture you’re swimming in.

The AI Corner of This Conversation

There’s a specific flavor of imposter syndrome that’s become very common recently, and it’s worth naming.

It sounds like: If I needed an AI to help me write that function, do I actually know what I’m doing?

Short answer: yes.

Knowing what to ask, reading the output critically, catching the code that runs but creates a problem nobody will understand in six months, knowing when to trust it and when to throw it out entirely, that is the skill. The tool doesn’t think for you. It generates. You decide.

Every generation of developers has had a version of this anxiety. Frameworks, cloud infrastructure, package managers, each one came with a voice that said the new way was somehow cheating. That voice has always been wrong. Use the tools. Know what they’re doing. Move on.

The Part That’s Actually Worth Keeping

Here’s the thing about imposter syndrome that never quite makes it into the productivity articles.

It mostly visits people who care.

The developer who lies awake thinking about whether their solution was the right one, who sits in a meeting quietly terrified of asking a dumb question, who reads their own PR comments with a faint sense of dread, that person gives a damn. That’s not nothing. That’s actually most of what separates good work from forgettable work.

The people who never feel any of this aren’t necessarily more competent. They’re often just less aware. Confidence without self-doubt is not always a flex. Sometimes it’s just a blindspot with good posture.

So the next time you’re in that meeting, sitting on a thought, running the mental simulation of everything that could go wrong if you say it out loud: say it anyway.

You’ve been right before. You just filed it under lucky.

How has imposter syndrome shown up in your career? Drop it in the comments. The more honestly we talk about this stuff, the less power it has over all of us.

Did you learn something good today as a developer?
Then show some love.
© Muhammad Usman
WordPress Developer | Website Strategist | SEO Specialist
Don’t forget to subscribe to Developer’s Journey to show your support.
Developer's Journey

Moving Beyond the Black Box: How I Built a Real-Time Voice Fitness Coach using Next.js 15, Convex, & Vapi.ai

yazar

Moving Beyond the Black Box: How I Built a Real-Time Voice Fitness Coach using Next.js 15, Convex, & Vapi.ai

I’ve used a lot of fitness apps. I’ve followed their calorie numbers, done their workout splits, and trusted their recommendations – all without ever understanding where any of it came from. The number just appeared, authoritative and unexplained, and I was supposed to trust it.

Eventually, I stopped. Not because the numbers were wrong. Because I had no reason to believe they were right.

That frustration became FitExplain – a full-stack web application that generates personalized fitness and nutrition plans through a voice conversation, and then shows you exactly how every number was calculated. No black box. No mystery algorithm. Just metabolic science, made visible.

This post is a complete technical breakdown of how I built it.

🧠 The Core Problem I Was Solving

Most fitness apps share a quiet design flaw: they treat the user like they can’t handle the truth.

You open the app, enter your weight and height, tap through a few goal screens, and get back a daily calorie target and a workout plan. The numbers look precise. They come with decimal points. But if you ask why 1,940 calories – why not 1,800, why not 2,100 – the app has no answer. It never does.

This isn’t accidental. It’s a design assumption: that authority is enough, that users will follow a number simply because the app said so.

Behavioral research consistently shows this assumption is wrong. People follow health guidance longer and more faithfully when they understand the reasoning behind it. Unexplained AI output, however accurate, produces shallow trust – and shallow trust doesn’t survive the friction of real life.

FitExplain was built around the opposite assumption: show your work.

🏗️ Architecture Overview: The Triple-Cloud Handshake

Before diving into each piece, here’s how the system fits together. I call it the Triple-Cloud Handshake: 

User's Browser
    │
    ├──── speaks to ──────► Vapi.ai (Voice AI)
    │                           │
    │                           └── webhook ──► Convex Backend
    │                                               │
    ├──── subscribes to ─────────────────────────► Convex (Reactive DB)
    │                                               │
    └──── authenticated by ──► Clerk ──webhook──►  Convex

Four independent cloud services, each best-in-class for its specific role:

Layer Technology Role
Frontend Next.js 15 + Vercel UI, routing, dashboard
Voice AI Vapi.ai Conversational data collection
Backend + DB Convex Persistence, AI orchestration, real-time push
Auth Clerk User identity, session tokens

The browser talks to Vapi for voice and to Convex for data. Clerk and Vapi each communicate with Convex via authenticated server-side webhooks — invisible to the user, but the backbone of the system.

🎙️ Layer 1: Vapi.ai — Voice as the Intake Form

The first question I had to answer was: why voice at all?

Most fitness apps use forms. Forms are fine — until you realize what they actually produce. A dropdown asking “How active are you?” with options like “Sedentary,” “Lightly Active,” and “Moderately Active” doesn’t capture useful data. Users don’t know what “moderately active” means in caloric terms, so they pick whatever sounds right and end up with a generic output.

Voice is different. A conversation can ask follow-up questions. It can confirm ambiguous answers. It can be designed to elicit specific, structured information — in a format that feels completely natural to the user.

The Vapi Workflow: 10 Nodes, 8 Parameters

I built a ten-node conversation workflow in Vapi that collects exactly eight parameters before allowing the session to close:

  1. Age – integer
  2. Current weight – numeric, with unit confirmation
  3. Height — numeric, with unit confirmation
  4. Existing injuries – free text, normalized
  5. Primary fitness goal – categorized (fat loss / muscle gain / endurance / general fitness)
  6. Preferred weekly training sessions – integer (1–7)
  7. Current fitness level – categorized (beginner / intermediate / advanced)
  8. Dietary restrictions or allergies – free text, normalized

The workflow uses guard conditions – Vapi will not advance to plan generation unless every required parameter has been captured and confirmed. If a user says something ambiguous like “I work out sometimes,” the workflow probes: “About how many days a week would you say?” The session runs between 30 and 77 seconds across all tested users.

The Race Condition I Spent an Hour Debugging

Here’s something that cost me a full debugging session: Vapi’s call.ended webhook fires before the call analysis is complete. I had built my Convex action to extract parameters from the analysis object – which wasn’t populated yet when the webhook arrived.

The fix was to listen for end-of-call-report instead of call.ended. The report event fires after analysis completes and includes the full structured output. A one-line change, but I only found it after reading the Vapi docs three times and adding console logs to every webhook handler. 

// convex/http.ts - webhook handler
export const vapiWebhook = httpAction(async (ctx, request) => {
  const payload = await request.json();

  // Use end-of-call-report, NOT call.ended
  if (payload.message?.type === "end-of-call-report") {
    const analysis = payload.message.analysis;
    const structuredData = analysis?.structuredData;

    if (structuredData) {
      await ctx.runAction(internal.fitplan.generatePlan, {
        callId: payload.message.call.id,
        userData: structuredData,
      });
    }
  }

  return new Response("OK", { status: 200 });
});

⚡ Layer 2: Convex — The Reactive Backend

This was the most interesting architectural decision in the project, and the one I’d recommend most strongly to other developers.

Why Not Just Use a REST API?

The standard approach would be: Vapi webhook hits an API route → API route calls Gemini → API route writes to a database → user refreshes the page to see results.

That works. But it means the user has to do something to see their plan. They have to know to refresh. The experience has a gap in it.

Convex eliminates the gap entirely.

How Convex Reactivity Works

In Convex, queries are subscriptions. When the frontend runs a query through the Convex React SDK, it opens a persistent WebSocket connection. Convex tracks which database documents that query depends on. The moment any of those documents change — from any mutation, anywhere in the system – Convex re-executes the query and pushes the updated result to all subscribed clients.

The developer writes a query function. Convex handles subscription, invalidation, and delivery automatically. 

// convex/fitplans.ts
export const getUserPlan = query({
  args: { userId: v.string() },
  handler: async (ctx, { userId }) => {
    return await ctx.db
      .query("fitPlans")
      .withIndex("by_user", (q) => q.eq("userId", userId))
      .order("desc")
      .first();
  },
});

// app/dashboard/page.tsx — React component
const plan = useQuery(api.fitplans.getUserPlan, { userId: user.id });

// 'plan' updates automatically when Convex pushes a new value.
// No polling. No useEffect with fetch. No page reload.

For FitExplain, this meant: the moment the Vapi webhook triggers a plan creation mutation in Convex, the user’s dashboard re-renders with the new plan. Latency between database write and screen update is WebSocket round-trip time plus Convex re-execution time — under 200ms combined.

The Schema: Keeping Gemini Honest

One subtle but important decision: I defined a strict Convex schema for fitness plans, then validated all Gemini output against it before writing to the database. This caught a bug that took me a while to find — Gemini was returning "2200" (string) instead of 2200 (integer) for calorie values. The Convex validator rejected it, which surfaced the issue immediately rather than letting corrupt data reach the frontend. 

// convex/schema.ts
fitPlans: defineTable({
  userId: v.string(),
  callId: v.string(),
  createdAt: v.number(),
  userProfile: v.object({
    age: v.number(),
    weightKg: v.number(),
    heightCm: v.number(),
    goal: v.string(),
    fitnessLevel: v.string(),
    weeklySessionTarget: v.number(),
    injuries: v.string(),
    dietaryRestrictions: v.string(),
  }),
  metabolics: v.object({
    bmr: v.number(),         // Mifflin-St Jeor result
    tdee: v.number(),        // BMR × activity multiplier
    targetCalories: v.number(), // TDEE adjusted for goal
    proteinGrams: v.number(),
    carbGrams: v.number(),
    fatGrams: v.number(),
  }),
  workoutPlan: v.array(v.object({
    day: v.string(),
    focus: v.string(),
    exercises: v.array(v.object({
      name: v.string(),
      sets: v.number(),
      reps: v.string(),
      rest: v.string(),
    })),
  })),
  mealPlan: v.array(v.object({
    meal: v.string(),
    foods: v.array(v.string()),
    calories: v.number(),
  })),
}).index("by_user", ["userId"]),

🤖 Layer 3: Google Gemini 1.5 Flash – Structured AI Output

Most LLM integrations treat the model as a conversational partner. FitExplain treats Gemini as a structured data generator.

The distinction matters. A conversational response (“Here’s your plan! On Monday, you should do…”) cannot be stored cleanly in a database or rendered in a structured dashboard UI. Schema-compliant JSON can.

The Prompt Engineering

The system prompt does three things:

  1. Instructs Gemini to calculate BMR using Mifflin-St Jeor explicitly
  2. Instructs Gemini to derive TDEE using the standard activity multiplier table
  3. Requires output as valid JSON matching a predefined schema – no prose, no markdown, no preamble 
// convex/fitplan.ts - Gemini prompt construction
const systemPrompt = `You are a fitness and nutrition AI that generates structured plans.
You MUST respond with ONLY valid JSON. No markdown. No explanation. No preamble.

Calculate BMR using Mifflin-St Jeor:
- Men: (10 × weight_kg) + (6.25 × height_cm) - (5 × age) + 5
- Women: (10 × weight_kg) + (6.25 × height_cm) - (5 × age) - 161

Apply TDEE multiplier:
- Sedentary (1-2 days/week): BMR × 1.2
- Lightly active (3 days/week): BMR × 1.375
- Moderately active (4-5 days/week): BMR × 1.55
- Very active (6-7 days/week): BMR × 1.725

Adjust target calories for goal:
- Fat loss: TDEE - 500
- Muscle gain: TDEE + 300
- Endurance/General: TDEE

Output JSON matching this exact schema: ${JSON.stringify(planSchema)}`;

The apiVersion Bug

The Gemini Node.js SDK was returning HTTP 404 errors on every call. The fix was non-obvious: I needed to pass apiVersion: 'v1' explicitly when initializing the client. The default was pointing to a preview endpoint that wasn’t available in my region. 

import { GoogleGenerativeAI } from "@google/generative-ai";

const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY!, {
  apiVersion: 'v1', // Required — default causes 404 in some regions
});

One line. Forty-five minutes to find it.

🔐 Layer 4: Clerk – Authentication Done Right

I chose Clerk over NextAuth for one reason: its webhook system integrates cleanly with Convex. When a user signs up, Clerk fires a user.created webhook to Convex, which creates a user document in the database. Session tokens from Clerk are verified in every Convex query and mutation, so user data is always scoped correctly. 

// convex/http.ts — Clerk webhook
export const clerkWebhook = httpAction(async (ctx, request) => {
  const event = await validateClerkWebhook(request);

  if (event.type === "user.created") {
    await ctx.runMutation(internal.users.createUser, {
      clerkId: event.data.id,
      email: event.data.email_addresses[0].email_address,
      name: `${event.data.first_name} ${event.data.last_name}`,
    });
  }

  return new Response("OK", { status: 200 });
});

📊 The XAI Part: Making the Math Visible

The Explainable AI component of FitExplain isn’t a separate feature – it’s built into the dashboard display. Every number shown to the user has a derivation trace: 

Your Daily Calorie Target: 1,940 kcal

How we calculated this:
├── BMR (Mifflin-St Jeor): 1,847 kcal
│   └── (10 × 75kg) + (6.25 × 178cm) - (5 × 24) + 5
├── TDEE (activity multiplier): 2,440 kcal
│   └── BMR × 1.325 (4 sessions/week = lightly-to-moderately active)
└── Fat loss adjustment: -500 kcal
    └── Standard 500kcal deficit targets ~0.5kg/week loss

This is the core design principle: explainability as engineering input, not post-hoc annotation. I didn’t generate a plan and then try to explain it afterward. I started from the explanation — the formula, the multiplier table, the adjustment logic — and used Gemini to instantiate that explanation for each individual user’s parameters.

The AI doesn’t replace the science. It applies it.

🧪 Testing Results

I ran 13 live voice sessions across users with different profiles:

Metric Result
Voice-to-dashboard latency 5–11 seconds
Convex cached query performance < 5ms
Sessions completed without error 13/13
Schema validation failures 2 (caught before DB write)
Contextually differentiated plans 13/13

The two schema validation failures were both the string/integer type mismatch on calorie values — Gemini occasionally returns numeric values as strings. I added a normalization step in the Convex action that coerces all numeric fields before validation, which resolved both failures.

🚀 The Full Tech Stack

Frontend:     Next.js 15 (App Router), TypeScript, Tailwind CSS
Deployment:   Vercel (automatic from GitHub)
Voice AI:     Vapi.ai (10-node workflow, guard-conditioned)
Database:     Convex (reactive, serverless, TypeScript-native)
AI Model:     Google Gemini 1.5 Flash (structured JSON output)
Auth:         Clerk (webhooks + session verification)

💡 What I’d Do Differently

1. Schema validation from day one. I added the Convex schema validator after encountering the string/integer bug. I should have defined it before writing any Gemini integration code. Strict schema-first development would have caught that issue in local testing rather than in a live session.

2. Vapi workflow versioning. The Vapi workflow is configured in their dashboard UI, which makes version control awkward. I’d build a script to export and commit the workflow JSON alongside the codebase from the start.

3. Gemini output temperature. I used the default temperature (1.0) throughout development. In retrospect, setting temperature to 0.3–0.5 for structured JSON generation would have produced more consistent outputs and fewer edge cases in the schema normalization layer.

🔭 What’s Next

The current system generates a plan once per voice session. The obvious next step is longitudinal tracking — storing multiple plans over time, detecting progress, and adjusting recommendations based on what’s changed. Convex’s reactive queries make this architecturally straightforward; it’s primarily a UI and prompt engineering problem.

Other directions I want to explore:

  • Wearable integration – pulling actual activity data from Apple Health or Garmin rather than relying on self-reported activity levels
  • Plan revision via voice – letting users update specific parameters (new injury, changed schedule) without redoing the full intake conversation
  • Nutrition logging – tracking actual meals against the generated plan, with Convex reactively updating macro targets based on what’s been logged

Wrapping Up

The thing I kept thinking about while building FitExplain is that the transparency problem in fitness apps isn’t really a hard technical problem. It’s a design choice. Someone decided that showing users the formula would confuse them, or slow them down, or make the app feel less magical.

I think that’s wrong. Showing the formula is what builds real trust — the kind that survives a bad week and a missed workout, instead of evaporating the moment the user questions the number.

With cloud-native tools available today — Vapi for voice, Convex for reactive data, Gemini for structured AI generation — you can build a system that is both intelligent and transparent without trading one for the other.

That’s what FitExplain does. And it’s what I think a lot more fitness software should do.

The full project is documented in my final year report at Manipal University Jaipur. If you have questions about any specific part of the implementation — the Vapi workflow design, the Convex schema, the Gemini prompt structure — drop them in the comments.

Tags: #ai #gemini #frontend #database

How to Recover Kafka DLQ Messages After a Schema Change Broke Your Consumer

yazar

The Problem Nobody Has a Tool For

Here is the scenario that wakes engineers up at 2am.

Your payment service has been running fine for months. The consumer reads events from a Kafka topic, processes them, sends confirmations. The status field is a plain String – "pending", "failed", "done".

A bug is discovered: the status field accepts invalid values like "ok" and "finished" from different producer teams, causing downstream analytics to break. The correct fix is converting status from String to a strict Enum with only four valid values: PENDING, PROCESSING, COMPLETED, FAILED.

The fix gets deployed at 9:45 PM.

Nobody checked the DLQ first.

There were 23,000 messages sitting in payments.dlq from an earlier consumer failure that afternoon. The new V2 consumer starts processing them. Each one crashes immediately: 

InvalidFormatException: Cannot deserialize value of type `PaymentStatus` 
from String "pending": not one of the values accepted for Enum class: 
[PENDING, PROCESSING, COMPLETED, FAILED]

All 23,000 messages are now permanently stuck. The V2 consumer cannot read them. They cannot be redriven as-is. PagerDuty fires at 2:00 AM.

Why Schema Registry Doesn’t Help Here

The first suggestion you’ll get is “just use Schema Registry.”

Schema Registry is a seatbelt. It prevents new schema-incompatible events from being produced. It does nothing for messages already sitting in your DLQ from before the schema change.

Those messages are stored as bytes in Kafka. Schema Registry has no knowledge of them. It cannot transform them. Cannot validate them. Cannot redrive them.

The 23,000 stuck messages are entirely your problem.

What Engineers Actually Do (The Manual Reality)

I posted this exact scenario on r/apachekafka and asked how senior teams handle it. Four experienced engineers replied. Not one named a tool.

Their answers:

BroBroMate (Senior Engineer): “Something I’ve done in the past is just throw a quick Kafka Streams app up to do mass transformations… And easy to unit test before rolling out.”

KTCrisis: “You could also spin up a v2 topic for the new consumers and keep a specific v1 consumer around just to drain the DLQ. But it adds a new topic and a dedicated consumer for a one-shot issue.”

Every solution described is either:

  • Building a temporary application from scratch, using it once, deleting it
  • Keeping zombie infrastructure alive until the queue drains
  • Writing a throwaway Python script with no tests, no audit trail, no validation

The time cost: 3-8 hours of a senior engineer’s time. At 2am. Every single incident.

The Gap No Existing Tool Fills

I looked at every existing solution:

DLQMan (irori-ab) – routes messages based on exception type headers. No transformation engine. If your messages have the wrong data format, DLQMan cannot help.

Confluent Control Center – has basic redrive. No schema transformation. Requires full Confluent Enterprise stack.

kafka-rewind-tools – handles the redrive part. You still need to write the mutation yourself.

Custom Kafka Streams apps – what everyone builds from scratch, uses once, and throws away.

The gap: no tool handles the transformation step between “message has wrong format” and “message is safe to redrive”.

Building DLQ Revive

I spent 5 weeks building the tool that should exist.

DLQ Revive is an open-source Kafka Dead Letter Queue mutation and redrive engine. Here is what it does and the architectural decisions behind each feature.

1. Browse DLQ Messages Safely

// WRONG - what most people do
consumer.subscribe(List.of("payments.dlq"));  // joins consumer group!

// CORRECT - what DLQ Revive does
consumer.assign(List.of(new TopicPartition("payments.dlq", 0)));
consumer.seek(new TopicPartition("payments.dlq", 0), fromOffset);

Using subscribe() joins your application to Kafka’s consumer group. Kafka can then trigger a rebalance and assign partitions to your read-only viewer – stealing them from your production consumer. Your 2am debugging tool accidentally takes down your production pipeline.

assign() + seek() bypasses group membership entirely. You read exactly what you want. The production consumer is untouched. DLQ Revive never calls commitSync() in view mode.

Reads are paginated at max 100 messages per API call. No full topic loads. No JVM OutOfMemoryError on topics with 500,000 stuck messages.

2. Transform Schema With JSONata

The transformation step is what makes DLQ recovery possible without writing custom code.

DLQ Revive uses JSONata – a declarative JSON-to-JSON mapping language. To fix the String to Enum problem from our scenario: 

{
  "orderId": orderId,
  "amount": amount,
  "currency": currency,
  "status": $uppercase(status),
  "processedAt": $now()
}

One expression. Applies to all 23,000 messages. Preview 5 samples before committing. See the before and after side by side.

Why not Groovy? User-submitted Groovy on your backend can call Runtime.getRuntime().exec("rm -rf /"). It is an RCE vulnerability built into the product if you allow it. JSONata is purely declarative – it has no access to the file system, network, or system commands. Zero RCE surface.

3. Validate Before Redriving

BroBroMate’s advice from that Reddit thread was exactly right:

“Whatever way you do it, validate every ‘fixed’ record against an agreed good schema before producing it – there’s nothing worse than dumping another X thousand bad messages on the DLQ.”

DLQ Revive validates each transformed message against the target schema before producing. Failed validation shows per-message errors. You see exactly which messages will fail before redriving a single one.

4. Idempotency Guard – Pod-Restart Safe

-- Before producing any message:
SELECT 1 FROM redrive_log 
WHERE topic = ? AND partition = ? AND offset = ?

-- If not found, insert THEN produce:
INSERT INTO redrive_log (topic, partition, offset, redriven_at, redriven_by)
VALUES (?, ?, ?, NOW(), ?)

The redrive_log table has a UNIQUE(topic, partition, offset) constraint.

If your application crashes halfway through redriving 10,000 messages and restarts, it will not reprocess the messages it already produced. In a fintech payment pipeline, that is the difference between a successful recovery and double-charging 5,000 customers.

5. Full Audit Trail

Every browse and redrive action is logged to PostgreSQL with timestamp, user, session ID, message count, and source/target topics. For teams in regulated industries (PCI-DSS, SOC 2), this is the compliance evidence that a manual throwaway script can never provide.

Quick Start

git clone https://github.com/Saifulhuq01/dlq-revive.git
cd dlq-revive
docker compose -f docker/docker-compose.yml up -d
# Open http://localhost:4200

That starts Kafka, Zookeeper, PostgreSQL, the Spring Boot backend, and the Angular dashboard together.

Stack: Java 17, Spring Boot 3.x, Apache Kafka 3.x, Angular 13+, PostgreSQL 15, JSONata, Docker Compose, GitHub Actions CI.

The Kafka Consumer Safety Details

A few things worth understanding if you are building Kafka tooling:

Why assign() instead of subscribe():
subscribe() participates in consumer group coordination. Kafka’s group coordinator can reassign partitions at any time during a rebalance. For a read-only tool this is dangerous – you don’t want Kafka moving partitions between your tool and the production consumer mid-operation.

assign() with seek() gives you direct partition access. No group membership. No rebalance risk. The offset you seek to is exactly where you start reading. You consume exactly limit records and stop.

Why pagination at the Kafka offset level:
Standard REST pagination (page 1, page 2) doesn’t work well for Kafka topics because the “next page” needs to know the exact offset where the previous page ended. DLQ Revive uses cursor-based pagination: fromOffset=0&limit=50 returns messages at offsets 0-49. The next call uses fromOffset=50. This is memory-safe regardless of topic size.

Why no commitSync() in view mode:
Committing offsets in view mode would change the consumer group’s position – affecting what the production consumer reads next if they share a group ID. DLQ Revive uses a unique, isolated consumer group and never commits in view mode.

What’s Next

The open-source core handles:

  • Paginated DLQ browsing
  • JSONata schema transformation
  • Preview before redriving
  • Idempotency-safe redrive
  • Full PostgreSQL audit trail
  • One-command Docker setup

Free tier limit: 100 messages per redrive session. This is enough for testing and small incidents. Bulk redrive (>100 messages) and team features are in the cloud tier.

GitHub: github.com/Saifulhuq01/dlq-revive

MIT licensed. If you’ve dealt with DLQ schema recovery at work and want to try it against a real scenario, I’d genuinely value your feedback. Especially on the JSONata approach and whether the safety guarantees hold up under your specific Kafka setup.

Mohammed Saifulhuq – Apache Fineract contributor, building DLQ Revive

From Spec-Driven Development to Attractor-Guided Engineering

yazar
abstractor.png

nop-chaos-flux is a low-code runtime framework built on React 19, Zustand 5, and Vite 8. It is inspired by Baidu AMIS but independently implemented from scratch based on innovative design principles. It includes a JSON Schema compiler and runtime, as well as design tools such as a report designer and Flow Designer.

The initial version of this project was completed by a single programmer in two months, developed entirely through AI automation. Unlike typical AI-assisted development, as features were continuously added, the project did not suffer from quality degradation. Instead, module boundaries became clearer, document categorization more stable, testing and auditing increasingly captured real issues, and code quality continuously improved.

I recently gave some internal sharing sessions and named this practice Attractor-Guided Engineering (AGE). Later, during code reviews, I found that many people did not truly understand it. Some operated in a way close to casual vibe coding, while others adopted a spec-driven development style similar to OpenSpec, leaning overall toward task-driven, feature-by-feature development. As the workload accumulated, inconsistencies in the code steadily built up, and deviations from the architectural design became the norm.

This article analyzes the differences between AGE and spec-driven development, and what ordinary business projects can learn from the large-scale AI engineering practices of nop-chaos-flux.

Core Hierarchy

At the heart of AGE lies the following conceptual hierarchy:

State Space → Attractor → Trajectory → Control

  • State Space: All possible implementation states the system can evolve into under current constraints.
  • Attractor: The stable structure to which the system is repeatedly pulled back during long-term evolution.
  • Trajectory: The actual evolutionary path preserved after each round of generation, verification, and correction.
  • Control: Various mechanisms that continuously influence the trajectory through local signals.

All possible combinations of code, documentation, and tests that the repository could evolve into under existing constraints constitute the state space. The continuous actions of humans, AI, review, CI, and document updates form the evolutionary rules. The live repo history created by the superposition of the two is the trajectory. The attractor is the stable structure to which the system is repeatedly pulled back over long-term iteration.

abstractor.png

Large-scale AI development is essentially a controlled convergence problem for a dynamical system. AI expands the state space extremely fast. The key is not to add guardrails everywhere, but to first figure out: what structure should the system be pulled back to over the long term?

An attractor is not a fully written-out end point, not a roadmap, and not a “scope of permitted activity.” It is a structure implicitly defined by a small set of high-order constraints: local implementations can vary, but the whole will be pulled back to the same type of form by these relationships.

In nop-chaos-flux, what first defines the attractor is not the plan, lint, test, audit, or any single spec file, but the architecture docs with precedence under docs/architecture/. In the following text, these architecture docs with owner and precedence are referred to as owner docs.

docs/architecture/README.md explicitly states: docs/architecture/ is the current final-state architecture baseline, flux-design-principles.md is the governing-principles anchor, frontend-programming-model.md holds the top-level primitives and core-boundary precedence, and other normative docs hold local precedence within their respective topics.

These documents do not enumerate all correct implementations. They define a stable structure like an equation: what primitives exist, which dependency directions are legal, which owner boundaries must not be broken, and which old patterns no longer belong to the correct state space. They first define the cluster of states that can sustainably exist, so that subsequent implementation, testing, planning, and auditing have a reference.

Plan, verification, audit, logs, bugs, testing all come later. They are not the attractor itself, but the harness that keeps the system trajectory continuously close to the attractor.

Flux/AGE is not centered on spec evolution, but on attractor definition and trajectory convergence.

Real Development History

“Architecture docs define the attractor” might sound like ordinary document governance at first, but the difference becomes clear if you carefully examine the actual development records in docs/logs/.

For example, docs/plans/371-deep-audit-2026-05-19-owner-routing-plan.md is a typical entry point. It does not directly change the code, but converges the 64 retained findings from the 2026-05-19 deep audit into an owner-routing baseline. Each finding has a unique owner bucket, priority, successor plan, and owner-doc obligation. No ownerless items, no multiply-owned items, no silent downgrade to vague follow-ups.

Its role is to first freeze how a round of deviations should be resolved, after which the subsequent execution plans close the local trajectory to concrete outcomes. docs/plans/382-deep-audit-2026-05-19-table-and-crud-owner-state-and-event-contract-plan.md resolves the table/CRUD owner state and event payload: Current Baseline declares the live status of explicit empty-array owner state and event payload, execution items fix the empty-array fallback, focused tests prove the result, docs/components/table/design.md and docs/components/crud/design.md synchronize the owner contract, and finally repo-wide pnpm typecheck, pnpm build, pnpm lint, pnpm test all pass, with the independent closure audit ses_1bd9ed593ffeVpkho4lb4wPR6p recording Verdict: acceptable.

docs/plans/388-deep-audit-2026-05-19-form-tree-widget-accessibility-plan.md demonstrates another type of closure: visible node roving focus, ArrowUp/ArrowDown/Home/End navigation, loading aria-busy/aria-describedby for input-tree and tree-select all have focused proof, but the plan explicitly rules No owner-doc update required. This shows that a plan is not just a to-do list; it is also responsible for judging whether this change modifies the owner-doc baseline.

docs/plans/400-deep-audit-2026-05-19-test-harness-reliability-plan.md and docs/bugs/62-e2e-shared-websocket-error-suppression-fix.md demonstrate the role of memory harness. The problem was not a business function failure, but that the E2E shared fixture’s filter scope was too broad, filtering out real transport/runtime failures as well. The fix not only changes the code, but also adds tests/e2e/fixtures-hard-gate.spec.ts to prove it now fails, and records a rule in the bug note: if future tests genuinely expect a WebSocket failure, a per-test allowance must be used, and fixture-wide suppression must not be restored.

docs/logs/2026/05-19.md and docs/logs/2026/05-20.md are not mere daily logs. They record the focused proof of each closure slice, owner-doc rulings, repo-wide gates, independent audits, and full-green baselines.

These materials form a real chain: owner docs define the long-term structure, audits discover deviations, plans route and freeze ownership, focused proof demonstrates local results, owner-doc rulings decide whether to update, logs/bugs/testing preserve cross-session memory, and closure audits re-judge completion status from the live repo.

This is the harness in AGE. The goal is not to make each task appear more complete, but to make the system trajectory continuously return to the vicinity of the attractor.

Where OpenSpec’s Structure Excels

OpenSpec is a representative spec-driven development framework that structures a category of work well:

  • openspec/specs/ stores the behavioral specifications of current capabilities.
  • openspec/changes/ stores proposed changes, including proposal.md (why change, what changes, impact), design.md (technical design when necessary), and tasks.md (implementation checklist).
  • changes/<name>/specs/ records spec deltas using ADDED/MODIFIED/REMOVED/RENAMED.
  • Archive applies the deltas back to the main specs through fixed rules.

This mechanism lowers the cost of updating specifications. OpenSpec does not require the AI to freely understand the entire requirement and then arbitrarily modify documents each time. Instead, it uses fixed sections, requirement header matching, and delta application to merge structured changes back into the main specs. This approach is well-suited for scenarios where behavioral specifications are clear and need to be parseable, archivable, and re-writable.

Where OpenSpec Is Limited

OpenSpec’s spec format (Requirement, Scenario, SHALL/MUST, delta sections) is tool-parseable friendly, but is not flexible enough as a way to organize general project knowledge.

A lot of repository knowledge does not naturally grow into requirement/scenario form:

  • Architecture hierarchy and precedence.
  • History of terminology misunderstandings.
  • External research materials (PPT, discussion records).
  • Analysis conclusions from a source code audit.
  • Phenomena discovered through manual testing.
  • The elimination path of a complex bug.
  • Why a plan cannot be closed.

If everything must first be translated into a spec/change to gain an official place, the conversion cost is high, and the original context is also lost. External documents, discussion records, analysis reports, and bug retrospectives could directly become repository memory. Forcing them into requirement/scenario format tends to flatten the information.

In terms of execution verification, the two reflect different tool philosophies. OpenSpec’s tasks.md is a checklist; /opsx:verify is an optional agent verification skill (requires manually enabling expanded workflows). archive checks task status and prompts about risks, but can still proceed after user confirmation—strict closure is left to the team’s own conventions.

Flux plan, on the other hand, bakes independent closure audit in as a required step: you must start from the current baseline, write clear Goals, Non-Goals, execution items (each annotated with Fix/Decision/Proof), and Closure Gates; before marking completed, an independent sub-agent or independent reviewer must go back to the live repo and perform a closure audit. docs/plans/361-slot-contract-host-manifest-and-owner-doc-closure-plan.md puts together the current baseline, the finding ownership matrix, execution phases, closure gates, the deferred items that have been ruled on, and independent review evidence. This is not a checklist, but an execution contract that strictly defines closure conditions and requires independent audit. It emphasizes not what was done, but to what standard something counts as truly landed.

Why Spec-Driven Easily Becomes Task-Oriented

The default structure of spec-driven development easily pulls the attention of both humans and AI back to “how to complete this change.”

A change typically has a proposal, design, tasks, and delta specs. This is much better than verbal requirements, but it is naturally organized around one change: why change, what to change, how to do it, which checklist items are done. Over time, spec-driven development can easily become a more formal task dispatch system.

This task orientation has a hidden risk: specs are updated, tasks are checked, archive is done, and both AI and humans get a strong sense of completion. But whether the system is truly closer to the long-term structure is not proven by the change itself. It must be verified by looking back at the architecture docs, the live repo, tests, logs, and independent audits.

If one tries to long-term maintain specs to the level of precision in Flux, the cost is usually higher. What Flux maintains is not a single specification tree, but a set of repository memories with different responsibilities: architecture docs define the attractor, plans define local trajectory closure, logs record evolution, bugs preserve complex defect diagnosis, testing preserves manual findings, analysis preserves research judgments, and audit is responsible for looking back at the live repo. Cramming all of this into a spec/change structure incurs continuous conversion cost.

Spec-driven can serve as a local harness within AGE to manage behavioral spec evolution. But it should not replace architecture docs, nor should it replace the separate repository memories of plan closure, logs, bugs, testing, and audit.

What If You Just Need to Look Up an Owner Doc?

This reveals a fundamental difference between the two approaches.

If, in Flux, you simply want to check the current owner contract of the renderer runtime, you can directly route from docs/index.md to docs/architecture/renderer-runtime.md and the relevant references. You do not need to create a change, do not need a proposal, do not need tasks, and do not need to rewrite the question into a spec delta.

If it’s just research, it might go into docs/analysis/. If it’s just a discovery from manual testing, it goes into docs/testing/ first. If it’s a complex bug, it goes into docs/bugs/. If it’s an execution closure, then a plan is written. A stable baseline change then goes back to the owner doc.

Flux’s document organization is free but not arbitrary. Free means: it does not force all knowledge to first enter the same artifact workflow. Constraint means: each type of material has a clear responsibility and cannot arbitrarily compete for factual status.

OpenSpec’s organization is more structured. It is good at putting proposed behavioral changes into a change package and then writing spec deltas back to main specs. But general repository knowledge routing is not the main problem it aims to solve by default.

Tasks Are Not Plans

OpenSpec’s tasks.md does not correspond to Flux’s plans.

tasks.md at most corresponds to the execution checklist fragment inside a Flux plan, or the todo list of the current session. It can help AI avoid missing steps, but it does not answer these questions:

  • What is the current live repo baseline?
  • What is explicitly not being done this time?
  • What evidence can prove it is truly complete?
  • Which defects cannot be downgraded to follow-ups?
  • Who independently verifies the completion status?

The value of a Flux plan is not in “having more tasks,” but in it being a closure contract for a local trajectory.

So tasks are necessary for AI, but tasks are not the source of truth. - [x] only indicates that the executor claims an item is complete. The real proof must return to the current code, tests, owner docs, and independent review.

Sources of Change Go Beyond Spec Changes

OpenSpec’s path is very clear: organize behavioral spec evolution around specs and changes.

Flux’s sources of change are broader:

  • New requirements raised by users.
  • Architectural drift discovered by reading source code.
  • Owner boundary problems discovered by audits.
  • Interaction problems exposed by testing issues.
  • Historical regression risks recorded in bug notes.
  • Design judgments derived from external research or analysis.
  • A plan closure audit overturning a previous judgment of “completed.”

These sources should not all be transformed into spec deltas first. They can first land in analysis, testing, bugs, logs, plans, or directly update architecture docs. Whether they ultimately need to become a behavioral contract depends on whether they are part of the long-term baseline.

This is the significance of Flux’s organizational freedom in documentation. Information from different sources first stops at different positions, rather than being shoved entirely into a single spec evolution channel.

Why Files-In, Files-Out Matters

I have emphasized many times in sharing sessions that a best practice for AI-driven development is files-in, files-out. This is not just documentation fastidiousness—the attractor and the harness need a physical carrier.

Files-in: do not leave input only in the chat window. Even if the content is messy, first write it to a requirements file, analysis file, or plan file, and then @ this file in OpenCode.

Files-out: do not end with output just printed in the window. Important conclusions, analyses, plans, test records, bug diagnoses, and architectural constraints are all written into docs/, categorized by responsibility:

  • Architecture rules → docs/architecture/
  • Change closures → docs/plans/
  • Execution trajectory → docs/logs/
  • Manual testing findings → docs/testing/
  • Complex defect diagnosis → docs/bugs/
  • Research judgments → docs/analysis/

The point is not “write more documents.” Once output is categorized and persisted, it enters the repository memory. The chat window is temporary context; files are the repository memory.

Code Is the Current Implementation Fact, Not the Only Fact

Code is the source of truth for the current implementation. Types and tests protect the current behavior. The architecture owner docs with precedence define the attractor—the structure the system should converge to over the long term. Logs, bugs, analysis, and testing record the trajectory and externalize memory.

These do not conflict. Conflicts arise in two situations:

  • When documents start regurgitating implementation details that easily rot.
  • When the completed state of a document is taken as proof that the code is already correct.

OpenSpec’s behavior-first boundary actually avoids the first problem to some extent: behavioral specs should write verifiable behavior, not internal implementation details. Flux further emphasizes the second problem: even if the document describes the correct behavior, it cannot replace the verification of the current repository. True closure must return to code, tests, owner docs, logs, and audit evidence.

The Real Difference

What OpenSpec solves: how to use structured specs/changes/delta/archive to organize the update of behavioral specifications, focusing on the evolution flow of specs.

What AGE solves: when AI rapidly expands the state space, how the system first defines an attractor, and then uses harnesses such as plan, verification, audit, logs, bugs, analysis, and testing to make the trajectory continuously converge.

The two tools solve problems at different levels. OpenSpec’s strength is the structured specification workflow. Flux/AGE’s strength is using architecture owner docs with precedence to define the attractor, and then using free but responsibility-bounded document routing and a strict closure harness to make the repository converge over the long term.

Moving from spec-driven development to Attractor-Guided Engineering is not moving from “writing specs” back to “not writing specs,” but moving from “organizing changes into spec updates” toward “defining where the system should converge over the long term, and making every round of AI generation continuously pulled back by this structure.”

What Ordinary Projects Can Directly Adopt

There is no need to copy the full governance of Flux.

First, files-in, files-out. Write input into a file first before @-ing it to AI; output goes into docs/, not left only in the chat window. This step takes just a few minutes, but it gives AI traceable repository memory instead of relying on conversational context.

Second, write an architecture document for the core business that defines the business attractor. Do not write a code guide. Instead, write down the sources of truth, state transitions, ownership boundaries, and common misunderstandings.

Third, write lightweight plans for complex changes. Include the current baseline, goals, what is not being done, an execution checklist, proof items, and closure conditions. Before completion, have an independent session or independent reviewer re-verify.

Fourth, turn recurring problems into repository memory. Manual issues go into testing, complex bugs go into bugs, research conclusions go into analysis, and high-frequency errors go into audit scripts.

Specification updates are a necessary layer. System convergence still needs an attractor.

nop-chaos-flux is open source: