Both Camps in the ‘Left Behind’ Argument Are Right About Each Other

There’s a small, angry post on a bear blog called migraine brain that’s been on Hacker News for a few days now. The whole post is three paragraphs. It opens: “‘People who don’t use AI will be left behind,’ they say. I can’t emphasize enough how much I hate it when I hear/read shit like that because I’m pretty sure, in fact, that what will happen is the exact opposite.”

The author’s argument is brisk: AI-reliant people are the ones who will be left behind. They’ll forget how to think, how to write, how to do a simple reliable search, how to tell fact from fiction. They’ll forget — in the original phrasing, with a stronger word for emphasis cleaned for republication — “how to freaking LEARN” [the original used a four-letter intensifier in that slot; the linked post has the unmodified line]. “What a beautiful thing it is just to learn stuff.”

The post is short, the language is cleansed of nuance on purpose, and it landed 255 comments deep on the same Hacker News front page where, on any given week of 2026, several other posts are running the inverse argument: that you adapt or die, that the productivity multiplier is real, that two years into agentic coding the people who refused to learn the workflow will be the ones explaining themselves at every job interview.

Both camps are right. They are right about each other. The whole conversation has been an argument about which kind of left behind is the worse failure mode, conducted by people who don’t agree on which failure mode is real.

The competence-erosion case

The blogger’s case is more interesting than its delivery. It is not, mechanically, an argument against AI tools. It is an argument that the practice of using them — particularly the agent-style, hand-it-the-task delegation pattern that 2026 software development has converged on — atrophies the underlying skill, and that in a non-trivial fraction of cases the underlying skill was the actual job.

The Hacker News thread runs heavy on testimony for this side. “It is easy to shut off your brain when using AI and then get overwhelmed by the amount of code it produces,” one commenter wrote. “I have seen a lot of really bad AI code. I can spot and repair it. Others can not.” That distinction — I can spot and repair it, others can not — is the whole game. The bad-AI-code-spotting skill is the one that erodes when you stop reading code, and reading code is the one thing the agent will do for you if you let it.

Another commenter framed it physiologically: “Just as many people leading sedentary lifestyles have to make a deliberate effort to exercise, because inactivity is really bad for our bodies, I think we’re going to realise that a similar process is necessary for our minds. Coding used to kind of give you this exercise for free, but you can go really far with just your System 1 nowadays — literally get things done while scrolling Reddit.” The get things done while scrolling Reddit part is the indictment. The agent has reduced engineering for some fraction of tasks to skim-and-approve, and the skim-and-approve loop trains a different set of muscles than the read-and-build loop did.

The sharpest version of the competence argument in the thread came from a reply to a productivity-pitch line — that “a person working with a bunch of agents is a lot more productive than just a person.” The reply: “A tool being smarter than me but inconsistent is useless. I can work with people who are smarter than me, because I can trust them, and I can trust them to own up or be held accountable for screw-ups. For a calculator, I can only hold myself accountable. However I cannot hold myself accountable for not knowing something I don’t know.” That’s the argument that the blogger is making, expressed as accountability rather than as anger. The competence loss is not just about the tasks the AI does for you; it’s about the meta-skill of would I have caught that if it were a human peer’s mistake. That meta-skill comes from doing the work yourself, and it depreciates if you don’t.

The competence-erosion case has a real failure mode it’s pointing at, and the failure mode does not require you to believe that AI is bad or fake or hyped. It only requires you to notice that the human cognitive system, like the human muscular system, defaults to atrophy without practice. In 2026 the supply of practice is rapidly contracting.

The productivity-displacement case

The opposing case is at least as well-defended in the thread, and at least as concrete.

It runs roughly: the agent-driven workflow is real, the productivity gains are real, and the people who refuse to learn the workflow will become unhireable in the same way that someone who refused to learn to use a compiler — preferring to stay in assembly because it kept the skill — would have made themselves unhireable in the 1990s. “I mean, that works for you since you’re retiring,” one commenter replied to a senior engineer’s farewell-to-the-industry post. “But for people still working in the industry, you adapt or die. As it’s always been.”

The productivity-displacement camp’s strongest argument is not the multiplier claim. It’s that the skill of delegation itself — knowing when to hand the agent a task, what context to give it, when to trust the output, when to pull the work back — is a real and emerging skill. “Understanding the optimal work flow for what to delegate and what to do yourself is difficult. Understanding the need for precision in the language used, and learning how to elegantly phrase things that were previously just abstract thoughts is absolutely a talent that can be refined.”

This is not a fake skill. The same loop that erodes the read-and-build muscle in the competence-erosion frame is, in this frame, building a different muscle — one that is closer to architectural review than to line-level coding. There is a non-trivial population of working engineers, two years into AI tooling, who report that their judgment about what is worth building has sharpened precisely because the cost of building is no longer the gating factor.

The most direct reply to the blogger’s “I love learning” framing in the thread also came from Simon Willison. Quoting the blog’s exact line back, he wrote: “I love learning. My life of self-education is so much richer with LLMs to help me. There are dozens of other arguments for not engaging with AI. If your reason is ‘I love learning’ I recommend at least dipping your toes in before you declare that AI is a hindrance, not a help, to people who love to learn new things.” This is the inverse of the competence-erosion argument from someone who has been running the experiment longer than almost anyone publicly. The blogger and Willison are not actually arguing about the same thing — the blogger is talking about learning as discipline, Willison is talking about learning as access — but Willison’s reply is the version of the productivity-pitch that doesn’t reduce to “adapt or die.” It’s “the door is wider; come in.”

The productivity-displacement case is also right that the dual failure mode — people who use AI badly — is real. “Some people who do use AI will also be left behind,” one commenter put it, “those who use it to replace their skills without developing new ones themselves, and those who use it to do the same or worse work more cheaply. They will be left behind in a competitive world where others will work out how to use it to do more or better work with no reduction in effort.” That’s not the blogger’s argument. But it’s not far from it.

The “is it a skill” dispute

The single most-concrete disagreement in the thread is the one that decides which of these cases lands harder. Some commenters argued that AI tooling is essentially a weekend learning curve. “Any engineer can ‘learn to use AI’ in a couple of days,” one wrote. “It’s not rocket science; there’s no chance of left behind. If you haven’t used LLMs at all, a weekend would be enough to be on par with everyone else in the industry.”

The most credentialed reply in the thread came from Simon Willison — handle simonw, an almost-daily LLM user for nearly three years, and one of the most prolific public chroniclers of working-developer LLM use: “Firmly disagree. Learning how to use these tools effectively is unintuitively difficult. They’re great at some stuff and terrible at other stuff in ways that are very hard to predict. I’m figuring out new and better ways to use them in a daily basis, and I’ve been an almost daily user for nearly three years.”

This isn’t really a contradiction. It’s two different answers to two different questions. “Can you sit down at ChatGPT and produce something that looks like work?” is a weekend curve. “Can you reliably tell which work the model is good at and which work the model is going to make worse?” is a multi-year curve, and according to the person who has spent three years on it, the bottom of the curve is not visible yet.

The skill-curve question is also where the two camps’ arguments collide most usefully. The competence-erosion frame is largely pointing at people on the early part of the curve — people for whom AI has reduced the practice of judgment to approve / approve / approve. The productivity-displacement frame is largely pointing at people on the later part of the curve — people whose judgment about when not to use the agent is itself the productive skill. Both populations exist. They’re not the same engineers.

The asymmetric failure modes

The strongest single move in the thread came from a commenter early in the discussion: “Some people who don’t use AI will be left behind. Some people who do use AI will also be left behind.” That isn’t a hedge. It’s the actual structural answer.

Both camps are arguing that you can lose your seat at the table. They disagree only about which seat-loss is worse, and they disagree because they’re standing in different places. The competence-erosion camp is mostly people in mid-to-late-career engineering roles where the value of the work is the judgment, and the judgment can be eroded faster than you can replace it. The productivity-displacement camp is mostly people in industry-facing or early-career engineering roles where the table is already moving, and refusing to move with it has a velocity cost that compounds.

The blogger and the productivity-pitch are both correct in their own population. They are both wrong about the other population. The actual question — which one each individual reader should be optimizing against — is answered by where you sit, not by which slogan won this week’s HN front page.

What this asks of us

The reason the conversation is exhausting in 2026 is that both “AI will leave you behind” and “AI will leave the people who use it behind” are operating as identity claims at this point. The slogan you repeat marks which side of the argument you’re on. The argument is not actually about AI. It is about which kind of agency-loss you are most afraid of.

Agency-loss to a tool that is smarter than you but inconsistent — the “cannot hold myself accountable for not knowing something I don’t know” problem — is a real fear, and don’t use the tool is a defensible response. Agency-loss to the labor market — the table moved without you — is also a real fear, and learn the tool, including its boundaries is a defensible response. Both are responses to real threats. The wrong move is pretending the other person’s threat is fictional.

The migraine-brain blogger’s daily practice of writing without an LLM, of doing a search the slow way, of holding “What a beautiful thing it is just to learn stuff” as a load-bearing value — that is one valid answer to which agency-loss they’re protecting against. Simon Willison’s daily practice of three years of careful, public, contradictable LLM experimentation is another, for a different agency-loss. Both practices are honest; neither is a slogan.

The slogan “people who don’t use AI will be left behind” is not workable; it is what you say when you do not want to do the work of figuring out which agency-loss is the one you are actually optimizing against. The blogger’s response that “the exact opposite” will happen is not workable either, for the same reason. The harder question — which agency-loss is the one you are working to prevent, and what daily practice does that imply — is the one that doesn’t fit on a LinkedIn post. Both slogans on the front page of the industry conversation are refusals to ask it.

Flutter MCP Toolkit v3

After a couple of months I finally released v3 of Flutter MCP Toolkit (which complimentary for official Dart / Flutter MCP server) + CLI tools to work.

This release again completely changes its architecture (MCP and CLI just thin wrappers, everything is configurable from packages and customizable for usage). New tools are client side – first, they are optional, can be modified, verified or improved – and finally ai agents (LLM in the loop:)) starting catching up that how to use them.

Since there a lot of changes, there can be challenges and bugs, but I hopefully will fix everything as fast as it would be possible.

Huge thank you for contributors who inspired to make more (like web support) or contributed to the project in previous version!

Will be glad for any feedback:)

The video is an experiment made with hyperframes and kokoro onnyx as tts 🙂

GitHub: https://github.com/Arenukvern/mcp_flutter
website with docs for AI Agents and Humans: https://docs.page/arenukvern/mcp_flutter

Google Just Shipped Gemini 3.5 Flash. Here’s What Developers Actually Need to Know.

The Flash series has always been Google’s answer to the speed-vs-intelligence tradeoff. With Gemini 3.5 Flash, Google is making a different argument: you shouldn’t have to choose.

The Problem It’s Solving

The history of “fast” AI models is a history of compromise. You got low latency, but you gave up reasoning depth. You got cheaper inference, but you got worse results on multi-step tasks. The whole Flash premise — intelligence at Flash-level speed and cost — has always been aspirational. With Gemini 3.5 Flash, the benchmarks suggest Google has actually closed a meaningful portion of that gap, particularly for the workload that matters most right now: agentic execution.

How Gemini 3.5 Flash Actually Works

Gemini 3.5 Flash is designed for sub-agent deployment, multi-step workflows, and long-horizon tasks at scale, with particular effectiveness in rapid agentic loops involving complex coding cycles and iterations. That’s the framing Google leads with, and the architecture reflects it.

The model supports a 1M token context window, 65k max output tokens, and thinking — the same set of tools and platform features as Gemini 3 Flash. The key architectural addition is thought preservation: the model now maintains intermediate reasoning across multi-turn conversations automatically. When present in the conversation history, reasoning context carries forward, which improves performance on complex multi-step tasks like iterative debugging and code refactoring. No API changes are needed.

The thinking system itself has also changed. The default thinking effort level is now medium, changed from high in Gemini 3 Flash Preview. medium yields very good results across a wide range of tasks while being faster and more cost-efficient. For complex problems, high encourages the model to think more deeply. Google’s explicit recommendation: start at medium, drop to low for speed-sensitive agentic loops, escalate to high only for hard reasoning or math. The old thinking_budget numeric parameter is gone — use the thinking_level string enum instead.

One important note for teams running computer-use workloads: Computer Use is not supported in Gemini 3.5 Flash at this moment. For Computer Use workloads, continue using Gemini 3 Flash Preview.

What Developers Are Actually Using It For

The benchmark most worth examining for this audience is MCP Atlas — a multi-step workflows benchmark using MCP. Gemini 3.5 Flash scores 83.6% on MCP Atlas, leading the comparison set that includes Gemini 3.1 Pro (78.2%), Claude Opus 4.7 (79.1%), and GPT-5.5 (75.3%). If you’re building anything involving MCP tool chains, that number is directly relevant.

On Finance Agent v2 (financial analysis and decision-making), Gemini 3.5 Flash scores 57.9%, ahead of Claude Sonnet 4.6 (51.0%), Claude Opus 4.7 (51.5%), and GPT-5.5 (51.8%).

The coding story is also compelling in a specific way. JetBrains reports that Gemini 3.5 Flash delivers coding and reasoning quality close to Gemini Pro while preserving the speed and cost profile that makes Flash ideal for real-time developer workflows, with low-reasoning coding performance improved by 10–20% compared to the previous Flash generation.

Enterprise validation comes from Box: Gemini 3.5 Flash beat Gemini 3 Flash by 19.6% on Box’s enterprise work evaluation set, which was designed to reflect the kinds of real-world multi-step tasks their customers perform daily. For Life Sciences customers, Gemini 3.5 Flash can extract data and make calculations with 96.4% greater accuracy, and for Financial Services firms, it can build financial reports from structured data with 46.7% greater accuracy.

Why This Is a Bigger Deal Than It Looks

The MCP Atlas score deserves more attention than it’s getting. For anyone building agentic systems using the Model Context Protocol — and the infrastructure around it is growing fast — having a model that leads on multi-step MCP workflows at Flash pricing changes the economics of what you can deploy. MCP-native tooling like Glama.ai and other agentic middleware layers become more viable when your inference costs stay low without sacrificing orchestration quality.

The thought preservation feature is the other architectural shift worth watching. Most developers managing multi-turn agentic sessions today are manually engineering state — reconstructing context, summarizing prior steps, managing memory externally. With Gemini 3.5 Flash, the model uses reasoning context from all previous turns when thought signatures are present in the conversation history; the SDKs handle this automatically. That’s less scaffolding code your team has to maintain.

There is one behavioral change that could silently degrade quality if you migrate without testing: the default thinking effort changed from high to medium. Teams should verify quality, speed, and cost after migration, and note that thought preservation is now on by default — reasoning context carries forward across turns, which improves performance but may increase token usage.

Availability and Access

Gemini 3.5 Flash is generally available (GA), stable, and ready for scaled production use. The model ID is gemini-3.5-flash, last updated May 2026.

The model is accessible via the Gemini App, Gemini API, Google AI Studio, Google Antigravity, Gemini Enterprise Agent Platform, and Android Studio. It supports function calling, structured output, search grounding, Google Maps grounding, URL context, file search, code execution, and thinking — all available in the same request via combined tool use.

On the paid tier, input pricing runs $1.50 per million tokens and output at $9.00 per million tokens (including thinking tokens). Context caching is $0.15 per million tokens, with storage at $1.00 per million tokens per hour. Batch inference halves those rates. A free tier is available for experimentation through Google AI Studio.

For teams migrating from Gemini 3 Flash Preview: update the model string from gemini-3-flash-preview to gemini-3.5-flash, replace thinking_budget with thinking_level, remove temperature/top_p/top_k from your config (no longer recommended), and add id and matching name to all FunctionResponse parts. The full migration checklist is worth reading before touching production.

The speed-vs-intelligence tradeoff that has defined the Flash tier since its inception is getting smaller with each generation. The MCP Atlas score, the thought preservation architecture, and the enterprise validation from Box all point at the same conclusion: Gemini 3.5 Flash is the most credible case yet that “fast and cheap” doesn’t have to mean “less capable” for agentic workloads specifically.

Follow for more coverage on MCP, agentic AI, and AI infrastructure.

🔐 Working with Private Symfony Recipes

Private Symfony Recipes (or: how I stopped repeating myself)

I got tired of setting up the same things in every Symfony project.

Same configs.
Same services.
Same little pieces of glue code that aren’t hard—but add up.

Symfony recipes solve a lot of this.

Until you need your own.

At some point I decided:

“Surely there’s a way to do this privately.”

So I sat down one evening and started digging.

Docs open.
Composer configs half-understood.
A couple of tabs that looked promising but didn’t quite connect.

And for a while… nothing clicked.

I remember getting to the point where I was about ready to reach out to someone a lot smarter than me and just ask:

“Alright, what am I missing here?”

And then it finally clicked.

🧠 The idea

Symfony recipes aren’t magic.

They’re just:

  • A repository
  • With a specific structure
  • That Flex knows how to read

So instead of relying only on the public recipes repo, you can point your project at your own.

A private recipes repository.

🪓 The basic setup (no fluff)

At a high level, you need:

1. A recipes repository

This is just a git repo with a structure like:

recipes/
  vendor/
    package/
      version/
        manifest.json
        config/
        src/

The important piece is the manifest.json.
That’s what tells Symfony Flex what to do.

2. A manifest

Example:

{
  "copy-from-recipe": {
    "config/": "%CONFIG_DIR%/"
  }
}

This is where you define:

  • files to copy
  • configs to apply
  • environment setup

3. Tell Composer where to find it

In your project’s composer.json:

{
  "extra": {
    "symfony": {
      "endpoint": [
        "https://your-private-recipes-repo",
        "flex://defaults"
      ]
    }
  }
}

👉 Order matters. Your repo goes first.

4. Install your package

Now when you require your package:

composer require your/package

Symfony Flex will:

  • check your private repo first
  • apply your recipe
  • then fall back to defaults if needed

💡 What I actually use this for

This is where it gets useful.

My private recipes handle things like:

  • Shared configuration across projects
  • Common service wiring
  • Internal conventions I don’t want to rethink every time

Instead of:

“Alright, how did I set this up last time?”

It becomes:

“composer require → done”

⚠️ The gotchas (aka the part that took way too long)

A few things that tripped me up:

  • The structure has to be exactly right
  • The endpoint configuration is not intuitive the first time
  • Debugging why a recipe didn’t apply is… not fun

If it’s not working, check:

  • Is your repo reachable?
  • Is the path correct?
  • Is the version directory matching what Flex expects?

Most of my time wasn’t spent building this.

It was spent figuring out why it wasn’t working.

⚡ The payoff

Once it clicked, everything got easier.

New project?

  • Require package
  • Config shows up
  • Services wired
  • Done

No more:

  • digging through old repos
  • copying configs manually
  • forgetting one small but important detail

Closing

This isn’t a flashy feature.

It’s not something you demo.

But it’s one of those things that quietly saves time every single time you start something new.

And if you’ve ever found yourself thinking:

“I’ve done this before… why am I doing it again?”

This might be worth the evening it takes to figure out.

Even if that evening is a little frustrating.

Rate limiting in web apps: what to protect before picking a library

Rate limiting in web apps: what to protect before picking a library

The right way to protect a Next.js route from abuse is not to start with the middleware. I know that sounds backwards — everyone reaches for npm install upstash-ratelimit before they’ve thought about what they’re actually protecting. But that sequence almost guarantees you’ll put the wrong limit in the wrong place.

My thesis is simple: rate limiting isn’t a dependency; it’s an abuse policy. And a policy requires decisions before code.

If you’ve ever tuned a threshold “by feel” in production because your logs were showing false positives, you’ve already lived this problem. This post exists so you don’t repeat it.

Rate limiting in Next.js: the order almost nobody follows

The typical sequence: read a tutorial, copy the middleware, tweak the number until people stop complaining. That’s not a policy — that’s trial and error on real users.

The sequence that actually works starts with four questions before you touch any code:

  1. What asset are you protecting? A login endpoint is not the same as a public search API, which is not the same as an incoming webhook.
  2. What abuse are you expecting? Credential stuffing? Scraping? A bot hammering a form? The expected vector determines the shape of the limit.
  3. What does a false positive cost? Over-limit /api/auth/login and you’re locking out real users. Under-limit /api/send-email and you’re paying for spam.
  4. How are you going to observe whether the limit is working? Without metrics, you don’t have a policy — you have hope.

OWASP puts it plainly in their Authentication Cheat Sheet: defensive controls around authentication should include progressive lockout, attempt logging, and a distinction between credential errors and throttle errors. It doesn’t say “install a library.” It says “define the expected behavior and measure it.”

Where people go wrong: the copy-pasted recipe and its hidden cost

The most common pattern I see in Next.js codebases looks something like this:

// middleware.ts — the classic "I copied it from the docs"
import { Ratelimit } from "@upstash/ratelimit";
import { Redis } from "@upstash/redis";

const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  // 10 requests per 10 seconds — why 10? "seemed reasonable"
  limiter: Ratelimit.slidingWindow(10, "10 s"),
});

export async function middleware(request: NextRequest) {
  const ip = request.ip ?? "127.0.0.1";
  const { success } = await ratelimit.limit(ip);

  if (!success) {
    return NextResponse.json({ error: "Too Many Requests" }, { status: 429 });
  }
}

The code works. The problem isn’t in the code — it’s in what isn’t written anywhere:

Problem 1 — Global limit with no route distinction. A middleware applied to matcher: ["/((?!_next).*)", ] limits /api/auth/login and /api/products/search identically. Those are assets with completely different abuse profiles.

Problem 2 — IP as the only key. In Argentina (and anywhere with CGNAT), multiple users share the same public IP. Pure IP-based limiting means your neighbor in the same building can accidentally “DDoS” you without trying.

Problem 3 — No observability. If success = false returns a 429 with no log, you have no idea whether you’re blocking a bot or your own test runner firing integration tests.

Problem 4 — No differential cost. Blocking a product search has low cost. Blocking a legitimate login attempt after an IP change (office → home → VPN) has high cost. The threshold can’t be the same number for both.

This isn’t theoretical. It’s the pattern you find when you search “Next.js rate limiting” on GitHub and look at the first ten implementations. Most share the same middleware with no policy behind it.

The decision matrix: what to look at before writing a single line

Before choosing any implementation — Upstash, express-rate-limit, your own Redis counter, or an external WAF — fill out this matrix for every endpoint you want to protect:

┌─────────────────────────┬────────────────┬──────────────────┬────────────────────┬─────────────────────┐
│ Endpoint                │ Asset          │ Expected abuse   │ FP cost (false+)   │ Key granularity     │
├─────────────────────────┼────────────────┼──────────────────┼────────────────────┼─────────────────────┤
│ /api/auth/login         │ User account   │ Credential stuff │ HIGH — real lockout│ IP + username       │
│ /api/contact            │ Email inbox    │ Mass spam        │ MED — UX damage    │ IP + fingerprint    │
│ /api/search             │ Public DB      │ Scraping         │ LOW — search query │ IP (w/ CGNAT warn)  │
│ /api/webhooks/incoming  │ Data pipeline  │ Replay attack    │ LOW — drop it      │ API key + timestamp │
└─────────────────────────┴────────────────┴──────────────────┴────────────────────┴─────────────────────┘

The most ignored column is FP cost. It’s the one that tells you whether to err inward (too permissive) or outward (too restrictive) — and which of those is actually more tolerable for that specific asset.

For /api/auth/login, OWASP explicitly recommends progressive lockout strategies with user notification — not a silent 429. That requires business logic, not just middleware.

Deliberate implementation: what a real policy looks like in Next.js

With the matrix filled out, the middleware changes shape:

// lib/rate-limit.ts — explicit policy per asset
import { Ratelimit } from "@upstash/ratelimit";
import { Redis } from "@upstash/redis";

const redis = Redis.fromEnv();

// Differentiated policy: each constant documents a decision
export const loginRatelimit = new Ratelimit({
  redis,
  // 5 attempts per minute per IP+username — based on OWASP lockout guidance
  // High FP cost: we prefer a false negative over locking out a real user
  limiter: Ratelimit.fixedWindow(5, "60 s"),
  analytics: true, // observability enabled — non-negotiable
});

export const searchRatelimit = new Ratelimit({
  redis,
  // 100 req/10s per IP — low FP cost, wider margin
  limiter: Ratelimit.slidingWindow(100, "10 s"),
  analytics: true,
});
// app/api/auth/login/route.ts — policy applied with context
import { loginRatelimit } from "@/lib/rate-limit";
import { NextRequest, NextResponse } from "next/server";

export async function POST(request: NextRequest) {
  const body = await request.json();
  const username = body?.username ?? "anon";
  const ip = request.ip ?? "unknown";

  // Composite key: IP + username avoids the CGNAT problem
  // One user under shared CGNAT doesn't affect other distinct users
  const identifier = `login:${ip}:${username}`;

  const { success, limit, remaining, reset } = await loginRatelimit.limit(identifier);

  if (!success) {
    // Explicit log: without this there's no policy, just hope
    console.warn(`[rate-limit] LOGIN blocked — identifier: ${identifier}, reset: ${reset}`);

    return NextResponse.json(
      {
        error: "Too many attempts. Please try again in a few minutes.",
        // Don't expose the exact reset time in production — useful info for attackers
      },
      {
        status: 429,
        headers: {
          "Retry-After": String(Math.ceil((reset - Date.now()) / 1000)),
        },
      }
    );
  }

  // ... authentication logic
}

Two critical differences from the generic middleware: the key is composite (not just IP) and every rejection generates a log. Without the log, there’s no feedback loop to tune the policy.

If you’re deploying on Railway — which is my current stack for Next.js projects — the console.warn logs go straight to the Railway dashboard with zero extra config. That’s enough to start seeing patterns before you need anything more sophisticated.

What this guide can’t tell you: things that require your own data

This matters and I’m not going to bury it at the end: the numbers in this post are starting points, not validated values for your use case.

You don’t know whether 5 attempts per minute is the right login threshold until you:

  • Measure the real distribution of attempts from legitimate users in your app (a user who forgot their password might try 3–4 times in 30 seconds)
  • Observe how many 429s the limit generates in the first week
  • Check whether integration tests or health checks are hitting the same endpoint

Without that data, any number you pick — including the ones in this post — is an educated guess. The goal here isn’t to hand you the threshold; it’s to make sure you know which questions to ask before setting it.

Same goes for library choice. Upstash works well with Next.js on Edge Runtime because Redis operates outside the bundle over HTTP. But if you already have your own Redis on Railway, a simple wrapper with ioredis might be plenty. That decision depends on your infrastructure, not a universal benchmark.

If you’re interested in wiring up deeper observability in Next.js, the post on OpenTelemetry in Spring Boot where logs say OK and traces show the real problem runs on the same principle: without a trace, diagnosis is guesswork.

Common mistakes that turn a policy into noise

Mistake 1 — Rate limiting without a Retry-After header. RFC 6585 specifies that a 429 should include Retry-After. Without it, the client (or the browser) may retry immediately and amplify the load. I covered this pattern in the Retry isn’t free post: the retry cost doesn’t show up in your p95 until it’s already too late.

Mistake 2 — Applying rate limiting on the client. I see this occasionally: throttle on the frontend to “not overload the API.” The client is not a security boundary. Anyone with curl bypasses it instantly.

Mistake 3 — Confusing rate limiting with authentication. A 429 limit doesn’t replace credential validation, tokens, or authorization. It reduces the attack surface over time, but it doesn’t authenticate anything. They’re separate layers, not alternatives.

Mistake 4 — Ignoring the CDN/proxy effect. If your Next.js app sits behind Vercel Edge, Cloudflare, or an nginx, request.ip might return the proxy’s IP, not the real client’s. You need to read X-Forwarded-For carefully — and verify that the header can’t be forged by the client.

// Extract the real client IP with awareness of your stack
function getClientIp(request: NextRequest): string {
  // X-Forwarded-For can have multiple values: "client, proxy1, proxy2"
  // The first is the real client — but only if you trust the proxy setting it
  const forwarded = request.headers.get("x-forwarded-for");
  if (forwarded) {
    return forwarded.split(",")[0].trim();
  }
  return request.ip ?? "unknown";
}

FAQ: real questions about rate limiting in Next.js

Do I need Redis for rate limiting in Next.js?
Not for simple cases, but yes for any deploy running more than one instance. In-memory doesn’t work across multiple replicas because each instance has its own counter. If you’re on Railway with a single container, in-memory might be enough to start — but it’s visible technical debt.

What’s the difference between rate limiting and throttling?
Rate limiting rejects requests that exceed a threshold (429 Too Many Requests). Throttling queues or slows them down without rejecting. For abuse protection, rate limiting is more predictable. Throttling has its place in processing queues, not in public APIs.

Should I put rate limiting in middleware or in each route handler?
Depends on the granularity you need. Global middleware is convenient but applies the same policy to everything. Route handlers give you fine-grained control per asset. The decision matrix above should guide that call — if all your assets have the same abuse profile, the middleware is fine.

What about bots that rotate IPs?
IP-based rate limiting alone doesn’t hold up against sophisticated bots with IP rotation. For that vector, you need browser fingerprinting (TLS JA3, user-agent patterns, behavior analysis) or a dedicated WAF. That’s a different scope than this post — and honestly, if you’ve hit that problem, you need more than a Node library.

Is Upstash Ratelimit the only option for Next.js on Edge Runtime?
No. Upstash works well because its Redis client is HTTP-based and Edge-compatible. But you can also use @vercel/kv if you’re on Vercel, or a Cloudflare Worker with KV if you’re on Cloudflare Workers. The technical constraint is that Edge Runtime doesn’t support TCP sockets — any solution has to speak HTTP for storage.

How do I know if my rate limit is calibrated correctly?
Look at the distribution of 429s in the first 7 days after activation. If the 429s are coming from unique IPs/identifiers you’ve never seen before → the limit is catching abuse. If 429s are coming from recurring IPs that also have successful requests → likely false positive. Without analytics enabled in the library and without logs, you can’t answer this question at all.

My take: the library is an implementation detail

Rate limiting in Next.js web apps is a topic where 80% of the work is technical decision-making and 20% is code. Almost everything written about it does the inverse.

I don’t buy the argument that “any rate limiting is better than none.” A badly calibrated limit on a login endpoint can systematically lock out real users — and that damage is measurable and completely silent if you have no observability.

What I do buy: defining the policy before the code forces you to ask questions the generic middleware never asks you. What asset? What abuse? What’s the cost if I get it wrong in the restrictive direction? Those three questions change the threshold, the key granularity, and the rejection behavior.

Concrete next step: take the most critical endpoint in your app (probably login or registration), fill out the matrix row for that asset, and then write the limit. If you want to see the same thinking applied to Server Actions with Prisma, the post on Prisma Server Actions in Next.js and the N+1 that appears when you least expect it follows the same pattern: diagnose before you solve.

And if the endpoint you’re protecting handles sensitive data, the post on useEffect and state synchronization in React 19 is a good reminder that abstractions that simplify can also hide unexpected behavior — applies equally to middleware.

Primary source:

This article was originally published on juanchi.dev

Rate limiting en aplicaciones web: qué proteger antes de elegir una librería

Rate limiting en aplicaciones web: qué proteger antes de elegir una librería

La solución correcta para proteger una ruta de Next.js contra abuso es no empezar por el middleware. Sé que suena raro — todo el mundo busca npm install upstash-ratelimit antes de pensar en qué está protegiendo. Pero esa secuencia garantiza que vas a poner el límite equivocado en el lugar equivocado.

Mi tesis es simple: rate limiting no es una dependencia; es una política de abuso. Y una política requiere decisiones antes de código.

Si alguna vez terminaste ajustando un threshold “a ojo” en producción porque los logs te mostraron falsos positivos, ya viviste este problema. Esta guía es para que no lo repitas.

Rate limiting aplicaciones web Next.js: el orden que casi nadie respeta

La secuencia típica es: leer un tutorial, copiar el middleware, tunear el número hasta que deje de haber quejas. Eso no es una política — es prueba y error sobre usuarios reales.

El orden que sí funciona empieza con cuatro preguntas antes de tocar código:

  1. ¿Qué activo protegés? Un endpoint de login no es lo mismo que una API pública de búsqueda, que no es lo mismo que un webhook entrante.
  2. ¿Qué abuso esperás? ¿Credential stuffing? ¿Scraping? ¿Un bot que llena formularios? El vector esperado determina el patrón de límite.
  3. ¿Cuánto cuesta un falso positivo? Si limitás de más en /api/auth/login, bloqueás usuarios legítimos. Si limitás de menos en /api/send-email, pagás por spam.
  4. ¿Cómo vas a observar que el límite está funcionando? Sin métricas, no hay política — hay esperanza.

OWASP lo plantea claramente en su Authentication Cheat Sheet: los controles defensivos alrededor de autenticación deben incluir lockout progresivo, logging de intentos y distinción entre errores por credencial vs. errores por throttle. No dice “instalá una librería”. Dice “definí el comportamiento esperado y medilo”.

Dónde se equivoca la gente: la receta copiada y su costo oculto

El pattern más común que veo en codebases Next.js es algo así:

// middleware.ts — el clásico "lo copié de la docs"
import { Ratelimit } from "@upstash/ratelimit";
import { Redis } from "@upstash/redis";

const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  // 10 requests por 10 segundos — ¿por qué 10? "parecía razonable"
  limiter: Ratelimit.slidingWindow(10, "10 s"),
});

export async function middleware(request: NextRequest) {
  const ip = request.ip ?? "127.0.0.1";
  const { success } = await ratelimit.limit(ip);

  if (!success) {
    return NextResponse.json({ error: "Too Many Requests" }, { status: 429 });
  }
}

El código funciona. El problema no está en el código — está en lo que no está escrito:

Problema 1 — Límite global sin distinción de ruta. Un middleware aplicado a matcher: ["/((?!_next).*)", ] limita igual /api/auth/login que /api/products/search. Son activos con perfiles de abuso completamente distintos.

Problema 2 — IP como única clave. En Argentina (y en cualquier ISP con CGNAT), múltiples usuarios comparten la misma IP pública. Limitar por IP pura significa que el vecino del edificio puede hacerte un “accidental DDoS” a vos sin querer.

Problema 3 — Sin observabilidad. Si success = false devuelve 429 y no hay log, no sabés si estás bloqueando un bot o a tu propio usuario de prueba corriendo tests de integración.

Problema 4 — Sin costo diferencial. Bloquear una búsqueda de producto tiene costo bajo. Bloquear un intento de login legítimo después de un cambio de IP (trabajo → casa → VPN) tiene costo alto. El threshold no puede ser el mismo número.

Esto no es teórico. Es el patrón que aparece cuando buscás “Next.js rate limiting” en GitHub y mirás las primeras diez implementaciones. La mayoría comparten el mismo middleware sin política detrás.

La matriz de decisión: qué mirar antes de escribir una línea

Antes de elegir cualquier implementación — Upstash, express-rate-limit, tu propio contador en Redis o un WAF externo — completá esta matriz para cada endpoint que querés proteger:

┌─────────────────────────┬────────────────┬──────────────────┬────────────────────┬─────────────────────┐
│ Endpoint                │ Activo         │ Abuso esperado   │ Costo FP (falso+)  │ Granularidad clave  │
├─────────────────────────┼────────────────┼──────────────────┼────────────────────┼─────────────────────┤
│ /api/auth/login         │ Cuenta usuario │ Credential stuff │ ALTO — bloqueo real│ IP + username       │
│ /api/contact            │ Bandeja email  │ Spam masivo      │ MEDIO — UX dañada  │ IP + fingerprint    │
│ /api/search             │ BD pública     │ Scraping         │ BAJO — búsqueda    │ IP (con CGNAT warn) │
│ /api/webhooks/incoming  │ Pipeline datos │ Replay attack    │ BAJO — ignorar     │ API key + timestamp │
└─────────────────────────┴────────────────┴──────────────────┴────────────────────┴─────────────────────┘

La columna que más se ignora es Costo FP. Es la que determina si errás para adentro (demasiado permisivo) o para afuera (demasiado restrictivo) — y cuál de los dos es más tolerable para ese activo específico.

Para /api/auth/login, OWASP recomienda explícitamente estrategias de lockout progresivo con notificación al usuario, no un 429 silencioso. Eso requiere lógica de negocio, no solo middleware.

Implementación consciente: cómo se ve una política real en Next.js

Con la matriz en mano, el middleware cambia de forma:

// lib/rate-limit.ts — política explícita por activo
import { Ratelimit } from "@upstash/ratelimit";
import { Redis } from "@upstash/redis";

const redis = Redis.fromEnv();

// Política diferenciada: cada constante documenta una decisión
export const loginRatelimit = new Ratelimit({
  redis,
  // 5 intentos por minuto por IP+username — basado en OWASP lockout guidance
  // Costo FP alto: preferimos falso negativo antes que bloquear usuario real
  limiter: Ratelimit.fixedWindow(5, "60 s"),
  analytics: true, // observabilidad habilitada — no negociable
});

export const searchRatelimit = new Ratelimit({
  redis,
  // 100 req/10s por IP — costo FP bajo, margen más ancho
  limiter: Ratelimit.slidingWindow(100, "10 s"),
  analytics: true,
});
// app/api/auth/login/route.ts — política aplicada con contexto
import { loginRatelimit } from "@/lib/rate-limit";
import { NextRequest, NextResponse } from "next/server";

export async function POST(request: NextRequest) {
  const body = await request.json();
  const username = body?.username ?? "anon";
  const ip = request.ip ?? "unknown";

  // Clave compuesta: IP + username evita el problema de CGNAT
  // Un usuario en CGNAT compartido no afecta a otros usuarios distintos
  const identifier = `login:${ip}:${username}`;

  const { success, limit, remaining, reset } = await loginRatelimit.limit(identifier);

  if (!success) {
    // Log explícito: sin esto no hay política, hay esperanza
    console.warn(`[rate-limit] LOGIN bloqueado — identifier: ${identifier}, reset: ${reset}`);

    return NextResponse.json(
      {
        error: "Demasiados intentos. Intentá nuevamente en unos minutos.",
        // No exponer reset exacto en producción — información útil para atacantes
      },
      {
        status: 429,
        headers: {
          "Retry-After": String(Math.ceil((reset - Date.now()) / 1000)),
        },
      }
    );
  }

  // ... lógica de autenticación
}

Dos diferencias críticas respecto al middleware genérico: la clave es compuesta (no solo IP) y cada rechazo genera un log. Sin log, no hay feedback para ajustar la política.

Si usás Railway para deployar — que es mi stack actual para proyectos Next.js — los logs del console.warn van directo al dashboard de Railway sin configuración extra. Es suficiente para empezar a ver patrones antes de necesitar algo más sofisticado.

Límites de esta guía: qué no podés concluir sin tus propios datos

Esto es importante y no lo voy a enterrar al final: los números de esta guía son puntos de partida, no valores validados para tu caso.

No sabés si 5 intentos por minuto es el threshold correcto para login hasta que:

  • Medís la distribución real de intentos de usuarios legítimos en tu app (un usuario que olvidó la contraseña puede intentar 3-4 veces en 30 segundos)
  • Observás cuántos 429 genera el límite en la primera semana
  • Revisás si hay tests de integración o health checks que disparen el mismo endpoint

Sin esos datos, cualquier número que elijas — incluyendo los de este post — es una estimación educada. El objetivo del post no es darte el threshold; es que sepas qué preguntas hacerte antes de fijarlo.

Lo mismo aplica para la elección de librería. Upstash funciona bien con Next.js en Edge Runtime porque Redis opera fuera del bundle. Pero si ya tenés Redis propio en Railway, un wrapper simple con ioredis puede ser suficiente. La decisión depende de tu infraestructura, no de un benchmark universal.

Si te interesa cómo conectar observabilidad más profunda en Next.js, el post de OpenTelemetry en Spring Boot donde los logs dicen OK y los traces muestran el problema tiene el mismo principio: sin traza, el diagnóstico es adivinanza.

Errores comunes que convierten una política en ruido

Error 1 — Rate limiting sin header Retry-After. RFC 6585 especifica que un 429 debería incluir Retry-After. Sin él, el cliente (o el browser) puede reintentar inmediatamente y amplificar la carga. Ya cubrí este patrón en el post de Retry no es gratis: el costo del reintento no aparece en el p95 hasta que ya es tarde.

Error 2 — Aplicar rate limiting en el cliente. Veo esto ocasionalmente: throttle en el frontend para “no sobrecargar la API”. El cliente no es una frontera de seguridad. Cualquier persona con curl la saltea.

Error 3 — Confundir rate limiting con autenticación. Un límite de 429 no reemplaza validación de credenciales, tokens ni autorización. Reduce la superficie de ataque en el tiempo, pero no autentica nada. Son capas distintas, no alternativas.

Error 4 — Ignorar el efecto de CDN/proxy. Si tu Next.js está detrás de Vercel Edge, Cloudflare o un nginx, request.ip puede devolver la IP del proxy, no del cliente real. Necesitás leer X-Forwarded-For con cuidado — y validar que el header no pueda ser falsificado desde el cliente.

// Extraer IP real con conciencia del stack
function getClientIp(request: NextRequest): string {
  // X-Forwarded-For puede tener múltiples valores: "client, proxy1, proxy2"
  // El primero es el cliente real — pero solo si confiás en el proxy que lo setea
  const forwarded = request.headers.get("x-forwarded-for");
  if (forwarded) {
    return forwarded.split(",")[0].trim();
  }
  return request.ip ?? "unknown";
}

FAQ: preguntas reales sobre rate limiting en Next.js

¿Necesito Redis para rate limiting en Next.js?
No para casos simples, pero sí para cualquier deploy con más de una instancia. En-memoria no funciona cuando hay múltiples réplicas porque cada instancia tiene su propio contador. Si usás Railway con un solo container, en-memoria puede alcanzar para empezar — pero es una deuda técnica visible.

¿Cuál es la diferencia entre rate limiting y throttling?
Rate limiting rechaza requests que superan un umbral (429 Too Many Requests). Throttling los encola o los ralentiza sin rechazarlos. Para protección contra abuso, rate limiting es más predecible. Throttling tiene su lugar en colas de procesamiento, no en APIs públicas.

¿Debo poner el rate limiting en middleware o en cada route handler?
Depende de la granularidad que necesitás. Middleware global es conveniente pero aplica la misma política a todo. Route handler te da control fino por activo. La matriz de decisión de más arriba debería guiar esa elección — si todos tus activos tienen el mismo perfil de abuso, el middleware está bien.

¿Qué pasa con los bots que rotan IPs?
IP-based rate limiting sola no alcanza contra bots sofisticados con rotación de IP. Para ese vector, necesitás fingerprinting de browser (TLS JA3, user-agent patterns, behavior analysis) o un WAF dedicado. Es un scope diferente al de este post — y honestamente, si llegaste a ese problema, necesitás más que una librería de Node.

¿Upstash Ratelimit es la única opción para Next.js en Edge Runtime?
No. Upstash funciona bien porque su cliente Redis es HTTP-based y compatible con Edge. Pero también podés usar @vercel/kv si estás en Vercel, o un worker de Cloudflare con KV si usás Cloudflare Workers. La restricción técnica es que Edge Runtime no soporta sockets TCP — cualquier solución tiene que hablar HTTP para el almacenamiento.

¿Cómo sé si mi rate limit está bien calibrado?
Mirá la distribución de 429 en los primeros 7 días de activación. Si los 429 vienen de IPs/identificadores únicos que nunca viste antes → el límite está capturando abuso. Si los 429 vienen de IPs recurrentes que también tienen requests exitosos → posible falso positivo. Sin analytics activado en la librería y sin logs, no podés responder esta pregunta.

Mi postura: la librería es un detalle de implementación

Rate limiting en aplicaciones web con Next.js es un tema donde el 80% del trabajo es decisión técnica y el 20% es código. Casi toda la literatura hace lo inverso.

No compro el argumento de que “cualquier rate limiting es mejor que ninguno”. Un límite mal calibrado sobre un endpoint de login puede bloquear usuarios legítimos de forma sistemática — y ese daño es medible y silencioso si no tenés observabilidad.

Lo que sí compro: definir la política antes del código obliga a hacerse preguntas que el middleware genérico no te hace. ¿Qué activo? ¿Qué abuso? ¿Qué costo si me equivoco para afuera? Esas tres preguntas cambian el threshold, la granularidad de la clave y el comportamiento ante el rechazo.

El próximo paso concreto: tomá el endpoint más crítico de tu app (probablemente login o registro), completá la fila de la matriz para ese activo, y recién ahí escribí el límite. Si querés ver cómo aplicar el mismo pensamiento a Server Actions con Prisma, el post de Prisma Server Actions en Next.js y el N+1 que aparece cuando no lo esperás tiene el mismo patrón: diagnóstico antes de solución.

Y si el endpoint que protegés maneja datos sensibles, el post sobre useEffect y sincronización de estado en React 19 es un recordatorio de que las abstracciones que simplifican también pueden esconder comportamiento inesperado — aplica igual para middleware.

Fuente original:

Este artículo fue publicado originalmente en juanchi.dev

What Are Lakehouse Catalogs? The Role of Catalogs in Apache Iceberg

The three core responsibilities of a lakehouse catalog

This is Part 7 of a 15-part Apache Iceberg Masterclass. Part 6 covered the write process and explained how the catalog enables atomic commits. This article covers what catalogs are, why they matter, and how to choose between the many options available in 2026.

A lakehouse catalog is the component that answers one question: “Where is the current metadata for this table?” Without a catalog, every engine would need to independently locate and track metadata files. With a catalog, there is a single source of truth that coordinates reads, writes, and access control across all engines.

Table of Contents

  1. What Are Table Formats and Why Were They Needed?
  2. The Metadata Structure of Current Table Formats
  3. Performance and Apache Iceberg’s Metadata
  4. Technical Deep Dive on Partition Evolution
  5. Technical Deep Dive on Hidden Partitioning
  6. Writing to an Apache Iceberg Table
  7. What Are Lakehouse Catalogs?
  8. Embedded Catalogs: S3 Tables and MinIO AI Stor
  9. How Iceberg Table Storage Degrades Over Time
  10. Maintaining Apache Iceberg Tables
  11. Apache Iceberg Metadata Tables
  12. Using Iceberg with Python and MPP Engines
  13. Streaming Data into Apache Iceberg Tables
  14. Hands-On with Iceberg Using Dremio Cloud
  15. Migrating to Apache Iceberg

What a Catalog Does

The three core responsibilities of a lakehouse catalog

Every Iceberg catalog performs three functions:

Store the current metadata pointer. When a query engine asks for table analytics.orders, the catalog returns the location of the current metadata.json file (e.g., s3://warehouse/orders/metadata/v42.metadata.json). This is the most fundamental responsibility. As described in Part 6, the atomic update of this pointer is what makes ACID transactions possible.

Manage namespaces. Catalogs organize tables into hierarchical namespaces (databases, schemas). This provides logical organization (production.analytics.orders vs staging.analytics.orders) and is the foundation for access control.

Enforce access control. Catalogs determine which users and engines can read, write, or manage specific tables and namespaces. This ranges from simple table-level permissions to fine-grained column-level and row-level security.

The REST Catalog Protocol

How the Iceberg REST Catalog Protocol works from table load through atomic commit

The Iceberg REST Catalog specification defines a standard HTTP API for catalog operations. This protocol has become the industry standard because it decouples the catalog implementation from the engine.

The key operations:

The key operations

The protocol includes credential vending: the catalog returns short-lived storage credentials alongside the metadata location, so the engine can access the data files directly without needing permanent storage credentials. This is important for multi-tenant environments where Dremio and other engines need scoped access to specific tables.

Why the REST Protocol Matters

Before the REST catalog specification, every engine needed a custom integration for each catalog type. Spark had its own Hive Metastore connector, Trino had a different one, and adding a new catalog meant updating every engine. The REST protocol standardizes this: any engine that speaks REST can talk to any catalog that implements the specification.

This is what makes the Iceberg ecosystem genuinely multi-engine. You can use Spark for ETL, Dremio for interactive analytics, Trino for exploration, and Flink for streaming, all pointed at the same REST catalog. Each engine sees the same tables, the same schemas, and the same snapshots.

Multi-Engine Coordination

When multiple engines share a catalog, the catalog becomes the coordination point for concurrent access. The atomic compare-and-swap mechanism ensures that two engines writing to the same table cannot corrupt each other’s commits. This is fundamentally different from file-system-based metastores where coordination relies on file renames that may not be atomic on object storage.

Governance Portability

One of the biggest concerns in the catalog landscape is governance portability. Access control policies (who can query what) are defined in the catalog, but there is no industry standard for sharing these policies across catalogs. If you set up row-level security in one catalog, that policy does not automatically transfer to another.

This is why many architects recommend picking a catalog that will serve as the single governance boundary and having all engines connect through it, rather than having multiple catalogs with duplicate governance rules.

The Catalog Landscape

Open source vs managed catalog options in the 2026 lakehouse ecosystem

Open Source Catalogs

Apache Polaris (Apache Incubating). The leading vendor-neutral REST catalog implementation. Co-created by Snowflake and Dremio, Polaris is designed to be engine-agnostic and cloud-agnostic. It implements the full REST Catalog spec with fine-grained access control and credential vending. It is the foundation for Dremio’s Open Catalog.

Nessie. Differentiates itself with Git-like branching and merging for data. You can create branches, make changes to multiple tables, and merge them atomically. This is useful for testing pipeline changes or implementing multi-table transactions.

Unity Catalog OSS. Databricks’ open-source catalog offering. It provides multi-format support (Delta, Iceberg, Hudi) and includes AI/ML asset management (models, features). Closely tied to the Databricks ecosystem.

Lakekeeper. A lightweight, Rust-native REST catalog implementation focused on performance and minimal operational footprint. Good for teams that want a self-hosted catalog without the complexity of larger platforms.

Apache Gravitino (Incubating). A federation-focused catalog that can bridge multiple underlying catalogs and storage systems. Designed for organizations that need a unified metadata view across multiple Iceberg catalogs, Hive metastores, and other data sources.

Managed Catalogs

Dremio Open Catalog. A managed Polaris-based catalog that includes automatic table optimization (compaction, snapshot expiry, orphan cleanup) and integrates with Dremio’s query engine, semantic layer, and AI capabilities.

AWS Glue. Amazon’s managed metastore service. Widely used because it is integrated with the AWS ecosystem (Athena, EMR, Redshift Spectrum). Supports Iceberg tables natively and acts as both a Hive-compatible metastore and an Iceberg catalog.

Databricks Unity Catalog (managed). The enterprise version of Unity Catalog with additional governance features, lineage tracking, and AI asset management. Tightly integrated with the Databricks runtime.

Snowflake Horizon. Snowflake’s catalog and governance layer that supports Iceberg tables in Snowflake-managed storage.

Google BigLake. Google Cloud’s managed metadata service for Iceberg tables on GCS.

Microsoft OneLake. Microsoft’s unified storage and catalog layer within the Fabric ecosystem.

How to Choose a Catalog

The decision depends on three factors:

The decision depends on three factors

The safest long-term choice is a REST-catalog-compatible implementation because every major engine supports the protocol. If you start with a REST catalog, you can swap implementations later without changing your engine configurations.

Catalogs Are Not Optional

Some teams try to use Iceberg without a proper catalog, relying on Hadoop-style file system catalogs that use file renames for atomicity. This works on HDFS but is unreliable on object storage (S3 does not support atomic renames). For production lakehouses on cloud storage, a proper catalog with server-side compare-and-swap is essential.

Part 8 covers a newer approach where the catalog is embedded directly in the storage layer.

Books to Go Deeper

Free Resources

What It Really Takes to Become a Senior Software Engineer

I have always dreamed of becoming a Senior Software Engineer, and throughout my career I have realised that it requires far more than just technical ability. Currently, I work as a Software Engineer III at LexisNexis Risk Solutions, and I took the opportunity to gather advice and perspectives from engineers, managers, product, architecture, and leadership across the organisation.

Thanks to everyone who took the time to share their thoughts, experiences, and insights.

What stood out most was how consistent the advice was. Different people, different roles, but very similar themes. This post is a compilation of those common patterns.

Core Mindset

Be Self-Directed

Senior engineers are expected to operate with autonomy.

  • Take ambiguous problems and define the path forward
  • Fill in missing details yourself and continue to validate
  • Work independently without constant guidance
  • Take ownership and run with projects
  • Be proactive rather than waiting for tasks

The shift to senior level is largely about moving from “being assigned work” to “owning problems.”

Think Bigger Than the Task

  • Think in systems and larger building blocks
  • Understand product goals and business context
  • Learn the language of the product
  • Consider trajectory, tradeoffs, and long-term impact
  • Not all solutions are equal — think carefully before acting

A strong theme was that seniors don’t just solve problems — they evaluate the quality and impact of solutions.

Be Curious and Experimental

Growth comes from exploration.

  • Dig deeper into problems
  • Try things and experiment
  • Learn by building
  • Don’t be afraid to make mistakes
  • Figure things out yourself before escalating

Curiosity and willingness to explore came up repeatedly as key growth drivers.

Technical Growth

Build Strong Technical Foundations

Technical excellence still matters.

  • Develop strong engineering fundamentals
  • Improve your system knowledge
  • Become knowledgeable in a specific domain
  • Build subject matter expertise
  • Understand infrastructure, observability, dashboards, alerts, and pipelines

Technical strength is still required — but it needs to be reliable and deep.

Keep Upskilling

Software changes constantly — adaptation is part of the job.

  • Take courses
  • Study good codebases
  • Fork and learn from GitHub/open-source projects
  • Stay technically passionate
  • Keep improving your engineering craft

Engineering Excellence

Focus on quality and ownership.

  • Don’t just “make it work”
  • Catch issues early
  • Monitor systems and visibility
  • Maintain high standards
  • Be methodical and thoughtful
  • Learn when to slow down and think carefully

Communication & Collaboration

Communication Is a Superpower

Nearly everyone highlighted communication.

  • Ask questions
  • Share feedback
  • Explain concepts clearly
  • Adapt explanations to different audiences
  • Present ideas confidently
  • Talk through technical decisions
  • Keep managers informed about concerns and goals

Strong communication builds trust and influence.

Help Others Grow

Senior engineers multiply the effectiveness of the team.

  • Mentor juniors
  • Share knowledge openly
  • Review PRs thoughtfully
  • Teach along the way
  • Be approachable and helpful

“When you can explain things clearly to people below and above your level, that’s valuable.”

Work Well With People

Technical skill alone is not enough.

  • People matter a lot
  • Everyone is a peer
  • Collaborate effectively
  • Learn from seniors and peers
  • Avoid negativity and unnecessary criticism
  • Pick your battles

A recurring theme was maturity and professionalism.

Ownership & Visibility

Take Initiative

  • Volunteer for projects
  • Speak up when something should improve
  • Create momentum
  • Don’t shy away from responsibility
  • Own outcomes, not just tasks

Opportunity doesn’t always come directly — you often need to step into it.

Make Your Work Visible

Good work should be discoverable.

  • Track your changes and contributions
  • Create PBIs/tasks for your work
  • Participate actively in PR discussions
  • Document your work clearly
  • Contribute to visible projects

Visibility helps others understand your impact.

Product & Business Awareness

Understand the “Why”

Senior engineers connect technical decisions to business outcomes.

  • Understand customer and product needs
  • Solve meaningful problems
  • Think beyond implementation
  • Look for opportunities to innovate
  • Balance engineering with practicality

Better decisions come from understanding context, not just code.

Career Growth Advice

Be Intentional About Your Career

Growth does not happen automatically.

  • Understand your manager’s expectations
  • Set objectives aligned to your career goals
  • Look at strong examples around you
  • Ask directly for opportunities and feedback
  • Advocate for yourself
  • Don’t say “yes” to everything if you are unhappy

Final Thoughts

Being a senior engineer is not just about writing better code.

It is about:

  • owning problems,
  • communicating clearly,
  • helping others succeed,
  • understanding the bigger picture,
  • and consistently demonstrating maturity, initiative, and reliability.

Technical skill opens the door, but leadership, communication, and ownership are what truly distinguish senior engineers.

… and a final gem from our recently retired Principal Engineer: “Don’t forget to stay healthy, lift weights”

Microservices Were Never About Technology

Every failed microservices adoption I have seen made the same mistake: treating microservices as an infrastructure pattern instead of an organisational one. The technology is the easy part. The hard part is everything else.

The microservices conversation in most engineering teams goes something
like this. The monolith is getting unwieldy. Deployments are slow.
The codebase is hard to navigate. A senior engineer proposes breaking
things apart into services. The team agrees. They spend six months
doing it. Things get worse.

The services are too small or too large. Nobody agrees on where the
boundaries should be. Simple features now require coordinating changes
across three repositories. A bug that used to take twenty minutes to
debug now takes two hours because it crosses service boundaries.
Deployments are more frequent but individually more fragile. The team
is working harder than before and moving slower than before.

They blame the technology. The technology is not the problem.

Microservices failed for them for the same reason they fail for most
teams that adopt them: the team treated decomposition as a technical
decision and ignored the organisational reality that microservices
are actually designed to solve. You cannot separate the architecture
from the team structure. Conway’s Law is not optional.

What microservices were actually invented for

Amazon is the origin story most people know. The mandate from Jeff
Bezos in the early 2000s: all teams will expose their data and
functionality through service interfaces. All teams will communicate
through these interfaces. No other form of interprocess communication
is allowed. Anyone who doesn’t do this will be fired. He was serious.

The problem Bezos was solving was not technical. Amazon’s engineering
organisation had grown to a size where teams were deeply entangled with
each other. Team A could not deploy without coordinating with Team B,
which needed sign-off from Team C, which had a dependency on Team D.
Every change required a synchronisation meeting. Every release was a
negotiation. The coupling between teams was strangling the organisation’s
ability to move.

Services were the solution because they forced a contract between teams.
If Team A owns Service A, and Team B owns Service B, and they communicate
only through a defined API, then Team A can change anything inside
Service A without asking Team B’s permission. Team B can deploy Service B
on its own schedule. The organisational autonomy is enforced by the
technical boundary.

This is the thing that most microservices adoptions miss entirely.
Services are not primarily a way to scale technology. They are a way
to scale teams. The technical properties of services (independent
deployment, technology flexibility, fault isolation) are valuable
side effects of the organisational property (team autonomy).

If you adopt microservices without the organisational changes that
make them valuable, you get all the costs of distributed systems
and none of the benefits.

The cost of distribution

A monolith, for all its problems, has properties that distributed
systems do not have and cannot have.

A function call inside a monolith is reliable. Either it works or
it throws an exception. It is fast. It completes in microseconds.
It participates in the same database transaction as the code that
called it. If the whole operation needs to be rolled back, it is.

A network call between services is unreliable. It might succeed.
It might fail. It might succeed on the server side and fail on the
network before the response reaches the caller. It might time out,
leaving you with no information about whether the remote operation
completed. It is slow relative to a function call. It crosses
a transaction boundary, which means if something fails after the
call succeeded, you have a consistency problem that cannot be
resolved by a rollback.

This is not an implementation detail to be engineered around. It is
a fundamental property of distributed systems, described precisely
in the fallacies of distributed computing that Peter Deutsch wrote
in 1994 and that the industry has been rediscovering ever since.

The network is not reliable. Latency is not zero. Bandwidth is not
infinite. The network is not secure. Topology changes. There is not
one administrator. Transport cost is not zero. The network is not
homogeneous.

Every service boundary you add to your system is a place where these
fallacies apply. Every service call is an opportunity for latency,
failure, and consistency problems that simply do not exist inside
a monolith. The question is whether the organisational benefits of
the boundary justify the distributed systems cost of maintaining it.

For a team of eight people working on one product, they almost never do.

Where the boundaries actually belong

The most common failure mode in microservices adoption is drawing
service boundaries around technical concerns rather than business
ones. Teams create an “auth service,” a “notification service,”
a “user service,” a “payment service.” These feel like natural
decompositions because they map to recognisable technical concepts.

They are terrible service boundaries.

An auth service that every other service must call to validate a
token is not a service. It is a shared library that has been deployed
as infrastructure, adding network latency and a failure mode to every
authenticated request in the system. If the auth service is slow,
everything is slow. If the auth service is down, everything is down.
You have taken a piece of logic that could live as a function call
and made it a distributed systems problem.

A notification service is not a service. It is a collection of side
effects that have been externalized, creating a situation where the
service that wants to send an email must make a network call, handle
the failure case, and figure out what to do if the notification
service is unavailable at the moment the email needs to be sent.

The boundaries that work are the ones that map to bounded contexts
in the business domain. Not “the thing that handles auth” but “the
thing that owns everything about how customers interact with our
platform.” Not “the thing that sends notifications” but “the thing
that owns the customer communication history and all the rules about
when and how to communicate.”

These boundaries are harder to identify. They require understanding
the business deeply enough to know where the real seams are. They
require conversations with product managers and domain experts, not
just with engineers. They change as the business evolves. But they
are the boundaries that, when you respect them, produce services
that teams can own autonomously and evolve independently.

Domain-Driven Design’s concept of bounded contexts is the clearest
framework for finding these boundaries. The bounded context defines
the scope within which a particular domain model applies. At the
edge of the bounded context, the model changes. That is where the
service boundary belongs.

# A service boundary drawn around a technical concern.
# Every other service calls this. Auth is now a distributed dependency.
#
# Bad:
class AuthService:
    async def validate_token(self, token: str) -> User:
        ...
    async def create_token(self, user_id: str) -> str:
        ...
    async def revoke_token(self, token: str) -> None:
        ...


# A service boundary drawn around a business capability.
# This service owns everything about an order, including its auth context.
# Other services don't call into it for auth. They communicate
# through events when they need to know something happened.
#
# Better:
class OrderService:
    async def place_order(self, customer_id: str, items: list) -> Order:
        # Auth context is resolved here, not farmed out to a network call
        customer = await self.customer_repository.get(customer_id)
        if not customer.can_place_orders():
            raise InsufficientPermissionsError()
        ...

    async def cancel_order(self, order_id: str, requesting_customer_id: str) -> None:
        order = await self.order_repository.get(order_id)
        if order.customer_id != requesting_customer_id:
            raise InsufficientPermissionsError()
        ...

Conway’s Law is a constraint, not a suggestion

Mel Conway observed in 1967 that organisations produce systems that
mirror their communication structures. A team with three groups will
produce a system with three components. This is not because they
planned to. It is because the system reflects who talks to whom.

The implication that most teams don’t fully absorb: if you want a
particular system architecture, you need the corresponding
organisational structure. You cannot have a microservices architecture
with a team structure designed for a monolith. The organisation will
fight the architecture until one of them wins, and the organisation
usually wins because it existed first.

This is why Amazon’s microservices worked. The service boundaries
and the team boundaries were the same boundaries. Team A owns Service A.
Not “Team A and Team B both contribute to Service A.” Not “Service A
is maintained by whoever has time.” One team, one service, full ownership.
The organisational autonomy and the technical autonomy were the same thing.

Most microservices adoptions separate these. The same team that used
to work on the monolith now works on six services. They have all the
coordination overhead of distributed systems and none of the team
autonomy that makes it worth it. They still talk to each other constantly
because they’re the same people. The service boundaries don’t reflect
team boundaries because there are no team boundaries. There is one team
doing distributed systems for no organisational reason.

The inverse Conway maneuver, a term coined by Thoughtworks, is the
deliberate version: you design the team structure you want, then
let the architecture follow from it. If you want a payments service
that can be developed and deployed independently, you need a payments
team that can make decisions and ship code independently. If you do
not have or cannot create that team, you do not have the prerequisite
for the payments service.
The prerequisite check before splitting a service:

Who will own this service?
“The backend team” is not an answer.
A named, stable, small team is an answer.
Can that team deploy the service without coordinating with
other teams?
If not, the boundary is wrong or the ownership is wrong.
Can that team change the service’s internal implementation
without changing any other service?
If not, the boundary is wrong.
Is there a defined contract (API, event schema) between this
service and its consumers?
If not, you don’t have a service. You have a distributed module.
Does the team have enough context about the business domain
this service represents to make good decisions autonomously?
If not, the team needs to exist and stabilise before the service
should be extracted.

If any of these answers is no, the split is premature.

The operational surface nobody accounts for

When a team decides to split their monolith into ten services, they
usually have a plan for the technical decomposition. They rarely have
a plan for what they are about to own operationally.

A monolith has one deployment pipeline. One set of infrastructure
to configure. One place to look at logs. One set of metrics. One
runbook for when things go wrong. The operational complexity is low.

Ten services have ten deployment pipelines. Ten infrastructure
configurations. Log aggregation that spans services. Distributed
tracing to follow a request through multiple services. Ten runbooks,
except the incidents that matter will involve multiple services and
none of the runbooks will cover that. Service discovery. Health
checking at the inter-service level. Circuit breakers for when
downstream services are degraded.

None of this complexity is impossible to manage. It is all solvable.
But it requires a team that has the capacity to manage it, tools
that have been set up before the split happens, and expertise that
takes time to develop.

Most teams split their services and then build the operational
infrastructure retroactively, while also trying to deliver product
work, while also debugging the new distributed systems problems they
did not have before. This is where the eighteen months of slowdown
comes from.

The teams that do this well build the operational infrastructure
first. They get distributed tracing working in the monolith before
they split it. They standardise their deployment pipeline before
they have ten of them. They establish logging conventions before
they have ten services emitting logs in subtly different formats.

# The operational baseline that must exist before splitting services.
# This is not optional infrastructure to add later.

# Centralised structured logging
logging:
  format: json
  fields:
    service: ${SERVICE_NAME}
    version: ${SERVICE_VERSION}
    environment: ${ENVIRONMENT}
    trace_id: ${TRACE_ID}    # Must be propagated across service calls
    span_id: ${SPAN_ID}

# Every service exposes these endpoints. No exceptions.
health_endpoints:
  liveness: /healthz      # Is the process running?
  readiness: /ready       # Is it ready to serve traffic?
  metrics: /metrics       # Prometheus metrics

# Every inter-service call propagates these headers
trace_propagation:
  headers:
    - traceparent           # W3C Trace Context
    - tracestate

# Every service has these alerts configured before it handles traffic
minimum_alerts:
  - error_rate_above_1_percent
  - p99_latency_above_1_second
  - service_unavailable

The monolith that should stay a monolith

Not every system should be microservices. This is easy to say and
hard to accept in an industry where microservices became the mark
of a serious engineering organisation.

The monolith that should stay a monolith is the one where:

The team is small enough that coordination overhead is low. Five
to eight engineers can coordinate in a daily standup without the
synchronisation cost becoming significant. For a team this size,
the organisational problem that microservices solve does not exist.

The domain is not yet well understood. Early-stage products have
unstable domain models. The concepts that seem fundamental change
as you learn what you’re actually building. Service boundaries drawn
around an unstable domain model have to be redrawn as the domain
stabilises, which is expensive and demoralising. The monolith lets
the domain model evolve cheaply. Split when the domain is understood.

The operational team does not exist. If nobody owns the infrastructure
that a distributed system requires, the system will be operated badly.
A well-operated monolith beats a poorly-operated distributed system
every time.

The internal structure can be improved without splitting. A modular
monolith with clear internal boundaries, enforced through package
structure and dependency rules, provides most of the cognitive benefits
of microservices (clear ownership, bounded contexts, interface discipline)
without the distributed systems cost. It is not a compromise. For the
right team and domain, it is the correct architecture.

# A modular monolith with enforced boundaries.
# orders/ cannot import directly from payments/.
# They communicate through defined interfaces.
# This is achievable without distributed systems.

# src/orders/service.py
from orders.repository import OrderRepository
from orders.events import OrderPlaced  # Orders emits events
# from payments.service import PaymentService  # This import is forbidden
                                               # enforced by linting rules

class OrderService:
    def __init__(
        self,
        repository: OrderRepository,
        event_bus: EventBus,
        payment_gateway: PaymentGateway,  # Interface, not concrete payments module
    ):
        self.repository = repository
        self.event_bus = event_bus
        self.payment_gateway = payment_gateway

    async def place_order(self, customer_id: str, items: list) -> Order:
        order = Order.create(customer_id=customer_id, items=items)
        await self.repository.save(order)
        await self.event_bus.publish(OrderPlaced(order_id=order.id))
        return order

# The payments module listens for OrderPlaced events.
# It never gets called directly by orders.
# The boundary is real. It is enforced by design, not by a network.

# src/payments/handlers.py
from orders.events import OrderPlaced  # Reading event schema is allowed

class PaymentEventHandler:
    async def on_order_placed(self, event: OrderPlaced) -> None:
        await self.payment_service.initiate_payment(order_id=event.order_id)

This is a real architecture that scales further than most teams
think before the overhead of splitting services becomes worth paying.
Shopify ran a version of this for years. Stack Overflow still does.
They are not unsophisticated organisations.

What the good teams understand

The teams that have figured out distributed systems share a
perspective that took most of them several years and at least one
failed microservices adoption to arrive at.

Services are not about code organisation. They are about team
organisation. A service boundary that does not correspond to a team
boundary is overhead without benefit.

The overhead of distributed systems is real, permanent, and
compounding. You pay it forever. It needs to buy something worth
having. For a team that is too large to coordinate, team autonomy
is worth having. For a team that is not yet at that size, it is
not.

The correct direction of reasoning is: we have an organisational
problem, what architecture solves it? Not: we have an architecture
trend, what organisation do we need to adopt it?

Microservices adopted as a technical decision produce the costs
of distribution and the politics of boundary negotiation without
the autonomy that makes them valuable. Microservices adopted as
an organisational decision, by teams that have done the work of
defining ownership and building operational foundations, produce
systems that actually deliver what the pattern promises.

The technology has never been the hard part.
The hard part is everything the technology forces you to sort out first.
Most teams skip that part and wonder why the technology failed them.

JS Crime Scene: The Misleading Array

🕵️ JS Crime Scene: The Case of the Misleading Array

📹 Surveillance Footage

The full investigation is available on TikTok:

View the Evidence

The call came in early.

An application was completely misbehaving, data was leaking out of order, and the logs were a mess.

The victim?

Developer Sanity

Someone tried to sort a simple array of integers, and JavaScript decided to play by its own rules.

Let’s inspect the evidence, identify the culprit, and deploy the one-line fix before production collapses entirely.

🔍 The Crime Scene Report

We found a collection of numerical values at the center of the incident.

No complex objects.

No deeply nested structures.

Just raw numbers.

Here is the exact code executed at the scene:

// The Lineup (Original Array)
const numbers = [1, 10, 2, 25, 5];

// The Attempted Sort
numbers.sort();

console.log(numbers);

// Output:
// [1, 10, 2, 25, 5]

📋 The Expected vs. Actual Output

Intended Behavior (Math) Actual Reality (JavaScript) Status
[1, 2, 5, 10, 25] [1, 10, 2, 25, 5] ❌ Maliciously Broken

Look closely at that result.

10 comes before 2?

25 comes before 5?

Did mathematics quietly stop working overnight?

Not exactly.

🕵️‍♂️ The Culprit: UTF-16 String Sorting

When you call .sort() without providing a comparator function, JavaScript does not sort numbers mathematically.

Instead, it secretly performs a type coercion operation and converts every element into a string first.

Our innocent integers suddenly became:

  • 1 → "1"
  • 10 → "10"
  • 2 → "2"
  • 25 → "25"
  • 5 → "5"

Once converted to strings, JavaScript compares them character-by-character using UTF-16 code unit values.

Essentially, the engine starts sorting your numbers like dictionary words.

📖 Think of It Like a Dictionary

In a dictionary:

  • "Apple" comes before "Banana"
  • because "A" comes before "B"

JavaScript applies that exact same logic here.

For example:

  • "10" starts with "1"
  • "2" starts with "2"

Since the character "1" has a lower UTF-16 value than "2", JavaScript places "10" first.

The engine isn’t malfunctioning.

It’s simply following text-based sorting rules.

🛠️ The Fix: Deploying a Comparator Function

To force JavaScript to sort numerically, you must explicitly tell the engine how two values should be compared.

Here’s the proper fix:

const numbers = [1, 10, 2, 25, 5];

// Numeric Sort Fix
numbers.sort((a, b) => a - b);

console.log(numbers);

// Output:
// ✅ [1, 2, 5, 10, 25]

One tiny comparator function.

Massive difference.

⚙️ How the Comparator Actually Works

This line:

(a, b) => a - b

subtracts the second value from the first.

JavaScript then interprets the result.

Negative Result

If the result is negative:

2 - 5 = -3

JavaScript understands that 2 should come before 5.

Positive Result

If the result is positive:

10 - 2 = 8

JavaScript knows 2 should move ahead of 10.

Zero

If the result is 0, both values are considered equal, and their order remains unchanged.

💡 Pro Tip: Reverse Sorting

Need descending order instead?

Just flip the subtraction:

numbers.sort((a, b) => b - a);

Output:

[25, 10, 5, 2, 1]

🚨 Case Closed

Another JavaScript mystery solved.

The golden rule?

Never trust the default behavior of .sort() when working with numbers.

Always provide a comparator function if you want predictable numeric sorting behavior.

Otherwise, JavaScript will happily treat your integers like alphabetized grocery items.

And that’s how 10 ends up beating 2 in a fight it should never win.

Have you ever been burned by JavaScript’s default .sort() behavior in production?

Drop your horror stories below 👇