Mert

The Industrialization of ML: A Deep Dive into Uber’s AI Platform Architecture

yazar

This article serves as a technical follow-up to our KubeCon 2026 coverage, providing a comprehensive deep dive into the architecture and evolution of Uber’s machine learning platform.

When Uber presented at KubeCon Europe 2026, the numbers they shared silenced the room: 1 million+ diverse workloads deployed onto 200 Kubernetes clusters, 20,000 models trained monthly, 5,300 models actively in production, and over 30 million peak predictions per second.

For most organizations, achieving even 1% of that scale is a multi-year roadmap. Uber’s platform doesn’t just support their business; it is their business. From surge pricing and ETA estimation to fraud detection and Generative AI-driven customer support, machine learning sits in the critical path of every user interaction.

But Uber didn’t arrive at this architecture overnight. Their journey from scattered Python scripts to a globally federated, Kubernetes-native AI control plane is a masterclass in platform engineering.

Here is the deep dive into how Uber industrialized machine learning, the bottlenecks they hit along the way, and the architectural blueprints they’ve proven at hyperscale.

1. The Pre-Platform Era: The Fragmentation Tax (Pre-2017)

Before 2017, data science at Uber looked like data science at most fast-growing startups today: entirely fragmented.

  • The How: Data scientists worked on individual laptops or dedicated EC2 instances using a fragmented toolkit (R, scikit-learn, bespoke Python scripts).
  • The What: Each team built separate, one-off systems to pull data, train models, and serve predictions.
  • The Bottleneck: Models could only be as large as what fit on a single machine. Once a model was trained, “deploying” it often meant handing an opaque pickle file to a backend engineering team to rewrite in Java or Go.

This lack of standardization meant high operational friction. Teams couldn’t easily share features, monitor model drift, or scale prediction serving. Uber realized that building custom infrastructure for every ML use case was economically and operationally unsustainable. They needed a centralized factory.

2. Michelangelo: Standardizing the ML Factory (2017–2022)

To solve the fragmentation tax, Uber built Michelangelo, an end-to-end internal machine learning platform designed to democratize ML across the company. The goal was to standardize the entire lifecycle—from data prep to model deployment.

Michelangelo introduced several architectural patterns that have since become industry standard:

  • The Centralized Feature Store: Instead of every team writing their own Spark jobs to calculate “user’s trip frequency in the last 30 days,” features were calculated once, stored, and shared.
  • Offline vs. Online Split: Michelangelo cleanly separated batch feature computation (using Apache Spark and Hive for historical data) from real-time feature computation (using Apache Kafka and Flink for streaming data like GPS coordinates).
  • Deployment Standardization: Models were deployed in three specific modes: Offline (Spark batch jobs for overnight predictions), Online (load-balanced API endpoints responding in <10ms), and Library (embedded directly into microservices for the absolute lowest latency).

Michelangelo was a massive success, bringing hundreds of use cases into production. However, as the industry shifted toward Deep Learning and Large Language Models (LLMs), Michelangelo’s underlying orchestration layer began to crack under the weight.

3. Hitting the Wall: The Kubernetes & Ray Migration (2023–2024)

By mid-2023, Uber’s ML workloads were primarily running on a legacy job gateway service called MADLJ (Michelangelo Deep Learning Jobs). While functional, it forced ML engineers to manually handle resource management—choosing specific regions, zones, and clusters based on GPU availability.

This led to the “stranded compute” problem: Cluster A would be operating at 100% capacity with a massive queue of training jobs, while Cluster B sat 50% empty because engineers hadn’t manually targeted it.

To prepare for the Generative AI boom, Uber executed a massive architectural shift: moving the entire ML platform to Kubernetes and Ray.

Curing Stranded Compute via Federation

Uber decoupled the user experience from the infrastructure. They introduced a Global Control Plane built on standard Kubernetes architecture.

  • Developers now submit declarative jobs (via a Python-native workflow service called Uniflow) simply stating: “I need to train this PyTorch model on 8 A100 GPUs.”
  • The Global Control Plane’s custom Job Controller automatically scans dozens of regional Kubernetes clusters (the Local Control Plane), identifies available capacity, and schedules the Ray workers accordingly.

Overcoming ETCD Limits with Transparent Persistence

Scaling Kubernetes to handle 100+ purpose-built Custom Resource Definitions (CRDs) representing the ML lifecycle introduced a new problem: etcd (Kubernetes’ default datastore) choked under the high-cardinality metadata of 30 million predictions a second.
To solve this, Uber engineered a transparent storage abstraction. While the system interacts with standard Kubernetes objects via the API, the underlying metadata is seamlessly synchronized with a horizontally scalable MySQL backend, completely bypassing ETCD’s limitations.

4. The GenAI & Agentic Era (2024–2026)

With a federated Kubernetes and Ray foundation in place, Uber was uniquely positioned to absorb the immense compute requirements of Generative AI and Agentic systems.

Uber leverages a hybrid hardware approach: heavily utilizing on-prem A100 GPU clusters alongside Google Cloud H100 instances. To maximize GPU utilization (MFU – Model Flops Utilization) when training massive open-source models (like Llama or Mixtral), the platform engineering team implemented severe infrastructure-level optimizations:

  • Distributed Memory Offloading: Because GPU memory is prohibitively expensive, Uber implemented advanced CPU offloading—keeping active computations on the GPU while shifting optimizer states to CPU RAM or NVMe SSDs. This effectively doubled training throughput and allowed them to train models that previously wouldn’t fit in VRAM.
  • Software/Hardware Co-design: By utilizing optimized frameworks like TensorRT-LLM tuned specifically for their H100 instances, Uber achieved a 2x improvement in response latency and a 6x boost in throughput.

The Shift to Agentic AI

Most recently, Uber has expanded beyond simple GenAI content generation into Agentic AI—systems capable of autonomous task decomposition, multi-agent collaboration, and real-time adaptability. By combining generative capabilities with their massive data annotation and testing engines (like uLabel and uTest), Uber is building systems where GenAI provides creative options, and Agentic logic evaluates, selects, and executes them reliably.

5. The Architecture Blueprint

Today, Uber’s ML platform can be distilled into four highly decoupled layers:

  1. Hardware Layer (Layer 0): A hybrid mix of on-premise A100 clusters and cloud-based H100 instances, connected via 100GB/s high-bandwidth networking.
  2. Orchestration Layer (Layer 1): Kubernetes handles the primitive scheduling and hardware constraints, while Ray (via the KubeRay operator) distributes the actual mathematical workloads across the worker nodes.
  3. Federation Layer (Layer 2): A global control plane that treats dozens of individual Kubernetes clusters as a single, unified compute mesh, dynamically routing workloads to eliminate idle GPU time.
  4. Developer Experience (Layer 3): Python-native workflows (Uniflow) and centralized Feature Stores that allow data scientists to focus entirely on modeling rather than infrastructure plumbing.

6. The Lesson for the Enterprise

Uber’s architectural journey validates a crucial reality for modern platform engineering: AI scale exposes design flaws. An architecture that works for 1,000 predictions an hour will spectacularly collapse at 30 million predictions a second.

The primary takeaway from Uber’s Michelangelo evolution is that successful, scalable AI is not fundamentally about having the smartest neural network. It is about robust data plumbing and distributed state management. By treating machine learning not as a special, fragile science project, but as standard, declarative, Kubernetes-native infrastructure, Uber has built the blueprint for the next decade of enterprise AI.

References & Further Reading

This article draws from sessions and discussions at KubeCon + CloudNativeCon EU 2026, including Agentics Day, Open Source SecurityCon, and contributions from the CNCF TAG Security community.

By Soumia, a developer advocate focused on making complex infrastructure legible — through writing, speaking, and helping technical and non-technical audiences find common ground. I work at the intersection of cloud-native systems, AI, and editorial craft. — LinkedIn · Portfolio

The Hive Mind: Scaling Multi-Agent AI State with AWS Lambda and Amazon EFS

yazar
Image dfdf

If you are building a multi-agent AI system on AWS, you will quickly hit a massive, hidden architectural wall: State Transfer.

In a multi-agent framework, AI agents are constantly reading, writing, and debating over a shared context. Agent A (The Researcher) reads 50 pages of documentation. Agent B (The Coder) writes a massive script based on that research. Agent C (The Critic) reviews it.

The payload passing between these agents is enormous.

If you try to build this using standard serverless patterns, you immediately hit physical constraints:

  • AWS Step Functions has a strict 256KB payload limit.
  • Amazon DynamoDB has a strict 400KB item size limit (and gets expensive if you continuously overwrite massive text blocks).
  • Amazon S3 has no size limits, but it is an atomic object store. You cannot stream or append data to an existing S3 object. You have to wait for Agent A to completely finish generating its 10,000-token output, save the entire file to S3, and only then can Agent B download it to start working.

This atomic wait-time creates a massive latency bottleneck.

To build a true, real-time “Hive Mind” for your AI agents, you need to abandon standard databases and object stores. You need to give your serverless functions a shared, POSIX-compliant file system.

Here is how to architect a real-time, shared memory bus for multi-agent systems using AWS Lambda and Amazon EFS (Elastic File System).

The Pivot: Serverless Shared Memory

Amazon EFS is a fully managed, elastic NFS file system. While it is often used for legacy EC2 migrations, AWS added the ability to mount EFS directly to Lambda functions.

When you mount an EFS drive (e.g., to /mnt/hivemind) across a fleet of 100 concurrent Lambda functions, it acts as a shared, low-latency network drive.

Because EFS is POSIX-compliant, it supports byte-level appending and file locking.

This completely changes how LLMs communicate. Agent A can use the LLM streaming API to stream generated tokens directly into a text file on the EFS drive. Because it is a standard file system, Agent B can literally open that same file from a completely different Lambda instance and start reading the “thoughts” of Agent A as they are being written, milliseconds later.

The Architecture: The EFS Hive Mind

Image dfdf

How the Execution Flow Works

Let’s look at how two agents interact synchronously without ever touching a database or S3.

Image dddd

The CTO Perspective: Why This Pattern Wins

When engineering leaders see this architecture, the reaction is usually one of disbelief: “We can give our serverless AI agents a shared, real-time POSIX file system so they can read each other’s ‘thoughts’ synchronously without any database overhead?”

Yes. Here is why this tradeoff is incredibly powerful for AI workloads:

1. Bypassing Payload Limits

You no longer care about the 256KB Step Functions limit or the 400KB DynamoDB limit. Your Step Function only passes the file path (e.g., {"context_path": "/mnt/hivemind/task_99.txt"}). The actual context whether it’s 50 kilobytes or 50 gigabytes of source code lives on the mounted drive.

2. Microsecond File Access vs. Network API Calls

Downloading a 50MB context file from S3 at the start of a Lambda execution requires an HTTPS API call, TCP handshake, and data transfer time. With EFS, the file is already mounted to the local directory. Reading it uses standard Python open() or Node fs.readFile() commands. The OS handles the caching, resulting in single-digit millisecond latency.

3. The Economics of EFS

DynamoDB charges for Write Capacity Units (WCUs). If you are streaming AI tokens and saving state to DynamoDB every second, your WCU costs will explode.
Amazon EFS Standard storage costs $0.30 per GB-month. Using EFS Elastic Throughput, you pay roughly $0.03 per GB of data transferred. Because AI text generation is large in token count but tiny in actual megabytes, using EFS as a transient scratchpad is remarkably cheap.

Engineering Reality Check: Tradeoffs & Constraints

This is a highly advanced architectural pattern. If you deploy it, you must design around these AWS realities:

1. The VPC Requirement

To mount Amazon EFS, your AWS Lambda functions must be connected to a VPC (Virtual Private Cloud). Historically, putting Lambda in a VPC caused massive cold starts. Thankfully, AWS solved this years ago with Hyperplane ENIs. The cold start penalty for a VPC Lambda is now negligible, but you will still need to manage subnets, security groups, and NAT Gateways if your agents need internet access to reach external APIs.

2. Zombie Data Cost

EFS is persistent storage. If your AI agents generate 10GB of temporary scratchpad files a day and you never delete them, you will pay for that storage forever.
The Fix: You must implement a lifecycle policy or a nightly cron job (EventBridge + Lambda) that runs rm -rf /mnt/hivemind/tmp/* for any files older than 24 hours.

3. Concurrency and File Locking

While POSIX allows concurrent reads, concurrent writes to the exact same file from different Lambdas can result in interleaved, corrupted text. If Agent A and Agent B are writing to the Hive Mind simultaneously, they must write to isolated files (e.g., agent_a_out.txt and agent_b_out.txt), or you must implement strict fcntl file locking in your code.

The Bottom Line

As we push Multi-Agent AI systems into production, we are rediscovering old computer science problems. Moving massive amounts of state between distributed compute nodes is hard.

Databases and object stores are the wrong tools for real-time, streaming AI context. By attaching Amazon EFS to AWS Lambda, you combine the infinite horizontal scaling of serverless compute with the raw, byte-level speed of a shared POSIX file system.

Give your AI swarm a true Hive Mind.

How are you managing shared context and state transfer in your multi-agent AI systems? Have you hit the DynamoDB/Step Function size limits yet? Let’s discuss in the comments!

Building a Cozy Ambient Study App with React at 14

yazar

I’ve been building a cozy ambience app with React, and what started as a small experiment has slowly evolved into something much larger than I originally planned. The original idea was simple: create a space where people could mix ambient sounds like rain, wind, thunder, coffee shop noise, and fire crackling to help them focus, relax, study, or simply exist comfortably in the background of their day. Most productivity apps feel overly rigid or overwhelming, so I wanted this project to feel softer — more atmospheric than task-oriented — almost like opening a digital room instead of a tool.

Right now the project is still in its early stages, probably around 5–10% complete, but the core experience is already starting to take shape. The main system revolves around a real-time ambient sound engine where every sound runs independently with its own volume controls, allowing users to build layered soundscapes dynamically. Instead of forcing users into predefined experiences, the goal is to let them shape their own environment depending on their mood.

On top of that, I implemented a preset system that allows users to instantly switch between different ambience setups without rebuilding mixes every session. So instead of manually adjusting rain, thunder, wind, and background sounds every time, users can move between environments like rainy library nights, cozy coffee shops, deep focus spaces, or sleep atmospheres with a single click. Over time, I realized that these transitions mattered just as much as the sounds themselves because smoothness and flow heavily affect how immersive an app feels.

One of my favorite systems so far is the live background engine. Instead of static wallpapers, the app uses animated and dynamic backgrounds that actively reinforce the mood of each ambience setup. Combined with layered audio, subtle UI motion, and soft transitions, the app starts feeling less like a normal productivity tool and more like an interactive environment you can stay inside for hours.

To make the experience feel even more cohesive, I also integrated adaptive accent generation using node-vibrant. The app analyzes live backgrounds and extracts dominant colors to generate dynamic accent tones automatically. This means the interface subtly changes depending on the current environment, helping the UI feel visually connected to the ambience itself rather than relying on fixed themes. Small details like this ended up becoming surprisingly important because atmosphere relies heavily on consistency between visuals, motion, audio, and interaction.

The UI design itself is intentionally minimal and slightly glassy. I wanted it to stay soft and immersive without becoming distracting or visually heavy. The layout is currently split into a main ambience area and a side panel where presets, controls, and future systems live. One thing I learned very quickly while building this project is that creating a “calm” interface is actually much harder than making something flashy. Every small detail matters — spacing, animations, transitions, blur intensity, responsiveness, and even how quickly controls react to input.

Recently, I also started experimenting with weather integration so the ambience can subtly react to real-world conditions. The goal is to make the experience feel more connected to the user’s actual environment instead of existing as something completely isolated. Another feature that unexpectedly became part of the project was a lightweight chat system. It wasn’t originally planned, but I liked the idea of quiet interaction existing naturally inside the space without turning the app into a loud social platform. The intention is not to build another messaging app, but to create a small sense of shared presence while still preserving the calm atmosphere.

On the technical side, most of the audio handling is currently client-side using React state and refs, with each sound operating as its own controllable instance. I also started moving toward proper backend querying and scalable data handling instead of relying heavily on frontend filtering as the project grows. Earlier versions used simpler approaches, but the more features I added, the more I realized how quickly “vibe-based” apps become technically complicated under the hood.

One of the biggest lessons from this project has been learning not to overengineer too early. At the start, I tried planning every feature and system upfront, but the app evolved much more naturally once I focused on building step by step and letting the experience shape itself over time. A lot of the best ideas — including live environments, adaptive theming, and social elements — were never part of the original concept.

I’m 14 years old, and this has easily been one of the biggest and most educational projects I’ve worked on so far. Beyond improving my React skills, this project taught me a huge amount about product design, atmosphere-driven UX, scalability, audio systems, frontend architecture, and how emotion can influence the way people interact with software.

The long-term goal is simple: create a digital space people can open at any time and instantly feel calmer, more focused, and more comfortable in.

Building FateFolio: An AI Toolkit for Structured Eastern Metaphysics Reports

yazar

I recently built FateFolio (https://fatefolio.com), an AI-powered toolkit for structured Eastern metaphysics reports.

The idea is simple: take traditional systems like Bazi, I Ching, Feng Shui, palm reading, face reading, and auspicious date selection, then turn them into clear, reviewable, exportable reports instead of vague one-off answers.

I did not want FateFolio to feel like a mysterious black box. I wanted it to feel more like a structured analysis dashboard.

Why I built it

A lot of AI products in this space have the same problem.

You type a question into a chatbot, get a fluent answer, and then you are left wondering:

  • What input did the answer rely on?
  • Which part is traditional calculation and which part is interpretation?
  • Can I save the result?
  • Can I compare it with another report later?
  • How confident should I be about the output?

That was the core product problem I wanted to solve.

FateFolio treats these traditional systems as frameworks for self-reflection and decision support. It is not designed to replace professional advice or guarantee outcomes. The goal is to make the output easier to understand, easier to revisit, and easier to use as a personal reflection tool.

What FateFolio does

FateFolio currently includes several modules.

Bazi

The Bazi module creates a Four Pillars report from birth date, optional birth time, gender, and timezone.

It focuses on things like:

  • Four Pillars chart
  • Heavenly Stems and Earthly Branches
  • Five Elements balance
  • Ten Gods
  • Life themes
  • 12-month rhythm
  • Input uncertainty when birth time is missing

This was one of the most interesting modules to design because part of the result is deterministic calculation, while another part is explanation and interpretation.

So the product has to separate the two clearly.

I Ching

The I Ching module lets users ask a focused question, choose a casting method, and receive a structured reading.

The product supports multiple casting methods, including coin casting, time casting, character casting, Plum Blossom, yarrow stalks, direction casting, and sound casting.

The report is not meant to give a single absolute answer. It is meant to help users reflect on momentum, risks, timing, and possible next steps.

A good I Ching experience depends heavily on question quality, so the interface encourages open-ended questions instead of binary prediction questions.

For example, instead of asking:

Will I succeed?

A better question would be:

What should I consider before moving forward with this opportunity?

That small UX detail changes the quality of the whole experience.

Feng Shui

The Feng Shui module works with room photos or floor plans.

Users can upload an image, select the scene type, choose goals like sleep quality, career, relationships, focus, or energy flow, and receive practical suggestions.

The report focuses on visible layout issues first:

  • Door and window positions
  • Bed or desk placement
  • Lighting
  • Flow
  • Furniture relationships
  • Yin-Yang and Five Elements balance

I wanted this module to feel useful even for people who are not deeply familiar with Feng Shui. So the output tries to explain the reasoning behind each suggestion instead of only saying “move this object here.”

Face Reading and Palm Reading

The face reading and palm reading modules are image-based.

For face reading, the product looks at observable facial features and turns them into self-observation notes. For palm reading, it detects palm lines, hand shape, and visual patterns, then generates a structured report based on traditional palmistry concepts.

These modules required extra care around privacy and boundaries.

Images are processed in memory and are not stored by default. The product also does not perform facial recognition, identity matching, or medical diagnosis.

That boundary is important.

The purpose is self-reflection, not labeling people.

Auspicious Date Selection

The date selection module helps users compare possible dates for events such as weddings, moving, business openings, travel, and other life events.

Instead of giving only one “best” date, it returns multiple date options with reasoning, cautions, and planning notes.

This is another place where product design matters.

A date may be symbolically favorable, but real-world constraints still matter: venues, people, travel, cost, preparation, and timing. The report makes that explicit.

Why structured output matters

The most important design decision in FateFolio is that the product is not just a chat interface.

Every module is designed around a report structure:

  • Summary
  • Details
  • Recommendations
  • Confidence
  • Boundaries
  • Exportable data

This makes the output easier to review later.

It also makes the product easier to improve because each module can follow a stable schema. The user is not just receiving a paragraph of text. They are receiving a report with sections, priorities, and uncertainty notes.

For deterministic inputs, FateFolio also uses input hashing to make outputs more stable and reproducible where possible.

That does not mean every AI-generated sentence will always be identical. But the underlying calculation and report structure can stay consistent.

The product principle: traditional input, modern constraints

One challenge with building this type of product is tone.

If the tone is too mystical, it can feel vague.
If the tone is too technical, it loses the cultural context.
If the product overclaims, it becomes irresponsible.
If the product underexplains, it becomes useless.

So the product principle became:

Traditional frameworks, modern constraints.

That means:

  • Explain uncertainty clearly
  • Avoid fear-based language
  • Avoid absolute claims
  • Keep professional boundaries
  • Give practical suggestions
  • Make reports saveable and exportable
  • Make privacy part of the product, not an afterthought

FateFolio is for entertainment, self-reflection, and decision support. It is not medical, legal, financial, or professional advice.

Privacy was part of the product from the beginning

For image-based modules, privacy is especially important.

FateFolio processes uploaded images in memory and does not store them by default. Local history is stored in the browser unless the user chooses to export it.

That design choice affects the entire product experience.

It means users can try a reading without needing an account. It also means the product does not have to collect more personal data than necessary.

For a product that deals with birth data, personal questions, face photos, palm photos, and home layouts, this is not a small detail.

It is central to trust.

What I learned while building it

The biggest lesson from building FateFolio is that AI products need more than a prompt.

A prompt can generate text.
A product needs structure.

For this kind of AI application, the hard parts are:

  • Input design
  • Output schema
  • Safety boundaries
  • User expectations
  • Privacy flow
  • Error handling
  • Explanation quality
  • Repeatability
  • Cultural sensitivity

The AI model is only one part of the system.

The real product is the workflow around it.

What is next

There are still many things I want to improve.

Some directions I am exploring:

  • Better report comparison
  • More transparent confidence scoring
  • More multilingual support
  • Better export formats
  • More educational glossary content
  • Follow-up questions after a report
  • A cleaner history dashboard

The long-term goal is to make FateFolio feel less like a novelty tool and more like a structured personal insight workspace.

A place where users can ask a question, generate a report, review the reasoning, understand the limits, and come back later with more context.

Final thoughts

FateFolio is my attempt to combine traditional Eastern metaphysical systems with modern AI product design.

Not as a prediction engine.
Not as a replacement for real-world judgment.
Not as professional advice.

But as a structured, privacy-conscious, AI-assisted reflection tool.

For me, the interesting part is not just what the AI says.

It is how the product frames the question, structures the answer, explains uncertainty, and helps the user think more clearly.

Validate JWTs from Multiple Issuers in kgateway

yazar

Production APIs often need to accept tokens from more than one identity provider for example, a tenant’s own Auth0 tenant and Google Workspace for internal tools. kgateway’s JWTPolicy resource lets you declare multiple issuers in one policy and attach it to any HTTPRoute, so you don’t need a separate gateway per IdP.

This guide walks through a working, reproducible configuration. By the end you will have a policy that validates tokens from two issuers, rejects mismatched audiences, and forwards selected claims as upstream headers.

What is a JWT?

A JSON Web Token (JWT) is a compact, self-contained credential that an identity provider (IdP) issues to a user or service after they authenticate. Instead of your API checking a username and password on every request, the client attaches a JWT and your API trusts it because it was cryptographically signed by someone it already trusts.

Think of it like a signed event wristband. The venue (IdP) checks your ID once at the gate and gives you a wristband. Staff inside the venue (your APIs) can verify the wristband is genuine without phoning the front gate again. The wristband also says which areas you can access and it expires at midnight.

Structure of a JWT

A JWT is three Base64URL-encoded JSON objects joined by dots: 

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9   ← Header
.
eyJzdWIiOiJ1c2VyXzEyMyIsImVtYWlsIjoiYWxpY2VAZXhhbXBsZS5jb20i...  ← Payload
.
SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c  ← Signature
Part What it contains
Header Algorithm (RS256) and token type. Tells the verifier how to check the signature.
Payload Claims about the user and the token who issued it, who it’s for, when it expires.
Signature Cryptographic proof the token hasn’t been tampered with. Verified against the IdP’s public key.

You can paste any JWT into jwt.io to decode it instantly.

What’s inside the payload?

The payload is a JSON object of claims statements about the token and its subject. Some are standard; some are custom fields added by your IdP. 

{
  "iss": "https://my-tenant.auth0.com/",  // issuer — who created the token
  "sub": "user_123",                       // subject — the user's unique ID
  "aud": "my-api",                         // audience — which service this token is for
  "exp": 1717000000,                       // expiry — Unix timestamp
  "email": "[email protected]",            // custom claim added by Auth0
  "roles": ["admin", "editor"]             // custom claim for RBAC
}

How signature verification works (JWKS)

JWTs signed with RS256 use asymmetric cryptography: the IdP signs tokens with a private key that only it holds, and publishes the corresponding public keys at a well known URL called the JWKS endpoint (JSON Web Key Set). Anyone including kgateway can fetch these public keys and verify that a token was genuinely issued by that IdP and hasn’t been altered since.

This means kgateway never needs to call back to your IdP on every request. It fetches the JWKS once, caches the keys, and verifies signatures locally at the data plane making validation fast and offline capable.

Why this matters for multi-issuer setups: Each IdP has its own JWKS endpoint and its own signing keys. kgateway can hold keys from multiple providers simultaneously, matching each incoming token to the right key by checking its iss claim first.

What you’ll build

An HTTPRoute on /api that:

  • Accepts RS256-signed JWTs from Auth0 and Google
  • Enforces aud: my-api on tokens from both providers
  • Forwards the sub and email claims as X-User-Id and X-User-Email headers to your upstream service

Before you begin

  • kgateway ≥ 1.2 installed in your cluster
  • kubectl access with permissions to create custom resources
  • An Auth0 tenant with an API audience configured
  • A Google OAuth 2.0 client ID

How kgateway validates JWTs

Validation happens in the Envoy data plane before a request ever reaches your upstream. On each request, kgateway:

  1. Extracts the bearer token from the Authorization: Bearer <token> header (configurable to cookies or query params).
  2. Resolves the matching issuer by comparing the token’s iss claim against each issuer declared in JWTPolicy.spec.providers. The first match wins.
  3. Fetches and caches JWKS from the provider’s jwks_uri. Keys are cached per the cacheDuration you set and never re-fetched mid-request.
  4. Validates claims and signature verifies exp, nbf, aud, and the cryptographic signature. Any failure returns 401 Unauthorized immediately.
  5. Forwards claims as headers injects declared claims into request headers so your upstream can make authorization decisions without reparsing the JWT.

Step 1: Create the JWTPolicy

The JWTPolicy is a namespace scoped custom resource that declares which issuers to trust, where to fetch their public keys, and which claims to forward upstream. Create a file named jwt-policy.yaml: 

apiVersion: gateway.kgateway.dev/v1alpha1
kind: JWTPolicy
metadata:
  name: multi-issuer-policy
  namespace: default
spec:
  providers:

    # Provider 1: Auth0 tenant
    - name: auth0
      issuer: https://my-tenant.auth0.com/     # note the trailing slash
      audiences:
        - my-api
      remoteJwks:
        uri: https://my-tenant.auth0.com/.well-known/jwks.json
        cacheDuration: 10m
      claimsToHeaders:
        - claim: sub
          header: X-User-Id
        - claim: email
          header: X-User-Email

    # Provider 2: Google
    - name: google
      issuer: https://accounts.google.com      # no trailing slash
      audiences:
        - my-api
      remoteJwks:
        uri: https://www.googleapis.com/oauth2/v3/certs
        cacheDuration: 5m
      claimsToHeaders:
        - claim: sub
          header: X-User-Id
        - claim: email
          header: X-User-Email

⚠️ Issuer strings must be exact. The issuer field is compared character-for-character against the token’s iss claim. Auth0 includes a trailing slash in its tokens; Google does not. A mismatch here means every token from that provider will be rejected, even if the signature is valid.

Step 2: Attach the policy to your HTTPRoute

Reference the policy via an annotation on your HTTPRoute. You do not need to modify the route’s rules: 

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: api-route
  namespace: default
  annotations:
    gateway.kgateway.dev/jwt-policy: multi-issuer-policy
spec:
  parentRefs:
    - name: my-gateway
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /api
      backendRefs:
        - name: my-service
          port: 8080

Step 3: Apply and verify

# Apply both resources
$ kubectl apply -f jwt-policy.yaml -f httproute.yaml

# Confirm the policy is accepted by the control plane
$ kubectl get jwtpolicy multi-issuer-policy 
    -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'
True

# Test with a valid Auth0 token
$ curl -H "Authorization: Bearer $AUTH0_TOKEN" https://my-gateway/api/health
200 OK

# Test rejection: no token → 401
$ curl https://my-gateway/api/health
401 Unauthorized

# Confirm upstream receives the forwarded headers
$ kubectl logs deploy/my-service | grep X-User-Id
X-User-Id: user_123

⚠️ JWKS caching on first request: kgateway fetches JWKS the first time a token from a given issuer arrives. If the jwks_uri is unreachable at that moment, the request fails with 503. Use a cacheDuration of at least 5m in production never 0s outside of development.

Claim validation reference

Claim Validated automatically Notes
iss Yes Must exactly match a declared provider’s issuer. First match wins; no fallback.
aud Yes, if configured Token must contain at least one value from the audiences list. Omit the field to skip audience validation (not recommended in production).
exp Yes Expired tokens are rejected with 401. Clock skew tolerance is 60 s by default.
nbf Yes Tokens with a future nbf (not-before) are rejected.
sub, email, roles No kgateway does not validate custom claims. Use claimsToHeaders to forward them and enforce access rules in your upstream service.

Next steps

  • Use a local JWKS secret: Mount JWKS as a Kubernetes secret for air gapped or high security environments.
  • Claim based routing: Route requests to different backends based on forwarded claim headers.
  • Full OIDC with Auth0: Add the authorization code flow for browser facing applications.
  • Monitor validation errors: Surface JWT rejection rates in Prometheus and set alerting thresholds.

Samsung ile sendika arasında müzakereler yeniden başlayacak

yazar
Güney Kore merkezli Samsung Electronics ile sendika, ikramiye artışı talebiyle planlanan grev öncesinde hükümet arabuluculuğunda yeniden müzakere masasına oturma kararı aldı. Olası grevin ulusal ekonomiye 66,9 milyar dolar zarar verebileceği belirtiliyor.

Bursa'da bir ilk, Togg T10F taksi oldu

yazar
Bursa’nın İznik ilçesinde genç girişimci Arda Barkın Sümer, dedesinin hediye ettiği Togg T10F modelini taksiye çevirdi. Elektrikli araç sayesinde yakıt ve bakım maliyetlerinden tasarruf eden Sümer, vatandaşların sırf fotoğraf çektirmek için aracı çağırdığını belirtti. Yerli ve milli araç, ilçede kısa sürede ilgi odağı oldu.