Zitadel Has a Free API: Self-Hosted Identity and Auth That Replaces Auth0

What is Zitadel?

Zitadel is an open-source identity management platform — a self-hosted alternative to Auth0, Okta, and Firebase Auth. It provides authentication, authorization, and user management with a comprehensive API.

Free for unlimited users when self-hosted.

Quick Start

docker run --name zitadel -p 8080:8080 
  ghcr.io/zitadel/zitadel:latest start-from-init 
  --masterkey "MasterkeyNeedsToHave32Characters" 
  --tlsMode disabled

The Management API

export ZITADEL_URL="https://your-zitadel.com"
export TOKEN="your-pat-token"

User Management

# Create user
curl -X POST "$ZITADEL_URL/management/v1/users/human" 
  -H "Authorization: Bearer $TOKEN" 
  -H "Content-Type: application/json" 
  -d '{
    "userName": "[email protected]",
    "profile": {
      "firstName": "John",
      "lastName": "Doe",
      "displayName": "John Doe"
    },
    "email": {
      "email": "[email protected]",
      "isEmailVerified": true
    },
    "password": "SecureP@ss123!"
  }'

# List users
curl -X POST "$ZITADEL_URL/management/v1/users/_search" 
  -H "Authorization: Bearer $TOKEN" 
  -d '{"queries": [], "limit": 100}'

# Get user by ID
curl "$ZITADEL_URL/management/v1/users/USER_ID" 
  -H "Authorization: Bearer $TOKEN"

Projects and Applications

# Create project
curl -X POST "$ZITADEL_URL/management/v1/projects" 
  -H "Authorization: Bearer $TOKEN" 
  -d '{"name": "My Web App"}'

# Create OIDC application
curl -X POST "$ZITADEL_URL/management/v1/projects/PROJECT_ID/apps/oidc" 
  -H "Authorization: Bearer $TOKEN" 
  -d '{
    "name": "Frontend",
    "redirectUris": ["http://localhost:3000/callback"],
    "responseTypes": ["OIDC_RESPONSE_TYPE_CODE"],
    "grantTypes": ["OIDC_GRANT_TYPE_AUTHORIZATION_CODE"],
    "appType": "OIDC_APP_TYPE_WEB",
    "authMethodType": "OIDC_AUTH_METHOD_TYPE_NONE"
  }'

Roles and Authorization

# Add role to project
curl -X POST "$ZITADEL_URL/management/v1/projects/PROJECT_ID/roles" 
  -H "Authorization: Bearer $TOKEN" 
  -d '{"roleKey": "admin", "displayName": "Administrator"}'

# Grant role to user
curl -X POST "$ZITADEL_URL/management/v1/users/USER_ID/grants" 
  -H "Authorization: Bearer $TOKEN" 
  -d '{"projectId": "PROJECT_ID", "roleKeys": ["admin"]}'

OIDC/OAuth2 Integration

// Next.js with next-auth
import NextAuth from "next-auth";

export default NextAuth({
  providers: [{
    id: "zitadel",
    name: "Zitadel",
    type: "oidc",
    issuer: process.env.ZITADEL_URL,
    clientId: process.env.ZITADEL_CLIENT_ID,
    clientSecret: process.env.ZITADEL_CLIENT_SECRET,
  }],
});

Features

  • Multi-tenancy: Manage multiple organizations
  • OIDC/OAuth2/SAML: Industry-standard protocols
  • MFA: TOTP, WebAuthn, SMS
  • Social Login: Google, GitHub, Apple, etc.
  • Branding: Custom login pages per organization
  • Audit Logs: Full event sourcing

Zitadel vs Auth0

Feature Zitadel (self-hosted) Auth0
Price Free (unlimited) Free to 7,500 MAU
Users Unlimited 7,500 then $23/mo
SSO/SAML Included Enterprise plan
Multi-tenancy Built-in Organizations add-on
Self-hosted Yes No

Need auth infrastructure or identity management setup?

📧 [email protected]
🔧 My tools on Apify Store

Auth0, Clerk, or self-hosted? What’s your auth stack?

The $4.87 Spec: How Local Session Storage Cuts AI Costs by 89%

A simple file-based memory system for AI coding sessions turned a $45 multi-session rebuild into a single $4.87 conversation. Here’s the architecture, the data, and why context management is the most undervalued problem in AI-assisted development.

The Problem Nobody Talks About

Every AI coding assistant has the same dirty secret: context evaporates.

You spend 3 hours in a session with your AI pair programmer. You explore APIs, validate assumptions, make design decisions, discover edge cases. Then the session ends. Tomorrow, you start from zero.

The next session costs just as much — not because the work is hard, but because the AI has to rediscover everything it already knew.

Tracked across 53 sessions over 6 weeks on a platform engineering project. The waste was staggering.

What Was Measured

The setup: an AI coding assistant used for infrastructure automation — managing on-call schedules, incident response tooling, and platform operations across multiple cloud environments. The work is context-heavy: API integrations, team structures, design decisions, and operational processes that span weeks of iterative design.

One project required 9 sessions over 2 weeks to produce a specification document. Here’s what actually happened:

Session Focus Key Outputs
1–2 API discovery, shift structure mapping 10 engineer IDs resolved, 9 shift UUIDs mapped
3 Architecture pattern discovery Fundamental design change (member-swap → scheduled absence)
4 22 use cases gathered from stakeholder Design decisions D-1 through D-9
5 5 integration POCs executed 3 passed, 2 blocked (enterprise auth)
6 Auth blocker solved Novel zero-auth approach discovered
7 20-point design review with stakeholder Corrections on every major section
8 6th POC + gap analysis 17 missing items identified, spec outline approved
9 Spec writing 792-line spec + 285-line execution plan

Session 9 — the one that produced the actual deliverable — consumed all the knowledge from sessions 1–8. Without persistent storage, session 9 would need to:

  1. Re-discover API endpoints, UUIDs, and shift structures (sessions 1–2)
  2. Re-learn the scheduled absence pattern (session 3)
  3. Re-gather 22 use cases (session 4)
  4. Re-run or re-verify 6 POCs (sessions 5–6)
  5. Re-apply 20 points of stakeholder feedback (session 7)
  6. Re-do the gap analysis (session 8)

Conservative estimate: 6–8 sessions at $5–6 each = $35–45 just to rebuild context before writing a single line.

Actual cost of session 9 with local storage: $4.87.

The Architecture: Three Files

The system is embarrassingly simple. Three markdown files per project, stored locally (never in the AI provider’s cloud, never in the remote repository):

.local/agent/
├── current.md                    # Session state (what's active, what's next)
├── praveen-style.md              # Operating manual (style, decisions, anti-patterns)
└── projects/
    └── <project-name>/
        ├── log.md                # Chronological session history
        ├── reference.md          # Verified facts, API endpoints, IDs
        └── open-questions.md     # Decision tracker

current.md — The Recovery Point

This is the only file the AI reads at session start. It contains:

  • Active projects with one-line status and file pointers
  • TODO list with priorities and owners
  • Resume instructions — what was done last session, what’s next
  • File index — when to load each file (on-demand, not preloaded)

562 lines. Updated at the end of every session. If a session crashes, this file is the recovery point.

log.md — The Session History

Chronological log of every session: what was done, what was decided, what was discovered. Each entry has:

  • Context (why this session happened)
  • Decisions made (with rationale)
  • Key findings (especially surprises)
  • Open items carried forward

For the project that produced the spec, this file grew to 1,040 lines across 9 sessions. It’s the primary source material — the AI reads it when it needs to understand why a decision was made, not just what was decided.

reference.md — The Verified Facts

API endpoints, authentication patterns, ID mappings, integration test results — anything that was verified against a live system. This file exists because LLMs hallucinate, and the most dangerous hallucinations are the ones that look like API documentation.

Every entry in this file was confirmed by an actual API call or system query. When the AI reads this file, it’s reading facts, not assumptions.

396 lines for the project in question. Includes: 10 verified API endpoints, 10 engineer identity mappings, 4 placeholder account UUIDs, 9 shift structures, 6 POC results with evidence, and a complete decision register.

The Rules That Make It Work

The files alone aren’t enough. Five rules — discovered through painful trial and error — prevent the system from degrading:

Rule 1: Read current.md First, Everything Else On-Demand

The AI reads current.md at session start. That’s it. Every other file is loaded only when the current task requires it. This prevents context window pollution — loading 11,000+ lines of project knowledge into a conversation about a single API endpoint.

Rule 2: Separate State from History from Facts

  • current.md = what’s happening now (mutable, updated every session)
  • log.md = what happened (append-only, never edited retroactively)
  • reference.md = what’s true (verified facts, updated only when facts change)

This separation means the AI loads only the type of knowledge it needs. Writing a spec? Load log.md for design history. Making an API call? Load reference.md for endpoints. Starting a new session? Just current.md.

Rule 3: Update Before Session End

The AI updates current.md resume instructions before every session close. This is non-negotiable. If the session crashes after the update, the next session can recover. If it crashes before, one session of context is lost — not everything.

Rule 4: Archive When Files Get Large

When log.md exceeds ~500 lines, older sessions are archived to archive/. The active file stays manageable. The archive is there if deep historical context is needed (rare — maybe 5% of sessions).

Rule 5: Never Store in Provider Cloud

All files live in .local/ (gitignored). They never go to the AI provider’s servers, never go to the remote repository. This is about control: the user owns the context, decides what persists, and can move between AI providers without losing institutional knowledge.

The Numbers

Single Session ROI

Metric Without Storage With Storage
Context rebuild 6–8 sessions ($35–45) 0 sessions ($0)
Spec writing 1 session ($5–6) 1 session ($4.87)
Total $40–51 $4.87
Saving 86–89%

Cumulative Impact (53 Sessions, 6 Weeks)

Metric Value
Active projects 10
Total project knowledge 11,812 lines across 36 files
Semantic memory chunks 961 (indexed for search)
Files indexed 51
Estimated sessions saved 40–60 (context rebuilds avoided)

What the Storage Contains

Category Lines Examples
Session histories ~6,200 Design decisions, POC results, stakeholder feedback
Reference data ~2,100 API endpoints, verified IDs, integration patterns
Operating manuals ~760 Style guides, decision-making patterns, anti-patterns
Session state ~560 Active projects, TODOs, resume instructions
Decision trackers ~190 Open questions with status and resolution
Total ~11,812

The Semantic Memory Layer

On top of the three-file system, a lightweight semantic search layer handles cross-project recall:

# 196 lines of Python
# sentence-transformers (all-MiniLM-L6-v2, 384-dim embeddings)
# numpy .npz + chunks.json storage
# Index time: ~12s for 961 chunks
# Search time: <3s
# Cost: $0 (local model, no API calls)

This handles the “I know this was solved in a different project” problem. The AI searches across all project files semantically before starting work that might duplicate past effort.

961 chunks indexed from 51 files. The index is 2.5MB total. Re-indexed after every session save.

What Doesn’t Work

Stuffing Everything Into the System Prompt

The obvious first attempt: load all project files at session start. The problems:

  1. Context window waste — 11,000 lines is ~40,000 tokens. That’s a significant chunk of the context window consumed before the conversation even starts.
  2. Attention dilution — LLMs pay less attention to content in the middle of long contexts. Critical facts buried in page 15 of 20 get missed.
  3. Cost — Every message in the conversation includes the full system prompt. Token costs scale linearly.

On-demand loading (Rule 1) solved all three.

Relying on the AI’s “Memory” Features

Some AI providers offer built-in memory or “project knowledge” features. Tried these too. The problems:

  1. Opacity — You can’t see exactly what was stored or how it’s retrieved.
  2. Vendor lock-in — Switch providers, lose everything.
  3. Granularity — Built-in memory stores summaries. Real work needs verbatim API endpoints, exact UUIDs, precise decision rationale. Summaries lose the details that matter.
  4. No version control — Local files are in a git-ignored directory, but they could be version-controlled. Built-in memory can’t be diffed, branched, or rolled back.

RAG Over Everything

Full RAG (vector database, chunking pipeline, retrieval-augmented generation) is overkill for this use case. The semantic search layer here is 196 lines of Python with a local embedding model. It indexes in 12 seconds and searches in 3. No database server, no embedding API costs, no infrastructure.

The three-file system handles 95% of cases. Semantic search handles the remaining 5% (cross-project recall). A full RAG stack would add complexity without proportional benefit.

The Compression Effect

The most interesting outcome isn’t cost savings — it’s knowledge compression.

Session 9 consumed 2,221 lines of pre-existing context (across 5 files) and produced 1,077 lines of structured output. That’s a 2.2:1 compression ratio. But the real compression happened across all 9 sessions:

Input Lines
8 sessions of iterative design ~4,000 (including dead ends)
API documentation and POC logs ~1,500
Stakeholder feedback (20+ points) ~800
Total raw input ~6,300
Output Lines
Spec (25 sections) 792
Execution plan (9 tasks) 285
Total structured output 1,077

5.8:1 compression from scattered session notes to structured specification. The local storage system made this possible because:

  1. Nothing was lost between sessions (no re-discovery)
  2. Dead ends were recorded once and never repeated (session 5 documented 10 failed auth approaches — session 9 didn’t retry any of them)
  3. Decisions were recorded with rationale (session 9 didn’t re-debate closed questions)

Why This Matters for the Industry

The current AI coding assistant landscape is focused on:

  • Model intelligence — bigger models, better reasoning
  • Tool use — code execution, file editing, web search
  • Context windows — 128K, 200K, 1M tokens

Nobody is seriously working on session-to-session knowledge persistence as a first-class feature. The assumption seems to be that bigger context windows solve the problem. They don’t.

A 1M token context window means you can load 11,000 lines of project knowledge. It doesn’t mean you should. Attention mechanisms degrade with context length. Cost scales linearly. And the fundamental problem remains: who decides which knowledge to load, when?

The three-file system described here is a manual solution to what should be an automated one. The rules discovered empirically — separate state from history from facts, load on-demand, update before session end, archive when large — should be built into every AI coding tool.

What a Real Solution Looks Like

  1. Automatic session persistence — Every session’s decisions, discoveries, and dead ends are captured without manual effort.
  2. Typed knowledge stores — Separate “what’s true” (facts) from “what happened” (history) from “what’s next” (state). Different retrieval strategies for each.
  3. On-demand retrieval — Load context based on the current task, not the current project. If I’m writing an API integration, load verified API endpoints. If I’m writing a spec, load design decisions.
  4. Cross-session deduplication — If a question was asked and answered in session 3, don’t let session 7 ask it again.
  5. Provider-agnostic storage — The knowledge belongs to the user, not the AI provider. Portable, version-controllable, inspectable.

The team that builds this well — not as a feature bolted onto a chat interface, but as a core architectural primitive — wins the AI coding assistant market. Because the cost of intelligence is dropping (model prices halve every 6 months). The cost of context is the durable competitive advantage.

Try It Yourself

You don’t need any special tooling. Create three files:

# In your project root (gitignored)
.local/
├── current.md      # "Read this at session start"
├── log.md          # Append after every session
└── reference.md    # Verified facts only

Start every AI session with: “Read .local/current.md first.”

End every session with: “Update .local/current.md with resume instructions.”

After 5 sessions, measure how often you’re re-explaining context. After 10 sessions, calculate the cost of sessions with vs without the files.

The ROI will speak for itself.

Data from 53 sessions across 6 weeks on a platform engineering project. Total local storage: 11,812 lines across 36 files. Measured cost savings: 86–89% per context-heavy session. The entire memory system is 196 lines of Python and three markdown files.

Dokku Has a Free API: Build Your Own Heroku on a $5 VPS

What is Dokku?

Dokku is the smallest PaaS implementation you’ve ever seen. It’s a self-hosted Heroku alternative that runs on a single server. Push code with git push — Dokku builds, deploys, and manages your apps.

100% free. 100% open source.

Install

wget -NP . https://dokku.com/bootstrap.sh
sudo DOKKU_TAG=v0.34.8 bash bootstrap.sh

Deploy Your First App

# On your local machine:
git remote add dokku dokku@your-server:my-app
git push dokku main

That’s it. Dokku detects your language, builds the app, and deploys it.

The CLI API

App Management

# Create app
dokku apps:create my-app

# List apps
dokku apps:list

# Destroy app
dokku apps:destroy my-app

# View logs
dokku logs my-app -t  # Follow logs

# Run one-off command
dokku run my-app python manage.py migrate
dokku run my-app rails console

Domains & SSL

# Add domain
dokku domains:add my-app myapp.com
dokku domains:add my-app www.myapp.com

# Auto SSL with Let's Encrypt
dokku letsencrypt:enable my-app

# Auto-renewal
dokku letsencrypt:cron-job --add

Environment Variables

# Set env vars
dokku config:set my-app DATABASE_URL=postgres://... SECRET_KEY=abc123

# View env vars
dokku config:show my-app

# Unset
dokku config:unset my-app DEBUG

Databases (Plugins)

# Install PostgreSQL plugin
sudo dokku plugin:install https://github.com/dokku/dokku-postgres.git postgres

# Create database
dokku postgres:create my-db

# Link to app (sets DATABASE_URL automatically)
dokku postgres:link my-db my-app

# Backup
dokku postgres:export my-db > backup.sql

# Redis
sudo dokku plugin:install https://github.com/dokku/dokku-redis.git redis
dokku redis:create my-cache
dokku redis:link my-cache my-app

Scaling

# Scale processes
dokku ps:scale my-app web=2 worker=1

# Set resource limits
dokku resource:limit my-app --memory 512m --cpu 1

Docker Options

# Mount volumes
dokku storage:mount my-app /var/lib/dokku/data/storage:/app/uploads

# Set Docker options
dokku docker-options:add my-app deploy "--restart=always"

Network & Ports

# Set port mapping
dokku ports:add my-app http:80:3000
dokku ports:add my-app https:443:3000

# View ports
dokku ports:list my-app

Buildpacks vs Dockerfile

# Use buildpacks (auto-detect)
# Just push code — Dokku detects Node.js, Python, Ruby, etc.

# Use Dockerfile
# Just include a Dockerfile in your repo — Dokku uses it automatically

# Use Docker image
dokku git:from-image my-app myorg/my-image:latest

Zero-Downtime Deploys

# Enable zero-downtime checks
dokku checks:enable my-app

# Custom health check
# In your app root, create CHECKS:
# /health  200  OK

Popular Plugins

Plugin Purpose
postgres PostgreSQL databases
redis Redis instances
letsencrypt Auto SSL certificates
mongo MongoDB databases
mysql MySQL databases
rabbitmq Message queues
elasticsearch Search engine

Dokku vs Alternatives

Feature Dokku Coolify CapRover
Interface CLI Web UI Web UI
Deploy git push Git/Docker Git/Docker
Footprint ~256MB ~1GB ~512MB
Plugins 100+ Built-in Limited
Maturity 10+ years 3 years 5 years

Need a lightweight deployment solution or server automation?

📧 [email protected]
🔧 My tools on Apify Store

git push to deploy — is Dokku the best PaaS for indie hackers? Let me know!

Woodpecker CI Has a Free API: Self-Hosted CI/CD That’s Actually Simple

What is Woodpecker CI?

Woodpecker CI is a community fork of Drone CI — a lightweight, container-based CI/CD system. It’s simple, fast, and self-hosted. Every pipeline step runs in a Docker container.

No YAML complexity of GitHub Actions. No vendor lock-in.

Setup with Docker Compose

# docker-compose.yml
version: '3'
services:
  woodpecker-server:
    image: woodpeckerci/woodpecker-server:latest
    ports:
      - 8000:8000
    volumes:
      - woodpecker-server-data:/var/lib/woodpecker/
    environment:
      WOODPECKER_OPEN: true
      WOODPECKER_HOST: https://ci.myserver.com
      WOODPECKER_GITHUB: true
      WOODPECKER_GITHUB_CLIENT: ${GITHUB_CLIENT}
      WOODPECKER_GITHUB_SECRET: ${GITHUB_SECRET}
      WOODPECKER_ADMIN: myuser

  woodpecker-agent:
    image: woodpeckerci/woodpecker-agent:latest
    depends_on: [woodpecker-server]
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      WOODPECKER_SERVER: woodpecker-server:9000
      WOODPECKER_AGENT_SECRET: ${AGENT_SECRET}

volumes:
  woodpecker-server-data:
docker compose up -d

Pipeline Configuration

# .woodpecker.yml
steps:
  - name: install
    image: node:20
    commands:
      - npm ci

  - name: lint
    image: node:20
    commands:
      - npm run lint

  - name: test
    image: node:20
    commands:
      - npm test

  - name: build
    image: node:20
    commands:
      - npm run build
    when:
      branch: main

  - name: deploy
    image: plugins/docker
    settings:
      repo: myorg/my-app
      tags: latest
      username:
        from_secret: docker_username
      password:
        from_secret: docker_password
    when:
      branch: main
      event: push

The REST API

export WP_URL="https://ci.myserver.com/api"
export WP_TOKEN="your-token"

# List repos
curl -s "$WP_URL/user/repos" 
  -H "Authorization: Bearer $WP_TOKEN" | jq '.[].full_name'

# Get pipeline status
curl -s "$WP_URL/repos/myorg/my-app/pipelines/latest" 
  -H "Authorization: Bearer $WP_TOKEN" | jq '.status'

# Trigger a build
curl -X POST "$WP_URL/repos/myorg/my-app/pipelines" 
  -H "Authorization: Bearer $WP_TOKEN" 
  -H "Content-Type: application/json" 
  -d '{"branch": "main"}'

# List secrets
curl -s "$WP_URL/repos/myorg/my-app/secrets" 
  -H "Authorization: Bearer $WP_TOKEN" | jq '.[].name'

# Add secret
curl -X POST "$WP_URL/repos/myorg/my-app/secrets" 
  -H "Authorization: Bearer $WP_TOKEN" 
  -H "Content-Type: application/json" 
  -d '{"name": "deploy_key", "value": "secret-value"}'

Services (Databases in CI)

steps:
  - name: test
    image: node:20
    commands:
      - npm test

services:
  - name: postgres
    image: postgres:16
    environment:
      POSTGRES_DB: test
      POSTGRES_PASSWORD: test

  - name: redis
    image: redis:7

Services run alongside your pipeline — real databases for integration tests.

Matrix Builds

matrix:
  NODE_VERSION:
    - 18
    - 20
    - 22

steps:
  - name: test
    image: node:${NODE_VERSION}
    commands:
      - npm ci
      - npm test

Woodpecker vs Others

Feature Woodpecker GitHub Actions Jenkins
Self-hosted Yes No (runners yes) Yes
Config YAML (simple) YAML (complex) Groovy
Container-native Yes Partial Plugin
Resource usage ~50MB RAM N/A 1GB+ RAM
UI Clean, modern GitHub-integrated Dated

Need CI/CD setup or DevOps automation?

📧 [email protected]
🔧 My tools on Apify Store

What CI/CD do you self-host? Comment below!

Token Approval Hygiene in 2026: Why Your Old approve(MAX_UINT256) Is a Ticking Time Bomb

Token Approval Hygiene in 2026: Why Your Old approve(MAX_UINT256) Is a Ticking Time Bomb

Over $50 million was drained from DeFi users in the first three months of 2026 alone — not through flashy flash loans or oracle manipulation, but through something far more mundane: stale token approvals.

The SwapNet exploit ($13.5M), Aperture Finance hack ($3.67M), and dozens of smaller incidents share a common thread. Attackers didn’t need to break cryptography or find zero-days. They exploited permissions users had already granted — sometimes months or years earlier.

This article is a practitioner’s guide to token approval security: what goes wrong, why the standard patterns are dangerous, and how to build approval-safe protocols and habits in 2026.

The Anatomy of an Approval Attack

Every ERC-20 approve() call creates a standing permission: “Contract X can move up to Y of my tokens.” When you approve type(uint256).max, you’re saying “Contract X can take everything, forever.”

Here’s the attack flow that drained SwapNet users:

1. User swaps on Matcha Meta → routed through SwapNet
2. SwapNet's router requests unlimited approval for USDC
3. User approves (standard UX, most users click through)
4. Months pass. User forgets about the approval.
5. Attacker finds arbitrary-call vulnerability in SwapNet's 0x87395540()
6. Attacker calls transferFrom(victim, attacker, victim_balance)
7. Approved tokens vanish. No phishing needed. No signature required.

The critical insight: the vulnerability was in the protocol, but the damage was enabled by the approval. If users had granted one-time or bounded approvals, the attack surface would have been near zero.

Why MAX_UINT256 Approvals Persist

Despite years of warnings, unlimited approvals remain the default across DeFi. Here’s why:

1. Gas Optimization Theater

// The "gas-saving" argument
// "Why pay for approve() every transaction when you can approve once?"
token.approve(router, type(uint256).max); // One tx, ~46k gas
// vs.
token.approve(router, exactAmount);       // Every swap, ~46k gas each

On Ethereum mainnet at 30 gwei, each approval costs ~$2-4. For frequent traders, unlimited approval saves maybe $50/year. But it creates an indefinite liability against their entire token balance.

2. UX Friction Aversion

Wallets and dApps optimize for conversion. Every additional transaction is a dropout point. So the industry standardized on “approve once, use forever” — prioritizing convenience over security.

3. The “Audited Protocol” Fallacy

Users assume audited protocols are safe forever. But:

  • Audits are point-in-time snapshots
  • Upgradeable proxies can change logic post-audit
  • Closed-source contracts (like SwapNet) can’t even be publicly audited
  • Team key compromises can alter contract behavior

The 2026 Approval Security Stack

Layer 1: Protocol-Level — Permit2 and Bounded Approvals

Uniswap’s Permit2 contract fundamentally changes the approval model:

// Old pattern: approve each protocol individually
USDC.approve(uniswapRouter, MAX);
USDC.approve(sushiRouter, MAX);
USDC.approve(oneInchRouter, MAX);
// 3 unlimited approvals = 3 attack surfaces

// Permit2 pattern: one approval to Permit2, then signed permits
USDC.approve(permit2Address, MAX); // Only 1 approval needed
// Each swap uses a signed, time-bounded permit
permit2.permitTransferFrom(
    ISignatureTransfer.PermitTransferFrom({
        permitted: ISignatureTransfer.TokenPermissions({
            token: address(USDC),
            amount: swapAmount   // Exact amount, not MAX
        }),
        nonce: uniqueNonce,
        deadline: block.timestamp + 30 minutes  // Expires quickly
    }),
    transferDetails,
    msg.sender,
    signature
);

Why this matters: Even if a protocol using Permit2 gets exploited, the attacker can only use permits that haven’t expired. No more infinite standing approvals to drain.

Layer 2: Contract Design — The Approval-Minimizing Pattern

If you’re building a protocol, design for minimal approval surface:

// BAD: Require users to pre-approve your contract
function swap(address token, uint256 amount) external {
    IERC20(token).transferFrom(msg.sender, address(this), amount);
    // ... swap logic
}

// GOOD: Pull exact amounts via Permit2 with deadline
function swapWithPermit(
    address token,
    uint256 amount,
    uint256 deadline,
    bytes calldata signature
) external {
    require(block.timestamp <= deadline, "Permit expired");
    permit2.permitTransferFrom(
        /* signed permit for exact amount with deadline */,
        ISignatureTransfer.SignatureTransferDetails({
            to: address(this),
            requestedAmount: amount
        }),
        msg.sender,
        signature
    );
    // ... swap logic
}

For Solana programs, the equivalent is using delegated token accounts with explicit limits:

// Solana: Approve with explicit amount and optional delegate authority
use anchor_lang::prelude::*;
use anchor_spl::token::{self, Approve};

pub fn approve_bounded(ctx: Context<ApproveCtx>, amount: u64) -> Result<()> {
    // Only approve the exact amount needed for this operation
    token::approve(
        CpiContext::new(
            ctx.accounts.token_program.to_account_info(),
            Approve {
                to: ctx.accounts.token_account.to_account_info(),
                delegate: ctx.accounts.delegate.to_account_info(),
                authority: ctx.accounts.owner.to_account_info(),
            },
        ),
        amount, // NOT u64::MAX
    )?;
    Ok(())
}

// After operation completes, revoke the delegation
pub fn revoke_after_use(ctx: Context<RevokeCtx>) -> Result<()> {
    token::revoke(
        CpiContext::new(
            ctx.accounts.token_program.to_account_info(),
            token::Revoke {
                source: ctx.accounts.token_account.to_account_info(),
                authority: ctx.accounts.owner.to_account_info(),
            },
        ),
    )?;
    Ok(())
}

Layer 3: User Habits — The Approval Hygiene Checklist

For individual users, implement these practices:

Monthly Approval Audit:

  1. Visit revoke.cash or app.unrekt.net
  2. Connect each wallet you use
  3. Sort approvals by “value at risk” (unlimited approvals × token balance)
  4. Revoke any approval where:
    • You haven’t used the protocol in 30+ days
    • The approved contract is unverified/closed-source
    • The approval is unlimited and the protocol supports bounded approvals

Per-Transaction Discipline:

  • Enable “One-Time Approvals” on aggregators that support it (1inch, Matcha)
  • Use wallet extensions like Pocket Universe or Fire that simulate and warn about approval transactions
  • When approving, manually edit the amount in MetaMask (click the pencil icon on the approval screen)

Emergency Response:
If a protocol you’ve approved gets hacked:

1. DON'T PANIC — but move fast
2. Go to revoke.cash immediately
3. Revoke the affected contract's approvals
4. Check if the exploiter has already used your approval
5. If funds are still there, move them to a fresh wallet

The Approval Attack Surface: By the Numbers (Q1 2026)

Incident Loss Root Cause Approval Factor
SwapNet/Matcha $13.5M Arbitrary call in closed-source contract Unlimited USDC approvals
Aperture Finance $3.67M Arbitrary call, no input validation Unlimited approvals for “Instant Liquidity”
Various phishing ~$30M+ Wallet drainer scripts, permit signatures Users signing malicious permit2 messages
Venus Protocol $3.7M Oracle manipulation + donation attack N/A (lending protocol, different vector)

Notice the pattern: the largest approval-based losses come from closed-source contracts. Open-source contracts get audited by the community; closed-source ones are black boxes where you’re trusting the team’s competence entirely.

Solana-Specific Approval Risks

Solana’s token model is different from ERC-20, but the approval risks are analogous:

Token Delegate Authority

SPL tokens use delegate instead of approve. A delegated account can transfer up to the approved amount. The risk: some Solana dApps request delegation of your full token balance.

Token-2022 Transfer Hooks

The newer Token-2022 standard introduces transfer hooks — arbitrary code that executes on every transfer. A malicious or compromised transfer hook could:

  • Silently redirect tokens during legitimate transfers
  • Log all transfer destinations for phishing targeting
  • Implement time-delayed drains that activate weeks after approval

CPI-Based Approval Chains

When a Solana program makes a Cross-Program Invocation (CPI) to the token program with your signing authority, it can approve delegates on your behalf. Always verify that programs you interact with don’t grant unexpected delegations in their CPI chains.

Building an Approval-Safe Protocol: Checklist for Developers

□ Use Permit2 or equivalent bounded-approval mechanism
□ Never request unlimited approvals in your frontend
□ If using upgradeable proxies, implement approval revocation in upgrade hooks
□ Time-bound all delegated permissions (24h max for routine operations)
□ Emit clear events on approval changes for monitoring tools
□ Open-source your token-handling contracts (even if core logic is proprietary)
□ Document your approval model in user-facing documentation
□ Implement a one-click "revoke all" function in your dApp
□ Add approval amount to transaction simulation previews
□ Conduct approval-focused audit (separate from general security audit)

The Path Forward: ERC-7715 and Native Approval Expiry

The Ethereum community is working on ERC-7715 (Grant Permissions), which would introduce native expiry for token approvals at the protocol level. Instead of approvals persisting forever until explicitly revoked, they would automatically expire after a specified duration.

Combined with account abstraction (ERC-4337) enabling batch revocations and session keys, the approval landscape is shifting. But these standards are months from widespread adoption. In the meantime, the security practices outlined above are your best defense.

Key Takeaways

  1. Unlimited approvals are technical debt — they accumulate silently and explode when a protocol gets compromised
  2. Closed-source contracts with approvals are the highest-risk combination — you can’t verify what your approval enables
  3. Permit2 is the current best practice — use it if you’re building, prefer protocols that use it if you’re trading
  4. Monthly approval audits are non-negotiable — 10 minutes on revoke.cash can save your portfolio
  5. The SwapNet exploit was not novel — approval-based drains have been happening since 2020. The only novel part was the scale.

The DeFi ecosystem has spent billions on smart contract auditing while leaving the approval model fundamentally broken. Until ERC-7715 and similar standards achieve adoption, your security is only as good as your approval hygiene.

Building something in DeFi? I audit smart contracts with a focus on approval surfaces and access control. Find me on dev.to or Hashnode.

CDKTF Has a Free API: Write Terraform Infrastructure with TypeScript Instead of HCL

What is CDKTF?

CDKTF (Cloud Development Kit for Terraform) lets you write Terraform infrastructure using TypeScript, Python, Go, C#, or Java instead of HCL. You get the full Terraform ecosystem — thousands of providers — with real programming language features.

Quick Start

npm install -g cdktf-cli
cdktf init --template=typescript --providers=aws
cdktf deploy

Define Infrastructure

import { Construct } from "constructs";
import { App, TerraformStack, TerraformOutput } from "cdktf";
import { AwsProvider } from "@cdktf/provider-aws/lib/provider";
import { S3Bucket } from "@cdktf/provider-aws/lib/s3-bucket";
import { Instance } from "@cdktf/provider-aws/lib/instance";
import { SecurityGroup } from "@cdktf/provider-aws/lib/security-group";

class MyStack extends TerraformStack {
  constructor(scope: Construct, id: string) {
    super(scope, id);

    new AwsProvider(this, "aws", { region: "us-east-1" });

    // S3 bucket
    const bucket = new S3Bucket(this, "my-bucket", {
      bucket: "my-app-assets-2026",
      tags: { Environment: "production" },
    });

    // Security group
    const sg = new SecurityGroup(this, "web-sg", {
      name: "web-server-sg",
      ingress: [
        { fromPort: 80, toPort: 80, protocol: "tcp", cidrBlocks: ["0.0.0.0/0"] },
        { fromPort: 443, toPort: 443, protocol: "tcp", cidrBlocks: ["0.0.0.0/0"] },
      ],
      egress: [
        { fromPort: 0, toPort: 0, protocol: "-1", cidrBlocks: ["0.0.0.0/0"] },
      ],
    });

    // EC2 instance
    const server = new Instance(this, "web-server", {
      ami: "ami-0c55b159cbfafe1f0",
      instanceType: "t3.micro",
      vpcSecurityGroupIds: [sg.id],
      tags: { Name: "Web Server" },
    });

    new TerraformOutput(this, "bucket-name", { value: bucket.bucket });
    new TerraformOutput(this, "server-ip", { value: server.publicIp });
  }
}

const app = new App();
new MyStack(app, "my-infrastructure");
app.synth();

Real Programming — Loops and Functions

// Create multiple environments with a loop
const environments = ["dev", "staging", "production"];

for (const env of environments) {
  new S3Bucket(this, `bucket-${env}`, {
    bucket: `myapp-${env}-assets`,
    tags: { Environment: env },
  });
}

// Reusable function
function createVpc(stack: TerraformStack, name: string, cidr: string) {
  return new Vpc(stack, name, {
    cidrBlock: cidr,
    enableDnsHostnames: true,
    tags: { Name: name },
  });
}

const devVpc = createVpc(this, "dev-vpc", "10.0.0.0/16");
const prodVpc = createVpc(this, "prod-vpc", "10.1.0.0/16");

Constructs (Reusable Components)

class WebService extends Construct {
  public readonly url: string;

  constructor(scope: Construct, id: string, props: {
    port: number;
    instanceType?: string;
    environment: string;
  }) {
    super(scope, id);

    const sg = new SecurityGroup(this, "sg", {
      ingress: [{
        fromPort: props.port,
        toPort: props.port,
        protocol: "tcp",
        cidrBlocks: ["0.0.0.0/0"],
      }],
    });

    const instance = new Instance(this, "server", {
      ami: "ami-0c55b159cbfafe1f0",
      instanceType: props.instanceType || "t3.micro",
      vpcSecurityGroupIds: [sg.id],
      tags: {
        Name: `${id}-server`,
        Environment: props.environment,
      },
    });

    this.url = instance.publicIp;
  }
}

// Use like a building block
new WebService(this, "api", { port: 3000, environment: "prod" });
new WebService(this, "admin", { port: 8080, environment: "prod", instanceType: "t3.small" });

Testing Infrastructure

import { Testing } from "cdktf";

describe("MyStack", () => {
  it("should create an S3 bucket", () => {
    const app = Testing.app();
    const stack = new MyStack(app, "test");
    const synth = Testing.synth(stack);

    expect(Testing.toHaveResource(synth, "aws_s3_bucket")).toBeTruthy();
  });

  it("should use t3.micro for instances", () => {
    const app = Testing.app();
    const stack = new MyStack(app, "test");
    const synth = Testing.synth(stack);

    expect(
      Testing.toHaveResourceWithProperties(synth, "aws_instance", {
        instance_type: "t3.micro",
      })
    ).toBeTruthy();
  });
});

Deploy

# Synthesize to Terraform JSON
cdktf synth

# Plan changes
cdktf diff

# Deploy
cdktf deploy

# Destroy
cdktf destroy

CDKTF vs Terraform HCL vs Pulumi

Feature CDKTF Terraform HCL Pulumi
Language TS/Python/Go HCL TS/Python/Go
Ecosystem Terraform providers Terraform providers Own providers
State Terraform state Terraform state Pulumi state
Testing Jest/pytest Terratest Jest/pytest
Maturity GA Mature GA

Need cloud infrastructure automation or IaC consulting?

📧 [email protected]
🔧 My tools on Apify Store

CDKTF, Pulumi, or plain Terraform? Let me know your pick!

Telnyx Python SDK Compromised on PyPI: Malware Delivered via Steganography in WAV Files, Mitigation Strategies Discussed

Incident Overview

The Telnyx Python SDK on the Python Package Index (PyPI) was compromised in a sophisticated supply chain attack executed by the threat actor TeamPCP. Malicious versions of the SDK (4.87.1 and 4.87.2) were uploaded to PyPI, engineered to execute arbitrary code immediately upon import, ensuring cross-platform compatibility. The attack’s technical sophistication is exemplified by its payload delivery mechanism: WAV steganography, which embeds malicious data within the frame metadata of ostensibly benign audio files.

Below is a detailed technical analysis of the attack:

  • Upon import, the compromised package fetches a .wav file from a command-and-control (C&C) server located at 83.142.209.203:8080. The WAV file maintains structural validity, concealing malicious data within its frame metadata.
  • The malware extracts the hidden data from the frame metadata, applies base64 decoding, and performs an XOR operation using the initial bytes as the encryption key. This process reconstructs the payload by reversing the deformation applied to the encoded data structure, transforming it into executable code.
  • The payload execution diverges based on the target operating system:
    • Windows: The payload deploys msbuild.exe into the Startup folder, ensuring persistence by leveraging the system’s boot sequence.
    • Linux/macOS: A staged Python loader fetches the WAV file, extracts the second-stage payload, and executes it via stdin. The payload then AES-encrypts sensitive data and exfiltrates it, causing anomalous outbound traffic that saturates network interfaces.

The C&C server employs endpoints such as /hangup.wav and /ringtone.wav, masquerading malicious files as benign audio to evade detection. Traditional security tools fail to identify the steganographic payload due to the WAV files’ structural validity, which circumvents anomaly detection mechanisms.

This attack exposes critical vulnerabilities in software supply chain security:

  • PyPI’s verification process: The absence of rigorous integrity checks enabled the malicious package to be published, undermining trust in open-source ecosystems.
  • Outbound traffic monitoring: Inadequate monitoring of network activity allowed exfiltration to proceed undetected, expanding the attack surface across compromised systems.
  • Steganography detection: Overreliance on traditional security tools, which lack metadata and frame data analysis capabilities, created a critical blind spot in threat detection.

For affected systems, immediate mitigation measures include downgrading to a verified safe version of the SDK, rotating compromised cryptographic secrets, and implementing continuous monitoring for outbound traffic to the identified C&C IP. This incident underscores the imperative for robust package verification protocols, advanced steganography detection capabilities, and proactive threat hunting within open-source repositories to fortify software supply chain resilience.

Technical Analysis: Deconstructing the Telnyx Python SDK Supply Chain Attack

The recent compromise of the Telnyx Python SDK on PyPI by TeamPCP exemplifies the escalating sophistication of supply chain attacks. This incident underscores the critical need for robust security frameworks within open-source ecosystems. The attack’s technical ingenuity lies in its exploitation of WAV steganography for payload delivery, a method that seamlessly embeds malicious code within legitimate audio files. This analysis dissects the attack’s mechanisms, its cross-platform persistence strategies, and the broader implications for software supply chain security.

Payload Delivery Mechanism: WAV Steganography Explained

The attack’s efficacy hinges on the covert embedding of malicious code within the frame metadata of WAV files. The causal mechanism unfolds as follows:

  • Stealth Mechanism: The WAV file retains its audio functionality, rendering it indistinguishable from benign files to both human inspection and conventional security tools.
  • Execution Process:

    • The compromised package retrieves a WAV file (e.g., /hangup.wav or /ringtone.wav) from the command-and-control (C2) server at 83.142.209.203:8080.
    • The frame metadata is parsed to extract hidden bytes encoded in base64.
    • The initial bytes serve as an XOR key, decrypting the embedded payload.
    • The decrypted data is reconstructed into executable code, which is dynamically executed upon import, bypassing static analysis tools.
  • Operational Impact: The malicious code executes seamlessly, leveraging the disparity between human auditory perception and machine data interpretation to evade detection.

This technique parallels the physical process of encoding information within the grooves of a vinyl record, where the primary function (audio playback) remains intact while secondary data (the payload) is concealed within the structure.

Cross-Platform Persistence and Exfiltration Strategies

The payload employs OS-specific tactics to ensure persistence and maximize exfiltration efficiency:

Operating System Attack Strategy
Windows * Persistence Mechanism: A malicious instance of msbuild.exe is placed in the Startup folder, ensuring execution on system boot. * Operational Impact: Unauthorized processes spawn at startup, consuming system resources and maintaining persistent access.
Linux/macOS * Exfiltration Mechanism: A staged Python loader retrieves the WAV file, extracts the payload, and executes it via stdin. Sensitive data is encrypted using AES and exfiltrated to the C2 server. * Operational Impact: Network interfaces are saturated with outbound traffic, degrading system performance and masking malicious activity.

The attack’s risk amplification stems from its dual-pronged approach: persistence ensures prolonged access, while exfiltration exploits network resources, creating a feedback loop that complicates detection and remediation.

C2 Server Obfuscation and Detection Challenges

The C2 server employs disguised endpoints (e.g., /hangup.wav, /ringtone.wav) to obfuscate malicious activity. The causal chain is as follows:

  • Obfuscation Mechanism: Security tools prioritize file type over metadata analysis, failing to flag WAV files as suspicious.
  • Detection Gap: Steganography detection requires deep packet inspection, a capability rarely implemented in standard security pipelines.
  • Operational Impact: Malicious traffic masquerades as legitimate audio downloads, evading traditional monitoring systems.

This tactic mirrors the Trojan horse strategy, where the WAV file serves as the innocuous exterior, concealing the malicious payload within.

Mitigation Strategies: Addressing Root Causes

The incident exposes systemic vulnerabilities within PyPI’s ecosystem, necessitating proactive, multi-layered defenses:

  • Package Integrity: Implement mandatory cryptographic signing and integrity checks to prevent unauthorized package uploads.
  • Traffic Monitoring: Deploy advanced tools capable of identifying anomalous WAV file downloads and detecting encrypted exfiltration patterns.
  • Steganography Detection: Integrate metadata analysis into security pipelines to uncover hidden payloads within benign file formats.

For compromised systems, immediate remediation includes downgrading to verified package versions, rotating compromised secrets, and monitoring for communication with the C2 server at 83.142.209.203:8080. The broader imperative is clear: supply chain security demands anticipatory, stratified defenses, not reactive patches.

Mitigation and Prevention: Fortifying the Software Supply Chain Against Advanced Adversaries

The compromise of the Telnyx Python SDK on PyPI exemplifies a paradigm shift in supply chain attacks, leveraging sophisticated techniques such as WAV steganography to bypass conventional defenses. This incident underscores the critical need for a multi-layered security approach that addresses both technical vulnerabilities and the evolving tactics of threat actors. By deconstructing the attack’s methodology, we can engineer resilient defenses at the intersection of code integrity, data inspection, and network behavior analysis.

1. Package Integrity: Cryptographic Signing as a Foundational Defense

The absence of mandatory code signing in PyPI’s upload process enabled the undetected publication of malicious versions 4.87.1 and 4.87.2. To mitigate this vulnerability:

  • Mechanical Process: Enforce cryptographic signing using asymmetric key pairs. Each package upload must include a detached signature, verified against the maintainer’s public key. This ensures end-to-end integrity, preventing tampering during transit or storage.
  • Causal Mechanism: Cryptographic signing shifts the trust model from implicit reliance on repository infrastructure to explicit proof of authorship. By requiring a valid signature, the attack surface is reduced, as unauthorized uploads become computationally infeasible without access to the private key.
  • Edge Case Mitigation: In the event of key compromise, implement multi-signature requirements for critical packages. This necessitates multiple valid signatures, exponentially increasing the complexity of unauthorized modifications.

2. Steganography Detection: Unmasking Covert Payloads in WAV Metadata

The malicious use of WAV steganography exploited the gap between human-perceptible audio and machine-interpretable metadata. To counter this technique:

  • Mechanical Process: Deploy deep packet inspection (DPI) tools capable of analyzing WAV frame metadata for anomalies, such as base64 encoding or XOR-based obfuscation patterns.
  • Causal Mechanism: Steganography leverages the perceptual gap between human auditory processing and machine data interpretation. DPI bridges this gap by extending inspection beyond the audio stream to metadata layers, where covert payloads reside.
  • Edge Case Mitigation: To address adaptive steganography, employ machine learning models trained on datasets of benign and malicious WAV metadata. These models detect subtle deviations indicative of embedded payloads, even as steganographic techniques evolve.

3. Network Traffic Monitoring: Detecting Exfiltration Through Behavioral Anomalies

83.142.209.203:8080 remained undetected due to the absence of anomaly detection mechanisms. To prevent such incidents:

  • Mechanical Process: Implement behavioral analytics solutions that establish baselines of normal network activity. Flag deviations such as sudden increases in WAV file transfers or AES-encrypted traffic.
  • Causal Mechanism: Data exfiltration generates anomalous network patterns, particularly on Linux/macOS systems where payloads saturate bandwidth. Behavioral analytics tools identify these patterns, triggering alerts before data egress occurs.
  • Edge Case Mitigation: If attackers employ domain fronting, block traffic to known command-and-control (C2) IPs and enforce DNS filtering to prevent resolution of suspicious domains.

4. Remediation: Neutralizing Compromised Systems

For systems compromised by versions 4.87.1 or 4.87.2, execute the following steps:

  • Code Rollback: Downgrade to a verified safe version. This interrupts the malicious execution chain by removing compromised code.
  • Secret Rotation: Replace all API keys and tokens associated with the compromised package. This severes the attacker’s access vector to sensitive systems.
  • Persistence Removal: On Windows, delete msbuild.exe from startup folders. On Linux/macOS, terminate rogue Python processes and remove staged payloads from temporary directories.

5. Proactive Threat Hunting: Anticipating Adversarial Tactics

Reactive defenses are insufficient against proactive adversaries. Adopt a threat-hunting posture:

  • Mechanical Process: Utilize YARA rules to scan repositories for WAV files with steganographic indicators. Continuously update rules to incorporate emerging techniques.
  • Causal Mechanism: Threat hunting transcends detection beyond signature-based approaches, identifying threats through behavioral and structural anomalies before execution.
  • Edge Case Mitigation: If attackers evade static analysis, employ sandboxed execution environments to observe runtime behavior without exposing production systems.

The Telnyx incident is not a failure of technology but a failure of anticipatory security design. By dissecting the physical and logical mechanisms of the attack—from cryptographic integrity violations to steganographic payload delivery—we can engineer defenses that are both reactive and proactive. The software supply chain’s resilience is determined by its weakest link. Through rigorous integrity checks, deep data inspection, and behavioral monitoring, every link can be fortified against advanced adversaries.

Kamal 2 Has a Free API: Deploy Web Apps to Any Server Without Kubernetes

What is Kamal?

Kamal (formerly MRSK) is a deployment tool from the creators of Rails (37signals/Basecamp). It deploys containerized web apps to any server — bare metal, VPS, or cloud — without Kubernetes complexity.

Kamal 2 powers HEY.com and Basecamp in production.

Quick Start

gem install kamal
kamal init
kamal setup
kamal deploy

Configuration

# config/deploy.yml
service: my-app
image: myorg/my-app

servers:
  web:
    hosts:
      - 192.168.1.1
      - 192.168.1.2
    options:
      memory: 512m
  worker:
    hosts:
      - 192.168.1.3
    cmd: bundle exec sidekiq

registry:
  username: myorg
  password:
    - KAMAL_REGISTRY_PASSWORD

env:
  clear:
    RAILS_ENV: production
    DATABASE_URL: postgres://db-host/myapp
  secret:
    - RAILS_MASTER_KEY
    - STRIPE_SECRET_KEY

proxy:
  ssl: true
  host: myapp.com
  app_port: 3000

accessories:
  db:
    image: postgres:16
    host: 192.168.1.4
    port: 5432
    env:
      clear:
        POSTGRES_DB: myapp_production
      secret:
        - POSTGRES_PASSWORD
    directories:
      - data:/var/lib/postgresql/data
  redis:
    image: redis:7
    host: 192.168.1.4
    port: 6379
    directories:
      - data:/data

Zero-Downtime Deploys

# Deploy with zero downtime
kamal deploy

# What happens:
# 1. Build Docker image locally
# 2. Push to registry
# 3. Pull on servers
# 4. Start new container
# 5. Health check passes
# 6. Switch traffic (kamal-proxy)
# 7. Stop old container

The built-in kamal-proxy handles traffic switching — no nginx config needed.

Key Commands

# First-time setup (installs Docker, kamal-proxy)
kamal setup

# Deploy latest code
kamal deploy

# Rollback to previous version
kamal rollback

# View logs
kamal app logs
kamal app logs -f  # Follow

# Run console
kamal app exec -i 'bin/rails console'

# Run one-off task
kamal app exec 'bin/rails db:migrate'

# Manage accessories
kamal accessory boot db
kamal accessory logs db

# Check server details
kamal details

Kamal 2 New Features

Built-in Proxy (replaces Traefik)

Kamal 2 ships with kamal-proxy — a purpose-built reverse proxy:

proxy:
  ssl: true
  host: myapp.com
  buffering:
    requests: true
    responses: true
    max_request_body: 10m
  response_timeout: 30
  healthcheck:
    interval: 3
    path: /up
    timeout: 3

Aliases

# config/deploy.yml
aliases:
  console: app exec -i 'bin/rails console'
  logs: app logs -f
  shell: app exec -i bash
kamal console  # Opens Rails console on production
kamal logs     # Follows production logs

Multi-App on Single Server

# app1/config/deploy.yml
proxy:
  host: app1.com

# app2/config/deploy.yml  
proxy:
  host: app2.com

Multiple apps share the same server, kamal-proxy routes by hostname.

Cost Comparison

Setup Monthly Cost
Heroku (2 dynos) $50-100
AWS ECS/Fargate $50-150
Kubernetes (EKS) $75-200+
Kamal + Hetzner VPS $5-20

Kamal + a $5 VPS = production-ready deployment.

Need help deploying apps or setting up CI/CD pipelines?

📧 [email protected]
🔧 My tools on Apify Store

Kamal, Docker Compose, or Kubernetes? Share your deployment stack!

How AI workloads changed the queue I was already building

I did not start glide-mq because of AI.

I started it because I needed a queue, wanted mechanics I liked better than what was out there, and wanted it built on top of Valkey Glide.

Up through v0.13, that is basically what it was: a feature-rich queue.

Then I kept building AI systems on top of it.

That is where the shape started changing.

The queue itself was usually fine. The pain was everything around it. Long-running jobs that were not actually stuck. Streaming that wanted to be part of the job instead of a side channel. Budget checks that needed to happen before the spend, not after. Token-aware rate limits. Pause/resume because real flows sometimes need a human or have to wait for CI.

Different project. Same pile of glue.

After enough rounds, the pattern stops looking normal. The queue is doing the easy part. Everything AI-specific is leaking out around it.

That is what pushed glide-mq in a different direction.

AI workloads expose the wrong assumptions in normal queue design

Most queues were built for ordinary background work.

AI workloads are not ordinary background work.

They stream. They run long without being stuck. They spend money while they execute. They wait on humans. They hit limits based on tokens, not job count. They can fail semantically while every operational metric stays green.

If the queue does not understand that shape, the missing behavior does not disappear. It ends up patched on from the outside.

That is the part that started feeling wrong to me.

Not because queues are bad. Because a lot of the assumptions underneath them were built for a different class of workload.

1. Locks are the wrong model for long AI jobs

Most queues assume the worker proves it is alive by renewing a lock. Miss the renewal, retry the job.

That is reasonable for short jobs.

It is a bad model for LLM workloads.

A long generation is not dead. A reasoning-heavy call sitting on a hard prompt for 90 seconds is not dead. But a normal queue cannot tell the difference, so it retries, and now you are paying twice for the same work.

Nothing crashed. Nothing exploded. You just had the wrong execution model.

The usual workaround is increasing a global lock timeout.

That is also wrong.

A tiny classifier and a two-minute generation should not share the same timeout assumptions just because they live in the same queue.

So in glide-mq, lock duration is per job:

await queue.add("classify", { text: "short" }, { lockDuration: 10_000 });
await queue.add("research", { topic: "complex" }, { lockDuration: 180_000 });

Same queue. Different expectations.

That should be normal.

2. Budget control belongs in the execution path

The ugly failures in agent systems are often not technical failures.

The requests succeed.
The responses are valid.
The logs look clean.
The dashboards stay green.

And meanwhile the system is stuck in some useless loop spending money.

That is why budget control does not belong only in dashboards, analytics, or some side service that tells you later what happened. By then the spend is already gone.

The queue is the choke point. Every step passes through it. That is where the budget check belongs.

So I put budgets on flows:

await flowProducer.add(
  {
    name: "pipeline",
    queueName: "ai",
    data: { topic: "research" },
    children: [
      { name: "search", queueName: "ai", data: {} },
      { name: "analyze", queueName: "ai", data: {} },
      { name: "draft", queueName: "ai", data: {} },
    ],
  },
  {
    budget: {
      maxTotalTokens: 50_000,
      maxTotalCost: 2.0,
      tokenWeights: { reasoning: 2.0 },
      onExceeded: "fail",
    },
  },
);

When a job reports usage, the budget check happens atomically in Valkey. If the flow is out of budget, the next step stops there.

Not later. Not after the invoice. At the point where it still matters.

3. Streaming should not be a second system

If the model is producing the result incrementally, that stream is part of the job.

Treating it as a separate system is already the smell.

But most queues only understand one shape: job starts, job finishes, here is the result.

LLMs do not behave like that.

So people bolt on pub/sub. Or WebSockets. Or SSE through some different route. Then reconnect logic. Then ordering. Then another pile of glue to keep the stream state and the job state from drifting apart.

Now the job lives in one place and the live output lives somewhere else.

That split is artificial.

So in glide-mq, the stream stays on the job:

const worker = new Worker(
  "ai",
  async (job) => {
    for await (const token of generateTokens(job.data.prompt)) {
      await job.streamChunk("token", token);
    }
    await job.streamChunk("done");
    return { completed: true };
  },
  { connection },
);

const chunks = await queue.readStream(jobId, { block: 5000 });

No extra pub/sub layer. No second system just to watch a job do its work. The stream is attached to the job, stored in Valkey, and resumable after disconnect.

That is the model that actually matches the workload.

The rest is the same mismatch in different clothes

Once you accept that AI workloads are a different class of work, the rest stops looking like extra features and starts looking like missing queue behavior.

Pause/resume for human approval.
Wake the flow back up when CI finishes.
Fallbacks across models.
Rate limiting based on tokens instead of job count.
Usage tracking that does not break when the next model adds a new token category.

These are not edge cases. They are part of the shape of the work.

That is what v0.14 is really about

Not AI branding. Not pretending glide-mq started as an AI queue from day one.

It did not.

It started as the queue I wanted to have.

Then building real AI systems on top of it kept exposing the same gaps, and eventually patching around them started feeling like the wrong move.

So v0.14 moves those behaviors into the queue

That is what changed.

glide-mq is still a queue. But v0.14 is where it started absorbing the behaviors that AI systems kept forcing into side systems.

  • Per-job lockDuration so long jobs stop fighting short ones.
  • job.reportUsage() so budgets and accounting live in the execution path.
  • job.streamChunk() so streaming stays attached to the job.
  • job.suspend() and queue.signal() for human-in-the-loop flows.
  • Ordered fallbacks.
  • Token-aware throttling.
  • Flow budgets that fail before the spend gets worse.

That is the direction now.

Not queue plus five things you will bolt on later anyway.

The queue should understand more of the workload it is running.

Final thought

I do not think this is only a glide-mq story.

I think AI workloads are exposing the wrong assumptions in a lot of older tooling.

The problem is not just that queues need a few more integrations. The problem is that many of the abstractions we still lean on were designed for ordinary jobs, requests, and background work. AI systems have a different shape, and when the abstraction does not match, the missing behavior leaks out into glue.

That is the part I stopped wanting to patch from the outside.

npm install glide-mq

GitHub | Examples | Docs

Pulumi Has a Free API: Infrastructure as Code with Real Programming Languages

What is Pulumi?

Pulumi lets you define cloud infrastructure using real programming languages — TypeScript, Python, Go, C#, Java — instead of YAML or HCL. You get loops, conditionals, functions, type checking, and IDE autocomplete for your infrastructure.

The open-source engine is completely free.

Quick Start

curl -fsSL https://get.pulumi.com | sh
pulumi new aws-typescript
pulumi up

Define Infrastructure in TypeScript

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// S3 bucket with website hosting
const bucket = new aws.s3.Bucket("my-site", {
  website: {
    indexDocument: "index.html",
    errorDocument: "error.html",
  },
});

// Upload files
new aws.s3.BucketObject("index", {
  bucket: bucket.id,
  key: "index.html",
  content: "<h1>Hello Pulumi!</h1>",
  contentType: "text/html",
});

// CloudFront CDN
const cdn = new aws.cloudfront.Distribution("cdn", {
  origins: [{
    domainName: bucket.websiteEndpoint,
    originId: bucket.arn,
    customOriginConfig: {
      httpPort: 80,
      httpsPort: 443,
      originProtocolPolicy: "http-only",
      originSslProtocols: ["TLSv1.2"],
    },
  }],
  enabled: true,
  defaultCacheBehavior: {
    targetOriginId: bucket.arn,
    viewerProtocolPolicy: "redirect-to-https",
    allowedMethods: ["GET", "HEAD"],
    cachedMethods: ["GET", "HEAD"],
    forwardedValues: { queryString: false, cookies: { forward: "none" } },
  },
  restrictions: { geoRestriction: { restrictionType: "none" } },
  viewerCertificate: { cloudfrontDefaultCertificate: true },
});

export const url = pulumi.interpolate`https://${cdn.domainName}`;

Automation API (Programmatic Control)

Pulumi’s Automation API lets you manage stacks from your own code:

import { LocalWorkspace } from "@pulumi/pulumi/automation";

async function deploy() {
  const stack = await LocalWorkspace.createOrSelectStack({
    stackName: "production",
    projectName: "my-infra",
    program: async () => {
      const bucket = new aws.s3.Bucket("my-bucket");
      return { bucketName: bucket.id };
    },
  });

  await stack.setConfig("aws:region", { value: "us-east-1" });

  const result = await stack.up({ onOutput: console.log });
  console.log(`Bucket: ${result.outputs.bucketName.value}`);
}

async function destroy() {
  const stack = await LocalWorkspace.selectStack({
    stackName: "production",
    projectName: "my-infra",
    program: async () => {},
  });
  await stack.destroy({ onOutput: console.log });
}

Build internal developer platforms, self-service portals, or CI/CD pipelines with full infrastructure control.

Component Resources (Reusable Modules)

class StaticSite extends pulumi.ComponentResource {
  public readonly url: pulumi.Output<string>;

  constructor(name: string, args: { path: string; domain?: string }, opts?: pulumi.ComponentResourceOptions) {
    super("custom:StaticSite", name, {}, opts);

    const bucket = new aws.s3.Bucket(`${name}-bucket`, {
      website: { indexDocument: "index.html" },
    }, { parent: this });

    // Upload all files from path
    // ... (sync logic)

    this.url = bucket.websiteEndpoint;
    this.registerOutputs({ url: this.url });
  }
}

// Use it like any other resource
const site = new StaticSite("docs", { path: "./dist" });
export const siteUrl = site.url;

Multi-Cloud

import * as gcp from "@pulumi/gcp";
import * as azure from "@pulumi/azure-native";

// Same language, any cloud
const gcpBucket = new gcp.storage.Bucket("gcp-bucket");
const azureBlob = new azure.storage.StorageAccount("azure-storage", {
  resourceGroupName: "my-rg",
  sku: { name: "Standard_LRS" },
  kind: "StorageV2",
});

Pulumi vs Terraform

Feature Pulumi Terraform
Language TypeScript/Python/Go HCL
IDE Support Full (autocomplete, types) Limited
Testing Standard test frameworks Terratest
Loops/Conditions Native language count/for_each
State Free (Pulumi Cloud) Terraform Cloud ($)
Reuse npm/pip packages Modules (HCL)

Need infrastructure automation or cloud architecture help?

📧 [email protected]
🔧 My tools on Apify Store

Pulumi or Terraform — which do you prefer? Comment below!