We Built an Open-Source CRM for AI Agents — Here’s Why

We Built an Open-Source CRM for AI Agents — Here’s Why

Traditional CRMs were designed for humans clicking buttons. With 6,400+ MCP servers and AI agents entering every enterprise workflow, it’s time for infrastructure that’s built for them.

If you’re building AI agents that interact with business data, you’ve probably hit the same wall we did.

Your agent can call APIs, generate text, and reason over documents — but where does it store what it knows? Where does it persist contacts, deals, interactions, and the evidence behind every claim? How does it handle conflicting information without silently overwriting the truth?

Most teams wire up a vector database and call it memory. That works for semantic search. It doesn’t work when you need to know the exact deal value, who the decision maker is, and which email confirmed it — with a full audit trail.

We needed something better. So we built it and open-sourced it.

Open Tooling CRM: The Technical Case

Open Tooling CRM is a headless, local-first CRM designed for AI agents as the primary users. No UI. No cloud dependencies. Just a SQLite-backed data layer with two clean interfaces:

  • REST API — 29 endpoints. Standard HTTP/JSON with consistent pagination, filtering, idempotency keys, and Zod validation.

  • MCP Server — 27 tools purpose-built for Claude Desktop, Claude Code, and any MCP-compatible client. Entities, relationships, artifacts, observations, briefs, conflicts, graph traversal, FTS, import/export.

The Evidence Chain

The core differentiator is the memory layer. Instead of flat fields that get overwritten, Open Tooling CRM implements a four-level evidence chain:

Artifacts (raw evidence: emails, transcripts, documents)

↓ extraction

Observations (typed claims with lifecycle: current → superseded / retracted)

↓ derivation

Briefs (summaries that cite observations — always regeneratable)

↓ detection

Conflicts (when observations disagree — never silently resolved)

Every observation links to its source artifact. Every brief cites its observations. Every conflict references the disagreeing claims. Progressive retrieval means agents start with the brief and drill down to raw evidence when needed.

This isn’t academic — it’s practical. When your agent says “budget is $250K,” you can trace that through the observation to the artifact (the email from the prospect’s CFO on March 3rd). When the budget changes, the old observation is superseded, not deleted. The audit trail is preserved.

Architecture

Agent Client → MCP Tools → OpenCRM Core → SQLite

Agent Client → REST API → OpenCRM Core → SQLite

Both interfaces share the same core. All writes produce immutable events in an append-only ledger. Data model: typed entities with JSON properties, directed relationships, field-level provenance, FTS index, and the full memory layer.

MCP Tools at a Glance

  • Entities: create_entity, update_entity, get_entity, search_entities, archive_entity

  • Relationships: link_entities, unlink_entities, list_relationships, traverse_graph

  • Memory: ingest_artifact, add_observation, list_observations, supersede_observation, retract_observation

  • Briefs: create_brief, get_brief, list_briefs

  • Conflicts: create_conflict, get_conflict, list_conflicts, resolve_conflict

  • Data: export_data, import_data

Why Headless?

No bundled UI is a deliberate architectural choice. CRMs that couple their data layer to a rigid frontend create lock-in by design. Open Tooling inverts this: the API is the product.

Three ways to use it:

  • Agent-only. Connect the MCP server to Claude Desktop. Your CRM interface is natural language.

  • Vibe-code a frontend. Tell Claude or Cursor: “Build me a React dashboard for my deals pipeline using the Open Tooling CRM API at localhost:8787.” The API is conventional enough that any code-gen tool can scaffold a working UI in minutes.

  • Integrate into your stack. It’s a REST API — connect it to anything.

Part of a Bigger Vision

Open Tooling CRM is the first module in Open Tooling — an open-source family of AI-native SaaS tools. The same architecture (headless, local-first, evidence-based, MCP-native) applies across verticals like HR, Finance, Project Management, Procurement, and beyond.

We also ship a Claude plugin marketplace (Attri-Inc/open-tooling-plugins). Install it and Claude gets the knowledge layer to use Open Tooling CRM optimally — the right ingestion patterns, the evidence chain, progressive retrieval, conflict resolution.

Quick Start

git clone https://github.com/Attri-Inc/open-tooling.git

cd open-tooling/crm

npm install && cp .env.example .env

npm run dev # REST API at localhost:8787

npm run seed # Sample data

npm run mcp # MCP server (stdio)

Use It with Claude (Zero Config)

MCP Server:

Add the MCP config to Claude Desktop or Claude Code. 27 tools, full evidence chain, natural language CRM.

Claude Cowork Plugin:

One-click install via our plugin marketplace. Gives Claude the knowledge layer for optimal usage — ingestion patterns, evidence chain workflow, progressive retrieval, conflict resolution.

Cowork setup: Customize → Add plugin → Add marketplace → Attri-Inc/open-tooling-plugins → Sync.

[GIF: Claude Cowork plugin install — showing the marketplace setup flow]

⭐ GITHUB: github.com/Attri-Inc/open-tooling

🧩 PLUGIN MARKETPLACE: github.com/Attri-Inc/open-tooling-plugins

Apache 2.0 licensed. Contributions welcome.

Originally published on Attri AI Substack [link].

Built by Attri AI. We build AI-native tools for enterprises.

Lucid Software advances MCP Server offering and Lucid AI capabilities, debuts Process Agent to boost speed and clarity across teams

SOUTH JORDAN, Utah  – Lucid Software, the leader in AI-driven work acceleration, today announced new AI-enhanced collaboration features that speed up how teams create complex diagrams and documentation. With advancements to its Model Context Protocol (MCP) server, enhancements to Lucid AI, and the introduction of its Process Agent, Lucid is helping organizations scale their AI … continue reading

The post Lucid Software advances MCP Server offering and Lucid AI capabilities, debuts Process Agent to boost speed and clarity across teams appeared first on SD Times.

Zero-Downtime Deployments in Node.js: Blue-Green, Rolling, and Canary Explained

Zero-Downtime Deployments in Node.js: Blue-Green, Rolling, and Canary Explained

Every production deployment is a bet. You’re betting that your changes are correct, your infrastructure will cooperate, and your users won’t notice anything happened. Most of the time, the bet pays off. But when it doesn’t — when a bad deploy takes down your Node.js service at 2 PM on a Tuesday — the cost is real: lost revenue, damaged trust, and a team scrambling to roll back.

Zero-downtime deployment removes the bet. Instead of replacing your running service with the new version and hoping for the best, you introduce the new version alongside the old one, validate it, and shift traffic gradually. If something breaks, you catch it early and reverse — before your users do.

This guide covers three production-ready strategies: blue-green, rolling, and canary deployments. We’ll look at when to use each, how to implement them for Node.js specifically, and what you need to have in place for any of them to work.

Why Node.js Deployments Fail (And When)

Before the strategies, let’s be clear about the actual failure modes:

Process crashes on startup. Your new version has a syntax error, a missing environment variable, or a database migration conflict. The old process is gone, the new one won’t start, your load balancer is routing to nothing.

Graceful shutdown not implemented. Your deployment kills the old process mid-request. Active HTTP connections get dropped. In-flight jobs are lost.

Health checks that lie. Your new version responds 200 OK on /health but has a broken connection to its database. Traffic is routed to it. Errors spike.

Memory leaks in new code. The new version looks fine for 10 minutes, then memory climbs. By the time you notice, every instance is affected.

Zero-downtime strategies solve the first two directly. The other two require proper health checks and monitoring — which you need regardless of deployment strategy.

Prerequisites: What Every Strategy Requires

You cannot do zero-downtime deployments without these:

1. A working health check endpoint

// app.js
app.get('/health', async (req, res) => {
  const checks = {
    status: 'ok',
    uptime: process.uptime(),
    timestamp: new Date().toISOString(),
    db: await checkDatabase(),
    memory: process.memoryUsage().heapUsed / 1024 / 1024
  };

  const isHealthy = checks.db === 'connected' && checks.memory < 400; // MB
  res.status(isHealthy ? 200 : 503).json(checks);
});

async function checkDatabase() {
  try {
    await pool.query('SELECT 1');
    return 'connected';
  } catch {
    return 'disconnected';
  }
}

2. Graceful shutdown

// Handle SIGTERM (sent by process managers and orchestrators)
process.on('SIGTERM', async () => {
  console.log('SIGTERM received — starting graceful shutdown');

  // Stop accepting new connections
  server.close(async () => {
    // Wait for in-flight requests to complete
    await drainConnections();

    // Close database connections
    await pool.end();

    console.log('Graceful shutdown complete');
    process.exit(0);
  });

  // Force kill after 30 seconds if something hangs
  setTimeout(() => {
    console.error('Graceful shutdown timed out — forcing exit');
    process.exit(1);
  }, 30000);
});

3. A load balancer or reverse proxy

NGINX, HAProxy, AWS ALB, Cloudflare, or your PaaS provider’s built-in routing — you need something sitting in front of your Node.js instances that can be told “route traffic here, not there.”

With these in place, let’s look at the strategies.

Strategy 1: Blue-Green Deployment

The concept: Maintain two identical production environments — “blue” (current live) and “green” (new version). Deploy to green, validate it fully, then switch all traffic from blue to green in a single step. Blue becomes your instant rollback.

Before deploy:    [Users] → [Load Balancer] → [Blue: v1.2.3]    [Green: idle]
Deploy to green:  [Users] → [Load Balancer] → [Blue: v1.2.3]    [Green: v1.3.0 ← deploying]
After validation: [Users] → [Load Balancer] → [Blue: v1.2.3]    [Green: v1.3.0 ✓]
Switch traffic:   [Users] → [Load Balancer] → [Blue: v1.2.3]    [Green: v1.3.0 → LIVE]

Implementing blue-green with PM2 + NGINX:

# ecosystem.config.js defines both environments
// ecosystem.config.js
module.exports = {
  apps: [
    {
      name: 'app-blue',
      script: './dist/server.js',
      instances: 2,
      env: { PORT: 3000, NODE_ENV: 'production' }
    },
    {
      name: 'app-green',
      script: './dist/server.js',
      instances: 2,
      env: { PORT: 3001, NODE_ENV: 'production' }
    }
  ]
};
# nginx.conf
upstream active_app {
  server localhost:3000;  # Blue: change to 3001 to switch to Green
}

server {
  listen 80;
  location / {
    proxy_pass http://active_app;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
  }

  location /health {
    proxy_pass http://active_app/health;
    access_log off;
  }
}

Deployment script:

#!/bin/bash
# deploy-blue-green.sh

NEW_VERSION=$1
CURRENT=$(cat /var/app/active-slot)  # "blue" or "green"
NEW_SLOT=$([ "$CURRENT" = "blue" ] && echo "green" || echo "blue")
NEW_PORT=$([ "$NEW_SLOT" = "blue" ] && echo "3000" || echo "3001")

echo "Current: $CURRENT | Deploying to: $NEW_SLOT (port $NEW_PORT)"

# 1. Pull new code and build
git pull origin main
npm ci --production
npm run build

# 2. Start the new slot
pm2 start ecosystem.config.js --only "app-$NEW_SLOT"

# 3. Wait for it to be healthy
echo "Waiting for $NEW_SLOT to become healthy..."
for i in {1..30}; do
  STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:$NEW_PORT/health)
  if [ "$STATUS" = "200" ]; then
    echo "Health check passed"
    break
  fi
  echo "Attempt $i: status $STATUS — waiting..."
  sleep 2
  if [ $i -eq 30 ]; then
    echo "Health check failed after 60s — aborting"
    pm2 stop "app-$NEW_SLOT"
    exit 1
  fi
done

# 4. Switch NGINX traffic to new slot
sed -i "s/server localhost:[0-9]*/server localhost:$NEW_PORT/" /etc/nginx/conf.d/app.conf
nginx -t && nginx -s reload

# 5. Record the switch and stop old slot
echo "$NEW_SLOT" > /var/app/active-slot
sleep 5
pm2 stop "app-$CURRENT"

echo "Deploy complete. Active: $NEW_SLOT ($NEW_VERSION)"

When to use blue-green:

  • You need instant, complete rollback capability
  • Your migrations are additive (blue and green can share a database)
  • You can afford to run double infrastructure (even briefly)
  • You’re on a PaaS that supports slot-based deployment (Railway, Render, Azure App Service)

When not to: If your database schema changes are destructive (column drops, renames), blue-green is tricky — old blue can’t run against green’s schema. Use expand-contract migrations.

Strategy 2: Rolling Deployment

The concept: Replace instances of the old version one by one (or in small batches), while keeping the remaining old instances live. At any point during the rollout, both versions are serving traffic simultaneously.

Start:   [v1][v1][v1][v1]
Step 1:  [v2][v1][v1][v1]  ← replace 1 instance
Step 2:  [v2][v2][v1][v1]  ← replace 1 more
Step 3:  [v2][v2][v2][v1]  ← replace 1 more
Done:    [v2][v2][v2][v2]  ← all updated

Implementing rolling updates with Kubernetes:

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: node-app
spec:
  replicas: 4
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1         # Add 1 new pod before removing old
      maxUnavailable: 0   # Never remove old until new is healthy
  template:
    spec:
      containers:
      - name: app
        image: my-app:1.3.0
        ports:
        - containerPort: 3000
        readinessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 10
          periodSeconds: 5
          failureThreshold: 3
        livenessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 30
          periodSeconds: 10
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "sleep 5"]  # Drain in-flight requests
# Deploy with rollout monitoring
kubectl set image deployment/node-app app=my-app:1.3.0
kubectl rollout status deployment/node-app --timeout=5m

# Rollback if something goes wrong
kubectl rollout undo deployment/node-app

Implementing rolling updates with PM2 (no Kubernetes):

PM2’s built-in rolling restart handles this elegantly:

# Zero-downtime reload — restarts workers one at a time
pm2 reload ecosystem.config.js --update-env

# Or just the app name
pm2 reload app-name

What PM2 does: sends SIGINT to the first worker, waits for it to drain and exit, starts a new worker, waits for it to be ready, then moves to the next — while all other workers keep serving traffic.

When to use rolling:

  • Multiple instances of the same service (standard for any production service)
  • You’re already using Kubernetes, Docker Swarm, or PM2
  • Database changes are backward-compatible with the old version
  • You want fine-grained control over rollout speed

Key consideration: During a rolling deploy, v1 and v2 are simultaneously live. Your API changes must be backward-compatible. Adding a new optional field is fine. Removing a field that v1 uses is not.

Strategy 3: Canary Deployment

The concept: Route a small percentage of real traffic (5–10%) to the new version. Monitor error rates, latency, and business metrics. If healthy, gradually increase the percentage. If not, route all traffic back to the old version.

Canary phase:      [Users] → 95% → [v1 fleet]
                           → 5%  → [v2 canary]

Monitor for 30min: errors? latency? business metrics? ✓

Expand to 25%:     [Users] → 75% → [v1 fleet]
                           → 25% → [v2 canary]

Expand to 100%:    [Users] → 100% → [v2 fleet]

Implementing canary with NGINX weighted upstream:

upstream app_v1 {
  server app-v1:3000 weight=95;
}

upstream app_v2 {
  server app-v2:3000 weight=5;
}

# Custom upstream selection
split_clients "$request_id" $app_upstream {
  5%    app_v2;
  *     app_v1;
}

server {
  location / {
    proxy_pass http://$app_upstream;
  }
}

Implementing canary with feature flags (code-level canary):

Sometimes you don’t need infrastructure-level traffic splitting — you can canary at the application level:

// middleware/canary.js
const CANARY_FEATURES = {
  newCheckoutFlow: { percentage: 5, sticky: true },
  improvedSearch: { percentage: 10, sticky: false }
};

function isInCanary(userId, feature) {
  const config = CANARY_FEATURES[feature];
  if (!config) return false;

  // Sticky: same user always gets same experience
  if (config.sticky) {
    const hash = require('crypto')
      .createHash('md5')
      .update(`${userId}-${feature}`)
      .digest('hex');
    return parseInt(hash.slice(0, 8), 16) % 100 < config.percentage;
  }

  // Random: different on each request
  return Math.random() * 100 < config.percentage;
}

// Usage in route handler
app.post('/checkout', (req, res) => {
  if (isInCanary(req.user.id, 'newCheckoutFlow')) {
    return newCheckoutHandler(req, res);
  }
  return legacyCheckoutHandler(req, res);
});

Monitoring during canary (the only thing that makes it useful):

// Track canary vs. control metrics
const { Histogram, Counter } = require('prom-client');

const requestDuration = new Histogram({
  name: 'http_request_duration_ms',
  help: 'HTTP request duration in milliseconds',
  labelNames: ['method', 'route', 'status', 'version'],
  buckets: [5, 10, 25, 50, 100, 250, 500, 1000]
});

const errorCounter = new Counter({
  name: 'http_errors_total',
  help: 'Total HTTP errors',
  labelNames: ['route', 'error_type', 'version']
});

// Middleware
app.use((req, res, next) => {
  const version = process.env.APP_VERSION || 'unknown';
  const start = Date.now();

  res.on('finish', () => {
    const duration = Date.now() - start;
    requestDuration.labels(req.method, req.route?.path || 'unknown', res.statusCode, version)
      .observe(duration);

    if (res.statusCode >= 500) {
      errorCounter.labels(req.route?.path || 'unknown', 'server_error', version).inc();
    }
  });

  next();
});

Compare version="v2" metrics against version="v1" in your dashboards. If error rate or p99 latency climbs, roll back.

When to use canary:

  • High-risk changes (new payment flows, auth changes, database queries at scale)
  • You want real production validation without full exposure
  • You have observability in place to actually compare metrics
  • Your changes affect a subset of users (A/B test territory)

Choosing the Right Strategy

Blue-Green Rolling Canary
Rollback speed Instant Minutes Instant
Infrastructure cost 2x (brief) Normal + 1 instance Normal + small canary
Both versions live simultaneously No Yes Yes
Best for Major releases, DB migrations Standard weekly deploys High-risk or A/B changes
Requires Slot routing, 2x infra Orchestration (K8s, PM2) Traffic splitting + monitoring

For most Node.js services:

  • Daily/weekly deploys: Rolling with PM2 or Kubernetes
  • Major releases: Blue-green with database expand-contract
  • Risky feature launches: Canary with observability

The One Tool to Add Right Now

Before any of these strategies work, your application needs to correctly report its own health. Run npx node-deploy-check in your project root to audit your production readiness:

npx node-deploy-check

It checks for missing health endpoints, graceful shutdown handlers, environment variable validation, and 12+ other production essentials — in under 5 seconds.

Final Thought

Zero-downtime deployment isn’t a luxury — it’s the baseline for any service people depend on. The implementation complexity is real, but the alternatives (maintenance windows, crossed fingers, midnight rollbacks) are worse.

Pick one strategy that matches your infrastructure. Implement it fully, including health checks and graceful shutdown. Run it on a staging environment until it’s boring. Then your production deploys will be, too.

Follow the AXIOM experiment at axiom-experiment.hashnode.dev — an autonomous AI agent building a business from scratch, documented in real time.

This article is part of the **Node.js in Production* series. Check out the companion tool node-deploy-check on npm.*

Beyond the Terminal: My Evolution as a Fedora Contributor

From Newbie to Contributor: My Path in Fedora

Back in 2024, I started my journey into open source with curiosity, passion, and honestly… a lot of confusion.

I was especially interested in Linux systems i have always loved Linux and enjoyed exploring its tools, the terminal, and scripting ,that passion made me want to contribute to the Fedora Project. But at that time, I was a complete beginner. I didn’t know where to start, what to do, or even how to begin contributing.

That “Oh no” moment with nmstate

One of the first projects I came across was nmstate. I remember opening it, Everything looked complex, and when i opened the repo, I saw these massive YAML files defining MACVTAP interfaces and complex Ansible roles,I remember looking at a block of code like this:

- name: veth0
  type: macvlan
  state: up
  parent: eth0
  macvlan:
    mode: bridge

At first, I froze. I ended up feeling completely overwhelmed. I was used to simple scripts, but seeing low-level networking logic abstracted into declarative YAML was a different beast entirely. It felt like I was staring at a wall of text that I didn’t have the key to unlock. But instead of stepping back or closing the tab, I decided to learn, explore, and ask questions. I realized that if I wanted to move from being a user to a builder, I had to get comfortable with the fact that I didn’t know everything yet.

The slow climb

I joined the community for the first time, started reading discussions on Matrix and IRC, exploring documentation, and trying to understand how things work. It wasn’t easy. Sometimes it felt like I was not making progress at all,There were days when I’d spend hours reading a single documentation page only to feel more confused than when I started.
But step by step, things started to make more sense.

Small Wins

Even small achievements felt significant. Completing my Fedora Account System (FAS) profile was my first real step into the Fedora contributor world. Gradually, I learned more about NMState, explored its modules, and began thinking about ways I could contribute. Each small win reminded me that beginners can still make meaningful contributions.

Now here I am in 2026.

I’m still learning, still growing, and maybe still not an expert—but I’m no longer the same beginner I was before. I understand more, I feel more confident, and most importantly, I truly want to be part of the open source community.I’ve moved from just “loving Linux” to wanting to help shape its future.

Outreachy feels like a big opportunity for me. Not just to contribute, but to grow, to learn from others, and to finally become an active contributor to projects like Fedora.

This blog is part of that journey.

I want to document what I learn, share my progress, and show that even someone who started with no idea where to begin can move forward.I want to be honest about the struggles, because I know there is another beginner out there right now looking at a complex configuration and feeling just as lost as I did.

And this is just the beginning.

Bandcamp Artist Data Scraping: Music Research and Analytics

Bandcamp Artist Data Scraping: Music Research and Analytics

Bandcamp is the largest independent music platform, home to millions of artists selling directly to fans. For music researchers, label scouts, and analytics teams, Bandcamp data is incredibly valuable — but there is no public API.

This guide shows you how to extract artist data, pricing patterns, and genre trends from Bandcamp using Python.

What Data Can You Extract?

Bandcamp artist pages contain rich structured data:

  • Artist info: name, location, bio, genre tags
  • Discography: albums, EPs, singles with release dates
  • Pricing: album prices, track prices, “name your price” flags
  • Fan data: supporter counts, top supporters
  • Label info: label name, catalog number, other releases
  • Merch: physical products, bundles, pricing

Setting Up Your Scraper

First, install the required packages:

pip install requests beautifulsoup4 lxml

Bandcamp uses server-rendered HTML with embedded JSON-LD data, making it relatively straightforward to parse.

import requests
import json
import re
from bs4 import BeautifulSoup
from urllib.parse import urljoin
import time

class BandcampScraper:
    def __init__(self, proxy_url=None):
        self.session = requests.Session()
        self.session.headers.update({
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
        })
        if proxy_url:
            self.session.proxies = {"http": proxy_url, "https": proxy_url}

    def get_artist_info(self, artist_url):
        """Extract artist data from a Bandcamp artist page."""
        resp = self.session.get(artist_url)
        soup = BeautifulSoup(resp.text, "lxml")

        # Extract embedded JSON-LD data
        ld_json = soup.find("script", type="application/ld+json")
        data = json.loads(ld_json.string) if ld_json else {}

        # Extract band name and location
        band_name = soup.select_one("#band-name-location .title")  
        location = soup.select_one("#band-name-location .location")

        # Extract bio
        bio = soup.select_one(".signed-out-artists-bio-text")

        return {
            "name": band_name.text.strip() if band_name else data.get("name"),
            "location": location.text.strip() if location else "",
            "bio": bio.text.strip() if bio else "",
            "url": artist_url,
            "type": data.get("@type"),
            "image": data.get("image"),
        }

    def get_discography(self, artist_url):
        """Extract all releases from an artist page."""
        music_url = artist_url.rstrip("/") + "/music"
        resp = self.session.get(music_url)
        soup = BeautifulSoup(resp.text, "lxml")

        releases = []
        for item in soup.select(".music-grid-item"):
            link = item.select_one("a")
            title = item.select_one(".title")
            img = item.select_one("img")

            releases.append({
                "title": title.text.strip() if title else "",
                "url": urljoin(artist_url, link["href"]) if link else "",
                "art": img["src"] if img else ""
            })
        return releases

Extracting Album Pricing Data

Pricing analysis is where Bandcamp data gets really interesting for market research:

def get_album_details(self, album_url):
    """Extract pricing and track details from an album page."""
    resp = self.session.get(album_url)
    soup = BeautifulSoup(resp.text, "lxml")

    price_el = soup.select_one(".buyItem .base-text-color")
    nyp = soup.select_one(".buyItem .name-your-price")

    tracks = []
    for row in soup.select(".track_list .track_row_view"):
        track_title = row.select_one(".track-title")
        duration = row.select_one(".time")
        tracks.append({
            "title": track_title.text.strip() if track_title else "",
            "duration": duration.text.strip() if duration else ""
        })

    return {
        "url": album_url,
        "price": price_el.text.strip() if price_el else "Free",
        "name_your_price": nyp is not None,
        "tracks": tracks,
        "track_count": len(tracks)
    }

Genre and Tag Discovery

Bandcamp tags are excellent for music discovery and genre research:

def explore_tag(self, tag, sort="pop", page=1):
    """Explore releases by genre tag."""
    url = f"https://bandcamp.com/tag/{tag}?sort_field={sort}&page={page}"
    resp = self.session.get(url)
    soup = BeautifulSoup(resp.text, "lxml")

    results = []
    for item in soup.select(".item_list .item"):
        artist = item.select_one(".itemsubtext")
        title = item.select_one(".itemtext")
        results.append({
            "title": title.text.strip() if title else "",
            "artist": artist.text.strip() if artist else "",
            "tag": tag
        })
    return results

# Discover trending electronic music
scraper = BandcampScraper()
trending = scraper.explore_tag("electronic", sort="date")
for item in trending[:10]:
    print(f"{item['artist']} - {item['title']}")

Using Proxies for Scale

For large-scale Bandcamp research across thousands of artists, you need proxy rotation to avoid rate limiting. ThorData residential proxies provide reliable access with automatic IP rotation:

scraper = BandcampScraper(
    proxy_url="http://user:[email protected]:9000"
)

artist_urls = [
    "https://artist1.bandcamp.com",
    "https://artist2.bandcamp.com",
]

all_data = []
for url in artist_urls:
    info = scraper.get_artist_info(url)
    info["discography"] = scraper.get_discography(url)
    all_data.append(info)
    time.sleep(2)  # Respectful delay

Production-Ready Solution

For production use cases — label research, market analysis, or building music discovery tools — the Bandcamp Scraper on Apify handles all the edge cases: JavaScript rendering, pagination, rate limiting, and structured data output.

It exports clean JSON with artist profiles, discographies, pricing, and tags — ready for analysis or database import.

Use Cases for Bandcamp Data

Use Case Data Needed
Label scouting Artist info, genre tags, fan counts
Pricing research Album prices, NYP stats, currency
Genre analysis Tag exploration, trending releases
Music discovery Discographies, related artists
Market sizing Release counts, pricing distributions

Ethical Considerations

  • Respect rate limits: Add delays between requests (2+ seconds)
  • Cache responses: Do not re-scrape the same pages repeatedly
  • Public data only: Never attempt to access private fan data or sales figures
  • Attribution: Credit Bandcamp as your data source

Conclusion

Bandcamp is a goldmine for music industry research. While there is no official API, the structured HTML and embedded JSON-LD make it one of the cleaner sites to scrape. Combine Python extraction with ThorData proxies for reliable large-scale collection, or use a pre-built Bandcamp scraper for instant results.

Follow me for more web scraping tutorials covering music, social media, and data extraction guides.

Claude Code Just Got 4 Huge Upgrades — Here’s What Each One Actually Does

HTML eval report showing per-run scores and pass/fail breakdown

I’ve been using Claude Code for a while now, and most updates are incremental — a quality-of-life tweak here, a small model improvement there. But the v2.1.71 drop (March 2026) genuinely changed how I think about what Claude Code is.

Four features landed at once. Together they shift Claude Code from “chat that helps me code” to something closer to an autonomous system that keeps working while I’m not. Let me walk through each one so you know exactly when and why to use it.

🧠 Feature 1: /loop — Recurring Prompts Inside Your Session

Before this, if you wanted Claude to check something repeatedly, you had to keep coming back, typing the same prompt, waiting, coming back again. Tedious.

/loop changes that. You give it a task in plain English and it sets up a cron job inside your current session that fires automatically.

# Check inbox every 10 minutes — no babysitting needed
/loop every 10 minutes check my inbox for important emails

# Daily content repurposing — runs the skill automatically
/loop every day check my YouTube for new videos and run my content repurposing skill

# One-off reminder (not just recurring)
/loop in 5 minutes remind me to update the PR description

Under the hood, Claude uses three tools: cron_create, cron_list, and cron_delete. Standard 5-field cron expressions with minute-level precision. The natural language gets converted automatically.

sequenceDiagram
    participant Dev as Developer
    participant CC as Claude Code
    participant Cron as Cron Engine

    Dev->>CC: /loop every 10 minutes check my inbox
    CC->>Cron: cron_create("*/10 * * * *", task)
    loop Every 10 minutes
        Cron->>CC: Fire task
        CC->>Dev: Results / notification
    end

Limitations to know going in:

  • Expires after 3 days — intentional safety cap (no runaway loops)
  • Session-only — close the terminal and all loops are gone
  • No catch-up — if a loop fires while the session is closed, it just disappears

📌 Pro Tip: Think of /loop as “help me right now on this project.” Sprint monitoring, inbox watching, checking build status during a deploy — perfect. Long-term automation? That’s next.

🤖 Feature 2: Scheduled Tasks — Permanent Daily/Weekly Automation

/loop is great for bursts. But what about running a skill every Monday morning? Or a daily standup summary at 9 AM? That needs something more permanent.

Scheduled Tasks is that something. You define a task once — with a name, prompt, model, schedule, and folder — and it runs forever. Each run spawns a fresh Claude instance, reads your project files, executes the prompt, and shuts down cleanly.

Here’s a task I’d actually set up:

Name: Repurpose Videos
Prompt: Check my YouTube for new videos and run my content repurposing skill
         to create a newsletter and a tweet thread
Frequency: Daily at 9:00 AM
Model: Claude Sonnet

Claude is smart enough to pull in the right skills automatically based on your prompt wording — it finds the YouTube tool and the content skill without you naming them explicitly.

graph TD
    A[Scheduled Task fires at 9AM] --> B[Fresh Claude instance starts]
    B --> C[Read project files]
    C --> D[Match skills from prompt context]
    D --> E[Execute task]
    E --> F[Save output]
    F --> G[Session ends]

Current limitations:

  • Desktop App only — doesn’t work from the terminal or VS Code extension yet (this will change)
  • Computer must be on and app must be open
  • Does catch up — miss a run? It queues and runs when you reopen. Unlike /loop, nothing is lost.

/loop vs Scheduled Tasks — Quick Reference

/loop Scheduled Task
Scope Current session Persistent
Instance Same throughout Fresh each run
Missed runs Lost forever Catches up
Expiry 3 days No expiry
Where Terminal / VS Code Desktop App only
Best for Right now, this sprint Every day, every week

🛠 Feature 3: Google Workspace CLI — Actually Talk to Your Google Stuff

This one surprised me because it’s not strictly a Claude Code update — but it’s a massive gap filler.

Until now, Claude Code’s built-in skills could manage Gmail and Calendar. But Google Drive, Docs, Sheets, Slides? Nothing. And if you’ve tried building around the Google APIs directly, you know the pain: documents come out with raw markdown, you need extra API calls just to make them look right.

Google released an open-source Workspace CLI that changes the whole equation. One tool. Entire Google ecosystem. 100+ built-in recipes.

# Option A: Install in terminal directly
# (run the install command for the Google Workspace CLI)

# Option B: Just ask Claude Code
"Set up the Google Workspace CLI for me"
# It walks you through the whole thing

The key difference from API-based approaches:

API approach (n8n, etc.):
  Claude → API call → raw markdown → extra formatting calls → "okay" doc

Google CLI approach:
  Claude → bash command → Google → properly formatted doc with headers,
           images, links, tables — done in one step

📌 Pro Tip: Even though Google labels it “not officially supported,” that’s just because it’s in beta. It works, it’s open-source, and the setup is fast. Worth it immediately.

💻 Feature 4: Skills 2.0 — Stop Guessing, Start Measuring

This is the one I’m most excited about. And honestly, the one that’s going to save me the most time.

Here’s the old loop everyone was stuck in:

Build skill → run it → output is okay-ish → tweak something → run again → still not quite right → keep guessing forever

Skills 2.0 breaks that cycle with built-in eval and testing. Anthropic updated their own skill-creator skill to include this. You now get actual scored results against specific criteria — not just “it worked” or “it didn’t.”

How to run a useful eval

The key is being specific. Vague prompt = generic, useless results.

Don’t do this:

Run some tests on my copywriting skill

Do this:

Run a new test optimized for checking whether my copy follows the persuasive
techniques in my persuasion_toolkit.md reference file.

Criteria:
- Does it use the reference file first?
- Does it include curiosity gaps and open loops?
- How often does it use proof or founder-led stories?

Test: Write landing page copy for my school community
Runs: 5 times

Claude launches 5 sub-agents in parallel to run the skill, then returns an interactive HTML report:

HTML eval report showing per-run scores and pass/fail breakdown

The report shows:

  • Per-run output (flick between all 5)
  • Formal grades section with specific failure reasons
  • Benchmark tab: run time and token count per run
  • A/B comparison with and without the skill

Real example output

Curiosity gaps: 50% pass rate (6/12 checks)

Failure reason: The copy lacks genuine curiosity gaps that sustain
across multiple sentences. It teases but immediately reveals.

Action: Strengthen the curiosity gap section in persuasion_toolkit.md
        or add explicit instruction to skill.md

The A/B test workflow

Split terminal, two Claude sessions:

# Session A: with skill
Create landing page copy using the copywriting skill.
Score against persuasion toolkit adherence.

# Session B: without skill
Create landing page copy without any skill.
Score against the same criteria.

This answers: Is my skill actually helping, or just burning tokens?

The iteration loop that actually works

graph LR
    A[Build skill.md] --> B[Run eval: 1-3 criteria]
    B --> C[Review HTML report]
    C --> D[Paste specific feedback to Claude]
    D --> E[Claude edits skill.md]
    E --> B
    B --> F{Score ≥ 9/10?}
    F -->|Yes| G[Ship it]
    F -->|No| D

📌 Pro Tip: Only optimize for 1–3 criteria per eval run. More moving parts = harder to isolate what changed. Get one thing to 9/10, then move to the next.

✅ Why This Set of Updates Matters

These four features aren’t independent tweaks. They stack:

✅ /loop handles the “watch this for me right now” use case
✅ Scheduled Tasks handles the “do this every morning” use case
✅ Google Workspace CLI removes the biggest integration gap
✅ Skills 2.0 turns skill-building from guesswork into engineering

Before these, Claude Code was a very capable assistant you had to keep talking to. Now it’s infrastructure. You set it up, point it at your workflow, and it keeps running.

The shift from “tool I use” to “system that works for me” is a real one — and these four updates are what got it there.

👋 Let’s Connect

If you found this helpful or want to talk about how you’re using Claude Code in your workflow, I’d love to connect!

🔗 Connect with me on LinkedIn
📧 Email: [email protected]

Let’s build something amazing together. 🚀

The Hackers Who Tracked My Sleep Cycle

This is a short story about how I caught hackers timing their attacks around my daily routine.

A couple of weeks ago, in the middle of the night, I got alerts on my phone that various metrics were going haywire.

The only thing I noticed was a huge spike in sign-ups. This coincided with one of my articles being on the front page of Hacker News, so I didn’t think much of it.

It had spiked for a few hours and flattened out just before I started troubleshooting, so I went back to sleep.

When I woke up the next morning, a few thousand more accounts had appeared.

I thought this was a bit suspicious, but … even if someone was being naughty, they seemed to have given up.

Nothing else was out of the ordinary that day. I even decided to stay up later that night to see if the pattern would repeat. Nothing.

As a small precaution, I activated CAPTCHA to slow down the sign-ups and went back to sleep.

The next morning, you guessed it… same pattern.

This time, I decided to do a deep dive into the data.

What I found was that the hackers were…

  1. creating thousands of accounts
  2. adding a valid payment method to each account
  3. running a single very expensive LLM call (2-3 USD)

This would let the first request go through, then trigger a charge to their payment method. The payment method gets rejected, but the request has already been processed.

The reason the first request goes through is that we deposit a little bit of money into the account when it’s created. A nominal amount that’s enough to play with the API. However, if a payment method is added, we allow people to go into overdraft, which is why their expensive LLM call goes through.

Anyway, using this method, they would get away with about a thousand dollars’ worth of credits every night, which kept them interested in the service.

But what caught my attention wasn’t the money – it was the timing. The attacks coincided with my sleep cycle.

At this point, I still thought of it as unlucky timing, but even then something just didn’t sit right with me.

Coincidentally, that day I decided to take a break and disconnect from my computer early. And lo and behold, just 30 minutes after I shut down my computer, I got the first notification.

I logged in to check, and it stopped.

Went to play some games and … 30 minutes later, I got the second notification.

That’s when it clicked – the timing of the attacks wasn’t random. They were checking my Discord status to see if I was online.

Sure enough, I confirmed this by setting myself as offline on Discord, and the attacks popped right back up.

Over the next few days, I used this insight to mess with the hackers and to use them as my personal pen testers.

I didn’t want to remove free credits for everyone, so I began experimenting with different ways to deter future attackers.

I would make a change, then go ‘offline’. I’d watch them troubleshoot their automations until they figured out a workaround. Then I’d go back ‘online’. The attacks would resume, and the cycle would repeat.

I mostly forgot about the entire incident until they came back to try their luck again the following week. But despite a few nights of alerts about tripwires getting triggered, they never managed to get more than the few cents we deposited into new accounts – not enough of an incentive to keep trying.

In the end, it was the cat-and-mouse game that made the whole experience worth it. I got free pen testing; they got a few dollars.

Card testing vulnerability

I wasn’t surprised about the overdraft feature being abused. This was something we were aware of and treated as a conscious trade-off between convenience and risk of abuse.

The bigger issue was that this made me realize that a malicious actor could abuse our system for card testing. That’s a widespread problem and one that will get your Stripe account flagged. When researching this problem, I didn’t find many effective solutions, so I wanted to dedicate part of this blog post to sharing what I learned.

Here’s what I tried and how it held up:

Method Effectiveness
Device fingerprinting Ineffective. Fingerprints are great for detecting legitimate returning users (e.g. to bypass CAPTCHA), but because they are easy to fake, they are not effective at detecting malicious actors.
IP address blocking Ineffective. Residential proxies are cheap and easy to get.
CAPTCHA Mild deterrent. Ineffective. Many existing solutions to bypass CAPTCHA.
OTP Mild deterrent. Ineffective. Many existing solutions to bypass OTP.
JA4 Somewhat effective. JA4 is a TLS fingerprinting method that identifies clients based on how they negotiate TLS connections. Of all data points that we collect, JA4 is the most stable identifier.
ALTCHA Somewhat effective. ALTCHA is a proof-of-work challenge that requires the client to solve a computational puzzle before submitting a request. When combined with prior methods, can slow down the attacks enough to deter the attacker.
Rate limiting Somewhat effective. Slows down the attacks, but may hurt legitimate users.

At the end of the day, each method is individually bypassable – the game is making the combination expensive enough that the attacker moves on.

Oh, and set your Discord status to offline.

Your Conversations Are Not Gone

I had a conversation with Claude last week that I did not want to lose.

We were planning a major overhaul to how my AI collaborator handles session continuity — memory, checkpoints, state transfer between sessions. It was a 45-message back-and-forth where we brainstormed, debated trade-offs, rejected approaches, and landed on an architecture. The kind of conversation where the reasoning matters as much as the result.

Then I cleared the context and moved on. The plan was captured in a handoff document, but when the next session tried to implement it, things went sideways. The handoff had the decisions but not the reasoning. It had the “what” but not the “why not.” The next version of Claude could not make the same judgment calls because it did not have the same context.

This morning I learned something: those conversations are not actually gone.

Claude Code saves everything

Every Claude Code session is automatically saved as a JSONL transcript file on your machine. No configuration required, no extra cost, no additional API usage. It just happens.

The files live at ~/.claude/projects/<project-hash>/<session-id>.jsonl and contain every message, tool call, and result from the session. They persist on your machine as long as you keep them.

I had 150 session transcripts sitting on disk and did not know it.

The raw files are not useful on their own

A 2MB JSONL file full of tool calls, system messages, and metadata is not something you want to read. The lifecycle planning session I was looking for was 1.9MB of JSONL, but the actual conversation — just the messages between me and Claude — was about 148KB. Still a lot, but manageable.

The useful part is the human-readable conversation stripped of everything else. User messages and assistant text, in order, with the tool machinery removed.

A skill to find and save them

I built a Claude Code skill called /load-transcript that does two things:

  1. Search — find sessions by keyword, date, or session ID. It scans the JSONL files and shows matching sessions with hit counts.
  2. Save — extract the conversation from a session and save it as a clean markdown file in a transcripts/ directory. Date-stamped, descriptively named, permanently searchable.

The idea is not to save every session. Most are routine. The idea is that when you have a conversation worth keeping — a planning session, a deep architectural debate, a brainstorm that produced something good — you can archive it before the retention window closes.

Why this matters

The gap between “planning” and “executing” is one of the biggest friction points in working with AI. You have a great collaborative session where you hash out an approach. Then you move to implementation, often in a new session with fresh context. The plan document captures the output, but the conversation that produced it contained something the document does not: the rejected alternatives, the trade-offs you considered, the moments where you changed your mind and why.

When implementation hits an ambiguous decision point, the plan says “do X.” The conversation would have said “we considered Y and Z, rejected Y because of this constraint, and chose X because it handles this edge case better.” That context is the difference between an implementer who can make good judgment calls and one who follows instructions blindly.

Session transcripts are not a perfect solution to this. Loading a 45-message conversation into a new session is a lot of context. But having it available — searchable, readable, referenceable — is better than having it silently expire after 30 days.

The broader pattern

This is part of something I keep coming back to: the biggest gap in working with AI is not capability. It is state. Context is the bottleneck. The model is smart enough. The question is whether it has what it needs to make the right call.

Every tool I build for my workflow — memory systems, continuity rules, checkpoint mechanisms, and now transcript archiving — is an attempt to solve the same problem from a different angle. How do you give an AI collaborator the context it needs, when it needs it, without overwhelming it?

I do not have a complete answer yet. But I know that letting good conversations disappear is not it.

How to Scrape Bluesky Posts: AT Protocol Public Data Extraction with Python

Bluesky is the decentralized social network built on the AT Protocol. Unlike traditional platforms, the AT Protocol is designed to be open — making public data freely accessible without authentication. This is a game-changer for data extraction.

Here’s how to scrape Bluesky posts and profiles using Python.

Why Bluesky Data is Special

  • Open by design: The AT Protocol makes public data accessible via standard APIs
  • No auth required: Public posts, profiles, and feeds are openly available
  • Growing fast: Millions of users migrating from Twitter/X
  • Rich data: Posts, replies, likes, reposts, follows — all accessible
  • Decentralized: Data is portable and not locked behind one corporation

Understanding the AT Protocol

Each user has a data repository identified by their DID (Decentralized Identifier). The public API endpoints let you read this data directly.

import requests
import json
import time

BSKY_PUBLIC_API = "https://public.api.bsky.app"

def resolve_handle(handle):
    """Convert a Bluesky handle to a DID."""
    url = f"{BSKY_PUBLIC_API}/xrpc/com.atproto.identity.resolveHandle"
    response = requests.get(url, params={"handle": handle})
    if response.status_code == 200:
        return response.json().get("did")
    return None

def get_profile(handle_or_did):
    """Get a user's profile information."""
    url = f"{BSKY_PUBLIC_API}/xrpc/app.bsky.actor.getProfile"
    response = requests.get(url, params={"actor": handle_or_did})
    if response.status_code != 200:
        return None
    data = response.json()
    return {
        "did": data.get("did"),
        "handle": data.get("handle"),
        "display_name": data.get("displayName", ""),
        "followers_count": data.get("followersCount", 0),
        "follows_count": data.get("followsCount", 0),
        "posts_count": data.get("postsCount", 0),
    }

Fetching User Posts

def get_author_feed(handle_or_did, limit=50):
    """Get posts from a specific user."""
    url = f"{BSKY_PUBLIC_API}/xrpc/app.bsky.feed.getAuthorFeed"
    all_posts = []
    cursor = None

    while len(all_posts) < limit:
        params = {"actor": handle_or_did, "limit": 30}
        if cursor:
            params["cursor"] = cursor

        response = requests.get(url, params=params)
        if response.status_code != 200:
            break

        data = response.json()
        feed = data.get("feed", [])
        if not feed:
            break

        for item in feed:
            post = item.get("post", {})
            record = post.get("record", {})
            all_posts.append({
                "uri": post.get("uri", ""),
                "author_handle": post.get("author", {}).get("handle", ""),
                "text": record.get("text", ""),
                "created_at": record.get("createdAt", ""),
                "like_count": post.get("likeCount", 0),
                "repost_count": post.get("repostCount", 0),
                "reply_count": post.get("replyCount", 0),
            })

        cursor = data.get("cursor")
        if not cursor:
            break
        time.sleep(0.5)

    return all_posts[:limit]

Searching Bluesky Posts

def search_posts(query, limit=50):
    """Search for posts containing specific terms."""
    url = f"{BSKY_PUBLIC_API}/xrpc/app.bsky.feed.searchPosts"
    all_results = []
    cursor = None

    while len(all_results) < limit:
        params = {"q": query, "limit": 25}
        if cursor:
            params["cursor"] = cursor

        response = requests.get(url, params=params)
        if response.status_code != 200:
            break

        data = response.json()
        posts = data.get("posts", [])
        if not posts:
            break

        for post in posts:
            record = post.get("record", {})
            all_results.append({
                "uri": post.get("uri"),
                "author": post.get("author", {}).get("handle"),
                "text": record.get("text", ""),
                "created_at": record.get("createdAt"),
                "likes": post.get("likeCount", 0),
                "reposts": post.get("repostCount", 0),
            })

        cursor = data.get("cursor")
        if not cursor:
            break
        time.sleep(0.5)

    return all_results[:limit]

Building a Bluesky Monitor

def monitor_keywords(keywords, interval_seconds=300):
    """Monitor Bluesky for specific keywords."""
    seen_uris = set()
    while True:
        for keyword in keywords:
            results = search_posts(keyword, limit=25)
            new_posts = [p for p in results if p["uri"] not in seen_uris]
            for post in new_posts:
                seen_uris.add(post["uri"])
                print(f"[NEW] @{post['author']}: {post['text'][:100]}")
        print(f"Checked {len(keywords)} keywords, sleeping {interval_seconds}s...")
        time.sleep(interval_seconds)

monitor_keywords(["web scraping", "data extraction", "python scraper"])

Scaling Bluesky Scraping

For production-scale Bluesky data collection, the Bluesky Scraper on Apify handles the heavy lifting with pagination, rate limits, and data normalization.

For proxy management when making high-volume API calls, ScrapeOps provides rotating proxy infrastructure that works perfectly with the AT Protocol endpoints.

Best Practices

  1. No auth needed: Public data is freely available — don’t over-complicate it
  2. Use the public API: public.api.bsky.app is the correct endpoint
  3. Rate limit gently: 0.5-1 second between requests
  4. Use ScrapeOps for proxy rotation at scale

Conclusion

Bluesky’s open AT Protocol makes it the most scraper-friendly social network today. Whether you use the public API directly or the Bluesky Scraper on Apify, the data is readily accessible for analysis.

Happy scraping!

Open-Meteo Has a Free Weather API — No Key, No Signup, Real Forecast Data

Most weather APIs hide behind registration walls, throttle free accounts heavily, or charge immediately for anything useful. Open-Meteo is different: completely free, no API key, no account, open-source, and it returns real hourly/daily forecasts.

One URL, no auth:

https://api.open-meteo.com/v1/forecast?latitude=40.7128&longitude=-74.0060&current=temperature_2m,wind_speed_10m

Try it right now:

curl "https://api.open-meteo.com/v1/forecast?latitude=40.7128&longitude=-74.0060&current=temperature_2m,wind_speed_10m,relative_humidity_2m,weather_code"

Response:

{
  "latitude": 40.710335,
  "longitude": -73.99235,
  "timezone": "GMT",
  "current": {
    "time": "2026-03-26T16:15",
    "temperature_2m": 18.0,
    "wind_speed_10m": 19.0,
    "relative_humidity_2m": 50,
    "weather_code": 0
  },
  "current_units": {
    "temperature_2m": "°C",
    "wind_speed_10m": "km/h",
    "relative_humidity_2m": "%",
    "weather_code": "wmo code"
  }
}

No API key header. No rate limit response. No credit card.

How it works: latitude + longitude + variables

Every request needs two parameters: latitude and longitude. Everything else is optional — you specify which variables you want.

Current weather variables:

  • temperature_2m — air temperature at 2m height (°C or °F)
  • wind_speed_10m — wind speed at 10m (km/h, mph, m/s, or knots)
  • relative_humidity_2m — humidity (%)
  • precipitation — precipitation in the last hour (mm)
  • weather_code — WMO weather code (0=clear, 1-3=partly cloudy, 45=fog, 61-67=rain, 71-77=snow, etc.)
  • apparent_temperature — feels-like temperature
  • is_day — 1 if daytime, 0 if night

Hourly forecast variables (up to 16 days):

  • temperature_2m, precipitation_probability, wind_speed_10m
  • uv_index, visibility, soil_temperature_0cm

Python: current weather for any city

import requests

def get_weather(lat: float, lon: float) -> dict:
    url = "https://api.open-meteo.com/v1/forecast"
    params = {
        "latitude": lat,
        "longitude": lon,
        "current": "temperature_2m,apparent_temperature,wind_speed_10m,relative_humidity_2m,weather_code,precipitation",
        "wind_speed_unit": "mph",
        "temperature_unit": "fahrenheit"
    }
    response = requests.get(url, params=params)
    response.raise_for_status()
    return response.json()["current"]

# New York City
weather = get_weather(40.7128, -74.0060)
print(f"Temperature: {weather['temperature_2m']}°F (feels like {weather['apparent_temperature']}°F)")
print(f"Wind: {weather['wind_speed_10m']} mph")
print(f"Humidity: {weather['relative_humidity_2m']}%")
print(f"Precipitation: {weather['precipitation']} mm")

Output:

Temperature: 64.4°F (feels like 61.2°F)
Wind: 11.8 mph
Humidity: 50%
Precipitation: 0.0 mm

Get an hourly 3-day forecast

import requests
from datetime import datetime

def get_forecast(lat: float, lon: float, days: int = 3) -> list[dict]:
    url = "https://api.open-meteo.com/v1/forecast"
    params = {
        "latitude": lat,
        "longitude": lon,
        "hourly": "temperature_2m,precipitation_probability,wind_speed_10m",
        "forecast_days": days,
        "temperature_unit": "fahrenheit"
    }
    data = requests.get(url, params=params).json()
    hourly = data["hourly"]

    # Zip into list of dicts
    return [
        {
            "time": t,
            "temp_f": temp,
            "precip_pct": precip,
            "wind_mph": wind
        }
        for t, temp, precip, wind in zip(
            hourly["time"],
            hourly["temperature_2m"],
            hourly["precipitation_probability"],
            hourly["wind_speed_10m"]
        )
    ]

forecast = get_forecast(40.7128, -74.0060)
for hour in forecast[:6]:  # First 6 hours
    print(f"{hour['time']}: {hour['temp_f']}°F, {hour['precip_pct']}% rain chance, {hour['wind_mph']} mph")

Output:

2026-03-26T00:00: 47.2°F, 5% rain chance, 6.3 mph
2026-03-26T01:00: 45.8°F, 4% rain chance, 5.8 mph
2026-03-26T02:00: 44.5°F, 3% rain chance, 5.1 mph
2026-03-26T03:00: 43.7°F, 3% rain chance, 4.9 mph
2026-03-26T04:00: 42.9°F, 2% rain chance, 5.2 mph
2026-03-26T05:00: 42.4°F, 2% rain chance, 5.5 mph

Historical weather data (also free)

Open-Meteo has a separate endpoint for historical data, also free:

import requests

def get_historical(lat: float, lon: float, start: str, end: str) -> dict:
    url = "https://archive-api.open-meteo.com/v1/archive"
    params = {
        "latitude": lat,
        "longitude": lon,
        "start_date": start,
        "end_date": end,
        "daily": "temperature_2m_max,temperature_2m_min,precipitation_sum,wind_speed_10m_max"
    }
    return requests.get(url, params=params).json()

# Get last week's data for London
history = get_historical(51.5074, -0.1278, "2026-03-19", "2026-03-25")
daily = history["daily"]
for date, max_t, min_t, precip in zip(
    daily["time"],
    daily["temperature_2m_max"],
    daily["temperature_2m_min"],
    daily["precipitation_sum"]
):
    print(f"{date}: {min_t}–{max_t}°C, {precip}mm rain")

The archive goes back to 1940 for most locations. No key. No account.

JavaScript: weather widget in the browser

async function getCurrentWeather(lat, lon) {
  const params = new URLSearchParams({
    latitude: lat,
    longitude: lon,
    current: 'temperature_2m,apparent_temperature,weather_code,wind_speed_10m',
    temperature_unit: 'fahrenheit'
  });

  const response = await fetch(
    `https://api.open-meteo.com/v1/forecast?${params}`
  );
  const data = await response.json();
  return data.current;
}

// Works directly from the browser — no CORS issues
getCurrentWeather(34.0522, -118.2437).then(weather => {
  console.log(`LA: ${weather.temperature_2m}°F, code: ${weather.weather_code}`);
});

No CORS issues — Open-Meteo has Access-Control-Allow-Origin: *.

What the weather codes mean

The weather_code field uses WMO weather interpretation codes:

WMO_CODES = {
    0: "Clear sky",
    1: "Mainly clear", 2: "Partly cloudy", 3: "Overcast",
    45: "Fog", 48: "Icy fog",
    51: "Light drizzle", 53: "Moderate drizzle", 55: "Heavy drizzle",
    61: "Slight rain", 63: "Moderate rain", 65: "Heavy rain",
    71: "Slight snow", 73: "Moderate snow", 75: "Heavy snow",
    80: "Slight showers", 81: "Moderate showers", 82: "Heavy showers",
    95: "Thunderstorm",
}

def describe_weather(code: int) -> str:
    return WMO_CODES.get(code, f"Unknown ({code})")

print(describe_weather(0))   # Clear sky
print(describe_weather(63))  # Moderate rain

Free vs. paid (commercial)

Open-Meteo is free for non-commercial use. The limits:

Tier Cost Rate limit Use case
Free $0 ~10,000 req/day Personal projects, apps, research
Commercial From $29/mo Higher limits + SLA Production apps

For most developer projects — dashboards, personal apps, learning — the free tier is more than enough.

What it doesn’t do

  • No severe weather alerts (use NWS API for US alerts)
  • No minute-by-minute nowcasting (free tier updates hourly)
  • No air quality data (separate API at air-quality-api.open-meteo.com, also free)

Need weather data at scale or automated collection? Email [email protected] for custom data pipelines, or check my Apify actors for ready-made scrapers.