Mert

Data centers will drive 40% of electricity demand growth through the end of the decade, according to Goldman Sachs. Most conversations about fixing that focus on chips, cooling, and renewable power contracts. There’s a faster, cheaper intervention most teams haven’t touched: how they process data.

Shifting AI workloads from batch processing to real-time streaming can meaningfully cut energy consumption — with no new hardware required.

“Systems no longer need to be sized for the worst-case burst capacity; they can scale dynamically in response to actual throughput.”

What’s actually wrong with batch

Batch processing is still the dominant model for data analysis — data accumulated, staged, then processed in scheduled bursts. That creates sharp compute spikes. Infrastructure has to be provisioned for peak load, which means capacity sitting idle between runs, cooling systems taxed during bursts, then quiet again until the next cycle.

It’s the difference between flooring the accelerator from a standstill versus steady highway cruising. Same destination, very different fuel bill. Electricity prices jumped 6.9% last year. That math is getting worse.

Why streaming changes the energy profile

Streaming architectures — Apache Kafka, Apache Flink — process data continuously as it arrives. The compute load flattens. Instead of provisioning for worst-case bursts, you scale dynamically against actual throughput.

There are downstream gains too. Streaming typically cleans and deduplicates data in transit before it hits storage. Leaner data means lighter queries and less disk I/O — both energy-intensive operations. And because individual systems process independently in a decoupled event-driven setup, you avoid cascading compute loads that tightly integrated batch pipelines tend to generate.

Why AI workloads specifically benefit

AI agents need current data. Static datasets refreshed on batch cycles lead to stale context or force reprocessing. In many setups, the batch pipeline is the bottleneck — not the models themselves.

Streaming fixes both problems: lower energy footprint and fresher inputs for your models.

Where to start

Not everything needs to migrate at once. The strongest first candidate is preprocessing for AI workloads — drop a stream processor in front of your AI pipeline to filter, aggregate, and normalize data before it reaches the model. You get leaner inputs, lower GPU/CPU load, and a measurable energy reduction.

From there, identify which batch jobs create the sharpest demand spikes and evaluate whether real-time processing is practical. The migration happens at the software layer — no new hardware, no waiting for power contracts.

The hardware improvements are already underway. The software conversation is overdue.

Source: The New Stack — Warren Vella

✏️ Drafted with KewBot (AI), edited and approved by Drew.

Two weeks ago, my crypto signal API silently failed for 22 hours.

No errors. No exceptions. No crash. The service kept running, logs continued to flow, my deployment dashboard showed everything green. I only noticed when I happened to check the database and realized no new data had been written for almost a full day.

The culprit? My WebSocket connection to Binance. It was “connected” — but it hadn’t received a message in hours.

This is the silent staleness problem. And TCP keepalive can’t catch it.

If you’ve ever built a system that consumes a long-lived WebSocket feed (price data, chat messages, IoT telemetry, log streams), you’re vulnerable to this exact failure mode. Here’s what’s happening and how to fix it.

The illusion of “connected”

When your client opens a WebSocket connection, the underlying TCP socket goes through a handshake. From then on, “connected” really means: there’s an open TCP socket between you and the server, and TCP believes the route is alive.

That’s it.

TCP keepalive (when enabled) sends periodic empty packets to verify the route. The OS does this for you. If the route is broken, you’ll eventually get a connection-closed error.

But here’s what TCP can’t see:

• Whether the application on the other end is still pushing messages
• Whether a proxy or load balancer between you and the server has dropped your subscription
• Whether a backend bug stopped emitting events while keeping the connection open

Your WebSocket can look perfectly healthy at the TCP layer while application data has stopped flowing entirely.

In my case, Binance’s WebSocket gateway accepted my connection, accepted my subscriptions, and then stopped pushing ticker updates. The TCP socket was fine. The OS was fine. My code was fine. The data was gone.

Why naive fixes don’t work

The first instinct is: “I’ll just reconnect on error.” But the application never errors. No exception fires. The connection is perfectly alive — there’s just nothing coming through.

The second instinct: “I’ll add a watchdog timer that pings the server.” This is closer to right but has a flaw — many services (including most exchange feeds) don’t respond to client pings on data WebSockets. Your ping goes out, returns silence, and you can’t distinguish “server doesn’t ping back” from “server is broken.”

The third instinct: “I’ll send a subscribe message and check for confirmation.” This catches startup failures but not mid-stream failures.

What actually works is much simpler:

Track the time of the last message received. If it exceeds a threshold, the stream is stale — regardless of what TCP thinks.

Implementing message-level staleness detection

Here’s the pattern in Python:

import asyncio
import json
import time
import websockets

STALENESS_TIMEOUT_SECONDS = 60  # tune to your feed's expected frequency

class StaleStreamError(Exception):
    pass

async def consume_stream(url, subscribe_message):
    while True:
        try:
            async with websockets.connect(url) as ws:
                await ws.send(json.dumps(subscribe_message))

                last_message_at = time.time()

                async def monitor_staleness():
                    while True:
                        await asyncio.sleep(STALENESS_TIMEOUT_SECONDS)
                        age = time.time() - last_message_at
                        if age > STALENESS_TIMEOUT_SECONDS:
                            await ws.close()
                            raise StaleStreamError(
                                f"No message for {age:.1f}s "
                                f"(threshold: {STALENESS_TIMEOUT_SECONDS}s)"
                            )

                staleness_task = asyncio.create_task(monitor_staleness())

                try:
                    async for message in ws:
                        last_message_at = time.time()
                        await handle_message(message)
                finally:
                    staleness_task.cancel()

        except websockets.exceptions.ConnectionClosed:
            print("Connection closed, reconnecting...")
        except StaleStreamError as e:
            print(f"Staleness detected: {e}, reconnecting...")

        await asyncio.sleep(1)  # backoff before retry

The key insight: define “alive” at your application level, not the OS level.

Your feed might have natural quiet periods (markets close, low-traffic hours), so tune the threshold. A 60-second timeout might be too aggressive for IoT telemetry; a 5-minute timeout might be too lenient for a high-frequency ticker.

A good heuristic: set your timeout to 3-5x the expected gap between messages during your slowest periods.

What about exchange-provided heartbeats?

Some WebSocket protocols include explicit heartbeats — small periodic messages that confirm both parties are alive at the application layer. Binance Futures, for example, sends a ping every few minutes; you respond with a pong.

These help. But they don’t solve the staleness problem on their own, because:

1. Heartbeats might keep working while data subscription has died (different code paths on the server)
2. Some feeds don’t include heartbeats at all
3. Even with heartbeats, you still need staleness logic for the data stream specifically

Treat heartbeats as one input, not the source of truth. Your real signal is: “Am I getting the kind of message I subscribed to?”

Reconnect logic that doesn’t make things worse

When you detect staleness and reconnect, consider:

• Exponential backoff: if the server is genuinely down, don’t hammer it with reconnect attempts
• Jitter: if 1000 clients all detect staleness at the same instant (after a server outage), randomized retry intervals prevent a thundering herd
• State recovery: for stateful feeds (order books, subscription channels), you might need to resync state after reconnect
• Alerting: if you’ve had to reconnect more than N times in M minutes, something deeper is broken — page yourself

The 22-hour lesson

The bug that hit me wasn’t subtle — it’s a known failure mode in long-lived streaming systems. But I’d built my service assuming “WebSocket connected = data flowing,” and that assumption silently broke when the assumption became false.

What fixed it for good:

1. Message-level staleness detection (the pattern above)
2. External health monitoring — a small endpoint that returns last_signal_age_seconds so UptimeRobot can alert me when it crosses a threshold
3. Application-level alerting — a separate cron that emails me if no events fire for N hours during peak hours

If you’re consuming a long-lived WebSocket today and you don’t have all three of these, you’re vulnerable to the same silent failure. The fix is not expensive. The bug, when it hits, is.

About me

I’m building LeadEdge — a cross-exchange crypto signal API for trading bots. The WebSocket consumer pattern above ships in our open-source integration examples as a drop-in for anyone building on top of similar streaming feeds.

The full validation methodology with 9.4M live price updates and 90.7% follow-through on ETH cross-exchange signals is documented here.

I’ve been building Finterm, a financial terminal
that runs entirely in a browser tab, on Cloudflare
Workers via @opennextjs/cloudflare. The promise
is the obvious one — global edge runtime,
near-zero cold starts, pay-per-request pricing.
The catch is that Workers don’t run Node.js. Every
package in your dependency tree has to work in
the Workers runtime, and a surprising number of
common ones don’t.

This is a rough log of what I had to rip out and
what I replaced it with, plus a couple of patterns
that turned out to be necessary on Workers but
irrelevant on Node.

## bcrypt → Argon2id via @noble/hashes

bcryptjs reaches for crypto.randomBytes and a
few other Node primitives that aren’t polyfilled
in Workers. Argon2id is the modern alternative
anyway, and @noble/hashes exposes a pure-JS
implementation that runs anywhere and lines up
with current OWASP guidance. The migration was a
one-line change in the password hasher plus a
re-hash on next login.

## cheerio / jsdom / parse5 → htmlparser2

For the news reader I needed to extract article
bodies from arbitrary HTML. cheerio pulls in
parse5, which doesn’t run on Workers, and
jsdom is even further away from compatibility.
htmlparser2 is SAX-style streaming and pure JS,
so it runs anywhere — but you have to walk the
events yourself rather than querying a tree. Worth
it for the runtime portability.

## The Yahoo Finance crumb handshake

Not a Workers issue specifically, but I hit it
harder because of Workers. Yahoo’s quoteSummary,
options, and profile endpoints require a session
cookie and a crumb token bound to that session.
The dance: first you GET https://fc.yahoo.com/,
which returns 404 with Set-Cookie for A1 and A3;
then you GET /v1/test/getcrumb with those
cookies, which returns the crumb as plain text;
then every gated request needs that cookie plus
&crumb=... appended to the URL.

  const seed = await fetch('https://fc.yahoo.com/',
  { headers: BROWSERY });
  const cookie = seed.headers.getSetCookie()
    .map(c => c.split(';')[0])
    .filter(c => c.startsWith('A1=') ||
  c.startsWith('A3=') || c.startsWith('A1S='))
    .join('; ');
  const crumbRes = await fetch('https://query1.finan
  ce.yahoo.com/v1/test/getcrumb', {
    headers: { ...BROWSERY, Cookie: cookie },
  });
  const crumb = (await crumbRes.text()).trim();

A few gotchas. The getcrumb endpoint returns
text/plain, not JSON, so you can’t send an
Accept: application/json header — it’ll 406. The
User-Agent has to look browsery; a plain
curl/8.x or your fetch library’s default gets
rate-limited fast. The cookie and crumb stay valid
for roughly thirty minutes, after which you
re-handshake. I cache the pair per-isolate and
refresh on the first 401.

## Single outbound gateway

Workers isolates are short-lived but they do
persist module state across requests on a single
isolate, which makes the obvious caching patterns
work better than on a per-request serverless
model.

Every upstream HTTP call in the app goes through
one route — /api/data/[...path]. The route looks
up the endpoint by path, runs the provider
function, attaches Cache-Control: s-maxage=N,
stale-while-revalidate=86400, and lets the
Cloudflare edge cache do the heavy lifting. A
four-tier per-IP rate limiter only fires on cache
misses, so a popular ticker can be served to
thousands of users without ever hitting Yahoo or
SEC again that minute.

This pattern also gave me a single chokepoint for
an SSRF guard, so user-supplied paths can’t reach
back into Cloudflare’s metadata service or some
internal hostname.

## Pop-outs that share React state

Off the Workers topic but a fun one. Each window
in the dashboard has a popout button. It calls
window.open('/popout.html'), then React-renders
the same component switch into the popout’s
document. The popout writes back to the parent’s
workspace state through a callback, so edits
round-trip in real time.

Two non-obvious things had to change in the
window-body components. Any addEventListener or
matchMedia has to be bound to the popout’s own
window (I expose this through a
useOwnerWindow(ref) hook). And any
document.createElement for an offscreen canvas
has to go through
someExistingNode.ownerDocument.createElement so
the node belongs to the popout’s DOM rather than
the parent’s.

## Result

Finterm is live at
finterm.xyz — open it, type
a ticker, and a chart, SEC fundamentals, options
chain with Greeks, and analyst estimates open as
draggable windows. Everything described above runs
in Cloudflare’s edge: no Node server, no cold
starts to speak of.

Happy to go deeper on any of the above if there’s
interest.

So I wanted to learn how to use SQL databases and what better way to than to build one and learn how things works, well couldn’t cover as much as planned still touched core components and basic queries. Here is how I started.

Resources that I referred to

• Architecture of a Database System (majorly)
• database-management-systems-raghu-ramakrishnan (for query planner &
  optimization)

Source

https://github.com/lightsigma96/db_scratch

cmake -S . -B build
cmake --build build

./build/db_scratch

docker build -t db_scratch .

docker run -it db_scratch

./build/db_scratch

Storage Manager

This is the core of disk-based database, it’s function is to interact with disk,
handle various db files and manage pages.

Buffer Pool

Buffer Pool caches pages from heap files in memory (which are responsible for storage of rows), pages are small fragments of heap file which are of fixed size (4kb in my case).

Reading or Writing from heap file on disk can be slow, hence pages from heap
file are loaded into memory (RAM) through buffer pool which is much faster.

Now obviously we cannot load entire file into memory, as that that can lead to out of memory error.

Hence we load file in pages into buffer pool which sits in memory.

Buffer Pool has a page replacement policy which evicts certain pages when buffer
pool is full.

In my case I implemented a simple FIFO replacement policy which tracks pages in
order of insertion into buffer pool, which checks if the page is pinned (pages which are frequently used or are being used by some component are pinned, pin count shows how many components are using that page), if yes then send them back in the queue, if no then flush them (write to disk) and evict them.

Buffer pools also contains a Disk Operator responsible for writing to file and
other things (implementations can vary).

Access Methods

Access Methods provides with necessary data structure and algorithms to organize
and access data.

On heap file rows are stored one after another and heap scan from access methods scan each one of them to find correct row based on given SARG.

SARGs (Search Arguments) describes condition for finding a row, they contain
column, operator and constant.

Heap scan may require scanning all rows across multiple pages resulting in O(n)
search complexity, hence a index scan can be implemented using B+ Trees.

B+ Tree consists of leaf nodes and internal nodes, Leaf nodes store key and RID (Row Identifier) which points to the exact row in heap file,

B+ Tree reduces index traversal cost from scanning many pages to traversing a small number of pages proportional to tree height, after which corresponding heap pages can be accessed using RID.

The downside of index scan is that index keys are built on specific columns.If query predicates do not use indexed columns then planner may fallback to heap scan.

Query Planner and Optimizer

After Buffer Pool and Access Methods, rows can be stored on disk and retrieved,
next comes the query planner & optimizer which makes plan for execution of query
and optimizes that plan.

Lexer & Parser

Queries are tokenized into keywords, identifiers, literals and special symbols
which are then passed to parser.

Parser is responsible for checking grammar and construct AST based on tokens
passed.

Parser’s implementation can vary, in my case Lexer provides a tokenizer which is
used to iterate token forward/backward, and peek tokens.

Parser uses these token to check grammar of different queries.

Once this has been done Parser produces an AST specific to each query.

Planner and Optimizer

The AST produced by Parser is consumed by Planner which has a plan defined for
each query.

For example plan for select query involves three steps (from top to bottom):

Projection : Selects specific columns from tuples returned by child operators.

Filter : Based on SARG matching, this steps filters out rows returned by heap scan.

Scan : This calls heap scan and returns the output above in the chain.

(Rows flow upward from Scan → Filter → Projection)

Each operator pulls one tuple at a time from its child operator.

Optimization depends on type of query, for example if a index for columns exists then index scan can be used instead of heap scan.

Catalog Manager

Catalog Manager is logically separate from query execution components and is
responsible for schema metadata management.

This is used to enforce schema and handle writing and reading to/from a schema file.

To write schema to a file a serializer has to be made which writes schema in
format:

• Schema name size
• Schema name
• – heap page offset
• – heap page id
• – Table name size
• – Table name
• – – Column size
• – – Column name

The only caveat here is, before every name I had to store it’s size as I have used C++ string over here which stores pointer to actual string allocated in memory, so if I directly write it on file, I would be writing garbage memory address.

Deserializer reads this back from file.

Main Server

This initializes all the necessary objects like buffer manager, access methods,
query planner, catalog manager.

I have two pipelines, DB Pipeline, TUI Pipeline, initially I wanted to have a
client-server architecture with a tcp server requesting DB but instead I thought
of going with direct function calls.

DB Pipeline

This function is responsible for orchestration of database pipeline (A TCP server can also be created which would be responsible for preprocessing cli input and calling db).

First the input query is tokenized and checked for correctness by lexer and parser.

If query is valid then AST is generated which is then goes through their respective processing (like for SELECT query it is passed through select plan and create table is passed through create table provided by catalog manager).

A response is finally returned by this function, implementation for this pipeline can be different.

With all the things above a database can support basic things like creating schema,table,columns executing basic queries like SELECT, INSERT, DELETE (same architecture can be extended to support JOINS and Stored Procedure).

Some other core components included:

• WAL (Write Ahead Log): Ensures durability and crash recovery by logging changes before modified pages are written to disk. Follows ACID properties.
• Lock Manager: Used to gain locks for multi-threading.

Every time I debugged a deeply nested API response, I scrolled. I counted brackets. I lost my place. After the third or fourth time of doing this for the same Stripe webhook, I gave up and built a thing: paste JSON in one side, see it as an interactive graph on the other.

The result is jsonbloom.com — free, runs entirely in the browser, no signup. This post is about the architecture choices behind it, because most of them turned out to be smaller decisions than I expected.

Why not react-flow or d3-hierarchy?

Both are great libraries. I tried both before writing anything custom. The problem is that they’re designed for a much broader interaction model than what a JSON viewer actually needs.

• react-flow is ~150kb gzipped on its own. It supports node dragging, edge editing, mini-maps, custom handles — none of which a JSON visualizer needs.
• d3-hierarchy gives you the layout math, but you still bring your own renderer, your own collapse logic, your own interaction layer.

What a JSON visualizer actually needs is narrow:

1. Render objects and arrays as boxes
2. Draw edges from a parent to each child
3. Collapse / expand subtrees
4. Pan and zoom the whole canvas
5. Edit a leaf value inline

That’s it. No drag-to-rearrange, no node merging, no custom node types. Once I wrote that list down, the case for a custom renderer wrote itself: ~5kb of code that does exactly these five things, versus 150kb of code that does many more things less directly.

The architecture

Astro (static shell, SSG)
└── React island (client:only)
    ├── CodeMirror editor on the left
    └── Custom SVG graph on the right

The whole landing page — hero, feature cards, FAQ — is plain Astro components rendered at build time. Only the workspace (editor + graph) is a React island, mounted with client:only because there’s no useful SSR for it. This gets you:

• ~30kb of JS on the marketing pages (zero React there)
• Fast first paint even on slow connections
• The interactive part loads in parallel while users read the hero

If you’ve used Astro this is the obvious play. If you haven’t: it’s the single biggest perf win for a “landing page + interactive tool” site, and it’s almost free to adopt.

The layout

JSON is a tree. The naive thing is to lay it out like an org chart: root at the top, children below. That works for small payloads and breaks immediately for anything realistic — a 50-key object becomes 50 boxes in a row.

The pattern I settled on:

• Objects and arrays are boxes in a left-to-right tree
• Primitive values are pinned next to their key inside the parent box, not as separate nodes
• Each level is its own column; siblings stack vertically

This collapses what would be hundreds of nodes into tens. A typical Stripe event renders as maybe 8–15 boxes instead of 80.

Layout is one pass:

function layout(node, depth = 0, y = 0) {
  node.x = depth * COLUMN_WIDTH;
  node.y = y;
  let childY = y;
  for (const child of node.children) {
    const h = layout(child, depth + 1, childY);
    childY += h + GAP;
  }
  node.height = Math.max(NODE_MIN_H, childY - y);
  return node.height;
}

That’s it. No d3, no force simulation. It’s deterministic, fast, and you can collapse a subtree by walking the same recursion and skipping its children.

The hard parts

I underestimated three things.

1. Inline editing that stays in sync. The editor on the left is the source of truth. The graph on the right is a view. When you click a value in the graph and edit it, the change has to propagate back to the editor’s text — but the editor has cursor position, undo history, syntax highlighting. You can’t just setValue on every edit because you’d nuke the user’s cursor every keystroke they made elsewhere.

The fix was to make the graph send a patch (JSON pointer + new value) rather than a full document, and have the editor apply patches as small targeted dispatches. CodeMirror’s transaction API makes this clean once you stop thinking of the editor as a “text input.”

2. Collapsing without relayout flicker. First version: collapse a node → re-run layout for the entire tree → React re-renders everything. The result was a visible jump even on small payloads.

The fix: layout is incremental. Each subtree owns its own height. When you collapse a node, you only walk up the tree to fix the y-offsets of its later siblings, not the whole document.

3. Cycles. Most people don’t realise that JavaScript objects can have cycles but JSON cannot. If you’re parsing user input you don’t have this problem — JSON.parse would reject it. But if you ever support eval-like input (I had this in an early version) you absolutely have to detect cycles before walking, or your renderer locks up.

What I would do differently

• I should have started with the editor → graph protocol (patches) from day one. I started with “the graph re-derives from the parsed JSON every render” and that was fine until it wasn’t.
• Hand-rolled SVG was the right call for layout but a slightly less right call for the editing controls. I should have used HTML overlays positioned over the SVG for the inline editors — <input> inside <foreignObject> has just enough cross-browser jank to be a recurring annoyance.

Stack summary

• Astro for the shell
• React for the workspace island
• CodeMirror 6 for the editor
• Plain SVG + a ~200-line layout function for the graph
• No state management library — everything is local React state plus the JSON document as the single source of truth

Try it

jsonbloom.com — paste any JSON in. I’d especially love feedback on:

• Performance with your biggest real-world payloads (a few MB and up)
• Edge cases in the inline editor
• Anything that feels off when you collapse a deeply nested subtree

It’s free, no signup, and your JSON never leaves your browser.