Making Dynamic MDX Blogs Work with OpenNext on Cloudflare Workers

Before and after diagram showing an MDX blog moving from runtime filesystem reads to a build-time registry for OpenNext on Cloudflare Workers

I ran into a small but annoying problem while deploying a Next.js MDX blog to Cloudflare Workers with OpenNext.

The blog worked locally. The build passed. OpenNext even listed the blog routes during the build.

Then I opened the deployed site, and the blog page was empty.

The issue was not MDX. It was not frontmatter. It was not a missing route. The real problem was that my blog code was still thinking like a normal Node.js app, while Cloudflare Workers runs from a bundled Worker output.

The Short Version

If your MDX blog works in next dev but shows empty pages or missing posts on Cloudflare Workers, check if you read blog files with node:fs at request time.

This is the safer pattern:

  • Keep writing posts as .mdx files.
  • Parse the files during the build.
  • Generate a small TypeScript file with post metadata.
  • Generate another TypeScript file that statically imports every MDX post.
  • Render posts from that generated registry in production.

That way the Worker does not need to scan your content/blog folder at runtime.

Before and after diagram showing an MDX blog moving from runtime filesystem reads to a build-time registry for OpenNext on Cloudflare Workers

What Broke

The old setup looked something like this:

import fs from "node:fs";
import path from "node:path";
import matter from "gray-matter";

const postsDir = path.join(process.cwd(), "content", "blog");

export function getAllPosts() {
  return fs.readdirSync(postsDir).map((file) => {
    const raw = fs.readFileSync(path.join(postsDir, file), "utf8");
    const { data } = matter(raw);

    return {
      slug: file.replace(/.mdx$/, ""),
      title: data.title,
      description: data.description,
    };
  });
}

This feels fine in local development because the source files are right there on disk.

But after OpenNext builds the app for Cloudflare Workers, the app is running from a Worker bundle. Cloudflare does support node:fs through a virtual filesystem, but that filesystem is not the same as having your project folder mounted in production. Files inside the Worker bundle are readable under /bundle, and /tmp is temporary for a request.

So I stopped treating the source folder as runtime data.

The Fix

I moved blog discovery to build time.

The build step reads the MDX files once, parses the frontmatter, and writes generated TypeScript files that the app can import normally.

The mental model is simple:

Authoring:
  content/blog/*.mdx

Build:
  read MDX files
  parse frontmatter
  generate metadata
  generate static imports

Runtime:
  import generated modules
  render the matching MDX component

This keeps the nice file-based writing flow, but removes runtime filesystem reads from the deployed Worker.

Step 1: Generate Blog Metadata

I use a script before the build. It reads the .mdx files and creates metadata for the blog index, sitemap, RSS feed, and page metadata.

// scripts/generate-blog-data.mjs
import fs from "node:fs";
import path from "node:path";
import matter from "gray-matter";

const root = process.cwd();
const postsDir = path.join(root, "content", "blog");
const outputDir = path.join(root, "src", "lib", "blog");

const files = fs.readdirSync(postsDir).filter((file) => file.endsWith(".mdx"));

const posts = files.map((file) => {
  const slug = file.replace(/.mdx$/, "");
  const raw = fs.readFileSync(path.join(postsDir, file), "utf8");
  const { data, content } = matter(raw);

  const words = content.trim().split(/s+/).filter(Boolean).length;

  return {
    slug,
    title: data.title,
    description: data.description,
    date: data.date,
    readingTime: `${Math.max(1, Math.ceil(words / 225))} min read`,
  };
});

fs.mkdirSync(outputDir, { recursive: true });

fs.writeFileSync(
  path.join(outputDir, "generated-posts.ts"),
  `export const allBlogPosts = ${JSON.stringify(posts, null, 2)} as const;n`
);

The important part is not the exact script. The important part is when it runs.

It runs before next build, not when someone opens /blog.

Step 2: Generate Static MDX Imports

The next file is the one that makes the Worker build reliable.

Instead of doing this:

await import(`../../../content/blog/${slug}.mdx`);

I generate static imports:

// src/lib/blog/generated-components.ts
import type { ComponentType } from "react";

import Post0 from "../../../content/blog/first-post.mdx";
import Post1 from "../../../content/blog/second-post.mdx";

const blogPostComponents: Record<string, ComponentType> = {
  "first-post": Post0,
  "second-post": Post1,
};

export function getPostComponent(slug: string) {
  return blogPostComponents[slug] ?? null;
}

This gives Next.js and OpenNext a clear import graph. They can see the MDX files, compile them, and include them in the Worker output.

That is much easier to trust than a variable import path.

Step 3: Render From the Registry

The blog route becomes a lookup, not a file scan.

import { notFound } from "next/navigation";
import { getPostComponent } from "@/lib/blog/generated-components";
import { allBlogPosts } from "@/lib/blog/generated-posts";

export function generateStaticParams() {
  return allBlogPosts.map((post) => ({ slug: post.slug }));
}

export default async function BlogPostPage({ params }) {
  const { slug } = await params;
  const post = allBlogPosts.find((item) => item.slug === slug);
  const PostContent = getPostComponent(slug);

  if (!post || !PostContent) {
    notFound();
  }

  return (
    <article>
      <h1>{post.title}</h1>
      <p>{post.description}</p>
      <PostContent />
    </article>
  );
}

Now the production route only depends on bundled code.

No fs.readdirSync. No process.cwd(). No runtime gray-matter.

Step 4: Run the Script Before Every Build

I wired the generator into the build scripts:

{
  "scripts": {
    "prebuild": "node scripts/generate-blog-data.mjs",
    "build": "next build",
    "prebuild:cf": "node scripts/generate-blog-data.mjs",
    "build:cf": "opennextjs-cloudflare build -c wrangler.jsonc"
  }
}

Now when I add a new .mdx file, the next build updates the generated metadata and component registry.

I still get the same writing flow:

content/blog/my-new-post.mdx

But the deployed Worker gets predictable imports.

Step 5: Test the Worker Build

I do not stop at next build for this kind of bug.

next build can pass while the Worker output still behaves differently. So I check the Cloudflare build too:

pnpm build:cf

Then I preview the Worker locally:

pnpm exec opennextjs-cloudflare preview -c wrangler.jsonc

And I test three routes:

curl -I http://localhost:8787/blog
curl -I http://localhost:8787/blog/my-real-post
curl -I http://localhost:8787/blog/not-a-real-post

The result I want:

/blog                 200
/blog/my-real-post    200
/blog/not-a-real-post 404

The fake post matters. A working blog should render real posts and still reject bad slugs.

Quick Debug Checklist

  • Does any blog route import node:fs?
  • Does /blog call fs.readdirSync during a request?
  • Does the post page use a variable MDX import path?
  • Does the generated registry include every .mdx file?
  • Does pnpm build:cf list the expected blog routes?
  • Does the local Worker preview return 200 for a real post?
  • Does it return 404 for a fake post?

Where the Content Lives

The full post content still lives in MDX files.

The generated metadata file only stores things like:

  • slug
  • title
  • description
  • date
  • reading time
  • headings, if you need a table of contents

I do not put the whole article body into JSON. That gets messy fast, especially with code blocks, custom MDX components, and imports.

Instead, the body is compiled from the MDX module:

content/blog/my-post.mdx
        |
        v
import Post from "../../../content/blog/my-post.mdx"
        |
        v
<Post />

That is the clean split:

  • metadata is generated as plain data
  • content is rendered as compiled MDX

A Small SEO Note

The technical fix gets the pages to render. SEO still depends on whether the page is useful.

For this kind of technical post, I try to keep the basics simple:

  • use a clear title
  • describe the problem in the first few lines
  • use short sections
  • write headings that say what the section is about
  • add examples that someone can copy into their project
  • link to the official docs when they matter
  • avoid padding the post to hit a word count

That last point is important. Google does not require a magic word count. A shorter post that solves the problem is better than a long post that makes the reader dig.

Final Thought

The fix was mostly a change in where the work happens.

Before, the Worker had to discover blog files at request time.

After, the build discovered the files once, generated a registry, and gave the Worker normal imports to render.

That made the blog simple again. I can still add posts as .mdx files, but production no longer depends on reading my source folder at runtime.

Further Reading

The Folly of Relying on Traditional Payment Platforms for the African Creator Economy

The Problem We Were Actually Solving

When our platform began to gain traction with African creators, we encountered an unexpected problem. Many users were unable to receive payments through traditional platforms due to restricted market access and geographic limitations. This led to user churn and frustrated customers. The more we dug, the more we realized that traditional payment platforms were not designed with the nuances of emerging markets in mind.

What We Tried First (And Why It Failed)

Initially, we attempted to bypass these limitations by implementing multiple payment gateways (MPGs) on our platform. We integrated with every available payment method in the hopes that someone, somehow, would magically be able to receive payments. However, this approach ultimately led to a cluttered user experience, increased complexity, and even more user churn. MPGs required endless configuration, debugging, and maintenance, not to mention the hefty fees associated with each individual gateway.

The Architecture Decision

After months of experimentation and frustration, we made a critical decision. We would build our own unchained commerce solution, decoupling payment processing from traditional platforms. This required a custom-built payment processing engine that could handle local African payment networks, mobile money, and other unique payment methods. It was a daunting task, but one that ultimately gave us control over the payment experience and removed the geographical restrictions that had been frustrating our users.

What The Numbers Said After

Following the implementation of our unchained commerce solution, we saw a significant increase in user engagement and conversion rates. Our payment process became seamless, and users were able to receive payments without the hassle of geographic restrictions. Our metrics showed a 30% increase in user retention, a 25% boost in average order value, and a decrease in customer support tickets by 40%. These numbers validated our decision to break free from traditional payment platforms.

What I Would Do Differently

In retrospect, I wish I had prioritized building our own unchained commerce solution from the beginning. This would have saved us months of frustration and countless engineering hours. However, I would also take a more nuanced approach to integrating multiple payment gateways. While it may not have been the best solution, it did expose us to the complexities of payment processing and the importance of catering to local payment networks. In the future, I would invest more time in researching local payment methods and exploring partnership opportunities with niche payment providers. After all, in the world of digital commerce, the most effective solutions are often those that tailor themselves to the unique needs of emerging markets.

AI agents don’t have a memory problem. They have an architecture problem.

.klickd cover

.klickd cover Every session,
the LLM starts fresh. The user re-explains their role, their constraints, their preferences, what they were doing last time. Then the session ends, and next time: same thing.

The industry has diagnosed this correctly — statelessness is a real limitation. But the solutions being built mostly share the same premise: that memory is a service you connect to. I think that premise is wrong, and it shapes everything downstream.

The actual cost of statelessness

This isn’t just a UX annoyance. A 2026 study by Pichay measuring 857 production AI sessions found that 21.8% of input tokens are “structural waste” — context that has to be re-established on every session because nothing persists. Nearly a quarter of your token budget, on every call, going toward re-explaining what should already be known.

For casual chat, that’s tolerable. For workflows where context is dense and high-stakes — a lawyer switching between matters, a developer moving between codebases, a clinician picking up a patient thread — the cost compounds. And it’s paid on every session, indefinitely.

What everyone else built

The market’s answer has been centralized memory stores. Mem0 just closed $24M in funding (October 2025) to build “the memory layer for AI.” Letta/MemGPT persists agent state in a server-side database. Zep builds a temporal knowledge graph of user interactions. SAMEP and MemTrust add encryption layers on top of server-side storage.

These are all genuinely useful tools. They solve the statelessness problem for most use cases. But they share an architecture: your context lives on their infrastructure, retrieval is query-scoped, and access is controlled by the service provider.

Even the solutions that advertise encryption — SAMEP, MemTrust — encrypt server-side. The data leaves the client before any cryptographic protection is applied. You’ve traded “AI forgets you” for “your memory is a managed cloud service.” For many applications that’s fine. For sensitive workflows, it’s a different risk surface, not a smaller one.

The question that didn’t get asked

What if memory is a file, not a service?

Not metaphorically. Literally: a single encrypted file, owned by the user, that travels with them across sessions and across models. The LLM reads it at session start, updates it at session end, and the file lives wherever the user puts it.

{
  "format": "klickd/v1",
  "encrypted_payload": "<AES-256-GCM ciphertext>",
  "kdf": "argon2id",
  "salt": "<per-file salt>",
  "nonce": "<GCM nonce>"
}

The key insight: persistent context doesn’t require a server. It requires a standard. A shared format that any model can read and any client can write.

What we built

We built .klickd around this premise. The architecture is deliberately minimal:

  • AES-256-GCM encryption, Argon2id key derivation. Client-side only. The key is derived from a passphrase that never leaves the device. There is no server that could be subpoenaed, breached, or decommissioned.
  • Provider-agnostic. The same .klickd file works with GPT-4o, Claude, Gemini, Llama. It’s not bound to any model provider’s infrastructure or format.
  • Zero-server. There is no backend storing context. The file is the memory. If the file doesn’t exist on your machine, the context doesn’t exist anywhere.

On personalization quality: our LLM-judge benchmark (Zenodo, DOI: 10.5281/zenodo.20320480) — run across 23 test lots and 115 profiles, using qwen3-32b as judge — showed an average improvement of +13.9 points over baseline, with a range of +12.8 to +19.2. This is with llama-3.3-70b-versatile as the model under test. Results are published as-is; methodology and raw data are in the report.

For legal and regulated workflows specifically: the file-per-context model makes cross-matter contamination structurally impossible — not enforced by query scoping or ACLs, but by physical separation. Discovery compliance changes shape: you produce the file, or you don’t. There’s no “server logs” ambiguity.

The honest tradeoffs

This architecture gives up things that matter in other contexts.

You lose:

  • Centralized governance and server-side revocation
  • Query analytics and usage telemetry
  • Multi-tenant management at scale
  • Cross-device sync without a separate sync layer

You gain:

  • Zero trust surface: there is nothing to breach on the provider side
  • GDPR-native by architecture: personal data doesn’t leave the client, so data residency and right-to-erasure are trivially satisfied
  • Portability: the file works with any model, now and in the future

This is not a universal solution. It is the right solution for a specific class of use cases: privacy-sensitive, cross-model, user-owned context. If you’re building a consumer product where the vendor needs to manage memory at scale, use Mem0 or Zep — they’re well-engineered for that. If you’re building for a context where the user owns the data and the service provider should have zero access, the server-side model is architecturally incompatible with that requirement, regardless of how good the encryption story is.

Is this a new standard?

The field probably needs a portable, encrypted, open context format the way it needed JWT for auth tokens or RSS for feed syndication — a shared abstraction that any tool can read and write, owned by no single vendor.

We’re not claiming .klickd is that standard. It’s a proof of concept that the abstraction is viable. The memory-file spec is open: https://github.com/Davincc77/klickdskill

The question I keep coming back to: if the AI ecosystem converged on server-side memory because that’s what was easy to build first, not because it’s the right primitive — what does the right primitive actually look like? And is the file abstraction the right level, or is there something better?

Curious what others think, especially those who’ve hit the limits of query-scoped retrieval in production.

Digital Exclusion Is a Technical Problem

The Problem We Were Actually Solving

Our platform relied on traditional payment gateways like Stripe and PayPal, which were easily integrated into our system but excluded a large portion of our users. In Kenya and Ethiopia, for example, M-Pesa is the most widely used mobile payment system, but it’s not supported by our current payment gateways. This meant that creators in these countries couldn’t receive payments for their digital products, making our platform inaccessible to them.

What We Tried First (And Why It Failed)

We tried to add support for M-Pesa by integrating a third-party API, but it was a disaster. The API was slow, unreliable, and had high latency, causing our platform to timeout and fail. We also encountered issues with currency conversion, which led to incorrect exchange rates and lost revenue for our creators. On top of that, the API was chargeable, which added unnecessary expense to our operations. We realized that relying on a third-party API wasn’t a viable solution.

The Architecture Decision

After weeks of research and experimentation, we decided to build our own payment gateway from scratch using the Unchained Commerce protocol. This protocol allowed us to create a decentralized payment system that supported multiple payment methods, including M-Pesa. We built a microservices architecture with separate components for payment processing, currency conversion, and reconciliation. This design allowed us to scale easily, reduce latency, and improve overall system reliability.

What The Numbers Said After

After deploying our new payment gateway, we saw a significant increase in creator adoption in Kenya and Ethiopia. Our platform’s payment success rate improved from 70% to 95%, and our revenue from these countries increased by 300%. Our creators were thrilled to have access to M-Pesa, and our platform’s user base expanded exponentially.

What I Would Do Differently

If I were to do it again, I would have invested more time in researching and understanding the nuances of the payment systems in each country. We assumed that M-Pesa was the only payment method needed in Kenya and Ethiopia, but there are other popular payment systems like Tingg and WorldRemit. I would have also considered partnering with local payment providers to ensure a seamless experience for our creators.

Our decision to build our own payment gateway was a costly and time-consuming one, but it paid off in the end. We learned that digital exclusion is a technical problem that requires a technical solution. By building a decentralized payment system that supports multiple payment methods, we were able to reach a wider audience and create a more inclusive platform.

Sustainable open source requires sustainable revenue. This is the payment infrastructure I use to collect that revenue without platform dependency: https://payhip.com/ref/dev9

Texas Just Opened a Privacy Investigation Into Meta’s AI Glasses — and It Raises a Question Every Camera App Has to Answer

On May 20, 2026, Texas Attorney General Ken Paxton opened a formal investigation into Meta’s AI Glasses — the Ray-Ban and Oakley camera-equipped eyewear — over how the device records, processes, and stores facial geometry and ambient video. Two days earlier, the ACLU and 75 civil-liberties organizations sent Meta an open letter warning that the company’s reported plan to ship facial recognition on the same glasses would “create serious risks for abuse victims, immigrants, LGBTQ+ people, protesters, [and] workers.” The Texas AG’s office described the central concern as the device’s documented “always enabled” mode, in which the glasses continuously process video for Meta’s AI products — with a recording-indicator LED that is “easily hidden” and “not active during the always-enabled mode.”

This isn’t a niche policy story. It’s the second time in less than two years Texas has taken Meta to court over the same underlying issue: capturing biometric data from people who never consented. The previous round ended in 2024 with a $1.4 billion settlement.

If you build, ship, or even use a camera-bearing piece of consumer technology, this is a moment to look at your own surface area and ask the question Paxton is asking Meta: where does the video go, who can see it, and did anyone agree to that?

Why this is a watershed and not a one-off

The legal text matters less than the precedent. A state AG just declared that:

  1. An “always enabled” recording mode, by itself, is a regulatable consumer-protection issue when the LED indicator isn’t reliably visible.
  2. Cloud-side processing of ambient video — even when framed as “AI features” — falls inside biometric-privacy law if facial geometry is derivable from it.
  3. The 2024 Meta settlement was not a one-time event; future biometric capture is on the table again.

That last point is the structural shift. Until this week, the operating assumption inside most consumer-hardware roadmaps was that a 2024-style consent banner and a small LED indicator were enough. Texas just signaled that they aren’t, at least when the device is wearable and the camera is on by default.

A wearable camera is not a unique animal. The question is whether the same logic applies to camera apps, dash cams, smart doorbells, and any other “always on” recording product. There is no obvious reason it wouldn’t.

The architecture question every recording product now has to answer

The Texas filing is really a stress-test of three architectural choices that any product with a camera can be measured against:

1. Is recording opt-in per session, or always-on by default?
Meta’s vulnerability here is “always enabled.” The opposite end of the spectrum is “the user explicitly starts each recording, and there is a persistent on-screen indicator that the camera is active.” The latter is the model a regulator can defend.

2. Where does the footage live — locally, in a vendor cloud, or fed into a model?
Cloud storage of camera footage isn’t illegal. But it inverts the burden of proof. Once the footage is in your vendor’s cloud, the vendor has to demonstrate it isn’t being used for anything the user didn’t consent to. Locally-stored footage carries no such burden — the user holds the only copy.

3. Is biometric inference happening, and if so, where?
On-device biometric inference (Face Unlock, Apple’s Vision Pro hand-tracking) is broadly tolerated. Off-device biometric inference of third parties captured in your camera frame is the actual line Texas is drawing. If your product processes video in a remote system, you need an affirmative answer for what is and isn’t being inferred from each frame, and what’s being retained.

These three questions — Opt-in or always-on? Local or cloud? Inference on-device or off? — now form the baseline a state AG appears willing to enforce.

What this looks like in a phone-based camera

We build Background Camera RemoteStream, an Android app that turns the phone you already own into a security/monitoring camera. The architectural choices we made years ago map directly onto the three questions above, and they look very different from the Meta Glasses model:

  • No always-on mode. The user opens the app and starts the recording. The app does not auto-record, does not auto-start at boot, and does not capture anything unless the user has explicitly initiated a session.
  • Local-only storage. Recorded video is written to the phone’s internal storage. There is no cloud upload, no off-device backup, and no vendor server holding a copy. We have no copy of your footage. We can’t have one — the architecture doesn’t support it.
  • No account, no tracking. The app does not require a Google or Meta or Super Funicular account. There’s no sign-in screen. There’s no analytics SDK shipping ambient telemetry. There’s no advertising ID being attached to recordings.
  • YouTube Live streaming is user-initiated. If you want to stream to YouTube Live, you set it up yourself, using your own YouTube account and your own RTMP key. The stream goes from your phone directly to YouTube. We’re not in the middle.
  • Screen-off recording is intentional, not covert. Recording continues when the screen is off because that’s a battery and stealth-of-the-phone-itself feature (the phone looks idle to anyone glancing at it). The camera LED on the phone behaves the way Android requires.

The point isn’t that we’re better than Meta. The point is that the Texas filing makes “where does the video go” a question consumers can now expect to ask, and a product that can answer it crisply is in a different category from one that can’t.

What developers should take from this

If you build anything that captures video, audio, or biometric-derivable data, three things are worth doing this week:

  1. Audit your default state. Is recording opt-in or always-on? If always-on, what’s the affirmative justification, and is the user indicator reliable and unconcealable?
  2. Audit your storage boundary. Trace one frame of recorded video through your system. Where does it go, how long does it live, who has access, and what models touch it?
  3. Audit your inference surface. What is your product deriving from camera input that the user didn’t explicitly ask you to derive? Whatever that list is, decide whether it survives the question “what if the Texas AG asks.”

The Meta Glasses investigation will take months to play out, and the legal outcome isn’t the interesting part. The interesting part is that the architectural questions are now public. Camera products that can answer them honestly are going to find that the conversation has shifted in their favor.

If you want to see what a “no cloud, no account, no tracking” Android camera looks like in practice, Background Camera RemoteStream is the one we ship. Source on our positioning and how we think about privacy lives at superfunicular.com.

5 things Railway’s 8 hour outage should change about how you think about redundancy

Railway runs on Google Cloud, AWS, and its own metal.

So when I first saw that Railway was down for hours, my first thought was probably the same as yours.

“How does a multi cloud platform go dark like that?”

Then I read the incident report, the Hacker News discussion, and the follow up coverage. And the real lesson is uncomfortable.

This was not really a cloud outage.

The servers did not all die. AWS did not die. Railway Metal did not die. Google Cloud infrastructure itself did not have to collapse.

What failed was much higher up the stack.

The account.

Google Cloud placed Railway’s production account into suspended status incorrectly as part of an automated action. Railway says this happened around 22:20 UTC on May 19, and the platform was not fully recovered until the next morning. (https://blog.railway.com/p/incident-report-may-19-2026-gcp-account-outage)

That should make every CloudOps, platform, SRE, and engineering leader stop for a minute.

Because most redundancy plans are built for the wrong failure.

We design for dead VMs.
We design for unavailable zones.
We design for regional failover.
We design for database replicas.

But what do we do when the provider says, incorrectly or automatically, “your account is no longer allowed to exist normally”?

Not much, usually.

1. This was not a cloud outage. It was an account suspension

That is the first big lesson.

A lot of people hear “cloud outage” and instantly think of regions, zones, load balancers, or broken hardware. But Railway’s case was different.

Google Cloud’s automated systems suspended Railway’s production account. Railway says this was incorrect, and that the action was part of a wider automated event affecting many accounts. (https://blog.railway.com/p/incident-report-may-19-2026-gcp-account-outage)

That kind of failure does not look like a server going unhealthy.

It looks like identity, billing, trust, abuse detection, policy, support, and account control all becoming part of your availability story.

Your health checks can say everything is fine.

Your multi zone architecture can be green.

Your workloads can still technically exist.

But if the account is restricted, your beautiful infrastructure diagram does not matter much.

This is the part many teams do not model.

They model “what if eu west 1 is down?”

They rarely model “what if our production cloud account is frozen by an automated system at 11 PM?”

And honestly, that second one is scarier.

Because you do not debug it with kubectl.

You debug it with support tickets, escalation paths, account managers, legal trust, and luck.

2. The control plane was the real single point of failure

Railway had workloads on AWS and Railway Metal that were still running during the incident. But users still saw errors.

Why?

Because the routing control plane was hosted on Google Cloud.

Railway’s edge proxies needed that control plane to know where workloads lived. They had cached route data for a while, but once the cache expired, the edge could not keep routing properly. Railway’s community update said route cache expiry caused the incident to spread beyond GCP hosted workloads and affect the wider platform. (https://station.railway.com/community/what-we-know-so-far-may-19th-2026-86354cdd)

This is the second lesson.

Your data plane can be redundant while your control plane is still fragile.

And this is where a lot of “multi cloud” thinking becomes a little fake.

You can run compute in three places.
You can run storage in two places.
You can have Kubernetes clusters everywhere.

But if the scheduler, routing map, identity service, deployment API, config database, or certificate automation lives in one provider, your multi cloud story may only be multi cloud on paper.

The thing customers see as “the product” is often not the workload.

It is the control plane around the workload.

For Railway, customers were not just buying raw compute. They were buying routing, builds, deployments, dashboard access, APIs, orchestration and platform magic.

And the platform magic had a dependency.

That dependency became the outage.

3. Getting the account back is not the same as getting the service back

This one is very important.

According to Railway, Google reversed the suspension shortly after escalation. But recovery still took hours because account restoration did not automatically bring everything back cleanly. Persistent disks, compute instances, networking and orchestration layers had to be restored and verified step by step. (https://blog.railway.com/p/incident-report-may-19-2026-gcp-account-outage)

This is the part people underestimate.

A provider can say, “access restored.”

But your system still has to wake up.

Disks need to attach.
Networks need to behave.
Queues need to drain.
Deployments need to stop stampeding.
Databases need to agree again.
Caches need to be repopulated.
Humans need to verify what is safe.

That is not instant.

And in a complex platform, bringing things back too fast can be worse than bringing them back slowly.

Railway also throttled queued deploys during recovery, which sounds boring, but it is actually the responsible move. Because after an outage, your own backlog becomes traffic. And that traffic can flatten the recovering system.

So the real RTO is not:

“How fast can the provider undo the mistake?”

It is:

“How fast can we safely restore the whole chain after the provider undo the mistake?”

Small difference in words.

Huge difference in reality.

4. Recovery can create a second outage

This is probably my favorite lesson from the whole incident, because it is so real.

When Railway started recovering, queued retries and user activity came back in a burst. That burst hit GitHub OAuth and webhook flows hard enough that GitHub rate limited Railway. So logins and builds had problems again, even after the original Google Cloud issue was no longer the main blocker. (https://blog.railway.com/p/incident-report-may-19-2026-gcp-account-outage)

That is painful.

The first outage came from one provider.

The second problem appeared during recovery, from another dependency.

This happens more often than teams admit.

After an outage, everything tries to catch up.

Cron jobs wake up.
Webhooks retry.
CI pipelines restart.
Users refresh dashboards.
Workers pull old messages.
Integrations suddenly see a wall of traffic.

And then some other system says, “this looks abusive.”

Now your recovery has become its own incident.

This is why serious resilience is not just failover.

It is controlled recovery.

Backpressure matters.
Retry budgets matter.
Queue draining matters.
Circuit breakers matter.
Rate limit awareness matters.
Runbooks matter.

And boring old institutional memory matters even more.

Railway had already hardened parts of the GitHub rate limit path after a prior incident, which helped reduce damage this time. That is not luck. That is the value of learning properly from past pain.

5. Most teams insure the wrong half of the risk

The Railway incident is not the first time account level cloud risk became real.

In 2024, UniSuper, a major Australian pension fund, had a serious Google Cloud incident where its private cloud environment was deleted because of a misconfiguration. Google later published details saying backups in Google Cloud Storage and third party backup software helped restoration. (https://cloud.google.com/blog/products/infrastructure/details-of-google-cloud-gcve-incident)

So no, account level and provider control plane risk is not some imaginary edge case.

It happens.

But most companies still talk about redundancy like this:

“We use multiple clouds.”

Ok, but what does that mean?

Does it mean workloads can run somewhere else?

Or does it mean you can actually operate the business if one provider account disappears?

Those are very different things.

Flexera’s 2026 State of the Cloud report shows multi cloud is still a major enterprise pattern, and its report is based on 753 cloud decision makers. (https://info.flexera.com/CM-REPORT-State-of-the-Cloud?lead_source=Organic+Search) But in practice, many companies are multi cloud for procurement, politics, analytics, or workload placement.

Not always for true survivability.

True survivability asks much harder questions.

Can we deploy without this provider?
Can we route without this provider?
Can we authenticate without this provider?
Can we restore backups without this provider?
Can we contact support fast enough?
Can we prove ownership if an automated trust system flags us?
Can we keep serving read only traffic if the control plane dies?
Can we rebuild from another account, another org, or another provider?

That is not as sexy as “active active multi cloud.”

But it is probably more useful.

The real takeaway

Railway did have redundancy.

Just not for the layer that failed.

And that is the uncomfortable lesson for the rest of us.

Redundancy at the compute layer does not protect you from account suspension.

Multi region databases do not protect you from provider level identity actions.

Healthy servers do not help when routing control planes cannot tell traffic where to go.

And getting your cloud account back does not mean your service is back.

The next resilience review should not only ask:

“What happens if a region dies?”

It should also ask:

“What happens if our cloud provider suspends our production account by mistake tonight?”

I Inherited 47,000 Lines of Terraform Spaghetti — Here’s How I Untangled It Without Burning Production

sanjaysundarmurthy image

The Slack Message That Ruined My Monday

“Hey, the previous platform team left. Here’s the repo. Good luck 🫡”

I stared at the Git repository. 47,000 lines of Terraform. One state file. Zero modules. Variables named x, temp2, and my personal favorite — DO_NOT_TOUCH_ask_raj. Raj had left the company two years ago.

If you’ve been a Senior DevOps Engineer for more than a year, you’ve inherited something like this. Maybe not 47K lines, but you’ve opened a main.tf that made you question your career choices.

This isn’t a “Terraform best practices” article. Those are written by people who’ve never had to run terraform plan on a 3,000-resource state file at 2 AM while the VP of Engineering watches.

This is a survival guide.

Anti-Pattern #1: The Monolith State File (aka “The Single Point of Career Failure”)

What I Found

# main.tf — 8,400 lines
# "Managed" networking, compute, databases, DNS, IAM, monitoring,
# and somehow... a CloudFront distribution for a marketing site
# that was decommissioned in 2023.

resource "aws_vpc" "main" { ... }
resource "aws_instance" "api_server_1" { ... }
resource "aws_instance" "api_server_2" { ... }
# ... 200 more instances ...
resource "aws_rds_instance" "prod_db" { ... }
resource "aws_iam_role" "god_mode" { ... }  # yes, really

A single terraform apply touched everything. Networking, databases, compute, DNS — all entangled like Christmas lights in January. One typo in a security group rule? Congratulations, your plan just showed 847 resources to evaluate, and Terraform decided your RDS instance needs replacing.

The Real Danger

This isn’t just messy — it’s operationally catastrophic. Here’s what happens:

  • terraform plan takes 14 minutes. Developers stop running it.
  • State file locking means only one person can work at a time.
  • Blast radius of any mistake = the entire infrastructure.
  • New team members are terrified to touch anything (rightfully so).

How I Fixed It (Without Downtime)

Step 1: State Surgery with terraform state mv

# First, I mapped resource dependencies visually
terraform graph | dot -Tsvg > infra-dependency-map.svg

# Then, split by domain boundaries
terraform state mv 'aws_vpc.main' -state-out=networking/terraform.tfstate
terraform state mv 'aws_subnet.public[0]' -state-out=networking/terraform.tfstate
terraform state mv 'aws_subnet.public[1]' -state-out=networking/terraform.tfstate

Step 2: Introduce State Boundaries by Blast Radius

I split into five state files based on change frequency and blast radius:

Layer Contents Change Frequency Blast Radius
foundation VPC, Subnets, Route Tables Monthly Critical
security IAM, KMS, Security Groups Weekly Critical
data RDS, ElastiCache, S3 Rare Catastrophic
compute ECS/EKS, ASGs, ALBs Daily High
edge CloudFront, Route53, WAF Weekly Medium

Step 3: Wire Them Together with Remote State Data Sources

# In compute/main.tf
data "terraform_remote_state" "networking" {
  backend = "s3"
  config = {
    bucket = "company-terraform-state"
    key    = "foundation/terraform.tfstate"
    region = "us-east-1"
  }
}

resource "aws_ecs_service" "api" {
  # Reference networking outputs safely
  network_configuration {
    subnets = data.terraform_remote_state.networking.outputs.private_subnet_ids
  }
}

Result: terraform plan went from 14 minutes to 45 seconds. Team velocity tripled. I stopped getting 2 AM pages about state locks.

Anti-Pattern #2: The Copy-Paste Empire (aka “Modules at Home”)

What I Found

environments/
├── dev/
│   └── main.tf      # 1,200 lines
├── staging/
│   └── main.tf      # 1,200 lines (95% identical to dev)
├── prod/
│   └── main.tf      # 1,200 lines (90% identical... with 47 "hotfixes")
└── dr/
    └── main.tf      # 1,200 lines (copied from prod 8 months ago, never updated)

Four copies of the same infrastructure with subtle drift. Staging had a security group rule that prod didn’t. DR was missing three services entirely. Nobody knew which differences were intentional.

Why This Kills Senior Engineers

You can’t diff your way out of this. The files have diverged in ways that are both intentional (prod has larger instances) and accidental (someone fixed a bug in dev but forgot to propagate it). You have no source of truth.

The Refactoring Strategy That Actually Works

Don’t try to unify everything at once. I learned this the hard way after a failed “big bang” refactor that took 3 sprints and broke staging for a week.

Instead, use the Strangler Fig pattern:

# modules/api-platform/main.tf
variable "environment" {
  type = string
  validation {
    condition     = contains(["dev", "staging", "prod", "dr"], var.environment)
    error_message = "Environment must be dev, staging, prod, or dr."
  }
}

variable "config" {
  type = object({
    instance_type    = string
    min_capacity     = number
    max_capacity     = number
    enable_waf       = bool
    multi_az         = bool
    backup_retention = number
  })
}

locals {
  # Environment-specific defaults that document WHY they differ
  env_config = {
    dev = {
      instance_type    = "t3.medium"
      min_capacity     = 1
      max_capacity     = 2
      enable_waf       = false
      multi_az         = false
      backup_retention = 1
    }
    prod = {
      instance_type    = "m5.xlarge"
      min_capacity     = 3
      max_capacity     = 20
      enable_waf       = true
      multi_az         = true
      backup_retention = 35
    }
  }
}

The key insight: Every environment difference should be documented in code as a conscious decision, not hidden in a 1,200-line file as an accidental divergence.

Anti-Pattern #3: The terraform apply -auto-approve YOLO Pipeline

What I Found in .gitlab-ci.yml

deploy_prod:
  stage: deploy
  script:
    - terraform init
    - terraform apply -auto-approve  # 🚨 WHAT
  only:
    - main

No plan artifact. No approval gate. No diff review. Push to main → infrastructure changes in production. The commit history told the horror story:

fix: revert the revert of the fix
fix: actually fix prod this time
fix: ok THIS one fixes it
revert: revert everything from today

What Senior Engineers Actually Need

# .github/workflows/terraform.yml
name: "Terraform"

on:
  pull_request:
    paths: ['infrastructure/**']
  push:
    branches: [main]
    paths: ['infrastructure/**']

jobs:
  plan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Terraform Plan
        id: plan
        run: |
          terraform init
          terraform plan -no-color -out=tfplan 
            -detailed-exitcode 2>&1 | tee plan_output.txt
        continue-on-error: true

      - name: Comment Plan on PR
        uses: actions/github-script@v7
        if: github.event_name == 'pull_request'
        with:
          script: |
            const fs = require('fs');
            const plan = fs.readFileSync('plan_output.txt', 'utf8');
            const truncated = plan.length > 60000 
              ? plan.substring(0, 60000) + 'nn... truncated ...' 
              : plan;
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: `## Terraform Plan Outputn```n${truncated}n````
            });

      - name: Upload Plan Artifact
        uses: actions/upload-artifact@v4
        with:
          name: tfplan
          path: tfplan

  apply:
    needs: plan
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    environment: production  # Requires manual approval
    steps:
      - uses: actions/checkout@v4

      - name: Download Plan
        uses: actions/download-artifact@v4
        with:
          name: tfplan

      - name: Terraform Apply
        run: terraform apply tfplan  # Apply ONLY the reviewed plan

The non-negotiable rules:

  1. Plans are generated on PR and attached as artifacts.
  2. Humans review the diff before any production apply.
  3. Apply uses the exact plan that was reviewed (not a new plan).
  4. The production environment requires manual approval from a senior engineer.

Anti-Pattern #4: Secrets in State (The Ticking Compliance Bomb)

What I Found

resource "aws_db_instance" "prod" {
  engine               = "postgres"
  instance_class       = "db.r5.2xlarge"
  username             = "admin"
  password             = "Pr0d_P@ssw0rd_2022!"  # I wish I was joking
  publicly_accessible  = true                    # I really wish I was joking
}

The password was in the .tf file, the state file, the plan output, and the Git history. Four places to leak from. And publicly_accessible = true was the cherry on this dumpster fire sundae.

The Fix (That Also Passes Audit)

# Use a data source to pull secrets at plan/apply time
data "aws_secretsmanager_secret_version" "db_password" {
  secret_id = "prod/rds/master-password"
}

resource "aws_db_instance" "prod" {
  engine              = "postgres"
  instance_class      = "db.r5.2xlarge"
  username            = "admin"
  password            = data.aws_secretsmanager_secret_version.db_password.secret_string
  publicly_accessible = false

  # Prevent Terraform from detecting password "drift"
  lifecycle {
    ignore_changes = [password]
  }
}

But that’s not enough. The state file still contains sensitive values. The complete solution:

# backend.tf
terraform {
  backend "s3" {
    bucket         = "company-terraform-state"
    key            = "prod/data/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true                          # SSE-KMS encryption
    kms_key_id     = "arn:aws:kms:us-east-1:xxx:key/yyy"
    dynamodb_table = "terraform-state-lock"
  }
}

Plus strict S3 bucket policies, access logging, and never giving developers direct state file access. Use terraform output instead.

Anti-Pattern #5: The “God Resource” With 200 Lines of Nested Blocks

What I Found

resource "aws_ecs_task_definition" "api" {
  family                   = "api"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = 1024
  memory                   = 2048
  execution_role_arn       = aws_iam_role.ecs_execution.arn
  task_role_arn            = aws_iam_role.ecs_task.arn

  container_definitions = jsonencode([
    {
      name  = "api"
      image = "company/api:latest"  # 🚨 LATEST TAG IN PROD
      portMappings = [{ containerPort = 8080 }]
      environment = [
        { name = "DB_HOST", value = "prod-db.cluster-xxx.us-east-1.rds.amazonaws.com" },
        { name = "DB_NAME", value = "production" },
        { name = "REDIS_URL", value = "prod-redis.xxx.cache.amazonaws.com:6379" },
        # ... 45 more environment variables hardcoded here ...
      ]
      logConfiguration = {
        logDriver = "awslogs"
        options = {
          "awslogs-group"         = "/ecs/api"
          "awslogs-region"        = "us-east-1"
          "awslogs-stream-prefix" = "api"
        }
      }
      # ... 80 more lines of health checks, mount points, ulimits ...
    }
  ])
}

The problems compound:

  • Environment variables are hardcoded (not sourced from SSM/Secrets Manager).
  • latest tag means deployments are non-reproducible.
  • The jsonencode blob is untestable and un-diffable in PR reviews.
  • One change to any env var triggers a full task definition replacement.

The Refactored Version

# Use templatefile for complex JSON — it's testable and readable
resource "aws_ecs_task_definition" "api" {
  family                   = "api-${var.environment}"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = var.task_cpu
  memory                   = var.task_memory
  execution_role_arn       = aws_iam_role.ecs_execution.arn
  task_role_arn            = aws_iam_role.ecs_task.arn

  container_definitions = templatefile("${path.module}/templates/api-container.json.tpl", {
    image_tag     = var.image_tag  # Pinned, passed from CI/CD
    environment   = var.environment
    db_host       = data.aws_ssm_parameter.db_host.value
    redis_url     = data.aws_ssm_parameter.redis_url.value
    log_group     = aws_cloudwatch_log_group.api.name
    aws_region    = data.aws_region.current.name
  })
}

The Refactoring Playbook (Do This Monday)

After untangling this mess across three months, here’s the sequence that works:

Week 1: Triage and Protect

# 1. Enable state file encryption and locking NOW
# 2. Add branch protection — no direct pushes to main
# 3. Run terraform plan and SAVE the output as your baseline
terraform plan -no-color > baseline_plan_$(date +%Y%m%d).txt

# 4. Enable detailed audit logging on your state bucket

Week 2-4: Split the Monolith

# Use terraform state list to inventory everything
terraform state list > all_resources.txt
wc -l all_resources.txt  # Mine had 2,847 resources

# Group by service domain
grep "aws_vpc|aws_subnet|aws_route" all_resources.txt > networking.txt
grep "aws_iam|aws_kms" all_resources.txt > security.txt
grep "aws_rds|aws_elasticache|aws_s3" all_resources.txt > data.txt
grep "aws_ecs|aws_alb|aws_autoscaling" all_resources.txt > compute.txt

Week 5-8: Modularize Incrementally

Move one service at a time into a module. After each move:

  1. Run terraform plan — it should show zero changes.
  2. If plan shows changes, you have a bug. Fix it before moving on.
  3. Get a PR review from another senior engineer.
  4. Apply and monitor for 24 hours.

Week 9-12: Harden the Pipeline

  • Add terraform validate and tflint to CI.
  • Add checkov or tfsec for security scanning.
  • Implement drift detection (scheduled plan that alerts on differences).
  • Add cost estimation with infracost.

The Drift Detection Cron That Saved Us

This is the thing nobody talks about. Even after a perfect refactor, drift happens. Someone clicks in the console. An auto-remediation tool makes changes. A Lambda modifies a security group.

# .github/workflows/drift-detection.yml
name: "Drift Detection"

on:
  schedule:
    - cron: '0 6 * * 1-5'  # Every weekday at 6 AM

jobs:
  detect-drift:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        layer: [foundation, security, data, compute, edge]
    steps:
      - uses: actions/checkout@v4

      - name: Terraform Plan (Drift Check)
        id: plan
        working-directory: infrastructure/${{ matrix.layer }}
        run: |
          terraform init
          terraform plan -detailed-exitcode -no-color > plan.txt 2>&1
          echo "exitcode=$?" >> $GITHUB_OUTPUT
        continue-on-error: true

      - name: Alert on Drift
        if: steps.plan.outputs.exitcode == '2'
        run: |
          # Exit code 2 = changes detected (drift!)
          curl -X POST "${{ secrets.SLACK_WEBHOOK }}" 
            -H 'Content-type: application/json' 
            -d "{"text":"🚨 Drift detected in *${{ matrix.layer }}* layer. Check the plan output."}"

We caught 3 unauthorized console changes in the first week alone.

Parting Wisdom for the Senior Engineer Who Just Inherited a Mess

  1. Don’t refactor everything at once. You’ll break things and lose credibility.

  2. Document what you find before you fix it. Screenshot the horrors. You’ll need them for the post-mortem and for your performance review.

  3. Get buy-in from leadership BEFORE you start. “I need 3 sprints for tech debt” is a hard sell. “Our current setup means any infrastructure change has a 40% chance of causing an incident” gets budget approved.

  4. Every terraform state mv should be a separate, reviewed PR. Not because it’s technically necessary, but because when something breaks at step 37 of 50, you want a clean git history to bisect.

  5. The goal isn’t perfect Terraform. The goal is Terraform that your team can safely operate at 2 AM. If a junior engineer can’t run terraform plan without fear, your refactor isn’t done.

TL;DR for the Scrollers

Anti-Pattern Fix Priority
Monolith state file Split by blast radius and change frequency P0
Copy-paste environments Modules + environment configs P1
-auto-approve in CI Plan artifacts + manual approval gates P0
Secrets in state/code Secrets Manager + encrypted state + ignore_changes P0
God resources with inline JSON templatefile + SSM parameters P2
No drift detection Scheduled plan with alerting P1

If you’ve ever stared at a Terraform codebase and whispered “who did this?!” into the void — you’re not alone. We’ve all been there. The good news? It’s fixable. One state move at a time.

Found this useful? Follow me for more battle-tested DevOps content. I write about the stuff that actually happens in production — not the happy path from the docs.

How I Turned My Terminal Into an Aesthetic Experience

I’ve always been addicted to staring at a screen all day, writing — whether it’s code or a story. But this passion comes with a cost, at least for me. I struggle with eye strain and painful migraines, and it genuinely hurts to use these devices, no matter how many “protective layers” I add. By that, I mean things like enabling dark mode, blocking blue light from screens, and wearing anti-reflective glasses. They help, sure — but not enough. A fraction of the discomfort always remains.

And it gets even worse WHEN WORKING IN THE TERMINAL.

All that plain white output gets me dizzy. It forces me to take frequent breaks throughout the day. I just can’t stand looking at walls of unstyled, uncolored text. Ironically, it ends up being distracting.

Of course, I’ve tried to fix this. I added an Oh My Zsh syntax-highlighting plugin, switched to most as a pager for man pages, and installed tools like bat, eza, and grc.

And yes — it helped.

My terminal looks better now, and spending time in it has become easier.

But still… some commands remain stubbornly plain.

Maybe this sounds like overreacting — and maybe you’re right — but I just want a fully colored terminal experience. You can’t blame me for that, right?

That’s why I built this function, colored-help, which automatically colorizes help output to improve readability via bat. At first, I implemented it with a Zsh preexec hook, but I ran into strange issues: commands wouldn’t exit cleanly with q or Ctrl+C, and — help outputs would linger on the screen. So, I turned to my beloved ChatGPT, which introduced me to ZLE widgets — and that changed everything.

Terms explained

bat — A modern replacement for cat with syntax highlighting, Git integration, and prettier output formatting.

eza — A modern alternative to ls with colors, icons, tree views, and improved file metadata display.

most — A terminal pager similar to less, but with syntax highlighting and support for multiple windows.

grc — A tool that adds color and formatting to the output of many common command-line programs.

ZLE (Zsh Line Editor) — The interactive command-line editor built into Zsh. It allows you to create custom widgets, keybindings, and shell behaviors.

preexec hook — A Zsh hook function that runs automatically right before a command executes.

Why am I sharing this?

Because I’d really appreciate your feedback. What do you think of my approach? Is there a better way to implement this? I’m also thinking of extending it. Are there other tools or “10x hacks” I might be missing?

Please don’t hesitate to share your thoughts — I’d genuinely love to hear them.

Synchronous Functions in Dart

This series of post will try to explain a complex topic: concurrent and parallel programming, in Dart. I think the only way to deal with that is using the Erlang VM (BEAM), but Clojure and other functional languages are usually doing better job on this part. Unfortunately, to me, most of other languages using OOP don’t offer a great abstraction to concurrency and parallelism, but during the last decade, things are changing a bit.

Anyway, before talking about concurrency, parallelism, or even distributed systems, everything starts with synchronous call, the “standard” way to execute a function or a procedure on a computer. In dart, synchronous function call and generators functions need to me studied a little bit.

Firstly, all function without any specific keyword are all defined as synchronous:

int addition(int x, int y) {
  return x + y;
}

If the keyword sync was present in the language, the previous snippet would be equivalent to the one below. Don’t try to compile it, it will fail.

// could be equivalent to the following
// snippet with the sync keyword. But this
// function will not work, sync keyword is
// invalid during compilation
int addition_sync(int x, int y) sync {
  return x + y;
}

When adding the addition() function inside the main() entry-point, it will directly return the value. In fact, you can’t really see this because it’s quick, but the computer will block a little bit to wait the result of the computation.

int main() {
  return addition(1, 2);
}

A synchronous function will always wait for any kind of return before continuing the execution of the program. For example, if an infinite loop is created, the function will never return, and the function will block the whole program. The following snippet, when executed, will never return because of the while loop defined in the infinite_loop() function. The only way to close this program is to kill it (for example using Ctrl-C or sending another signal on UNIX systems).

int infinite_loop() {
  while (true) {
    // do something
  }
  return 0;
}

int main() {
  return infinite_loop();
}

Great, we know how to create synchronous functions. Now, let have a look on a specific keyword: sync*. This one is a syntactic sugar to help developers creating generators, a function that return many elements, one at a time. The best example is the List class (or []). This data type – a list – is seen as a generator, and more precisely an Iterable.

A function is a generator if its body is marked with the sync* or async* modifier. […] Generator functions are a sugar for functions that produce collections in a systematic way, by lazily applying a function that generates individual elements of a collection. Dart provides such a sugar in both the synchronous case, where one returns an iterable, and in the asynchronous case, where one returns a stream.

— Dart Language Specification, Chapter 9, page 23

Let create our first generator from scratch, let call it int_generator(). It will take 2 integers as arguments and will return the list of elements between them. You can see it as a kind of primitive range or sequence function.

Iterable<int> int_generator(int x, int y) sync* {
  if (y<x) throw "y<x"; 
  while (x <= y) {
    yield x;
    x++;
  }
}

int main() {
  for (var i in int_generator(0, 10)) {
    print(i);
  }
  return 0;
}

In the first place, instead of returning a simple int this function is returning an Iterable<int>. Why? Because we are using yield, but we will explain that a bit later. Next, the function definition looks like a classic one, but the sync* keyword is added just after. It means this function will be a generator.

Then, the body of the function where a first step is to avoid creating an infinite loop by creating a simple guard throwing an exception if the second argument is less than the first argument. Immediately after, a while loop is created, where the first argument is incremented until the first argument is less or equal to the second one.

The yield statement adds an object to the result of a generator function.

— Dart Language Specification, Chapter 18, Section 16, page 213

In this loop, the keyword yield followed by variable (or any valid terms) is used. This specific expression is important, because it will adds the object into the result of the generator. In other words, if another function is calling the previous function inside a loop, the sequence of integer will be printed to the screen.

Let execute that from the console.

$ dart run bin/int_generator.dart
0
1
2
3
4
5
6
7
8
9
10

yield is not the only unique term here able to put an object inside a generator, yield* (yield-each) can also be used to push a series of objects, in fact a subtype of Iterable class.

The yield-each statement adds a series of objects to the result of a generator function.

— Dart Language Specification, Chapter 18, Section 17, page 214

Iterable<int> int_generator(int x, int y) sync* {
  if (y<x) throw "y<x"; 
  yield* [x,y];
}

int main() {
  print(int_generator(0, 10));
  for (var i in int_generator(0, 10)) {
    print(i);
  }
  return 0;
}

Let execute this code.

$ dart run bin/int_each_generator.dart 
(0, 10)
0
10

To be honest, I don’t currently understand why it returns a record on the first part of the program. I was reading the specification but I was unable to find and answer. I will let this problem for future-me. A quick note though: a generator cannot return a value.

Case ⟨Generator functions⟩. It is a compile-time error if a return statement of the form return e; appears in a generator function.

— Dart Language Specification, Chapter 18, Section 12, page 211

Based on the documentation, it also seems yield* can be used to improve recursive function performance. Here the one of the previous example implemented using recursive function call.

Iterable<int> int_rec_generator(int x, int y) sync* {
  if (x<=y) {
    yield x;
    yield* int_rec_generator(x+1, y);
  }
}

int main() {
  print(int_rec_generator(0, 10));
  return 0;
}

Let run it now.

$ dart run bin/int_recursive_generator.dart 
(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)

Same result than before, but it was using a recursive function. If I remember correctly, calling recursively a function in Dart with tail queue optimization and trampoline is a very bad idea (it will consume a lot of stack and memory). Anyway, let switch to another topic: Iterable class.

Iterable class

We know how to create generators, and then Iterable from scratch, but what we can do with this kind of objects? Let read the definition from the API documentation.

A collection of values, or “elements”, that can be accessed sequentially.

— Dart Language Documentation, Iterable class

Not all features will be illustrated there, only the one looking interesting and useful for my own use cases.

  • Iterate.generate() constructor creates a new Iterable object using Integer by default (like a Range/Sequence). The second argument offers a way to change the type returned based on the integer sequence produced (a bit like a map()).
int main() {
  var x = Iterable.generate(10);
  print("generate(10): $x");

  var y = Iterable.generate(26, (int x) => String.fromCharCode(x+65));
  print("generate(10, _): $y");

  return 0;
}
$ dart run bin/t.dart 
generate(10): (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
generate(10, _): (A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z)
  • Iterable.contains() method returns true if the object passed in the first argument is present in the collection of objects managed by the Iterable.
$ cat bin/contains.dart 
void main() {
  var a = Iterable.generate(26, (int x) => String.fromCharCode(x+65));
  print('"A"? ${a.contains("A")}');
  print('65? ${a.contains(65)}');
}
$ dart bin/contains.dart 
"A"? true
65? false
  • Iterable.elementAt() method returns the element present at the position defined by the first argument. If the position does not exist, it throws a RangeError exception.
$ cat bin/elementAt.dart 
void main() {
  var x = Iterable.generate(10);
  print(x.elementAt(0));
  print(x.elementAt(9));
  print(x.elementAt(1024));
}
$ dart bin/elementAt.dart 
0
9
Unhandled exception:
RangeError (index): Index out of range: index should be less than 10: 1024
#0      IndexError.check (dart:core/errors.dart:520:7)
#1      _GeneratorIterable.elementAt (dart:core/iterable.dart:928:16)
#2      main (file:///dart/t/bin/elementAt.dart:5:11)
#3      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:314:19)
#4      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:193:12)
  • Iterable.every() method returns true if the collection of items satisfies the anonymous function passed in argument.
$ cat bin/every.dart 
void main() {
  var x = Iterable.generate(10);
  print(x.every((y) => y >= 0 ));
  print(x.every((y) => y == String ));
}
$ dart bin/every.dart 
true
false
  • Iterable.fold() method reduces the collection of items using a combine function present in second argument and an initial value defined in the first argument.
void main() {
  Iterable<int> i = Iterable.generate(10);
  print(i.fold<int>(1, (v, e) => v + e));

  var s = Iterable.generate(26, (int x) => String.fromCharCode(x+65));
  print(s.fold<String>("string: ", (v, e) => "${v} ${e}"));
}
$ dart bin/fold.dart 
46
string:  A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
  • Iterable.forEach() method iterates of the collection of item. This method returns nothing.
void main() {
  var list = [1,2,3,4, "end"];
  list.forEach((x) => print(x));
}
$ dart bin/forEach.dart 
1
2
3
4
end
  • Iterable.join() method joins the collection of item with a String and converts all object in the collection t String.
void main() {
  var list = ["start", 1, 2, 3, "end"];
  print(list.join(", "));
}
$ dart bin/join.dart 
start, 1, 2, 3, end
  • Iterable.map() method modifies each element of the collection by calling a function defined in the first argument.
void main() {
  var m = [
    {'a': 1},
    {'b': 2}
  ];
  print(
    m.map(
      (x) => {x.values.first: x.keys.first}
    )
  );
}
$ dart bin/map.dart 
({1: a}, {2: b})
  •  Iterable.reduce() methods is similar to Iterable.fold() excepts it does not have an initial value defined.
void main() {
  Iterable<int> list = Iterable.generate(20);
  print(list.reduce((a, e) {
    print("$a $e");
    if (e%2==0) {
      return a + e;
    }
    else {
      return a;
    }
  }));
}
$ dart bin/reduce.dart 
0 1
0 2
2 3
2 4
6 5
6 6
12 7
12 8
20 9
20 10
30 11
30 12
42 13
42 14
56 15
56 16
72 17
72 18
90 19
90

Way more methods are available, but it was not expected to go so deep in this post. If you want to go even deeper, you will need to read the test cases and the Dart implementation:

With these 2 links, you will have more than enough to understand how things are working.

Cover Image by Will H McMahan on Unsplash