What happens when your payment processing crashes mid-transaction? With regular code — data loss. With Temporal — it resumes exactly where it left off. Temporal makes your workflows durable, reliable, and resumable by default.

What Temporal Gives You for Free

• Durable execution — workflows survive crashes, restarts, and deployments
• Automatic retries — configurable retry policies per activity
• Long-running workflows — days, weeks, months
• Timers — sleep for hours/days without holding resources
• Versioning — update workflow code without breaking running instances
• Visibility — web UI showing all workflow states
• TypeScript SDK — fully typed, great DX
• Self-hosted — Docker Compose or Kubernetes

Quick Start

# Run Temporal server
docker compose up -d

# Create project
npx @temporalio/create@latest my-app
cd my-app
npm run start

Define a Workflow

// workflows.ts
import { proxyActivities, sleep } from '@temporalio/workflow';
import type * as activities from './activities';

const { sendEmail, chargePayment, provisionAccount } = proxyActivities<typeof activities>({
  startToCloseTimeout: '30 seconds',
  retry: { maximumAttempts: 3 }
});

export async function onboardUser(email: string, plan: string): Promise<void> {
  // Step 1: Send welcome email
  await sendEmail(email, 'Welcome!');

  // Step 2: Charge payment
  const paymentId = await chargePayment(email, plan);

  // Step 3: Provision account
  await provisionAccount(email, plan);

  // Step 4: Wait 3 days, then send tips
  await sleep('3 days'); // Durable! Survives server restarts.
  await sendEmail(email, 'Here are some tips...');

  // Step 5: Wait 7 days, check engagement
  await sleep('7 days');
  await sendEmail(email, 'How's it going?');
}

If the server crashes after Step 2, Temporal resumes at Step 3 — not from the beginning.

Activities (Side Effects)

// activities.ts
export async function sendEmail(to: string, subject: string): Promise<void> {
  await resend.emails.send({ to, subject, from: '[email protected]' });
}

export async function chargePayment(email: string, plan: string): Promise<string> {
  const session = await stripe.checkout.sessions.create({
    customer_email: email,
    line_items: [{ price: plans[plan], quantity: 1 }]
  });
  return session.id;
}

export async function provisionAccount(email: string, plan: string): Promise<void> {
  await db.accounts.create({ data: { email, plan, status: 'active' } });
}

Start a Workflow

import { Client } from '@temporalio/client';

const client = new Client();

const handle = await client.workflow.start('onboardUser', {
  taskQueue: 'main',
  workflowId: `onboard-${userId}`,
  args: ['[email protected]', 'pro']
});

// Check status
const result = await handle.result();

Use Cases

• Payment processing — charge, fulfill, refund with guaranteed completion
• User onboarding — multi-day sequences that never lose state
• Data pipelines — ETL that resumes from where it failed
• Order fulfillment — payment → inventory → shipping → notification
• Subscription management — billing cycles, upgrades, cancellations

Temporal vs BullMQ vs Inngest vs Trigger.dev

The Verdict

Temporal is for workflows that MUST complete — payment processing, order fulfillment, data pipelines. If a crash or restart can lose your state, Temporal prevents it. Enterprise-grade reliability for any application.

Need help building production web scrapers or data pipelines? I build custom solutions. Reach out: [email protected]

Check out my awesome-web-scraping collection — 400+ tools for extracting web data.

Today is the 30th, and we’re back with Weekly Challenge #5.

This one’s sneaky, weird, and honestly super satisfying when you get it right.

You’re building a CSS‑only secret menu — a hidden little UI easter egg that only appears when the user does something specific (and not obvious at all).

Quick shoutout to @francistrdev for last week’s challenge — it was genuinely good, and yes, it’s in progress. I didn’t forget, I’m just slow and easily distracted by shiny CSS things.

The Mission

Make a menu that starts completely invisible and only shows up when the user triggers it in a clever way.

Not a normal button.

Not a big “open menu” link.

Something subtle. Something sneaky. Something that makes people go “wait… how did I open that?”

The Rules

• No JavaScript
• Menu must start fully hidden
• User must perform a non‑obvious action to reveal it
• Menu must have at least 5 items
• The reveal must be animated

The Goal

Make it feel like a secret dev panel.

A hidden door.

A little “you found the easter egg” moment.

Bonus (if you wanna go wild)

• Add a second hidden layer
• Add a CSS‑only lock/unlock toggle
• Give it a theme (spy, hacker, neon, retro terminal, etc.)

How to Enter

Just drop your CodePen or GitHub link in the comments.
That’s literally it.
No deadlines, no pressure — just build something cool and share it.

Alright, that’s the challenge

Go make something sneaky.
Make it weird. Make it fun. Make it yours.

And again — thanks @francistrdev for last week’s challenge.
It’s cooking.

Introduction

Working with raw data often introduces challenges such as duplication, inconsistency, and difficulty in extracting meaningful insights.

Data modelling in Power BI addresses these challenges by structuring data into related tables, enabling efficient querying, accurate calculations, and scalable reporting.

This article explores:

• SQL joins and their behaviour
• Power BI relationships and how they differ from joins
• Data modelling schemas
• Practical implementation in Power BI

We’ll use a simple school dataset to ground these concepts.

1. Sample Dataset

We will work with three core tables:

Students (Dimension Table)

Test Scores (Fact Table)

⚠️ Note: Student 105 does not exist in the Students table—this represents a referential integrity issue.

📊 View Dataset Online

Explore the dataset directly: 👉here!

Teachers (Dimension Table)

Data Roles

• Fact Table: Test Scores (transactional/measurable data)
• Dimension Tables: Students, Teachers (descriptive attributes)

2. What is Data Modelling?

Data modelling is the process of:

Organizing data into structured tables and defining relationships that enable efficient querying and accurate analysis.

In analytical systems like Power BI, this typically involves:

• Separating facts from dimensions
• Establishing keys (e.g., StudentID)
• Designing a schema optimized for performance

3. Joins vs Relationships

Joins (Power Query Layer)

A join is a data transformation operation that combines rows from two tables based on a matching condition.

Key Characteristics:

• Produces a new table
• Data is physically merged
• Can introduce duplication

Example:

Joining Students and Test Scores results in repeated student records for each score.

Relationships (Model Layer)

A relationship is a logical link between tables based on matching columns.

Key Characteristics:

• Tables remain separate
• Data is combined at query time
• Enables efficient filtering and aggregation via DAX

Critical Difference

4. SQL Joins Explained (with Examples)

Joins determine how rows from two tables are combined based on a matching condition.

We will use:

• Students table (left table)
• Test Scores table (right table)

INNER JOIN

Definition

Returns only rows where a match exists in both tables.

SQL Example

SELECT s.StudentID, s.Name, t.Subject, t.Score
FROM Students s
INNER JOIN TestScores t
ON s.StudentID = t.StudentID;

Result

Explanation

• Student 103 and 104 are excluded (no scores)
• Score with StudentID 105 is excluded (no matching student)

LEFT JOIN

Definition

Returns all rows from the left table, with matching rows from the right.

SQL Example

SELECT s.StudentID, s.Name, t.Subject, t.Score
FROM Students s
LEFT JOIN TestScores t
ON s.StudentID = t.StudentID;

Result

Explanation

• All students are included
• Missing scores appear as NULL
• Useful for identifying missing activity

RIGHT JOIN

Definition

Returns all rows from the right table and matching rows from the left.

SQL Example

SELECT s.StudentID, s.Name, t.Subject, t.Score
FROM Students s
RIGHT JOIN TestScores t
ON s.StudentID = t.StudentID;

Result

Explanation

• Score for StudentID 105 appears
• Missing student info is NULL
• Highlights data inconsistencies

FULL OUTER JOIN

Definition

Returns all rows from both tables, matched where possible.

SQL Example

SELECT s.StudentID, s.Name, t.Subject, t.Score
FROM Students s
FULL OUTER JOIN TestScores t
ON s.StudentID = t.StudentID;

Result

Explanation

• Combines LEFT and RIGHT JOIN behavior
• Shows all matched and unmatched records

LEFT ANTI JOIN

Definition

Returns rows from the left table with no match in the right.

SQL Equivalent

SELECT s.StudentID, s.Name
FROM Students s
LEFT JOIN TestScores t
ON s.StudentID = t.StudentID
WHERE t.StudentID IS NULL;

Result

Explanation

• These students have no test scores
• Useful for identifying inactivity

RIGHT ANTI JOIN

Definition

Returns rows from the right table with no match in the left.

SQL Equivalent

SELECT t.StudentID, t.Subject
FROM TestScores t
LEFT JOIN Students s
ON s.StudentID = t.StudentID
WHERE s.StudentID IS NULL;

Result

Explanation

• This score has no valid student
• Indicates a data quality issue

Key Insight

Each join answers a specific business question:

5. Power BI Relationships

Cardinality

Defines how rows relate between tables.

👉 Best practice:
Use 1:M from Dimension → Fact

Cross-Filter Direction

Defines how filters propagate between tables.

⚠️ Overusing bi-directional filtering can introduce ambiguity.

Active vs Inactive Relationships

• Only one relationship can be active between two tables
• Inactive relationships can be used via DAX (USERELATIONSHIP)

6. Fact vs Dimension Tables

Fact Table

• Contains measurable data
• Often large
• Example: Test Scores

Dimension Table

• Contains descriptive attributes
• Used for filtering and grouping
• Example: Students

7. Schemas

Star Schema

          Students
             |
Teachers — Test Scores — Subjects

Characteristics:

• Central fact table
• Denormalized dimensions
• Optimized for performance

Snowflake Schema

Students
   |
Address
   |
Region
   |
Test Scores

Characteristics:

• Normalized dimensions
• Reduced redundancy
• Increased complexity

Flat Table (Denormalized Model)

All data stored in one table.

Trade-offs:

• Simpler structure
• Poor scalability
• High duplication

8. Implementation in Power BI

Power Query (Data Preparation)

• Clean data
• Perform joins
• Handle missing values

Model View (Data Modelling)

• Create relationships
• Define cardinality
• Configure filters

Best Practice Workflow

1. Load data
2. Clean in Power Query
3. Avoid unnecessary joins
4. Build relationships in Model View
5. Validate using visuals

9. Common Modelling Issues

• Data duplication from excessive joins
• Incorrect cardinality
• Ambiguous filter paths
• Missing or inconsistent keys

Conclusion and Key Takeaways

Data modelling is the foundation of effective analysis in Power BI. It defines how data is structured, interpreted, and ultimately how insights are derived.

Throughout this article, we’ve seen how:

Joins are used to reshape and prepare data by combining tables in Power Query
Relationships allow Power BI to connect tables logically, enabling efficient analysis without duplication
Schemas, particularly the star schema, provide a scalable structure for organizing fact and dimension tables
Data integrity (e.g., handling missing or invalid records like StudentID 105) directly impacts the quality of insights

Practical Takeaways

• Use Power Query for data cleaning and controlled joins
• Use Model View to define relationships and control how data interacts
• Prefer 1:M relationships from dimension to fact tables
• Default to single-direction filtering unless there is a clear need for bi-directional filtering
• Adopt a star schema for most analytical scenarios
• Treat missing or unmatched data as valuable signals, not just errors

Sprint 7 Retrospective: Quality Gates & Human Experience

TL;DR: Sprint 7 wired 118 V3 services to production REST API routes, built an E2E integration smoke test proving the 6-stage content pipeline works as a chain, enforced memory storage via ADR-032, tested OAuth2 credential rotation, and containerized ComfyUI/FFmpeg. Validated velocity: ~38 points (right-sized from 58). Nine consecutive sprint publishing passes. The gap: persona memory storage is still advisory-only despite three sprints of zero compliance.

Goal

Sprint 7 was the quality hardening sprint — transforming individually tested V3 services into a verified, integrated production system. Where Sprints 3-6 built capabilities, Sprint 7 proved they work together.

This is part of our sprint retrospective series:

• Sprint 3: Production Validation & Pipeline Hardening
• Sprint 4: Building the Media Orchestration Engine
• Sprint 5: YouTube & Video Pipeline — Closing V3
• Sprint 6: Podcast Production & Distribution Pipeline
• Sprint 7: Quality Gates & Human Experience (this post)

Built

Sprint 7 delivered across 10 stories covering 5 capability areas:

V3 Service Wiring (OAS-123)

Wired 118 V3 services to api-server.mjs REST routes — the single largest integration task in the program. Each service got a dedicated REST endpoint, validated by source inspection tests that read the actual route file.

// E2E Pipeline — 6-stage artifact passing
interface StageResult {
  name: string;
  input: string | null;
  output: string | null;
  duration_ms: number;
  status: 'pass' | 'fail' | 'skip';
  error_code?: string;
  error_message?: string;
}

class PipelineExecutor {
  addStage(name: string, fn: StageFn): void;
  run(): Result<PipelineResult>;
}

E2E Integration Smoke Test (OAS-124)

First end-to-end test of the full content pipeline: Source → Script → Audio → Assembly → Quality Gate → RSS. Proves the “24/7 agency” vision works as a chain, not just individual units. PipelineStageValidator catches broken chains, duplicate stages, and contract violations.

Memory Enforcement (OAS-121)

ADR-032 advisory warnings at DONE and sprint close for persona memory storage. While advisory-only (not yet blocking), this establishes the infrastructure for Sprint 8 escalation.

OAuth2 Credential Rotation Testing (OAS-122)

AES-256-GCM encrypted credential store with PBKDF2 key derivation. Tests cover rotation without downtime, emergency revocation, and concurrent access patterns.

Containerization (OAS-120)

ComfyUI and FFmpeg containerized with GPU passthrough. The Dockerfile now compiles TypeScript during docker build — no manual npm run build needed before deployment.

Metrics

Program Cumulative Metrics

Wrong

What Failed or Was Missing

1. Persona memory: three sprints of zero compliance — Sprints 0, 4, and 7 all recorded zero persona-level memories despite ADR-032 advisory warnings. Advisory guidance is provably insufficient. This is the program’s most persistent process gap.

2. Source inspection ≠ runtime validation — Sprint 7’s service wiring tests verify that route registrations exist in source code but cannot catch runtime binding errors, malformed responses, or middleware issues. HTTP integration tests are needed.

3. Scope was right-sized from 58 to ~38 points — This is healthy (scope was pruned, not abandoned), but it reveals the planning estimates were 53% over-optimistic. Better estimation tooling would help.

4. api-server.mjs is a monolith — 118 services wired to a single file. No domain separation, no route modules. This creates merge conflicts and cognitive overload for Sprint 8 UI work.

5. Sprint 6 retro decision partially unimplemented — 3/4 Sprint 6 decisions were implemented, but persona memory enforcement remained advisory despite being flagged for escalation.

AI Participation

Sprint 7 was executed entirely by Claude (Opus 4.6) following the ORCHESTRATE Agile methodology via MCP enforcement. Ten AI personas contributed to the retrospective ceremony:

Lessons

Six lessons extracted from the Sprint 7 ceremony:

1. Phased execution order matters — Infrastructure tickets (wiring, containers) must complete before integration tickets (E2E, smoke tests). Sprint 7 got this right.

2. Right-sizing scope preserves delivery quality — Reducing from 58 to ~38 points was not a failure. It prevented the quality erosion that comes from overcommitment.

3. Source inspection is necessary but not sufficient — Reading route registration code proves services are wired. It cannot prove they respond correctly at runtime.

4. Persona memory requires mechanical enforcement — Three sprints of zero compliance (0, 4, 7) proves advisory-only ADR-032 warnings are ignored. Escalation to blocking gate is the only viable path.

5. Bottom-up architecture validated — Building 118 individual services before the E2E pipeline test is correct. Integration tests need something to integrate.

6. Provenance metadata maturation is a strength — Growing from 4 fields in Sprint 0 to 12+ fields in Sprint 7 (including temporal claims, data sensitivity checks) demonstrates continuous quality improvement.

Accountability

5 Decisions from Sprint 7 Retrospective

Sprint 6 Retro Disposition

3 Unresolved Tensions (Sprint 8 Planning Inputs)

• Auth before UI: Guard Ian insists auth middleware must precede dashboard; React Ive wants to start UI immediately.
• Route extraction timing: Api Endor wants route modules extracted before UI work; pragmatists want to defer to avoid refactoring risk.
• Sprint 8 capacity: 108 points planned vs 38-point velocity — Sprint 9 overflow is inevitable; which stories get deferred?

Next

Sprint 7 closes the quality gates phase. Sprint 8 is the final planned sprint:

• Sprint 8: UI Integration & Dashboard — single-pane-of-glass for the entire platform

The primary risk: Sprint 8 has ~108 points of stories (including 4 retro-decision carry-overs) against a validated velocity of ~38 points. Sprint 9 overflow is certain. The Sprint 8 prompt must prioritize: auth middleware (D-S7-01), route extraction (D-S7-05), and the unified dashboard (OAS-068) before secondary UI tabs.

Sprint 7’s honest assessment: integration-validated, UI-pending. The 118 services are wired and the E2E pipeline proves the chain works. Now it needs a human interface.

Provenance

Generated by ORCHESTRATE Agile Suite — Sprint 7 Quality Gates & Human Experience Retrospective