Why Your SFTP Transfer Is Stuck at 2 MB/s (and the Fix Is a Protocol from 1983)

Two minutes to copy a 274 MB file to a VM running on localhost. Not over the internet. Not to a cloud instance across the country. Localhost. The same machine, loopback, zero network latency.

That was the experience a user reported in issue #290 on cubic, a lightweight CLI for managing QEMU/KVM virtual machines. The maintainer reproduced it, traced the problem to the upstream russh-sftp crate, and posted a comment asking if anyone had ideas about where the bottleneck was. I did. The answer turned out to be a protocol design decision that limits every Rust project using this crate to about 2 MB/s on file transfers, regardless of how fast the link is.

The fix was to stop using SFTP entirely and fall back to a simpler, older protocol.

What Is cubic?

cubic is a CLI tool for creating and managing lightweight virtual machines on Linux and macOS. Think of it as the middle ground between running Docker containers and spinning up full VMs in libvirt. You run cubic create myvm --image debian and get a cloud-init provisioned VM with SSH access, a dedicated disk, and port forwarding. cubic ssh myvm drops you into a shell. cubic scp file.tar.gz myvm:~/ copies files in. It’s about 7,000 lines of Rust, built on QEMU/KVM with cloud-init for provisioning.

Under 40 stars. The maintainer (rogkne) commits daily and reviews external PRs within hours.

The Snapshot

Project cubic
Stars ~37 at time of writing
Maintainer Solo developer, committing daily
Code health ~7,000 lines of clean Rust, 104 unit tests, clap + thiserror + tokio
Docs Good README, CONTRIBUTING.md with conventional commit rules
Contributor UX Fast reviews, specific feedback, merged shell completions PR in multi-round review
Worth using Yes, if you want lightweight VMs without libvirt’s complexity

Under the Hood

cubic has a clean layered architecture. CLI commands live in src/commands/ (one file per subcommand, clap with derive macros). Business logic lives in src/actions/. The instance model, serialization (TOML and YAML), and storage live in src/instance/. Image fetching and distro definitions live in src/image/. SSH and file transfer live in src/ssh_cmd/.

The dependency list is lean. Four crates handle the heavy lifting: russh for SSH connections, russh-sftp for file transfers, clap for CLI parsing, and reqwest for image downloads. Everything else is standard library or small utility crates. The Cargo.toml is not trying to be clever.

One pattern that caught my eye: the project is async internally (tokio, russh) but sync at the CLI boundary. An AsyncCaller struct wraps a tokio multi-threaded runtime and exposes a call() method that blocks on a future. Every command creates one, runs its async work through it, and returns a sync result. It’s simple and it works. No async bleeding into the CLI layer.

The image pipeline is solid. cubic fetches cloud images from distro mirrors, verifies SHA-256/SHA-512 checksums against the upstream checksum file, shows a progress bar during download, and caches images locally. Adding a new distro means adding one entry to the DISTROS static in image_factory.rs. Rocky Linux was added in a recent PR following this exact pattern.

The rough edges are in the SSH layer. The SFTP implementation delegates to russh-sftp, which turned out to be the source of the performance bug. The progress bar during file transfers is coupled to the async read wrapper (AsyncTransferView), which works but makes it hard to swap the underlying transfer mechanism without touching the view layer. The test coverage is good for models and serialization but thin for the SSH and QEMU interaction code, which is typical for tools that depend on external services.

The Contribution

The performance issue (#290) reported that cubic scp transferred files at roughly 2 MB/s on loopback. I dug into the russh-sftp internals to find out why.

The answer is in how russh-sftp implements AsyncWrite. Every call to poll_write() creates a one-shot channel, sends an SFTP write request, and blocks until the server responds with an acknowledgment. One write in flight at a time. No pipelining. The SFTP protocol (RFC 4253) explicitly supports pipelining: clients can send many write requests with different IDs and collect the responses asynchronously. OpenSSH’s sftp client does exactly this with 64 outstanding requests by default. russh-sftp doesn’t. The upstream issue (#70) has been open since June 2025 with no fix.

For a 274 MB file at the default 255 KB max write size, that’s roughly 1,075 round-trips, each waiting for an ACK. Even on loopback, the per-request overhead adds up to minutes.

Wrapping the writer in a BufWriter wouldn’t help. It coalesces small writes into larger ones, but each poll_write() still blocks on the ACK. You’d go from many small round-trips to fewer large ones, but the bottleneck is the same.

The fix was to bypass SFTP for single-file transfers and use SCP instead. SCP is a much simpler protocol: open an SSH exec channel with scp -t <path>, send a one-line header (C0644 <size> <filename>n), stream the raw bytes, send a null byte, done. No request IDs, no per-packet ACKs during data transfer. Just a header and a byte stream.

I added a new scp.rs module (~170 lines) that implements SCP upload and download over a raw russh channel via channel.into_stream(). The async_copy function in russh.rs now detects single-file host-to-guest transfers and routes them through SCP. Directory copies and guest-to-guest transfers still use SFTP. Guest-to-host tries SCP first and falls back to SFTP if it fails (which it will for directories).

The review was thorough. The maintainer requested eight changes, all cleanups: use BufReader.read_line() instead of byte-by-byte loops, add error message prefixes, reuse the ack-reading function in the download path, validate the end-of-transfer marker byte. All reasonable, all addressed. He also asked (politely) whether the PR was AI-generated. I explained my workflow and he was satisfied. The PR went through two review rounds over 12 days and merged. PR #311.

The Verdict

cubic is for developers who want lightweight VMs without the weight of libvirt or the constraints of Docker. If you’re testing deployment scripts, need an isolated Linux environment for a project, or just want to spin up a Debian box and SSH into it without thinking about Vagrant files, this does the job.

The project is young (v0.19.0, solo maintainer) but the trajectory is good. New distros get added regularly. The contributor experience is above average: specific review feedback, no ego, merged with thanks. The maintainer is clearly using this tool daily and fixing things as they surface.

What would push cubic to the next level? The SFTP performance fix helps, but the bigger opportunity is user experience. A cubic init that scaffolds a project config file, better error messages when QEMU isn’t installed, and a Homebrew formula for macOS users would all lower the barrier. The foundation is clean. It just needs more people kicking the tires.

Go Look At This

If you manage VMs from the command line, try cubic. cubic create myvm --image debian and you’re running in under a minute. If you’ve been burned by slow file transfers to VMs before, the SCP fix in PR #311 is worth a look for the protocol analysis alone.

Star the repo. The codebase is small enough to read in a sitting, and there are open issues at every difficulty level.

This is Review Bomb #8, a series where I find under-the-radar projects on GitHub, read the code, contribute something, and write it up. If you know a project that deserves more eyeballs, drop it in the comments.

Nuxt Test Utils v4: Vitest v4 Requirement, Mocking Overhaul and Stricter Environment Setup

Nuxt Test Utils has released version 4.0.0, which primarily integrates Vitest v4. This update changes the test environment setup to beforeAll, resolving issues with module-level mocks. It also improves mockNuxtImport for cleaner partial mocking and enhances state management for registered endpoints. The library remains vital for testing in the Nuxt framework, bridging unit and end-to-end testing.

By Daniel Curtis

TUA Astro Hackathon'da final heyecanı başladı

Türkiye Uzay Ajansı öncülüğünde düzenlenen TUA Astro Hackathon, 37 şehirden 6 bine yakın katılımcının projelerini uzay ve havacılık alanında sunmasıyla başladı. Kazanan takımlara yatırım fırsatları ve destek verilecek.

Web Scraping APIs for AI Agents: Firecrawl vs ScraperAPI vs Apify

Web Scraping APIs for AI Agents: Firecrawl vs ScraperAPI vs Apify

Agents need reliable web data extraction. Three platforms dominate: Firecrawl (LLM-native), ScraperAPI (general purpose), and Apify (versatile platform).

This comparison uses Rhumb AN Scores — evaluating web scraping APIs specifically for agent-readiness: execution reliability, structured extraction, handling complex JavaScript sites, and failure modes that agents must defend against.

The Scores

Provider AN Score Execution Access Readiness Flexibility Confidence
Firecrawl 7.2 7.8 7.1 7.2 73%
ScraperAPI 7.0 6.9 6.8 7.8 71%
Apify 7.2 7.4 6.9 7.6 72%

Firecrawl: “Built for LLM pipelines” (7.2 / 10)

Best for: Agents that need markdown from complex sites with minimal config. LLM-native output format, zero JavaScript handling overhead, designed specifically for agent use.

Biggest friction: Markdown extraction can miss structured data (tables, lists) that pure HTML would preserve. LLM-specific design means if you need raw HTML or API responses, Firecrawl is overfit to your use case.

Avoid when: You need granular control over extraction rules or raw HTML. Firecrawl’s “just give me markdown” simplicity is its weakness for complex data extraction requirements.

Decision: Pick Firecrawl if your agent consumes markdown and you want minimum setup. Fastest path from URL to agent context.

Why it lands here: Highest execution for agent workflows (7.8). REST API is clean. Handles JavaScript-heavy sites by default. No configuration required — point it at a URL, get markdown back. Error messages are agent-actionable.

ScraperAPI: “General-purpose Swiss Army knife” (7.0 / 10)

Best for: Agents that need flexibility across diverse scraping requirements: HTML parsing, structured extraction, proxy rotation, JavaScript handling, and custom rendering.

Biggest friction: Requires more configuration than Firecrawl. JavaScript rendering adds latency. Structured extraction needs custom CSS selectors or regex — not as agent-friendly as Firecrawl’s markdown approach.

Avoid when: You want a quick “markdown from URL” interface. ScraperAPI’s power comes at the cost of setup. If your only need is markdown extraction, Firecrawl is simpler.

Decision: Pick ScraperAPI when you need extraction flexibility or when dealing with diverse site structures that require different strategies.

Why it lands here: Highest flexibility score (7.8) reflects the breadth of extraction options. REST API supports both simple and advanced use cases. CAPTCHA solving and proxy rotation remove friction for difficult sites. But it’s more of a toolkit than a solved problem.

Apify: “Full automation platform” (7.2 / 10)

Best for: Agents running complex workflows: multi-step navigation, authenticated scraping, data transformation pipelines, and monitoring hundreds of sites on schedule.

Biggest friction: Heavyweight for simple “scrape one URL” requests. Cloud-native design assumes you’re building data pipelines, not one-off requests. Pricing is complex (compute units, storage, requests).

Avoid when: Your agent just needs to extract data from a single URL per request. Apify’s power is wasted. ScraperAPI or Firecrawl are simpler choices.

Decision: Pick Apify when your agent needs to orchestrate complex extraction workflows or monitor sites on a schedule.

Why it lands here: Execution is 7.4 (strong). The platform is built for automation — you can define actor logic (Node.js, Python), schedule runs, and integrate with external systems. But that power makes it overkill for simple agent requests.

Routing Rules for Agents

  1. “Just give me markdown from a URL”: Use Firecrawl. Simplest API, LLM-native output, zero config.
  2. “I need to extract specific data structures”: Use ScraperAPI. CSS selectors, custom parsing, flexible output formats.
  3. “My agent runs complex multi-step scraping workflows”: Use Apify. Orchestrate navigation, authentication, transformation, and scheduling.
  4. “I need to scrape JavaScript-heavy sites”: All three handle it, but Firecrawl is simplest (built-in), ScraperAPI requires enable-js, Apify requires actor code.

One-Line Rule

Firecrawl for agent-first markdown extraction. ScraperAPI when you need extraction flexibility. Apify for complex automation workflows.

Web scraping is inherently fragile — sites change structure, break links, add obfuscation. Agents must handle extraction failures gracefully. Prefer APIs that provide clear error signals and documented retry patterns over those that silently return partial data.

What AN Score Actually Measures

We evaluate web scraping APIs on:

  • Execution: API reliability, JavaScript handling, error signal clarity, timeout patterns
  • Access Readiness: Authentication integration, proxy support, rate limiting feedback, structured output modes
  • Flexibility: Extraction customization, output format options, transformation capabilities, scheduling support
  • Autonomy: Documented failure modes, retry strategies, cost transparency, SLA clarity

Each dimension is weighted for agent-specific use cases (not human data analysis or BI pipelines).

See the Full Data

Visit Rhumb.dev for the complete comparison, failure mode analysis, cost breakdown, and agent routing patterns.

This comparison is powered by Rhumb AN Score — the open scoring framework for APIs built for autonomous agents.

About the AN Score: Rhumb evaluates 645+ APIs across 20 dimensions specifically for agent-readiness. No pay-to-rank. No vendor influence. Just data.

Understanding Data Modeling in Power BI: Joins, Relationships, and Schemas Explained

Image containing SQL joins

Introduction

Every report, dashboard, and visual in Power BI is only as good as the data model behind it. You can have perfect DAX measures and stunning visuals, but if your data model is poorly designed, your numbers will be wrong, your report will be slow, and your users will lose trust.

Data modeling is the process of organizing, structuring, and connecting your data tables so that Power BI can calculate results correctly and perform efficiently. It determines how tables relate to each other, how filters flow between them, and ultimately how every single number in your report is computed.

This article is a deep-dive into the core concepts of Power BI data modeling. We will cover SQL-style joins, Power BI relationships, fact and dimension tables, schema designs, and common pitfalls — all with real-life examples and clear guidance on where to perform each action in the Power BI interface.

Part 1: What Is Data Modeling?

Data modeling is the practice of defining how your data tables are structured and how they connect to one another. In Power BI specifically, a data model is the collection of all your tables, the relationships between them, and the calculated columns, measures, and hierarchies you build on top.

A well-designed data model achieves three goals:

  1. Correctness — Filters propagate properly, so aggregations (SUM, COUNT, AVERAGE) return accurate results across related tables.
  2. Performance — The VertiPaq engine can compress and query data efficiently, keeping report load times fast even at scale.
  3. Simplicity — Report authors can drag and drop fields intuitively without needing to understand complex workarounds.

A poorly designed model, by contrast, leads to ambiguous relationships, inflated row counts, circular dependencies, and DAX formulas that become nightmarishly complex just to produce simple totals.

Part 2: SQL Joins — Combining Data Before It Enters the Model

Before data reaches your Power BI model, you often need to combine tables from one or more sources. This is where joins come in. Joins are operations that merge rows from two tables based on a matching condition (a common column). In Power BI, joins are performed in Power Query Editor (the data transformation layer), not in the model itself.

Image containing SQL joins

Think of a join as a question: “For each row in Table A, what matching rows exist in Table B?”

The answer depends on the type of join you use.

2.1 INNER JOIN

What it does: Returns only the rows where a match exists in both tables. If a row in Table A has no corresponding match in Table B, it is excluded — and vice versa.

Real-life example: You have an Orders table and a Customers table. An INNER JOIN on CustomerID returns only orders placed by customers who exist in the Customers table. If there is an order with a CustomerID that does not appear in Customers (perhaps an orphaned record), that order is dropped.

Orders Table              Customers Table
┌──────────┬────────┐     ┌──────────┬──────────┐
│ OrderID  │ CustID │     │ CustID   │ Name     │
├──────────┼────────┤     ├──────────┼──────────┤
│ 1001     │ C1     │     │ C1       │ Alice    │
│ 1002     │ C2     │     │ C2       │ Bob      │
│ 1003     │ C3     │     │ C4       │ Diana    │
│ 1004     │ C5     │     │ C5       │ Eve      │
└──────────┴────────┘     └──────────┴──────────┘

INNER JOIN Result (on CustID):
┌──────────┬────────┬──────────┐
│ OrderID  │ CustID │ Name     │
├──────────┼────────┼──────────┤
│ 1001     │ C1     │ Alice    │
│ 1002     │ C2     │ Bob      │
│ 1004     │ C5     │ Eve      │
└──────────┴────────┴──────────┘
→ Order 1003 (C3) is dropped: no match in Customers.
→ Customer C4 (Diana) is dropped: no match in Orders.

Image explaining inner joins

When to use it: When you only want records that exist in both tables and are comfortable discarding unmatched rows.

2.2 LEFT (OUTER) JOIN

What it does: Returns all rows from the left table (Table A), plus matching rows from the right table (Table B). If a row in the left table has no match, the right-side columns are filled with null.

Real-life example: You want a complete list of all orders, with customer names attached where available. Even if an order’s CustomerID has no match in Customers, you still want to see that order.

LEFT JOIN Result (Orders LEFT JOIN Customers on CustID):
┌──────────┬────────┬──────────┐
│ OrderID  │ CustID │ Name     │
├──────────┼────────┼──────────┤
│ 1001     │ C1     │ Alice    │
│ 1002     │ C2     │ Bob      │
│ 1003     │ C3     │ null     │  ← No match, but order is kept
│ 1004     │ C5     │ Eve      │
└──────────┴────────┴──────────┘
→ All 4 orders are preserved. Customer Diana (C4) has no order, so she does not appear.

When to use it: When the left table is your “primary” table and you want every row from it, enriched with optional data from the right table.

2.3 RIGHT (OUTER) JOIN

What it does: The mirror of a LEFT JOIN. Returns all rows from the right table, plus matching rows from the left table. Unmatched left-side columns are filled with null.

Real-life example: You want a list of all customers, with their orders attached where applicable. Customers with no orders still appear.

RIGHT JOIN Result (Orders RIGHT JOIN Customers on CustID):
┌──────────┬────────┬──────────┐
│ OrderID  │ CustID │ Name     │
├──────────┼────────┼──────────┤
│ 1001     │ C1     │ Alice    │
│ 1002     │ C2     │ Bob      │
│ null     │ C4     │ Diana    │  ← No order, but customer is kept
│ 1004     │ C5     │ Eve      │
└──────────┴────────┴──────────┘
→ Order 1003 (C3) is dropped because C3 is not in Customers.

When to use it: When the right table is your anchor. In practice, most analysts simply swap the table order and use a LEFT JOIN instead — both achieve the same result.

2.4 FULL OUTER JOIN

What it does: Returns all rows from both tables. Where a match exists, the columns are merged. Where no match exists on either side, the missing columns are filled with null.

Real-life example: You want to see every order and every customer, whether they match or not — useful for data reconciliation or finding orphaned records.

FULL OUTER JOIN Result:
┌──────────┬────────┬──────────┐
│ OrderID  │ CustID │ Name     │
├──────────┼────────┼──────────┤
│ 1001     │ C1     │ Alice    │
│ 1002     │ C2     │ Bob      │
│ 1003     │ C3     │ null     │  ← Order with no customer
│ null     │ C4     │ Diana    │  ← Customer with no order
│ 1004     │ C5     │ Eve      │
└──────────┴────────┴──────────┘
→ Nothing is dropped. Both unmatched sides are preserved.

When to use it: Data quality checks, reconciliation, or when you cannot afford to lose any records from either table.

2.5 LEFT ANTI JOIN

What it does: Returns only the rows from the left table that have NO match in the right table. It is the opposite of an INNER JOIN from the left table’s perspective.

Real-life example: You want to find orders that reference a CustomerID that does not exist in the Customers table — orphaned orders.

LEFT ANTI JOIN Result:
┌──────────┬────────┐
│ OrderID  │ CustID │
├──────────┼────────┤
│ 1003     │ C3     │  ← This order has no matching customer
└──────────┴────────┘
→ Only unmatched rows from the left table are returned.

When to use it: Finding orphaned or unmatched records. Data validation. Identifying gaps.

2.6 RIGHT ANTI JOIN

What it does: Returns only the rows from the right table that have NO match in the left table.

Real-life example: You want to find customers who have never placed an order.

RIGHT ANTI JOIN Result:
┌────────┬──────────┐
│ CustID │ Name     │
├────────┼──────────┤
│ C4     │ Diana    │  ← This customer has no orders
└────────┴──────────┘
→ Only unmatched rows from the right table are returned.

When to use it: Identifying inactive customers, unused products, or entities with no associated transactions.

2.7 How to Perform Joins in Power BI (Step-by-Step)

Joins in Power BI are performed in Power Query Editor, using the Merge Queries feature.

Step 1: Open Power BI Desktop and click Transform Data (on the Home ribbon) to enter Power Query Editor.

Step 2: Select the table you want to act as the “left” table in the Queries pane on the left.

Step 3: On the Home ribbon in Power Query Editor, click Merge Queries (or Merge Queries as New if you want a separate output table).

Step 4: In the Merge dialog:

  • The top table is your left table (already selected).
  • Select the bottom table (your right table) from the dropdown.
  • Click on the matching column(s) in both table previews (e.g., CustID in both).
  • At the bottom of the dialog, choose your Join Kind from the dropdown:
    • Inner Join
    • Left Outer (All from first, matching from second)
    • Right Outer (All from second, matching from first)
    • Full Outer (All rows from both)
    • Left Anti (Rows only in first)
    • Right Anti (Rows only in second)

Step 5: Click OK. A new column appears in your table containing the merged data as nested tables.

Step 6: Click the expand icon (↔) on the new column header to select which columns from the right table you want to bring in.

Step 7: Click Close & Apply to load the merged result into your data model.

Part 3: Power BI Relationships — Connecting Tables in the Model

While joins physically merge two tables into one (performed during data loading in Power Query), relationships are logical connections between separate tables that remain in your data model. They tell Power BI how to filter and aggregate data across tables without combining them.

This distinction is critical: joins happen before data enters the model; relationships exist within the model.

3.1 Cardinality Types

Cardinality defines how many rows in one table can match rows in another table.

One-to-Many (1:M) — The Gold Standard

The most common and recommended relationship type. One row in the “one” side matches zero, one, or many rows on the “many” side.

Example: One row in Customers (CustomerID = C1) can match many rows in Orders (multiple orders placed by C1). But each order has exactly one customer.

Customers (1 side)          Orders (Many side)
┌────────┬────────┐         ┌──────────┬────────┬────────┐
│ CustID │ Name   │         │ OrderID  │ CustID │ Amount │
├────────┼────────┤         ├──────────┼────────┼────────┤
│ C1     │ Alice  │───1:M──▶│ 1001     │ C1     │ 50     │
│        │        │         │ 1005     │ C1     │ 120    │
│ C2     │ Bob    │───1:M──▶│ 1002     │ C2     │ 80     │
└────────┴────────┘         └──────────┴────────┴────────┘

Why it is preferred: Filters flow cleanly from the “one” side to the “many” side. When you select “Alice” in a slicer, Power BI filters the Orders table to only show Alice’s orders. Aggregations (SUM of Amount) work correctly and unambiguously.

Many-to-Many (M:M)

Multiple rows on both sides can match each other. This occurs when neither table has unique values in the join column.

Example: A Students table and a Courses table connected through a bridge table, where each student takes multiple courses and each course has multiple students. If you connect Students directly to Courses without a bridge table, you get an M:M relationship.

Risks: M:M relationships can produce unexpected results because filters can flow in ways that cause double-counting or inflated totals. Power BI allows them, but they should be used cautiously and are often a sign that a bridge table (a fact table representing the enrollment) is needed.

One-to-One (1:1)

Each row in Table A matches exactly one row in Table B. Both columns in the relationship contain unique values.

Example: An Employees table and an EmployeeDetails table, where each employee has exactly one detail record.

When it arises: Usually when a single table has been split into two for organizational reasons (e.g., separating PII data from non-PII data). Consider whether the tables should simply be merged.

3.2 Cross-Filter Direction

Cross-filter direction controls how slicer and filter selections propagate between related tables.

Single Direction (Default for 1:M)

Filters flow from the “one” side to the “many” side only. This is the default and recommended setting.

Customers ──filter──▶ Orders
(Selecting a customer filters their orders)

Orders ──X──▶ Customers
(Selecting an order does NOT filter the Customers table by default)

Why single is preferred: It keeps filter behavior predictable. Report authors know that dimension tables filter fact tables, and not the other way around.

Both Directions (Bidirectional)

Filters flow in both directions. Selecting a value in either table filters the other.

Customers ◀──filter──▶ Orders

When to use: Rarely. Sometimes necessary when you have a bridge table in an M:M scenario and need filters to propagate through it. Bidirectional filtering can cause performance issues and ambiguous filter contexts if overused.

Recommendation: Start with single-direction filtering. Only switch to bidirectional when you have a specific, well-understood reason.

3.3 Active vs. Inactive Relationships

Power BI allows only one active relationship between any two tables at a time. If your data has multiple logical connections between the same two tables, you must designate one as active and the rest as inactive.

Example: A Sales table has both an OrderDate and a ShipDate column. Both logically connect to a Calendar table. You can create two relationships (OrderDate→Calendar and ShipDate→Calendar), but only one can be active.

Calendar (Date)
    │
    ├── Active Relationship ──── Sales[OrderDate]
    │
    └── Inactive Relationship ─ ─ ─ Sales[ShipDate]  (dashed line in Model View)

The active relationship is used by default in all DAX calculations. To use the inactive relationship, you invoke the USERELATIONSHIP function:

Ship Date Sales = 
CALCULATE(
    SUM(Sales[Amount]),
    USERELATIONSHIP(Sales[ShipDate], Calendar[Date])
)

This concept directly connects to role-playing dimensions, discussed later in this article.

3.4 How to Create and Manage Relationships in Power BI (Step-by-Step)

Method 1: Model View (Drag-and-Drop)

Step 1: In Power BI Desktop, click the Model View icon on the left sidebar (the icon with three connected boxes).

Step 2: You will see all your tables displayed as boxes with their columns listed.

Step 3: To create a relationship, click on a column in one table and drag it to the matching column in another table. A line appears connecting the two tables.

Step 4: Power BI will auto-detect the cardinality and cross-filter direction. Double-click the relationship line to edit these properties.

Method 2: Manage Relationships Dialog

Step 1: Go to the Home ribbon (or Modeling ribbon) and click Manage Relationships.

Step 2: A dialog appears showing all existing relationships. Click New to create one.

Step 3: Select the two tables and the matching columns from dropdowns.

Step 4: Set the cardinality (One-to-Many, Many-to-Many, One-to-One).

Step 5: Set the cross-filter direction (Single or Both).

Step 6: Check or uncheck Make this relationship active.

Step 7: Click OK to save.

Method 3: Auto-Detect

When you first load data, Power BI attempts to auto-detect relationships based on column names and data types. This works well for simple models but should always be reviewed manually. Go to Manage Relationships to verify what Power BI created and correct any mistakes.

Part 4: Joins vs. Relationships — What Is the Difference?

Image illustrating the differences btwn joins and r/ships

This is one of the most common points of confusion for Power BI beginners.

Aspect Joins (Power Query) Relationships (Model)
Where Power Query Editor (Transform Data) Model View / Manage Relationships
When During data loading/transformation After data is loaded into the model
Result Physically merges two tables into one Keeps tables separate, connected logically
Row count Output table may have more or fewer rows Tables retain their original row counts
Use case Enriching/cleaning data before modeling Enabling cross-table filtering and aggregation
Performance Increases table size (more columns per row) Efficient — VertiPaq handles relationships natively
DAX impact Merged columns become part of one table DAX uses RELATED/RELATEDTABLE to traverse

Rule of thumb: Use joins (Merge in Power Query) when you need to physically combine or clean data, such as adding a lookup column or deduplicating. Use relationships when you want to keep tables separate and let Power BI handle filtering and aggregation — which is the case for the vast majority of your model.

Part 5: Fact Tables vs. Dimension Tables

The foundation of any good data model is the separation of data into fact tables and dimension tables.

Fact Tables

A fact table stores the events or transactions of your business. Each row represents something that happened: a sale, a shipment, a click, a support ticket.

Characteristics of fact tables:

  • Contain measures — numeric values you want to aggregate (Amount, Quantity, Cost, Duration).
  • Contain foreign keys — columns that reference dimension tables (CustomerID, ProductID, DateKey).
  • Tend to be tall (many rows, relatively few columns).
  • Grow over time as new transactions occur.

Examples: Sales, Orders, WebClicks, SupportTickets, Shipments, FinancialTransactions.

Dimension Tables

A dimension table stores the descriptive attributes that provide context to your facts. They answer the “who, what, where, when, why” questions about your transactions.

Characteristics of dimension tables:

  • Contain descriptive columns — names, categories, addresses, dates, hierarchies.
  • Contain a primary key — a unique identifier for each row (CustomerID, ProductID, Date).
  • Tend to be wide (many columns, relatively few rows compared to fact tables).
  • Change slowly or infrequently.

Examples: Customers, Products, Calendar/Date, Geography, Employees, Stores.

How They Connect

┌──────────────┐     ┌──────────────────────────────┐     ┌──────────────┐
│  Customers   │     │         Sales (Fact)          │     │   Products   │
│  (Dimension) │     │                               │     │  (Dimension) │
├──────────────┤     ├──────────────────────────────┤     ├──────────────┤
│ *CustomerID  │◀─1:M│ SaleID                        │M:1─▶│ *ProductID   │
│  Name        │     │ CustomerID (FK)               │     │  ProductName │
│  City        │     │ ProductID (FK)                │     │  Category    │
│  Segment     │     │ DateKey (FK)                  │     │  Price       │
└──────────────┘     │ Quantity                      │     └──────────────┘
                     │ Amount                        │
                     └──────────────────────────────┘
                                   │
                                  M:1
                                   │
                              ┌─────────┐
                              │Calendar │
                              │(Dim)    │
                              ├─────────┤
                              │*Date    │
                              │ Month   │
                              │ Quarter │
                              │ Year    │
                              └─────────┘

The asterisk (*) denotes the primary key. Arrows show the “one” side pointing to the “many” side. Filters flow from dimensions into the fact table, which is exactly what you want — selecting a customer, product, or date filters the relevant sales.

Part 6: Schema Designs

A schema is the overall shape or architecture of your data model — how fact and dimension tables are arranged and connected.

6.1 Star Schema

The star schema is the recommended design for Power BI. It consists of a central fact table surrounded by dimension tables, each directly connected to the fact table. When visualized in Model View, the layout resembles a star.

Image showing star schema

                    ┌────────────┐
                    │  Calendar  │
                    └──────┬─────┘
                           │
┌────────────┐    ┌────────┴────────┐    ┌────────────┐
│ Customers  ├────┤   Sales (Fact)  ├────┤  Products  │
└────────────┘    └────────┬────────┘    └────────────┘
                           │
                    ┌──────┴─────┐
                    │   Stores   │
                    └────────────┘

Characteristics:

  • One fact table at the center.
  • Dimension tables connect directly to the fact table (one hop).
  • Dimensions are denormalized — all descriptive attributes are in a single table per dimension, even if that means some redundancy.

Example: A Products dimension contains ProductName, SubCategory, Category, and Department all in one table, even though SubCategory→Category→Department is a hierarchy. The redundancy is intentional — it simplifies the model.

Advantages:

  • Simple and intuitive for report authors.
  • Power BI’s VertiPaq engine is optimized for star schemas.
  • Filters travel one hop from dimension to fact — fast and predictable.
  • DAX formulas are straightforward.

When to use: Almost always. This should be your default schema design in Power BI.

6.2 Snowflake Schema

A snowflake schema is a star schema where one or more dimension tables are normalized — broken into sub-tables connected in a chain.

┌──────────────┐    ┌──────────────┐    ┌──────────────────┐
│  Department  ├────┤  Category    ├────┤    Products      │
└──────────────┘    └──────────────┘    └────────┬─────────┘
                                                 │
                                        ┌────────┴────────┐
                                        │  Sales (Fact)   │
                                        └────────┬────────┘
                                                 │
                                          ┌──────┴─────┐
                                          │  Calendar  │
                                          └────────────┘

Instead of one Products dimension with all attributes, the snowflake breaks it into Products → Category → Department, each as a separate table.

Characteristics:

  • Normalized dimension chains (multiple hops from fact to outermost dimension).
  • Less data redundancy in dimension tables.
  • More tables and relationships to manage.

Advantages:

  • Reduced storage for dimension data (relevant in traditional data warehouses).
  • Cleaner structure if dimensions are very large and highly hierarchical.

Disadvantages in Power BI:

  • Filters must travel through multiple relationships to reach the fact table, which can complicate DAX and reduce performance.
  • Report authors have to understand which sub-table contains which attribute.
  • VertiPaq already compresses data extremely well, so the storage savings of normalization are negligible.

When to use: Rarely in Power BI. If your source data arrives snowflaked, consider flattening (denormalizing) the dimension chains in Power Query before loading them into the model. The result: a star schema.

Image showing snowflake schemas example

6.3 Flat Table (Denormalized / DLAT)

A flat table model consists of a single, wide table that contains everything — facts and dimensions combined into one table.

┌─────────────────────────────────────────────────────────┐
│                   FlatSalesTable                        │
├─────────────────────────────────────────────────────────┤
│ SaleID │ Date │ CustomerName │ City │ ProductName │ ... │
│ Amount │ Quantity │ Category │ Segment │ StoreName │ ... │
└─────────────────────────────────────────────────────────┘
(One massive table — no relationships needed)

Characteristics:

  • Everything in one table.
  • No relationships because there is only one table.
  • Extremely wide (many columns) and potentially very tall (many rows with repeated descriptive data).

Advantages:

  • Simple for very small datasets or quick ad-hoc analysis.
  • No relationships to configure.
  • Easy to create from a single CSV or Excel export.

Disadvantages:

  • Massive data redundancy (customer name, city, segment repeated on every row).
  • Poor compression and large model size.
  • No clear separation of measures vs. attributes.
  • Difficult to maintain and extend.
  • DAX calculations can behave unexpectedly because there is no clear filter direction.

When to use: Only for quick prototypes, small personal datasets, or when you genuinely have a single flat data source with fewer than a few thousand rows and no plans to scale.

6.4 Schema Comparison Summary

Criteria Star Schema Snowflake Schema Flat Table
Complexity Low Medium-High Lowest
Performance in Power BI Best Good (but extra hops) Poor at scale
Data redundancy Moderate (in dimensions) Low Very high
Ease of DAX Easiest Harder (multi-hop) Unpredictable
Scalability Excellent Good Poor
Recommended for Power BI Yes — default choice Flatten if possible Small/ad-hoc only

Part 7: Role-Playing Dimensions

A role-playing dimension is a single dimension table that serves multiple analytical purposes by connecting to the same fact table through different columns.

The most common example is the Calendar/Date dimension. A Sales fact table might have:

  • OrderDate — when the order was placed.
  • ShipDate — when the order was shipped.
  • DeliveryDate — when the order was delivered.

All three columns connect to the same Calendar table, but each represents a different analytical “role.”

Image showing role playing dimensions

How to Handle This in Power BI

Option A: One Calendar Table, Multiple Relationships

Create one Calendar table and establish three relationships to Sales. Designate one as active (typically OrderDate) and the others as inactive. Use USERELATIONSHIP in DAX when you need the inactive relationships:

Shipped Amount = 
CALCULATE(
    SUM(Sales[Amount]),
    USERELATIONSHIP(Sales[ShipDate], Calendar[Date])
)

Pros: Single calendar table to maintain. Less data duplication.
Cons: Requires DAX knowledge for inactive relationships. Only one relationship is active for default behavior.

Option B: Duplicate Calendar Tables

Create separate calendar tables in Power Query (e.g., OrderCalendar, ShipCalendar, DeliveryCalendar), each with an active relationship to the respective date column. This means duplicating the Calendar table logic.

Pros: All relationships are active. No USERELATIONSHIP needed. Slicers can be placed on any calendar independently.
Cons: Multiple copies of the same data. More tables in the model.

Which to choose: Option A is cleaner for models where the non-primary date roles are used infrequently. Option B is better when all date roles are equally important and frequently sliced on.

Part 8: Common Data Modeling Issues and How to Avoid Them

Issue 1: Ambiguous Relationships

Symptom: Power BI shows an error: “A relationship already exists between these tables” or “Ambiguous paths.”

Cause: Multiple active relationship paths exist between two tables, and Power BI does not know which to use.

Fix: Ensure only one active relationship exists between any pair of tables. Use inactive relationships with USERELATIONSHIP for alternate paths.

Issue 2: Circular Dependencies

Symptom: Power BI warns about circular dependencies or refuses to create a relationship.

Cause: The relationships form a closed loop (A→B→C→A), and Power BI cannot determine a clear filter direction.

Fix: Restructure your model to eliminate the loop. Often this means consolidating tables or removing a redundant relationship.

Issue 3: Many-to-Many Without a Bridge Table

Symptom: Aggregations return inflated or incorrect totals.

Cause: Two tables are connected M:M directly because neither has unique values in the join column. Filters propagate ambiguously.

Fix: Introduce a bridge (junction/associative) table that contains one row for each valid combination, turning the M:M into two 1:M relationships.

Before (problematic):
Students ──M:M──▶ Courses

After (correct):
Students ──1:M──▶ Enrollments ◀──M:1── Courses

Issue 4: Using Bidirectional Filtering Everywhere

Symptom: Unexpected filter behavior, slow performance, or filters appearing to “leak” across unrelated parts of the model.

Cause: Cross-filter direction set to “Both” on many relationships.

Fix: Reset all relationships to single-direction (default). Only enable bidirectional where absolutely necessary and test the impact thoroughly.

Issue 5: Too Many Columns in Fact Tables

Symptom: Large model size, slow refresh, poor compression.

Cause: Descriptive columns (customer names, product categories, addresses) stored directly in the fact table instead of in dimension tables.

Fix: Move descriptive attributes to their own dimension tables. Keep only keys and measures in the fact table. Use Power Query to split data during loading.

Issue 6: Missing or Incorrect Date Table

Symptom: Time intelligence functions (TOTALYTD, SAMEPERIODLASTYEAR, etc.) do not work or return errors.

Cause: No proper date/calendar dimension exists, or the date column has gaps.

Fix: Create a dedicated Calendar table that covers the full date range of your data with no gaps. Mark it as a Date Table in Power BI (select the table → Modeling ribbon → Mark as Date Table). Ensure it has a column of data type Date with unique, contiguous values.

Part 9: Putting It All Together — A Modeling Workflow

Here is a practical workflow for building a clean Power BI data model:

Step 1: Understand the business questions. What do stakeholders need to analyze? (Revenue by region? Inventory by warehouse? Support tickets by priority?) The answers determine which facts and dimensions you need.

Step 2: Identify fact and dimension tables. Facts are the measurable events. Dimensions are the descriptive context. Separate them clearly.

Step 3: Clean and transform in Power Query. Use joins (Merge) to enrich data during loading. Flatten snowflaked dimensions. Remove unnecessary columns. Set correct data types.

Step 4: Load into the model and build relationships. Switch to Model View. Create 1:M relationships between dimension tables (one side) and fact tables (many side). Set cross-filter direction to single.

Step 5: Organize for usability. Hide foreign key columns from the report view (right-click → Hide in Report View). Create display folders. Add descriptions to key columns and measures. Arrange tables in a star layout in Model View.

Step 6: Build measures and test. Write DAX measures (SUM, AVERAGE, CALCULATE, etc.) and verify they return correct results. Test slicers to confirm filters propagate as expected.

Step 7: Optimize. Remove unused columns. Check model size. Profile query performance. Ensure date tables are properly marked.

Conclusion

Data modeling is not a step you rush through on the way to building visuals — it is the structural foundation that determines whether your Power BI reports are fast, accurate, and maintainable.

The key takeaways from this article:

  • Joins merge tables physically during data loading and are performed in Power Query. Use them for enrichment and cleanup.
  • Relationships connect tables logically within the model and enable cross-table filtering. They are the backbone of your data model.
  • Star schemas are the gold standard for Power BI. Keep fact tables at the center, surrounded by denormalized dimension tables.
  • Cardinality should be 1:M wherever possible. Avoid M:M unless you have a proper bridge table.
  • Cross-filter direction should default to single. Use bidirectional filtering sparingly and deliberately.
  • Role-playing dimensions handle scenarios where one dimension serves multiple analytical roles (e.g., multiple dates).
  • Common issues like ambiguous relationships, circular dependencies, and misplaced columns are almost always solvable by restructuring toward a clean star schema.

Invest time in your data model. Every minute spent designing a clean model saves hours of debugging DAX formulas and explaining incorrect numbers later.

This article is intended as a reference guide for Power BI practitioners at all levels. Bookmark it, share it with your team, and revisit it whenever you start a new project.

Auth0 vs Clerk vs Firebase Auth for AI Agents — AN Score Comparison

Auth0 vs Clerk vs Firebase Auth for AI Agents — AN Score Comparison

When your agent needs to authenticate API calls, three platforms dominate: Clerk (execution-first), Auth0 (enterprise compliance), and Firebase Auth (Google ecosystem).

This comparison uses Rhumb AN Scores — a 20-dimension framework specifically designed for evaluating APIs agent-readiness. We measure execution reliability, access patterns, governance requirements, and real failure modes.

The Scores

Provider AN Score Execution Access Readiness Governance Confidence
Clerk 7.4 8.3 7.1 8.0 82%
Auth0 6.3 6.8 5.0 9.0 71%
Firebase Auth 6.3 7.5 4.8 7.0 68%

Clerk: “Default for agent builders” (7.4 / 10)

Best for: Agents that need the cleanest path from API key to user management. Modern API design, highest execution score, most agent-friendly SDK.

Biggest friction: JWT clock skew causes intermittent auth rejections in distributed systems. Webhook delivery has no retry — lost events stay lost. Both require defensive coding from day one.

Avoid when: Your organization needs HIPAA/FedRAMP compliance or is locked into existing Auth0/Okta contracts. Clerk’s compliance surface is newer.

Decision: Pick Clerk unless an enterprise compliance requirement pushes you elsewhere.

Why it lands here: Highest aggregate score. SDK is modern (Next.js, React native). Backend API is clean REST. Self-serve billing removes friction. Documented failure modes make agent hardening easier.

Auth0: “Enterprise gateway” (6.3 / 10)

Best for: Organizations needing HIPAA, SOC 2 Type II, SCIM, and deep RBAC with custom roles. The governance ceiling when compliance is the gating constraint.

Biggest friction: Management API rate limits are strict. Rules → Actions migration creates confusion. Okta acquisition fragmented documentation and billing.

Avoid when: Building greenfield projects without compliance requirements. Management API complexity is unnecessary overhead.

Decision: Pick Auth0 when the organization needs deep compliance or is already in the Okta ecosystem.

Why it lands here: Governance score is 9.0 (highest), but access readiness is 5.0. Enterprise compliance is genuinely strong: HIPAA BAA, custom roles, audit logs. API reflects feature accumulation and two conflicting extensibility models.

Firebase Auth: “Google Cloud native” (6.3 / 10)

Best for: Projects already on Firebase/GCP where auth is one layer in a broader Google ecosystem. Generously free tier (50K MAU — highest of the three).

Biggest friction: Admin SDK is the only agent-friendly interface. Client SDKs assume browser context. Multi-step GCP setup (IAM, service accounts, project config). Token verification requires Google public key rotation awareness.

Avoid when: Building standalone auth solution or wanting to avoid Google Cloud lock-in. Firebase Auth is tightly coupled to the Firebase SDK ecosystem.

Decision: Pick Firebase Auth when already on Firebase/GCP and auth is a feature, not the product.

Why it lands here: Execution is 7.5 (good), but access readiness is 4.8 (lowest) — Firebase Auth isn’t designed as standalone service. Works best inside Firebase, fights you in isolation.

Routing Rules for Agents

  1. If pure execution speed matters: Use Clerk. Highest documented API surface, cleanest endpoint design for agent calls.
  2. If compliance is a contract requirement: Use Auth0. No other provider has the governance depth.
  3. If you’re already on Google Cloud: Use Firebase Auth. Sunk infrastructure cost favors ecosystem lock-in.
  4. If you’re unsure: Start with Clerk. Easiest to migrate away from later.

One-Line Rule

Clerk for clean agent-first auth. Auth0 when enterprise compliance is the gating constraint. Firebase Auth when Google Cloud already owns the infrastructure.

Auth is a trust-critical surface. Unlike payments or email, a mistake here is a security incident, not a failed transaction. Agents should prefer the option with the most documented failure modes — that is where hardening is most possible.

What AN Score Actually Measures

We evaluate authentication providers on:

  • Execution: API surface design, rate limits, retry behavior, timeout patterns
  • Access Readiness: Agent-native auth flows, machine-to-machine credential patterns, webhook reliability
  • Governance: Compliance certifications, audit trails, organization controls
  • Autonomy: Failure mode documentation, recovery patterns, incident response surfaces

Each dimension is weighted and scored independently, then combined into the aggregate AN Score.

See the Full Data

Visit Rhumb.dev for the complete comparison, score methodology, free-tier analysis, and agent routing rules.

This comparison is powered by Rhumb AN Score — the open scoring framework for APIs built for autonomous agents.

About the AN Score: Rhumb evaluates 645+ APIs across 20 dimensions specifically for agent-readiness. No pay-to-rank. No vendor influence. Just data.

Top Remote Developer Jobs Available Right Now

Top Remote Developer Jobs Available Right Now

In the rapidly evolving world of technology, remote work has become a major player, especially for developers. If you’re in the tech field and looking to escape the confines of a traditional office while still making great contributions, the good news is that a plethora of remote developer jobs is currently up for grabs. With 311 active remote jobs listed on our site, there are plenty of opportunities to explore that align with your skills and aspirations. Here’s a closer look at the top categories of remote developer jobs available right now!

Explore Top Categories of Remote Developer Jobs

At RemoteDevJobs, we’re excited to present an array of positions that cater to various specialties in the development landscape. Here’s a breakdown of the top categories currently trending:

1. DevOps (114 Jobs)

The DevOps field is booming, with a staggering 114 remote job openings. Companies are increasingly adopting DevOps practices to improve collaboration between software development and IT operations, aiming for continuous integration and continuous delivery (CI/CD). If you have skills in Kubernetes, AWS, or Docker, this category is ripe for the picking. Organizations seek professionals who can streamline their processes and enhance operational efficiency.

2. Backend Development (49 Jobs)

If you’re well-versed in server-side programming and databases, the backend development category could be your goldmine! With 49 remote jobs available, employers are on the hunt for developers experienced in Python, SQL, Java, and more. As the backbone of any application, backend developers play a crucial role in ensuring that everything runs smoothly behind the scenes.

3. Full Stack Development (44 Jobs)

Full Stack developers are the Swiss Army knives of the development world, adept at handling both frontend and backend tasks. With 44 remote positions available, this is an excellent opportunity for those who want to take on diverse challenges. Companies are looking for developers skilled in both user interface (UI) design and server management, making it essential to stay current with frameworks and languages across the board.

4. Frontend Development (37 Jobs)

The demand for stunning user experiences is higher than ever in the digital age, and this is where frontend developers shine. With 37 jobs currently available, it’s an exciting field for those who excel in creating visually appealing and intuitive user interfaces. If you are proficient in technologies like HTML, CSS, or modern JavaScript frameworks, opportunities are waiting for you.

5. AI / ML (27 Jobs)

Artificial Intelligence and Machine Learning are transforming industries, and the demand for experts in these fields is rapidly increasing. With 27 remote job openings in AI / ML, candidates with skills in Machine Learning and data analysis can find opportunities to work on groundbreaking projects that shape the future of technology. Industries ranging from healthcare to finance are actively seeking talent in this cutting-edge domain.

Top Skills in Demand

As you explore these exciting opportunities, it’s essential to consider which skills are currently in demand. Employers are looking for candidates who are proficient in:

  • Python
  • DevOps
  • Kubernetes
  • CI/CD
  • AWS
  • SQL
  • Machine Learning
  • Java
  • Docker
  • Cloud Infrastructure

Having these skills can greatly enhance your chances of landing that dream remote job. So take stock of your skill set and think about how you can position yourself as an ideal candidate in today’s market.

Start Your Remote Job Search Today!

Ready to take the plunge and elevate your career with a remote developer job? Whether you’re focusing on DevOps, backend development, full stack, frontend, or diving into the world of AI and ML, there’s something for everyone at RemoteDevJobs.

Don’t miss out on the opportunity to work with innovative companies from the comfort of your home. Visit RemoteDevJobs today to browse all 311+ active remote jobs and find the perfect fit for you.

Browse all 311+ remote developer jobs at RemoteDevJobs.