Mert

“Data Extension” is the generic term, but SFMC supports several DE types with different behaviors. Knowing the distinctions saves you from trying to send email from a lookup DE or forgetting to refresh a Filtered DE before a campaign.

Here’s the reference we hand to every new engineer.

1. Sendable DE

The main type used for sending email. Requirements:

• Is Sendable = true in Properties
• A field of type EmailAddress
• A Send Relationship mapping a field (usually Primary Key) to the Subscriber Key

This is the DE you pick when you send a campaign or set up a Journey with Data Extension entry source. No send can happen without one.

2. Lookup DE (Non-sendable)

Reference data that AMPscript Lookup() pulls into email templates at render time. Common examples:

• Sales rep directory: SalesRep_DE with RepID, name, email, phone
• Product catalog: Product_DE with SKU, name, description, price
• Store locations: Store_DE with StoreID, address, hours

Not sendable, no EmailAddress field needed. Used by the email, not as the source of the send.

3. Filtered DE

A DE created by applying a Data Filter to another DE – point-and-click segmentation without SQL. Example: “subscribers from Master_DE where MemberTier = Gold” produces a Gold-only Filtered DE.

Important: Filtered DEs need to be refreshed to reflect the current state of the source DE. They don’t auto-update.

Refresh via:

• Filter Activity in Automation Studio (scheduled or ad hoc)
• Manually in Email Studio

If the source DE changes (new imports, updates) but the Filtered DE isn’t refreshed, the Filtered DE holds stale data. Campaigns targeting it send to outdated segments.

For anything more complex than single-attribute filtering (joins, calculations), use SQL Query Activity writing to a standard DE, not a Filtered DE.

4. Random DE

Splits a source DE into N equal random chunks. Use case: A/B/N testing where you need 10 equal groups to test 10 email variants.

No SQL needed. Configure the split percentages in the UI; SFMC assigns rows randomly.

Caveats:

• The “random” assignment happens at creation; refreshing re-randomizes, potentially putting the same subscriber into different buckets.
• For more sophisticated random assignment (e.g. Journey Builder’s Random Split), the Journey tool is usually the better fit.

5. Shared DE

A DE placed in the Shared Data Extensions folder in the parent Business Unit. Multiple child BUs in the Enterprise account can access the DE without copying it.

Use when:

• The same reference data (product catalog, store list) is needed across multiple brands or regions.
• You want a single source of truth rather than syncing copies between BUs.

Access permissions are set via Shared Data Extension Permissions – which BUs can read/write.

Watch for unintended cross-BU writes: if two BUs can write to the same Shared DE, coordinate schemas and import schedules.

6. Send Log DE

A special DE that logs every email send – who received what, when, subject line, etc. Useful for:

• Audit requirements beyond SFMC’s default 10-day retention
• Join with other data to build custom reporting
• Investigate specific send events

Created from the TriggeredSendDataExtension template when used with Triggered Sends.

Caveat: Test Sends don’t write to Send Log. Only production sends do. If you’re testing and expecting the Send Log to populate, it won’t.

7. DE with Data Retention Policy

Any DE can have a retention policy configured on creation:

• Delete records older than X days – SFMC auto-purges rows past the threshold
• Delete data and DE after X days – whole DE goes away

Use for:

• Temporary event signup DEs (auto-purge 30 days after event)
• PII-heavy DEs that shouldn’t retain data beyond a defined window
• Any Send Log or archive DE to prevent unbounded growth

Set retention at creation if possible. Adding it later works but doesn’t apply retroactively to existing rows until the next automation evaluation.

Picking the right type on a new DE

NeedDE TypeSend email from this listSendableReference data AMPscript will look upLookupSimple attribute-based segmentFilteredA/B test random splitsRandomCross-BU shared referenceSharedAudit / custom trackingSend LogAuto-purge old dataDE with Retention Policy

Most projects end up with a mix: one or two Sendable DEs, several Lookup DEs, possibly a Shared DE for multi-brand setups, and retention policies on anything transient.

Takeaway

Naming the DE type right in your head before creating it prevents architecture rebuilds. The decision takes seconds; fixing an incorrectly-typed DE after it’s been loaded with data can take a day.

Designing SFMC data architecture? Our Salesforce team ships Data Extension layouts, shared-BU patterns, and retention strategies on production engagements. Get in touch ->

See our full platform services for the stack we cover.

A deal desk dashboard showed the running total of opportunities per account. Total amount per account was a rollup column. Users opened the dashboard, saw a total, made a decision. Then someone added a new opportunity and checked the same account. The account’s total did not change.

Fifteen minutes later, refresh – still not changed. An hour later – changed. The rollup column was working exactly as documented, and the team had mistaken it for a real-time aggregate.

Here is the difference between calculated and rollup columns, and the pattern we now use when the dashboard needs to be current.

The two column types

Calculated column: computes its value every time the row is read. If the column adds two other columns, every time anything loads the row (form view, API call, view row), Dataverse computes the sum. Results are always current because they are re-derived on demand.

Rollup column: computes its value on a schedule. The default schedule is every 12 hours, configurable down to one hour per column. A rollup that sums child record amounts reads the current state of children when the schedule fires, stores the result, and serves that stored value until the next schedule.

Both store their definition (the formula) in the solution. Both look identical to consuming code. The runtime behavior is radically different.

When each one is correct

Calculated columns are correct when:

• The formula depends only on fields on the same row, or on a parent’s fields (via lookup)
• The computation is cheap (arithmetic, string concatenation, conditionals)
• You need the value to be current every time it is read

Rollup columns are correct when:

• The formula aggregates across many child records (SUM, COUNT, AVG, MIN, MAX)
• The underlying data changes frequently but the aggregate does not need to be current within seconds
• Scheduling async recomputation is acceptable

A calculated column that aggregates children is not allowed. Dataverse blocks it – the reason is performance, reading an account would trigger a query across all its opportunities every load. Rollup is the platform’s answer: compute the aggregate ahead of time, serve it cached.

Why the async nature bites

Teams who want a rolling total of child records are usually asked for it by a business user who sees the number on a dashboard and expects it to match reality. When a user tells the team “the total on the Account form should show all linked Opportunity amounts,” the team builds a rollup column. The dashboard shows the rollup.

The user then creates an opportunity, looks at the account, and the total is stale. From the user’s perspective, the number is wrong. From the platform’s perspective, the number is accurate for the last time the schedule fired.

The fix is either to lower the user’s expectations (the dashboard updates on a schedule) or to change the implementation.

Real-time totals: three alternatives

When the aggregate genuinely needs to be current:

Alternative 1: plugin that updates a stored total on every child change.

A post-operation plugin on Opportunity (Create, Update on amount, Delete) recalculates acme_total_opportunities on the parent Account and writes it. Works for modest volumes – an account with 200 opportunities, each updating rarely, is fine. Fails for accounts with 20,000 opportunities updating frequently – every child change becomes a parent write, and the Dataverse execution pipeline starts throttling.

Good for: master-detail relationships with medium fanout.

Alternative 2: calculated column summing via the lookup.

You cannot aggregate children from a parent via a calculated column, but you can compute the child’s contribution per row using a calculated column on the child. acme_attributable_amount = IF(acme_is_counted, amount, 0) on Opportunity, then use that in any consumer query. The consumer does the aggregation at read time.

Works for: dashboards that can run FetchXML with aggregate operators (SUM the acme_attributable_amount across children in a single query). Fails if the consumer is a Dataverse form or view that cannot run aggregate queries.

Good for: Power BI reports, custom dashboards, anywhere you control the read query.

Alternative 3: rollup with one-hour schedule plus “Refresh” button.

Keep the rollup for the baseline case. Add a button on the form that explicitly triggers a rollup recalculation via the CalculateRollupField SDK request. User experiences it as “I just added a child, now I hit Refresh, the total updates.”

Works for: dashboards with bursty usage patterns – the baseline is acceptable, explicit refresh covers the “just changed it” case.

Good for: sales management screens, deal desk reviews, scenarios where users understand the refresh semantics.

The audit we run on rollups

On any project with more than three rollup columns, we audit quarterly:

1. Is the rollup still in use? Rollups accumulate – someone adds one for a requirement that later gets dropped, the column stays. Deprecated rollups still consume schedule slots.
2. Is the schedule frequency right? A rollup set to 12 hours when the underlying data changes every hour is both wrong (stale) and wasteful (the 12-hour window sees many state changes).
3. Is the formula still correct? Filters in rollup definitions reference specific statecode/statuscode values. When those values change semantics in later releases, rollups silently compute the wrong thing.

The audit takes 20 minutes per project. Half the time, at least one rollup can be deleted; a third of the time, at least one schedule needs adjusting.

Gotcha: deleting a child does not trigger rollup

A rollup of child-record amounts updates when a child is created or its amount changes. It does not update when a child is deleted, until the scheduled refresh fires.

The window between “child deleted” and “rollup refreshed” shows an overstated total. This is especially painful in audit scenarios – a user deletes a duplicate, the total still shows the duplicate’s contribution for up to twelve hours.

The fix is either a plugin on the child Delete event (post-operation, trigger a rollup recalculation explicitly via CalculateRollupFieldRequest) or a tighter schedule. We usually go with the plugin when the table supports deletions, because the time-lag is user-visible.

The rule we enforce

Rollup columns ship with a form tooltip: “This total refreshes every N hours. Click Refresh after adding new records to see updated totals immediately.”

The tooltip is one field-description line, it costs nothing to add, and it inoculates the “why is the number stale” ticket that would otherwise land every other week. Every rollup we ship has it. Every one we inherit gets it added on the first pass.

Manufacturers running Dynamics 365 Supply Chain Management almost always also run a dedicated Manufacturing Execution System (MES) on the shop floor. Production order updates, inventory movements, quality tests, and traceability data flow between them continuously. The integration has to be low-latency (shop floor runs on seconds, not hours), high-throughput (hundreds of events per minute at peak), and reliable (lost messages mean lost traceability).

Three integration patterns come up in evaluations. Two have documented failure modes.

The options that don’t fit manufacturing

Nightly batch jobs via Data Management Framework. Designed for bulk data movement, not real-time signaling. Production orders complete hours before D365 knows about it. Real-time inventory view is always lagging. Traceability data arrives after the batches have shipped.

Custom OData polling with a loop that queries MES every few seconds. Introduces polling overhead for no latency benefit, and MES systems aren’t typically designed to handle heavy poll loads. Also creates a custom code dependency that needs maintenance.

Database-level triggers on the MES database pushing directly to F&O’s database. Breaks supportability completely. D365 F&O is a managed platform – direct database writes aren’t supported, aren’t upgrade-safe, and will break the next time Microsoft changes schema. Also creates a security nightmare (MES has privileged write access to F&O’s database?).

The only answer that fits the requirements is Azure middleware between the two systems.

The Azure-native pattern

Logic Apps or Service Bus as middleware between MES and D365, with F&O Business Events on the D365 side.

What each piece does:

Azure Service Bus for the guaranteed-delivery, ordered messaging. Production-order status updates, inventory moves, quality-test results flow through Service Bus queues with FIFO ordering per production order.

Azure Logic Apps for the orchestration where branching and transformation happen. A pick-complete event from MES fires a Logic App that transforms the payload, updates inventory in D365, and triggers the next production-flow message back to MES.

F&O Business Events for the D365-side publishing. When a production order is created, released, or completed in F&O, a business event fires to Service Bus or Event Grid. MES subscribers pick it up.

Custom Services on F&O for the inbound – when MES has a state change D365 needs to record, the Logic App (or Function) calls a custom service endpoint on F&O. Custom services are designed for low-latency targeted writes, unlike data entities which are bulk-optimized.

Traceability architecture

Traceability is specific in manufacturing – regulators and customers need to know which raw materials went into which finished-goods batch. D365’s batch tracking combines with MES’s shop-floor batch recording to produce the full lineage. The integration ensures:

• MES tracks the physical movement (machine X processed batch Y at time Z)
• D365 records the ERP-level batch (raw material batch A consumed in production order B producing finished batch C)
• Integration correlates the two via batch numbers and production order references
• Recall scenarios can trace backward from sold finished goods to source materials, or forward from suspect materials to affected finished goods

The integration isn’t just about moving data – it’s about keeping the correlation intact under all failure modes.

High-throughput considerations

At manufacturing scale (large plants with multiple lines, each firing events per minute), throughput planning matters:

• Service Bus sizing – standard tier suffices for most deployments; Premium only when message volumes exceed the standard tier’s throughput units
• Logic Apps concurrency – configured per workflow, default is 20 concurrent runs; high-throughput flows need higher
• F&O write capacity – custom services are faster than data entities for single-record writes; batching is appropriate when MES aggregates multiple updates
• Dead-letter monitoring – alerts when Service Bus DLQ gets non-zero entries; usually indicates transformation errors that need human review

Reliability patterns

Manufacturing can’t afford lost messages. The architecture carries:

• Retry with exponential backoff in Logic Apps for transient failures
• Dead-letter queues for poison messages
• Idempotency keys on custom service calls so retry doesn’t double-record
• Correlation IDs flowing end-to-end for cross-system debugging
• Monitoring dashboards on message throughput and latency per queue

Direction-specific patterns

Flows in each direction have different shapes:

MES → D365 (updates ERP from shop floor):

• MES publishes to Service Bus topic
• Logic App subscribes, transforms, calls F&O custom service
• F&O updates inventory, production order status, quality records

D365 → MES (issues work to shop floor):

• F&O business event on production order release
• Event flows to Service Bus
• MES subscriber picks up the work package, distributes to lines

Bidirectional correlation:

• Correlation IDs in headers
• Handshake patterns for critical state transitions (e.g., “order ready” → “order accepted by line”)

Where custom code fits

Not everything is Logic Apps. Sometimes:

• Azure Functions for complex transformations Logic Apps can’t express cleanly
• Custom services in F&O for writes standard entities don’t cover
• Durable Functions for long-running orchestrations (multi-day production runs)

Each has a clear use case. Reach for them when the declarative tool hits a limit, not as default.

What ships with the architecture

A working MES-to-D365 integration has:

• Service Bus queues and topics partitioned by production-order or line
• Logic Apps for each directional flow with transformation logic
• Business events published on production order lifecycle
• Custom services on F&O for inbound writes from MES
• Dead-letter queue monitoring with alerting
• Traceability validation – spot-check recall scenarios quarterly
• Runbook for extending the integration when a new MES module is added

The pattern is architect-grade because manufacturing systems won’t tolerate the simpler options. Azure middleware is the supported, scalable, maintainable middle.

A client needs to expose a “calculate the loyalty rebate for this customer” operation. It reads three Dataverse tables, applies some business rules, writes a result. Every consumer – the Dynamics web app, a Power Automate flow, an external integration – should call the same operation.

Three places we could put it. Three different cost, latency, and scale profiles. Here is the matrix we now run on every “new operation” request.

The three options

Custom Action (legacy): a process defined in Dataverse that can be invoked through the SDK. Steps are Workflow Activity Actions. Old-school but still widely deployed.

Custom API (modern): the successor to custom actions. Defined as Dataverse entity rows (customapi, customapiRequestParameter, customapiResponseProperty), backed by a plugin that implements the logic. Exposed through the Web API with a typed OpenAPI schema.

Azure Function: fully external .NET function, invoked from Power Automate or direct HTTP. Runs in its own compute, scales separately, has its own pricing.

The three solve overlapping problems. The choice is about latency, cost structure, and who maintains the code.

Decision matrix

DimensionCustom ActionCustom APIAzure FunctionLatencyLow (in-process)Low (in-process)Medium (cross-service)Concurrency limitDataverse’sDataverse’sFunction App’sTimeout2 minutes2 minutes10 minutes (Consumption), configurable higherCostIncluded in DataverseIncluded in DataversePer-execution + computeLong-running workNoNoYes (durable functions)External dependency callsLimited (sandbox)Limited (sandbox)Full flexibilityOpenAPI schemaNoYesManualInvocable from flowYesYes (first-class)Yes (HTTP connector)Maintenance.NET + Workflow.NET.NET (preferred)

When each one is correct

Custom Action is correct when you are maintaining an existing system that already uses them. For new work, skip them – custom APIs are the modern equivalent with better tooling.

Custom API is correct when:

• The logic is primarily Dataverse queries and writes
• Latency matters (under 500ms per call)
• The caller is inside the Microsoft stack (flow, another plugin, Dynamics app)
• The operation completes in under 2 minutes

Azure Function is correct when:

• The logic calls external services with uncertain latency
• The operation may take longer than 2 minutes
• Durability, retries, or checkpointing matter (durable functions)
• The code needs packages the Dataverse sandbox blocks (file IO, sockets, native dependencies)
• You want the compute cost on a different meter from Dataverse

Cost reality

Custom APIs consume Dataverse capacity. A high-volume custom API (tens of thousands of calls per day) is paid for by your existing Dataverse licensing – no incremental per-call charge.

Azure Functions on Consumption plan: $0.000016 per GB-second and $0.20 per million executions. A typical custom function costing 128MB for 300ms runs at roughly $0.00000006 per call. A million calls per month is around $0.25 in compute plus $0.20 in executions – effectively free.

The cost math usually favors custom API for Dataverse-heavy work and Azure Function for external-dependency-heavy work. Mixing the two via a custom API that calls a function (rare pattern) pays both bills.

Gotchas per option

Custom API gotcha: the plugin that implements a custom API registers against a synthetic “Custom API message” step. Debugging is via plugin trace logs, same as any plugin. Deployment is through the solution. OpenAPI schema is auto-generated from the custom api row definitions – mis-typing a parameter name once in Dataverse and once in the plugin code produces a runtime failure that the solution checker does not catch.

Azure Function gotcha: cold start on Consumption plan can add 2-5 seconds for the first call after idle. For a user-facing interactive call, this is unacceptable. Options: Premium plan (always warm, higher baseline cost), Dedicated App Service plan, or accept the cold-start delay if the call is async.

Authentication gotcha for Azure Function: calling Dataverse from an Azure Function requires auth. The cleanest pattern is a managed identity on the Function App with an application user in Dataverse. Tutorials that use a user’s personal credentials in app settings are security incidents waiting to happen.

The decision we ran last month

Requirement: “Calculate and apply a loyalty rebate on every Order created.”

• Reads three tables (Customer, OrderHistory, RebateProgram)
• Writes a calculated value back to the Order
• Must run on every order creation
• Expected volume: 2,000 orders per day, bursty up to 50/minute
• Latency target: under one second on order save

Walking the matrix:

• Latency: sub-second is tight for Azure Function with cold start. Custom API stays in-process.
• External dependencies: none. All Dataverse.
• Long-running: no, under 500ms estimated.
• Cost: Dataverse is already paid for; Azure Function would be cheap but not free.

Custom API won clearly. Implementation was a plugin registered against a custom API definition, called from both a post-operation plugin on Order Create (synchronous, blocks save) and directly from a Power Automate flow (for retroactive recalculation).

A year later, the client wanted to integrate with an external tax service that takes 3-4 seconds per call. We did not put that in the same custom API – the 2-minute timeout plus the unpredictability of the external service made it fragile. We built a separate Azure Function for the tax call, invoked async from a Power Automate flow. Two tools for two different latency profiles, as it should be.

The rule of thumb

If the operation stays inside Dataverse, start with a custom API. You get latency, integrated ALM, and zero incremental cost. You can always move it to an Azure Function later if the requirements shift.

If the operation has any external dependency with unpredictable behavior, start with an Azure Function. You get the durability patterns and the 10-minute timeout that the Dataverse sandbox cannot give.

If you find yourself using custom actions for new work, stop. Custom APIs have everything custom actions did plus a schema and better tooling.

A founder pinged us with a UX problem disguised as an engineering question. His team had launched an AI chat product. Users were abandoning the conversation before the agent finished responding. The team had measured p95 response latency at 31 seconds. Their assumption was that they needed to switch to a faster model.

The actual model was responsible for about 35% of the total latency. The other 65% was sequential tool calls, unnecessary intermediate LLM steps, and a missing streaming layer. Switching to a smaller model would have cut maybe 5 seconds off the worst case while degrading response quality.

We made four changes that did not touch the model. P95 latency dropped from 31 seconds to 8 seconds. User abandonment rate dropped 70%. The model stayed the same.

This is the latency stack that most teams either do not measure or do not know how to optimize. Here is the pattern.

Where the time actually goes

Before any optimization, instrument the agent and trace where time is spent. The breakdown for a typical multi-step agent looks something like:

• LLM calls (cumulative): 30-50% of total latency
• Tool calls (cumulative): 30-40% of total latency
• Network and serialization overhead: 5-10%
• Application logic between steps: 5-10%

The LLM call portion is hard to optimize without changing the model. The other 50-70% is where most of the win is, and most teams do not look there because the LLM feels like the obvious bottleneck.

The founder’s agent had:

• 8 LLM calls per response (planner + 4 tool dispatchers + critic + 2 retries)
• 6 tool calls (4 distinct tools, with 2 retries)
• All sequential
• No streaming back to the user

Total p95 was 31 seconds. The model was responsible for 11 seconds of that. The other 20 seconds were structural.

Change 1: Parallelize independent tool calls

The biggest win in most agent systems is making tool calls concurrent.

The founder’s agent had a step where it needed to look up the user’s account information, the customer’s order history, and the related support tickets. The original code did this sequentially:

1. Look up account (1.5s)
2. Look up orders (2.0s)
3. Look up tickets (1.8s)

Total: 5.3 seconds, all wait time.

The fix was async-await with parallel dispatch:

1. Dispatch all three lookups concurrently
2. Wait for all to complete
3. Total: 2.0 seconds (the slowest of the three)

That single change saved 3.3 seconds per request. The actual code change was 8 lines.

The pattern: any time the agent calls multiple tools and the results are independent, those tool calls can run concurrently. Use asyncio.gather in Python, Promise.all in JavaScript. Modern agent frameworks (LangGraph, CrewAI Flows) support this natively.

Audit your agent for serial tool calls that could run in parallel. There is almost always one or two.

Change 2: Cut the unnecessary intermediate steps

The original agent had a planner LLM that generated a plan, then a separate critic LLM that reviewed the plan, then the executor.

The critic was rejecting roughly 8% of plans, which meant 92% of the time it was an extra LLM call (3 seconds) for nothing. And of the 8% it rejected, the planner produced a similar plan on the next iteration anyway, suggesting the critic was not adding much real value.

We removed the critic, kept the planner, and added validation rules instead (does the plan reference real tools, does it have any cycles, does it stay under the step limit). The validation runs in milliseconds, not seconds, and catches the same class of bad plans the critic was catching.

3 seconds saved per request, on average. Quality measured no different on the eval set.

The pattern: every LLM call in the chain should be earning its keep. If a step rejects only a small fraction of inputs and the alternative is a deterministic check, the deterministic check is faster and usually as good.

Change 3: Stream the response back to the user

The original agent waited for the full response to be generated before returning anything to the user. The user saw a loading spinner for 31 seconds, then the full response appeared.

Streaming changes the perceived latency dramatically. The user starts seeing tokens within 1-2 seconds, even if the full response takes 8. The agent is not actually faster, but the UX feels faster, and abandonment rate drops because users get visible feedback.

The implementation is a few lines of code. The OpenAI and Anthropic APIs both support streaming natively. The frontend needs to handle server-sent events (SSE) or websockets. The user experience improvement is large and immediate.

For multi-step agents, you can also stream intermediate progress: “Looking up your account… Checking order history… Generating response…” This is qualitatively different from a spinner. Users will wait 8 seconds for visible progress; they will not wait 8 seconds for a spinner.

Change 4: Cache aggressively

Some queries repeat. Some intermediate steps repeat across queries. Both are cacheable.

The founder’s agent had a “policy lookup” tool that retrieved from a small KB of company policies. The policies changed maybe once a month. The agent was hitting the KB and running a vector search every time. We added a simple in-memory cache with a 5-minute TTL. The cache hit rate was 35%, saving 0.8 seconds per cached request.

Semantic caching is the more advanced version: cache LLM responses keyed on the embedding of the query. If a new query is semantically similar to a recent one, return the cached response. We use this carefully (only for queries with high confidence and low staleness risk), but it can save 5+ seconds on cache hits.

Aggressive caching: every tool call and every LLM call should be evaluated for cacheability. Most are. Most teams do not bother. The latency improvement compounds.

What the founder shipped

After all four changes:

• Parallel tool calls: -3 seconds
• Removed critic LLM: -3 seconds
• Streaming: perceived latency dropped from 31s to 1-2s for first token
• Caching: -2 seconds average
• Total p95: 31s → 8s

User abandonment rate dropped 70% in the first two weeks. Customer satisfaction scores went up. The model and the agent’s core logic did not change.

The team’s original instinct (switch to a smaller model) would have saved maybe 5 seconds while degrading quality. The structural changes saved 23 seconds while keeping quality the same.

The latency optimization checklist

Before approving “switch to a faster model” as the latency fix, walk through:

• Are tool calls running sequentially when they could run in parallel? Almost always at least one set is serial that could be concurrent.
• Is every LLM step earning its keep? Critics, validators, refiners that fire on every request might be removable in favor of deterministic checks.
• Is the user seeing intermediate progress, or just a spinner? Streaming alone changes the perceived latency more than any model swap.
• Are repeated queries cached? Tool calls, LLM calls, retrieved chunks: most are cacheable with TTL.
• Are smaller models being used for non-critical steps? Use GPT-4o for the final answer if quality matters; use GPT-4o-mini for the planner, the router, the validator.

Most agent systems have 50-70% latency improvement available without touching the primary model. Audit before you swap.

The model swap is the last resort

Switching to a smaller, faster model can be the right call for some agents. But it is the last resort, not the first. The cost is response quality, and that cost is permanent. The structural improvements are free.

The pattern Sapota recommends: optimize the structure first, measure the new latency, then decide whether a model swap is still needed. Often it is not. When it is, you have a much smaller gap to close, and the trade-off is more justified.

If your agent’s latency is killing UX

If your team has launched an AI agent and users are abandoning conversations before the response arrives, the issue is rarely the model. It is usually the structure.

Sapota offers a one-week latency audit that traces your agent’s actual time spent, identifies the parallelizable tool calls, the removable LLM steps, and the missing caching opportunities, and ships the optimizations as working code. We have done this for chat products, customer support tools, and research assistants. The typical improvement is 60-75% latency reduction without changing the model.

Reach out via the AI engineering page with your current p95 latency and a sample trace if you have one. If you do not have traces, we will install observability first and audit second.

A founder we work with had been stuck on the same problem for two months. Their RAG retrieval recall was sitting at 58%. They had tried OpenAI’s embedding-3-small, then embedding-3-large, then BGE-M3, then Voyage. Each swap added a couple of points, then the curve flattened. The team was about to start fine-tuning their own embedding model.

We told them to stop and add a reranker first. The number went from 58% to 81% in a single afternoon. The fine-tuning project was cancelled.

This is the moment most teams discover that the bottleneck was never the embedding model. It was the architecture choice of using a single embedding per chunk to begin with. Late interaction is the family of techniques that fixes it, and it is the one most teams skip because the name sounds intimidating.

What a single embedding per chunk loses

A bi-encoder (which is what every standard embedding model is) takes a chunk of text, compresses it into a single fixed-length vector, and stores it. At query time, the user’s question is also compressed into a single vector, and similarity is computed between the two.

The compression is the problem. A 500-token chunk that mentions five different concepts gets averaged into one vector. The vector represents the chunk roughly, but it loses the distinction between “this chunk is mostly about X with a brief mention of Y” and “this chunk is mostly about Y with a brief mention of X.” When the user query is about Y, both chunks look equally relevant by cosine distance, even though one is the right answer and the other is noise.

This is why every benchmark of “best embedding model” shows diminishing returns past a certain point. The embedding model is doing the best it can with the information bottleneck of a single vector. The architecture is the limit.

How late interaction works

ColBERT (the original, 2020) keeps the per-token embeddings instead of pooling them into one vector. A 500-token chunk becomes 500 vectors. A 10-token query becomes 10 vectors. At scoring time, you compute the maximum similarity between each query token and any chunk token, then sum those max scores into the final relevance score.

The math is the same dot products you would do for any vector search. The difference is that “how well does this query match this chunk” is now a sum of “for each query token, what is the best matching chunk token,” which preserves the fine-grained signal that pooling threw away.

In practice this looks like:

• Query “GPT-4o pricing breakdown” tokenizes to roughly 4 tokens.
• Each token finds its best match in the candidate chunk.
• “GPT-4o” matches the “GPT-4o” token in the chunk strongly.
• “pricing” matches “cost” or “price” or “pricing” in the chunk.
• “breakdown” matches “table” or “structure” or “breakdown.”
• The summed max scores give a relevance number that reflects all four query terms, not just the average semantic similarity.

This is what catches the recall ceiling.

Two patterns we ship

Sapota uses ColBERT in two patterns depending on the corpus size and latency budget.

Pattern 1: ColBERT as a reranker over bi-encoder retrieval. The first stage is a standard bi-encoder vector search returning the top 50 candidates. The second stage is ColBERT reranking those 50 down to the top 5. This is the pattern we use for most production deployments. The first stage is fast (millisecond range, scales to billions of vectors). The second stage is slow but only runs on 50 candidates, not the full index.

Pattern 2: ColBERT as the only retriever. For corpora under a few million chunks, ColBERT can be the primary retriever using PLAID or similar index structures that make late-interaction search tractable at scale. Latency is higher than a bi-encoder (10x to 50x depending on index size), but recall is the highest of any retrieval method we have benchmarked.

We default to Pattern 1 unless the corpus is small enough that Pattern 2 is feasible and the recall lift justifies it.

ColPali: the same trick for documents

ColPali extends the late-interaction idea to entire document pages treated as images. Instead of token-level embeddings of text, it uses patch-level embeddings of an image of the page (each page split into a 32×32 grid of patches). Query tokens match against image patches using the same MaxSim mechanism.

The implications:

• The OCR step disappears. The model sees the page as a vision-language model would.
• Layout, charts, tables, and figures are all preserved as part of the same embedding space.
• Cross-modal queries (text query against visual content) work natively.

The cost is storage (1024 vectors per page vs 1 vector per chunk) and indexing speed (vision encoder inference is GPU-bound). Binary quantization brings the storage cost down by 32x and the latency down by an order of magnitude, which is what makes ColPali production-viable.

For document-heavy corpora (research papers, financial filings, slide decks, regulatory submissions), ColPali outperforms both bi-encoder text RAG and CLIP-based multimodal RAG on published benchmarks. We use it when the corpus is genuinely visual and the budget supports the storage and GPU inference cost.

The cost conversation

Late interaction is not free. The honest trade-off:

• Storage. ColBERT chunks store roughly 100x more vectors than bi-encoder chunks (one per token vs one per chunk). ColPali stores 1024 vectors per page. Plan for this in vector DB sizing.
• Index time. Building the index takes longer because there are more vectors to compute. Not catastrophic, usually a few hours for a million-chunk corpus on a single GPU.
• Query time. A reranker pattern adds 50ms to 200ms to the p50 latency, depending on how many candidates the reranker consumes. A pure-ColBERT setup adds more.
• Operational complexity. Vector databases that support multi-vector with MaxSim comparators are a smaller set than the ones that support standard cosine search. Qdrant supports it natively. Most others do not.

The number to weigh against this is the recall lift. In every audit Sapota has run where the team’s recall plateaued in the 50% to 70% range, adding late-interaction reranking pushed it into the 80% to 90% range. That delta is the difference between “the AI is unreliable” and “the AI is the best search interface we have.”

When NOT to add late interaction

A reranker is not always the answer. Skip it when:

• Recall is already above 90% (you are past the useful range; further gains come from prompt or generation work).
• The corpus is small enough (under 50,000 chunks) that a cross-encoder reranker (BGE-reranker, Jina-reranker) gets you most of the same lift with less infrastructure.
• The latency budget is hard-sub-100ms (a real-time conversational interface where the reranker overhead breaks the UX).
• The team does not have a vector DB that supports multi-vector with MaxSim, and migrating is not on the roadmap.

For most production RAG systems sitting at recall in the 60% to 75% range, late interaction is the next move. Cross-encoder rerankers are the lighter alternative if the team is not ready for the full ColBERT setup.

What changed for the founder

The fix took half a day. We added a Jina-reranker stage between their existing Qdrant retrieval and the LLM call. Recall jumped from 58% to 81%. Faithfulness (because the LLM was now seeing better context) went from 0.79 to 0.93. The fine-tuning project was cancelled the same week.

The next conversation is whether to upgrade the cross-encoder to a full ColBERT setup, which would push recall another 4 to 6 points based on what we have seen on similar corpora. For their current scale and budget, the cross-encoder is the right floor. Full ColBERT is the v2.

If your retrieval has plateaued

If your team has been swapping embedding models and watching the recall curve flatten, the bottleneck is almost certainly the architecture, not the model. Sapota runs a one-week reranker integration engagement that adds the cross-encoder or ColBERT stage as a working PR plus a side-by-side eval against the current setup.

Reach out via the AI engineering page with the recall numbers you are seeing and the embedding models you have already tried. The diagnosis is usually the same conversation.

A founder asked us last month which mobile framework was “the best.” We get the question often enough that we have a rehearsed answer: none of them is best. Each fits a specific set of constraints, and picking the wrong one for your specific situation usually costs 2 to 4 months of rework eighteen months in.

The framework matters less than the fit. The fit depends on your timeline, your team, your design fidelity needs, and whether you are optimizing for the next twelve weeks or the next five years. Here is the decision framework Sapota walks every founder through.

The three options on the table

For most B2C and B2B mobile products in 2026, the realistic shortlist is FlutterFlow, React Native (with Expo), or Flutter native (writing Dart directly). Native iOS / Android (Swift, Kotlin) is still the right call for some specific cases, but it is rarely the default anymore. We will cover when native does win at the end.

The three frameworks differ on three dimensions that matter most: development speed, runtime control, and team future-proofing.

FlutterFlow

A visual development tool built on top of Flutter. You design screens by dragging components, configure data sources visually, and FlutterFlow generates Dart code under the hood. Custom logic drops into Dart custom actions and custom widgets when the visual editor cannot express what you need.

Strengths:

• Fastest path to a shippable MVP. 12-week marketplace MVPs are realistic for a 3-engineer team.
• Pixel-perfect design implementation. Figma to FlutterFlow is closer to one-to-one than any other framework we have used.
• Theming and white-label setup are first-class concepts. Multi-tenant apps with brand variables ship in days, not weeks.
• The founder can open the editor and request changes that you implement in minutes. Tightest design-to-development feedback loop in the industry.

Where it pushes back:

• Anything custom-rendered (canvas drawing, complex animations, game-like UI) needs Dart custom widgets, which pulls you out of the visual flow.
• State management at scale (3+ user roles, multi-tenant theming, real-time updates) outgrows app-state primitives. You end up moving state to the backend and treating the app as a thin client.
• Performance profiling is shallow. No equivalent to React Native’s Flipper or Flutter’s DevTools. You debug performance through guesswork.
• Vendor lock-in is real. Migrating off FlutterFlow means rebuilding in Flutter or another framework, not exporting and continuing.

Pick FlutterFlow when:

• MVP timeline is under 16 weeks
• 80%+ of screens are standard mobile UI patterns (auth, lists, details, forms)
• Backend is decoupled (Supabase, Firebase, custom API)
• Your team accepts that custom Dart will be needed for the bespoke 20%
• Visual fidelity matters and you want predictable results across iOS and Android

Skip FlutterFlow when:

• The product needs heavy real-time interactivity (multiplayer games, collaborative drawing, live markets)
• Your engineering team will need to maintain the app for 5+ years and you want them comfortable with the underlying framework directly
• You already have a strong React Native or Flutter team; the framework switch cost is higher than the visual development savings

React Native (with Expo)

Facebook’s framework for building native mobile apps using JavaScript and React. With Expo’s tooling (EAS Build, EAS Update, OTA patches), you get a managed workflow that handles a lot of native build complexity automatically.

Strengths:

• Largest ecosystem of any cross-platform framework. Almost any native API has a JavaScript wrapper. Stuck on something? There is a npm package or a community fix.
• JavaScript / TypeScript familiarity. Your web team can ramp on mobile faster than learning Dart or Swift.
• OTA updates via Expo. Push bug fixes without going through App Store review. For early-stage products iterating weekly, this is significant.
• Mature debugging tools (Flipper, React DevTools, native debuggers).
• Best long-term hire-ability. JavaScript engineers are everywhere.

Where it pushes back:

• Performance ceiling lower than Flutter or native for graphics-heavy or animation-heavy apps. The JavaScript bridge is faster than it used to be (with Hermes and the new architecture) but still not native.
• Native module compatibility issues when you upgrade React Native. Major upgrades are rarely smooth. We budget 2-4 weeks for upgrades on production apps.
• Visual fidelity work is more iterative than FlutterFlow. Designs render close but you spend more time on platform-specific edge cases (Android shadows, iOS safe areas, font rendering).
• Easy to ship a slow app accidentally. The framework forgives sloppy patterns until production load reveals them.

Pick React Native when:

• Your team has strong JavaScript / TypeScript background
• The product needs OTA update capability (frequent iteration without store review)
• You want hire-ability and ecosystem maturity over raw performance
• The app is content-heavy or transactional (most B2B SaaS, content apps, e-commerce)
• You expect 3+ years of active development with team turnover

Skip React Native when:

• The app is graphics or animation-heavy
• You need consistent 60+ fps under load
• Your team has no JavaScript expertise and you would be teaching from scratch
• The product is highly platform-specific (deep iOS-only or Android-only integrations)

Flutter native

Writing Dart directly against Flutter’s framework, no visual builder. Same underlying technology as FlutterFlow but with full control over every line of code.

Strengths:

• Best performance of the three options. Compiles to native code, no JavaScript bridge, predictable 60+ fps even with complex UI.
• Single codebase compiles to iOS, Android, web, desktop. The most truly cross-platform option of the three.
• Beautiful animation framework. If your app’s differentiator is motion design or visual polish, Flutter native gives you the most expressive primitives.
• Strong typing with Dart catches more bugs at compile time than JavaScript.
• DevTools profiler is excellent. Performance debugging is straightforward.

Where it pushes back:

• Slower to ship MVP than FlutterFlow. You write every screen by hand. For an MVP that FlutterFlow ships in 12 weeks, Flutter native usually takes 16-20.
• Smaller ecosystem than React Native. Most native APIs have Flutter packages, but the long tail is smaller. You will write more native plugins yourself.
• Hire-ability is improving but still behind JavaScript. Senior Flutter engineers in 2026 are easier to find than in 2022 but still command a premium.
• Dart is a mid-popularity language. Comfortable for most engineers within a few weeks but not a transferable skill outside Flutter.

Pick Flutter native when:

• Performance requirements are strict (60+ fps under all load conditions)
• The app’s design includes complex animations or motion as a differentiator
• You want one codebase across iOS, Android, web, and possibly desktop
• The team is already Flutter-fluent or willing to invest in becoming so
• Long-term ownership of the codebase matters more than fastest-possible MVP

Skip Flutter native when:

• Your timeline is under 16 weeks for v1
• You have no Flutter or Dart experience and the team’s bandwidth to ramp up is limited
• The product is mostly standard CRUD screens (FlutterFlow ships these faster)
• You need OTA updates as a core capability (Flutter has this but ecosystem is less mature than Expo)

When native (Swift, Kotlin) still wins

We bring up native iOS / Android less often than we used to, but it has not disappeared. The cases where we still recommend native:

• Heavy platform-specific integrations: Apple Pay deep integration, ARKit / RealityKit, Watch app, CarPlay, App Clips, Live Activities. Same on Android: deep Auto, Wear OS, advanced camera features. Cross-platform frameworks reach these eventually but always with delay.
• Performance-critical apps with platform-specific optimization: heavy image processing, on-device ML, AR / VR. Native gives you direct access to Metal, Core ML, ARKit without bridge overhead.
• Apps where the team is already native and the cross-platform switch costs more than maintaining two codebases: rare but real. A team that has shipped 5 years of Swift will lose more by switching than by writing two apps.
• Projects where the App Store review process is hostile to cross-platform (some categories of apps face stricter review when they look or behave non-native).

For most B2C and B2B SaaS products in 2026, native is overkill. The cross-platform frameworks have closed the gap on 95% of use cases.

The decision matrix

Eight criteria, four options each.

MVP speed (how fast you can ship v1):

• FlutterFlow: Fastest
• React Native: Fast
• Flutter native: Medium
• Native (Swift/Kotlin): Slowest

Performance ceiling (frame rate under load):

• FlutterFlow: Medium
• React Native: Medium
• Flutter native: High
• Native: Highest

Hire-ability in 2026 (how easy to find engineers):

• FlutterFlow: Niche
• React Native: High
• Flutter native: Medium
• Native: High per platform (Swift or Kotlin separately)

OTA updates (ship fixes without store review):

• FlutterFlow: Yes via FlutterFlow Cloud
• React Native: Yes via Expo
• Flutter native: Limited
• Native: Not supported

Long-term codebase health (5+ year ownership):

• FlutterFlow: Risky, vendor lock-in
• React Native: Stable
• Flutter native: Stable
• Native: Stable

White-label / multi-tenant theming:

• FlutterFlow: Native first-class concept
• React Native: Doable with effort
• Flutter native: Doable with effort
• Native: Doable with effort

Custom animations (motion-heavy products):

• FlutterFlow: Limited
• React Native: Medium
• Flutter native: Strongest
• Native: Strongest per platform

Visual fidelity to Figma:

• FlutterFlow: Highest
• React Native: Medium-High
• Flutter native: High
• Native: Highest per platform

How Sapota approaches the choice

The mistake we see most often is teams picking a framework based on what their current developers happen to know, then spending the next two years working around its limitations. The framework should fit the product, not the team’s existing comfort zone.

Our mobile engineers are trained across all four stacks (FlutterFlow, React Native, Flutter native, and the native platforms). When a new project lands, we run a fit assessment that scores the project against each framework on the criteria above. The recommendation comes from the assessment, not from “what we happen to specialize in.”

This sometimes means we recommend FlutterFlow when the founder expected React Native, or React Native when they expected Flutter. The conversation is occasionally awkward (founders have read articles, formed preferences) but it saves the months of rework that come from picking the wrong framework. We have done that rework on inherited projects often enough to know what it costs.

The cross-training is deliberate. A vendor who only knows FlutterFlow will recommend FlutterFlow. A vendor who only knows React Native will recommend React Native. The unbiased recommendation is the differentiator, and the only way to give it honestly is to be fluent across the options.

A simple decision tree

If you do not want to walk through the full matrix, the rough heuristic:

• MVP in under 16 weeks, standard CRUD screens, single brand or simple white-label → FlutterFlow
• 3+ year codebase, JavaScript team, frequent iteration with OTA → React Native
• Performance-critical, animation-heavy, or codebase needs to span iOS / Android / web → Flutter native
• Heavy platform-specific integration or your team is already native and shipping → Native

When in doubt, FlutterFlow for MVP, React Native for the production rebuild if you outgrow FlutterFlow’s limits.

If you are picking a stack right now

If your team is debating mobile frameworks and the conversation is not converging, the issue is usually that nobody has put the project’s specific constraints on the table next to each framework’s strengths. The decision is not abstract; it is a concrete fit for this product, this timeline, this team.

Sapota offers a one-week mobile stack assessment that takes your product spec, scores it against the four options, and returns a recommendation with the reasoning written down. We have done this for marketplaces, internal tools, content apps, and B2B SaaS products. The recommendation is unbiased because our engineers are trained across all four stacks.

Reach out via the mobile app development page with a description of what you are building, your timeline, and your team’s current language background. The first conversation usually surfaces the right fit within thirty minutes.