Distributors processing hundreds of purchase orders daily across multiple warehouses need approval workflows that handle real variation: item categories with different approval thresholds, vendor risk tiers that escalate approval, amount-based routing, and exception handling for urgent procurements that can’t wait for the normal chain.

Three approaches get proposed. Two fail the “hundreds of POs daily” test the moment they’re tried at scale.

What doesn’t scale

Basic workflows with manual override allowances. Fine for small volumes where individual attention fits. Fails when buyers override thresholds casually to keep POs moving, eroding the internal controls the workflow exists to enforce. Audit findings follow.

Allow urgent orders to bypass workflows entirely, log approvals via email. Creates a parallel approval system that nobody can audit. Urgent becomes “whenever a buyer feels pressured.” Internal controls become advisory. Inevitable SOX or internal-audit finding.

Excel for procurement rules externally, upload approved POs to D365. Shifts the problem outside the system entirely. No enforcement in D365, no audit trail of why specific POs were approved, manual data entry at upload time. Scales poorly and introduces data integrity risk.

The pattern that handles the volume

Procurement and Sourcing workflows configured with rule-based conditions, exception escalations, and procurement category hierarchies.

Components of the working architecture:

Procurement categories are the taxonomy that drives category-specific approval. Office supplies, IT hardware, capital equipment, raw materials – each category can have its own approval rules. The category is set on each PO line; the rule engine evaluates against the line’s category.

Rule-based workflow conditions on the PO workflow:

• Amount thresholds (stepped: under $X, $X-Y, over Y)
• Category-specific thresholds (capital goes up faster than office supplies)
• Vendor performance rating (poor-rated vendor requires senior approval)
• First-time vendor flag (always requires senior approval)
• Buyer-specific overrides (certain buyers have higher blanket authority)

Exception escalation for urgent procurement:

• “Urgent” flag on the PO triggers a fast-path workflow with delegated approval
• Senior-buyer delegation list configured for after-hours approvals
• Exception is still workflow-approved; just routes differently than normal

Approval hierarchies tied to position – the approver is the person in a specific organizational position, not a named individual. When the person in the position changes, the workflow still works without reconfiguration.

Workflow scalability at hundreds-per-day volume

Hundreds of POs per day tests workflow performance. Specific configuration matters:

• Batch evaluation for workflow conditions – avoid complex X++ in conditions; precompute rule-relevant values in custom fields
• Parallel approval where multiple approvers can act simultaneously
• Conditional approval so low-risk POs skip unnecessary steps
• Auto-approval for well-defined routine categories within tight thresholds (e.g., office supplies under $500 to an approved vendor – no human approval needed)

Auto-approval isn’t laziness when it’s explicitly configured for scenarios that present no controls risk. The workflow still generates the audit record.

Vendor performance integration

Using vendor performance in approval rules requires:

• Vendor performance scoring in the vendor master, refreshed by a scheduled job pulling PO on-time, quality, and completeness metrics
• Score thresholds defining the risk tiers (green, yellow, red)
• Workflow condition reading the vendor’s current tier at PO submission
• Override with justification for buying from a red-tier vendor (sometimes unavoidable; requires extra approval + documented reason)

This makes vendor performance a live signal, not a quarterly review finding.

Exception handling for urgent procurement

The urgent path has to be fast and still auditable:

• Urgent flag requires buyer to record justification
• Fast-track approval routes to senior buyer or VP of procurement
• After-hours delegation – when primary approver is unavailable, pre-configured delegate can approve
• Post-hoc review – urgent POs flagged for procurement-committee review next week

Compliance accepts this because the audit trail is intact. Internal controls aren’t bypassed; they’re expedited.

Reporting and continuous improvement

The workflow produces data for ongoing tuning:

• Approval time per category – where’s the bottleneck?
• Urgent-flag frequency per buyer – is one buyer over-using the escape hatch?
• Vendor-performance-driven escalations – which vendors trigger frequent extra approvals? Time to negotiate or replace them.
• Threshold analysis – are the current thresholds too loose or too tight based on actual approval reject rates?

Quarterly workflow review with procurement leadership uses this data to refine rules.

Integration with budget controls

PO approval and budget control work together but aren’t the same. Budget control blocks POs that would exceed budget; approval workflow routes POs that are within budget for sign-off. Both need to fire:

• Budget check first – if it fails, PO can’t submit for approval
• Approval workflow second – if budget check passes, routes per rules
• Budget update on approval – reserves the amount so subsequent POs see reduced availability

Standard functionality, but the sequence and interaction need architect attention during configuration.

What ships with the pattern

A working high-volume PO approval setup has:

• Procurement category hierarchy defined and maintained
• Rule-based approval workflows with category, amount, vendor, and buyer dimensions
• Urgent-procurement fast path with documented controls
• Auto-approval configured for well-defined routine scenarios
• Vendor performance scoring integrated into approval conditions
• Budget control integrated with workflow
• Reporting on approval metrics for quarterly review

Standard D365 workflow engine. The architect’s work is in the rules, categories, and governance – not in custom workflow code.

One of the most confusing bugs in SFMC: the customer updated their email in the client’s system, the nightly file import brought the new address into the Data Extension, but emails still go to the old address.

We’ve seen this on multiple engagements. Takes someone who’s seen it before to diagnose in under an hour.

The two-list architecture

SFMC keeps email addresses in two independent places:

1. Data Extension – the email column of whichever DE the send uses.
2. All Subscribers List – the account-level master list, keyed by Subscriber Key.

When SFMC sends, and the subscriber’s Subscriber Key already exists in the All Subscribers List, it uses the email from the All Subscribers List – not the email in the Data Extension.

So even if the DE has the new email, the All Subscribers List still holds the old one. SFMC follows the old one, email goes to the wrong address.

The mental model: think of the DE as a working copy and All Subscribers as the master register. The Send Engine never trusts the working copy when the master has an entry for that Subscriber Key – the master’s value always wins.

Why it’s designed this way

The All Subscribers List is SFMC’s single source of truth for who a subscriber is at the account level. Subscribers can belong to multiple DEs and multiple sends, but their “identity” (Subscriber Key -> email) lives once in All Subscribers.

When you send from a DE, SFMC reconciles the DE rows against All Subscribers. If there’s a conflict (different email for the same Subscriber Key), the All Subscribers version wins.

This is actually useful in normal operation – it prevents inconsistent email addresses drifting between multiple DEs. It’s only surprising when you’re trying to update the email.

Audit: how many wrong sends are queued up?

Before fixing the pipeline, find out the size of the existing problem. Run this SQL Query Activity against your sending DE joined to the _Subscribers data view to count subscribers whose DE email no longer matches the All Subscribers email:

SELECT de.SubscriberKey, de.EmailAddress AS DE_Email, sub.EmailAddress AS AllSubscribers_Email FROM [Your_Sending_DE] de INNER JOIN _Subscribers sub ON de.SubscriberKey = sub.SubscriberKey WHERE de.EmailAddress <> sub.EmailAddress

Each row in the result is a subscriber currently set up to receive sends at the wrong address. If the count is high, that’s also the size of the silent goodwill loss already running through your account.

Save the result into a sync DE (call it Subscriber_Email_Sync) – it’s the input for the fix below.

The fix: update the All Subscribers List too

Updating the DE alone isn’t enough. You also need to push the email change into the All Subscribers List. Three methods, in order of how we usually deploy them:

Method 1: Scheduled sync via Import Activity

For ongoing pipelines, build an Automation that pushes the audit DE into All Subscribers:

Schedule (daily or hourly, after the upstream DE refresh) -> SQL Query Activity (the audit query above, output to Subscriber_Email_Sync DE) -> Import File Activity (target: All Subscribers List, NOT a DE) (update type: Add and Update) (key: Subscriber Key) (fields: SubscriberKey, EmailAddress)

The Import Activity targeting All Subscribers is the critical part. By default the destination picker drops you into a DE – you have to switch to All Subscribers explicitly at step 1 of the activity setup. This is the single click most teams miss the first time and then spend an hour wondering why the email column never updates.

Method 2: Direct UI import for one-off fixes

If you just need to fix a single batch of stale records (not set up an ongoing pipeline), use the Import Subscriber Wizard from Email Studio > Subscribers > All Subscribers > Import. Upload a file with Subscriber Key + EmailAddress. Update type: Add and Update.

The wizard does the same thing as the Import Activity – this method just skips the Automation Studio plumbing for ad-hoc cleanups.

Method 3: AMPscript UpsertContacts for real-time

For real-time updates from a Cloud Page, API call, or Journey Custom Activity, AMPscript has UpsertContacts() that pushes into the All Subscribers side directly:

%%[ var @rows set @rows = CreateArray() set @row = CreateObject(“Property”) SetObjectProperty(@row, “ContactKey”, @subscriberKey) SetObjectProperty(@row, “EmailAddress”, @newEmail) Add(@rows, @row) UpsertContacts(@rows) ]%%

Use this when DE updates are too slow for the use case (e.g. a customer changes email on a self-service portal and immediately triggers a confirmation email).

What about brand new subscribers?

If a Subscriber Key doesn’t yet exist in the All Subscribers List, SFMC auto-creates the record using the email from the DE on the first send. No override problem because there’s nothing to override.

The bug only appears when the Subscriber Key already exists with an old email value – which is also why it tends to surface late. Things look fine for weeks while the audience is mostly first-send subscribers, until enough customers update their email and a few of them complain.

Diagnosing it on a live ticket

When marketing ops files a “customer reports wrong address” ticket, the first place to check is All Subscribers – not the DE. Order of operations:

1. Query the DE for that Subscriber Key. Confirm DE has the new email.
2. Open Subscribers > All Subscribers in Email Studio, search for the Subscriber Key. Read the email value stored there.
3. If DE has new email but All Subscribers has old, you’ve found it.
4. Test: edit the All Subscribers record manually (it’s editable in the UI), do a test send. If it goes to the new address, diagnosis confirmed.

The screenshot shows the All Subscribers panel with the Subscriber Key column visible — this is the table you read against, not the DE, when diagnosing wrong-address tickets.

This is the single shift that saves teams about a day per case – skip debugging the DE, skip the import logs, go straight to All Subscribers.

Prevention on new engagements

Include an All Subscribers sync in the data pipeline design from day one. Whenever customer email updates flow into SFMC, they should update both the relevant DE(s) and the All Subscribers List. A separate Import Activity targeting All Subscribers runs alongside the DE import, on the same schedule.

Document this in the runbook so it doesn’t get dropped when someone new picks up the engagement. We add it to every SFMC handover doc precisely because it’s the kind of thing that takes a fresh engineer a full day to rediscover otherwise.

Takeaway

The DE is not the source of truth for email addresses – the All Subscribers List is. Update both when email addresses change. Build the sync into the data pipeline from day one, not as a patch after the first angry client call.

Shipping an SFMC subscriber management pipeline? Our Salesforce team sets up All Subscribers sync, Contact Builder attributes, and identity resolution on production engagements. Get in touch ->

See our full platform services for the stack we cover.

The Power Apps Component Framework (PCF) lets you drop a custom React component onto a Dataverse form or a Canvas app and treat it as if it were native. The official getting-started docs walk you through a working “hello world” in 20 minutes. The gap from there to something you would put into a managed solution and ship to production is wider than the tutorials admit.

We have built eleven PCF controls across client projects. Each taught us something we baked into a template repo. The current version of that template – React 18, TypeScript strict mode, Jest, ESLint, and a CI pipeline that produces a solution zip – is what we reach for on every new control.

Here is what is in it and why.

The shape of the template

Nothing exotic. The details matter more than the shape.

Why React and TypeScript (strict)

PCF originally shipped with plain TypeScript and DOM-native rendering. React is optional but heavily supported – Microsoft’s Fluent UI library assumes React, and the productivity gain for anything beyond a simple input is significant.

We use React 18 with the concurrent rendering features (createRoot rather than ReactDOM.render). This matters for controls that need transitions or suspense boundaries. If you ship with React 17, you opt out of features the newer Fluent UI components assume.

TypeScript strict mode is non-negotiable. PCF exposes ComponentFramework.Context, and the input/output types are generated from the manifest. Without strict mode, you can pass the wrong type to context.parameters.xxx and find out at runtime. With strict mode, the compiler rejects it.

The ControlManifest decisions

Three choices in the manifest make the template shareable across projects:

1. Virtual control (): the modern shape, React-aware, required for React 18 compatibility.
2. Dataset-bound or field-bound variants of the template: two separate starter branches because the Context API is subtly different between the two.
3. Resource declarations include a localization strings file (resx): every user-facing string gets a key, never hardcoded. Internationalization later is free.

Services: the wrapper layer over context.webAPI

PCF exposes context.webAPI for Dataverse calls. The raw API is callback-style, untyped, and verbose. Wrapping it in a typed service with Promise-returning methods is five minutes of work that pays off on every subsequent call.

Components import the service, never the raw webApi. Testing becomes possible because the service is a pure class that can be mocked.

The test harness

PCF’s runtime environment is not Node – it is a host page in Dynamics or a Canvas app. You cannot run a PCF control outside of that host directly. But you can test the React components and the service classes in isolation.

Jest plus @testing-library/react covers 80% of what matters:

The pattern: pass services in as props, not as new DataverseService(context.webApi) inside the component. Dependency injection at the top level makes every component test-mockable.

We require 70% line coverage on new PCFs. Not because the number is magic, but because a coverage floor forces the team to structure code so it can be tested.

ESLint rules that earn their slot

The base config we use:

no-floating-promises catches the most common PCF bug: calling webApi.retrieveRecord without awaiting it, which silently fails on error. no-explicit-any forces proper typing of the context API surface. exhaustive-deps catches stale-closure bugs in useEffect.

The CI pipeline

Three jobs run on every pull request:

1. Install + build: npm ci then npm run build (compiles TypeScript and runs the PCF build step). Fails if build errors.
2. Test: npm test (Jest) + coverage report. Fails if coverage drops below 70%.
3. Solution pack: uses pac pcf push or the CreatePowerPlatformSolution pipeline task to produce a solution zip artifact.

The solution zip is the artifact we deploy from. A pull request into main triggers a build that produces a versioned zip; the release pipeline then imports that zip into the target environment.

Things we stopped doing

• Bundling Fluent UI’s entire core library. The size hit on the PCF bundle is noticeable on slow connections. We import only the components we use (Button, TextField, specific icons).
• Running TypeScript in –strict false mode to “move faster.” It was always slower in the end.
• Skipping manifest versioning. The PCF manifest has a version field. Bumping it on every deploy was annoying until we had a PCF fail to load because the cached version matched the new version number – forcing users to hard-refresh. Now the version bumps automatically in CI.

When to not use PCF at all

Not every customization needs a PCF. Model-driven forms cover 80% of requirements with out-of-the-box components. Canvas apps offer drag-and-drop custom layouts with Power Fx. PCF is for:

• Interactive visualizations that Power Fx cannot produce (complex charts, map overlays)
• Third-party JavaScript library integrations (Chart.js, specific mapping SDKs)
• Custom form controls that replace a native field with richer UX (a date range picker, a search-autocomplete)

If the requirement maps to a Fluent UI or out-of-box component, use the native option. PCF adds a build pipeline, a deployment artifact, and a code surface to maintain. The threshold for choosing PCF should be “no other Power Platform option fits.”

The deliverable the client actually cares about

After all the engineering, the client handoff is: “here is a managed solution zip. Import it and your new field control appears on the form.” They do not know or care about React 18, strict mode, or Jest.

But every time they ask for a change six months later, the coverage, the linting, and the test suite are what make the change possible in hours instead of days. That is the real value of the template.

If you are starting your first PCF, skip past the hello-world tutorials and invest half a day in this shape. The hello-world code will become the first thing you have to throw away.

A SaaS founder asked us to second-opinion a roadmap his CTO had come back with after an agentic RAG vendor pitch. The plan was eighteen tools, a planner LLM, an executor LLM, a critic LLM, and a synthesis LLM, with autonomous multi-step reasoning over their support and CRM data. The vendor’s slides showed accuracy lifts from 67% to 89%. The implied infrastructure bill was around 5x their current model spend.

The product was a real-time chat assistant for their internal sales team. The latency budget was three seconds end-to-end. The team had not measured what fraction of production queries actually required multi-step reasoning.

We told the founder to ship a router first and the agent second, only on the queries that needed it. That conversation cut the projected agentic budget by roughly 80% while preserving most of the accuracy lift. Here is the framework.

What agentic RAG actually does

Naive RAG is a single retrieval followed by a single generation: embed the query, fetch the top-k, prompt the LLM, return the answer. Total cost is roughly two LLM calls (one to embed, one to generate). Total latency is the sum of the embedding, vector search, and generation.

Agentic RAG breaks the query into sub-tasks and runs the retrieve-and-reason loop multiple times. A typical pattern:

1. Planner breaks the query into sub-questions.
2. Retrieval runs once per sub-question.
3. Reasoner synthesizes intermediate results.
4. Executor decides whether to retrieve more, run a calculation, or call a tool.
5. Critic checks the answer against the question and triggers a revision if needed.
6. Synthesizer combines everything into the final response.

Each step is an LLM call. A typical agentic loop runs 3 to 8 LLM calls per query. Some patterns (ReAct with reflection, multi-agent collaboration) run 10 to 20.

The accuracy lift is real. On genuinely multi-step queries, agentic RAG outperforms naive RAG by 20 to 40 percentage points in published benchmarks and in the audits we have run. The cost lift is also real: 5x to 15x the LLM spend per query, plus 3x to 10x the latency.

The fraction-of-queries question

The conversation the vendor pitch never starts with: what fraction of production queries actually require multi-step reasoning?

For most B2B SaaS products we have audited, the breakdown looks roughly like this:

• 65 to 75% of queries are single-hop lookups. “What is the refund policy?” “How do I export reports?” “What does error code 502 mean?” These are textbook RAG questions. A naive pipeline answers them correctly at 85% recall and a fraction of the agentic cost.
• 15 to 25% of queries are multi-step. “Which enterprise customers complained about pricing in Q4?” “Compare the deployment guides for AWS and Azure and tell me which is faster to set up.” These benefit from agentic patterns. The accuracy lift is real and the latency hit is acceptable because the queries are inherently slower-thinking.
• 5 to 10% of queries are not RAG questions at all. They are SQL queries disguised as natural language (“total MRR by region last quarter”) or transactional requests (“create a ticket for me”). These should be routed to a dedicated agent or directly to the database, not to a RAG pipeline.

Running an agentic pipeline on the 65% to 75% of queries that are single-hop is paying 10x cost for no accuracy lift. The agent loop just does more steps to arrive at the same answer the naive pipeline produces.

What we ship instead

Sapota’s default for any production RAG product where agentic capabilities are on the roadmap:

Step 1: build the naive RAG pipeline first. Hybrid search, reranker, faithfulness gate, evaluation. This is the floor. Everything else is layered on top.

Step 2: add a query router. A small fast LLM (Llama 3.1 8B is plenty) classifies each incoming query as single-hop, multi-hop, or structured. This costs one extra LLM call per query, around 50ms of latency. The router accuracy needs to be measured and tracked, because misroutes are expensive in both directions.

Step 3: dispatch by class. Single-hop queries go to the naive pipeline. Multi-hop queries go to the agentic pipeline. Structured queries go to a SQL agent or whatever non-RAG tool fits.

Step 4: build the agentic pipeline only for the queries that need it. This is where the vendor pitch starts. The point is that the agent is not the entire product. It is one branch the router dispatches to.

The cost math: if 70% of queries hit the naive pipeline at $0.005 per query and 25% hit the agent at $0.05 per query, the blended average is $0.016 per query. If everything hits the agent, it is $0.05. The router pays for itself many times over.

When the agent is genuinely the product

Some products are built around multi-step reasoning by design. The agent is not an upgrade, it is the core value proposition. Cases we have seen and recommend full-agent for:

• Research assistants that take a complex question and produce a structured report (think: AI consultant for due diligence). The user expects 30-second responses. Multi-step reasoning is the value.
• Code agents that read a codebase, plan a refactor, and execute it across multiple files. The agent’s planning and tool-use capabilities are the entire product.
• Customer support agents that need to read account history, query backend systems, generate responses, and escalate when uncertain. The decision-making is the work.

For these products, the question is not whether to use agents, but how to keep the agent loop efficient (fewer steps, smarter tools, better caching).

For everything else (most B2B SaaS RAG products, most internal knowledge assistants, most customer-facing chatbots), the router-then-agent pattern is the right architecture.

The latency conversation

Agentic RAG is slow. A 5-step loop with 2-second LLM calls is a 10-second response. For batch use cases (overnight report generation, async customer email triage), this is fine. For real-time chat, it is the end of the user experience.

Mitigation patterns we use:

• Streaming intermediate steps to the user so the perceived latency is the time to first token, not the time to final answer. Users will wait if they see progress.
• Smaller models in the loop. The planner and the critic do not need GPT-4. Llama 3.1 8B or Haiku is enough. Reserve the expensive model for the final synthesis.
• Aggressive caching of intermediate results. Sub-questions that recur across queries (which they do, in narrow domains) hit a cache instead of re-running.
• Concurrent tool calls where the steps are independent. Most agent frameworks support parallel tool dispatch. Many implementations do not use it.
• Hard step limits. Cap the loop at 5 steps. If the agent has not converged by then, return a partial answer with a “I could not fully answer this; here is what I found” caveat. Better than a 30-second timeout.

These are the techniques that make agentic patterns viable for products with sub-10-second latency budgets.

What we sent the founder

The recommendation was three lines:

1. Ship the naive pipeline first. Spend two weeks getting it to 85% recall on the eval set.
2. Add the router as the second sprint. Measure misroute rate; aim for under 5%.
3. Build the agent only for the multi-hop branch. Start with a 3-step ReAct loop. Add reflection only if the eval shows it is worth the cost.

The vendor’s eighteen-tool plan was a v3 conversation, not a v1. By the time the team has the routing and the agent for the multi-hop branch working, they will know which tools they actually need. Most of the eighteen will not survive contact with the production query log.

The founder’s CTO sent the vendor a polite “let’s revisit this in six months” email.

If a vendor is pitching agents at you

If your team is being told that agentic RAG is the next quarter’s roadmap, the question to ask before signing is what fraction of your production queries actually need it. If the vendor cannot answer that with your data, they are pitching a product, not a solution.

Sapota offers a one-week query distribution audit that takes your production query log, classifies the queries by complexity, and produces a sized recommendation for which fraction needs agents and which is better served by naive RAG with a router. The output is a budget-and-architecture document the team can take to the vendor, the board, or both.

Reach out via the AI engineering page with your current RAG architecture and approximate query volume. The diagnostic conversation almost always reframes the conversation from “agents or not” to “how much of the product is agents and what is the rest.”

Your engineering interview loop was designed for a world where the bottleneck was writing code. The best engineers in 2026 are not the fastest coders — they’re the ones with the best judgment about what to build, how to direct AI toward the right outcome, and when to override what it produced.

The job title still says “engineer,” but the role has fractured. The person you actually need is part architect, part product thinker, part operator, and part team multiplier — designs systems before touching a keyboard, kills bad ideas before they become features, plans for the 3am page before it fires, makes the people around them better. AI writes the code. The human decides whether the code should exist at all.

Augment Code put it well in their hiring framework: the human role has shifted from author to architect and editor. You define intent, make trade-off decisions, set guardrails, and serve as the last line of quality. Raw coding ability no longer separates exceptional engineers from competent ones — regardless of whether they’re writing backends, frontends, data pipelines, or infrastructure.

CoderPad’s 2026 State of Tech Hiring confirms this from the demand side — technical assessments are up 48% globally, and 82% of developers find AI at least somewhat useful. Companies leading in AI are hiring more engineers, not fewer. The bottleneck isn’t code generation. It’s judgment, systems thinking, and the ability to work alongside AI without losing control of the outcome.

If your interview loop still centers on “can this person write a balanced binary tree on a whiteboard,” you’re selecting for a skill that AI does better than any human. Here’s what to select for instead.

The rubric

Six dimensions, each with sub-criteria you score independently. 3 = strong, 2 = adequate, 1 = weak. Open the scorecard (Google Sheet).

1. Architecture & design

Can the candidate think at the level of systems, not just functions? These apply whether someone is building APIs, frontends, data pipelines, or infrastructure.

• Trade-off analysis under ambiguity — can they explain why one approach beats another in terms the business cares about, not just technical preference?
• Failure mode awareness — do they design for the 3am outage, or only the happy path?
• Scale reasoning — do they ask about constraints (current bottlenecks, data patterns, cost) before proposing solutions?
• API and contract design — can they design interfaces that other teams can integrate safely?
• Data modeling and storage decisions — do they choose the right storage for the access pattern, not just the storage they know?

2. AI-assisted execution

Can the candidate direct AI tools toward the right outcome — and catch it when the output is wrong?

• Directs AI toward well-defined subtasks — uses AI for implementation while keeping control of the overall design. Treats AI as a tool, not an oracle.
• Critically reviews and debugs AI output — can explain why AI-generated code is or isn’t correct. This is the single strongest signal.
• Clear boundary between AI-delegated and human-owned work — knows which tasks to hand off and which to protect. No boundary means no judgment.
• Understands AI limitations in production contexts — knows where models hallucinate, where context windows break, where AI-generated code introduces security or concurrency risks.

3. Systems & operations

Does the candidate understand what makes code production-worthy?

• Observability — thinks about monitoring, alerting, and SLOs from day one. Monitors what the user cares about, not what the default dashboard provides.
• Debugging methodology — has a systematic approach: logs, metrics, traces, reproduction. Asks what changed recently before guessing.
• Cost awareness — considers compute, storage, and API call costs as design constraints, not afterthoughts.
• Incident response and on-call mindset — has a mental model for what happens after the code is merged. Designs for operability.

4. Product & problem selection

Does the candidate solve the right problems, or just the ones assigned?

• Asks about users and business context before building — wants to know the success metric, not just the spec.
• Pushes back on problem framing when appropriate — would rather solve a simpler problem well than an interesting problem poorly.
• Chooses simple solutions over interesting ones — the boring choice that saves the team real time or pain.

5. Learning velocity

How fast does the candidate adapt when tools and practices change underneath them?

• Has adopted and discarded tools based on results — not just tried things, but evaluated and made deliberate choices.
• Can articulate what changed in their workflow and why — adoption without displacement usually means the tool isn’t actually being used.
• Stays current without hype-chasing — runs personal experiments but doesn’t chase every new release.

6. Culture & collaboration

AI amplifies individual output — but a 10x individual who can’t collaborate is a net negative. Score this separately from technical ability.

• Curiosity — asks questions, explores unfamiliar territory, wants to understand the system beyond their immediate scope.
• Navigates disagreement constructively — describes what they learned from technical disagreements, not just how they won them.
• Communicates technical decisions to non-engineers — can explain trade-offs in terms a PM or exec can act on.
• Shares context on AI-generated code with the team — flags AI-generated sections in PRs, documents reasoning, takes responsibility for the review burden.
• Mentors or lifts others — impact isn’t just individual output. Improves the people around them.
• Holds opinions loosely — has strong technical convictions but adapts to team norms and new information.

The questions

Fourteen questions, grouped by what they assess. Each one includes what you’re listening for and the red flag.

Architecture & design

1. “Walk me through a system you designed that had to handle a non-obvious failure mode. What was the failure, and how did your design account for it?”

Listen for: specifics about the failure mode, the trade-off they made, and whether they designed for it proactively or reactively. Strong candidates talk about the failure before it happened.

Red flag: only discusses happy-path architecture.

2. “You’re designing a service that needs to process 10x its current load within 6 months. Where do you start?”

Listen for: questions before answers. Strong candidates ask about current bottlenecks, data access patterns, and cost constraints before proposing solutions. They think about what not to change as much as what to change.

Red flag: immediately proposes a technology (“just use Kafka”) without understanding the constraints.

3. “Tell me about a time you chose a boring solution over an interesting one. Why?”

Listen for: a clear trade-off between technical elegance and operational simplicity. The best answer is a story where the boring choice saved the team real time or pain.

Red flag: can’t think of one. Engineers who always choose the interesting solution are expensive to operate alongside.

AI-assisted execution

4. “Walk me through your daily workflow. Where does AI fit in, and where doesn’t it?”

Listen for: specificity. Strong candidates can name exact tools, describe which tasks they delegate to AI, and — critically — which tasks they don’t. The boundary matters more than the tools.

Red flag: vague answers like “I use Copilot for everything.” No boundary means no judgment.

5. “Tell me about a time AI-generated code was wrong in a way that wasn’t obvious. How did you catch it?”

Listen for: a real story with a real failure mode. Strong candidates describe a subtle bug — a race condition, a missed edge case, a security hole — that they caught through review, testing, or domain knowledge. This is the single strongest signal for AI-assisted work: the ability to be the quality backstop.

Red flag: “that hasn’t really happened to me.” It has. They just didn’t notice.

6. “If I gave you a new codebase you’ve never seen and asked you to add a feature using AI tools, how would you approach it?”

Listen for: a process that starts with understanding, not prompting. Strong candidates talk about reading the existing code, understanding the architecture, then using AI for implementation — not pasting requirements into a chat window and hoping.

Red flag: leads with “I’d prompt the AI to…” without any mention of understanding the system first.

Systems & operations

7. “Your service is throwing 500s for 2% of requests. Walk me through your debugging process.”

Listen for: a systematic approach — logs, metrics, traces, reproduction. Strong candidates ask what changed recently, check deployment history, and think about partial failures. Let them use AI tools during the discussion. Watch whether they use AI to accelerate diagnosis or to replace thinking.

Red flag: guesses without data. “It’s probably the database” is not a debugging process.

8. “How do you decide what to monitor in a new service?”

Listen for: a framework, not a checklist. Strong candidates think about SLOs, user-facing metrics, and leading indicators of failure — not just CPU and memory. They monitor the thing the user cares about, not the thing the infrastructure provides by default.

Red flag: “whatever the default dashboard gives us.”

Product & problem selection

9. “Tell me about a feature you decided not to build. What was the reasoning?”

Listen for: business judgment. The feature was technically feasible but wrong for the user, the timeline, or the system’s current maturity. Strong candidates kill their own ideas.

Red flag: can’t think of a feature they chose not to build. This suggests they build whatever’s asked without filtering.

10. “A PM asks you to add a feature that you think is a bad idea. What do you do?”

Listen for: pushback with evidence, not ego. Strong candidates describe how they’d frame the concern — data, user impact, operational cost — and what they’d do if overruled. The answer to “what if they still want it” matters as much as the initial pushback.

Red flag: “I’d just build it, they’re the PM.” Or the opposite: “I’d refuse.” Neither is a real answer.

Learning velocity

11. “What AI tool or workflow did you adopt in the last 6 months that meaningfully changed how you work? What did it replace?”

Listen for: a concrete change in behavior, not a tool name. Strong candidates describe what they stopped doing when they adopted the new thing. Adoption without displacement usually means the tool isn’t actually being used.

Red flag: names a tool but can’t describe the workflow change. “I started using Cursor” is not an answer. “I stopped writing boilerplate tests by hand because Cursor generates them and I review them” is.

12. “What’s something AI tools are bad at today that people assume they’re good at?”

Listen for: nuance and firsthand experience. Strong candidates have bumped into real limitations — context window issues, hallucinated APIs, broken concurrency patterns — and can describe them specifically. This question reveals whether someone has used AI tools enough to know where they break.

Red flag: “AI is pretty good at everything now.” It is not.

Culture & collaboration

13. “Tell me about a technical disagreement you had with a teammate. How did it resolve, and would you do anything differently?”

Listen for: respect for the other person’s view, willingness to change their mind, and a resolution that wasn’t just “I was right.” Strong candidates describe what they learned, not just how they won.

Red flag: every story ends with them being right. Or they can’t recall a disagreement — which means they either avoid conflict or don’t notice it.

14. “How do you share context with your team when you’ve used AI to generate a significant chunk of code?”

Listen for: a process — PR descriptions that call out AI-generated sections, documentation of the reasoning, flagging areas that need extra review. Strong candidates recognize that AI-generated code shifts the review burden to the team and own that.

Red flag: “I just push it like any other code.” That’s how AI-generated bugs become the team’s problem.

Red flags to watch across the loop

A few patterns that span the whole loop:

Tool fluency without technical depth. The candidate is fast with AI tools but can’t explain the code they produce. This is the most dangerous hire in 2026 — someone who ships faster but embeds hidden defects. Canva’s interview team found the same thing: the best candidates don’t just prompt — they ask clarifying questions, use AI for well-defined subtasks, and critically review output.

No boundary between AI tasks and human tasks. Every engineer needs a clear mental model of what they delegate and what they don’t. If someone uses AI for everything indiscriminately, they’ve abdicated judgment.

Resistance to AI as identity. Some candidates treat not using AI as a badge. In 2026, this is like refusing to use an IDE. It doesn’t demonstrate skill — it demonstrates rigidity.

Can’t debug AI output. If you ask them to review AI-generated code and they can’t identify issues, they’re a liability in a codebase where AI writes the first draft.

The heuristic

You’re not hiring a coder anymore. The engineer you need in 2026 is an architect who designs systems before writing them, a product thinker who kills bad ideas before they become tickets, an operator who plans for the 3am page, and a teammate who makes everyone around them better. AI handles the typing. You’re hiring for everything else. Interview for judgment, not keystrokes — the candidate who pushes back on AI output is worth more than the one who prompts their way to a solution.

tl;dr

The pattern. Engineering interviews still optimize for raw coding ability — a skill AI now handles — while ignoring the architect, product thinker, operator, and collaborator the role actually demands.
The fix. Score candidates across six dimensions (architecture, AI-assisted execution, systems & ops, product taste, learning velocity, culture) with specific sub-criteria, and use questions that reveal how they direct, evaluate, and override AI output.
The outcome. You hire engineers who think like architects, challenge like PMs, operate like SREs, and use AI as a tool — not engineers who either ignore it or depend on it blindly.

Originally published at b0gy.com

Go’s built-in testing package
provides a powerful, minimalist framework for writing unit tests without external dependencies.
Here are the testing fundamentals, project structure, and advanced patterns to build reliable Go applications.

Why Testing Matters in Go

Go’s philosophy emphasizes simplicity and reliability. The standard library includes the testing package, making unit testing a first-class citizen in the Go ecosystem. Well-tested Go code improves maintainability, catches bugs early, and provides documentation through examples. If you’re new to Go, check out our Go Cheat Sheet for a quick reference of the language fundamentals.

Key benefits of Go testing:

• Built-in support: No external frameworks required
• Fast execution: Concurrent test execution by default
• Simple syntax: Minimal boilerplate code
• Rich tooling: Coverage reports, benchmarks, and profiling
• CI/CD friendly: Easy integration with automated pipelines

Project Structure for Go Tests

Go tests live alongside your production code with a clear naming convention:

myproject/
├── go.mod
├── main.go
├── calculator.go
├── calculator_test.go
├── utils/
│   ├── helper.go
│   └── helper_test.go
└── models/
    ├── user.go
    └── user_test.go

Key conventions:

• Test files end with _test.go
• Tests are in the same package as the code (or use _test suffix for black-box testing)
• Each source file can have a corresponding test file

Package Testing Approaches

White-box testing (same package):

package calculator

import "testing"
// Can access unexported functions and variables

Black-box testing (external package):

package calculator_test

import (
    "testing"
    "myproject/calculator"
)
// Can only access exported functions (recommended for public APIs)

Basic Test Structure

Every test function follows this pattern:

package calculator

import "testing"

// Test function must start with "Test"
func TestAdd(t *testing.T) {
    result := Add(2, 3)
    expected := 5

    if result != expected {
        t.Errorf("Add(2, 3) = %d; want %d", result, expected)
    }
}

Testing.T methods:

• t.Error() / t.Errorf(): Mark test as failed but continue
• t.Fatal() / t.Fatalf(): Mark test as failed and stop immediately
• t.Log() / t.Logf(): Log output (only shown with -v flag)
• t.Skip() / t.Skipf(): Skip the test
• t.Parallel(): Run test in parallel with other parallel tests

t.Log is for human-readable test diagnostics. In running services, log/slog and JSON-friendly records are usually a better match for aggregation and incident debugging. See Structured Logging in Go with slog for Observability and Alerting.

Table-Driven Tests: The Go Way

Table-driven tests are the idiomatic Go approach for testing multiple scenarios. With Go generics, you can also create type-safe test helpers that work across different data types:

func TestCalculate(t *testing.T) {
    tests := []struct {
        name     string
        a, b     int
        op       string
        expected int
        wantErr  bool
    }{
        {"addition", 2, 3, "+", 5, false},
        {"subtraction", 5, 3, "-", 2, false},
        {"multiplication", 4, 3, "*", 12, false},
        {"division", 10, 2, "/", 5, false},
        {"division by zero", 10, 0, "/", 0, true},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            result, err := Calculate(tt.a, tt.b, tt.op)

            if (err != nil) != tt.wantErr {
                t.Errorf("Calculate() error = %v, wantErr %v", err, tt.wantErr)
                return
            }

            if result != tt.expected {
                t.Errorf("Calculate(%d, %d, %q) = %d; want %d", 
                    tt.a, tt.b, tt.op, result, tt.expected)
            }
        })
    }
}

Advantages:

• Single test function for multiple scenarios
• Easy to add new test cases
• Clear documentation of expected behavior
• Better test organization and maintainability

Running Tests

Basic Commands

# Run tests in current directory
go test

# Run tests with verbose output
go test -v

# Run tests in all subdirectories
go test ./...

# Run specific test
go test -run TestAdd

# Run tests matching pattern
go test -run TestCalculate/addition

# Run tests in parallel (default is GOMAXPROCS)
go test -parallel 4

# Run tests with timeout
go test -timeout 30s

Test Coverage

# Run tests with coverage
go test -cover

# Generate coverage profile
go test -coverprofile=coverage.out

# View coverage in browser
go tool cover -html=coverage.out

# Show coverage by function
go tool cover -func=coverage.out

# Set coverage mode (set, count, atomic)
go test -covermode=count -coverprofile=coverage.out

Useful Flags

• -short: Run tests marked with if testing.Short() checks
• -race: Enable race detector (finds concurrent access issues)
• -cpu: Specify GOMAXPROCS values
• -count n: Run each test n times
• -failfast: Stop on first test failure

Test Helpers and Setup/Teardown

Helper Functions

Mark helper functions with t.Helper() to improve error reporting:

func assertEqual(t *testing.T, got, want int) {
    t.Helper() // This line is reported as the caller
    if got != want {
        t.Errorf("got %d, want %d", got, want)
    }
}

func TestMath(t *testing.T) {
    result := Add(2, 3)
    assertEqual(t, result, 5) // Error line points here
}

Setup and Teardown

func TestMain(m *testing.M) {
    // Setup code here
    setup()

    // Run tests
    code := m.Run()

    // Teardown code here
    teardown()

    os.Exit(code)
}

Test Fixtures

func setupTestCase(t *testing.T) func(t *testing.T) {
    t.Log("setup test case")
    return func(t *testing.T) {
        t.Log("teardown test case")
    }
}

func TestSomething(t *testing.T) {
    teardown := setupTestCase(t)
    defer teardown(t)

    // Test code here
}

Mocking and Dependency Injection

Interface-Based Mocking

When testing code that interacts with databases, using interfaces makes it easy to create mock implementations. If you’re working with PostgreSQL in Go, see our comparison of Go ORMs for choosing the right database library with good testability.

// Production code
type Database interface {
    GetUser(id int) (*User, error)
}

type UserService struct {
    db Database
}

func (s *UserService) GetUserName(id int) (string, error) {
    user, err := s.db.GetUser(id)
    if err != nil {
        return "", err
    }
    return user.Name, nil
}

// Test code
type MockDatabase struct {
    users map[int]*User
}

func (m *MockDatabase) GetUser(id int) (*User, error) {
    if user, ok := m.users[id]; ok {
        return user, nil
    }
    return nil, errors.New("user not found")
}

func TestGetUserName(t *testing.T) {
    mockDB := &MockDatabase{
        users: map[int]*User{
            1: {ID: 1, Name: "Alice"},
        },
    }

    service := &UserService{db: mockDB}
    name, err := service.GetUserName(1)

    if err != nil {
        t.Fatalf("unexpected error: %v", err)
    }
    if name != "Alice" {
        t.Errorf("got %s, want Alice", name)
    }
}

Popular Testing Libraries

Testify

The most popular Go testing library for assertions and mocks:

import (
    "github.com/stretchr/testify/assert"
    "github.com/stretchr/testify/mock"
)

func TestWithTestify(t *testing.T) {
    result := Add(2, 3)
    assert.Equal(t, 5, result, "they should be equal")
    assert.NotNil(t, result)
}

// Mock example
type MockDB struct {
    mock.Mock
}

func (m *MockDB) GetUser(id int) (*User, error) {
    args := m.Called(id)
    return args.Get(0).(*User), args.Error(1)
}

Other Tools

• gomock: Google’s mocking framework with code generation
• httptest: Standard library for testing HTTP handlers
• testcontainers-go: Integration testing with Docker containers
• ginkgo/gomega: BDD-style testing framework

When testing integrations with external services like AI models, you’ll need to mock or stub those dependencies. For example, if you’re using Ollama in Go, consider creating interface wrappers to make your code more testable.

Benchmark Tests

Go includes built-in support for benchmarks:

func BenchmarkAdd(b *testing.B) {
    for i := 0; i < b.N; i++ {
        Add(2, 3)
    }
}

// Run benchmarks
// go test -bench=. -benchmem

Output shows iterations per second and memory allocations.

Best Practices

1. Write table-driven tests: Use the slice of structs pattern for multiple test cases
2. Use t.Run for subtests: Better organization and can run subtests selectively
3. Test exported functions first: Focus on public API behavior
4. Keep tests simple: Each test should verify one thing
5. Use meaningful test names: Describe what is being tested and expected outcome
6. Don’t test implementation details: Test behavior, not internals
7. Use interfaces for dependencies: Makes mocking easier
8. Aim for high coverage, but quality over quantity: 100% coverage doesn’t mean bug-free
9. Run tests with -race flag: Catch concurrency issues early
10. Use TestMain for expensive setup: Avoid repeating setup in each test

Example: Complete Test Suite

package user

import (
    "errors"
    "testing"
)

type User struct {
    ID    int
    Name  string
    Email string
}

func ValidateUser(u *User) error {
    if u.Name == "" {
        return errors.New("name cannot be empty")
    }
    if u.Email == "" {
        return errors.New("email cannot be empty")
    }
    return nil
}

// Test file: user_test.go
func TestValidateUser(t *testing.T) {
    tests := []struct {
        name    string
        user    *User
        wantErr bool
        errMsg  string
    }{
        {
            name:    "valid user",
            user:    &User{ID: 1, Name: "Alice", Email: "[email protected]"},
            wantErr: false,
        },
        {
            name:    "empty name",
            user:    &User{ID: 1, Name: "", Email: "[email protected]"},
            wantErr: true,
            errMsg:  "name cannot be empty",
        },
        {
            name:    "empty email",
            user:    &User{ID: 1, Name: "Alice", Email: ""},
            wantErr: true,
            errMsg:  "email cannot be empty",
        },
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            err := ValidateUser(tt.user)

            if (err != nil) != tt.wantErr {
                t.Errorf("ValidateUser() error = %v, wantErr %v", err, tt.wantErr)
                return
            }

            if err != nil && err.Error() != tt.errMsg {
                t.Errorf("ValidateUser() error message = %v, want %v", err.Error(), tt.errMsg)
            }
        })
    }
}

Useful Links

• Official Go Testing Package Documentation
• Go Blog: Table-Driven Tests
• Testify GitHub Repository
• GoMock Documentation
• Learn Go with Tests
• Go Code Coverage Tool
• Go Cheat Sheet
• Structured Logging in Go with slog for Observability and Alerting
• Comparing Go ORMs for PostgreSQL: GORM vs Ent vs Bun vs sqlc
• Go SDKs for Ollama – comparison with examples
• Building CLI Applications in Go with Cobra & Viper
• Go Generics: Use Cases and Patterns

Conclusion

Go’s testing framework provides everything needed for comprehensive unit testing with minimal setup. By following Go idioms like table-driven tests, using interfaces for mocking, and leveraging built-in tools, you can create maintainable, reliable test suites that grow with your codebase.

These testing practices apply to all types of Go applications, from web services to CLI applications built with Cobra & Viper. Testing command-line tools requires similar patterns with additional focus on testing input/output and flag parsing.

Start with simple tests, gradually add coverage, and remember that testing is an investment in code quality and developer confidence. The Go community’s emphasis on testing makes it easier to maintain projects long-term and collaborate effectively with team members.

See the App Architecture hub for related guides on Go project structure, dependency injection, API design, and integration patterns.

Moving from traditional coding to agentic coding requires a large mindset shift.

As a developer, your job transitions into an orchestrator and reviewer role. You design the ticket, hand it off to an agent like Claude Code to implement, review what was done, and request changes.

A major side effect of this shift is that traditional agile velocity estimations can no longer be calculated the way they once were.

Historically, cost estimates focused mostly on the time it would take a human developer to build out the code. With agentic coding, the time to write becomes an almost non-factor; even a large ticket can take 30 to 60 minutes to implement, instead of days. However, that doesn’t mean you can push out production-ready software in that window.

First, the developer must collaborate with the agent to design what it is going to do. After that comes the heaviest time sink: code review. Reviewing what an AI wrote to ensure it maintains architectural standards and strictly achieves the business intent requires an entirely different type of energy.

The bottleneck is no longer how fast a developer can write code; it is the developer’s cognitive capacity to review it thoroughly.

A notorious adage in software development describes this perfectly (often called Parkinson’s Law of Triviality, or Bikeshhedding): a 5-line code change gets deeply scrutinized in a PR, while a 10,000-line change gets checked off without much thought. This stems directly from the massive cognitive strain that goes into a large code review.

With agentic coding, this cognitive fatigue arises much sooner in the process; occurring continuously during development rather than being back-loaded to a post-development pull request.

Instead of measuring velocity in terms of implementation time, teams must start viewing it through the lens of cognitive complexity.

How do you calculate cognitive complexity?

After a lot of thought about this I think the complexity should be broken up into 5 categories, and within those categories, a rating of low, medium, high.

• Planning complexity
• Review surface area
• Cognitive load
• Test complexity
• Ambiguity

1. Planning Complexity

How difficult will the planning session be? How many design decisions need to be locked down before the AI can start?

A simple CRUD endpoint with an obvious implementation path is Low. A feature that touches three architectural layers, requires a new abstraction, or has a data migration is High. Planning complexity determines how long the /plan session takes and how much rework happens if it goes wrong.

2. Review Surface Area

How many files will a reviewer need to read? How many lines of code?

This is measurable. Under five files and 200 lines is Low; a focused change a reviewer can hold in their head. Over ten files or 500+ lines is High; and at that point you’re asking reviewers to maintain the full picture of a system they didn’t design while looking for subtle AI mistakes.

3. Cognitive Load

Review surface area is a reading-time problem. Cognitive load is a comprehension problem. They’re different.

A three-file change that touches your authentication and permission system can be harder to review than a fifteen-file change that adds a new resource with standard CRUD. Cognitive load measures how much system knowledge a reviewer must hold simultaneously to catch the errors an AI is most likely to make.

An isolated, self-contained change is Low. Something that requires understanding a state machine, a set of permission rules, and an async coordination pattern simultaneously is High; because the risk isn’t that any individual piece is hard, it’s that one wrong assumption about how the pieces interact creates a bug that passes all the tests.

4. Test Complexity

AI writes tests too, but someone still has to design the test scenarios, verify that the mocks actually reflect the system’s behaviour, and confirm that edge cases are covered. More scenarios, more external service stubs, and more integration requirements all increase the human overhead here.

5. Ambiguity

This one is different from the others. When a human developer hits an ambiguous requirement mid-implementation, they make a judgment call or ask a question. When an AI hits an ambiguous requirement, it makes a confident guess; and the guess is often plausible enough that nobody catches it in review.

High ambiguity in a ticket doesn’t just add a fixed overhead. It multiplies all the other costs, because a poorly-defined ticket is likely to produce a result that needs reworking; and rework means running the entire planning-review-QA cycle again.

Turning Scores Into a Number

Measuring dimensions is useful, but planning boards need a number. Here’s a formula that works.

Score each execution dimension (Planning, Review Surface, Cognitive Load, Test Complexity) using a non-linear scale; Low=1, Medium=3, High=6; and sum them. This is your base score, ranging from 4 to 24.

The non-linearity is deliberate. Complexity doesn’t scale evenly: a reviewer holding one complex mental model isn’t just 50% more taxed than one holding none; they’re three to four times as taxed, because there’s no margin for error. High=6 reflects that disproportionate cost. Two Highs together compound further still: a reviewer must hold both difficult models simultaneously in a single pass, which is qualitatively harder than doing each separately.

Then apply an ambiguity multiplier:

• Low ambiguity: ×1.0 (no adjustment; the ticket is well-defined)
• Medium ambiguity: ×1.25 (some gaps; expect some iteration)
• High ambiguity: ×1.5 (rework likely; the AI will miss the intent)

Round the result to the nearest integer, then map it to velocity points.

The calibration anchor matters here: these points represent human overhead hours, not implementation hours. Using a conventional 1 point = 4 hours sprint velocity, the mapping looks like this:

If your team uses a different anchor (2 hours per point, 8 hours per point), slide the boundary ranges accordingly; the formula stays the same, only the lookup table shifts.

These aren’t implementation hours. They’re human-overhead hours; the time that real people spend planning, reviewing, and verifying work the AI produced.

Calculating on a ticket by ticket basis

This formula isn’t straightforward, and asking developers to calculate this will take time and experience with agentic coding, Once they are familiar with their work habits and experiences with agentic coding, they will be able to figure it out just fine.

Until that happens, Product managers and Dev managers still are going to want to know with some accuracy, what the likely ticket velocity will be, and calculating it based on the amount of coding required won’t arrive at the right answer (or maybe it will but for the wrong reasons)

For that reason I created a small skill for Claude Code that takes a PM built ticket requirement list, and goes through the planning of the ticket, using both the tickets ask and context of the code base, to pre plan the ticket and calculate the weights. This could be something run by a Dev Manager, or Principal developer, or team lead before the ticket reaches a backlog meeting. It could highlight changes that need to be made before presenting the ticket to the team, potential vertically slicing the ticket if its a high complexity, or fold other tickets into one if it’s low.

The plan generated by the estimator wouldn’t be the final plan, that should be left to the developers to work through, and deliver. Developers have their own systems for building features, even when AI is involved. However, an initial plan can give Product Managers the information they need to create sprints that deliver in an agentic space.

Alleviating bottlenecks

Developers can help some of the cognitive overheads by implementing processes during development. Specifically, teams can create agentic review skills that compare generated code against the claude.md specifications, architectural rules, and original ticket plan before a human ever sees the code.

Doing this will mean that the AI agent can self review its code and find holes in it before the developer needs to look at it. Having a First-pass AI skill can help remove the obvious issues and leave the developer only needing to look at the deeper things, rather than having to look for both.

Helpful visual tool

I am providing a link to a helpful estimate tool, so you can see the difference various combinations make to the overall velocity. Hopefully seeing it in action will help it make sense!

https://lawrence72.github.io/claude-tools/calculator/

Enterprise deals do not have ten line items. They have two hundred, sometimes more. The same CPQ quote page that loads in 2 seconds for a small quote takes 30 to 45 seconds for a large one. Sales reps avoid editing because every change triggers a recalculation that hangs the browser. Quotes get split into multiple smaller quotes (because the system cannot handle one big one), which then have to be reconciled manually.

The bottleneck is rarely “CPQ is slow in general.” It is usually one of five specific places, and the fix preserves the pricing logic rather than throwing it away.

Bottleneck 1: Price Rules with broad evaluation scope

Price Rules evaluate on quote calculation. Each rule with broad criteria (e.g., “applies to all quote lines”) iterates over every line. With 200 lines and 30 Price Rules, that is 6000 evaluations per recalculation.

The diagnostic: count Price Rules in the org. Count rules with no scope filter or with overly broad scope filters. The combination of high rule count and broad scope is the smoking gun.

The fix: tighten Price Rule scope. Each rule should apply to the smallest set of quote lines it needs. A rule that adjusts discount on a specific product family should filter to that product family, not run for every line. The scope filter is free; not having one is expensive on every recalculation.

Sapota has seen orgs reduce recalculation time by 60 percent through scope tightening alone, without changing any business logic.

Bottleneck 2: Product Rules running unnecessarily

Product Rules validate quote line combinations (constraints like “Product A requires Product B” or “Product C is incompatible with Product D”). They fire on every change, evaluating the full set of lines.

A poorly scoped Product Rule that runs across all lines for every change adds visible latency at the 200-line scale. Same diagnostic as Price Rules: high count, broad scope, high cost per recalculation.

The fix: convert validation-style Product Rules to CML Constraints in RLM (if the migration is in scope), or in CPQ Classic, narrow the evaluation scope per rule. The pattern is identical to Price Rule tightening.

Bottleneck 3: Custom Quote Calculator Plugin (QCP)

CPQ Classic supports a Quote Calculator Plugin written in Apex that runs during calculation. The QCP can add custom logic that the standard pricing engine does not handle natively.

Three patterns make QCP slow at scale:

• The plugin iterates over quote lines multiple times when one pass would suffice. Refactor to a single pass.
• The plugin makes SOQL queries inside loops. Bulkify all queries before the loop.
• The plugin processes lines in serial when parallel processing is safe. Use Apex async patterns where possible.

The QCP is the most common deep performance bottleneck. Code review by someone who knows both CPQ behavior and Apex optimization typically produces a 3 to 5x improvement.

Bottleneck 4: Synchronous calculation on every change

CPQ Classic recalculates on every quote line change. For 200 lines, this means a full recalculation when the rep edits a single discount.

CPQ supports asynchronous calculation: defer the recalculation until the rep explicitly triggers it. The trade-off is that the rep sees stale numbers between changes, but for large quotes this is preferable to a 30-second hang on every keystroke.

The configuration: enable “Calculate quote on calculate action only” in CPQ Configuration Settings. Train reps to click Calculate after a batch of changes rather than after each one. The change is operationally meaningful but acceptable on quotes large enough to need it.

For mid-size quotes (50 to 100 lines), keep synchronous calculation. The threshold for switching depends on the specific org’s rule complexity, but 100 to 150 lines is the common breakpoint.

Bottleneck 5: Smart Approvals queries on large quotes

Approval evaluation runs on quote submission. Smart Approvals (RLM) and Advanced Approvals (CPQ Classic) both evaluate rules against the full quote state. With 200 lines and complex approval criteria, evaluation latency adds to the perceived slowness.

The diagnostic: time the approval evaluation separately from the calculation. If submission takes 30 seconds and calculation took 2, the bottleneck is approval logic, not pricing.

The fix: review approval rule scope. Approval criteria should run on aggregated quote-level state (total amount, discount percentage) rather than iterating per line. Per-line approval criteria are rarely the right design and almost never necessary.

For RLM with Smart Approvals, the new parallel routing feature also helps: independent approval branches can run concurrently rather than serially.

A debugging sequence

The order to investigate a slow quote:

1. Measure first. Open browser devtools, time the recalculation. Note where the time goes (CPU during calculation, network during save, render after response).
2. Check Price Rule count and scope. Quick win if the count is high and scope is broad.
3. Check Product Rule scope. Same diagnostic.
4. Review QCP code if it exists. This is the deep dive; budget more time.
5. Try async calculation on a sample quote. If acceptable to the team, the configuration change is small.
6. Time approval evaluation separately. Address if it is the dominant factor.

Most slow-quote engagements resolve at step 2 or 3 with scope tightening, plus async calculation for the largest quotes. The QCP rewrite is the heavier intervention, reserved for cases where steps 2 to 5 are not enough.

Architecture decisions that prevent the problem

The patterns that keep quotes fast as they scale:

• Tight scope on every Price Rule and Product Rule. Default to “applies to this product family” rather than “applies to all lines.”
• QCP only when standard rules cannot do the job. Most QCP code can be replaced with better-scoped Price Rules.
• Async calculation as the default for new orgs anticipating large quotes. Sync is fine until it is not.
• Approval criteria at the quote level. Per-line approvals are an anti-pattern.
• Modular product bundles. A 200-line quote that is actually 20 bundles is faster to manage than 200 individual lines.

Quote performance is one of the most frequent reasons Sapota’s Salesforce team gets called in for an audit. The fixes are repeatable, and the gains are large. A 30-second quote becoming a 5-second quote changes how sales reps use the system.

Debugging quote performance in Salesforce CPQ or RLM? Sapota’s Salesforce team holds the Revenue Cloud Consultant credential and handles performance audits on production engagements. Get in touch ->

See our full platform services for the stack we cover.

This is a submission for the Gemma 4 Challenge: Build with Gemma 4

What I Built

I live with my wife and our 1.5 year old daughter. Both me and my wife are working and usually have busy schedules. My wife manages the stock keeping of grocery and daily essentials and our cook and maid frequently ask her to order stuff that is running out. These requests usually come on-the-fly, and my wife has to keep a mental note and then add it to her ordering list. Later when she gets time, she adds items to cart (usually the likes of Amazon, Blinkit etc). Many times she even forgets items and we end up compromising on some dishes which can’t be prepared because some ingredients are out of stock.

I built Piclu, a product that captures voice notes, and installed it in our kitchen. Now our cook and maid just press a button and leave a voice note whenever something comes to their mind. No runtime interruptions to my wife. The voice notes get transcribed and structured with Gemma 4 E4B, and the shopping list stays populated for my wife to see whenever she gets time. She then orders as appropriate. Our cook and maid are also happier as they don’t have to wait for my wife to be around to share stock outs and they also now have fewer things to keep in their memory.

For the voice capture – I programmed an M5Stack Atom Echo as a Push-To-Talk device and plugged it into the kitchen socket. It records the audio locally and then pushes it to the backend endpoint. Everything runs locally on a small home server, no cloud, so the audio never leaves home.

A fair question here would be: why not just ask the cook and maid to write down the requests or send a WhatsApp voice note? Two reasons. Writing it down is more friction than it sounds when your hands are busy in the kitchen, and not everyone is comfortable writing. And even if they did write it down somewhere, my wife would still have to read through it and add each item to the cart one by one. We tried WhatsApp voice notes as well but it was too much of an ask when they are in the middle of work and in a hurry.

Demo

In the video you can see the full loop – the helper presses the button by the counter and leaves a note in Hindi (“nimbu aur tamatar khatam hai”, out of lemons and tomatoes), Gemma 4 transcribes it onto the list locally, a second Gemma 4 pass turns the notes into a structured shopping list, and each item shows up on its own as a tickable to-do.

Code

PTT source ─► POST /api/items/freeform_audio   (multipart: audio)
           ─► app HTTP
           ─► brain gRPC over Unix socket
           ─► recordings.Store (seals .webm on disk + emits SealedEvent)
           ─► processor:  ffmpeg decode (WebM/Opus → 16 kHz mono PCM)
                          → Gemma 4 E4B pass 1 — TRANSCRIBE (multimodal audio →
                            Hinglish text,

The open sourced repo has the Gemma brain (transcription and structuring), the web UI, and the single audio endpoint that the M5Stack pushes to. It also has the full transcription and extraction eval I talk about below.

Under the hood it is a Go app and brain talking over a Unix socket, the cart and notes update live over SSE, and the whole thing runs from a Docker Compose. Gemma 4 runs locally through a libmtmd / llama.cpp build, with a Lemonade adapter for faster GPU inference (the eval uses the same adapter), and the eval is fully reproducible from the repo.

How I Used Gemma 4

Gemma powers 2 key steps in the note capturing flow – transcription and structuring. Both run as local Gemma 4 E4B passes on the home server.

Step 1: Transcription

When I started building this, I tried a few open source models and put them against each other on a small eval dataset I prepared from our own voice notes – Whisper, indic-whisper from AI4Bharat, and Gemma 4. The pure speech-to-text models had difficulty understanding the accents of our cook and maid when they spoke about our household items. Gemma 4 uniquely stands out, because of its native multimodality – it takes the audio and a text prompt together, so unlike a pure STT model I can actually tell it about what it is about to hear. Across the open-source landscape, Gemma 4 E2B/E4B are among the few models that support native audio and text input within a single prompt.

Among the Gemma variants, the bigger ones (26B MoE and 31B Dense) don’t take audio at all, and the smaller E2B is not accurate enough, its word error rate comes out above 1.0 and it drops about 36% of the items during structuring. So E4B was the variant that is both multimodal on audio and accurate enough for this job. In the eval it also beat Whisper-Large-v3, the gold standard for STT with global language support, both on accuracy and on staying quiet on noise (Whisper tends to invent text on silence, which would quietly pollute the list). Here is the head to head:

Lower WER and CER are better (WER is word error rate, CER is character error rate; CER is the fairer read here because Romanized Hindi has many valid spellings). No-speech detection is how often the model correctly stayed silent or flagged the clip as noise instead of inventing words – the shipped E4B prompt teaches it those no-speech flags, which is why it handles 93%. Full method and the rest of the numbers are in the repo eval.

The context I give it is specific to our home. I live in Gurugram, India and our cook and maid are from West Bengal and Bihar respectively. I prompt E4B with what is being said, the purpose, by whom and intended for whom, so it can steer the predicted tokens keeping the intent in mind. That one change is the single biggest lever – in my eval, adding this context reduced the word error rate by more than half compared to a plain “transcribe this” (the no-context row in the table above). The relevant part of the transcription prompt looks like this:

You are an expert audio transcriber specializing in {{native_language}} dialects.
You receive voice notes from members of a household in {{city}}, {{country}} —
typically the cook, household helps, or family members — addressed to the
household manager regarding the shopping list (stock requests, brand preferences,
quantity changes, cancellations).

The full prompts (transcription and structuring) are in the repo under internal/prompts.

Step 2: Structuring

This pass turns the notes into clean items to be ordered. Notes can arrive with linked context as well, for e.g:

• Note 1: The dishwash liquid is running out.
• Note 2: The current dishwash liquid is not good, the earlier one was better.

I keep an SQL table keyed by a name_id, with kind, size, quantity, stock_state and notes. I pass the new transcript and the existing table items to Gemma and it generates a small DSL that the code then parses to insert or update items in the table:

• Note 1 returns: UPSERT name_id="dishwash-liquid" name="dishwash liquid" stock_state="low"
• Note 2 returns: UPSERT name_id="dishwash-liquid" name="dishwash liquid" notes="current one is not good, the earlier one was better"

Both notes map to the same name_id, so the table ends up with a single merged row:

dishwash-liquid stock_state="low" notes="current one is not good, the earlier one was better"

If a note is not about the shopping list at all, Gemma emits NO_OP and nothing changes. In the eval, this structuring pass had a recall of 1.0 (it found every product) and an F1 of 0.99, and it never invented items from non-shopping notes.

Next I plan to build MCP servers for the online ordering platforms and then eventually auto populate the carts with the items to be ordered, contextualized using the order history on respective platforms. I am bullish that Gemma 4 can deliver here as well – it will come down to how well it does tool calling and how much its chain-of-thought thinking can optimize the ordering. Let’s see what happens, fingers crossed.

The Challenge of Selling Specialized Products Online

Whether you’re building a store for premium knives, artisanal goods, or other niche products, e-commerce developers face unique challenges. Unlike mass-market retailers, niche stores have smaller audiences but higher margins—making every conversion count. If you’re tasked with building or optimizing a specialty store, here’s what the data shows matters most.

1. Product Photography & Rich Media Strategy

Developers often overlook this, but product images are your most critical UX element for high-value items.

• High-resolution, multiple angles: Implement lazy loading with WebP/AVIF formats for fast load times
• Zoom functionality: Use libraries like react-medium-image-zoom for detailed inspection
• Video embeds: Short clips (15-30s) showing product in use can increase conversion by 40%+
• Descriptive alt text: Each image needs proper alt text for SEO, not just “product.jpg”

// Lazy-loaded image with modern formats
<img 
  src="knife-thumb.webp" 
  srcset="knife-small.webp 480w, knife-large.webp 1200w"
  loading="lazy"
  alt="Damascus steel hunting knife with 5-inch stainless blade"
/>

2. Structured Data & Rich Snippets

Schema markup is non-negotiable for niche products. Implement:

• Product schema: name, price, availability, rating
• AggregateRating: display star ratings if you have reviews
• BreadcrumbList: essential for category navigation and SEO
• FAQPage schema: answer common questions (materials, maintenance, warranty)

Well-optimized sites like nazis-duncis.lv/ show how proper schema implementation helps specialty retailers rank for competitive keywords.

3. Core Web Vitals: Performance is Non-Negotiable

For niche products, serious buyers spend time researching. Slow sites kill conversions.

• LCP (Largest Contentful Paint): aim for < 2.5s
• INP (Interaction to Next Paint): keep below 200ms
• CLS (Cumulative Layout Shift): < 0.1 to prevent layout jank

Run Lighthouse audits regularly. Optimize image delivery, defer non-critical JavaScript, and use edge caching for static assets.

4. SEO for Long-Tail Keywords

Niche products dominate through long-tail keywords. Create category pages with:

• 800-1200 words of guide content (not just product grids)
• Natural keyword integration: avoid keyword stuffing; write for humans first
• Contextual internal links: guide readers between related products and buying guides
• FAQ sections: answer real customer questions with structured data

5. Trust Signals for Premium Items

Buyers spending real money need confidence:

• Customer reviews with photos: Verified purchase badges increase trust by 73%
• Clear return policy: Make it prominent and hassle-free
• SSL/TLS 1.3: Non-negotiable for payment security
• Credible “About” page: Share your expertise and why customers should buy from you

Key Takeaway

E-commerce for niche products requires balancing technical excellence with psychology. Focus on rich media, blazing-fast performance, structured data, and genuine trust signals. The margins are there—your job is to remove friction.

Test everything. Measure conversions. Iterate.