ARES: Cut LLM Agent Reasoning Costs 52% Per Step

Agentic tasks are expensive because most steps don’t need heavy reasoning. Opening a URL, clicking a button, or reading a form field requires almost no chain-of-thought. But if you run a multi-step agent with fixed “high” reasoning throughout, you pay for deep reasoning on every trivial step.

ARES (Adaptive Reasoning Effort Selection, arXiv:2603.07915) addresses this directly. Rather than a fixed reasoning level across all steps, ARES uses a lightweight router to predict the minimum viable reasoning effort for each step — given what’s happened so far in the agent’s context. The result: 52.7% fewer reasoning tokens on TAU-Bench Retail, 41.8% fewer on BrowseComp-Plus, and 45.3% fewer on WebArena, with accuracy maintained against the fixed-high baseline.

Effloow Lab implemented a Python rule-based approximation of the ARES router and validated the step-level effort allocation pattern. See [data/lab-runs/ares-adaptive-reasoning-effort-llm-agents-2026.md] for the full PoC output. Full pipeline execution (fine-tuned router + LLM API + agent harness) was not run due to missing API keys and GPU resources.

The Problem ARES Solves

Modern reasoning-capable LLMs — Claude Sonnet 4 extended thinking, GPT-o3 high effort, Gemini 3.5 Flash dynamic thinking — all support configurable reasoning levels. A “high” setting triggers deep chain-of-thought reasoning. A “low” setting produces fast responses with minimal internal scratchpad.

The naive approach to cost reduction is to drop all steps to “low.” This works for simple tasks but fails for agents: when a step requires conditional logic (should I paginate? is this the right product?), low-effort reasoning often picks the wrong branch, and the downstream steps compound the error.

The other naive approach is to stay at “high” everywhere. This preserves accuracy but is expensive — and unnecessary. ARES benchmarks show that in a typical WebArena task, roughly 40% of steps need “low” effort (navigation, field entry), 30% need “medium,” and only 30% genuinely require “high.”

The cost difference between levels is large. On Claude Sonnet 4 with extended thinking, a high-effort step might burn 1,800 reasoning tokens; a low-effort navigation step uses 200. At scale — thousands of agent tasks per day — the delta is significant.

How ARES Works

The ARES framework has three components:

1. Data generation pipeline: For each training task, ARES runs the agent at all three effort levels (high, medium, low) per step and labels each step with the minimum level that produced a correct outcome. This creates a dataset of (step_context, min_effort_level) pairs.

2. Router fine-tuning: A small classification model is trained on the labeled dataset to predict the minimum effort level given the current step context and interaction history.

3. Plug-and-play integration: At inference time, before each step, the router predicts the effort level. The agent then runs that step using only that reasoning budget.

The router is explicitly “lightweight” — the paper does not specify the model size, but the framing suggests something in the 100M–1B parameter range that adds negligible latency compared to the LLM itself.

Benchmark Results

All results from arXiv:2603.07915, compared to a fixed-high-effort baseline:

Benchmark Task type Token reduction Accuracy vs baseline
TAU-Bench Retail Tool-use agents 52.7% Maintained
BrowseComp-Plus Deep-research agents 41.8% Maintained
WebArena Web navigation agents 45.3% Maintained

“Maintained” means ARES matches fixed-high accuracy — the paper does not report that ARES degrades accuracy. This is the key claim: you get the cost reduction without a task success penalty.

The paper also compares ARES against alternative approaches: always-low effort (low cost, low accuracy), random effort selection (neither goal achieved), and static medium effort (partial reduction, partial accuracy loss). ARES outperforms all three on the cost-accuracy tradeoff frontier.

PoC: Implementing the Router Logic

Effloow Lab implemented a rule-based approximation of the ARES router to validate the step-level allocation pattern. The real ARES router is a fine-tuned model; this PoC uses explicit rules derived from the paper’s described features (context depth, branching presence, navigation type, tool complexity).

from dataclasses import dataclass
from typing import Literal

ReasoningLevel = Literal["high", "medium", "low"]

@dataclass
class AgentStep:
    step_id: int
    description: str
    context_depth: int      # how many prior steps referenced
    has_branching: bool     # requires conditional logic
    is_navigation: bool     # simple URL or click
    tool_complexity: int    # 0=none, 1=simple, 2=complex

def classify_step(step: AgentStep) -> ReasoningLevel:
    """
    Rule-based approximation of the ARES fine-tuned classifier.
    Real implementation fine-tunes a small model on (step_context, min_effort) pairs.
    """
    score = 0.0
    if step.tool_complexity == 2:  score += 3.0
    if step.has_branching:         score += 2.0
    if step.context_depth >= 5:    score += 1.5
    if step.tool_complexity == 1:  score += 1.0
    if step.context_depth >= 3:    score += 0.5
    if step.is_navigation:         score -= 2.5
    if step.context_depth == 0:    score -= 1.0

    if score >= 3.0:   return "high"
    if score >= 1.0:   return "medium"
    return "low"

Applied to a simulated 8-step WebArena task (open URL → locate search → enter query → parse results → decide pagination → extract data → validate → write output):

Step 1: low   — Open target URL (navigation, no context)
Step 2: low   — Locate search input (navigation)
Step 3: medium — Enter search query (tool_complexity=1)
Step 4: medium — Parse result listing (tool_complexity=1)
Step 5: high  — Decide whether to paginate (branching)
Step 6: high  — Extract structured data (tool_complexity=2)
Step 7: high  — Validate data completeness (branching, deep context)
Step 8: high  — Write result to output (tool_complexity=1, deep context)

Fixed-high baseline: 14,400 reasoning tokens
ARES router:          9,000 reasoning tokens
Token reduction:       37.5%

The PoC achieves 37.5% reduction against the paper’s 45.3% on WebArena — expected, since the rule-based heuristic is less precise than a fine-tuned model. The directional result confirms the paper’s core claim: navigation steps consistently route to low, branching/complex steps consistently route to high, and the per-step granularity drives meaningful savings.

Implementing ARES for Your Agent

The paper describes ARES as “plug-and-play for any LLM agent.” In practice, integration requires two pieces:

Step 1: Collect per-step effort labels. Run your agent tasks at all three effort levels and record which minimum level produced a correct step. This is the labeling pipeline. For a production agent, you’d run this on a representative sample of 100–500 tasks.

Step 2: Train a router. Fine-tune a small classification model on your labeled dataset. The input is the step context (current task description + prior N steps). The output is the effort level class. A 125M-parameter classifier is likely sufficient given the paper’s “lightweight” framing.

If fine-tuning is out of scope, a rule-based router like the PoC above recovers a meaningful portion of the savings. Our PoC gets ~82% of the full reduction (37.5% / 45.3% on WebArena). For high-volume production agents this is still worth deploying while collecting data for a fine-tuned version.

For the agent harness itself, the AutoTTS test-time scaling guide and the Chain of Draft minimal reasoning guide cover complementary approaches to reasoning cost reduction that can stack with ARES.

Limitations and What the Paper Doesn’t Address

The paper focuses on accuracy-vs-cost tradeoff, not latency. Using a router adds one inference call per step. For very fast agents (sub-second steps), the router overhead may negate some savings — the paper does not benchmark this.

The data generation pipeline requires running each step at all three effort levels, which means roughly 3x the usual agent evaluation cost to build the training set. For teams with existing agent evaluation infrastructure, this is manageable; for teams starting from scratch, it’s a meaningful upfront investment.

The router is trained on your specific agent tasks and benchmarks. Generalization to different task distributions is not studied — a router trained on WebArena may not transfer well to TAU-Bench without retraining.

ARES is worth implementing when:

  • You have a production agent running thousands of multi-step tasks per day
  • Your agent already uses a reasoning-capable model with configurable effort levels
  • You have a labeled evaluation set to train a router from
  • Even a 30–50% token reduction translates to meaningful cost savings at your volume

Use a rule-based approximation first when:

  • You want to test the approach before investing in fine-tuning infrastructure
  • Your tasks have a clear navigation/trivial vs. decision/complex split
  • A 35–40% reduction (rule-based) vs 45–52% (fine-tuned) is acceptable for now

FAQ

What is ARES?

ARES (Adaptive Reasoning Effort Selection, arXiv:2603.07915) is a framework that predicts the minimum reasoning effort needed for each step of a multi-step LLM agent task, reducing reasoning token cost 41–52% while maintaining task accuracy.

How does ARES differ from just using a lower reasoning level?

A static low-effort setting reduces accuracy significantly. ARES routes dynamically — high for decision steps, low for navigation. This preserves accuracy while cutting cost only where reasoning is genuinely unnecessary.

Does ARES require a specific LLM?

No. The paper describes it as plug-and-play for any LLM that supports configurable reasoning levels. This includes Claude extended thinking, GPT-o3 high/medium/low, and Gemini dynamic thinking.

What benchmarks did ARES use?

TAU-Bench Retail (tool-use agents), BrowseComp-Plus (deep-research agents), and WebArena (web navigation agents). All showed 40–52% token reduction with maintained accuracy against fixed-high baseline.

Where can I find the ARES paper?

arXiv:2603.07915 — submitted March 9, 2026. Available at arxiv.org/abs/2603.07915.

Gemini 3.5 Flash

I watched the Google I/O 2026 Developer Keynote, and honestly… the launch of Gemini 3.5 surprised me in more ways than one.

This model is faster AND more powerful than Gemini 1.5 Pro on nearly all benchmarks—including reasoning, coding, and long-running tasks.

What’s impressive is the performance-to-price ratio: it costs less than half the price of comparable models on the market. Ideal for those who want to integrate AI into their products without blowing their API budget.

Gemini 3.5 Flash is already available now on all Google products and APIs. And Gemini 3.5 Pro, even more powerful, is coming in June.

The era of agents has just accelerated.

youtube link

One MCP server, four clients, zero translation apps open in browser tabs

I built konid after translating work emails in Japanese for six months and realizing I was getting through each email but learning nothing. Same literal output every time, no sense of whether I was being too stiff or too casual, no audio to check whether I’d read the pronunciation right in my head.

The specific thing konid does differently: it returns three options per query, ordered casual to formal, with the register named and a comparison of the cultural nuance between them. Not “here’s your translation” — more like “here are the three ways a native speaker might say this, and here’s why you’d pick each one.” Audio plays through speakers via node-edge-tts, no external API key involved.

The angle I want to cover here is MCP client coverage, because this is the part that took the most work and matters most to how I actually use it.

konid runs as an MCP server. Install once:

claude mcp add konid-ai -- npx -y konid-ai

That gets you Claude Code. The same server config works in Cursor, VS Code Copilot, Windsurf, Zed, and JetBrains with their respective MCP settings. For Claude Cowork — the multi-agent workspace — you point it at the same server process. For ChatGPT, Developer mode accepts the hosted endpoint directly:

https://konid.fly.dev/mcp

The reason this matters: I switch between Claude Code (coding sessions), Claude Cowork (longer research), and ChatGPT depending on the task. Before, that meant context-switching to a browser tab, pasting text, getting a flat answer, closing the tab. Now the same tool is available wherever I’m already working. The language coach doesn’t move; I move.

I’ve tested it against 13+ languages including Mandarin, Japanese, Korean, Arabic, Spanish, French, and Hindi. The register distinction is sharpest in the languages that formalize it most explicitly — Japanese keigo levels, Korean speech styles, Arabic formal vs. colloquial — but it’s useful in Spanish and French too where casual vs. formal lands differently depending on the relationship.

MIT licensed. Source and full setup docs at https://github.com/robertnowell/konid-language-learning.

ERP Payment Security: Reducing PCI Scope Through Staged Authentication Architecture

Why Conventional ERP Payment Architecture Creates Unnecessary Scope

Most ERP payment integrations follow a deceptively simple flow:

Customer provides card data
→ ERP system receives full PAN + CVV + billing address
→ ERP forwards to payment gateway
→ Gateway returns authorization + token
→ ERP stores token for future transactions

The problem is in step two. The ERP system — with its order management modules, AR workflows, customer service interfaces, and recurring billing engines — receives the full Primary Account Number. Under PCI DSS v4.0, every system component that stores, processes, or transmits cardholder data, or that can impact the security of cardholder data, is in scope for the Cardholder Data Environment (CDE) assessment.

For a typical mid-market Dynamics 365 F&SCM environment, that means a non-trivial portion of your ERP is PCI-scoped: the order entry module, the AR payment journal, the customer service portal, the billing automation engine. Each one requires security controls, documentation, and audit validation.

The question architects should be asking is not “how do we secure all these systems?” but “why are all these systems receiving card data at all?”

Staged Authentication: The Architectural Alternative

Here’s the architectural contrast:

CONVENTIONAL FLOW:
[Customer] –full PAN + CVV–> [ERP Order Entry]
[ERP Order Entry] –full PAN–> [Payment Gateway]
[Payment Gateway] –token–> [ERP Token Store]

Result: ERP touched full PAN. ERP is in PCI scope.

STAGED AUTHENTICATION FLOW:
[Customer] –partial identifier + order metadata–> [ERP Order Entry]
[ERP] –session token + partial ref–> [Secure Hosted Page]
[Customer] –remaining card digits + out-of-wallet factor–> [Secure Hosted Page]
[Secure Hosted Page] –auth request–> [Payment Gateway]
[Payment Gateway] –persistent token + auth result–> [ERP]

Result: ERP never touched full PAN. ERP scope dramatically reduced. which typically occurs after full PAN receipt. Here, full PAN assembly never happens within the merchant’s systems at all.

PCI DSS v4.0 Scope Reduction: What Changes

Understanding scope reduction requires knowing how PCI DSS defines a Cardholder Data Environment (CDE).

Per PCI DSS v4.0 requirements, scope includes any system component that:
● Stores, processes, or transmits cardholder data or sensitive authentication data
● Can impact the security of cardholder data (connected-to systems)
● Provides security services to in-scope systems

Under a conventional integration, your ERP order entry module, AR interface, customer service portal, and recurring billing engine are likely all in scope because they interact with systems that handle full PANs.

Under a staged authentication model:

The merchant-side components handle only partial card fragments and order metadata. These do not constitute cardholder data under PCI DSS definitions (a partial PAN — fewer than the first 6/last 4 digits — falls outside the definition of a PAN that requires protection). This dramatically reduces which ERP components are in scope.

Practical SAQ impact:
For mid-market merchants completing SAQ-D (the most comprehensive questionnaire, applicable to merchants who don’t qualify for simpler SAQ types), staged authentication can shift eligibility toward SAQ-A or SAQ-A-EP, depending on integration specifics. SAQ-A applies when all payment page functions are outsourced to PCI-compliant third parties and the merchant doesn’t receive, process, or store card data. The staged model, implemented correctly, satisfies this structure for the card entry portion.

This has meaningful operational consequences — fewer systems to audit, lower QSA engagement costs, and reduced scope for security controls implementation.

Token Lifecycle for ERP-Native Recurring Transactions

Tokenization in an ERP context has requirements that simple e-commerce flows don’t. ERPs manage long-term customer relationships, recurring billing cycles, intercompany transactions, and multi-channel payment surfaces. The token architecture needs to be persistent, portable across modules, and consistent across channels.

INITIAL TRANSACTION — token issuance

function processInitialPayment(orderId, customerId, sessionRef) {

// ERP has only: orderId, customerId, partial card ref, session token
// No full PAN ever enters this function

const authResult = await paymentGateway.completeAuth({
sessionRef: sessionRef,
orderId: orderId,
amount: order.totalAmount
})

// Gateway returns persistent network token
const persistentToken = authResult.networkToken || authResult.gatewayToken

// Store token on customer/account record in ERP
await erp.customerPaymentMethods.upsert({
customerId: customerId,
token: persistentToken,
maskedPan: authResult.maskedPan,
cardNetwork: authResult.network,
expiryMonth: authResult.expiryMonth,
expiryYear: authResult.expiryYear
})

return { authorized: true, token: persistentToken }
}

SUBSEQUENT TRANSACTIONS — token reuse

function processRecurringPayment(customerId, invoiceId, amount) {

const paymentMethod = await erp.customerPaymentMethods.get(customerId)

// ERP submits token — gateway translates to PAN internally
const authResult = await paymentGateway.chargeToken({
token: paymentMethod.token,
amount: amount,
invoiceRef: invoiceId
})

await erp.paymentJournals.post({
invoiceId: invoiceId,
amount: amount,
maskedPan: paymentMethod.maskedPan,
token: paymentMethod.token,
authCode: authResult.authorizationCode,
transactionId: authResult.transactionId
})

return authResult
}

Key architectural properties:

  • ERP code path never contains a full PAN
  • Payment journals store token references only
  • Gateway handles token-to-PAN translation
  • AR and billing modules operate entirely on tokens

Authentication Factor Design: Beyond CVV/AVS

Standard CNP (card-not-present) verification relies on three factors that share a critical weakness: all of them can be captured in a single data breach.

Conventional CNP verification:
✓ Card number (PAN)
✓ CVV
✓ Billing address (AVS)

Attack vector:
one compromised card file → all three factors satisfied

Staged authentication adds out-of-wallet factors that are not co-located with card data:

✓ Partial card reference
✓ Remaining card digits on isolated channel
✓ Email/phone on account
✓ Session binding

Attack vector:
compromised card file → cannot complete Stage 2

This aligns architecturally with the 3D Secure 2.0 (3DS2) model, which uses risk signals — device fingerprinting, behavioral analytics, transaction context — at the completion stage to authenticate without friction for legitimate users.

The staged authentication pattern for ERP-native transactions follows the same principle: multi-factor authentication is structural, not a bolt-on challenge triggered only when something looks suspicious.

Dynamics 365 Implementation Architecture

For teams building on Dynamics 365 F&SCM or Business Central, the architectural decisions that matter most:

Native payment framework vs. middleware connector:

The Dynamics 365 payment framework allows payment logic to run natively within the AX/D365 stack. This matters for reconciliation accuracy and scope containment.

External connectors that route ERP payment data through middleware create additional PCI scope questions.

Payment journal design:

A well-designed integration populates journal lines with:

  • Customer account reference
  • Invoice reference
  • Amount
  • Currency
  • Masked PAN (first 6 / last 4)
  • Payment token reference
  • Authorization code
  • Gateway transaction ID

NOT:

  • full PAN
  • CVV
  • track data

*Omnichannel token consistency:
*

If payment surfaces include native order entry, B2B portals, mPOS/ePOS terminals, and customer service telephony, the token issued at any channel must be recognizable across all others.

This requires a common token vault that all channels reference.

Red Maple’s native Dynamics 365 payment integration implements this architecture — split-entry authentication, gateway-agnostic token storage, and payment journal population with masked PANs only.

Fraud Pattern Disruption

Understanding the security value of staged authentication requires mapping it to how modern fraud patterns actually work.

Card testing attacks:
Automated bots submit micro-transactions across merchants to validate stolen card numbers. A staged architecture has no conventional auth endpoint. The bot cannot complete Stage 2 without the out-of-wallet factor.

**Account takeover (ATO):
**An attacker who compromises a merchant account may find token references stored in the customer record. Tokens without the associated verification factor are non-functional.

Insider threat:
This is the most underestimated vector in ERP environments. In a staged model, there are no full PANs in the database to extract. The insider threat surface is eliminated structurally, not mitigated procedurally.

Summary: When Staged Authentication Is (and Isn’t) the Right Fit

Worth evaluating when:

  • Your ERP has multiple modules interacting with payment data
  • You’re processing B2B recurring billing
  • Your workflow includes manual order entry
  • You need omnichannel token consistency
  • You’re on Dynamics 365 environments

Consider alternatives when:

  • Your payment surface is a standalone hosted checkout
  • Transaction volume is low
  • You don’t need ERP-native workflows

The core architectural principle remains consistent:

Reduce attack surface by reducing what your systems receive, not just by controlling what they store. reduce the attack surface by reducing what your systems touch, not just by protecting what they hold.

Further Reading
● PCI DSS v4.0 Requirements and Testing Procedures (PCI Security Standards Council)
● PCI DSS Tokenization Guidelines
● SAQ eligibility matrix — PCI SSC website
● EMVCo Token Specification
● Red Maple Clever Division documentation — redmaple.com

Supply Chain Data Flow: Why Errors Happen in ERP Systems

Supply Chain Data Flow: Where Do Problems Begin?

While developing a manufacturing ERP, there was a persistent inconsistency in reporting shipment delays. Initially, I tried to optimize database queries, but the problem was deeper. At the root of these delays was an unhealthy data flow throughout the supply chain. Critical information, such as when the raw materials needed to produce a product would arrive, when production would be completed, and when the product would be ready for shipment, was getting lost or incorrectly processed at different points in the system. This situation reduced operational efficiency and increased costs.

Such problems require not only software developers but also a fundamental understanding of operational processes. In the real world, an ERP system is not just a pile of code; it’s a complex set of flows that reflects an organization’s operations. Every step of the supply chain – supplier selection, raw material intake, production planning, inventory management, quality control, packaging, and finally, shipment – is tightly interconnected. Any disruption in any link of this chain can affect the entire system.

The First Point of Contact: Data Entry and Consistency

Most often, problems start at the very first point of data entry. Entering a supplier order into the system, recording a goods receipt, or creating a production order… When these operations are done manually, the probability of human error increases. Inputs like incorrect product codes, missing quantities, or erroneous date information directly affect all subsequent operations. For example, entering a raw material with the wrong unit (like kilograms instead of pieces) can lead to completely wrong results in production planning.

In one case we experienced with our manufacturing ERP, a key raw material stock was consistently showing as insufficient. Upon detailed investigation, we realized that our colleague in goods receipt had recorded the incoming 10 pallets of product as a total of 10 pallets, rather than entering each pallet individually. This was a simple manual entry error, but it directly impacted the next day’s production plan, preventing the planned production of 500 units. These types of errors are common in daily operations and can lead to significant cost increases.

ℹ️ Best Practices for Data Entry

  • Tighten data validation mechanisms: Check the format, range, and logical consistency of data entered into fields.
  • Detail user authorization: Ensure each user can only access and modify data within their authority.
  • Increase automation: Reduce manual data entry wherever possible. Use barcode scanners, RFID tags, or integrated systems.
  • Invest in training: Provide regular training to users to ensure they use the systems correctly and completely.

The Gap In Between: Integration and Communication Problems

The supply chain doesn’t just consist of a company’s internal processes; it involves multiple stakeholders: suppliers, manufacturers, logistics companies, and customers. Data exchange between these stakeholders often happens through different systems. In situations where integration is weak or non-existent, serious disruptions in data flow occur. When transferring information from one system to another becomes manual, delays and errors are inevitable.

At a large e-commerce company where I once worked, a custom middleware was used to transfer orders to the ERP system. Orders first landed in the e-commerce platform, then to the middleware, and then to the ERP. During a weekend incident, the e-commerce platform received an unexpected update, and its API changed. Since the middleware couldn’t immediately detect this change, orders couldn’t be transferred to the ERP. As a result, on Monday morning, chaos ensued with hundreds of orders missing and incorrect information being given to customers. These types of integration issues are more common, especially when systems with different technology stacks come together.

# Simple API integration example (erroneous scenario)
import requests
import json

def send_order_to_erp(order_data):
    erp_api_url = "https://api.example-erp.com/v1/orders"
    headers = {"Authorization": "Bearer YOUR_API_KEY"}

    try:
        response = requests.post(erp_api_url, data=json.dumps(order_data), headers=headers)
        response.raise_for_status() # Raises an exception for HTTP errors
        print("Order sent to ERP successfully.")
        return response.json()
    except requests.exceptions.RequestException as e:
        print(f"Error sending order: {e}")
        # Error logging or retry mechanisms can be added here
        return None

# Example order data
order = {
    "order_id": "ECOMM12345",
    "customer_id": "CUST987",
    "items": [
        {"product_id": "PROD001", "quantity": 2},
        {"product_id": "PROD005", "quantity": 1}
    ],
    "order_date": "2026-05-20T10:00:00Z"
}

# If the API changed and this code wasn't updated, the send_order_to_erp function will error out.
# send_order_to_erp(order)

In this scenario, the raise_for_status() method of the requests library is used to catch error codes from the API. If the API’s endpoint changes or authorization details become invalid, this function will raise an error, and the order cannot be transmitted to the ERP. It is critical to establish a robust error management and monitoring mechanism for such situations.

Production Planning and Inventory Management Dilemma

One of the most critical stages of the supply chain is production planning. An accurate production plan prevents idle capacity and ensures timely fulfillment of customer demands. However, this planning must be based on accurate and up-to-date inventory information. If there’s a discrepancy between the stock quantity shown in the system and the physical stock quantity, the production plan can go awry. This can lead to either overproduction (increasing inventory costs) or production stoppages due to insufficient raw materials (delivery delays).

At a manufacturing company, we were using an AI model to optimize weekly production plans. The model determined the most efficient production route using current inventory data from the ERP. However, for a month, the plans generated by the model were inconsistent. After detailed analysis, we discovered a bug in the inventory management module that was recording returned products in a separate “returns stock” area instead of deducting them from the main stock. This caused the available stock quantity shown in the ERP to be lower than the physical stock. The AI model, assuming the current stock was accurate, was making incorrect plans. Fixing this bug and synchronizing the returns stock with the main stock took about 4 days.

⚠️ The Delicate Balance Between Inventory Management and Production Planning

The synchronization between these two modules is the backbone of the supply chain. Even a small deviation in stock can lead to large fluctuations in production plans. Especially in industries with variable demand and short lead times, perfect synchronization is vital.

Lack of Reporting and Visibility

Many of the problems experienced throughout the supply chain stem from a lack of adequate reporting and visibility. When there are no clear answers to questions like which product is where, when it will be delivered, or at which stage there’s a delay, proactive measures become impossible. Standard ERP reports often provide general information but lack the depth needed to analyze the dynamic nature of the supply chain and potential bottlenecks.

On a project for one of our clients, we wanted to improve their shipping processes. Their self-generated reports only showed completed shipments. However, they didn’t provide clear information about the status of products that were in transit, waiting at customs, or being prepared in the warehouse. Therefore, understanding why a product’s delivery was delayed required manually gathering information from multiple sources, which led to an average of 3-4 hours of overtime.

To solve this problem, we developed a custom dashboard that tracked the lifecycle of an order within the ERP. This dashboard visualized every stage, from the moment the order was received, to the creation of the production order, the preparation of raw materials for shipment, the packaging of the product, and finally, its dispatch. It also showed the expected times and current delays for each stage. This allowed us to instantly see which orders were at risk. For instance, we could see that an order was waiting for 2 days at the raw material procurement stage and contact the relevant supplier to resolve the issue.

# Example of log analysis with Bash command (detecting delays)
# Default log file: /var/log/erp/shipment_tracker.log
# Using grep and awk to find delayed orders

# Example log format:
# 2026-05-20 09:15:32 INFO [SHIPMENT_TRACKER] Order ID: ORD7890, Status: WAITING_FOR_MATERIAL, Expected completion: 2026-05-19 17:00:00

LOG_FILE="/var/log/erp/shipment_tracker.log"
CURRENT_DATE="2026-05-20" # Current date

echo "Orders expected yesterday but still pending:"
awk -v current_date="$CURRENT_DATE" '
  $3 == "ORDER_ID:" { order_id = $4 }
  $3 == "STATUS:" && order_id {
    status = $4
    # If the order status is "WAITING_FOR_MATERIAL" or similar and the completion date has passed
    if (status ~ /WAITING|PENDING/) {
      # Find the expected completion date
      match($0, /Expected completion: ([0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2})/, completion_time)
      if (completion_time[1]) {
        if (completion_time[1] < current_date) {
          print "Order ID: " order_id ", Status: " status ", Expected: " completion_time[1]
        }
      }
    }
    order_id = "" # Reset order_id
  }
' "$LOG_FILE"

This script scans the records in the shipment_tracker.log file and lists orders that have passed their delivery date but are still in a waiting or pending status. Simple log analyses like this can help quickly identify the root cause of a problem. The awk command is very powerful for text processing and is ideal for such tasks.

Cost Increases and Lost Opportunities

The ultimate consequence of errors in the supply chain data flow is direct cost increases and lost opportunities. Delayed deliveries lead to customer dissatisfaction, which in turn reduces brand value in the long run. Holding excess inventory increases storage costs and ties up capital. Production stoppages mean wasted labor and machine costs, as well as lost revenue from unsold products.

While examining the ERP system of a textile company, I found that their annual inventory costs were 15% higher. The primary reason for this was the inability to properly manage stock levels of fabrics used in production. Frequent last-minute orders were placed, which increased express shipping costs and led to purchases from suppliers at higher prices. Analysis revealed that lack of visibility in the raw material supply chain and inaccurate forecasting were causing this situation.

To resolve the issue, we began sharing weekly demand forecasts in closer collaboration with suppliers. We also implemented a mechanism that generated automatic alerts when stock levels in the system dropped below a certain critical point. As a result of these changes, we achieved approximately a 10% reduction in annual inventory costs and a 30% decrease in the number of urgent orders. This not only resulted in cost savings but also improved customer satisfaction.

💡 The Relationship Between Efficiency and Cost

Increased efficiency in the supply chain directly translates to reduced costs. Every delay, every error negatively impacts cash flow. Therefore, supply chain optimization is not just an operational improvement but also a strategic financial investment.

Conclusion: A Data-Driven and Integrated Approach

The way to overcome errors in supply chain data flow is through a data-driven and integrated approach. Systems need to be designed not only from a software perspective but also by considering operational processes. Consistency and visibility must be ensured at every step, from data entry to integration, production planning, inventory management, and reporting.

This often requires reviewing and improving existing systems or integrating them with new ones. Automation, data validation, user training, and strong inter-stakeholder communication are the cornerstones of this process. It’s important to remember that ERP systems are just tools; what truly matters is effectively using these tools to seamlessly connect all links of the supply chain. In future posts, I will examine these integrations in more detail.

How do you prevent “AI spaghetti code” when orchestrating with LLMs?

Hi everyone!

I’ve reached a point in my workflow where I barely write code line-by-line anymore, I orchestrate AI agents. I use tools like Superpower, detailed specs, and a structured prompt architecture: a folder with specific .md rules for different scopes (Front, Back, Git…) and a global claude.md for overall context.

My rules are supposed to be rock solid. The high-level architecture stays fine. But I’m hitting a massive pain point: localized spaghetti code and micro-debt.

The reality is: Sometimes, if I don’t take 45 minutes to really review the code, I miss the spaghetti code. Then later, when I actually open the file, I’m just like “shit, wtf is this?!”

It feels like sub-agents lose the memory/context of the parent rules the moment they spawn for a micro-task, and they still write shit or over-complex code.

So I am wondering, am I missing something?

How can I make Superpower run sub-agents without losing context/Rules, or is there a better approach to coding with agents?

Would love to hear how you guys keep your codebase clean without having to create a specific task to spot messy code three weeks after it’s done just to fix it.

I just want the AI to write good code directly from the start.

Sony, yeni “State of Play” etkinliği duyurdu

State of Play

State of PlaySony bugün 2 Haziran tarihli yeni “State of Play” etkinliğini duyurdu. 60 dakika süreceği açıklanan etkinlikte “Marvel’s Wolverine” başta olmak üzere PS5’e gelecek çok sayıda oyun hakkında yeni bilgiler verilecek ve görüntüler paylaşılacak. 15 Eylül’de çıkacak “Wolverine”, Insomniac Games tarafından geliştiriliyor ve uzun süredir büyük bir merakla bekleniyor.

Sony, yeni “State of Play” etkinliği duyurdu yazısı ilk olarak Teknoloji haberleri – LOG üzerinde yayınlandı.

Stellantis ve Çinli Dongfeng Motor, küresel pazarlara ortak üretim yapacak

Stellantis ve Çinli Dongfeng Motor, küresel pazarlara ortak üretim yapacak

Stellantis ve Çinli Dongfeng Motor, küresel pazarlara ortak üretim yapacakÇok sayıda otomobil devini bünyesinde barındıran Stellantis ve Çinli Dongfeng Motor, küresel pazarlara ortak üretim yapacak. Bu konuda şirketten yapılan resmi açıklamada, “Stellantis, var olan köklü iş birliklerinin kapsamını genişleterek sürdürülebilir büyüme hamlesini güçlendirmeye devam ediyor. Bu kapsamda Stellantis ve Dongfeng Motor Corporation, Çin pazarı ve küresel satışlar için Çin’de Peugeot ve Jeep araçlarının ortak üretimini kapsayan stratejik iş birliği anlaşması ile 34 yıllık ortaklıklarını daha da genişlettiklerini duyurdu. Stellantis ve Dongfeng Group ayrıca, iki tarafın ölçek, uzmanlık ve Ar-Ge yetkinliklerinden faydalanarak iş birliklerini daha da güçlendirmek amacıyla bağlayıcı olmayan stratejik bir iş birliği anlaşması (MoU) imzaladı. İmzalanan stratejik iş birliği anlaşması kapsamında ve gerekli onay süreçleri ile ilgili uygulama anlaşmalarının tamamlanmasına bağlı olarak, Dongfeng Peugeot Citroën Automobile Co Ltd (DPCA) ortak girişiminin, Wuhan fabrikasında 2027 yılından itibaren ilk etapta Peugeot markalı iki tamamen yeni enerji aracı üretmesi planlanıyor. Bu modeller, Peugeot’nun 2026 Pekin Otomobil Fuarı’nda tanıttığı yeni konsept araçların […]

Stellantis ve Çinli Dongfeng Motor, küresel pazarlara ortak üretim yapacak yazısı ilk olarak Teknoloji haberleri – LOG üzerinde yayınlandı.

Things they don’t tell you about building (or buying) an SBC

Was helping a friend’s team evaluate Session Border Controller options recently, and the conversation reminded me how much weird, undocumented knowledge SBC work involves. Sharing some of it here because most of the public material out there is either marketing brochures or RFC walls of text.
This isn’t an intro to what an SBC does. If you’re reading this you probably already know it sits at the edge of a VoIP network and handles NAT, security, interop, and a few other things. The point of this post is the stuff that bites you only after you’ve actually deployed one.

1. “Just put an SBC in front of it” hides a lot of complexity
Half the SIP advice on the internet ends with “put an SBC in front of it” as if that’s a one-step fix. It isn’t. The SBC is the thing that does the hard work, which means all the complexity that was hiding in NAT, codec negotiation, and SIP normalization now lives inside the SBC config instead.

A poorly configured SBC will silently break calls in ways that look like softphone bugs, network issues, or even DNS problems. Spent two weeks once chasing a “softphone bug” that turned out to be the SBC stripping a SIP header that the downstream PBX needed.

Lesson: when something breaks weirdly, suspect the SBC first, not last.

2. SIP normalization is where most real-world deployments live
Different SIP devices, providers, and softphones implement SIP slightly differently. Polycom does one thing. Yealink does another. Some softphones add custom headers. Some carriers expect a very specific From header format. Some downstream PBXes choke on whitespace in odd places.

A serious chunk of SBC configuration is just “rewrite this SIP message into the format the next hop expects.” It’s not glamorous. It’s not what the marketing slides talk about. But it’s where 60% of real SBC tuning time goes.

If you’re building or configuring one, get comfortable with header manipulation, URI rewriting, and SDP modification early.

3. Topology hiding is more than a buzzword
Every SBC vendor talks about topology hiding. What they mean in practice is: rewriting Via headers, Record-Route headers, From and To URIs so the other side can’t see your internal infrastructure.
Important because:

Exposed internal IPs help attackers fingerprint your setup.
Some attackers specifically target SBCs they can identify by vendor banner.
Some downstream networks reject calls that show internal routing hops.

In production, every outgoing call should look like it’s coming from the SBC’s public IP and nothing else. If you’re seeing internal IPs leak in production traffic, your topology hiding is broken regardless of what the config screen claims.

4. Fraud is real and it happens fast
The first time you put a public-facing SBC into production, you will see scan traffic within minutes. SIP scanners are constant, automated, and aggressive. They’ll try every common extension, every default password, every well-known account name.

If you have a misconfigured registration endpoint or an open trunk, fraud calls can start within hours. International premium rate numbers. Calls to obscure country codes that nobody at the company has ever called. The bill arrives at the end of the month, and it’s usually large.

Defenses that actually matter:

  • Disable anonymous trunks completely
  • Rate-limit registration attempts per source IP
  • Block geographically (most fraud comes from specific country ranges)
  • Block international dialing patterns by default and whitelist countries individually
  • Monitor for unusual call patterns in real-time, not in monthly reports

Don’t trust the vendor’s defaults. Audit them yourself.

5. RTP is where the “interesting” performance problems live
SIP signaling is small. RTP is constant, real-time, and bandwidth-heavy. Most SBC performance issues at scale are actually RTP path issues.

Things that matter:

  • Whether the SBC is doing media anchoring (relaying RTP) or just signaling
  • Codec transcoding cost (Opus to G.711 conversion is CPU-expensive, do it sparingly)
  • Jitter buffer behavior under load
  • DTLS-SRTP handshake performance for WebRTC calls

A signaling-only SBC scales easily to thousands of concurrent registrations. The moment you start anchoring media or transcoding codecs, hardware budgets change drastically.

If you’re sizing infrastructure, model your transcoding load carefully. It’s the thing that most often surprises teams in production.

6. WebRTC support is harder than it looks
If your SBC needs to bridge WebRTC clients to traditional SIP, you’re

  • signing up for a different category of complexity.
  • WebRTC uses DTLS-SRTP, traditional SIP usually uses plain SRTP or unencrypted RTP.
  • ICE candidates need to be properly translated to/from SIP/SDP.
  • Some browsers behave differently in edge cases.
  • WebSocket connection management adds another layer of state to track.

A lot of older SBCs were retrofitted for WebRTC and it shows. If WebRTC matters to your deployment, test it explicitly with realistic scenarios, not just “it connected once.”

7. Logs grow faster than you expect
An SBC at production volume will generate gigabytes of logs per day. SIP messages, RTP statistics, security events, performance counters. You need a real log pipeline.

  • Send logs to a centralized system (ELK, Loki, Splunk – whatever your team already uses)
  • Set retention policies before you fill up disks
  • Be ready to capture full SIP traces selectively when something breaks, because debugging without traces is misery
  • Watch your log levels – debug logging in production will crush both disk and CPU

The teams I’ve seen do this well treat the SBC like any other production service: monitoring, alerting, dashboards. The teams that don’t, end up chasing problems blind.

8. Open source vs commercial isn’t a binary choice
Common open-source SBC options: Kamailio (often configured as one), OpenSIPS, drachtio (programmable, Node.js-based). Commercial options: AudioCodes, Oracle (Acme Packet), Ribbon, Sansay, and several others.

The honest tradeoffs:

  • Open source: total control, zero license cost, but you own all the operational burden
  • Commercial: support contracts, certified hardware, but expensive and sometimes inflexible

A surprising number of large carriers run open-source SBCs in production. Kamailio in particular powers a huge amount of real-world VoIP infrastructure that you’d think runs on commercial gear.

Pick based on your team’s expertise, not just the feature checklist.

Closing
SBC work is one of the more underrated parts of VoIP infrastructure. It’s not glamorous, the documentation across the industry is patchy, and a lot of the real knowledge lives in the heads of engineers who’ve debugged enough production issues to know which questions to ask.
If you’re getting started with SBC development or evaluation, the best advice I can give: don’t trust the defaults, test under real network conditions, and assume fraud will find you within hours of going live.

If you’ve shipped SBC work and run into things I haven’t covered, comments are open. Always interested in comparing notes on this stuff.

Organizing the Use Cases of AWS Step Functions and Glue Workflow for ETL Processing with AWS Glue Jobs

Original Japanese article: Glue JobのETL処理におけるAWS Step FunctionsとGlue Workflowの使い分けを整理する

Introduction

I’m Aki, an AWS Community Builder (@jitepengin).

When building data pipelines on AWS, deciding which workflow orchestration tool to use is an important architectural decision.
Especially when designing ETL pipelines centered around Glue Jobs, the following two services are commonly compared:

  • AWS Step Functions
  • AWS Glue Workflow

Both services are workflow orchestration tools that manage job dependencies and execute processes sequentially.
However, since they are designed with different strengths and philosophies, choosing the right one based on your requirements is critical.

In this article, I’ll organize the characteristics, advantages, disadvantages, and use cases of these two services, and discuss how to decide which one to choose.

Personally, I really like Glue Workflow, but I’ve had fewer opportunities to use it recently, which is honestly a bit disappointing.
I also like Glue Python Shell, but compared to standard Glue Jobs, I rarely get to use it these days either…

Major Workflow Orchestration Tools

AWS provides multiple workflow orchestration services, but the two most commonly compared for Glue Job–based data pipelines are the following:

Service Overview
AWS Step Functions AWS managed workflow orchestration service. Can flexibly orchestrate a wide range of AWS services such as Lambda, Glue, ECS, and more
AWS Glue Workflow Glue-native workflow feature. Defines pipelines using Glue Jobs, Crawlers, and Triggers

AWS also offers Amazon Managed Workflows for Apache Airflow (MWAA).
MWAA becomes a strong option when you need highly complex dependency management or cross-cloud orchestration. However, in this article, I’ll focus specifically on comparing Step Functions and Glue Workflow.

Recently, many teams have also adopted workflow tools such as Airflow, Dagster, and Prefect. As always, selecting the right tool depends heavily on your requirements and goals.

AWS Step Functions

Overview

(Figure above: Visual workflow definition using Workflow Studio)

AWS Step Functions is a workflow orchestration service that defines workflows using Amazon States Language (ASL), a JSON/YAML-based language.

It integrates with over 200 AWS services including Lambda, Glue, ECS, SNS, and DynamoDB, and supports flexible workflow controls such as conditional branching, parallel execution, and retry handling.

Features

Step Functions provides two workflow types:

Type Characteristics
Standard Workflow Long-running execution, audit logs, exactly-once execution guarantee
Express Workflow High throughput, low cost, optimized for short-lived processing (at-least-once)

In the context of data pipelines, Standard Workflow is generally the more common choice.

Step Functions also provides AWS Step Functions Workflow Studio, a visual editor that allows workflows to be built through a GUI.

Advantages

  • Multi-service integration: Easily integrates Glue with other AWS services such as Lambda, ECS, SNS, and more
  • Flexible control flow: Supports conditional branching (Choice), parallel execution (Parallel/Map), and advanced error handling (Catch/Retry)
  • High observability: Execution history and per-step states can be visually inspected in the console
  • Event-driven integration: Easily triggered through EventBridge using S3 uploads or schedules
  • Parallel execution support (Map): Well suited for large-scale processing such as file-level or partition-level parallel execution. Distributed Map is especially useful for high-scale parallel workloads
  • Infrastructure as Code support: Can be fully managed through CloudFormation, CDK, or Terraform

Disadvantages

  • Learning curve: Requires understanding ASL, and complex workflows can become verbose
  • Cost: Standard Workflow pricing is based on state transitions, so costs can increase with larger workflows
  • Glue integration setup: IAM roles, parameter passing, and Glue Job integration must be configured manually

Use Cases

Step Functions is a strong fit for scenarios such as:

  • Hybrid pipelines combining Glue with Lambda, ECS, or other AWS services
  • Complex workflows requiring branching, parallel processing, or dynamic parameter passing
  • Pipelines that require SNS notifications or compensating actions on failures
  • Teams managing infrastructure strictly through IaC
  • Large-scale data platforms involving multiple teams

AWS Glue Workflow

Overview

(Figure above: Visual workflow definition using Glue Workflow)

AWS Glue Workflow is Glue’s native workflow orchestration feature.

It defines pipelines using three primary components:

  • Glue Jobs
  • Glue Crawlers
  • Glue Triggers

Pipelines can be configured directly from the Glue console GUI, making it easy to build Glue-centric ETL workflows quickly.

Features

The major components of Glue Workflow are as follows:

Component Role
Glue Job Actual ETL processing using Spark or Python Shell
Glue Crawler Scans data sources such as S3 and registers table metadata into the Data Catalog
Glue Trigger Defines execution conditions such as schedules, events, or conditional dependencies

Glue Triggers support three types:

  • SCHEDULED
  • ON_DEMAND
  • CONDITIONAL

This allows chained execution patterns, such as triggering downstream jobs based on upstream job success or failure.

Advantages

  • Native Glue integration: Seamlessly integrates with Glue Jobs and Crawlers with minimal additional setup
  • Simple configuration: DAG-style workflows can be built intuitively through the console GUI
  • Low cost: No additional charge for the workflow itself beyond Glue Job execution costs
  • Easy crawler integration: Natural fit for workflows that update the Data Catalog after ETL execution
  • Glue Data Catalog integration: Job execution metadata and lineage can be managed centrally within Glue

Disadvantages

  • Glue-only orchestration: Cannot directly orchestrate Lambda, ECS, or other non-Glue services
  • Limited event-driven capabilities: Primarily Trigger- and schedule-based, making advanced event integration less flexible than Step Functions
  • Limited control flow: Weak support for advanced branching and dynamic parameter handling
  • Observability limitations: Detailed execution logs often require separate CloudWatch investigation
  • More difficult IaC management: CloudFormation and Terraform management can become cumbersome compared to Step Functions
  • Limited parallel execution control: Not ideal for fine-grained parallelization or Map-style orchestration
  • Weaker retry/re-execution control: Re-running only failed portions of a workflow is less flexible than in Step Functions

Use Cases

Glue Workflow is well suited for:

  • Simple ETL pipelines composed entirely of Glue Jobs and Crawlers
  • Periodic ingestion pipelines into S3-based data lakes with automatic catalog updates
  • Small teams or early-stage projects that need fast implementation
  • Organizations that prefer operating primarily through the Glue console

Which One Should You Choose?

Based on the characteristics of both services, the selection criteria can generally be summarized as follows.

When to Choose Glue Workflow

  • Your ETL pipeline consists only of Glue Jobs and Crawlers
  • Simple sequential execution and conditional triggers are sufficient
  • You want to build quickly (prototypes or small-scale projects)
  • Your operations are centered around the Glue console
  • You want to minimize costs

When to Choose Step Functions

  • You need integration with Lambda, ECS, or other AWS services
  • You require branching, parallel processing, or advanced error handling
  • You are adopting an EventBridge-centric event-driven architecture
  • You want strict Infrastructure as Code management
  • Multiple teams are involved in operating the data platform
  • Observability and audit logging are important

Summary of Decision Criteria

Perspective Glue Workflow Step Functions
Supported Services Glue only Broad AWS integration
Control Flow Simple Flexible and advanced
Observability △ ◎
Ease of Configuration ◎ △ (learning curve)
Cost Low Depends on state transitions
IaC Management △ ◎
Crawler Integration ◎ △ (manual setup required)

Glue Workflow is fundamentally Trigger-oriented and is not designed for advanced event orchestration like Step Functions.

Unless Glue Workflow specifically satisfies your requirements better, choosing Step Functions is generally the safer long-term option.
Personally, when designing architectures, I often start by considering Step Functions first.

That said, Glue Workflow remains a strong choice when the requirement is simply:

“I want to quickly build a Glue-centric ETL pipeline.”

Combining Both Services

Instead of choosing one or the other exclusively, it is also possible to invoke Glue Workflow from Step Functions.

For example, Step Functions can handle preprocessing and postprocessing with Lambda, while delegating the ETL core to Glue Workflow.

However, this introduces additional complexity because workflow state coordination between the two services must be managed carefully.
If simplicity is important, standardizing on one orchestration tool is generally easier operationally.

Caveats When Invoking Glue Workflow from Step Functions

1. Completion Detection Requires Polling

Step Functions provides a convenient .sync integration pattern for Glue: StartJobRun, which waits for job completion automatically.

However, Glue: StartWorkflowRun does not support the .sync integration pattern.

While you can invoke Glue Workflow through SDK integration, Step Functions will immediately proceed to the next state without waiting for completion.
As a result, you must implement custom polling logic to repeatedly check the WorkflowRun status.

2. Polling Logic Becomes Complex

You typically need to implement a loop like:

  • Wait
  • GetWorkflowRun
  • Choice (RUNNING / COMPLETED / FAILED)

This increases both the number of states and the verbosity of the ASL definition.

3. Error Handling Becomes More Complicated

Glue Workflow status is returned at the WorkflowRun level rather than the individual Job level.

As a result, identifying which specific Glue Job failed requires additional parsing logic against the GetWorkflowRun response.

Because of these complications, although combining both services is technically possible, I generally recommend avoiding Step Functions → Glue Workflow orchestration unless there is a compelling reason.

One possible use case is when you need to extend an existing Glue Workflow–based system incrementally.
Even then, rebuilding the orchestration directly in Step Functions using existing Glue Jobs often feels cleaner.

Migration Considerations

Some teams initially adopt Glue Workflow during the early stages of a project and later migrate to Step Functions as the data platform grows.

When migrating, the following considerations become important:

  • Workflow definitions must be rewritten: Glue Trigger definitions need to be converted into ASL
  • IAM roles must be redesigned: Step Functions requires permissions to invoke Glue Jobs
  • Extensive testing is necessary: Existing jobs must be validated carefully after migration

Migration is certainly possible, but it is not trivial.
This is why it is important to consider future extensibility and maintainability from the beginning when selecting your orchestration tool.

Conclusion

In this article, I organized the differences between AWS Step Functions and Glue Workflow for ETL orchestration.

To summarize:

  • Glue Workflow: Glue-native, simple, low-cost, and ideal for rapidly building straightforward ETL pipelines
  • Step Functions: Better suited for multi-service orchestration, advanced workflow control, observability, and large-scale pipelines

Neither service is universally “better.”
The right choice depends on your use cases, organizational structure, operational requirements, and future scalability needs.

For small-scale ETL pipelines, Glue Workflow is often sufficient. However, as data platforms evolve, requirements such as exception handling, notifications, conditional branching, and integrations with other services tend to grow over time. In many cases, architectures gradually move toward Step Functions as complexity increases.

A practical strategy can be to start simple with Glue Workflow during the early stages, and later migrate to Step Functions when requirements become more sophisticated.

That said, considering future migration costs, building on Step Functions from the beginning can also be a very reasonable approach.

I hope this article helps anyone currently evaluating workflow orchestration options on AWS.