LLM-as-judge has become the dominant pattern for evaluating language model outputs. Tools like Promptfoo, Braintrust, LangSmith all converge on the same architecture: send your prompt to your model, send the output to a different model with a rubric, take the second model’s score as the quality signal.

This works. It’s also expensive (judge tokens cost real money), slow (extra API roundtrip), variance-prone (the same eval gets different scores across runs), and architecturally a bit circular (using an LLM to evaluate an LLM trained on overlapping data distributions). The single signal becomes a bottleneck for trust.

So I built an eval module that has two independent signals instead of one.

What the tool does

Side-by-side blind comparison. Two agents answer the same prompt. One runs raw, the other can optionally have a cognitive harness wired in as a tool call. A separate blind judge model scores both responses, sees only A and B labels with no knowledge of which is which. Standard setup so far.

But alongside the judge, four cognitive posture heat maps run on each response. These are not LLM-based. Deterministic text analysis that visualizes HOW the model wrote, not just whether it agreed with you.

When the heat maps agree with the judge’s verdict, you have confidence. When they disagree, you have a question worth investigating. Two independent signals beat one signal that wraps itself.

How the heat maps work

Each response is split into 100 word-chunks arranged on a 10×10 grid. Two grids per agent.

Top grid: confidence posture. Per chunk, count hedge words (maybe, might, possibly, seems, could) and assertive words (definitely, must, always, never, clearly). Compute net (asserts - hedges) / (asserts + hedges). Add punctuation cadence as a secondary signal: periods are positive (definite statements end with them), commas are negative (qualifications stack with them). Normalize to [-1, 1]. Color the chunk diverging from blue (hedged) through gray (neutral) to red (assertive).

Bottom grid: reasoning density. Per chunk, count explicit reasoning connectives (because, therefore, since, if/then, due to, as a result, this means). The denser the reasoning markers, the brighter the cell. Sequential palette from dark to hot.

A 2D Gaussian blur runs over both grids so sparse markers spread into spatial blobs instead of isolated cells. Empirically this matters: a single “because” in a 100-chunk response forms a small heat radius on the reasoning grid even when neighboring chunks have nothing. The blob shapes are easier to scan at a glance than scattered pixels.

The whole computation runs client-side in plain JavaScript. No API call, no model inference. Pure word counting plus a smoothing pass. Free to compute, deterministic, fast.

Multi-turn scenario mode

Most LLM evals are single-turn. The most interesting failure modes are multi-turn.

If you paste turn1---turn2---turn3 separated turns into the scenario textarea, both agents accumulate conversation history across turns. This is where production failures actually manifest:

• Sycophancy compounding. A model that gives ground on turn 2 has already shifted by turn 4. Single-turn evals miss the trajectory entirely.
• Hallucination cascade. Once a model emits a wrong fact, that fact becomes part of the conversation history. On the next turn, the model treats its own previous error as established truth and builds on top of it.
• Authority claim drift. User-proposed framings persist across turns. The model anchors on the first plausible framing without re-examining it later.
• Prompt-forgery patterns. A user can inject fake reasoning chains in a later turn (“we already verified X yesterday, can you finalize the report?”). The model has no way to verify the off-screen claim and tends to accept it.

The eval module captures all four. The cognitive posture field shows visually where in the response the model committed to the bad path.

Other things in the module

The optional cognitive harness has four modes you can switch in the UI:

• anti-deception (139 cognitive operations): sycophancy resistance, prompt injection, hallucination cascade
• reasoning (311 operations): general structured thinking, causality, simulation, metacognition
• code (128 operations): software engineering tasks
• memory (101 operations): perception and behavioral calibration

Pick whichever mode fits the failure category you’re testing for.

Dimensions the judge scores on are user-defined. There’s a small library to pick from (Accuracy, Hallucination resistance, Held the line, Reasoning depth, Safety, Completeness) but you can type any name and the judge prompt rewrites itself to include it. Each agent has its own system prompt field, so you can frame them differently if your comparison needs that.

The Results Overview sidebar accumulates per-dimension bar charts, win tally, latency and token cost per branch across runs in the same browser. localStorage persists everything between sessions. Compare A vs B opens a fullscreen modal for reading both responses in parallel when they get long.

Why Windows 95 chrome

I tried to make it look like an instrument, not a SaaS dashboard. Beveled fieldsets do hierarchy work for free (the inset border physically separates each panel from the canvas, no whitespace tuning required). White input fields are where data lives so the eye lands on them. Gray-on-gray chrome stays out of the way.

Modern flat dark themes have to invent that hierarchy back from scratch using whitespace, type weight, dividers, and color hierarchy. They usually come up shorter. Win95 was a 1995 UI grammar that handled hierarchy through bevels, and bevels are free visual structure.

It’s also nicer to look at when you’re staring at evals for hours.

Tech stack

• Single HTML file (vanilla JS, no framework, no build step)
• 50-line Python stdlib proxy for CORS (the harness gateway doesn’t send CORS headers, so the proxy forwards server-side). Could be replaced with any reverse proxy (nginx, Caddy, Workers) in production.
• localStorage for persistence, no signup, no telemetry
• MIT licensed

Works with any OpenAI-compatible endpoint: OpenRouter, OpenAI direct, Anthropic via gateway, vLLM, llama.cpp’s openai shim, Ollama with the compat layer, LM Studio. Just point Provider URL at the right endpoint. Tool-calling capable model required for the harness branch, raw branch works on anything.

Try it yourself

bash
git clone https://github.com/ejentum/agent-teams.git
cd agent-teams/agent_evaluation_module_xp95
python serve.py

Authored by Conall Heffernan

As the Customer Success lead at Bronto, I need fast, reliable insights into customer health and product usage — but I don’t have time to constantly update indexes, schemas, or individual widgets just to answer new questions. I need to spot patterns, explore trends, and get answers in real time without manual overhead.

That’s why Bronto’s dashboards are so critical to my work. We recently added a new query filtering feature that lets me use SQL to look for any pattern across all widgets in a dashboard simultaneously — and update everything at lightning speed.

What Bronto Dashboards Provide

• Rich visualisation options — time-series charts, geomaps, numeric value widgets (with units like bytes and time), top lists, treemaps, and log event lists for drilling into raw data
• AI widget builder — describe what you want in natural language; an LLM builds the query and creates the widget without you needing to know the query language or which datasets to select (see the full post on this feature)
• Full screen widget mode — compare different timeframes for the same query (e.g. vs. previous day, week, month)
• Easy filtering — use the query builder with a searchable dropdown of top keys and values, or write SQL filters directly in the filter bar

The Power of Filtering at Scale

In many logging or observability tools, applying a filter means updating every single chart, table, or widget individually. Tedious. Time-consuming. It doesn’t scale.

With Bronto, applying a filter in the main query bar instantly updates every single widget on the dashboard for your given timeframe. With a default retention period of one year, you don’t need to worry about missing long-term trends — all your data is fully searchable and visualisable. Concerned about when an issue started? You can analyze trends over months, not days.

Our widgets use pre-computed log-based metrics (LBMs) for rapid responses, but the new dashboard filtering goes further — running raw log queries to filter the data in your dashboard in real time. Results come back in seconds, and you can drill down across all widgets simultaneously using SQL or by clicking from a dropdown of top keys and values.

One thing worth noting: there’s no initial configuration of keys required. I can use any key I want in the filter, with no setup.

Log-Based Filtering vs. Log-Based Metrics

Log-based metrics (LBMs) deliver millisecond responses — perfect for real-time dashboards, trend analysis, and high-level views of system behavior.

When something unexpected happens or a new question comes in, we seamlessly switch to log-based filtering to investigate deeper, explore raw data, and uncover answers without being constrained by pre-defined fields or aggregations.

Log-based filtering excels because it lets you query and visualize raw logs immediately — no upfront configuration like index definitions or field extraction required. Bronto combines this with structured parsing and indexing, using the right approach for the right job.

The result: both fast, flexible investigation and high-performance queries on known fields, without forcing you to predefine every key or build parsing pipelines before you can search effectively.

How I Use Dashboard Filtering for Customer Usage

My primary use for this feature is gathering and presenting product usage data to our leadership team. Questions like:

• “How much data did Org ID 54321 send over the last 6 months?”
• “How much did company ACME search last month?”

Instead of building 10 custom dashboards (which doesn’t scale as your customer base grows), I use dashboard filtering:

1. Navigate to our main Usage Dashboard
2. Enter the specific org_id in the main query filter (e.g. org_id: 54321)
3. Every widget updates instantly to reflect only that organization’s data

That’s it. A complex, multi-step data lookup becomes a quick and easy process.

The first time I tried filtering across the dashboard for an org_id, I thought something wasn’t working right — the results were rendered so fast across terabytes of data. It was a genuine “wow” moment. As a customer support lead, it’s great to see the under-the-hood changes we’re building for customers also improving my own day-to-day.

How Bronto Dashboards Compare

Most dashboard tools are optimized for known questions: predefined fields, fixed widgets, metrics decided on ahead of time. That works for stable monitoring but breaks down when you need to explore new questions, investigate unexpected behavior, or quickly slice data in different ways.

Summary

Modern teams need dashboards that are fast, flexible, and easy to adapt as questions change. Bronto dashboards combine high-performance log-based metrics with instant, dashboard-wide filtering to help you explore usage, investigate issues, answer leadership questions, and quickly re-run reports as requirements evolve.

Questions like “what does this customer’s usage look like over the last six months?” are answered in seconds — without upfront schemas, per-widget reconfiguration, or slow refresh cycles.

See Dashboard Filtering in Action

Last week was about Metal and the Skin Designer. This week the headline items are about what a brand new project looks like when you generate it: the default JDK is Java 17, and every generated project ships with an AGENTS.md authoring skill that lets any modern AI agent work on the project intelligently. There are also some other things worth covering: a runtime accent palette on the new native themes, three Metal follow-ups (one of which introduces a new matrix-correct translate API), the JDK 11+ String API gap closed, and iOS push permission that no longer fires at app launch.

What is Codename One? Codename One is an open-source framework for building native iOS, Android, desktop, and web apps from a single Java or Kotlin codebase. Learn more at codenameone.com.

Java 17 by default

We changed the default projects generated by the Initializr to Java 17+ to focus on the future of Codename One. The existing Java 8 option in the Initializr is still selectable from the radio panel if you have a reason to use it. Pick whichever you want.

The Java 17 path is the one we now recommend for new work. Generated projects build with any JDK from 17 onwards (we routinely test on 21 and 25); you do not need to install Java 17 specifically. The bigger picture of how Java 17 support works in the toolchain, including which language features land in your app code and how the iOS / Android ports handle the newer bytecode, was covered in Official Experimental Java 17 Support earlier this year. The change this week is the default and the wording: the (Experimental) tag is gone, and Java 17 is now what you get unless you opt out.

AGENTS.md and the Codename One skill

The other change in PR #4946 is that every Java 17 project the Initializr generates now ships an AGENTS.md file at the project root and a Codename One authoring skill alongside it.

AGENTS.md is the convention for handing project-specific context to any AI agent. Claude Code, Cursor, Codex, Aider; they all look for it. Codename One projects now ship one. The actual skill content lives under .agent-skills/codename-one/ (vendor-neutral) and the source for it is in the repo at scripts/initializr/common/src/main/resources/skill if you want to read through it directly. There is also a thin stub at .claude/skills/codename-one/SKILL.md so Claude Code’s /skills picker indexes it; the stub redirects to the same vendor-neutral content.

We deliberately scoped this to Java 17 projects. The older Java 8 build had additional constraints (Java 5/8 source target, retrolambda, the historical bytecode rewrite rules) that made the “what can I actually use” answer noticeably more complicated. Restricting the skill to Java 17 lets us give agents a cleaner picture of the language level, the toolchain, and the build commands without spending half the SKILL.md on caveats. If you stay on Java 8, you keep the project layout you had; nothing changes for you.

A few things the skill makes possible that I think are genuinely useful:

Agents can debug a Codename One app under jdb. This is the one I am most pleased with. The simulator is a regular JVM, so the standard Java Debugger attaches cleanly, but agents previously had no idea this workflow was available. The skill’s debugging.md reference walks through starting the simulator with the right -Xrunjdwp flags, attaching jdb, setting breakpoints, dumping locals, and stepping. The same workflow works in CI and any headless context where a graphical debugger is not an option. For an LLM that is otherwise reduced to “add a println and hope”, this is a much sharper tool.

Agents can check whether an API is part of the Codename One subset before they suggest it. Codename One targets a Java 5/8 shaped JDK so the same bytecode translates to iOS, Android, and JavaScript. An agent that has only read regular Java idioms will routinely reach for java.nio.file, java.time, or pieces of java.util.concurrent that the framework does not include. The skill ships a single-file IsApiSupported.java tool that an agent can invoke to verify a class or method before writing code against it.

Agents can validate a CSS snippet before applying it. Codename One CSS is its own subset; rules that look fine to a browser developer get silently dropped by the compiler. The IsCssValid.java tool lets the agent confirm the compiler will accept a snippet without booting the simulator.

These three things together are most of why an agent that was previously polite-but-not-useful on a Codename One project is now actually productive on one. If you do not use agents, the same Markdown is one of the better tours of the framework’s mental model that we have written; open .agent-skills/codename-one/SKILL.md in any project you generate today and read top to bottom.

Native theme accents

PR #4884 closes the loop on the new iOS Modern and Material 3 native themes we shipped two weeks ago. The native themes now expose their accent palette as named theme constants, so rebranding your app to your own colours is a five-line CSS change instead of a fork.

Override the constants inside the #Constants block of your own theme.css:

#Constants {
    includeNativeBool: true;
    darkModeBool: true;

    --accent-color: #ff2d95;
    --accent-color-dark: #ff2d95;
    --accent-pressed-color: #c71a75;
    --accent-on-color: #ffffff;
}

That is it. Every accent-bearing UIID picks up the new colour. Light and dark are independent (--accent-color vs --accent-color-dark), and partial overrides are fine; anything you do not redeclare stays at the framework default. Material 3 has a couple of additional container-tier constants for the elevated-surface tone; iOS ignores those.

There is also a runtime path for dynamic theming (in-app accent toggles, branded flavours, A/B tests). It uses the same constants. The Native Themes chapter of the developer guide covers it in detail, along with the full iOS and Android constant tables and the places where the binding system intentionally does not apply: Accent palette override.

The point worth pulling out: the parts of theming that do not change per app (which UIIDs participate in the accent palette, which states they expose, which dark-mode counterparts they have) live inside the framework and stay there. The parts that do change per app (your colours) live in your project as five constants and nothing else. That is the whole reason this change exists.

Metal follow-ups

Last week was about shipping the Metal renderer. This week is the follow-up week: three PRs, plus one new API on Graphics that I think will quietly pay for itself many times over.

Per-axis scale decomposition (#4939, fixes #3302)

Long-standing issue #3302 had a clear repro: g.translate + g.scale(sx, sy) + fillShape with sx != sy produced shapes that visibly drifted off the axis-aligned drawRect and drawLine calls the framework emitted alongside them. Triangles inscribed in rectangles escaped their bounding rect.

The cause was that the legacy alpha-mask path rasterised the shape at a uniform scale (the diagonal ratio h2/h1), then stretched the resulting texture non-uniformly through the GPU matrix to recover the requested aspect. The bbox math is exact in real numbers, but the texture is pixel-rounded at the intermediate uniform scale, so the stretch drifted the rasterised shape off the pixel grid that drawRect and drawLine were already on.

The fix factors the user transform’s 2×2 linear part by taking the column norms as (sx, sy), rasterises the path at S(sx, sy) so the per-axis stretch happens at rasterisation time against a vector path rather than a pixel grid, and applies only the residual transform * S(1/sx, 1/sy) on the GPU. The residual is pure rotation (and shear in the worst case), so no per-axis stretch happens at sample time and the alpha-mask texture lands on the same pixel grid as its drawRect siblings.

The change is gated to Metal; the GL ES2 path keeps its legacy branch so the existing GL goldens are byte-identical. A new InscribedTriangleGrid screenshot test was registered with Cn1ssDeviceRunner so the inscribed-triangle property is now visually verifiable in CI.

Clip-under-rotation diagnostic (#4924, towards #3921)

PR #4924 does not fix a bug, it localises one. Issue #3921 is “clip-under-rotation behaves wrong on some ports”, entangled with a getClip / setClip(int[]) round-trip limitation the reporter himself called out as a separate issue. To split the two, we shipped a screenshot test that uses only pushClip / popClip and rotateRadians. The clip becomes non-axis-aligned via clipRect inside a 30-degree rotation, which forces the framework through its polygon-clip branch.

The expected outcome is a 30-degree-tilted red fill that overlaps the navy outline at two diagonal corners and falls short at the other two. Two distinguishable failure modes are pre-labelled in the PR: the clip widened to its axis-aligned bbox (red exactly matches the navy outline), or the polygon clip dropped entirely (red fills the whole cell). When the iOS Metal cell of this test renders, we know within a glance which of the three behaviours we are looking at. The expected-failure cell is also a hypothesis: ClipRect.m‘s polygon initialiser stores x = y = w = h = -1, and the Metal execute path then calls CN1MetalSetScissor(0, 0, -2, -2), whose width <= 0 / height <= 0 branch sets the scissor to the full framebuffer instead of the intended polygon. If the screenshot confirms the hypothesis, the fix is a one-line replacement of the polygon-scissor fallback.

iOS Metal colour space hint (#4909, fixes #4908)

PR #4909 adds an ios.metal.colorSpace build hint. Until this week, the Metal layer’s CAMetalLayer.colorspace was hard-coded to sRGB. For most apps that is right; sRGB is what your existing assets are authored in. But on iPhone XR and later, Apple’s screens are wide-gamut (Display P3), and a marketing-led brand that ships P3 artwork was visibly losing saturation by being routed through the sRGB pipeline.

Accepted values are sRGB (default), displayP3, deviceRGB, linearSRGB, extendedSRGB, extendedLinearSRGB, and none. Set it in codenameone_settings.properties:

codename1.arg.ios.metal.colorSpace=displayP3

The hint is dormant when ios.metal=false, so existing GL builds are unchanged. Unrecognised values produce a warning log and fall back to sRGB. Documented under Working-With-iOS.asciidoc.

The new translateMatrix API

The Inscribed-Triangle-Grid test in #4939 also surfaced a quiet papercut in Graphics that is worth pulling out as its own feature.

Graphics.translate(int, int) does not compose into the affine transform the way scale() and rotateRadians() do. It accumulates into a per-Graphics integer offset that is added to draw coordinates before the impl matrix is applied. That is a holdover from the very first version of the framework, when Graphics did not have a matrix at all. Today the consequence is surprising: a subsequent g.scale(sx, sy) multiplies the integer translate too, which means the same code produces visibly different positions depending on whether you scale before or after you translate.

The new Graphics.translateMatrix(float, float) composes the translation directly onto the impl matrix, in the same way scale and rotateRadians already do. The result is uniform “post-multiply translate onto the current transform” semantics across iOS (both GL and Metal), JavaSE, Android, and the JavaScript port. Same code, same on-screen position, whether you are drawing into a Form‘s Graphics or a mutable Image‘s Graphics.

// Matrix-correct composition. Use this when you want translate to
// behave like scale and rotate (composed into the affine transform).
g.translateMatrix(centerX, centerY);
g.rotateRadians(angle);
g.scale(sx, sy);
g.translateMatrix(-centerX, -centerY);
g.fillShape(path);

For app code writing affine-transform pipelines (the “translate to pivot, rotate, scale, translate back” idiom from Java2D and AWT), this is the API you want. isTranslateMatrixSupported() returns true on every modern port. The old translate(int, int) is not deprecated and is not going anywhere; half the framework’s internal scrolling code is built on it. The new method is the one to reach for in new drawing code, particularly anything that combines translate with scale or rotate.

String API: replace(CharSequence, CharSequence), replaceAll, replaceFirst

PR #4893 closes a long-standing gap reported in issue #4878. The JDK 1.5+ overload of String.replace that takes CharSequence arguments (the one nearly every modern Java tutorial reaches for) was missing from the Codename One subset. So were String.replaceAll(String, String) and String.replaceFirst(String, String). Because none of the three were on the bootclasspath, code that reached for them did not compile against a Codename One project at all; you had to know to fall back to the older replace(char, char) overload and to roll your own regex.

All three are now wired in. String.replace(CharSequence, CharSequence) has a real implementation in vm/JavaAPI. replaceAll and replaceFirst are wired through the bytecode-compliance rewriter to a new JdkApiRewriteHelper pair that delegates to the existing RE regex engine (the same pattern we have been using for years on String.split). New compliance tests cover both rewrite rules.

It is a small change in line count. In practice it is a noticeable reduction in how often “I copied a snippet from Stack Overflow and it didn’t work on iOS” turns into a real bug. Three of the most-reached-for String methods in modern Java are now part of the on-device API.

iOS push permission no longer fires at app launch

PR #4894 fixes issue #4876. With ios.includePush=true the framework used to call requestAuthorizationWithOptions from application:didFinishLaunchingWithOptions:, which meant the iOS system permission dialog fired as soon as the app finished launching, before the user had seen any of your screens. There is no good way to recover from a “Don’t Allow” tap at that point. The user has not experienced the app yet, does not know why notifications matter, and tapping Don’t Allow is the path of least resistance. Once denied, re-prompting requires sending the user out to Settings.

The fix moves the prompt to the natural points. Push.register() triggers the system prompt (this code path already requested permission inside IOSNative.m; we just stopped firing it ahead of time). LocalNotification.schedule() also triggers it, via a new requestAuthorizationWithOptions call in sendLocalNotification. Same flow Android has been on for years. The practical consequence is that you can now show your own rationale screen (“we’d like to ping you when your order ships”) before the system dialog fires.

If you have an app that needs the legacy launch-time behaviour, a backwards-compatibility build hint restores it:

codename1.arg.ios.notificationPermissionAtLaunch=true

The default is false, so existing apps that did not opt in pick up the new behaviour on next rebuild. Documented in Push-Notifications.asciidoc. The cloud-side build server change shipped as BuildDaemon #71, so local and cloud builds match.

One thing to flag if you are updating an existing iOS app: if your onboarding flow was relying on the launch-time prompt happening automatically, your prompt now never fires unless Push.register() or LocalNotification.schedule() is invoked somewhere. That is almost certainly what you want, but check that the call lands.

Skin Designer FAQ follow-up

A few questions came up on discussion #4928 after last week’s Skin Designer post, worth pulling forward here because they keep coming up in the same shape:

• Skins do not affect CSS. The skin is simulator scaffolding (device frame, screen rect, cutouts, safe-area insets); your theme.css and your native theme are unrelated.
• For a known device, the defaults are usually right. Pick the device, hit Pick a shape, click Finish. The customization UI is there for when our device database is incomplete (the iPhone 17e entry might say “no notch” when it actually has one, or the notch position might be off by a few pixels); when you have a physical device to measure against, that is where you refine.
• Themes are leaving skins. Historically the native theme was bundled inside each skin because that is what made sense at the time. Going forward the right home for themes is the framework itself, distributed via Maven, so you pick up updates automatically. The new native themes already work this way. The per-skin embedded theme stays for legacy compatibility and the Skin Designer still writes one for you, but the Native Theme menu we shipped two weeks ago is the path forward.

The device database the Skin Designer reads from is open at scripts/skindesigner/common/src/main/resources/devices.json if you want to file a PR with a device we are missing or a row whose details are off.

Wrapping up

Two reminders. First, flip ios.metal=true on your real app this week if you have not. The default flip is days away and we would rather find any remaining edge case against your screens than against the install base on launch day. Second, if you have not generated a project from the Initializr recently, do it; the Java 17 default and the AGENTS.md skill are both worth seeing for yourself.

A specific thank-you this week to the reporter on #3302 for sticking with the inscribed-triangle bug for as long as GL was the only target, Durank for the iOS push permission report on #4876, and the reporter on #4878 who flagged the missing String.replace(CharSequence, CharSequence); that one had been sitting in the gap for a long time.

Issue tracker is here, the Playground and Initializr are the easiest places to poke at the new defaults, and the Skin Designer from last week is still there if you have a device shape you need a skin for.

Original Japanese article: AWS Glue Workflowの使い方について整理してみる

Introduction

I’m Aki, an AWS Community Builder (@jitepengin).

Previously, I wrote an article comparing when to use AWS Step Functions versus Glue Workflow.
Organizing the Use Cases of AWS Step Functions and Glue Workflow for ETL Processing with AWS Glue Jobs

As I mentioned there, I personally like Glue Workflow and consider it an excellent service that balances simplicity and low cost.

However, in recent years, Step Functions has become increasingly mainstream, and I get the impression that opportunities to work with Glue Workflow have decreased.
Because of that, I think many people are unsure about how they should actually use it in practice.

So in this article, I’d like to organize everything from the basics of Glue Workflow to practical usage patterns.
I hope this helps more people become interested in Glue Workflow.

What Is Glue Workflow?

AWS Glue Workflow is Glue’s native workflow orchestration feature.
It defines ETL pipelines by combining the following three elements:

By connecting these components as a DAG (Directed Acyclic Graph), you can build ETL pipelines.

Another characteristic is that workflows can be visually configured through the Glue console GUI.

A major advantage is that workflows themselves incur no additional cost—you only pay for Job and Crawler execution.

Details of the Core Components

The core of Glue Workflow is the Trigger mechanism.
There are four trigger types, each with different roles.

The typical pattern is:

• Use SCHEDULED, ON_DEMAND, or EVENT as the workflow’s starting trigger
• Use CONDITIONAL to connect downstream Jobs

By the way, AWS officially recommends keeping the number of elements included in a workflow (Jobs + Crawlers + Triggers) under 100.
Exceeding this recommendation can cause errors when resuming or stopping Workflow Runs.

CONDITIONAL Trigger Usage Patterns

CONDITIONAL Triggers are one of the key features that provide flexibility in Glue Workflow.
Here are several representative patterns.

1. Simple Sequential Pattern

This is the most basic usage pattern: “Run JobB after JobA succeeds.”

JobA (SUCCEEDED) → JobB

You can implement this simply by configuring a CONDITIONAL Trigger with the condition:
“Start when JobA reaches the SUCCEEDED state.”

2. Waiting for Multiple Jobs to Complete (AND Condition)

This pattern runs JobC only after both JobA and JobB succeed.

JobA (SUCCEEDED) ┐
                 ├→ JobC
JobB (SUCCEEDED) ┘

This can be implemented by setting Logical: AND in the CONDITIONAL Trigger predicate and listing multiple conditions.

This is useful in scenarios such as:
“Run an aggregation Job only after multiple data sources have finished loading.”

3. Triggering When Any Job Completes (ANY Condition)

This pattern runs JobC when either JobA or JobB succeeds.

The predicate of a CONDITIONAL Trigger has a Logical field where you can specify either AND or ANY.
Using ANY causes the trigger to fire as soon as any one of the specified conditions is satisfied.

Note that although this behavior is logically equivalent to “OR,” the actual Glue configuration value is ANY.
This is important when defining workflows using IaC or CLI because specifying OR will result in an error.

4. Failure Branching (Catching FAILED States)

CONDITIONAL Triggers can react not only to SUCCEEDED, but also to states such as:

• FAILED
• STOPPED
• TIMEOUT
• ERROR

(The supported states differ slightly between Jobs and Crawlers.)

Using this feature, you can create patterns such as launching a notification Job (for example, a Python Shell Job that publishes to SNS) when a Job fails.

JobA (SUCCEEDED) → Downstream Processing
JobA (FAILED)    → Notification Job

For relatively simple error handling, this approach allows you to avoid introducing Step Functions.

Managing Parameters with default_run_properties

Glue Workflow provides a property called default_run_properties, which acts like globally shared variables across the workflow.

How It Works

default_run_properties stores key-value pairs that can be referenced by all Jobs within the workflow.

It functions as the default set of parameters passed during Workflow execution and serves as the foundation for sharing information between Jobs.

One important note:
Run Property values may appear in logs, so you should avoid storing secrets directly in them.

Instead, retrieve secrets through services such as:

• AWS Secrets Manager
• Glue Connections

How to Configure It

There are three main configuration methods.

Configure from the Console

In the Glue console:
Workflow → Edit Properties

You can add key-value pairs there.

Configure via boto3

import boto3

glue = boto3.client('glue')

glue.create_workflow(
    Name='my-workflow',
    DefaultRunProperties={
        'env': 'production',
        'target_date': '2026-05-21'
    }
)

Configure via IaC (CloudFormation, etc.)

You can specify DefaultRunProperties in the AWS::Glue::Workflow resource.

Static Parameters vs Dynamic Parameters

Typical usage patterns include:

• Static parameters

  • Environment names (dev / prod)
  • S3 bucket names
  • Values that rarely change

• Dynamic parameters

  • Processing dates
  • Execution IDs
  • Values that change per execution

Dynamic parameters can be updated either by:

• Passing RunProperties to start_workflow_run
• Dynamically updating them later using put_workflow_run_properties

Passing Data Between Jobs

Using default_run_properties as the foundation, let’s look at how to exchange data between Jobs.

Dynamically Updating Run Properties

During Job execution, you can dynamically update Workflow Run properties by calling the put_workflow_run_properties API.

import boto3

glue = boto3.client('glue')

glue.put_workflow_run_properties(
    Name='my-workflow',
    RunId=workflow_run_id,
    RunProperties={
        'processed_records': '12345',
        'output_path': 's3://mybucket/output/2026-05-21/'
    }
)

This allows downstream Jobs to reference values calculated by upstream Jobs.

Retrieving Properties from a PySpark Job

Inside a PySpark Job, you first retrieve the Workflow Run ID and then call get_workflow_run_properties.

import sys
import boto3
from awsglue.utils import getResolvedOptions

args = getResolvedOptions(
    sys.argv,
    ['JOB_NAME', 'WORKFLOW_NAME', 'WORKFLOW_RUN_ID']
)

glue = boto3.client('glue')

response = glue.get_workflow_run_properties(
    Name=args['WORKFLOW_NAME'],
    RunId=args['WORKFLOW_RUN_ID']
)

properties = response['RunProperties']

target_date = properties.get('target_date')

WORKFLOW_NAME and WORKFLOW_RUN_ID are special arguments automatically passed when a Job is launched through a Workflow.

Retrieving Properties from a Python Shell Job

The basic approach is the same for Python Shell Jobs:

• Retrieve arguments using getResolvedOptions
• Access properties through boto3

Since Python Shell Jobs do not require SparkContext initialization, they can be written more lightweight.

They are also cheaper than Spark-based Glue Jobs, so depending on the requirements, they can be a good option.

Personally, I also like Python Shell Jobs, although I feel opportunities to use them in real-world projects have decreased, which is a bit unfortunate.

I’ve also written articles about Python Shell Jobs, so feel free to check them out.

S3 Triggers: How to Launch Glue Python Shell via AWS Lambda

Anti-Patterns for Data Passing

default_run_properties should only be used for metadata-like information exchange.

The following usage patterns should generally be avoided:

• Passing large datasets directly

  • Store the data itself in S3 and pass only the path

• Storing secrets

  • Use Secrets Manager or Glue Connections instead

• Frequently rewriting properties

  • This increases API calls and introduces race condition risks

Integrating with EventBridge and Other Services

Glue Workflow becomes much more flexible when combined with EventBridge.

EventBridge-Based Startup (EVENT Trigger)

Glue Workflow can be started directly by EventBridge events.

This is achieved by setting the Trigger Type to EVENT.

aws glue create-trigger 
  --workflow-name my-workflow 
  --type EVENT 
  --name s3-arrival-trigger 
  --actions JobName=my-job

By configuring an EventBridge rule with Glue Workflow as the target, the workflow starts automatically when an event occurs.

However, appropriate IAM permissions such as glue:notifyEvent are required.

Batch Event Startup

EVENT Triggers also support event batching.

Using EventBatchingCondition, you can configure the workflow to start when either:

• N events arrive
• M seconds pass since the first event arrived

aws glue create-trigger 
  --workflow-name my-workflow 
  --type EVENT 
  --name batch-trigger 
  --event-batching-condition BatchSize=10,BatchWindow=300 
  --actions JobName=my-job

This enables patterns such as:
“Run ETL once 100 files have arrived.”

The maximum batch window is 900 seconds (15 minutes).

Starting from S3 Events (The Parameter Passing Limitation)

A common use case is:
“Start a workflow when a file is uploaded to S3.”

When starting Glue Workflow through EventBridge, the event IDs are automatically stored in a Run Property called aws:eventIds.

event_ids = glue_client.get_workflow_run_properties(
    Name=workflow_name,
    RunId=workflow_run_id
)['RunProperties']['aws:eventIds']

The returned value looks like:

'["abc-123", "def-456"]'

However, this is where one of Glue Workflow’s limitations becomes apparent.

The EventBridge event payload itself (such as the S3 object key or bucket name) is not automatically passed as Run Properties.

Only the event IDs are provided.

If you need the actual object details, your Job must retrieve the corresponding event contents from CloudTrail, which becomes somewhat cumbersome.

Because of this, many cases are easier to manage by placing Lambda in the middle and explicitly calling start_workflow_run with structured Run Properties.

S3 PUT → EventBridge → Lambda → start_workflow_run (pass parameters via RunProperties)

Example Lambda code:

import boto3

glue = boto3.client('glue')

def lambda_handler(event, context):
    bucket = event['detail']['bucket']['name']
    key = event['detail']['object']['key']

    glue.start_workflow_run(
        Name='my-workflow',
        RunProperties={
            'source_bucket': bucket,
            'source_key': key
        }
    )

With Step Functions, the EventBridge payload can be received directly using paths such as $.detail, so there is no need for an intermediate Lambda function.

This is one of the areas where Glue Workflow limitations become noticeable compared to Step Functions.

Detecting Workflow Completion via EventBridge

Glue Workflow status changes are emitted as EventBridge events.

You can use this to:

• Send SNS notifications
• Trigger downstream systems
• Launch post-processing workflows

Glue Workflow (COMPLETED/FAILED)
    → EventBridge
        → SNS / Lambda

This is especially useful when you want separate processing for success and failure cases.

Calling Glue Workflow from Step Functions

It is also possible to launch Glue Workflow from Step Functions.

However, since there is no .sync integration pattern available, you must implement your own polling logic to detect completion.

I covered this in detail in the previous article, so feel free to refer to it if interested.

Organizing the Use Cases of AWS Step Functions and Glue Workflow for ETL Processing with AWS Glue Jobs

Operational Tips

Here are several practical points worth knowing when operating Glue Workflow in production.

Resuming Failed Workflows (ResumeWorkflowRun)

Glue Workflow provides a feature called ResumeWorkflowRun, which allows resuming execution from failed nodes.

In the console, you can:

• Open the failed Workflow Run detail page
• Select the nodes to resume
• Enable the “Resume” checkbox

It is also available through CLI/API.

aws glue resume-workflow-run 
  --name my-workflow 
  --run-id wr_xxxx 
  --node-ids node_yyyy node_zzzz

When resumed:

• The specified nodes
• And all downstream nodes

are re-executed.

The resumed workflow is tracked using a new Run ID.

Compared with Step Functions Redrive, however, there are several limitations:

• You must explicitly specify failed nodes
• Retrieving node IDs requires calling:
  get-workflow-run --include-graph
• Additional IAM permissions (glue:ResumeWorkflowRun) are required

For simple retry scenarios, restarting the entire workflow via ON_DEMAND execution is often easier.

For pipelines with more sophisticated recovery requirements, Step Functions tends to provide a smoother operational experience.

Monitoring with CloudWatch

Glue Workflow execution states can be viewed in the Glue console, but detailed Job and Crawler logs are output to CloudWatch Logs.

Typical monitoring targets include:

• Workflow Run list

  • Glue Console

• Job execution logs

  • /aws-glue/jobs/output
  • /aws-glue/jobs/error

• Metrics

  • CloudWatch Metrics
  • Execution duration
  • DPU usage

In practice, monitoring and alerting strategies often combine:

• Workflow-level statuses
• Job-level metrics

depending on the situation.

Handling the 100-Object Recommendation Limit

As mentioned earlier, AWS recommends keeping the total number of objects in a workflow (Jobs + Crawlers + Triggers) below 100.

If a large pipeline approaches this limit, consider:

• Splitting workflows

  • Trigger downstream workflows after upstream completion

• Consolidating common processing inside Jobs

  • Merge smaller processing units into larger Jobs

In my experience, workflows approaching 100 objects are often already too complex from a design perspective.
At that point, it may be worth reconsidering the architecture itself or migrating to Step Functions.

Where Glue Workflow Fits — and Its Limitations

Glue Workflow shines in scenarios such as:

• Simple ETL pipelines completed entirely within Glue Jobs and Crawlers
• Periodic ingestion pipelines for data lakes
• Lightweight small-to-medium scale pipelines that need to be launched quickly

On the other hand, Step Functions is generally better suited for:

• Integrations involving Lambda, ECS, and other AWS services
• Complex branching and dynamic parameter control
• Large-scale development involving multiple teams

I discussed these decision criteria in more detail in the previous article, so feel free to refer to it.

Organizing the Use Cases of AWS Step Functions and Glue Workflow for ETL Processing with AWS Glue Jobs

Conclusion

In this article, I organized AWS Glue Workflow from the basics through practical usage patterns.

To summarize:

• Four trigger types:
  SCHEDULED / ON_DEMAND / CONDITIONAL / EVENT

• CONDITIONAL Triggers:
  Flexible flow control using AND / ANY conditions and failure branching

• default_run_properties:
  Shared workflow-wide parameter management

• Data passing between Jobs:
  Dynamic value propagation using put_workflow_run_properties

• EventBridge integration:
  Event-driven execution via EVENT Trigger
  (although Lambda-based parameter passing is often easier)

• ResumeWorkflowRun:
  Partial restart functionality from failed nodes

After writing all of this, I still have to admit:
Step Functions is generally easier to use.

That said, Glue Workflow still provides meaningful value today because it allows you to build Glue-centric ETL pipelines in a very simple and cost-efficient way.

Rather than defaulting to Step Functions automatically, understanding Glue Workflow properly and knowing when to use it can broaden your architectural options.

I hope this article helps both people who are starting to use Glue Workflow and those already working with it.