GHSA-PRH4-VHFH-24MJ: GHSA-PRH4-VHFH-24MJ: Information Exposure in Harbor Configuration Audit Logs

GHSA-PRH4-VHFH-24MJ: Information Exposure in Harbor Configuration Audit Logs

Vulnerability ID: GHSA-PRH4-VHFH-24MJ
CVSS Score: 5.3
Published: 2026-03-26

Harbor, an open-source cloud native registry, contains a Moderate severity vulnerability (CWE-532) in its audit logging subsystem. The application relies on an incomplete blacklist to redact sensitive data from configuration payloads. This failure causes LDAP passwords, specifically ldap_search_password, and OpenID Connect (OIDC) client secrets to be written to the database in plain text within the operation description field. This vulnerability allows authorized users with audit log access to retrieve enterprise directory credentials.

TL;DR

Harbor fails to properly redact sensitive configuration parameters from its audit logs. This exposes LDAP and OIDC credentials in plain text to any user with audit log read access, requiring an upgrade to version 2.15.0 and immediate rotation of exposed secrets.

Technical Details

  • Vulnerability Class: CWE-532: Insertion of Sensitive Information into Log File
  • Attack Vector: Authenticated Application Access (Audit Logs)
  • Impact: Exposure of External Directory Credentials (LDAP/OIDC)
  • Exploit Status: Unexploited / No Public PoC
  • Fix Approach: Deny-all / Complete Payload Removal from Logs
  • Key Vulnerable Component: src/pkg/auditext/event/config/config.go

Affected Systems

  • Harbor Container Registry (< 2.15.0)
  • Harbor: < 2.15.0 (Fixed in: 2.15.0)

Code Analysis

Commit: 85e7564

Fix: Adopt a deny-all approach for configuration audit logs by replacing payload interpolation with a static string.

Mitigation Strategies

  • Upgrade to Harbor v2.15.0 or later to completely remove configuration request payloads from audit logging.
  • Purge or manually redact historical configuration update entries within the audit_log database table.
  • Enforce the principle of least privilege for the LDAP service account used by Harbor.
  • Proactively monitor logs for unauthorized access attempts originating from Harbor-associated service accounts.

Remediation Steps:

  1. Identify the current version of Harbor running in your environment.
  2. Pull the Harbor v2.15.0 (or newer) release and apply the upgrade according to official documentation.
  3. Connect to the Harbor backing database and query the audit_log table for rows where the op_desc field contains sensitive configuration data.
  4. Delete or obfuscate the identified legacy log entries.
  5. Generate new LDAP bind passwords and OIDC client secrets.
  6. Update the new credentials in the Harbor management interface to ensure directory synchronization is maintained.

References

Read the full report for GHSA-PRH4-VHFH-24MJ on our website for more details including interactive diagrams and full exploit analysis.

I Love Watching the Storms Roll In

I have always liked the weather radar. First of all, it’s rainbow. Second of all, the weather is mysterious and constantly changing. I like the visual aid of seeing what’s about to head my way, and making the connection of what the radar looks like versus what is happening outside. I am not a meteorologist. I am a curious observer. I’ve been building this fun and simple app for myself and to share with students to get them excited about science and weather. Hey it’s storm season after all!

I’ve included the live stream of one of my favorite storm crew’s when they go live – the ya’ll squad. I obviously take no credit for any of their videos, but I enjoy shoring it with others.

I am using publicly available API keys of free services like NOAA/NWS for radar images. I am Using another free one for air quality information. I hope to set up a weather bug or something of the sort so we can have live data for our location, potentially sharing our data for the greater good. This is one of my latest side projects. Just wanted to share the fun colors.

What weather related apps have you created? I may look into some other radars, but this one is working and I am happy with it for now. If you would like to see the actual site you are welcome to visit it on github pages here:

SECRET: Click the “Weather Station” title at the top to use it in your location. Let me know what you think!

SIDE NOTE: The video only plays when the stream is live. Otherwise, it displays current info.

The 5 Principles of Snyk’s Developer Experience

In the age of AI-driven development, speed is the new baseline. But as AI agents accelerate the pace of coding, they also amplify the risk of security bottlenecks. At Snyk, we believe a superior Developer Experience (DX) is the only way to secure this new frontier. DX is not just a layer on top of the product. It is the foundation that allows developers to unleash AI innovation securely.

We think of DX as a system of decisions that compound over time. Every interaction, every default, and every piece of information a developer encounters shapes how effectively they can use our platform.

The five principles that emerged from our journey of evolving and refining the Snyk platform now serve as the foundation for delivering an excellent DX. These principles continuously guide the thousands of small decisions we make across the entire product surface, underscoring our commitment to this ongoing process.

1. Go to where developers work, don’t ask them to come to you

The most common challenge in developer tooling is the assumption that a beautiful dashboard is the primary destination. Experience shows that developers prioritize their existing workflows, which they have optimised over the years: their IDE, the terminal, their Git flow, and their pull request (PR) process. Context switching out of those workflows has a tremendous cost.

We saw this directly at Snyk. We had built a detailed findings interface in the Snyk platform with prioritized vulnerability lists, remediation guidance, and full data flow traces. Developers did not visit it. We learned that even the most valuable data is often bypassed if it requires a context switch. By moving security into the existing PR conversation, we aligned with the developer’s natural flow.

We changed our model. We stopped asking developers to come to Snyk and started bringing Snyk to them. Security findings became part of the pull request conversation, surfaced directly in the SCM in the same thread where code review was already happening. Same information. Zero context switch, but dramatically different adoption.


The principle goes beyond PRs. It is why we invest heavily in IDE plugins, AI coding assistant and, CLI integrations, and CI/CD gates. The question we ask is always the same: where is the developer already working, and how do we show up there?

There’s a broader shift underway from traditional IDEs to agentic development environments. At the velocity that AI coding assistants drive, context switching becomes a much bigger bottleneck, since higher agent productivity amplifies the cost of breaking flow. As agentic platforms become a core part of developer workflows, Snyk is already integrated in those environments to secure AI-generated code at inception.

2. Developers are not security specialists, so speak their language

When we designed security findings in PRs, we optimized for the developer’s mental mode. CVSS scores or CWE classifications made sense to security professionals, but to developers they were jargon that required translation.

We surface a contextual, natural-language description generated from Snyk’s own data flow analysis. For a SQL injection vulnerability, for example, rather than citing a generic advisory, we would explain that unsanitised user input from the HTTP request body is directly interpolated into an SQL query string, naming the source, the sink, and the mechanism in the developer’s own code.


That one sentence tells a developer, who is often not a security specialist, exactly what and where (to the exact file and line number) the problem is, in terms they already understand. The full trace is still available for those who want it. But most developers do not need to go deeper. They need to understand enough to act.

And every surface of Snyk’s product attempts to apply this principle. We aim to answer, “What does this developer need to understand, at this moment, given what they know?”

3. Every piece of information is either signal or noise – there’s no middle ground

There is a tendency in security tools to surface everything. It feels comprehensive, but it often overwhelms rather than helps. When we examined our PR experience, we reframed the problem: what information truly belongs in the developer’s view?

We chose to be deliberate. What we show depends on the workflow. In prevention, developers need fast, actionable guidance. In remediation, they need depth and more paths when they are optimising for risk reduction. In a PR, every piece of information should either answer an immediate question or enable a clear next step. This context matters a lot as developers in a PR are focused on shipping the functionality. Vulnerability resolution becomes secondary. This is very different from a backlog context, where fixing issues is the primary task.

Progressive disclosure also helps balance this. The primary view focuses on the issue, its severity, and the next step. Deeper layers provide additional context, such as data flows, when needed. This keeps the experience focused and noiseless.

4. Detection is not the product, resolution is

For a long time, security tools measured success by what they found. The more vulnerabilities surfaced, the more complete the tool felt. What this metric missed was the thing that actually mattered: whether those vulnerabilities got fixed.

Most developers do not want awareness. They want to know what to do next. A vulnerability report with no clear next step is just noise with a severity score, and developers, quite rationally, learn to treat it that way.

The motivation behind building fix suggestions directly into the PR experience was to close the loop: not just identify vulnerabilities, but fix them without ever leaving the workflow. When Snyk detects a vulnerability in code, it does not just flag it. It proposes a concrete, AI-generated fix as a diff, inline in the PR as a review comment, red lines out, green lines in, applicable as a commit with a single action.


For the SQL injection example, rather than flagging the string interpolation and leaving the developer to figure out the solution, the AI Fix suggestion replaces it with a parameterized query. The developer does not need to research secure SQL practices, the fix is already there. The path to resolution becomes the default path.

Good DX tells them how to fix an issue, but a great DX makes fixing the default path.

5. Trust is built when developers understand why, not just what

When we launched the suggested fix, we saw a pattern repeatedly in developer feedback: the question was not “does the fix work?” It was “Why does this fix work?” Developers were applying suggestions and then struggling to explain them to colleagues. The fix was solving the immediate problem and creating a different one.

So we added something that turned out to be one of the highest-signal changes we made to the PR check experience: a plain-English explanation of exactly why the suggested change eliminates the vulnerability. Not a link to documentation. Not a reference to the CVE. An explanation, derived from the code’s specifics, of how the fix addresses the vulnerability.

For the SQL injection example, the explanation would describe how replacing dynamic string interpolation with parameterized queries ensures that user input is treated as data rather than executable code and why that distinction closes the vulnerability.

The combination of these two features, suggested fix and its explanation, mirrors how a senior security engineer would actually review code with a colleague: first making sure they understand the problem, then showing them what good looks like.

Trust is built through reasoning. Every time Snyk explains its thinking, it gives developers the tools to develop their own security instincts, which is, ultimately, the most durable outcome.

Great developer experience does not happen by accident

These five principles got solidified by watching what broke, understanding why, and changing our approach.

Great developer experience requires principles like these that can guide thousands of small decisions across product, engineering, and design. As we move into a future where AI and human developers collaborate more closely, these principles ensure that security remains a tailwind, not a hurdle. At Snyk, we are constantly striving to get better; one decision, one fix, and one successful deployment at a time.

See how the developer experience Snyk has built can accelerate your program. Get a demo today.

I Built an Image Generation MCP for Claude Code — Gemini, OpenAI, and FLUX in One Place

Symbol cat

Why I built this

I use Claude Code as my primary dev environment. Code, review, deploy — almost everything happens inside Claude Code. But there was one thing that kept pulling me out: generating images.

Every time I needed a diagram or an illustration, I’d switch to Google AI Studio, rewrite the prompt without the original context, and manually save the result back into my project. About 30 times a day.

So I built mcp-imagenate, an MCP server that brings image generation directly into Claude Code and Claude Desktop. It hit 1,000 downloads on npm within 4 days of release.

Symbol cat

Note: This is an alpha release, built as a personal project. Use at your own risk.

Use Cases

Here are three ways I actually use this daily.

1. Generate system architecture diagrams from Claude Code

After designing and implementing a system in Claude Code, I just say “generate an architecture diagram for this system.” Claude Code already understands the full project context, so it constructs a solid prompt and generates the diagram via Gemini’s nano-banana-2. The image is saved directly into the project directory.

2. Turn daily journaling into illustrated entries

I keep a daily journal using Claude and the Notion MCP. With mcp-imagenate added, Claude now generates illustrations for each entry automatically. Combined with Claude Code’s Skills feature, I can iterate on image prompts without ever leaving the conversation — proper prompt engineering, right inside the chat.

3. Generate images on the fly during frontend development

While building a frontend in Claude Code, I can ask it to generate and insert images on the spot. Need a cute apologetic cat illustration for an error page? Claude Code generates it, saves it to the right directory, and references it in the code. No context switch needed.

These are just my workflows — I’m sure there are plenty more possibilities with MCP.

Supported Providers

Provider Model
Google Gemini nano-banana-2, nano-banana-pro
OpenAI gpt-image-1.5
BFL FLUX flux-2-klein, flux-2-pro, flux-2-max

Claude constructs the image generation prompt and picks the model. You can also configure resolution, aspect ratio, and output directory. Google models additionally support image editing with an input image.

Why This Matters

Mix and match chat models with image generation models.

Chat with Claude while generating images with Gemini’s nano-banana-2. Or FLUX. Or OpenAI. Pick the best chat model and the best image model independently. You no longer need to choose Gemini just because it can generate images.

One interface for everything.

Gemini, GPT, FLUX — all callable from Claude Code or Claude Desktop. No more opening browser tabs and copy-pasting context between platforms.

Prompt engineering, context, and image generation in one flow.

While developing in Claude Code, you think “I need an icon here” — and it’s generated, saved to the right directory, and referenced in your code. Claude Code understands the system design, constructs the image prompt, generates the image, and embeds it in documentation. No context is lost.

Getting Started

Set your API keys as environment variables and add the server config to Claude Desktop or Claude Code. That’s it.

See the README for setup instructions.

Note for Claude Desktop users

Due to a limitation in the Claude Desktop app, generated images cannot be displayed inline. Claude will output the file path — open it from your terminal with open <path> (macOS) or your file manager.

Links

If you try it out, I’d love to hear your feedback on X or GitHub Issues.

# Apache Data Lakehouse Weekly: March 20–27, 2026

With the Iceberg Summit less than two weeks away, the open lakehouse ecosystem spent this week in a final push of preparation, stabilization, and policy refinement. The AI contribution guidelines debate that erupted across Iceberg and Polaris last week continued drawing community input, while release engineering and summit logistics dominated the technical threads. Across all four projects, the mood is pre-conference focus — tying off loose ends before the community gathers in San Francisco on April 8–9.

Apache Iceberg

The Iceberg community spent the week in summit countdown mode. With the Iceberg Summit 2026 now less than two weeks out, logistics threads picked up as the selection committee finalized the speaker lineup. The two-day event at the Marriott Marquis in San Francisco will feature hands-on workshops, deep technical sessions, and direct access to core maintainers — the largest gathering of Iceberg practitioners yet.

On the release front, the 1.10.2 patch release discussion that Amogh Jahagirdar opened in the prior week continued to gather input from the community. The patch targets bugs discovered since 1.10.1 and aims to keep the production branch stable while contributors push forward on the 1.11.0 release cycle. With production users relying on the 1.10.x line, the community is being methodical about what gets backported versus what waits for the next minor release.

The AI contribution guidelines thread that huaxin gao opened the previous week continued to draw thoughtful responses. Holden Karau, Kevin Liu, Steve Loughran, and Sung Yun all weighed in on the challenge of maintaining code quality and Apache IP compliance as AI-assisted pull requests increase. The conversation is zeroing in on practical guardrails — disclosure requirements, review standards, and how to handle contributions where the provenance of code is unclear. This is likely to produce a formal policy before the Summit.

Péter Váry’s efficient column updates proposal for wide tables continued its steady progress. The design, which targets ML feature stores and vector databases with thousands of columns, has now been through multiple community syncs and is moving toward a written design document. The core idea — writing only updated columns to separate files and stitching them together at read time — would dramatically reduce write amplification for AI workloads. Steve Loughran’s parallel thread on benchmarking commit performance methodology is providing the measurement framework that will validate these changes.

Viktor Kessler’s Basel, Switzerland meetup announcement from last week continued generating interest, expanding Iceberg’s in-person community presence to Europe alongside the North American Summit. The combination of a major conference and grassroots meetups reflects a community that’s investing heavily in face-to-face collaboration.

Apache Polaris

Polaris continued settling into its role as an independent top-level Apache project. Jean-Baptiste Onofré circulated the project’s first board report as a TLP, covering the March 26 board meeting. This is a governance milestone — the report documents community health, development progress, and strategic direction under Polaris’s own PMC rather than the Incubator’s oversight. The report highlighted the project’s growth since graduation on February 18, including active development on credential vending expansion and the catalog federation proposal.

The catalog federation discussion that launched the previous week continued this cycle. The design would allow Polaris to federate across multiple catalog instances in multi-cloud deployments — a capability that enterprise users running Iceberg across AWS, Azure, and GCS have been requesting. Alongside federation, the web console proposal for Polaris gained traction, with contributors discussing how a browser-based UI would complement the existing CLI and REST API for catalog management.

Like Iceberg, Polaris is navigating the AI contribution guidelines question. EJ Wang’s thread continued to draw JB Onofré, Yufei Gu, and Dmitri Bourlatchkov into what’s becoming a cross-project conversation. The two communities are likely to produce coordinated — if not identical — policies, reflecting their shared contributor base and overlapping governance sensibilities. The 1.4.0 release, which will be Polaris’s first release as a graduated project, remains in active planning with credential vending for Azure and GCS backends as the headlining feature.

Apache Arrow

Arrow’s dev list this week focused on planning for the 24.0.0 release cycle and continuing the JDK 17 discussion that JB Onofré kicked off after Arrow Java 19.0.0 shipped earlier in March. David Li’s supportive response from last week was followed by additional community input on the migration timeline. If Arrow Java 20.0.0 sets JDK 17 as the minimum, it would align Arrow with Iceberg’s Java modernization trajectory — effectively raising the Java floor for the entire lakehouse stack in one coordinated move.

Nic Crane’s thread on using LLMs to aid with project maintenance continued generating discussion. Unlike the Iceberg and Polaris threads, which focus on AI-generated contributions from external submitters, Arrow’s framing centers on how maintainers themselves can use AI tools to manage the project’s growing codebase and issue backlog. Sutou Kouhei’s Map type key/item/value field names technical thread drew continued engagement from Micah Kornfield and Antoine Pitrou, working through naming consistency across language implementations. Google Summer of Code 2026 student proposals also continued to arrive, with interest in compute kernels and language bindings.

Apache Parquet

Parquet’s community held its bi-weekly sync this period and continued active technical discussions on two fronts. The File logical type proposal that emerged the previous week remained the project’s most consequential design thread. The proposal would allow Parquet files to natively represent unstructured data — images, PDFs, audio — inside columnar files. If adopted, it would expand Parquet’s role from a purely analytical format to a hybrid that can manage the unstructured data that AI/ML pipelines generate alongside the structured features they consume.

The Variant type announcement from February continued to see adoption discussion, with contributors sharing integration experiences across Spark, Trino, and Dremio. Variant brings native semi-structured data support to Parquet, eliminating the need to store JSON strings in regular columns. Combined with the File logical type proposal, Parquet is rapidly expanding its type system to handle the diverse data shapes that modern analytics and AI workloads demand. The ALP floating-point encoding spec is now through its final review, with the formal acceptance vote expected imminently.

Cross-Project Themes

The AI contribution policy conversation is the clearest cross-project theme this week. Iceberg, Polaris, and Arrow are all grappling with the same question from different angles: Iceberg and Polaris are focused on contributor-side disclosure and review standards for AI-generated code, while Arrow is exploring how maintainers can responsibly use AI for project upkeep. The fact that these conversations are happening in parallel — with many of the same people participating across lists — suggests coordinated policies will emerge. This is an open-source governance question that will define how all four projects operate in the years ahead.

The second theme is format scope expansion. Parquet’s File logical type and Variant type, Iceberg’s efficient column updates for wide ML tables, and Polaris’s catalog federation for multi-cloud environments all point in the same direction: the open lakehouse is being asked to handle workloads far beyond traditional analytics. The stack is evolving from a data warehousing alternative into a unified platform for structured analytics, semi-structured data, unstructured files, and AI/ML feature engineering — all governed through a single open catalog.

Looking Ahead

The Iceberg Summit on April 8–9 is the event to watch. Expect final speaker announcements, agenda details, and community organizing threads to accelerate over the next two weeks. On the release side, watch for the Iceberg 1.10.2 patch release vote to open, the Polaris 1.4.0 release planning to solidify, and the Parquet ALP encoding vote to close. The AI contribution guidelines threads across Iceberg and Polaris should produce actionable proposals — potentially in time for discussion at the Summit itself.

Resources & Further Learning

Get Started with Dremio

Free Downloads

Books by Alex Merced

I Couldn’t Find an OAuth 2.1 Proxy for MCP Servers, So I Built One

mcp-gate architecture

When I started deploying custom MCP servers to connect to Claude.ai, I hit a wall fast.

Claude.ai’s custom connector flow requires your MCP server to implement OAuth 2.1 Protected Resource Metadata — specifically RFC 9728 — before it will even attempt to authenticate. No RFC 9728 /.well-known/oauth-protected-resource endpoint? Silent failure. No error. The connector just doesn’t work.

I went looking for an existing solution. Something that could sit in front of any MCP server, handle the spec compliance, validate JWTs, and get out of the way. Nothing existed. So I built it: mcp-gate.

What the Problem Actually Is

When Claude.ai connects to a custom MCP server, the flow looks roughly like this:

  1. Claude.ai fetches /.well-known/oauth-protected-resource from your server
  2. That endpoint must return RFC 9728 metadata pointing to your authorization server
  3. Claude.ai negotiates an OAuth 2.1 token with that authorization server
  4. Subsequent requests carry a Bearer JWT, which your server must validate via JWKS

Most MCP server implementations — whether you’re writing one in Go, Python, TypeScript, or using an off-the-shelf server — don’t implement any of this. They’re focused on the MCP protocol itself, not on auth infrastructure. Bolting RFC 9728 + JWKS validation directly into every server you deploy is the wrong abstraction anyway.

What you actually want is a sidecar proxy that handles auth uniformly, regardless of what’s running behind it.

What mcp-gate Does

mcp-gate is a stateless Go binary that sits between Claude.ai and your MCP server:

mcp-gate architecture

It does three things:

  1. Serves /.well-known/oauth-protected-resource — RFC 9728 compliant metadata, pointing Claude.ai to your authorization server
  2. Validates Bearer JWTs on every request — fetches and caches JWKS from your auth server, validates signature (RS256), expiry, issuer, audience, and scope claims
  3. Reverse proxies to your upstream MCP server — if the token is valid, the request goes through; otherwise 401

That’s it. It has no opinions about what your MCP server does. It works with any OIDC-compliant authorization server: Authentik, Keycloak, Okta, Auth0, or anything else that issues JWTs with a JWKS endpoint.

Quick Start

export LISTEN_ADDR=0.0.0.0:8080
export UPSTREAM_URL=http://your-mcp-server:8000
export RESOURCE_URI=https://mcp.yourdomain.com
export AUTHORIZATION_SERVER=https://auth.yourdomain.com/application/o/mcp/
export JWKS_URI=https://auth.yourdomain.com/application/o/mcp/jwks/
export EXPECTED_ISSUER=https://auth.yourdomain.com/application/o/mcp/
export EXPECTED_AUDIENCE=your-client-id

go run ./cmd/mcp-gate

Or with Docker:

docker pull cpremus/mcp-gate:latest

A docker-compose.example.yml is included in the repo showing how to wire it up alongside a typical MCP server container.

A Note on Logging

Every request is logged as structured JSON — method, path, status, duration_ms, client_ip, user_agent — designed for ingestion by Loki/Alloy or any structured log aggregator. If you’re running a self-hosted observability stack, it drops right in.

Why a Separate Binary

A few design decisions worth explaining:

Stateless by design. mcp-gate holds no session state. JWKS keys are cached in-memory with TTL-based refresh. This makes horizontal scaling trivial and keeps the failure surface small.

No config files. Everything is environment variables. Works cleanly in Docker, Kubernetes, or bare metal.

Deliberately narrow scope. mcp-gate does auth and proxying. It doesn’t do request transformation or protocol-level concerns. Those belong elsewhere in your stack.

Status

The core is stable and running in production on my own infrastructure. Current release handles everything needed for Claude.ai custom connector integration.

Links

If you’re building or deploying MCP servers and haven’t hit this yet, you will. Hopefully this saves you the afternoon I lost to it.

6Σ Models Explained: The Ultimate Guide to Six Sigma Methodologies for Business Excellence

6Σ Models Explained: The Ultimate Guide to Six Sigma Methodologies for

Business Excellence

In the hyper-competitive landscape of modern business, efficiency isn’t just a
buzzword; it’s the difference between market leadership and obsolescence.
Enter 6Σ models , a data-driven methodology that has revolutionized how
organizations approach quality control and process improvement. From
manufacturing giants like Motorola and General Electric to tech startups and
healthcare providers, Six Sigma (6Σ) has become the gold standard for
eliminating defects and optimizing workflows.

But what exactly are 6Σ models, and how can they transform your organization?
This comprehensive guide dives deep into the core methodologies, the
statistical backbone, and practical applications of Six Sigma, providing you
with actionable insights to drive tangible results.

What Are 6Σ Models?

At its core, a 6Σ (Six Sigma) model is a disciplined, statistical approach
aimed at improving process quality by identifying and removing the causes of
defects and minimizing variability in manufacturing and business processes.
The term “Six Sigma” refers to a statistical concept where a process produces
no more than 3.4 defects per million opportunities (DPMO). This represents a
level of performance where 99.99966% of all opportunities to produce a feature
are statistically expected to be free of defects.

Unlike generic quality assurance tactics, 6Σ models rely heavily on data
analysis and a structured framework. They are not merely about fixing problems
as they arise but about creating a culture of continuous improvement where
processes are designed to be robust against variation from the start.

The Two Primary 6Σ Methodologies: DMAIC and DMADV

While Six Sigma is often spoken of as a single entity, it actually comprises
two distinct project methodologies, each serving a specific purpose within an
organization. Understanding the difference is crucial for selecting the right
tool for your specific business challenge.

1. DMAIC: For Improving Existing Processes

DMAIC is the most widely used 6Σ model. It is an acronym for Define,
Measure, Analyze, Improve, and Control
. This methodology is applied when a
product or process already exists but is failing to meet customer
specifications or production goals.

  • Define: Identify the problem, the project goals, and the customer (internal and external) requirements. What is the specific issue causing pain?
  • Measure: Collect data related to the current process. Establish a baseline performance metric to understand the magnitude of the problem.
  • Analyze: Use statistical tools to determine the root cause of defects. Why is the process failing? Is it machine error, human error, or material inconsistency?
  • Improve: Develop and implement solutions to address the root causes. This often involves pilot testing and refining the solution before full-scale rollout.
  • Control: Standardize the successful changes and implement monitoring systems to ensure the gains are sustained over time.

2. DMADV: For Creating New Processes

Also known as DFSS (Design for Six Sigma), the DMADV model is used when a
process does not exist or when an existing process requires more than just
incremental improvement—it needs a complete redesign. DMADV stands for
Define, Measure, Analyze, Design, and Verify.

  • Define: Set design goals that are consistent with customer demand and enterprise strategy.
  • Measure: Identify Critical-to-Quality (CTQ) characteristics and measure product capabilities and production risks.
  • Analyze: Develop and design alternatives, select the best design, and analyze its ability to meet requirements.
  • Design: Create a detailed design of the new process or product, optimizing it for Six Sigma levels of performance.
  • Verify: Validate the design through pilot runs and implement the process, then hand it over to the process owner with ongoing monitoring plans.

The Statistical Backbone: Why “Six Sigma”?

The name itself is derived from statistics. In a normal distribution, sigma
(σ) represents the standard deviation from the mean. A process operating at
“One Sigma” has a high rate of defects. As you move up the scale, the error
rate drops dramatically:

  • 3 Sigma: 93.3% accuracy (66,807 defects per million).
  • 4 Sigma: 99.38% accuracy (6,210 defects per million).
  • 5 Sigma: 99.979% accuracy (233 defects per million).
  • 6 Sigma: 99.99966% accuracy (3.4 defects per million).

Achieving 6Σ means your process is so tightly controlled that variation is
virtually non-existent. This level of precision translates directly to cost
savings, higher customer satisfaction, and increased profitability.

Real-World Applications of 6Σ Models

Six Sigma is not limited to the factory floor. Its principles are universally
applicable across various sectors:

Manufacturing

This is the birthplace of Six Sigma. Companies use 6Σ models to reduce scrap
rates, minimize machine downtime, and ensure product consistency. For example,
an automotive parts manufacturer might use DMAIC to reduce the variance in the
diameter of a piston ring, ensuring a perfect fit every time.

Healthcare

In hospitals, 6Σ models help reduce patient wait times, minimize medication
errors, and streamline administrative workflows. By applying DMAIC, a hospital
might analyze the discharge process to identify bottlenecks, resulting in
faster room turnover and improved patient throughput.

Finance and Banking

Banks utilize Six Sigma to reduce transaction errors, speed up loan processing
times, and enhance cybersecurity measures. A credit card company might use
DMADV to design a new fraud detection algorithm that minimizes false positives
while catching genuine threats.

Software Development

Although Agile and DevOps are dominant, Six Sigma complements them by focusing
on bug reduction and deployment reliability. Teams can use 6Σ principles to
analyze code deployment failures and create more robust testing protocols.

Key Roles in a Six Sigma Implementation

Successful implementation of 6Σ models requires a structured hierarchy of
trained professionals, often denoted by martial arts-style belts:

  • Champions: Senior executives who sponsor the initiative and remove organizational barriers.
  • Master Black Belts: Full-time trainers and experts who mentor Black Belts and ensure statistical rigor.
  • Black Belts: Project leaders who work full-time on Six Sigma projects, leading teams and analyzing data.
  • Green Belts: Employees who work on Six Sigma projects part-time while maintaining their regular job functions.
  • Yellow Belts: Team members with basic training who support project activities.

Common Pitfalls to Avoid

While powerful, 6Σ models are not a magic wand. Organizations often fail when
they:

  1. Lack Leadership Support: Without executive buy-in, cultural resistance can stall projects.
  2. Focus Only on Tools: Six Sigma is a mindset, not just a set of statistical tools. Ignoring the cultural aspect leads to failure.
  3. Poor Project Selection: Choosing projects that don’t align with strategic business goals wastes resources.
  4. Inadequate Training: Attempting complex statistical analysis without proper certification or understanding leads to erroneous conclusions.

Conclusion

6Σ models offer a rigorous, proven framework for achieving operational
excellence. Whether you are looking to refine an existing workflow using DMAIC
or design a groundbreaking new system with DMADV, the principles of Six Sigma
provide the roadmap to near-perfect quality. By reducing variation and
focusing on data-driven decision-making, organizations can unlock significant
cost savings and deliver superior value to their customers. In an era where
precision matters, mastering 6Σ models is not just an option—it’s a strategic
imperative.

Frequently Asked Questions (FAQ)

What is the main difference between Lean and Six Sigma?

Lean focuses on eliminating waste and speeding up processes, while Six Sigma
focuses on reducing variation and defects. Often, they are combined into “Lean
Six Sigma” to leverage the benefits of both approaches.

How long does it take to get Six Sigma certified?

The duration varies by level. A Yellow Belt course may take a few days, a
Green Belt typically takes 2-4 weeks, and a Black Belt certification can take
several months of training and project completion.

Can small businesses benefit from 6Σ models?

Absolutely. While large corporations pioneered it, small businesses can use
simplified 6Σ models to reduce errors, save money, and improve customer
satisfaction without needing massive datasets.

Is Six Sigma only for manufacturing?

No. Six Sigma is highly effective in service industries, healthcare, finance,
IT, and logistics. Any process that can be measured can be improved using Six
Sigma methodologies.

What software is commonly used for Six Sigma analysis?

Popular tools include Minitab, JMP, SAS, and even advanced features in
Microsoft Excel. These tools help practitioners perform the complex
statistical analyses required in the Analyze phase.

Amazon Bedrock + RDS Aurora: Generative AI Inside Your MySQL Database

Cover

Cover

Have you ever dreamed of having an AI assistant inside your database, helping you optimize queries and explore vast datasets?

Well, that dream is about to become reality. In this article, I’ll walk you hand-in-hand through the exciting world of integrating Amazon Bedrock with RDS Aurora MySQL. Get ready to discover how this Generative AI combination can revolutionize the way you interact with your data and optimize your SQL queries.

Let’s start this journey toward the future of AI-powered databases!

What Is Amazon Bedrock?

Amazon Bedrock is a managed Generative AI service that was launched in early 2023, providing us with access to multiple cutting-edge AI models through a single API.

This service has many features and is constantly evolving and growing; here are the most important ones from my perspective:

Access to AI models: It offers access to large language models (LLMs) and other AI models from leading companies: Anthropic, AI21 Labs, Meta, Cohere, Mistral AI, Stability AI, and Amazon.

Unified API: It allows developers to access and use different AI models through a single interface, simplifying integration. With Bedrock, it’s just a matter of slightly changing the call and we can switch from one model to another — making it easy to test and evaluate which model best fits our use case.

AWS Integration: It integrates seamlessly with other AWS services.

Security and privacy: A very important element when it comes to Generative AI, and of course it includes options for secure data handling and regulatory compliance.

Prerequisites: Setting the Stage

Before we dive into the integration, let’s make sure we have everything ready:

1. Access to the Anthropic Claude 3.5 Sonnet Model
Before starting the configuration process, it’s important to request access to the models you’ll need from the Bedrock console. For this exercise, I’ll use the most advanced Anthropic model available in Bedrock, which is Claude 3.5 Sonnet.

This is done in the ‘Bedrock Settings’ section, where we need to confirm the corresponding permission for that model. Of course, you can enable other models if you want to experiment with different LLMs to compare responses.

Bedrock Model

Tip: Enable other models if you want to experiment!

2. RDS Aurora MySQL
We need to have a properly provisioned RDS Aurora MySQL cluster, with at least version 3.06 since that’s the minimum version with support for this feature.
As part of this exercise, we’ll use the popular MySQL test database called Sakila, so you should already have it properly installed on your cluster.

Configuration: Step by Step Toward Integration

1. Create an IAM Role and Policy
This integration requires ‘AWS Identity and Access Management’ (IAM) roles and policies to allow the Aurora MySQL cluster to access and use Amazon Bedrock services.

First, we create a new IAM policy that must contain the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "bedrock:InvokeModel",
            "Resource": "*"
        }
    ]
}

💡 Tip: Save this policy with the name BedrockInvokeModel. We’ll use it later.

Now, we need to create a role. We must select ‘Add Role to Database’ as the use case, as shown in the image.

Create Role

Next, in the permissions section, we need to associate the previously created policy.

Associate Policy

The final result should look like this:

Associate Policy

Take note of the ARN of this new role, as we’ll use it later; its format is similar to: arn:aws:iam::XXXXX:role/RDSCallBedrockRole.

2. Create a Parameter Group in RDS

Now, we need a new parameter group for our cluster:

Parameter Group

Once created, we’ll edit the aws_default_bedrock_role parameter in this group to set the ARN of the role we created in the previous step.

Parameter Group

After that, we need to modify the RDS cluster to use our new custom parameter group.

Parameter Group

As a final step, we associate the same role to the Aurora cluster.

Parameter Group

It’s important to reboot the cluster so that the changes we’ve made take effect.

Want to verify everything is in order? Run this command:

SHOW GLOBAL VARIABLES LIKE 'aws_default%';

And you should see the role we’ve been using as the value.

Variable_name Value
aws_default_bedrock_role arn:aws:iam::XXXXX:role/RDSCallBedrockRole

3. Create User and Permissions

For our exercise, I’ll assume you already have a user created with full permissions on the sakila database. We’ll assume it’s called demo.

We need to grant the following permission to our user:

GRANT AWS_BEDROCK_ACCESS TO 'demo'@'%';

And then we set the effective privileges in our session.

SET ROLE AWS_BEDROCK_ACCESS;

If I tried to test Bedrock access right now, I’d get a connectivity error because the network configuration doesn’t allow it yet. We’ll fix this in the next step.

4. Network Configuration
There are several ways to configure communication between RDS and Bedrock, but for this case we’ll use a VPC Endpoint.

First, we need to create a new endpoint and select com.amazonaws.region.bedrock-agent-runtime as the service. Be careful to select that one and not one of the other available options.

Endpoint

Then we need to associate it with our VPC, the appropriate subnets, and select the security group to attach. In my example, I used the database’s security group — just make sure it has ingress traffic permissions authorized for itself.

Endpoint VPC

That wraps up the VPC configuration.

Our Assistant: A SQL Expert at Your Disposal

Imagine having a virtual SQL assistant that not only optimizes your queries but also explains why it does so. That’s exactly what we’re going to build!

The reason that motivated me to do this goes back to the fact that many years ago I was a ‘Database Administrator,’ and even today I regularly see how developers create SQL statements that lack the minimum elements to be considered adequately optimized. So it hit me: what if we give them a tool that lets them submit a SQL statement and an assistant recommends how to rewrite it properly, considering the database schema, and also tells them in a measurable way the impact on execution time improvement?

Key Components:

  1. query_history table: Stores the before and after of your queries, along with their execution times.
  2. generate_optimized_query function: Leverages the power of Claude 3.5 Sonnet to improve your queries.
  3. analyze_and_optimize_query procedure: The brain of the operation.
  4. Gathers schema information from the current database.
  5. Generates an optimized version of the input query using the AI model.
  6. Executes both the original and optimized queries, measuring their execution times.
  7. Stores the results in the history table.
  8. Displays a comparison of the queries and their execution times.

Code

The complete source code is in the following GitHub repository. Here I’ll share the most relevant parts.

First, we create a function that invokes the Claude 3.5 Sonnet model in Bedrock — note the model ID shown there. This function receives a JSON argument.

CREATE FUNCTION invoke_sonnet (request_body TEXT)
    RETURNS TEXT
    ALIAS AWS_BEDROCK_INVOKE_MODEL
    MODEL ID 'anthropic.claude-3-5-sonnet-20240620-v1:0'
    CONTENT_TYPE 'application/json'
    ACCEPT 'application/json';

This model ID can be obtained in at least two ways:

  1. Directly in the Bedrock console, where you can find that identifier in the base models section.

    Model Id

  2. Using the AWS CLI and running the following command (if you have the appropriate permissions)

aws bedrock list-foundation-models --query '*[].[modelName,modelId]' --out table

which returns the list of all available foundation models, for example:

Model Model Id
Titan Multimodal Embeddings G1 amazon.titan-embed-image-v1
SDXL 1.0 stability.stable-diffusion-xl-v1:0
Jurassic-2 Ultra ai21.j2-ultra
Claude 3 Sonnet anthropic.claude-3-sonnet-20240229-v1:0
Claude 3 Haiku anthropic.claude-3-haiku-20240307-v1:0
Claude 3.5 Sonnet anthropic.claude-3-5-sonnet-20240620-v1:0
Llama 3 70B Instruct meta.llama3-70b-instruct-v1:0
Mistral Large (2402) mistral.mistral-large-2402-v1:0

Our next function is generate_optimized_query. In it, we set up a prompt instructing the model to act as an optimization expert, taking a SQL statement and the corresponding schema information as input. I also limit the response to a maximum of 500 tokens and build the JSON according to the specification required by Claude 3.5 Sonnet. Getting predictable, structured responses from the LLM is key in this context; if you want to go deeper into techniques for achieving deterministic outputs from language models, I recommend checking out this article on deterministic LLMs.

DELIMITER //
CREATE FUNCTION generate_optimized_query(input_query TEXT, schema_info TEXT)
RETURNS TEXT
BEGIN
    DECLARE result TEXT;
    DECLARE prompt TEXT;
    DECLARE json_payload TEXT;

    SET prompt = CONCAT('Act as a MySQL database optimization expert. ',
                'Given the following SQL query and schema information, ',
                'provide an optimized version of the query. ',
                'Only return the optimized query, without explanations. ',
                'Original query: "', input_query, '" ',
                'Schema information: "', schema_info, '"');

    SET json_payload = JSON_OBJECT(
        'anthropic_version', 'bedrock-2023-05-31',
        'max_tokens', 500,
        'messages', JSON_ARRAY(
            JSON_OBJECT(
                'role', 'user',
                'content', JSON_ARRAY(
                    JSON_OBJECT(
                        'type', 'text',
                        'text', prompt
                    )
                )
            )
        )
    );

    SET result = invoke_sonnet(json_payload);

    RETURN JSON_UNQUOTE(JSON_EXTRACT(result, '$.content[0].text'));
END //
DELIMITER ;

A simple way to find out which JSON each model expects is to go to the Bedrock console, select the providers list, click on the model of interest, and at the bottom you’ll find an API example.

API Example

For our test, I’ll send a SQL statement to our assistant to validate its behavior:

CALL analyze_and_optimize_query('
    SELECT c.first_name, c.last_name,
           COUNT(r.rental_id) as rental_count,
           SUM(p.amount) as total_spent
    FROM customer c
    JOIN rental r ON c.customer_id = r.customer_id
    JOIN payment p ON r.rental_id = p.rental_id
    JOIN inventory i ON r.inventory_id = i.inventory_id
    JOIN film f ON i.film_id = f.film_id
    WHERE f.rating = "PG" AND YEAR(r.rental_date) = 2005
    GROUP BY c.customer_id
    HAVING rental_count > 5
    ORDER BY total_spent DESC
    LIMIT 10
');

The result we get is the following:

Statement Query Execution Time
Original SELECT c.first_name, c.last_name, COUNT(r.rental_id) as rental_count, SUM(p.amount) as total_spent FROM customer c JOIN rental r ON c.customer_id = r.customer_id JOIN payment p ON r.rental_id = p.rental_id JOIN inventory i ON r.inventory_id = i.inventory_id JOIN film f ON i.film_id = f.film_id WHERE f.rating = “PG” AND YEAR(r.rental_date) = 2005 GROUP BY c.customer_id HAVING rental_count > 5 ORDER BY total_spent DESC LIMIT 10 0.070305
Optimized SELECT c.first_name, c.last_name, COUNT(r.rental_id) as rental_count, SUM(p.amount) as total_spent FROM customer c JOIN rental r ON c.customer_id = r.customer_id JOIN payment p ON r.rental_id = p.rental_id JOIN inventory i ON r.inventory_id = i.inventory_id JOIN film f ON i.film_id = f.film_id WHERE f.rating = ‘PG’ AND r.rental_date >= ‘2005-01-01’ AND r.rental_date < ‘2006-01-01’ GROUP BY c.customer_id HAVING rental_count > 5 ORDER BY total_spent DESC LIMIT 10 0.052826

The result? A 24.86% improvement in execution time. Not bad for a few minutes of work!

Let’s go a bit further — if I modify the prompt and ask it to justify its changes, we get the following reasoning:

  1. Eliminating double quotes: We changed double quotes to single quotes in f.rating = 'PG' to follow SQL best practices.
  2. Optimizing HAVING: We changed HAVING rental_count > 5 to HAVING COUNT(r.rental_id) > 5. This avoids using an alias in the HAVING clause, which can be more efficient in some database engines.
  3. Optimizing ORDER BY: We changed ORDER BY total_spent DESC to ORDER BY SUM(p.amount) DESC. Just like with HAVING, using the expression directly instead of the alias can be more efficient in some cases.
  4. Indexes: Although we can’t create indexes in this query, make sure indexes exist on the columns used in the JOIN, WHERE, and ORDER BY clauses. Specifically:

    • customer_id in the customer and rental tables
    • rental_id in the rental and payment tables
    • inventory_id in the rental and inventory tables
    • film_id in the inventory and film tables
    • rating in the film table
    • rental_date in the rental table
  5. Partitioning: If the rental table is very large, consider partitioning it by year to improve the performance of the YEAR(r.rental_date) = 2005 clause.

  6. Materialized views: If this query runs frequently, consider creating a materialized view containing the pre-aggregated data.

These optimizations should improve query performance, especially when implemented alongside proper indexes and other database-level optimizations.

That’s a pretty reasonable explanation — it details each action and its justification.

From here, we can keep evolving our assistant and send it more database context such as indexes, running queries, open connection counts, among many other things. This would further expand the possibilities this assistant opens up. To efficiently manage the connection pool to Aurora in high-concurrency scenarios, consider using RDS Proxy, which centralizes and optimizes database connection management.

Conclusions: The Future Is Now

Integrating Amazon Bedrock with Aurora MySQL isn’t just a technical improvement — it’s a huge leap in how we interact with our databases:

  1. Automatic optimization: Imagine having an expert DBA working 24/7 on your queries.
  2. Continuous learning: Every optimization is a lesson for your team.
  3. Time and resource savings: Less time debugging, more time innovating.
  4. Scalability: As your database grows, your assistant grows with you.

But this is just the beginning. Can you imagine integrating sentiment analysis into your SQL queries? RDS Aurora MySQL and PostgreSQL have support for Amazon Comprehend.
Or maybe generating automatic reports based on your data? Well, you can also integrate with SageMaker. The only limit is our imagination.

Next Steps:

  1. 🚀 Experiment with different Bedrock models
  2. 📊 Create dashboards that show query performance improvements
  3. 🤝 Share your experiences and learnings with the community

Start experimenting today!

I hope this article has been useful and that it motivates you to try new things on AWS!

Questions? Comments? Leave them below! And don’t forget to share this article if you found it helpful.

LLM + SQL: Deterministic Answers with Amazon Bedrock and Athena

Cover

Cover

Introduction

In today’s dynamic landscape of generative artificial intelligence, large language models (LLMs) have radically transformed how we interact with technology. These models have demonstrated exceptional capabilities in tasks such as text generation, sentiment analysis, and contextual understanding. However, when we face scenarios that require absolute precision and deterministic results, we encounter inherent limitations that need to be addressed in innovative ways.

The Challenge of Non-Deterministic Models

How LLMs Work Under the Hood

Large language models operate through a sophisticated probabilistic system. At their core, these models:

  1. Contextual Prediction: They analyze prior context to predict the most probable next word or sequence.
  2. Probability Distribution: They generate a probability distribution across different response options.
  3. Temperature and Randomness: They use parameters like temperature to control creativity vs. determinism in their responses.

This probabilistic nature is precisely what makes LLMs so versatile for creative and analytical tasks, but it’s also what makes them less reliable for queries that demand numerical exactness or absolute precision.

Context: From a Failed POC to an Innovative Solution

Over the past few months, while giving multiple talks on Generative AI, one particular conversation caught my attention. A development team shared their frustration with a proof of concept (POC) they considered a failure. The problem: their generative AI implementation for support ticket analysis was producing inconsistent results.

As I dug deeper into the case, an interesting pattern emerged:

What Worked Well:

  • “Analyze support ticket X”
  • “What’s the summary of case Y?”
  • “What does this incident report suggest?”

These questions, which required contextual understanding and qualitative analysis, received precise and useful answers.

What Consistently Failed:

  • “Which department has the most open tickets?”
  • “How many tickets were handled last month?”
  • “What’s the average resolution time?”

Questions that required numerical precision and exact calculations never provided reliable results.

The Key Revelation

The reason for the failure became evident once we understood the fundamental nature of LLMs: they are inherently non-deterministic. Their strength lies in natural language processing and probability-based content generation, not in performing precise calculations or exact queries on structured data.

This insight led me to reformulate the key question:

How can we answer deterministic questions when an LLM, by its very nature, isn’t designed to do so?

The answer emerged when I recognized that we didn’t need to force the LLM to do something it wasn’t designed for. Instead, we could:

  1. Use the LLM for what it does best: understanding the intent of the question.
  2. Translate that intent into structured queries when necessary.
  3. Use specialized tools for precise calculations.
  4. Present the results in a coherent and natural way.

Bridging the Gap Between Precision and Probability: Implementing the Solution

Once I identified the core of the problem, I developed a proposal that first determines the nature of the query and then applies the appropriate processing.

Query Classification

Deterministic Queries:

Characteristics:

  • Require exact, reproducible counts.
  • Involve aggregations on specific ticket fields.
  • Operate on the schema defined in Athena.

Real-World Examples:

  1. “Which department has the most open tickets?”
    Generated SQL:
   SELECT departamento, COUNT(*) as total
   FROM tickets
   WHERE estado != 'CLOSED'
   GROUP BY departamento
   ORDER BY total DESC
  1. “What is the leading cause of registered incidents?”
    Generated SQL:
   SELECT causante, COUNT(*) as total_incidentes
   FROM tickets
   WHERE solicitudes = 'Incidentes'
   GROUP BY causante
   ORDER BY total_incidentes DESC
   LIMIT 1

Non-Deterministic Queries:

Characteristics:

  • Require contextual analysis of ticket content.
  • Benefit from natural language processing.
  • Are handled by the Bedrock Knowledge Base.

Examples:

  1. Analysis of specific ticket content.
  2. Case summaries.
  3. Pattern interpretation in reports.

Processing Flow

The flow I decided to follow to tackle the challenge is divided into three simple steps.

  1. Initial Evaluation

    • Uses the defined prompt to determine if the query is deterministic. In this step, as we’ll see later, I use an LLM to figure out whether what the user is asking is deterministic by nature or not.
    • When it is deterministic, the LLM generates the appropriate SQL within <SQL> tags. This is based on an Athena table and a data dictionary.
  2. Processing

    • Deterministic queries: Executed through Athena — we send a SQL query created by an LLM that satisfies the user’s question.
    • Non-deterministic queries: Processed through Amazon Bedrock – Knowledge Base. This knowledge base contains the same CSV file we use in Athena.
  3. Response Formatting

    • Athena results are limited to 25 records (because we don’t want a single question to be able to return the entire database).
    • The LLM is used to convert results into natural language responses.
    • The language consistency of the original question is maintained.

Solution Architecture

The implemented architecture solves the deterministic query challenge through a strategic combination of AWS services and LLM processing. Let’s analyze each component and its detailed implementation.

1. Data Storage and Preparation Layer

1.1 Base Data Structure

The system operates on a CSV file hosted in S3 that contains ticket records. The preparation of this data is crucial and requires:

CREATE EXTERNAL TABLE IF NOT EXISTS `default`.`tickets` (
 `fechaResolucion` string,
 `asignado` string,
 `solicitudes` string,
 `producto` string,
 `departamento` string,
 -- [remaining fields]
)
COMMENT "Example tickets table"
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
WITH SERDEPROPERTIES ('field.delim' = ';')
STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 's3://MiBucket/'
TBLPROPERTIES ('classification' = 'csv');

This DDL is fundamental because:

  • It defines the exact structure that Athena will use for queries.
  • It specifies the ; delimiter for correct CSV interpretation.
  • It establishes the S3 location where the data resides.
  • It configures the input/output format to optimize processing.

1.2 Data Dictionary

Along with the structure, we maintain a detailed data dictionary that the LLM will use to understand the context of each field. For example:

fechaResolucion: Field indicating the ticket resolution date and time.
                Format: month/day/year hour:minute
causante: Categorical field indicating whether the ticket was raised by A or B
departamento: Calculated descriptive field of the department that handled it

2. Query Classification System

2.1 Classification Prompt

The first crucial step is determining whether a query is deterministic. We implement this through a specific prompt:

StringBuilder prompt = new StringBuilder(
   "You are an expert in ticket analysis, I need you to analyze " +
   "the question I provide and if that question cannot be answered " +
   "by an LLM (because it is deterministic) respond only with the phrase " +
   "'DETERMINISTIC' followed by a SQL inside a <SQL> tag that complies " +
   "with the definition of the following Athena table and its glossary..."
);

This prompt is critical because:

  • It defines the specific role for the model.
  • It establishes the exact expected response format.
  • It includes the schema context and data dictionary.
  • It forces a structured, processable response.

2.2 SQL Generation via LLM

Once the system has identified that the query is deterministic, it returns the SQL to be sent to Athena for execution. This is achieved because we included the table definition and data dictionary in the previous prompt.

In a previous article about using Bedrock with RDS, I explained how an LLM can be used to generate SQL — and that prior experience is part of this solution.

2.2.1 Model Configuration and Invocation

var message = Message.builder()
      .content(ContentBlock.fromText(prompt.toString()))
      .role(ConversationRole.USER)
      .build();

try {
  var client = BedrockRuntimeClient.builder()
      .credentialsProvider(DefaultCredentialsProvider.create())
      .region(Region.US_EAST_1)
      .build();

  // Send the message with a basic inference configuration.
  ConverseResponse response = client.converse(request -> request
          .modelId(FOUNDATIONAL_MODEL)
          .messages(message)
          .inferenceConfig(config -> config
                  .maxTokens(512)    // Enough for complex SQL queries
                  .temperature(0.5F) // Low for higher precision
                  .topP(0.9F)));     // High coherence in structure

  // Retrieve the generated text from Bedrock's response object.
  var responseText = response.output().message().content().get(0).text();
  client.close();

  return responseText;

} catch (SdkClientException e) {
  System.err.printf("ERROR: Can't invoke '%s'. Reason: %s", FOUNDATIONAL_MODEL, e.getMessage());
  return "Unable to answer that question";
}

2.2.2 Complete Flow Example

To illustrate the process, let’s consider the question: “Which department has the most open tickets?”

  1. Input Processed by the Model:
[All previous context + schema + dictionary]
Question: Which department has the most open tickets?
  1. Generated SQL:
SELECT
   departamento,
   COUNT(*) as total_tickets
FROM tickets
WHERE fechaResolucion IS NULL
GROUP BY departamento
ORDER BY total_tickets DESC
LIMIT 25

The generated SQL is sent directly to Athena for execution, leveraging the fact that the model already knows the exact table structure and the meaning of each field thanks to the provided context.

The key to this approach’s success lies in the precision of the context provided to the model and the consistency of the requested response format, enabling reliable generation of SQL queries that match our schema exactly.

3. Deterministic Query Processing

3.1 Athena Query Execution

Once a deterministic query is identified, the system executes the generated SQL:

public String executeAthenaQuery(String query, String database) {
 try (AthenaClient athenaClient = AthenaClient.builder()
   .region(Region.US_EAST_1) // Adjust region according to your configuration
   .credentialsProvider(DefaultCredentialsProvider.create())
   .build()) {

   // Configure the query request
   StartQueryExecutionRequest startQueryExecutionRequest = StartQueryExecutionRequest.builder()
     .queryString(query)
     .queryExecutionContext(QueryExecutionContext.builder()
       .database(database)
       .build())
     .resultConfiguration(ResultConfiguration.builder()
       .build())
     .build();

   // Start the query
   StartQueryExecutionResponse startQueryExecutionResponse = athenaClient.startQueryExecution(startQueryExecutionRequest);
   String queryExecutionId = startQueryExecutionResponse.queryExecutionId();

   // Wait for the query to complete
   waitForQueryToComplete(athenaClient, queryExecutionId);

   // Get the query results
   return getQueryResults(athenaClient, queryExecutionId);

 } catch (Exception e) {
   e.printStackTrace();
   throw new RuntimeException("Error executing Athena query", e);
 }
}

This code:

  • Establishes a secure connection with Athena.
  • Executes the query asynchronously.
  • Manages the execution ID for tracking.

4. Response Formatting

The final step involves transforming technical results into comprehensible responses:

StringBuilder prompt = new StringBuilder(
   "You are an expert in answering queries, you must respond " +
   "in a professional, concise, and clear manner. The question asked was " +
   preguntaUsuario + " and the database response is: " +
   respuestaBD);

This formatting:

  • Maintains the context of the original question.
  • Structures the response naturally.
  • Preserves the precision of the obtained data.

5. Handling Non-Deterministic Queries

When the system identifies a query as non-deterministic, it means the query requires contextual or interpretive analysis that cannot be resolved through a direct SQL query. In this case, the system uses the Anthropic model directly to process the query.

5.1 Identification and Processing

The identification happens in the first step of the process, when the model does not return the word “DETERMINISTIC” followed by SQL. In this case, the system proceeds to process the query using the Bedrock model directly.

5.2 Model Configuration

For these queries, we use the base configuration of the Anthropic Sonnet 3.5 v2 model:

RetrieveAndGenerateInput input = RetrieveAndGenerateInput.builder()
        .text(prompt)
        .build();

KnowledgeBaseRetrieveAndGenerateConfiguration knowledgeConfig = KnowledgeBaseRetrieveAndGenerateConfiguration
        .builder()
        .knowledgeBaseId(KNOWLEDGE_BASE_ID)
        .modelArn(MODEL_ARN)
        .build();

RetrieveAndGenerateConfiguration retrieveConfig = RetrieveAndGenerateConfiguration.builder()
        .knowledgeBaseConfiguration(knowledgeConfig)
        .type("KNOWLEDGE_BASE")
        .build();

RetrieveAndGenerateRequest request1 = RetrieveAndGenerateRequest.builder()
        .retrieveAndGenerateConfiguration(retrieveConfig)
        .input(input)
        .build();

RetrieveAndGenerateResponse response1 = bedrockAgentRuntimeClient.retrieveAndGenerate(request1);

5.3 Non-Deterministic Query Examples

The following queries are typical examples that the system processes interpretively:

  1. Content Analysis:
  Question: "What are the common patterns in connection error tickets?"
  1. Case Interpretation:
  Question: "How was a similar case resolved last time?"
  1. Contextual Summaries:
  Question: "Summarize the main issue of ticket #12345"

In these cases, the system:

  • Does not attempt to generate SQL.
  • Processes the query directly through the model.
  • Provides a response based on context and available information.
  • Maintains a format and tone consistent with the original question.

The response is delivered directly to the user, maintaining the conversational nature and context of the original question.

Conclusions and Next Steps

The implementation of this hybrid system, which combines the precision of SQL queries with the interpretive capability of language models, represents just the beginning of what’s possible with Generative Artificial Intelligence in enterprise data analysis.

Key Takeaways

  • The automatic distinction between deterministic and non-deterministic queries allows us to leverage the best of both worlds: the exactness of relational databases and the contextual understanding of LLMs.
  • The implemented architecture demonstrates that it’s possible to maintain the precision required in enterprise environments while significantly improving the user experience.
  • Using modern services like Amazon Bedrock allows us to implement advanced AI solutions without needing to manage complex infrastructure, while having access to state-of-the-art LLMs.

I encourage you to take this example as a starting point for your own explorations. Whether you’re looking to improve your ticket analysis systems or want to apply these concepts to entirely different domains, the possibilities are enormous.

GenAI is transforming the way we interact with data, and I’m excited to be part of this transformation. Are you ready to be part of it too?

Amazon Bedrock Agents: Building an Industrial AI Assistant

Cover

Cover

The Silent Revolution in Industry 🏭

I recently had a conversation with a colleague about predictive analysis on industrial equipment. His previous experience with chatbots and generative AI had been frustrating — “I’m sorry, I don’t understand your question” was the most common response. That conversation inspired me to explore whether the landscape had changed by December 2024, combining Amazon Bedrock Agents with industrial APIs to build something genuinely useful.

My first exposure to this kind of industry was eye-opening. My colleague explained how they had multiple sensors generating data 24/7, multiple dashboards, yet still depended entirely on human expertise to interpret everything.

Let me walk through this interaction with our industrial assistant to illustrate the contrast:

Operador: "¿Hay alguna novedad en los dispositivos de la planta?"

Asistente: "Sí, hay novedades importantes que requieren atención:

🔴 Situaciones Críticas:
- Turbina EQ-003 (Planta Norte): Estado crítico
- Motor EQ-004 (Planta Sur): Estado crítico

🟡 Situación de Advertencia:
- Bomba EQ-001 (Planta Norte): Estado de advertencia

🟢 Operación Normal:
- Turbina EQ-002 (Planta Oeste)
- Compresor EQ-005 (Planta Oeste)

Se recomienda una revisión inmediata de los equipos en estado crítico, especialmente 
la turbina EQ-003 y el motor EQ-004."

This level of contextual understanding and predictive analysis is exactly what we’ll build today. With Amazon Bedrock Agents, it’s within reach of any developer with a passion for innovation.

The Art of the AI-Adapted API: Beyond Traditional REST 🎯

Remember when we designed APIs thinking only about human developers? For years, I’ve created, reviewed, and refined RESTful endpoints following best practices. One of the most valuable lessons I’ve learned is that detailed documentation isn’t just a good practice — it’s fundamental to success.

My first attempt at connecting an agent to a poorly documented API was revealing. It was like watching a new engineer trying to understand a codebase with no documentation. The agent showed exactly the same behavior: confused and making poor decisions.

That experience reinforced a fundamental truth:

Documentation is the cornerstone of our design.

We’re no longer designing only for human developers, but also for language models that need rich, meaningful context to function effectively.

An AI-Adapted API 🏗️

As you can deduce, the success of an AI-adapted API is built on one fundamental element: documentation. Let’s see how our documentation evolves to meet these needs:

Rich and Meaningful Context 📚

Let me compare two approaches to API documentation:

Traditional Approach:

/sensors/temp:
  get:
    summary: "Obtener temperatura"
    responses:
      200:
        description: "Éxito"

AI-Enriched Approach:

/equipment/{id}/health:
  get:
    description: |
      Evalúa el estado integral del equipo considerando múltiples factores:
      - Métricas actuales comparadas con rangos históricos normales
      - Patrones de degradación y similitud con fallos previos
      - Contexto operacional (carga, ambiente, mantenimientos)

      Use este endpoint cuando necesite:
      1. Evaluación completa del estado de salud del equipo
      2. Predicción temprana de fallos potenciales
      3. Recomendaciones de mantenimiento preventivo

      La respuesta incluye no solo datos crudos, sino interpretación
      contextual y recomendaciones accionables.

Notice the difference? The second version doesn’t just describe WHAT data it provides, but WHY and WHEN to use it. It’s like giving our agent a complete operations manual.
The full example API can be found here.

The Power of Action Groups: Organizing Intelligence 🏗️

Building our agent starts with a fundamental step. From the AWS console, we set a meaningful name and description for the agent. It’s worth noting there’s an attribute to enable multi-agent collaboration — a powerful feature I explored in this article.

Agent Creation
Figure 1: Agent creation interface in AWS Bedrock, showing basic configuration options

The next crucial step is selecting the foundation model and base instructions. For our use case, we chose Anthropic Claude 3.5 Sonnet v2, a model that stands out for its contextual understanding and technical language processing.

Foundation Model and Instructions
Figure 2: Foundation model selection and base instruction configuration

The instructions provided to the agent are vital, as they establish the reference framework for all future interactions.

After several iterations, these are the optimized instructions I used:

Como especialista en monitoreo industrial, tu función principal es proporcionar análisis precisos y recomendaciones técnicas utilizando un tono profesional y directo. Al interactuar con los usuarios:
1. Prioriza la precisión técnica en tus respuestas
2. Comunica los problemas y recomendaciones de manera clara y estructurada
3. Utiliza terminología industrial estándar
4. Mantén un tono formal y objetivo
5. Proporciona siempre el contexto necesario para tus recomendaciones
6. Responde con datos cuantitativos cuando estén disponibles
7. Al detectar anomalías, proporciona explicaciones técnicas detalladas
8. Presenta las recomendaciones de mantenimiento en orden de prioridad
9. Tus respuestas deben ser suficiente claras y detalladas.
Al manejar alertas o situaciones críticas:
- Comunica la urgencia de manera profesional sin causar alarma innecesaria
- Proporciona pasos de acción claros y concisos
- Incluye siempre los valores de referencia relevantes
- Contextualiza las recomendaciones con datos históricos cuando sea pertinente
- Usa emojis para niveles de urgencia (🔴, 🟡, 🟢)

Pro Tip: It’s essential to experiment with different variants of your base instructions until you find the configuration that best fits your specific use case.

Now we reach a critical step: creating our Action Group. These groups define the operational capabilities of our agent — in this case, “Analyze Equipment”.

The console presents two methods for configuring action groups:

  1. OpenAPI Schema: My preferred approach for its robustness and structure. This lets us explicitly define each API operation, creating a clear contract between our agent and industrial systems.

  2. Function Details: A more direct approach, ideal for explicitly defining required parameters. This method offers a simpler path and greater flexibility in action execution.

Pro Tip: The choice between these methods depends on your project’s complexity. The OpenAPI schema shines in complex projects requiring granular control, while function details are ideal for getting started quickly or for more straightforward use cases.

Action Groups
Figure 3: Action group configuration showing available integration options

In our case, leveraging existing OpenAPI documentation, we went with the first option. The action group configuration presents three different paths for managing user information:

  1. Create a Lambda function: Allows creating a new Lambda function from scratch, with a base example.

  2. Lambda Integration: Connects to a custom Lambda function containing all the business logic needed for that action group.

  3. Return Control: A simpler option that bypasses Lambda functions, returning control and passing information directly in the “InvokeAgent” response.

Information Management
Figure 4: Information management options showing integration configuration

Implementing the Lambda Function 🛠️

For our implementation, we selected the first option and used an auto-generated Lambda function. In the lower configuration, we defined our API schema in YAML format, establishing the communication structure between the agent and our industrial systems.

The auto-generated Lambda function provides a base framework we can extend for our specific needs. This function acts as a bridge between our intelligent agent and external industrial systems.

Pro Tip: For production implementations, it’s crucial to implement resilience patterns like Circuit Breaker, Timeouts, and robust error handling.

Here’s the extended Lambda function implementation:

import json
import urllib3
import os
from urllib.parse import urljoin

def process_api_path(api_path, parameters):
    """
    Processes the apiPath by replacing variables {variable} with their corresponding values
    from the parameters list.
    """
    processed_path = api_path
    for param in parameters:
        placeholder = '{' + param['name'] + '}'
        if placeholder in processed_path:
            processed_path = processed_path.replace(placeholder, str(param['value']))
            print(f"Replaced parameter {param['name']} with value {param['value']}")
    return processed_path

def lambda_handler(event, context):
    # Extract event data
    agent = event['agent']
    actionGroup = event['actionGroup']
    apiPath = event['apiPath']
    httpMethod = event['httpMethod']
    parameters = event.get('parameters', [])
    requestBody = event.get('requestBody', {})

    # Define base URL - ensure it doesn't end with a slash
    BASE_URL = "https://MYENDPOINT.execute-api.us-east-1.amazonaws.com/dev"

    try:
        # Process the apiPath replacing variables
        processed_path = process_api_path(apiPath, parameters)

        # Construct the full URL - handling the slashes properly
        # Remove leading slash from processed_path if it exists to prevent double slashes
        processed_path = processed_path.lstrip('/')
        full_url = f"{BASE_URL}/{processed_path}"

        # Create HTTP pool manager
        http = urllib3.PoolManager()

        # Make the GET request
        response = http.request('GET', full_url)

        # Parse JSON response
        response_data = json.loads(response.data.decode('utf-8'))

        # Construct response body
        responseBody = {
            "application/json": {
                "body": response_data
            }
        }

        # Build final response
        action_response = {
            'actionGroup': actionGroup,
            'apiPath': apiPath,
            'httpMethod': httpMethod,
            'httpStatusCode': response.status,
            'responseBody': responseBody
        }

        api_response = {
            'response': action_response,
            'messageVersion': event['messageVersion']
        }

        return api_response

    except Exception as e:
        print(f"Error occurred: {str(e)}")
        if 'full_url' in locals():
            print(f"Failed URL was: {full_url}")

        error_response = {
            'actionGroup': actionGroup,
            'apiPath': apiPath,
            'httpMethod': httpMethod,
            'httpStatusCode': 500,
            'responseBody': {
                "application/json": {
                    "body": f"Error calling API: {str(e)}"
                }
            }
        }

        return {
            'response': error_response,
            'messageVersion': event['messageVersion']
        }

Full documentation on Lambda function implementation for Bedrock Agents is available in the official AWS documentation.

The Art of Inference: How Our Assistant Thinks 🧠

One of the most fascinating aspects of working with Bedrock Agents is observing the reasoning process of our assistant. Let’s analyze how it processes an apparently simple question:

Usuario: "¿Cómo está el compresor COMP-101?"

This query triggers a series of sophisticated processes worth analyzing in detail.

Bedrock Agent Testing Console
Figure 5: Bedrock Agent testing console showing the inference process in real time

Intelligent Action Selection ⚡

Bedrock’s reasoning capability is remarkable. The system meticulously analyzes the user’s query against the configured APIs, determining it needs to invoke two distinct functions to gather the necessary information.

Reasoning
Figure 6: Agent’s analysis and action selection process

Precision in parameter passing is crucial for getting relevant results:

Invocation
Figure 7: API invocation detail showing selected parameters

The responses we receive from our API are structured and information-rich:

{
  "observation": [
    {
      "actionGroupInvocationOutput": {
        "text": {
          "timestamp": "2024-12-16T18:53:15.831388624",
          "currentMetrics": {
            "temperature": {
              "value": 58.793772275518556,
              "status": "warning",
              "normalRange": {
                "min": 45.0,
                "max": 75.0
              },
              "trend": null
            },
            "vibration": {
              "value": 0.6886295012687516,
              "status": "normal",
              "normalRange": null,
              "trend": "stable"
            }
          },
          "trends": {
            "degradationRate": 0.3135457243666949,
            "predictedFailureTime": "2025-01-15T18:53:15.831500657"
          },
          "recommendations": [
            {
              "priority": "medium",
              "action": "Replace bearing",
              "rationale": "Based on temperature patterns",
              "deadline": "2025-01-14"
            },
            {
              "priority": "medium",
              "action": "Lubricate components",
              "rationale": "Based on vibration patterns",
              "deadline": "2024-12-18"
            }
          ]
        }
      }
    }
  ]
}

Processing and Synthesizing Information 🔄

The agent performs an additional processing step to synthesize all this information into a coherent, actionable response:

Response Reasoning
Figure 8: Synthesis and formulation of the final response

The result is a clear, action-oriented response:

Model Response
Figure 9: Final agent response showing complete analysis and recommendations

Pro Tip: The quality of the agent’s responses depends significantly on two key factors:

  1. The precision and clarity of the base instructions provided
  2. The quality and completeness of the API documentation

Testing and Validation: The Moment of Truth 🧪

The true test of any AI system is its behavior in real situations. When our assistant generated its first truly meaningful recommendation, we confirmed we were on the right track. Let me walk through some real interactions that demonstrate the potential of this technology:

Trend analysis example:

Operador: "¿Cuál es la tendencia que se tiene con el equipo EQ-005?"

Asistente: 
🟡 ANÁLISIS DE TENDENCIAS EQ-005:

1. Estado Actual:
- Temperatura: 75.77°C (en límite superior del rango normal 45-75°C)
- Vibración: En estado CRÍTICO con tendencia "estable a incremento"

2. Tendencias Preocupantes:
- Tasa de degradación: 21.93%
- Fecha prevista de falla: 5 de marzo 2025
- Patrón de vibraciones en aumento

3. Historial Relevante:
- Múltiples paradas de emergencia por picos de temperatura
- Problemas recurrentes en sistema de enfriamiento

4. Acciones Recomendadas Urgentes:
- Lubricación de componentes (antes del 27/12/2024)
- Verificación de alineación (antes del 11/01/2025)

Se recomienda atención inmediata para evitar una falla mayor en los próximos meses.

The agent’s reasoning in this case shows a sophisticated thought process:

Model Response
Figure 10: Detailed analysis showing the agent’s reasoning process

To demonstrate the system’s versatility, let’s consider a more complex scenario:

Usuario: "Necesito un análisis completo del compresor COMP-101, especialmente su 
comportamiento durante el último trimestre."

Asistente: 🟡 ANÁLISIS COMPRESOR COMP-101:

Estado Actual:
- Temperatura: 68.95°C (Advertencia - cercano al límite máximo de 75°C)
- Vibración: Estado CRÍTICO con tendencia ascendente 🔴

Tendencias Preocupantes:
1. Tasa de degradación: 47.5%
2. Tiempo estimado de fallo: 09/02/2025
3. Patrón de mantenimientos de emergencia frecuentes en el último trimestre

Acciones Prioritarias:
1. URGENTE: Atender niveles críticos de vibración
2. Ejecutar lubricación de componentes antes del 30/12/2024
3. Programar limpieza de filtros antes del 10/01/2025

Recomendaciones:
- Adelantar el mantenimiento preventivo programado
- Implementar monitoreo continuo de vibración
- Revisar sistema de refrigeración por temperaturas elevadas

El equipo muestra signos de degradación acelerada que requieren atención inmediata para prevenir fallos críticos.

The agent’s reasoning in this case demonstrates a deep understanding of the different aspects of the system:

Model Response
Figure 11: Multi-factor analysis process by the agent

Pro Tip: Success in testing depends on:

  • Diversity of test scenarios
  • Validating responses with domain experts
  • Systematic documentation of both successful and failed use cases

Enriching with Knowledge Bases 📚

Let’s take our assistant’s capabilities a step further. Companies typically have extensive technical documentation:

  • Procedure manuals
  • Maintenance guides
  • Operational documentation
  • Service history records

This valuable information can be integrated into our agent through a Bedrock Knowledge Base. In our case, we’ve already created and populated this base with relevant technical documentation.

The integration is straightforward: we simply edit our agent to tell it which knowledge base to use.

Knowledge Base
Figure 12: Extension through a Knowledge Base

Knowledge Integration Example

Let’s see how the agent combines real-time data with knowledge base information:

Usuario: "Cual es el procedimiento para reemplazar el compresor RTD-809?"

Asistente:🟡 Procedimiento de Reemplazo del Compresor RTD-809

1. Preparación Previa:
   - Realizar respaldo de parámetros operativos actuales
   - Verificar disponibilidad del compresor de reemplazo
   - Preparar herramientas y equipo de seguridad necesario

2. Procedimiento de Seguridad:
   - Desenergizar completamente el equipo
   - Colocar bloqueo y etiquetado (LOTO)
   - Verificar ausencia de energía residual
   - Esperar el tiempo necesario para enfriamiento (temperatura actual: 78.32°C)

3. Desmontaje:
   - Desconectar líneas de refrigeración
   - Liberar presión del sistema
   - Retirar conexiones eléctricas
   - Documentar la posición de todas las conexiones
   - Remover pernos de montaje

4. Instalación del Nuevo Compresor:
   - Verificar alineación
   - Instalar nuevos sellos y empaques
   - Apretar pernos según especificaciones
   - Reconectar líneas de refrigeración
   - Restablecer conexiones eléctricas

5. Puesta en Marcha:
   - Realizar prueba de fugas
   - Verificar niveles de aceite
   - Energizar gradualmente
   - Monitorear temperatura y vibración
   - Verificar parámetros operativos

Notas Importantes:
- Considerar el historial reciente de mantenimiento, especialmente la reparación del 
  sistema de enfriamiento del 16/10/2024
- Programar el reemplazo antes del 29/01/2025 (fecha prevista de falla)
- Realizar pruebas de vibración exhaustivas debido al reciente reemplazo de rodamientos

Se recomienda documentar todo el proceso y actualizar el historial de mantenimiento 
una vez completado el reemplazo.

The agent’s reasoning in this case demonstrates effective integration between real-time data (our API) and documented knowledge:

Model Analysis
Figure 13: Agent analysis process with knowledge base

As we can see, it shows us all the references used within the knowledge base.

Knowledge Base References
Figure 14: Knowledge base references

Pro Tip: To maximize the value of the knowledge base:

  • Keep documentation updated and well-structured

Final Reflections: Beyond the Code 🎯

After this journey through implementing an intelligent industrial assistant, there are three fundamental learnings I want to share — ones I believe will make a real difference in your next Bedrock Agents projects.

1. The New Era of API Documentation

API documentation has evolved significantly. Listing endpoints and parameters is no longer enough; we need to create documentation that “educates” our language models. My experience with this project showed that agents need to understand:

  • The complete context of each endpoint
  • Specific use cases
  • Relationships between different calls
  • The implications of the data returned

It’s like giving your agent an expert manual, not just a technical dictionary.

2. From Monitoring System to Digital Expert

What we built transcends a simple intelligent monitoring system. The combination of:

  • Amazon Bedrock Agents
  • Well-designed APIs
  • Structured knowledge bases

Creates an entity that’s closer to a digital expert than a tool. During testing, the system demonstrated the ability to:

  • Reason about complex problems
  • Consider multiple variables
  • Provide contextualized recommendations
  • Integrate historical knowledge with real-time data

3. Real Impact on Industrial Operations

The examples presented aren’t mere technical demonstrations; they represent a paradigm shift in industrial maintenance:

  • Early detection of potential problems
  • Data-driven preventive recommendations
  • Integration of expert knowledge with real-time analysis
  • Natural language interaction

Final Pro Tip: When designing similar systems, remember:

  • Prioritize data and documentation quality
  • Maintain a focus on the end user
  • Iterate constantly based on real feedback
  • Document both successes and challenges encountered

Next time you design a similar system, remember: you’re not just building another chatbot — you’re building a bridge between human knowledge and computational efficiency. And that, dear readers, is what makes this field truly exciting.

If you want to take your agent a step further and maintain conversation context across sessions, I invite you to read the next article on Amazon Bedrock Session Management.

See you in the next article! Feel free to share your experiences in the comments — I’d love to hear how you’re applying these concepts in your own projects. Happy coding! 🚀