A practical guide for engineers, cybersecurity teams, and DevSecOps leaders deciding whether to run large AI models locally, in private cloud, or through secure enterprise AI platforms.

The Real AI Question Is Not Only “How Fast?” It Is “How Secure, Sustainable, and Useful?”

Many engineering and security teams are asking the same question:

Do we need bigger GPUs to use AI safely and effectively, or do we need a better architecture?

The answer is usually both, but not in equal measure.

AI performance is no longer only about model quality. It is also about infrastructure, data security, operational cost, governance, and ownership. Engineers want fast inference. Cybersecurity teams want data control. DevSecOps teams want repeatable deployment pipelines. Business leaders want value without uncontrolled spending.

A powerful GPU can make an AI model respond faster, support larger prompts, and serve more users. But a GPU alone does not make an AI system secure. A local 70B model can still expose sensitive data if access control, logging, patching, prompt filtering, and retention policies are weak.

At the same time, a well-designed cloud or enterprise AI platform can be secure if the organization applies the right controls: data classification, contractual review, network isolation, identity integration, monitoring, and audit logging.

The goal is not to make AI adoption harder. The goal is to make it safer, more realistic, and more useful.

For many organizations, the future will not be “local AI versus cloud AI.” It will be a controlled mix of local inference, private cloud, managed AI platforms, secure APIs, retrieval-augmented generation, human review, and strong governance.

1. Why GPUs Matter for AI Performance

A GPU is designed to perform many mathematical operations in parallel. That makes it well suited for AI workloads, especially deep learning models that depend heavily on matrix multiplication.

CPUs are excellent at general-purpose computing. They manage operating systems, application logic, orchestration, networking, I/O, and many sequential tasks. GPUs, by contrast, are built for parallel computation. AI models benefit from this because neural networks apply similar operations across very large tensors.

For large language models, GPU performance affects four practical areas.

Inference speed: how quickly the model generates output.

Concurrency: how well the platform handles multiple users or requests.

Context size: how much prompt, document, code, or retrieval context the model can process effectively.

Model choice: whether the platform can run smaller models, 30B-class models, 70B-class models, or larger architectures.

This is why modern AI infrastructure is designed around high-bandwidth memory, GPU interconnects, tensor acceleration, and specialized data center systems.

But performance is not free. More GPU capacity usually means more power, cooling, rack planning, lifecycle management, monitoring, and operational discipline.

2. GPU, VRAM, CPU, RAM, Storage, and Network: How They Work Together

AI performance depends on the whole system, not only the GPU model.

GPU: The Parallel Compute Engine

The GPU performs the heavy mathematical operations used for inference, embeddings, attention calculations, and model execution. For training and fine-tuning, the GPU workload is heavier because the system also processes gradients, optimizer states, and large batches of data.

Most enterprise teams do not start by training frontier models. They usually start with inference, RAG, prompt engineering, model evaluation, and limited fine-tuning. That still requires careful GPU planning, but it is different from building a model from scratch.

VRAM: Where the Model and Working Context Live

VRAM is often the real constraint.

The model must fit into GPU memory along with runtime overhead, KV cache, prompt context, and sometimes additional components. If the model does not fit into VRAM, the system may offload work to CPU RAM. That can work, but performance usually drops because system RAM is slower for GPU-heavy inference.

A simple way to think about it:

GPU compute determines how fast the model can work. VRAM determines how much of the model and conversation it can hold while working.

Quantization can reduce memory requirements. A 70B model loaded in lower precision may fit into far less VRAM than the same model in FP16, but quality, latency, compatibility, and accuracy must be tested against real use cases.

CPU: The Coordinator

The CPU still matters. It handles orchestration, tokenization, preprocessing, API services, security agents, logging, storage access, request routing, authentication, and integration with enterprise systems.

A weak CPU paired with a strong GPU can create bottlenecks, especially when AI services are wrapped inside APIs, Kubernetes workloads, vector databases, authentication layers, and observability pipelines.

RAM, Storage, and Network: The Supporting Infrastructure

System RAM supports the operating system, application services, model loading, CPU offloading, document processing, vector database operations, and caching.

Storage matters because model files, embeddings, logs, datasets, and audit trails can grow quickly.

Network matters because AI platforms often depend on retrieval systems, identity providers, API gateways, SIEM pipelines, and distributed GPU infrastructure. In larger environments, data movement can become a major bottleneck.

High-end AI systems are no longer just “servers with GPUs.” They are integrated platforms combining GPUs, high-bandwidth memory, CPUs, storage, networking, interconnects, and management software.

3. How AI Is Reshaping the GPU Industry

AI has changed what buyers expect from GPU vendors.

For years, many people associated GPUs with graphics, gaming, video rendering, and scientific computing. AI changed the center of gravity. The most valuable GPU systems are now judged by memory capacity, memory bandwidth, tensor performance, interconnect speed, power efficiency, software ecosystem, availability, and data center integration.

Three shifts matter.

VRAM Is Strategic

For AI workloads, memory capacity can be as important as raw compute. Larger models need more VRAM. Longer context windows need more memory. Multi-user inference needs additional headroom.

This pushes the industry toward larger high-bandwidth memory configurations and specialized AI systems.

Interconnects Matter More

When a model does not fit on one GPU, it may need to be split across multiple GPUs. That requires fast interconnects such as NVLink or other high-performance fabrics. Without strong interconnects, multiple GPUs may not deliver the expected performance because too much time is spent moving data between devices.

Power and Cooling Are Now Security and Resilience Concerns

A single high-end AI system can consume significant power. A rack of AI servers can create facility-level power and cooling demands. This affects resilience, capacity planning, cost, and availability.

This does not mean AI is a problem. It means AI is becoming real infrastructure. Real infrastructure needs engineering discipline.

4. Should You Buy the Best GPUs to Run 30B or 70B Models Locally?

The honest answer is: not always.

Buying high-end GPUs feels like control. You own the hardware. You keep data local. You avoid sending sensitive prompts to external platforms. You can tune the environment for your own needs.

But local AI also creates responsibilities.

You must manage hardware, drivers, CUDA compatibility, model serving, patching, physical security, backups, user access, monitoring, logging, cooling, power, lifecycle replacement, and incident response.

For a lab, this may be manageable. For production, it becomes a platform.

When Local 30B or 70B Models Make Sense

Local models are worth considering when:

• sensitive data cannot leave your controlled environment;
• offline capability is required;
• engineers can operate GPU systems safely;
• open-weight model experimentation is important;
• internal assistants need stronger control over logging, retention, and model behavior;
• security or compliance requirements make external API use difficult.

For many teams, a 30B-class model can be a practical middle ground. It may support code assistance, documentation search, security triage summaries, and technical Q&A when paired with good retrieval and guardrails.

A 70B-class model can improve reasoning and language quality in some cases, but it increases infrastructure complexity. It may require multiple GPUs, quantization, or enterprise-grade systems depending on performance targets.

When Local 70B May Be Overkill

A local 70B model may not be the best first investment when:

• the use case is simple summarization or classification;
• the team has not built data governance controls;
• workload volume is occasional;
• a secure enterprise AI platform meets the requirement;
• the organization lacks GPU operations experience.

In security work, the quality of the data pipeline often matters more than the size of the model. A smaller model with strong retrieval and approved internal context can outperform a larger model guessing from general knowledge.

5. The Cost-to-Overhead Ratio Many AI Projects Underestimate

The cost of local AI is not just the GPU purchase price.

A practical cost model should include:

• hardware acquisition;
• power and cooling;
• rack space and facility planning;
• driver, firmware, and operating system maintenance;
• model serving software;
• monitoring and logging;
• identity integration;
• vulnerability management;
• backups and recovery;
• engineering support;
• lifecycle replacement.

A useful internal formula is:

AI infrastructure value = useful secure output ÷ total cost of ownership

The numerator is not “tokens generated.” It is useful secure output: faster investigations, better code review support, improved documentation search, safer customer support workflows, reduced manual triage, or faster compliance evidence preparation.

The denominator is not only the invoice. It includes engineering time, operational burden, security risk, and future maintenance.

For many organizations, the best answer is hybrid:

• use local or private infrastructure for sensitive data;
• use managed enterprise AI platforms for approved productivity workflows;
• use smaller local models for classification, redaction, routing, and offline tasks;
• use larger models only where quality improvement justifies the cost.

6. Data Security Is the Real Trust Boundary

AI systems handle prompts, documents, source code, tickets, logs, customer records, incident reports, vulnerability data, legal content, and business plans.

These are not harmless strings of text. They may contain secrets, personal data, intellectual property, credentials, system architecture, or regulated information.

The main security question is not only:

Where is the model running?

The better question is:

What data enters the AI system, where does it go, who can access it, how long is it retained, and how can we prove it?

This is where GPU strategy becomes security architecture.

A local GPU can reduce third-party exposure, but it does not automatically solve data security. You still need identity controls, data classification, prompt and output handling rules, logging, retention controls, vulnerability management, and incident response.

A cloud AI platform can be acceptable, but only if the organization understands data handling, retention, training use, access controls, encryption, regional processing, administrative visibility, and audit logging.

7. Main AI Data Security Risks Engineers Should Design For

Sensitive Data Leakage

Users may paste secrets, API keys, customer information, source code, architecture diagrams, or incident details into AI tools.

The fix starts with data classification. Not every AI platform should be allowed to process every data type.

Prompt Injection

Prompt injection matters in RAG systems, browser assistants, email copilots, ticket triage tools, and AI agents. A malicious instruction hidden inside a document can attempt to override system instructions or manipulate model behavior.

The fix is layered control: content filtering, instruction hierarchy, retrieval isolation, output validation, least-privilege tools, and human approval for sensitive actions.

Insecure Output Handling

AI output should not be blindly executed.

This matters in DevSecOps pipelines where AI may generate scripts, Kubernetes manifests, Terraform, SQL queries, firewall rules, or CI/CD changes.

Treat AI output as untrusted until it is reviewed, tested, scanned, and approved.

Model and Supply Chain Risk

Open-weight models, containers, libraries, tokenizers, plugins, and inference servers are part of the software supply chain.

Use trusted sources, hash verification, vulnerability scanning, signed containers where possible, controlled registries, dependency review, and repeatable deployment pipelines.

Excessive Agency

AI agents become risky when they can take action without boundaries. An assistant that reads documentation is lower risk than an assistant that modifies IAM policies, deploys infrastructure, opens firewall rules, or closes security alerts.

Use scoped permissions, approval workflows, transaction logging, rate limits, and separation between recommendation and execution.

8. Comparing AI Platform Options

Public AI SaaS

Public AI platforms are easy to adopt and often provide excellent model quality. They can be suitable for general writing, brainstorming, non-sensitive coding help, and productivity tasks when allowed by policy.

The concern is data handling. Before using public AI platforms for enterprise content, verify retention terms, training usage, encryption, enterprise controls, identity integration, regional options, and audit logs.

Enterprise AI Platforms

Enterprise AI platforms usually provide stronger controls such as SSO, administrative policy, audit features, contractual protections, and sometimes private connectivity.

Do not assume “enterprise” means safe for all data. Map the platform against internal data classification and compliance obligations.

Private Cloud AI

Private cloud AI runs models inside an organization’s cloud environment. This can provide stronger IAM integration, network control, encryption, logging, and workload isolation.

This model works well for organizations already mature in cloud security and DevSecOps.

On-Premises or Local AI

Local AI provides maximum control when implemented correctly. It can be valuable for sensitive workloads, regulated environments, research, legal review, security operations, or confidential engineering.

The trade-off is maintenance. A local model with poor governance can be less secure than a well-managed enterprise cloud platform.

Hybrid AI

Hybrid AI is likely the most practical model for many organizations.

Use local AI for sensitive data. Use enterprise AI platforms for approved workflows. Use smaller models for classification and redaction. Use larger models where quality improvement justifies the cost. Use RAG to ground responses in approved internal sources.

9. How to Improve Data Security Without Slowing AI Adoption

Blocking AI usually creates shadow IT. The better approach is to provide secure, approved paths.

Classify AI Use Cases by Data Sensitivity

Create simple categories:

• public data;
• internal business data;
• confidential engineering data;
• customer or regulated data;
• security-sensitive data;
• secrets and credentials.

Then define which AI platforms may process each category.

Build an AI Security Gateway

An AI security gateway can enforce policy before prompts reach a model. It can inspect prompts, detect secrets, redact sensitive content, apply allow or deny rules, log usage, and route requests to approved platforms.

This is useful in hybrid environments where teams use multiple models and providers.

Use RAG Carefully

RAG lets the model answer using approved internal documents rather than relying only on model memory.

For security and DevSecOps teams, RAG can connect AI to runbooks, architecture standards, control mappings, incident response procedures, secure coding guidelines, and approved knowledge bases.

The risk is authorization. A user should not retrieve documents they are not allowed to access.

Keep Humans in the Approval Loop

AI can recommend. Humans should approve high-impact actions.

This matters for production deployments, firewall changes, IAM changes, incident closure, vulnerability risk acceptance, customer-facing responses, and compliance evidence submission.

Log AI Activity for Audit and Detection

Security teams should be able to answer:

• who used the AI system;
• what platform was used;
• what data category was involved;
• whether sensitive data was detected;
• what documents were retrieved;
• what action was recommended;
• whether any action was executed;
• which human approved it.

These logs should feed SIEM or security analytics where appropriate.

Secure the AI Software Supply Chain

Treat models, prompts, plugins, vector databases, containers, and inference APIs as production components.

Apply source verification, container scanning, dependency scanning, secrets scanning, infrastructure-as-code review, access control testing, model version tracking, change management, and rollback planning.

10. Decision Framework: Local GPU or Secure AI Platform?

Before buying GPUs or signing a platform contract, ask these questions.

Workload

• What model size do we actually need?
• Would a smaller model with RAG work?
• How many concurrent users do we need to support?
• What latency is acceptable?
• Do we need offline capability?
• Are we fine-tuning, or only running inference?

Data Security

• What data will users submit?
• Does the platform retain prompts?
• Can submitted data be used for training?
• Where is data processed?
• Can we enforce SSO and role-based access?
• Are audit logs available?
• Can we prevent secrets from being submitted?
• Can we integrate with DLP or SIEM?

Operations

• Who owns patching?
• Who monitors GPU utilization?
• Who handles failed jobs?
• Who updates models?
• Who validates model output?
• Who pays for power and cooling?
• Who responds if the AI service is abused?

Cost

• Is usage steady or occasional?
• Would cloud GPU rental be cheaper during experimentation?
• Can GPU capacity be shared across teams?
• What is the expected hardware lifecycle?
• What is the cost of downtime?
• What is the cost of a data leak?

These questions prevent expensive hardware purchases for unclear outcomes.

11. Balanced Recommendation for 30B and 70B Local Models

Start with use cases, not hardware.

Classify the data.

Test smaller models first.

Add RAG with approved internal sources.

Measure accuracy, latency, and user value.

Introduce 30B-class models where quality requires it.

Use 70B-class models only where the improvement is proven.

Consider local or private deployment for sensitive data.

Use enterprise cloud AI where contractual and technical controls are acceptable.

Invest in logging, policy, and governance early.

A 70B local model can be powerful. It can also be expensive and operationally demanding. The right question is not “Can we run it?” The better question is:

Can we run it securely, reliably, and usefully enough to justify the overhead?

12. What This Means for Cybersecurity and DevSecOps Teams

AI should be treated as a production capability, not a side tool.

For cybersecurity teams, AI can support alert summarization, phishing analysis, incident timeline drafting, vulnerability explanation, control mapping, and analyst assistance. But it must not become an uncontrolled channel for sensitive logs, credentials, or customer data.

For DevSecOps teams, AI can help review code, generate tests, explain build failures, write infrastructure templates, and improve documentation. But AI-generated output should pass through the same quality gates as human-generated code.

For engineers, AI can reduce repetitive work and improve learning speed. But the best results come when AI is connected to accurate internal context and protected by sensible guardrails.

The mature path is not fear. It is engineering.

Security Review Checklist

Before approving an AI platform or local GPU deployment, ask:

• Have we defined approved and prohibited data types?
• Do users authenticate through enterprise identity?
• Can we enforce role-based access?
• Are prompts and outputs logged appropriately?
• Can sensitive data be detected or redacted?
• Is model access separated by environment?
• Are RAG documents permission-aware?
• Are AI-generated actions reviewed before execution?
• Are containers, models, and dependencies scanned?
• Can we trace model version, prompt template, and retrieval source?
• Is there an incident response process for AI misuse?
• Do we know the total cost of ownership?
• Can we explain residual risk to leadership?

This checklist is not meant to slow adoption. It is meant to make adoption sustainable.

Practical Takeaway

GPUs are shaping the AI industry because AI performance depends heavily on parallel compute, VRAM, memory bandwidth, and interconnects.

But successful enterprise AI is not just a GPU decision. It is a security architecture decision.

A strong GPU can help a 30B or 70B model run faster. Strong governance helps ensure the model is safe to use. Strong DevSecOps practices make deployment repeatable. Strong monitoring makes the platform accountable. Strong data controls make AI acceptable for real business use.

The practical strategy is to match model size, platform choice, and GPU investment to the sensitivity of the data and the value of the workload.

Use local AI where control matters most.

Use enterprise AI platforms where managed security and scale make sense.

Use RAG to ground responses.

Use human review for high-impact decisions.

Use logging and policy to keep trust measurable.

Final Thought

AI is not asking engineering and security teams to choose between innovation and control. It is asking us to design better systems.

The GPU gives AI speed. VRAM gives it room to work. CPU and RAM keep the platform coordinated. Security gives the whole system permission to operate in the real world.

The organizations that succeed will not simply buy the biggest GPUs. They will build AI environments where performance, data protection, cost, and operational trust work together.

That is the real AI advantage: not just faster answers, but safer and more reliable decisions.

Eight years into a DevOps career and I’d never built a personal site.

I’d architected and helped build out dozens of IaaS environments at work. I’d reviewed enough PRs touching CI/CD pipelines to have opinions about every action in the GitHub Marketplace. But Azure Static Web Apps, Cosmos DB, the whole serverless front-of-house stack? Those I knew from architecture diagrams and not from deploying one myself, which is a different relationship to a piece of software than people sometimes assume. For the half hour someone might spend googling my name, all I had was a LinkedIn page and a generic Outlook email pointing nowhere. The shoemaker’s children, etc.

The Cloud Resume Challenge was the excuse. The 16 official steps got me a working portfolio at rlwilliamson.dev. The next several rounds of “well, just one more thing” got me everything else: a blog, custom subpages, an A-grade security posture, a handful of easter eggs, and a README that is, charitably, ambitious in scope. This post is the build log.

What the Cloud Resume Challenge is

Forrest Brazeal‘s Cloud Resume Challenge is a 16-step gauntlet that walks you from “I want to learn the cloud” to a real, running, hand-built portfolio site backed by a serverless visitor counter and deployed via CI/CD. The official path covers a cloud cert, an HTML/CSS resume, a static website with HTTPS and a custom DNS, some JavaScript for interactivity, a database, an API, a server-side language (typically Python), tests, infrastructure as code, source control, frontend and backend CI/CD, and a blog post to wrap it up.

Thousands of people have done it. The format is opinionated by design. Forrest wants you to fight through the same problems professional cloud engineers fight, not to copy-paste your way to a working site. What makes mine worth reading isn’t the 16 steps. It’s the part where I refused to stop, and a project that should have ended in two weekends went on for a couple of months.

Why I did it

A certification tells you I’ve passed an exam. Live infrastructure tells you I can actually run things. The challenge sits at the spot where those two converge. There’s a real running counter and a real deploy pipeline somebody can poke at, plus the version-controlled source for all of it on GitHub. Either side standing alone is easy to fake. Both at once, hard.

I also wanted to spend some personal time in the parts of the Azure stack one layer below my day job. I’m a DevOps engineer where I help build and execute Azure CI/CD and DevSecOps practices for a tax portfolio. I help build out the platform developers work on, which means I spend most of my time in pipelines and IaaS, not in the application-layer Azure services. Cosmos DB doesn’t show up at work. Neither does the Functions runtime, in any way I’d consider hands-on. The challenge gave me a chance to run them both in anger, on someone else’s credit card. (My credit card. The “someone else” was the past version of me who said “yeah, sure, $12 for the domain.”)

Last reason: it sounded fun, in the slightly-unhinged way that “let me wire all this together with my own hands instead of clicking through a portal” is fun for people who like this kind of work. If that doesn’t immediately make sense to you, you are not the target audience for this post and I hope you have a nice day.

The architecture

The whole thing is one Azure Static Web App. A browser hits rlwilliamson.dev. Azure Front Door (which fronts every SWA whether you ask for it or not) terminates TLS and routes the request. SWA serves the static HTML directly for the resume and the custom subpages. For /blog/*, it serves a pre-built Astro static site that gets compiled into the same frontend/ directory at CI time. Astro’s static output is just HTML, so there’s no separate hosting and no runtime cost. For /api/*, SWA’s built-in proxy hands the request to a managed Python Function on the same SWA deployment, which reads or writes a single document in Cosmos DB serverless.

There’s no separate API hostname, no CORS configuration, no cookie-domain weirdness. The /api proxy is one of the cleanest “API + frontend on the same domain” patterns I’ve used in Azure, in the strict sense that I never had to write a single redirect rule to make it work.

Everything stateful lives in Cosmos DB serverless mode. That choice was the difference between “fractions of a cent per month” and ~$24 minimum monthly, and I’ll come back to it.

How it works

The frontend

Single-file HTML. Inline CSS. Inline JavaScript. No bundler, no framework, no build step for the resume page itself. One network request to the document, then a few for the photo, the font, and the favicon.

I’d push back hard on this for a team project. Inline CSS doesn’t scale, build steps exist for good reasons. For a single-page personal site it’s a feature: no FOUC, no chunked-asset cache invalidation problems, no source-map drift. The whole page can be cracked open in DevTools and understood in five minutes, which is exactly what you want when the audience is engineers.

The visual personality comes from a Linux-style boot animation that prints kernel log lines, then a typing terminal hero that cycles through whoami, cat current-role.txt, ls -la skills/, history | grep migration, and contact --short. The avatar opens a lightbox on click. Each role expands from a one-line summary to full bullets when you click it. There’s a built-in print-to-PDF mode that produces a clean traditional resume via the browser’s print dialog. Dark mode is the default. Light mode is one toggle away and persists in localStorage.

I’ll save the easter eggs for later in this post. There are a few. I’m not going to say what or where.

The API

The visitor counter is a Python 3.11 Azure Function on the V2 programming model, exposed at /api/counter. It does exactly one thing: increments a single document in Cosmos DB and returns the new count. The interesting part is the self-healing init:

# api/db.py (abbreviated)
from azure.cosmos.exceptions import CosmosResourceNotFoundError

def get_or_create_counter(container):
    try:
        return container.read_item(item="counter", partition_key="counter")
    except CosmosResourceNotFoundError:
        return container.create_item({"id": "counter", "count": 0})

Cold start, fresh container, missing document, and db.py creates it on the fly. You never have to seed the database manually. This is also the pattern I use for tests: spin up a mock container, the function creates its own initial state, no fixtures.

Tests live in api/tests/ and run as a pytest job in CI:

cd api
pip install -r requirements-dev.txt
pytest -v

The Cosmos client is mocked, so the tests don’t need a live connection and don’t accidentally rack up RUs. The CI workflow gates the deploy job on needs: test, so a red pytest run blocks production. This is a 5-line change from the default SWA workflow and it’s worth its weight.

The infrastructure (Bicep, not Terraform)

The official challenge mentions Terraform as the default IaC choice. I went with Bicep, on purpose, for three reasons:

1. Azure-native. Bicep is a first-party Microsoft DSL. It speaks Azure resource types fluently without a provider layer in between, and new Azure resource types light up in Bicep faster than they do in the Terraform provider.
2. No state file. Bicep deployments use Azure Resource Manager’s own deployment history as the source of truth. There’s no terraform.tfstate to keep in a storage account with a lock, no state corruption to recover from. ARM tracks every deployment for you.
3. what-if. az deployment group what-if is the cleanest “show me what’s going to change” preview I’ve used. It’s not perfect (drift detection on some resource types is fuzzy), but for a small project it’s plenty.

Here’s the Cosmos resource definition:

resource cosmosAccount 'Microsoft.DocumentDB/databaseAccounts@2024-05-15' = {
  name: cosmosAccountName
  location: location
  kind: 'GlobalDocumentDB'
  properties: {
    databaseAccountOfferType: 'Standard'
    capabilities: [
      { name: 'EnableServerless' }
    ]
    locations: [
      { locationName: location }
    ]
    consistencyPolicy: {
      defaultConsistencyLevel: 'Session'
    }
  }
}

That EnableServerless capability is the entire billing-mode story in one line. Get it right and you pay rounding-error dollars per month. Get it wrong and you pay the provisioned-throughput minimum. More on that in the war stories.

Deploying changes:

az deployment group what-if 
  --resource-group rg-azure-resume 
  --template-file infra/main.bicep 
  --parameters infra/main.bicepparam

az deployment group create 
  --resource-group rg-azure-resume 
  --template-file infra/main.bicep 
  --parameters infra/main.bicepparam

The Static Web App itself is not in Bicep. SWA’s GitHub integration is built around a workflow file that gets created when you connect a repo through the Azure portal, and trying to manage SWA as a Bicep resource while ALSO managing the workflow file from GitHub is a recipe for split-brain. The community pattern is “Bicep for the backend (Cosmos), portal-managed for the SWA + workflow.” I followed it.

CI/CD

The GitHub Actions workflow has three jobs:

1. test. Checks out, installs requirements-dev.txt, runs pytest -v. About 30 seconds.
2. build_and_deploy_job runs only if test passed. Checks out, gets an OIDC token, sets up Node 22, runs npm ci and npm run build in blog/, copies blog/dist/* into frontend/blog/, then runs the standard Azure/static-web-apps-deploy@v1 action which ships frontend/ plus the Python Function. About 2 minutes.
3. close_pull_request_job runs when a PR closes. Uses OIDC auth (same pattern as the build job) to call the SWA deploy action with action: close, which deletes the preview environment.

The Astro build steps and the OIDC-on-close-PR setup are both non-default. Without the build steps you can’t have a static blog co-located with the resume. Without OIDC on the close job, preview environments accumulate until you hit the 10-environment Free tier cap and your next PR build fails.

The full daily loop, from terminal:

git checkout -b feat/something-new
# ... edit, test locally ...
git commit -m "feat: ..."
git push -u origin feat/something-new

gh pr create --base main --title "..." --body "..."
gh pr checks --watch && gh pr merge --squash --delete-branch

git checkout main && git pull origin main && git fetch --prune

Everything from “push” to “merged and live in production” runs in the terminal. The browser exists for one purpose: looking at the running site after the deploy.

What I changed from the standard challenge

The 16 steps are the floor. Here’s what I added on top:

The easter eggs need a quick caveat. They’re not on the page to show off. They’re there because anyone reading a DevOps engineer’s portfolio site is the kind of person who opens DevTools out of curiosity, and rewarding that curiosity feels right. The README mentions they exist; that’s all. Go find them.

The blog deserves its own note. The official challenge ends with “write a blog post about the build,” usually meaning a one-shot Medium or Dev.to article. I wanted a real ongoing blog, version-controlled in the same repo as the rest of the site, with proper tooling. Astro 6 with the Content Layer API was the right call: plain markdown for the posts, type-safe Zod schemas for the frontmatter, Shiki for code blocks, auto-generated RSS and sitemap. The blog builds in CI alongside the rest of the site. There’s no second deploy target. The dev.to version you are currently reading is a mirror, with canonical_url pointing at the original.

The security headers piece was a rabbit hole. SWA’s default deployment ships with no staticwebapp.config.json and therefore no security headers, which scores a B-minus on securityheaders.com. Adding CSP, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy gets you to A. Adding the cross-origin trinity (COOP, COEP, CORP) keeps the A grade and unlocks cross-origin isolation in the browser, which is required for SharedArrayBuffer and high-resolution timers if you ever need them. The COEP gotcha: it blocks external resources that don’t send Cross-Origin-Resource-Policy: cross-origin. Every <img> and <script> from a CDN has to either send that header or be loaded with crossorigin="anonymous". Took me an afternoon to track down the broken image that didn’t.

War stories

The challenge spec is honest about the fact that things will break. It just doesn’t tell you which things, when, or in what order. Here are five of mine, in rough order of how stupid I felt while debugging each one.

The DigiCert CAA record cliff

The first time I bound my custom domain to SWA, validation hung. Eighteen hours, no error, no progress bar, no telemetry. The portal showed “Validating” the way a person says “I’m working on it” once they have stopped working on it.

Azure Static Web Apps issues custom-domain TLS certs through DigiCert. DigiCert respects CAA records, which are DNS records that authorize which Certificate Authorities are allowed to issue certs for a domain. My DNS had no CAA records at all, which most people read as “anyone can issue,” but DigiCert’s stricter read is “you have not explicitly authorized me, and I will wait.” Forever, apparently.

The fix is one record:

Type: CAA
Host: @
Tag: issue
Value: digicert.com
TTL: 5 min

Within ten minutes of adding that record, validation completed and SSL provisioned. Total fix time, zero. Total debug time, one Saturday afternoon.

The lesson isn’t “remember CAA records.” It’s “Azure’s certificate provisioning has zero observability into what’s actually happening upstream at the CA, and you need to learn the CA’s quirks even though you never directly interact with it.” DigiCert is the most common CA Azure uses for managed certs. Their docs on CAA are buried.

The close_pull_request_job OIDC saga

This one ate a full debugging session and is worth its own write-up.

The default GitHub Actions workflow that Azure generates when you connect a SWA to a repo has two jobs: build_and_deploy_job and close_pull_request_job. Both call the same Azure/static-web-apps-deploy@v1 action. Both need to authenticate to Azure. They authenticate differently.

The build job uses both a static deployment token (from the repo secret AZURE_STATIC_WEB_APPS_API_TOKEN_*) AND an OIDC github_id_token. The close job, in the default workflow, uses only the static token. When the static token gets rotated or invalidated, the build job keeps working via OIDC. The close job silently fails.

A broken close job means preview environments never get cleaned up. They accumulate on every PR. SWA Free tier has a 10-environment cap. Once you hit it, new PR builds start failing with “max environments reached.” Your CI red-X count goes up. You go looking for what’s wrong with the build job. The build job is fine. The cleanup is broken.

Common online advice: add skip_deploy_on_missing_secrets: true to the close job. This silences the red X without fixing anything. Preview environments still accumulate, you just don’t get notified anymore. It is the CI/CD equivalent of putting electrical tape over the check-engine light.

The actual fix is to add OIDC auth to the close job, mirroring the build job. Three coordinated changes:

close_pull_request_job:
  permissions:
    id-token: write           # <-- 1. permission to mint a token
    contents: read
  runs-on: ubuntu-latest
  steps:
    - name: Install OIDC Client from Core Package    # <-- 2. mandatory install
      run: npm install @actions/[email protected] @actions/http-client

    - name: Get Id Token                             # <-- 3. mint the token
      uses: actions/github-script@v6
      id: idtoken
      with:
        script: |
          const coredemo = require('@actions/core')
          return await coredemo.getIDToken()
        result-encoding: string

    - name: Close Pull Request
      id: closepullrequest
      uses: Azure/static-web-apps-deploy@v1
      with:
        azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_* }}
        action: "close"
        github_id_token: ${{ steps.idtoken.outputs.result }}

The “Install OIDC Client” step is non-obvious but mandatory. Without it, actions/github-script‘s require('@actions/core') throws MODULE_NOT_FOUND because the package isn’t bundled with the action. The build job has this step too, which is easy to overlook when you’re copy-pasting from one job to the other.

Once OIDC is wired through, preview cleanup is automatic. Every closed PR auto-deletes its environment. The 10-env cap becomes a non-issue.

Cosmos serverless vs. provisioned

Cosmos DB has two billing modes: provisioned throughput (you pay for RU/s capacity you’ve reserved, whether or not you use it) and serverless (you pay per request, no reservation, no minimum). They look identical in the portal. They are not.

Provisioned mode has a 400 RU/s minimum, which works out to roughly $24/month at the cheapest. For a visitor counter that handles maybe a few requests per day from the people I’ve sent the link to, this is a comically wrong fit, like leasing a forklift to move one box of cereal. Serverless mode on the same workload rounds down to zero. The site is too new to have any Cosmos billing line items yet, but the projected number is “you will not notice it.”

The portal default for new databases is provisioned. You have to actively pick serverless during creation. There is no “switch from provisioned to serverless later.” You would have to migrate the data to a brand new account. If you create a Cosmos account in provisioned mode for a low-traffic personal project, you will pay $24/month minimum until you delete and recreate, and you will learn this on billing day.

Bicep makes this a one-line choice and a one-line documentation footnote:

capabilities: [
  { name: 'EnableServerless' }
]

The lesson is: read the billing-mode docs first, then pick the resource type. Cosmos is the most obvious offender, but Application Insights, Log Analytics, and Azure Front Door all have similar billing-mode forks where the wrong default at provisioning time costs you orders of magnitude more than you needed to spend.

EnableCanary and the region picker that wasn’t

When I first tried to provision Cosmos through the portal, every region dropdown showed exactly two regions, both with names ending in EUAP. Neither supported Cosmos. The portal claimed Cosmos was unavailable.

It turned out my subscription had the Microsoft.Resources/EnableCanary feature flag registered. This feature opts a subscription into “early-access (EUAP) regions” for testing pre-release Azure features. It also, very helpfully, filters every region dropdown across the portal to show only those EUAP regions, on the apparent theory that if you opted into preview features once you can never have wanted East US again.

Fix:

az feature unregister --namespace Microsoft.Resources --name EnableCanary
az provider register --namespace Microsoft.Resources

The provider re-registration is the part the docs don’t mention, and without it the feature-flag change doesn’t take effect on the next portal session.

I have no idea how that flag got registered on my subscription. The most likely answer is “I clicked something during an unrelated preview enrollment a year ago and forgot.” Welcome to the EnableCanary club.

The squash-merge that didn’t deploy

Once, after a clean PR merge to main, the production deploy didn’t run. The site stayed on the previous version. gh run list --branch=main --event=push showed no recent run. The Actions tab showed nothing happening. There was no error to look at, because there was nothing to error.

GitHub’s webhook delivery for push events occasionally drops. It is rare but real. The merge fires a push event, the webhook is supposed to fire to Actions, and sometimes the delivery silently fails. There is no retry. The push effectively didn’t happen as far as Actions is concerned.

The fix is to make another push event:

git pull origin main
git commit --allow-empty -m "chore: retrigger deploy"
git push origin main

The empty commit costs nothing and forces a fresh webhook. The deploy ran on the next push without complaint.

The deeper lesson: production deploys need a “did we actually ship?” sanity check that doesn’t rely on the deploy log being green. For this site, that’s:

curl "https://rlwilliamson.dev/?nocache=$(date +%s)" | grep "your latest change"

The cache-busting query string forces a fresh fetch through Azure Front Door’s edge cache. If your latest change isn’t in the response, the deploy didn’t ship, regardless of what Actions says.

What I learned

A few things stuck.

The combo of gh pr checks --watch && gh pr merge --squash --delete-branch is the solo-dev gold path. The browser is for reading code and viewing the live site. Everything else (PR creation, status, merge, branch cleanup) runs from the terminal. Once you’ve internalized that, going back to clicking buttons in GitHub’s UI feels like walking when you could be biking.

Inline CSS is a sin for any team project and a feature for a single-page personal site. One file, one request, no FOUC, no source maps to leak, no build step. The “every page should have an external stylesheet” rule exists for reasons that don’t apply when there is one page.

The challenge is structurally a feedback loop. You build the smallest possible thing that exercises every layer of the stack (frontend, backend, database, IaC, CI/CD, DNS, certs, observability), and then every problem you hit teaches you something specific and load-bearing about one of those layers. A textbook tells you “Cosmos has serverless and provisioned modes.” Building this site teaches you that the difference between those modes is $24 in real money on your real credit card on real billing day, and you will not forget it.

Personal projects with running infrastructure teach you what your gaps are in a way that work projects often can’t. At work, you have a team. There is always someone who already knows the answer to the Cosmos billing question or the OIDC quirk. On your personal project, you are the team. You are the on-call. You are the entire stack-trace reader and the entire postmortem author and the only attendee at the postmortem. The gaps stand out, because they are the only thing in the room. You fill them in.

Tech stack

• Hosting: Azure Static Web Apps (Free tier)
• Compute: Azure Functions (Python 3.11, V2 programming model)
• Database: Azure Cosmos DB for NoSQL (Serverless capacity)
• IaC: Bicep
• CI/CD: GitHub Actions with OIDC auth on both build and close-PR jobs
• Frontend: HTML / CSS / JavaScript (single-file resume), JetBrains Mono via Google Fonts
• Blog: Astro 6 with Content Layer API, Shiki (Tokyo Night), @astrojs/rss, @astrojs/sitemap, reading-time
• DNS: Namecheap
• SSL: DigiCert via SWA managed cert
• Tests: pytest with mocked Cosmos client, gated via needs: test
• Commit signing: SSH with a dedicated id_ed25519_signing key
• Total cost: $0/month plus $12/year for the domain

Where to next

The repo is at github.com/rlwilliamson-dev/azure-resume. Fork it, adapt it, rip out the parts you don’t want, ignore the parts you find self-indulgent. The README has a section called How this differs from the standard Cloud Resume Challenge that maps every addition I made back to the base spec, which should make cherry-picking easier.

If you’re hiring senior DevOps or platform engineers and you’ve read this far, my contact is in the site footer at rlwilliamson.dev. Otherwise, thanks for reading. If you take the Cloud Resume Challenge yourself, I’d genuinely like to see what you build. Send me a link, even if it’s just the 16 steps. Especially then.

The 16 steps are the floor.

Building a production-ready RAG (Retrieval-Augmented Generation) application from scratch is a very difficult and time consuming.

You first get a new idea for a project or business but before you can write a single line of your actual core business logic, you find yourself spending weeks fighting with multimodal file parsing, configuring vector extensions, designing complex database architecture, and securing multi-tenant storage.

The idea behind building DocuIntel was to solve that exact infrastructure headache. I wanted a template where I could just drop in my API keys, run a single setup script, and have a fully functioning, secure AI document portal ready to go.

Here is a deep dive into the architecture, some of the technical hurdles I ran into, and how I solved them.

The Multimodal Architecture
DocuIntel needs to process a wide variety of inputs—PDFs, Word docs, images (JPG/PNG), and even audio transcriptions (MP3s).

At first I was routing different file types through different third party parsing libraries like I was using tesseract OCR for extracting text from from images and pdf-parser for PDF files.
But instead of doing that, I offloaded the heavy lifting directly to Gemini 2.0 Flash. Gemini’s native multimodal capabilities handle OCR and layout analysis beautifully.

To ensure consistent, high-quality search results, I hardcoded a rigorous master SYSTEM_PROMPT inside the backend processing route (/api/process/route.ts). It forces the AI to behave like a clean data architect:

const SYSTEM_PROMPT = `
  You are an expert document parser for DocuIntel. 
  Your goal is to extract text with 100% accuracy for semantic search indexing.

  RULES:
  1. PRESERVE STRUCTURE: Use Markdown for headings, subheadings, and lists.
  2. TABLES: Convert all data tables into clean Markdown table format.
  3. NO CHAT: Do not say "Here is the text" or "I have processed the file." 
  4. NOISE REDUCTION: Ignore headers, footers, and page numbers.
  5. SMART METADATA: At the very end of your output, add a section called '---METADATA---' 
     and list 5-10 key entities or topics (e.g., 'Company: Acme Corp', 'Date: 2024-01-01').
`;

Why the Automated Metadata Tag Matters?
By forcing Gemini to extract smart metadata tags right into the text, the vector embedding model (gemini-embedding-2) captures high-value concepts. If a user searches for a specific company or date, the semantic search will surface the document even if that detail only appeared once in fine print.

Navigating the New Supabase Security Standards
Supabase has shifted to a “Secure by Default” model which will come into force by 30th May 2026.

Previously, creating a table in the public schema automatically exposed it to the Data API. Now, new tables require explicit grants, or your frontend client libraries (supabase-js) will get a 42501 permission error—even if Row-Level Security (RLS) is enabled.

To future-proof the setup, my SQL initialization script applies explicit grants directly to the authenticated user roles right after creating the tables and vector search RPC functions:

-- Create the documents table with 3,072-dimensional vector support
create table if not exists public.documents (
  id uuid primary key default uuid_generate_v4(),
  file_name text not null,
  file_url text not null,
  content text,               
  user_id uuid references auth.users(id),
  user_email text not null,   
  embedding vector(3072),     -- Optimized for high-res gemini-embedding-2
  created_at timestamp with time zone default timezone('utc'::text, now())
);

-- EXPLICIT GRANTS (Fixes 42501 Permission Errors)
grant select, insert, update, delete on table public.documents to authenticated;
grant all on table public.documents to service_role;

Hardened Multi-Tenant Storage Policies
Since the app deals with private user documents i.e. User A must never be able to discover or access User B’s files, I have structured the Supabase Storage bucket so that every uploaded file is dynamically sandboxed into a folder named exactly after the user’s unique authenticated ID (auth.uid()).

Here are the folder-level RLS storage policies that enforce that rule on every single request:

-- Restrict file uploads to the user's own UID folder
create policy "Users can upload their own documents"
on storage.objects for insert to authenticated
with check (
  bucket_id = 'documents' AND 
  (storage.foldername(name))[1] = auth.uid()::text
);

-- Restrict file reading to the user's own UID folder
create policy "Users can view their own documents"
on storage.objects for select to authenticated
using (
  bucket_id = 'documents' AND 
  (storage.foldername(name))[1] = auth.uid()::text
);

Keeping the Frontend Clean & Readable
On the frontend (Next.js 14 + TypeScript + Tailwind CSS), I wanted a highly scalable UI. A minor but common mess I see in boilerplate projects is massive, nested ternary operators in the JSX to handle dynamic file icons.

To keep things clean and performant, I grouped the file extension arrays and used standard JavaScript .includes() methods to dynamically assign Lucide React icons using a single class utility string:

Wrapping Up:
Building these layers from scratch took extensive testing, debugging environment variables, and reading through updated security documentation. But once it’s configured, it works like magic: a user uploads an asset, Gemini reads and indexes it with a 3,072-dimension vector embedding, and you can instantly query your documents conceptually rather than just matching keywords.

If you are planning to build an AI document product for a client or launching your own micro-SaaS, you don’t have to spend your weekend configuring this infrastructure from zero.

I’ve packaged this exact production-ready foundation—complete with the 1-click database initialization script, Next.js frontend, and pre-configured API routes—into a developer-friendly boilerplate.

https://dhritiman.gumroad.com/l/docuintel