GamesRadar’a açıklamalarda bulunan CD Projekt Red’in üst düzey yöneticilerinden Jarosław Ruciński ve Adrian Fulneczek, çıkışı fiyaskoya dönüşen Cyberpunk 2077’deki hataları, aktif olarak geliştirme aşamasında olan The Witcher 4 ve Cyberpunk 2’de tekrarlamayacaklarını söyledi. “Dersimizi aldık” diyen yöneticiler, Cyberpunk 2077’deki teknik hataların tekrarlanmaması için yeni bir geliştirme sürecini devreye aldıklarını aktardı. Ruciński, geçmişin aksine bilgi birikiminin artık belirli ekiplere özel olmadığını ve öğrenilenlerin/yapılanların şirket içi ortak bir varlığa dönüştüğünü ifade ediyor. “Örneğin, The Witcher üzerinde çalışan bir ekip belirli bir sorun için çözüm ürettiğinde, Cyberpunk ekibi de bunu görebiliyor. Bu sayede diğer ekip geliştirilen çözümden yararlanabiliyor, kodu kendi projesine dahil edebiliyor ve üzerinde küçük değişiklikler yapabiliyor” diyen Ruciński, daha önce başlarına büyük sorun açan dokümantasyon sürecini de iyileştirdiklerini belirtiyor. The Witcher 4 ve Cyberpunk 2’nin ne zaman çıkacağı henüz tam olarak bilinmiyor.

CD Projekt Red: “The Witcher 4 ve Cyberpunk 2’de Cyberpunk 2077’deki hataları tekrarlamayacağız.” yazısı ilk olarak Teknoloji haberleri – LOG üzerinde yayınlandı.

It was a Tuesday afternoon. Nothing felt different about the deploy. Same process as always. Green tests. Approved PR. Merged to main. Deployed.
Seventeen minutes later the alerts started.

What Happened

We had a background job that processed orders and updated inventory counts across our marketplace. The job ran every few minutes and had been running reliably for months without a single issue.

I had made what I genuinely believed was a minor change. A refactor. Cleaner logic, same behavior. I had tested it locally. The tests passed. A colleague reviewed it and approved it. Everything looked fine.

What I had not accounted for was a race condition that only appeared under concurrent load.

In local testing and in our staging environment, the job ran sequentially. In production it ran concurrently across multiple workers. The refactored logic I had written assumed sequential execution. Under concurrent load two workers would occasionally read the same inventory record at the same time, both calculate an update based on the same stale value, and both write back — the second write overwriting the first.

The result was inventory counts that drifted further from reality with every job run. Orders were being accepted for products that were no longer in stock. The customer facing side kept working normally which is why the alerts took seventeen minutes to fire. By the time we caught it the damage was done.

Three hours to identify, roll back, audit the affected records, correct the data, and verify the fix. Three hours of degraded service. A handful of orders that had to be manually cancelled and refunded. A postmortem that took longer than the incident itself.

All of it from a change I had described in the PR as “minor refactor, no behavioral changes.”

The Mistakes I Made

Looking back there were several distinct failures that each independently could have prevented this.

I did not read the existing tests carefully enough.

The existing tests covered the sequential case because the original code had been written with sequential execution in mind. The tests passed because I had not broken the sequential behavior — I had broken the concurrent behavior that the tests never covered. I looked at green tests and assumed coverage. Green tests mean the tested cases pass. They say nothing about the cases that were not tested.

I assumed staging was representative of production.

Our staging environment runs a single worker. Production runs multiple. This difference had never mattered before because nothing we had deployed before this change was sensitive to concurrency. I knew staging was a single worker. I did not think about whether that mattered for this specific change. It did.

I described the change inaccurately in the PR.

“Minor refactor, no behavioral changes” is the kind of description that makes reviewers less careful not more. My colleague approved it quickly because the description signaled low risk. If I had described it accurately — “refactoring the inventory update logic, please check whether the new approach handles concurrent writes correctly” — the review would have been different. The description I wrote was not intentionally misleading. It reflected my own incorrect assessment of the risk. That is worse in some ways. I had convinced myself it was minor before I convinced anyone else.

I had no concurrency testing in my local verification process.

I ran the job locally once and watched it complete successfully. I did not run it concurrently. I did not run it under load. I did not simulate multiple workers. Running a background job once in a local environment and calling it tested is not testing the job — it is testing one execution path under ideal conditions.

What We Changed After

We added a staging environment worker count that matches production.

This sounds obvious in retrospect. It was not obvious until it cost us three hours. Staging now runs the same number of workers as production for any service where concurrency matters. The environment is still not a perfect replica but the concurrency profile now matches.

We added explicit concurrency tests for any job that touches shared state.

Not just unit tests for the logic. Integration tests that spin up multiple workers, run them simultaneously against the same test data, and verify the outcome is consistent. These tests are slower. They are also the only tests that would have caught this specific failure.

We changed how we describe PR risk in reviews.

We added a required field to our PR template: “Concurrency considerations.” It can be filled with “N/A — this change does not touch shared state” which takes ten seconds. For changes that do touch shared state it forces the author to think about and articulate the concurrency implications before a reviewer sees the code. The reviewer then knows to focus there specifically.

We introduced a pre-deploy checklist for background jobs.

Not a long checklist. Four questions. Does this job touch shared state? Does the logic assume sequential execution? Has it been tested under concurrent load? Does the rollback procedure work without manual data correction? The checklist takes two minutes. The incident it is designed to prevent took three hours.

The Thing I Keep Coming Back To

The bug itself was not the most interesting part of this story.
The most interesting part is how many independent checkpoints it sailed through without being caught.

I did not catch it in my own review of the change. My colleague did not catch it in the PR review. The automated tests did not catch it. The staging deployment did not catch it.

Each of those checkpoints failed for a different reason. My self review failed because I had already decided the change was minor and was not looking carefully. The PR review failed because my description set a low risk frame that the reviewer accepted. The automated tests failed because they covered the wrong execution model. Staging failed because the environment did not match production on the one dimension that mattered.
A failure that gets through four independent checkpoints is not bad luck. It is a systems problem. The checkpoints were not actually independent — they were all downstream of my initial incorrect assessment that the change was low risk. Once I had decided it was minor, every subsequent checkpoint was biased toward confirming that assessment.

This is the uncomfortable version of the lesson. It is not just “write better tests” or “stage more carefully.” It is that your own confidence in a change actively degrades the quality of the checks that follow it. The times you are most certain a change is safe are the times you most need to deliberately stress test that certainty.

What I Do Differently Now

Before merging anything that touches background jobs, queues, or shared state I now ask one question regardless of how minor the change feels:
What would have to be true about the execution environment for this to fail silently?

Not loudly. Not with an immediate error that gets caught in tests. Silently — in a way that produces correct looking output under normal conditions and wrong output under specific conditions that do not exist in development or staging.

That question changes how I look at a change. It forces me to think about the gap between the environment where I tested and the environment where it will run.

It would have caught this one. It has caught two potential issues since. Neither of them made it to production.
The best lesson from a production incident is not a new process. It is a new question you ask yourself before you need the process.

Exact Solution is a certified refurbished electronics marketplace shipping across Europe. We stock the best refurbished laptops from Apple, Dell, HP, and Lenovo — all fully tested and ready to ship.

Most developers approach XRPL event monitoring like it’s a simple polling problem. They spin up a script that checks wallet balances every few minutes, maybe set up a cron job to scan for new transactions. When they need real-time updates, they build a WebSocket listener that connects directly to the ledger.

This approach breaks down fast. WebSocket connections drop. Rate limits kick in. Transaction parsing gets complex when you’re dealing with DEX trades, escrow releases, and payment channel updates. The script that worked fine in development starts missing events in production.

The correct frame: XRPL event monitoring is an infrastructure problem, not a feature problem.

You need reliable delivery, retry logic, signature verification, and dead-letter queues. You need to handle 22 different event types across seven categories without your application logic becoming a mess of conditional statements. Most importantly, you need this infrastructure to work when your application scales beyond a few users.

I built XRNotify because every XRPL developer was solving the same infrastructure problem independently. The results were predictable: brittle listeners, missed events, no monitoring, and hours spent debugging webhook delivery failures instead of building product features.

The Infrastructure Layer XRPL Was Missing

XRNotify sits between the XRP Ledger and your application. It maintains persistent connections to the network, parses transaction data in real-time, and delivers structured events to your endpoints via webhooks.

The delivery mechanism uses exponential backoff retry with dead-letter queues for failed deliveries. Every payload includes HMAC-SHA256 signatures for verification. When your endpoint is down for maintenance, events queue up and deliver in order once you’re back online.

The event categorization covers wallet activity, DEX operations, escrow management, payment channels, NFT transfers, network state changes, and validator updates. Instead of parsing raw XRPL transaction data, you receive structured JSON with the specific information your application needs.

Beyond Simple Notifications

The real value emerges when you consider XRNotify as part of a larger data infrastructure. Network state data flows to The Neutral Bridge for financial research. Anomaly patterns feed into H.U.N.I.E.’s intelligence layer. The same infrastructure that delivers your application webhooks also powers the circuit breaker logic in H.U.N.I.E. Sentinel.

This isn’t accidental architecture. XRPL generates massive amounts of transaction data, but most of it gets processed once and discarded. XRNotify captures that data at the moment it becomes available, then routes it to multiple systems that extract different types of value.

Your application gets the specific events it needs. The research infrastructure gets comprehensive network state data. The security layer gets real-time anomaly detection. The same infrastructure investment serves multiple purposes.

Technical Implementation

The system runs on Next.js 14 with PostgreSQL for persistence and Redis for event queuing. Node.js workers handle the XRPL connections and webhook delivery. XRPL.js provides the ledger interface, but the heavy lifting happens in the retry logic and delivery infrastructure.

The webhook endpoints receive POST requests with JSON payloads. Each payload includes event metadata, transaction details, and the HMAC signature for verification. Your application validates the signature, processes the event, and returns a 200 status code. If validation fails or processing errors occur, the retry mechanism handles redelivery.

Event filtering happens at the subscription level. You specify which wallets to monitor, which event types to receive, and which endpoint should handle each category. The infrastructure handles the complexity of maintaining multiple XRPL connections, parsing different transaction types, and routing events to the correct destinations.

The Result

Developers connect their systems to XRPL events without building listener infrastructure. Applications respond to network activity in real-time without polling delays. The webhook delivery guarantees mean events reach your system even during deployment downtime or temporary outages.

This is infrastructure that works when you need it to work. No missed events, no parsing edge cases, no webhook delivery debugging at 2 AM.

https://www.xrnotify.io

As part of my volunteering with AbilityNet, I recently attended a training session on motor neurone disease (MND) — and it quietly shifted how I think about what technology is actually for.

What is MND?

MND affects nerves in the brain and spinal cord called motor neurones — the nerve cells that tell your muscles what to do. It’s a rare but progressive condition. There is no cure, and it’s usually life-shortening.

Around 5,000 people in the UK are living with MND at any one time. Six people are diagnosed every single day.

Voice banking

Voice banking was something I hadn’t encountered before this session. It allows a person to record their own voice or create personalised messages for use with speech-generating devices — and the MND Association recommends starting early, even before speech is significantly affected.

The goal is to preserve something that can’t be replaced once it’s gone: the sound of you.

Research shows that “preserving identity” is the overarching motivation for people who choose to voice bank. Yet only around 12% of people diagnosed with MND in the UK actually go through with it — because the process has historically been too long, too hard, and too clinical.

That gap between what’s available and what people can actually access is exactly where technology and human support need to meet.

Good technology serves the people

You don’t need to be in tech to show up for this. Anyone can volunteer, learn, and help bridge that gap.

If you want to understand more about what living with MND actually looks like, this is a good place to start:

Further reading:

• NHS — Motor Neurone Disease
• MND Association
• AbilityNet volunteer programme

Stop Paying $20/Month for Email You Could Run for $1.50

Most teams handle programmatic email one of two ways.

The first: sign up for SendGrid or Mailgun, grab an API key, and start sending. It works great — until you hit 50K emails and the bill jumps to $20/month, then you add inbound parsing and it’s another plan tier, then you need bounce webhooks and suddenly you’re deep in a third-party dashboard you barely understand, paying for features you don’t use.

The second: use Google Workspace’s catch-all rule to funnel everything into one inbox. Free-ish, but completely dumb — no processing, no routing, no automation. Every contact form submission and support@ email lands in the same pile for a human to sort manually.

There’s a better way that most AWS developers walk right past. Amazon SES catches the mail, S3 stores it, Lambda processes it. The logic lives in your codebase, the infrastructure is fully in your AWS account, and the bill last month was $1.48.

Let me show you exactly how it works.

The idea in one diagram

That’s it. SES stores the raw email in S3, then fires Lambda. Lambda rewrites the headers and re-sends via SES. Your team gets the email; they hit Reply; it goes back to whoever sent it.

The S3 step isn’t optional — it’s actually the smart part. Every email is durably stored before Lambda touches it. Parser bug? Replay from S3. Need an audit trail? It’s there, with 30-day auto-expiry built in.

This is not a replacement for Google Workspace or AWS WorkMail

Worth saying clearly before going further.

Google Workspace and AWS WorkMail are mailbox products — your team logs in, reads emails, manages a calendar, sends from their phone. That’s a completely different problem from what this stack solves.

The honest recommendation: run both. Keep Workspace or WorkMail for your team’s human mailboxes. Use this stack for everything your application touches — the support@ alias that should file a ticket, the contact@ that should notify Slack, the noreply@ that sends password resets.

They’re not competing. They solve different problems.

Why not just use SendGrid / Postmark / Mailgun?

Nothing wrong with them. But look at the math for a typical small company:

SES is $0.10 per 1,000 emails. That rate has not changed since 2011. At 500K sends, you’re paying $50 while Postmark charges $650 for the same volume. The gap is real.

The catch: SES is infrastructure, not a product. No drag-and-drop campaign builder, no deliverability dashboard (unless you pay extra for VDM). If you want someone to hold your hand through email, pay for managed. If you’re comfortable in AWS, this stack is hard to beat.

Keep Google Workspace or Microsoft 365 for your human mailboxes. This pattern is for the email your code handles — contact forms, support@ aliases, alerts, password resets.

What the deploy actually creates

One SAM template provisions everything:

• S3 bucket — stores raw emails under inbox/, auto-deletes after 30 days
• SES identity — verifies your domain with DKIM signing enabled
• Route 53 records — MX, SPF, DMARC, and all 3 DKIM CNAMEs, created automatically
• Lambda + IAM — reads from S3, forwards via SES
• Receipt rule — catches all mail at your domain, stores then invokes Lambda
• Custom resource — activates the rule set automatically so you don’t have to

sam build
sam deploy --guided 
  --parameter-overrides 
    EmailDomain=yourcompany.com 
    ForwardTo=team@yourcompany.com 
    ForwardFrom=relay@yourcompany.com 
    HostedZoneId=ZXXXXXXXXXXXXX

One command. Everything wired up.

The code

template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Serverless email infrastructure (SES + S3 + Lambda)

Parameters:
  EmailDomain:
    Type: String
    Description: Domain configured to receive email (e.g. yourcompany.com)
  ForwardTo:
    Type: String
    Description: Address to forward incoming mail to
  ForwardFrom:
    Type: String
    Description: Verified sender used as the From on forwards
  HostedZoneId:
    Type: String
    Description: Route 53 Hosted Zone ID for the domain

Resources:

  # -------------------------------------------------------
  # S3 — raw email storage
  # -------------------------------------------------------
  EmailBucket:
    Type: AWS::S3::Bucket
    Properties:
      LifecycleConfiguration:
        Rules:
          - Id: ExpireRawEmails
            Status: Enabled
            ExpirationInDays: 30

  EmailBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref EmailBucket
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Sid: AllowSESPuts
            Effect: Allow
            Principal:
              Service: ses.amazonaws.com
            Action: s3:PutObject
            Resource: !Sub '${EmailBucket.Arn}/*'
            Condition:
              StringEquals:
                AWS:SourceAccount: !Ref AWS::AccountId

  # -------------------------------------------------------
  # Lambda — parse and forward
  # -------------------------------------------------------
  ProcessEmailFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: python3.12
      Handler: app.lambda_handler
      CodeUri: ./src/
      MemorySize: 256
      Timeout: 30
      Environment:
        Variables:
          FORWARD_TO: !Ref ForwardTo
          FORWARD_FROM: !Ref ForwardFrom
          BUCKET_NAME: !Ref EmailBucket
          BUCKET_PREFIX: inbox/
      Policies:
        - S3ReadPolicy:
            BucketName: !Ref EmailBucket
        - Statement:
            - Effect: Allow
              Action: ses:SendRawEmail
              Resource: '*'

  ProcessEmailFunctionSESPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !GetAtt ProcessEmailFunction.Arn
      Principal: ses.amazonaws.com
      SourceAccount: !Ref AWS::AccountId

  # -------------------------------------------------------
  # SES — identity, receipt rule set and rule
  # -------------------------------------------------------
  SESEmailIdentity:
    Type: AWS::SES::EmailIdentity
    Properties:
      EmailIdentity: !Ref EmailDomain
      DkimAttributes:
        SigningEnabled: true

  ReceiptRuleSet:
    Type: AWS::SES::ReceiptRuleSet
    Properties:
      RuleSetName: !Sub '${AWS::StackName}-rules'

  ReceiptRule:
    Type: AWS::SES::ReceiptRule
    DependsOn:
      - EmailBucketPolicy
      - SESEmailIdentity
    Properties:
      RuleSetName: !Ref ReceiptRuleSet
      Rule:
        Name: store-and-process
        Enabled: true
        ScanEnabled: true
        Recipients:
          - !Ref EmailDomain
        Actions:
          - S3Action:
              BucketName: !Ref EmailBucket
              ObjectKeyPrefix: inbox/
          - LambdaAction:
              FunctionArn: !GetAtt ProcessEmailFunction.Arn
              InvocationType: Event

  # -------------------------------------------------------
  # Route 53 — all DNS records created automatically
  # -------------------------------------------------------
  DkimRecord1:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref HostedZoneId
      Name: !Sub "${SESEmailIdentity.DkimDNSTokenName1}"
      Type: CNAME
      TTL: 300
      ResourceRecords:
        - !Sub "${SESEmailIdentity.DkimDNSTokenValue1}"

  DkimRecord2:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref HostedZoneId
      Name: !Sub "${SESEmailIdentity.DkimDNSTokenName2}"
      Type: CNAME
      TTL: 300
      ResourceRecords:
        - !Sub "${SESEmailIdentity.DkimDNSTokenValue2}"

  DkimRecord3:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref HostedZoneId
      Name: !Sub "${SESEmailIdentity.DkimDNSTokenName3}"
      Type: CNAME
      TTL: 300
      ResourceRecords:
        - !Sub "${SESEmailIdentity.DkimDNSTokenValue3}"

  MXRecord:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref HostedZoneId
      Name: !Sub "${EmailDomain}."
      Type: MX
      TTL: 300
      ResourceRecords:
        - !Sub "10 inbound-smtp.${AWS::Region}.amazonaws.com"

  SPFRecord:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref HostedZoneId
      Name: !Sub "${EmailDomain}."
      Type: TXT
      TTL: 300
      ResourceRecords:
        - '"v=spf1 include:amazonses.com -all"'

  DMARCRecord:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref HostedZoneId
      Name: !Sub "_dmarc.${EmailDomain}."
      Type: TXT
      TTL: 300
      ResourceRecords:
        - !Sub '"v=DMARC1; p=none; rua=mailto:dmarc@${EmailDomain}"'

  # -------------------------------------------------------
  # Custom resource — activates the rule set automatically
  # -------------------------------------------------------
  ActivateRuleSetFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: python3.12
      Handler: index.handler
      InlineCode: |
        import boto3, cfnresponse
        def handler(event, context):
          try:
            if event['RequestType'] in ('Create', 'Update'):
              boto3.client('ses').set_active_receipt_rule_set(
                RuleSetName=event['ResourceProperties']['RuleSetName']
              )
            cfnresponse.send(event, context, cfnresponse.SUCCESS, {})
          except Exception as e:
            cfnresponse.send(event, context, cfnresponse.FAILED, {'Error': str(e)})
      Policies:
        - Statement:
            - Effect: Allow
              Action: ses:SetActiveReceiptRuleSet
              Resource: '*'

  ActivateRuleSet:
    Type: AWS::CloudFormation::CustomResource
    DependsOn: ReceiptRuleSet
    Properties:
      ServiceToken: !GetAtt ActivateRuleSetFunction.Arn
      RuleSetName: !Ref ReceiptRuleSet

Outputs:
  BucketName:
    Description: S3 bucket storing raw incoming emails
    Value: !Ref EmailBucket
  RuleSetName:
    Description: Active receipt rule set
    Value: !Ref ReceiptRuleSet
  LambdaFunction:
    Description: Email processing function
    Value: !GetAtt ProcessEmailFunction.Arn

src/app.py

The key insight here is to mutate the original message in-place rather than building a new MIME object from scratch. This preserves attachments, encoding, and HTML content without any extra parsing. The critical step is stripping the headers that would cause DKIM verification to fail on the re-sent message.

import os
import email
import logging

import boto3

logger = logging.getLogger()
logger.setLevel(logging.INFO)

s3  = boto3.client("s3")
ses = boto3.client("ses")

FORWARD_TO    = os.environ["FORWARD_TO"]
FORWARD_FROM  = os.environ["FORWARD_FROM"]
BUCKET_NAME   = os.environ["BUCKET_NAME"]
BUCKET_PREFIX = os.environ["BUCKET_PREFIX"]  # "inbox/"

# These headers are tied to the original message signature.
# Keeping them causes DKIM failure on the forwarded copy.
HEADERS_TO_REMOVE = [
    "DKIM-Signature",
    "Sender",
    "Return-Path",
    "Message-ID",
]


def lambda_handler(event, context):
    for record in event["Records"]:
        mail    = record["ses"]["mail"]
        receipt = record["ses"]["receipt"]

        msg_id     = mail["messageId"]
        recipients = receipt["recipients"]
        s3_key     = f"{BUCKET_PREFIX}{msg_id}"

        logger.info("Processing %s for recipients %s", msg_id, recipients)

        raw = s3.get_object(Bucket=BUCKET_NAME, Key=s3_key)["Body"].read()
        msg = email.message_from_bytes(raw)

        original_from    = msg.get("From", "unknown")
        original_subject = msg.get("Subject", "(no subject)")

        for header in HEADERS_TO_REMOVE:
            if header in msg:
                del msg[header]

        # Rewrite From so SES accepts the send
        if "From" in msg:
            msg.replace_header("From", FORWARD_FROM)
        else:
            msg["From"] = FORWARD_FROM

        # Reply-To preserves the original sender — your team hits Reply
        # and it goes back to the person who sent it, not to your relay address
        if "Reply-To" in msg:
            msg.replace_header("Reply-To", original_from)
        else:
            msg["Reply-To"] = original_from

        # Subject prefix tells you which alias received it
        original_recipient = recipients[0] if recipients else ""
        new_subject = f"[{original_recipient}] {original_subject}"
        if "Subject" in msg:
            msg.replace_header("Subject", new_subject)
        else:
            msg["Subject"] = new_subject

        if "To" in msg:
            msg.replace_header("To", FORWARD_TO)
        else:
            msg["To"] = FORWARD_TO

        ses.send_raw_email(
            Source=FORWARD_FROM,
            Destinations=[FORWARD_TO],
            RawMessage={"Data": msg.as_bytes()},
        )
        logger.info("Forwarded %s → %s → %s", msg_id, original_recipient, FORWARD_TO)

    return {"statusCode": 200}

Deploying it

Prerequisites: AWS CLI configured, SAM CLI v1.129+, a domain in Route 53.

1. Get your hosted zone ID

aws route53 list-hosted-zones 
  --query "HostedZones[?Name=='yourcompany.com.'].{Id:Id,Name:Name}"

2. Build and deploy

sam build
sam deploy --guided 
  --parameter-overrides 
    EmailDomain=yourcompany.com 
    ForwardTo=team@yourcompany.com 
    ForwardFrom=relay@yourcompany.com 
    HostedZoneId=ZXXXXXXXXXXXXX

SAM will ask a few questions on first run — accept the defaults, confirm the changeset. The deploy creates everything: S3, Lambda, SES identity, all DNS records, and activates the rule set automatically.

3. Request production access

New SES accounts are sandboxed — you can only send to verified addresses until you request production access. Go to SES Console → Account dashboard → Request production access, describe your use case in two sentences. AWS typically responds within 24 hours.

4. Test it

Send an email to any address at your domain and watch the logs:

aws logs tail /aws/lambda/YOUR_STACK_NAME-ProcessEmailFunction 
  --follow 
  --region us-east-1

You should see the message ID logged, then a forwarded confirmation within a couple of seconds.

If domain verification stays Pending

After deploy, SES verifies your domain by polling the DKIM CNAME records. It usually completes within 5–10 minutes. If it’s been 30+ minutes and still shows Pending, check that DNS is actually resolving:

# Replace with one of your actual DKIM token values
nslookup -type=CNAME abc123._domainkey.yourcompany.com 8.8.8.8

If that resolves correctly, nudge SES to recheck immediately rather than waiting for its next polling cycle:

aws ses verify-domain-identity 
  --domain yourcompany.com 
  --region us-east-1

# Then check status
aws ses get-identity-verification-attributes 
  --identities yourcompany.com 
  --region us-east-1

If DNS isn’t resolving at all, the CNAME records weren’t created cleanly — check the CloudFormation events for any Route 53 errors during deploy.

A few gotchas learned the hard way

SES only allows one active rule set per region per account. If you have multiple environments in the same region, they share this single slot. The solution is one rule set with multiple rules — one rule per domain — rather than multiple rule sets competing for the active slot.

New accounts start in sandbox mode. You can only send to verified addresses until you request production access. Do it on day one — AWS usually approves within 24 hours and there’s no reason to wait.

Not every region supports inbound. Stick to us-east-1, us-west-2, or eu-west-1. Pick your region before pointing your MX record.

Watch your bounces. SES will throttle your account if you keep sending to addresses that bounce. Enable Account-Level Suppression in the SES console — it’s free and it protects you automatically.

Want to go further?

This same stack is the backbone for a contact form endpoint — a Lambda Function URL receives the POST, validates the fields, and calls ses.send_email. Same verified domain, same IAM, a few dozen extra lines. Add a honeypot field and you get basic bot protection for free.

You can also extend the forwarder to route different aliases to different people (support@ → helpdesk, billing@ → finance) by checking recipients[0] in the Lambda and maintaining a simple routing table in the code or in DynamoDB.

The foundation is solid. Build on it.

The Problem We Were Actually Solving

I was trying to sell digital products online from a country that was blocked by most major payment platforms, including PayPal, Stripe, Gumroad, and Payhip. As the founder of a small business, this was a major obstacle to my ability to reach customers and generate revenue. I spent countless hours researching alternative payment solutions, but every door I tried to open seemed to be met with a new set of restrictions and requirements. It was clear that this was a platform problem, not a problem with my business or my products.

What We Tried First (And Why It Failed)

My first approach was to try to use VPNs and other workarounds to pretend to be located in a country that was not blocked by these platforms. However, this approach was not only risky and potentially illegal, but it also did not work. The payment platforms had sophisticated systems in place to detect and prevent this type of activity, and I was quickly flagged and banned from using their services. I also tried to use smaller, lesser-known payment platforms, but these often had their own set of problems, including high fees, poor customer support, and limited functionality. After months of trying different solutions, I was on the verge of giving up.

The Architecture Decision

It was at this point that I made the decision to take a completely different approach. Instead of trying to work with existing payment platforms, I decided to build my own custom payment solution using a combination of open-source software and local payment providers. This was a daunting task, as it would require significant development and testing resources, but I was convinced that it was the only way to truly bypass the restrictions and achieve my business goals. I chose to use a modular architecture, with separate components for payment processing, order management, and customer management. This would allow me to easily integrate with different payment providers and switch between them as needed. I also decided to use a microservices-based approach, with each component running as a separate service. This would provide greater flexibility and scalability, and would allow me to quickly respond to changes in the market or in the requirements of my business.

What The Numbers Said After

The results of this decision were nothing short of astonishing. Within six months, I was able to launch my custom payment solution and start processing transactions. The fees were significantly lower than those charged by the major payment platforms, and the customer support was vastly improved. I was also able to customize the solution to meet the specific needs of my business, including integrating with my existing order management and customer management systems. In terms of metrics, I saw a 30% increase in sales within the first year, and a 25% decrease in payment processing fees. The solution also provided greater scalability and reliability, with uptime of 99.99% and average transaction processing time of under 2 seconds. I used tools such as Prometheus and Grafana to monitor the performance of the system, and was able to quickly identify and resolve any issues that arose.

What I Would Do Differently

In retrospect, I would have made the decision to build a custom payment solution much earlier. I would have also invested more in testing and quality assurance, as this would have helped to identify and resolve issues more quickly. I would also have chosen to use more cloud-based services, such as AWS Lambda and API Gateway, to provide greater scalability and flexibility. Additionally, I would have implemented more robust security measures, such as two-factor authentication and encryption, to protect sensitive customer data. However, overall, I am extremely satisfied with the decision to build a custom payment solution, and I believe that it has been a key factor in the success of my business. I used tools such as Jenkins and GitLab CI/CD to automate the testing and deployment of the solution, and was able to quickly and easily make changes to the system as needed.