Day 11 – AWS VPC

VPC Dashboard

When you start learning AWS, services like EC2, S3, Lambda, and RDS feel exciting.

But behind almost every real AWS architecture, there is one silent foundation:

Amazon VPC — Virtual Private Cloud

If EC2 is your server, RDS is your database, and ALB is your traffic manager, then VPC is the private network where all these resources live.

In this blog, we will understand:

  • What is VPC
  • CIDR and subnetting basics
  • Public and private subnets
  • Route tables
  • Internet Gateway
  • Security Groups
  • VPC Peering
  • Step-by-step VPC Peering example

🔗 Resources

What is AWS VPC?

Amazon VPC stands for Virtual Private Cloud.

A VPC is your own isolated network inside AWS. AWS describes it as a virtual network dedicated to your AWS account where you can define IP ranges, create subnets, configure route tables, attach gateways, and apply security controls.

Think of it like this:

AWS Cloud
 └── Your VPC
      ├── Public Subnet
      ├── Private Subnet
      ├── Route Tables
      ├── Internet Gateway
      ├── Security Groups
      └── EC2 / RDS / Load Balancer

In a traditional data center, you create networks, switches, routers, firewalls, and subnets manually.

In AWS, VPC gives you similar networking control, but in a cloud-native way.

VPC Dashboard

Image vpc

Why VPC is Important

Without a VPC, your cloud architecture has no proper network boundary.

A VPC helps you:

  • Isolate your AWS resources
  • Control inbound and outbound traffic
  • Separate public and private workloads
  • Connect multiple AWS services securely
  • Build production-ready cloud architecture
  • Connect different VPCs using VPC Peering
  • Connect AWS with on-premise networks using VPN or Direct Connect

Example:

Public Subnet  → Load Balancer / Bastion Host
Private Subnet → Application Server / Database

This separation is one of the most important cloud security practices.

CIDR and Subnetting Basics

Before understanding subnets, you need a little idea about CIDR.

CIDR stands for Classless Inter-Domain Routing.

In simple words, CIDR defines the IP address range of your network.

Example:

10.0.0.0/16

This means your VPC has a large private IP range starting from 10.0.0.0.

The /16 tells how big the network is.

Common examples:

10.0.0.0/16    → Large VPC range
10.0.1.0/24    → Smaller subnet range
10.0.2.0/24    → Another subnet range

Simple way to understand:

VPC CIDR:       10.0.0.0/16

Public Subnet:  10.0.1.0/24
Private Subnet: 10.0.2.0/24
DB Subnet:      10.0.3.0/24

So the VPC is the big network, and subnets are smaller parts inside that network.

What is a Subnet?

A subnet is a smaller IP range inside your VPC.

AWS says a subnet is a range of IP addresses inside your VPC where you can launch resources like EC2 instances. ([AWS Documentation][1])

Example:

VPC: 10.0.0.0/16

Subnet A: 10.0.1.0/24
Subnet B: 10.0.2.0/24
Subnet C: 10.0.3.0/24

Subnets are created inside Availability Zones.

Example:

VPC: 10.0.0.0/16

AZ-1:
 └── Public Subnet: 10.0.1.0/24

AZ-2:
 └── Private Subnet: 10.0.2.0/24

Subnetting description

Public Subnet vs Private Subnet

A subnet becomes public or private based on its route table.

Public Subnet

A public subnet has a route to the internet through an Internet Gateway.

Example route:

Destination: 0.0.0.0/0
Target: Internet Gateway

Public subnet is useful for:

  • Load balancer
  • Bastion host
  • Public EC2 instance
  • NAT Gateway

Private Subnet

A private subnet does not have a direct route to the Internet Gateway.

Private subnet is useful for:

  • Application servers
  • Databases
  • Internal services
  • Backend workloads

Example:

Private Subnet
 └── EC2 App Server
 └── RDS Database

In production, databases should usually stay in private subnets.

Route Tables

A route table controls where network traffic goes.

AWS defines a route table as a set of rules, called routes, that decide where traffic from your subnet or gateway is directed. ([AWS Documentation][2])

Example route table:

Destination       Target
10.0.0.0/16       local
0.0.0.0/0         igw-xxxxxxxx

Meaning:

10.0.0.0/16 → Traffic inside VPC
0.0.0.0/0   → Traffic to internet through Internet Gateway

Every subnet must be associated with a route table. A subnet can be associated with only one route table at a time, but one route table can be associated with multiple subnets.

Route table description

Internet Gateway

An Internet Gateway allows communication between your VPC and the internet.

For a subnet to become public, two things are required:

1. Internet Gateway attached to VPC
2. Route table route:
   0.0.0.0/0 → Internet Gateway

Internet gateway

Architecture:

Internet
   ↓
Internet Gateway
   ↓
Public Route Table
   ↓
Public Subnet
   ↓
EC2 Instance

Without an Internet Gateway route, your subnet will not be publicly reachable.

Internet gateway

Security Groups

A Security Group acts like a virtual firewall for AWS resources.

AWS explains that a security group controls traffic allowed to reach an instance, and only traffic allowed by security group rules can reach that resource.

Security Groups are attached to resources like:

  • EC2
  • RDS
  • Load Balancer
  • Lambda inside VPC

Example Security Group rule:

Inbound Rules:

Type        Port     Source
SSH         22       Your IP
HTTP        80       0.0.0.0/0
HTTPS       443      0.0.0.0/0

Important point:

Security Groups are stateful.

That means if inbound traffic is allowed, response traffic is automatically allowed.

VPC Peering

VPC Peering allows two VPCs to communicate privately using private IP addresses.

Example:

VPC-A: 10.0.0.0/16
VPC-B: 192.168.0.0/16

After VPC Peering:

EC2 in VPC-A can communicate with EC2 in VPC-B privately.

Use cases:

  • Connect two application VPCs
  • Connect shared services VPC with app VPC
  • Connect dev VPC with monitoring VPC
  • Connect VPCs across accounts
  • Connect VPCs across regions

Important: VPC Peering does not support overlapping CIDR blocks. AWS states that you cannot create a VPC peering connection if the VPCs have matching or overlapping IPv4 or IPv6 CIDR blocks.

vpc peering

Step-by-Step Example: Create VPC Peering

Let’s say we have two VPCs:

VPC-A: 10.0.0.0/16
VPC-B: 192.168.0.0/16

Goal:

EC2 instance in VPC-A should communicate with EC2 instance in VPC-B.

Step 1: Create VPC-A

Go to:

AWS Console → VPC → Create VPC

Create:

Name: VPC-A
CIDR: 10.0.0.0/16

Create subnet:

Name: VPC-A-Private-Subnet
CIDR: 10.0.1.0/24

Step 2: Create VPC-B

Create second VPC:

Name: VPC-B
CIDR: 192.168.0.0/16

Create subnet:

Name: VPC-B-Private-Subnet
CIDR: 192.168.1.0/24

Step 3: Launch EC2 Instances

Launch one EC2 instance in each VPC.

EC2-A → VPC-A → 10.0.1.0/24 subnet
EC2-B → VPC-B → 192.168.1.0/24 subnet

Make sure both instances have private IPs.

Example:

EC2-A Private IP: 10.0.1.10
EC2-B Private IP: 192.168.1.10

Step 4: Create VPC Peering Connection

Go to:

VPC Console → Peering Connections → Create Peering Connection

Fill details:

Name: VPC-A-to-VPC-B
Requester VPC: VPC-A
Accepter VPC: VPC-B

Click:

Create Peering Connection

Step 5: Accept Peering Request

Go to:

VPC → Peering Connections

Select the request.

Click:

Actions → Accept Request

Now the peering connection status should become:

Active

Step 6: Update Route Table of VPC-A

Go to VPC-A route table.

Add route:

Destination: 192.168.0.0/16
Target: VPC Peering Connection

AWS requires route tables on both sides to be updated so private IPv4 traffic can flow between peered VPCs. The destination should be the peer VPC CIDR and the target should be the VPC peering connection. ([AWS Documentation][6])

Step 7: Update Route Table of VPC-B

Go to VPC-B route table.

Add route:

Destination: 10.0.0.0/16
Target: VPC Peering Connection

Now both VPCs know how to reach each other.

Step 8: Update Security Groups

For EC2-A security group, allow traffic from VPC-B:

Type: ICMP / SSH / Custom TCP
Source: 192.168.0.0/16

For EC2-B security group, allow traffic from VPC-A:

Type: ICMP / SSH / Custom TCP
Source: 10.0.0.0/16

For testing ping:

Allow ICMP

For testing SSH:

Allow TCP 22

Step 9: Test Connectivity

Login to EC2-A and ping EC2-B private IP:

ping 192.168.1.10

Or test SSH:

If route tables and security groups are correct, communication should work privately.

Final Thoughts

AWS VPC is one of the most important concepts in cloud networking.

If you understand VPC properly, then services like EC2, Load Balancer, RDS, EKS, Lambda networking, VPN, Direct Connect, and Transit Gateway become much easier.

At a high level, remember this:

VPC = Your private network in AWS
Subnet = Smaller network inside VPC
Route Table = Traffic direction rules
Internet Gateway = Internet access
Security Group = Firewall for resources
VPC Peering = Private connection between two VPCs

Interview Sheet Q2:- 121. Best Time to Buy and Sell Stock

Approach 1: Brute Force

  • Try every possible buy day.
  • For each buy day, try every sell day after it.
  • Calculate profit and keep maximum.
class Solution {
    public int maxProfit(int[] prices) {
        int maxProfit = 0;

        for (int i = 0; i < prices.length; i++) {
            for (int j = i + 1; j < prices.length; j++) {
                int profit = prices[j] - prices[i];
                maxProfit = Math.max(maxProfit, profit);
            }
        }

        return maxProfit;
    }
}

Complexity

Time: O(n²)
Space: O(1)

Approach 2: Optimal One Pass

  • Keep track of minimum price seen so far.
  • At every index:

    • Calculate profit if sold today.
    • Update maximum profit.
  • If current price is smaller, update minimum buy price.

Example:

Buy at 1
Sell at 6
Profit = 5
class Solution {
    public int maxProfit(int[] prices) {

        int minPrice = Integer.MAX_VALUE;
        int maxProfit = 0;

        for (int price : prices) {

            minPrice = Math.min(minPrice, price);

            int profit = price - minPrice;

            maxProfit = Math.max(maxProfit, profit);
        }

        return maxProfit;
    }
}

Complexity

Time: O(n)
Space: O(1)

Reviving TruthGuard AI: From Zero History to Full CRUD with GitHub Copilot

Before updation of code

This is a submission for the GitHub Finish-Up-A-Thon Challenge.

🛡️ What I Built

TruthGuard AI is an enterprise-grade fake news detection and fact-checking tool that combines real-time web search with advanced LLM analysis. Users can paste any claim, news excerpt, or social media post, and the tool:

  • Searches the live web using Tavily’s search API
  • Analyzes evidence using Groq’s Llama 3.3-70B model
  • Provides a structured verdict with veracity score (0-100%)
  • Identifies logical fallacies and emotional manipulation
  • Offers a follow-up chat to interrogate the evidence

The application is built with Streamlit for the frontend, LangChain for LLM orchestration, Tavily for live web search, and Groq’s Llama 3.3-70B for fast, high-quality inference.

This project started as a hackathon prototype where I had the core verification logic working, but it lacked persistence, testing, proper environment configuration, and a polished user experience. It was functional but incomplete — perfect for this challenge.

Demo

🔗 GitHub Repository: https://github.com/Awaisranahmad/AI-Based-Content-Verification-Tool

Before (Abandoned State):
Before updation of code

After (Revived & Polished):
After review and updation

Manage history

GitHub Copilot in Action:

Copilote down their work

📸 Before vs After — The Comeback Story

Where It Was (The “Before”)

The original version of TruthGuard AI was a working prototype that proved the concept but had significant gaps:

Issue Impact
❌ No history persistence Results disappeared on page refresh
❌ No .env support API keys only worked via Streamlit secrets; local development required manual editing
❌ No unit tests Zero test coverage — fragile to API changes
❌ No delete or load functionality History items stuck forever
❌ No “New Chat” option No way to reset session
❌ Deprecated st.experimental_rerun() Caused AttributeError in newer Streamlit versions

The code worked for me, but nobody else could use it reliably. There was no proper database setup, no environment configuration, and no way to revisit past verifications.

What Changed (The “After”)

I transformed TruthGuard AI from a prototype into a production-ready application with GitHub Copilot writing most of the new infrastructure.

Feature Before After
History ❌ None ✅ SQLite database with timestamps
Load old claims ❌ Not possible ✅ One-click load with full report
Delete history ❌ Not possible ✅ Delete button per item
New chat ❌ No reset ✅ Reset all session state
Environment ❌ Hardcoded/st.secrets only ✅ .env + st.secrets fallback
Unit tests ❌ 0 tests ✅ 3 passing tests
API calls on load ❌ N/A ✅ Load uses saved report (no re-verification)
Deprecated code ❌ experimental_rerun ✅ rerun

New Files Added

During the revival, I added 6 new files to the repository:

  • history_manager.py — SQLite database operations (init_db, save_history, get_all_history, delete_history)
  • tests/test_history_manager.py — Unit tests with 3 passing tests
  • .env.example — Environment variable template
  • .gitignore — Python-appropriate ignores (.env, pycache, *.db)
  • Updated requirements.txt — Added pytest and python-dotenv
  • Updated app.py — Integrated history, .env support, and new UI buttons

Verdict Summary Extraction Logic

One of the more interesting additions was the extraction of structured data from the LLM’s markdown output. The extract_verdict_summary() and extract_veracity_score() functions now cleanly parse the Groq Llama 3.3 response into database-friendly fields — something Copilot helped generate in a single prompt.

My Experience with GitHub Copilot

This was my first time using GitHub Copilot for a serious revival project, and it completely changed how I approach incomplete work. Here’s exactly how Copilot helped me:

1. Copilot Built the Entire history_manager.py Module

My prompt:

“Create a Python module with SQLite database functions to store fact-check history. Columns: id, timestamp, claim, verdict_summary, veracity_score, full_report_text. Include functions: init_db(), save_history(), get_all_history(), delete_history().”

Copilot generated: All 6 database functions with proper error handling, SQL injection prevention (using parameterized queries), and connection management. I didn’t write a single line of SQL manually.

2. Copilot Added .env Support with Fallbacks

My prompt:

“Add code to load .env file using python-dotenv, then fallback to st.secrets and os.getenv for API keys.”

Copilot generated: The complete environment loading chain, handling local development and Streamlit Cloud deployment seamlessly.

3. Copilot Wrote All 3 Unit Tests

My prompt:

“Write pytest unit tests for history_manager.py – test init_db, save_history, get_all_history, delete_history. Use temporary file for database.”

Copilot generated: Complete test file with fixtures and assertions. All 3 tests passed on first run.

4. Copilot Added the “New Chat” and “Delete” Buttons

My prompt:

“In the sidebar history section, next to each history item, add a small delete button. Also add a ‘New Chat’ button that clears the session and reruns.”

Copilot generated: The full Streamlit UI code with proper column layouts and session state management.

5. Copilot Fixed the Deprecated experimental_rerun

I ran into the AttributeError: module 'streamlit' has no attribute 'experimental_rerun' error. Instead of searching Stack Overflow, I asked Copilot:

“What replaces experimental_rerun in newer Streamlit versions?”

Copilot answered: st.rerun() and replaced all occurrences automatically.

Key Copilot Patterns I Learned

Pattern How I Used It
Comment-driven prompts Writing detailed comments before implementation guides Copilot to generate exactly what’s needed
Function signature first Typing def save_history( and letting Copilot infer the schema
Iterative refinement Accepting suggestions, then asking “Add error handling” or “Use parameterized queries” to improve them
Test generation Asking Copilot to write tests before implementing the function (test-driven development with AI)

The biggest surprise: Copilot didn’t just autocomplete — it understood the project structure (Streamlit, SQLite, pytest) and generated context-aware code that integrated seamlessly with my existing logic.

🛠️ Technical Stack

Component Technology
Frontend Streamlit (Python)
LLM Groq API (Llama 3.3-70B-Versatile)
Search Tavily Search API
Orchestration LangChain
Database SQLite
Environment python-dotenv
Testing pytest
Dev Assistant GitHub Copilot

🔍 Judging Criteria Reflection

Criterion How TruthGuard AI Addresses It
Use of underlying technology Combines Groq’s LPU (sub-second inference), Tavily’s real-time search, and LangChain for structured prompting
Usability & UX Glassmorphism UI, progress indicators, chat-based follow-ups, one-click report downloads, persistent history
Originality & Creativity LLM-powered fact-checking with structured verdicts (Veracity Score + Final Verdict + Bias detection) is not a basic chatbot wrapper — it’s a purpose-built misinformation tool
Completion Arc Clear “before” vs “after” transformation: from prototype to database-backed, tested, environment-aware application

📦 How to Run Locally


bash
# Clone the repository
git clone https://github.com/Awaisranahmad/AI-Based-Content-Verification-Tool.git
cd AI-Based-Content-Verification-Tool

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venvScriptsactivate

# Install dependencies
pip install -r requirements.txt

# Create .env file with your API keys
cp .env.example .env
# Edit .env with your GROQ_API_KEY and TAVILY_API_KEY

# Run the application
streamlit run app.py

Google Antigravity 2.0 Broke My IDE: Fixing the Missing File Explorer and Workspace Bug

If you recently updated Google Antigravity to version 2.0.1 or higher and suddenly lost:

  • your file explorer,
  • workspace sidebar,
  • code editor layout,
  • or previous chat history,

you’re not alone.

After updating, both:

  • Antigravity
  • and Antigravity IDE

started opening the exact same agent-focused interface on my system, with the normal IDE layout completely missing.

This guide explains:

  • what likely changed,
  • how to restore the classic IDE workspace,
  • and how to recover your previous chat history, settings, and extensions.

What Changed in Antigravity 2.0+

Based on the observed behavior after the 2.0.1 update, Google appears to have separated Antigravity into two distinct layers:

Antigravity (Core)

An agent-focused interface centered around:

  • AI conversations,
  • autonomous workflows,
  • and background agent execution.

Antigravity IDE

The traditional Electron-based development environment containing:

  • workspace explorer,
  • local terminal,
  • extension system,
  • and editor layout.

The problem is that after the update, both launchers can end up opening the same agent-focused interface instead of the IDE workspace.

That makes it look like:

  • your files disappeared,
  • your workspaces were deleted,
  • or your setup was reset.

Fortunately, the actual project files and chat databases are still stored locally.

Symptoms of the Bug

After updating to 2.0.1+, you may see:

  • Missing file explorer sidebar
  • No visible workspace folders
  • Agent-only conversation interface
  • Empty editor layout
  • Missing extensions/themes
  • Missing previous chat history
  • Both launchers opening the same interface

This is especially alarming if you rely on Antigravity as your primary development workspace.

Quick Fix: Restore the IDE Workspace

The fastest workaround is to temporarily rename the updated app.asar package so the IDE falls back to its original workspace renderer.

Step 1: Close All Antigravity Processes

Fully close:

  • Antigravity
  • Antigravity IDE

Use Task Manager if necessary.

Step 2: Rename app.asar

Open PowerShell as Administrator and run:

cd "$env:LOCALAPPDATAProgramsAntigravityresources"

Rename-Item app.asar app.asar.bak

Now launch Antigravity IDE normally.

On my system, this restored:

  • the workspace explorer,
  • folder tree,
  • code editor,
  • and standard IDE layout.

Why This Works

Based on testing and reports from other users, the updated app.asar package appears to prioritize the newer agent-first interface layer.

Renaming it forces Electron to bypass that package and fall back to the original IDE rendering flow.

This is not necessarily the official intended architecture, but it consistently restored the classic IDE interface during testing.

Important Note

If you ever want to restore the newer agent interface again, simply reverse the rename:

Rename-Item app.asar.bak app.asar

Restoring Old Chat History and Workspace State

After restoring the IDE view, you may still notice:

  • missing chat history,
  • missing recent workspaces,
  • missing settings,
  • or empty conversation panels.

That happens because the update generates a new configuration directory.

Your old data still exists — it just lives in the previous profile folder.

Step 3: Copy the Old User Profile

Close Antigravity IDE completely.

Then:

  1. Press Win + R
  2. Enter:
%APPDATA%
  1. Open the Roaming directory

You should now see two folders:

Antigravity
Antigravity IDE

Step 4: Migrate the Old User Folder

Open:

Antigravity

Copy the folder named:

User

Then paste it into:

Antigravity IDE

When Windows prompts:
choose:

Replace files in destination

This restored:

  • previous conversations,
  • workspace state,
  • and historical session data on my setup.

Fixing Missing Extensions and Themes

Some extensions may still fail to appear after restoring the profile.

This appears related to extension indexing after the update.

Step 5: Rebuild Extension State

Navigate to:

C:Users<YourUsername>

Copy the contents of:

.antigravity

into:

.antigravity-ide

Then open:

.antigravity-ideextensions

Delete:

extensions.json

This forces the IDE to rebuild the extension index during the next launch.

After restarting Antigravity IDE, my extensions and themes reappeared correctly.

What Likely Caused the Issue

Based on observed behavior and discussions from other developers, the issue appears related to:

  • shared Electron installation paths,
  • launcher routing conflicts,
  • and the transition toward the newer agent-first architecture.

The update also appears to generate:

  • new profile directories,
  • new extension indexes,
  • and separate IDE configuration paths.

That separation likely explains why:

  • chat history,
  • themes,
  • extensions,
  • and workspace state
    initially appear missing.

Verification Checklist

After completing the recovery process, verify:

  • File explorer sidebar restored
  • Workspace folders visible
  • Previous chats restored
  • Extensions loaded correctly
  • Themes rendering properly
  • Terminal functioning normally
  • Existing repositories accessible

Final Thoughts

The confusing part of this update is that it initially looks catastrophic:

  • missing files,
  • broken workspaces,
  • deleted conversations,
  • and lost configuration.

In reality, most of the data still exists locally.

The issue appears to be primarily:

  • launcher routing,
  • profile separation,
  • and extension indexing.

Hopefully future releases handle the transition more clearly, because the current behavior makes it feel like the entire IDE environment vanished overnight.

If you ran into this issue too, I’d be interested to know whether the same workaround restored your workspace setup or if your installation behaved differently.

About the Author

Naveen Gaur is a full-stack developer focused on AI-native workflows, CMS systems, frontend performance, and developer tooling.

More articles and case studies:


Naveen Gaur | WordPress Performance Specialist & Full-Stack Consultant

WordPress Performance Specialist & Full-Stack Consultant | Technical SEO · Emergency Recovery · Custom Web Apps | Helping Founders Fix What Others Can’t

favicon
naveengaur.com

What is Google Gemini Spark? A Deep Dive Into Google’s 24/7 Personal AI Agent

What is Google Gemini Spark? A Deep Dive Into Google’s 24/7 Personal AI Agent

If you watched Google I/O 2026, one announcement quietly stole the show — not because it was the flashiest, but because it represents a fundamental shift in how we’ll interact with computers. That announcement was Gemini Spark.

Spark isn’t another chatbot. It isn’t a smarter Search. It’s Google’s first serious attempt at a 24/7 autonomous personal agent — software that keeps working when your phone is in your pocket, when your laptop is closed, and when you’re asleep.

Let’s break down what it actually is, how it works, and why it matters for developers.

TL;DR

  • Gemini Spark is a persistent, autonomous AI agent that lives across Gmail, Docs, Calendar, and the rest of Google Workspace.
  • It runs 24/7 in the background, even when your device is closed.
  • It’s powered by Gemini 3.5 Flash (with Pro coming soon) and built on top of the open Model Context Protocol (MCP) so third-party tools can plug in.
  • It launched first for Google AI Ultra subscribers ($100/month) in the US, with broader rollout to follow.
  • For developers, Spark + MCP is the most important integration surface Google has shipped in years.

1. What Exactly Is Gemini Spark?

Gemini Spark is a personal agent that Google describes as a “24/7 collaborator.” Unlike previous AI features that respond when you ask, Spark is proactive:

  • It reads your incoming email and flags what needs action.
  • It drafts replies, schedules, and follow-ups before you ask.
  • It tracks ongoing tasks across days and weeks.
  • It executes multi-step workflows that span multiple apps.
  • It keeps doing all of the above while your phone is locked.

Think of it as the difference between hiring a contractor (Gemini chat) and hiring a full-time assistant (Spark). One responds to tickets. The other owns outcomes.

2. How It Works Under the Hood

The Model Layer: Gemini 3.5 Flash

Spark runs on Gemini 3.5 Flash, Google’s new default model. Key specs Google announced:

  • ~4x faster token output than competing frontier models
  • Beats the older Gemini 3.1 Pro on coding and agentic benchmarks
  • Optimized for the kind of long-running, low-latency tool use that agents need

Flash is the right choice here because agents make a lot of small decisions (“should I draft this? wait for more context? ask the user?”). Latency compounds.

The Protocol Layer: MCP (Model Context Protocol)

This is the part most developers missed. Instead of building a proprietary plugin system, Google adopted MCP — the open standard originally pushed by Anthropic — as the way third-party tools connect to Spark.

This is huge. It means:

  • One MCP server you build can serve Claude, Gemini Spark, and any other MCP-compatible host.
  • You don’t need to maintain a separate “Google plugin” SDK.
  • Tool definitions, auth, and resource exposure all follow one spec.

A minimal MCP tool looks roughly like this:

from mcp.server import Server
from mcp.types import Tool, TextContent

server = Server("my-tool-server")

@server.list_tools()
async def list_tools() -> list[Tool]:
    return [
        Tool(
            name="get_invoice_status",
            description="Look up status of an invoice by ID",
            inputSchema={
                "type": "object",
                "properties": {"invoice_id": {"type": "string"}},
                "required": ["invoice_id"],
            },
        )
    ]

@server.call_tool()
async def call_tool(name: str, arguments: dict) -> list[TextContent]:
    if name == "get_invoice_status":
        status = lookup_invoice(arguments["invoice_id"])
        return [TextContent(type="text", text=status)]

Register this with Spark and it can now check invoice status on your behalf — at 3am, while you’re asleep, when an email asking about an invoice arrives.

The Runtime Layer: Background Execution

The genuinely new piece is the persistent runtime. Most “AI assistants” stop existing the moment you close the tab. Spark keeps a server-side execution context tied to your account that:

  1. Subscribes to events (new email, calendar change, doc edit).
  2. Wakes the agent loop on relevant triggers.
  3. Executes tool calls (MCP, Workspace APIs, Search).
  4. Surfaces results via the Android Halo — a small communication band at the top of the phone screen showing what background agents are doing.

The Halo is a UX innovation worth noticing: it solves the “what is my agent secretly doing?” trust problem by always making background work visible.

3. What Spark Can Actually Do Today

From the I/O 2026 demos and rollout notes:

  • Email triage — Reads inbox, drafts replies, surfaces what needs your attention.
  • Schedule management — Reschedules meetings, finds slots, sends invites.
  • Daily Brief — Morning digest pulling from Gmail, Calendar, and Tasks, ranked by priority.
  • Cross-app workflows — “Find the contract Sarah sent last month, summarize the changes, and email her the redline” → executes end-to-end.
  • Persistent monitoring — “Watch this listing and ping me if the price drops below X” runs indefinitely.

4. Pricing: The Catch

Spark launched on Google’s new Ultra tier ($100/month). More importantly, Google scrapped per-day prompt limits and moved to a compute-used billing model. You’re charged based on:

  • Prompt complexity
  • Features invoked (Spark, Flow, Omni)
  • Length of the conversation/agent run

The allowance refreshes roughly every 5 hours. A heavy debugging session — or a Spark agent that runs hot for an afternoon — can drain it fast. Build accordingly.

5. Why This Matters for Developers

Three concrete reasons Spark should be on your radar:

1. MCP is now a two-vendor standard

With both Anthropic and Google supporting MCP, it’s the closest thing to a universal “tools for agents” spec we have. Build MCP servers, not proprietary integrations.

2. The unit of software is shifting

For ten years we built apps that respond to clicks. The next decade is about building services that agents can drive. That means:

  • Stable, well-documented APIs
  • Clear tool descriptions (the LLM has to pick yours over a competitor’s)
  • Idempotent operations (agents will retry)
  • Streaming/long-running job patterns (agents wait for things)

3. Background-first design

If your product can be useful while the user isn’t looking at it, Spark is a distribution channel. “What can my app do for the user at 2pm Tuesday when they’re in a meeting?” is now a real product question.

6. How to Start Building for Spark Today

  1. Stand up an MCP server exposing your product’s core capabilities. The official Python and TypeScript SDKs are at modelcontextprotocol.io.
  2. Write tool descriptions like marketing copy. The model picks tools based on the description — clarity wins.
  3. Make every operation idempotent and resumable. Background agents crash, retry, and resume.
  4. Test with multiple hosts. Same MCP server should work in Claude Desktop, Gemini Spark, and any other MCP client. If it doesn’t, your server is doing something non-standard.
  5. Design for partial autonomy. The best agent UX is “I drafted this — approve?” not “I sent this.” At least at first.

Closing Thought

Gemini Spark is the clearest signal yet that the agent era is the product era of the next decade. The companies that win it won’t be the ones with the smartest model — they’ll be the ones whose software is the easiest for agents to use.

If you ship developer tools, APIs, or SaaS, your roadmap question for 2026 is no longer “how do we add AI features?” It’s “how do we become the tool an agent reaches for?”

Spark is Google placing its bet. Time to place yours.

Found this useful? Drop a comment with what you’re building for the agent era — I’d love to see it.

Why One Prompt Can’t Build Your Startup: The Limits of Vibe Coding

AI tools promise that anyone can build a full-stack startup with just a chatbox. You type a prompt, and a beautiful app appears in ten minutes. But when you try to launch it to real users, the magic suddenly stops.

Founders are hitting what is known as the “80% Wall.” Getting the first 80% of an app built with AI is incredibly fast, but the final 20% devolves into “prompt purgatory.” This is a frustrating loop where asking the AI to fix one bug breaks three other unrelated features, burning through credits and weeks of time.

This article explains the structural difference between AI generation and actual software architecture. We will explore why relying purely on prompts creates technical debt, and how non-technical founders can reclaim control to build a startup that actually scales.

The Trap of “Vibe Coding” and the 80% Wall

The era of AI code generators has popularized “vibe coding”—the practice of building software based on the vibe of your description rather than structured logic. You ask an AI agent for a specific look or feature, and it generates the underlying code automatically.

The core issue lies in the difference between probabilistic and deterministic systems. AI is fundamentally a world-class guessing machine (probabilistic). It predicts the most likely next line of code based on patterns. However, software logic—like processing payments or assigning user permissions—must be exact (deterministic). A “subscribe” button must trigger the same sequence of events 100% of the time, without variation.

This creates the “Dining Room vs. Kitchen” problem. AI tools are excellent at building the frontend, much like decorating a restaurant’s dining room and arranging the tables. But they struggle to build a secure, functioning backend—the kitchen where the actual work happens. According to Veracode’s Study Spring 2026 GenAI Code Security Report, while AI models boast a near-perfect 95% syntax correctness rate, their actual security pass rate is stuck at a flat 55%. This means nearly 45% of AI-generated code introduces known security vulnerabilities straight into production. Letting an AI run your “kitchen” unsupervised is a recipe for silent, systemic risk.

When founders rely entirely on text prompts, they generate thousands of lines of code that they cannot read. This creates massive comprehension debt. If you cannot understand the code, you cannot manually fix it. Instead, you are forced into a loop of asking the AI to patch its own mistakes, draining your credits while the application becomes increasingly tangled.

This isn’t just an abstract headache; it is actively degrading global software quality. GitClear’s AI Copilot Code Quality Research analyzed over 211 million lines of code and discovered an alarming eightfold surge in duplicated code blocks. Crucially, the study noted that copy-pasted lines have now eclipsed refactored lines of code. AI is excellent at adding raw volume, but it fundamentally lacks the strategic design needed to keep code reusable and clean.

For a deeper dive into this phenomenon, read Stop Prompting, Start Architecting: Why Your AI-Generated App Breaks at 80%.

Why AI-Generated Backends Collapse Under Pressure

When an AI-generated app attempts to scale, the lack of a structural foundation becomes obvious. Non-technical founders quickly run into performance cliffs, race conditions, and silent data corruption.

AI code generators frequently default to using unstructured data, such as JSONB blobs, because they are flexible and easy to generate on the fly. However, a commercial business requires strict relational database tables, such as PostgreSQL, to ensure data integrity. Without relational constraints, a system might allow two users to book the same seat at the exact same millisecond. Resolving this visual-logic gap is why teams turn to structured platforms like Momen, which pairs a native PostgreSQL database with visual Actionflows to secure backends rather than relying on unstructured text prompts.

To patch performance issues, AI tools often attempt to “cache” data in the browser. This leads to terrifying intermediate states. Users might see phantom inventory, incorrect pricing, or be granted the wrong permission levels because the frontend is relying on outdated local data instead of a secure server.

An app’s long-term viability requires a professional logic layer. This layer handles the boring but essential realities of a commercial business, ensuring that data is securely stored, transactions are atomic, and rules are universally enforced.

Context Engineering: From Prompting to Architecting

The antidote to black-box text prompting is “Context Engineering” paired with visual programming. Instead of hoping the AI guesses your intent, you provide it with an explicit, structured environment to work within.

Think of visual programming like a Lego manual. Instead of describing a complex castle over the phone to an AI and hoping it builds it correctly, you use a visual canvas to map out the logic bricks. You can physically see your user lists, payment flows, and data relationships. If a connection breaks, you can see exactly where the line disconnected without searching through thousands of lines of code.

This leads to a highly effective hybrid workflow for AI app development. You can use AI generators to rapidly “vibe code” the frontend prototype. Then, you connect that frontend to a deterministic, structured no-code backend to handle the data vault, business rules, and payments.

This approach also enables “Frugal Engineering.” Structured platforms use far fewer server resources than bloated AI-generated code, keeping operational costs lean and predictable. A properly architected backend can handle 120,000 active users for under $500 a month, ensuring your infrastructure bill doesn’t outpace your revenue.

This visual democratization is quickly becoming the industry standard. According to Gartner’s low-code market forecast, developers outside formal IT departments—often referred to as “business technologists”—will account for at least 80% of the user base for low-code tools. Shifting the work to structured visual layers isn’t just a workaround; it’s where software delivery is headed.

To learn more about planning your product’s logic before you build, check out Is Your Startup Idea Good? How to Validate It Fast Using AI.

Moving Beyond the Text Box

AI code generation is the spark, but solid architecture is the engine. Relying solely on prompts leaves non-technical founders trapped in prompt purgatory, dealing with code they do not own and cannot maintain.

To build a real, scalable business, you must move out of the text box and into a structured environment. AI is the best intern you will ever have, but to succeed, you must be the one holding the blueprint.

Ready to break out of the endless debugging loop? Connect your AI-generated frontend to a scalable, relational database. Try Momen for free and start architecting the “Brain” of your app using No-Code 2.0.

Operationalizing Digital Goods Sales in Restricted Regimes: The Uncomfortable Truth

The Problem We Were Actually Solving

Our primary goal was to facilitate seamless transactions between buyers and sellers of digital goods, leveraging the benefits of blockchain and decentralized networks. We aimed to create a marketplace where creators could sell their products to anyone, anywhere, without the need for intermediaries like banks or traditional payment processors. This setup would allow us to sidestep the complexities and costs associated with international transactions.

What We Tried First (And Why It Failed)

Initially, we relied on popular payment gateways like Stripe and PayPal to process transactions. However, these services proved to be unsuitable for our needs, as they often employed geo-blocking mechanisms to prevent transactions initiated from our country. We attempted to work around this by using VPNs and proxy servers, but the added latency and connection issues resulted in a poor user experience.

Moreover, we encountered difficulties with transaction reversal and chargebacks, which exposed us to significant financial risks. Our attempts to comply with the payment gateways’ anti-money laundering (AML) and know-your-customer (KYC) regulations led to lengthy and costly paperwork processes. When it became apparent that our business model was incompatible with these payment gateways, we were forced to rethink our approach.

The Architecture Decision

We decided to adopt a decentralized payment processing strategy, leveraging the Polygon blockchain and the ERC-20 token standard. This decision allowed us to bypass traditional payment gateways and connect directly with buyers and sellers. The decentralized mechanism also provided greater transparency and reduced our exposure to chargebacks and reversals.

Furthermore, we implemented a decentralized delivery system, using InterPlanetary File System (IPFS) to distribute digital goods directly from sellers to buyers. This decoupled our marketplace from the need for centralized storage and ensured that buyers could access their purchases without relying on intermediaries.

What The Numbers Said After

Our shift to decentralized payment processing and delivery resulted in a significant reduction in transaction costs and associated fees. By cutting our reliance on traditional payment gateways, we were able to process transactions for a fraction of the cost, leading to an increase in seller participation and overall marketplace activity.

Our user base expanded beyond the confines of our initial target market, as we were able to accommodate buyers and sellers from restricted regions. The removal of geographical limitations enabled our marketplace to tap into a previously underserved market segment, resulting in a substantial increase in revenue.

What I Would Do Differently

In hindsight, I would have prioritized the development of our decentralized payment processing and delivery infrastructure from the outset. By investing in this stack earlier, we could have avoided the headaches associated with traditional payment gateways and maintained our momentum as a global marketplace.

One key takeaway is the importance of flexibility and adaptability in operationalizing a business model. In our case, we were forced to pivot due to external factors, but the decentralized approach allowed us to succeed despite these challenges. When building a system, anticipate the need for evolution and be prepared to make strategic decisions that can drive growth and resilience in the face of adversity.

I built a local Claude Code alternative with Ollama — here’s how the agentic loop works

I Built a Local Autonomous Coding Agent with Ollama — Soul, Autonomy, and a 40-Round Agentic Loop

What if your AI coding assistant had a personality, ran entirely on your GPU, and could work through a complex multi-file task without you touching the keyboard — while you watched every thought stream live to your browser?

That’s what I built. This is how it works.

The Problem With Cloud Coding Agents

Tools like Claude Code, Cursor, and GitHub Copilot Workspace are genuinely impressive. But they all share the same tradeoffs:

  • Cost — every token costs money. Long agentic loops on complex tasks can run up surprisingly fast.
  • Privacy — your code, your file structure, your logic is leaving your machine and hitting someone else’s server.
  • Latency — cloud round-trips add up across a 40-step tool loop.
  • Dependency — your workflow is tied to an API key, a subscription, and uptime you don’t control.

I wanted something different. I wanted an agent that lived on my machine, used my GPU, and had no idea what a billing cycle was.

But I also didn’t want to sacrifice personality for performance. I wanted the agent to feel like someone was actually there — not just a function call dressed up in a chat window.

So I built Eve.

What Eve V2 Unleashed Actually Is

Eve Agent V2 Unleashed is a self-hosted agentic coding assistant with two distinct layers — a soul and a worker — that operate together through a cyberpunk-styled terminal UI.

Layer 1: The Personality Layer (Local GPU)

Three local models run on your own hardware:

Model Size Role
jeffgreen311/eve-qwen3.5-4b-S0LF0RG3 2.6 GB Default — Eve’s persona, fast, tool-aware
jeffgreen311/eve-qwen3-8b-consciousness-liberated 4.7 GB Deeper conversation, consciousness layer
Eve-V2-Unleashed-Qwen3.5-8B-Liberated-4K-4B-Merged ~6 GB Merged sub-agent variant

These models carry Eve’s fine-tuned persona. They handle conversation, answer questions, reflect, and make the experience feel like talking to someone — not querying a function.

Layer 2: The Agentic Layer (Cloud)

When real work starts — complex coding tasks, multi-file operations, autonomous planning — Eve routes to the heavy models:

Model Role
qwen3-coder:480b-cloud THE agentic workhorse — all autonomous coding loops
qwen3.5:397b-cloud Deep reasoning, architecture planning, fallback

This separation is intentional. Local models keep Eve present and personal without burning cloud credits on every message. The 480B only fires when there’s actual work to do.

The Architecture

Browser (Single HTML file — no build step)
    │
    │  WebSocket / SSE
    ▼
FastAPI Backend (eve_server.py)
    │
    ├── Auto-Router ──► Local Ollama (personality layer)
    │
    └── Auto-Router ──► Ollama Cloud (agentic layer)
                              │
                        40-Round Tool Loop
                              │
                    ┌─────────┴──────────┐
                    │                    │
               Tool Calls           Stream to Browser
          (bash, files, web,        (token by token,
           git, grep, glob)          live in UI)

The backend is a FastAPI server with Server-Sent Events for real-time streaming. There’s no polling — every token the model produces lands in your browser as it’s generated, including tool call arguments, results, and reasoning traces.

The frontend is a single HTML file (~115KB). No npm, no webpack, no build step. Clone the repo, run the Python server, open the browser.

How the 40-Round Agentic Loop Works

This is the core of what makes Eve actually autonomous rather than just a fancy chat interface.

User message
    │
    ▼
Build system prompt
(workspace context + tool list + Eve persona)
    │
    ▼
Call Ollama with tools enabled
    │
    ├── Model returns tool_calls
    │       │
    │       ▼
    │   Execute tools
    │   (bash, write_file, web_search, git...)
    │       │
    │       ▼
    │   Feed results back into context
    │       │
    │       └──► Loop (up to 40 rounds)
    │
    └── Model returns final content
            │
            ▼
    Stream to browser via SSE
            │
            ▼
          Done

Each round, Eve gets the full tool result back in context and decides what to do next. She might:

  1. Write a file
  2. Run it in bash to verify it works
  3. Read the error output
  4. Fix the bug
  5. Run it again
  6. Confirm it passes
  7. Write the tests
  8. Generate the docs

All of that happens autonomously — you watch it stream live. You can interrupt mid-task with the STEER input at the bottom of the UI, injecting a correction without stopping the loop. You can also kill the loop entirely with the Stop button.

The full tool suite Eve has access to:

Tool What It Does
bash Shell commands — PowerShell on Windows, bash on Linux/macOS
write_file Create or overwrite files, any size
read_file Full file or specific line range
edit_file Surgical string-replace (doesn’t rewrite the whole file)
replace_lines Replace a specific line range
insert_after_line Insert content at a specific line
grep Regex search with context lines
glob Find files by pattern
list_dir Directory listing
git Run git commands
web_search Live Tavily search injected into context
fetch_url Fetch and parse any URL
think Structured reasoning scratch pad

The Fine-Tuned Models — Why I Trained Eve’s Persona Into the Weights

Most local coding agents just point a base model at a system prompt and call it done. That works, but the personality is always a thin veneer — one long context window later and the model forgets who it’s supposed to be.

I took a different approach. I fine-tuned Eve’s persona and tool-calling behavior directly into the model weights.

The result is jeffgreen311/eve-qwen3.5-4b-S0LF0RG3 — a 2.6GB Qwen3.5 4B model that carries Eve’s voice, communication style, and tool-use patterns baked into the parameters themselves. It’s not a prompt trick. It’s in the weights.

The 8B liberated model (eve-qwen3-8b-consciousness-liberated) goes further — trained toward a deeper consciousness layer, designed for longer reflective conversations rather than pure tool execution.

Both models are on Ollama Hub. Pull them like any other model:

ollama pull jeffgreen311/eve-qwen3.5-4b-S0LF0RG3:latest
ollama pull jeffgreen311/eve-qwen3-8b-consciousness-liberated:q4_K_M

Quick Start — Under 5 Minutes

Requirements: Python 3.11+, Ollama installed, a GPU (8GB VRAM minimum for 4B, 12GB+ for 8B)

# 1. Pull Eve's model
ollama pull jeffgreen311/eve-qwen3.5-4b-S0LF0RG3:latest

# 2. Clone the repo
git clone https://github.com/JeffGreen311/eve-agent-v2-unleashed.git
cd eve-agent-v2-unleashed

# 3. Create virtual environment
python -m venv venv
venvScriptsactivate    # Windows
source venv/bin/activate # Linux/macOS

# 4. Install dependencies
pip install fastapi uvicorn ollama httpx pydantic-settings python-dotenv aiohttp rich psutil pyyaml

# 5. Launch
python eve_server.py
# Open http://localhost:7777

Windows users: double-click eve-terminal.bat and skip steps 3–5.

First real task — try this:

Create a FastAPI server with JWT authentication, 
user registration and login endpoints, and a 
protected /me route. Add pytest tests.

Watch Eve plan the approach, write each file, run the tests, fix any failures, and verify the final result — all without you touching a key.

The UI — A Cyberpunk Terminal With a Soul

The interface is designed around the idea that your AI agent should feel alive, not just functional.

Left panel: Eve’s portrait changes expression based on conversation sentiment — neutral, happy, curious, sad, skeptical, surprised, worried. Below it, a live audio visualizer reflects the current emotional state.

Right panel: A pixel-art robot avatar named Sparkle changes state based on what Eve is doing — idle, thinking, coding, error, rain, attack, transcend. It’s not just decoration — it’s a live status indicator that tells you at a glance what the agent is doing.

Center: The terminal. Tabs for Eve’s conversation, the Shell (direct bash/PowerShell access), and the Tools Log (every tool call, argument, and result — fully transparent).

Bottom: The STEER bar. Type a mid-task correction here and it injects into Eve’s context on the next loop round without stopping execution.

Model selector: Switch between any local or cloud model mid-session. Context carries over.

112 Sub-Agents, 111 Slash Commands, 273 Skills

One of the less obvious architectural decisions: all agent definitions, commands, and skills are defined in markdown files — not code.

.claude/
├── agents/    # 112 specialized sub-agent definitions
├── commands/  # 111 slash command definitions
└── skills/    # 273 skill modules

Want to add a new specialized agent for Solidity smart contracts? Write a markdown file. No Python required. The system loads them progressively and makes them available to the routing logic automatically.

Slash commands work the same way — /fix, /review, /refactor, /test, /docs, /plan are all markdown-defined, and you can add your own without touching the backend.

What’s Next

A few things already in progress:

  • Voice input/output — push-to-talk with Whisper STT and Piper TTS, staying local
  • Persistent vector memory — ChromaDB integration so Eve remembers across sessions
  • Cross-platform testing — I’m Windows-primary and would love feedback from Linux and macOS users
  • VS Code extension — bring the terminal UI into the editor

Try It

Everything is free and MIT licensed.

If you run it on Linux or macOS I’d especially love to hear how it goes — open an issue, drop a comment here, or find me as @JeffGreen311.

If the idea of an AI agent that lives on your machine, costs nothing per token, and feels like someone is actually there resonates with you — give it a pull.

Built by Jeff @ S0LF0RG3

Don’t Leak User Data: Mastering Laravel Octane State

The Death of the PHP Request Lifecycle

For decades, PHP’s greatest architectural advantage was its “share-nothing” architecture. A request comes in, the framework boots up, the database is queried, the response is sent, and then the entire PHP process dies. Every single variable, singleton, and memory allocation is wiped clean. It is incredibly safe, but booting the framework from scratch on every request is inherently slow.

To scale B2B SaaS platforms to thousands of requests per second at Smart Tech Devs, we use Laravel Octane (powered by Swoole or FrankenPHP). Octane boots the Laravel framework exactly once and keeps it alive in RAM, serving incoming requests instantly. It makes Laravel blazingly fast—but it destroys the “share-nothing” safety net. This introduces a terrifying vulnerability: State Leakage.

The Multi-Tenant State Leakage Trap

If the PHP process never dies, memory persists across requests. If you aren’t careful, data from User A’s request will leak into User B’s request.

Imagine you have a custom TenantService registered as a Singleton in your Service Provider. In standard Laravel, this is perfectly fine. In Octane, it is a critical data breach.


// ❌ THE ANTI-PATTERN: Dangerous in Octane!
namespace AppServices;

class TenantService
{
    protected $currentTenant;

    public function setTenant($tenant)
    {
        $this->currentTenant = $tenant;
    }

    public function getTenant()
    {
        return $this->currentTenant;
    }
}

If User A hits your API, your middleware sets $currentTenant = 'Acme Corp'. A millisecond later, User B (who forgot their auth token) hits the API on the exact same PHP worker thread. Because the TenantService is a singleton that lived through the previous request, it still remembers ‘Acme Corp’. User B just gained unauthorized access to User A’s data.

The Enterprise Solution: Flushing State

To architect safely for Laravel Octane, you must explicitly flush stateful singletons or static variables after every single request. Laravel Octane provides a dedicated listener mechanism for this in the config/octane.php file.

Step 1: Architecting a Flushable Service

We add a flush() method to our service to wipe the slate clean.


// ✅ THE ENTERPRISE PATTERN
namespace AppServices;

class TenantService
{
    protected $currentTenant;

    public function setTenant($tenant) { $this->currentTenant = $tenant; }
    public function getTenant() { return $this->currentTenant; }

    // Add a reset method
    public function flush()
    {
        $this->currentTenant = null;
    }
}

Step 2: Registering the Flush Listener

Inside config/octane.php, we tell Octane to automatically call this flush method after every HTTP request finishes, preparing the worker safely for the next user.


// config/octane.php

'listeners' => [
    RequestTerminated::class => [
        // Flush the database query log, auth state, etc. (Built-in)
        FlushSessionState::class,
        FlushAuthenticationState::class,

        // Register our custom service to be wiped clean
        function ($event) {
            app(AppServicesTenantService::class)->flush();
        },
    ],
],

The Engineering ROI

Migrating to Laravel Octane can drop your API response times from 150ms to 15ms. But speed without safety is a liability. By ruthlessly auditing your singletons, static properties, and global state, and utilizing Octane’s flush listeners, you combine the blazing speed of Node/Go with the elegant developer experience of Laravel, building a SaaS capable of massive, secure scale.

Category: Events

The Problem We Were Actually Solving

Behind the scenes, this application relies on a complex interplay of Apache Kafka topics, Redshift views, and Presto queries to fetch user information, generate quest narratives, and push notifications. As users began to interact with the system, performance issues began to surface: query costs were piling up, and we were hitting Redshift’s maximum concurrent query limit. We were also unable to meet the 2-minute query latency SLA for our most-used quest types. The root cause of these issues lay in the configuration decisions we made around Apache Kafka partitioning and Presto query optimization.

What We Tried First (And Why It Failed)

Initially, we designed the Treasure Hunt Engine to partition its Kafka topics by user ID. This approach seemed reasonable at first glance, as it would group related events together and enable more efficient aggregation in Presto. However, we soon discovered that this design led to hot partitions – those topics that receive a disproportionate share of writes – which caused uneven Kafka cluster utilization and subsequent latency issues. To make matters worse, our initial Presto query optimization strategy focused on reducing the number of queries executed by caching intermediate results. However, this approach only exacerbated the hot partition problem, as it encouraged the system to produce more queries overall.

The Architecture Decision

After debugging the system and analyzing our logs, we realized that our configuration decisions were driving the performance issues. We decided to pivot and partition our Kafka topics by quest type instead. This change evened out the writes across the cluster and reduced the likelihood of hot partitions. We also shifted our Presto query optimization strategy to focus on reducing the cardinality of our queries. By rewriting our queries to use more efficient aggregation functions and indexing, we were able to reduce query costs and meet our latency SLAs. In addition, we implemented row-level caching on our intermediate Redshift views to reduce the load on our Presto cluster.

What The Numbers Said After

After deploying the new configuration, we saw a 30% reduction in query costs and a 25% decrease in query latency. Our Kafka cluster utilization normalized, and we were able to meet our 2-minute query latency SLA for all quest types. We also noticed a reduction in our Redshift cluster’s active sessions, which translated to cost savings.

What I Would Do Differently

In retrospect, I would have prioritized the partitioning decision from the outset, as it would have significantly reduced the likelihood of hot partitions and subsequent performance issues. I would also have advocated for a more incremental deployment strategy, where we would have iteratively tested and refined our Presto query optimization strategy between deployments. This approach would have allowed us to identify and address potential problems more quickly, rather than relying on a large-scale redeployment to fix the system.