Mert

Introduction

“Fluidize your AI subscription quotas and maximize the value of every penny.”

This is the No.73 article in the “One Open Source Project per Day” series. Today, we are exploring Sub2API (also known as CRS2).

With the rise of native AI power-tools like Claude Code and GitHub Copilot, many developers find themselves with multiple AI subscriptions (such as Claude Pro or OpenAI Plus). However, these subscriptions often come with usage limits or idle quotas. How can you consolidate these scattered subscription resources and distribute costs efficiently across different tools and users? Sub2API provides a perfect open-source solution.

It is more than just a simple forwarder; it is a full-featured API proxy platform that handles the entire pipeline — from account management and quota distribution to automated billing and built-in payments. It is particularly well-suited for team sharing, “carpooling” (shared-cost usage), or individual multi-account integration.

What You Will Learn

• The core positioning of Sub2API and the pain points it solves.
• Supported mainstream models and subscription types.
• Core features: Multi-account management, intelligent scheduling, and Token-level billing.
• Rapid deployment methods: Script installation and Docker Compose.
• How to use Sub2API to build your own AI API relay service.

Prerequisites

• Basic understanding of AI model APIs (OpenAI, Claude, Gemini, etc.).
• Fundamental Linux command-line experience.
• Familiarity with Docker or containerized deployment concepts.

Project Background

Project Overview

Sub2API is an AI API gateway platform developed in Go. Its core logic is to “pool” AI subscriptions from various channels (including OAuth-authenticated accounts, Session Keys, or standard API Keys).

With Sub2API, you can:

1. Aggregate Resources: Plug in multiple Claude Pro or OpenAI accounts and output a single, unified standard API.
2. Share Costs: Supports a “carpooling” mechanism with a built-in billing system to charge by usage.
3. Seamless Integration: The generated APIs work flawlessly with native tools like Claude Code, OpenClaw, and more, without complex client-side configuration.

Author/Team

• Maintained by: Wei-Shaw
• Ecosystem: The project enjoys active community support, including a mobile admin console (sub2api-mobile) and other surrounding tools.

Project Data

• 📄 Core Repository: Wei-Shaw/sub2api
• 🛠️ Tech Stack: Go (Gin, Ent), Vue 3, PostgreSQL, Redis
• 📄 License: LGPL-3.0
• 📈 Stats: Over 22k Stars on GitHub (Note: May include historical repository data or high community interest).

Main Features

Core Utility

Sub2API solves the “resource island” problem in AI usage:

Upstream Account Pool (Claude, OpenAI, Gemini)
      ↓ Integration
Sub2API Platform Layer (Auth, Billing, Load Balancing, Session Persistence)
      ↓ Unified Distribution
Downstream Applications (IDEs, Chat clients, Scripts)

Key Features

1. Multi-Account Management: Supports various upstream account types and automatically handles session persistence.
2. Precise Billing: Token-level usage tracking and cost calculation with customizable rates.
3. Smart Scheduling: Supports Sticky Sessions and load balancing to ensure continuity in long conversations.
4. Built-in Payment System: Native support for Alipay, WeChat Pay, Stripe, etc., allowing users to top up autonomously.
5. Concurrency & Rate Limiting: Configure per-user and per-account limits to protect your resources.
6. Admin Dashboard: Provides an intuitive Web UI for real-time monitoring and management.

Quick Start

Method 1: One-Click Script Installation (Recommended)

Recommended for clean Ubuntu/Debian systems:

curl -sSL https://raw.githubusercontent.com/Wei-Shaw/sub2api/main/deploy/install.sh | sudo bash

Notes:

• Requires PostgreSQL 15+ and Redis 7+ to be pre-installed.
• The script installs the binary to /opt/sub2api and creates a systemd service.

Method 2: Docker Compose Deployment

# Create deployment directory
mkdir -p sub2api-deploy && cd sub2api-deploy

# Download and run deployment preparation script
curl -sSL https://raw.githubusercontent.com/Wei-Shaw/sub2api/main/deploy/docker-deploy.sh | bash

# Spin up services
docker compose up -d

Once installed, access the admin dashboard at http://YOUR_SERVER_IP:8080.

Detailed Analysis

Architecture: Why is this more than just a Reverse Proxy?

The design priority of Sub2API is “Account State Management.” Traditional reverse proxy tools (like Nginx) lack the ability to understand application-layer sessions.

1. Sticky Sessions:
   For tools like Claude Code that require context continuity, Sub2API uses the session_id in the Header to lock a request to a specific upstream account, ensuring conversations aren’t interrupted by account switching.

Note: If using Nginx as a reverse proxy, ensure underscores_in_headers on; is enabled to support session headers.

1. Pooling Logic:
   The system abstracts multiple accounts into a single “resource pool.” When one account hits a Rate Limit, the scheduler automatically routes traffic away from it, maximizing uptime.

2. Integrated Ecosystem:
   While most relay tools require an external payment gateway, Sub2API’s built-in integration significantly reduces the operational complexity for small teams or community-led “carpools.”

Technical Stack

• Backend: Go ensures high concurrency handling and ease of deployment.
• Database: PostgreSQL handles complex relationships and billing records.
• Cache: Redis manages rate limiting and real-time state synchronization.

Address and Resources

Official Resources

• 🌟 GitHub: Wei-Shaw/sub2api
• 📖 Documentation: Detailed guides on deployment, payment configuration, and API usage are available in the repository.
• 🖥️ Demo: https://demo.sub2api.org/

Target Audience

• Carpool Leads: Organizers looking to split the cost of Claude/OpenAI Plus.
• Developers / Power Users: Individuals wanting to consolidate multiple account quotas for native CLI tools.
• Internal Enterprise Teams: Teams needing to distribute and audit AI resource usage internally.

Summary and Outlook

Key Takeaways

1. Clear Focus: Specialized in converting subscription-based quotas into standard API services.
2. All-in-One: A closed-loop for management, scheduling, billing, and payments.
3. Deployment Friendly: Multiple options including script-based and Docker-based setups.
4. Reliable Performance: Built on a solid Go/PostgreSQL/Redis foundation suitable for medium-to-large distribution.

One-Sentence Review

Sub2API is currently the most tightly integrated open-source solution combining “resource aggregation” with a “commercial model,” making it a powerful tool for achieving “subscription freedom.”

Visit my homepage for more useful insights and interesting products.

The Problem We Were Actually Solving

In hindsight, we were trying to solve a multi-faceted problem that went beyond just event handling. We needed to ensure that the game state was always up-to-date, even when players were offline or experienced network latency. We also needed to prevent cheating by detecting and penalizing players who tried to manipulate the game state. On top of that, we had to ensure that the system could scale to handle thousands of concurrent players and millions of events per second.

We knew that event-driven architectures were the way to go, but we didn’t fully appreciate the tradeoffs involved in choosing the right event streaming platform.

What We Tried First (And Why It Failed)

Initially, we chose Apache Kafka as the event streaming platform, given its popularity and strong community support. However, we soon ran into issues with Kafka’s built-in limitations, such as high latency and limited topic partitioning capabilities. Our system was consistently experiencing 5-second lag, which compromised the overall gaming experience.

We tried to work around these issues by tweaking Kafka’s configuration parameters, but it was a losing battle. We also suffered from periodic crashes due to Kafka’s inability to handle high-throughput batch processing.

The Architecture Decision

After several weeks of tinkering, we decided to switch to a combination of Apache Pulsar and Redis. We created multiple event streams for different aspects of the game, such as player updates, game state changes, and chat messages. This allowed us to decouple each component and scale them independently, reducing the overall system latency.

We also implemented a Redis caching layer to store the game state, which reduced the load on our database and minimized the time spent retrieving data from the event streams. By using Redis as a caching layer, we were able to increase the time-to-live (TTL) of the cached data to 1 second, allowing us to provide the latest game state to players even when they experienced network latency.

What The Numbers Said After

Our switch to Apache Pulsar and Redis resulted in a significant reduction in system latency, from 5 seconds to under 50ms. Our event streaming platform was able to handle 10 million events per second, and our Redis caching layer was able to cache millions of game state records.

Perhaps more importantly, we saw a significant reduction in crashes and downtime, which was critical for maintaining the reliability of our system.

What I Would Do Differently

In retrospect, I would have taken the time to better understand the tradeoffs involved in choosing an event streaming platform. While Apache Kafka has its strengths, such as fault tolerance and high-throughput batch processing, it’s not the best choice for all applications.

I would have also explored alternative caching solutions, such as Memcached or Amazon ElastiCache, before settling on Redis. Additionally, I would have implemented a more robust testing framework to simulate high-throughput workloads and verify the performance of our event streaming platform under various conditions.

Ultimately, the key takeaway from this experience is that there is no one-size-fits-all solution for event-driven architectures, and careful consideration must be given to the specific requirements and constraints of each application.

At first, I thought AI coding tools were mainly competing on code generation quality.

But after building several AI-assisted projects, I noticed something more important:

The biggest bottleneck is no longer coding itself.

It’s context management.
Once projects grow larger, AI tools start behaving very differently.
What matters now is:

• repo understanding
• multi-file coordination
• remembering dependencies
• preserving architecture consistency
• rollback safety
• workflow orchestration
  In small demos, most tools feel impressive.
  But once my projects passed around 40+ files, the differences became much clearer.
  For example:
  Windsurf helped me move faster during:
• rough prototyping
• brainstorming
• UI iteration
• quick experimentation
  But Codex became much stronger for:
• repo-wide cleanup
• multi-file refactoring
• dependency-aware edits
• context-heavy modifications
  One thing I learned:
  The future AI coding battle probably won’t be:
  “Which model writes code better?”
  It will be:
  “Which system manages large-scale context better?”
  That changes how I evaluate AI tools now.
  I no longer judge them only by:
• raw code output
• benchmark scores
• first impressions
  I pay much more attention to:
• long-session stability
• repo awareness
• workflow continuity
• architecture preservation
  AI coding is slowly becoming less about autocomplete…
  …and more about operating systems for development workflows.

Curious how other developers are experiencing this.

Introduction

A symbolic constant in any programming language is a name — a symbol — that can be used to stand in for a constant — a literal value. Programming languages inherited symbolic constants from mathematics that has many of them grouped by specific field of study. Examples include: π (pi), c (speed of light), e (Euler’s number), G (gravitational constant), h (Plank’s constant), etc. While those exact constants can be defined and used in programs, many programs define program-specific constants. Using constants is better than using magic numbers.

Both C and C++ have acquired multiple ways to specify symbolic constants as their respective languages have evolved over the decades, namely:

1. Macros (via #define).
2. Enumerations.
3. const.
4. constexpr.

Knowing which of the ways to use in a particular case can be quite the conundrum.

Macros

Originally, C only had macros, specifically, object-like macros, e.g.:

#define BUF_SIZE  8192

Macros are adequate, but not good. Why? Macros in general ignore scope, so you typically have to give macros very specific (long) names to avoid collision.

Enumerations

Enumerations in C and C++ are better, especially for declaring a set of related constants. In C++ with enum class, they can even be scoped to avoid collisions; in C, however, they’s still in the global scope.

The other caveat is that they can be constants only for integral values.

const

As I described for C and C++, you can use const for constants, e.g.:

static unsigned const BUF_SIZE = 8192;

char BUF[ BUF_SIZE ];

int main() {
  char local_buf[ BUF_SIZE ];
  // ...
}

In C++, that will compile just fine without warning; in C, it’ll either be accepted with warnings or rejected entirely, especially if you disable language extensions. Why? Because const is a misnomer since it really means immutable, not constant, and C is more picky about it.

While the declaration of BUF might be accepted, the declaration of local_buf will either be considered a variable length array (VLA) (that, as I pointed out, you should probably never use), or rejected since VLAs are an optional feature and not all compilers support them (notably, Microsoft’s C compiler doesn’t).

A common work-around in C (prior to C23, see below) is to (ab)use enum:

enum {
  BUF_SIZE = 8192
};

That is, use a nameless enumeration. The advantage is that enumeration constants really are constant.

constexpr

If you’re using C++11 or later, or C23 or later, there’s constexpr. Unlike const, constexpr really means constant. This is by far the best option for declaring constants:

constexpr unsigned BUF_SIZE = 8192;

Conclusion

To summarize:

• If you’re declaring a set of related, integral constants, use enum (in C) or enum class (in C++).
• Otherwise, use constexpr if you can.
• Otherwise, use const.
• Otherwise, use #define as a last resort.

In Part 1, we built a USC campus assistant by pasting a five-line knowledge base directly into the prompt. That works when “the data” fits in your head. It stops being cute the moment the campus handbook, club docs, and workshop notes all want a seat at the same prompt window.

The fix is retrieval — store the chunks once, and at query time pull only the few that look relevant. That’s what RAG (Retrieval-Augmented Generation) actually means once you strip away the marketing.

This post takes the assistant from Part 1 and bolts on a real retriever, using NVIDIA’s hosted embedding model. No vector database, no LangChain, no abstraction layer. A Python list and NumPy are enough to understand what’s actually happening. Once you’ve seen the moving parts, swapping in pgvector or Pinecone later is a fifteen-minute job.

I’m B Torkian, NVIDIA Developer Champion at USC. Same workshop series, same campus, one more capability added.

What you’re adding

User question → embed query → compare to stored chunks → pick top-k → send only those to the LLM → answer

The model call itself barely changes. The work is in steps 2–4: turn text into vectors, compare vectors, return the closest chunks.

Why the manual approach from Part 1 breaks

In Part 1, the entire knowledge base sat inside the prompt:

campus_info = """
The USC AI Club meets every Thursday at 5 PM...
The USC GPU computing lab is open Monday to Friday...
...
"""

Five lines is fine. But every model has a context window, and every token costs money and latency. You don’t want to paste the entire USC student handbook into every question — most of it is irrelevant to “when does the AI Club meet?”

Retrieval is the answer to “which 3 paragraphs out of 3000 are actually about this question?” You compute that before calling the LLM, then send only the winners.

What an embedding actually is

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Two texts that mean similar things land near each other in vector space. Two texts that mean different things land far apart.

NVIDIA’s nv-embedqa-e5-v5 is an embedding model tuned specifically for question-answer retrieval. It has a quirk worth knowing about up front — it treats queries and passages differently. You tell it which one you’re embedding via an input_type parameter. Getting this wrong is the most common beginner mistake — it still runs, but retrieval quality drops noticeably.

• input_type='passage' → use for the documents you store
• input_type='query' → use for the user’s question at search time

That’s it. Same model, two modes.

Step 1: Set up the client and ask() from Part 1

If you’re continuing from Part 1, you already have these defined and can skip this cell. If you’re starting fresh, paste this in first — everything later builds on it.

%pip install -q openai numpy

import os, getpass
from openai import OpenAI

if not os.getenv('NVIDIA_API_KEY'):
    os.environ['NVIDIA_API_KEY'] = getpass.getpass('Paste your NVIDIA API key (starts with nvapi-): ')

client = OpenAI(
    base_url='https://integrate.api.nvidia.com/v1',
    api_key=os.environ['NVIDIA_API_KEY'],
)

MODEL = 'meta/llama-3.1-8b-instruct'

def ask(system_prompt, user_message):
    response = client.chat.completions.create(
        model=MODEL,
        messages=[
            {'role': 'system', 'content': system_prompt},
            {'role': 'user',   'content': user_message},
        ],
        temperature=0.3,
        max_tokens=400,
    )
    return response.choices[0].message.content

client calls NVIDIA’s API Catalog. ask() is the same chat-completion shape from Part 1. The retriever we’re about to build slots in next to these, not instead of them.

Step 2: Build a small knowledge base and embed it as passages

import numpy as np

EMBED_MODEL = 'nvidia/nv-embedqa-e5-v5'

knowledge_base = [
    {'title': 'USC AI Club meeting',
     'text': 'The USC AI Club meets every Thursday at 5 PM in the engineering building, room 204.'},
    {'title': 'USC GPU lab hours',
     'text': 'The USC GPU computing lab is open Monday to Friday from 10 AM to 6 PM.'},
    {'title': 'NVIDIA Developer Program',
     'text': 'USC students can join the NVIDIA Developer Program for free.'},
    {'title': 'Next USC workshop',
     'text': 'The next USC AI Club workshop will cover Retrieval Augmented Generation (RAG).'},
    {'title': 'USC AI/ML office hours',
     'text': 'Office hours for the USC AI/ML faculty are Tuesdays 2-4 PM.'},
    {'title': 'USC robotics lab',
     'text': 'The USC robotics lab requires safety training before students can use the soldering station.'},
    {'title': 'USC tutoring',
     'text': 'Peer tutoring for introductory Python at USC is available Wednesdays from 1 PM to 3 PM.'},
]

def embed_texts(texts, input_type='passage'):
    response = client.embeddings.create(
        model=EMBED_MODEL,
        input=texts,
        extra_body={'input_type': input_type},
    )
    return [np.array(item.embedding, dtype=np.float32) for item in response.data]

# Embed every chunk once, as a passage. Store the vector alongside the text.
embeddings = embed_texts([item['text'] for item in knowledge_base], input_type='passage')
for item, embedding in zip(knowledge_base, embeddings):
    item['embedding'] = embedding

print(f'Embedded {len(knowledge_base)} chunks. Vector dim:', embeddings[0].shape[0])

Two things to notice:

• The OpenAI Python client doesn’t have a native field for NVIDIA’s input_type, so we pass it through extra_body. That’s the right way to send provider-specific arguments without forking the client.
• We’re storing the embeddings in plain Python dicts. For seven chunks this is fine. For seven thousand, you’d reach for a vector database (and the only thing that changes is where the vectors live; the cosine math is identical).

Step 3: Retrieve the top-k chunks for a question

def cosine_similarity(a, b):
    denominator = np.linalg.norm(a) * np.linalg.norm(b)
    if denominator == 0:
        return 0.0
    return float(np.dot(a, b) / denominator)

def retrieve_context(question, k=3):
    question_embedding = embed_texts([question], input_type='query')[0]

    scored = []
    for item in knowledge_base:
        score = cosine_similarity(question_embedding, item['embedding'])
        scored.append((score, item))

    scored.sort(key=lambda pair: pair[0], reverse=True)
    top_items = [item for score, item in scored[:k]]

    return 'n'.join(f"- {item['text']}" for item in top_items)

Three things are happening here:

1. The question is embedded as a query, not a passage. This is the part beginners trip over. Same model, different mode.
2. Cosine similarity scores how close the question vector is to each stored chunk vector. Numbers near 1.0 mean very similar; numbers near 0 mean unrelated.
3. Top-k picks the highest-scoring chunks. Three is a reasonable default for a tiny knowledge base; tune it for yours.

There is no magic in step 3. A vector database would do the same comparison but use indexing tricks to do it fast at scale.

Step 4: Plug retrieval into the same ask() from Part 1

def ask_with_retrieval(question):
    context = retrieve_context(question)

    system_prompt = f"""You are a USC campus assistant. Answer ONLY using the
context below. If the answer is not in the context, say
"I don't have that information — check with the USC AI Club."

CONTEXT:
{context}
"""

    return ask(system_prompt, question)

for question in [
    'Where does the USC AI Club meet?',
    'When can I get Python tutoring at USC?',
    'What is the wifi password?',
]:
    print(f'Q: {question}')
    print(f'Context:n{retrieve_context(question)}')
    print(f'A: {ask_with_retrieval(question)}n')

Run it. Three things to read carefully:

• The first question retrieves the AI Club chunk and answers from it. Good.
• The second retrieves the tutoring chunk and answers from it. Notice that “Python tutoring” doesn’t appear verbatim in the stored text — the chunk says “introductory Python” — but the embedding model knows those are semantically close. That’s the whole point of vector search over keyword search.
• The wifi question retrieves three chunks anyway (top-k always returns k items), but none of them contain a password. The assistant falls back to the refusal line because the ONLY using the context rule forces it to. That’s the guardrail from Part 1 doing its job — and it’s exactly the bridge into Part 3.

Step 5: What you actually did

You replaced the hand-picked campus_info string from Part 1 with a real retrieval step. The model call is identical, and the system prompt follows the same guardrail pattern — answer only from the provided context, otherwise fall back. The only structural change is that {context} now comes from a function instead of a hardcoded constant.

That swap is the entire mental model behind RAG. Real production systems add chunking strategies, hybrid search, re-ranking, and a vector database — but the spine stays the same: embed once, embed query, compare, pass top-k to the LLM.

In your own work, the seven-line knowledge_base becomes hundreds of paragraphs scraped from PDFs, lecture notes, club Slack archives, Notion pages, or a wiki. The retriever code doesn’t change. The dict-with-vector storage gets replaced by something like pgvector, Qdrant, or Pinecone the moment you outgrow a Python list.

Get the code

Repo: github.com/torkian/nvidia-nim-workshop
One-click Colab: Open the Part 1 notebook — paste the Part 2 cells underneath.

MIT licensed. I run this at USC — fork it, swap the knowledge base for your school, your club, your project, and run it wherever you are.

Previously / next in this series

• Part 1: Build Your First AI App with NVIDIA NIM in 30 Minutes
• Part 3 (next): Add Guardrails So It Doesn’t Lie — a two-layer approach using prompt scope + a tiny verifier call. The fallback line that fired on the wifi question above is the foundation we build on.

The Problem We Were Actually Solving

At first glance, our platform was simply a successful e-commerce solution. But the truth was more complex. We had designed it to bypass local payment gateways, because they were either unreliable or outright forbidden in many countries, including Ghana. Instead, we routed payments through international gateways, which provided us with the scale and speed we needed. However, this made our platform inaccessible to a significant portion of our user base in Ghana. We were trying to solve the problem of scaling payments without considering the needs of the users in countries we were trying to include.

What We Tried First (And Why It Failed)

Initially, we tried to fix this issue by developing a separate plugin for Ghanaian payment gateways, but it was a failure. The plugin was complex to implement, and support was spotty at best. Users in Ghana often encountered a barrage of errors, from “card declined” to “payment failed.” It was clear we needed a different approach. We thought localizing payment options was the answer, but what we didn’t realize was that this would only address the symptoms of a deeper problem, not the root cause.

The Architecture Decision

After weeks of research, we finally made a breakthrough. We decided to integrate our platform with local payment providers directly, removing the need for international gateways altogether. This would not only resolve the issue but also improve our overall performance, as transactions would be processed locally and in real-time. We realized that our system’s design had been driven by the constraints of international payments rather than the needs of our users. By shifting our focus to local providers, we could tap into a more extensive network of payment options and improve our overall user experience.

What The Numbers Said After

After deploying the new architecture, we saw a significant uptick in sales from Ghana. The average transaction time dropped by over 70%, and the overall success rate for payments increased by 25%. These numbers told us that we had made the right decision. Our users in Ghana could now easily purchase digital products using their preferred payment methods. This change not only improved our user experience but also opened up new revenue streams for creators.

What I Would Do Differently

In retrospect, we should have considered the needs of our international users from the outset. We were so focused on scaling payments that we overlooked the consequences of our design choices on users in countries like Ghana. If I were to do it again, I would prioritize user experience and inclusivity much earlier in the development process. This would have saved us months of tinkering with workarounds and potentially prevented the disappointment of our users in Ghana.

A small question can become a big database scan when an AI agent writes the query.

“Show me customers at risk” sounds harmless.

But depending on schema context, the agent might join:

• accounts
• subscriptions
• invoices
• usage events
• support tickets
• notes
• activity logs

Then it may retry when the first query does not answer the question.

For production MCP database servers, row limits should be treated as a safety boundary, not a UI preference.

Useful defaults:

• preview first, not full export
• aggregate before raw rows when possible
• enforced row limits per tool/intent
• page cursors for continuation
• explicit “more rows exist” metadata
• structured refusal for unbounded requests
• audit logs with rows scanned and rows returned

Longer version: Row limits for AI database agents

The model should never summarize 50 preview rows as if it saw the whole database.

The fastest way to leak sensitive data through an AI database agent is to expose columns the model never needed.

Table access is too broad.

A customer table can contain useful business fields and risky fields at the same time:

• account name
• plan
• renewal date
• usage trend
• email
• phone
• private notes
• raw payloads
• billing references
• internal flags

The agent may need the first four. It probably does not need the rest.

For production MCP database access, I would rather expose approved projections than raw tables:

• approved views for common business questions
• masked/pseudonymized fields by default
• aggregate-only modes for trend analysis
• structured refusal for blocked columns
• audit logs showing exactly which columns were returned

Longer version: Column-level permissions for AI database agents

The model should not be the thing deciding whether a sensitive field is safe to see.