Most AI tools for creators have the memory of a goldfish. You tell them your niche, your audience, your style — and next session, they ask again. I got tired of that loop, so I built ContentPilot: a content strategy agent that retains everything a creator shares, detects what actually works from their performance data, and adapts over time. The uncomfortable part? Once it learns enough about you, its recommendations are better than the generic ones you’d get from a human strategist who met you yesterday.
This article is the story of how I wired persistent agent memory and intelligent model routing into a single FastAPI process, what design decisions made the system actually useful, and why the most interesting feature turned out to be one I didn’t plan.
How it hangs together
ContentPilot is a monolithic Python backend (FastAPI) with a React frontend. The core loop is simple: a creator talks to the agent, the agent recalls relevant memories, generates a response using the right LLM for the task, then extracts and stores any new facts before responding.
Under the hood, three systems do the real work:
-
Hindsight — a persistent memory layer from Vectorize that stores, recalls, and synthesizes creator knowledge across sessions. It’s not a vector database I query with embeddings. It’s an opinionated memory system with its own
retain, recall, and reflect primitives.
-
cascadeflow — a runtime routing layer that selects which LLM to use for each call based on task complexity, and tracks cost and latency per request.
-
A 5-stage agent pipeline — classify → recall → synthesize → generate → retain — where each stage has a dedicated model tier and purpose.
There is no database. No PostgreSQL, no SQLite, no Redis. Hindsight is the persistence layer. That was a deliberate decision: the memory system isn’t a feature bolted onto the product — the memory system is the product.
ContentPilot’s architecture — a monolithic FastAPI backend with Hindsight as the only persistence layer and cascadeflow routing across three LLM providers. The frontend talks to the API Router, which feeds into the 5-stage pipeline. cascadeflow and Hindsight sit as sibling services below the pipeline — one choosing the model, the other providing memory. There is no database tier because the memory system serves that role entirely.
The pipeline: five stages, three model tiers
Every user message runs through five stages. This isn’t over-engineering — each stage exists because lumping everything into one LLM call produced inconsistent results and made cost management impossible.
# Stage 1: Classify (cheapest model — ~$0, ~120ms)
# Determines task type, complexity score, and memory topics
classification = classify(message) # llama-3.1-8b-instant
# Stage 2: Recall (no LLM cost — Hindsight API)
memories = recall(message, classification["topics"])
# Stage 3: Synthesize (standard model — ~$0.0003)
# Pattern detection with confidence scores
patterns = synthesize(memories) # llama-3.3-70b
# Stage 4: Generate (routed by cascadeflow)
# Model selected based on complexity score
response = generate(message, memories, patterns) # gpt-4o-mini or claude-sonnet
# Stage 5: Retain (cheapest model again)
# Extract key facts, tag them, store in Hindsight
retain(message, response) # llama-3.1-8b-instant
The classification stage runs on the cheapest model available — Groq’s llama-3.1-8b-instant, which costs effectively nothing and responds in ~120ms. Its only job is to produce a complexity score between 0 and 1 and identify what memory topics to query. That score drives cascadeflow’s routing logic:
def select_model(classification: dict) -> tuple[str, str]:
"""Returns (model_tier, human_readable_reason)."""
task_type = classification["task_type"]
complexity = classification["complexity"]
if complexity < 0.3:
return "standard", f"Low complexity ({complexity:.2f}) — standard model sufficient"
elif complexity < 0.6:
if task_type in ("strategy", "analysis", "performance_review"):
return "enhanced", f"Strategic task at moderate complexity ({complexity:.2f}) — enhanced reasoning needed"
return "standard", f"Non-strategic task at moderate complexity ({complexity:.2f}) — standard sufficient"
elif complexity < 0.8:
return "enhanced", f"High complexity ({complexity:.2f}) — escalating to enhanced model"
else:
return "premium", f"Maximum complexity ({complexity:.2f}) — premium model for best strategic output"
Simple questions (“what should I post today?”) hit Groq at $0.0003. Multi-factor strategy requests (“analyze my last month’s performance and redesign my content mix”) escalate to OpenAI or Anthropic. Every routing decision includes a human-readable reason that gets exposed to the frontend.
Here’s what a typical 12-request session looks like with and without routing:
|
Without routing (all gpt-4o-mini) |
With cascadeflow routing |
| Simple Q&A (8 requests) |
8 × $0.003 = $0.024 |
8 × $0.0003 = $0.0024 |
| Strategy tasks (3 requests) |
3 × $0.003 = $0.009 |
3 × $0.003 = $0.009 |
| Complex strategy (1 request) |
1 × $0.003 = $0.003 |
1 × $0.01 = $0.01 |
| Classify + Retain overhead |
$0 |
24 × $0.0001 = $0.0024 |
| Total |
$0.036 |
$0.014 |
| Quality on complex tasks |
Medium |
Premium (Claude) |
The result: ~61% cost reduction, and the hardest questions get a better model than before. 80% of requests stay on the cheapest tier, and the user sees exactly why the system chose to spend more when it did.
Each user message flows through five stages — classify, recall, synthesize, generate, retain — each with a dedicated model and cost profile. Only one stage (generate) uses a variable-cost model. The rest are fixed-cheap. The classify and retain stages both use the 8b model at essentially zero cost. Recall hits Hindsight with no LLM cost. Synthesize runs pattern detection on the 70b model. Generate is the only stage where cascadeflow routes to a more expensive model based on complexity.
The core story: memory that compounds
The part that surprised me was how much the agent’s usefulness compounds over time. Session 1, you get generic advice. By session 5, after feeding it 20 posts of performance data, the agent doesn’t just remember your numbers — it synthesizes patterns with confidence scores.
Here’s how Hindsight’s memory bank gets configured:
self._hindsight_call(
"create_bank",
bank_id=MEMORY_BANK_ID,
name="Content Strategist",
mission=(
"You are a content strategy memory system. Track creator profiles, "
"content performance, audience insights, and strategy preferences. "
"Prioritize actionable patterns: what content formats work best, "
"optimal posting times, audience demographics, and growth trends. "
"Always connect past performance data to future recommendations."
),
disposition={
"skepticism": 2,
"literalism": 3,
"empathy": 4,
},
)
That disposition object is interesting. The Hindsight docs describe it as shaping how the memory system interprets and weighs incoming information. Low skepticism means it trusts the data you feed it. High empathy means it weighs creator context and intent heavily. In practice, this means when a creator says “my audience loves tough love,” Hindsight doesn’t just store that as a string — it influences how future reflect() calls contextualize performance data.
Every conversation triggers a retain cycle. The agent doesn’t store the raw chat — it runs a cheap extraction pass first:
def _extract_key_info(self, user_msg: str, assistant_msg: str) -> str:
extraction_prompt = f"""Extract the key facts worth remembering from this exchange.
Focus on: creator profile info, content preferences, performance data, goals,
audience details, platform preferences, and any specific strategies discussed.
User said: {user_msg}
Assistant responded: {assistant_msg[:500]}
Return ONLY the key facts as a bulleted list. If there are no notable facts, return "NONE"."""
response = self.groq_client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[{"role": "user", "content": extraction_prompt}],
max_tokens=300,
temperature=0.1,
)
return response.choices[0].message.content.strip()
This is a double LLM call pattern — one for the response, one for the memory extraction — and it’s worth the extra $0.0003. Without it, the memory bank fills with conversational noise. With it, it accumulates structured, retrievable facts: engagement rates, content format preferences, audience behavior observations.
The synthesis stage then takes those individual facts and produces confidence-scored patterns:
{
"pattern": "conflict hooks outperform neutral hooks",
"confidence": 0.87,
"supporting_posts": 14,
"contradicting_posts": 2,
"trend": "strengthening"
}
Confidence scores strengthen as more data flows through the retain→synthesize loop. Individual performance metrics get retained with a performance tag. Once 5+ posts accumulate, the synthesizer detects a pattern at 0.65 confidence. As 9 more posts arrive and get re-synthesized, confidence climbs to 0.87. The agent doesn’t just remember — it becomes more certain about what works, and it tells you exactly how certain it is.
Results: same question, different session, different answer
The concrete payoff shows up when you ask the same question twice — once cold, once after the agent has accumulated context.
Session 1 (empty memory):
“What should I post this week?”
Agent: Generic fitness content advice. “Post 3-4 reels per week. Use trending audio. Mix educational and entertaining content.”
Session 5 (20 posts of data accumulated):
“What should I post this week?”
Agent: “Based on 14 posts (confidence: 0.87), your conflict hooks drive 3.2x engagement vs neutral openers. Your audience peaks 6-9 PM EST. Use your power-word vocabulary — DESTROYS, FAILING, STOP — they’re your signature. Avoid context-before-tension openings, which cause a 40% drop at the 15-second mark. Here are 3 specific reel ideas using your top-performing pattern…”
The second response references specific vocabulary, cites confidence scores, and names a retention issue tied to a specific timestamp. None of that was hardcoded. It came from Hindsight’s recall() pulling 8 hook-pattern memories and 3 audience-behavior insights, then the synthesizer turning them into scored patterns that the generator used as context.
Strategy recommendations are tracked as outcomes — validated strategies get higher confidence, and failed ones trigger alternatives. A recommendation (“use contradiction hooks”) is stored via Hindsight’s retain() with a strategy_outcome tag. When the creator posts using that advice and reports results, the outcome (“+38% retention”) is retained with the same tag. Validated outcomes feed back into future recommendations with higher confidence scores, closing the learning loop.
The use case I didn’t anticipate: Creator DNA as an auto-responder
Here’s what caught me off guard. As the memory bank grew, the agent didn’t just know what the creator should post — it knew how they talk.
Think about what happens when a fitness creator or a public speaking coach puts “DM me for help” in their bio. Hundreds of DMs pour in. Some are serious prospects. Some are just curious. Some are tire-kickers. The creator can’t reply to all of them personally, and a generic chatbot sounds nothing like them.
ContentPilot’s memory bank accumulates what I call Creator DNA — tone, vocabulary, pacing, catchphrases, controversy tolerance, storytelling patterns. After enough interactions, the agent can field those inbound DMs in the creator’s voice. It can differentiate a serious prospect from a casual browser, answer FAQs the way the creator would phrase it, and flag high-intent leads for personal follow-up. The creator’s audience gets a response that sounds right because the agent has internalized months of conversation context.
This isn’t a prompt template. The Creator DNA profile is built from observed patterns stored in Hindsight with a creator_dna tag — things like “uses provocative, confrontational tone,” “short punchy sentences,” “favors problem-agitate-solve structure.” When the agent generates a DM response, it recalls that profile and mirrors it. The creator’s followers can’t tell the difference because the context isn’t manufactured — it’s extracted from real interactions.
How does the memory system learn how a person speaks? Through the same retain→recall→reflect cycle that handles everything else. Every time the creator chats with ContentPilot, the extraction pass picks up stylistic signals alongside factual ones. Over 20+ sessions, the vocabulary list grows, the tone profile sharpens, the storytelling patterns emerge. Hindsight’s reflect() synthesizes these into a coherent profile that gets richer each session. The key insight is that voice and style are just another category of memory — they don’t need a separate system. They live in the same bank, tagged differently, and get recalled when the task requires them.
Three-tier graceful degradation
One design decision I’m genuinely proud of: the system works at three levels of availability.
Tier 1: Full stack — Hindsight + Groq + cascadeflow. Full memory, intelligent routing, cost tracking.
Tier 2: Groq only — Hindsight unavailable. The agent stores memories locally in an in-memory list, serves responses without persistent recall, and still functions as a capable strategist.
Tier 3: No API keys at all. The agent returns hardcoded but plausible strategy responses and still retains memories locally for the session.
if not self.groq_client:
fallback_response = self._demo_response(message)
self._retain_memory(
content=f"Demo-mode conversation: creator asked '{message}'.",
context="content strategy session",
)
return {"response": fallback_response, "metrics": {...}}
This isn’t heroic engineering. It’s ~30 lines of fallback paths. But it means the system never shows a blank screen or a stack trace to a creator. The worst case is “slightly less personalized advice” — which is still better than most tools on a good day.
Lessons learned
1. Memory extraction is worth the double-call cost. Storing raw conversations in a memory bank creates noise that degrades recall quality fast. Running a cheap extraction pass ($0.0003 per message) before retaining produces dramatically better recall results a week later. The key is keeping the extraction model’s temperature near zero — you want facts, not creative interpretation.
2. Classification-based routing pays for itself immediately. Before adding the 5-stage pipeline, every request hit the same model. With the classifier, 80% of requests stay on Groq’s cheapest tier. The total cost per session dropped ~61% with no user-perceptible quality difference on simple queries. cascadeflow made this straightforward — the routing logic is ~20 lines of Python, and every decision is traceable.
3. Hindsight’s reflect() does something different from recall(). I initially treated reflect() as a fancier recall. It’s not. recall() retrieves relevant memories. reflect() synthesizes them — it’s the difference between “here are 14 data points about hook performance” and “conflict hooks outperform neutral hooks by 3.2x with increasing confidence.” The synthesize stage in my pipeline exists because I realized reflect() was doing the most valuable work and I wanted to control when it ran.
4. Thread-per-memory-call is ugly but correct for async FastAPI + sync clients. Hindsight’s Python client uses synchronous HTTP under the hood. Calling it inside an async def endpoint blocks Uvicorn’s event loop. I wrapped every Hindsight call in a daemon thread with an 8-second timeout. It’s not elegant, but it hasn’t caused a single production issue, and the alternative — rewriting the client to be async — wasn’t worth the time.
5. The agent’s best feature was emergent. I built ContentPilot to give content strategy advice. The Creator DNA auto-responder capability — fielding DMs in the creator’s voice, qualifying leads, mirroring their tone — wasn’t in any spec. It fell out of the memory system working well enough that stylistic patterns became queryable context. The lesson: if your agent memory is good enough, features you didn’t plan for become possible without additional code.
ContentPilot uses Hindsight for persistent agent memory and cascadeflow for runtime LLM routing. The full source is available on GitHub.
