Nowadays, most people seem to over-engineer their RAG stack. They jump straight to embeddings, vector databases, and reranking pipelines. Meanwhile, their users just want to find the doc that says“How to reset my password.”
In engineering, there’s always the right tool for the right problem. In AI Retrieval Systems it’s not different.
Before we dive into recipes, let’s establish when you should use each approach. The key factors are:
1. Data Freshness Requirements- Real-time updates (news, social media) favor approaches with easy re-indexing. Daily or weekly updates work well with hybrid approaches. A stable corpus (monthly or quarterly updates) makes pre-embedding sensible.
2. Corpus Characteristics- High churn (more than 10% changes daily) means you should avoid full pre-embedding. Stable documents work fine with pre-embedding. Long-tail distribution (90% never accessed) means on-the-fly wins.
3. Query Patterns- Keyword-heavy queries should start with full-text search. Semantic or conversational queries benefit from embeddings. Mixed patterns need hybrid approaches.
4. Scale & Performance- Less than 1000 queries per day means simple approaches are sufficient. 1K to 10K queries per day requires selective optimization. More than 10K queries per day justifies full optimization.
5. Team Capabilities- No ML expertise means stay with full-text plus query rewriting. Some ML experience makes hybrid search manageable. Having an ML team available makes advanced approaches viable.
Now, let’s look at the recipe book. Start at the top. Move down only when you have data proving you need to.
Good old BM25. Elasticsearch. Postgres full-text search. The stuff that existed before“embedding”became a verb.
You’re just starting out. Your users write keyword-style queries(”pandas merge dataframe”). Exact matches matter(”invoice #12345”). You want zero ML complexity. Your corpus has proprietary terminology(more on this later).
Zero API costs. Fast (under 10ms). Easy to debug (you can see exactly why a document matched). Surprisingly effective (handles many use cases).No chunking strategy needed– works with full documents.No evaluation complexity– easy to test and validate. No model deprecation risk (BM25 doesn’t change).