It started with a simple experiment. I took a group of models, including GLM-5.2, Kimi K2.7 and others, and ran them on the same evaluations. Then I measured what would happen if, for each problem, you somehow knew in advance which models would be useful and how their outputs should be combined.That hypothetical system performed substantially better than any individual model in the pool. Of course, it is not something you can actually deploy because it relies on knowing which decisions were good after seeing the result. Echo is my attempt to recover some of that advantage without having that information in advance.For each request, Echo decides how much computation to allocate, which models should participate, and how their work should be combined. Some prompts may only need a relatively small amount of inference, while others benefit from multiple models working on different parts of the problem.One thing that surprised me while building it was how complementary the models are. A model that is clearly weaker overall can still be extremely useful on particular problems or as part of a combination.On my first evaluation mix, Echo consistently performed better than the best individual model in its pool. It also reached roughly the same aggregate result as Fable, which I used as one of the stronger comparison systems, at around one third of the inference cost.There are still some cases where Echo makes the wrong allocation or combination decision. I’m currently spending a lot of time understanding those failures, as well as testing whether the same approach holds up on coding and agentic tasks where measuring the quality of each decision becomes much harder.I built a chat interface (echo.tracerml.ai) and an OpenAI-compatible API (https://echo.tracerml.ai/docs/api) so the system can be tested outside the evaluation setup.Here is a short/high level video on how it works: https://www.youtube.com/watch?v=lJFJSvOdXhgI wrote up the evaluation methodology, individual model results, costs and current limitations here: https://echo.tracerml.ai/evalI would love for you to try it! Especially if you hit any weird failure cases or places where the allocation looks unintuitive.
That hypothetical system performed substantially better than any individual model in the pool. Of course, it is not something you can actually deploy because it relies on knowing which decisions were good after seeing the result. Echo is my attempt to recover some of that advantage without having that information in advance.For each request, Echo decides how much computation to allocate, which models should participate, and how their work should be combined. Some prompts may only need a relatively small amount of inference, while others benefit from multiple models working on different parts of the problem.One thing that surprised me while building it was how complementary the models are. A model that is clearly weaker overall can still be extremely useful on particular problems or as part of a combination.On my first evaluation mix, Echo consistently performed better than the best individual model in its pool. It also reached roughly the same aggregate result as Fable, which I used as one of the stronger comparison systems, at around one third of the inference cost.There are still some cases where Echo makes the wrong allocation or combination decision. I’m currently spending a lot of time understanding those failures, as well as testing whether the same approach holds up on coding and agentic tasks where measuring the quality of each decision becomes much harder.I built a chat interface (echo.tracerml.ai) and an OpenAI-compatible API (https://echo.tracerml.ai/docs/api) so the system can be tested outside the evaluation setup.Here is a short/high level video on how it works: https://www.youtube.com/watch?v=lJFJSvOdXhgI wrote up the evaluation methodology, individual model results, costs and current limitations here: https://echo.tracerml.ai/evalI would love for you to try it! Especially if you hit any weird failure cases or places where the allocation looks unintuitive.
For each request, Echo decides how much computation to allocate, which models should participate, and how their work should be combined. Some prompts may only need a relatively small amount of inference, while others benefit from multiple models working on different parts of the problem.One thing that surprised me while building it was how complementary the models are. A model that is clearly weaker overall can still be extremely useful on particular problems or as part of a combination.On my first evaluation mix, Echo consistently performed better than the best individual model in its pool. It also reached roughly the same aggregate result as Fable, which I used as one of the stronger comparison systems, at around one third of the inference cost.There are still some cases where Echo makes the wrong allocation or combination decision. I’m currently spending a lot of time understanding those failures, as well as testing whether the same approach holds up on coding and agentic tasks where measuring the quality of each decision becomes much harder.I built a chat interface (echo.tracerml.ai) and an OpenAI-compatible API (https://echo.tracerml.ai/docs/api) so the system can be tested outside the evaluation setup.Here is a short/high level video on how it works: https://www.youtube.com/watch?v=lJFJSvOdXhgI wrote up the evaluation methodology, individual model results, costs and current limitations here: https://echo.tracerml.ai/evalI would love for you to try it! Especially if you hit any weird failure cases or places where the allocation looks unintuitive.
One thing that surprised me while building it was how complementary the models are. A model that is clearly weaker overall can still be extremely useful on particular problems or as part of a combination.On my first evaluation mix, Echo consistently performed better than the best individual model in its pool. It also reached roughly the same aggregate result as Fable, which I used as one of the stronger comparison systems, at around one third of the inference cost.There are still some cases where Echo makes the wrong allocation or combination decision. I’m currently spending a lot of time understanding those failures, as well as testing whether the same approach holds up on coding and agentic tasks where measuring the quality of each decision becomes much harder.I built a chat interface (echo.tracerml.ai) and an OpenAI-compatible API (https://echo.tracerml.ai/docs/api) so the system can be tested outside the evaluation setup.Here is a short/high level video on how it works: https://www.youtube.com/watch?v=lJFJSvOdXhgI wrote up the evaluation methodology, individual model results, costs and current limitations here: https://echo.tracerml.ai/evalI would love for you to try it! Especially if you hit any weird failure cases or places where the allocation looks unintuitive.
On my first evaluation mix, Echo consistently performed better than the best individual model in its pool. It also reached roughly the same aggregate result as Fable, which I used as one of the stronger comparison systems, at around one third of the inference cost.There are still some cases where Echo makes the wrong allocation or combination decision. I’m currently spending a lot of time understanding those failures, as well as testing whether the same approach holds up on coding and agentic tasks where measuring the quality of each decision becomes much harder.I built a chat interface (echo.tracerml.ai) and an OpenAI-compatible API (https://echo.tracerml.ai/docs/api) so the system can be tested outside the evaluation setup.Here is a short/high level video on how it works: https://www.youtube.com/watch?v=lJFJSvOdXhgI wrote up the evaluation methodology, individual model results, costs and current limitations here: https://echo.tracerml.ai/evalI would love for you to try it! Especially if you hit any weird failure cases or places where the allocation looks unintuitive.
There are still some cases where Echo makes the wrong allocation or combination decision. I’m currently spending a lot of time understanding those failures, as well as testing whether the same approach holds up on coding and agentic tasks where measuring the quality of each decision becomes much harder.I built a chat interface (echo.tracerml.ai) and an OpenAI-compatible API (https://echo.tracerml.ai/docs/api) so the system can be tested outside the evaluation setup.Here is a short/high level video on how it works: https://www.youtube.com/watch?v=lJFJSvOdXhgI wrote up the evaluation methodology, individual model results, costs and current limitations here: https://echo.tracerml.ai/evalI would love for you to try it! Especially if you hit any weird failure cases or places where the allocation looks unintuitive.
I built a chat interface (echo.tracerml.ai) and an OpenAI-compatible API (https://echo.tracerml.ai/docs/api) so the system can be tested outside the evaluation setup.Here is a short/high level video on how it works: https://www.youtube.com/watch?v=lJFJSvOdXhgI wrote up the evaluation methodology, individual model results, costs and current limitations here: https://echo.tracerml.ai/evalI would love for you to try it! Especially if you hit any weird failure cases or places where the allocation looks unintuitive.
Here is a short/high level video on how it works: https://www.youtube.com/watch?v=lJFJSvOdXhgI wrote up the evaluation methodology, individual model results, costs and current limitations here: https://echo.tracerml.ai/evalI would love for you to try it! Especially if you hit any weird failure cases or places where the allocation looks unintuitive.
I wrote up the evaluation methodology, individual model results, costs and current limitations here: https://echo.tracerml.ai/evalI would love for you to try it! Especially if you hit any weird failure cases or places where the allocation looks unintuitive.
I would love for you to try it! Especially if you hit any weird failure cases or places where the allocation looks unintuitive.
Anyhow, this kinda reminds me of that quote about architecture: “We replaced our monolith with micro services so that every outage could be more like a murder mystery.”
reply
The benchmarks are here → https://echo.tracerml.ai/eval/They are not good benchmarks but at least they exist.
They are not good benchmarks but at least they exist.
reply
I just wish this were solving an actual problem rather than being a fairly transparent attempt to say something approximating, “Hey VCs, OpenRouter just became a unicorn but I can basically vibe code it”Calling it “Fable-level” feels intellectually lazy / dishonest, but then again, what do you expect when there’s so much money on the table.
Calling it “Fable-level” feels intellectually lazy / dishonest, but then again, what do you expect when there’s so much money on the table.
reply
Eval tests while giving general indicators might not be similar for each use case.
reply
As I understand it, in an MoE model, you essentially have hundreds of smaller sub-models (“experts”) that are good at different tasks, and for every generated token, a single “master” model chooses which ones are most relevant to participate, and you only activate them.
reply
reply
I have often wondered how tools like GHCP choose the best model for the job when set to “auto”.
reply
reply
reply
While an LLM isn’t what you’d traditionally consider a weak learner, the theorems on learning systems clearly point to them being so in this context. The feigned surprise at combining them to yield better results seems disingenuous.Even so, the work to predict which models are best suited for which task, how to delegate, and how to combine their outputs is interesting, especially if you’re placing a cost minimization objective on it. That said, this isn’t too far off from what many AI labs are already doing.
Even so, the work to predict which models are best suited for which task, how to delegate, and how to combine their outputs is interesting, especially if you’re placing a cost minimization objective on it. That said, this isn’t too far off from what many AI labs are already doing.
reply
It’s the natural move that happened after people realized you couldn’t throw away half a century of AI research.Most of these focus on costs. But it is simply the case that the one-shot output did not scale for harder problems on workflows.
Most of these focus on costs. But it is simply the case that the one-shot output did not scale for harder problems on workflows.
reply
reply
reply
reply
reply
reply
reply
reply
