For the past few weeks, I’ve been playing withgpt-5.6-luna. It isshockinglycapable, fast, and smart. I regularly see it do ~100 tps, and rip around my codebase, email, and knowledge base.
Of course, the biggest thing with luna is thecost. I’ve tried running some fairly complicated research threads, and it’s pretty tough to run up a large bill. Even having it search across thousands of emails, I end up with an API cost in the tens of cents.
With GLM 5.3, we even have a new option at the Pareto frontier.
When doing coding work, I almost always reach for the most expensive and capable models (Fable 5, 5.6 Sol). So it’s been easy to miss the progress the small fast models have made.
One thing a few investors I’ve talked with have mentioned: “It’s weird we’re not seeing more consumer AI companies. Why is that?”
There’s a straightforward answer: token costs.
In the times before AI, the playbook for big consumer apps looked like this…
This roughly describes most of the big consumer companies (Google, Facebook, Snapchat, etc.).1
But what if you want to add AI to your product? Well, now you have some real inference costs on every request! Suddenly the amount of capital required increases dramatically.
A pet eval of mine is to build a daily news site, personalized to me:
With the previous generation of models (Sonnet class), you’d spend ~$1 to get anywhere. Charging $30/mo is untenable for a consumer app. There’s obviously a lot we can optimize here, but if you’re charging what the WSJ or The Economist charges, you’d better be delivering similar value.
But looking at luna, the results are pretty decent, and the average cost is ~$0.10. Now we’re talking!