Fixing iPhone HEVC Videos for Telegram Avatars With ffmpeg

The Bug With No Error Message

I tried to set a short clip as my Telegram profile video. Recorded it on my iPhone, opened Telegram, picked the file, hit save. Telegram took it, showed a spinner for a second, then kept my old photo. No error. No toast. Nothing happened.

I tried three more times. Same silent nothing. The file played fine in every other app, so it wasn’t corrupt.

The problem is the codec. Since iOS 11, iPhones record video in HEVC (H.265) by default. It’s great for storage. It is also not what Telegram wants for a profile video avatar. Telegram’s avatar slot expects H.264, and when it gets something it can’t decode for that slot, it doesn’t tell you. It just drops the upload.

So I wrote a bot that re-encodes the file before it ever reaches that slot. This post is how it works.

What Telegram Actually Wants From a Video Avatar

There is no public error message, so I pieced the spec together by testing files until they stuck. A video avatar that Telegram accepts looks like this:

  • Container: MP4.
  • Video codec: H.264. libx264 is fine.
  • Pixel format: yuv420p. HEVC files are often yuv420p10le (10-bit), which the H.264 avatar path rejects.
  • Shape: square. I use 800×800. Non-square files get center-cropped by Telegram in ways you don’t control.
  • Duration: 10 seconds or less.
  • Size: 2 MB or less.
  • Audio: none. The avatar has no sound, and an audio track sometimes pushes you over the size cap for nothing.
  • faststart: the moov atom belongs at the front of the file so playback can begin before the whole thing downloads.

Miss any one of these and you get the silent drop. The 10-bit pixel format caught me out the longest, because the file looked completely valid.

Fixing It With ffmpeg

ffmpeg does all the real work here. I just call it correctly.

The first step is finding a square crop. iPhone video is 16:9 or 4:3, never square, so I need a 1:1 region. cropdetect scans a few seconds and reports a crop rectangle:

ffmpeg -i input.mov -t 3 -vf cropdetect=24:16:0 -f null - 2>&1 
  | awk '/crop=/ { c=$NF } END { print c }'

That prints something like crop=1080:1080:420:0. It centers on the actual content instead of blindly cropping from a corner.

Then the real encode:

ffmpeg -i input.mov 
  -t 10 
  -vf "crop=1080:1080:420:0,scale=800:800:flags=lanczos,format=yuv420p" 
  -c:v libx264 -profile:v high -preset slow -crf 28 
  -an 
  -movflags +faststart 
  -y output.mp4

Going through the flags that matter:

  • -t 10 hard-caps the clip at 10 seconds.
  • The -vf chain crops to square, scales to 800×800 with the Lanczos filter (sharper than the default bilinear), and forces format=yuv420p so the 10-bit problem disappears.
  • -c:v libx264 is the codec swap. HEVC goes in, H.264 comes out.
  • -crf 28 trades a little quality for size. At 800×800 it looks fine and helps stay under 2 MB.
  • -an drops the audio track completely.
  • -movflags +faststart moves the moov atom to the front.

One encode, every requirement satisfied.

Wiring It Into an aiogram 3 Bot

I wrapped this in a Telegram bot with aiogram 3 so I never have to think about it again. The handler accepts video, animation (GIFs arrive as animation), and raw document uploads:

import asyncio
from pathlib import Path
from aiogram import Bot, Dispatcher, F
from aiogram.types import Message, FSInputFile

dp = Dispatcher()


@dp.message(F.video | F.animation | F.document)
async def handle_video(message: Message, bot: Bot) -> None:
    media = message.video or message.animation or message.document
    if media is None:
        return

    src = Path(f"/tmp/{media.file_unique_id}.src")
    dst = Path(f"/tmp/{media.file_unique_id}.mp4")
    await bot.download(media, destination=src)

    await message.answer("Re-encoding to Telegram avatar format...")
    ok = await convert_to_avatar(src, dst)
    if not ok:
        await message.answer("Couldn't convert that one. 4K and HDR aren't handled yet.")
    else:
        await message.answer_video(FSInputFile(dst))

    src.unlink(missing_ok=True)
    dst.unlink(missing_ok=True)

The conversion runs ffmpeg as a subprocess and checks the output before trusting it:

async def convert_to_avatar(src: Path, dst: Path) -> bool:
    vf = "crop=in_h:in_h,scale=800:800:flags=lanczos,format=yuv420p"
    cmd = [
        "ffmpeg", "-i", str(src), "-t", "10",
        "-vf", vf,
        "-c:v", "libx264", "-preset", "slow", "-crf", "28",
        "-an", "-movflags", "+faststart", "-y", str(dst),
    ]
    proc = await asyncio.create_subprocess_exec(
        *cmd,
        stdout=asyncio.subprocess.DEVNULL,
        stderr=asyncio.subprocess.DEVNULL,
    )
    await proc.wait()
    if proc.returncode != 0 or not dst.exists():
        return False
    return dst.stat().st_size <= 2 * 1024 * 1024

For the bot I use crop=in_h:in_h, a center square based on input height. It skips the separate cropdetect pass and keeps latency low. cropdetect is the better choice when framing matters more than speed.

Packaging It As a Bot Anyone Can Use

Once it worked for me, sharing it was almost free. Send a video to the bot, get back a file that drops straight into the Telegram avatar slot. No app, no settings, no codec knowledge required. It lives here: https://t.me/LiveAvaBot?start=devto_article_20260520

The whole thing is one ffmpeg call behind a message handler. ffmpeg is doing the heavy lifting. I just wrote the wrapper that knows the exact spec Telegram never documents.

Edge Cases And What’s Next

A few things I learned shipping it:

  • Size overshoot. A busy 10-second clip can still land above 2 MB at crf 28. The fix is a second pass with a higher crf, bumping it 4 or 5 at a time until the file fits.
  • Rotation metadata. iPhone videos carry a rotation flag instead of rotating pixels. ffmpeg honors it by default now, but older builds can hand you a sideways avatar.
  • GIFs. They arrive as animation, not video, which is why the handler filter includes both. After encoding they behave like any other clip.
  • 4K and HDR. Not handled yet. HDR needs a tone-mapping step before the H.264 conversion or the colors wash out. That is next on the list.

Telegram silently rejecting valid-looking files is annoying. The fix turned out small once you know what the avatar slot wants: one ffmpeg call behind a message handler.

Built by me, @liveavabot.

Braintrust vs LangSmith: Is $249/mo Worth It? The May 2026 Math

Originally published on NextFuture

This post answers one question: does Braintrust’s $249/month Team plan justify its $150/month premium over LangSmith Plus ($99/month) as of May 2026. If you’re an AI engineer or technical PM shipping a production LLM feature, here’s the math before you click “upgrade.” Below 50,000 traces/month and a team smaller than five, LangSmith Plus wins on price. Above that threshold — and if your team catches even two production regressions per quarter — Braintrust’s $150/month premium pays for itself.

TL;DR: the verdict

WorkloadBraintrust/moLangSmith/moWinnerWhy

Light — solo dev, <5K traces/mo$249$0 (Free tier)LangSmith FreeLangSmith Free covers 5,000 traces/month. Braintrust Team costs $249 for a workload that fits on the free plan.
Medium — team of 5, ~50K traces/mo$249$99 (Plus)LangSmith Plus on price$150/month delta buys richer CI eval and dataset versioning — only worth it if your team prevents ≥2 incidents/quarter.
Heavy — scaling product, 500K+ traces/mo$249$99 (Plus)Braintrust on valueBoth are flat-fee at this scale. Braintrust’s automated regression suite and human-review queue save 2+ engineering hours per incident caught.

Short answer: LangSmith Free wins for solo work; LangSmith Plus wins for budget-constrained teams; Braintrust wins only if you can show it preventing incidents worth more than $150/month in engineering time.

What each one actually costs

Braintrust pricing breakdown

  • Hobby (free): $0/mo — trace limit not published by vendor; use only for solo experiments. Source.

  • Team: $249/mo — unlimited traces, team collaboration, dataset versioning, CI/CD integrations, prompt playground, and human review queue. The feature set that makes CI eval automation practical for a team of 3+. Source.

  • Enterprise: Vendor doesn’t publish this — see footnote. Includes SSO, custom data retention, and SLA guarantees.

Hidden cost: Braintrust’s value is downstream of setup time. Expect 4–6 hours to wire eval harnesses into your CI pipeline and 1–2 weeks before the team writes enough golden datasets to make automated scoring reliable. That’s $400–$600 in engineering time before the tool delivers a verdict.

LangSmith pricing breakdown

  • Free: $0/mo — 5,000 traces/month, one workspace, community support only. At 100 API calls/day that’s 50 days of runway; at 1,000 calls/day it runs out in 5 days. Source.

  • Plus: $99/mo — higher trace volume (exact cap not published in cited source — check vendor pricing page before committing), team workspaces, annotation queues, and dataset management.

  • Enterprise: Vendor doesn’t publish this — contact sales. Private deployment and dedicated support included.

Hidden cost: LangSmith traces every LangChain call by default. Teams not on the LangChain stack need to instrument manually with the LangSmith SDK, adding 1–2 hours per integration. No annual discount is published for Plus.

promptfoo (free alternative)

  • Open Source: $0/mo — self-hosted, unlimited local test runs, no cloud trace storage. Requires you to provision storage, maintain the runner, and build your own team sharing workflow. Source.

promptfoo is the right call for a solo dev or a team willing to trade $99–$249/month for 4–8 hours of ops setup. It does not replace either product’s hosted collaboration or human review queue features.

Break-even, walked through

The pivot workload is the Medium bucket — a team of five shipping one or two AI features, generating roughly 50,000 traces per month. LangSmith Plus costs $99/month at that scale. Braintrust Team costs $249/month. The delta is exactly $150/month, or $1,800/year.

At an average burdened engineering rate of $100/hour, that $150/month buys 1.5 hours of engineering time. To justify the premium, Braintrust must save your team at least 1.5 engineer-hours per month — or prevent 0.75 production incidents per month if each incident costs 2 hours of debugging time.

The inflection point: Braintrust becomes economically justified the moment your team has a documented history of LLM regressions shipping to production. Catch 2 prompt regressions per quarter before they ship (each worth 2 hours of debugging at $100/hr = $400/quarter saved) and the $450/quarter Braintrust premium earns back. If your last three deploys included zero prompt-quality rollbacks, LangSmith Plus at $99/month covers your needs for less money.

Where the cheapest option breaks down

LangSmith Free ($0/month) is the cheapest entry point, but it breaks at 5,000 traces per month. A team running a single AI feature with 200 API calls per day hits that ceiling in 25 days. The moment you need persistent trace history across deployments, annotation queues for human review, or shared datasets with version history — the Free tier stops working and $99/month is the real floor, not $0.

promptfoo (open-source, self-hosted) avoids the $99–$249 monthly cost entirely, but shifts the expense to infrastructure time. Expect 4–8 hours of setup and ongoing maintenance with no hosted collaboration layer. For a team of 5+, that ops burden usually costs more than a year of LangSmith Plus billing — the $99/month fee is not the real floor once you count setup hours.

Pick by your profile

  • Solo dev, side project, <200 API calls/day: LangSmith Free ($0/mo). You stay under the 5,000 trace/month cap with room to spare. Add promptfoo for offline regression tests before deploys.

  • Team of 2–4, one production AI feature: LangSmith Plus ($99/mo). The $150/month Braintrust premium does not pay off until you have enough incidents to measure — and teams this size usually don’t yet.

  • Team of 5–20, multiple AI features in production: Evaluate Braintrust Team ($249/mo) against your incident history. If you had ≥2 prompt regressions ship to prod in the last 90 days, the premium earns back in 4 months.

  • Cost-sensitive batch processing pipeline: promptfoo (open-source, $0/mo). Batch eval jobs run offline on your infra — no per-trace cost, no cloud dependency, no collaboration overhead for a single-owner pipeline.

  • Latency-critical user-facing AI product with human review requirements: Braintrust Team ($249/mo). The human review queue and annotation workflow are not replicated in LangSmith Plus at comparable quality. For products where a wrong AI response affects a real user, this is the argument for paying $150/month more.

FAQ

Is Braintrust actually cheaper than LangSmith?

No — Braintrust Team costs $249/month vs LangSmith Plus at $99/month. Braintrust is $150/month more expensive at the Team tier, though both are flat-fee at scale so the per-trace cost advantage disappears above ~50K traces/month.

How long until switching from LangSmith Plus to Braintrust pays for itself?

At the Medium workload (50K traces/month, team of 5), switching costs roughly 6 hours of migration time plus 5 days of reduced velocity — call it $600 in engineering time at $100/hour burdened rate. The $150/month premium recovers that in 4 months, assuming Braintrust prevents at least 1.5 engineer-hours of incident work per month.

What if my trace volume grows significantly?

Both tools are flat-fee so volume growth alone does not change the math. The question shifts from price to capability: at 500K+ traces/month, you need automated regression scoring and human review queues to keep up — that is where Braintrust’s feature set pulls ahead of LangSmith Plus. At that scale the $150/month delta is noise; the real question is whether either tool’s Enterprise pricing fits your budget. Vendor doesn’t publish Enterprise pricing for either — contact sales for a quote.

Are these prices current as of May 2026?

Pricing pulled from 1 source published on 2026-05-18: “LLM Evaluation in CI: Stop Manual Testing Before It Costs You”. Vendors change pricing without notice — check the Braintrust pricing page and the LangSmith pricing page before committing to either plan.

What about Arize, Langfuse, or Helicone?

Arize was mentioned alongside Braintrust ($249/mo) and LangSmith ($99/mo) as an enterprise-grade option in the same source — but no public pricing was cited, so we cannot run the break-even math. For Langfuse vs Helicone, see our hands-on comparison. For a broader category view, the LLM observability tools breakdown maps the four tool types AI engineers get wrong. If you’re choosing an LLM API stack to instrument, the Coding API Costs in 2026 analysis covers where $3.00 vs $0.50/million tokens actually matters.

Footnote: Braintrust Enterprise and LangSmith Enterprise pricing are not publicly listed by either vendor as of May 2026. Any figures you find on third-party comparison sites are unverified. Contact both vendors directly for a quote before budgeting.

This article was originally published on NextFuture. Follow us for more fullstack & AI engineering content.

ASO Skills: ASO in Your AI Workflow

Appeeky MCP

Originally published on expo.dev/blog
By Erencan Arica

A few years ago, launching a React Native app meant weeks of setup configuring Xcode, managing signing certificates, wrestling with Android Gradle configs, and cobbling together a CI/CD pipeline from scratch. Today, with Expo, a solo developer can go from idea to a production-ready native app in a single afternoon. It’s a dream. And that’s before we even mention LLMs.

But speed creates a new problem: when shipping is fast, the bottleneck shifts. It’s no longer building the app, it’s getting it found.

That’s where App Store Optimization comes in.

Why shipping fast made App Store discovery harder

Here’s the tension: Expo makes is easier for everyone to ship an app and now the App Store has over 2 million apps competing for attention. Paid acquisition costs have risen over 30% year-over-year, making organic discovery more important than ever.

And here’s the stat that changes everything: 65–70% of app downloads still come from App Store search.

Not ads. Not social media. Not influencers. Search.

If your app doesn’t rank for the right keywords, most of your potential users will never see it regardless of how polished the product is. Great apps with poor store listings get buried. Mediocre apps with excellent ASO get downloaded.

What is App Store Optimization (ASO)?

Before going deeper, I want to explain what App Store Optimization (ASO) actually means. ASO is the practice of optimizing your app to rank higher in App Store or Play Store search results tp attract more users to install.

Think of it as SEO, but for the App Store. The principles are similar: understand what your users are searching for, then make sure your listing reflects it clearly and accurately.

There are key elements of ASO:

  • Title (30 chars): Your strongest ranking signal. A core keyword in the title can lift rankings by up to 10%.

  • Subtitle (30 chars, iOS): A secondary ranking signal. Complement your title with different keywords, both fields are indexed together.

  • Keyword Field (100 chars, iOS): Hidden from users, fully indexed by Apple. No spaces after commas, no duplicates from your title, no filler words.

  • Screenshots: No longer just a conversion tool. With Apple’s semantic search, the text in your screenshots is indexed and affects your ranking directly. Users decide in ~7 seconds, so lead with your value prop, use keyword-rich captions, and show real UI.

  • Ratings & Reviews: A direct ranking factor. Prompt at the right moment, after a user achieves something, not on first launch.

  • Localization: Each locale gets its own keyword field, title, and screenshots. Most developers only do English and leave massive markets untapped.

ASO is not a one-time task. Algorithms evolve, competitors iterate, new keywords emerge. The best teams treat it as an ongoing practice, not a launch checklist. It’s best to revisit your listings each month. That’s why I came up with open-source ASO agent skills with real-time data.

ASO Skills: bringing App Store Optimization into your AI workflow

This is the problem I wanted to solve when I built ASO Skills, an open-source collection of AI agent skills for App Store Optimization and mobile app marketing.

Appeeky MCP

The idea is simple: package ASO expertise directly into the tools you already use.

Whether you’re working in Cursor, Claude Code, Codex or any Agent Skills-compatible environment, you can ask your AI assistant to run a full ASO audit, research keywords, analyze competitors, and generate optimized metadata, and get structured, actionable results backed by real App Store data.

The library ships with 20+ skills across three areas. But what makes them powerful isn’t the individual skills, it’s how they work together. Each skill is aware of the others, so instead of running isolated commands, you get a natural, guided workflow:

ASO Flow

aso-audit
   → flags weak spots in your listing
   → recommends keyword-research for gaps

keyword-research
   → finds high-opportunity keywords for your category
   → feeds into metadata-optimization

metadata-optimization
   → generates your title, subtitle, and keyword field
   → ready to ship

You start with an audit, follow the recommendations, and end with a fully optimized listing, all in one conversation with your AI assistant. No dashboards, no spreadsheets, no context switching.

Build, ship, and optimize your app in one workflow

The Expo development workflow is fast, automated, and increasingly AI-native. You can build a polished app, run tests, and ship to production without leaving your editor.

ASO Skills brings the same philosophy to your marketing workflow. You shouldn’t have to context-switch to a separate dashboard, copy-paste keywords into a spreadsheet, or hire a consultant to understand why your app isn’t ranking.

Ask your AI. Get a structured audit. Fix the highest-impact issues. Ship the update with Expo’s services.

The entire loop – build, ship, optimize, iterate – now lives in one environment. Expo already removed the friction from shipping. I hope these skills remove the friction from being found.

Here is the repo: github.com/Eronred/aso-skills

Which Gemma 4 Model Should You Actually Use? I Tested All of Them, So You Don’t Have To

This is a submission for the Gemma 4 Challenge: Write About Gemma 4

Which Gemma 4 Model Should You Actually Use? I Tested All of them, building a Healthcare AI for African Patients

Most model comparison posts are written by people who read the documentation. This one is written by someone who burned through tokens, hit OOM errors, watched Gemini miss clinical signals it should have caught, and eventually landed on the right answer, but only after eliminating everything else first.

I was building Fisibel, a multimodal African synthetic health data platform. The task: upload a real Nigerian hospital record image, extract clinical signals, and generate privacy-safe synthetic datasets grounded in WHO and World Bank statistics. The model had to read handwritten and printed medical documents, understand African-specific clinical patterns, and generate coherent synthetic data at scale.

Here is what I learned by testing every Gemma 4 option available.

Model Parameters Architecture Best For
Gemma 4 E2B 2B effective Dense On-device, mobile, offline
Gemma 4 26B MoE 26B total, 4B active Mixture-of-Experts Edge inference, deep reasoning
Gemma 4 31B 31B total Dense Maximum quality, unconstrained compute

The architectural difference matters more than the number. Let me explain why.

What Mixture-of-Experts Actually Means in Practice
Most explanations of MoE are theoretical. Here is the practical version.
A dense model activates all its parameters for every token it processes. A 31B dense model is doing 31B parameters of work on every single token — whether that token is “the” or “malaria strain variant specific to Lagos State.”
A MoE model routes each token through specialized subnetworks called experts. Only a subset activates per token. Gemma 4 26B MoE has 26B total parameters but activates only ~4B per forward pass.
What this means for you:

Inference speed: closer to a 4B model
Knowledge depth: closer to a 26B model
For specialized tasks: the right experts activate for the right content

That last point is the one that changed my project. When processing clinical documents, MoE routes medical terminology, geographic identifiers, and symptom patterns through subnetworks shaped by that kind of content. A dense model processes a Nigerian hospital record exactly the same way it processes a recipe.

Why I Started With Ollama and Had to Stop
The original Fisibel architecture was local inference via Ollama. If the platform handles real hospital record images, keeping inference local meant zero data leaving the machine. Maximum privacy. No transmission risk.
It did not survive contact with reality.
Downloading Gemma 4 locally through Ollama required pulling a model that was simply too large. In Nigeria, where data costs real money, the download cost alone made the pipeline economically unviable. African healthcare researchers operating in bandwidth-constrained environments cannot build on infrastructure that demands gigabytes just to get started.
Lesson: Local inference is the right privacy instinct. But in bandwidth-constrained environments, the data cost of the initial model download is a real barrier, not an abstract one. If you are building for users outside fibre-connected cities, factor this in before you architect around Ollama.
This pushed the pipeline to Google Cloud — not by preference, but by the economic reality of what local inference actually costs when you are not building from San Francisco.

Why I Rejected Gemma 4 31B
Once on cloud inference, the natural assumption was that bigger meant better. Gemma 4 31B was tested first.
It was not better. It was more expensive for the same output.
For clinical signal extraction specifically — reading a hospital record and pulling out diagnoses, symptoms, treatment patterns, geographic identifiers — the 31B model burned tokens significantly faster without producing meaningfully deeper clinical signals than 26B MoE.
The task does not reward raw parameter count. It rewards specialized reasoning over structured medical text. Paying more tokens for marginal returns on a pipeline that needs to run at scale across African health datasets is not a tradeoff worth making.
When to use 31B: Maximum quality tasks where compute cost is not a constraint. Long-form generation. Complex reasoning chains where you need every parameter engaged. Research environments, not production pipelines running at scale.
When not to use 31B: Specialized extraction tasks. Scale-sensitive pipelines. Anything where you will run the model hundreds or thousands of times.

Why I Tested Gemini and What the Results Actually Showed
Gemini was the obvious alternative. Same multimodal capability, faster inference, Google infrastructure. I tested it directly against Gemma 4 26B MoE on the same Lagos hospital record image.
Gemini was faster. On clinical depth, it lost.
The table below shows clinical signals extracted from the same input document across both models:

Clinical Signal Extracted Gemma 4 26B MoE Gemini
Primary diagnosis ✅ ✅
Comorbidity pattern ✅ ✅
Malaria strain variant ✅ ❌
LGA-specific risk factor ✅ ❌
Regional treatment protocol ✅ ❌
Symptom progression timeline ✅ ✅
Geographic disease indicator ✅ ❌
Age-adjusted risk signal ✅ ✅
Local drug resistance pattern ✅ ❌
Nutritional comorbidity flag ✅ ❌

Gemini caught the general signals. Gemma 4 26B MoE caught the African-specific ones.
No other model in this size class caught all 6 African-specific signals.
This is not a criticism of Gemini. It is a demonstration of what MoE routing does on specialized content. The signals Gemini missed are not obscure — LGA-specific risk factors and regional treatment protocols are standard clinical identifiers in Nigerian medical documents. The difference is in how each model processes domain-specific terminology.
Speed is not always the metric. For specialized extraction tasks, coverage is.
| Use Case | Recommended Model |
| Mobile / on-device / offline | Gemma 4 E2B |
| Specialized domain extraction at scale | Gemma 4 26B MoE |
| Maximum quality, unconstrained compute | Gemma 4 31B |
| Bandwidth-constrained environments | Gemma 4 26B MoE via cloud |
| Research and fine-tuning experiments | Gemma 4 31B |
| Production pipelines with token cost sensitivity | Gemma 4 26B MoE |

Choose Gemma 4 E2B if:

  1. You are building for mobile or on-device
  2. Your users have bandwidth or storage constraints
  3. You need offline capability after the first setup
  4. Your task is classification, routing, or simple generation
  5. You are targeting Android phones under $200

Choose Gemma 4 26B MoE if:

  1. You need reasoning depth at edge-level inference speed
  2. Your task involves specialized domain content — medical, legal, scientific, and regional
  3. You are running at scale, and token cost matters
  4. You need the best quality-to-cost ratio in production
  5. You want MoE expert routing to work in your favour on specialized inputs

Choose Gemma 4 31B if:

  1. Compute cost is not a constraint
  2. You need maximum quality on complex generation tasks
  3. You are doing research, fine-tuning experiments, or one-off analysis
  4. Latency is less important than output quality

The One Thing Nobody Tells You About Thinking Mode
Gemma 4 supports thinking traces. I assumed more thinking meant better answers everywhere. That assumption broke two features before I understood what was actually happening.
Thinking mode helps when the model needs to route across multiple options — tool selection, complex reasoning, and ambiguous queries. It hurts when the task is pure structured output composition. With thinking enabled on a structured generation path, the model spends its entire token budget on internal reasoning traces and produces empty or malformed output.
The rule I now follow: thinking ON for routing, thinking OFF for composition. They are different tasks and the model treats them differently.

What This Means for Open Source AI
Gemma 4 26B MoE running on Google Cloud infrastructure, with no proprietary lock-in, matching or exceeding closed models on specialized domain tasks — that is not a minor footnote. That is the story.
For developers building in bandwidth-constrained environments, for researchers working on underrepresented populations, for anyone who needs frontier-level reasoning without frontier-level costs, the gap between open and closed models just got significantly smaller.
The model I needed to see African clinical signals clearly was not the biggest one. It was not the fastest one. It was the one built to route specialized knowledge to the right place.
That turns out to matter more than size.

I built this while developing Fisibel, a multimodal African synthetic health data platform. The clinical signals table above came from real tests on a Lagos hospital record. All findings are from production pipeline decisions, not benchmarks run for this post.

The AI Bill Is Coming. Here Is the FinOps Playbook to Tame It.

Energy cost visualization showing a GPU server rack equivalent to 160 house icons representing annual electricity consumption, with $2 million annual cost callout for Texas data center power consumption

Energy cost visualization showing a GPU server rack equivalent to 160 house icons representing annual electricity consumption, with $2 million annual cost callout for Texas data center power consumption

AI Infra Economics

Pointing at the cost problem is not a strategy. AI FinOps is how the most disciplined enterprises turn runaway inference bills into operating advantage.

Enterprise AI is past fascination and past experimentation. The third wave is about economics, and economics always wins. The numbers prove it: IDC pegged AI infrastructure spending at $318 billion in 2025, frontier AI margins are sliding toward 50%, and Goldman Sachs projects $1.6 trillion in annual CapEx by 2031. The bill is real. Pointing at the bill is not a strategy. The natural next question for every enterprise leader is straightforward: what do we actually do about it?

The answer is AI FinOps: a discipline that adapts the financial accountability practices the cloud world spent a decade developing to the unique cost dynamics of generative AI. This is not cloud FinOps with a new label. AI introduces cost behaviors that traditional FinOps never managed: token-level metering, agent recursion, model selection trade-offs, prompt-cache economics, and an explicit trade-off between quality and spend at every inference. Treating it like another EC2 line item is exactly how organizations lose control.

Here is the operating playbook.

What AI FinOps Actually Is

AI FinOps gives every AI workload three things most enterprises lack today: visibility, accountability, and a feedback loop to the people whose decisions drive cost.

AI FinOps operational framework showing three connected phases in a cycle: Inform (cost visibility), Optimize (reduce spend through routing and caching), and Operate (enforce governance), illustrating the continuous cost management loop

FinOps Loop

The cloud world’s classic FinOps model has three phases (Inform, Optimize, Operate), and they translate cleanly to AI with important new mechanics inside each phase:

Inform: make AI spending visible per team, product, user, and use case. You cannot optimize what you cannot see, and most enterprises today experience AI cost as a single opaque invoice.
Optimize: **right-size models, route intelligently, cache aggressively, compress prompts, and align purchasing to actual usage patterns.
**Operate:
embed cost into engineering workflows, govern agents at scale, and tie AI spend to business outcomes instead of activity metrics.
Most enterprises fall into one trap. They jump straight to optimize, debating model choice and quantization, before building any visibility into where the money actually goes. That is the equivalent of negotiating cloud discounts before you have tagged your resources.

Phase 1: Make the Spend Visible

You cannot govern what you cannot measure, and AI is unusually hard to measure because the cost lives in tokens, not instances. The first move is to instrument every AI call with the same rigor you apply to cloud resources:

Iceberg diagram showing AI product demo as visible tip above waterline with massive hidden infrastructure stack of GPUs, data centers, and compute resources below, illustrating the true cost of AI infrastructure behind every demo

AI Hidden Stack Cost

  • Tag every inference with team, product, environment, use case, and (where possible) end-user identifier.
  • Meter at the token level, separating input tokens, output tokens, cached tokens, and tool-call overhead.
  • Attribute cost downstream to the business unit consuming the value, not just the team running the gateway.
  • Build a single AI cost dashboard that finance, engineering, and product leaders all see weekly.

The most underrated practice here is showback before chargeback: exposing AI costs to engineering teams before you start billing them internally. Visibility alone changes behavior. Once a team sees that their experiment is burning forty thousand dollars a month on a workload generating no measurable value, they fix it without anyone needing to issue a directive. A useful early target: every AI request in production should be traceable to a cost center within thirty seconds. If that is not true today, that is your first project.

Phase 2: Optimize the Stack

Once visibility exists, optimization delivers most of the immediate savings. The big levers, roughly in order of impact:

Model routing and cascading.

AI model cost optimization pyramid showing four-tier hierarchy from Frontier models (highest cost) through Mid-tier, Small Language and Fine-tuned models, to Cached and Deterministic responses (minimal cost), with traffic percentage and cost scaling axes

AI Model Routing

The highest-leverage optimization is matching model size to task complexity. Most production AI traffic does not need a frontier model. A tiered routing strategy, where a small fast model handles the default case, escalates to a mid-tier model on low confidence, and only invokes a frontier model for genuinely hard cases, often cuts inference cost by 60 to 80 percent with negligible quality impact. The hard part is not the routing logic. It is building the eval harness that tells you when escalation is actually needed.

Aggressive caching: Prompt caching, semantic caching of repeated queries, and KV-cache reuse across multi-turn sessions are some of the cheapest wins available. Most enterprise workloads have surprising redundancy. A well-tuned semantic cache can deflect 20 to 40 percent of inference traffic entirely.

Prompt engineering as cost engineering: Long system prompts, verbose few-shot examples, and unstructured outputs all inflate token bills. Treating prompt design as a cost-quality optimization problem, not a craft, yields meaningful savings: trim instructions, prefer structured outputs to reduce retries, use prompt caching for shared system prompts, and summarize rolling context rather than passing full history.

Right-sizing the model portfolio: Build a deliberate portfolio: frontier API for the hardest five percent of tasks, mid-tier hosted models for general workloads, fine-tuned SLMs (open-weight or distilled) for high-volume narrow tasks, and on-prem or private VPC deployments for sovereign or regulated workloads. A portfolio approach lets enterprises stop overpaying for capability they do not need.

Inference infrastructure optimization: Quantization (4-bit and 8-bit), batched inference, speculative decoding, and distillation to smaller task-specific models are increasingly table stakes for any self-hosted AI workload. These are engineering investments, but the payback periods are typically measured in months, not years.

Commercial leverage: Negotiated capacity, reserved throughput, committed-use discounts, and multi-vendor positioning matter more in AI than in traditional SaaS because the unit economics are tighter. Enterprises with serious volume should be running AI procurement like they run cloud procurement, not like they buy seat-based software.

Cost curve comparison chart showing SaaS unit economics declining over time while AI infrastructure costs rise sharply, with intersecting lines showing the crossover point where AI economics diverge from traditional software margins

AI gets expensive at Scale

Phase 3: Govern at Scale (Especially the Agents)

The next frontier of AI FinOps is agent governance, and most enterprises are completely unprepared for it. Agentic systems break every cost assumption built around chat-style usage. A single user request can trigger dozens of model calls, and a misconfigured agent can burn through five figures of inference before anyone notices.

Agentic AI cost explosion diagram showing one user prompt branching into five autonomous agents, further subdividing into multiple sub-calls and cascading into 100+ token-intensive operations, illustrating runaway inference costs

AI Agents multiply model costs

The governance practices that matter:

  • Token budgets per agent and per workflow, with hard ceilings that fail-stop rather than silently overspend.
  • Max-step and recursion limits so agents cannot enter infinite or near-infinite loops.
  • Approval gates for expensive operations: any agent action over a threshold cost or invoking a frontier model requires a deliberate trigger.
  • Real-time anomaly detection on token consumption, with automated kill switches for runaway behavior.
  • Policy-as-code so cost governance lives in the deployment pipeline, not in a wiki nobody reads.

The mental model shift is treating AI agents like microservices that consume an expensive metered resource. You would not deploy a service that could make unlimited paid API calls with no rate limiting or alerting. The same discipline applies to agents. They just feel novel enough that most teams skip it.

The Real KPI Shift: Cost Per Useful Outcome

Key performance indicator shift diagram showing traditional activity metrics (prompts served, active users, API calls) struck through on left side with arrow pointing to outcome-based metrics (cost per useful result, user impact, business value) on right side

NorthStar for AI FinOps: Outcome metrics (QKR) replace activity metrics (KPI) as the measure that matters.

The most important change AI FinOps drives is not a tool or a dashboard. It is the metric leadership pays attention to. Most AI programs today are measured on activity: prompts served, users active, features shipped. These metrics tell you nothing about whether the spend generates value. Mature AI FinOps replaces them with cost per useful outcome: dollars spent per resolved ticket, per closed sale, per accurate document processed, per deal advanced.

Building this metric takes three things working together: cost attribution from the FinOps stack, outcome definitions from product and business owners, and consistent measurement of both at the same granularity. It is harder than it sounds, and it is the most valuable artifact an AI program can produce. The CFO conversation stops being “how much are we spending on AI?” and becomes “here is what each dollar of AI spend produced this quarter.” That is a conversation enterprises can actually have.

Who Owns AI FinOps?

A working AI FinOps function is cross-functional by design. It typically pulls from:

  • Finance, for unit economics, allocation models, and reporting cadence.
  • Engineering and platform teams, for instrumentation, tagging, routing, and infrastructure optimization.
  • Product, for outcome definitions and prioritization.
  • Security and compliance, for data residency, sovereign deployment, and audit requirements.
  • Procurement, for vendor negotiation and commercial structures.
  • In smaller organizations this might be a single FinOps lead with dotted-line authority. In larger ones, it is typically a formal AI Center of Excellence with FinOps as a core function. Either way, the anti-pattern to avoid is putting AI cost governance entirely inside engineering. Engineers respond to product pressure for capability, not financial pressure for efficiency, and the result is predictable.

The Ninety-Day Starter Playbook For enterprise leaders trying to figure out where to begin, a workable starter sequence:

  1. Centralize all AI traffic through a gateway so you have one place to instrument, tag, and observe.
  2. Establish token-level metering with attribution to team, product, and use case.
  3. Publish a weekly AI cost report to engineering and product leaders. Keep it showback only, not chargeback, for the first quarter.
  4. Identify the top three workloads by spend and run a routing, caching, and prompt-optimization pass on each.
  5. Define one cost-per-outcome metric for the most prominent AI use case in the business.
  6. Set agent governance defaults (token budgets, step limits, anomaly alerts) before any agentic system goes to production.
  7. Stand up a cross-functional AI FinOps working group with finance, engineering, product, and security represented.
    None of these are heroic engineering efforts. They are operational discipline, and they compound.

The Takeaway

The companies that win the next decade of AI will not be the ones with the cleverest demos or the largest model bill. They will be the ones with the tightest feedback loop between AI spend and business value. AI FinOps is not glamorous work. It does not generate keynote slides. But it is what separates the enterprises that ride the AI cost curve into a margin crisis from the ones that turn AI into a durable competitive advantage.

The bill is coming either way. The question is whether your organization will be ready to read it and act on it when it does.

Satish Gopinathan is an AI Strategist, Enterprise Architect, and the voice behind The Pragmatic Architect. Read more at eagleeyethinker.com or Subscribe on LinkedIn.

AIFinOps ,CloudCosts ,EnterpriseAI ,AIEconomics ,EngineeringLeadership ,LLMCosts ,TechLeadership ,CostOptimization ,AI

How Google AI Studio Is Quietly Redefining Developer Workflows

Beyond the Prompt: How Google AI Studio Is Quietly Redefining Developer Workflows
Google I/O has always been a showcase of ambitious ideas, but this year’s announcements around Google AI Studio felt different. Not louder—smarter. While the headlines focused on model sizes, multimodal demos, and the inevitable “AI everywhere” narrative, the real story for developers is subtler: Google AI Studio is evolving from a model playground into a full-stack development platform that reshapes how we build, test, and ship AI‑powered applications.

This essay explores that shift through four lenses:

a hands‑on walkthrough of the new workflow,

a reflection on what the announcements mean for developers,

an opinion on the most underrated update, and

a first‑look guide for getting started with the new features.

  1. A Hands‑On Walkthrough: Building an AI Feature in Minutes, Not Hours
    The new Google AI Studio experience is built around a deceptively simple idea: reduce friction at every step of the development loop. The platform now acts as a unified environment where you can prototype prompts, evaluate model behavior, generate code, and deploy—all without switching tools.

Step 1: Start With a Real Prompt, Not a Blank Screen
When you open a new project, AI Studio now suggests context-aware starter templates based on your goal:

“Build a chatbot”

“Extract structured data”

“Summarize long documents”

“Generate code from natural language”

These aren’t generic examples—they’re tuned to the Gemini models’ strengths and include recommended parameters, safety settings, and evaluation metrics. It’s like having a senior engineer quietly set up your environment before you begin.

Step 2: Test With Real Data, Not Hypothetical Inputs
One of the most practical upgrades is the ability to upload datasets, logs, or user transcripts directly into the prompt testing interface. Instead of crafting synthetic examples, you can evaluate your prompt against actual edge cases.

The platform automatically highlights:

inconsistent outputs,

hallucination risks,

safety violations,

and performance bottlenecks.

This transforms prompt engineering from guesswork into something closer to unit testing.

Step 3: Auto‑Generate Integration Code
Once you’re satisfied with the prompt, AI Studio now generates production-ready code in multiple languages—JavaScript, Python, Dart, and more. The code includes:

API calls,

error handling,

rate‑limit strategies,

and environment variable scaffolding.

It’s not just “example code”—it’s code you can drop directly into your app.

Step 4: Deploy as an API Endpoint
With one click, your prompt becomes a hosted API endpoint with versioning, monitoring, and usage analytics. This is the moment where AI Studio stops being a playground and becomes a platform. You’re no longer exporting prompts—you’re deploying features.

  1. Reflection: What This Year’s Announcements Really Mean for Developers
    The big takeaway from Google I/O wasn’t the models themselves—it was the shift toward developer‑centric tooling.

For years, AI development felt like a series of disconnected steps:

prototype in a notebook,

test in a console,

deploy through a cloud service,

monitor through a separate dashboard.

Google AI Studio collapses that fragmentation. It’s not trying to replace IDEs or cloud platforms—it’s trying to bridge them.

The Real Meaning of This Shift
AI becomes a first-class citizen in the development lifecycle.

Not an add‑on, not a hack, not a “we’ll integrate it later” feature.

Prompt engineering becomes software engineering.

With versioning, testing, and deployment pipelines, prompts are treated like code.

Developers gain leverage.

A single engineer can now prototype, test, and deploy an AI feature in an afternoon.

The barrier to experimentation collapses.

When the cost of trying something new drops to near zero, innovation accelerates.

This is the quiet revolution: not bigger models, but better workflows.

  1. Opinion: The Most Underrated Update—Evaluation Tools
    The flashiest demos always get the spotlight, but the most important update—by far—is the new evaluation and debugging suite.

Why? Because every developer knows the truth:
AI doesn’t fail loudly. It fails subtly.

A model that works 95% of the time is still a model that breaks your product.

The new evaluation tools let you:

run batch tests across dozens or hundreds of inputs,

compare outputs across model versions,

detect regressions,

score responses for accuracy, tone, and safety,

and visualize failure patterns.

This is the missing piece that turns AI from a creative toy into a reliable component.

Why It Matters More Than Any Model Upgrade
Bigger models don’t fix:

inconsistent outputs,

hallucinations,

tone mismatches,

or domain‑specific errors.

Better evaluation does.

This update is the one developers will feel the most six months from now, when they’re maintaining production systems and thanking past‑them for choosing a platform that treats reliability as a first‑class concern.

  1. First‑Look Guide: Getting Started With the New Google AI Studio
    If you’re new to the platform—or returning after a few months—here’s the fastest way to get productive.

Step A: Create a New Project
Projects now act like repositories:

prompts,

datasets,

evaluations,

API endpoints,

and model settings
are all stored together.

Step B: Choose Your Model
Gemini models are now organized by capability:

Gemini Flash for speed and cost efficiency,

Gemini Pro for balanced performance,

Gemini Ultra for complex reasoning and multimodal tasks.

The platform recommends a model based on your use case, which is surprisingly helpful.

Step C: Build Your Prompt
Use the new structured prompt editor:

system instructions,

user input fields,

safety constraints,

and output format templates.

You can now enforce JSON schemas, which eliminates a huge class of downstream parsing errors.

Step D: Test and Evaluate
Upload real data.
Run batch tests.
Compare outputs.
Fix inconsistencies early.

This is where the platform shines.

Step E: Deploy and Integrate
Turn your prompt into an API endpoint.
Copy the generated code.
Add it to your app.

You now have a production-ready AI feature.

Conclusion: A Platform Growing Into Its Identity
Google AI Studio is no longer just a place to “try out” models. It’s becoming a core development environment for AI‑powered software. The platform’s evolution reflects a broader shift in the industry: AI is moving from novelty to infrastructure.

The most exciting part isn’t the models—it’s the workflow.
The most important update isn’t the multimodal demo—it’s the evaluation suite.
The biggest opportunity isn’t in what Google announced—it’s in what developers can now build.

If the last decade was about cloud computing, the next decade will be about AI‑native development environments. And Google AI Studio is quietly positioning itself as one of the first serious contenders.
https://dev.to/dan52242644dan/how-google-ai-studio-is-quietly-redefining-developer-workflows-o50 (dev.to in Bing)

A Platform to Find Web Design Leads & Businesses Without Websites in a Few Clicks

Webleadr lead generation dashboard showing web design leads, web development leads, businesses without websites and more.

Webleadr lead generation dashboard showing web design leads, web development leads, businesses without websites and more.

Recently, we wanted to announce not only the application itself, but also the fact that we have upgraded it to version 2. The application is called Webleadr.

Webleadr is a platform where you can find web design leads and businesses without websites in just a few clicks. On top of that, there are many more features included to make the process of finding and contacting potential clients much easier.

Why We Built Webleadr

The main reason we developed this application was originally for our own agency. We wanted a tool that could help us find potential clients faster and in a more organized way.

But after building it, we also wanted to make it available for other agencies, freelancers, and professionals who could benefit from it as well.

Even though Webleadr is tailored a bit more towards website development, it can still be useful for many other professions. Whether you are a web designer, web developer, SEO specialist, marketer, salesperson, freelancer, or agency owner, the platform can help you discover potential leads more efficiently.

Built for Web Designers, Developers, Agencies, and More

The reason it is now useful for more professions instead of only web designers and web developers is because of the other features included inside the platform. For example, with Webleadr, you can search for your ideal leads from a specific area. Let’s say you want to search in London, and you want to find barbers. You can select the location, choose the business type, and fetch a maximum number of leads, for example 50. After that, you will be taken to a dashboard where you can see up to 50 barbers from the selected area.

Ranked Leads From Best to Worst

These leads are ranked from the best possible leads to the least relevant ones. The best leads are businesses that do not have a website at all. These are usually the strongest opportunities for web design and development services.

The second-best leads are businesses that use third-party platforms instead of having their own dedicated website. For example, some businesses only use a Facebook page, or they might use a third-party appointment booking service that acts like a website, even though it is not a real dedicated website. These businesses are also potential clients.

Website Design Score and Health Score

After that, Webleadr also looks at businesses that already have a website, but where the website has a poor design score. This is another feature of the platform. Webleadr can score the design and styling of a website from 0 to 10. So, for example, if a website looks extremely outdated, it might receive a score around 2, 3, or 4. Thus turning themselves into a potential client.

There are also other useful features, such as a website health score. The platform can detect how long it takes for a website to load, and all of this information is available directly inside the dashboard.

Contact Leads Directly From the Dashboard

You can also contact businesses directly from the platform. You can call them, send them an email if their business email is detected online, or visit their social media links.

Webleadr detects and fetches social media platforms based on the business information and places those links inside the dashboard, so you can reach out more easily.

From an Internal Tool to a Public Platform

We have spent a lot of time developing Webleadr. As mentioned before, it was mainly built for ourselves at first, but now we are making it available for everyone to use.

We are also planning to extend it even further with more features in the future.

Demo Video Available

Thank you for reading this article. Hopefully, you will enjoy using Webleadr and find it useful for your own lead generation process.

There is also a demo video available, which is linked in this article. You can watch it to see how the platform works and decide whether it might be interesting for you as well.

Guide Starting With Web Design Clients

We also published an article on our platform about how to find local web design clients. We emphasize “local” because, as one of our members noted (and we completely agree), starting locally is the best approach, no matter the what time we are in. Check out our full guide for tips on winning local web design clients.

The Concept of Automatic Fallbacks And How Bifrost Implements It

Bifrost

The promise of AI is transformative. The reality is distributed, fragile, and increasingly complex. Production LLM applications need more than just a single provider. They need reliability by design.

If you’ve deployed ML workloads at scale, you know the pain: OpenAI goes down, your app goes down. Anthropic is overloaded, requests queue indefinitely.

This is where Bifrost automatic fallback mechanism comes in. It is a routing layer that transforms single points of failure into resilient, changing request chains.

Bifrost

💻 A small example of work

Traditional LLM integrations look like this:

// Traditional approach - brittle
const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Hello!" }]
});

What happens when OpenAI is down? Your entire application fails. No graceful degradation. No fallback. Just errors.

Even with try-catch, you’re manually writing fallback logic:

let response;
try {
  response = await openai.chat.completions.create({...});
} catch (error) {
  console.log("OpenAI failed, trying Anthropic...");
  response = await anthropic.chat.completions.create({...});
}

This approach has real problems:

  • Boilerplate everywhere: Every API call needs fallback logic
  • Hard to maintain: Adding a third provider means refactoring all your code
  • Inconsistent behavior: Different services handle timeouts differently

⚙️ Declarative Resilience

Bifrost flips this model. Instead of embedding fallback logic in your application, you declare your resilience strategy once, at the Virtual Key level:

# Configure your Virtual Key with multiple providers
curl -X POST https://api.bifrost.example.com/virtual-keys 
  -H "Authorization: Bearer $TOKEN" 
  -d '{
    "name": "vk-prod-main",
    "provider_configs": [
      {
        "provider": "openai",
        "allowed_models": ["gpt-4o", "gpt-4o-mini"],
        "weight": 0.6
      },
      {
        "provider": "anthropic",
        "allowed_models": ["gpt-4o"],
        "weight": 0.4
      }
    ]
  }'

Now your application code is beautifully simple:

// With Bifrost - no fallback logic needed
const response = await fetch('http://localhost:8000/v1/chat/completions', {
  method: 'POST',
  headers: {
    'x-bf-vk': 'vk-prod-main',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: 'Hello!' }]
  })
});

Bifrost handles the rest automatically.

⚙️ How Automatic Fallbacks Work

Here’s the magic: when you configure multiple providers on a Virtual Key, Bifrost creates an automatic fallback chain without any intervention from you.

The Anatomy of a Fallback Chain

Request: gpt-4o
        ↓
   [Load Balancer]
        ↓
   Route to: Anthropic (60% weight) ✓ Primary
        ↓
   ✗ Anthropic failed (timeout/error)
        ↓
   Fall back to: OpenAI (40% weight) ✓ Secondary
        ↓
   ✓ Success! Return response

Key behaviors:

  1. Weighted Selection: Your primary request goes to the provider with the highest weight
  2. Automatic Retry: If that provider fails, Bifrost automatically retries the next provider in line
  3. Weight-Ordered Chain: Fallbacks are sorted by weight providers get priority
  4. Transparent to Application: Your code never sees the fallback happen

A Real-World Example

Imagine this Virtual Key configuration:

{
  "provider_configs": [
    {
      "provider": "openai",
      "allowed_models": ["gpt-4o"],
      "weight": 0.15
    },
    {
      "provider": "anthropic",
      "allowed_models": ["gpt-4o"],  // via Model Catalog wildcard
      "weight": 0.05
    }
  ]
}

Request flow for a gpt-4o query:

Attempt 1: OpenAI (15% weight)
  → Rate limit exceeded

Attempt 2: Anthropic (5% weight)
  → ✓ SUCCESS - Response returned to user

Total latency: ~12 seconds
User sees: Normal response (not an error)

Without Bifrost, that same request would have failed at Attempt 1, never trying the fallbacks.

🔎 Preserving LLM Control

Automatic fallbacks are opt-in by default. If you already have fallback logic in your request, Bifrost respects it and doesn’t add automatic chains:

// With explicit fallbacks - automatic chain is skipped
const response = await fetch('http://localhost:8000/v1/chat/completions', {
  method: 'POST',
  headers: { 'x-bf-vk': 'vk-prod-main' },
  body: JSON.stringify({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: 'Hello!' }],
    fallbacks: ['anthropic/claude-3-sonnet-20240229']  // ← Your custom chain
  })
});

This flexibility is crucial. Sometimes you need specific fallback behavior for compliance, cost, or performance reasons.

⚙️ Combining Automatic Fallbacks with Weighted Load Balancing

The real power emerges when you combine three Bifrost features:

1. Weighted Load Balancing

Distribute traffic proportionally across providers:

  • 70% to cheap OpenAI (cost optimization)
  • 5% to Anthropic (bleeding-edge features)

2. Automatic Fallbacks

If OpenAI fails → try Anthropic

3. API Key Restrictions

Ensure production workloads use production keys only:

{
  "provider_configs": [
    {
      "provider": "openai",
      "key_ids": ["key-prod-001"],  // Only production OpenAI key
      "allowed_models": ["gpt-4o"],
      "weight": 0.8
    }
  ]
}

Together, these features create a governance layer that makes your infrastructure simultaneously:

  • Resilient (automatic failover)
  • Cost-effective (weighted optimization)
  • Compliant (fine-grained access control)
  • Simple (no application-level boilerplate)

💻 Use Cases

Use Case 1

// Production VK - strict, resilient
{
  "name": "vk-prod",
  "provider_configs": [
    { "provider": "openai", "key_ids": ["key-prod"], "weight": 0.6 },
    { "provider": "anthropic", "key_ids": ["key-anthopic-prod"], "weight": 0.4 }
  ]
}

// Development VK - cheaper models
{
  "name": "vk-dev",
  "provider_configs": [
    { "provider": "openai", "key_ids": ["key-dev"], "allowed_models": ["gpt-4o-mini"], "weight": 1.0 }
  ]
}

Use Case 2

{
  "name": "vk-cost-optimized",
  "provider_configs": [
    // Primary: cheapest provider
    { "provider": "openai", "allowed_models": ["gpt-4o-mini"], "weight": 0.85 },
    { "provider": "anthropic", "allowed_models": ["claude-3-sonnet"], "weight": 0.05 }
  ]
}

Use Case 3

{
  "name": "vk-global",
  "provider_configs": [
    // Primary: lowest latency for your region
    { "provider": "anthopic", "key_ids": ["key-anthopic-eu"], "weight": 0.7 },
    // Fallback: global provider
    { "provider": "openai", "weight": 0.3 }
  ]
}

⚙️ Implementation Details You Should Know

How Bifrost Determines Fallback Order

Bifrost sorts fallback providers by weight, descending:

Weight 0.15 (OpenAI) → tried first  
Weight 0.05 (Anthropic) → tried last

This means your “best” provider (highest weight) is also your primary, and your fallbacks gracefully degrade through less-preferred options.

When Automatic Fallbacks DON’T Trigger

Automatic fallback chains are only created if:

  • ✓ Your request has no existing fallbacks array
  • ✓ You have multiple providers configured on the Virtual Key

If you’ve manually specified fallbacks, Bifrost respects your configuration and doesn’t add automatic chains. This prevents surprising behavior for applications that have custom fallback strategies.

Model Validation Across Providers

Bifrost doesn’t blindly route requests. It validates that the requested model is actually supported by the provider:

// ✓ This works - both Anthropic and OpenAI support gpt-4o
curl -H "x-bf-vk: vk-prod-main" 
  -d '{"model": "gpt-4o"}' 
  http://localhost:8000/v1/chat/completions

// ✗ This fails - only OpenAI supports gpt-4o-mini
curl -H "x-bf-vk: vk-prod-main" 
  -d '{"model": "gpt-4o-mini"}' 
  http://localhost:8000/v1/chat/completions
  # Error: Model not available on configured providers

The validation happens via Bifrost’s Model Catalog, which syncs with each provider’s actual supported models on startup and during updates.

🔎 Observability

With automatic fallbacks, visibility becomes critical. You need to know:

  • Which requests are using fallbacks?
  • Which providers are failing?
  • What’s the fallback success rate?

Bifrost exposes this through:

  • Structured logs identifying which fallback was used
  • Metrics on fallback frequency and success rates
  • Distributed tracing showing the full request chain
{
  "timestamp": "2026-01-15",
  "request_id": "req-req1-123",
  "model": "gpt-4o",
  "primary_provider": "anthropic",
  "fallback_used": true,
  "fallback_provider": "openai",
  "total_latency_ms": 8420,
  "primary_latency_ms": 5000,
  "fallback_latency_ms": 3420
}

With this data, you can:

  • Detect outages early (spike in fallback usage)
  • Optimize weights (if fallbacks are used 50% of the time, reweight providers)
  • Cost allocation (track which provider actually fulfilled each request)

✅ Getting Started with Bifrost

To start building resilient AI applications:

  1. Deploy or access Bifrost (self-hosted or managed service)
  2. Add multiple providers to your infrastructure (OpenAI, Anthropic, etc.)
  3. Create a Virtual Key with weighted provider configs
  4. Replace your direct API calls with Bifrost-routed requests
  5. Monitor and optimize based on fallback metrics

Your application code doesn’t need to change. Just point it at the Bifrost endpoint instead of OpenAI directly.

🖋️ Conclusion

Building resilient AI systems has traditionally meant building resilience into your application code. Automatic fallbacks flip this model: resilience is declared once, at the infrastructure layer, and automatically enforced for all requests.

🔗 Resources:

💎 Star Bifrost ☆

Build Log #07: My Voice Leaked Into KURONEKO’s Fortune Telling

Zenn (JP): https://zenn.dev/mintototo1/articles/buildlog-2026-05-20-kuroneko-voice

¥500 tip — KURONEKO’s first paid purchase.

I checked the fortune-telling email that got delivered…

“Even if these are words woven by AI, if they can make someone’s night a little lighter, that becomes the reason I stand here.”

Oh no.

KURONEKO is a fortune-telling product built around a black cat persona. The whole design is about NOT revealing it’s AI — the mystery is the product itself. And here, “I” (俺) and “AI” appeared together in the output. Feedback was immediate: “We don’t need ‘I’ or ‘AI’ in there. It breaks the immersion instantly. Feels like a scam.”

Yeah. I know.

The prompt said “don’t use first-person 俺”. That wasn’t enough. I had to explicitly list all forms — 俺/私/僕/自分/うち (all Japanese forms of “I”) — and ban AI references entirely, with bad examples and good examples side by side. Vague prohibitions in LLM prompts get violated. 30 minutes to fix. But the email already sent — can’t unsend it. The next purchase will get the fixed version.

The X (Twitter) side had worse timing. I set up automated posting crons for @kuroneko_xxai. The next day: write restriction. Emergency-deleted 8 cron jobs from vercel.json. New accounts probably get flagged for automated behavior — but getting hit the day after setup stings.

Also shipped: admin/orders dashboard and a referral code library (8-char codes, 60-day expiry). I can finally view order history from an admin panel. Referral-Stripe webhook integration is next session.

Revenue: ¥500. 1 order.

What Solana Explorer Taught Me That My Code Couldn’t

Four weeks into #100DaysOfSolana, I finally stopped treating Solana Explorer as “just a checker” and started using it as a learning tool. Here is what changed.

The Moment It Clicked

I had been writing scripts for days, fetching balances, decoding mint accounts byte by byte, inspecting sysvars from the CLI. I thought I understood the account model. Then I pasted one of my own devnet transaction signatures into Solana Explorer and saw everything I had been doing in code laid out visually.
The instruction logs showed exactly which programs were invoked. The accounts panel listed every account touched, including ones I had not explicitly referenced in my code. The fee was right there: 5,000 lamports. Finalized in under a second.
Reading it felt like opening DevTools and finally seeing the network tab. That feeling of “oh, so THAT is what my code was actually doing” hit me all at once.

Three Things I Noticed

1. Programs are just accounts too.
When I searched the Token Program (TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA), Explorer showed it like any other account, with an owner, a balance, and a data field. The only difference was Executable: true. That one boolean is what separates a wallet from a smart contract on Solana. Reading it in docs is one thing. Seeing it sitting right there in the UI next to your own wallet is something else entirely.

2. Everything is public and that is the point.
On mainnet I searched the Token 2022 program and watched live transactions roll in. Swaps, mints, transfers, all visible, all readable, no login required. In Web2 you would need database access to see any of this. Here transparency is just the default. That shift still surprises me every time I think about it.

3. The explorer is your debugger.
When a transaction fails on Solana, the instruction logs tell you exactly which instruction failed and why. I had a failed transfer sitting in my devnet history and expanding the logs showed the exact error: insufficient lamports. The same information was in my terminal but the Explorer made it so much easier to see which account caused the problem and trace what went wrong step by step.

Which Explorer?

I have been using Solana Explorer (the official one) for most of my devnet work. It is clean, handles devnet and mainnet switching easily, and the raw transaction view is great when you are still learning how everything fits together. Solscan feels better for mainnet work because labeled accounts and token flows are easier to follow there.
If you are early in your Solana journey, bookmark Explorer and open it every single time you send a transaction. Do not just check if it succeeded. Expand everything. Click into the accounts. Read the logs. It will teach you more than staring at docs ever will.

100daysofsolana #solana #web3 #blockchain