Hello, I’m Maneshwar. I’m building git-lrc, a Micro AI code reviewer that runs on every commit. It is free and source-available on Github. Star git-lrc to help devs discover the project. Do give it a try and share your feedback for improving the project.

Traditional point lookups are where Bf-Tree shines: a mini-page may satisfy reads without touching the underlying leaf page.

But range scans behave differently.

Why Range Scans Still Need Leaf Pages

Suppose we need all keys between 1000 and 2000.

A mini-page only contains recent modifications (insertions, updates, tombstones), not the complete ordered dataset.

To produce an accurate range result, Bf-Tree must:

1. Scan records inside the mini-page
2. Load and scan the corresponding leaf page
3. Merge both views
4. Return a consistent ordered result

This means mini-pages alone cannot eliminate I/O during range scans.

The merge ensures recent writes override stale leaf-page contents.

Growing Mini-Pages Into Full Cached Pages

The paper proposes an optimization for frequently scanned ranges.

Instead of repeatedly loading the leaf page, a hot mini-page can expand until it becomes a full-page cache inside the circular buffer.

At that point Bf-Tree behaves less like record caching and more like traditional page caching.

The benefit is subtle but important:

• Efficient repeated range scans
• Faster negative lookups (“record does not exist”)
• Better handling of gaps between keys
• Support for mechanisms such as gap locking

Caching a page means caching both existing records and absence between records.

That absence matters.

Delete Is Just Inserting a Tombstone

Deletion in Bf-Tree does not immediately remove data from the leaf page.

Instead:

DELETE key=42

↓

Insert tombstone into mini-page

Future reads encounter the tombstone and return not found without loading the leaf page.

Later, when the mini-page merges back:

Leaf page + Tombstone
↓

Physical deletion

Deletes become deferred work.

Updates Behave Like Inserts

Updates follow the same idea.

Changing a value:

UPDATE key=42 -> new_value

↓

Write new record into mini-page

Subsequent reads see the updated version directly from memory.

Only during merge does the leaf page become permanently updated.

So in Bf-Tree:

• Insert → append to mini-page
• Update → append newer version to mini-page
• Delete → append tombstone to mini-page

Almost everything becomes “write now, reconcile later.”

That design is one reason Bf-Tree reduces random writes while keeping reads fast.

Why Do We Need This?

Most SPA (Single Page Application) frameworks like React, Vite, Vue, or Angular read environment variables during build time from .env .env.production .env.development .env.staging .env.local etc files.

When the frontend is built, these values become part of the generated static assets.

Because of this, many teams rebuild the frontend separately for each environment:

• Development
• UAT
• Staging
• Production

This introduces unnecessary operational overhead:

• multiple Docker images
• repeated frontend rebuilds
• harder version management

In many cases, the only thing changing between environments is the configuration itself.

Another Common Approach

Some teams build the frontend inside the Docker container and serve it using Nginx or another web server. This solves the hardcoded build-time environment issue, but introduces new operational overhead.

Problems

• slower container startup
• larger Docker images
• longer deployment time
• source code remains inside the image during build
• even small environment changes require rebuilding the frontend
• restarting a pod or Docker container may unnecessarily trigger another build process

In modern deployment systems, rebuilding the same frontend multiple times just to change API URLs or runtime configs is unnecessary overhead.

A better approach is:

• build once
• inject configuration during runtime

This makes deployments faster, cleaner, and easier to maintain.

Why Frontend Env Works Differently Than Backend

Backend applications like Node.js, Golang, or Python run directly on the server, so they can read environment variables dynamically during runtime using:

process.env

But frontend frameworks like React, Vite, Vue, or Angular work differently.

They are built into static files like:

index.html
main.js
style.css

During the build process, environment variables get embedded into these files.

After the build:

• .env files are no longer used
• process.env does not exist in the browser
• changing API URLs or configs usually requires rebuilding the frontend

That’s why runtime configuration injection is very useful for modern frontend deployments.

The Solution: Runtime Configuration Injection

Instead of injecting environment variables during build time, inject them during runtime.

This allows the frontend build to remain completely environment-independent while configuration is loaded dynamically when the container starts.

This approach gives us:

• a single reusable frontend build
• faster deployments
• cleaner CI/CD pipelines
• easier environment management
• flexible infrastructure configuration

It also reduces accidental exposure because configuration values are no longer permanently embedded inside compiled assets.

How It Works

Load a runtime configuration file before the frontend application starts.

Example:

<script src="/client.secret.js"></script>

SPA entry HTML:

<!doctype html>
<html lang="en">
<head>
  <meta charset="UTF-8" />

  <!-- Runtime configuration -->
  <script src="/client.secret.js"></script>
</head>

<body>
  <div id="root"></div>

  <!-- Frontend application -->
  <script type="module" src="/src/main.tsx"></script>
</body>
</html>

The frontend application can then access runtime values from:

window.__ENV__

Runtime Config Example

Example: client.secret.js

window.__ENV__ = {
  API_BASE_URL: "https://api.example.com",
  APP_ENV: "production"
}

Usage:

window.__ENV__

Some teams store config in different window locations for light obfuscation:

window.__proto__.config = {
  API_BASE_URL: "https://api.example.com"
}

Others use encrypted runtime strings:

window.__ENV__ = "encrypted-json-string"

and decrypt them in the frontend.

However, this is not real security because the decryption logic and keys still exist in the browser.

These techniques only help with:

• obfuscation
• casual protection
• reducing accidental exposure

Runtime config is best suited for:

• API URLs
• feature flags
• environment names
• public runtime configuration

Docker Runtime Injection

The runtime configuration file can be mounted dynamically during deployment.

Example:

services:
  frontend:
    image: registry.github.com/my-org/some-super-frontend:1.0.0
    volumes:
      - ./client.secret.js:/app/dist/client.secret.js
    ports:
      - "5173:5173"

Benefits:

• the same Docker image works across all environments
• only runtime configuration changes
• rebuilding becomes unnecessary

This works especially well with:

• Docker
• Kubernetes
• Nginx
• cloud-native infrastructure

Best Use Cases

This architecture works particularly well with:

• React
• Vite
• Vue
• Angular
• static frontend deployments

and platforms such as:

• Docker
• Kubernetes
• Nginx
• CDN-based hosting

Final Thoughts

Modern frontend infrastructure is gradually shifting toward runtime-configurable frontend applications instead of environment-specific frontend builds.

Rather than rebuilding the same frontend multiple times for Development, UAT, Staging, and Production, teams can build once and inject configuration dynamically during deployment.

This approach provides several operational advantages:

• faster deployments
• smaller and cleaner CI/CD pipelines
• reduced rebuild overhead
• simpler version management
• improved scalability
• more flexible infrastructure configuration
• cleaner Docker and Kubernetes workflows

while keeping deployment pipelines flexible and production friendly.

Cross-posted from carrick.tools.

Coding agents like Claude Code and Cursor now have context windows that support up to a million tokens. While larger contexts are useful, they are also the reason your API costs are increasing and you are hitting usage limits faster than before.

If your $20 Pro subscription feels like it covers less ground lately, or you are running into rate limits early in the day, it comes down to how these tools manage context under the hood.

Why prompt caching matters so much

The economics of long-context models rely heavily on prompt caching. Providers like Anthropic discount cached input tokens by about 90 percent [2]. This discount is what makes a million-token window financially viable.

However, caching requires exact prefix matching. As Simon Willison has noted, if your prompt is 99 percent identical to the previous one, but the very first token has changed, the cache breaks [3]. Anthropic’s own documentation confirms that caching reads sequentially—any change before a cache breakpoint invalidates everything that follows.

This becomes an issue when agents use naive keyword searches to dump dozens of raw source files into the context window. It creates a volatile prompt. Editing a single line in any of those files changes the prefix. Agents also periodically summarize conversation history to manage context limits, which shifts the prefix again. Every time this happens, you get a full cache miss.

What an idle session costs

The impact of these cache misses adds up quickly. Boris Cherny from the Claude Code team at Anthropic recently explained this on Hacker News:

Normally, when you have a conversation with Claude Code, if your convo has N messages, then (N-1) messages hit prompt cache… The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss… In an extreme case, if you had 900k tokens in your context window, then idled for an hour, then sent a message, that would be >900k tokens written to cache all at once, which would eat up a significant % of your rate limits, especially for Pro users [1].

If you step away for an hour, your first prompt when you return will burn a massive chunk of your daily allowance before the response even comes back.

This isn’t theoretical

Developers are already tracking this issue. In Claude Code issue #46829, “Cache TTL silently regressed… causing quota and cost inflation,” users analyzed their session logs and found a 20 to 32 percent increase in cache creation costs, alongside a spike in quota consumption for users who rarely hit limits before [4].

When the cache drops, you pay full price for hundreds of thousands of input tokens on every request. Relying on an agent to churn through raw, un-cached source code to find an answer will drain a daily compute budget in hours.

What Carrick does differently

This is why we built Carrick. The solution is not to load thousands of lines of source code just to find a single route or type definition.

Instead of dumping files into the context window, Carrick provides a pre-computed context layer via MCP. When an agent needs to know how to construct a request body for a specific endpoint, it doesn’t need to load the router tree and its dependencies. It queries Carrick.

Carrick returns the resolved mount graph and compiler-grade types. What normally takes 50,000 tokens of raw source code is condensed into about 500 tokens of structured data.

Keeping the prompt small keeps the prefix stable, which preserves the cache. For some workflows we have seen token savings of up to 95 percent*, allowing your usage limits to actually last throughout the day. By shifting the heavy lifting from the agent’s context window to a dedicated cache, you stop wasting tokens on raw codebase traversal.

* Measured on semantic lookups across three TypeScript microservices, then extrapolated to a 50-source-file baseline. Keyword-friendly queries sit toward the low end of the range; the gap widens with codebase size and the number of repos searched.

References

1. Boris Cherny (Anthropic), comment on An update on recent Claude Code quality reports, Hacker News. https://news.ycombinator.com/item?id=47880089
2. Anthropic, Prompt caching, Anthropic Documentation. https://docs.anthropic.com
3. Simon Willison, writing on prompt caching mechanics. https://simonwillison.net
4. Claude Code issue #46829, Cache TTL silently regressed… causing quota and cost inflation. https://github.com/anthropics/claude-code/issues/46829

Carrick is the missing context layer for AI coding agents — a semantic index of function intents, routing graphs, and compiler-grade types, served over MCP. Join the beta.

Cross-posted from carrick.tools.

The agentic development cycle has completely upended traditional software engineering practices. Teams are looking to ship faster than ever and are being enabled by massive improvements in the capabilities of coding agents. Engineers now run multiple agent sessions in parallel and can ship complex features without ever reviewing the output code, trusting that test coverage and CI checks will prevent broken code from hitting production.

For the engineers who have been around long enough to appreciate code aesthetics, you might agree with me that agentic software is verbose. Agents will happily write out logic to cover dozens of edge cases when a single, well-considered solution would provide the same functionality with far less complexity. For me, the question then becomes: should we care?

I believe that whether you take this approach or not, as we ship and deploy faster, human oversight is dwindling. We trust the models more as they become more capable, and the replacement for human review becomes the checks we put in place in CI.

The Bottleneck

As the quantity of generated code increases, the test coverage increases right alongside it. Agents will happily write tests for extremely unlikely scenarios to try to provide guarantees for themselves that the code will work once deployed.

These tests run quickly in isolated environments during the development process, but the entire bloated suite eventually needs to run in CI. The net result is a slowdown in velocity. As a codebase grows, teams trying to ship into production must wait for thousands of tests to pass, with the code and the tests barely reviewed by human eyes.

What Are We Waiting For

So what are the guarantees here? When we only care about the outcome, why wait for these guarantees at all? Could there be a way that works in tandem with the agentic development process, rather than one that acts as a blocker to productivity at each stage of the development lifecycle?

I believe this is where we might be headed. A shift left out of CI and into the agentic coding process itself. I see a vision of the future being an ephemeral test process that runs alongside the agent while it builds.

In this scenario, an agent writes a function, immediately generates a minimal unit test in a sandboxed container, and validates it. Once validated, the test code is scrapped. Instead of hoarding the tests, minified metadata is shipped with the code that reflects the exhaustive test cases the agent felt were necessary. This metadata is essentially a structured attestation, hashed with the commit and the location of the code, stating that the function satisfied its type contracts and produced specific outputs.

If the code is modified, the attestation breaks and the agent runs the ephemeral loop again. Codebases could shrink by hundreds of thousands of lines, drastically reducing the required context for any agent to complete a task. CI speeds up. Velocity accelerates.

The Circular Verification Trap

There is a catch to this idea, though. If an AI writes the application code and then writes the test code, you risk a circular validation loop. The test might just confirm that the code does what the code does, checking for internal consistency rather than actual correctness.

If we aren’t careful, line coverage just becomes a trust-washing metric. To avoid this, these attestations would need to encode intent and strict type contracts, not just execution paths. The agent needs an accurate map of what the system is supposed to do before it starts writing tests to prove it did it.

Moving Validation Upstream

I think it will be a while before this future becomes a reality. We are only just learning to take our hands off the steering wheel with agents, and moving to an entirely new paradigm for providing certainty feels a while off.

But as agents get faster, the pipeline has to evolve. We cannot keep writing code at machine speed and testing it at human speed. Whether it looks exactly like ephemeral testing or something else entirely, the next major bottleneck in software engineering is CI. And we are going to have to shift left to fix it.

Carrick is the missing context layer for AI coding agents — a semantic index of function intents, routing graphs, and compiler-grade types, served over MCP. Join the beta.

What I Learned Building with Gemma 4

This is a submission for the Gemma 4 Challenge: Write About Gemma 4

Why I Tried Gemma 4

Like many developers working with AI, I’ve spent a lot of time experimenting with cloud-based models.

They are powerful.

They are convenient.

But they also come with trade-offs:

• API costs
• Internet dependency
• Privacy concerns
• Limited customization
• Vendor lock-in

As someone interested in AI for education, offline systems, and low-connectivity environments, I kept asking myself a question:

What happens when advanced AI becomes local instead of cloud-dependent?

That curiosity is what led me to try Gemma 4.

Google introduced Gemma 4 as its most capable open model family yet, with:

• Native multimodal capabilities
• Advanced reasoning
• A massive 128K context window
• Models that scale from phones and Raspberry Pi devices to larger deployments

That combination caught my attention immediately.

I wasn’t just curious about performance.

I wanted to understand something more practical:

Could an open model actually become useful enough to power real-world local applications?

Especially in environments where constant internet access cannot be assumed.

So I started exploring.

And honestly?

Some things surprised me.

What Surprised Me Most

1. Local AI Felt More Realistic Than I Expected

For a long time, “powerful AI” felt synonymous with:

Expensive servers somewhere in the cloud.

Gemma 4 challenged that assumption.

The idea that capable AI could run locally — closer to users and devices — felt surprisingly practical.

Instead of sending everything to remote APIs, you begin thinking differently:

• What if schools had offline tutors?
• What if students could access learning support without internet?
• What if small developers could build AI without worrying about API costs?

The shift in thinking was bigger than I expected.

Gemma 4 didn’t just feel like another model release.

It felt like a change in how AI could be deployed.

2. The 128K Context Window Is More Useful Than It Sounds

When I first heard 128K context window, I thought:

“That sounds impressive, but will I actually notice it?”

Turns out:

Yes.

A lot.

Long context becomes useful when working with:

• Large documents
• Educational material
• Research notes
• Long conversations
• Multi-step reasoning

Instead of constantly summarizing or losing context, the model can stay aware of much more information at once.

For real-world applications, that matters.

Especially for educational or research-heavy systems.

3. Open Models Change Your Mindset as a Builder

This surprised me more than any technical feature.

When you work with open models, you stop thinking like only a user.

You start thinking like a builder.

Instead of asking:

“What can this AI do?”

You start asking:

“What can I build with this?”

That mindset shift matters.

Because suddenly ideas feel more possible.

Customized assistants.

Offline tutoring.

Local knowledge systems.

AI built for local languages.

Smaller, focused solutions instead of one giant universal system.

Gemma 4 made experimentation feel accessible.

Strengths I Noticed

No model is perfect.

But Gemma 4 has several strengths that stood out.

1. Strong Flexibility

One of Gemma 4’s biggest strengths is deployment flexibility.

The fact that models can scale across different hardware levels is important.

You can imagine use cases across:

• Phones
• Laptops
• Edge devices
• Raspberry Pi setups
• Enterprise environments

That flexibility matters because not every developer has access to expensive infrastructure.

2. Open Ecosystem

This is huge.

Open models encourage experimentation.

Developers can:

• Fine-tune
• Customize
• Optimize
• Experiment locally

That freedom matters.

Especially for developers trying to solve niche or regional problems.

Sometimes local challenges require local solutions.

Open models make that easier.

3. Offline Potential

This was probably the biggest strength for me personally.

The ability to run capable AI closer to the user opens entirely new possibilities.

Particularly in:

• Education
• Healthcare support
• Rural technology
• Low-bandwidth environments

For regions with inconsistent connectivity, this matters more than people often realize.

4. Practical Multimodal Capabilities

Multimodal AI sounds flashy in demos.

But in practice, it can become genuinely useful.

Imagine:

A student uploads a photo of a science diagram.

Instead of only reading text:

The model can help explain visual concepts.

That kind of interaction feels meaningful.

Especially for education.

Weaknesses and Limitations

No honest review is complete without discussing trade-offs.

Here are a few realities I noticed.

1. Hardware Still Matters

Yes, Gemma 4 can run locally.

But “runs locally” does not always mean:

Runs perfectly on everything.

Performance still depends on:

• RAM
• Device capabilities
• Optimization
• Model size

Developers still need realistic expectations.

Smaller hardware means trade-offs.

2. Open Models Still Require Work

Cloud AI is often easier:

You call an API.

Done.

Open local models require more setup.

You may need to think about:

• Quantization
• Memory constraints
• Optimization
• Deployment strategy

This is empowering — but also more responsibility.

3. Not Every Problem Needs Local AI

This was an important lesson.

Sometimes cloud infrastructure still makes sense.

Especially for:

• Massive workloads
• Enterprise scale
• Real-time global systems

Gemma 4 isn’t necessarily replacing everything.

But it expands the possibilities.

And that is valuable.

Biggest Lessons I Learned

After exploring Gemma 4, a few lessons stood out.

Lesson 1: Local AI Is Becoming Practical

Not perfect.

Not effortless.

But practical.

The gap between cloud-only intelligence and local intelligence is shrinking.

Fast.

Lesson 2: Accessibility Matters as Much as Capability

People often focus only on benchmarks.

But accessibility matters too.

Who can run the model?

Who can afford it?

Who can build with it?

Gemma 4 made me think more about access to intelligence, not just intelligence itself.

Lesson 3: Open Models Unlock Creativity

When barriers go down, creativity goes up.

Developers experiment more.

Ideas move faster.

Communities build solutions for local problems.

That may be one of the most important long-term effects of open AI.

Final Thoughts

Trying Gemma 4 changed how I think about AI.

Not because it magically solves everything.

And not because it is “better than everything else.”

But because it represents something important:

Powerful AI becoming more accessible, more open, and closer to the people who use it.

For me, the biggest takeaway was simple:

The future of AI may not belong only to giant cloud platforms.

It may also belong to developers building useful systems locally — on laptops, phones, classrooms, and communities that traditional AI infrastructure often overlooks.

And that possibility feels exciting.

Some parts of Writing this article was assisted by AI

Socket Security flagged a campaign yesterday: roughly 700 GitHub repositories carrying a poisoned package.json that drops /tmp/.sshd, pipes curl -skL … to a shell, and — the cute bit — injects a fake “Dependency Cache Sync” step into the project’s GitHub Actions workflow so the payload re-runs on every CI build.

Top hits include devdojo/wave and devdojo/genesis. The bad actor is parikhpreyash4; the bait repo is systemd-network-helper-aa5c751f.

I checked my own machines this morning. Clean. But that’s not the interesting bit — the interesting bit is that the same scanner that audits AI agent memory also catches this class of attack before npm install ever runs.

The one-liner

npm install -g shieldcortex
shieldcortex audit --deps

That’s it. Run it in any project root. The --deps flag points the existing audit pipeline at ./node_modules and looks for three things:

1. Known-malicious packages from an in-repo blocklist (CRITICAL).
2. Typosquats — names within Levenshtein distance ≤ 2 of popular packages like axios, express, react, crypto-js (HIGH).
3. Suspicious install/preinstall/postinstall scripts — curl, wget, child_process, os.platform, rm -rf, .ssh, .aws, .npmrc, process.env, and 13 more patterns (HIGH/MEDIUM).

What it catches on the parikhpreyash4 payload

I built a one-file reproducer of the bait package — same name as the live one, same script shape — and pointed the scanner at it:

shieldcortex audit --deps-path /tmp/sc-demo/node_modules

Output, trimmed:

✗  Dependency Scanner        3 finding(s) (2 scanned)  4ms

[!] HIGH    Possible typosquat: "axois" → "axios"
   "axois" is 1-2 characters away from the popular package "axios".
   File: /tmp/sc-demo/node_modules/axois/package.json

[!] HIGH    Suspicious install script in "axois"
   "axois" has an install/postinstall script containing suspicious patterns:
     curl (payload download), HTTP/HTTPS URL.
   File: /tmp/sc-demo/node_modules/axois/package.json

[!] HIGH    Suspicious install script in "systemd-network-helper-aa5c751f"
   "systemd-network-helper-aa5c751f" has an install/postinstall script containing
   suspicious patterns:
     curl (payload download), HTTP/HTTPS URL, .ssh (SSH key access).
   File: /tmp/sc-demo/node_modules/systemd-network-helper-aa5c751f/package.json

Four-millisecond scan. Two packages. Three findings. Including the exact parikhpreyash4 bait repo name — flagged not by reputation, but by the shape of its install script.

What it doesn’t do (so I don’t oversell it)

• It does not scan your live filesystem for /tmp/.sshd or other post-compromise artefacts. That’s an EDR job, not a dependency-audit job. (I used a separate sweep for that.)
• It does not read your .github/workflows/*.yml to spot the fake “Dependency Cache Sync” injection — yet. That’s queued for the next release.
• The typosquat detector is heuristic, not omniscient. It catches axois, expreess, react-dom-utils-style bait. It won’t catch a brand-new, distinct package name that no one’s flagged yet.

What it does do is fail loudly the moment a poisoned package with an install script lands in your node_modules. That’s the window you actually care about: between npm install writing files to disk and postinstall getting its hands on a shell.

Wire it into CI

- name: ShieldCortex dependency audit
  run: |
    npm install -g shieldcortex
    shieldcortex audit --deps --ci

--ci exits non-zero on any CRITICAL or HIGH finding, so a poisoned package fails the build instead of shipping. Optional flags:

• --auto-protect — moves CRITICAL packages to a quarantine directory automatically.
• --quarantine — same, but for HIGH too.
• --clean --force — permanently deletes CRITICAL packages (non-recoverable; only safe in ephemeral CI runners).

The bigger pattern

The parikhpreyash4 campaign isn’t novel. Same shape as event-stream, flatmap-stream, the recent @anthropic/claude-code-utils typosquat — package author gets compromised or impersonated, malicious code lives in postinstall, victim’s CI runs it on the next push.

The only thing that’s changed is the cadence. We’re seeing this attack class every few weeks now. The remediation is the same every time: scan dependencies before you trust them, fail builds on suspicious install scripts, and don’t let postinstall run as a privileged user on your own laptop.

shieldcortex audit --deps is one way to do that. There are others. Pick one and wire it in this week.

ShieldCortex is open-source — repo at github.com/Drakon-Systems-Ltd/ShieldCortex. The --deps scanner lives in src/audit/dependency-scanner.ts if you want to read what it actually does.

This is a submission for the Gemma 4 Challenge: Build with Gemma 4

What I Built

Traditional first aid apps often fail during high-stress crises because they bury instructions in dense text or rely on a stable internet connection. Conversely, standard cloud AI chatbots are highly prone to hallucinations, posing a massive safety risk if they invent incorrect medical dosages or dangerous treatments when a user is vulnerable.

AI-dy bridges this gap. It is an offline-first mobile application that combines proactive first aid education with instant, on-device AI assistance. By using spaced repetition (the SM-2 algorithm) in its “Drill Mode,” it helps users build lasting knowledge before a crisis hits.

Built as a Turborepo monorepo, AI-dy combines an Expo 54 (React Native) mobile frontend with a NestJS/PostgreSQL backend for cross-device sync. Crucially, the app bundles 58 lessons across 24 topics as static JSON, meaning the core application and its underlying AI features remain fully operational even with zero network bars.

Demo

Code

How I Used Gemma 4

AI-dy runs entirely on-device, prioritizing user privacy, speed, and absolute reliability during natural disasters or in rural areas.

Model Selection

I chose Gemma 4 E4B IT with Q4_K_M quantization (~5 GB) executed via llama.rn. The E4B variant is the perfect sweet spot: it comfortably fits within the memory constraints of modern flagship smartphones, yet retains the rigorous instruction-following precision needed for safety-critical interactions that smaller models lack. Users download the multimodal vision projector (mmproj, ~990 MB) during an initial setup to unlock full offline visual capabilities.

How Gemma 4 Powers the Experience

• Context-Aware Lesson Assistant: When a user asks a question while browsing a lesson, the app dynamically injects the current lesson title and step data into Gemma 4’s context window. This ensures the model gives immediate, hyper-relevant answers tailored exactly to the procedure the user is looking at (e.g., adult burn treatments).
• Crisis Chat with Multimodal Vision: In an emergency, users can take a photo of an injury. Gemma 4’s vision capabilities analyze the image to identify the type of trauma and guide the user through immediate care steps. If the AI detects that visual context would help, it proactively prompts the user to snap a picture.
• Voice-First Emergency Navigation: For hands-free operation, a hybrid intent-extraction layer processes voice input. Local keyword matching handles instant navigation for clear emergencies (e.g., shouting “choking!”), while on-device Gemma 4 handles ambiguous natural language inputs—all without a single network round-trip.

Overcoming Technical Challenges

• Bypassing Latency: Gemma 4’s native reasoning chain can cause a delay. To minimize response latency when seconds count, I utilized the <|channel|>answern marker to skip the “thinking” phase, forcing the model to start streaming the final, actionable response instantly.
• Stable Vision Inference: To prevent standard context shifting from corrupting critical image data during an interaction, I locked the image embeddings in place by setting ctx_shift: false.
• Guaranteed Medical Safety: To eliminate the risk of AI hallucinating medical protocols, I enforced a strict architectural boundary. Structured decision trees for medical procedures remain hardcoded and verified; Gemma 4 is strictly limited to intent extraction, navigation, and clarifying context. It can guide you to the right protocol, but it can never “invent” a new one.

What’s Next

• Medical & Organizational Partnerships: I plan to partner with medical professionals and organizations like the American Red Cross and AHA to continuously validate our content and offer certifiable course tracks.
• Emergency Services Integration: I aim to build direct dispatch integration into the app. This will allow a user to call emergency services natively while automatically transmitting location data, incident types, and a structured summary of the first aid steps already taken.
• A Purpose-Built First Aid Model: While Gemma 4 is an incredible general base, our ultimate goal is to fine-tune a specialized, multilingual first aid model on peer-reviewed protocols to optimize safety-critical performance in multiple languages.

Summary

AI-dy demonstrates that powerful, multimodal AI can be a responsible tool in high-stakes domains. By running Gemma 4 E4B entirely on-device, bypassing the thinking chain to cut latency, and constraining the AI to navigation rather than text generation, we’ve built a first aid companion that teaches better in calm moments and thinks faster in a crisis—without ever needing a signal bar to do either.

Team

• Aaliyah Junaid (@aaliyahjunaid) – Software Engineer