Gas Town by Kilo multi-agent orchestrator now available

Gas Town by Kilo, a cloud-hosted version of a multi-agent orchestrator that turns a single developer into an engineering team, is generally available today.  By running it on Kilo Cloud with the Kilo Gateway behind it, it removes the need to self-host and adds managed infrastructure, elastic scaling and more than 500 models run through … continue reading

The post Gas Town by Kilo multi-agent orchestrator now available appeared first on SD Times.

Why I made my paid product fully free for personal use

I spent two months thinking about pricing before I let myself write the pricing page.

That is not a humblebrag, it slowed everything down. The product was technically ready in March. By April it could have been on sale. But every time I sat down to define the free tier, something did not sit right, and I would close the tab and go fix a bug instead.

What unstuck it was not a framework, a benchmark, or a YC essay. It was one question I could not answer to my own satisfaction. I want to write about it, because the question is generalizable and I think a lot of solo makers building tools quietly trip over it.

A bit of context first. I have been a technical product manager at large tech companies for many years, always on B2B SaaS, always with a team, always for someone else’s company. The product I was pricing is Notch, a screen recorder for Mac and Windows. It is the first desktop app I have shipped under my own name. So this was the first time I was personally on the line for a pricing decision, with my own money funding the development and my own name on the receiver.

The freemium playbook I almost ran

When I sketched the pricing card the first time, it looked almost exactly like what every other screen recorder does:

  • Free tier with a 5-project cap, limited by 1080p output quality, and a watermark on exports
  • Paid tier at $49 one-time that removes both

This is the freemium playbook by default. The free tier is a trial in disguise. Generous enough to demo the product, restrictive enough that anyone using it seriously has to upgrade. Loom does it. Screen Studio does it. Camtasia does it. Tella does it. So did the spreadsheet I was modelling.

The numbers added up. The conversion funnel made sense. I built the watermark into the export pipeline. I implemented the project counter and a “you’ve hit your limit” modal. I tested both. They worked.

I just did not want to ship them.

The question I could not answer

The discomfort was not moral. I had convinced myself the freemium model is honest enough, since the limits are visible upfront. The discomfort was about a specific user I kept picturing.

That user is someone like me five or ten years ago: an engineer recording their first screencast tutorial for their blog, a student doing a class project, a hobbyist showing their indie game on Mastodon. Someone who is not going to make a dollar from the thing they are recording. Someone for whom $49, even one-time, is meaningful money.

Why am I asking them to pay?

I sat with that question for two weeks. Every answer felt thin.

“Because the software costs me money to maintain.”

OK, true, but that user is not loading my support queue. They are not going to email me. They are using the product silently.

“Because if I let them export for free, they will never upgrade.”

They were never going to upgrade. They do not have a use case for the Commercial license, they are not running paid work through it. The “upgrade” framing is a trick I would be playing on someone who could not fall for it.

“Because it is the standard model in the category.”

The least convincing answer of all. “Everyone does it” has never been a strong reason to do anything.

The honest version of the question kept reformulating itself in my head:

Why should someone pay me if they are not making money from this themselves?

I did not have a good answer. So I stopped trying to find one.

What I changed

I removed the project cap. I removed the watermark. I removed all limits. The personal tier of Notch is fully free. No degraded export, no nag screens, no asterisks. You install the app, you hit record, you export your video. Done.

The Commercial license is still there. It is $49 one-time, with team pricing for 3 and 10 seats. The split is not “free for limited use, paid for everything”. It is “free for personal use, paid for revenue-generating use”. The line is the purpose of the recording, not the feature set.

This means:

  • A student making a class project: free, forever, with full features.
  • A bootcamp graduate recording tutorials for their portfolio: free.
  • An OSS maintainer recording a feature demo for a GitHub issue: free.
  • A teacher recording an async lesson for their students: free.
  • A consultant making client deliverables: Commercial license.
  • A SaaS team recording onboarding videos: Commercial license.
  • A YouTuber monetizing their channel: Commercial license.

The free tier and the Commercial tier are the same software. The license is what changes.

The objection I expected

I had a draft of this post that opened with the predictable objection: “But how will you actually make money?”

I cut it because I think it deserves a real answer, not a defensive setup. Here is the real answer.

Most non-monetizing users were never going to pay anyway. Walking into a freemium model assuming they are conversion-able is wishful. The honest math: if 95% of free-tier users would never pay regardless of how much you crippled the free tier, your “free tier” was effectively gifting them a worse product than they could have had, and resenting them for not converting.

Commercial users are different. They have a budget line for tools. They are comparing me to Loom at $18/user/month or Screen Studio at $108/year. A one-time $49, or $29/user for a 10-pack, is the cheapest thing in their procurement queue. They are going to pay.

Goodwill compounds. The student who got Notch for free for a year, who graduates and goes to a SaaS company that needs internal training videos, that is the person who pitches Notch to their boss. Not because of switching costs, but because they already know it works and trust it. That kind of advocacy does not show up in a conversion funnel, is almost impossible to A/B test, and stacks over years.

The “real free tier” is a marketing position. It is also a moat. Every competitor who runs the freemium-with-limits playbook is implicitly telling their non-monetizing users “we tolerate you, for now”. That is not a feeling people seek out.

The deeper reason

I will be honest about the part that is not a pricing argument.

I went into product management years ago because I love watching the results of my work get used. Not “purchased”. Used. The thing I find satisfying about shipping is people getting value. If I could only sell products and never see them in someone’s hands, I would have stayed in e-commerce.

Building “another commercial product”, one more entry in a category, optimized for revenue capture, was not a strong enough reason for me to spend a year of my evenings on something. It is a thing I could have done with my day job’s time instead. The reason I built Notch is that I kept wanting it to exist and nothing else delivered it. The reason I am shipping it is that I want other people to have it too.

A pricing model that restricts the people most likely to enjoy the product, in order to maximize revenue from the people also most likely to enjoy the product, is solving for the wrong thing.

The team and Commercial side will pay for itself. I would rather underearn from the model I chose than overcharge through one that fights my own reason for building.

What I’d tell another maker

Three things, briefly.

Pricing reveals what you actually think your product is for. If the only way the math works is by limiting the people who would love your product most, the math might be wrong.

The freemium playbook is the default, not the answer. It works for some categories, usually ones with high marginal cost per user, strong network effects, or genuine paid value the free tier cannot replicate. Most maker tools are none of those.

Trust your discomfort. I spent two months avoiding writing a pricing page because the model I was about to ship felt wrong. That feeling was data. I should have listened to it sooner.

If you are curious what the product looks like with this pricing applied: Notch is on Mac and Windows, fully free for personal use, no account, no cloud, no nags. Commercial license is one-time, $49.

I am happy to argue any of the above in the comments. I think I am right, but I have been wrong about pricing for a living.

May the force be with you.

Build Your Own CLI Agent: A Step-by-Step Guide

Build Your Own CLI Agent — terminal chat with Ink and Mozaik

I love JavaScript. Even though I have learned several languages, JavaScript has become my first choice when building new projects. In this post I’ll walk through a small terminal chat app where a language model can run shell commands for you — something you can actually run locally. At the end there’s a GitHub template if you’d rather clone than copy.

Two libraries do most of the heavy lifting:

  • Ink draws the chat in the terminal — the input box, scrolling text, and key handling — using React so you are not building a TUI from scratch.
  • Mozaik connects the model, tool calls, and conversation memory. Think of it as the wiring between “the model said something,” “a tool finished,” and “show that in the UI,” so your screen code does not have to know every detail of the API.

Ink is what the user sees; Mozaik is what coordinates the agent behind the scenes.

What we are building

One Node command that:

  1. Shows a simple chat UI in the terminal.
  2. Forwards what you type to a hosted language model.
  3. Lets the model run terminal commands through a single tool (run_command) when it needs to inspect the machine or run a build.
  4. Prints the model’s replies (and optional “calling a tool…” hints) in that same UI — without pushing API calls into every component.

The goal is a clean split: the terminal view stays simple; the agent and tools stay in one place and are easier to test or swap later.

How the pieces fit together

Mozaik runs everything on a shared AgenticEnvironment: when the model speaks, asks for a tool, or a tool returns a result, every piece that has joined that environment can hear about it. For this CLI you mainly care about two kinds of participant:

Role Typical base class Responsibility
Agent BaseAgentParticipant Remembers the conversation, asks the model for the next step, and runs tools (like run_command) when the model asks for them.
Observer / UI bridge BaseObserverParticipant Listens for assistant text and tool activity from the agent and forwards it into Ink through small callbacks — so the screen updates when the model speaks or starts a tool, without owning the agent loop.

In this project, typing in Ink calls session.send, which hands your text to the agent. You could add other participant types later (for example streaming stdin), but a straight send-to-agent path keeps the tutorial easy to follow.

┌─────────────┐     session.send()      ┌──────────────────┐
│  Ink (app)  │ ──────────────────────► │  TerminalAgent   │
└─────────────┘                         │  (agent loop)    │
       ▲                                └────────┬─────────┘
       │ callbacks                            │ join()
       │                                        ▼
┌─────────────┐                         ┌──────────────────┐
│  UIUpdater  │ ◄── onExternal* ─────── │ AgenticEnvironment│
│  (observer) │                         └──────────────────┘
└─────────────┘

Step 1 — Bootstrap the runtime (cli.tsx)

The entry file is intentionally tiny: load environment variables, maybe print a one-line usage hint, then hand off to Ink with render(<App />). Keep agent logic out of here — only bootstrapping.

#!/usr/bin/env node
import React from "react";
import { render } from "ink";
import meow from "meow";
import dotenv from "dotenv";
import path from "node:path";
import { fileURLToPath } from "node:url";
import App from "./app.js";

const here = path.dirname(fileURLToPath(import.meta.url));
dotenv.config({
  quiet: true,
  path: [
    path.resolve(process.cwd(), ".env"),
    path.resolve(here, "..", "..", ".env"),
    path.resolve(here, "..", "..", "..", ".env"),
  ],
});

meow(
  `
    Usage
      $ your-cli

    Starts an interactive chat with the agent.
`,
  { importMeta: import.meta },
);

render(<App />);

Step 2 — Compose the session (session.ts)

This file is the “control room.” You create the model, the conversation memory, the thing that runs tools, and the shared environment, then plug in your agent and your UI helper. Everything with awkward names (OpenAIInferenceRunner, ModelContext, and so on) lives here so the React side stays small. When you are done, the UI only needs one method: send(message), forwarded to the agent’s onMessage.

import {
  AgenticEnvironment,
  Gpt54,
  ModelContext,
  OpenAIInferenceRunner,
  DefaultFunctionCallRunner,
} from "@mozaik-ai/core";
import { terminalTools } from "./terminal/tools.js";
import { TerminalAgent } from "./terminal/agent.js";
import { UIUpdater } from "./ui-updater.js";

export type AgentSession = {
  send: (message: string) => void;
};

export type AgentListeners = {
  onAssistantText: (text: string) => void;
  onFunctionCall?: (name: string) => void;
};

export function createAgentSession(listeners: AgentListeners): AgentSession {
  const functionCallRunner = new DefaultFunctionCallRunner([...terminalTools]);
  const inferenceRunner = new OpenAIInferenceRunner();

  const context = ModelContext.create("cli-agent");
  const model = new Gpt54();
  model.setTools([...terminalTools]);

  const environment = new AgenticEnvironment();
  const agent = new TerminalAgent(
    inferenceRunner,
    functionCallRunner,
    environment,
    context,
    model,
  );
  const uiUpdater = new UIUpdater(listeners);

  agent.join(environment);
  uiUpdater.join(environment);
  environment.start();

  return {
    send: (message: string) => agent.onMessage(message),
  };
}

Step 3 — The agent loop (terminal/agent.ts)

This is the heart of the app. When someone sends a message, you record it, let the model think, and if it wants to run a tool you run it and feed the result back — then the model gets another turn until it answers in plain language.

In practice:

  • On new user text → add to conversation and ask for the next model response.
  • When the model asks for a tool → remember the call is in flight, run it, store the outcome.
  • When every outstanding tool has finished → ask the model again so it can either reply or request another step.

There is no Ink or terminal drawing in this file — only memory and orchestration — so you can change the UI later without touching the agent.

import {
  BaseAgentParticipant,
  UserMessageItem,
  FunctionCallItem,
  AgenticEnvironment,
  ModelContext,
  GenerativeModel,
  InputStream,
  InferenceRunner,
  FunctionCallRunner,
  FunctionCallOutputItem,
  DeveloperMessageItem,
} from "@mozaik-ai/core";

const programmaticAgentInputStub: InputStream = {
  async *stream() {},
};

export class TerminalAgent extends BaseAgentParticipant {
  private pendingCalls = new Set<string>();

  constructor(
    inferenceRunner: InferenceRunner,
    functionCallRunner: FunctionCallRunner,
    private readonly environment: AgenticEnvironment,
    private readonly context: ModelContext,
    private readonly model: GenerativeModel,
  ) {
    super(programmaticAgentInputStub, inferenceRunner, functionCallRunner);
  }

  override onMessage(message: string): void {
    const developerMessage = DeveloperMessageItem.create(
      `You are a terminal agent. You can run commands in the terminal to help the user with their request.`,
    );

    this.context
      .addContextItem(developerMessage)
      .addContextItem(UserMessageItem.create(message));
    this.runInference(this.environment, this.context, this.model);
  }

  override onFunctionCall(item: FunctionCallItem) {
    this.pendingCalls.add(item.callId);
    this.context.addContextItem(item);
    this.executeFunctionCall(this.environment, item);
  }

  override onFunctionCallOutput(item: FunctionCallOutputItem) {
    this.context.addContextItem(item);
    this.pendingCalls.delete(item.callId);
    if (this.pendingCalls.size === 0) {
      this.runInference(this.environment, this.context, this.model);
    }
  }
}

Step 4 — Tools the model can use (terminal/tools.ts)

Tools are how you tell the model what it is allowed to do outside of chat text. Each tool has a name, a short description the model can read, argument shapes, and an invoke function that runs on your machine.

Here we expose one tool: run_command, which executes a shell command and returns output so the model can use it on its next turn.

import { Tool } from "@mozaik-ai/core";
import { Terminal } from "./terminal.js";

const terminal = new Terminal();

export const terminalTools: Tool[] = [
  {
    name: "run_command",
    description: "Run a command in the terminal.",
    parameters: {
      type: "object",
      properties: {
        command: {
          type: "string",
          description: "The command to run in the terminal.",
        },
        cwd: { type: "string", description: "The current working directory." },
      },
      required: ["command", "cwd"],
    },
    strict: true,
    type: "function",
    invoke: async (args: { command: string; cwd: string }) => {
      const result = await terminal.runCommand(args.command, args.cwd);
      return result;
    },
  },
];

Step 5 — Feed the terminal UI (ui-updater.ts)

The observer sits between Mozaik and Ink. When the agent produces text the user should see, this class forwards it through a callback; when a tool starts, it can add a small status line (for example “calling run_command“).

You stay subscribed to external events so you hear what the agent is doing, not duplicate the agent’s own work — one clear owner of the loop and one clear owner of the display.

import {
  Participant,
  FunctionCallItem,
  ModelMessageItem,
  BaseObserverParticipant,
} from "@mozaik-ai/core";

type Listeners = {
  onAssistantText: (text: string) => void;
  onFunctionCall?: (name: string) => void;
};

export class UIUpdater extends BaseObserverParticipant {
  constructor(private readonly listeners: Listeners) {
    super();
  }

  override onFunctionCall(item: FunctionCallItem) {
    this.listeners.onFunctionCall?.(item.toJSON()?.name ?? "tool");
  }

  override onExternalFunctionCall(
    _source: Participant,
    item: FunctionCallItem,
  ) {
    this.listeners.onFunctionCall?.(item.toJSON()?.name ?? "tool");
  }

  override onExternalModelMessage(_source: Participant, item: ModelMessageItem) {
    const text = item.content?.text ?? "";
    if (text) this.listeners.onAssistantText(text);
  }
}

Step 6 — Wire Ink (app.tsx)

The Ink layer holds chat history in normal React state, builds the session once so you do not reconnect on every render, and on submit appends the user message then calls session.send. Anything the observer hears arrives through the callbacks you passed in when creating the session.

import React, { useMemo, useRef, useState } from "react";
import { useApp } from "ink";
import { createAgentSession } from "./session.js";

type ChatMessage = {
  id: number;
  role: "user" | "assistant" | "system";
  content: string;
};

export default function App() {
  const { exit } = useApp();
  const [messages, setMessages] = useState<ChatMessage[]>([]);
  const nextId = useRef(0);

  const appendMessage = (role: ChatMessage["role"], content: string) => {
    setMessages((previous) => [
      ...previous,
      { id: nextId.current++, role, content },
    ]);
  };

  const session = useMemo(
    () =>
      createAgentSession({
        onAssistantText: (text: string) => {
          appendMessage("assistant", text);
        },
        onFunctionCall: (name: string) => {
          appendMessage("system", `calling tool: ${name}`);
        },
      }),
    [],
  );

  const handleSubmit = (value: string) => {
    const trimmed = value.trim();
    if (!trimmed) return;
    appendMessage("user", trimmed);
    session.send(trimmed);
  };

  // …render messages + <TextInput onSubmit={handleSubmit} /> …
}

Step 7 — Credentials and build

Put your API key in .env (the template expects OPENAI_API_KEY; check Mozaik if you change model or provider). Then install, build, and run the compiled CLI (or npm link if you want a global command).

cp .env.example .env
# add OPENAI_API_KEY=sk-...

npm install
npm run build
npm start

Starter repository

Prefer a working tree over copy-paste? There is a template repo with the same layout this article walks through — agent, observer, tools, and Ink UI already split into files.

Scaffold a fresh project without copying the template’s full git history:

npx degit jigjoy-ai/cli-agent-starter my-cli-agent
cd my-cli-agent
git init
git add .
git commit -m "Initial commit"
npm install

Replace jigjoy-ai/cli-agent-starter with your fork or canonical URL if it moves; replace my-cli-agent with your package name. Then edit package.json (name, bin), tweak source/cli.tsx / source/app.tsx branding, and start adding participants and tools.

GitHub alternative: enable Template repository in the repo settings and use Use this template — you get a first commit snapshot with a clean history for a new repo.

Starter template: github.com/jigjoy-ai/cli-agent-starter

Where to go next

  • Add BaseHumanParticipant if you want stdin streaming as first-class InputStream input.
  • Add a second BaseAgentParticipant and use onExternal* handlers to log or aggregate multi-agent chatter.
  • Swap Gpt54 / runners for other models supported by Mozaik as the ecosystem grows.

You now have a concise path from “blank Node project” to Ink front end + Mozaik event bus + tool-running agent — with a degit-friendly repo to hit the ground running.

Learn more

Mozaik: github.com/jigjoy-ai/mozaik

Related: Reactive Agents in Mozaik · Mozaik 3.0 — Agentic Environment

Read the full article on JigJoy: jigjoy.ai/blog/build-your-own-cli-agent

Tune in and Join the Google I/O 2026 Writing Challenge: $1,000 in Prizes!!

Google I/O 2026 is here! Join us today and tomorrow (May 19 & 20) as we livestream Google’s keynotes and sessions right on DEV. Plus, we’re launching a new writing challenge right alongside the event!

Running through May 24, the Google I/O Writing Challenge invites you to dive into the announcements as they drop, then experiment and write about what has you most excited from this year’s releases, keynotes, and sessions.

Tune In to Google I/O 2026

Catch everything as it unfolds live here on DEV, May 19 and 20:

Note: If you miss the livestreams you will be able to watch the recorded keynotes on YouTube.

Our Prompt

Explore the Google I/O 2026 announcements and pick the release, update, or livestream session that speaks to you, and tell us about it.

We encourage you to play around with any new products (if applicable) to form your opinions, but even if you’re in the “reading the docs” phase, we want your take. Share your highlights and critiques with us!

 

Google I/O Challenge Submission Template

 

Your submission might look like:

  • A hands-on tutorial or walkthrough of something you tried
  • A written reflection on an announcement or keynote point and what it means for developers
  • An opinion piece on the update you think is most underrated or overlooked
  • A first-look or “getting started” guide for a new tool or feature

The most compelling submissions will go beyond summary to offer a genuine perspective for the community.

Judging Criteria

  • Clarity & Quality: Is it well-written, engaging, and easy to follow?
  • Usefulness: Will the community learn something or be inspired by reading it?
  • Depth & Insight: Does it go beyond surface-level summary to offer a genuine perspective?

Prizes

Five winners will each receive:

  • $200 USD
  • DEV++ Membership
  • Exclusive DEV badge

All participants with a valid submission will receive a completion badge.

How To Participate

Explore the Google I/O 2026 announcements, then publish a post on DEV using the submission template provided above.

Please review our judging criteria, rules, guidelines, and FAQ page before submitting so you understand our participation guidelines and official contest rules such as eligibility requirements.

Important Dates

  • May 19: Google I/O Writing Challenge begins
  • May 20: Google I/O wraps
  • May 24: Submissions due at 11:59 PM PDT
  • June 4: Winners Announced

We can’t wait to read what you discover. Questions about the challenge? Ask them below.

Good luck and happy writing!

When to use Data API vs RPC API in Workflow Engine NEO: a .NET architect’s guide

Data API vs RPC API decision canvas

I am Mike Lukinov, co-founder of Optimajet. Since 2022, I have been closely involved in customer architecture and developer experience work around Workflow Engine by Optimajet. I sit in most customer architecture meetings where Workflow Engine NEO questions repeat. The pattern I see most often when teams adopt Workflow Engine NEO is this: an architect picks the RPC API for every integration because “the RPC API is the obvious API.” Six months later that team has a BI dashboard, a mobile app, and a partner integration all calling the same RPC surface, and a security review is asking why a read-only analyst token can start and cancel processes. This is the article I wrote instead of answering each case from scratch.

Workflow Engine NEO ships two HTTP surfaces, the Data API and the RPC API; the choice between them is a security and blast-radius decision, not a syntax preference. Everything below is personal opinion, grounded in those customer conversations. Throughout I mark each block as Documented behavior, Architectural recommendation, or Risk if ignored, so you can separate Optimajet facts from opinion. The framing should hold even if you pick a different boundary.

Version note. The code examples below target Workflow Engine NEO v21.0.0 and later, where Web API authorization uses a compact WorkflowApiPermissions claim and the builder API. If you are on v20.x or earlier, check the matching documentation for the older per-operation claim format; do not copy v21 snippets into a v20 project.

What the Workflow Engine NEO Data API and RPC API actually do

Documented behavior. According to the Workflow Engine Web API setup documentation retrieved 2026-05-12, each surface has a one-sentence definition:

“RPC API: Remotely manage Workflow Engine Runtime instances in single-tenant or multi-tenant modes.”
“RESTful Data API: Interact safely with the Workflow Engine database without disrupting internal processes.”

The Web API Core is provided by OptimaJet.Workflow.Api for .NET 8.0, and Data access is added through a provider-specific package such as OptimaJet.Workflow.Api.Mssql, OptimaJet.Workflow.Api.Postgres, OptimaJet.Workflow.Api.Mysql, OptimaJet.Workflow.Api.Oracle, OptimaJet.Workflow.Api.Sqlite, or OptimaJet.Workflow.Api.Mongo. Workflow Engine NEO is a licensed feature set on top of base Workflow Engine starting from v19.0.0; the RPC API and Web API multitenancy were introduced in v19.0.0 per the 19.0.0 release notes. The two surfaces share the same multi-tenant header, Workflow-Api-Tenant-ID, the same WorkflowApiPermissions claim model, and are exposed through the same Workflow Engine Web API service with a generated OpenAPI specification. What differs is the operations branch each one exposes, and that difference is where the design intent lives.

The Web API requires a Workflow Engine license key with the API/NEO option enabled; without the key, the API does not start. The Data/RPC split is therefore a paid Workflow Engine NEO capability; base Workflow Engine and the free Workflow Engine Community Edition do not include either API, per the Workflow Engine pricing and editions page.

When to use the Workflow Engine NEO Data API

Architectural recommendation. Use the Data API as the default integration boundary for reporting, audit export, BI dashboards, and support tools that should observe workflow state without driving runtime behavior. The Data API is the right place to surface workflow data to consumers whose blast radius you want to keep small.

Documented behavior, important clarification. The Data API is not strictly read-only. The documented Data API surface includes create, update, and delete operations for several entities, among them global parameters, workflow schemes, process parameters, timers, transitions, approvals, and inbox entries. Calling the Data API does not move a process through state transitions the way an RPC command does, but the Data API can mutate the underlying data. Read-only consumers must therefore receive only the specific get and get-collection permissions for the entities they need, not a blanket allow over the Data API surface.

Architectural recommendation, with v21 code. A practical scope for a read-only analyst persona uses the v21 WorkflowApiPermissions builder API to deny everything by default, scope to a single tenant, and then explicitly allow concrete read operations:

var claim = permissions.BuildClaim(builder => builder
    .DenyAllOperations()
    .DenyAllTenantsExcept("TenantA")
    .Allow(
        "workflow-api.data.schemes.get",
        "workflow-api.data.schemes.get-collection",
        "workflow-api.data.processes.get",
        "workflow-api.data.processes.get-collection",
        "workflow-api.data.processes.parameters.get",
        "workflow-api.data.processes.parameters.get-collection",
        "workflow-api.data.processes.transitions.get",
        "workflow-api.data.processes.transitions.get-collection",
        "workflow-api.data.processes.approvals.get",
        "workflow-api.data.processes.approvals.get-collection",
        "workflow-api.data.processes.inbox-entries.get",
        "workflow-api.data.processes.inbox-entries.get-collection"
    ));

This token reads schemes, process records, parameters, transitions, approvals, and inbox entries inside TenantA; it cannot mutate any Data entity or reach other tenants. For cross-tenant analyst access, replace DenyAllTenantsExcept("TenantA") with AllowAllTenants() as an explicit administrative decision.

Note on process history. Process-history operations (get-process-history, get-process-history-count) live under workflow-api.rpc.*, not the Data API. A reporting client that needs them should receive them as scoped read-only RPC exceptions.

When to use the Workflow Engine NEO RPC API

Architectural recommendation. Use the RPC API when the caller is meant to drive the runtime: start instances, execute commands, advance state. Bulk operations live exclusively on the RPC API, so any client that needs to start or progress many processes in one round-trip belongs here.

Documented behavior. The operations branch workflow-api.rpc covers, among others, workflow-api.rpc.bulk-create-instance, workflow-api.rpc.bulk-execute-command, workflow-api.rpc.bulk-get-process-instance, workflow-api.rpc.get-available-commands, workflow-api.rpc.delete-instance, and workflow-api.rpc.get-process-history, per the Workflow Engine documentation. Bulk operations, including both read-like bulk retrieval and runtime-changing operations, live under the RPC API branch.

The RPC API is the right surface for:

  • Polyglot integration. A Node.js worker, a Python ETL job, or a Java partner service that needs to start and progress Workflow Engine processes does so over plain HTTP using the RPC API. The host application stays on .NET, but the callers do not have to.
  • Front-end clients that drive workflows directly, such as a React or Angular UI that maps user actions onto workflow-api.rpc.execute-command or workflow-api.rpc.bulk-execute-command.
  • Backend microservices that orchestrate Workflow Engine state from outside the host process.
    Risk if ignored. Each RPC operation has a separate permission identifier. A token authorised for workflow-api.rpc.execute-command does not automatically gain access to workflow-api.rpc.delete-instance. In practice, however, teams tend to grant broad RPC allow lists to operational clients, which is where the real blast radius shows up. Keep execute-command, delete-instance, runtime control, and bulk operations in separate permission profiles, and only combine them inside dedicated admin tokens that you audit.

How WorkflowApiPermissions enforces the Data/RPC split at the claim level

Documented behavior. The Workflow Engine NEO permission model treats the Data/RPC split as a security primitive. Each operation has a concrete identifier under either workflow-api.data.* or workflow-api.rpc.*, and tokens are issued by passing concrete identifiers to the WorkflowApiPermissions builder. The Workflow Engine documentation states verbatim: “Operations are hierarchical. More specific rule wins on conflict.”

Architectural recommendation. Build claims from concrete operation identifiers. The pattern works for both Data API and RPC API personas:

Persona Builder pattern What the token can do
BI analyst, Data API only Deny all operations, allow concrete Data API operations ending with .get or .get-collection Read schemes, process records, parameters, transitions, approvals, and inbox entries; cannot create, update, or delete
Workflow operator, RPC without destructive ops Deny all operations, allow concrete RPC operations except delete-instance Start instances, execute commands, read; cannot delete
Admin excluding one tenant Allow all operations, then AllowAllTenantsExcept("TenantX") Full admin access except the excluded tenant
Tenant-scoped admin Allow all operations, then DenyAllTenantsExcept("TenantA") Full admin access inside one tenant only

Recipe table compiled from the v21 WorkflowApiPermissions builder examples in the Workflow Engine documentation, retrieved 2026-05-12.

Risk if ignored. The same compact format describes tenant access. A note on multi-tenancy from the Workflow Engine documentation: “The tenant header selects the tenant context only. It is not an authorization source. When security services are enabled, the current WorkflowApiPermissions claim must also allow the selected tenant. If the tenant permission is missing… the request is rejected with 403 Forbidden.” Architects who treat the Workflow-Api-Tenant-ID header as an authorization signal will eventually ship a tenant-cross-bleed bug. The header is routing; the claim is authorization.

A note for v21.0.0 readers: IWorkflowApiPermissions was redesigned in v21.0.0 and the synchronous API was removed across both the Data API and the RPC API, per the Workflow Engine release notes retrieved 2026-05-12. Earlier per-operation claim helpers such as AllowAll, AllowList, DenyAll, DenyList, GetAllPermissions, and ValidatePermissions were removed; use the compact WorkflowApiPermissions claim and the BuildClaim builder API from v21.0.0 onward.

Note: some public security pages still show pre-v21 helpers like AllowList(...). Follow the v21 release notes; treat older snippets as v20.x reference.

A decision table: which Workflow Engine NEO API does my caller need?

Architectural recommendation. The table below is the one I draw on whiteboards in support calls.

Use case Prefer Why Permission style
BI dashboard Data API Reads process records, parameters, transitions, approvals, and related Data API entities without runtime commands Only concrete get and get-collection permissions
Read-only support panel Data API Lower operational blast radius No create, update, or delete permissions
Audit and evidence export Data API Snapshot of state without affecting running work Read-only Data API allow list
External service starts workflows RPC API Requires runtime command execution Only create-instance, execute-command, possibly get-available-commands
Front-end UI executes commands RPC API Maps user actions onto runtime transitions Per-user token scoped by tenant and command set
Admin runtime control RPC API Requires start, stop, delete, status operations Separate privileged token, audited

Data API vs RPC API decision canvas

Decision canvas compiled by the author from the Workflow Engine NEO operation branches, 2026-05-12.

Risk if ignored. In multi-tenant deployments, avoid relying on an implicit default tenant for external clients. Prefer requiring every caller to send Workflow-Api-Tenant-ID, and ensure the token’s WorkflowApiPermissions claim also allows that tenant. Default-deny on the operation tree, allow only the concrete operations the caller’s job needs, deny destructive operations explicitly, per the Workflow API core services documentation.

Take this to your team

If you are the architect deciding how to surface Workflow Engine NEO to the rest of your organisation, here is the short version to bring to a design review.

Decision rule. Pick the Data API for any caller whose job description does not include changing runtime state. Pick the RPC API when the caller must drive the runtime, accepting that the token will carry a larger blast radius. Scope every token, on both APIs, from concrete operation identifiers via the v21 builder API; do not rely on conceptual branch names, and do not rely on the absence of UI affordances. The Data API is safer as an integration boundary, but only with a narrow allow list of read permissions. The API itself is not automatically read-only.

Candidate filter. With one trusted caller and no audit, BI, or analyst workload on the horizon, you can put everything behind the RPC API for now. With more than one consumer of workflow state, such as an analyst tool, a mobile app, or a partner service, split early. Retrofitting permission boundaries after the first quarter in production is expensive.

Cost of getting it wrong. A single RPC token with a broad allow list that lands in a BI tool can move workflow state in ways the audit log records but nobody watches. The same is true of a Data API token granted at branch level when the consumer needs only reads. I have seen both patterns in customer support conversations. The fix is organisational, not code: revoke the token, audit downstream callers, and explain to compliance why a read tool had write access for an unknown window.

Anticipated pushbacks.

  • “REST is simpler than two surfaces.” Both APIs are REST/HTTP and share an OpenAPI document; one generator produces one client. The operation tree is what you scope against, so the choice adds a permission boundary, not a protocol.
  • “We can just gate on roles in the host app.” Role checks in the host app hold only until somebody calls Workflow Engine NEO directly with a valid token. WorkflowApiPermissions enforces the boundary at the runtime, regardless of caller.
  • “We do not have multi-tenancy, so the security model is overkill.” The Data/RPC split is independent of multi-tenancy. A single-tenant deployment still benefits from a read-only Data API token for analytics and a separate privileged profile for destructive RPC operations.

A personal note from the author

Thank you for reading. If you have shipped Workflow Engine NEO and you split your Data and RPC clients differently from how I described, or chose not to split at all, I would like to hear about it. The support conversations I draw from are biased toward teams who are already debating the trade-off, and there are deployment patterns I will not see from where I sit.

The Data API and the RPC API are part of the licensed Workflow Engine NEO tier and I have a commercial interest in your choice. Treat the framing above as a starting point, not the end. If you disagree with the decision rule, please leave a comment below or push back on LinkedIn; that feedback will make the next article in this series sharper.

If you would rather walk through your own architecture, book a demo at workflowengine.io. Come with a concrete task and constraints, and we will help you scope a free POC against Workflow Engine NEO so you can validate the Data/RPC split on your real data before committing.

The Synthetic Insider

Most security teams still think in terms of external attackers.

Phishing emails.
Malware payloads.
Exploited servers.
Compromised endpoints.

But while studying modern offensive security models lately, I’ve started thinking more about something else entirely:

What happens when future intrusions stop looking external at all?

Not malware.

Not noisy exploits.

Not ransomware.

But operations that slowly evolve into what looks almost indistinguishable from legitimate internal activity.

I’ve been referring to this idea as the Synthetic Insider model.

An intrusion approach where the attacker’s objective is no longer just access…

but operational assimilation.

The Shift Away From “Breaking In”

Traditional intrusion models usually follow visible stages:

  • initial compromise
  • privilege escalation
  • lateral movement
  • persistence
  • exfiltration

Defenders learned to monitor many of these patterns.

But modern infrastructures are changing the environment attackers operate in.

Cloud ecosystems.
Identity federation.
SaaS-heavy operations.
Remote work.
AI-assisted workflows.
Machine-to-machine trust.

The more interconnected environments become, the more difficult it becomes to distinguish:

  • legitimate operational behavior
    from
  • malicious operational behavior.

And I think that distinction is going to become one of the biggest cybersecurity problems of the next decade.

Why Identity Changes Everything

In older environments, location mattered.

Internal network = trusted.
External network = suspicious.

That model is collapsing.

Now:

  • identities move everywhere
  • sessions persist across devices
  • APIs communicate continuously
  • SaaS platforms inherit trust automatically

Which means modern offensive operations increasingly revolve around:

  • permissions
  • trust chains
  • valid sessions
  • operational normalcy

not loud exploitation.

The goal is no longer:
“Can I break the system?”

It’s becoming:
“Can I become operationally indistinguishable from the system?”

The Dangerous Part Isn’t Access

It’s adaptation.

A sophisticated intrusion today may not immediately:

  • deploy malware
  • disable systems
  • trigger alerts

Instead, it may spend time learning:

  • workflow timing
  • communication patterns
  • access behaviors
  • approval logic
  • operational habits

At that point, the intrusion stops behaving like an external threat.

It starts behaving like a synthetic internal presence.

That’s where detection becomes extremely difficult.

Because technically:
nothing looks obviously malicious anymore.

SaaS + APIs Quietly Accelerate This Problem

One thing I’ve noticed while researching modern infrastructures is how much trust now exists between systems that security teams rarely fully visualize.

Applications trust:

  • APIs
  • automation workflows
  • service accounts
  • cloud roles
  • AI integrations
  • synchronization systems

And many of these interactions occur constantly in the background with minimal human oversight.

That creates an environment where malicious activity can increasingly hide inside:

  • legitimate automation
  • trusted integrations
  • approved workflows

not just compromised machines.

AI May Intensify Operational Blending

As organizations integrate AI deeper into infrastructure, another layer appears.

AI systems now:

  • summarize information
  • automate decisions
  • trigger actions
  • access internal data
  • interact with APIs

Which creates an interesting offensive-security question:

What happens when attackers stop targeting users directly…

and instead target the operational trust surrounding AI-driven systems?

At that point, intrusion may become less about technical compromise…

and more about influencing trusted automated behavior.

Why I Think This Matters

A lot of current security visibility still focuses on:

  • malware signatures
  • endpoint telemetry
  • exploit behavior
  • suspicious binaries

But the Synthetic Insider model lives somewhere else entirely.

Inside:

  • identity systems
  • workflow legitimacy
  • machine trust
  • operational patterns
  • approved infrastructure

And I honestly think many organizations are still psychologically preparing for attacks that are louder than what future intrusions may actually look like.

Final Thought

The most advanced offensive operations of the future may not look like attacks in the traditional sense.

No dramatic exploits.

No obvious payloads.

No visible chaos.

Just activity that slowly becomes so operationally normal that defenders struggle to separate trust from compromise.

And personally, I think that invisible transition from “external attacker” to “synthetic insider” is going to redefine how modern red teaming is understood.

Black Cipher
Exploring offensive security beyond visible intrusion.

Allient Inc. to Present at Upcoming Webinar on Engineering Thermally-Optimized Joints for Humanoids

The webinar will explore the key challenges in humanoid joint design and how motor data should be interpreted in a system-level context rather than relying on standard specifications. It will also look at practical engineering approaches to motor selection and performance optimization for humanoid and advanced robotics applications, with a focus on achieving reliable, thermally efficient performance.

How I built a developer portfolio template in Next.js 15 — typography-first design decisions

I’ve been building and selling Next.js templates as a side project. This is my fifth one: Craft, a developer portfolio template with a typography-first approach.

Here are all the decisions I made.

Why typography-first?

Most dev portfolio templates use gradients, animations, and card grids to grab attention. I wanted to try the opposite — let the type do the work, and keep everything else minimal.

Font pairing: Lora (serif) for headings, JetBrains Mono for labels and code, Inter for body text. Three families that complement each other without fighting.

The serif gives headings an editorial quality that stands out from the usual Inter/Geist sans-serif stack.

The blinking cursor

The hero has an animated blinking cursor after the tech stack line:

function Cursor() {
  const [visible, setVisible] = useState(true)
  useEffect(() => {
    const id = setInterval(() => setVisible(v => !v), 530)
    return () => clearInterval(id)
  }, [])
  return (
    _
  )
}

530ms interval feels natural — not too fast, not too slow. Hidden from screen readers via aria-hidden.

The bento grid for projects

Same “1px gap trick” I use in all my templates: grid background = border color, gap = 1px, no borders on individual cards.

.grid {
  display: grid;
  grid-template-columns: repeat(3, 1fr);
  gap: 1px;
  background: var(--border-sub); /* this IS the border */
  border-radius: var(--r-lg);
  overflow: hidden;
}

/* First project gets more visual weight */
.wide { grid-column: span 2; }

The experience timeline

Pure CSS — no JavaScript, no library:

.timeline {
  position: relative;
}

/* The vertical line */
.timeline::before {
  content: '';
  position: absolute;
  left: 56px;
  top: 6px; bottom: 6px;
  width: 1px;
  background: var(--border);
}

.item {
  display: grid;
  grid-template-columns: 56px 16px 1fr;
  gap: 0 20px;
}

The three-column grid: year column (fixed width) → dot column (fixed width) → content. The ::before pseudo-element on the list creates the connector line behind all dots.

All content in one file

src/lib/constants.ts exports everything the buyer will want to edit:

  • Personal info (name, role, tagline, bio, email)
  • Social links
  • Stack (with highlight: true for primary skills)
  • Projects (with live URL, repo URL, metrics)
  • Experience
  • Services
  • SEO metadata

No need to touch any component file. Change constants.ts and the entire site updates.

Real favicon included

This is the thing most templates skip. I include:

  • public/favicon.ico — browser tab icon
  • public/favicon.svg — modern SVG version

And in layout.tsx:



Both are required. Some browsers prefer .ico, modern browsers prefer .svg.

Live demo: https://craft-portfolio-rose.vercel.app

Available on Gumroad for $39: https://devmaya.gumroad.com/l/craft

HABIT VERSE – Lifestyle & Game · Full Stack · AI-Powered · 3D WebGL

Inspiration

Most habit tracking apps fail not because users forget their goals — but because they stop caring. Streaks feel like data entry. There’s no emotional stake. We asked ourselves: what if missing a habit didn’t just break a streak — it cracked your entire world apart? That question became HabitVerse.

What it does

HabitVerse is a full-stack web app where your daily habits build a living, breathing 3D planet. Complete your sleep, hydration, exercise, and study goals — watch terrain rise, oceans fill, forests grow, and your sky turn golden. Miss them — storms form, land cracks, and your world begins to die.

There are no charts. No spreadsheets. The planet IS the dashboard.

Every habit appears as an orbiting moon around your planet. Click a moon, check in your habit, and watch the terrain morph in real time with particle bursts and animations. Miss a habit and the ground literally cracks with a screen shake.

The Planet Oracle — powered by Gemini AI — narrates your planet’s state like a nature documentary, referencing your actual streak data and generating personalized daily challenges.

The Time Machine lets you scrub through your planet’s entire history — watching mountains grow, oceans fill, or seeing the exact moment you fell off track.

How we built it

  • Frontend: React + Vite, Three.js via React Three Fiber for the 3D planet, GSAP for camera animations, Framer Motion for UI spring animations, Tailwind CSS
  • Backend: Node.js + Express, MongoDB with Mongoose, JWT authentication
  • AI: Gemini AI API powering the Planet Oracle with full habit context in every prompt
  • Database: Supabase Edge Functions for the Oracle chat route
  • Deploy: Vercel (frontend), Railway (backend)

Every single interaction has a micro-animation. Nothing in HabitVerse is static.

Challenges we faced

  • Building a real-time 3D planet that morphs terrain based on live habit data using Three.js morph targets was the hardest technical challenge — syncing planet state with the database without performance drops required careful optimization.
  • Designing the Planet Oracle prompt so Gemini always responds with personalized, dramatic narration — never generic advice — required multiple iterations of the system prompt with strict context injection.
  • Making the Time Machine scrubber smoothly morph the planet between historical snapshots required storing daily planet state as JSON and interpolating between morph targets on the fly.

What we learned

  • How to build and animate 3D WebGL experiences in the browser using Three.js
  • How to design AI prompts that feel personal and contextual rather than generic
  • That emotional design is the most powerful retention mechanic — users return not because of reminders, but because they genuinely don’t want their planet to die
  • Full-stack architecture with Supabase Edge Functions for secure AI API proxying

What’s next for HabitVerse

  • Native mobile app (iOS/Android) with push notifications tied to planet health decay
  • Multiplayer planets — build a shared world with friends or teammates
  • Voice check-ins via the Planet Oracle
  • Custom habit categories and AI-generated habit suggestions based on your goals
  • Export your planet’s evolution as a shareable timelapse video

Logging Your AI Events (from Ollama) in Bronto

Searching for log events containing

Authored by David Tracey

Many software companies are investigating the use of Large Language Models (LLMs) in their products. At Bronto we’ve announced our Bronto Labs initiative, with AI features including auto-parsing, AI dashboard creation, and Bronto Scope for error investigation.

This post explores a different angle: using logs in the development of AI applications. We’ll focus on Ollama — an open source tool for running LLMs locally — and show how to pipe its logs into Bronto for search and analysis.

LLMs are complex, non-deterministic systems. Beyond traditional logging use cases (performance monitoring, API usage), their unpredictable nature increases the need for logging — particularly to record and track responses to prompts. Individual log events can be large when they include a full prompt or response. Meta found this problem significant enough at their scale to build a dedicated Meta AI Logging Engine.

The fundamental requirements for logging AI applications are:

  • Ability to handle large log events
  • Ability to handle high volumes at low cost
  • Ability to search across high volumes quickly

These are exactly the requirements Bronto was designed to meet.

Setting Up Ollama

Recommended specs:

  • 16GB RAM (8GB works for smaller models)
  • 12GB disk space for Ollama and basic models
  • Modern CPU with at least 4 cores (8 preferred)
  • Optional: GPU for improved performance

Install and Run the Server

Install from ollama.com/download for your OS, then start the server:

ollama serve

You’ll see output including the default port it’s listening on (11434).

Download and Run a Model

# Pull a model from the registry
ollama pull gemma:2b

# List downloaded models
ollama list

# Run a model interactively
ollama run gemma:2b

The run command gives you a >>> prompt where you can enter prompts or /help for commands.

Sending Ollama Logs to Bronto

Step 1: Configure Ollama Logging to File

Stop the server and restart it writing logs to a file:

ollama serve > /your_log_path/.ollama/logs/server.log 2>&1

For more detailed debug logs, add to your shell profile (.zprofile etc.):

export OLLAMA_LOG_LEVEL=DEBUG
export OLLAMA_DEBUG=true

To redirect model client logs:

# stderr only (keeps console interactive)
ollama run gemma:2b 2>>/your_log_path/.ollama/logs/gemma.log

# both stdout and stderr (API use only — disables console input)
ollama run gemma:2b > /your_log_path/.ollama/logs/gemma.log 2>&1

Verify logs are flowing:

tail -f /your_log_path/.ollama/logs/server.log

Step 2: Install OpenTelemetry Collector

Download for your platform from opentelemetry.io. Example for Mac ARM64:

curl --proto '=https' --tlsv1.2 -fOL 
  https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.114.0/otelcol-contrib_0.114.0_darwin_arm64.tar.gz

chmod +x otelcol-contrib
mv otelcol-contrib /usr/local/bin/otelcol

# Verify
otelcol --version

Step 3: Configure OpenTelemetry to Forward to Bronto

Create /etc/otelcol/config.yaml:

receivers:
  filelog/Ollama_Server:
    include:
      - /your_log_path/.ollama/logs/server.log
    resource:
      service.name: LaptopServer
      service.namespace: Ollama

  filelog/Ollama_Gemma:
    include:
      - /your_log_path/.ollama/logs/gemma.log
    resource:
      service.name: LaptopGemma
      service.namespace: Ollama

processors:
  batch:

exporters:
  otlphttp/brontobytes:
    logs_endpoint: "https://ingestion.us.bronto.io/v1/logs"
    compression: none
    headers:
      x-bronto-api-key: replace_this_with_your_bronto_apikey

service:
  pipelines:
    logs:
      receivers: [filelog/Ollama_Server, filelog/Ollama_Gemma]
      processors: [batch]
      exporters: [otlphttp/brontobytes]
  # Useful for debugging:
  # telemetry:
  #   logs:
  #     level: "debug"
  #     output_paths: [/your_log_path/otelcol/debug.log]

Validate and run:

otelcol validate --config=/etc/otelcol/config.yaml
otelcol --config=/etc/otelcol/config.yaml

A Simple Ollama API Program

The Python script below (ollama-log-demo.py) uses the Ollama API to send prompts against a log file and print the response. Example usage:

# Summarize 100 lines of CDN logs
python3 ollama-log-demo.py 100lines-CDN-log.csv 
  --model "gemma:2b" 
  --prompt "You have been given 100 lines from a CDN log in CSV format. Summarise the logs provided."

# Find errors and suggest fixes
python3 ollama-log-demo.py 100lines-search-log.csv 
  --model "gemma:2b" 
  --prompt "Find errors in this log and suggest how to fix them"

The final line of each Ollama response includes useful performance metadata:

Field Description
total_duration Total time spent generating the response
load_duration Time spent loading the model (nanoseconds)
prompt_eval_count Number of tokens in the prompt
prompt_eval_duration Time spent evaluating the prompt (nanoseconds)
eval_count Number of tokens in the response
eval_duration Time spent generating the response (nanoseconds)
context Conversation encoding — pass in next request to maintain memory
response Empty if streamed; full response if not streamed

Model notes from testing: gemma:2b is good for summarizing but tends to give high-level summaries even when asked for specifics. mistral takes longer but produces more detailed, data-specific responses. Defining the right prompt for your use case is key.

Searching Ollama Logs in Bronto

Ollama server logs include a mix of structured and unstructured entries:

Standard log levels:

INFO [main] HTTP server listening | hostname="127.0.0.1" port="11434"
level=INFO source=sched.go:714 msg="new model will fit in available VRAM"
level=DEBUG source=memory.go:103 msg=evaluating library=metal gpu_count=1

Model and resource logs:

llm_load_print_meta: max token length = 93
llama_model_loader: - kv 0: general.architecture str = gemma
level=INFO source=server.go:105 msg="system memory" total="8.0 GiB" free="1.2 GiB"

Even a small test with short prompts generates surprisingly large log volumes — 244 events totaling ~2MB in our test. Bronto handles these unstructured and semi-structured formats natively, and you can add a custom parser to make them more convenient to search and view.

Example searches in Bronto:
Fig.1 — Searching for log events containing “tokens”
Searching for log events containing

Fig.2 — Searching for log events containing “prompt”
Searching for log events containing
Fig.3 — Grouping by prompt evaluation time per task_id

Grouping by prompt evaluation time per task_id

Conclusion

This post introduced Ollama as an example of an LLM system and explained why AI applications create unique logging challenges — large events, high volumes, non-deterministic outputs, and distributed agents. We walked through setting up Ollama locally, configuring OpenTelemetry to forward logs to Bronto, and writing a simple Python API program to experiment with prompts against log data.

Future posts will develop the theme further with other AI systems including AWS Bedrock.

Appendix: ollama-log-demo.py

import argparse
import json
import requests


def print_ollama_stats(json_response):
    load_duration = json_response.get("load_duration")
    if load_duration:
        print("n--- load_duration = ", load_duration)

    total_duration = json_response.get("total_duration")
    if total_duration:
        print("n--- total_duration = ", total_duration)

    eval_duration = json_response.get("eval_duration")
    if eval_duration:
        print("n--- eval_duration = ", eval_duration)

    prompt_eval_duration = json_response.get("prompt_eval_duration")
    if prompt_eval_duration:
        print("n--- prompt_eval_duration = ", prompt_eval_duration)

    prompt_eval_count = json_response.get("prompt_eval_count")
    if prompt_eval_count:
        print("n--- prompt_eval_count = ", prompt_eval_count)

    eval_count = json_response.get("eval_count")
    if eval_count:
        print("n--- eval_count = ", eval_count)


def examine_log_with_prompt(file_path, input_prompt, input_model):
    with open(file_path, 'r') as file:
        log_data = file.read()

    req_params = {
        "model": input_model,
        "prompt": f"{input_prompt}nn{log_data}"
    }

    try:
        # Update localhost URL to match your Ollama API endpoint
        response = requests.post(
            "http://localhost:11434/api/generate",
            headers={"Content-Type": "application/json"},
            data=json.dumps(req_params),
            stream=True
        )
        if response.status_code == 200:
            print("n--- Processing Successful Ollama Response ---")
            line_count = 0
            for line in response.iter_lines():
                if line:
                    try:
                        json_line = line.decode('utf-8')
                        line_count += 1
                        json_response = json.loads(json_line)
                        print(json_response["response"], end='', flush=True)
                    except json.JSONDecodeError as e:
                        print(f"Error decoding JSON on line {line_count + 1}: {e}")
                    except UnicodeDecodeError as e:
                        print(f"Error decoding line to UTF-8 on line {line_count + 1}: {e}")
            if line_count == 0:
                print("No JSON lines found or response was empty.")
            print("n--------------------------------------------------")
            print_ollama_stats(json_response)
            print("n--------------------------------------------------")
        else:
            print(f"nError - Response Status code: {response.status_code}")
            print(response.text)
    except Exception as e:
        print(e)


def main():
    parser = argparse.ArgumentParser(description="Ollama API Demo for Logs")
    parser.add_argument('file', type=str, help='Path to the log file to be examined')
    parser.add_argument('--model', type=str, help='Model to use in analysis', default=None)
    parser.add_argument('--prompt', type=str, help='Prompt to send to model', default=None)
    args = parser.parse_args()
    examine_log_with_prompt(args.file, args.prompt, args.model)


if __name__ == "__main__":
    main()

Explore Bronto’s AI Features