How We Reduced LLM Costs Without Touching Model Quality

One of the fastest ways to destroy an AI system in production is uncontrolled token growth.

Most demos ignore this problem because they run small prompts against clean datasets. Real enterprise systems do not behave like that.

Once multiple integrations start running together, token usage grows faster than most teams expect.

We started seeing it after several enterprise pipelines went live at the same time.

• Slack ingestion
• Email synchronization
• CRM updates
• Meeting transcripts
• Internal ticket systems
• Knowledge base sync jobs

Everything was feeding into the same operational AI layer.

At first, nothing looked broken.

Responses were accurate.
Latency was acceptable.
Users were happy.

But infrastructure metrics told a different story.

Prompt sizes were growing continuously.
Costs increased every week.
Some requests carried massive amounts of unnecessary context.

The issue was not the model itself.

The issue was everything surrounding the model.

The Real Problem Was Context Inflation

A single request slowly turned into this:

• duplicated conversation history
• overlapping retrieval chunks
• unnecessary metadata
• old execution traces
• repeated system instructions
• temporary tool outputs nobody needed anymore

The worst part was that response quality barely changed.

We were spending more money to process noise.

That forced us to look at the architecture instead of blaming model pricing.

What We Changed

We Stopped Treating Retrieval Like Free Context

Initially, retrieval output was pushed directly into prompts.

That works during early development.

It breaks during long-running enterprise operation.

Vector search systems naturally return overlapping information. As datasets grow, overlap increases even more.

We added a preprocessing layer before prompt assembly.

Now every retrieval result passes through:

• semantic deduplication
• overlap removal
• metadata cleanup
• token budgeting
• context prioritization

This immediately reduced prompt size across production workloads.

The important part was that output quality stayed almost identical.

That was the moment we realized how much useless data was entering the system.

We Split Operational Memory From Reasoning Memory

This changed the architecture more than anything else.

Most AI systems mix all state together:

• chat history
• tool outputs
• execution logs
• retry traces
• retrieval data
• audit metadata

The model does not need all of that for reasoning.

So we separated memory into layers.

Operational memory stores infrastructure state:

• retries
• execution traces
• audit logs
• system metadata

Reasoning memory stores only the information required for inference.

That separation reduced context pollution heavily.

It also made debugging easier because infrastructure concerns stopped leaking into model reasoning.

We Reduced Prompt Complexity

Large prompts feel productive.

They usually are not.

Over time we noticed many system prompts were repeating the same instructions in different wording.

That increased tokens without improving reliability.

Instead of adding more prompt logic, we moved more control into infrastructure logic.

We added:

• structured validation layers
• schema enforcement
• routing constraints
• tool permission boundaries
• deterministic execution rules

The result was smaller prompts with more predictable behavior.

The infrastructure became responsible for operational control instead of pushing everything into the model.

We Added Token Observability Everywhere

This should exist in every production AI system.

Without token observability, cost problems stay invisible for weeks.

We now track:

• token usage per tenant
• token usage per integration
• retrieval expansion rates
• average context growth
• abnormal cost spikes after deployments

One deployment accidentally tripled token usage because a serializer started injecting entire API payloads into conversation state.

The system still worked.

Nobody noticed immediately.

Without observability, we would have discovered it only after billing increased significantly.

The Bigger Lesson

Most enterprise AI cost problems are not model problems.

They are architecture problems.

The expensive part is usually not inference itself.

It is:

• poor memory design
• uncontrolled retrieval
• duplicated context
• oversized prompts
• weak operational boundaries

Reducing waste matters more than constantly changing models.

We did not downgrade quality.

We did not switch providers.

We fixed the infrastructure around the model.

That changed the economics of the system far more than any prompt optimization ever did.

I usually write CUDA code in C++, but recently I have been spending more time with Python, especially PyTorch and Numba.

Numba is interesting because it lets you write a real GPU kernel directly in Python. You can decorate a function with @cuda.jit, launch it with kernel[grid, block](...), and Numba compiles it down to GPU machine code that runs on the actual hardware.

The good news is that GPUFlight can profile Python GPU programs as well.

In this post, I’ll profile a simple Numba matrix multiplication kernel with GPUFlight. Then I’ll read the report step by step and show how the report points to a real optimization: shared-memory tiling.

One important note before we start: this example uses GPUFlight’s deeper profiling mode with SASS-level metrics and PC sampling. So the duration numbers in the report should not be treated as clean baseline kernel timing. They include profiling overhead. The main goal here is not to benchmark Numba against an optimized library like cuBLAS. The goal is to show how GPUFlight helps explain what is happening inside the kernel.

Setup

Both GPUFlight and Numba can be installed from PyPI. On a fresh Linux machine:

sudo apt-get install -y python3.12-venv
python3 --version            # expect Python 3.12.x

python3 -m venv ~/gpufl-venv
source ~/gpufl-venv/bin/activate

pip install --upgrade pip
pip install gpufl "numba-cuda[cu13]"

python -c "import gpufl; print('gpufl', gpufl.__version__)"

You should see something like:

gpufl 1.x.x

At the time I am writing this, the version is 1.0.2.

Before using the profiler, it is a good idea to confirm that Numba can find your GPU:

python -c "from numba import cuda; print('cuda available:', cuda.is_available()); cuda.detect()"

Now we are ready to run a Python CUDA application with GPUFlight.

The sample kernel

Here is the sample code I am using:

import gpufl as gfl
from gpufl.report import generate_report
from numba import cuda
import numpy as np
import math
import os

@cuda.jit
def matmul_kernel(A, B, C):
    row, col = cuda.grid(2)

    if row < C.shape[0] and col < C.shape[1]:
        tmp = 0.0

        for k in range(A.shape[1]):
            tmp += A[row, k] * B[k, col]

        C[row, col] = tmp

LOG_PATH = "./gfl_logs"

gfl.init(
    app_name="matmul_sample",
    log_path=LOG_PATH,
    sampling_auto_start=True,
    system_sample_rate_ms=100,
    profiling_engine=gfl.ProfilingEngine.PcSamplingWithSass,
)

try:
    N = 2048

    A = cuda.to_device(np.random.rand(N, N).astype(np.float32))
    B = cuda.to_device(np.random.rand(N, N).astype(np.float32))
    C = cuda.to_device(np.zeros((N, N), dtype=np.float32))

    tpb = (16, 16)
    bpg = (math.ceil(N / tpb[0]), math.ceil(N / tpb[1]))

    with gfl.Scope("matrix_mul_compute", "math"):
        for _ in range(10):
            matmul_kernel[bpg, tpb](A, B, C)

    _ = C.copy_to_host()
    print("[OK] compute finished")

finally:
    gfl.shutdown()

    print(
        generate_report(
            os.path.dirname(LOG_PATH) or ".",
            log_prefix=os.path.basename(LOG_PATH),
            top_n=10,
        )
    )

This is a very simple matrix multiplication kernel.

Each thread computes one output element. For each element, the thread walks through one full row of A and one full column of B.

This is intentionally not optimized. I want to start with a simple kernel, because it makes the profiling report easier to understand.

Let’s run it and see what GPUFlight tells us.

===============================================================================
                           GPU Flight Session Report
                       Generated: 2026-05-22 05:05:33 UTC
===============================================================================

===============================================================================
  Session Summary
===============================================================================
  Application:          matmul_sample
  Session ID:           565d3c32-86cc-415d-8642-9c140f856f2b
  Duration:             17.91 s
  GPU Device:           NVIDIA GeForce RTX 5060 Laptop GPU
    SMs:                26
    Registers/Block:    65536

===============================================================================
  Kernel Execution Summary
===============================================================================
  Total Kernels:        10
  Unique Kernels:       1
  Total GPU Time:       17.40 s
  GPU Busy:             97.2%
  Avg Duration:         1.74 s
  Median Duration:      1.74 s
  Min Duration:         1.71 s
  Max Duration:         1.78 s

===============================================================================
  Top 10 Kernels by Total GPU Time
===============================================================================
  #   Kernel                                   Calls       Total         Avg         Max
  --------------------------------------------------------------------------------------
  1   __main__::matmul_kernel                     10     17.40 s      1.74 s      1.78 s

===============================================================================
  Kernel Details (Top 10)
===============================================================================

  __main__::matmul_kernel
  =======================
    Grid:               (128,128,1)
    Block:              (16,16,1)
    Occupancy:          100.0%
    Reg Occupancy:      100.0%
    SMem Occupancy:     100.0%
    Warp Occupancy:     100.0%
    Block Occupancy:    100.0%
    Limiting Resource:  warps
    Registers/Thread:   40
    Shared Memory:      0 B dyn + 0 B static

===============================================================================
  Memory Transfer Summary
===============================================================================
  Total Transfers:      4
  Total Bytes:          64.0 MB

  Direction      Count     Total Bytes    Avg Throughput
  ------------------------------------------------------
  HtoD               3         48.0 MB        11.68 GB/s
  DtoH               1         16.0 MB         4.40 GB/s

===============================================================================
  System Metrics
===============================================================================
  GPU Metrics:
    Utilization:        avg 96.6%  peak 100%  min 0%
    Temperature:        avg 53.4 C  peak 58 C
    Power:              avg 71.0 W  peak 75.6 W
    VRAM Usage:         peak 1105 MiB
    SM Clock:           avg 2631 MHz  peak 2790 MHz

  Host Metrics:
    CPU Utilization:    avg 8.6%  peak 29.1%
    RAM Usage:          peak 27593 / 32189 MiB (85.7%)

===============================================================================
  Scope Summary
===============================================================================
  Scope Timing:
  Scope                          Calls       Total         Avg         Max
  ------------------------------------------------------------------------
  matrix_mul_compute                 1   195.21 ms   195.21 ms   195.21 ms

  GPU Time by Scope:
  Scope                          Kernels      GPU Time         Avg
  ----------------------------------------------------------------
  matrix_mul_compute                  10       17.40 s      1.74 s

===============================================================================
  Profile / SASS Analysis
===============================================================================

  SASS Metrics Summary:
  Metric                                                   Total
  --------------------------------------------------------------
  smsp__sass_thread_inst_executed                   2235815690240
  smsp__sass_inst_executed                           69869240320
  smsp__sass_sectors_mem_global                      45654999040
  smsp__sass_sectors_mem_global_ideal                13427015680

  Thread Divergence Analysis:
    Warp Instructions:    69869240320
    Thread Instructions:  2235815690240
    Avg Threads/Warp:     32.0 / 32
    Warp Efficiency:      100.0%

Now let’s read the report carefully.

A profiling report is only useful if we can turn it into a decision. So instead of just looking at numbers, I usually ask a few questions.

1. Is the GPU actually busy?

Yes.

The report shows:

GPU Busy:             97.2%
GPU Util avg:         96.6%
Total GPU Time:       17.40 s
Duration:             17.91 s

This means the GPU was working for almost the entire run. Out of 17.91 s of wall-clock time, 17.40 s were spent running GPU kernels.

The SM clock is also boosted to 2631 MHz, and power is around 71.0 W, which is close to the laptop GPU’s power limit.

So this is not a case where the CPU is too slow, the input data is too small, or the GPU is waiting for work. The GPU is busy.

That means if we want to improve performance, we need to look inside the kernel.

2. How long did each profiled launch take?

The report shows:

Avg Duration:         1.74 s
Median Duration:      1.74 s
Min Duration:         1.71 s
Max Duration:         1.78 s

However, this number needs to be read carefully.

This run includes deeper profiling, including SASS-level metrics and sampling. That means the measured duration includes profiling overhead. So I should not treat 1.74 s as the clean baseline runtime of the kernel.

I would not use this number alone to claim how fast or slow the raw Numba kernel is. But it is still useful as the runtime under this profiling configuration.

3. Is the problem occupancy?

Probably not.

The report shows:

Occupancy:          100.0%
Reg Occupancy:      100.0%
SMem Occupancy:     100.0%
Warp Occupancy:     100.0%
Block Occupancy:    100.0%
Limiting Resource:  warps

This tells us the GPU has enough active warps. The SMs are not sitting empty because we launched too few threads.

Occupancy is not the same thing as performance, but in this case low occupancy does not look like the main problem.

4. Is the problem thread divergence?

Also no.

The report shows:

Avg Threads/Warp:     32.0 / 32
Warp Efficiency:      100.0%

This means every warp is using all 32 threads. There is no meaningful branch divergence here.

That makes sense because the kernel is simple. The 16 x 16 block and 128 x 128 grid map cleanly to the 2048 x 2048 output matrix.

So far, the report says:

• The GPU is busy.
• Occupancy is high.
• Warp efficiency is perfect.

So now we need to look at memory behavior.

5. What do the memory sectors say?

This is the most useful part of the report:

SASS Metrics Summary:
Metric                                                   Total
--------------------------------------------------------------
smsp__sass_thread_inst_executed                   2235815690240
smsp__sass_inst_executed                           69869240320
smsp__sass_sectors_mem_global                      45654999040
smsp__sass_sectors_mem_global_ideal                13427015680

The important two numbers are:

smsp__sass_sectors_mem_global          45,654,999,040
smsp__sass_sectors_mem_global_ideal    13,427,015,680

The kernel is accessing about 45.7B global memory sectors, while the ideal number is about 13.4B.

That is roughly:

45.7 / 13.4 ≈ 3.4x

So the kernel is moving about 3.4x more global memory traffic than the ideal case.

Another way to read it:

13.4 / 45.7 ≈ 29%

The memory access efficiency is only around 29%.

This is the real story.

The naive kernel makes each thread re-read values from global memory. Many threads need overlapping data from A and B, but the kernel does not reuse that data efficiently. So the same data crosses the memory system again and again.

The GPU is busy, the warps are full, and the lanes are active. But the memory access pattern is wasteful.

6. The fix: shared-memory tiling

For this kind of matrix multiplication kernel, the classic fix is shared-memory tiling.

Instead of letting each thread repeatedly read everything from global memory, each block cooperatively loads a tile of A and a tile of B into shared memory. Then the threads reuse those values many times before loading the next tile.

Here is the improved kernel:

from numba import cuda, float32

TPB = 16

@cuda.jit
def matmul_kernel_perf(A, B, C):
    sA = cuda.shared.array((TPB, TPB), dtype=float32)
    sB = cuda.shared.array((TPB, TPB), dtype=float32)

    x, y = cuda.grid(2)

    tx = cuda.threadIdx.x
    ty = cuda.threadIdx.y

    tmp = float32(0.0)

    n_tiles = (A.shape[1] + TPB - 1) // TPB

    for i in range(n_tiles):
        sA[ty, tx] = 0.0
        sB[ty, tx] = 0.0

        if y < A.shape[0] and (tx + i * TPB) < A.shape[1]:
            sA[ty, tx] = A[y, tx + i * TPB]

        if x < B.shape[1] and (ty + i * TPB) < B.shape[0]:
            sB[ty, tx] = B[ty + i * TPB, x]

        cuda.syncthreads()

        for j in range(TPB):
            tmp += sA[ty, j] * sB[j, tx]

        cuda.syncthreads()

    if y < C.shape[0] and x < C.shape[1]:
        C[y, x] = tmp

Now let’s run the same profiling mode again.

===============================================================================
                           GPU Flight Session Report
                       Generated: 2026-05-22 05:20:40 UTC
===============================================================================

===============================================================================
  Session Summary
===============================================================================
  Application:          matmul_sample_perf
  Session ID:           d44e5478-ba19-4cd1-b3cf-f6d31ab8b0ca
  Duration:             2.90 s
  GPU Device:           NVIDIA GeForce RTX 5060 Laptop GPU
    SMs:                26
    Registers/Block:    65536

===============================================================================
  Kernel Execution Summary
===============================================================================
  Total Kernels:        10
  Unique Kernels:       1
  Total GPU Time:       2.22 s
  GPU Busy:             76.4%
  Avg Duration:         221.64 ms
  Median Duration:      216.89 ms
  Min Duration:         215.38 ms
  Max Duration:         250.06 ms

===============================================================================
  Top 10 Kernels by Total GPU Time
===============================================================================
  #   Kernel                                   Calls       Total         Avg         Max
  --------------------------------------------------------------------------------------
  1   __main__::matmul_kernel_perf                10      2.22 s   221.64 ms   250.06 ms

===============================================================================
  Kernel Details (Top 10)
===============================================================================

  __main__::matmul_kernel_perf
  ============================
    Grid:               (128,128,1)
    Block:              (16,16,1)
    Occupancy:          100.0%
    Reg Occupancy:      100.0%
    SMem Occupancy:     100.0%
    Warp Occupancy:     100.0%
    Block Occupancy:    100.0%
    Limiting Resource:  warps
    Registers/Thread:   37
    Shared Memory:      0 B dyn + 2.0 KB static

===============================================================================
  Memory Transfer Summary
===============================================================================
  Total Transfers:      4
  Total Bytes:          64.0 MB

  Direction      Count     Total Bytes    Avg Throughput
  ------------------------------------------------------
  HtoD               3         48.0 MB         9.87 GB/s
  DtoH               1         16.0 MB         4.45 GB/s

===============================================================================
  System Metrics
===============================================================================
  GPU Metrics:
    Utilization:        avg 74.9%  peak 100%  min 0%
    Temperature:        avg 43.0 C  peak 48 C
    Power:              avg 51.0 W  peak 76.1 W
    VRAM Usage:         peak 958 MiB
    SM Clock:           avg 2180 MHz  peak 2812 MHz

  Host Metrics:
    CPU Utilization:    avg 16.0%  peak 46.0%
    RAM Usage:          peak 27019 / 32189 MiB (83.9%)

===============================================================================
  Scope Summary
===============================================================================
  Scope Timing:
  Scope                          Calls       Total         Avg         Max
  ------------------------------------------------------------------------
  matrix_mul_compute_perf            1   330.58 ms   330.58 ms   330.58 ms

  GPU Time by Scope:
  Scope                          Kernels      GPU Time         Avg
  ----------------------------------------------------------------
  matrix_mul_compute_perf             10        2.22 s   221.64 ms

===============================================================================
  Profile / SASS Analysis
===============================================================================

  SASS Metrics Summary:
  Metric                                                   Total
  --------------------------------------------------------------
  smsp__sass_thread_inst_executed                   298005299200
  smsp__sass_inst_executed                            9312665600
  smsp__sass_sectors_mem_global                       1347420160
  smsp__sass_sectors_mem_global_ideal                 1347420160

  Thread Divergence Analysis:
    Warp Instructions:    9312665600
    Thread Instructions:  298005299200
    Avg Threads/Warp:     32.0 / 32
    Warp Efficiency:      100.0%

The result is much better under the same profiling configuration.

The full session duration goes down from 17.91 s to 2.90 s.

Total GPU time goes down from 17.40 s to 2.22 s.

The average profiled kernel duration goes down from 1.74 s to 221.64 ms.

Again, these are still profiled durations, not clean baseline timings. But because both runs use the same deep profiling mode, this comparison is still useful. It tells us the tiled version behaves much better under the same measurement setup.

7. What changed?

The most important change is in the memory-sector metrics.

Naive version:

smsp__sass_sectors_mem_global          45,654,999,040
smsp__sass_sectors_mem_global_ideal    13,427,015,680

Tiled version:

smsp__sass_sectors_mem_global           1,347,420,160
smsp__sass_sectors_mem_global_ideal     1,347,420,160

In the naive kernel, actual global memory sectors were about 3.4x higher than ideal.

In the tiled kernel, actual and ideal global memory sectors are the same.

That is exactly what we wanted to see.

The optimized kernel also uses shared memory:

Shared Memory:      0 B dyn + 2.0 KB static

That means each block is now reusing data through shared memory instead of repeatedly pulling the same values from global memory.

Instruction count also drops a lot:

Naive thread instructions:  2,235,815,690,240
Tiled thread instructions:    298,005,299,200

So the optimized kernel is not only reducing memory traffic. It is also doing much less total instruction work.

Summary

This example is not a full benchmark. I am not comparing Numba against cuBLAS, and I am not claiming these numbers are the raw kernel runtimes. The run uses SASS-level profiling and sampling, so there is overhead.

But the report is still useful because both versions were measured with the same profiling mode. More importantly, the report explains why the naive kernel is slow.

The first version had:

• high GPU utilization,
• 100% occupancy,
• 100% warp efficiency,
• but very inefficient global memory access.

That means the problem was not lack of work or branch divergence. The problem was the memory access pattern.

After changing the kernel to use shared-memory tiling:

• total profiled GPU time dropped from 17.40 s to 2.22 s,
• average profiled kernel time dropped from 1.74 s to 221.64 ms,
• global memory sectors dropped from 45.65B to 1.35B,
• and actual global memory sectors matched the ideal number.

So the main takeaway is not just “the optimized kernel is faster.”

The more important takeaway is that GPUFlight helped point to the right fix. The report showed that the naive kernel was wasting memory bandwidth, and the optimized version confirmed that shared-memory tiling reduced that waste.

That is the workflow I want GPUFlight to support:

Run your program normally, collect useful GPU metrics, and turn the report into a concrete optimization decision.

What a decade of observing hybrid warfare ecosystems reveals about where we are now.
There is a moment, when you have been watching something long enough, where the pattern stops feeling like analysis and starts feeling like memory.

I have spent well over a decade monitoring the intersection of hybrid warfare operations, dark web criminal ecosystems, and the information environments that connect them. I did not start this work from a think tank, an intelligence agency, or a university. I started it from inside the communities being studied observing in real time, from a NATO Eastern Flank position, as the architecture of modern information warfare was being assembled around me.

What I want to share here is not academic. It is observational. And the observation that matters most right now is this:

The actors who built the first generation of hybrid warfare infrastructure are still operational. They are significantly more capable. And the population they are targeting is significantly more susceptible than it was when this started.

The Architecture That Was Being Built While No One Was Watching
When the first coordinated information operations appeared across European social media platforms in early 2014, the analysis that followed treated them as a novel phenomenon. What direct community monitoring revealed was something different: the infrastructure had been under construction for months. Communities with ostensibly cultural or historical focus, cultivated across multiple languages simultaneously, activated as coordinated distribution networks within hours of a triggering geopolitical event. The narrative architecture was not assembled in response to events. It was prepared in advance and deployed on cue.

The simultaneity was the tell. Organic public sentiment does not appear in Romanian, Italian, Serbian, and Hungarian communities with culturally localized framing within eighteen hours of a triggering event. Coordination does.

What Western analysis missed at the time — and what took years to correctly categorize — was that this was not primarily a technology problem. It was a behavioral one. The operation did not create the anger it distributed. It found the anger that already existed, validated it, and directed it toward specific political outcomes. The emotional material was real. The grievances were genuine. The distribution was manufactured.

That distinction matters profoundly, because it has not changed. It has intensified.

The Criminal-Geopolitical Pipeline
One of the most consequential findings from sustained dark web monitoring over this period is the relationship between criminal financial infrastructure and geopolitical operational infrastructure. These are not separate systems. They became the same system operating at different layers.

The early phase of this integration, in the 2014–2016 period, was not state-directed. It was ideologically motivated. Actors who had existing criminal capabilities primarily operating in dark web markets, carding, ransomware revenue made independent decisions to channel that revenue toward conflict support operations they genuinely believed in. The ideology preceded the state relationship. The state identified these voluntary contributors after they had already paid.

This matters for detection because the standard analytical framework for state-criminal overlap assumes the state is the principal. In the Donbas model, the sequence was reversed: belief first, crime as financial infrastructure second, state identification and recruitment third. The pipeline ran from grassroots conviction through criminal capability to state-adjacent asset not the other direction.

By 2019–2020, the cryptocurrency infrastructure that had once been PayPal donation links on VK was running through Monero wallets, DEX routing, and multi-hop mixing protocols. The same community members who had been posting about conflict support in public Facebook groups in 2015 were using privacy coins and decentralized exchanges five years later. The ideological and criminal ecosystems did not separate as they matured. They grew together.

The criminal-geopolitical financial overlap documented throughout this period is not a historical artifact. It is the current operating model, adapted and refined over a decade of continuous use.

The Line That Does Not Exist
The boundary between state-sponsored threat actor and criminal operator is the analytical fiction that most institutional frameworks are least equipped to abandon.

The state does not direct its criminal ecosystem. It licenses it. The license is not a contract. It is an understanding: operate where you want, avoid certain targets, be available when asked, and law enforcement attention will remain structurally absent. From more than a decade of monitoring Russian-language criminal forums, the behavioral constraints of this license system are observable in aggregate forum behavior even when they are never explicitly stated. The asymmetry between actors who target Russian organizations and actors who do not is too consistent across too many actors over too long a period to be coincidental.

The criminal actor who begins declining certain transaction types, improving their operational security with a discipline that exceeds what experience-based learning produces, and demonstrating knowledge of target environments that their stated criminal methodology should not provide that actor is exhibiting the behavioral signature of state recruitment. The transition is not abrupt. It is a gradual accumulation of small improvements that individually have innocent explanations and collectively do not.

For analysts who understand this architecture, the line between espionage and cybercrime is not a classification problem. It is a deliberate strategic design.

Why People Are More Vulnerable Now Than They Were Then
This is the observation that I find most important to communicate and the one that receives the least attention in the policy and security discourse I encounter.

The effectiveness of influence operations is not primarily a function of their technical sophistication. It is a function of the emotional and cognitive material available in the target population. An operation that finds pre-existing grievances, validates them, and redirects the resulting emotional energy wins on the emotional register even when it loses on the factual one. Because the emotional register is where it was designed to operate.

In 2014, the emotional material available in most European target populations was moderate. Institutional trust in governments, in media, in European structures, in the transatlantic alliance was imperfect but functional. The information operations of that period had to work against populations for whom institutional counter-narratives still carried credibility.

That has changed.

The populations that information operations now target in Eastern Europe carry a decade more of accumulated institutional disappointment. The brain drain is lived experience, not a statistic. The EU membership benefits are perceived as unevenly distributed by the people who received the smaller share. The economic comparisons with Western Europe are documented in the daily bank account of every person who stayed behind while someone they knew emigrated. These are not manufactured grievances. They are real.

And real grievances are the only raw material that effective influence operations require.

The local influencer model that has replaced bot networks in Eastern Flank electoral interference cases works precisely because authenticity cannot be manufactured. A real person, with a real community following, sharing content that reflects positions they partially hold, paid in cryptocurrency for the reach but not for the conviction that person is not a fake. They are genuinely credible to their genuine audience. The payment buys distribution. The authenticity is real.

The shift from bot networks to real people with real grievances is the single most consequential operational evolution in the influence operation landscape since 2014. It is also the evolution that is hardest to detect, hardest to disrupt, and hardest to counter without producing the iatrogenic amplification cycle where institutional counter-messaging amplifies the operation’s central narrative among precisely the demographics most susceptible to it.

The Dark Web as Early Warning Layer
The operational intelligence insight that ten years of dark web monitoring has produced most consistently is this: the events that manifest on surface platforms in weeks are being planned and resourced in underground forums now.

The TikTok algorithmic seeding campaigns that achieved electoral effect in the 2024 Eastern Flank cycle were assembled from commercial dark web supply chains influencer recruitment posts, account farm purchases, content production services with political calibration — weeks before the content appeared. The platforms saw the amplification. The preparation was invisible to anyone who wasn’t watching where the preparation was occurring.

IAB listings for critical infrastructure and defense-adjacent targets in Eastern Flank member states that carry premium prices with no financial exploitation rationale — those listings are not noise. They are a signal that adversarial actors with state-level motivation have assessed specific targets as worth the investment. The financial logic is wrong for a criminal buyer. It is exactly right for a strategic one.

The intelligence gap that allows most operations to achieve surprise is not technical. It is the absence of monitoring where the preparation is occurring.

What Has Not Changed
The operational template deployed in early 2014 pre-positioned communities, culturally localized narrative architecture, exploitation of authentic grievances, dark web financial infrastructure, simultaneous multi-platform activation is the same template that is operationally active in 2026.

The platforms have changed. Facebook gave way to Telegram, Telegram to TikTok. The cryptocurrency infrastructure has evolved from primitive direct transfers to institutional-grade obfuscation. The content production capacity has been multiplied by AI integration that has removed the human resource constraints that previously limited campaign volume.

But the actors who understood this system when it was being built are still the actors running it. The communities that were cultivated in 2014 were never dismantled. They were never truly disrupted. They grew in the dark, funded by the same criminal financial ecosystem that was always their infrastructure, until geopolitical events made them visible to audiences that had been looking elsewhere.

The asymmetry between analysts who have been watching this continuously and institutions that are encountering it as a new problem is not a knowledge gap. It is a time gap. And the operational value of sustained longitudinal monitoring in dark web communities, in influence operation ecosystems, in the criminal-geopolitical overlap is precisely the baseline that makes the current signals readable.

The signals were already changing before the announcements were made. They always are.

Adrian Alexandru Stîngă is Lead Analyst A-01 at Aether Intel, a CTI research platform producing threat intelligence at the intersection of dark web ecosystems, hybrid warfare operations, and Eastern Flank security. The full AS-CTI-2026 series (30 reports, TLP:CLEAR) and the OBSIDIAN-TRACE deep-dive series are published at aether-intel.com.

All analysis reflects direct community-level observation. Where assessments draw on community-level intelligence that cannot be independently verified, confidence levels are explicitly documented in the underlying reports.

If you’ve spent any time wrestling with AI coding tools and agents in 2026, you’ve hit two walls:

1. Provider lock-in. Claude Code expects Anthropic. Codex expects OpenAI. Your shiny new agent framework wants whatever its README assumes.
2. Agent amnesia. Every session starts from zero. Your “AI assistant” doesn’t actually learn anything about you, your codebase, or the work you did yesterday.

Two open-source projects address those problems head-on — and they pair beautifully together.

• Hermes Agent (by Nous Research) — a self-improving AI agent with a built-in learning loop, multi-platform presence, and a serious tool ecosystem.
• Lynkr — a self-hosted universal LLM proxy that lets any AI tool talk to any model provider.

This post explains what each one is, why they exist, and shows you the exact steps to run Hermes through Lynkr so you can route Hermes to Databricks, Bedrock, Ollama, llama.cpp, Azure, OpenRouter — or all of them with automatic cost-tier routing.

What Is Hermes Agent?

Hermes is an open-source AI agent (MIT-licensed, built by Nous Research) that you actually live inside, not just call.

What makes it different from “yet another agent”:

• A closed learning loop. Hermes curates its own memory, autonomously creates skills (procedural memory) after complex tasks succeed, improves them during use, and searches its own past conversations via SQLite FTS5. It’s the only agent I’ve seen that gets meaningfully better the longer you use it.
• Lives where you do. A single gateway process plugs into Telegram, Discord, Slack, WhatsApp, Signal, Email, and a real terminal TUI. Send a voice memo from your phone, get a transcribed answer back, continue the same thread from your laptop later.
• Runs anywhere. Seven terminal backends — local, Docker, SSH, Singularity, Modal, Daytona, Vercel Sandbox. Run it on a $5 VPS or a GPU cluster. Modal/Daytona give you serverless persistence — hibernates when idle, wakes on demand.
• Built-in cron. “Every weekday at 8am, summarize my GitHub notifications and send to Telegram.” That’s a one-line cron job in natural language.
• Delegates and parallelizes. Spawns isolated subagents for parallel workstreams; results come back without flooding your context.
• Provider-agnostic by design. OpenRouter, Nous Portal, NovitaAI, NVIDIA NIM, Xiaomi MiMo, z.ai/GLM, Kimi/Moonshot, MiniMax, Hugging Face, OpenAI, or your own endpoint. Switch with hermes model — no code changes.

Architecture in one paragraph

The core is AIAgent in run_agent.py — a synchronous tool-calling loop over OpenAI-format messages. model_tools.py orchestrates ~40 built-in tools auto-discovered from tools/. The CLI (cli.py, ~11k LOC) handles slash commands, prompt_toolkit input, Rich rendering, and a data-driven skin engine. Provider profiles live under plugins/model-providers/<name>/ and contribute base_url, env_vars, api_mode, and fallback_models — the runtime resolver merges those with custom_providers from config.yaml to figure out where to send each request. That last detail is what makes Lynkr integration trivial.

Install Hermes in one line

curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash

Then hermes to start chatting.

What Is Lynkr?

Lynkr is a self-hosted Node.js proxy that sits between any AI coding tool and any LLM provider. One environment variable change, and your tool works with whatever backend you want.

Claude Code / Cursor / Codex / Cline / Continue / Hermes / Vercel AI SDK
                                |
                              Lynkr  (http://localhost:8081)
                                |
   Ollama | Bedrock | Databricks | OpenRouter | Azure | OpenAI | llama.cpp | LM Studio | z.ai | Vertex | Moonshot

What’s actually inside

I went through the source. Lynkr is more than a “translate request, forward, translate response” proxy:

• Format conversion. Anthropic ↔ OpenAI ↔ Codex Responses API ↔ Databricks ↔ Bedrock — handled in src/clients/ (openai-format.js, responses-format.js, databricks.js, bedrock-utils.js, etc.).
• Tier-based routing. src/routing/ analyzes prompt complexity, agentic intent, risk, and latency, then routes to a TIER_SIMPLE / TIER_STANDARD / TIER_COMPLEX model. Cheap stuff goes to Ollama; gnarly stuff goes to a frontier cloud model. This is where the headline “60–80% cost savings” comes from.
• Resilience. Circuit breaker (cockatiel), retries, DNS logging, prompt cache injection.
• MCP integration + Code Mode. Auto-discovers MCP servers and can collapse 100+ MCP tool definitions into 4 meta-tools (~96% token reduction).
• Observability built in. Telemetry, latency tracking, usage reporting (lynkr usage shows AI spend and tier savings), trajectory export as JSONL for training (lynkr trajectory).
• 699 passing tests. Routing, format conversion, streaming, error resilience, memory store, prompt cache — it’s seriously tested for a side-project proxy.

Install Lynkr in one line

curl -fsSL https://raw.githubusercontent.com/Fast-Editor/Lynkr/main/install.sh | bash

Or via npm: npm install -g pino-pretty && npm install -g lynkr.

Why Use Them Together?

Hermes already supports a long list of providers natively. Why bolt Lynkr in front?

Three concrete reasons:

1. Unify your enterprise creds

Your company has a Databricks endpoint serving Claude, an AWS Bedrock account with cross-region inference profiles, an Azure OpenAI deployment, and a private Ollama box. With Lynkr, all of those live behind one OpenAI-compatible URL. Hermes points at that URL and stops caring which backend is serving the request.

2. Automatic cost-tier routing

This is the killer feature. Hermes can switch models with /model, but Lynkr will switch per request based on complexity. Simple tool calls and short prompts go to free local Ollama. Heavy reasoning goes to your premium cloud model. You don’t think about it — Lynkr’s complexity-analyzer.js and risk-analyzer.js decide.

Run lynkr usage afterward to see the actual savings.

3. Centralized observability for every agent + tool

If you run Hermes + Claude Code + Cursor + Codex all on the same machine — and a lot of us do — Lynkr becomes a single chokepoint for spend, telemetry, prompt caching, and trajectory capture across all of them. You get one usage report instead of four dashboards.

How to Use Lynkr With Hermes

The integration is genuinely 3 minutes of work because both tools speak OpenAI-compatible HTTP.

Step 1: Start Lynkr with a backend

Pick whatever provider you want Lynkr to route to. For a local-first setup:

# .env in your Lynkr directory (or just exports)
export MODEL_PROVIDER=ollama
export OLLAMA_MODEL=qwen2.5-coder:latest
export OLLAMA_ENDPOINT=http://localhost:11434

lynkr start

Or for tier routing across providers:

export TIER_SIMPLE=ollama:qwen2.5-coder:latest
export TIER_STANDARD=openrouter:anthropic/claude-3.5-haiku
export TIER_COMPLEX=bedrock:anthropic.claude-3-5-sonnet-20241022-v2:0
export OPENROUTER_API_KEY=sk-or-...
export AWS_BEDROCK_API_KEY=...
lynkr start

Lynkr now listens on http://localhost:8081 (OpenAI-compatible) and http://localhost:8081/v1/messages (Anthropic-compatible).

Step 2: Register Lynkr as a custom provider in Hermes

Hermes resolves providers through plugins/model-providers/<name>/ profiles plus a custom_providers list in your ~/.hermes/config.yaml. Add an entry:

custom_providers:
  - name: lynkr
    base_url: http://localhost:8081/v1
    api_mode: chat_completions
    env_var: LYNKR_API_KEY      # any string works — Lynkr doesn't validate
    models:
      - auto                    # Lynkr's tier router picks the actual model
      - qwen2.5-coder:latest
      - anthropic/claude-3.5-sonnet

Then set the key (any value):

hermes config set env.LYNKR_API_KEY sk-lynkr

Step 3: Point Hermes at Lynkr

hermes model custom:lynkr/auto

Or interactively: run hermes model, pick custom:lynkr, choose auto.

That’s it. Every Hermes turn now flows through Lynkr, which routes to the right backend based on tier and complexity. Run a few turns, then:

lynkr usage

…and you’ll see the per-tier spend breakdown and dollars saved versus a single-frontier-model baseline.

Bonus: voice memo → Hermes → Lynkr → cheapest model

Because Hermes already has Telegram and voice memo transcription wired in, this whole stack means:

Record a voice memo on your phone → Hermes transcribes it → routes the request through Lynkr → Lynkr picks Ollama for the “what time is it in Tokyo” parts and Sonnet for the “refactor this function” parts → reply comes back to your phone.

You built that in 5 minutes with two npm/bash installers and a YAML edit.

When NOT to Use Lynkr With Hermes

Being honest:

• You only use one provider. Hermes already supports it natively. Adding Lynkr is extra latency and another process to babysit.
• You need streaming reasoning tokens from a specific model. Make sure Lynkr’s format converter for that provider preserves what you need — it does for most cases, but verify before betting on it.
• You’re on a constrained environment. Lynkr is Node 20+. Hermes is Python 3.11. That’s two runtimes on a Raspberry Pi.

For everything else — multi-provider workflows, enterprise creds, cost optimization, observability — the combination is hard to beat.

TL;DR

Links

• Hermes Agent: https://github.com/NousResearch/hermes-agent
• Hermes docs: https://hermes-agent.nousresearch.com/docs
• Lynkr: https://github.com/Fast-Editor/Lynkr
• Lynkr docs: https://fast-editor.github.io/Lynkr/

If you build something with this combo, drop a comment — I’d love to see what stacks people are putting together.

Approach 1: Brute Force

• Try every possible buy day.
• For each buy day, try every sell day after it.
• Calculate profit and keep maximum.

class Solution {
    public int maxProfit(int[] prices) {
        int maxProfit = 0;

        for (int i = 0; i < prices.length; i++) {
            for (int j = i + 1; j < prices.length; j++) {
                int profit = prices[j] - prices[i];
                maxProfit = Math.max(maxProfit, profit);
            }
        }

        return maxProfit;
    }
}

Complexity

Time: O(n²)
Space: O(1)

Approach 2: Optimal One Pass

• Keep track of minimum price seen so far.

• At every index:

  • Calculate profit if sold today.
  • Update maximum profit.

• If current price is smaller, update minimum buy price.

Example:

Buy at 1
Sell at 6
Profit = 5

class Solution {
    public int maxProfit(int[] prices) {

        int minPrice = Integer.MAX_VALUE;
        int maxProfit = 0;

        for (int price : prices) {

            minPrice = Math.min(minPrice, price);

            int profit = price - minPrice;

            maxProfit = Math.max(maxProfit, profit);
        }

        return maxProfit;
    }
}

Complexity

Time: O(n)
Space: O(1)

What is Google Gemini Spark? A Deep Dive Into Google’s 24/7 Personal AI Agent

If you watched Google I/O 2026, one announcement quietly stole the show — not because it was the flashiest, but because it represents a fundamental shift in how we’ll interact with computers. That announcement was Gemini Spark.

Spark isn’t another chatbot. It isn’t a smarter Search. It’s Google’s first serious attempt at a 24/7 autonomous personal agent — software that keeps working when your phone is in your pocket, when your laptop is closed, and when you’re asleep.

Let’s break down what it actually is, how it works, and why it matters for developers.

TL;DR

• Gemini Spark is a persistent, autonomous AI agent that lives across Gmail, Docs, Calendar, and the rest of Google Workspace.
• It runs 24/7 in the background, even when your device is closed.
• It’s powered by Gemini 3.5 Flash (with Pro coming soon) and built on top of the open Model Context Protocol (MCP) so third-party tools can plug in.
• It launched first for Google AI Ultra subscribers ($100/month) in the US, with broader rollout to follow.
• For developers, Spark + MCP is the most important integration surface Google has shipped in years.

1. What Exactly Is Gemini Spark?

Gemini Spark is a personal agent that Google describes as a “24/7 collaborator.” Unlike previous AI features that respond when you ask, Spark is proactive:

• It reads your incoming email and flags what needs action.
• It drafts replies, schedules, and follow-ups before you ask.
• It tracks ongoing tasks across days and weeks.
• It executes multi-step workflows that span multiple apps.
• It keeps doing all of the above while your phone is locked.

Think of it as the difference between hiring a contractor (Gemini chat) and hiring a full-time assistant (Spark). One responds to tickets. The other owns outcomes.

2. How It Works Under the Hood

The Model Layer: Gemini 3.5 Flash

Spark runs on Gemini 3.5 Flash, Google’s new default model. Key specs Google announced:

• ~4x faster token output than competing frontier models
• Beats the older Gemini 3.1 Pro on coding and agentic benchmarks
• Optimized for the kind of long-running, low-latency tool use that agents need

Flash is the right choice here because agents make a lot of small decisions (“should I draft this? wait for more context? ask the user?”). Latency compounds.

The Protocol Layer: MCP (Model Context Protocol)

This is the part most developers missed. Instead of building a proprietary plugin system, Google adopted MCP — the open standard originally pushed by Anthropic — as the way third-party tools connect to Spark.

This is huge. It means:

• One MCP server you build can serve Claude, Gemini Spark, and any other MCP-compatible host.
• You don’t need to maintain a separate “Google plugin” SDK.
• Tool definitions, auth, and resource exposure all follow one spec.

A minimal MCP tool looks roughly like this:

from mcp.server import Server
from mcp.types import Tool, TextContent

server = Server("my-tool-server")

@server.list_tools()
async def list_tools() -> list[Tool]:
    return [
        Tool(
            name="get_invoice_status",
            description="Look up status of an invoice by ID",
            inputSchema={
                "type": "object",
                "properties": {"invoice_id": {"type": "string"}},
                "required": ["invoice_id"],
            },
        )
    ]

@server.call_tool()
async def call_tool(name: str, arguments: dict) -> list[TextContent]:
    if name == "get_invoice_status":
        status = lookup_invoice(arguments["invoice_id"])
        return [TextContent(type="text", text=status)]

Register this with Spark and it can now check invoice status on your behalf — at 3am, while you’re asleep, when an email asking about an invoice arrives.

The Runtime Layer: Background Execution

The genuinely new piece is the persistent runtime. Most “AI assistants” stop existing the moment you close the tab. Spark keeps a server-side execution context tied to your account that:

1. Subscribes to events (new email, calendar change, doc edit).
2. Wakes the agent loop on relevant triggers.
3. Executes tool calls (MCP, Workspace APIs, Search).
4. Surfaces results via the Android Halo — a small communication band at the top of the phone screen showing what background agents are doing.

The Halo is a UX innovation worth noticing: it solves the “what is my agent secretly doing?” trust problem by always making background work visible.

3. What Spark Can Actually Do Today

From the I/O 2026 demos and rollout notes:

• Email triage — Reads inbox, drafts replies, surfaces what needs your attention.
• Schedule management — Reschedules meetings, finds slots, sends invites.
• Daily Brief — Morning digest pulling from Gmail, Calendar, and Tasks, ranked by priority.
• Cross-app workflows — “Find the contract Sarah sent last month, summarize the changes, and email her the redline” → executes end-to-end.
• Persistent monitoring — “Watch this listing and ping me if the price drops below X” runs indefinitely.

4. Pricing: The Catch

Spark launched on Google’s new Ultra tier ($100/month). More importantly, Google scrapped per-day prompt limits and moved to a compute-used billing model. You’re charged based on:

• Prompt complexity
• Features invoked (Spark, Flow, Omni)
• Length of the conversation/agent run

The allowance refreshes roughly every 5 hours. A heavy debugging session — or a Spark agent that runs hot for an afternoon — can drain it fast. Build accordingly.

5. Why This Matters for Developers

Three concrete reasons Spark should be on your radar:

1. MCP is now a two-vendor standard

With both Anthropic and Google supporting MCP, it’s the closest thing to a universal “tools for agents” spec we have. Build MCP servers, not proprietary integrations.

2. The unit of software is shifting

For ten years we built apps that respond to clicks. The next decade is about building services that agents can drive. That means:

• Stable, well-documented APIs
• Clear tool descriptions (the LLM has to pick yours over a competitor’s)
• Idempotent operations (agents will retry)
• Streaming/long-running job patterns (agents wait for things)

3. Background-first design

If your product can be useful while the user isn’t looking at it, Spark is a distribution channel. “What can my app do for the user at 2pm Tuesday when they’re in a meeting?” is now a real product question.

6. How to Start Building for Spark Today

1. Stand up an MCP server exposing your product’s core capabilities. The official Python and TypeScript SDKs are at modelcontextprotocol.io.
2. Write tool descriptions like marketing copy. The model picks tools based on the description — clarity wins.
3. Make every operation idempotent and resumable. Background agents crash, retry, and resume.
4. Test with multiple hosts. Same MCP server should work in Claude Desktop, Gemini Spark, and any other MCP client. If it doesn’t, your server is doing something non-standard.
5. Design for partial autonomy. The best agent UX is “I drafted this — approve?” not “I sent this.” At least at first.

Closing Thought

Gemini Spark is the clearest signal yet that the agent era is the product era of the next decade. The companies that win it won’t be the ones with the smartest model — they’ll be the ones whose software is the easiest for agents to use.

If you ship developer tools, APIs, or SaaS, your roadmap question for 2026 is no longer “how do we add AI features?” It’s “how do we become the tool an agent reaches for?”

Spark is Google placing its bet. Time to place yours.

Found this useful? Drop a comment with what you’re building for the agent era — I’d love to see it.