This was a release-heavy week across the lakehouse projects. Iceberg pushed 1.11.0 through a fourth release candidate while shipping a 1.10.2 patch in parallel. The V4 spec discussion accelerated with two simultaneous votes on content stats and relative paths. The Iceberg Go client kicked off a 0.6.0 vote. Parquet held a vote on format changes for IEEE 754 total order semantics and stood up a brand-new Footer Working Group. Arrow released ADBC 23 and Arrow 24.0.0, and the community spent serious time debating donation of an Erlang implementation and dictionary-encoded extension types. Polaris was relatively quiet on the dev list after its 1.4.1 patch went out on May 1, but the community continued discussions around catalog federation and the path to 1.5.0.

The connecting theme this week was format evolution under pressure. Iceberg V4 is no longer a far-off design exercise. Specific votes are landing. Parquet is restructuring how the format committee handles its most fundamental data structure, the footer. Arrow is grappling with whether to expand its extension type system to handle increasingly complex use cases coming from the engines that depend on it. These projects are all making decisions now that engines and storage providers will have to live with for years.

Apache Iceberg

The big story on the Iceberg dev list this week was the simultaneous push on three different release tracks and two V4 spec votes that ran in parallel. The community is moving fast.

Aihua Xu opened the Iceberg 1.11.0 RC4 vote after several prior release candidates surfaced issues that needed fixes. Binding +1 votes came in from Russell Spitzer, Eduard Tudenhöfner, Amogh Jahagirdar, and Kevin Liu, along with non-binding +1s from Steven Wu, huaxin gao, Yufei Gu, Jean-Baptiste Onofré, Yuya Ebihara, Neelesh Salian, Andrei Tserakhau, and Bryan Keller. The discussion threaded through environment-specific test failures and a few late patches that had to be cherry-picked to the release branch. Aihua handled multiple status updates and the eventual successful close. Going from RC1 in late April to RC4 in mid-May suggests this release had more friction than the team would have liked, but the spread of binding voters across companies is the right signal for a healthy major release.

While 1.11.0 was working through its final candidate, Amogh Jahagirdar shepherded the 1.10.2 patch release. He kicked off the Apache Iceberg 1.10.2 RC1 vote and collected +1s from Russell Spitzer, Kevin Liu, Yufei Gu, huaxin gao, Yuya Ebihara, Aihua Xu, Neelesh Salian, roryqi, and Eduard Tudenhöfner before announcing the release of 1.10.2 midweek. Kevin Liu confirmed the announcement. The fact that the project can run a 1.10.x patch release and a 1.11.0 major release in parallel reflects the maturity of the release infrastructure that the community has built over the past year. Two years ago, a parallel release would have been hard to coordinate. Now it is routine.

The V4 spec is where the more consequential work happened. Daniel Weeks opened the vote on relative paths in v4, drawing immediate engagement from Amogh Jahagirdar, Anoop Johnson, Anurag Mantripragada, Alex Stephen, Gang Wu, roryqi, Fokko Driesprong, Andrei Tserakhau, and Junwang Zhao. Relative paths are one of those structural changes that look small in the spec but have major implications for portability. Tables written with relative paths can be moved between buckets, regions, or storage backends without rewriting every manifest. For organizations running disaster recovery or migrating clouds, this is a real operational improvement. The volume of votes in the first few days of the thread suggests broad community support.

Running in parallel, Eduard Tudenhöfner opened the vote on Content Stats representation in v4. Russell Spitzer, Talat Uyarer, Wing Yew Poon, Steven Wu, Amogh Jahagirdar, and Daniel Weeks all participated, and Eduard followed up with two additional status messages. Content stats matter because they directly affect query planning. Better stats let engines prune more aggressively, which means lower scan costs and faster queries. Getting the representation right in the format means every engine that supports v4 gets the same benefit.

Ryan Blue stepped back into the dev list with a proposal to add an unregister table endpoint to the REST spec. The thread drew responses from Russell Spitzer, Ajantha Bhat, Péter Váry, Jean-Baptiste Onofré, Yufei Gu, Amogh Jahagirdar, Steven Wu, Neelesh Salian, Daniel Weeks, and Steve Loughran. Today, removing a table from a catalog without dropping the data is awkward. Some catalogs treat it as drop with delete-data-false. Others have custom endpoints. Adding a standard unregister operation to the REST spec means every Iceberg client gets the same behavior regardless of catalog implementation. That is the kind of cleanup work that does not generate headlines but pays off for every operator who has ever had to migrate tables between catalogs.

Prashant Singh proposed a closely related change with the addition of an X-Iceberg-Client-Capabilities header to the REST spec. Sung Yun and Daniel Weeks weighed in. Client capability negotiation is the kind of protocol detail that becomes critical as the ecosystem grows. A REST catalog needs to know what features a calling client supports before deciding which response shape to return. Without a header like this, the catalog ends up either returning the lowest common denominator or breaking older clients. The Iceberg REST spec is starting to look a lot like the HTTP spec did in its early years, with each iteration filling in the practical details that come up only in production deployments.

On the language client side, Matt Topol opened the Iceberg Go 0.6.0 RC0 vote. Andrei Tserakhau, Neelesh Salian, Kevin Liu (three messages), Tanmay Rauth, and Gang Wu joined the thread. Iceberg Go has been quietly building up steady momentum as a serious alternative for tools that do not want a JVM dependency. The release cadence is consistent. Each version closes more of the gap with the Java reference implementation. The number of new contributors voting on releases is one of the better signals of a healthy language client community.

Anurag Mantripragada continued his work on column-level updates with a new proposal for column update file representation. Gang Wu replied with feedback. Efficient column updates are one of the few remaining gaps in Iceberg’s update story. The V4 design work is reportedly opening up new possibilities for handling updates without rewriting entire data files. This thread is one to watch closely as V4 work continues.

A few other discussions worth flagging. Kevin Liu raised a spec ambiguity question on the Avro schema for the day partition transform fields in manifests. These ambiguity threads tend to come from someone trying to implement Iceberg in a new language and running into edge cases the original Java implementation glossed over. Resolving them in the spec means every future implementation gets the same answer.

Steven Wu’s vote on adding the CatalogObjectIdentifier schema to OpenAPI drew responses from Yufei Gu, Russell Spitzer, huaxin gao, Christian Thiel, Alexandre Dutra, Jean-Baptiste Onofré, Steve, Ajantha Bhat, Daniel Weeks, Renjie Liu, and Péter Váry. Long thread, many committers engaged. The Iceberg REST OpenAPI specification is where a lot of the standardization work between catalog implementations actually happens. When Polaris, Lakekeeper, and other REST catalogs all agree on the same OpenAPI shape, multi-engine interoperability gets easier for everyone.

Henry Haiying Cai posted a Kafka Connect discussion thread on worker coordinator progress detection. Kafka Connect remains an under-discussed but heavily used piece of the Iceberg ecosystem, and ongoing reliability work there matters for streaming ingestion pipelines.

The bottom line on Iceberg this week. The project shipped a patch release, drove a major release through a final candidate, opened two V4 spec votes, ran a Go client release vote, and discussed standards-level REST changes. That is a sustained pace of work that very few open source data projects can match.

Apache Polaris

Polaris had a quieter week on the dev list than usual, which is the expected pattern after a release. Apache Polaris 1.4.1 shipped on May 1 as a patch release on the 1.4.0 line that came out April 21. The 1.4.1 patch addressed storage URI handling and a few smaller fixes. The release cadence for the Polaris community has settled into a roughly monthly minor or patch release with quarterly major versions, which is a sustainable pace for a project that graduated from the Apache Incubator only in February.

The graduation context is worth keeping in mind when looking at this week’s activity. Polaris is now an Apache Top-Level Project rather than a podling. That means the project’s governance and infrastructure operate under the full Apache umbrella, with its own PMC and direct relationship with the Apache board. The transition has not changed the day-to-day cadence on the dev list, but it has changed what kinds of discussions happen. Release voting no longer needs IPMC approval. PMC member additions and committer promotions happen through the project’s own process. The community has fewer process gates to work through. That tends to translate into faster feature iteration over time.

The 1.4.0 release that landed in late April was substantive. It introduced granular control over how Polaris interacts with cloud storage, enabling multi-tenancy with support for separate encryption keys and IAM identities across catalogs. Iceberg metrics persistence to the Polaris database moved from preview to stable, giving operators a way to capture ScanReports and CommitReports directly without piping them through an external metrics system. The Helm chart picked up a JSON schema for validation of values files, which is the kind of detail that turns out to matter a lot when you are trying to operationalize the catalog at scale.

Catalog federation continues to be one of the most-discussed areas of the Polaris codebase even when there is not a specific dev list thread driving it. Federation lets a single Polaris instance act as a routing layer for tables that live in other Iceberg REST catalogs, Hive Metastores, or other systems. For organizations consolidating multiple catalog sources without forcing a big-bang migration, federation is the answer. The 1.4 release built on the federation work from 1.1 and 1.2, adding extension points that make it easier to add new catalog types without rewriting the core federation logic.

The path to Polaris 1.5.0 is starting to take shape based on signals from the broader community. Generic tables, which let Polaris manage non-Iceberg formats like Delta Lake and Hudi through a unified catalog interface, are likely to see additional polish in the next release. Open Policy Agent integration for external authorization will continue to mature. And the work on persisting events to multiple backends, which arrived in preview form in 1.2 and was expanded in 1.4, is one of those features that turns the catalog into a real source of operational truth for governance teams.

Worth noting for Polaris watchers. The Snowflake engineering blog published a detailed analysis of the 1.4 release that covers the security and metrics work in more depth. The Dremio blog also covered Polaris’s positioning as the catalog standard for Iceberg lakehouses and agentic analytics, framing how the catalog layer connects to AI agents that need to query and act on lakehouse data.

The pattern across both pieces is that Polaris is being positioned not just as an Iceberg catalog but as the governance and discovery layer for the lakehouse as a whole. That framing has implications for what gets prioritized in the next release. Features that help AI agents discover tables, understand schemas, and reason about data quality become first-class citizens. Features that help human governance teams manage policy, audit access, and trace data lineage become equally important. The dev list has not yet had the formal proposal threads for these features, but the trajectory is clear from the community discussions and from how vendors building on Polaris are talking about the project.

Apache Arrow

The Arrow dev list was as busy as it usually is. Arrow handles releases on a different cadence than Iceberg or Polaris, with multiple sub-projects releasing on their own schedules under the same project umbrella. This week saw release votes on multiple Rust crates, .NET, Go, and the main monorepo.

Andrew Lamb opened the Arrow Rust 58.3.0 RC1 vote, the Arrow Rust 57.3.1 RC1 vote, and the Arrow Rust 56.2.1 RC1 vote within a few days of each other. Three concurrent patch and minor releases across three different version lines is unusual even for arrow-rs, which has a famously fast release cadence. Ed Seidl, Bryce Mecum, Raúl Cumplido, and L. C. Hsieh voted on multiple of the threads. All three votes passed and the releases shipped. Arrow Rust is now on a release cycle measured in days rather than weeks, which means downstream projects that depend on it have to think harder about how aggressively they pin versions.

The DataFusion Python Bindings 31.0.0 vote also closed earlier in the period, continuing the steady release pace on the DataFusion side of the Arrow project. DataFusion is increasingly the engine of choice for new analytical tools built in Rust, and its release cadence reflects that. Every release pulls more contributors in and adds more SQL surface area.

The donation discussions on the dev list this week were the most interesting strategic signals. Sutou Kouhei opened the vote to donate Apache Arrow Erlang, drawing responses from Benjamin Philip, Curt Hagenlocher, Matt Topol, and David Li. Erlang is a language used heavily in telecom, distributed systems, and high-availability backend services. Adding an official Arrow implementation for Erlang opens up zero-copy data exchange for systems built on the BEAM virtual machine. The discussion was constructive, with Benjamin Philip following up on grant document specifics over several messages.

In parallel, Rok Mihevc opened the vote to donate pyarrow-stubs to Apache Arrow. Antoine Pitrou, Dewey Dunnington, Matt Topol, Raúl Cumplido, Nic Crane, Adam Reeve, Ian Cook, David Li, Sutou Kouhei, Alenka Frim, Jacob Wujciak, Joris Van den Bossche, wish maple, and L. C. Hsieh all participated. PyArrow stubs are the Python type annotations that IDEs and type checkers use to give developers autocomplete and error checking when they write code against pyarrow. The fact that this was a community project being formally donated to Arrow shows how much the broader Python ecosystem depends on these stubs, and how the community has stepped up to maintain them.

Dewey Dunnington opened an IPC representation discussion for dictionary-encoded extension types. Antoine Pitrou and Matt Topol joined. Extension types are one of those features that look niche on the surface but matter enormously in practice. They let Arrow carry domain-specific type information like geographic coordinates, UUIDs, or fixed-precision decimals without breaking interoperability with consumers that do not understand the extension. Combining extension types with dictionary encoding is technically tricky, and getting the IPC representation right means downstream engines can handle these types correctly when they cross network or process boundaries.

Antoine Pitrou opened a similarly foundational discussion with the question of whether to restrict field, schema, and custom metadata to UTF-8. Rusty Conover, Raphael Taylor-Davies, and Dewey Dunnington responded. This is the kind of spec question that has to be answered very carefully because the answer affects every existing Arrow implementation. Most production code already treats this metadata as UTF-8 in practice, but the format does not explicitly require it. Tightening the spec to match practice is good cleanup work.

Raúl Cumplido announced Apache Arrow 24.0.0 after passing the vote. The release includes work across the C++, Python, R, and Java implementations. The Arrow monorepo has slowed its release cadence to roughly once per quarter while the language-specific implementations like Rust, Go, and .NET release on their own schedules. The split-cadence approach lets the fast-moving language implementations ship features without being held back by the rest of the project, while the monorepo releases provide stable points for the C++ ecosystem to build against.

David Li announced Apache Arrow ADBC 23. ADBC, the Arrow Database Connectivity standard, continues to gain traction as an Arrow-native alternative to ODBC and JDBC. The release notes were not detailed in the announcement but the steady cadence speaks for itself. ADBC drivers exist for an increasing number of databases, and the API is stabilizing.

A few smaller items rounded out the week. Curt Hagenlocher announced Apache Arrow .NET 23.0.0 after the RC0 vote passed. Matt Topol announced Apache Arrow Go 18.6.0. Tornike Gurgenidze opened a thoughtful discussion thread on an ADBC Partitioned Bulk Ingest API with a response from Curt Hagenlocher. Dan Mattheiss proposed AVX2 optimization work for the parquet bloom filter with responses from Antoine Pitrou and wish maple. Antoine Pitrou announced an in-person Arrow and Parquet meetup in Paris. And Jarek Potiuk opened a discussion about Arrow involvement in the Community Over Code Glasgow 2026 hackathon, with positive responses from Nic Crane, Antoine Pitrou, Raúl Cumplido, and Jean-Baptiste Onofré.

The state of Arrow remains strong. The project has the broadest contributor base of any of the four projects covered here, with regular contributions across roughly a dozen languages and multiple sub-projects. The release management work that Raúl Cumplido and Sutou Kouhei handle quarter after quarter is the unsung infrastructure that makes the whole ecosystem function.

Apache Parquet

Parquet had the most consequential governance moment of the week. Jiayi Wang announced the kickoff of the Parquet Footer Working Group. Raúl Cumplido joined the discussion, and Jiayi followed up. The Parquet footer is the central metadata structure in every Parquet file. It tells readers what columns exist, what their types are, where the row groups live, what statistics are available, and how the data is encoded. Anything that changes the footer changes Parquet itself.

Standing up a formal working group for the footer signals that the community sees a need for structured discussion on what the footer should evolve into. Recent threads on the dev list reinforce this. Divjot Arora opened a discussion on Parquet Footer Options. Daniel Weeks proposed support for non-contiguous pages in Parquet, drawing extensive responses from Andrew Bell, Adrian Garcia Badaracco, Micah Kornfield, Will Edwards, and Andrew Lamb. Steve Loughran raised concerns about hardening variant readers, and Antoine Pitrou responded. Each of these threads touches the footer in some way. A working group with a defined scope and charter is a reasonable way to handle a body of work that big.

Gang Wu opened the vote on PARQUET-2249, the format change for IEEE 754 total order and NaN counts. Micah Kornfield, Steve Loughran, and Ed Seidl voted. Total order and NaN handling are exactly the kind of numerical correctness issues that lurk in floating-point data for years until they cause a query to silently produce wrong answers. Resolving them at the format level means engines do not have to invent their own conventions, and means data that crosses engine boundaries behaves predictably.

Russell Spitzer opened an automated release discussion for Parquet. Arnav Balyan, Gang Wu, and Fokko Driesprong all joined. Release automation is the kind of internal project that pays off every release cycle. Polaris went through a similar automation push last year and now releases on a much more predictable cadence as a result. The Parquet project handling its own automation thread suggests the community is ready to move from the artisanal release model to something more repeatable.

Gang Wu announced the release of Apache Parquet Java 1.17.1 after the RC0 vote passed. The release passed binding votes from Steve Loughran, Fokko Driesprong, Russell Spitzer, Daniel Weeks, and Xinli shang. Manu Zhang had earlier kicked off the discussion on the next parquet-java release, with input from Steve Loughran, Aaron Niskode-Dossett, Fokko Driesprong, Julien Le Dem, Gang Wu, and Rahil C. That kind of thread, where multiple committers weigh in on scope before the actual release work begins, is exactly the right governance pattern for a format-defining library.

Micah Kornfield announced Ed Seidl as a new Parquet committer. Andrew Lamb, Gang Wu, and Raúl Cumplido offered congratulations. Ed has been a steady contributor across both Parquet and Arrow for some time. The promotion to committer reflects the work he has put in across a number of release votes and discussion threads.

Arnav Balyan opened two discussion threads that point to where the Parquet community is thinking about AI-assisted contribution. The first was a discussion on AI tooling policy for Parquet, drawing a response from Fokko Driesprong. The second was a discussion on adding AGENTS.md to parquet-java, with input from Aaron Niskode-Dossett, Andrew Lamb, and Micah Kornfield. AGENTS.md is the emerging convention for documenting how AI coding agents should interact with a codebase. The fact that an Apache format project is having this conversation openly on its dev list, rather than letting individual contributors quietly use AI tools without disclosure, is the right approach. It is also a topic the Iceberg community has been working through for several months.

Micah Kornfield opened a discussion on remaining open spec-level questions for ALP. Andrew Lamb engaged. ALP is the adaptive lossless floating-point compression scheme that has been making its way through Parquet for the past year. It promises significant compression improvements on floating-point data versus the existing compression schemes. Getting the spec right matters because once ALP is in the format, every reader has to be able to decode it.

Other threads of note. Andrew Lamb opened a discussion on where the VariantJsonParser should live. Ed Seidl opened a discussion on making path_in_schema optional. Ismaël Mejía requested code reviews on Java performance optimization work, with engagement from Steve Loughran and Fokko Driesprong. Andrew Bell opened a discussion on wide schemas in Parquet. Each of these is the kind of practical improvement work that compounds over time.

Parquet is at an interesting point in its life cycle. The format is mature enough that nobody questions whether it should exist. The community is now wrestling with how to handle format evolution carefully, how to bring in new contributors at scale, and how to balance the needs of the established Java reference implementation against the growing number of native implementations in Rust, Go, and other languages. The Footer Working Group is the most visible sign of this maturation. The AI tooling policy discussion is another.

Cross-Project Themes

Three patterns connect the work across all four projects this week.

The first is format evolution under load. Iceberg V4 votes ran this week on relative paths and content stats. Parquet stood up a Footer Working Group and held a vote on IEEE 754 total order. Arrow discussed restricting metadata to UTF-8 and resolving IPC representation for dictionary-encoded extension types. These are all spec-level changes that ripple through the ecosystem. The fact that all three formats are evolving at the same time is not coincidental. The lakehouse architecture has matured to the point where the gaps between the formats are now visible in production. Each project is addressing the issues it can address, and the changes need to compose cleanly with each other.

The second is the rise of formal working groups and process discipline. Parquet now has a Footer Working Group. Iceberg has been running working groups on Kafka Connect, partition stats, and Python bindings for months. Polaris has clear schedule-driven release cadence and graduated to TLP. Arrow has multiple parallel release tracks managed by different committers. These projects are not just shipping features. They are building the institutional infrastructure that lets large communities collaborate without stepping on each other. That kind of process work is invisible from the outside but essential to long-term project health.

The third is the open conversation about AI tooling and disclosure. Parquet’s discussion this week on AI tooling policy and AGENTS.md echoes similar discussions in Iceberg earlier this year. The community is figuring out how to handle AI-assisted contributions in a way that is transparent, that preserves the chain of provenance for the code, and that does not slow down legitimate contributors. The fact that these discussions are happening openly on dev lists rather than being handled through informal back channels is the right approach. The decisions made over the next few months will set the pattern for how Apache projects handle AI-assisted development for years.

Looking Ahead

Next week should bring the close of the Iceberg 1.11.0 release if RC4 holds up, which would be a meaningful milestone for the project’s 2026 release cadence. The Iceberg V4 votes on relative paths and content stats are likely to close and move into implementation. The Parquet Footer Working Group will likely publish a charter and meeting schedule. Polaris watchers should expect early signals on what is heading into 1.5.0. Arrow has multiple smaller release votes likely to land in the next seven days, and the Erlang donation vote will close.

The bigger picture is that the lakehouse stack is maturing in step with the AI demand pulling on it. Every one of these format and protocol decisions affects how AI agents will be able to query, update, and reason about lakehouse data in the years ahead. The community is doing the unglamorous work now so the platforms can do the headline work later.

Resources and Further Learning

Get Started with Dremio

• Try Dremio Free – Build your lakehouse on Iceberg with a free trial
• Build a Lakehouse with Iceberg, Parquet, Polaris and Arrow – Learn how Dremio brings the open lakehouse stack together

Free Downloads

• Apache Iceberg: The Definitive Guide – O’Reilly book, free download
• Apache Polaris: The Definitive Guide – O’Reilly book, free download

Books by Alex Merced

• Architecting an Apache Iceberg Lakehouse
• Enabling Agentic Analytics with Apache Iceberg and Dremio
• The 2026 Guide to Lakehouses, Apache Iceberg and Agentic AI
• The Book on Using Apache Iceberg with Python

Published by SAGEWORKS AI — Maun, Botswana

There are moments when you are building something and you realize the tool you need does not exist. Not a library. Not a framework. Not a configuration layer. The language itself is missing.

That is what happened with WebScript.

The Problem That Started This

The browser is a spatial environment. Every element has a position. Every transition has a curve. Every layout is a geometric relationship between objects in space.

And yet every language we use to program the browser is either document-oriented (HTML), cascade-oriented (CSS), imperative and procedural (JavaScript), or a type system overlay (TypeScript). Not one of them treats the browser as what it actually is: a spatial field where entities exist in geometric relationship to each other.

You want a panel that orbits the cursor with a harmonic depth transition? You write 40 lines of JavaScript. You want three cards positioned at phi-distributed angles on a circle? You calculate the trigonometry by hand, write the DOM manipulation, wire the animation frame, and manage the state.

The gap between what you mean and what you have to write is enormous.

WebScript closes that gap. It is the language that should have existed for the spatial web from the beginning.

What WebScript Is

WebScript is a declarative constraint-based spatial DSL that compiles simultaneously to HTML, CSS, SVG, JavaScript, TypeScript, and SQL.

Not a framework. Not a library. Not a preprocessor. A compiled language with its own grammar, type system, compiler pipeline, and browser runtime.

Four properties define it:

Declarative — you describe spatial relationships, not procedures.

Constraint-based — properties are bound to live equations, not static values. When an input changes, every dependent property updates automatically.

Spatial — geometry and position are first-class citizens of the language. orbit, distance, harmonic, phi are not imported utilities. They are primitives.

Relational — entities know each other. A panel knows its viewport. A card knows its cursor. A node knows its cluster. The language graph is a live relational system.

The Grammar — Six Constructs, No More

WebScript has exactly six sentence patterns. This is the complete language:

Construct 1 — Entity Declaration

card {
  depth    : 300
  rotate   : harmonic(0.5)
  position : orbit(cursor)
}

Construct 2 — Relational Binding

The of particle comes from Mandarin topic-comment grammar. It declares ownership, containment, and membership.

panel of viewport {
  sync    : parent.rhythm
  opacity : field(0.85)
  depth   : phi * 300
}

Particles available: of · at · as · by · through · into · from

Construct 3 — Equation Constraint

A live reactive binding. Re-evaluates whenever its inputs change.

card.rotation = cursor.angle * harmonic(0.4)
card.scale    = distance(origin) ^ 0.5

Construct 4 — Classifier Generation

From Mandarin measure words. Declares batch spatial generation.

7 orbital nodes
3 harmonic layers
1 depth field

7 orbital nodes generates seven positioned entities distributed on an orbital path. The mathematics of the distribution is handled by the language.

Construct 5 — Fetch Declaration (WSQL)

Declares a data retrieval that simultaneously generates a SQL query and a spatial layout.

fetch nodes from sales_data {
  where  : revenue > harmonic(0.5)
  order  : spatial(distance(origin))
  limit  : fibonacci(6)[5]
  map    : orbital(result.count)
  radius : phi^i * 40
  color  : hue(revenue.normalized)
}

One declaration. One SQL query. One spatial layout. They are the same thing.

Construct 6 — Page Boundary

Redefines the coordinate origin and containment boundary of a spatial environment.

page {
  boundary : circle(phi * viewport.width)
  origin   : viewport.center
}

The Compilation Model

Every WebScript declaration compiles to all six targets simultaneously from a single relational graph.

Take one declaration:

panel of viewport {
  depth    : phi * 300
  rotation : harmonic(cursor.angle * 0.4)
  position : orbit(cursor, phi * 80)
}

This produces:

HTML

<div class="ws-panel" data-ws-depth="485.4" data-ws-orbit="cursor">

CSS

.ws-panel {
  perspective: 485.4px;
  transform: rotate(var(--ws-panel-rotation));
}

SVG

<circle data-entity="panel" />

positioned at dynamically calculated orbit coordinates.

JavaScript

document.addEventListener('mousemove', e => {
  const angle = Math.atan2(e.clientY - cy, e.clientX - cx);
  panel.style.setProperty('--ws-panel-rotation',
    (0.5 - Math.cos(Math.PI * (angle * 0.4)) / 2) + 'rad');
});

TypeScript

interface WSPanel extends WSEntity {
  depth    : number;
  rotation : WSHarmonic;
  position : WSOrbit;
}

SQL

SELECT *, atan2(cursor_y - panel_y, cursor_x - panel_x) AS orbital_angle
FROM panels

One source. Six manifestations. All mathematically coherent because they all derive from the same relational constraint graph.

The Type System

WebScript has a spatial type system. Every value belongs to one of these types:

WSScalar      → a number, possibly equation-derived
WSEquation    → a live expression that resolves to a scalar
WSEntity      → a named spatial object in the graph
WSParticle    → a relational operator
WSClassifier  → a spatial type qualifier (orbital, harmonic, depth, fibonacci)
WSFunction    → a built-in spatial function
WSBoundary    → a page boundary equation
WSField       → a spatial influence area
WSConstraint  → a reactive binding between entity.property and expression
WSNode        → a data-carrying spatial entity (from WSQL)

The Built-in Function Library

These are the only built-in functions in v0.1. Every one is a spatial or mathematical primitive:

harmonic(t)          0.5 − cos(π·t) / 2
                     Easing. Maps [0,1] to [0,1] with smooth curve.

orbit(target, r)     Position on circle of radius r around target

distance(a, b)       Euclidean distance between two entities

field(value)         Spatially typed scalar wrapper

phi                  1.618033988749895

pi                   3.14159265358979

fibonacci(n)         First n Fibonacci numbers as array

hue(angle)           HSL color derived from spatial angle

decay(rate)          Trail fade multiplier for motion history

harmonic_scale(n)    Distortion scale from polygon side count

No bloat. No utility sprawl. A closed set of spatial primitives that compose into everything else.

The Compiler Pipeline

Five stages. Clear build path.

Stage 1 — Lexer: Reads WebScript source. Produces a token stream. The six grammar constructs are the complete token set.

Stage 2 — Parser: Takes tokens. Produces the Abstract Syntax Tree as a JSON-native structure. Every WS declaration maps 1:1 to a JSON node. This is the bidirectional bridge — the AST is simultaneously the JSON API and the AI-generatable format.

Stage 3 — Graph Resolver: Takes the AST. Builds the relational constraint graph. Resolves entity references, particle relationships, and equation dependencies.

Stage 4 — Target Compilers: Six separate compilers, each consuming the runtime graph simultaneously.

Stage 5 — Runtime Engine: The browser-injected webscript.js. Approximately 8-12kb minified. Executes constraint equations reactively. Maintains the spatial index. Updates CSS custom properties when inputs change.

What WebScript Is Not

These boundaries are permanent:

WebScript does not replace JavaScript’s control flow. No if, for, while, function in WebScript. JavaScript handles all procedural logic.

WebScript does not handle data validation, authentication, business logic, or server-side operations. These remain in their own layers.

WebScript does not transform HTML into geometry. HTML keeps its rectangular document model. WebScript projects geometry inside it.

WebScript does not replace CSS’s cascade, specificity, or responsive design system. It generates mathematically coherent values for CSS to apply.

WebScript does not attempt to be general-purpose. It is a domain-specific language for one domain: spatial relational computation in the browser.

The Intellectual Lineage

WebScript draws from four sources and is none of them:

CSS — the declarative model, the property-value syntax, the idea that you describe what you want rather than how to achieve it.

GLSL — mathematical spatial thinking as a first-class mode. The browser as a field of values rather than a document of nodes.

GraphQL — the unified query philosophy. One declaration surfaces in multiple layers simultaneously. The query and the presentation are the same expression.

Mandarin grammar — the topic-comment structure and classifier system. panel of viewport reads as a topicalized statement: “speaking of the viewport, here is the panel.” 7 orbital nodes uses measure-word logic — a spatial classifier before a noun.

The result is something none of those are: a language that treats the browser as a spatial field and programs it as a relational mathematical system.

Where It Goes

WebScript is one language with many native communities. The grammar does not change between them. The demonstrations do.

For web game developers: a declarative spatial layer for entity positions, collision constraints, camera following, and particle systems. The first language where game spatial logic is declared rather than procedurally written.

For data visualization engineers: a single declaration that simultaneously produces a SQL query and a spatial layout. The gap between data and geometry, closed.

For creative coders: the mental model of p5.js — fields, forces, particles, harmonics — expressed as a constraint grammar rather than a procedural script. Browser-native. No wrapper required.

For TypeScript developers: generated typed interfaces for every spatial entity. The mathematical coherence of the layout enforced at compile time.

For spatial / XR web: a declarative syntax for scene graphs, proximity constraints, and gaze-reactive positioning that reads as clearly as CSS but reasons in three dimensions.

For educators: the first programming language where spatial mathematics and visual output are the same syntax. A geometry class and a programming class simultaneously.

The Repository

The v0.1 implementation is live.

The repository contains the full language specification, compiler architecture, type system, runtime engine, standard library, WSQL bridge, CLI, playground, and test suite.

Source, specification, and build path: https://github.com/PEACEBINFLOW/webscript

The implementation is MIT licensed. The specification is Creative Commons Attribution 4.0 — anyone may implement WebScript, but Peace Thabiwa must be credited as the language designer.

The name WEBSCRIPT and the ⟁ symbol are trademarks of SAGEWORKS AI.

The Statement

The browser has been a spatial environment for thirty years.

It has never had a spatial language.

WebScript is that language.

Peace Thabiwa
SAGEWORKS AI — Maun, Botswana
https://github.com/PEACEBINFLOW/webscript

This is a submission for the Gemma 4 Challenge: Write About Gemma 4

Shipping on Gemma 4: chain-of-thought leakage, MoE-vs-Dense, and on-device pragmatism

I built and shipped Curio Kid, a kid-safe multimodal Android app where my 6-year-old asks Luna (a Gemma-4-powered tutor) anything by text, voice, or camera. The product story is in my other submission. This post is the engineering writeup — three things about Gemma 4 that I had to actually work around in production, with the code and reasoning behind each fix.

If you’re about to ship a Gemma 4 app, these are the three traps I’d want to know about on day one.

1. Chain-of-thought leakage is real, and it hits the user

Gemma 4 is good at following structured system prompts. Too good, sometimes. Give it a strict persona spec and it will occasionally show you the rubric while answering. In my testing, a meaningful slice of responses came back like this:

**Intent:** child wants to know why the sky is blue
**Tone check:** warm, age-5 vocabulary, no jargon
**Final Polish:**

Great question! The sky is blue because…

The “Final Polish:” line is the give-away — Gemma is narrating its own polishing step before giving the answer. For a chatbot aimed at a 6-year-old, that’s not a quirk; it’s a UX bug.

The naïve fix doesn’t work

The obvious instinct is “tell the model not to do this in the system prompt.” I tried. My prompt now contains a half-page of “never write section labels like ‘Final Polish’, ‘Self-Correction’, ‘Reasoning’, ‘Plan’… your very first word must be part of the actual answer”. It helps. It doesn’t eliminate.

The reason it can’t eliminate the issue: instruction-following is a soft constraint. The same model that’s smart enough to follow a 50-line persona spec is also smart enough to think about how to follow it — and sometimes the thinking ends up on the wire.

The real fix: a two-stage cleaner that knows the difference between meta and content

I ended up with a 100-line response sanitiser (LunaAI.kt) that does three things in order:

Stage 1 — Anchor detection. Look for “final answer” / “polished response” / “answer:” anchors on their own line. If present, throw away everything before the last one and keep only what follows. This handles the dominant failure mode (model planned out loud, then gave the real answer at the bottom).

val finalAnchorLine = Regex(
    "(?im)^\s*\*{0,2}\s*" +
        "(?:final(?:\s+(?:polish|answer|response|reply|draft|version))?" +
        "|polished(?:\s+(?:answer|reply|response))?" +
        "|the\s+answer|answer|response|reply)" +
        "\s*\*{0,2}\s*:\s*$",
)
val afterAnchor = finalAnchorLine.findAll(raw).lastOrNull()
    ?.let { raw.substring(it.range.last + 1) } ?: raw

Stage 2 — Paragraph-level meta filter. Split by blank lines, drop any paragraph that contains chain-of-thought prose cues — phrases like “the prompt says…”, “I’ll treat the question as…”, “drafting…”, “let me revise…”. This catches the case where the model narrates its process in prose instead of with labels.

Stage 3 — Line-level scrub. A safety net for leaks embedded inside an otherwise-good paragraph: bullet/label lines like * Intent: … or **Tone check:** ….

The non-obvious part: what not to filter

The interesting design problem isn’t writing the regex; it’s making sure you don’t kill legitimate content. Three rules I learned the hard way:

• Never filter on the word “think” alone. Phrases like “Let me think of a fun example!” are exactly the warm tone you want from a kid’s tutor. My meta-regex matches “let me / I’ll / I will / I should” only when followed by a planning verb (plan, draft, rewrite, revise, polish, reconsider, interpret). “Let me think of” slips through. Good.
• Only apply bullet-stripping when a leak is detected nearby. Gemma sometimes does legitimately produce a bulleted list when the kid asks “give me three facts about pandas”. You don’t want to scrub bullets unconditionally; you want to scrub them only when other meta-leakage is already visible on the page.
• Have a fallback for “scrubbed to nothing”. If filtering empties the response, return “Hmm, let me think about that another way — could you ask me again?” — not a blank bubble.

Takeaway

If you’re building a user-facing app on Gemma 4 — especially with kids, customer support, or anywhere “the model thinking out loud is bad UX” — assume CoT leakage will happen and ship a sanitiser. A sanitiser is also dramatically cheaper than fine-tuning, and it composes with whichever model variant you swap in next month.

2. MoE vs Dense: how I actually chose between gemma-4-26b-a4b-it and gemma-4-31b-it

The Gemma 4 family ships three architectures for very different deployment targets:

For a kid-facing multimodal chat app I shipped the 26B MoE as the default and the 31B Dense as an opt-in “thinker” mode. Both have the 256K context window and Apache-2.0 licence, so the choice is purely about latency vs. depth.

Why MoE wins the default slot

The MoE’s superpower isn’t raw size — it’s that only a slice of experts is activated per token. You pay ~4B of compute per token while keeping 26B of capacity available across the network. For my workload that translated into three concrete wins:

1. First-token latency that feels like chat, not like batch inference. Streaming starts in well under a second on Google AI Studio. A 6-year-old’s patience is shorter than the inverse of his curiosity rate, so this matters more than benchmark scores.
2. Multimodal in the same model. No separate vision pass, no second API call for the image. “What kind of bug is this?” with a photo attached is one request.
3. 256K context lets the Curiosity Digest be a one-shot. End of the day, I cat the whole transcript into a single prompt and ask Gemma to produce a structured digest. No RAG, no map-reduce summarisation. The whole “parent dashboard” feature is ~30 lines of glue because of this.

Why Dense earns its own button

For the questions that are genuinely hard — “why do mirrors flip left-and-right but not up-and-down?” is the canonical kid-stumper — the 31B Dense produces noticeably better multi-step reasoning. It’s slower and pricier per call, so it’s not the right default for “explain photosynthesis in three sentences”, but it’s the right tool when the kid trips into something philosophical.

The mental model I’d suggest for picking

Forget the parameter count for a second and ask three questions about your workload:

• Is latency-to-first-token a UX requirement? → MoE.
• Are you doing multimodal in the same call? → MoE (with image input).
• Do you measurably gain on the hardest 10% of your prompts when you swap to Dense? → ship both, give users a toggle.

Don’t pick on price; pick on what your prompts actually need. The MoE is the right answer for most chat workloads. The Dense is the right answer when you can articulate the reasoning gap.

3. On-device pragmatism: cloud-first isn’t a cop-out

The most photogenic Gemma 4 demos run E2B on a Pixel or a Raspberry Pi. They’re amazing. They’re also not the right default for a consumer Android app you want real families to use.

Two realities pushed me cloud-first:

1. Not every phone can run Gemma 4 locally. A multi-gigabyte model needs the RAM, the storage, and the NPU/GPU to be worth the wait. Older flagships, mid-range phones, and the hand-me-down tablet a kid actually gets to use aren’t there yet. Gating an app on “must own a current flagship” defeats the point of an accessible kids’ app.
2. Quality matters more than offline-ness for a 6-year-old. A child being confidently told “the moon is made of cheese” by an under-cooked tiny model is a worse experience than a 2-second wait over Wi-Fi for the 26B MoE.

The architecture trick that lets you defer the choice

What I’d recommend for anyone shipping today: don’t pick between cloud and on-device. Pick a backend interface and write three implementations.

interface LlmBackend {
    suspend fun ask(
        systemPrompt: String,
        history: List<ChatTurn>,
        userText: String,
        image: Bitmap?,
        modelName: String,
    ): String

    suspend fun summarise(
        systemPrompt: String,
        rawHistoryText: String,
        modelName: String,
    ): String
}

In Curio Kid that interface has three implementations: GoogleAiStudioBackend, OpenRouterBackend, and a LocalGemmaBackend stub that throws a friendly “on-device Gemma 4 isn’t installed on this phone yet” until a MediaPipe .task file is wired in. Same system prompt, same response cleaner, same UI for all three. The provider is a single enum in EncryptedSharedPreferences and a one-tap toggle in Settings.

The pay-off: when E2B becomes the right default — when phones catch up, when multimodal lands in MediaPipe, when battery cost makes sense — I change one factory method. The persona, the safety prompt, the digest pipeline, the cleaner all carry over. The same kid talking to the same Luna; the model just moved into the phone.

That’s the on-device pragmatism: don’t bet on offline-first when your users can’t run it, but don’t lock yourself out of it either. Bet on the abstraction.

Three small SDK gotchas I’d want to have known on day one

While I’m here: three concrete Gemini-SDK-on-Android landmines that cost me an evening each.

• The 80-second socket timeout is hard-coded. RequestOptions doesn’t expose a knob to change it. If Gemma 4 takes longer than 80 seconds to start emitting tokens, you’ll get a SocketTimeoutException even though the model is fine. Fix: use generateContentStream instead of generateContent. The read timer resets on each chunk, so as long as tokens are flowing you never trip the cap.
• MAX_TOKENS throws, it doesn’t return the partial text. The Kotlin SDK raises ResponseStoppedException from the .text convenience getter when finish reason ≠ STOP. You have to catch it and walk candidates[0].content.parts for TextParts yourself to recover the 90%-complete answer the user nearly got.
• A 500 from the upstream model often surfaces as MissingFieldException from kotlinx-serialization. When the Gemini backend has a hiccup it returns JSON that the SDK’s strict deserialiser doesn’t recognise, and the exception you see is the serialisation failure, not the underlying 500. Worth normalising every error class through a single friendlyError() mapper that walks the cause chain — the real problem is usually two layers down.

TL;DR

Three lessons from shipping on Gemma 4:

1. CoT leakage is a UX problem, not a prompt problem. Ship a sanitiser. Be careful what you scrub.
2. MoE is the right default for chat; Dense is the right tool for hard reasoning. Give users the toggle, pick by latency-and-multimodal vs. reasoning-on-the-hardest-10%.
3. Cloud-first isn’t a cop-out, but architecting for on-device later is non-negotiable. A LlmBackend interface with three implementations buys you the option.

The Gemma 4 family is the first open-model release where I genuinely had to think about which member to ship for which job — that’s a great problem to have. If you’re building on it, I hope these save you a weekend each.

Code is at github.com/sann3/curio-kid if you want to read the cleaner, the backend interface, or the friendly-error mapper in full. Happy to answer questions in the comments.

Thanks to the DEV team and Google for the challenge!

How DropZap Handles Instagram and TikTok Downloads: A Technical Walkthrough

DropZap is a free social media video downloader built with Next.js 14. It handles Instagram, TikTok, Twitter/X, Facebook, Reddit, Pinterest, and Threads — all from one interface. I want to walk through the technical approach behind the two most complex platforms: Instagram and TikTok.

Stack Overview

• Framework: Next.js 14 (App Router)
• Language: TypeScript
• Download engine: yt-dlp via Node child process
• Audio merging: ffmpeg (for Reddit, and some formats)
• UI: Tailwind CSS + shadcn/ui
• Hosting: Render / Railway (Docker container)

Instagram Download Flow

Instagram is complex because it serves three distinct content types that require different handling:

1. Reels and Video Posts

The API route /api/stream receives the Instagram post URL from the client. The server spawns yt-dlp with the URL and --format bestvideo+bestaudio/best flags.

// Simplified — actual route handles cookies + proxies
const result = await execa('yt-dlp', [
  '--format', 'bestvideo+bestaudio/best',
  '--merge-output-format', 'mp4',
  '--output', '-',  // pipe to stdout
  instagramUrl
]);

The raw bytes are streamed back to the client via a ReadableStream. This avoids writing temp files to disk on the server, keeping the container stateless.

2. Photos (Single Image Posts)

For photo posts, yt-dlp extracts the direct CDN URL of the JPEG. We then proxy the CDN response to the client with Content-Disposition: attachment headers so the browser triggers a download instead of navigating to the image.

3. Carousels (Multi-Slide Posts)

Carousels are the trickiest. A single Instagram carousel URL can contain 2–20 slides. yt-dlp returns them as a playlist of individual entries.

The server:

1. Runs yt-dlp --dump-json to get the manifest of all slide URLs
2. Fetches each slide in parallel using Promise.all
3. Packages them into a ZIP using the archiver npm package
4. Streams the ZIP to the client

import archiver from 'archiver';

const archive = archiver('zip', { zlib: { level: 5 } });
// pipe archive to response
res.setHeader('Content-Type', 'application/zip');
archive.pipe(res);

for (const [i, slideUrl] of slides.entries()) {
  const imageStream = await fetch(slideUrl).then(r => r.body);
  archive.append(imageStream, { name: `slide-${i + 1}.jpg` });
}

await archive.finalize();

TikTok Download Flow

TikTok is more aggressive about blocking programmatic access than Instagram. The key challenges:

Watermark Removal

TikTok serves two versions of each video:

• A “watermarked” version (with the floating TikTok logo + username)
• A “no-watermark” version accessible via a different CDN path

yt-dlp knows which endpoint to hit for the no-watermark version because it parses TikTok’s internal API response, which includes a play_addr_h264 field pointing to the clean copy. This is the same mechanism that tools like ssstik and SnapTik use.

Rate Limiting and Anti-Bot

TikTok’s API endpoints rotate and require specific request headers (User-Agent, Referer, and a session token derived from the device signature). yt-dlp maintains an updater that tracks these header requirements and patches them when TikTok changes their scheme — which happens roughly every 2–4 weeks.

This is why keeping yt-dlp updated is critical. In the Dockerfile:

# Always pull the latest yt-dlp binary at build time
RUN pip install --upgrade yt-dlp

On production, a weekly cron job runs yt-dlp --update-to nightly to stay ahead of TikTok API changes.

Rate Limiting on the DropZap Side

The app applies a server-side rate limiter (1 request per 5 seconds per IP) using an in-memory Map to track timestamps. No Redis needed for the traffic levels a free tool receives — if this ever needed to scale horizontally, swapping to Redis would be a one-line change.

// lib/rate-limit.ts
const requests = new Map<string, number>();

export function checkRateLimit(ip: string): boolean {
  const last = requests.get(ip) ?? 0;
  const now = Date.now();
  if (now - last < 5000) return false; // 5 second window
  requests.set(ip, now);
  return true;
}

Performance Decisions

Code splitting: Every downloader tab is loaded via next/dynamic with ssr: false. Only Instagram (the default tab) is statically bundled. This cut unused-JS budget by ~317 KiB in a PageSpeed audit.

Streaming responses: Video bytes are streamed from yt-dlp‘s stdout directly to the HTTP response. The server never writes a temp file to disk. This matters both for latency and for running on containers with limited ephemeral storage.

No client-side state persistence: The download history feature was removed after user feedback that it confused people. localStorage had a habit of showing stale “pending” entries. Simpler is better.

Try It

The live tool is at dropzap.digital. Instagram is the default tab; TikTok is next. Paste any public post URL, hit Download.

If you’re building something similar and have questions about any specific part of the architecture — cookie handling, the archiver pipeline, the yt-dlp subprocess management — ask in the comments.

The image of the dedicated developer is… well, a lot of images. Often it’s romanticized – hunched over a glowing screen, fueled by caffeine, relentlessly coding. Instagram might show late nights and “grind culture,” but that’s not the real experience of burnout. For those of us deep in the technical trenches, it manifests very differently.

It’s not a sudden crash. It’s a slow erosion, starting with the things that used to energize you. As a software developer building an AI/ML-focused business, with a workstation capable of handling demanding workloads (Ryzen 9 9950X3D + RTX 5090 – ), I initially thought I was built for long hours. I was wrong. The issue wasn’t simply the amount of time, but the increasing cognitive demands of the work itself.

It starts with technical debt. Not just knowing you have it, but the feeling that addressing it is increasingly difficult. It’s the creeping realization that every new feature adds another layer of complexity to a fragile system. You start with a simple script, a quick fix, a “works on my machine” moment… and then it multiplies. That initial convenience turns into a web of dependencies you dread touching. It’s not about the amount of work, it’s the feeling that the work isn’t moving you forward.

Then comes the operational weight. We’re building increasingly complex systems. The expectation isn’t just “getting something working” it’s building something that lasts. This means dealing with CI/CD pipelines, database migrations without downtime, and the constant pressure to scale. You’re not just coding, you’re building and maintaining the infrastructure around that code, and increasingly finding yourself reacting to issues as they arise.

The AI landscape adds another layer. We’re entering a new era of AI orchestration. The promise of AI-powered tools is immense, but it also creates a pressure to adopt everything. The illusion that simply having AI solves problems is pervasive. You become a manager of AI agents as much as a developer, and that adds significant cognitive load.

Here’s what it feels like, specifically:

• Constant context switching: Jumping between addressing urgent production issues, designing new features, and evaluating the latest AI tools.
• Sleep disturbances: Light, restless sleep, and multiple other entries in the same dream log. You’re mentally replaying code reviews and potential outages instead of resting, which then leads to…
• Difficulty concentrating: …a fuzzy brain and inability to focus, even when you do sit down to code.
• Loss of curiosity: The things that used to excite you about technology – learning a new language, exploring a new framework – feel draining.
• Decreased code quality: You start taking shortcuts, prioritizing speed over maintainability, often due to feeling overwhelmed by the workload.
• A sense of futility: The feeling that, no matter how much you do, you’re not making a meaningful impact. Like you’re building in a vacuum.

Burnout isn’t a badge of honor. It’s a signal that something is fundamentally broken. It’s time to reassess, prioritize, and remember why you started building in the first place. It’s time to move from just doing the work, to building a system that supports you while you do it.