Centralising Upsun Technical References with Meta-Upsun

Managing technical reference data is a grind when information for runtimes and services is scattered everywhere. It leads to inconsistent automation and inaccurate documentation.

We built Meta-Upsun to fix this. It is a read-only and versioned API that centralizes runtime, service, region, and PHP data into one public source.

Why it matters for your workflow:

  • Provides a single source of truth for consistency
  • Automates documentation to keep it accurate
  • Builds in reliability and validation from the start
  • Aligns registry status with upstream support

This shift improves the developer ecosystem and leverages agility for better automation.

Check out the full article here:


Centralising Upsun Technical References with Meta-Upsun – Upsun Developer

How Upsun centralised runtime, service, region, and PHP reference data into a single public API to improve consistency, automation, and documentation accuracy.

favicon
developer.upsun.com

NEXUS‑GAMES: Building a Free Retro & Arcade Hub — Looking for Collaborators


 I’ve been working on NEXUS‑GAMES 👉 https://nexus-games.onrender.com
Image description) — a free online hub for HTML5 and JS‑based retro and arcade experiences. The project is live, but I want to take it further with new features, better performance, and more community input.

Here’s what I’ve built so far:
1️⃣ HTML5 Canvas + JavaScript for nostalgic gameplay.
2️⃣ Responsive UI/UX design for smooth browser controls.
3️⃣ Deployment on Render with analytics tracking.
4️⃣ Open‑source repo available here: GitHub Project.

💡 I’m now looking for collaborators — developers, designers, or anyone passionate about browser games — to help expand the project. Whether it’s adding new titles, improving performance, or experimenting with multiplayer features, your contributions would be hugely valuable.

👉 Try the games at https://nexus-games.onrender.com

👉 Check out the code at GitHub https://github.com/bahati-sage

Why Gemma 4 Matters for Africa and Low-Connectivity Regions

Gemma4 unlocking healthcare, education and underserved communities

Why Gemma 4 Matters for Africa and Low-Connectivity Regions

This is a submission for the Gemma 4 Challenge: Write About Gemma 4

Exploring how Gemma 4 could unlock offline AI for education, healthcare, and underserved communities.

Gemma4 unlocking healthcare, education and underserved communities

What Happens When Frontier AI No Longer Requires Constant Internet?

For years, the AI revolution has largely belonged to places with fast internet, powerful cloud infrastructure, and organizations that can afford expensive API bills.

If you wanted advanced AI, the assumption was simple:

You needed the cloud.

But what if that assumption is starting to change?

With the release of Gemma 4, Google’s newest family of open models, we are entering a world where capable AI can run locally, on devices ranging from laptops and phones to Raspberry Pi setups and enterprise systems.

That shift matters everywhere.

But it may matter most in regions where internet access is expensive, inconsistent, or unavailable.

In places across Africa and other low-connectivity regions, Gemma 4 is more than another AI model release.

It represents a different possibility:

What if world-class AI could work even when the internet doesn’t?

The Hidden Problem with Modern AI

Most people interact with AI through cloud services.

You type a prompt.

Your request travels across the internet.

A remote server processes it.

Then a response comes back.

This works well — when infrastructure exists.

But many parts of the world face challenges that cloud-first AI quietly assumes away:

  • Expensive internet costs
  • Unstable connections
  • Power interruptions
  • Limited access to high-end cloud infrastructure
  • Schools and institutions operating under tight budgets

In many communities, connectivity is not always guaranteed.

And if AI only works online, access to intelligence becomes unequal.

This creates an uncomfortable question:

Does useful AI only belong to places with reliable internet?

Gemma 4 suggests a different answer.

Why Gemma 4 Feels Different

Gemma 4 stands out because it combines several things that rarely exist together:

✅ Open models

✅ Advanced reasoning

✅ Native multimodal capabilities

✅ 128K context window

✅ Ability to run on smaller hardware

Instead of forcing developers into expensive cloud-only systems, Gemma 4 offers something more flexible.

You can run it:

  • Locally on laptops
  • On edge devices
  • On phones
  • On Raspberry Pi systems
  • In large-scale deployments

This flexibility changes what becomes possible in environments with limited connectivity.

And that matters.

A lot.

1. Offline AI in Schools Could Change Education

Imagine a rural school with:

  • Limited internet
  • Few teachers
  • Outdated textbooks
  • Large classrooms

Now imagine students still having access to an intelligent tutor.

Not through expensive subscriptions.

Not through constant internet access.

But locally.

A Gemma 4-powered educational assistant could:

Explain difficult concepts

A student struggling with algebra or chemistry could ask questions repeatedly without fear of embarrassment.

Adapt explanations

Some students learn visually.

Some prefer examples.

Some need simpler explanations.

AI tutoring becomes personalized.

Work without internet

This may be the most important part.

Students should not lose access to learning because connectivity disappears.

In many places, educational inequality is partially an infrastructure problem.

Local AI changes that equation.

Instead of asking:

“How do we bring internet to every student first?”

We can also ask:

“How much education can we bring offline?”

That question becomes realistic with models like Gemma 4.

2. Rural Healthcare Could Benefit from Local Intelligence

Healthcare systems in many low-resource regions face major constraints:

  • Doctor shortages
  • Long travel distances
  • Limited specialist access
  • Connectivity issues

Of course, AI should never replace medical professionals.

But local AI can become a support tool.

Imagine healthcare assistants that can:

  • Summarize medical guidelines
  • Help interpret symptoms
  • Translate medical information
  • Assist with record organization
  • Support health education

Most importantly:

Sensitive medical data could remain local.

This matters for privacy.

Instead of constantly sending patient information to external cloud services, institutions may choose systems that process information closer to where care happens.

Imagine a healthcare worker in a rural clinic accessing medical explanations and educational support without depending on stable internet.

That possibility matters.

3. Low-Bandwidth Environments Deserve Better Technology

Many modern software experiences assume:

Fast Wi-Fi is normal.

But globally, that is not always reality.

Developers sometimes build products assuming unlimited bandwidth.

Reality says otherwise.

In low-bandwidth environments:

  • Pages fail to load
  • Cloud tools become unreliable
  • AI systems stop working entirely

This is where local models become powerful.

If intelligence exists directly on the device:

  • Latency drops
  • Costs fall
  • Dependence on connectivity decreases

The experience becomes more reliable.

And reliability matters more than flashy features.

Because in many underserved regions:

Technology that works consistently beats technology that only works under perfect conditions.

4. The Cost of Cloud AI Is a Bigger Problem Than We Admit

Cloud AI is powerful.

But it is also expensive.

Developers quickly run into:

  • API usage fees
  • Token costs
  • Scaling expenses
  • Infrastructure limitations

For startups, schools, NGOs, and small developers, these costs add up fast.

Gemma 4 introduces another option:

Run capable models locally.

That changes the economics.

Instead of paying continuously for every request, organizations can invest in hardware and run workloads closer to home.

For regions with tighter budgets, this matters tremendously.

Innovation should not belong only to organizations with large cloud budgets.

A small school, local startup, or nonprofit should still have access to advanced intelligence.

Gemma 4 helps make that possible.

5. Why Open Models Matter

There is another reason Gemma 4 feels important:

It is open.

That matters because developers can:

  • Experiment freely
  • Customize systems
  • Fine-tune for local needs
  • Adapt to local languages
  • Build domain-specific assistants

This becomes especially valuable in Africa.

Many local challenges are unique.

Languages are diverse.

Educational contexts differ.

Healthcare realities differ.

Agricultural advice differs.

An open model means developers can adapt intelligence to local problems instead of waiting for one-size-fits-all solutions.

Imagine localized AI assistants trained to support:

  • Farmers with crop advice
  • Students preparing for exams
  • Teachers managing classrooms
  • Small businesses learning digital skills
  • Communities accessing information in local languages

That is the real promise of open AI.

The Bigger Shift: From Cloud Dependence to Local Intelligence

For a long time, the future of AI looked centralized.

Massive servers.

Massive companies.

Massive compute.

But models like Gemma 4 hint at another future:

Distributed intelligence.

A future where capable AI exists:

  • In schools
  • In hospitals
  • On personal devices
  • In remote regions
  • Without permanent internet access

That future feels especially meaningful for underserved communities.

Because access to intelligence should not depend entirely on connectivity.

And perhaps the most exciting part is this:

Developers everywhere can now participate.

Not just large tech companies.

Not just well-funded startups.

But local builders solving local problems.

Final Thoughts

Gemma 4 is impressive because of its multimodal abilities, reasoning capabilities, and massive context window.

But what excites me most is something simpler:

Accessibility.

The ability to run powerful intelligence locally could reshape who benefits from AI.

For developers in Africa and other low-connectivity regions, the question is no longer:

“Can we participate in the AI future?”

The better question may be:

“What can we now build because frontier AI finally works closer to home?”

And that may be the most exciting shift of all.

If AI is going to shape the future, then that future should not belong only to places with perfect infrastructure.

It should belong to everyone.

Including the places that need it most.

Some parts of this Article was written by CHATGPT

Writing to an Apache Iceberg Table: How Commits and ACID Actually Work

The Iceberg write process from data file creation through the atomic catalog pointer swap

This is Part 6 of a 15-part Apache Iceberg Masterclass. Part 5 covered hidden partitioning. This article walks through the exact steps an engine takes when writing data to an Iceberg table, when the write becomes visible, and how concurrent writers are handled.

Understanding the write process is critical because it explains why Iceberg can provide ACID guarantees on top of object storage, something that seems impossible when you consider that S3, ADLS, and GCS have no built-in transaction support. The answer is that ACID lives entirely in the metadata layer, not in storage.

Table of Contents

  1. What Are Table Formats and Why Were They Needed?
  2. The Metadata Structure of Current Table Formats
  3. Performance and Apache Iceberg’s Metadata
  4. Technical Deep Dive on Partition Evolution
  5. Technical Deep Dive on Hidden Partitioning
  6. Writing to an Apache Iceberg Table
  7. What Are Lakehouse Catalogs?
  8. Embedded Catalogs: S3 Tables and MinIO AI Stor
  9. How Iceberg Table Storage Degrades Over Time
  10. Maintaining Apache Iceberg Tables
  11. Apache Iceberg Metadata Tables
  12. Using Iceberg with Python and MPP Engines
  13. Streaming Data into Apache Iceberg Tables
  14. Hands-On with Iceberg Using Dremio Cloud
  15. Migrating to Apache Iceberg

The Six Steps of a Write

The Iceberg write process from data file creation through the atomic catalog pointer swap

Every write operation (INSERT, DELETE, UPDATE, MERGE) follows the same six-step sequence:

Step 1: Write Data Files

The engine writes new Parquet (or ORC/Avro) files to object storage. These files are placed in the table’s data directory but are not yet referenced by any metadata. At this point, they are invisible to all readers. They are just orphan files sitting in storage.

Step 2: Create Manifest Entries

For each new data file, the engine creates a manifest entry containing the file path, file size, row count, partition values (computed using the table’s partition transforms), and column-level statistics (min, max, null count).

Step 3: Create or Update Manifest Files

The engine bundles manifest entries into Avro-format manifest files. If the write affects only a single partition, it may create one new manifest. If it touches many partitions, it may create multiple manifests. Existing manifests from previous snapshots that were not modified are carried forward by reference, not copied.

Step 4: Create a Manifest List

A new manifest list (Avro) is created that references all manifests for this snapshot: the new manifests from Step 3 plus the unchanged manifests inherited from the previous snapshot. This manifest list represents the complete state of the table after this write.

Step 5: Create New Metadata File

A new metadata.json file is written, containing the table schema, partition spec, properties, and the snapshot list. The new snapshot (pointing to the manifest list from Step 4) is appended to the list. The previous metadata.json remains in storage, unchanged.

Step 6: Atomic Commit (The Pointer Swap)

The engine asks the catalog to update its pointer from the old metadata.json to the new one. This is a compare-and-swap operation: the catalog checks that the current pointer matches what the engine expects, and only then updates it.

This is the exact moment the transaction commits. Before the swap, readers see the old snapshot. After the swap, readers see the new snapshot. There is no in-between state.

Why This Provides ACID Guarantees

How ACID works through the atomic metadata pointer swap

The pointer swap mechanism delivers all four ACID properties:

Atomicity. The entire write is visible or invisible. If the engine crashes after writing data files but before the pointer swap, the data files are orphans. They exist in storage but no metadata references them. Readers never see partial writes. A cleanup process (covered in Part 10) can remove these orphans later.

Consistency. The new metadata.json contains a valid schema, valid partition specs, and consistent statistics. The catalog only accepts the swap if the metadata file is well-formed.

Isolation. Readers load a specific snapshot and operate on it for the duration of their query. Even if a new snapshot is committed while they are reading, their query continues to see the snapshot they started with. This is snapshot isolation, and it happens naturally because each snapshot is immutable.

Durability. Once the catalog confirms the pointer swap, the new state is persisted. The metadata file and all data files are already in durable object storage. The catalog’s own persistence layer (a database for REST catalogs, a metastore for Hive) provides the durability guarantee for the pointer itself.

Concurrent Writes: Optimistic Concurrency Control

How two concurrent writers are resolved through optimistic concurrency with retry on conflict

When two engines write to the same table simultaneously, Iceberg uses optimistic concurrency control (OCC):

  1. Both writers read the current metadata (say v1.metadata.json) and begin their writes independently.

  2. Writer A finishes first and successfully swaps the catalog pointer from v1 to v2.

  3. Writer B attempts to commit by swapping from v1 to v3. The catalog detects that the current pointer is v2, not v1. The swap fails.

  4. Writer B retries. It reads v2.metadata.json and checks whether its changes conflict with Writer A’s changes:

  • No conflict (different partitions): Writer B’s new files affect partition region=west, and Writer A’s changes affected region=east. The changes are compatible. Writer B rebases its manifest list to include Writer A’s manifests and creates a new v3.metadata.json that reflects both writes. The swap from v2 to v3 succeeds.

  • Conflict (same files modified): Both writers modified the same data files (e.g., both deleted rows from the same file). The changes cannot be automatically merged. Writer B’s operation fails with a conflict error.

This model works well for append-heavy workloads (multiple jobs writing to different partitions), which is the dominant pattern in data lakes. Dremio handles concurrent writes and automatic retries through its engine, and its Open Catalog provides the atomic compare-and-swap through the REST catalog protocol.

Delete and Update Operations

Iceberg supports three strategies for modifying existing rows:

Copy-on-Write (COW)

The engine reads the affected data files, removes or modifies the target rows, and writes entirely new files containing the result. The old files are removed from the manifest (marked as deleted), and the new files are added. This is simple but expensive for large files when only a few rows change.

Merge-on-Read (MOR) with Position Delete Files

Instead of rewriting data files, the engine writes a small “position delete file” that lists the file path and row positions of deleted rows. At read time, the engine reads both the data file and the delete file, filtering out deleted rows during scan. This makes writes fast but adds read-time overhead.

Merge-on-Read with Deletion Vectors (Iceberg v2+)

Deletion vectors are a compact bitmap representation of deleted rows within a file. They are more storage-efficient than position delete files and faster to evaluate during reads. Engines like Dremio and Spark use deletion vectors for row-level updates in production.

Merge-on-Read with Deletion Vectors

What Happens to Old Data?

After a commit, the previous snapshot’s data files are not deleted. They remain in storage and are referenced by the old snapshot. This enables:

  • Time travel: Query the table as of any retained snapshot
  • Rollback: Revert the table to a previous snapshot if a bad write is detected
  • Incremental reads: Process only the files that changed between two snapshots

Eventually, old snapshots are expired (removed from the metadata) and their orphan files are cleaned up. This maintenance is covered in Part 10.

The Catalog’s Role in Commits

The catalog is the gatekeeper of consistency. Without a catalog providing atomic compare-and-swap, concurrent writers could overwrite each other’s commits. The choice of catalog affects write reliability:

  • REST catalogs (Dremio Open Catalog, Polaris) provide server-side CAS operations
  • Hive Metastore uses database-level locking for CAS
  • AWS Glue provides CAS through its API
  • Hadoop Filesystem catalogs use file-system rename atomicity (less reliable on object storage)

Part 7 covers the catalog landscape in detail.

Books to Go Deeper

Free Resources

Meslek lisesi öğrencileri atık boruları sanata dönüştürdü

Samsun’da Atakum Mesleki ve Teknik Anadolu Lisesi öğrencileri, atık çelik boru ve fittings parçalarını geri dönüştürerek retro aplik, şamdan ve çeşme görünümlü aydınlatmalar yaptı. Proje, sıfır atık ve geri dönüşüm temasıyla farkındalık oluşturmayı hedefliyor.