main
Branches
Tags
Go to file
Code
Open more actions menu
Folders and files
Name
Name
Last commit message
Last commit date
Latest commit
History
155 Commits
155 Commits
.github
.github
checks
checks
containers
containers
kernel-configs
kernel-configs
modules
modules
nixos-anywhere
nixos-anywhere
overlays
overlays
packages
packages
patches
patches
playbooks
playbooks
scripts
scripts
templates/
dgx-spark
templates/
dgx-spark
tests
tests
vendor
vendor
.gitignore
.gitignore
CLAUDE.md
CLAUDE.md
LICENSE
LICENSE
README.md
README.md
flake.lock
flake.lock
flake.nix
flake.nix
usb-configuration-base.nix
usb-configuration-base.nix
usb-configuration.nix
usb-configuration.nix
View all files
Repository files navigation
README
MIT license
More
items
Nix and NixOS on the DGX Spark
Try DGX Spark playbooks using Nix on DGX OS, or install NixOS on your DGX
Spark for the full Nix experience. The repository provides USB images and a
NixOS module with settings for DGX Spark systems.
This works on the NVIDIA DGX Spark itself and also on the Asus Ascent GX10.
See my 5 minute lightning talk from
Planet Nix
for an intro:
Using Nix on DGX OS (Ubuntu)
You can use the dev shells and playbooks in this repo on NVIDIA DGX OS (Ubuntu)
without installing NixOS.
Setup
Install Nix
using the
official installer
:
sh
<(
curl -L https://nixos.org/nix/install
)
–daemon
Alternatively, you can use the
Determinate Nix Installer
:
curl –proto
'
=https
'
–tlsv1.2 -sSf -L https://install.determinate.systems/nix
|
sh -s — install
Enable flakes and nix-command
by adding to
/etc/nix/nix.conf
:
experimental-features = nix-command flakes
Enable the graham33 Cachix cache
— see
Caching
below.
Running on Non-NixOS (e.g. DGX OS)
On non-NixOS systems, Nix-built CUDA applications need
nix-gl-host
to find the host GPU
drivers. The playbook devshells handle this automatically — container-based
playbooks don't need it, and Nix-native playbooks wrap their commands with
nixglhost
so no manual intervention is required.
For other devshells (e.g.
cuda
),
nixglhost
is available and can be used
to prefix commands manually:
nix develop .
#
cuda
nixglhost deviceQuery
NixOS on the DGX Spark
Warning
Only DGX OS can boot from the factory firmware. You need to
update firmware
before installing NixOS.
USB Boot Image
Build the USB image:
nix build .
#
usb-image
sudo dd if=
$(
echo result/iso/
*
.iso
)
of=/dev/your_usb_disk_device bs=1M status=progress
sync
The image includes two kernel options, selectable from the GRUB boot menu:
NixOS
(default) – NVIDIA's specialised kernel for DGX Spark with full GPU support and working Ethernet
NixOS (standard-kernel)
– Standard NixOS 6.17 kernel (Ethernet has problems)
Booting
Disable Secure Boot in the DGX Spark BIOS and boot from the USB drive.
You can then follow the installation instructions in the NixOS manual:
https://nixos.org/manual/nixos/stable/#sec-installation-manual
Using the DGX Spark module
This module provides configurable DGX Spark hardware support with options for kernel selection.
Module configuration options
hardware
.
dgx-spark
=
{
enable
=
true
;
# Enable DGX Spark hardware support
useNvidiaKernel
=
true
;
# Use NVIDIA kernel (default: true)
}
;
Using NVIDIA kernel
hardware
.
dgx-spark
.
enable
=
true
;
# Uses NVIDIA kernel by default
The NVIDIA kernel is a custom build optimised for NVIDIA DGX Spark systems. The kernel configuration is generated from NVIDIA's Debian annotations and compared with NixOS defaults to produce a minimal, maintainable configuration.
The module also enables the DGX Dashboard web interface at
http://localhost:11000
, providing GPU telemetry and system monitoring.
Using standard NixOS kernel (has some issues with networking)
hardware
.
dgx-spark
=
{
enable
=
true
;
useNvidiaKernel
=
false
;
# Use standard NixOS 6.17 kernel
}
;
Kernel configuration management
The kernel configuration is generated from NVIDIA's Debian annotations and stored in
kernel-configs/nvidia-dgx-spark-.nix
. This terse configuration only contains options that differ from NixOS defaults, reducing verbosity by ~82%.
To regenerate the kernel configuration:
nix run .
#
generate-kernel-config
This:
Fetches the NVIDIA kernel source from GitHub
Builds the NixOS baseline kernel config
Compares NVIDIA’s annotations with NixOS defaults
Generates a terse config file with only the differences
Regeneration is needed when:
NVIDIA kernel version changes (update
kernel-configs/nvidia-kernel-source.nix
)
NixOS common-config changes (nixpkgs update)
You can also use a local kernel source for development:
nix run .
#
generate-kernel-config — –kernel-source /path/to/NV-Kernels
Importing in other projects
Other projects can import this flake and use the DGX Spark module:
{
inputs
.
dgx-spark
.
url
=
“github:graham33/nixos-dgx-spark”
;
outputs
=
{
nixpkgs
,
dgx-spark
,
…
}
:
{
nixosConfigurations
.
mySystem
=
nixpkgs
.
lib
.
nixosSystem
{
modules
=
[
dgx-spark
.
nixosModules
.
dgx-spark
{
# Enable DGX Spark support
hardware
.
dgx-spark
.
enable
=
true
;
# Optionally use standard kernel: useNvidiaKernel = false;
}
# your other modules
]
;
}
;
}
;
}
Quick start NixOS template
For a complete NixOS configuration template specifically designed for DGX Spark
systems, you can use the template:
#
Create a new directory for your NixOS configuration
mkdir my-dgx-spark-config
cd
my-dgx-spark-config
#
Initialise with the DGX Spark template
nix flake init -t github:graham33/nixos-dgx-spark#dgx-spark
This creates a complete NixOS configuration with:
flake.nix
– Flake configuration that imports the DGX Spark module
configuration.nix
– Main system configuration optimised for DGX Spark
hardware-configuration.nix
– Hardware configuration template
Customising the template
After initialising the template, you need to:
Generate hardware configuration and update template:
#
Generate hardware config to a temporary location to get the real UUIDs
sudo nixos-generate-config –root /mnt –dir /tmp/nixos-config
#
Copy the real hardware UUIDs and settings from the generated file
#
Replace the placeholder UUIDs in hardware-configuration.nix with actual
#
values from /tmp/nixos-config/hardware-configuration.nix
Edit
configuration.nix
to customise:
Change hostname from
dgx-spark
to your preferred name
Update username from
nixos
to your preferred username
Add your SSH public keys for remote access
Set your timezone and locale preferences
Add any additional packages you need
Deploy the configuration to /etc/nixos:
#
Copy your configuration to /etc/nixos
sudo cp -r
.
/etc/nixos/
#
Apply the configuration
sudo nixos-rebuild switch –flake /etc/nixos#dgx-spark
Firmware updates
As
noted
in #32, only DGX OS can boot from the factory firmware. If NixOS can’t boot
from the factory firmware, you need to update firmware from DGX OS first.
The module enables
fwupd
for firmware updates. NVIDIA
publishes DGX Spark firmware to the Linux Vendor Firmware Service (LVFS). To
check for available updates:
fwupdmgr get-updates
To install available updates:
fwupdmgr update
Playbooks
This repository includes devshells for NVIDIA DGX Spark playbooks from
https://build.nvidia.com/spark
:
Playbook
Description
Type
Tested on NixOS
Tested on DGX OS
ComfyUI
Run ComfyUI with Stable Diffusion 1.5 for AI image generation
🟢 Full Nix²
✅
✅
Connect Two Sparks
Connect two DGX Spark systems via QSFP
🟢 Full Nix²
✅
☑️¹
DGX Dashboard
Set up DGX Dashboard for system monitoring
🟢 Full Nix²
✅
☑️¹
FLUX.1 Dreambooth
FLUX.1 Dreambooth LoRA fine-tuning
🟠 Container³
✅
✅
Multi-Agent Chatbot
Build and deploy a multi-agent chatbot
🟠 Container³
✅
✅
Multi-modal Inference
Run multi-modal inference with vision-language models
🟠 Container³
✅
✅
NCCL for Two Sparks
Multi-node GPU communication with NCCL
🟢 Full Nix²
✅
☑️¹
NVFP4
FP4 model quantisation with TensorRT Model Optimizer
🟠 Container³
✅
OpenShell
Secure long-running AI agents with OpenShell sandbox
🔵 Nix + Container⁴
✅
PyTorch Fine-tuning Container
Fine-tune models with PyTorch on DGX Spark
🟠 Container³
✅
PyTorch Fine-tuning Nix
Fine-tune models with PyTorch (Nix native, no containers)
🟢 Full Nix²
✅
✅
Speculative Decoding
Speculative decoding for faster inference
🟠 Container³
✅
TRT-LLM
TensorRT-LLM for optimised inference
🟠 Container³
✅
✅
vLLM Container
Run vLLM inference server with Qwen2.5-Math-1.5B-Instruct model
🟠 Container³
✅
✅
vLLM Nix
Run vLLM inference server natively (Nix native, no containers)
🟢 Full Nix²
✅
✅
¹ Pre-installed on DGX OS
²
Full Nix
: Fully reproducible — all dependencies installed via Nix
³
Container
: Nix provides podman, but containers are pulled/built at runtime
⁴
Nix + Container
: CLI tools packaged via Nix, but containers are managed by the tool at runtime
Caching
Flox CUDA binary cache (recommended)
Flox
distributes pre-built CUDA packages for
aarch64-linux
with NVIDIA’s permission
.
This includes cudatoolkit, nccl, cuDNN, PyTorch, and other CUDA
dependencies — dramatically reducing build times.
If you use the DGX Spark NixOS module
, the Flox cache is configured
automatically as a substituter.
For standalone Nix
(e.g. on DGX OS), add to
/etc/nix/nix.conf
:
extra-substituters = https://cache.flox.dev
extra-trusted-public-keys = flox-cache-public-1:7F4OyH7ZCnFhcze3fJdfyXYLQw/aV7GEed86nQ7IsOs=
graham33 Cachix cache
For packages built by this repo (e.g. dgx-dashboard, openshell):
cachix use graham33
Install cachix first if needed:
https://docs.cachix.org/installation
See
https://nixos.wiki/wiki/CUDA
for general CUDA caching details.
nixos-anywhere (Experimental)
Warning
This has not been tested yet. Use at your own risk.
You can install NixOS on a DGX Spark remotely using
nixos-anywhere
. This is
useful for headless setups where you have SSH access to the target machine.
nix run github:nix-community/nixos-anywhere — –flake github:graham33/nixos-dgx-spark#dgx-spark root@
This partitions the NVMe disk and installs NixOS with the DGX Spark module
enabled. You may want to customise
nixos-anywhere/configuration.nix
(e.g. to
add SSH keys or change the hostname) — clone the repo and point
–flake
at
your local checkout instead.
To test the disk configuration in a VM without installing:
nix run github:nix-community/nixos-anywhere — –flake .
#
dgx-spark –vm-test
See the
nixos-anywhere documentation
for full details and requirements.
License
MIT License – see
LICENSE
for details.
About
Use Nix and NixOS on your DGX Spark!
Resources
Readme
MIT license
Activity
Stars
114
stars
Watchers
4
watching
Forks
6
forks
Report repository
Releases
Packages
Contributors
Languages