Module 6 · OS fundamentals for AI systems

Vera Rubin, and reading any new machine

Each NVIDIA generation changes the numbers: link speeds, memory sizes, domain size. The OS model mostly stays the same. This module lines up every machine you have or will meet, from your 4090 to Vera Rubin NVL72, on the same axes: how the GPU reaches CPU memory, who manages GPU memory, how big the NVLink domain is, and how slow the way out is. Then it works through what Vera Rubin's numbers change for offload, MoE and inference serving.

≈35 min reading3 interactive diagramsDrill ≈10 min per machineBuilds on M1–M5

Vera Rubin figures come from NVIDIA's January–March 2026 platform announcements and partner datasheets. NVIDIA marks them preliminary. Nothing here about Rubin has been measured, and Rubin Ultra/Kyber details changed between announcements.

1Six machines, one model

What it is. Four questions place any GPU machine for the OS. (1) How does the GPU reach CPU memory? Through PCIe and pinned buffers, or coherently with ATS. (2) Who manages GPU memory? The driver alone, or the kernel too (NUMA mode). (3) How big is the NVLink domain? That's the set of GPUs that can load and store into each other's memory. (4) What is the way out? The NIC bandwidth per GPU, and whether it can DMA to GPU memory. Step through the machines below and watch which answers change.

Fig 1 · The same four questions, generation by generation interactive

Vendor peak bandwidths. "Per GPU" figures divide shared links (a GB200 Grace's C2C serves two GPUs). Dashed CPU–GPU link = not coherent (PCIe).

Why it matters for AI workloads: the answers decide which tricks are available. Zero-copy reads of CPU memory and cheap offload need (1) = coherent. CPU data spilling into HBM and HBM counted in cgroups need (2) = kernel. A TP/EP degree above 8 at NVLink speed needs (3) ≥ 72. Gradient sync cost is set by (4). When a new system ships, answer these four first. The drill does it with one command.

2Side by side

	RTX 4090 box	DGX Spark	HGX H100	GB200 NVL72	GB300 NVL72	Vera Rubin NVL72
CPU	x86 (i9-13900K)	GB10 Arm, 10 X925 + 10 A725	2 × x86 (e.g. Xeon 8480C)	Grace, 72 × Neoverse V2	Grace	Vera, 88 Olympus cores / 176 threads
CPU memory	64 GB DDR5	128 GB LPDDR5x, shared with GPU, 273 GB/s	up to 2 TB DDR5	≤480 GB LPDDR5X, ≤512 GB/s per Grace	same	≤1.5 TB LPDDR5X (SOCAMM), ≤1.2 TB/s per Vera
CPU↔GPU	PCIe Gen4 x16	on-package C2C; one physical memory	PCIe Gen5 x16	NVLink-C2C 900 GB/s bidir, Grace ↔ 2 GPUs	same	NVLink-C2C 1.8 TB/s, Vera ↔ 2 GPUs
Coherent / ATS	no	yes (integrated)	no	yes	yes	yes
GPU memory	24 GB GDDR6X, ~1 TB/s	(the same 128 GB)	80 GB HBM3, 3.35 TB/s	186 GB HBM3e, ~8 TB/s	288 GB HBM3e (20 TB per rack), ~8 TB/s	288 GB HBM4, ~22 TB/s
NVLink domain	none	none	8 GPUs, 900 GB/s each (NVLink 4)	72 GPUs, 1.8 TB/s each (NVLink 5)	72, 1.8 TB/s	72, 3.6 TB/s each (NVLink 6); 260 TB/s rack
Scale-out per GPU	—	ConnectX-7, 200 Gb/s per box	ConnectX-7 400 Gb/s	ConnectX-7 400 Gb/s	ConnectX-8 800 Gb/s	ConnectX-9 1.6 Tb/s
NVLink ÷ NIC (per GPU)	—	—	18×	36×	18×	18×
GPUDirect RDMA	no (GeForce)	no (use cudaHostAlloc)	yes	yes	yes	expected yes
OS hosts per domain	1	1	1	18	18	18 (expected)

The NVLink ÷ NIC row uses bidirectional NVLink against unidirectional NIC line rate, i.e. 1.8 TB/s ÷ 50 GB/s = 36 for GB200. That's the ratio that sets how much it's worth keeping traffic inside the domain. (Module 5 §7 used per-direction NVLink, 18×. Pick one convention and stick to it.)

GB300: NVIDIA's rack page lists 17 TB LPDDR5X and 2,592 Neoverse V2 cores, the same Grace configuration as GB200, so C2C is assumed unchanged. Its 20 TB of rack HBM works out to ≈278 GB per GPU against the 288 GB usually quoted. Vera Rubin rows are NVIDIA's preliminary platform figures. I haven't found a public statement of whether Vera exposes HBM4 as NUMA nodes in the same way, or whether CDMM is the default there. The R615 / CUDA 13.4 change to CDMM-by-default suggests it will be.

3Vera: 88 cores, 176 threads, "spatial" multithreading

What it is. Vera is NVIDIA's first custom Arm core design (Olympus, Armv9.2), replacing Grace's licensed Neoverse V2. Each Vera has 88 cores, 2 MB L2 per core, a large shared L3, up to 1.5 TB of LPDDR5X on replaceable SOCAMM modules at up to 1.2 TB/s, PCIe Gen6 and CXL 3.1. The OS-relevant novelty is Spatial Multithreading: two hardware threads per core, so Linux sees 176 CPUs. NVIDIA describes it as physically partitioning core resources between the two threads rather than time-slicing them, the way x86 Hyper-Threading does.

Why it matters for AI workloads: Module 3 measured what an x86 SMT sibling does to a pinned launch thread: 5.4 ms → 8.9 ms per step when the sibling was busy. Grace had no SMT, so "pin the launch thread to a core" was the whole recipe. On Vera, every core has a sibling again, and the question becomes how much partitioning protects the thread you care about. Until that's measured, use the x86 recipe: leave the launch thread's sibling idle (isolate the core, not just the CPU), and give data workers whole cores.

Where this shows up in your stack Topology-aware binding: lscpu -e and /sys/devices/system/cpu/cpu*/topology/thread_siblings_list tell you which logical CPUs share a core, the same as on x86. Expect torchrun's default OMP_NUM_THREADS=1 and NUMA binding (--numa-binding in recent PyTorch, or numactl) to matter even more with 176 logical CPUs per socket. drills/m3_cpu.sh jitter is the experiment to run on a Vera node: pinned with the sibling idle vs pinned with the sibling noisy.

How much Spatial Multithreading isolates siblings in practice (caches, load/store queues, memory bandwidth) isn't documented beyond NVIDIA's description. The M3 jitter drill on real hardware is the test.

4NVLink-C2C at 1.8 TB/s: when does offload pay?

What it is. A Vera Rubin superchip is 1 Vera + 2 Rubin GPUs: 1.5 TB of LPDDR5X plus 2 × 288 GB of HBM4. NVIDIA markets that as "2 TB of fast memory" because C2C makes the LPDDR coherent and GPU-addressable at 1.8 TB/s. Offload means keeping optimizer states, KV cache or cold weights in CPU memory and moving them over that link. It is limited by the link and by CPU memory bandwidth, which both GPUs on the superchip share.

Fig 2 · Offload cost per training step, by machine interactive

Scenario Params per GPU (billions) Step time without offload (s)

Per-GPU bandwidth: link per direction ÷ GPUs sharing it, and CPU memory ÷ GPUs sharing it. ZeRO-Offload: 2 B/param of bf16 gradients down, 2 B/param of bf16 params up, ~30 B/param of CPU DRAM traffic for Adam (read p, m, v, g; write p, m, v, bf16 p). Streamed: 12 B/param each way (fp32 p, m, v), 24 B/param of DRAM traffic. Bandwidth only: no overlap with compute, no CPU FLOP limit. Spark: the "link" is its shared LPDDR5x.

Why it matters for AI workloads: on PCIe boxes, offload adds seconds per step and only makes sense when the model doesn't fit at all. With coherent C2C, streamed optimizer states for a 10B-parameter shard cost a fraction of a second, and they can overlap with backward. The limit moves from the link to CPU memory bandwidth per GPU, and that's the number Vera roughly doubled.

Where this shows up in your stack FSDP2's CPUOffloadPolicy and DeepSpeed ZeRO-Offload implement the CPU-side scenario. Inference engines' KV-cache offload (CPU swap space, LMCache-style tiers) implements the third. On coherent machines, check whether these libraries use pinned cudaHostAlloc buffers and copy engines (they work everywhere), or can take advantage of direct SM access to LPDDR (coherent only). Also check where the CPU-side buffers land. In NUMA mode an unbound offload buffer can end up in HBM (M5 §3), which defeats the purpose.

5NVLink 6 and MoE all-to-all

What it is. NVLink 6 doubles per-GPU NVLink bandwidth to 3.6 TB/s and keeps the 72-GPU domain. A Vera Rubin switch tray has four NVLink 6 switch chips (28.8 TB/s) with in-network compute for SHARP reductions, and the rack totals 260 TB/s. NVIDIA's headline claim is up to 2× all-to-all throughput for MoE. All-to-all is the collective where domain size matters most: with uniform routing, a token's expert is outside your NVLink domain with probability (EP − d)/EP.

Fig 3 · MoE dispatch + combine time per layer interactive

Expert-parallel size Tokens per GPU Hidden size Top-k · dtype

Uniform routing, so each GPU sends tokens·k·hidden·bytes·(EP−1)/EP per direction, and the same again for combine. The share going to GPUs inside the NVLink domain uses NVLink (per-direction peak); the rest uses the GPU's NIC; the two run in parallel. Combine is assumed BF16 when dispatch is FP8. Ignores latency, load imbalance and NVLink-to-NIC forwarding. Use it for ratios.

Why it matters for AI workloads: on HGX H100, EP = 64 sends 7/8 of the dispatch traffic over 50 GB/s NICs, so all-to-all dominates the layer. On NVL72 the same EP stays on NVLink. On Vera Rubin, NVLink doubles again, so for EP ≤ 72 the all-to-all time halves. Beyond 72, the NIC term is back, and ConnectX-9's 4× over ConnectX-7 is what keeps it in check.

Where this shows up in your stack MoE dispatch kernels (DeepEP-style, or symmetric-memory all-to-all in PyTorch) use NVLink load/store inside the domain and RDMA outside it. Those are exactly the M5 §6 fabric-handle path and the M4 §5 registered-memory path. Set the EP group to the NVLink domain: on NVL72, ranks in the same rack and the same clique.

6NICs, DPUs and the KV-cache memory tiers

What it is. Vera Rubin pairs each GPU with 1.6 Tb/s of ConnectX-9 (800 Gb/s ports) and adds BlueField-4: a DPU that packages a 64-core Grace with a ConnectX-9, 128 GB of its own memory, NVMe-oF offload (NVIDIA claims 20 M 4K IOPS) and inline crypto. NVIDIA positions it as the host for an inference context memory storage tier: KV cache that has fallen out of GPU and CPU memory but is cheaper to fetch than to recompute.

Fig 4 · Where a KV-cache block can live on Vera Rubin (per GPU, preliminary peaks)

Bars are capacity per GPU on a log scale. The right-hand column is the bandwidth to reach that tier from the GPU. Moving down a tier: ~10× more capacity, ~10× less bandwidth.

Why it matters for AI workloads: long-context, multi-turn inference is bounded by KV-cache capacity. Every tier below HBM is an OS-managed resource from earlier modules. LPDDR via C2C is NUMA placement and pinning (M5 §3–4). The DPU tier is RDMA, registration and queue pairs (M4 §4–5), with the storage stack running on the DPU's own Arm cores instead of the host's. Recompute versus fetch is a bandwidth comparison you can now make with the numbers above.

Where this shows up in your stack NIXL / LMCache / vLLM KV connectors are the software that moves blocks between these tiers. The OS checks are the same as M4: memlock and registration limits for RDMA buffers, /dev/infiniband in containers, and NUMA-local pinned buffers.

BlueField-4 and the context-memory storage tier are described in NVIDIA's platform blog. How applications address that tier (block, object or KV API) wasn't specified in what I could find.

7Rubin CPX and disaggregated serving

What it is. Rubin CPX is a separate GPU for the prefill (context) phase of inference: up to 30 PFLOPS NVFP4 with 128 GB of GDDR7 instead of HBM, and attention acceleration. Prefill is compute-bound, decode is memory-bandwidth-bound, so NVIDIA splits them: a Vera Rubin NVL144 CPX rack has 144 CPX GPUs, 144 Rubin GPUs and 36 Vera CPUs. CPX builds the KV cache, and Rubin GPUs with HBM4 decode from it.

Why it matters for AI workloads: disaggregation turns the KV cache into data that moves between processes, GPUs and possibly hosts on every request. That's the M5 §6 problem (sharing GPU memory across processes and kernels) and the M4 §5 problem (registered memory for RDMA), on the critical path of time-to-first-token.

How CPX GPUs attach to the rack (NVLink domain membership, PCIe, NIC) and therefore which path KV transfers take wasn't in NVIDIA's public material I found. Also note the naming: NVIDIA's 2025 roadmap called the base Vera Rubin rack "NVL144" (counting GPU dies), while its 2026 platform pages call it "NVL72" (counting packages). The CPX rack kept "NVL144". Check which count a spec sheet uses before comparing.

8Day-one checklist for any new machine

The first hour on unfamiliar hardware, in order. Each line is a module you've done.

Check	Command	Module
CPU, cores, SMT siblings, page size	`lscpu`, `getconf PAGESIZE`	M1 §2, M2 §2, M3 §6
NUMA nodes, which have CPUs, which hold GPU memory	`numactl -H`, `bash drills/m5_rack.sh numa`	M1 §3, M5 §3
GPU ↔ CPU ↔ NIC topology	`nvidia-smi topo -m`	M1 §4–5
Coherence class and fabric support	`bash drills/m5_rack.sh bw` (section 1)	M5 §4, §6
Coherent memory mode	`grep Coherent /proc/driver/nvidia/params`	M5 §3
Kernel memory policy (THP, init_on_alloc, NUMA balancing)	`bash drills/m5_rack.sh tune`	M2 §5, M5 §5
Host↔GPU copy bandwidth, pinned vs pageable	`python3 drills/m1_h2d_bandwidth.py`	M1 §6
Launch jitter with and without SMT-sibling noise	`bash drills/m3_cpu.sh jitter`	M3 §6
RDMA devices, GDR, memlock, container limits	`ibv_devinfo`, `sh drills/m4_container_check.sh`	M4 §5–7
Fabric, IMEX, NCCL transport	`bash drills/m5_rack.sh fabric`, `NCCL_DEBUG=INFO`	M5 §6–7

Where this shows up in your stack bash drills/report.sh runs all the read-only checks and writes results/<hostname>.md, so every machine you have (4090 box, Spark, an 8×H100 node, a GB200 tray) gets the same one-page profile. The repo's CLAUDE.md tracks which machines still need a report and which "expected" numbers in these pages they should replace.

DDrill: profile every machine the same way

bash drills/report.sh            # read-only; writes results/$(hostname).md
bash drills/report.sh --quick    # skip the bandwidth/jitter runs (≈30 s)

Then compare the reports across machines against the table in §2. The four questions from §1 should be answerable from the first screen of each report.

Expected output

x86 + RTX 4090 measured on your box

## Four questions
1. PCIe, not coherent: pinned/managed
   memory only
2. driver only (discrete GPU)
3. none (1 GPU(s), no NVLink)
4. RDMA devices: none; GDR attribute: 0
## Headline numbers
H2D pageable / pinned  16.1 / 20.2 GB/s
SM read GPU memory     941.8 GB/s
SM read zero-copy      12.8 GB/s
plain malloc           n/a
fabric handle: operation not permitted

Vera Rubin NVL72 tray expected, not yet measured

## Four questions
1. coherent link with ATS (GPU walks
   CPU page tables)
2. driver only (CDMM), or kernel + driver
   (NUMA mode)
3. multi-node NVLink fabric: registered,
   clique N (4 local GPUs; links NV18?)
4. RDMA devices: mlx5_*; GDR attribute: 1
## Headline numbers
H2D pinned: C2C-bound, above GB200
SM read of plain malloc: hundreds of GB/s+
fabric handle create: OK (with channel)
nproc per tray: 2 × 176 = 352?

Vera Rubin values are predictions. Also unknown: how many Vera sockets each OS instance spans (GB200 is 2 per tray). NVIDIA's VR NVL72 compute tray holds 2 superchips, so 2 × 176 logical CPUs is a guess.

?Self-check

1. Name the four questions that place a GPU machine for the OS, and answer them for HGX H100 and GB200 NVL72.

How the GPU reaches CPU memory, who manages GPU memory, NVLink domain size, and the way out (NIC per GPU, GDR). HGX H100: PCIe Gen5, not coherent, so pinned buffers and copy engines; the driver only; 8 GPUs at 900 GB/s; ConnectX-7 400G per GPU with GPUDirect RDMA. GB200: coherent NVLink-C2C with ATS; the kernel (NUMA mode) or the driver (CDMM); 72 GPUs across 18 OS instances, IMEX required; ConnectX-7 400G per GPU with GDR.

2. Why might the launch-thread pinning recipe that worked on Grace need changing on Vera?

Grace has one hardware thread per core, so pinning a thread to a CPU gave it a whole core. Vera has two threads per core (176 logical CPUs). Pinning to a CPU now leaves a sibling that the scheduler can fill, and M3 showed a busy x86 sibling raised step time from 5.4 to 8.9 ms. NVIDIA says Vera partitions resources between siblings rather than time-slicing, which may reduce interference, but until it's measured, isolate the whole core: keep the sibling idle or give it only quiet work.

3. Use Fig 2: why is CPU-memory bandwidth, not the C2C link, often the offload bottleneck on GB200?

CPU-side Adam moves only ~4 B/param over the link but ~30 B/param through CPU DRAM. A Grace has ≤512 GB/s of LPDDR5X shared by two GPUs (~256 GB/s each), while the link gives each GPU ~225 GB/s per direction. 30 B/param of DRAM traffic over 256 GB/s outweighs 2 B/param each way over 225 GB/s by about 13×. Vera's 1.2 TB/s of LPDDR5X raises the per-GPU DRAM share to ~600 GB/s, and that's the bigger change for offload.

4. With uniform routing and EP = 144 on Vera Rubin NVL72, what fraction of dispatch traffic leaves the NVLink domain, and what limits it?

The domain holds 72 GPUs, so for each GPU, 71 of the 143 other EP ranks are inside it, and (144 − 72)/144 ≈ half the destinations are outside. Roughly half the bytes go over the NIC at 200 GB/s (ConnectX-9 1.6 Tb/s) while the other half go over NVLink at 1.8 TB/s per direction. The NIC half takes ~9× longer, so it sets the all-to-all time. That's why EP is usually capped at the domain size, or traffic is shaped hierarchically.

5. A spec sheet says "NVL144" and another says "NVL72" for Vera Rubin. How do you know whether they describe the same rack?

Check what's being counted. NVIDIA's 2025 roadmap counted GPU dies (each Rubin package has 2 compute dies, so 72 packages = "NVL144"), while its 2026 platform materials count packages ("NVL72"). Compare GPU package count, superchips per tray, HBM per GPU (288 GB) and rack totals (e.g. 20.7 TB of HBM4 for 72 packages). The CPX rack is different hardware: 144 Rubin + 144 CPX.

Saved in this browser and shown on the index page.

§Sources

NVIDIA developer blog, Inside the NVIDIA Rubin Platform: Six New Chips, One AI Supercomputer (Vera: 88 Olympus cores, Spatial Multithreading, 176 threads, ≤1.5 TB LPDDR5X at ≤1.2 TB/s, NVLink-C2C 1.8 TB/s, PCIe Gen6/CXL 3.1; Rubin: 288 GB HBM4 at 22 TB/s; NVLink 6 at 3.6 TB/s and 260 TB/s per rack; switch tray 28.8 TB/s; ConnectX-9; BlueField-4; superchip and compute-tray composition).
NVIDIA, DGX Vera Rubin NVL72 (20.7 TB HBM4, preliminary figures); Lenovo Press, Vera Rubin NVL72 datasheet (18 compute trays, 9 NVLink switch trays).
NVIDIA, GB200 NVL72 and GB300 NVL72 product pages.
NVIDIA newsroom (via press-release reprint), Rubin CPX: 30 PFLOPS NVFP4, 128 GB GDDR7, Vera Rubin NVL144 CPX rack composition.
Rubin Ultra / Kyber: DCD on the GTC 2025 roadmap (2H 2027, NVL576). Secondary reporting differs on later memory and rack details, so treat them as unsettled.
NVIDIA, DGX H100 (2 × Xeon 8480C, 8 × ConnectX-7) and H100 datasheet (80 GB HBM3, 3.35 TB/s, NVLink 4 900 GB/s).
Modules 1–5 of this course for the mechanisms. Your measurements: the 4090 numbers come from drills/m1_h2d_bandwidth.py, drills/m3_launch_jitter.py and drills/m5_coherent.cu.