Skip to content

L6 · Transport#

conceptstub

What it is#

The mechanism that moves bytes between machines with minimal CPU involvement: RDMA verbs, libfabric providers, and GPUDirect RDMA, which lets the network card read and write GPU memory directly.

Why it matters#

Kernel bypass and zero-copy transfers are what make inter-node collectives fast enough for distributed training. This layer spans the boundary between the container (user-space libraries) and the host (kernel drivers), so version mismatches here are a classic failure point.

In this module#

Moving bytes between GPUs across nodes without the kernel in the path.

Planned content

  • User space (libibverbs, libfabric, NCCL net plugins) vs kernel (ib_core, mlx5_ib, EFA)
  • Image/host version alignment
  • GPUDirect RDMA and peer memory