PyTorch Distributed

conceptstub

torchrun, process groups, DDP/FSDP, and elastic launch.

Planned content

• torchrun rendezvous on Kubernetes
• Process groups and backend selection
• Kubeflow Trainer / JobSet launch patterns