Reference

Curated tables and exhaustive lists that don’t fit a narrative but are useful: every config preset, every proven parallelism combination at each GPU count, every env var the framework reads.

• Available configs
• Parallelism recipes
• Benchmarks
• Environment variables

• Available configs — the full configs/train/*.toml and configs/model/*.toml tables, with “what this config exists to prove” per row.

• Parallelism recipes — (model, GPU count, parallelism) combinations that we’ve actually run end-to-end, indexed by model rather than by filename.

• Benchmarks — summaries and reproduction commands for the measured campaigns: dense 7B/13B/70B MFU scaling on 1–32 GPUs, weak scaling of 13B/70B out to 160 GPUs, a 175B dense run on 360 GPUs at 47.7% MFU, and MoE Expert Parallelism with per-sub-module FSDP wrapping. The full index lives in benchmarks/README.md.

• Environment variables — every env var the framework reads, grouped by source (torchrun / SLURM / NCCL / logging) with who-sets-what.

See also

• README § Training Configurations — the current configs table at the repo root.

• README § Benchmarks — the MFU table at the repo root.