Documentation
Set up and run your GPU server
Step-by-step guides from first SSH login to multi-GPU training: drivers and CUDA, ROCm, Docker, vLLM, NCCL, monitoring, storage and firewalls.
Popular:QuickstartServe an LLM with vLLMNCCL testsDocker with GPUs
Get started
- GPU server quickstart GPU server quickstart for new FullGPU servers. Log in over SSH, check the GPUs, update Ubuntu and run a first GPU container, with a check at each step.
- SSH into a GPU server How to SSH into a GPU server. Create an ed25519 key on macOS, Linux or Windows, connect, add teammates and disable password login. Copy-ready steps.
- VS Code Remote SSH and Jupyter on a GPU server Use VS Code Remote SSH on your GPU server, run Jupyter on a remote GPU through an SSH tunnel and move data with rsync and scp. Safe defaults, step by step.
- Ubuntu GPU image with CUDA preinstalled What our Ubuntu GPU image with CUDA preinstalled contains, how to check driver and toolkit versions, pick 22.04 or 24.04 and keep the driver pinned safely.
GPU software
- Check NVIDIA driver and CUDA version Check NVIDIA driver and CUDA version with nvidia-smi and nvcc, see how they must match and add a CUDA toolkit on Ubuntu 24.04 without a new driver.
- ROCm setup for AMD Instinct MI355X ROCm setup for AMD Instinct MI355X servers. Check GPUs with rocm-smi and amd-smi, set user groups, run PyTorch and vLLM in ROCm containers and test RCCL.
- Docker GPU containers with the NVIDIA Container Toolkit Docker GPU guide for the NVIDIA Container Toolkit. Use --gpus, select devices, reserve GPUs in Compose, fix PyTorch shared memory and run ROCm containers.
- Serve an LLM with vLLM Serve an LLM with vLLM on your GPU server. Set up Python with uv, install PyTorch and vLLM, test the OpenAI-compatible API and run it as a service.
- NCCL tests on a multi-GPU server Run NCCL tests on a multi-GPU server. Check NVLink topology, build nccl-tests, read busbw from all_reduce_perf and launch torchrun jobs across 8 GPUs.
- nvidia-smi monitoring commands nvidia-smi monitoring commands to monitor GPU usage on Linux. Live views, CSV logs, nvtop, DCGM exporter, ECC and Xid errors, plus rocm-smi equivalents.
Server setup
- Mount NVMe drives on Ubuntu Mount NVMe drives on Ubuntu. Find them with lsblk, format and add to fstab by UUID, or build an NVMe RAID 0 with mdadm. Plus health checks and backups.
- GPU server firewall setup GPU server firewall setup with ufw. Allow SSH first, filter IPv6, avoid the Docker port trap and secure a vLLM endpoint with TLS and an API key.
No match. Try another word, or ask an engineer — we reply 24/7.