NVIDIA Container Toolkit

The NVIDIA Container Toolkit enables GPU-accelerated containers by exposing NVIDIA hardware to the container runtime via a specialized library and CLI.

Engineers use the NVIDIA Container Toolkit to bridge the gap between host GPUs and isolated container environments. It functions by modifying the container runtime (like Docker or containerd) to inject the nvidia-container-cli and necessary drivers into the container at start time. This setup allows frameworks like PyTorch and TensorFlow to access CUDA cores and Tensor Cores without installing drivers inside the image. By standardizing the interface across local workstations and cloud clusters (including AWS and GCP), it ensures that high-performance workloads remain portable and scalable.

https://github.com/NVIDIA/nvidia-container-toolkit