NVML

NVIDIA Management Library (NVML): The C-based API for comprehensive monitoring and control of NVIDIA Data Center GPU hardware.

NVML (NVIDIA Management Library) is your direct programmatic interface for managing NVIDIA GPUs (e.g., A100, H100). This thread-safe, C-based API is the engine behind the `nvidia-smi` utility. Use it to query critical operational data: GPU utilization, power draw, temperature, and ECC error counts. Beyond monitoring, you gain control: manage ECC mode, set compute mode, and configure driver persistence. Deploy NVML for building robust, third-party management tools on Linux and Windows platforms, ensuring peak performance and system reliability.

https://docs.nvidia.com/deploy/nvml/index.html