Your AI infrastructure team is observing out-of-memory (OOM) errors during the execution of large deep learning models on NVIDIA GPUs. To prevent these errors and optimize model performance, which GPU monitoring metric is most critical?

GPU Memory Usage is the most critical metric to monitor to prevent out-of-memory (OOM) errors

and optimize performance for large deep learning models on NVIDIA GPUs. OOM errors occur when

a model’s memory requirements (e.g., weights, activations) exceed the GPU’s available memory

(e.g., 40GB on A100). Monitoring memory usage with tools like NVIDIA DCGM helps identify when

limits are approached, enabling adjustments like reducing batch size or enabling mixed precision, as

emphasized in NVIDIA’s "DCGM User Guide" and "AI Infrastructure and Operations Fundamentals."

Core utilization (B) tracks compute load, not memory. Power usage (C) relates to efficiency, not

OOM. PCIe bandwidth (D) affects data transfer, not memory capacity. Memory usage is NVIDIA’s key

metric for OOM prevention.

Reference:NVIDIA DCGM User Guide, AI Infrastructure and Operations Fundamentals

(www.nvidia.com).