They must keep an eye on the GPU performance indicators in order to comprehend the nature of their workload and streamline the ML development process We’re happy to report that Ops Agent now gathers metrics from NVIDIA GPUs on Compute Engine VMs to assist

The telemetry tool for Compute Engine that Google recommends for monitoring VM instances is called Cloud Ops Agent You can now get better insight into your NVIDIA GPUs and accelerated workloads thanks to key metrics from the NVIDIA Management Library (NVML) and sophisticated profiling data from the NVIDIA Data Center GPU Manager

Plan scaling by observing patterns to determine when to increase GPU capacity or modernize current GPUs

Change the collecting interval of your NVML and DCGM metrics using metrics processors, or filter out any unnecessary metrics. You can filter and only retain the metrics you need using metrics processors with NVML and DCGM metrics, and you can quickly adjust the collection interval of those metrics through the configuration file

Fill use our Metrics Explorer query builder or PromQL Utilizing GPU data gathered from both GKE GPU nodes and Compute Engine GPU VMs, our NVIDIA GPU Monitoring dashboard offers a single point of access to your entire GPU fleet in some text

Gather data from your workloads using the Prometheus and OpenTelemetry Protocol (OTLP) protocols

Automatically gather system logs from Windows and Linux virtual machines, such as syslog

Automatically gather host metrics like those for the CPU, RAM, and processes