Edge Developer Toolbox Developer Guide

ID 783775
Date 06/07/2024
Version 24.05
Confidential
Document Table of Contents

Metrics and Descriptions

Metrics

Metric

Description

CPU Usage (%/time)

Percent usage of CPU processing power for workload over time

CPU Usage distribution

CPU usage distribution of workload (histogram) - Showcases the CPU usage sample count in a specific bucket. It is the usage values (x-axis) over the number of samples collected during the workload (y -axis)

Avg CPU Usage (%)

Average Percent usage of CPU processing power for workload

Max CPU Usage (%)

Max Percent usage of CPU processing power for workload

Memory Availability (GB)/ Time

GB of RAM that is available on the chosen hardware for workload over time

Memory Availability Distribution

Memory distribution of workload (histogram) - Showcases the Memory Availability sample count in a specific bucket. It is the memory values (x-axis) over the number of samples collected during the workload (y -axis)

Avg Mem Available (GB)

Average GB of RAM that is available on the chosen hardware for workload

Max Mem Available (GB)

Max GB of RAM that is available on the chosen hardware for workload

IGPU-Specific Metrics

IGPU-Specific Metrics

Metric

Description

GPU Render Engine Usage (%)/ Time

Percent utilization of all render engines of GPU for workload over time

GPU Render Engine Usage Distribution

GPU render engine usage distribution of workload (histogram) - Showcases the GPU render engine usage sample count in a specific bucket. It is the usage values (x-axis) over the number of samples collected during the workload (y -axis)

Avg GPU Renderer Usage (%)

Average percent utilization of all render engines of GPU

Max GPU Renderer Usage (%)

Max percent utilization of all render engines of GPU

GPU Video Engine Usage (%)/Time

Percent utilization of video engines of GPU for workload over time

GPU Video Engine Usage Distribution

GPU video engine usage distribution of workload (histogram) - Showcases the GPU video engine usage sample count in a specific bucket. It is the usage values (x-axis) over the number of samples collected during the workload (y -axis)

Avg GPU Video Usage (%)

Average percent utilization of video engines of GPU

Max GPU Video Usage (%)

Max percent utilization of video engines of GPU

GPU Video Enhance Engine Usage (%)/Time

Percent utilization of video enhance engine of GPU for workload over time

GPU Video Enhance Engine Usage Distribution

GPU video enhance engine usage distribution of workload (histogram) - Showcases the GPU video enhance engine usage sample count in a specific bucket. It is the usage values (x-axis) over the number of samples collected during the workload (y -axis)

Avg GPU Video Enhance Usage (%)

Average percent utilization of video enhance engine of GPU

Max GPU Video Enhance Usage (%)

Max percent utilization of video enhance engine of GPU

DGPU-Specific Metrics

DGPU-Specific Metrics

Metric

Description

GPU Temperature (°C)

GPU temperature in degree Celsius, per tile

GPU Temperature Distribution

GPU temperature distribution of workload (histogram) - Showcases the GPU temperature sample count in a specific bucket. It is the temperature values (x-axis) over the number of samples collected during the workload (y -axis)

Avg GPU Temperature (°C)

Average GPU temperature in degree Celcius, per tile

Max GPU Temperature (°C)

Max GPU temperature in degree Celcius, per tile

GPU Power (W)

GPU power in watts, per GPU and per card

GPU Power Distribution

GPU power distribution of workload (histogram) - Showcases the GPU power sample count in a specific bucket. It is the power values (x-axis) over the number of samples collected during the workload (y -axis)

Avg GPU Power (W)

Average GPU power in watts, per GPU and per card

Max GPU Power (W)

Max GPU power in watts, per GPU and per card

GPU Memory Used (MB)/Time

Used GPU memory in bytes, per GPU tile for workload over time

GPU Memory Used Distribution

GPU memory usage distribution of workload (histogram) - Showcases the GPU memory usage sample count in a specific bucket. It is the usage values (x-axis) over the number of samples collected during the workload (y -axis)

Avg GPU Memory Used (MB)

Average Used GPU memory in bytes, per GPU tile

Max GPU Memory Used (MB)

Max Used GPU memory in bytes, per GPU tile

GPU Compute Engine Utilization (%)/Time

Percent utilization of all compute engines, per GPU tile for workload over time

GPU Compute Engine Utilization Distribution

GPU compute engine usage distribution of workload (histogram) - Showcases the GPU compute engine usage sample count in a specific bucket. It is the utilization values (x-axis) over the number of samples collected during the workload (y -axis)

Avg GPU Compute Engine Utilization (%)

Average percent utilization of all compute engines, per GPU tile

Max GPU Compute Engine Utilization (%)

Max percent utilization of all compute engines, per GPU tile

GPU Engine Group Utilization (%)/Time

Percent utilization of all engine groups present on device (i.e.: decoder and encoder), per GPU tile for workload over time

Avg GPU Engine Group Utilization (%)

Average percent utilization of all engine groups present on device (i.e.: decoder and encoder), per GPU tile

GPU Engine Group Utilization Distribution

GPU engine groups utilization distribution of workload (histogram) - Showcases all GPU engine groups usage sample count in a specific bucket. It is the utilization values (x-axis) over the number of samples collected during the workload (y -axis)

Max GPU Engine Group Utilization (%)

Max percent utilization of all engine groups present on device (i.e.: decoder and encoder), per GPU tile

Tokens Per Second (TPS)-Specific Metrics

“Tokens per second” measures the rate of processing or generating tokens. This metric is crucial for evaluating and benchmarking the efficiency and processing speed of LLMs, with higher values indicating faster and more efficient performance influenced by factors such as model complexity, hardware, tokenization strategy, and input data size.

Tokens Per Second (TPS)-Specific Metrics

Metric

Description

Avg Tokens/Sec/Prompt

Average tokens per second per prompt entered into the Q&A Assistant

Tokens/Sec Distribution

Tokens per Second Distribution: TPS distribution of workload (histogram) - Showcases the token sample count in a specific bucket. It is the token values (x-axis) over the number of samples collected during the workload (y -axis)

Avg Tokens/Sec

The average tokens per second for the entire run of the workload

Max Tokens/Sec

The max tokens per second for the entire run of the workload