Edge Video Infrastructure (EVI) KPI Test User Guide

ID Date Version Classification
826104 06/28/2024 1.0 Public

AI Inference

Deploy EVI first:

# uninstall to make the environment clean
cd $pkgdir && python3 $pkgbin --uninstall

# clean environment variables
unset ${!EVI_@}
# and configure
source /dev/stdin <<EOF
export EVI_aiInference_replicaCount=4
export EVI_aiInference_maxConcurrentWorkloadPerReplica=8
EOF

# install
python3 $pkgbin --install

Prepare for the KPI test:

mkdir -p kpi-test/ai && cd $_

# prepare 300 images from dataset and resize them for quantization
resize-image /home/data/images images 300
# prepare 3 images for kpi test
resize-image images out/images-resized 3

# because we can't redistribute yolov5, need to get it from official repo
export YOLO_MODEL=yolov5m
get-yolov5-for-openvino images

# generate pipeline configuration file for test
gen-pipeline

The following configurations (thread number) are specific for the target platform Intel® Xeon® Gold 6430 x 2. For other platforms, users can scale thread number according to the following formula:

factor = ("core number of target CPU" * "base freq of target CPU") / ("core number of 6430" * "base freq of 6430")

Start the KPI test and wait for the result:

# the default config for image kpi test is: threads:72, loops:100
# each iteration takes about 2 minutes
for i in {1..4}; do kpi-test-ai image | sed -n '/Total time:/,$p'; done
# 189.249
# every request has 3 images to process
# images per second should be: qps * 3

Need to redeploy EVI for video workloads and perform the KPI test:

cd $pkgdir && python3 $pkgbin --uninstall
unset ${!EVI_@}
export EVI_aiInference_replicaCount=5
export EVI_aiInference_maxConcurrentWorkloadPerReplica=8
python3 $pkgbin --install

cd kpi-test/ai
# the default config for video kpi test is: threads:88, loops:10
# each iteration takes about 5 minutes
for i in {1..4}; do kpi-test-ai video  | sed -n '/Total time:/,$p'; done
# fps per stream: 30.2409