Edge Video Infrastructure (EVI) KPI Test User Guide
AI Inference
Deploy EVI first:
# uninstall to make the environment clean cd $pkgdir && python3 $pkgbin --uninstall # clean environment variables unset ${!EVI_@} # and configure source /dev/stdin <<EOF export EVI_aiInference_replicaCount=4 export EVI_aiInference_maxConcurrentWorkloadPerReplica=8 EOF # install python3 $pkgbin --install
Prepare for the KPI test:
mkdir -p kpi-test/ai && cd $_ # prepare 300 images from dataset and resize them for quantization resize-image /home/data/images images 300 # prepare 3 images for kpi test resize-image images out/images-resized 3 # because we can't redistribute yolov5, need to get it from official repo export YOLO_MODEL=yolov5m get-yolov5-for-openvino images # generate pipeline configuration file for test gen-pipeline
The following configurations (thread number) are specific for the target platform Intel® Xeon® Gold 6430 x 2. For other platforms, users can scale thread number according to the following formula:
factor = ("core number of target CPU" * "base freq of target CPU") / ("core number of 6430" * "base freq of 6430")
Start the KPI test and wait for the result:
# the default config for image kpi test is: threads:72, loops:100 # each iteration takes about 2 minutes for i in {1..4}; do kpi-test-ai image | sed -n '/Total time:/,$p'; done # 189.249 # every request has 3 images to process # images per second should be: qps * 3
Need to redeploy EVI for video workloads and perform the KPI test:
cd $pkgdir && python3 $pkgbin --uninstall unset ${!EVI_@} export EVI_aiInference_replicaCount=5 export EVI_aiInference_maxConcurrentWorkloadPerReplica=8 python3 $pkgbin --install cd kpi-test/ai # the default config for video kpi test is: threads:88, loops:10 # each iteration takes about 5 minutes for i in {1..4}; do kpi-test-ai video | sed -n '/Total time:/,$p'; done # fps per stream: 30.2409