Edge Video Infrastructure (EVI) KPI Test User Guide

ID Date Version Classification
826104 06/28/2024 1.0 Public

Clustering

Deploy it first:

# uninstall first
cd $pkgdir && python3 $pkgbin --uninstall
# clean environment
unset ${!EVI_@}
# install again
python3 $pkgbin --install

Prepare UA-DETRAC dataset:

Download about 10G data, requiring a total of about 30G storage space.

# create folder
mkdir -p kpi-test/clustering && cd $_
# download dataset and extract
for f in test-data train-data Test-Annotations-XML Train-Annotations-XML; do
  wget -c https://detrac-db.rit.albany.edu/Data/DETRAC-$f.zip
done
(mkdir -p UA-DETRAC && cd $_ && unzip '../*.zip')

Extract objects (cropped images) from UA-DETRAC dataset:

# install dependencies first
sudo apt install -y imagemagick libimage-magick-perl libxml-parser-perl libparallel-forkmanager-perl cpanminus
sudo cpanm XML::Reader

# execute and wait for about 2 minutes
extract-object -o out/object-images UA-DETRAC
# there will be 1274055 images
ls out/object-images | wc -l

Generate feature vectors:

# create pipeline configuration file for feature vector extraction
gen-pipeline out feature

# start the task, it will take about 3 hours, be patient
rm -rf result && extract-fv

# convert the result format
convert-csv result/*/results.csv > features.csv

Import feature vectors in CSV format to HBase:

import-fv features.csv

Initialize database:

kubectl -n smartedge-apps exec -i deploy/evi-test-tool -- python3 insert_mock_data_to_psql.py

Start the test and check the result from the output:

CAUTION: wait extra time on each iteration to ensure data is written to the database

# each iteration takes about 5 minutes
for i in {1..4}; do kpi-test-clustering; done
# 2024-01-19_04:35:32,2024-01-19_04:35:37,1262089,00h 00m 4.71s,965290243,128,0.6,235,300