todoThis document is an example of using YOLOV8s object detection to test model mean average precision(mAP) on C3V.
| Table of Contents | ||
|---|---|---|
|
1. Prepare the environment (on Ubuntu)
| Code Block |
|---|
sudo apt update
sudo apt install python3-pip libopencv-dev libjsoncpp-dev
pip install pycocotools |
2. Cross-compile the sample (on Ubuntu)
First, you need to use the snnf_build.sh script to compile the SNNF-related library files, and then execute the following command：
| Code Block |
|---|
make -C unittest/mapTest/ && make install |
After the command execution is completed, the following log can be seen:
| Code Block |
|---|
---------------------------------------------------------------------------------
---------------------------------------------------------------------------------
total 444
-rwxr-xr-x 1 lory lory 118528 Jan 8 17:10 snnf_map_demo
-rwxr-xr-x 1 lory lory 332096 Jan 8 17:10 snnf_nnsample
---------------------------------------------------------------------------------
install
---------------------------------------------------------------------------------
total 36
drwxr-xr-x 2 lory lory 4096 Jan 8 17:10 bin
drwxr-xr-x 6 lory lory 4096 Jan 8 17:10 include
drwxr-xr-x 3 lory lory 4096 Jan 8 16:54 lib
drwxr-xr-x 7 lory lory 4096 Jan 8 17:10 resource
drwxr-xr-x 4 lory lory 4096 Jan 8 17:10 samples
-rwxr-xr-x 1 lory lory 473 Jan 8 17:10 snnf_build_samples.sh
-rwxr-xr-x 1 lory lory 1618 Jan 8 17:10 snnf_env.sh
-rwxr-xr-x 1 lory lory 430 Jan 8 17:10 snnf_run.sh
drwxr-xr-x 9 lory lory 4096 Jan 8 17:10 thirdparty
--------------------------------------------------------------------------------- |
the snnf_map_demo app was built completely, which is in the following path: release/bin/
The release code tree is like this:
| Code Block |
|---|
(base) lory@lory:~/code/snnf/release$ tree -L 2
.
├── bin
│ ├── snnf_map_demo
│ └── snnf_nnsample
├── include
│ ├── NNLogger.h
│ ├── NNModel.h
| ├── ...
│ └── type
├── lib
│ ├── libnnRoutines.a
│ ├── ...
│ └── plugin
├── resource
│ ├── config
│ ├── ...
│ └── video
├── samples
│ ├── CMakeLists.txt
│ ├── ...
│ └── unittest
├── snnf_build_samples.sh
├── snnf_env.sh
├── snnf_run.sh
└── thirdparty
├── ffmpeg
├── ...
└── pytorch |
3. Inference(on C3V)
Please prepare the coco dataset: 2017 Val images [5K/1GB] https://cocodataset.org/#download
Copy the whole release folder to the C3V, and copy the val2017 dataset to the C3V. Do the map test using the following command:
| Code Block |
|---|
. ./snnf_env.sh
./bin/snnf_map_demo ~/c3v/dataset/val2017 ./detection_results_coco_val2017.json |
snnf_map_demo needs to set two parameters.
param1: the dataset path of coco val2017.
param2: result file name, the detects results by yolov8s object detection.
After the command execution is completed, the following log can be seen:
| Code Block |
|---|
sunplus@ubuntu:~/c3v/app/snnf_release$ ./bin/snnf_map_demo ~/c3v/dataset/val2017 ./detection_results_coco_val2017.json
1736328302307|7f7baa3010|T|common: [app]dataset path:/home/sunplus/c3v/dataset/val2017 save file name:./detection_results_coco_val2017.json
2025-01-08 17:25:02
2025-01-08 17:31:20####################################################][5000 of 5000 | 100.00%][Avg: 75.53ms FPS: 13.24]
Time cost: 377.88s, Average FPS: 13.23 |
4. Evaluation(on Ubuntu)
Download the
| View file | ||
|---|---|---|
|
coco_eval.py
instances_val2017.json
Copy the detection_results_coco_val2017.json to Ubuntu.
Use coco_eval.py to evaluate:
| Code Block |
|---|
./coco_eval.py ./detection_results_coco_val2017.json |
The results are like this:
| Code Block |
|---|
~$python3 coco_eval.py ./detection_results_coco_val2017.json
loading annotations into memory...
Done (t=0.48s)
creating index...
index created!
Loading and preparing results...
DONE (t=3.46s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=44.35s).
Accumulating evaluation results...
DONE (t=10.41s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.439
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.605
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.476
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.246
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.487
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.600
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.350
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.580
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.633
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.429
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.697
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.786 |