This article demonstrates how to use VScode (Visual Studio Code) for NN model conversion in an Ubuntu PC and execution on a C3V Linux. The Ubuntu PC can manipulate the C3V Linux directly by VScode through internet. It is very convenient for the verification of NN model on a C3V Linux.

Table of Contents

1 Network Settings
2 Docker and Docker Compose Installation
3 VScode Download and Installation
4 VScode Configurations
- 4.1 Docker Image
- 4.2 devcontainer.json and docker-compose.yml
5 Activate the container in VScode
6 Remote - SSH (connect to the C3V Linux)
7 Samba Deployments
8 NN Model Conversion
- 8.1 import
- 8.2 quantize
- 8.3 inference
- 8.4 export
- 8.5 program compilation
9 Run on the C3V Linux

Network Settings

Prepare an Ubuntu PC and a C3V Linux. Configure them in the same LAN (Local Area Network) and connect both of them to the internet.

Docker and Docker Compose Installation

Download and install the updates for each outdated packages and dependency in the Ubuntu PC.

sudo apt update
sudo apt install gnupg lsb-release apt-transport-https ca-certificates curl gnupg-agent software-properti
es-common

Download GNU Privacy Guard Key of Docker.

sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

Add the repository to Apt sources.

echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

Install Docker and Docker Compose

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin

VScode Download and Installation

Go to https://code.visualstudio.com and click .deb to download .deb in the Ubuntu PC.

Then code_1.97.0-1738713410_amd64.deb downloaded will be used to install VScode with the command shown below.

sudo dpkg -i code_1.97.0-1738713410_amd64.deb

Use the command “code” to activate VScode. Then click “Extensions (Ctrl+Shift+X)” in VScode to search Dev Containers, Python and Remote Development to install them.

Dev Containers installation

Python installation

Remote Development installation

VScode Configurations

Docker Image

A docker image has been created for NN model conversion and the following command can be executed to pull it from zengping2024/c3v_npu:v0.0.5 in the Ubuntu PC.

docker pull zengping2024/c3v_npu:v0.0.5

devcontainer.json and docker-compose.yml

A devcontainer.json and a docker-compose.yml have been created for the docker and the container configurations in VScode. They can be pulled from https://github.com/peter-zen/ai_ide_devcontainer.git with the following command in the Ubuntu PC. After the download, the 2 files will be at ai_ide_devcontainer/.devcontainer/.

git clone -b develop https://github.com/peter-zen/ai_ide_devcontainer.git

The working folder of the container : /home/c3v/model_converter/models/mAP/

The shared folders in the host and the container : a. In the host : /path/ai_ide_devcontainer/ (where /path/ is the directory you download devcontainer.json and docker-compose.yml and here it is /home/adminuser/vincent/Docker/Q654/npu_VScode/) b. In the container : /home/c3v/model_converter/models/mAP/

Activate the container in VScode

In VScode, click “File“ → “Open Folder…[Ctrl+K Ctrl+O]” to select /path/ai_ide_devcontainer/ and click “Open”. (where /path/ is the directory you download devcontainer.json and docker-compose.yml and here it is /home/adminuser/vincent/Docker/Q654/npu_VScode/)

Then press “Ctrl+Shift+p” to search “Dev Containers: Reopen in Container” and click it to execute.

A container will be built in VScode based on the docker image, devcontainer.json and docker-compose.yml which have been downloaded earlier. It will also open a terminal at the bottom of the VScode window.

Remote - SSH (connect to the C3V Linux)

Press “Ctrl + Shift + p“ to search “Remote-SSH: Connect to Host…” in VScode and click it.

Key “sunplus@172.18.12.72” in and press “Enter“. (where sunplus is the user name of the C3V Linux and 172.18.12.72 is the IP address)

It will open a new window of VScode for the C3V Linux. Key the password (sunplus) in and press “Enter“ in the new window. It will download VS Code Server for the C3V Linux and connect to it.

Click “Terminal” → “New Terminal” in the new window of VScode to open a terminal for the C3V Linux.

Now we got 2 VScode windows with terminals and one is for the container in the Ubuntu PC and the other is for the C3V Linux. We can manipulate both of them in VScode of the Ubuntu PC.

Samba Deployments

The container has installed “samba” based on the docker image. The shared folder of samba is configured in /etc/samba/smb.conf and is /home/c3v/model_converter. The following commands in the C3V Linux are to create a new directory (/home/sunplus/npu/npu_c3v) and to mount samba shared folder of the container to it. Where 172.18.12.80 is the IP address of the container and 172.18.12.80/c3v_npu/ is the samba shared folder (/home/c3v/model_converter).

sudo mkdir -p npu/npu_c3v
sudo chown -R sunplus:sunplus npu
sudo mount -t cifs //172.18.12.80/c3v_npu npu/npu_c3v/ -o username=c3v,password=c3v,uid=$(id -u),gid=$(id -g)

NN Model Conversion

There are 3 NN model examples (yolov5s-detection, yolov8s-detection and yolov8s-pose) at /home/c3v/model_converter/models/ of the container.

Here yolov8s-pose will be taken to do model conversion. a. Copy /home/c3v/model_converter/models/yolov8s-pose to /home/c3v/model_converter/models/mAP.

The folder of yolov8s-pose includes 4 files as shown in the following figure.

b. Copy /home/c3v/model_converter/model_cvt_script.sh to /home/c3v/model_converter/models/mAP.(Where model_cvt_script.sh is the script to import, quantize, inference or export the NN model)

The NN model conversion will be done at /home/c3v/model_converter/models/mAP.

import

The following command will be used to import the NN model.

./model_cvt_script.sh import yolov8s-pose int8

import command

end import

quantize

Modify the scale value(1/255=0.003921569) of the yolov8s-pose_inputmeta.yml file, which is in yolov8s-pose.

scale: 0.003921569

The following command will be used to quantize the NN model.

./model_cvt_script.sh quantize yolov8s-pose int8

quantize command

end quantize

inference

The following command will be used to inference the NN model.

./model_cvt_script.sh inference yolov8s-pose int8

inference command

end inference

export

The following command will be used to export the NN model.

./model_cvt_script.sh export yolov8s-pose int8

export command

end export

program compilation

After exporting the NN model, the folder of yolov8s-pose_int8_nbg_unify will be created and it includes the NN application and a network_binary.nb file shown below. Copy /home/c3v/moder_converter/Makefile to yolov8s-pose_int8_nbg_unify.

Open Makefile and modify TOOLCHAIN and VIVANTE_SDK_DIR as shown below.

TOOLCHAIN?=/npu/arm-gnu-toolchain-12.3.rel1-x86_64--aarch64-none-linux-gnu/bin
VIVANTE_SDK_DIR?=/npu/vip9000sdk-6.4.18.5/6.4.18.5-patch

In /home/c3v/model_converter/models/mAP/yolov8s-pose/yolov8s-pose_int8_nbg_unify, use the following command to compile the NN application.

make BIN=yolov8s-pose_sample -j

After the compilation, it will create an executable file of yolov8s-pose_sample. And

Copy /home/c3v/model_converter/models/mAP/yolov8s-pose/yolov8s-pose_int8_nbg_unify/yolov8s-pose_sample and network_binary.nb to /home/c3v/model_converter/.
Copy /home/c3v/model_converter/models/mAP/yolov8s-pose/input.jpg to /home/c3v/model_converter/.

Run on the C3V Linux

At /home/sunplus/npu/npu_c3v/ of the C3V Linux, use the following command to run the NN model via the application.

./yolov8s-pose_sample network_binary.nb input.jpg

Automatic Deployment and Verification for NN Models Using Docker