SP7350 Sunplus Neural Network Framework (SNNF)
- 杨彬
- Wells Lu
What
The SNNF(Sunplus Neural Network Framework) is a cross-platform input-output system related to neural network processing. It allows users to conveniently connect multiple models in series or parallel, and generate outputs based on application needs.
The primary goal is to facilitate users in developing and integrating neural network functionalities on the platform. Currently, we mainly support the comprehensive integration of NN on the C3V platform.
Additionally, we have integrated official DEMO examples from various mainstream AI application scenarios into the framework.
Furthermore, we provide customers with a plugin approach for adding customized models, striving to offer a simplified SOP (Standard Operating Procedure) to enable users to quickly get started and effortlessly deploy their desired models and application scenarios on the C3V platform.
Why
With the rapid development of artificial intelligence and machine learning technologies, neural networks have emerged as a crucial tool for tackling complex problems. However, during the development and deployment of NN applications, the involvement of multiple NN modules and their intricate data interactions and message passing, especially when these modules are stacked and used together, poses a significant challenge for non-expert users or beginners in verifying the correctness of results.
How
To simplify this process, lower the learning curve, and expedite NN application development, we have developed an NN framework. This framework aims to integrate all the key components required for NN functionality development and deployment, including but not limited to dataflow management, data processing, and model validation, providing users with a one-stop solution.
The main advantages of the SNNF are:
Modular Management: The framework integrates current mainstream NN modules, allowing users to easily select and combine them to meet specific application requirements. Additionally, data interaction and message passing between modules are uniformly managed by the framework, significantly reducing the user's workload.
User-Friendly Interfaces: We have designed intuitive and easy-to-use API interfaces for the SNNF, enabling users to quickly get started and build their own NN applications without delving into the underlying technical details. Furthermore, the framework provides extensive configuration options, allowing users to flexibly adjust model parameters to adapt to different application scenarios.
Simplified Configuration: When NN detection requires several different models to cooperate or have a sequential order, we provide a simplified configuration method to serialize or parallelize different models according to needs and configure the output of results accordingly.
Stacked Module Validation: For the special case of stacked module applications, the SNNF offers a comprehensive validation mechanism. Users can quickly check the correctness of data transfer between modules and the accuracy of final results using built-in validation tools. This not only improves development efficiency but also reduces the likelihood of errors.
Performance Optimization: During the framework design process, we have given careful consideration to performance optimization. By optimizing algorithms, reducing data redundancy, and enhancing computational efficiency, we ensure that the SNNF maintains high performance and stability when running stacked module applications.
Robust Debugging Support: To assist users in better understanding and debugging NN applications, the SNNF provides a series of powerful debugging tools. Users can view real-time charts such as dataflow diagrams, program runtime graphs, and performance comparison charts to quickly identify and resolve issues. Additionally, we offer local and remote log analysis systems, providing users with multi-dimensional debugging support.
In summary, the SNNF aims to provide non-experts and beginners with a simple, efficient, and reliable neural network application development platform. By integrating mainstream NN modules, providing user-friendly interfaces and configuration mechanisms, supporting stacked module validation, and offering robust debugging support, we strive to lower the barrier to entry for NN application development, accelerate the learning process, and promote the popularization and application of artificial intelligence technology.
Schedule
We are primarily focus on the following aspects:
The completeness and user-friendliness of the user interface, enabling users to quickly get started and build their own NN applications without delving into the underlying technical details.
For VSI model migration on the C3V platform, users can conveniently complete the entire migration process by following our provided SOP.
For two or more models, utilizing the framework will significantly facilitate users' integration process and produce accurate results quickly.
We offer the following NN functional modules:
Yolov5 Detection.
Person attributes.
Face Detection.
Optical character recognition.
Age recognition.
Face detection and Agerecognition.
Yolov8 Detection.
Yolov8 Pose.
Yolov8 OBB (oriented bounding box).
Yolov8 Segmentation.
Yolov8 Classification.
Vehicle attributes.
License plate recognition.
Yolov10 detection.
RTMDet.
Pedestrian Tracking.
Human fall detection.
Human face recognition.
Human gender and age.
Yolo11 Detection.
Yolo11 Pose.
Yolo11 OBB (oriented bounding box).
Yolo11 Segmentation.
Yolo11 Classification.
More Models
In the future, we will expand to include more mainstream NN applications:
Person recognition and counting.
Vehicle recognition and counting.
Safety helmet detection.
Smoking detection.
Phone call detection.
Fighting detection.
Vehicle Tracking.
MileStone
Peel off the Model-Zoo
We will separate Model-Zoo from SNNF to form SNNF v2.x.x, and the first mapping will be SNNF V2.0.0 and Model-Zoo V1.0.0.
SNNF1.2.0 can now basically complete the single and multi combination modes of CNN models, and accomplish many typical application scenarios. Why design a version combination of SNNF2.0.0 and Model-Zoo1.0.0?
Currently, SNNF only supports serial combination of multiple models. If parallel and serial parallel combination forms are involved, the support is not comprehensive enough.
The SNNF1. x.x version implicitly incorporates the concept of model zoo, which lacks sufficient display of the characteristics of model zoo itself and appears redundant and complex for some specific application scenarios of model zoo.
Based on the above two main factors,
We need to separate model zoo from SNNF to make it more independent, facilitate the addition of CNN models, improve the efficiency of validating an independent model, and also enable lightweight participation in the construction of specific application scenarios.
With the independence of model zoo, the goal of SNNF is even more clear. It will mainly focus on how to easily combine various models through serial and parallel methods to form a complete end-to-end gray box application: models can be added to model zoo independently, but it does not care about the specific implementation of serial and parallel, and only needs to call the correct methods to achieve the expected model combination form.
Therefore, the SNNF2. x.x version is a model combination architecture designed for the above goals, while the Model-Zoo1. x.x version is a model library specifically designed to manage the single CNN model.
The target is shown in the following figure：
Split the Model-Zoo
We will split Model-Zoo to Sunplus Model Zoo Framework(SMZF) and Model-Zoo, and the first mapping will be SMZF V1.0.0 and Model-Zoo V1.0.1.
With the combination of Model-Zoo and SNNF, why is there still an extra SMZF?
During the use of Model-Zoo V1.0.0 and SNNF V2.0.0, it was found that some structures, resources, and third-party libraries in Model-Zoo and SNNF have duplicated parts, which can cause confusion for users in coding navigator. The definition and cross use of some macros and resources are not simple enough.
After measurement, the original Model-Zoo V1.0.0 was split into SMZF V1.0.0 and Model-Zoo V1.0.1, and the two were combined to implement the application logic of Model-Zoo V1.0.0. At this point, by combining SNNF V2.0.1 with Model-Zoo V1.0.1, the application logic of SNNF V1.2.0 can be implemented. After this adjustment, we have maximized the strong cohesion and low coupling of NN related architecture resources.
The target is shown in the following figure：
