A Progressive Compression Method for Full-Size Convolutional Kernels Oriented Toward Embedded Deployment

To address the issues of the expansion of the size of computational models and their adaptability to tasks in computer vision models designed for embedded deployment with low latency， we propose a progressive compression method for full-size convolutional kernels based on tensor decomposition and singular value truncation. We further propose an efficient and general accelerated deployment scheme based on neural processing unit （NPU） devices. Firstly， a tensor decomposition technique was applied to replace the commonly used 3×3 large-sized convolutional kernels with lightweight convolutional derived structures， while the energy distribution of downstream 1×1 convolution kernels responsible for channel combination was reshaped. Secondly， leveraging the induced energy decay characteristics， singular value decomposition （SVD） was applied to perform compression on 1×1 convolution kernels to construct a lightweight and deployment-friendly vision model through a progressive compression process. Finally， in response to the common challenges faced by embedded chips-namely relatively static computational resource scheduling and the difficulty of directly migrating NVIDIA-oriented quantization strategies， we propose a queue-based adaptive dynamic thread adjustment strategy and a layer-sensitivity-aware mixed quantization strategy. Through finer-grained， task-aware dynamic allocation of inference threads and the introduction of hardware feasibility weights for layer sensitivity evaluation， mixed quantization was ultimately achieved to fully exploit chip computational potential and realize low-latency real-time inference. Experimental results showed that the parameter compression ratios of YOLOv5s， YOLOv8s， YOLOv8n， and YOLOv11n models reached 2.4×， 3.9×， 1.3×， and 1.5×， respectively， with inference speed improvements generally exceeding 30 FPS over baseline models on an RTX 3070 GPU after progressive full-size convolution kernel compression. For ResNet50 and MobileNetV2 models， parameter compression ratios of 3.7× and 1.3× were achieved with accuracy losses within 1%， which validates the generality and robustness of the proposed method. After mixed quantization， the inference speeds of the YOLO series reached 215 FPS， 196 FPS， 209 FPS， and 203 FPS， achieving speedups by a factor of 1.5 to 2， while the number of parameters and memory overhead were further reduced to better meet embedded deployment requirements. Real-time task processing speeds of up to 112 FPS were achieved in deployment on the Huawei Ascend 310B chip despite limited hardware resources. These results demonstrate that the proposed approach effectively balances compression， deployment efficiency， and real-time performance， which makes it suitable for low-latency embedded vision applications.