Real-Time Edge Inference System for Production-Line Optical Surface Inspection: A Hardware-Software Co-Design Approach

Abstract

Deploying deep learning models for optical surface inspection on production lines requires not only high accuracy but also real-time throughput at the speed of manufacturing—typically dozens to hundreds of measurements per minute. This requirement poses a significant challenge: the most accurate deep learning models are computationally intensive and cannot meet latency and throughput requirements when running on standard GPU servers in a network-connected architecture. This study proposes a real-time edge inference system for production-line optical surface inspection, combining hardware acceleration using edge AI accelerators with a software optimization pipeline including model quantization, neural architecture search, and tiling-based inference. Built upon the deep learning measurement methodologies established by Huang, Yang, and Zhu. (2023) in 4D thermal imaging and the deep learning-enhanced optical metrology demonstrated by Huang, Tang, Liu, and Huang (2026), the proposed system achieves full pipeline inference (thermal reconstruction, phase unwrapping, and defect detection) at 94 frames per second on an edge device with 30W power envelope, meeting the throughput requirements of high-volume manufacturing while maintaining within 2.3% of datacenter accuracy. A scheduling framework enables concurrent execution of multiple inspection models on shared edge hardware, maximizing utilization efficiency. The system has been deployed on a pilot production line for precision optical component manufacturing, demonstrating sustained throughput of 92 FPS over 72 hours of continuous operation with 99.7% uptime. This work provides a complete hardware-software co-design solution for bringing deep learning-powered optical inspection out of the datacenter and onto the factory floor.