cpp-inspection-runtime

C++17 inference runtime for YOLO surface-defect models, built for the deployment side of inspection equipment: no Python at runtime, one static CLI, measured latency.

The model comes from surface-defect-yolo (YOLO11/YOLOv8 trained on NEU-DET, exported to ONNX). This repo loads the ONNX graph with ONNX Runtime, does letterbox preprocessing and NMS with OpenCV, and reads class names from the ONNX metadata, so any ultralytics detection export works unmodified.

Build

Dependencies: CMake ≥ 3.16, OpenCV, ONNX Runtime.

# macOS
brew install cmake opencv onnxruntime

# Ubuntu: apt install libopencv-dev, then either set -DONNXRUNTIME_ROOT
# to an extracted release from https://github.com/microsoft/onnxruntime/releases
# or install it system-wide.

cmake -B build -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

Run

# fetch a few NEU-DET validation images (not redistributed in the repo)
pip install kagglehub && python scripts/fetch_samples.py

# annotate images
./build/inspect --model models/best.onnx --input samples --out outputs

# latency benchmark (single image, repeated)
./build/inspect --model models/best.onnx --input samples/patches_295.jpg --bench 200

Options: --imgsz (default 640, must match the export), --conf (default 0.25), --bench N (N timed runs after warm-up).

What the code does

• src/detector.{hpp,cpp} — YoloDetector: letterbox to the export size, BGR→RGB NCHW float tensor, single session.Run, decode the (1, 4+nc, anchors) output head, class-wise NMS (cv::dnn::NMSBoxes), map boxes back through the letterbox transform.
• src/main.cpp — CLI: directory/file input, annotated image output, latency benchmark (mean/median/p95).

Benchmark

Model: models/best.onnx — YOLO11n trained on NEU-DET (mAP50 0.739), exported at imgsz 640, 10.1 MB.

metric	value
mean latency	18.9 ms
median	18.8 ms
p95	20.3 ms
throughput	52.8 fps

Measured on Apple silicon (CPU execution provider, 200 runs after warm-up). Numbers are end-to-end detect() calls: preprocessing + inference + NMS. A 320-px export of the same architecture runs at ~6.4 ms (155 fps) on the same machine when lower resolution is acceptable.

License

MIT. NEU-DET images are downloaded separately by scripts/fetch_samples.py and are not part of this repository.