Detector-agnostic, training-free adaptive inference for video object detection on edge GPUs
Video object detection on the edge runs an expensive detector on every frame, even when consecutive frames are nearly identical — yet naive frame skipping is content-blind and fails exactly when it matters: object entry, occlusion recovery, abrupt motion.
ALBIREO lets uncertainty decide. It wraps off-the-shelf detectors, tracks each object with a 10-dimensional Kalman filter, and invokes the real detector only when prediction uncertainty exceeds a threshold; skipped frames get predicted boxes at near-zero GPU cost. A rescue mechanism preserves confirmed objects through brief detector misses, and an empty-scene screen avoids detector calls on objectless frames. No detector modification or retraining is required.
Evaluated on BDD100K MOT across three architecturally distinct detectors (YOLO11x, YOLO26x, RF-DETR-Large) and two Jetson platforms (AGX Thor, AGX Orin):
AP@50 stays within ±1.2 pp of per-frame inference while total energy drops 12.1–17.6%.
On the primary YOLO26x configuration, AP@50 improves by +0.8 pp while energy falls 17.6% (Thor), with per-frame energy-delay product reduced by ~25%.
A fixed-interval skipping baseline with the same skip budget loses 8.6 pp AP@50.
Video object detection on edge devices requires running computationally expensive detectors over long frame streams, leading to high energy consumption and sustained GPU utilization. Although consecutive video frames contain substantial temporal redundancy, naive frame-skipping strategies are content-blind: they skip during critical moments such as object entry, occlusion recovery, and abrupt motion, causing substantial detection-quality loss. We present ALBIREO, a detector-agnostic, codec-free, adaptive inference framework that wraps off-the-shelf object detectors and decides when detector invocation can be safely skipped based on scene content and per-object temporal state. ALBIREO requires no detector modification or detector retraining and is designed as a drop-in efficiency layer for improving the accuracy-energy-latency tradeoff of edge video-detection pipelines. ALBIREO maintains a 10-dimensional Kalman filter (KF) for each active object state and invokes the detector only when prediction uncertainty exceeds a threshold. On skipped frames, bounding boxes are predicted from the KF state at near-zero GPU cost. A KF-based rescue mechanism preserves confirmed object states through brief detector misses, and a lightweight empty-scene screen further avoids full detector calls on objectless frames. We evaluate ALBIREO on the BDD100K MOT validation split using three architecturally distinct detectors (YOLO11x, YOLO26x, and RF-DETR-Large) on two NVIDIA Jetson platforms (AGX Thor and AGX Orin). Across all detector-platform configurations, ALBIREO keeps AP@50 within ±1.2 pp of per-frame inference while reducing total energy by 12.1-17.6%. On the primary YOLO26x configuration, ALBIREO improves AP@50 by +0.8 pp while reducing energy by 17.6% on Thor and 14.4% on Orin, and reduces per-frame energy-delay product by 24.9% on Thor and 26.1% on Orin.
@inproceedings{taherin2026albireo,title={ALBIREO: Adaptive, Energy-Efficient Inference Framework for Video Object Detection on the Edge},author={Taherin, Amir and Cano, José and Ren, Bin and Wang, Yanzhi and Kaeli, David},booktitle={ACM/IEEE Symposium on Edge Computing (SEC)},year={2026},}