Real-time visual perception within edge power budgets — from FPGA co-design to adaptive detector scheduling
Goal: deliver real-time visual perception on devices where every joule counts, by restructuring the computation — in hardware or at runtime — instead of simply shrinking models.
ALBIREO — adaptive video object detection
A detector-agnostic, training-free framework that wraps off-the-shelf detectors and skips detector invocations when a per-object Kalman state is confident enough; skipped frames get predicted boxes at near-zero GPU cost, with a rescue mechanism preserving objects through brief detector misses. On BDD100K MOT across three detector families and two Jetson generations, accuracy stays within ±1.2 pp of per-frame inference while energy drops 12.1–17.6% — and on the primary configuration accuracy improves by +0.8 pp while energy falls 17.6%. To appear at ACM/IEEE SEC 2026 (Taherin et al., 2026).
Algorithm–architecture co-design for 360° video
Real-time 360° AR/VR video rendering restructured to fit FPGA on-chip memory budgets: co-designing the rendering algorithm with the architecture enabled energy-efficient processing on a Zynq UltraScale+ MPSoC without performance loss versus commercial pipelines. Published at FPGA 2020 (Sun et al., 2020).
Video object detection on edge devices requires running computationally expensive detectors over long frame streams, leading to high energy consumption and sustained GPU utilization. Although consecutive video frames contain substantial temporal redundancy, naive frame-skipping strategies are content-blind: they skip during critical moments such as object entry, occlusion recovery, and abrupt motion, causing substantial detection-quality loss. We present ALBIREO, a detector-agnostic, codec-free, adaptive inference framework that wraps off-the-shelf object detectors and decides when detector invocation can be safely skipped based on scene content and per-object temporal state. ALBIREO requires no detector modification or detector retraining and is designed as a drop-in efficiency layer for improving the accuracy-energy-latency tradeoff of edge video-detection pipelines. ALBIREO maintains a 10-dimensional Kalman filter (KF) for each active object state and invokes the detector only when prediction uncertainty exceeds a threshold. On skipped frames, bounding boxes are predicted from the KF state at near-zero GPU cost. A KF-based rescue mechanism preserves confirmed object states through brief detector misses, and a lightweight empty-scene screen further avoids full detector calls on objectless frames. We evaluate ALBIREO on the BDD100K MOT validation split using three architecturally distinct detectors (YOLO11x, YOLO26x, and RF-DETR-Large) on two NVIDIA Jetson platforms (AGX Thor and AGX Orin). Across all detector-platform configurations, ALBIREO keeps AP@50 within ±1.2 pp of per-frame inference while reducing total energy by 12.1-17.6%. On the primary YOLO26x configuration, ALBIREO improves AP@50 by +0.8 pp while reducing energy by 17.6% on Thor and 14.4% on Orin, and reduces per-frame energy-delay product by 24.9% on Thor and 26.1% on Orin.
@inproceedings{taherin2026albireo,title={ALBIREO: Adaptive, Energy-Efficient Inference Framework for Video Object Detection on the Edge},author={Taherin, Amir and Cano, José and Ren, Bin and Wang, Yanzhi and Kaeli, David},booktitle={ACM/IEEE Symposium on Edge Computing (SEC)},year={2026},}
@inproceedings{sun2020energy,title={Energy-Efficient 360-Degree Video Rendering on FPGA via Algorithm-Architecture Co-Design},author={Sun, Qiuyue and Taherin, Amir and Siatitse, Yawo and Zhu, Yuhao},booktitle={Proceedings of the 2020 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA)},year={2020},doi={10.1145/3373087.3375317},}