Vision-language-action models from edge to cloud — and where world models take embodied intelligence next
Goal: understand and optimize the policies that let machines act in the physical world — across the full hardware spectrum robots actually ship with.
Cross-platform VLA scaling
Five representative vision-language-action models evaluated on the LIBERO benchmark from power-constrained edge SoCs to datacenter GPUs, measuring task accuracy jointly with latency, throughput, and peak memory. Architectural choices dominate throughput and memory; edge devices degrade non-linearly yet well-chosen edge configurations match or exceed older datacenter GPUs — challenging the assumption that robotic inference belongs in the cloud. Published at GLSVLSI 2026 (Taherin et al., 2026).
VOTE — efficient VLA optimization
A training framework that finetunes VLA models to emit far fewer action tokens, plus a voting-based ensemble over action trajectories: 39× faster inference than OpenVLA with 46 Hz throughput on edge platforms, at higher success rates (Lin et al., 2026).
World models
With collaborators, a unified perspective on world models organized by the cognitive functions each line of work innovates — memory, perception, language, reasoning, imagining, motivation, and metacognition — identifying motivation and metacognition as drastically under-researched and introducing epistemic world models for scientific discovery (Rupprecht* et al., 2026). From my systems perspective: predictive models of physical environments add sustained memory, latency, and energy pressure exactly where they can least be afforded — the edge.
Vision-Language-Action (VLA) models have emerged as powerful generalist policies for robotic control, yet their performance scaling across model architectures and hardware platforms, as well as their associated power budgets, remain poorly understood. This work presents an evaluation of five representative VLA models—spanning state-of-the-art baselines and two newly proposed architectures—targeting edge and datacenter GPU platforms. Using the LIBERO benchmark, we measure accuracy alongside system-level metrics, including latency, throughput, and peak memory usage, under varying edge power constraints and high-performance datacenter GPU configurations. Our results identify distinct scaling trends: (1) architectural choices, such as action tokenization and model backbone size, strongly influence throughput and memory footprint; (2) power-constrained edge devices exhibit non-linear performance degradation, with some configurations matching or exceeding older datacenter GPUs; and (3) high-throughput variants can be achieved without significant accuracy loss. These findings provide actionable insights when selecting and optimizing VLAs across a range of deployment constraints. Our work challenges current assumptions about the superiority of datacenter hardware for robotic inference.
@inproceedings{taherin2026crossplatform,title={Cross-Platform Scaling of Vision-Language-Action Models from Edge to Cloud GPUs},author={Taherin, Amir and Lin, Juyi and Akbari, Arash and Akbari, Arman and Zhao, Pu and Chen, Weiwei and Kaeli, David and Wang, Yanzhi},booktitle={Proceedings of the Great Lakes Symposium on VLSI (GLSVLSI)},year={2026},doi={10.1145/3787109.3816400},}
Recent large-scale Vision Language Action (VLA) models have shown superior performance in robotic manipulation tasks guided by natural language. However, current VLA models suffer from two drawbacks: (i) generation of massive tokens leading to high inference latency and increased training cost, and (ii) insufficient utilization of generated actions resulting in potential performance loss. To address these issues, we develop a training framework to finetune VLA models for generating significantly fewer action tokens with high parallelism, effectively reducing inference latency and training cost. Furthermore, we introduce an inference optimization technique with a novel voting-based ensemble strategy to combine current and previous action predictions, improving the utilization of generated actions and overall performance. Our results demonstrate that we achieve superior performance compared with state-of-the-art VLA models, achieving significantly higher success rates and 39x faster inference than OpenVLA with 46 Hz throughput on edge platforms, demonstrating practical deployability.
@inproceedings{lin2025vote,title={Vision-Language-Action Model Optimization for Real-Time Robotic Manipulation},author={Lin, Juyi and Taherin, Amir and Akbari, Arash and Akbari, Arman and Lu, Lei and Chen, Guangyu and Padir, Taskin and Yang, Xiaomeng and Srinivasa, Jayanth and Lin, Xue and Kaeli, David and Wang, Yanzhi and Zhao, Pu and Shen, Xuan},booktitle={Artificial Neural Networks and Machine Learning -- ICANN 2026 (Lecture Notes in Computer Science, Springer)},year={2026},pages={279--291},doi={10.1007/978-3-032-38404-1_23},}
This report of world models distinguishes prior works by the cognitive functions they innovate. Many works claim an almost human-like cognitive capability in their world models. To evaluate these claims requires a proper grounding in first principles from human and machine cognition theory. In moving towards human-like world models we present a conceptual unified framework for world models that fully incorporates all the cognitive functions (i.e., memory, perception, language, reasoning, imagining, motivation, and metacognition) and identify gaps in existing research as a guide for future states of the art. In particular, we find that motivation (especially intrinsic motivation) and metacognition remain drastically under-researched, and we propose concrete directions to address these gaps informed by active inference and global workspace theory. We also introduce epistemic world models, a new category encompassing agent frameworks for scientific discovery that operate over structured knowledge. Our taxonomy, applied to video, embodied, and epistemic world models, suggests research directions where prior taxonomies have not.
@misc{rupprecht2026human,title={Human Cognition in Machines: A Unified Perspective of World Models},author={Rupprecht, Timothy and Zhao, Pu and Taherin, Amir and Akbari, Arash and Akbari, Arman and He, Yumei and Imtiaz, Tooba and Duffy, Sean and Lin, Juyi and Chen, Yixiao and Chowdhury, Rahul and Nan, Enfu and Shen, Yixin and Cao, Yifan and Zeng, Haochen and Chen, Weiwei and Yuan, Geng and Dy, Jennifer and Ostadabbas, Sarah and Zhang, Xuan and Kaeli, David and Yeh, Edmund and Wang, Yanzhi},year={2026},}