Patch and Model Size Characterization for On-Device Efficient-ViTs on Small Datasets Using 12 Quantitative Metrics
Park J.-G. Amangeldi A. Fakhrutdinov N. Karzhaubayeva M. Zorbas D.
2025Institute of Electrical and Electronics Engineers Inc.
IEEE Access
2025#1325704 - 25722 pp.
Vision transformers (ViTs) have emerged as a successful alternative to convolutional neural networks (CNNs) in deep learning (DL) applications for computer vision (CV), particularly excelling in accuracy on large-scale datasets within high-performance computing (HPC) or cloud domains. However, in the context of resource-constrained mobile and edge AI devices, there is a lack of systematic and comprehensive investigations into the challenging optimizations for both device-agnostic (e.g., accuracy and model size) and device-related (e.g., latency, memory usage, and power/energy consumption) multi-objectives. To resolve this problem, we first 1) introduce five device-agnostic (DA) and seven device-related (DR) quantitative metrics, 2) using which we thoroughly characterize the effects of ViT hyper-parameters on small datasets in terms of patch size and model size, and then 3) propose a simple yet effective optimization technique called the hierarchical and local (HelLo) tuning method for efficient ViTs. The results show that our method achieves significant improvements of up to 85% in MACs, 67.2% in inference latency, 77.7% in train latency/time, 63.3% in GPU memory, 73.8% in energy consumption, and 263.0% in FoM, with minimal accuracy degradation (up to 2%).
characterization , Deep learning , edge-AI , efficient vision transformers (ViTs) , embedded systems , mobile devices , multi-objective optimization , on-device ML
Text of the article Перейти на текст статьи
Nazarbayev University, School of Engineering and Digital Sciences, Astana, 010000, Kazakhstan
Carnegie Mellon University, Heinz College of Information Systems and Public Policy, Pittsburgh, 15213, PA, United States
Nazarbayev University
Carnegie Mellon University
10 лет помогаем публиковать статьи Международный издатель
Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026