This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Authors: (1) Minghao Yan, University of Wisconsin-Madison; (2) Hongyi Wang, Carnegie Mellon University; (3) Shivaram Venkataraman, myan@cs.wisc.edu. Table of Links Abstract & Introduction Motivation Opportunities Architecture Overview Proble Formulation: Two-Phase Tuning Modeling Workload Interference Experiments Conclusion & References A. Hardware Details B. Experimental Results C. Arithmetic Intensity D. Predictor Analysis B EXPERIMENTAL RESULTS In this section, we further demonstrate the tradeoff between memory frequency and maximum GPU frequency by presenting an array of results. These results underline the interesting observation that the energy consumption patterns vary for the same model operating on different devices. Furthermore, even for the same model device pairing, the optimization landscape can be significantly influenced by the batch size. This underlines the complexities of energy optimization and the need for an adaptive framework that can take these factors into account. Figures 6 − 12 show the energy consumption patterns of EfficientNet and Bert on Jetson TX2 and Orin under various batch sizes. Table 7 shows the optimal CPU frequency and corresponding energy consumption reduction in image preprocessing.

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices: Experimental Results

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A Dynamic Programming Approach to Optimizing Signaling Strategies in Multi-phase Trials:

The Promise Of Edge Computing

PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices: Predictor Analysis

PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices: Architecture Overview

Proble Formulation: Two-Phase Tuning

Modeling Workload Interference

A Dynamic Programming Approach to Optimizing Signaling Strategies in Multi-phase Trials:

The Promise Of Edge Computing

PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices: Predictor Analysis

PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices: Architecture Overview

Proble Formulation: Two-Phase Tuning

Modeling Workload Interference

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps