Table of links ABSTRACT 1 INTRODUCTION 2 BACKGROUND: OMNIDIRECTIONAL 3D OBJECT DETECTION 3 PRELIMINARY EXPERIMENT 3.1 Experiment Setup 3.2 Observations 3.3 Summary and Challenges 4 OVERVIEW OF PANOPTICUS 5 MULTI-BRANCH OMNIDIRECTIONAL 3D OBJECT DETECTION 5.1 Model Design 6 SPATIAL-ADAPTIVE EXECUTION 6.1 Performance Prediction 5.2 Model Adaptation 6.2 Execution Scheduling 7 IMPLEMENTATION 8 EVALUATION 8.1 Testbed and Dataset 8.2 Experiment Setup 8.3 Performance 8.4 Robustness 8.5 Component Analysis 8.6 Overhead 9 RELATED WORK 10 DISCUSSION AND FUTURE WORK 11 CONCLUSION AND REFERENCES 9 RELATED WORK 9 RELATED WORK Adaptive object detection systems. Many computer vision systems designed for resource-constrained edge devices have achieved efficient resource utilization by adapting to the video content or computing budgets [14, 15, 19, 22, 50]. In parallel with these advancements, several works have specifically focused on adaptive object detection methods [3, 28, 49], facilitating valuable applications such as edge video analytics or mobile augmented reality. For instance, ApproxDet [49] presented a multi-branch framework that switches between a detector and a tracker based on awareness of video content characteristics and resource contention. Remix [28] brings video content adaptiveness by partitioning images and selectively applying neural networks with different scales. However, existing adaptive methods fall short in supporting resource-efficient omnidirectional 3D detection. This type of detection requires processing each camera view using different 3D perception capabilities, such as enhanced estimation of objects’ 3D location and velocity. To process each view optimally, Panopticus predicts its expected performance based on short-term future dynamics in the spatial distribution, which is crucial for mobile scenarios. Although Remix [28] utilizes performance estimation based on object distribution, it relies on long-term historical distribution and does not account for various object characteristics in 3D, which is unsuitable for our system’s setting 3D object detection on edge devices. With the rapid advancement of edge computing, there is an increasing demand for employing 3D object detection on edge devices. DeepMix [18] addressed the limitations of edge resources by delegating the compute-intensive 2D detection tasks to a server equipped with high-performance GPUs. The other lightweight tasks, such as estimating the 3D location of detected objects using a depth sensor, are efficiently handled by mobile devices. Another solution, PointSplit [37], proposed a parallel processing technique that utilizes edge NPU and GPU to facilitate on-device execution of RGB-D camera-based 3D detection. This approach exemplifies the trend of harnessing the power of specialized AI accelerators to meet the demands of edge computing [26, 27, 33, 59]. VIPS [44], designed for self-driving vehicles, introduced an edge-based system that collaborates with outdoor infrastructures equipped with computing units and LiDAR sensors. This strategy effectively extended the vehicles’ perception ranges by fusing data from the onboard system and the infrastructure. Apart from these efforts, Panopticus effectively initiates a self-contained, comprehensive perception system for resource-constrained edge devices. The system introduced a novel concept of camera-based omnidirectional 3D perception optimized for edge computing abilities, eliminating the need for depth sensors or computation offloading. Authors:

Jeho Lee
Chanyoung Jung
Jiwon Kim
Hojung Cha This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. Table of links Table of links ABSTRACT ABSTRACT ABSTRACT 1 INTRODUCTION 1 INTRODUCTION 1 INTRODUCTION 2 BACKGROUND: OMNIDIRECTIONAL 3D OBJECT DETECTION 2 BACKGROUND: OMNIDIRECTIONAL 3D OBJECT DETECTION 2 BACKGROUND: OMNIDIRECTIONAL 3D OBJECT DETECTION 3 PRELIMINARY EXPERIMENT 3 PRELIMINARY EXPERIMENT 3 PRELIMINARY EXPERIMENT 3.1 Experiment Setup 3.2 Observations 3.3 Summary and Challenges 4 OVERVIEW OF PANOPTICUS 4 OVERVIEW OF PANOPTICUS 4 OVERVIEW OF PANOPTICUS 5 MULTI-BRANCH OMNIDIRECTIONAL 3D OBJECT DETECTION 5 MULTI-BRANCH OMNIDIRECTIONAL 3D OBJECT DETECTION 5 MULTI-BRANCH OMNIDIRECTIONAL 3D OBJECT DETECTION 5.1 Model Design 6 SPATIAL-ADAPTIVE EXECUTION 6 SPATIAL-ADAPTIVE EXECUTION 6 SPATIAL-ADAPTIVE EXECUTION 6.1 Performance Prediction 5.2 Model Adaptation 6.2 Execution Scheduling 7 IMPLEMENTATION 7 IMPLEMENTATION 7 IMPLEMENTATION 8 EVALUATION 8 EVALUATION 8 EVALUATION 8.1 Testbed and Dataset 8.2 Experiment Setup 8.3 Performance 8.4 Robustness 8.5 Component Analysis 8.6 Overhead 9 RELATED WORK 9 RELATED WORK 9 RELATED WORK 10 DISCUSSION AND FUTURE WORK 10 DISCUSSION AND FUTURE WORK 10 DISCUSSION AND FUTURE WORK 11 CONCLUSION AND REFERENCES 11 CONCLUSION AND REFERENCES 11 CONCLUSION AND REFERENCES 9 RELATED WORK 9 RELATED WORK 9 RELATED WORK Adaptive object detection systems. Many computer vision systems designed for resource-constrained edge devices have achieved efficient resource utilization by adapting to the video content or computing budgets [14, 15, 19, 22, 50]. In parallel with these advancements, several works have specifically focused on adaptive object detection methods [3, 28, 49], facilitating valuable applications such as edge video analytics or mobile augmented reality. For instance, ApproxDet [49] presented a multi-branch framework that switches between a detector and a tracker based on awareness of video content characteristics and resource contention. Remix [28] brings video content adaptiveness by partitioning images and selectively applying neural networks with different scales. However, existing adaptive methods fall short in supporting resource-efficient omnidirectional 3D detection. This type of detection requires processing each camera view using different 3D perception capabilities, such as enhanced estimation of objects’ 3D location and velocity. To process each view optimally, Panopticus predicts its expected performance based on short-term future dynamics in the spatial distribution, which is crucial for mobile scenarios. Although Remix [28] utilizes performance estimation based on object distribution, it relies on long-term historical distribution and does not account for various object characteristics in 3D, which is unsuitable for our system’s setting 3D object detection on edge devices. With the rapid advancement of edge computing, there is an increasing demand for employing 3D object detection on edge devices. DeepMix [18] addressed the limitations of edge resources by delegating the compute-intensive 2D detection tasks to a server equipped with high-performance GPUs. The other lightweight tasks, such as estimating the 3D location of detected objects using a depth sensor, are efficiently handled by mobile devices. Another solution, PointSplit [37], proposed a parallel processing technique that utilizes edge NPU and GPU to facilitate on-device execution of RGB-D camera-based 3D detection. This approach exemplifies the trend of harnessing the power of specialized AI accelerators to meet the demands of edge computing [26, 27, 33, 59]. VIPS [44], designed for self-driving vehicles, introduced an edge-based system that collaborates with outdoor infrastructures equipped with computing units and LiDAR sensors. This strategy effectively extended the vehicles’ perception ranges by fusing data from the onboard system and the infrastructure. Apart from these efforts, Panopticus effectively initiates a self-contained, comprehensive perception system for resource-constrained edge devices. The system introduced a novel concept of camera-based omnidirectional 3D perception optimized for edge computing abilities, eliminating the need for depth sensors or computation offloading. Authors: Jeho Lee
Chanyoung Jung
Jiwon Kim
Hojung Cha Authors: Jeho Lee Chanyoung Jung Jiwon Kim Hojung Cha Jeho Lee Jeho Lee Chanyoung Jung Chanyoung Jung Jiwon Kim Jiwon Kim Hojung Cha Hojung Cha This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Arweave

Advancements in Adaptive and 3D Object Detection for Edge AI

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Edge Devices Are Getting Better at Seeing the World in 3D

This AI Model Gives Edge Devices Eyes on the Back of Their Heads

How 3D Object Detection Helps AI Avoid Obstacles

Optimizing Omnidirectional 3D Object Detection for Edge AI

Efficient 3D Object Detection for Edge Devices

Optimizing AI for Real-Time Object Tracking

Edge Devices Are Getting Better at Seeing the World in 3D

This AI Model Gives Edge Devices Eyes on the Back of Their Heads

How 3D Object Detection Helps AI Avoid Obstacles

Optimizing Omnidirectional 3D Object Detection for Edge AI

Efficient 3D Object Detection for Edge Devices

Optimizing AI for Real-Time Object Tracking

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps