Table of links ABSTRACT 1 INTRODUCTION 2 BACKGROUND: OMNIDIRECTIONAL 3D OBJECT DETECTION 3 PRELIMINARY EXPERIMENT 3.1 Experiment Setup 3.2 Observations 3.3 Summary and Challenges 4 OVERVIEW OF PANOPTICUS 5 MULTI-BRANCH OMNIDIRECTIONAL 3D OBJECT DETECTION 5.1 Model Design 6 SPATIAL-ADAPTIVE EXECUTION 6.1 Performance Prediction 5.2 Model Adaptation 6.2 Execution Scheduling 7 IMPLEMENTATION 8 EVALUATION 8.1 Testbed and Dataset 8.2 Experiment Setup 8.3 Performance 8.4 Robustness 8.5 Component Analysis 8.6 Overhead 9 RELATED WORK 10 DISCUSSION AND FUTURE WORK 11 CONCLUSION AND REFERENCES 4 OVERVIEW OF PANOPTICUS Motivated by our observations and insights, we present Panopticus, an omnidirectional 3D object detection system designed for resource-constrained edge devices. Panopticus aims to maximize detection accuracy while meeting the latency objective with a given edge device. Figure 6 illustrates the system architecture of Panopticus. To accommodate camera views with varying inference requirements, we propose an omnidirectional 3D detection model equipped with multiple inference branches. Each branch comprises different modules from BEV-based 3D detectors, providing flexibility in processing each camera view with distinct detection capabilities. The model architecture can be modified by simply detaching the branch modules to meet the diverse requirements accounting for latency and device. In the offline stage, runtime characteristics such as inference latency are profiled for each branch. Modules within a heavy branch that exceed latency or memory constraints are removed. At runtime, each of 𝑁 multi-view images at time 𝑡 is processed by the proper branch to generate accurate 3D bounding boxes. The optimal selection of branch and image combinations is determined by a spatial-adaptive execution scheduler. The scheduler estimates the expected accuracy and inference time for each branch-image pair, based on the predicted spatial distribution for the incoming frame. Then, the optimal 𝑁 image-branch pairs are selected, which maximizes overall accuracy while ensuring the total predicted latency is within the target latency. The detected boxes are used to update the tracked status of the surrounding objects, allowing the continuous observation of spatial characteristics. Authors:

Jeho Lee
Chanyoung Jung
Jiwon Kim
Hojung Cha This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. Table of links Table of links ABSTRACT ABSTRACT ABSTRACT 1 INTRODUCTION 1 INTRODUCTION 1 INTRODUCTION 2 BACKGROUND: OMNIDIRECTIONAL 3D OBJECT DETECTION 2 BACKGROUND: OMNIDIRECTIONAL 3D OBJECT DETECTION 2 BACKGROUND: OMNIDIRECTIONAL 3D OBJECT DETECTION 3 PRELIMINARY EXPERIMENT 3 PRELIMINARY EXPERIMENT 3 PRELIMINARY EXPERIMENT 3.1 Experiment Setup 3.2 Observations 3.3 Summary and Challenges 4 OVERVIEW OF PANOPTICUS 4 OVERVIEW OF PANOPTICUS 4 OVERVIEW OF PANOPTICUS 5 MULTI-BRANCH OMNIDIRECTIONAL 3D OBJECT DETECTION 5 MULTI-BRANCH OMNIDIRECTIONAL 3D OBJECT DETECTION 5 MULTI-BRANCH OMNIDIRECTIONAL 3D OBJECT DETECTION 5.1 Model Design 6 SPATIAL-ADAPTIVE EXECUTION 6 SPATIAL-ADAPTIVE EXECUTION 6 SPATIAL-ADAPTIVE EXECUTION 6.1 Performance Prediction 5.2 Model Adaptation 6.2 Execution Scheduling 7 IMPLEMENTATION 7 IMPLEMENTATION 7 IMPLEMENTATION 8 EVALUATION 8 EVALUATION 8 EVALUATION 8.1 Testbed and Dataset 8.2 Experiment Setup 8.3 Performance 8.4 Robustness 8.5 Component Analysis 8.6 Overhead 9 RELATED WORK 9 RELATED WORK 9 RELATED WORK 10 DISCUSSION AND FUTURE WORK 10 DISCUSSION AND FUTURE WORK 10 DISCUSSION AND FUTURE WORK 11 CONCLUSION AND REFERENCES 11 CONCLUSION AND REFERENCES 11 CONCLUSION AND REFERENCES 4 OVERVIEW OF PANOPTICUS 4 OVERVIEW OF PANOPTICUS Motivated by our observations and insights, we present Panopticus, an omnidirectional 3D object detection system designed for resource-constrained edge devices. Panopticus aims to maximize detection accuracy while meeting the latency objective with a given edge device. Figure 6 illustrates the system architecture of Panopticus. To accommodate camera views with varying inference requirements, we propose an omnidirectional 3D detection model equipped with multiple inference branches. Each branch comprises different modules from BEV-based 3D detectors, providing flexibility in processing each camera view with distinct detection capabilities. The model architecture can be modified by simply detaching the branch modules to meet the diverse requirements accounting for latency and device. In the offline stage, runtime characteristics such as inference latency are profiled for each branch. Modules within a heavy branch that exceed latency or memory constraints are removed. At runtime, each of 𝑁 multi-view images at time 𝑡 is processed by the proper branch to generate accurate 3D bounding boxes. The optimal selection of branch and image combinations is determined by a spatial-adaptive execution scheduler. The scheduler estimates the expected accuracy and inference time for each branch-image pair, based on the predicted spatial distribution for the incoming frame. Then, the optimal 𝑁 image-branch pairs are selected, which maximizes overall accuracy while ensuring the total predicted latency is within the target latency. The detected boxes are used to update the tracked status of the surrounding objects, allowing the continuous observation of spatial characteristics. Authors: Jeho Lee
Chanyoung Jung
Jiwon Kim
Hojung Cha Authors: Jeho Lee Chanyoung Jung Jiwon Kim Hojung Cha Jeho Lee Jeho Lee Chanyoung Jung Chanyoung Jung Jiwon Kim Jiwon Kim Hojung Cha Hojung Cha This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Optimizing AI for Real-Time Object Tracking

Optimizing Omnidirectional 3D Object Detection for Edge AI

Omnidirectional Technology

Too Long; Didn't Read

Make resilience your competitive advantage

Efficient 3D Object Detection for Edge Devices

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Next-Gen 3D Detection: Challenges, Innovations & Future Directions

This AI Model Gives Edge Devices Eyes on the Back of Their Heads

How 3D Object Detection Helps AI Avoid Obstacles

Optimizing Omnidirectional 3D Object Detection for Edge AI

Optimizing AI for Real-Time Object Tracking

Next-Gen 3D Detection: Challenges, Innovations & Future Directions

This AI Model Gives Edge Devices Eyes on the Back of Their Heads

How 3D Object Detection Helps AI Avoid Obstacles

Optimizing Omnidirectional 3D Object Detection for Edge AI

Optimizing AI for Real-Time Object Tracking

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps