Authors:
(1) Umberto Michieli, Samsung Research UK;
(2) Jijoong Moon, Samsung Research Korea;
(3) Daehyun Kim, Samsung Research Korea;
(4) Mete Ozay, Samsung Research UK. Authors: Authors: (1) Umberto Michieli, Samsung Research UK; (2) Jijoong Moon, Samsung Research Korea; (3) Daehyun Kim, Samsung Research Korea; (4) Mete Ozay, Samsung Research UK. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Few-Shot Personalized Instance Recognition
Object-Conditioned Bag of Instances
Experimental Results
Conclusion
References Few-Shot Personalized Instance Recognition Few-Shot Personalized Instance Recognition Object-Conditioned Bag of Instances Object-Conditioned Bag of Instances Experimental Results Experimental Results Conclusion Conclusion References References ABSTRACT Nowadays, users demand for increased personalization of vision systems to localize and identify personal instances of objects (e.g., my dog rather than dog) from a few-shot dataset only. Despite outstanding results of deep networks on classical label-abundant benchmarks (e.g., those of the latest YOLOv8 model for standard object detection), they struggle to maintain within-class variability to represent different instances rather than object categories only. We construct an Object-conditioned Bag of Instances (OBoI) based on multiorder statistics of extracted features, where generic object detection models are extended to search and identify personal instances from the OBoI’s metric space, without need for backpropagation. By relying on multi-order statistics, OBoI achieves consistent superior accuracy in distinguishing different instances. In the results, we achieve 77.1% personal object recognition accuracy in case of 18 personal instances, showing about 12% relative gain over the state of the art. my dog dog Object-conditioned Bag of Instances 1. INTRODUCTION Smart devices are starting to be ubiquitous in everyday life [1] and their users are demanding for instance-level personalized detection of vision systems mounted on such devices [2, 3]. For example, vacuum cleaners can now monitor the behavior of users’ specific pets, and stay away from those specific pets that are mostly scared by the robot’s noise [4]. Nonetheless, users do not provide many labeled examples, being a time-consuming operation. Therefore, we introduce a new task of few-shot instance-level personalization of object detection models to detect and recognize personal instances of objects (e.g., dog1 and dog2 rather than just dog). The limited availability of the data distinguishes our task from previous instance-level personalization attempts [5, 6]. To the best of our knowledge, previous works assume large availability of labelled data and finetune (FT) the models through computationally expensive updates. However, FT-based methods inevitably fail when few-shot samples are provided [7, 8, 9, 10]. In our work, we utilize the latest YOLOv8 [11] efficient detection model, and we enable personalized instance recognition via backpropagation-free Prototypes-based Few-Shot Learners (PFSLs), such as [12, 13]. In short, PFSLs learn a metric space in which classification is performed by computing distances to prototypical representations of each class. In this context, we extend PFSLs to support object-class conditioned search, and we call these approaches Object-conditioned Bag of Instances (OBoI), since they contain instance-level prototypes. Our approach enriches any OBoI method by augmenting localized encoder embeddings (EEs) of the input object via multi-order statistics to construct a richer metric space, where instance-specific patterns are separable. We compute augmented EEs (AEEs) via a reduction module similar to recent pooling schemes [14, 15, 16, 17] to characterize the distribution of the specific instances from the few-shot labelled data. A concurrent work [14] applies ensemble learning on multi-order features learned separately; however, their focus is neither personalized instance recognition nor object detection, and they require gradient-based training. A backpropagation-free approach, instead, could be especially useful where dynamic compilers are not available for the target hardware. Our OBoIs with AEEs significantly increase model personalization, alleviating neural collapse [18, 19], i.e., a state at which within-class variability of hidden layer outputs is completely lost due to the object-level optimization objective. Our main novelties are: Object-conditioned Bag of Instances We propose a novel task of few-shot personalization of object detectors to recognize instances of objects;


We extend PFSLs via object-level conditioning (OBoIs);


We further design a multi-order feature space where personal instances can be separated via a backpropagationfree metric learning on few-shot labelled user data only;


OBoIs provide superior results on both same and other domain data (11-22% and 7-18% relative gains respectively). We propose a novel task of few-shot personalization of object detectors to recognize instances of objects; We propose a novel task of few-shot personalization of object detectors to recognize instances of objects; We extend PFSLs via object-level conditioning (OBoIs); We extend PFSLs via object-level conditioning (OBoIs); We further design a multi-order feature space where personal instances can be separated via a backpropagationfree metric learning on few-shot labelled user data only; We further design a multi-order feature space where personal instances can be separated via a backpropagationfree metric learning on few-shot labelled user data only; OBoIs provide superior results on both same and other domain data (11-22% and 7-18% relative gains respectively). OBoIs provide superior results on both same and other domain data (11-22% and 7-18% relative gains respectively). This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Boost Few-Shot Instance Accuracy 12% Without Fine-Tuning

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

A Coarse-to-Fine Approach to Text-Guided 3D Localization Without Ground Truth

Apparently, Feeding ChatGPT With a Lot of Personal Information Makes It Better

Comparative Study of YOLOv5 and YOLOv8 for Challenging Aquatic Object Detection

Train YOLO in 10 Epochs: A Lean Recipe for Marine Micro-Object Detection

Need Precision Plankton Counts? Why YOLOv5 Shines, But YOLOv8 Adapts

A Coarse-to-Fine Approach to Text-Guided 3D Localization Without Ground Truth

Apparently, Feeding ChatGPT With a Lot of Personal Information Makes It Better

Comparative Study of YOLOv5 and YOLOv8 for Challenging Aquatic Object Detection

Train YOLO in 10 Epochs: A Lean Recipe for Marine Micro-Object Detection

Need Precision Plankton Counts? Why YOLOv5 Shines, But YOLOv8 Adapts

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps