Table of Links Abstract and 1 Introduction Abstract and 1 Introduction Related Works
2.1. Vision-and-Language Navigation
2.2. Semantic Scene Understanding and Instance Segmentation
2.3. 3D Scene Reconstruction


Methodology
3.1. Data Collection
3.2. Open-set Semantic Information from Images
3.3. Creating the Open-set 3D Representation
3.4. Language-Guided Navigation


Experiments
4.1. Quantitative Evaluation
4.2. Qualitative Results


Conclusion and Future Work, Disclosure statement, and References Related Works
2.1. Vision-and-Language Navigation
2.2. Semantic Scene Understanding and Instance Segmentation
2.3. 3D Scene Reconstruction Related Works 2.1. Vision-and-Language Navigation 2.1. Vision-and-Language Navigation 2.2. Semantic Scene Understanding and Instance Segmentation 2.2. Semantic Scene Understanding and Instance Segmentation 2.3. 3D Scene Reconstruction 2.3. 3D Scene Reconstruction Methodology
3.1. Data Collection
3.2. Open-set Semantic Information from Images
3.3. Creating the Open-set 3D Representation
3.4. Language-Guided Navigation Methodology Methodology 3.1. Data Collection 3.1. Data Collection 3.2. Open-set Semantic Information from Images 3.2. Open-set Semantic Information from Images 3.3. Creating the Open-set 3D Representation 3.3. Creating the Open-set 3D Representation 3.4. Language-Guided Navigation 3.4. Language-Guided Navigation Experiments
4.1. Quantitative Evaluation
4.2. Qualitative Results Experiments Experiments 4.1. Quantitative Evaluation 4.1. Quantitative Evaluation 4.2. Qualitative Results 4.2. Qualitative Results Conclusion and Future Work, Disclosure statement, and References Conclusion and Future Work, Disclosure statement, and References Conclusion and Future Work, Disclosure statement, and References 4.2. Qualitative Results To qualitatively demonstrate the effectiveness of the proposed O3D-SIM, this section includes visualizations of the model’s performance using select examples. These visualizations are displayed in Figure 5, illustrating the outcomes for two mapping sequences. Notably, the open-set capability of O3D-SIM enables the identification of objects that are typically undetectable by conventional pipelines relying on closed sets or predefined datasets, such as wheelchairs. The figure showcases a comparative analysis of our pipeline’s ability to accurately identify and segment various objects, including mannequins and mobile robots, against their actual counts. This comparison highlights situations where traditional methods, constrained by a limited set of recognizable objects, fall short. Our approach excels in recognizing instance-level semantics, accurately identifying 5 out of 6 table instances (with one false positive), underscoring its precision across both simulated and real-world data. This demonstrates the robustness of our pipeline, further evidenced by the clarity of the semantic map and the ease with which instance-level segmentation results can be visualized. While methods like VLMaps might identify a broader range of objects due to their open-set nature, and SI-Maps may detect multiple instances of the same object, O3D-SIM uniquely excels at both, offering a comprehensive solution. Authors:
(1) Laksh Nanwani, International Institute of Information Technology, Hyderabad, India; this author contributed equally to this work;
(2) Kumaraditya Gupta, International Institute of Information Technology, Hyderabad, India;
(3) Aditya Mathur, International Institute of Information Technology, Hyderabad, India; this author contributed equally to this work;
(4) Swayam Agrawal, International Institute of Information Technology, Hyderabad, India;
(5) A.H. Abdul Hafez, Hasan Kalyoncu University, Sahinbey, Gaziantep, Turkey;
(6) K. Madhava Krishna, International Institute of Information Technology, Hyderabad, India. Authors: Authors: (1) Laksh Nanwani, International Institute of Information Technology, Hyderabad, India; this author contributed equally to this work; (2) Kumaraditya Gupta, International Institute of Information Technology, Hyderabad, India; (3) Aditya Mathur, International Institute of Information Technology, Hyderabad, India; this author contributed equally to this work; (4) Swayam Agrawal, International Institute of Information Technology, Hyderabad, India; (5) A.H. Abdul Hafez, Hasan Kalyoncu University, Sahinbey, Gaziantep, Turkey; (6) K. Madhava Krishna, International Institute of Information Technology, Hyderabad, India. This paper is available on arxiv under CC by-SA 4.0 Deed (Attribution-Sharealike 4.0 International) license. This paper is available on arxiv under CC by-SA 4.0 Deed (Attribution-Sharealike 4.0 International) license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Quant

O3D-SIM Visualization: Accurately Identifying Unseen Objects and Instances

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

3D Mapping Initialization: Using RGB-D Images and Camera Parameters

It Is Okay If You Don't Know What You Like. We Do (feat. Deep Recommendation Algorithms)

10 Machine Learning, Data Science, and Deep Learning Courses for Programmers in 2020

10 Computer Vision Startups on Product Hunt with the Most Upvotes

10 Best Entry Level Machine Learning Tutorials

10 Best + Free Machine Learning Courses Collection

3D Mapping Initialization: Using RGB-D Images and Camera Parameters

It Is Okay If You Don't Know What You Like. We Do (feat. Deep Recommendation Algorithms)

10 Machine Learning, Data Science, and Deep Learning Courses for Programmers in 2020

10 Computer Vision Startups on Product Hunt with the Most Upvotes

10 Best Entry Level Machine Learning Tutorials

10 Best + Free Machine Learning Courses Collection

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps