Authors:
(1) Kun Lan, University of Science and Technology of China;
(2) Haoran Li, University of Science and Technology of China;
(3) Haolin Shi, University of Science and Technology of China;
(4) Wenjun Wu, University of Science and Technology of China;
(5) Yong Liao, University of Science and Technology of China;
(6) Lin Wang, AI Thrust, HKUST(GZ);
(7) Pengyuan Zhou, University of Science and Technology of China. Table of Links Abstract and 1. Introduction 2. Related Work 3. Method and 3.1. Point-Based rendering and Semantic Information Learning 3.2. Gaussian Clustering and 3.3. Gaussian Filtering 4. Experiment 4.1. Setups, 4.2. Result and 4.3. Ablations 5. Conclusion and References 3.2. Gaussian Clustering During experiments, we observed that employing 2D segmentation maps as the sole guide for learning 3D semantic information may lead to inaccuracies in the semantic information of some 3D Gaussians. These inaccuracies manifest either as 3D Gaussians approximating an initial state of uniform distribution across all categories or as exhibiting similar probabilities in a limited number of categories. To address this issue, and considering that objects are continuously distributed in space, we posit that each 3D Gaussian should typically be classified within the same category as other 3D Gaussians located within a certain proximity. To remedy the inaccuracies in semantic information, we refer to the KNN clustering algorithm. For a 3D scene with pre-learned semantic information, we initially retrieve the object code, denoted as o, of each 3D Gaussian used to represent the scene. These codes then undergo softmax processing to deduce the probability distribution of each 3D Gaussian across various categories. 3D Gaussians with maximum probability values max(sof tmax(o)) < β are selected. Finally, we fed the object codes of these selected 3D Gaussians along with their center coordinates into KNN for clustering. For a query 3D Gaussian, we calculate its distance from the surrounding 3D Gaussians, and the k 3D Gaussians closest in distance are selected, the object code of the query Gaussian is set to the mean of these 3D Gaussians’ object code. 3.3. Gaussian Filtering During experiments, We also found that after 3D semantic information learning and Gaussian clustering, some 3D Gaussians not belonging to the object intended for segmentation were incorrectly segmented out. We observed that these erroneously segmented 3D Gaussians are spatially distant from the rest of the segmented 3D Gaussians, as shown in Fig. 4(a). Therefore, we employ a statistical filtering algorithm similar to that used in point cloud segmentation to solve this problem. For each segmented Gaussian, we calculate its average distance D from the neighboring 3D Gaussians. Then, we compute the mean µ and variance σ of these average distances. Finally, we remove those 3D Gaussians whose average distance D > µ + σ from the current segmentation results. This paper is available on arxiv under CC 4.0 license. Authors: (1) Kun Lan, University of Science and Technology of China; (2) Haoran Li, University of Science and Technology of China; (3) Haolin Shi, University of Science and Technology of China; (4) Wenjun Wu, University of Science and Technology of China; (5) Yong Liao, University of Science and Technology of China; (6) Lin Wang, AI Thrust, HKUST(GZ); (7) Pengyuan Zhou, University of Science and Technology of China. Authors: Authors: (1) Kun Lan, University of Science and Technology of China; (2) Haoran Li, University of Science and Technology of China; (3) Haolin Shi, University of Science and Technology of China; (4) Wenjun Wu, University of Science and Technology of China; (5) Yong Liao, University of Science and Technology of China; (6) Lin Wang, AI Thrust, HKUST(GZ); (7) Pengyuan Zhou, University of Science and Technology of China. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2. Related Work 2. Related Work 3. Method and 3.1. Point-Based rendering and Semantic Information Learning 3. Method and 3.1. Point-Based rendering and Semantic Information Learning 3.2. Gaussian Clustering and 3.3. Gaussian Filtering 3.2. Gaussian Clustering and 3.3. Gaussian Filtering 4. Experiment 4.1. Setups, 4.2. Result and 4.3. Ablations 4.1. Setups, 4.2. Result and 4.3. Ablations 5. Conclusion and References 5. Conclusion and References 3.2. Gaussian Clustering During experiments, we observed that employing 2D segmentation maps as the sole guide for learning 3D semantic information may lead to inaccuracies in the semantic information of some 3D Gaussians. These inaccuracies manifest either as 3D Gaussians approximating an initial state of uniform distribution across all categories or as exhibiting similar probabilities in a limited number of categories. To address this issue, and considering that objects are continuously distributed in space, we posit that each 3D Gaussian should typically be classified within the same category as other 3D Gaussians located within a certain proximity. To remedy the inaccuracies in semantic information, we refer to the KNN clustering algorithm. For a 3D scene with pre-learned semantic information, we initially retrieve the object code, denoted as o, of each 3D Gaussian used to represent the scene. These codes then undergo softmax processing to deduce the probability distribution of each 3D Gaussian across various categories. 3D Gaussians with maximum probability values max(sof tmax(o)) < β are selected. Finally, we fed the object codes of these selected 3D Gaussians along with their center coordinates into KNN for clustering. For a query 3D Gaussian, we calculate its distance from the surrounding 3D Gaussians, and the k 3D Gaussians closest in distance are selected, the object code of the query Gaussian is set to the mean of these 3D Gaussians’ object code. max(sof tmax(o)) < β k 3.3. Gaussian Filtering During experiments, We also found that after 3D semantic information learning and Gaussian clustering, some 3D Gaussians not belonging to the object intended for segmentation were incorrectly segmented out. We observed that these erroneously segmented 3D Gaussians are spatially distant from the rest of the segmented 3D Gaussians, as shown in Fig. 4(a). Therefore, we employ a statistical filtering algorithm similar to that used in point cloud segmentation to solve this problem. For each segmented Gaussian, we calculate its average distance D from the neighboring 3D Gaussians. Then, we compute the mean µ and variance σ of these average distances. Finally, we remove those 3D Gaussians whose average distance D > µ + σ from the current segmentation results. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Effortless 2D-Guided, 3D Gaussian Segmentation: Gaussian Clustering and Filtering

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

15 Common Types of Unethical Behavior Found in Open-Source Projects

Effortless 2D-Guided, 3D Gaussian Segmentation: Abstract and Introduction

Effortless 2D-Guided, 3D Gaussian Segmentation: Related Work

Effortless 2D-Guided, 3D Gaussian Segmentation: Rendering and Semantic Information

Effortless 2D-Guided, 3D Gaussian Segmentation: Setups, Result and Ablations

15 Common Types of Unethical Behavior Found in Open-Source Projects

Effortless 2D-Guided, 3D Gaussian Segmentation: Abstract and Introduction

Effortless 2D-Guided, 3D Gaussian Segmentation: Related Work

Effortless 2D-Guided, 3D Gaussian Segmentation: Rendering and Semantic Information

Effortless 2D-Guided, 3D Gaussian Segmentation: Setups, Result and Ablations

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps