For engineers who perfected 2D data pipelines, moving to 3D can be a shock. What was once a solved problem of drawing 2D boxes has become a complex battle against sparse point clouds, clunky visualization, and ambiguous classifications.
A “simple” process often includes numerous challenges and hidden bottlenecks. How can they be solved, and what is the optimal solution for them? Let's try to find it “in the field”, by exploring the expertise and projects of the leading labeling provider and 3D platform.
Where the bottlenecks arise
3D labeling unlocks powerful capabilities for autonomous systems across various industries, including automotive, robotics, construction, and healthcare. So, where do a unique set of challenges arise? Point-cloud data is inherently unstable: reflections from glass or wet surfaces, weather-induced noise, and constantly moving objects can distort the scene or create “phantom” structures that never existed. Large-scale scans add additional complexity; aligning sensors, synchronizing frames, and maintaining geometric consistency across massive datasets requires meticulous control. Even more importantly, 3D annotation is more than marking points; it’s about correctly interpreting context. A LiDAR system doesn’t know that a storefront is transparent, that a cyclist may suddenly change posture, or that two nearly identical objects should be understood differently in the real world. Another frequent bottleneck appears long before annotation even begins: unclear goals. The level of detail, the attributes to capture, and the semantic relationships between objects must all be defined up front. Whether a bicycle needs detailed segmentation or a single cuboid with a movement vector shapes the entire architecture of the model. Individually, these challenges are typical for any complex point-cloud project. Together, they can slow down or completely block an AI pipeline. All this can be solved by a solution built on three pillars: experienced specialists, proper tooling, and clearly defined requirements.
Joint laboratory of humans and powerful tools
The announced partnership between Keymakr, a team specializing in high-precision labeling, and Segments.ai, a platform designed for enterprise-grade labeling of 3D, demonstrates how the combination of advanced tooling and human expertise elevates 3D data workflows and solves hidden bottlenecks. Working across 3D projects of different complexity and scale, the Keymakr and Segments.ai teams share their most interesting cases and creative solutions to challenging problems.
Solving weather-related point cloud distortions
One of the practical examples of solving bottlenecks came from weather-related noise. During a LiDAR-based infrastructure mapping project, heavy fog and drizzle caused unstable point density, making it difficult to distinguish real objects from atmospheric artifacts. Segments.ai’s built-in filters handled the first layer of cleanup by removing low-intensity and isolated points. However, the weather distortions were too heterogeneous for automatic tools alone: fog created diffuse point clusters, drizzle generated sharp but random “sparkles,” and reflective road surfaces added another layer of complexity. To address this, the Keymakr team applied a multi-step review workflow:
- Pattern-based visual analysis: annotators manually identified recurring noise signatures (fog patches, reflective glare, moisture artifacts).
- Iterative threshold tuning: Segments.ai engineers adapted filtering rules to environmental interference rather than generic noise.
- Layered cleanup: the dataset was filtered, reviewed, corrected, and then passed through an adjusted filter a second time.
- Cross-checking in 3D space: operators validated and cleaned regions in multiple angles and depths to ensure legitimate objects were preserved.
This human-machine loop dramatically improved the dataset. Even more importantly, it produced a repeatable workflow for weather-affected scans, a process both teams now use as a template when working with LiDAR captured in non-ideal conditions.
"Keymakr’s team makes full use of our platform to optimize speed and quality across complex projects. Their proactive communication and expert feedback help us continuously improve the platform. It’s always great to see the results they’re achieving for some of the customers we support together."
Otto Debals, CEO at Segments.ai
Restoring time synchronization in custom LiDAR formats
Another case involved a technical bottleneck related to time-based point cloud data. Unlike video, where frames are discrete and easy to align, point clouds are measured over time, meaning not a specific frame but a continuous recording in seconds. The client had a custom data format, and during upload, the frame-to-timestamp alignment was overlooked, causing the file to display incorrectly. As a result, the platform couldn’t recognize or process the dataset. The challenge was resolved jointly by the Keymakr technical team and Segments.ai engineers, who adjusted the time mapping logic and restored full compatibility. This collaboration salvaged the dataset and led to improvements in the platform’s time synchronization capabilities for future projects.
“We’re always looking for creative ways to use our partners’ tools, as our clients often request non-standard and complex tasks,” says Zoia Boiko, PM at Keymakr. “Working with unconventional formats and challenging cases helps Segments improve the platform, creating a better environment for everyone. So, we grow together, and our team finds its tools indispensable.”
Thus, Keymakr contributes the human component, attentiveness, creativity, and a deep understanding of how real-world data behaves. Segments brings the streamlining element - powering workflows, engineering solutions, and making labeling possible. When the annotation team tests real “field” cases on the platform, both companies create an environment where the tool adapts to the user and, in turn, shapes the workflows accordingly.
Resolving structural breaks in 3D road markings
One more bottleneck emerged in projects requiring highly accurate 3D road markings. In 2D workflows, a line can be interrupted and continued without affecting the overall structure, but in 3D, this is dangerous. Even a tiny artificial “gap” can be misinterpreted by an autonomous system as a lane ending. To address this, the Keymakr team developed a workflow based on continuous polylines with automated overlap removal. Each road marking was drawn as a single continuous object, after which Segments.ai automatically removed only the segments that fell within overlap zones. This prevented the model from reading overlaps as structural breaks, significantly improving consistency.
“In 3D, you can’t afford accidental gaps. A single break can distort the geometry of the entire scene and mislead an autonomous system. Our goal was to create a workflow where continuity is guaranteed by design, not left to chance,” — Zoia Boiko, PM at Keymakr.
The process was further strengthened by working inside an aggregated 3D scene using Segments.ai’s merge mode. Instead of thousands of disconnected frames, annotators worked with a unified, dense representation of the entire route, complete with edges, splits, merges, curbs, signs, and guardrails. This holistic view made road geometry clearer, transitions more logical, and line smoothing far more accurate.
After building the global “skeleton” of the annotation in merge mode, operators switched back to frame-level editing only to refine overlaps, without altering the underlying geometry. This kept datasets compact and avoided errors common in extensive multi-frame edits. What began as a structural bottleneck became a scalable method now reused across many 3D LiDAR projects.
All these cases from Keymakr and Segments.ai can be seen as examples of a joint laboratory where technology evolves in tandem with human expertise. While 3D labeling is often viewed as a purely technical process, in reality, it reflects something much deeper, teaching machines to understand the real world. Every point cloud captures a fragment of reality, and it is the human operator who determines what is meaningful, what is noise, and what truly exists in the scene. Together, these cases show that the future of 3D data work lies in a human-in-the-loop model, where people guide the system, shape the tools through real field challenges, and ultimately ensure reliability, accuracy, and trust in the final output.