Segment SEO

segmentseo

Startups of The Year 2023 Winners: North America 

Beyond the Smile: Decoding Emoji Lengths in JavaScript

This paper is available on arxiv under CC 4.0 license. Authors: (1) Evan Ning, Milton Academy & evan_ning24@milton.edu; (2) David R. Kaeli, Department of Electrical and Computer Engineering & Northeastern University kaeli@ece.neu.edu. Table of Links Introduction Sieve Optimizations Mathematical Optimization Implementation Results and Discussions Conclusions, Source Code & References 5. Results and Discussions Our Sieve algorithm implementation uses the POSIX threads C++ libraries [7]. The performance study was carried out on the system described in Table 2. By leveraging a high-performance computing cluster, our implementation aims not just to sieve primes efficiently, but to push the boundaries of what's possible in large-scale prime enumeration. The computation time is measured while varying the number of threads and the value N, which specifies the range that we are sieving to. The memory usage is 1,000,000 x 4B x 2 = 8MB for the two A_one and A_five arrays. Inspect the graph on the left, the computation time drops linearly against the number of threads until thread number = 10. In the right graph, we can see the linear correlation between the value of N and computation time. The result shows that our algorithm is highly scalable. The interplay between memory size and computational speed is a pivotal aspect of algorithm optimization. Our second set of experiments delved deeper by exploring how the sizes of the A_one and A_five arrays, which serve as the scratch memory during each thread's operations, influence the algorithm's overall performance. When the sizes of the scratch memory arrays increased, the number of iterations the program needs to complete its task decreased. Using a larger memory pool, more prime sieving results can be held at once, enabling more extensive sieving in a single pass. Thus, fewer passes or iterations are required to achieve the end goal. Our experiments ranged from using a tiny scratch memory size of 100k, suitable for environments with stringent memory restrictions, to a spacious 100M, tailored for systems where memory is abundant. The goal was to find a balance - a size that minimizes the iteration count, while also being mindful of memory usage constraints. The results from these experiments have shown some very interesting findings. At first glance, one might assume that a scratch memory size of 100M, the largest in our experiments, would achieve the best performance in terms of computation time. However, the performance was poor when using this configuration. We believe it is an attribute of the intrinsic memory constraints of what the parallel node can allocate efficiently. When the memory exceeds this limit, the operating system swaps pages, which is causing significant delays. This swapping of memory pages between the RAM and the disk is notoriously slow, introducing significant delays in execution. Hence, while having a large scratch memory may reduce iterations, the overheads introduced due to page swapping can negate those benefits. For the 100k array size, where each thread uses the least amount of memory, we ran into different challenges. The execution time grew as the number of threads increased, an anomaly unique to this configuration. A higher thread count implies that each thread's designated segment shrinks. When these segments approach, or dip below, the size of a cache line, cache synchronization becomes mandatory among threads. This synchronization can lead to stalls, as cache lines are evicted and fetched repeatedly, impacting performance. The best performance is achieved by 1M array size, as this amount of scratch memory can avoid cache collisions and page swapping. Overall, the cache and memory play a significant role in the overall performance. Choosing the right scratch memory size is a critical parameter to achieve good multithread performance.

Memory Efficient Multithreaded Incremental Segmented Sieve Algorithm: Results and Discussions

This paper is available on arxiv under CC 4.0 license. Authors: (1) Evan Ning, Milton Academy & evan_ning24@milton.edu; (2) David R. Kaeli, Department of Electrical and Computer Engineering & Northeastern University kaeli@ece.neu.edu. Table of Links Introduction Sieve Optimizations Mathematical Optimization Implementation Results and Discussions Conclusions, Source Code & References 4.0 Implementation In Figure 3, we provide a diagram of our multithreaded Sieve algorithm's execution flow when utilizing multiple threads. We show the thread structure and flow of the information. In Figure 4 we provide pseudocode for our implementation. In each iteration, the newly found prime numbers are stored in the Prime Number Data Store, which we currently store in a linked list. The primes are also stored in the Sieve Data Store, but only up to the square root of the numbe,r since we only need to sieve up to the square root of the number. This is because for any prime greater than the square root, P2 would be greater than N and there would be nothing to sieve.

Memory Efficient Multithreaded Incremental Segmented Sieve Algorithm: Implementation

This paper is available on arxiv under CC 4.0 license. Authors: (1) Evan Ning, Milton Academy & evan_ning24@milton.edu; (2) David R. Kaeli, Department of Electrical and Computer Engineering & Northeastern University kaeli@ece.neu.edu. Table of Links Introduction Sieve Optimizations Mathematical Optimization Implementation Results and Discussions Conclusions, Source Code & References 2. Sieve Optimizations Our implementation of the Sieve algorithm incorporates several optimizations to improve computational efficiency, selected specifically since we are mapping this algorithm to a multithreaded environment. We also consider a set of mathematical optimizations. 2.1 Memory Optimizations Primes greater than 3 lie only in the form of 6k+1 and 6k+5, which means they are either 1 more or 5 more than a multiple of 6, as shown in Figure 2. By considering only numbers in the form 6k+1 and 6k+5, memory usage can be significantly reduced. 2.2 Divide the Range If we are looking for primes up to a value of n, we can divide this range into segments. For instance, if n=10,000,000 and we have 10 threads, each thread would be assigned to handle 1,000,000 numbers. The main range can be split so that each thread works on contiguous segments of numbers, reducing the complexity of merging the results. 2.3 Multithreaded Tasks A thread, often termed a lightweight process, represents the smallest sequence of programmed instructions that can be managed independently by a scheduler. Each thread is responsible for sieving out multiples of primes in its assigned range. Due to memory constraints, it is not always possible to sieve very large numbers in a single thread. Utilizing consecutive chunks of memory for each thread can indeed help reduce the need for synchronization and thus can prevent memory contention. By allocating a distinct memory range for each thread, we ensure that no two threads are attempting to write to the same memory address at the same time, which is a common cause of race conditions. The segmented sieve approach sieves numbers in segments, making it memory efficient and parallel thread friendly. 2.4 Incremental Sieve An incremental formulation of the sieve generates primes indefinitely (i.e., without an upper bound) by interleaving the generation of primes where the multiples of each prime P are generated directly by counting from the square of the prime in increments of P. The generation must be initiated only when the prime's square is reached, to avoid adverse effects on efficiency.

Memory Efficient Multithreaded Incremental Segmented Sieve Algorithm: Sieve Optimizations

This paper is available on arxiv under CC 4.0 license. Authors: (1) Evan Ning, Milton Academy & evan_ning24@milton.edu; (2) David R. Kaeli, Department of Electrical and Computer Engineering & Northeastern University kaeli@ece.neu.edu. Table of Links Introduction Sieve Optimizations Mathematical Optimization Implementation Results and Discussions Conclusions, Source Code & References 3 Mathematical Optimization One of the novel contributions of our work is to create optimized mathematical operations, where each thread can work efficiently on its designated segment, ensuring that the sieve is executed in a parallelized and optimized manner. For any thread T, let the current starting index in the array be N and the starting value be M, assume we want to sieve for prime number P. • First, we find the greatest multiple of P less than M, denoted as Q. • Then, we find Q (mod6) = q, P (mod6) = p, and ⌊P/6⌋ = R. • Finally, we find the approximate index that a value of Q would be relative to N (this will be less than N). • From there, we can go through every possible value for p and q (note that p can only be 1 or 5). Let id_1 and id_5 be the indices to start sieving in the mod 1 and mod 5 arrays. Table 1 illustrates the operation performed for each p and q condition. Once we know the starting index, we can iterate by adding P to the index (essentially adding 6*P to the number). This guarantees that the new index will be the same remainder mod 6 and that it will be a multiple of P. : Assume that we are using a chunk size of 6,000 values and using 5 threads (0-4). Also assume that we are currently iterating through the group 12,000-18,000, which is mapped onto thread 2 of 5. The index range would be 400-599, and we would sieve from 14,400 – 15,599. Let us sieve for the prime 13. The value of R would be 2. The smallest multiple of 13 less than 14,400 is 14,391. 14,391 mod 6 = q = 3, and 13 mod 6 = p = 1. id_1 and id_5 would be 400-2 = 398, as 14,391 would be between those two values (14,389 and 14,393). id_1 = 398 + 4*2 + 1 = 407, and id_5 = 398 + 2*2 = 402. Converting these numbers back to integers, we see that we start sieving from 407*6 + 1 + 12,000 = 14,443 and 402*6 + 5 + 12,000 = 14,417, which we can confirm are multiples of 13 and are 1 and 5 mod 6, respectively. Example This method reduces redundant calculations, simplifies each step’s mathematical operation, and ensures that each thread is not just parallel but also efficiently parallel, making the best use of the computational resources.

Memory Efficient Multithreaded Incremental Segmented Sieve Algorithm: Mathematical Optimization

This paper is available on arxiv under CC 4.0 license. Authors: (1) Evan Ning, Milton Academy & evan_ning24@milton.edu; (2) David R. Kaeli, Department of Electrical and Computer Engineering & Northeastern University kaeli@ece.neu.edu. Table of Links Introduction Sieve Optimizations Mathematical Optimization Implementation Results and Discussions Conclusions, Source Code & References 1. Introduction Prime numbers are fundamental in number theory and play a significant role in various areas, from pure mathematics to practical applications. For example, public key encryption techniques, especially in cryptography (e.g., the Rivest Shamir Adelman encryption scheme (RSA) [1]), rely heavily on the novel properties of prime numbers. The degree of difficulty in factorizing very large numbers which are products of two prime numbers ensures the security of data encrypted using these methods. Even though prime numbers are fundamental in number theory, determining the primality of large numbers or generating long lists of prime numbers can be computationally challenging as primes are pseudorandomly distributed. Thus, despite several conjectures and theorems that lend insight into the distribution of prime numbers, there is no formula to compute the next prime. The prime sieve algorithm, more formally known as the "Sieve of Eratosthenes," is a well-known and efficient technique used to find prime numbers [2]. With the growing computational needs associated with prime enumeration tasks, traditional implementations of the Sieve algorithm remain a computational bottleneck, especially given the advances in multi-core processors. As the rate of data grows and computational demands rise, harnessing the power of multithread processing becomes imperative. Over the years, there have been prior studies that have delved into the challenge of parallelizing the Sieve of Eratosthenes. Some of these approaches try to exploit the capabilities of multicore processors, breaking down the computation task into chunks [3]. This approach lends itself to using parallelization libraries and middleware, including OpenMP (shared memory) and MPI (inter processor message queues) [4]. Some of the common drawbacks and challenges of parallel implementations are that too many threads accessing shared memory can lead to memory contention, causing bottlenecks and limiting the performance gains from parallelism. Furthermore, prior work has not adequately addressed scalability issues, such that when the prime number being searched for is very large, it eventually can exceed the limits of memory. Additionally, more optimizations can be made within each iteration with improvements in mathematical algorithms, which can lead to significant savings. We introduce a multithreaded implementation of the Segmented Sieve [3]. In our implementation, instead of handling large prime ranges in one iteration, the sieving process is broken down incrementally, which theoretically eliminates the challenges of working with large numbers, and can reduce memory usage, providing overall more efficient utilization over extended computations. In this work, we also explored some mathematical optimization to speed up computation. The code is implemented in POSIX threads C++ [6] and tested on a high-performance multi-core cluster [7]. We have been able to achieve a significant reduction in prime enumeration time, as well as a significant reduction in the associated memory footprint. 1.1 Sieve of Eratosthenes Prime numbers are hard to compute, which makes them indispensable in various fields, especially in cryptography. The security of many encryption systems depends on the computational difficulty of factorizing large numbers. The "Sieve of Eratosthenes," is an old, yet efficient, algorithm used to find all prime numbers up to a specified integer. The algorithm works by iteratively marking the multiples of each prime number given a range. The algorithm begins by creating a list of consecutive integers, from 2 up to n: (2, 3, 4, ..., n). Next, we set the current number, P, to the value 2. Then, remove all multiples of P beginning at P2 from the list, as all multiples of P less than P2 would have been removed in previous iterations already. For instance, the first time we run the algorithm, we remove all even numbers (except 2). Then, we remove all multiples of 3 (except 3), starting from 9 as 6 has already been removed, and so on. After each removal, we find the next number in the list after P that has not been removed. We set this new number to P and repeat the process. The Sieve of Eratosthenes can be expressed in pseudocode [5], as shown in Figure 1. The Sieve of Eratosthenes is especially efficient for computing all prime numbers up to a certain limit. Its time complexity is O(n log log n), making it one of the most efficient ways to find all primes up to a given limit n.

Memory Efficient Multithreaded Incremental Segmented Sieve Algorithm: Introduction

Imagine you're in the taxi business, cruising through the competitive landscape where companies have to make a big choice – whether to focus on regular folks (B2C) or on businesses (B2B). It's a tough call because most of the industry, like 60 to 70%, goes after regular customers. You might be wondering why more companies aren't jumping into the business customer pool. Let's take a peek at the taxi world, ruled by big players like , , , and Yandex Taxi. We'll see how they deal with regular customers who just want a quick, affordable, and reliable ride. On the flip side, there are corporate customers with a whole different wishlist – they're into service integration, detailed billing, and top-notch safety measures. It's like they have a whole set of demands because of their more complicated needs. Uber Lyft Bolt Now, if your company is thinking about leaning towards or away from the business side, this comparison is gold. Knowing what each group wants is key to customizing your services, sorting out how things run, and planning your marketing game. It's like having a map to navigate the different roads of customer demands. B2C segment Alright, picture this – you're in the hustle and bustle of B2C commerce, where businesses are all about selling stuff directly to regular folks like you and me. It's like a big party with all kinds of shops – from the local stores around the corner to those fancy online shops and even your favorite diners. They all want to be buddies with us, the customers, trying to make a direct connection. Now, for these businesses, the main goal is to make us happy by solving our problems or meeting our needs. They want their stuff to be not just cool but also competitive in the market. And here's the trick – they don't have to be the best at everything. What they need is their special thing, like a "secret ingredient" that makes people pick them. Take the ride-hailing gang, for example. Us regular folks care about the cost, how fast and reliable the ride is, the quality of the service, how easy the app is to use, and any cool extras that make the service even better. It's like a global rule – no matter where you are, everyone wants the most bang for their buck. That's the name of the game, everywhere. Imagine you're picking a ride, and there are two types of folks on your mind. First, some dig the VIP treatment – willing to shell out a bit more for a swanky experience. They crave a ride that screams quality. Then, there's the crew who's all about personalization. They want their ride to vibe with them, maybe choosing the tunes or deciding if they're down for a chat. Now, some companies get it. Like Wheely – they're all about rolling out the red carpet, focusing on those high-end customers. On the flip side, there's Gett, letting folks in Russia tailor their ride, choosing everything from the music to their preferred chat level. But here's the cool part – Yandex Go wants to be the superhero for everyone. They offer different service levels, from Economy to Premium, trying to catch the eye of all kinds of customers. No matter who they're targeting, these companies are on a mission to make us regular folks happy. It's like a dance, always switching things up to keep us, the customers, at the center. It's pretty awesome how each of them has their own way of making us feel special. B2B segment In the B2B (Business-to-Business) sector, companies primarily cater to other businesses, including manufacturers, wholesalers, and providers of business services. This model involves a transactional relationship where the end consumer is typically another business, acting as an intermediary. In the B2B realm, the focus shifts to creating value for business partners, rather than directly for individual consumers. For instance, in the taxi service industry, engaging B2B clients often entails offering transportation services to corporate employees rather than enabling other businesses to resell services. The emphasis here is on understanding and fulfilling the unique needs of business clients. A critical first step in addressing the B2B segment is to categorize it into sub-segments, each with distinct needs: (Small and Medium Enterprises): Representing the majority of businesses globally, SMEs usually have a smaller number of employees (less than 250). SMB/SME These are bigger companies with more employees and established internal processes and policies to manage operations effectively. Large enterprises: Addressing the needs of each segment requires a nuanced approach. For SMEs, the strategy might resemble B2C tactics due to their size and structure. Conversely, when dealing with large enterprises, it's crucial to tailor value propositions to internal decision-makers within the company, often valuing their needs as equal or sometimes more critical than the end users (passengers) themselves. SMB segment When it comes to small and medium-sized businesses (SMBs), they use taxi apps a bit differently. The person who decides in these businesses is often someone who either uses the app themselves or hears feedback from employees using it. Unlike apps made for regular folks, the ones for businesses need to think about what the company needs and what the actual users want. Getting this balance right is super important for making products that work well and get good results for the business. These business decision-makers might use the app for both work and personal stuff, trying to mix it into their business when they need to. Why focus on SMBs: If you're already doing well with regular customers (B2C), you don't need extra stuff to get into the SMB market. Companies might use different ways to get business clients, like making it easy for them to join or doing marketing to bring them in. And the cool part? You don't need to grow your team much to handle this new business. So, if your app doesn't match what regular customers love, these business clients might leave. But making things better for regular customers will also make the SMB folks happy. Challenges of the SMB segment Each joining business likely contributes a small incremental revenue due to the small size of the businesses, requiring a significant volume of companies to make a substantial impact. Instead of creating solutions with direct consideration of the end-user, developers must navigate through a complex network of business needs, stakeholder expectations, and profitability goals. The ultimate aim? To build a product that not only satisfies the immediate needs of the business but also supports its long-term strategic objectives. However, this business-centric focus does not diminish the importance of the end-user. On the contrary, understanding and addressing the end-user's issues are critical to the product's success and, by extension, the business outcomes it seeks to achieve. Moreover, feedback mechanisms inherent in B2B and B2B2C models add another layer of complexity. In direct-to-consumer (DTC) sales models, feedback can be direct and immediate, with users directly conveying their impressions, issues, or suggestions to the product team. In contrast, in B2B and B2B2C models, end-user feedback often goes through several layers—be it through their organizations, direct supervisors, or indirect channels—before reaching developers. This indirect feedback path can dilute the clarity of the user's voice, making it more challenging to identify and meet their true needs and preferences. Integrating direct user interaction tools into the product, such as interactive surveys, in-app feedback modules, or user forums, facilitates direct communication. This direct dialogue enables the collection of first-hand user feedback, providing valuable insights into their experience and needs. Enterprise segment Unlike small and medium-sized businesses (SMBs), getting into the enterprise market means dealing with big companies. It's not just about having a good taxi app; you need to connect with the people inside these companies, convince them, and make sure your app meets their needs. But here's the catch: the folks who decide on buying or developing the product might have different ideas than the ones actually using it. They might like features that seem cool but don't really matter to the people who use the app every day. To win over big clients, you've got to think about what the decision-makers want. It's not as simple as offering a cheap and reliable taxi app. You need to solve their specific problems, like giving them negotiated rates, a system to manage expenses, and tools to handle employees. Dealing with big companies also means spending a lot of money and having a special team to meet their unique needs, which might include customizing the product. How big companies work Most big companies have someone or a whole team making decisions. For example, a transportation department would: Figure out what transportation the company needs. Find transportation partners. Make sure costs are predictable and the terms are good. Run daily operations and make sure everyone who needs access has it. Manage and cut down on costs. But going after big companies has some good things: These companies often have branches in other countries or employees traveling for work. A taxi service that works internationally can make things way easier for their global business. Big companies need a taxi service that can do it all – local rides, long-distance trips – to fit all their needs. Working with big companies usually brings in a lot of money. Just one contract can mean getting hundreds or even thousands of clients. Lots of clients still want the same kinds of things. There's a good chance that not many others in your industry are focusing on big companies. Big companies aren't expecting fancy designs like you see in regular customer apps. If your product already looks modern, that could make you stand out in this market. For example, here's what some corporate platforms for booking taxis and flights look like…) Challenges include: Entering this segment requires your company to invest in several directions: Market research to assess its real size Hiring a team of experts Creating products and operational processes ready for B2B Meeting specific customer needs here and there Moreover, your company needs to understand the specific problems of these individuals in your industry. For the taxi market, typically relevant aspects include: Contractual, predictable rates: Unlike the B2C market, where service prices can fluctuate based on current demand and supply, most large companies expect fixed rates for the duration of the contract. Centralized expense management: Your product should provide centralized reporting and billing on agreed terms, ideally ready for integration with internal expense accounting systems. Centralized employee management: Provide the decision-maker with tools for centralized employee management within the service, ideally through integration with the internal HR system. Centralized expense management: A tool that allows limiting employees' use of trips according to internal policies. As an example, consider the key advantages global mobility platform  uses to attract the corporate segment in the UK: And here we come to the main challenges: First, get ready because stepping into this industry means you'll need a team of pros in sales and customer service. They'll be the experts in talking and keeping good relationships with big companies. Second, focus on making a product that suits the folks who make decisions in the company. Create something that helps manage the company's profile within your product. Third, remember, quick and small approaches usually don't work with big companies. They want your product to be perfect from the get-go, not getting better bit by bit. So, you'll need to put in a bunch of money upfront, way before you get the first big company to use your product Here's the lowdown: sometimes, a partner might ask for things that seem totally new and unexpected. You didn't plan on making these for regular customers or the businesses already using your app. But guess what? 80% of the time, those requests might not be that unique. It's worth digging into the market more to check if other folks need the same thing. For the other 20%, you've got a choice to make. Stick to your regular product and maybe lose this client, or make your product fit whatever they want. But here's the real deal: going all customizable can cost you more in the long run. The upside? If the top management in the company is happy with the tools in your product, little hiccups in your main product (like not finding a driver twice for employee X) won't make them ditch the deal. And in the taxi world, where a perfect ride isn't guaranteed, that's a win. But here's a heads-up: don't let the top management down. Another mess-up in getting a ride might make the travel manager think twice about keeping you around. So, get to know these VIPs and go the extra mile to make sure their trips (or any other deals) go smooth. Conclusion Instead of a lengthy conclusion, I propose three main takeaways from this article: Big companies want a full-fledged product from the get-go. They expect solutions not just for the basics but for different jobs in the company. And that often means tweaking the product, which can jack up the costs. Top management at companies want top-notch security in your product. That means spending more on checks, following rules, and maybe sticking to providers that meet their strict standards. Solving problems for big companies can spill over into regular people's lives, opening up new chances in the regular customer market.

Why Product-Marketers Ignore the Goldmine of B2B Segments: Lessons From the Taxi Industry

Uni-OVSeg: A Step Towards Efficient and Bias-Resilient Vision Systems

Authors:
(1) Zhaoqing Wang, The University of Sydney and AI2Robotics;
(2) Xiaobo Xia, The University of Sydney;
(3) Ziye Chen, The University of Melbourne;
(4) Xiao He, AI2Robotics;
(5) Yandong Guo, AI2Robotics;
(6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence;
(7) Tongliang Liu, The University of Sydney. Table of Links Abstract and 1. Introduction 2. Related works 3. Method and 3.1. Problem definition 3.2. Baseline and 3.3. Uni-OVSeg framework 4. Experiments 4.1. Implementation details 4.2. Main results 4.3. Ablation study 5. Conclusion 6. Broader impacts and References A. Framework details B. Promptable segmentation C. Visualisation 2. Related works Generic segmentation. Given an image, segmentation of specific visual concepts has remained an ongoing research topic in computer vision, as indicated by the extensive literature on it [23, 33, 44]. Generic segmentation mainly includes semantic segmentation [12, 44, 77, 79], instance segmentation [4, 7, 23], and panoptic segmentation [11, 11, 33, 56], related to different levels of granularity. In more detail, semantic segmentation [12, 20, 28, 38, 78] aims to assign a label to each pixel of the input image, according to their respective semantic classes. In addition, instance segmentation [53, 54, 59] attempts to distinguish different object instances of the same semantic class. Panoptic segmentation [9, 57, 74, 75] combines the characteristics of semantic segmentation and instance segmentation. Following a close-vocabulary assumption, previous works only can predict a predefined set of object categories. In this paper, we aim to build an advanced open-vocabulary segmentation system, which can categorise objects and stuff from an open set of vocabulary in the real world. Vision foundation models. Recent advancements in visual foundation models have diversified optimisation techniques across various learning paradigms. These developments range from vision-only pretraining [2, 24, 25, 61, 62] to joint vision-language pre-training [30, 48, 73], and extend to multi-modal frameworks that integrate visual prompting [1]. A prime example of this evolution is SAM [34], which shows the potential of extensive training for general segmentation, offering impressive generalisability and scalability. Despite its impressive capabilities, SAM cannot categorise predicted masks into different semantic classes, which is limited by the supervision of the image-mask pairs. More recently, Semantic-SAM [36] unifies different sources of human-annotated segmentation datasets and augments SAM by adding semantic labels and increased levels of granularity. In our work, our aim is to develop a more flexible vision foundation model, which can be trained with unpaired mask-text supervision (e.g., independent imagemask and image-text pairs) and can be easily adapted to different segmentation tasks. Open-vocabulary segmentation. Open-vocabulary segmentation counters the constraints of closed-vocabulary segmentation by allowing the segmentation of a diverse range of classes, even those unseen during training [21, 67, 68, 80]. Existing works [17, 66, 76] leverage the pretrained vision-language models (e.g., CLIP [48] and ALIGN [30]) to perform open-vocabulary segmentation. Most open-vocabulary segmentation methods commonly utilise human-annotated supervision (i.e., the image-masktext triplets) to generalise the capability of vision-language models from the image level to the pixel level. To reduce the dependency on this labour-intensive supervision, some weakly-supervised methods are proposed to use only text supervisions [46, 65]. They learn to group image regions into shaped segments, but struggle to distinguish different instances with the same semantic class and the segmentation performance is unsatisfactory [64, 85]. This dilemma drives our pursuit of more advanced open-vocabulary segmentation framework, where the aim is to enjoy as low annotation cost as possible and simultaneously achieve significant performance. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Zhaoqing Wang, The University of Sydney and AI2Robotics; (2) Xiaobo Xia, The University of Sydney; (3) Ziye Chen, The University of Melbourne; (4) Xiao He, AI2Robotics; (5) Yandong Guo, AI2Robotics; (6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence; (7) Tongliang Liu, The University of Sydney. Authors: Authors: (1) Zhaoqing Wang, The University of Sydney and AI2Robotics; (2) Xiaobo Xia, The University of Sydney; (3) Ziye Chen, The University of Melbourne; (4) Xiao He, AI2Robotics; (5) Yandong Guo, AI2Robotics; (6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence; (7) Tongliang Liu, The University of Sydney. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2. Related works 2. Related works 3. Method and 3.1. Problem definition 3. Method and 3.1. Problem definition 3.2. Baseline and 3.3. Uni-OVSeg framework 3.2. Baseline and 3.3. Uni-OVSeg framework 4. Experiments 4.1. Implementation details 4.1. Implementation details 4.2. Main results 4.2. Main results 4.3. Ablation study 4.3. Ablation study 5. Conclusion 5. Conclusion 6. Broader impacts and References 6. Broader impacts and References A. Framework details A. Framework details B. Promptable segmentation B. Promptable segmentation C. Visualisation C. Visualisation 2. Related works Generic segmentation. Given an image, segmentation of specific visual concepts has remained an ongoing research topic in computer vision, as indicated by the extensive literature on it [23, 33, 44]. Generic segmentation mainly includes semantic segmentation [12, 44, 77, 79], instance segmentation [4, 7, 23], and panoptic segmentation [11, 11, 33, 56], related to different levels of granularity. In more detail, semantic segmentation [12, 20, 28, 38, 78] aims to assign a label to each pixel of the input image, according to their respective semantic classes. In addition, instance segmentation [53, 54, 59] attempts to distinguish different object instances of the same semantic class. Panoptic segmentation [9, 57, 74, 75] combines the characteristics of semantic segmentation and instance segmentation. Following a close-vocabulary assumption, previous works only can predict a predefined set of object categories. In this paper, we aim to build an advanced open-vocabulary segmentation system, which can categorise objects and stuff from an open set of vocabulary in the real world. Generic segmentation. Vision foundation models. Recent advancements in visual foundation models have diversified optimisation techniques across various learning paradigms. These developments range from vision-only pretraining [2, 24, 25, 61, 62] to joint vision-language pre-training [30, 48, 73], and extend to multi-modal frameworks that integrate visual prompting [1]. A prime example of this evolution is SAM [34], which shows the potential of extensive training for general segmentation, offering impressive generalisability and scalability. Despite its impressive capabilities, SAM cannot categorise predicted masks into different semantic classes, which is limited by the supervision of the image-mask pairs. Vision foundation models. More recently, Semantic-SAM [36] unifies different sources of human-annotated segmentation datasets and augments SAM by adding semantic labels and increased levels of granularity. In our work, our aim is to develop a more flexible vision foundation model, which can be trained with unpaired mask-text supervision (e.g., independent imagemask and image-text pairs) and can be easily adapted to different segmentation tasks. Open-vocabulary segmentation. Open-vocabulary segmentation counters the constraints of closed-vocabulary segmentation by allowing the segmentation of a diverse range of classes, even those unseen during training [21, 67, 68, 80]. Existing works [17, 66, 76] leverage the pretrained vision-language models (e.g., CLIP [48] and ALIGN [30]) to perform open-vocabulary segmentation. Most open-vocabulary segmentation methods commonly utilise human-annotated supervision (i.e., the image-masktext triplets) to generalise the capability of vision-language models from the image level to the pixel level. To reduce the dependency on this labour-intensive supervision, some weakly-supervised methods are proposed to use only text supervisions [46, 65]. They learn to group image regions into shaped segments, but struggle to distinguish different instances with the same semantic class and the segmentation performance is unsatisfactory [64, 85]. This dilemma drives our pursuit of more advanced open-vocabulary segmentation framework, where the aim is to enjoy as low annotation cost as possible and simultaneously achieve significant performance. Open-vocabulary segmentation. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Stories, News & Mentions about SEGMENTSEO

Segmentseo’s Stories on HackerNoon