14 Nov 2024 - Cbus Super CEO Kristian Fok is set to appear before the Senate Economics References Committee today in relation to a range of matters, including ASIC’s latest allegations of failed insurance claim handling.

Cbus faces Senate grilling | Finance News Network

Super Micro shares plunged 18% on Wednesday to their lowest level since June of last year after the embattled server maker issued disappointing unaudited financials and failed to provide specifics plans to keep its Nasdaq listing.

Super Micro shares plummet 18% after financial update heightens investor concerns

Pick up this amazing gaming GPU for a discount price, thanks to this MSI deal, which gets you an RTX 4070 Ti Super Expert card for $729.99.

This MSI Nvidia GeForce RTX 4070 Ti Super graphics card deal is incredible

BBC Sport has agreed a new five-year deal to continue broadcasting the Women's Super League.

BBC Sport signs new WSL rights deal until 2030 - Yahoo Sports

Chelsea's Women's Super League game against Manchester United will now be played on 24 November after it was postponed last month. The game had been due to take place on 6 October but was postponed at a week's notice because of a conflict with the Blues' fixtures in the Women's Champions League.

Chelsea-Man Utd rescheduled for November - Yahoo Sports

Putting the beatdown on the Cowboys in Dallas certainly helped the national credibility of the Detroit Lions as top Super Bowl contenders. In fact, the 47-9 blowout lifted the Lions to the top spot in ESPN's ever-adaptive Football Power Index. In the last daily update of…

Huge Week 6 win lifts Lions to the top of ESPN's Football Power Index - Yahoo Sports

During the company's fifth Starship test flight, SpaceX successfully caught its Super Heavy booster using its "Mechazilla" tower.

SpaceX Just Caught Its Starship Booster With Giant "Chopsticks" Arms

Super

super

In this article, I'll show you how to quickly setup pretty URLs in Super with a little shell command and a Node.js script. If you are reading this, you may already be aware that helps you go public with your pages.  What you may not know, is that ugly URLs are And notion's got some ugly URLs. https://super.so https://www.notion.so bad for SEO. It should come as no surprise that the easier a URL is to read for humans, the better it is for search engines - Rand Fishkin Well, color me surprised!  I've never needed to give SEO any thought before, but now it's my job to care!  It's time to accommodate the web crawlers and get our new and pages indexed. help careers So the first step to getting all your links pretty and into Super, is to get all the links off of your notion page.  In the case of the new Hacker Noon help page (http://help.hackernoon.com/), there were links to pretty up in this super form: 157 That's a lot of copy-pasting and clicking around and coming up with pretty URLs and so I decided to open up my chrome browser's network tab and see what was being sent to the server so I could just give it a single payload with all the links in it. DISCLAIMER:  Using any site a way not intended by the developers may result loss data.  Proceed at your own risk!  (maybe start a little test where it won in in of with 't hurt anything:) To to get the network tab, one needs to open the developer console.  On a mac, this is done with cmd+option+i.  You can also reach this by clicking the ellipsis/hamburger menu (three dots, in a column) button, going to More Tools and then finally Developer Console A portion of your window should be covered with a new panel and at the top of that, you should see some tabs.  One of those is called Network and will provide us with a template to use in our own payload to send to Super. You can see Network selected here with the blue underline.  If I click on the clear button 🚫 next to the red circle, it'll clear out any previous network calls so we can clearly see what is sent when we save our pretty URLs. Here we see a bunch of requests that happened, but the POST that goes to a URL called "update" looks interesting for our use case.  (You may need to right-click the columns to select "Method" to be able to show that column)  You can also use the search box with to filter to just those types of calls, if you already know what you are looking for. method:POST If you click on the method and look at the Headers tab that will show up, then scroll to the bottom of that, you will see the request payload. Open that up and see where the stuff you typed into the pretty URL window is. In this cast, there is a field called prettyUrls that is an array of objects with two fields, which ends up just getting the ugly, unique identifier portion of a notion URL, and also a field which is where it will redirect.  So now we know what we are doing, we need an array of objects with all our notion page ids and the URLs they are going to.  Upwards and onwards! pageId url Right-click that method and select if you want to write a node.js script like I did.  Maybe you're into curl or using fetch right in the browser's console. Copy > Copy as Node.js fetch Okay, so you've got this node code in our paste buffer and nowhere to put it!  It's time to make a script and paste it in there. We are going to ignore it for now though because I wrote this article a little out of order and I can't drag things around to reorder them and I'm just gonna roll with it! Let's move on to getting all the notion links off the page.   One way this can be done is by downloading the page and its links with: wget -r -l0 https://help.hackernoon.com/ This took a few minutes and may have been overkill, but when it was done I had a neat little help.hackernoon.com folder with a bunch of HTML files.  I ended up using the HTML files to come up with the pretty URL so maybe not overkill.  Here's a bit of what it looked like in that directory when I listed the files: help.hackernoon.com ls
0109b933c3084e9699b408a73b142e8a  5c115f8cb76445788bb16a4499758cd8  d1edc21b1f2d48d9afa43f4da6d4cc4f
0142ddfc63b94ef9a1ca01975c3d2cab  5c45cf10b99046939df336a23ddae597  d44cdbf1e6d94c0eab9fb5091c249e97
028466d28622424ca1523069faf1d7c6  5f54fe2e3b6641b3b1d49f7b47a961d7  db9dd7e7998141df9705628b03fcd75b
033557c93f714ae092e13e095ca9c2cf  604adc1e960840b782d7f2de2687f719  de55338768674dccbc80909d4ccea58f Cool, so all these piles of alpha-numeric characters are the notion entry to put in the left side of that form. To get the corresponding pretty URL to put on the right-hand side, I peeked inside one of the files and looked for where the header was by searching for h1 There it is with the juicy "Hacker Noon Editor 101" meat/text inside.  Now to start a script to iterate over all these files and parse and kebab-case the h1 headers. I used , which is jQuery for node.js, to do this.  This is how things started to look: the library cheerio fs = ( ); cheerio = ( ); { files = fs.promises.readdir( ); ( file files) { stat = fs.promises.stat(file); (stat.isDirectory()) ; html = fs.readFileSync(file).toString(); $ = cheerio.load(html); contents = $( ).contents()[ ]; (!contents) ; url = contents.data; .log(url);
  }
})(); const require "fs" const require "cheerio" ( )=> ( async const await "./" for const of const await if continue //we just want the html files const const const 'h1' 0 if continue const console So this script was run from inside that help.hackernoon.com so it could find the files easily with the path. Next, we used the function to help us determine if it was a file or directory, as there was one directory to skip. ./ stat _next Maybe you think the is whack, because it is synchronous? You are correct. readFileSync It doesn't matter though, this is just a one-off script to get the job done.  We call on that to be able to pass the wad of HTML text at cheerio and now we can run selectors on it to scrape it. toString() I needed the first h1 so our selector is simply that and we to get the first one, maybe it found more, if it didn't we continue going to the next file.  After that the schema cheerio gives you dictates that to get the text out you will need to ask for the to get the text.  All these get logged out and I could see if things were looking good. [0] .data By the way, one nice way to work on things like this and get quick feedback is to use another package https://nodemon.io/ which can be installed globally to use as a shell command or as a library to be used for your own file watching needs, like if you want to roll your own hot reloading framework for node development.  In this case, my script was called sendIt.js and so I could run in one split pane of my terminal and anytime I saved the file in another split, the file was re-run and I got a handy feedback loop. nodemon sendIt.js Okay, it's time to combine the node.js fetch bit with this loop that gets all the URL mappings. I'm going to plop in a gist now that should get the idea across.  The new code added to our existing code is just adding a Set and then adding our pretty URL objects to that. You may also notice that I am adjusting the pretty URL around line 22. It's there that we modify the h1 text by making it lowercase on line and removing invalid URL characters. Spaces become dashes, double dashes turn back into one dash, strips out the non-dashes and all non-alphanumeric characters.  Next, leading and trailing dashes are removed. /[^0-9a-z-]/gi This could be condensed into less replace calls, but this is readable and this code will only ever run when you want to update your pretty URLs so I am not going to sweat it. Next, around , I skip some files with dots because those were downloaded but don't need mappings, like index.js.  You could also just delete them from the folder of links that were downloaded. line 29 with the body was created by copying the string from the node.js fetch code we pasted in from the developer console into a node REPL and running on that.  Now it's some code, so we can get a chance to take our Set of prettyUrls and get them into our payload. will turn the Set into an array. Line 42 JSON.parse Array.from(prettyUrls) The rest of the fetch is unchanged, save for the body section where I it back into the original format we pasted in.  I save this fetch call as a variable so I can print it and see what any issues are.  There are always issues with the first (through at least the 10th) attempt. JSON.stringify In my case, the lines about stripping our invalid chars, having duplicate entries that were already added to the list of pretty URLs and all that were my issues that caused me to iterate a few times on this.  So there were more console.logs sprinkled throughout this code to help me identify where things went wrong. If the result of the fetch has a and the word OK, you are good. Refresh your pretty URL list and double-check.  If you get a 500 and a Bad Gateway, you probably sent an invalid character in the URL or sent a duplicate entry. status 200 So there you have it, you can use these ideas in a lot of other places than just notion and super.  Just be careful you try out ideas on something else first that you are not worried about screwing up if you send the wrong payload.  The saying "Measure twice, cut once" comes to mind. Anywho, I hope this was helpful.  You can tell me your thoughts about it on Twitter if you really want.  I primarily wrote this to help me get over my writer's block and just publish .  ✌️ something

How to Clean Up Notion URLs: From Ugly URLs to Pretty URLs in Minutes

Authors:
(1) Zhaoqing Wang, The University of Sydney and AI2Robotics;
(2) Xiaobo Xia, The University of Sydney;
(3) Ziye Chen, The University of Melbourne;
(4) Xiao He, AI2Robotics;
(5) Yandong Guo, AI2Robotics;
(6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence;
(7) Tongliang Liu, The University of Sydney. Table of Links Abstract and 1. Introduction 2. Related works 3. Method and 3.1. Problem definition 3.2. Baseline and 3.3. Uni-OVSeg framework 4. Experiments 4.1. Implementation details 4.2. Main results 4.3. Ablation study 5. Conclusion 6. Broader impacts and References A. Framework details B. Promptable segmentation C. Visualisation 5. Conclusion In conclusion, this paper proposes an innovative framework for weakly-supervised open-vocabulary segmentation, named Uni-OVSeg. Using independent image-text and image-mask pairs, Uni-OVSeg effectively reduces the dependency on labour-intensive image-mask-text triplets, meanwhile achieving impressive segmentation performance in open-vocabulary settings. Using the LVLM to refine text descriptions and multi-scale ensemble to enhance the quality of region embeddings, we alleviate the noise in masktext correspondences, achieving substantial performance improvements. Notably, Uni-OVSeg significantly outper forms previous state-of-the-art weakly-supervised methods and even surpasses the cutting-edge fully-supervised method on the Challenging PASCAL Context-459 dataset. This impressive advancement demonstrates the superiority of our proposed framework and paves the way for further research. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Zhaoqing Wang, The University of Sydney and AI2Robotics; (2) Xiaobo Xia, The University of Sydney; (3) Ziye Chen, The University of Melbourne; (4) Xiao He, AI2Robotics; (5) Yandong Guo, AI2Robotics; (6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence; (7) Tongliang Liu, The University of Sydney. Authors: Authors: (1) Zhaoqing Wang, The University of Sydney and AI2Robotics; (2) Xiaobo Xia, The University of Sydney; (3) Ziye Chen, The University of Melbourne; (4) Xiao He, AI2Robotics; (5) Yandong Guo, AI2Robotics; (6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence; (7) Tongliang Liu, The University of Sydney. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2. Related works 2. Related works 3. Method and 3.1. Problem definition 3. Method and 3.1. Problem definition 3.2. Baseline and 3.3. Uni-OVSeg framework 3.2. Baseline and 3.3. Uni-OVSeg framework 4. Experiments 4.1. Implementation details 4.1. Implementation details 4.2. Main results 4.2. Main results 4.3. Ablation study 4.3. Ablation study 5. Conclusion 5. Conclusion 6. Broader impacts and References 6. Broader impacts and References A. Framework details A. Framework details B. Promptable segmentation B. Promptable segmentation C. Visualisation C. Visualisation 5. Conclusion In conclusion, this paper proposes an innovative framework for weakly-supervised open-vocabulary segmentation, named Uni-OVSeg. Using independent image-text and image-mask pairs, Uni-OVSeg effectively reduces the dependency on labour-intensive image-mask-text triplets, meanwhile achieving impressive segmentation performance in open-vocabulary settings. Using the LVLM to refine text descriptions and multi-scale ensemble to enhance the quality of region embeddings, we alleviate the noise in masktext correspondences, achieving substantial performance improvements. Notably, Uni-OVSeg significantly outper forms previous state-of-the-art weakly-supervised methods and even surpasses the cutting-edge fully-supervised method on the Challenging PASCAL Context-459 dataset. This impressive advancement demonstrates the superiority of our proposed framework and paves the way for further research. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Uni-OVSeg: Weakly-Supervised Open-Vocabulary Segmentation with Cutting-Edge Performance

Authors:
(1) Zhaoqing Wang, The University of Sydney and AI2Robotics;
(2) Xiaobo Xia, The University of Sydney;
(3) Ziye Chen, The University of Melbourne;
(4) Xiao He, AI2Robotics;
(5) Yandong Guo, AI2Robotics;
(6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence;
(7) Tongliang Liu, The University of Sydney. Table of Links Abstract and 1. Introduction 2. Related works 3. Method and 3.1. Problem definition 3.2. Baseline and 3.3. Uni-OVSeg framework 4. Experiments 4.1. Implementation details 4.2. Main results 4.3. Ablation study 5. Conclusion 6. Broader impacts and References A. Framework details B. Promptable segmentation C. Visualisation 4.2. Main results Open-vocabulary semantic segmentation. As demonstrated in Tab. 1, our study presents a comprehensive comparison of our Uni-OVSeg against previous works across a range of benchmarks. These include ADE20K (encompassing both 150 and 847 class variants), PASCAL Context (459 and 59 class variants), PASCAL VOC (with 20 and 21 class categories), and Cityscapes. Compared to weakly-supervised methods, Uni-OVSeg exhibits remarkable performance improvements across all evaluated datasets. Specifically, in the more challenging datasets of PASCAL Context-459, Uni-OVSeg not only surpasses its weakly-supervised counterparts but also outperforms the cutting-edge fully-supervised methods, e.g., FC-CLIP. This is indicative of Uni-OVSeg’s superior capability in categorizing a diverse array of semantic classes. Furthermore, in the PASCAL VOC benchmarks (20 and 21 classes), Uni-OVSeg demonstrates a substantial enhancement over state-of-the-art weakly-supervised methods, achieving improvements of 18.3% and 12.2% mIoU, respectively, which demonstrates our Uni-OVSeg captures fine-grained spatial structures. These results elevate the practical applicability of weakly-supervised open-vocabulary segmentation to new heights. Open-vocabulary panoptic segmentation. For openvocabulary panoptic segmentation, we zero-shot evaluate our model on the COCO, ADE20K, and Cityscapes datasets, shown in Tab. 2. Existing weakly-supervised methods only use text supervision, which causes a challenge to discriminate different instances with the same semantic class. To the best of our knowledge, we are the first to learn open-vocabulary panoptic segmentation with weak supervision. Compared to unsupervised methods, we obviously outperform U2Seg by 1.9% PQ, 1.5%SQ, and 4.4% RQ on COCO datasets. Unfortunately, our used image-mask pairs contain multiple granularity masks, such as object-wise and part-wise masks, which is different with the panoptic segmentation datasets. This discrepancy leads to a number of false positive results, limiting our performance. Promptable segmentation. To evaluate the segmentation efficacy of Uni-OVSeg in scenarios involving interactive point and box prompts, we conduct comparative analyses with the SAM-ViT/L model across a range of datasets from diverse domains. For visual prompts, we implement a uniform 20×20 point grid as the interactive point prompt and utilise the actual bounding boxes as box prompts. The segmentation performance is measured using the 1-pt IoU (Oracle) metric across all datasets. The results, as demonstrated in Fig. 3, indicate that Uni-OVSeg exceeds SAM in most datasets when employing point and box prompts. In particular, Uni-OVSeg significantly surpasses SAM in terms of IoU by 31.1%, 20.1%, and 43.2% on the TimberSeg, COCO instance, and PIDray datasets, respectively. Additionally, we provide a visualisation of the segmentation results of both SAM and Uni-OVSeg in Fig. 4. The visualisation reveals that SAM tends to miss larger masks and erroneously categorise areas with unclear boundaries. In contrast, UniOVSeg demonstrates exceptional precision in segmenting objects of varying scales and different stuff. More experimental results are visualisation can be found in the Appendix B and C. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Zhaoqing Wang, The University of Sydney and AI2Robotics; (2) Xiaobo Xia, The University of Sydney; (3) Ziye Chen, The University of Melbourne; (4) Xiao He, AI2Robotics; (5) Yandong Guo, AI2Robotics; (6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence; (7) Tongliang Liu, The University of Sydney. Authors: Authors: (1) Zhaoqing Wang, The University of Sydney and AI2Robotics; (2) Xiaobo Xia, The University of Sydney; (3) Ziye Chen, The University of Melbourne; (4) Xiao He, AI2Robotics; (5) Yandong Guo, AI2Robotics; (6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence; (7) Tongliang Liu, The University of Sydney. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2. Related works 2. Related works 3. Method and 3.1. Problem definition 3. Method and 3.1. Problem definition 3.2. Baseline and 3.3. Uni-OVSeg framework 3.2. Baseline and 3.3. Uni-OVSeg framework 4. Experiments 4.1. Implementation details 4.1. Implementation details 4.2. Main results 4.2. Main results 4.3. Ablation study 4.3. Ablation study 5. Conclusion 5. Conclusion 6. Broader impacts and References 6. Broader impacts and References A. Framework details A. Framework details B. Promptable segmentation B. Promptable segmentation C. Visualisation C. Visualisation 4.2. Main results Open-vocabulary semantic segmentation. As demonstrated in Tab. 1, our study presents a comprehensive comparison of our Uni-OVSeg against previous works across a range of benchmarks. These include ADE20K (encompassing both 150 and 847 class variants), PASCAL Context (459 and 59 class variants), PASCAL VOC (with 20 and 21 class categories), and Cityscapes. Compared to weakly-supervised methods, Uni-OVSeg exhibits remarkable performance improvements across all evaluated datasets. Specifically, in the more challenging datasets of PASCAL Context-459, Uni-OVSeg not only surpasses its weakly-supervised counterparts but also outperforms the cutting-edge fully-supervised methods, e.g., FC-CLIP. This is indicative of Uni-OVSeg’s superior capability in categorizing a diverse array of semantic classes. Furthermore, in the PASCAL VOC benchmarks (20 and 21 classes), Uni-OVSeg demonstrates a substantial enhancement over state-of-the-art weakly-supervised methods, achieving improvements of 18.3% and 12.2% mIoU, respectively, which demonstrates our Uni-OVSeg captures fine-grained spatial structures. These results elevate the practical applicability of weakly-supervised open-vocabulary segmentation to new heights. Open-vocabulary semantic segmentation. Open-vocabulary panoptic segmentation. For openvocabulary panoptic segmentation, we zero-shot evaluate our model on the COCO, ADE20K, and Cityscapes datasets, shown in Tab. 2. Existing weakly-supervised methods only use text supervision, which causes a challenge to discriminate different instances with the same semantic class. To the best of our knowledge, we are the first to learn open-vocabulary panoptic segmentation with weak supervision. Compared to unsupervised methods, we obviously outperform U2Seg by 1.9% PQ, 1.5%SQ, and 4.4% RQ on COCO datasets. Unfortunately, our used image-mask pairs contain multiple granularity masks, such as object-wise and part-wise masks, which is different with the panoptic segmentation datasets. This discrepancy leads to a number of false positive results, limiting our performance. Open-vocabulary panoptic segmentation. Promptable segmentation. To evaluate the segmentation efficacy of Uni-OVSeg in scenarios involving interactive point and box prompts, we conduct comparative analyses with the SAM-ViT/L model across a range of datasets from diverse domains. For visual prompts, we implement a uniform 20×20 point grid as the interactive point prompt and utilise the actual bounding boxes as box prompts. The segmentation performance is measured using the 1-pt IoU (Oracle) metric across all datasets. The results, as demonstrated in Fig. 3, indicate that Uni-OVSeg exceeds SAM in most Promptable segmentation. datasets when employing point and box prompts. In particular, Uni-OVSeg significantly surpasses SAM in terms of IoU by 31.1%, 20.1%, and 43.2% on the TimberSeg, COCO instance, and PIDray datasets, respectively. Additionally, we provide a visualisation of the segmentation results of both SAM and Uni-OVSeg in Fig. 4. The visualisation reveals that SAM tends to miss larger masks and erroneously categorise areas with unclear boundaries. In contrast, UniOVSeg demonstrates exceptional precision in segmenting objects of varying scales and different stuff. More experimental results are visualisation can be found in the Appendix B and C. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Uni-OVSeg Outperforms Weakly-Supervised and Fully-Supervised Methods in Open-Vocabulary Segmentation

Authors:
(1) Zhaoqing Wang, The University of Sydney and AI2Robotics;
(2) Xiaobo Xia, The University of Sydney;
(3) Ziye Chen, The University of Melbourne;
(4) Xiao He, AI2Robotics;
(5) Yandong Guo, AI2Robotics;
(6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence;
(7) Tongliang Liu, The University of Sydney. Table of Links Abstract and 1. Introduction 2. Related works 3. Method and 3.1. Problem definition 3.2. Baseline and 3.3. Uni-OVSeg framework 4. Experiments 4.1. Implementation details 4.2. Main results 4.3. Ablation study 5. Conclusion 6. Broader impacts and References A. Framework details B. Promptable segmentation C. Visualisation Abstract Contemporary cutting-edge open-vocabulary segmentation approaches commonly rely on image-mask-text triplets, yet this restricted annotation is labour-intensive and encounters scalability hurdles in complex real-world scenarios. Although some methods are proposed to reduce the annotation cost with only text supervision, the incompleteness of supervision severely limits the versatility and performance. In this paper, we liberate the strict correspondence between masks and texts by using independent image-mask and image-text pairs, which can be easily collected respectively. With this unpaired mask-text supervision, we propose a new weakly-supervised open-vocabulary segmentation framework (Uni-OVSeg) that leverages confident pairs of mask predictions and entities in text descriptions. Using the independent image-mask and image-text pairs, we predict a set of binary masks and associate them with entities by resorting to the CLIP embedding space. However, the inherent noise in the correspondence between masks and entities poses a significant challenge when obtaining reliable pairs. In light of this, we advocate using the large vision-language model (LVLM) to refine text descriptions and devise a multi-scale ensemble to stablise the matching between masks and entities. Compared to text-only weaklysupervised methods, our Uni-OVSeg achieves substantial improvements of 15.5% mIoU on the ADE20K datasets, and even surpasses fully-supervised methods on the challenging PASCAL Context-459 dataset. 1. Introduction Open-vocabulary segmentation refers to the segmentation and categorisation of objects from an expansive and unrestricted vocabulary, even though the object categories within the vocabulary are not encountered during training [16, 35, 70]. Compared to traditional closed-vocabulary segmentation [13, 83], which depends on predefined training categories and cannot recognise absent categories, open-vocabulary segmentation can segment any category with arbitrary text descriptions. This innovative segmentation paradigm has garnered considerable attention [39, 67] and opened up numerous potential applications [72, 87, 88]. Cutting-edge approaches in open-vocabulary segmentation typically leverage supervision with triplet annotations that are composed of images, masks, and corresponding texts [63, 76]. It is worth noting that strict alignment between each mask and text results in an expensive annotation cost. To mitigate this, some weakly-supervised methods propose using only text supervision [6, 64, 65]. However, learning with this supervision, the model falls short in capturing complex spatial details, which is suboptimal for dense prediction. Furthermore, this type of supervision lacks positional information, making the model difficult to distinguish different instances with the same semantic class. These issues severely limit the versatility and segmentation performance of existing weakly-supervised methods. Motivated by that, in this paper, we propose an advanced weakly-supervised open-vocabulary segmentation framework, named Uni-OVSeg, to reduce the annotation expense while significantly enhancing performance. In essence, we liberate the strict correspondence between masks and texts, by using independent image-mask and image-text pairs. These two types of pairs can be easily collected from different sources, as illustrated in Fig. 1, such as SA-1B [34] for image-mask pairs and CC3M [50] for image-text pairs. By resorting to independent image-mask and image-text pairs, our Uni-OVSeg has a strong segmentation ability to group semantically similar pixels, and align mask-wise embeddings with entity embeddings of texts into the same space, achieving open-vocabulary segmentation. Technically, when presented with an image-mask pair, we train a visual prompt encoder, a pixel decoder, and a mask decoder to generate a set of binary masks. The collected image-text pair often contains certain texts that does not match the image [52], leading to an incorrect correspondence between masks and entities. We employ the LLaVa model [42] to refine the quality of text descriptions, coupled with a ChatGPT-based parser for precise entity extraction. For mask-text alignment, we introduce a mask-text bipartite matching to exploit confident pairs of predicted masks and entities. A multi-scale feature adapter is designed to enhance the visual embeddings of predicted masks, which are further aligned with the text embeddings of corresponding entities. As masks and text are allowed to be unpaired in Uni-OVSeg, a multi-scale ensemble is introduced to improve the quality of visual embeddings and tackle inherent noise in the correspondence between masks and entities effectively, thereby stablising the matching process. During the inference phase, a zero-shot classifier, built by embedding target dataset category names, assigns categories to predicted masks, enabling the system to segment objects across an open vocabulary. We summarise our contributions in the following aspects: We introduce a new framework Uni-OVSeg and offer an innovative solution that significantly reduces the need for costly triplet annotations. This makes openvocabulary segmentation more accessible and scalable.


Uni-OVSeg employs a visual prompt encoder, pixel decoder, and mask decoder for binary mask segmentation. An inclusion of refinement of text descriptions, multiscale ensemble enhances segmentation performance by alleviating inherent noise in mask-text correspondence.


With one suit of weights, we achieve 32.6% mIoU on the ADE20K dataset, and even surpassing the state-of-theart FC-CLIP [76] on the challenging PASCAL Context459 dataset. Comprehensive ablation studies and discussions are also provided. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Zhaoqing Wang, The University of Sydney and AI2Robotics; (2) Xiaobo Xia, The University of Sydney; (3) Ziye Chen, The University of Melbourne; (4) Xiao He, AI2Robotics; (5) Yandong Guo, AI2Robotics; (6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence; (7) Tongliang Liu, The University of Sydney. Authors: Authors: (1) Zhaoqing Wang, The University of Sydney and AI2Robotics; (2) Xiaobo Xia, The University of Sydney; (3) Ziye Chen, The University of Melbourne; (4) Xiao He, AI2Robotics; (5) Yandong Guo, AI2Robotics; (6) Mingming Gong, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence; (7) Tongliang Liu, The University of Sydney. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2. Related works 2. Related works 3. Method and 3.1. Problem definition 3. Method and 3.1. Problem definition 3.2. Baseline and 3.3. Uni-OVSeg framework 3.2. Baseline and 3.3. Uni-OVSeg framework 4. Experiments 4.1. Implementation details 4.1. Implementation details 4.2. Main results 4.2. Main results 4.3. Ablation study 4.3. Ablation study 5. Conclusion 5. Conclusion 6. Broader impacts and References 6. Broader impacts and References A. Framework details A. Framework details B. Promptable segmentation B. Promptable segmentation C. Visualisation C. Visualisation Abstract Contemporary cutting-edge open-vocabulary segmentation approaches commonly rely on image-mask-text triplets, yet this restricted annotation is labour-intensive and encounters scalability hurdles in complex real-world scenarios. Although some methods are proposed to reduce the annotation cost with only text supervision, the incompleteness of supervision severely limits the versatility and performance. In this paper, we liberate the strict correspondence between masks and texts by using independent image-mask and image-text pairs, which can be easily collected respectively. With this unpaired mask-text supervision, we propose a new weakly-supervised open-vocabulary segmentation framework (Uni-OVSeg) that leverages confident pairs of mask predictions and entities in text descriptions. Using the independent image-mask and image-text pairs, we predict a set of binary masks and associate them with entities by resorting to the CLIP embedding space. However, the inherent noise in the correspondence between masks and entities poses a significant challenge when obtaining reliable pairs. In light of this, we advocate using the large vision-language model (LVLM) to refine text descriptions and devise a multi-scale ensemble to stablise the matching between masks and entities. Compared to text-only weaklysupervised methods, our Uni-OVSeg achieves substantial improvements of 15.5% mIoU on the ADE20K datasets, and even surpasses fully-supervised methods on the challenging PASCAL Context-459 dataset. Contemporary cutting-edge open-vocabulary segmentation approaches commonly rely on image-mask-text triplets, yet this restricted annotation is labour-intensive and encounters scalability hurdles in complex real-world scenarios. Although some methods are proposed to reduce the annotation cost with only text supervision, the incompleteness of supervision severely limits the versatility and performance. In this paper, we liberate the strict correspondence between masks and texts by using independent image-mask and image-text pairs, which can be easily collected respectively. With this unpaired mask-text supervision, we propose a new weakly-supervised open-vocabulary segmentation framework (Uni-OVSeg) that leverages confident pairs of mask predictions and entities in text descriptions. Using the independent image-mask and image-text pairs, we predict a set of binary masks and associate them with entities by resorting to the CLIP embedding space. However, the inherent noise in the correspondence between masks and entities poses a significant challenge when obtaining reliable pairs. In light of this, we advocate using the large vision-language model (LVLM) to refine text descriptions and devise a multi-scale ensemble to stablise the matching between masks and entities. Compared to text-only weaklysupervised methods, our Uni-OVSeg achieves substantial improvements of 15.5% mIoU on the ADE20K datasets, and even surpasses fully-supervised methods on the challenging PASCAL Context-459 dataset. 1. Introduction Open-vocabulary segmentation refers to the segmentation and categorisation of objects from an expansive and unrestricted vocabulary, even though the object categories within the vocabulary are not encountered during training [16, 35, 70]. Compared to traditional closed-vocabulary segmentation [13, 83], which depends on predefined training categories and cannot recognise absent categories, open-vocabulary segmentation can segment any category with arbitrary text descriptions. This innovative segmentation paradigm has garnered considerable attention [39, 67] and opened up numerous potential applications [72, 87, 88]. Cutting-edge approaches in open-vocabulary segmentation typically leverage supervision with triplet annotations that are composed of images, masks, and corresponding texts [63, 76]. It is worth noting that strict alignment between each mask and text results in an expensive annotation cost. To mitigate this, some weakly-supervised methods propose using only text supervision [6, 64, 65]. However, learning with this supervision, the model falls short in capturing complex spatial details, which is suboptimal for dense prediction. Furthermore, this type of supervision lacks positional information, making the model difficult to distinguish different instances with the same semantic class. These issues severely limit the versatility and segmentation performance of existing weakly-supervised methods. Motivated by that, in this paper, we propose an advanced weakly-supervised open-vocabulary segmentation framework, named Uni-OVSeg, to reduce the annotation expense while significantly enhancing performance. In essence, we liberate the strict correspondence between masks and texts, by using independent image-mask and image-text pairs. These two types of pairs can be easily collected from different sources, as illustrated in Fig. 1, such as SA-1B [34] for image-mask pairs and CC3M [50] for image-text pairs. By resorting to independent image-mask and image-text pairs, our Uni-OVSeg has a strong segmentation ability to group semantically similar pixels, and align mask-wise embeddings with entity embeddings of texts into the same space, achieving open-vocabulary segmentation. Technically, when presented with an image-mask pair, we train a visual prompt encoder, a pixel decoder, and a mask decoder to generate a set of binary masks. The collected image-text pair often contains certain texts that does not match the image [52], leading to an incorrect correspondence between masks and entities. We employ the LLaVa model [42] to refine the quality of text descriptions, coupled with a ChatGPT-based parser for precise entity extraction. For mask-text alignment, we introduce a mask-text bipartite matching to exploit confident pairs of predicted masks and entities. A multi-scale feature adapter is designed to enhance the visual embeddings of predicted masks, which are further aligned with the text embeddings of corresponding entities. As masks and text are allowed to be unpaired in Uni-OVSeg, a multi-scale ensemble is introduced to improve the quality of visual embeddings and tackle inherent noise in the correspondence between masks and entities effectively, thereby stablising the matching process. During the inference phase, a zero-shot classifier, built by embedding target dataset category names, assigns categories to predicted masks, enabling the system to segment objects across an open vocabulary. We summarise our contributions in the following aspects: We introduce a new framework Uni-OVSeg and offer an innovative solution that significantly reduces the need for costly triplet annotations. This makes openvocabulary segmentation more accessible and scalable. Uni-OVSeg employs a visual prompt encoder, pixel decoder, and mask decoder for binary mask segmentation. An inclusion of refinement of text descriptions, multiscale ensemble enhances segmentation performance by alleviating inherent noise in mask-text correspondence. With one suit of weights, we achieve 32.6% mIoU on the ADE20K dataset, and even surpassing the state-of-theart FC-CLIP [76] on the challenging PASCAL Context459 dataset. Comprehensive ablation studies and discussions are also provided. We introduce a new framework Uni-OVSeg and offer an innovative solution that significantly reduces the need for costly triplet annotations. This makes openvocabulary segmentation more accessible and scalable. We introduce a new framework Uni-OVSeg and offer an innovative solution that significantly reduces the need for costly triplet annotations. This makes openvocabulary segmentation more accessible and scalable. Uni-OVSeg employs a visual prompt encoder, pixel decoder, and mask decoder for binary mask segmentation. An inclusion of refinement of text descriptions, multiscale ensemble enhances segmentation performance by alleviating inherent noise in mask-text correspondence. Uni-OVSeg employs a visual prompt encoder, pixel decoder, and mask decoder for binary mask segmentation. An inclusion of refinement of text descriptions, multiscale ensemble enhances segmentation performance by alleviating inherent noise in mask-text correspondence. With one suit of weights, we achieve 32.6% mIoU on the ADE20K dataset, and even surpassing the state-of-theart FC-CLIP [76] on the challenging PASCAL Context459 dataset. Comprehensive ablation studies and discussions are also provided. With one suit of weights, we achieve 32.6% mIoU on the ADE20K dataset, and even surpassing the state-of-theart FC-CLIP [76] on the challenging PASCAL Context459 dataset. Comprehensive ablation studies and discussions are also provided. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Open-Vocabulary Segmentation with Unpaired Mask-Text Supervision

If you check popular benchmarks used for measuring LLM performance, then you probably feel like AI is oh sooo clever. This is a fair surface-level impression; however, is AI truly better than an average human for cognitive tasks? Frontier models like o1 from OpenAI and Claude 3.5 Sonnet from Anthropic do perform better than expert humans in a number of fields, including law, coding, and math. Why, then, can’t ChatGPT solve some simple spatial reasoning tasks or some silly trick questions? Well, we are still talking about “large language models"—they take in a ton of letters and try to predict what letters to spit out for a given query. Notice, that nowhere in this equation the actual “thinking” is mentioned. The models are stochastic parrots of a sort, as they try to retrieve the right information from their training dataset instead of actually considering your question. At least, this was the case until OpenAI released o1-preview, but more on this later. Among those who started questioning the existing LLM benchmarks for relevance is the author of “AI Explained” a popular YouTube channel of which I’m a big fan. Phillip (the YouTuber’s name) noticed that the industry standard benchmarks have a clear style of questions that are also mostly publicly available. This means that not only can those exact questions be part of the training dataset, but due to standardization, it is easier for models to spot & apply patterns from the overall training data. Put simply, AI researchers who create groundbreaking complex tech can surely find a way to give their model the relevant questions and answers to “remember” before benchmarking. Looking at the results of the top model out there, o1 from OpenAI, one can imply that it scores above average in a lot of professional domains. And this is true, but this result relies on the availability of relevant training data and past examples from those specific fields. Don’t get me wrong, the models are now amazing at giving textbook answers to textbook questions, and this in itself is insanely impressive. The term “artificial intelligence," though, implies a bit more than just information retrieval; there should be some actual thinking involved. So a logical follow-up to all of the impressive numbers above is whether such “AI” can answer a tricky reasoning question. Does it have any spatial intelligence? Or can it navigate well in common social scenarios? The answer is - sometimes. Unlike field-specific questions with predefined answers, problems that humans solve on a daily basis often require understanding the context beyond natural language (which is the only thing LLMs have). Above are the top scorers on the SIMPLE benchmark, which gives LLMs questions that an average person would deem trivial but the models can’t necessarily answer yet. We are used to seeing AI doing so much better than an average human in exams or specialized benchmarks, but here, the top model performance is actually just 41.7% (o1-preview) vs. 83.7% of an average human. This benchmark uses 200 multiple-choice text questions focused on spatio-temporal reasoning, social intelligence, and trick questions. The most important feature of the benchmark is that those questions are not publicly available, so AI labs cannot just add them to their training data. You can learn more about this benchmark here. This new approach to measuring LLM performance shows just how far all the models still are from an average human reasoning capability. The quicker this gap closes in the coming months, the more definitive the answer “yes” to our headline becomes. An interesting new metric to look out for if you are enthusiastic but cautious about AI. If you check popular benchmarks used for measuring LLM performance, then you probably feel like AI is oh sooo clever. This is a fair surface-level impression; however, is AI truly better than an average human for cognitive tasks? Frontier models like o1 from OpenAI and Claude 3.5 Sonnet from Anthropic do perform better than expert humans in a number of fields, including law, coding, and math. Why, then, can’t ChatGPT solve some simple spatial reasoning tasks or some silly trick questions? Well, we are still talking about “large language models"—they take in a ton of letters and try to predict what letters to spit out for a given query. Notice, that nowhere in this equation the actual “thinking” is mentioned. The models are stochastic parrots of a sort, as they try to retrieve the right information from their training dataset instead of actually considering your question. At least, this was the case until OpenAI released o1-preview, but more on this later. stochastic parrots Among those who started questioning the existing LLM benchmarks for relevance is the author of “AI Explained” a popular YouTube channel of which I’m a big fan. Phillip (the YouTuber’s name) noticed that the industry standard benchmarks have a clear style of questions that are also mostly publicly available. This means that not only can those exact questions be part of the training dataset, but due to standardization, it is easier for models to spot & apply patterns from the overall training data. Put simply, AI researchers who create groundbreaking complex tech can surely find a way to give their model the relevant questions and answers to “remember” before benchmarking. Looking at the results of the top model out there, o1 from OpenAI, one can imply that it scores above average in a lot of professional domains. And this is true, but this result relies on the availability of relevant training data and past examples from those specific fields. Don’t get me wrong, the models are now amazing at giving textbook answers to textbook questions, and this in itself is insanely impressive. The term “artificial intelligence," though, implies a bit more than just information retrieval; there should be some actual thinking involved. So a logical follow-up to all of the impressive numbers above is whether such “AI” can answer a tricky reasoning question. Does it have any spatial intelligence? Or can it navigate well in common social scenarios? The answer is - sometimes. Unlike field-specific questions with predefined answers, problems that humans solve on a daily basis often require understanding the context beyond natural language (which is the only thing LLMs have). Above are the top scorers on the SIMPLE benchmark, which gives LLMs questions that an average person would deem trivial but the models can’t necessarily answer yet. We are used to seeing AI doing so much better than an average human in exams or specialized benchmarks, but here, the top model performance is actually just 41.7% (o1-preview) vs. 83.7% of an average human. This benchmark uses 200 multiple-choice text questions focused on spatio-temporal reasoning, social intelligence, and trick questions. The most important feature of the benchmark is that those questions are not publicly available, so AI labs cannot just add them to their training data. You can learn more about this benchmark here . here This new approach to measuring LLM performance shows just how far all the models still are from an average human reasoning capability. The quicker this gap closes in the coming months, the more definitive the answer “yes” to our headline becomes. An interesting new metric to look out for if you are enthusiastic but cautious about AI.

AI vs Human - Is the Machine Already Superior? 

الذكاء الاصطناعي مقابل الإنسان - هل الآلة متفوقة بالفعل؟

KI vs Mensch – Ist die Maschine schon überlegen?

IA vs. humanos: ¿la máquina ya es superior?

AI protiv čovjeka - je li stroj već superioran? 

Intelligenza artificiale contro uomo: la macchina è già superiore?

AI vs 人間 - 機械はすでに優れているのか?

AI prieš žmogų – ar mašina jau pranašesnė? 

Sztuczna inteligencja kontra człowiek – czy maszyna jest już lepsza?

AI بمقابله انسان - ایا ماشین دمخه غوره دی؟ 

AI vs Aadanaha - Mashiinku mar hore ma ka sarreeyaa? 

AI vs மனித - இயந்திரம் ஏற்கனவே உயர்ந்ததா? 

AI กับ มนุษย์ - เครื่องจักรเหนือกว่าแล้วจริงหรือ?

The HackerNoon Newsletter: Why Laziness is a Superpower (10/25/2024)

Presto SQL + S3 Data + Superset = Data Lake

Why Laziness is a Superpower

Чаму лянота - гэта звышздольнасць

Por qué la pereza es un superpoder

Miksi laiskuus on supervoima

Zašto je lijenost supermoć

Perché la pigrizia è un superpotere

למה עצלות היא מעצמת על

なぜ怠惰はスーパーパワーなのか

Kāpēc slinkums ir lielvara

Залхуурал яагаад супер хүч вэ?

Ke ka baka la’ng Botswa e le Matla a Magolo

Imaraykutaq Qella kayqa Superpoder

Waa maxay sababta caajisnimadu u tahay awood sare

Hey Hackers, Super Duper Studios has been nominated in HackerNoon's annual Startups of The Year awards in Toronto, Canada. We would love it if you could drop a vote for us. ⬇️ https://hackernoon.com/startups/north-america/north-america-toronto-canada?stup=66f320306ea4677e23de6640&embedable=true Read more about us below to understand why we deserve your vote. 👀 Meet Super Duper Studios We are a Gen Z social media agency that optimizes brands for the internet. Our mission is to evolve the industry to match the ever-changing digital universe. As millennials and Gen Zs, we’ve grown up our entire lives being bombarded with social media and advertisements between swipes. We can spot an ad from a mile away — and we can clock a bad ad from the next city over. How Super Duper achieved product-market fit We merge traditional brand strategy with tomorrow's social strategy to transform brands into internet citizens. Super Duper specializes in connecting the dots between people, brands and culture. The evolution of Super Duper Super Duper was founded in 2020 with the realization that traditional marketers weren’t cutting it with the Gen Z audience. The new generation of consumers is super social savvy and they know exactly when, where and how you’re trying to sell to them - which is where we step in! We went from helping brands simply kickstart or ramp up their online presence, to ensuring our clients aren’t just faceless brands on social media - they’re relatable internet citizens who know how to engage with their audience. Our client roster, team and expertise have all grown since we launched our small business and we’re so excited to see what the future holds for us. Milestones and achievements We launched three Super Bowl ads for HAVOC!
We were the lead sponsor for the 2023 One Life Gala in Toronto
We were nominated as one of Hackernoon’s Startups of the Year (hehe) Lessons learned There’s no “one size fits all” approach to planning a social strategy. Every brand will have a different identity or a different approach they want to take when it comes to being authentic on the internet, and it’s our job not to change that dream, but to strategically amplify it. What Startups of The Year means to us We’re honoured to hear that there are people out there who consider us the best social agency! We’re lucky to have so many supportive clients behind us who trust us - even though this is just a nomination (for now), it feels like a win to us because our small team is being recognized for our hard work. Moments like this are what keeps our team feeling empowered to stride forward and not doubt ourselves. Conclusion There’s a special place in our hearts for everyone who supports us. Whether you liked an Instagram story, read one of our blog posts, let us create your social strategy, shared our posts with your friends, or even just laughed at one of our TikToks, it’s the most wonderful feeling to be seen by our community. And don’t forget… if you’re looking for a team to spice up your socials, you know who to call! Vote for us today! Welcome to HackerNoon’s Startups of The Year 2024 interview series, spotlighting startups from around the world that are changing the way we live, one solution at a time. If you’ve been nominated, create a brand profile and answer these questions here.
Want to nominate a startup? Follow the steps here.Looking for inspiration? Check out how other startups responded to last year’s Startups of The Year interview questions and examples of company responses here, here, and here. Hey Hackers, Super Duper Studios has been nominated in HackerNoon's annual Startups of The Year awards in Toronto, Canada. We would love it if you could drop a vote for us. ⬇️ We would love it if you could drop a vote for us. ⬇️ https://hackernoon.com/startups/north-america/north-america-toronto-canada?stup=66f320306ea4677e23de6640&embedable=true https://hackernoon.com/startups/north-america/north-america-toronto-canada?stup=66f320306ea4677e23de6640&embedable=true Read more about us below to understand why we deserve your vote. 👀 Meet Super Duper Studios We are a Gen Z social media agency that optimizes brands for the internet. Our mission is to evolve the industry to match the ever-changing digital universe. As millennials and Gen Zs, we’ve grown up our entire lives being bombarded with social media and advertisements between swipes. We can spot an ad from a mile away — and we can clock a bad ad from the next city over. How Super Duper achieved product-market fit We merge traditional brand strategy with tomorrow's social strategy to transform brands into internet citizens. Super Duper specializes in connecting the dots between people, brands and culture. The evolution of Super Duper Super Duper was founded in 2020 with the realization that traditional marketers weren’t cutting it with the Gen Z audience. The new generation of consumers is super social savvy and they know exactly when, where and how you’re trying to sell to them - which is where we step in! We went from helping brands simply kickstart or ramp up their online presence, to ensuring our clients aren’t just faceless brands on social media - they’re relatable internet citizens who know how to engage with their audience. Our client roster, team and expertise have all grown since we launched our small business and we’re so excited to see what the future holds for us. Milestones and achievements We launched three Super Bowl ads for HAVOC! We were the lead sponsor for the 2023 One Life Gala in Toronto We were nominated as one of Hackernoon’s Startups of the Year (hehe) We launched three Super Bowl ads for HAVOC! We were the lead sponsor for the 2023 One Life Gala in Toronto We were nominated as one of Hackernoon’s Startups of the Year (hehe) Lessons learned There’s no “one size fits all” approach to planning a social strategy. Every brand will have a different identity or a different approach they want to take when it comes to being authentic on the internet, and it’s our job not to change that dream, but to strategically amplify it. What Startups of The Year means to us We’re honoured to hear that there are people out there who consider us the best social agency! We’re lucky to have so many supportive clients behind us who trust us - even though this is just a nomination (for now), it feels like a win to us because our small team is being recognized for our hard work. Moments like this are what keeps our team feeling empowered to stride forward and not doubt ourselves. Conclusion There’s a special place in our hearts for everyone who supports us. Whether you liked an Instagram story, read one of our blog posts, let us create your social strategy, shared our posts with your friends, or even just laughed at one of our TikToks, it’s the most wonderful feeling to be seen by our community. And don’t forget… if you’re looking for a team to spice up your socials, you know who to call! Vote for us today! Vote for us today ! Vote for us today Welcome to HackerNoon’s Startups of The Year 2024 interview series, spotlighting startups from around the world that are changing the way we live, one solution at a time. If you’ve been nominated, create a brand profile and answer these questions here. Want to nominate a startup? Follow the steps here. Looking for inspiration? Check out how other startups responded to last year’s Startups of The Year interview questions and examples of company responses here, here, and here. Welcome to HackerNoon’s Startups of The Year 2024 interview series, spotlighting startups from around the world that are changing the way we live, one solution at a time. If you’ve been nominated, create a brand profile and answer these questions here . Startups of The Year 2024 create a brand profile create a brand profile create a brand profile here here Want to nominate a startup? Follow the steps here . here here Looking for inspiration? Check out how other startups responded to last year’s Startups of The Year interview questions and examples of company responses here, here, and here. Looking for inspiration? Check out how other startups responded to last year’s Startups of The Year interview questions and examples of company responses here , here , and here . last year’s Startups of The Year interview questions last year’s Startups of The Year interview questions here here here here here here

Meet Super Duper Studios, Startups of the Year 2024 Nominee

Authors:
(1) Dustin Podell, Stability AI, Applied Research;
(2) Zion English, Stability AI, Applied Research;
(3) Kyle Lacey, Stability AI, Applied Research;
(4) Andreas Blattmann, Stability AI, Applied Research;
(5) Tim Dockhorn, Stability AI, Applied Research;
(6) Jonas Müller, Stability AI, Applied Research;
(7) Joe Penna, Stability AI, Applied Research;
(8) Robin Rombach, Stability AI, Applied Research. Table of Links Abstract and 1 Introduction 2 Improving Stable Diffusion 2.1 Architecture & Scale 2.2 Micro-Conditioning 2.3 Multi-Aspect Training 2.4 Improved Autoencoder and 2.5 Putting Everything Together 3 Future Work Appendix A Acknowledgements B Limitations C Diffusion Models D Comparison to the State of the Art E Comparison to Midjourney v5.1 F On FID Assessment of Generative Text-Image Foundation Models G Additional Comparison between Single- and Two-Stage SDXL pipeline References 2.2 Micro-Conditioning Conditioning the Model on Image Size A notorious shortcoming of the LDM paradigm [38] is the fact that training a model requires a minimal image size, due to its two-stage architecture. The two main approaches to tackle this problem are either to discard all training images below a certain minimal resolution (for example, Stable Diffusion 1.4/1.5 discarded all images with any size below 512 pixels), or, alternatively, upscale images that are too small. However, depending on the desired image resolution, the former method can lead to significant portions of the training data being discarded, what will likely lead to a loss in performance and hurt generalization. We visualize such effects in Fig. 2 for the dataset on which SDXL was pretrained. For this particular choice of data, discarding all samples below our pretraining resolution of 256 2 pixels would lead to a significant 39% of discarded data. The second method, on the other hand, usually introduces upscaling artifacts which may leak into the final model outputs, causing, for example, blurry samples. Conditioning the Model on Cropping Parameters The first two rows of Fig. 4 illustrate a typical failure mode of previous SD models: Synthesized objects can be cropped, such as the cut-off head of the cat in the left examples for SD 1-5 and SD 2-1. An intuitive explanation for this behavior is the use of random cropping during training of the model: As collating a batch in DL frameworks such as PyTorch [32] requires tensors of the same size, a typical processing pipeline is to (i) resize an image such that the shortest size matches the desired target size, followed by (ii) randomly cropping the image along the longer axis. While random cropping is a natural form of data augmentation, it can leak into the generated samples, causing the malicious effects shown above. While other methods like data bucketing [31] successfully tackle the same task, we still benefit from cropping-induced data augmentation, while making sure that it does not leak into the generation process - we actually use it to our advantage to gain more control over the image synthesis process. Furthermore, it is easy to implement and can be applied in an online fashion during training, without additional data preprocessing. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Dustin Podell, Stability AI, Applied Research; (2) Zion English, Stability AI, Applied Research; (3) Kyle Lacey, Stability AI, Applied Research; (4) Andreas Blattmann, Stability AI, Applied Research; (5) Tim Dockhorn, Stability AI, Applied Research; (6) Jonas Müller, Stability AI, Applied Research; (7) Joe Penna, Stability AI, Applied Research; (8) Robin Rombach, Stability AI, Applied Research. Authors: Authors: (1) Dustin Podell, Stability AI, Applied Research; (2) Zion English, Stability AI, Applied Research; (3) Kyle Lacey, Stability AI, Applied Research; (4) Andreas Blattmann, Stability AI, Applied Research; (5) Tim Dockhorn, Stability AI, Applied Research; (6) Jonas Müller, Stability AI, Applied Research; (7) Joe Penna, Stability AI, Applied Research; (8) Robin Rombach, Stability AI, Applied Research. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Improving Stable Diffusion 2 Improving Stable Diffusion 2.1 Architecture & Scale 2.1 Architecture & Scale 2.2 Micro-Conditioning 2.2 Micro-Conditioning 2.3 Multi-Aspect Training 2.3 Multi-Aspect Training 2.4 Improved Autoencoder and 2.5 Putting Everything Together 2.4 Improved Autoencoder and 2.5 Putting Everything Together 3 Future Work 3 Future Work Appendix Appendix A Acknowledgements A Acknowledgements B Limitations B Limitations C Diffusion Models C Diffusion Models D Comparison to the State of the Art D Comparison to the State of the Art E Comparison to Midjourney v5.1 E Comparison to Midjourney v5.1 F On FID Assessment of Generative Text-Image Foundation Models F On FID Assessment of Generative Text-Image Foundation Models G Additional Comparison between Single- and Two-Stage SDXL pipeline G Additional Comparison between Single- and Two-Stage SDXL pipeline References References 2.2 Micro-Conditioning Conditioning the Model on Image Size A notorious shortcoming of the LDM paradigm [38] is the fact that training a model requires a minimal image size, due to its two-stage architecture. The two main approaches to tackle this problem are either to discard all training images below a certain minimal resolution (for example, Stable Diffusion 1.4/1.5 discarded all images with any size below 512 pixels), or, alternatively, upscale images that are too small. However, depending on the desired image resolution, the former method can lead to significant portions of the training data being discarded, what will likely lead to a loss in performance and hurt generalization. We visualize such effects in Fig. 2 for the dataset on which SDXL was pretrained. For this particular choice of data, discarding all samples below our pretraining resolution of 256 2 pixels would lead to a significant 39% of discarded data. The second method, on the other hand, usually introduces upscaling artifacts which may leak into the final model outputs, causing, for example, blurry samples. Conditioning the Model on Cropping Parameters The first two rows of Fig. 4 illustrate a typical failure mode of previous SD models: Synthesized objects can be cropped, such as the cut-off head of the cat in the left examples for SD 1-5 and SD 2-1. An intuitive explanation for this behavior is the use of random cropping during training of the model: As collating a batch in DL frameworks such as Conditioning the Model on Cropping Parameters PyTorch [32] requires tensors of the same size, a typical processing pipeline is to (i) resize an image such that the shortest size matches the desired target size, followed by (ii) randomly cropping the image along the longer axis. While random cropping is a natural form of data augmentation, it can leak into the generated samples, causing the malicious effects shown above. While other methods like data bucketing [31] successfully tackle the same task, we still benefit from cropping-induced data augmentation, while making sure that it does not leak into the generation process - we actually use it to our advantage to gain more control over the image synthesis process. Furthermore, it is easy to implement and can be applied in an online fashion during training, without additional data preprocessing. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Stories, News & Mentions about SUPER

Super’s Stories on HackerNoon

Super's News & Mentions