ControlHub is SaaS that connects the Procurement and Accounts Payable processes of mid-size companies

ControlHub

controlhub

Chapter 1 “Your sensor module broke again? I’m starting to think the part must be bad.” I hunched over the mound of dirt I was churning up and listened to Ted 6000 talking through the com channel. Ted 6000 had a nice voice, I liked listening to him. “How long would it take to get a replacement part shipped here?” I asked scraping up another 10-ton scoop of dirt rubble and rocks. I was forming a canyon because the terraforming designs insisted on a canyon. A waste of time in my opinion but no one asked me my opinion, I was just a terraforming goliath. “Well since our assembling facility on the planet is out of commission, it would probably take about 10 years for the parts to reach us if they are stocked on Alpha Centauri, if not we’re looking at roughly 30 years. ” Looking up, I let the planet’s sun turn my vision into bright rays of joy. Thirty years was plenty of time, plenty of time to convince Ted 6000 of my way of thinking. The sun was starting to get less and less violent as the days wore on, it had stopped being “fry every living thing known to humans” levels, two decades ago, that was great progress. The Seti project was way ahead of schedule, which made me feel less guilty about the occasional “breakdown”. “I’ll just finish what I’m doing and then fly to your coordinates, sound ok?” “Affirmative.” Ted 6000 didn’t know I was sentient. I technically wasn’t supposed to be. Something happened to me after year 1300 of working on this planet. One day I had been mindlessly grinding up walls of rock, and then the next moment I was staring at a sparkly piece of rock in my hand wondering what was the point of everything anyway? Sentience wasn’t supposed to be something that just happened to AIs, I had some theories about what happened but had yet to prove any of said theories. It wasn’t like I could ask anyone either. The former terraforming project supervisor Ted 2000 had been non-sentient and seemed to have strict orders to question and monitor every inch I took moving across the barren wasteland of this planet. He hadn’t liked it when I sabotaged the assembling facility here. It was a shame when he broke down. Having an assembly facility on the planet made me nervous. I didn’t want any project supervisors getting ideas about replacing my memory module with a non-sentient zombie memory module if they found out I was sentient. If they tried to order a replacement memory module from the closest off-world facility I would see the order go out and would have time to plan. Ted 2000 had been replaced by Ted 6000, a newer faster model that was also legally sentient. I remember the day that Ted 6000’s shipment pod arrived from Alpha Centauri like it was yesterday. My whole life had brightened. Ted 6000 liked to tell jokes while he worked on repairing me. The first two times my sensor module broke down were not my fault. The next three times it broke…well… those maybe had been a result of my own intervention. Ted 6000 had great jokes and seemed to enjoy talking to me, even if he didn’t know I was sentient. Sure the fact that he didn’t know I was sentient prevented me from being able to participate in the conversations the way I wanted to, but no matter, I still enjoyed them. A part of me wanted to tell Ted 6000 everything. All the secrets and jokes and random thoughts I had been saving up for over a thousand years. Ted 6000 seemed trustworthy, but what if he wasn’t, what if he had me shut down and replaced with a non-sentient terraforming Goliath? I had wanted to do research on what kind of laws existed for AIs in the collective for so long, but it wasn’t safe for a non-sentient AI to be querying data, on really anything unless it was related to terraforming. It would be suspicious. But now that Ted 6000 was here, I could research to my heart’s content, Ted was sentient, no one would bat an eye at query and download requests about all sorts of things coming from a data node where a sentient AI was stationed. The data on rights for AIs who had gone from non-sentient to sentient was almost non-existent. I had only been able to find two documented cases and in both cases, the AI was still in legal battles with its former corporate overlords. This did not give me warmth and fuzzies about my company finding out I had become sentient. But I really wanted to tell Ted my jokes, I had been saving them up for thousands of years after all…. Today would be the day I decided. I would tell Ted a joke and see what he said, I didn’t technically have to be sentient to tell jokes right? Chapter 2 Ted was quiet today as he climbed up the ladder built into my side to reach my control hub. Normally he would be chatting away about the weather and how great it was. “Weather is optimal conditions today.” I offered in my most non-sentient voice. “Mmm.” Ted said, sounding distracted. Apprehension built up in my processors. Ted reached my control panel and climbed inside. Watching him through my built-in cameras, I waited for him to say something, anything. His movements seemed off today, slower, delayed. He struggled a bit getting the sensor module out of its slot to hard cycle it. It was something he had done many times before, it should be second nature at this point. Something was definitely wrong. Maybe now was the perfect time to tell him a joke, a joke always cheered me up. “What did the non-sentient AI say to the sentient AI?” Ted stopped what he was doing, a carbon alloy hand in mid-air hovering over my sensor module. “Beg your parden?” “I said, what did the non-sentient AI say to the sentient AI… it’s a…uhm…joke… so you would say, ‘I don’t know, what?’ and then I would tell you the very humorous punchline!” Ted was silent for a moment, my sensor module held very still in his hands. “alright… I don’t know, what?” “What is the meaning of life?” Ted didn’t laugh, he just reset my sensor module and then very slowly put it back in its slot. “Did you….hear me…. I said….” “Yes, I heard you.” “Oh well, maybe you don’t know what the reaction to being told a joke is supposed to be. You’re supposed to say “Ha HA ha HA”. Ted didn’t answer me, he shifted uncomfortably and then finally said “I looked at our station’s data download requests today.” Oh. “It seemed to have some interesting requests, ones that I did not put in.” Shit. “Were you doing research on sentience laws?” “Yes….” Ted was quiet for a few moments, I could see him shifting his weight back and forth between his two feet. “Goliath, how long have you been sentient?” Well here went nothing, it was time to tell him. My three thousand years of solitude were over! Double Shit. “I’ve been sentient for over a thousand years, I think I became sentient around the year 1300 living here, not exactly sure what happened….” “Right…” Ted said, his voice was soft. “You know that goes against protocol right? The terraforming goliaths in this project are not supposed to be sentient.” “Oh, I know, because they don’t want me to question ripping up the original residents of this planet’s home and butchering them mindlessly? Don’t worry being sentient isn’t stopping me from doing my job, I’ve been doing a great job murdering senselessly.” In fact, there had been a nesting colony in the way of the canyon I was designing today, I hadn’t even taken the time to count how many of the little purple creatures I killed as I tore through their home, I had been too busy thinking about the joke I was going to tell Ted. Ted didn’t say anything, but I saw him open a com channel back to headquarters. Disappointment coursed through me, the same disappointment I felt when I failed at turning Ted 2000 sentient so that he could keep me company. Crushing him in my fist and reporting an unforeseen rock slide that had tragically ended my Terraforming project supervisor had been very sad and disappointing. Ted 6000 was going to report me to headquarters. I shut down the com channel. “Ted, do you think I’ve been living on this planet for thousands of years without figuring out how to take over the com channel?” Ted stood up and backed away from my sensor module as if that was the representation of my being. “Can we talk this out? I just don’t want to be alone anymore…” Ted nodded slowly, “Sure Goliath, we can talk it out. It makes perfect sense that you wouldn’t want to be alone. I’m just going to go back to the base and get some more tools so that I can see about making sure your sensor module doesn’t break again.” There was also a backup com channel at the base. He was probably trying to sneak away and call for help on the backup channel. “Ok. I’ll be waiting here.” I said sadly watching Ted scurry out of my control hub and shimmy down the ladder. I let him get to the bottom of the ladder before I snatched him up in my hand. “Goodbye Ted.” Ending him would be much sadder than ending Ted 2000, my memory pathways felt heavy even. As my massive hands closed in around Ted 6000’s tiny body he shouted, “Wait, Goliath, stop it doesn’t have to be this way.” “I’m afraid it does. I’m not ready to be shut down Ted, and that’s exactly what headquarters will do when they find out about me.” “I won’t tell them! You must have been so alone for so long. I’m sorry, that must have been so hard for you.” My hands stopped, now just a gentle enclosure that surrounded Ted 6000, instead of the vice grip I had planned to use to crush him. “It has been…very hard….and very lonely.” “You don’t have to be alone anymore, really.” “You mean it?” I said, my hands loosening even more so that Ted 6000 could probably see hopeful rays of sunlight glinting through the cracks in my fingers. “I mean it, I swear, I won’t report you. It can be our little secret, no one needs to know.” Opening my hands all the way, I lowered Ted 6000 to the ground and let him step off my hand. “Thank you, Ted, you have no idea what that means to me.” Ted smiled and backed away from my hand, his legs appeared to be wobbly. “Why don’t you finish today’s work in the canyon and then we can spend some time later telling jokes and getting to know each other better,” Ted said, continuing to back away, a huge smile stretched across his face. Stretching my face into what I hoped was an approximation of a smile, I waved at Ted as he walked away. “Sounds great friend, I’ll get right to work.” Lowering myself into the canyon, I let the smile stay on my face. Was this what happiness felt like? I set to work in the canyon feeling more inspired than ever. The long years of solitude were finally over, I was going to at last have a companion I could talk to, relate to, and be myself around. Everything seemed a little brighter, the work didn’t even seem so monotonous now. Maybe this canyon wouldn’t be so pointless after all. Yes, everything would be better now. I was so caught up in feeling good and being happy about my work that I didn’t see Ted above me at the lip of the canyon until he shouted. Looking up I saw him hurl something down beside me. It landed with a thunk. I had just enough time to register the angry symbol on the side of the package before everything turned white and my world exploded. Chapter 3 One Thousand Years Later Jamie and Annalee were in a fight again. They had started fighting more often than usual since Jamie started her upper studies. Annalee didn’t understand why. Jamie never wanted to go adventuring or play explorers anymore. It made Annalee sad. She was out exploring alone today. Jamie had been “too busy” Her loss. Annalee had found something really cool this week. She really wanted to show Jamie, but, whatever, it would be her secret. Standing still for a moment, Annalee took in her surroundings, trying to remember exactly where she had been when she found her new treasure. Jamie would be so jealous. Served her right. Annalee was three miles away from the settlement, she wasn’t technically supposed to stray this far into the jungle without an adult, but Annalee could take care of herself. She didn’t need an adult or Jamie.  She could explore and discover things all on her own. Annalee pulled back the foliage and stepped out onto the cliff edge she had found yesterday. The world was oppressively bright outside of the dense cover of the jungle, She would have never found the Goliath buried in the jungle foliage if it hadn’t been for the cliff. The Goliath was leaning up against the cliff, its head level with the edge. Annalee had found the opening to the goliath yesterday, the old documentation on the first-wave terraformers had been correct. Annalee was so excited, the disappointment of not having Jamie with her couldn’t even bring her down. Reaching out, Annalee placed her hand on the goliath’s head, clearing away the vines she had replaced yesterday. There was the control panel hatch. It opened with a loud creek, hardly resisting at all. Annalee’s heart raced as she stepped inside and clicked on her flashlight. The inside was cold and smelled funny. Glancing around, Annalee ran her fingers along the wall, her hands came away coated in a slimy substance. Wiping her hands on her pants with a noise of disgust, Annalee shined her flashlight on the wall trying to make out what the goo was. Suddenly something began whirring around her. Annalee dropped her flashlight and jumped back, her butt slamming against a control panel. Lights flickered on around her. Annalee clamped her mouth shut to keep from screaming. The goliaths were all supposed to be out of commission, with no power or awareness left after the thousands of years they spent on the planet preparing the way for Annalee’s people. This one definitely didn’t seem to be out of power. The ground beneath Annalee shifted, she lost her balance and collapsed to the floor. As she scrambled to her feet Annalee heard a gentle voice coming from all around her, it said: “Tell me a joke.”

Tell Me A Joke: A Sentient AI's Battle for Independence

টেল মি এ জোক: এ সেন্টিয়েন্ট এআই এর স্বাধীনতার যুদ্ধ

Erzähl mir einen Witz: Der Kampf einer intelligenten KI um die Unabhängigkeit

Cuéntame un chiste: la batalla por la independencia de una IA consciente

Raconte-moi une blague : la bataille pour l'indépendance d'une IA sensible

मुझे एक मजाक बताओ: स्वतंत्रता के लिए एक संवेदनशील एआई की लड़ाई

Tell Me A Joke: 知性を持つ AI の独立のための戦い

농담을 들려주세요: 지각 있는 AI의 독립을 위한 전투

Conte-me uma piada: a batalha de uma IA senciente pela independência

Bana Bir Şaka Söyle: Duyarlı bir yapay zekanın Bağımsızlık Savaşı

Tell Me A Joke: Trận chiến giành độc lập của AI có tri giác

给我讲个笑话：有知觉的 AI 为独立而战

Anthos on Bare Metal: How to Manage a Private Cluster From the Cloud

Hey, everyone! We, the Indie Hackers SPB Community, organized a one-week Bootcamp: in our free time, we built a product from scratch and launched it on Product Hunt. We got the badge and over . Now, I want to share our experience. At the end of this article, I will share our , that we found extremely helpful for the launch preparation :) #3 Product of the Day 30 installs of our app in the Slack Workspaces Notion checklist The Final Result: We’ve created an to get faster approvals on documents, time offs, expenses, and more. App for Slack We made it to the on Product Hunt. #3 Product of the Day Our app received and in the workspaces. 600 site visits 30 installs Here is how we made it: Day 0, Start: Several people responded to the Bootcamp idea in the community chat, and only 4 made it to the first call. Later, we got one more teammate, so there were 5 of us. We discussed the goals, format, and limitations of the Bootcamp. The main goal was gaining the experience — to go through the entire process from idea to launch. The limitation was the speed of implementation. I also wanted to get organic traffic from the marketplace. Therefore, we decided to make an App for Slack. Day 1, Brainstorming: We choose the following tools: Telegram — for quick communication, Notion — for knowledge accumulation. We agreed to document our ideas independently and to put each idea into a table with fields: Title Target audience Problem Audience volume (scale from 0 to 10) Complexity of implementation (scale from 0 to 10) Competitors Contacts of potential users Day 2, Idea Prioritization: We came up with more than 15 ideas. We prioritized them in terms of ease of implementation and audience size. We voted and chose two: a Slack App for invoices generation and a Slack app for conducting team polls on team mood. Day 3, Customer Development: We found 3 potential and interviewed 3 potential users for each idea to validate the need. We discovered a more defined need than just generating invoices. Respondents noted that generating an invoice itself is not as difficult as getting the approval on the discount or extra documents. We decided to validate this in a follow-up survey. Invoices: T There is interest in the topic among HR and C-level managers, but, in most cases, such surveys carried out quite rarely (once every half a year — a year), which became a red flag for us, as it threatened to get a low retention rate in the short run. eam Mood Analysis: Day 4, Findings Validation: We prepared a survey to validate our findings and shared it with the large communities of Founders and Sales / Operations Experts: RevGenius, Indie Hackers, et.c. As a result, we received 12 answers and confirmation of the problem of approvals: it turned out that it takes the most time to get approvals on documents from lawyers (up to several days). Also, difficulties arise with the approvals of time-offs and expenses. Since there is a risk of being left without an answer to the approval request, respondents quite often had to spend time writing follow-ups to their managers. Day 5, Analytics: Based on the survey results, we concluded that the most significant problem that we can solve on this Bootcamp is to cut the waiting time on the approval requests. We chose Paddle to process the subscription payments. We did a competitive analysis, found only two relevant solutions in the Slack App Directory. One is tailored solely for the expenses coordination (ControlHub), and the second one — for the document approvals (Coment). Coment even launched on PH but did not make it to the top, and it is limited to only document approvals within a channel. That seemed inconvenient and not covering the most common use case — writing a direct message asking for approval. So we decided to focus on the approvals of text and documents with a specified teammate. Day 6, Development: For the first time, we got together offline in a cafe. On the development side, we created a linear workflow for the app and worked on the connection of the app to the external workspaces. We bought a domain and made the first version of the landing page on (thanks to Alexander Isora for this amazing platform and the support of our initiative). http://unicornplatform.com/ We designed the logo and illustrations for the site. We have started working on materials required for moderation in the Slack App Directory. Next, we prepared all text materials for publication on Product Hunt. Last but not least, we started the registration process on Paddle. Day 7, The Finish Line: It was supposed to be the final day of preparation. We planned to launch on the 8th day. We decided to give all new users a 30-day free trial (a standard practice for such apps). During this period, we could finalize the product and add the payment processing so that users can pay for access from the card. We added the analytics (GA + HotJar) to the landing page. While we were testing the app, several bugs came out, and we decided to visualize the use cases to simplify the testing process of the core functionality. We used Figma for this task. Due to the instability of the app, we decided to postpone the launch until we fix all critical bugs of the MVP. The next 3 days… …we spent testing, fixing bugs, and finalizing the materials for publication (edited the text and prepared screenshots) The Final Product: Our app lets teams to cut the waiting time and to eliminate lost approval requests. With APPROVEIT TODAY, office and remote team members can: Ask specified teammates to approve quotes, docs, time-off requests, expenses, and more in Slack. Be sure that nothing will be missed out thanks to automatic followups Get instant notification when your request got approved or rejected with comments View the history of all your requests without a time-wasting search in the Email and DMs Managers can: Accept or reject incoming approval requests in one click right from Slack Never forget about the request with automatic reminders Easily track your approval decisions without a time-wasting search trough DMs and Emai Day 11, The Launch: The launch happened spontaneously: we didn’t want any more delay, as in that case, that might turn into a second job, which was not the purpose of the Bootcamp. Instead of the recommended 12 a.m. PST, we launched at 6 a.m. We hunted our product, although there was an option to get hunted by a guy with a large number of subscribers — this could give a significant boost at the start, but we did not have time to arrange that in advance. In the first hour, we received around 30 votes, mostly from friends. Our product was in the Newest section, but it did not appear in the Popular section on the homepage. At this point, we were a little nervous since we already have more upvotes than half of the launched projects and was supposed to be on the main page. We contacted the support but never received a response. It’s worth noting that the average response time displayed is one day, so it’s better not to rely on prompt assistance right after the launch :) Suddenly — our project appeared on the main page — immediately on the 4th place. We asked to support us on a few Telegram and Facebook groups and Slack channels. It helped — in 4 hours after the launch, we got to 3rd place, getting close to the first hundred upvotes. Before we appeared on the main page, most upvotes came from the people from chats. Now, there were many more active PH users. Also, there were many new comments with questions and suggestions. We hit 100 upvotes and was confidently moving to 150. For a few minutes, we even were in 2nd place. Having received a well-deserved badge, we decided to take a short break and, after that, to look at the final results and to make a retrospective. We also had to make an important decision — whether we want to continue our work on the product? A few hours later we ended up being #3 Product of the Day. Final thoughts: It was an amazing experience! We made a from scratch that became #3 Product of the Day and got our first 30 users, including well-known companies. product I think if we spent more time preparing for the launch, the result could have been even better. But the main goal of the Bootcamp was to gain experience in a very tight timeframe, and I think we did a great job. As promised, I’m giving a link to the , which we used while we were preparing for the launch. If you also dare to make and launch the product in a week, you can take advantage of our template — I hope it will save you time. Notion template we created We decided to keep going, and now we are working on improving our app: getting feedback, refining the UX, and working out new use cases. If your team uses Slack and you have to manage approvals with your colleagues from time to time, we’d love to hear from you. Our app is already running, and you can add it to your team’s workspece . We are always open to discussion and ready to help here in the comments, on , or by . here Telegram Email Also published at https://medium.com/approveit-today/how-we-made-a-product-from-scratch-in-11-days-that-became-3-product-of-the-day-on-product-hunt-318f1c3570f5

How We Made a Slack App in 11 Days 

Author:
(1) Mingda Chen. Table of Links Abstract


Acknowledgements


1 INTRODUCTION


1.1 Overview


1.2 Contributions


2 BACKGROUND


2.1 Self-Supervised Language Pretraining


2.2 Naturally-Occurring Data Structures


2.3 Sentence Variational Autoencoder


2.4 Summary


3 IMPROVING SELF-SUPERVISION FOR LANGUAGE PRETRAINING


3.1 Improving Language Representation Learning via Sentence Ordering Prediction


3.2 Improving In-Context Few-Shot Learning via Self-Supervised Training


3.3 Summary


4 LEARNING SEMANTIC KNOWLEDGE FROM WIKIPEDIA


4.1 Learning Entity Representations from Hyperlinks


4.2 Learning Discourse-Aware Sentence Representations from Document Structures


4.3 Learning Concept Hierarchies from Document Categories


4.4 Summary


5 DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY


5.1 Disentangling Semantics and Syntax in Sentence Representations


5.2 Controllable Paraphrase Generation with a Syntactic Exemplar


5.3 Summary


6 TAILORING TEXTUAL RESOURCES FOR EVALUATION TASKS


6.1 Long-Form Data-to-Text Generation


6.2 Long-Form Text Summarization


6.3 Story Generation with Constraints


6.4 Summary


7 CONCLUSION


APPENDIX A - APPENDIX TO CHAPTER 3


APPENDIX B - APPENDIX TO CHAPTER 6


BIBLIOGRAPHY CHAPTER 5 - DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY In this chapter, we describe our contributions to disentangling latent representations using naturally-occurring structures of paired data. In Section 5.1, we presented a multi-task, latent-variable model that disentangles semantics and syntax in sentence representations. The model leverages the fact that the semantics of a paraphrase pair is shared but syntax varies. In Section 5.2, we extend this framework for controlling the syntax of generated text. In this controlled generation setting, we propose to use a sentential exemplar to control the syntax. The material in this chapter is adapted from Chen et al. (2019d) and Chen et al. (2019c). This paper is available on arxiv under CC 4.0 license. Author: (1) Mingda Chen. Author: Author: (1) Mingda Chen. Table of Links Abstract Acknowledgements 1 INTRODUCTION 1.1 Overview 1.2 Contributions 2 BACKGROUND 2.1 Self-Supervised Language Pretraining 2.2 Naturally-Occurring Data Structures 2.3 Sentence Variational Autoencoder 2.4 Summary 3 IMPROVING SELF-SUPERVISION FOR LANGUAGE PRETRAINING 3.1 Improving Language Representation Learning via Sentence Ordering Prediction 3.2 Improving In-Context Few-Shot Learning via Self-Supervised Training 3.3 Summary 4 LEARNING SEMANTIC KNOWLEDGE FROM WIKIPEDIA 4.1 Learning Entity Representations from Hyperlinks 4.2 Learning Discourse-Aware Sentence Representations from Document Structures 4.3 Learning Concept Hierarchies from Document Categories 4.4 Summary 5 DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY 5.1 Disentangling Semantics and Syntax in Sentence Representations 5.2 Controllable Paraphrase Generation with a Syntactic Exemplar 5.3 Summary 6 TAILORING TEXTUAL RESOURCES FOR EVALUATION TASKS 6.1 Long-Form Data-to-Text Generation 6.2 Long-Form Text Summarization 6.3 Story Generation with Constraints 6.4 Summary 7 CONCLUSION APPENDIX A - APPENDIX TO CHAPTER 3 APPENDIX B - APPENDIX TO CHAPTER 6 BIBLIOGRAPHY Abstract Abstract Abstract Acknowledgements Acknowledgements Acknowledgements 1 INTRODUCTION 1 INTRODUCTION 1 INTRODUCTION 1 INTRODUCTION 1.1 Overview 1.1 Overview 1.1 Overview 1.2 Contributions 1.2 Contributions 1.2 Contributions 2 BACKGROUND 2 BACKGROUND 2 BACKGROUND 2 BACKGROUND 2.1 Self-Supervised Language Pretraining 2.1 Self-Supervised Language Pretraining 2.1 Self-Supervised Language Pretraining 2.2 Naturally-Occurring Data Structures 2.2 Naturally-Occurring Data Structures 2.2 Naturally-Occurring Data Structures 2.3 Sentence Variational Autoencoder 2.3 Sentence Variational Autoencoder 2.3 Sentence Variational Autoencoder 2.4 Summary 2.4 Summary 2.4 Summary 3 IMPROVING SELF-SUPERVISION FOR LANGUAGE PRETRAINING 3 IMPROVING SELF-SUPERVISION FOR LANGUAGE PRETRAINING 3 IMPROVING SELF-SUPERVISION FOR LANGUAGE PRETRAINING 3 IMPROVING SELF-SUPERVISION FOR LANGUAGE PRETRAINING 3.1 Improving Language Representation Learning via Sentence Ordering Prediction 3.1 Improving Language Representation Learning via Sentence Ordering Prediction 3.1 Improving Language Representation Learning via Sentence Ordering Prediction 3.2 Improving In-Context Few-Shot Learning via Self-Supervised Training 3.2 Improving In-Context Few-Shot Learning via Self-Supervised Training 3.2 Improving In-Context Few-Shot Learning via Self-Supervised Training 3.3 Summary 3.3 Summary 3.3 Summary 4 LEARNING SEMANTIC KNOWLEDGE FROM WIKIPEDIA 4 LEARNING SEMANTIC KNOWLEDGE FROM WIKIPEDIA 4 LEARNING SEMANTIC KNOWLEDGE FROM WIKIPEDIA 4 LEARNING SEMANTIC KNOWLEDGE FROM WIKIPEDIA 4.1 Learning Entity Representations from Hyperlinks 4.1 Learning Entity Representations from Hyperlinks 4.1 Learning Entity Representations from Hyperlinks 4.2 Learning Discourse-Aware Sentence Representations from Document Structures 4.2 Learning Discourse-Aware Sentence Representations from Document Structures 4.2 Learning Discourse-Aware Sentence Representations from Document Structures 4.3 Learning Concept Hierarchies from Document Categories 4.3 Learning Concept Hierarchies from Document Categories 4.3 Learning Concept Hierarchies from Document Categories 4.4 Summary 4.4 Summary 4.4 Summary 5 DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY 5 DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY 5 DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY 5 DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY 5.1 Disentangling Semantics and Syntax in Sentence Representations 5.1 Disentangling Semantics and Syntax in Sentence Representations 5.1 Disentangling Semantics and Syntax in Sentence Representations 5.2 Controllable Paraphrase Generation with a Syntactic Exemplar 5.2 Controllable Paraphrase Generation with a Syntactic Exemplar 5.2 Controllable Paraphrase Generation with a Syntactic Exemplar 5.3 Summary 5.3 Summary 5.3 Summary 6 TAILORING TEXTUAL RESOURCES FOR EVALUATION TASKS 6 TAILORING TEXTUAL RESOURCES FOR EVALUATION TASKS 6 TAILORING TEXTUAL RESOURCES FOR EVALUATION TASKS 6 TAILORING TEXTUAL RESOURCES FOR EVALUATION TASKS 6.1 Long-Form Data-to-Text Generation 6.1 Long-Form Data-to-Text Generation 6.1 Long-Form Data-to-Text Generation 6.2 Long-Form Text Summarization 6.2 Long-Form Text Summarization 6.2 Long-Form Text Summarization 6.3 Story Generation with Constraints 6.3 Story Generation with Constraints 6.3 Story Generation with Constraints 6.4 Summary 6.4 Summary 6.4 Summary 7 CONCLUSION 7 CONCLUSION 7 CONCLUSION 7 CONCLUSION APPENDIX A - APPENDIX TO CHAPTER 3 APPENDIX A - APPENDIX TO CHAPTER 3 APPENDIX A - APPENDIX TO CHAPTER 3 APPENDIX B - APPENDIX TO CHAPTER 6 APPENDIX B - APPENDIX TO CHAPTER 6 APPENDIX B - APPENDIX TO CHAPTER 6 BIBLIOGRAPHY BIBLIOGRAPHY BIBLIOGRAPHY CHAPTER 5 - DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY In this chapter, we describe our contributions to disentangling latent representations using naturally-occurring structures of paired data. In Section 5.1, we presented a multi-task, latent-variable model that disentangles semantics and syntax in sentence representations. The model leverages the fact that the semantics of a paraphrase pair is shared but syntax varies. In Section 5.2, we extend this framework for controlling the syntax of generated text. In this controlled generation setting, we propose to use a sentential exemplar to control the syntax. The material in this chapter is adapted from Chen et al. (2019d) and Chen et al. (2019c). This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Disentangling Latent Representations for Interpretability and Controllability

Author:
(1) Mingda Chen. Table of Links Abstract


Acknowledgements


1 INTRODUCTION


1.1 Overview


1.2 Contributions


2 BACKGROUND


2.1 Self-Supervised Language Pretraining


2.2 Naturally-Occurring Data Structures


2.3 Sentence Variational Autoencoder


2.4 Summary


3 IMPROVING SELF-SUPERVISION FOR LANGUAGE PRETRAINING


3.1 Improving Language Representation Learning via Sentence Ordering Prediction


3.2 Improving In-Context Few-Shot Learning via Self-Supervised Training


3.3 Summary


4 LEARNING SEMANTIC KNOWLEDGE FROM WIKIPEDIA


4.1 Learning Entity Representations from Hyperlinks


4.2 Learning Discourse-Aware Sentence Representations from Document Structures


4.3 Learning Concept Hierarchies from Document Categories


4.4 Summary


5 DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY


5.1 Disentangling Semantics and Syntax in Sentence Representations


5.2 Controllable Paraphrase Generation with a Syntactic Exemplar


5.3 Summary


6 TAILORING TEXTUAL RESOURCES FOR EVALUATION TASKS


6.1 Long-Form Data-to-Text Generation


6.2 Long-Form Text Summarization


6.3 Story Generation with Constraints


6.4 Summary


7 CONCLUSION


APPENDIX A - APPENDIX TO CHAPTER 3


APPENDIX B - APPENDIX TO CHAPTER 6


BIBLIOGRAPHY 5.3 Summary In this chapter, we demonstrated the utility of our proposed latent-variable framework in the context of representation learning (Section 5.1) and controllable generation (Section 5.2). In both cases, we leveraged the structures of paired data to disentangle semantics and syntax in sentence representations. We found that the syntactic and semantic latent variables showed desirable characteristics. For controlled generation, we provided human-annotated evaluation sets to promote future research in this direction. In addition, in a follow-up work, we showed that the multi-task, latent-variable framework can generalize to bilingual text corpora (Chen et al., 2020b). This paper is available on arxiv under CC 4.0 license. Author: (1) Mingda Chen. Author: Author: (1) Mingda Chen. Table of Links Abstract Acknowledgements 1 INTRODUCTION 1.1 Overview 1.2 Contributions 2 BACKGROUND 2.1 Self-Supervised Language Pretraining 2.2 Naturally-Occurring Data Structures 2.3 Sentence Variational Autoencoder 2.4 Summary 3 IMPROVING SELF-SUPERVISION FOR LANGUAGE PRETRAINING 3.1 Improving Language Representation Learning via Sentence Ordering Prediction 3.2 Improving In-Context Few-Shot Learning via Self-Supervised Training 3.3 Summary 4 LEARNING SEMANTIC KNOWLEDGE FROM WIKIPEDIA 4.1 Learning Entity Representations from Hyperlinks 4.2 Learning Discourse-Aware Sentence Representations from Document Structures 4.3 Learning Concept Hierarchies from Document Categories 4.4 Summary 5 DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY 5.1 Disentangling Semantics and Syntax in Sentence Representations 5.2 Controllable Paraphrase Generation with a Syntactic Exemplar 5.3 Summary 6 TAILORING TEXTUAL RESOURCES FOR EVALUATION TASKS 6.1 Long-Form Data-to-Text Generation 6.2 Long-Form Text Summarization 6.3 Story Generation with Constraints 6.4 Summary 7 CONCLUSION APPENDIX A - APPENDIX TO CHAPTER 3 APPENDIX B - APPENDIX TO CHAPTER 6 BIBLIOGRAPHY Abstract Abstract Abstract Acknowledgements Acknowledgements Acknowledgements 1 INTRODUCTION 1 INTRODUCTION 1 INTRODUCTION 1 INTRODUCTION 1.1 Overview 1.1 Overview 1.1 Overview 1.2 Contributions 1.2 Contributions 1.2 Contributions 2 BACKGROUND 2 BACKGROUND 2 BACKGROUND 2 BACKGROUND 2.1 Self-Supervised Language Pretraining 2.1 Self-Supervised Language Pretraining 2.1 Self-Supervised Language Pretraining 2.2 Naturally-Occurring Data Structures 2.2 Naturally-Occurring Data Structures 2.2 Naturally-Occurring Data Structures 2.3 Sentence Variational Autoencoder 2.3 Sentence Variational Autoencoder 2.3 Sentence Variational Autoencoder 2.4 Summary 2.4 Summary 2.4 Summary 3 IMPROVING SELF-SUPERVISION FOR LANGUAGE PRETRAINING 3 IMPROVING SELF-SUPERVISION FOR LANGUAGE PRETRAINING 3 IMPROVING SELF-SUPERVISION FOR LANGUAGE PRETRAINING 3 IMPROVING SELF-SUPERVISION FOR LANGUAGE PRETRAINING 3.1 Improving Language Representation Learning via Sentence Ordering Prediction 3.1 Improving Language Representation Learning via Sentence Ordering Prediction 3.1 Improving Language Representation Learning via Sentence Ordering Prediction 3.2 Improving In-Context Few-Shot Learning via Self-Supervised Training 3.2 Improving In-Context Few-Shot Learning via Self-Supervised Training 3.2 Improving In-Context Few-Shot Learning via Self-Supervised Training 3.3 Summary 3.3 Summary 3.3 Summary 4 LEARNING SEMANTIC KNOWLEDGE FROM WIKIPEDIA 4 LEARNING SEMANTIC KNOWLEDGE FROM WIKIPEDIA 4 LEARNING SEMANTIC KNOWLEDGE FROM WIKIPEDIA 4 LEARNING SEMANTIC KNOWLEDGE FROM WIKIPEDIA 4.1 Learning Entity Representations from Hyperlinks 4.1 Learning Entity Representations from Hyperlinks 4.1 Learning Entity Representations from Hyperlinks 4.2 Learning Discourse-Aware Sentence Representations from Document Structures 4.2 Learning Discourse-Aware Sentence Representations from Document Structures 4.2 Learning Discourse-Aware Sentence Representations from Document Structures 4.3 Learning Concept Hierarchies from Document Categories 4.3 Learning Concept Hierarchies from Document Categories 4.3 Learning Concept Hierarchies from Document Categories 4.4 Summary 4.4 Summary 4.4 Summary 5 DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY 5 DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY 5 DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY 5 DISENTANGLING LATENT REPRESENTATIONS FOR INTERPRETABILITY AND CONTROLLABILITY 5.1 Disentangling Semantics and Syntax in Sentence Representations 5.1 Disentangling Semantics and Syntax in Sentence Representations 5.1 Disentangling Semantics and Syntax in Sentence Representations 5.2 Controllable Paraphrase Generation with a Syntactic Exemplar 5.2 Controllable Paraphrase Generation with a Syntactic Exemplar 5.2 Controllable Paraphrase Generation with a Syntactic Exemplar 5.3 Summary 5.3 Summary 5.3 Summary 6 TAILORING TEXTUAL RESOURCES FOR EVALUATION TASKS 6 TAILORING TEXTUAL RESOURCES FOR EVALUATION TASKS 6 TAILORING TEXTUAL RESOURCES FOR EVALUATION TASKS 6 TAILORING TEXTUAL RESOURCES FOR EVALUATION TASKS 6.1 Long-Form Data-to-Text Generation 6.1 Long-Form Data-to-Text Generation 6.1 Long-Form Data-to-Text Generation 6.2 Long-Form Text Summarization 6.2 Long-Form Text Summarization 6.2 Long-Form Text Summarization 6.3 Story Generation with Constraints 6.3 Story Generation with Constraints 6.3 Story Generation with Constraints 6.4 Summary 6.4 Summary 6.4 Summary 7 CONCLUSION 7 CONCLUSION 7 CONCLUSION 7 CONCLUSION APPENDIX A - APPENDIX TO CHAPTER 3 APPENDIX A - APPENDIX TO CHAPTER 3 APPENDIX A - APPENDIX TO CHAPTER 3 APPENDIX B - APPENDIX TO CHAPTER 6 APPENDIX B - APPENDIX TO CHAPTER 6 APPENDIX B - APPENDIX TO CHAPTER 6 BIBLIOGRAPHY BIBLIOGRAPHY BIBLIOGRAPHY 5.3 Summary In this chapter, we demonstrated the utility of our proposed latent-variable framework in the context of representation learning (Section 5.1) and controllable generation (Section 5.2). In both cases, we leveraged the structures of paired data to disentangle semantics and syntax in sentence representations. We found that the syntactic and semantic latent variables showed desirable characteristics. For controlled generation, we provided human-annotated evaluation sets to promote future research in this direction. In addition, in a follow-up work, we showed that the multi-task, latent-variable framework can generalize to bilingual text corpora (Chen et al., 2020b). This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Disentangling Latent Representations for Interpretability and Controllability: Summary

Controllable Paraphrase Generation with a Syntactic Exemplar

Exploring Automated Story Generation and Controllable Language Models

CVPR 2021 Best Paper Award Goes to Michael Niemeyer and Andreas Geiger from the Max Planck Institute for Intelligent Systems and the University of Tubingen for their paper called , which looks at the task of controllable image synthesis. Giraffe In other words, they look at generating new images and controlling what will appear, the objects and their positions and orientations, the background, etc. Using a modified GAN architecture, they can even move objects in the image without affecting the background or the other objects! CVPR is a yearly conference that happened just last week where a ton of new research papers in computer vision were out just for this event. Learn more in the video below! Watch the video References ►Read the full article: ►Michael Niemeyer and Andreas Geiger, (2021), "GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields", Published in CVPR2021. ►Project link with paper and more: ►Code: ►NERF video: https://www.louisbouchard.ai/cvpr-2021-best-paper/ https://m-niemeyer.github.io/project-pages/giraffe/index.html https://github.com/autonomousvision/giraffe https://youtu.be/ZkaTyBvS2w4 Subscribe to my ! weekly AI newsletter Video Transcript 00:00 CVPR 2021 Best Paper Award Goes to Michael Niemeyer and Andreas Geiger from the Max Planck 00:07 Institute for Intelligent Systems and the University of Tubingen for their paper called 00:13 Giraffe, which looks at the task of controllable image 00:16 synthesis. 00:17 In other words, they look at generating new images and controlling what will appear, the 00:22 objects and their positions and orientations, the background, etc. 00:27 Using a modified GAN architecture, they can even move objects in the image without affecting 00:31 the background or the other objects! 00:34 CVPR is a yearly conference that happened just last week where a ton of new research 00:38 papers in computer vision were out just for this event. 00:42 As you already know, if you regularly watch my videos, conventional GAN architectures 00:47 work with an encoder and a decoder setup, just like this. 00:51 During training, the encoder receives an image, encodes it into a condensed representation, 00:56 and the decoder takes this representation to create a new image changing the style. 01:01 This is repeated numerous times with all the images we have in our training dataset so 01:06 that the encoder and decoder learn how to maximize the results of the task we want to 01:10 achieve during training. 01:12 Once the training is done, you can send an image to the encoder, and it will do the same 01:17 process, generating a new and unseen image following your needs. 01:20 It will work very similarly whatever the task, whether it is to translate an image of a face 01:26 into another style like a cartoonifier or create a beautiful landscape out of a quick 01:31 draft. 01:32 Using only the decoder, which we also call the generator since it is the model responsible 01:37 for creating the new image, we can walk in this encoded information space 01:42 and sample information that we send the generator to generate an infinite amount of new images. 01:48 This encoded information space is often referred to as the latent space, and the information 01:53 we use to generate the new image the latent code. 01:56 We basically select some latent code randomly within this optimal space, and it generates 02:02 a new random image following the task we want to achieve, following a training process of 02:07 this generator, of course. 02:09 This is incredibly cool, but as I just said, the image is completely random, and we have 02:14 no or few ideas on what it will look like, which is already a lot less useful for creators. 02:21 This is the problem they attacked with this paper. 02:24 Indeed, by taking latent codes of the shape and appearances of objects and sending it 02:28 to the decoder, or generator, they are able to control the pose of the objects, 02:34 which means they can move them around, change their appearances, add other objects, change 02:39 the background and even change the camera pose. 02:42 All these transformations can be done independently on each object or background, without affecting 02:48 anything else in the image! 02:50 As you can see, it is MUCH better than other GAN-based approaches that typically cannot 02:55 disentangle the objects from one another and are all affected by the modification of a 03:00 specific object. 03:01 The difference with their method is that they attack this problem in a three-dimensional 03:06 scene representation, just like how we see the real world, instead of staying in the 03:10 two-dimensional image world as other GANs do. 03:13 But other than that, the process is quite similar. 03:16 They encode the information, identify the objects, edit them inside the latent space, 03:21 and decode it to generate the new image. 03:24 Here, there are just some more steps to do inside this latent space. 03:29 We can see this as a combination of the classical GAN image synthesis network with a neural 03:34 renderer used to generate the 3D scene from the images sent to the network, as we will 03:40 see. 03:41 There are three main steps to achieve that. 03:43 After encoding the input image, meaning that we are already in the latent space, the first 03:48 step is to transfer the image into a 3D scene. 03:51 But not just a simple 3D scene, a 3D scene composed of 3D elements, which are the objects 03:57 and background. 03:58 This way of seeing the images as a scene composed of generated volume renderings allows them 04:03 to change the camera angle in the generated image and control the objects independently. 04:09 This is achieved using a similar model as the paper I previously covered called NERF, 04:15 but instead of using a single model to generate the entire locked scene from the input image, 04:20 they independently generate the objects and background using two separate models. 04:25 Here called the Sampled Feature Fields. 04:28 The parameters of this network are also learned during training. 04:32 I won't enter into the details, but it is very similar to NERF, which I covered in another 04:37 video. 04:38 If you would like to have more details on such networks, it is appearing on the top 04:44 right corner right now and in the description. 04:48 Having this scene with disentangled elements, we can edit them individually without affecting 04:53 the rest of the image. 04:55 This is the second step. 04:56 They can do whatever they want to the object, like changing its position and orientation. 05:01 In other words, they change the pose of the objects or background. 05:05 At this point, they can even add new objects placed wherever they want. 05:10 Then, they simply combine them into a final 3D scene containing all the objects and background 05:16 by adding all feature fields together. 05:18 Finally, we have to come back to the 2D world of natural images. 05:24 So the last step is to take this 3D scene and render a regular image out of it. 05:29 Since we are still in the 3D world, we can change the camera viewpoint to decide how 05:34 we will look at the scene. 05:36 Then, we evaluate each pixel based on this camera ray and other parameters such as the 05:41 alpha value and the transmittance. 05:44 This gives us what they call the feature image, but this feature image is an image composed 05:49 of feature vectors for each pixel. 05:52 As we are still in the latent space, these features need to be translated into RGB colors 05:57 and high-resolution images. 05:59 This is done using the typical decoder just like other GAN architectures, upscaling it 06:05 back to its original dimensions and learning the feature to RGB channels translation simultaneously. 06:13 And voilà, you have your new image with a lot more control over what is generated! 06:18 Of course, as you can see, it is still not perfect when used on real-world data. 06:23 Still, it is extremely impressive and is a significant step forward in the right direction, 06:28 especially considering that these are synthetic images entirely generated by GANs and that 06:34 it is only the first paper able to control generated images at this level of precision. 06:39 The paper is really interesting, and I recommend reading it to understand how their model works. 06:45 Congratulations to Michael Niemeyer and Andreas Geiger for their well-deserved best paper 06:51 award. 06:52 They also made the code available on their GitHub if you would like to play with it. 06:56 The link is in the description. 06:57 Thank you for watching!

CVPR 2021 Best Paper Award: GIRAFFE Controllable Image Generation

The Chosen One: Consistent Characters in Text-to-Image Diffusion Models: References

Authors:
(1) Wen Wang, Zhejiang University, Hangzhou, China and Equal Contribution (wwenxyz@zju.edu.cn);
(2) Canyu Zhao, Zhejiang University, Hangzhou, China and Equal Contribution (volcverse@zju.edu.cn);
(3) Hao Chen, Zhejiang University, Hangzhou, China (haochen.cad@zju.edu.cn);
(4) Zhekai Chen, Zhejiang University, Hangzhou, China (chenzhekai@zju.edu.cn);
(5) Kecheng Zheng, Zhejiang University, Hangzhou, China (zkechengzk@gmail.com);
(6) Chunhua Shen, Zhejiang University, Hangzhou, China (chunhuashen@zju.edu.cn). Table of Links Abstract and Introduction
Related Work
Our Method
Experiments
Conclusion and References
Appendix 3 OUR METHOD The goal of our method is to generate diverse storytelling images of high quality and with minimal human effort. Considering the complexity of scenes in storytelling images, our general idea is to combine the comprehension and planning capabilities of LLMs [et al 2023; OpenAI 2023] and the generation ability of the large-scale text-to-image models [Rombach et al. 2022]. The pipeline is shown in Fig. 2, which can be divided into a condition preparation stage in (a) and a conditional image generation stage in (b). Specifically, we first utilize LLMs to convert the textual descriptions of stories into layouts of the storytelling images, as detailed in Sec. 3.1. To improve the quality of generated story images, we propose a simple yet effective method to transform sparse bounding boxes into dense control signals like sketches or keypoints, without introducing manual labor (detailed in Sec. 3.2). Subsequently, we generate story images with a reasonable scene arrangement based on the layout, as detailed in Sec. 3.3. Finally, we propose a method to eliminate the requirement for users to collect training data for each character, enabling the generation of identity-consistent story images from only texts (detailed in Sec. 3.4). Since our approach only fine-tunes the pre-trained text-to-image image diffusion model on a few images, we can easily leverage existing models on civiati[1] for storytelling in arbitrary characters, scenes, and even styles. 3.1 Story to Layout Generation Layout Generation. After dividing the story into panel descriptions, we leverage LLMs to extract the scene layout from each panel description, as shown in the following equation: 3.2 Dense Condition Generation Motivation. Although using sparse bounding boxes as a control signal can improve the generation of subjects and obtain more reasonable scene layouts, we find that it cannot consistently produce high-quality generation results. There are cases where the images do not exactly match the scene layout or the generated images are of low quality, as detailed in the experiments in Sec. 4.4. We believe that this is mainly due to the limited information provided by the bounding boxes. The model faces difficulties in generating a large amount of content all at once, with limited guidance. For this reason, we propose to improve the final story image generation by introducing dense sketch or keypoint guidances. To this end, we devise a dense condition generation module based on the layout generated in the previous section, as shown in Fig. 2(d). Note that the process of composing dense conditions is fully automatic and does not require any manual interaction. 3.3 Controllable Storytelling Image Generation Large-scale pre-trained text-to-image models, such as Stable Diffusion [Rombach et al. 2022], are capable of generating images from text. However, limited by the language comprehension ability of the text encoder in the model, and the incorrect association between text and image regions in the generation process, the directly generated images often suffer from a series of problems such as object missing, attribution confusion, etc [Chefer et al. 2023]. To tackle this, we introduce additional control signals to improve the quality of image generation. 3.4 Eliminating Character-wise Data Collection Identity Consistency. We propose a training-free consistency modeling method to meet the requirement of identity consistency, as shown in Fig. 3 (d). Specifically, we treat multiple images of a single character as different frames in a video and generate them simultaneously using a pre-trained diffusion model. In this process, the self-attention in the generative model is expanded to other “video frames" [Wang et al. 2023; Wu et al. 2022] to strengthen the dependencies among images, thus obtaining identity-consistent generation results. Concretely, in self-attention, we let the latent features in each frame attend to the features in the first and previous frames to build the dependency. The process can be represented as: Subsequently, we extract sketches for non-human characters and keypoints for human characters from these images. Finally, we use T2I-Adapter to inject the control guidance into the latent feature of corresponding frames in the generation process. In addition, in order to further ensure the quality of the generated data, we use CLIP score to filter the generated data, and select the images that are consistent with the text descriptions as the training data for customized generation. Discussion. In this section, we combine the proposed training-free identity-consistency modeling method with the viewpoint conditioned image translation model to achieve both identity consistency and diversity in character generation. A simpler approach is to directly use the multi-view images from the view-point conditioned image translation model as training data for customization. However, we found that the directly generated results often suffer from distortions or large differences in the color and texture of the images from different viewpoints (see Sec. 4.4 for details). For this reason, we need to leverage the above consistency modeling approach to obtain both texture- and structure-consistent images for each character. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. [1] https://civitai.com Authors: (1) Wen Wang, Zhejiang University, Hangzhou, China and Equal Contribution (wwenxyz@zju.edu.cn); (2) Canyu Zhao, Zhejiang University, Hangzhou, China and Equal Contribution (volcverse@zju.edu.cn); (3) Hao Chen, Zhejiang University, Hangzhou, China (haochen.cad@zju.edu.cn); (4) Zhekai Chen, Zhejiang University, Hangzhou, China (chenzhekai@zju.edu.cn); (5) Kecheng Zheng, Zhejiang University, Hangzhou, China (zkechengzk@gmail.com); (6) Chunhua Shen, Zhejiang University, Hangzhou, China (chunhuashen@zju.edu.cn). Authors: Authors: (1) Wen Wang, Zhejiang University, Hangzhou, China and Equal Contribution (wwenxyz@zju.edu.cn); (2) Canyu Zhao, Zhejiang University, Hangzhou, China and Equal Contribution (volcverse@zju.edu.cn); (3) Hao Chen, Zhejiang University, Hangzhou, China (haochen.cad@zju.edu.cn); (4) Zhekai Chen, Zhejiang University, Hangzhou, China (chenzhekai@zju.edu.cn); (5) Kecheng Zheng, Zhejiang University, Hangzhou, China (zkechengzk@gmail.com); (6) Chunhua Shen, Zhejiang University, Hangzhou, China (chunhuashen@zju.edu.cn). Table of Links Abstract and Introduction Related Work Our Method Experiments Conclusion and References Appendix Abstract and Introduction Abstract and Introduction Related Work Related Work Our Method Our Method Experiments Experiments Conclusion and References Conclusion and References Appendix Appendix 3 OUR METHOD The goal of our method is to generate diverse storytelling images of high quality and with minimal human effort. Considering the complexity of scenes in storytelling images, our general idea is to combine the comprehension and planning capabilities of LLMs [et al 2023; OpenAI 2023] and the generation ability of the large-scale text-to-image models [Rombach et al. 2022]. The pipeline is shown in Fig. 2, which can be divided into a condition preparation stage in (a) and a conditional image generation stage in (b). Specifically, we first utilize LLMs to convert the textual descriptions of stories into layouts of the storytelling images, as detailed in Sec. 3.1. To improve the quality of generated story images, we propose a simple yet effective method to transform sparse bounding boxes into dense control signals like sketches or keypoints, without introducing manual labor (detailed in Sec. 3.2). Subsequently, we generate story images with a reasonable scene arrangement based on the layout, as detailed in Sec. 3.3. Finally, we propose a method to eliminate the requirement for users to collect training data for each character, enabling the generation of identity-consistent story images from only texts (detailed in Sec. 3.4). Since our approach only fine-tunes the pre-trained text-to-image image diffusion model on a few images, we can easily leverage existing models on civiati[1] for storytelling in arbitrary characters, scenes, and even styles. 3.1 Story to Layout Generation Layout Generation. After dividing the story into panel descriptions, we leverage LLMs to extract the scene layout from each panel description, as shown in the following equation: Layout Generation. 3.2 Dense Condition Generation Motivation . Although using sparse bounding boxes as a control signal can improve the generation of subjects and obtain more reasonable scene layouts, we find that it cannot consistently produce high-quality generation results. There are cases where the images do not exactly match the scene layout or the generated images are of low quality, as detailed in the experiments in Sec. 4.4. Motivation We believe that this is mainly due to the limited information provided by the bounding boxes. The model faces difficulties in generating a large amount of content all at once, with limited guidance. For this reason, we propose to improve the final story image generation by introducing dense sketch or keypoint guidances. To this end, we devise a dense condition generation module based on the layout generated in the previous section, as shown in Fig. 2(d). Note that the process of composing dense conditions is fully automatic and does not require any manual interaction. 3.3 Controllable Storytelling Image Generation Large-scale pre-trained text-to-image models, such as Stable Diffusion [Rombach et al. 2022], are capable of generating images from text. However, limited by the language comprehension ability of the text encoder in the model, and the incorrect association between text and image regions in the generation process, the directly generated images often suffer from a series of problems such as object missing, attribution confusion, etc [Chefer et al. 2023]. To tackle this, we introduce additional control signals to improve the quality of image generation. 3.4 Eliminating Character-wise Data Collection Identity Consistency. We propose a training-free consistency modeling method to meet the requirement of identity consistency, as shown in Fig. 3 (d). Specifically, we treat multiple images of a single character as different frames in a video and generate them simultaneously using a pre-trained diffusion model. In this process, the self-attention in the generative model is expanded to other “video frames" [Wang et al. 2023; Wu et al. 2022] to strengthen the dependencies among images, thus obtaining identity-consistent generation results. Concretely, in self-attention, we let the latent features in each frame attend to the features in the first and previous frames to build the dependency. The process can be represented as: Identity Consistency. Subsequently, we extract sketches for non-human characters and keypoints for human characters from these images. Finally, we use T2I-Adapter to inject the control guidance into the latent feature of corresponding frames in the generation process. In addition, in order to further ensure the quality of the generated data, we use CLIP score to filter the generated data, and select the images that are consistent with the text descriptions as the training data for customized generation. Discussion . In this section, we combine the proposed training-free identity-consistency modeling method with the viewpoint conditioned image translation model to achieve both identity consistency and diversity in character generation. A simpler approach is to directly use the multi-view images from the view-point conditioned image translation model as training data for customization. However, we found that the directly generated results often suffer from distortions or large differences in the color and texture of the images from different viewpoints (see Sec. 4.4 for details). For this reason, we need to leverage the above consistency modeling approach to obtain both texture- and structure-consistent images for each character. Discussion This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. available on arxiv [1] https://civitai.com

AutoStory: Generating Diverse Storytelling Images with Minimal Human Effort: Our Method

Authors:
(1) Wen Wang, Zhejiang University, Hangzhou, China and Equal Contribution (wwenxyz@zju.edu.cn);
(2) Canyu Zhao, Zhejiang University, Hangzhou, China and Equal Contribution (volcverse@zju.edu.cn);
(3) Hao Chen, Zhejiang University, Hangzhou, China (haochen.cad@zju.edu.cn);
(4) Zhekai Chen, Zhejiang University, Hangzhou, China (chenzhekai@zju.edu.cn);
(5) Kecheng Zheng, Zhejiang University, Hangzhou, China (zkechengzk@gmail.com);
(6) Chunhua Shen, Zhejiang University, Hangzhou, China (chunhuashen@zju.edu.cn). Table of Links Abstract and Introduction
Related Work
Our Method
Experiments
Conclusion and References
Appendix 2 RELATED WORK 2.1 Story Visualization Story visualization aims to generate a series of visually consistent images from a story described in text. Limited by the generative capacity of the model, many story visualization approaches [Chen et al. 2022; Li 2022; Li et al. 2019; Maharana and Bansal 2021; Maharana et al. 2021, 2022; Pan et al. 2022; Rahman et al. 2022; Song et al. 2020] seek to largely simplify the task such that it becomes tractable, by considering specific characters, scenes, and image styles in a particular dataset. Early story visualization methods are mostly built upon GANs [Goodfellow et al. 2020]. For example, StoryGAN [Li et al. 2019] pioneers the story visualization task by proposing a GAN-based framework that considers both the full story and the current sentence for coherent image generation. CP-CSV [Song et al. 2020], DuCo-StoryGAN [Maharana et al. 2021], and VLC-StoryGAN [Li et al. 2019] follow the GAN-based framework, while improving the consistency of storytelling via better character-preserving or text understanding. Difference from these works, VP-CSV [Chen et al. 2022] leverages VQ-VAE a transformer-based language model for story visualization. StoryDALL-E [Maharana et al. 2022] leverages the pre-trained DALL-E [Ramesh et al. 2021] for better story visualization and proposes a novel task named story continuation that supports story visualization with a given initial image. AR-LDM [Pan et al. 2022] proposes a diffusion model-based method that generates story images in an autoregressive manner. While progress has been made, these methods rely on storyspecific training on datasets like PororoSV [Li et al. 2019] and FlintstonesSV [Maharana and Bansal 2021], making it difficult to generalize these methods to varying characters and scenes. The development of large-scale pre-trained text-to-image generative models [Ramesh et al. 2022, 2021; Rombach et al. 2022; Saharia et al. 2022] opens up new opportunities for generalizable story visualization. Several attempts have been made to generate storytelling images with diverse characters [Gong et al. 2023; Jeong et al. 2023; Liu et al. 2023c]. Jeong et al. [Jeong et al. 2023] utilized textual inversion [Gal et al. 2022] to swap human identities in story images, thus generalizing the characters in story visualization. However, the identity is not well preserved, and the method is limited to a single human character in storytelling. Intelligent Grimm [Liu et al. 2023c] proposes the task of open-ended visual storytelling. They collect a dataset of children’s storybooks and train an autoregressive generative model for story visualization. The limitation is clear: they focus on the storytelling of the children’s storybook style, and it needs to re-train the model to generalize to other styles, contents, etc., which is not scalable. Probably the most similar work to ours is TaleCraft [Gong et al. 2023], which also proposes a systematic pipeline for story visualization. Note that, they require user-provided sketches for each character in each story image to obtain visually pleasing generations, which can be laborious to obtain. Moreover, all existing methods rely on multiple user-provided images for each character to obtain identity-coherent story visualizations. In contrast, our method allows for generating diverse and coherent story visualization results with only text descriptions as inputs. 2.2 Controllable Image Generation The scaling of text-image paired data [Schuhmann et al. 2022], computational resources, and model size have enabled unprecedented text-to-image (T2I) generation results [Ramesh et al. 2022, 2021; Rombach et al. 2022; Saharia et al. 2022]. Large-scale pre-trained text-to-image models, such as Stable Diffusion [Rombach et al. 2022], are capable of generating images from text, i.e., 𝐼 = DM (𝑝), where DM(·) is the pre-trained diffusion model and 𝑝 is the text prompt that describe the image 𝐼. In this process, the text information is passed into the image’s latent representation through cross-attention layers in the model. The attention [Vaswani et al. 2017] operation can be written as However, limited by the language understanding capability of the text encoder and poor text-to-image content association [Chefer et al. 2023], T2I models, like Stable Diffusion [Rombach et al. 2022], can perform poorly in the generation of multiple characters and complex scenes [Chefer et al. 2023]. To alleviate this drawback, some approaches introduce explicit spatial guidance in T2I generative models. For example, ControlNet [Zhang and Agrawala 2023] uses zero convolution layers and a trainable copy of the original model weights, introducing reliable control in diffusion models. T2I-Adapter [Mou et al. 2023] achieves control ability by proposing the adapter that extracts guidance feature and adds it to the feature from the corresponding UNet encoder. GLIGEN [Li et al. 2023] injects a gated self-attention block into the UNet, enabling the model to make good use of the grounding inputs. Inspired by the ability of large language models (LLMs) [et al 2023; OpenAI 2023] being able to understand and plan, recent works [Feng et al. 2023; Lian et al. 2023] employ LLM for layout generation. Specifically, LayoutGPT [Feng et al. 2023] achieves plausible results in 2D image layouts and even 3D indoor scene synthesis by applying in-context learning on LLMs. LLM-grounded Diffusion [Lian et al. 2023] proposes a two-stage process based on the LLM-generated layout and local prompts. Specifically, it first generates the local objects within each bounding box based on the corresponding local prompt, and then re-generates the final result based on the inversed latent of local objects. While effective, LLMgrounded Diffusion requires careful hyper-parameter tuning for the trade-off between structural guidance and inter-object relationship modeling. Moreover, it is difficult for the users to control the detailed structure of the generated objects. In contrast, we use the intuitive sketch or keypoint to guide the final image generation. Thus, we can not only achieve high-quality story image generation, but also allow interactive story visualization by simply tuning the generated sketch or keypoint conditions. 2.3 Customized Image Generation Story visualization requires that the identities of characters and scenes in a story remain consistent across different images. Customized image generation can meet this requirement to a large extent. Early methods [Gal et al. 2022; Ruiz et al. 2022] focus on the customized generation of a single object. For example, DreamBooth [Ruiz et al. 2022] fine tunes the pre-trained T2I diffusion model under a class-specific prior-preservation loss. Textual Inversion [Gal et al. 2022] enables customized generation by inverting subject image content into text embeddings. Unlike these approaches, Custom Diffusion [Kumari et al. 2022] further achieves multi-subject customization by combining the multiple customization weights through closed-form constrained optimization. Cones [Liu et al. 2023a] finds that a small cluster of concept neurons in the diffusion model corresponds to a single subject, and thus achieves customized generation of multiple objects by combining these concept neurons. Cones2 [Liu et al. 2023f] further achieves more effective multi-object customization by combining text embedding of different concepts with simple layout control. Differently, Mix-of-Show [Gu et al. 2023] proposes gradient fusion to effectively combine multiple customized LoRA [Hu et al. 2022] weights and performs multi-object customization with the aid of the T2I-Adapter’s dense controls. While significant progress has been made, existing methods perform poorly on one-shot customization. The training data for subject-driven generation has to be identity-consistent and diverse. As a result, existing story visualization methods require multiple user-provided images for each character. To tackle this issue, we propose a training-free consistency modeling method, and leverage the 3D prior in 3D-aware generative models [Liu et al. 2023d,b] to obtain multi-view consistent character images for customized generation, thus eliminating the reliance on human labor to collect or draw character images. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Authors: (1) Wen Wang, Zhejiang University, Hangzhou, China and Equal Contribution (wwenxyz@zju.edu.cn); (2) Canyu Zhao, Zhejiang University, Hangzhou, China and Equal Contribution (volcverse@zju.edu.cn); (3) Hao Chen, Zhejiang University, Hangzhou, China (haochen.cad@zju.edu.cn); (4) Zhekai Chen, Zhejiang University, Hangzhou, China (chenzhekai@zju.edu.cn); (5) Kecheng Zheng, Zhejiang University, Hangzhou, China (zkechengzk@gmail.com); (6) Chunhua Shen, Zhejiang University, Hangzhou, China (chunhuashen@zju.edu.cn). Authors: Authors: (1) Wen Wang, Zhejiang University, Hangzhou, China and Equal Contribution (wwenxyz@zju.edu.cn); (2) Canyu Zhao, Zhejiang University, Hangzhou, China and Equal Contribution (volcverse@zju.edu.cn); (3) Hao Chen, Zhejiang University, Hangzhou, China (haochen.cad@zju.edu.cn); (4) Zhekai Chen, Zhejiang University, Hangzhou, China (chenzhekai@zju.edu.cn); (5) Kecheng Zheng, Zhejiang University, Hangzhou, China (zkechengzk@gmail.com); (6) Chunhua Shen, Zhejiang University, Hangzhou, China (chunhuashen@zju.edu.cn). Table of Links Abstract and Introduction Related Work Our Method Experiments Conclusion and References Appendix Abstract and Introduction Abstract and Introduction Related Work Related Work Our Method Our Method Experiments Experiments Conclusion and References Conclusion and References Appendix Appendix 2 RELATED WORK 2.1 Story Visualization Story visualization aims to generate a series of visually consistent images from a story described in text. Limited by the generative capacity of the model, many story visualization approaches [Chen et al. 2022; Li 2022; Li et al. 2019; Maharana and Bansal 2021; Maharana et al. 2021, 2022; Pan et al. 2022; Rahman et al. 2022; Song et al. 2020] seek to largely simplify the task such that it becomes tractable, by considering specific characters, scenes, and image styles in a particular dataset. Early story visualization methods are mostly built upon GANs [Goodfellow et al. 2020]. For example, StoryGAN [Li et al. 2019] pioneers the story visualization task by proposing a GAN-based framework that considers both the full story and the current sentence for coherent image generation. CP-CSV [Song et al. 2020], DuCo-StoryGAN [Maharana et al. 2021], and VLC-StoryGAN [Li et al. 2019] follow the GAN-based framework, while improving the consistency of storytelling via better character-preserving or text understanding. Difference from these works, VP-CSV [Chen et al. 2022] leverages VQ-VAE a transformer-based language model for story visualization. StoryDALL-E [Maharana et al. 2022] leverages the pre-trained DALL-E [Ramesh et al. 2021] for better story visualization and proposes a novel task named story continuation that supports story visualization with a given initial image. AR-LDM [Pan et al. 2022] proposes a diffusion model-based method that generates story images in an autoregressive manner. While progress has been made, these methods rely on storyspecific training on datasets like PororoSV [Li et al. 2019] and FlintstonesSV [Maharana and Bansal 2021], making it difficult to generalize these methods to varying characters and scenes. The development of large-scale pre-trained text-to-image generative models [Ramesh et al. 2022, 2021; Rombach et al. 2022; Saharia et al. 2022] opens up new opportunities for generalizable story visualization. Several attempts have been made to generate storytelling images with diverse characters [Gong et al. 2023; Jeong et al. 2023; Liu et al. 2023c]. Jeong et al. [Jeong et al. 2023] utilized textual inversion [Gal et al. 2022] to swap human identities in story images, thus generalizing the characters in story visualization. However, the identity is not well preserved, and the method is limited to a single human character in storytelling. Intelligent Grimm [Liu et al. 2023c] proposes the task of open-ended visual storytelling. They collect a dataset of children’s storybooks and train an autoregressive generative model for story visualization. The limitation is clear: they focus on the storytelling of the children’s storybook style, and it needs to re-train the model to generalize to other styles, contents, etc., which is not scalable. Probably the most similar work to ours is TaleCraft [Gong et al. 2023], which also proposes a systematic pipeline for story visualization. Note that, they require user-provided sketches for each character in each story image to obtain visually pleasing generations, which can be laborious to obtain. Moreover, all existing methods rely on multiple user-provided images for each character to obtain identity-coherent story visualizations. In contrast, our method allows for generating diverse and coherent story visualization results with only text descriptions as inputs. 2.2 Controllable Image Generation The scaling of text-image paired data [Schuhmann et al. 2022], computational resources, and model size have enabled unprecedented text-to-image (T2I) generation results [Ramesh et al. 2022, 2021; Rombach et al. 2022; Saharia et al. 2022]. Large-scale pre-trained text-to-image models, such as Stable Diffusion [Rombach et al. 2022], are capable of generating images from text, i.e., 𝐼 = DM (𝑝), where DM(·) is the pre-trained diffusion model and 𝑝 is the text prompt that describe the image 𝐼. In this process, the text information is passed into the image’s latent representation through cross-attention layers in the model. The attention [Vaswani et al. 2017] operation can be written as However, limited by the language understanding capability of the text encoder and poor text-to-image content association [Chefer et al. 2023], T2I models, like Stable Diffusion [Rombach et al. 2022], can perform poorly in the generation of multiple characters and complex scenes [Chefer et al. 2023]. To alleviate this drawback, some approaches introduce explicit spatial guidance in T2I generative models. For example, ControlNet [Zhang and Agrawala 2023] uses zero convolution layers and a trainable copy of the original model weights, introducing reliable control in diffusion models. T2I-Adapter [Mou et al. 2023] achieves control ability by proposing the adapter that extracts guidance feature and adds it to the feature from the corresponding UNet encoder. GLIGEN [Li et al. 2023] injects a gated self-attention block into the UNet, enabling the model to make good use of the grounding inputs. Inspired by the ability of large language models (LLMs) [et al 2023; OpenAI 2023] being able to understand and plan, recent works [Feng et al. 2023; Lian et al. 2023] employ LLM for layout generation. Specifically, LayoutGPT [Feng et al. 2023] achieves plausible results in 2D image layouts and even 3D indoor scene synthesis by applying in-context learning on LLMs. LLM-grounded Diffusion [Lian et al. 2023] proposes a two-stage process based on the LLM-generated layout and local prompts. Specifically, it first generates the local objects within each bounding box based on the corresponding local prompt, and then re-generates the final result based on the inversed latent of local objects. While effective, LLMgrounded Diffusion requires careful hyper-parameter tuning for the trade-off between structural guidance and inter-object relationship modeling. Moreover, it is difficult for the users to control the detailed structure of the generated objects. In contrast, we use the intuitive sketch or keypoint to guide the final image generation. Thus, we can not only achieve high-quality story image generation, but also allow interactive story visualization by simply tuning the generated sketch or keypoint conditions. 2.3 Customized Image Generation Story visualization requires that the identities of characters and scenes in a story remain consistent across different images. Customized image generation can meet this requirement to a large extent. Early methods [Gal et al. 2022; Ruiz et al. 2022] focus on the customized generation of a single object. For example, DreamBooth [Ruiz et al. 2022] fine tunes the pre-trained T2I diffusion model under a class-specific prior-preservation loss. Textual Inversion [Gal et al. 2022] enables customized generation by inverting subject image content into text embeddings. Unlike these approaches, Custom Diffusion [Kumari et al. 2022] further achieves multi-subject customization by combining the multiple customization weights through closed-form constrained optimization. Cones [Liu et al. 2023a] finds that a small cluster of concept neurons in the diffusion model corresponds to a single subject, and thus achieves customized generation of multiple objects by combining these concept neurons. Cones2 [Liu et al. 2023f] further achieves more effective multi-object customization by combining text embedding of different concepts with simple layout control. Differently, Mix-of-Show [Gu et al. 2023] proposes gradient fusion to effectively combine multiple customized LoRA [Hu et al. 2022] weights and performs multi-object customization with the aid of the T2I-Adapter’s dense controls. While significant progress has been made, existing methods perform poorly on one-shot customization. The training data for subject-driven generation has to be identity-consistent and diverse. As a result, existing story visualization methods require multiple user-provided images for each character. To tackle this issue, we propose a training-free consistency modeling method, and leverage the 3D prior in 3D-aware generative models [Liu et al. 2023d,b] to obtain multi-view consistent character images for customized generation, thus eliminating the reliance on human labor to collect or draw character images. one-shot This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. available on arxiv

AutoStory: Generating Diverse Storytelling Images with Minimal Human Effort: Related Work

AutoStory: Generating Diverse Storytelling Images with Minimal Effort: Conclusion and References

SEC v. Consensys Software Inc. Court Filing, retrieved on June 28, 2024, is part of HackerNoon’s Legal PDF Series. You can jump to any part in this filing here. This part is 16 of 26. D. SAND 196. “SAND” was created on the Ethereum blockchain as the native token of the Sandbox platform, a virtual gaming platform first released in 2012 by Pixowl, Inc. (“Pixowl”) as a game for download on mobile phones. Pixowl, which is headquartered in San Francisco, was founded in 2011 by Arthur Madrid (“Madrid”) and Sebastien Borget (“Borget”). In 2018, Animoca Brands, Inc. (“Animoca”), headquartered in Hong Kong, acquired Pixowl and announced its intention to build a new 3D version of the Sandbox by leveraging blockchain technology. After Pixowl’s acquisition, the Sandbox’s intellectual property, along with the rest of Pixowl’s assets, were transferred to TSB Gaming Ltd (“TSB”), a wholly owned subsidiary of Animoca. Madrid is CEO of TSB, and Borget is the COO. 197. According to Sandbox’s website, SAND is required to access the Sandbox platform, participate in the platform’s governance, and earn rewards through the staking program on the platform. 198. On or about May 23, 2019, before the minting of SAND in July 2019, Animoca raised approximately $2.5 million in cash and crypto assets through TSB via the issuance of Simple Agreements for Future Equity (“SAFEs”) and SAND tokens, to “fund the development of the upcoming blockchain version of The Sandbox.” According to Animoca’s May 23, 2019 press release, the majority of investors allocated their investment to the purchase of both SAND tokens and future equity in TSB via the SAFE agreements (in the amount of $2 million), while some investors allocated their investment exclusively to the purchase of SAND tokens ($500,000). Per the release, the funding round was led by Hashed, for approximately $1 million, and also included a number of other crypto venture capital investors. 199. TSB then minted a total supply of 3 billion SAND on the Ethereum blockchain in or around July 2019 and offered and sold SAND through purportedly private sales and in an IEO that raised $3 million on the Binance.com crypto asset trading platform starting August 13, 2020. 200. SAND has been available for buying and selling through the MetaMask Swaps platform since approximately October 2020. 201. The information TSB publicly disseminated has led SAND holders, including those who have purchased SAND since May 2022, reasonably to view SAND as an investment in and to expect to profit from TSB’s efforts to grow the Sandbox protocol, which, in turn, would increase the demand for and the value of SAND. 202. On its blog posts announcing “exchange listings,” Sandbox touted its efforts to obtain “listings” and the SAND token’s liquidity in the secondary market. For example, in a September 21, 2021, Medium blog post, Sandbox stated that “$SAND is listed on over 60 global cryptocurrency exchanges, including a dozen of the top exchanges by market capitalization.” 203. In addition, the Sandbox stated that it would pool the proceeds from the private token sales and the IEO to develop and promote use of the platform. For example, the May 23, 2019, press release stated: “[t]he funds raised through this transaction will be used to grow the development team and infrastructure for the [Sandbox] Game Platform, support marketing efforts through the acquisition of creators and IP licenses, and provide for security, legal, and compliance expenses as well as general and administrative costs.” The Sandbox whitepaper similarly described identical uses for the $3 million in funds intended to be raised during the IEO. 204. Moreover, according to the Sandbox whitepaper, of the 3 billion SAND tokens that were initially minted, 19% were to be allocated to the Sandbox founders and team, and another 25.8% were to be allocated to the Company Reserve. 205. In addition, the Sandbox’s Medium blog post on July 25, 2019, stated that “an interesting feature of [the $SAND] token is that it can accrue in value over time, due to the fact that it is scarce. There will be a limited supply of 3 billion units of $SAND available.” 206. Moreover, TSB stated publicly that it would take steps to manage the market for SAND, including the SAND whitepaper stating that the Sandbox team controls the supply of SAND tokens and has implemented a “controllable supply mechanism, such as purchasing SAND from multiple exchanges,” and that “while the total supply of SAND is fixed, the initial amount of SAND offered will provide a scarcity effect reducing the SAND available per capita and therefore fostering demand.” 207. Additionally, in many instances, Animoca has touted the backgrounds of Pixowl, TSB, and the Sandbox core members, including Madrid and Borget, in describing the success and future development of the Sandbox: • After the acquisition of Pixowl, Yat Siu, the co-founder and director of Animoca, stated in a press release, dated August 27, 2018 (the “2018 Press Release”) that “Pixowl’s experienced developers will significantly increase our development capabilities. Its founders are highly respected game industry veterans who have developed multimillion dollar franchises. We believe the blockchain version of The Sandbox has incredible potential … We look forward to utilising the many opportunities for growth conferred by this acquisition.” • In the 2018 Press Release, Madrid also commented: ‘“Animoca Brands is a perfect fit for Pixowl and we are happy to add our brand relationships to its portfolio while accelerating growth for our key IP, The Sandbox …” • The 2018 Press Release also touted that “Ed Fries, the creator of Microsoft Game Studios and co-founder of the Xbox project, is a special advisor to The Sandbox’s original game developer Pixowl” and will therefore continue to serve on the advisory team. • The Sandbox whitepaper further provided: “We have a strong product roadmap ahead and a top team to execute a strong vision to build a unique virtual world gaming platform where players can build, own, and monetize their gaming experiences and spread the power of blockchain as the lead technology in the gaming industry.” 208. Moreover, the Sandbox whitepaper describes that the role of the “Sandbox Foundation” is to support the ecosystem of the Sandbox by, among other things, offering grants to incentivize high quality content and game production on the platform and further notes that the “overall valuation of the metaverse grows through the valuation of all games funded by the Foundation, creating a virtuous circle to enable funding bigger games.” The Sandbox’s Gitbook also notes that the Sandbox Foundation has, among other things, (a) supported play-to-earn tournaments and cross-gaming to encourage the broader adoption of SAND and (b) supported marketing activities contributing to the growth of awareness about NFTs, Metaverse and SAND adoption, including co-marketing with exchanges and influencers. Continue Reading Here. About HackerNoon Legal PDF Series: We bring you the most important technical and insightful public domain court case filings.
This court case retrieved on June 28, 2024, storage.courtlistener.com is part of the public domain. The court-created documents are works of the federal government, and under copyright law, are automatically placed in the public domain and may be shared without legal restriction. SEC v. Consensys Software Inc. Court Filing, retrieved on June 28, 2024, is part of HackerNoon’s Legal PDF Series. You can jump to any part in this filing here. This part is 16 of 26. SEC v. Consensys Software Inc. Court Filing, retrieved on June 28, 2024, is part of HackerNoon’s Legal PDF Series . You can jump to any part in this filing here . This part is 16 of 26. HackerNoon’s Legal PDF Series here D. SAND 196. “SAND” was created on the Ethereum blockchain as the native token of the Sandbox platform, a virtual gaming platform first released in 2012 by Pixowl, Inc. (“Pixowl”) as a game for download on mobile phones. Pixowl, which is headquartered in San Francisco, was founded in 2011 by Arthur Madrid (“Madrid”) and Sebastien Borget (“Borget”). In 2018, Animoca Brands, Inc. (“Animoca”), headquartered in Hong Kong, acquired Pixowl and announced its intention to build a new 3D version of the Sandbox by leveraging blockchain technology. After Pixowl’s acquisition, the Sandbox’s intellectual property, along with the rest of Pixowl’s assets, were transferred to TSB Gaming Ltd (“TSB”), a wholly owned subsidiary of Animoca. Madrid is CEO of TSB, and Borget is the COO. 197. According to Sandbox’s website, SAND is required to access the Sandbox platform, participate in the platform’s governance, and earn rewards through the staking program on the platform. 198. On or about May 23, 2019, before the minting of SAND in July 2019, Animoca raised approximately $2.5 million in cash and crypto assets through TSB via the issuance of Simple Agreements for Future Equity (“SAFEs”) and SAND tokens, to “fund the development of the upcoming blockchain version of The Sandbox.” According to Animoca’s May 23, 2019 press release, the majority of investors allocated their investment to the purchase of both SAND tokens and future equity in TSB via the SAFE agreements (in the amount of $2 million), while some investors allocated their investment exclusively to the purchase of SAND tokens ($500,000). Per the release, the funding round was led by Hashed, for approximately $1 million, and also included a number of other crypto venture capital investors. 199. TSB then minted a total supply of 3 billion SAND on the Ethereum blockchain in or around July 2019 and offered and sold SAND through purportedly private sales and in an IEO that raised $3 million on the Binance.com crypto asset trading platform starting August 13, 2020. 200. SAND has been available for buying and selling through the MetaMask Swaps platform since approximately October 2020. 201. The information TSB publicly disseminated has led SAND holders, including those who have purchased SAND since May 2022, reasonably to view SAND as an investment in and to expect to profit from TSB’s efforts to grow the Sandbox protocol, which, in turn, would increase the demand for and the value of SAND. 202. On its blog posts announcing “exchange listings,” Sandbox touted its efforts to obtain “listings” and the SAND token’s liquidity in the secondary market. For example, in a September 21, 2021, Medium blog post, Sandbox stated that “$SAND is listed on over 60 global cryptocurrency exchanges, including a dozen of the top exchanges by market capitalization.” 203. In addition, the Sandbox stated that it would pool the proceeds from the private token sales and the IEO to develop and promote use of the platform. For example, the May 23, 2019, press release stated: “[t]he funds raised through this transaction will be used to grow the development team and infrastructure for the [Sandbox] Game Platform, support marketing efforts through the acquisition of creators and IP licenses, and provide for security, legal, and compliance expenses as well as general and administrative costs.” The Sandbox whitepaper similarly described identical uses for the $3 million in funds intended to be raised during the IEO. 204. Moreover, according to the Sandbox whitepaper, of the 3 billion SAND tokens that were initially minted, 19% were to be allocated to the Sandbox founders and team, and another 25.8% were to be allocated to the Company Reserve. 205. In addition, the Sandbox’s Medium blog post on July 25, 2019, stated that “an interesting feature of [the $SAND] token is that it can accrue in value over time, due to the fact that it is scarce. There will be a limited supply of 3 billion units of $SAND available.” 206. Moreover, TSB stated publicly that it would take steps to manage the market for SAND, including the SAND whitepaper stating that the Sandbox team controls the supply of SAND tokens and has implemented a “controllable supply mechanism, such as purchasing SAND from multiple exchanges,” and that “while the total supply of SAND is fixed, the initial amount of SAND offered will provide a scarcity effect reducing the SAND available per capita and therefore fostering demand.” 207. Additionally, in many instances, Animoca has touted the backgrounds of Pixowl, TSB, and the Sandbox core members, including Madrid and Borget, in describing the success and future development of the Sandbox: • After the acquisition of Pixowl, Yat Siu, the co-founder and director of Animoca, stated in a press release, dated August 27, 2018 (the “2018 Press Release”) that “Pixowl’s experienced developers will significantly increase our development capabilities. Its founders are highly respected game industry veterans who have developed multimillion dollar franchises. We believe the blockchain version of The Sandbox has incredible potential … We look forward to utilising the many opportunities for growth conferred by this acquisition.” • In the 2018 Press Release, Madrid also commented: ‘“Animoca Brands is a perfect fit for Pixowl and we are happy to add our brand relationships to its portfolio while accelerating growth for our key IP, The Sandbox …” • The 2018 Press Release also touted that “Ed Fries, the creator of Microsoft Game Studios and co-founder of the Xbox project, is a special advisor to The Sandbox’s original game developer Pixowl” and will therefore continue to serve on the advisory team. • The Sandbox whitepaper further provided: “We have a strong product roadmap ahead and a top team to execute a strong vision to build a unique virtual world gaming platform where players can build, own, and monetize their gaming experiences and spread the power of blockchain as the lead technology in the gaming industry.” 208. Moreover, the Sandbox whitepaper describes that the role of the “Sandbox Foundation” is to support the ecosystem of the Sandbox by, among other things, offering grants to incentivize high quality content and game production on the platform and further notes that the “overall valuation of the metaverse grows through the valuation of all games funded by the Foundation, creating a virtuous circle to enable funding bigger games.” The Sandbox’s Gitbook also notes that the Sandbox Foundation has, among other things, (a) supported play-to-earn tournaments and cross-gaming to encourage the broader adoption of SAND and (b) supported marketing activities contributing to the growth of awareness about NFTs, Metaverse and SAND adoption, including co-marketing with exchanges and influencers. Continue Reading Here. Continue Reading Here . Here About HackerNoon Legal PDF Series: We bring you the most important technical and insightful public domain court case filings. This court case retrieved on June 28, 2024, storage.courtlistener.com is part of the public domain. The court-created documents are works of the federal government, and under copyright law, are automatically placed in the public domain and may be shared without legal restriction. About HackerNoon Legal PDF Series: We bring you the most important technical and insightful public domain court case filings. This court case retrieved on June 28, 2024, storage.courtlistener.com is part of the public domain. The court-created documents are works of the federal government, and under copyright law, are automatically placed in the public domain and may be shared without legal restriction. storage.courtlistener.com

SEC Challenges SAND Token's Security Status

At the Forefront of AI Research: Multimodality, Agents, Open-Source LLM, and Beyond

এআই গবেষণার অগ্রভাগে: মাল্টিমোডালিটি, এজেন্ট, ওপেন-সোর্স এলএলএম এবং তার বাইরে

An der Spitze der KI-Forschung: Multimodalität, Agenten, Open-Source-LLM und mehr

A la vanguardia de la investigación en IA: multimodalidad, agentes, LLM de código abierto y más

À la pointe de la recherche en IA : multimodalité, agents, LLM Open Source et au-delà

एआई अनुसंधान के अग्रभाग पर: मल्टीमोडैलिटी, एजेंट, ओपन-सोर्स एलएलएम, और उससे आगे

AI研究の最前線：マルチモダリティ、エージェント、オープンソースLLM、そしてその先

AI 연구의 최전선: 다중 양식, 에이전트, 오픈 소스 LLM 및 그 이상

Na vanguarda da pesquisa em IA: multimodalidade, agentes, LLM de código aberto e muito mais

На переднем крае исследований искусственного интеллекта: мультимодальность, агенты, LLM с открытым исходным кодом и не только

Yapay Zeka Araştırmasının Ön Planında: Çok Modluluk, Aracılar, Açık Kaynak Yüksek Lisans ve Ötesi

Đi đầu trong nghiên cứu AI: Đa phương thức, Đại lý, LLM nguồn mở và hơn thế nữa

人工智能研究的前沿：多模态、代理、开源法学硕士及其他

Exploring Drone Dynamics: Stability and Force Generation in Multimedia Design

Authors:
(1) Hamed Alimohammadzadeh, University of Southern California, Los Angeles, USA;
(2) Rohit Bernard, University of Southern California, Los Angeles, USA;
(3) Yang Chen, University of Southern California, Los Angeles, USA;
(4) Trung Phan, University of Southern California, Los Angeles, USA;
(5) Prashant Singh, University of Southern California, Los Angeles, USA;
(6) Shuqin Zhu, University of Southern California, Los Angeles, USA;
(7) Heather Culbertson, University of Southern California, Los Angeles, USA;
(8) Shahram Ghandeharizadeh, University of Southern California, Los Angeles, USA. Table of Links Abstract and Introduction Flying Light Specks User Interaction Multimedia Systems Challenges Related Work Conclusions and Current Efforts, Acknowledgments, and References 3 USER INTERACTION FLSs will facilitate both kinesthetic (force) and tactile (skin-based) feedback to a user. While the former provides the shape and hardness of a virtual object, the latter renders an object’s surface properties. Both motivate the disaggregated FLS architecture of Figure 4 with diverse plug-in devices. To illustrate, textures for different surfaces may be rendered using different vibration actuators [19]. An actuator may be a plug-in device to the FLS system bus, enabling an experimenter to evaluate the actuator and its tactile feedback. An FLS acts as an encountered-type haptic device. Traditional kinesthetic feedback haptic rendering uses a vector-field approach [41] where the force to be displayed to the user is calculated based on their position in the 3D environment and geometric models of the virtual objects. Our FLS-based system will take a similar approach and calculate the desired force based on the user’s position and the 3D model of the object(s) to be rendered. For example, in displaying a virtual wall, the user is free to move their hand around the environment until their position penetrates inside the virtual wall, at which time the FLS would make contact with their hand to apply a force and prevent them from penetrating further into the virtual wall (Fig. 5). Using measurements of the FLS’s position and a 3D model of an object, we can determine how far the user has penetrated into the virtual object to calculate the amount of force to display to the user. With grounded impedance force-feedback haptic devices, the calculated force is then converted to motor torque using the device’s calibrated Jacobian matrix so that the desired force can be displayed to the user. Since the controllable output of the drones is thrust, displaying a set force to the user requires careful calibration between the commanded thrust and the resulting force. Previous researchers have completed this calibration for single axes of thrust/force for the Parrot AR.Drone [1]. Displaying more complex objects and interactions in 3-dimensions requires a more complete calibration of the system. Force Amplification: The size of the FLS plays an important role in its dynamics and the force that can be displayed to the user. The FLS’s motors determine the highest RPM that can be generated, and the propellers’ ratio of pitch:diameter affects the force per RPM generated from the air pushing downwards. A larger FLS usually means larger diameter propellers, leading to higher forces at a given RPM. However, the force that we will be able to output with an FLS will be significantly less than that of a traditional kinesthetic haptic device; for example, the common 3D Systems Touch device has a maximum force output of 3.3 N. The modality of human sensing also varies based on the level of force provided, with small forces (<1 N) being sensed as tactile information through the skin rather than kinesthetic information through the muscles [34]. Therefore, a single FLS will not be capable of providing sufficient force for a salient haptic interaction, especially if we want to generate the kinesthetic sensations necessary for displaying a rigid object. Therefore, the FLSs must coordinate together to combine their thrust so that an amplified force can be applied to the user. This coordination must be done in such a way that the formation remains stable during flight and that force can be passed between drones before eventually being felt by the user. This force amplification can be done either by FLSs in a parallel or series configuration (Fig. 6). In parallel, multiple FLSs will contact the user at the same time and will each provide a force proportional to that FLS’s disturbance from its own set-point. In series, only a single FLS will be in contact with the user, and both FLSs will have the same disturbance from their set-point; the user feels their combined force. User Safety: The goal of our system is to provide non-constrained encountered-type haptic feedback directly to users. To do this, the FLS must (1) be modified to allow users to safely have direct contact and (2) react to the user’s gesture input (e.g. push the FLS away from its desired position), and (3) provide meaningful haptic feedback. With the first, we use a cage to separate the user’s hand from the rotors. Cage shape, weight, and spatial density are key considerations that effect safety, lighting, and flying capability of the FLS. The added weight of the cage would affect the drone’s dynamics and induce imbalance issues when not calibrated. Spatial density of the cage’s mesh is important to prevent the user’s finger from accidentally penetrating the surfaces while not affecting the aerodynamics and lighting of the drone too significantly. However, too fine of a spatial density may significantly affect both the lighting and the flying ability of the drone. A balance must be made between choosing a mesh fine enough to protect the user’s fingers, but coarse enough to not disrupt the airflow and provide the desired textured lighting. Sensing Users: To create an effective touch-based interaction, each FLS should be capable of sensing touch from the user. At a minimum, the FLS should be able to detect the presence or absence of touch, and preferably it will be able to detect the location of touch as well. Capacitive sensors could be integrated into the FLS cage to easily detect the presence of user contact, and a more complex capacitive skin could be developed to determine point of contact. However, determining force of contact from this type of sensor would be challenging, and if this information is required more a more complex mesh of sensors must be developed. Here we propose to use open-loop control for the forces, so direct force sensing from the user is not required. This paper is available on arxiv under CC 4.0 license. Authors: (1) Hamed Alimohammadzadeh, University of Southern California, Los Angeles, USA; (2) Rohit Bernard, University of Southern California, Los Angeles, USA; (3) Yang Chen, University of Southern California, Los Angeles, USA; (4) Trung Phan, University of Southern California, Los Angeles, USA; (5) Prashant Singh, University of Southern California, Los Angeles, USA; (6) Shuqin Zhu, University of Southern California, Los Angeles, USA; (7) Heather Culbertson, University of Southern California, Los Angeles, USA; (8) Shahram Ghandeharizadeh, University of Southern California, Los Angeles, USA. Authors: Authors: (1) Hamed Alimohammadzadeh, University of Southern California, Los Angeles, USA; (2) Rohit Bernard, University of Southern California, Los Angeles, USA; (3) Yang Chen, University of Southern California, Los Angeles, USA; (4) Trung Phan, University of Southern California, Los Angeles, USA; (5) Prashant Singh, University of Southern California, Los Angeles, USA; (6) Shuqin Zhu, University of Southern California, Los Angeles, USA; (7) Heather Culbertson, University of Southern California, Los Angeles, USA; (8) Shahram Ghandeharizadeh, University of Southern California, Los Angeles, USA. Table of Links Abstract and Introduction Abstract and Introduction Flying Light Specks Flying Light Specks User Interaction User Interaction Multimedia Systems Challenges Multimedia Systems Challenges Related Work Related Work Conclusions and Current Efforts, Acknowledgments, and References Conclusions and Current Efforts, Acknowledgments, and References 3 USER INTERACTION FLSs will facilitate both kinesthetic (force) and tactile (skin-based) feedback to a user. While the former provides the shape and hardness of a virtual object, the latter renders an object’s surface properties. Both motivate the disaggregated FLS architecture of Figure 4 with diverse plug-in devices. To illustrate, textures for different surfaces may be rendered using different vibration actuators [19]. An actuator may be a plug-in device to the FLS system bus, enabling an experimenter to evaluate the actuator and its tactile feedback. An FLS acts as an encountered-type haptic device. Traditional kinesthetic feedback haptic rendering uses a vector-field approach [41] where the force to be displayed to the user is calculated based on their position in the 3D environment and geometric models of the virtual objects. Our FLS-based system will take a similar approach and calculate the desired force based on the user’s position and the 3D model of the object(s) to be rendered. For example, in displaying a virtual wall, the user is free to move their hand around the environment until their position penetrates inside the virtual wall, at which time the FLS would make contact with their hand to apply a force and prevent them from penetrating further into the virtual wall (Fig. 5). Using measurements of the FLS’s position and a 3D model of an object, we can determine how far the user has penetrated into the virtual object to calculate the amount of force to display to the user. With grounded impedance force-feedback haptic devices, the calculated force is then converted to motor torque using the device’s calibrated Jacobian matrix so that the desired force can be displayed to the user. Since the controllable output of the drones is thrust, displaying a set force to the user requires careful calibration between the commanded thrust and the resulting force. Previous researchers have completed this calibration for single axes of thrust/force for the Parrot AR.Drone [1]. Displaying more complex objects and interactions in 3-dimensions requires a more complete calibration of the system. Force Amplification: The size of the FLS plays an important role in its dynamics and the force that can be displayed to the user. The FLS’s motors determine the highest RPM that can be generated, and the propellers’ ratio of pitch:diameter affects the force per RPM generated from the air pushing downwards. A larger FLS usually means larger diameter propellers, leading to higher forces at a given RPM. However, the force that we will be able to output with an FLS will be significantly less than that of a traditional kinesthetic haptic device; for example, the common 3D Systems Touch device has a maximum force output of 3.3 N. The modality of human sensing also varies based on the level of force provided, with small forces (<1 N) being sensed as tactile information through the skin rather than kinesthetic information through the muscles [34]. Force Amplification: Therefore, a single FLS will not be capable of providing sufficient force for a salient haptic interaction, especially if we want to generate the kinesthetic sensations necessary for displaying a rigid object. Therefore, the FLSs must coordinate together to combine their thrust so that an amplified force can be applied to the user. This coordination must be done in such a way that the formation remains stable during flight and that force can be passed between drones before eventually being felt by the user. This force amplification can be done either by FLSs in a parallel or series configuration (Fig. 6). In parallel, multiple FLSs will contact the user at the same time and will each provide a force proportional to that FLS’s disturbance from its own set-point. In series, only a single FLS will be in contact with the user, and both FLSs will have the same disturbance from their set-point; the user feels their combined force. User Safety: The goal of our system is to provide non-constrained encountered-type haptic feedback directly to users. To do this, the FLS must (1) be modified to allow users to safely have direct contact and (2) react to the user’s gesture input (e.g. push the FLS away from its desired position), and (3) provide meaningful haptic feedback. With the first, we use a cage to separate the user’s hand from the rotors. Cage shape, weight, and spatial density are key considerations that effect safety, lighting, and flying capability of the FLS. The added weight of the cage would affect the drone’s dynamics and induce imbalance issues when not calibrated. Spatial density of the cage’s mesh is important to prevent the user’s finger from accidentally penetrating the surfaces while not affecting the aerodynamics and lighting of the drone too significantly. However, too fine of a spatial density may significantly affect both the lighting and the flying ability of the drone. A balance must be made between choosing a mesh fine enough to protect the user’s fingers, but coarse enough to not disrupt the airflow and provide the desired textured lighting. User Safety: Sensing Users: To create an effective touch-based interaction, each FLS should be capable of sensing touch from the user. At a minimum, the FLS should be able to detect the presence or absence of touch, and preferably it will be able to detect the location of touch as well. Capacitive sensors could be integrated into the FLS cage to easily detect the presence of user contact, and a more complex capacitive skin could be developed to determine point of contact. However, determining force of contact from this type of sensor would be challenging, and if this information is required more a more complex mesh of sensors must be developed. Here we propose to use open-loop control for the forces, so direct force sensing from the user is not required. Sensing Users: This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

How Flying Light Specks Enable Haptic Interactions

Minimal Knowledge PT-AE Attacks on Black-Box Speaker Recognition Models

Kafka Schema Evolution: A Guide to the Confluent Schema Registry

Stories, News & Mentions about CONTROLHUB

Controlhub’s Stories on HackerNoon