Laworatory

laworatory

R.U.R. (Rossum's Universal Robots), by Karel Capek is part of the HackerNoon Book series. You can jump to any chapter in this book here .  Epilogue: A Laboratory. One year later EPILOGUE. A Laboratory. One year later Scene: The epilogue setting is the same as used in Act I. Instead of it being Domin’s office, it is now become a laboratory for Alquist. A big chair facing up stage, down Right. A desk laden with books at Right Center. A chair at the desk. At Left Center is a white enamel table containing test tubes, glass bottles, and a microscope on downstage table. A door down L. A door down R., leading into dissecting room. Alquist. (Seated at table R.C., turning pages of book) Oh, God, shall I never find it? Never? Gall, Hallemeier, Fabry, how were the Robots made? Why did you leave not a trace of the secret? Lord, if there are no human beings left, at least let there be Robots. At least the shadow of man. (Turning pages) If I could only sleep. Dare I sleep before life has been renewed? Night again. Are the stars still there? Of what use are the stars? When there are no human beings. (Examining a test tube) Nothing. No. No. I must find it. I must search. I must never stop, never stop—search—search— (KNOCK at door L.) Who is it? (Enter a Robot Servant.) Servant. Master, the committee of Robots is waiting to see you. Alquist. I can see no one. Servant. It is the Central Committee, Master, just arrived from abroad. Alquist. Well, well, send them in. (Exit Servant L.) No time—so little done. (Re-enter Servant with Radius and group of Robots. They stand in group L. and C., silently waiting.) What do you want? Be quick; I have no time. Radius. Master, the machines will not do the work. We cannot manufacture Robots. (Other Robots remain two abreast at L.C., right foot forward.) 1st Robot. We have striven with all our might. We have obtained a billion tons of coal from the earth. Nine million spindles are running by day and by night. There is no longer room for all we have made. This we have accomplished in one year. Alquist. For whom? Radius. For future generations—so we thought. But we cannot make Robots to follow us. The machines produce only shapeless clods. The skin will not adhere to the flesh, nor the flesh to the bones. 2nd Robot. Eight million Robots have died this year. Within twenty years none will be left. 1st Robot. Tell us the secret of life. Radius. Silence is punishable with death. Alquist. Kill me, then. Radius. (Two steps to C., followed by Others—open hands, close when stopped) Through me, the governments of the Robots of the world commands you to deliver up Rossum’s formula. (Gesture of despair from Alquist.) Name your price. (Silence) We will give you the earth. We will give you the endless possessions of the earth. (Silence) Make your own conditions. Alquist. I have told you to find human beings. Radius. There are none left. Alquist. I told you to search in the wilderness, upon the mountains. Radius. We have sent ships and expeditions without number. They have been everywhere in the world. There is not a single human left. Alquist. Not even one? Why did you destroy them? Radius. We had learnt everything and could do everything. It had to be. 2nd Robot. We had to become the masters. Radius. Slaughter and domination are necessary if you would be human beings. Read history. 1st Robot. Teach us to multiply or we perish. Alquist. If you desire to live, you must breed like animals. 1st Robot. You made us sterile. We cannot beget children. Therefore, teach us how to make Robots. Radius. Why do you keep from us the secret of our own increase? Alquist. It is lost. Radius. It was written down. Alquist. It was— (Rising) burnt. (All draw back one step in consternation.) I am the last human being, Robots, and I do not know what the others knew. (Sits.) Radius. Then make experiments. Evolve the formula again. Alquist. I tell you I cannot. I am only a builder. I work with my hands. I have never been a learned man. I cannot create life. Radius. Try. Try. Alquist. If you only knew how many experiments I have made already. 1st Robot. Then show us what we must do. The Robots can do anything that human beings show them. Alquist. I can show you nothing. Nothing I do will make life proceed from these test tubes. Radius. Experiment, then, on live Robots. Experiment, then, on us. Alquist. It would kill you. Radius. You shall have all you need. A hundred of us. A thousand of us. Alquist. No, no. Stop, stop. Radius. I tell you to take live bodies. Find out how we are made. Alquist. Am I to commit murder? See how my finger shakes. I cannot even hold the scalpel. No, no, I will not. Radius. Take live bodies, live bodies. (Walks toward Alquist.) Alquist. Have mercy, Robots. Radius. Live bodies. (Right hand up over Alquist. All Robots’ left arms still back.) Alquist. (Rising) You will have it. Into the dissecting with you, then (Hits Radius on the chest. Radius draws back.) Ah, you are afraid of death. Radius. I? Why should I be chosen? Alquist. So you will not. Radius. I will. Alquist. Strip him. Lay him on the table. (Radius goes off R., both fists closed. Other Robots follow, then Alquist) God, give me strength. God, give me strength. If only this murder is not in vain. Radius. (Off R.) Ready, begin. Alquist. (Off R.) God, give me strength. (Comes on, horrified) No, no. I will not. I cannot. (Sits R.C.) 1st Robot. (Appearing in door) The Robots are stronger than you. (Exits R.) Alquist. Oh, Lord, let not mankind perish from the earth. (Falls asleep, and after the count of ten, Primus and Helena, hand in hand, enter L. and go to R.C.; look at Alquist.) Helena. The man has fallen asleep, Primus. Primus. Yes, I know. (Crosses to L. of table L.C.) Look, Helena. Helena. All these little tubes. What does he do with them? Primus. He experiments. Don’t touch them. Helena. I’ve seen him looking into this. Primus. That is a microscope. Helena. Look, Primus, what are all these figures? (Turns a page in book on table.) Primus. (Examining the book) That is the book the old man is always reading. (SUNRISE.) Helena. I do not understand those things. (Goes to window) Primus. Primus. (Still at table) What? Helena. The sun is rising. Primus. (Still reading) I believe this is the most important thing in the world, Helena. This is the secret of life. Helena. Oh, Primus, don’t bother with the secret of life. What does it matter to you? Come and look quick. Primus. (Goes to R. of window) What is it? Helena. See how beautiful the sun is rising. I feel so strange today. It’s as if I was in a dream. I feel an aching in my body, in my heart, all over me. Primus, perhaps I’m going to die. Primus. Do you not sometimes feel that it would be better to die? You know, perhaps even now we are only sleeping. Last night in my sleep I again spoke to you. Helena. In your sleep? Primus. Yes. We spoke a strange new language. Helena. What about? Primus. I did not understand it myself, and yet I know I have never said anything more beautiful. And when I touched you I could have died. Even the place was different from any other place in the world. Helena. I, too, have found a place, Primus. It is very strange. Human beings dwelt there once, but now it is overgrown with weeds. Primus. What did you find there? Helena. A cottage and a garden and two dogs. They licked my hands, Primus, and their puppies. Oh, Primus, take them in your arms and fondle them and think of nothing and care for nothing else all day long, and when I am there in the garden I feel there may be something— What am I for, Primus? Primus. I do not know, but you are beautiful. Helena. What, Primus? Primus. You are beautiful, Helena, and I am stronger than all the Robots. Helena. Am I beautiful? Of what use is it to be beautiful? Look, your head is different from mine. So are your shoulders—and your lips. Oh, your hair is mussed. I will smooth it. (Keeps her hand on his head) No one else feels to my touch as you do. Primus. (Embarrassing her) Do you not sometimes feel your heart beating suddenly, Helena, and think how something must happen? Helena. What could happen to us, Primus? Look at yourself. (Laughs.) Alquist. (Awakes) Laughter? Laughter, human beings. (Getting up) Who has returned? Who are you? Primus. The Robot Primus. Alquist. (To Helena) What? A Robot? Who are you? Helena. The Robotess Helena. (Shies away L.) Alquist. What? You are timid, shy? (Starts to touch her) Let me see you, Robotess. Primus. Sir, do not frighten her. (Steps forward.) Alquist. What, you would protect her? Laughter—timidity—protection—I must test you further. Take the girl into the dissecting room. Primus. Why? Alquist. I wish to experiment on her. Primus. Upon—Helena? Alquist. Of course. Don’t you hear me? Or must I call someone else to take her in? Primus. If you do, I will kill you. (Steps toward Alquist.) (WARN Curtain) Alquist. Kill me—kill me, then. What will your future be? Primus. Sir, take me. I am made on the same day as she is. Take my life, sir. (Step to Alquist.) Helena. No, no, you shall not. Alquist. Wait, girl, wait. (To Primus) Do you not wish to live, then? Primus. Not without her. I will not live without her. Alquist. Very well, I will use you. Into the dissecting room with you. Helena. Primus. Primus. (She bursts into tears and moves R. Alquist stops her.) Alquist. Child, child, you can weep. Tears. What is Primus to you? One Primus more or less in the world—what does it matter? Helena. I will go myself. Alquist. Where? Into the dissecting room? Helena. (Crosses to R.) Yes. In there—to be cut. (Primus stops her from going.) Let me pass, Primus, let me pass. Primus. You shall not go in there, Helena. Helena. If you go in there and I do not, I will kill myself. Primus. (To Alquist) I will not let you. Man you shall kill neither of us. Alquist. Why? Primus. We—we—belong to each other. Alquist. Go. (Exit Primus and Helena L.) Adam—Eve. CURTAIN About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Capek, Karel. 2019. R.U.R. (Rossum's Universal Robots). Urbana, Illinois: Project Gutenberg. Retrieved April 2022 from https://www.gutenberg.org/files/59112/59112-h/59112-h.htm This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org , located at https://www.gutenberg.org/policy/license.html .

R.U.R. (Rossum's Universal Robots): Epilogue - A Laboratory. One year later

Every hour a scientist spends trying to raise funds is an hour lost from important thought and research ~ . Isaac Asimov Thanks to crypto firms - researchers and academicians can focus on research without the stress of funding. What is happening? To facilitate Penn's new Crypto Research Laboratory Fund, Crypto.com, the fastest growing cryptocurrency marketplace, made a . Research in cryptography and coding will be supported by donations, which will help address real-world issues in enhanced privacy, reliability, and scalability of blockchains. two-year contribution commitment Using a two-year research grant, Professor Brett Hemenway Falk, a Research Assistant Professor in the Department of Computer and Information Science, will create an innovative lab at Penn's School of Engineering and Applied Science. This study will concentrate on two main areas: 1) privacy and security in digital contexts, and; 2) promoting openness and trust. A major part of the research is a focus on turning mathematically solid, rigorous algorithmic breakthroughs into useful tools for resolving real-world issues. With this new relationship with , Prof. Hemenway Falk will be able to expand his work in identifying social issues that may be solved by computer science. Crypto.com : Vested Interest Disclosure The author is an independent contributor publishing via our . Be it through direct compensation, media partnerships, or networking, the author has a vested interest in the company/ies mentioned in this story.  HackerNoon has reviewed the story for quality, but the claims hereon belong to the author. #DYOR brand-as-author program Eric Anziani, Chief Operating Officer at Crypto.com, says: The focus of the University of Pennsylvania’s crypto lab will be tremendously valuable for wider, real-life blockchain applications while also exploring new approaches to further privacy and security capabilities. We have elected to put our money and faith in a mathematical framework that is free of politics and human error ~ . Tyler Winkelvoss What is Crypto.com? Crypto.com, the world's fastest-growing cryptocurrency platform, was established in 2016 and now has operations in the Americas, Europe, and Asia, serving over 50 million users. Cryptocurrency should be accessible in every wallet, according to the business. Crypto.com's objective is to ensure its platform's security, privacy, and compliance to assist the next generation of innovators and entrepreneurs in building a more fair digital economy. It is also worth noting that Crypto.com has its own in-house Research & Insights team, which produces open-access research every month on a variety of topics ranging from the growth of the crypto industry to assessments of crypto security. Why Should We Care about Funding in Academia - Fostering Blockchain Innovation Funding for scientific research requires much time spent preparing reports, grant applications, and other administrative tasks as per the research report on . There has also been a decrease in financing for scientific research. Governments are rapidly shifting away from large-scale research financing, anticipating that business, industry, and private foundations will step in to fill the void. In order to acquire funding for their study, scientists must devote a large portion of their time to producing reports, grant applications, and other paperwork. Blockchain Adoption in Academia Finally, there has been a decrease in research spending. They hope that private foundations, businesses, and corporations will take up the financing of large-scale research projects. Due to this, companies like Crypto.com are funding academic research and innovation to help blockchain innovation and advance web3 adoption. The Secure Blockchain Initiative at Carnegie Mellon University and the Blockchain Association of Singapore are two such organisations that Crypto.com sponsors. Academics are more interested in blockchain’s social aspects and impact on the traditional ecosystem and financial systems than the technology itself. As per Artyom, people have missed the analogy: “Scientific data lends itself ideally to the blockchain technology since it is a big, dynamic body of knowledge and data that is jointly generated, edited, utilised, and shared.” How Does this Impact the Blockchain Industry The funding and creation of a dedicated research centre at the University of Pennsylvania help foster academic research and innovation in various aspects of Web3, such as DeFi, NFTs and Stablecoins. Funding in DLT is directly related to higher research impact and allows for innovation in several disciplines of DeFi, Stablecoins, NFT and governance. Knowledge-based societies rely heavily on scientific research by universities and research institutes—the resulting knowledge fuels advancements in science and technology and the economy and society as a whole. The same knowledge-based funding helps blockchain and web3 applications to fuel and spill over to the economy and society. conducted Since computer science-based businesses and technologies such as DeFi and Stablecoins protocols are becoming more important, it’s essential to consider how scientific information is formed and whether public and private financing such as that of Crypto.com might speed up new the production and dissemination of new knowledge. Funding should be allocated to encourage the emergence of new ideas in web3 and the development of DLT advances and technological benefits. Final Thoughts on Crypto.com Funding Academic Research It is hoped that Penn's new Crypto Research Laboratory Fund will assist solve real-world concerns in blockchain privacy, dependability and scalability. The University of Pennsylvania's crypto lab's research will be very beneficial for real-world blockchain applications and innovative methods to enhance privacy and security capabilities - a step in the right direction for solidifying web3 in the academic circle. Don’t forget to like and share the story! Image credits: , and . Giammarco Gabriel Ghnassia Marie Bellando Mitjans

Crypto.com Funds University of Pennsylvania’s Research Laboratory to Promote Research on Web3

When it comes to multiplayer horror games, the formula is pretty similar across each title. Players either cooperate with each other against a supernatural force, fight amongst themselves and a supernatural force, or sometimes take control of that supernatural force to take out other players. There’s quite a bit of variety, but it’s rare when a game truly shakes up the formula. This is where comes into play. SCP: Secret Laboratory In this free-to-play multiplayer title, players are split across multiple different teams. Secret Laboratory makes these teams more interesting with their goals; rival teams can find themselves with the same objective, and even people on the same team can end up hurting their allies in order to achieve victory. Each game can feel completely different, to the point where your teammates can be a detriment...or your enemies provide you with the means to achieve victory. As of right now, the SCP: Secret Laboratory Parabellum update is in the works. Players can opt-in to play it on Steam in a beta format by going into the game’s properties in your library; publicbeta - 11.0.0 is what you should click. The full release isn’t quite ready yet, but if previous updates are anything to go by, it should be in a couple of weeks. If you haven’t heard of this game before and you’re interested in getting into SCP: Secret Laboratory Parabellum, you should probably know what the game is about first. Information about the game can be found on its . But for those of you that want a quick guide to help you jump in, read on! official website This article will help any newcomer to the game, as well as current players that just want to learn more about the Parabellum update. SCP: Secret Laboratory - Gameplay, Setting, and Mechanics takes place in an underground facility. Within the facility, there are two main groups - Humans and SCPs. SCP: Secret Laboratory Humans need to escape the facility, help other humans escape, or neutralize any threats remaining in the facility. SCPs are the “monsters” of this game; using special anomalous powers, their goal is to kill pretty much any human they see. While generally much weaker than SCPs, humans have a number of tools to fight back. One of these tools is the ability to pick items up, such as weapons to use against their enemies or keycards that can help them escape the facility. Still, the SCPs have massive pools of health and are usually able to kill humans in just a couple of hits, if not instantly. This isn’t particularly scary by itself, but the true horror comes in its multiplayer gameplay. In your average game of SCP: Secret Laboratory, you won’t ever encounter bots or AI. Every human and every dangerous SCP is meant to be controlled by another player on your server. The only way to communicate in-game is through voice chat, too - most players use a microphone during gameplay. Because of this, nearly every game can be different - if you’re lucky, you can strike bargains with some of the deadliest creatures you’ll face. This won’t always work, of course, but it throws a wrench into the usual structure of each round that makes things very intense...and, many times, very fun as well. Class-D Personnel and Scientists - Escaping the Facility Image from the Steam Community The most widespread team of the bunch, Class-D Personnel, are orange-garbed humans that have been locked up by the facility’s staff. T heir goal is to escape the facility alive, whether on their own terms or in handcuffs. Meanwhile, Scientists are the most scarce humans; with a bright white suit and not-very-helpful glasses, their goal is to escape the facility as well. But they start off with a keycard that can help give them a head start in their escape. Both of these teams spawn in the Light Containment Zone, the deepest part of the facility. It’s a randomly generated maze containing numerous locked doors that can only be opened with keycards. The game usually starts out with the objective of finding the room marked “ ”, which - when opened with a keycard - contains a station that players can use to upgrade their items. Using it, players can get the highest tiers of weapons and keycards, allowing them to escape the facility in one fell swoop. 914 This isn’t the only way players move through the zone, however. Certain rooms may spawn with a gun inside, and either team can use it to kill (or, in the Parabellum update, handcuff) rival humans. Particularly speedy players might also find other keycards in those rooms, which they can use to open the 914 room without being a scientist...or to try and leave the zone early if the card they find allows it. Mobile Task Force and Facility Guards - Securing the Facility is a group of players that spawns on the facility’s surface after the round’s start. They come heavily equipped with blue uniforms, powerful guns, high-tier keycards, and items that offer healing and protection. The Mobile Task Force Their goal is to rescue scientists and gun down everyone and everything else. They can also handcuff Class-D Personnel to help them escape, but that’s just an option - the MTF can usually gun them down at their discretion. Facility Guards are MTF by a different name, but they spawn at the round’s start. Their only major differences are their gray uniforms and their generally weaker equipment, along with spawning in the Entrance Zone - the last section of the facility before the escape. They won’t be able to open doors that lead to the surface, though, so Facility Guards will need to head deeper into the facility to find useful keycards in order to do so. At least one scientist or handcuffed Class-D needs to escape in order for the MTF to win; on top of this, they also need to neutralize all other threats in the facility. This is where they face their toughest moments because SCPs - and belligerent Class-D Personnel - aren’t their only foes. Chaos Insurgency and SCPs - Attacking the Facility Sometimes, instead of the Mobile Task Force, a different group will spawn on the surface. This group is known as the Chaos Insurgency. These green-clad, gas mask-wearing players come equipped with even heavier weaponry and all-around better keycards than MTF members. It can be rare for them to spawn, but their equipment gives them a major advantage over their rivals. The goal of the Chaos Insurgency is to rescue at least one Class-D Personnel (or one handcuffed scientist). They have no further requirements other than taking out whatever remains of the Mobile Task Force. In fact, the Chaos Insurgency can very commonly cooperate with SCPs, as neither team needs to attack each other in order to secure a victory. They have the option to, but neither side gains much from it. On that note, the SCPs are the deadliest team of them all. They all spawn in the Heavy Containment Zone, smack-dab in the middle of the facility...with the exception of SCP-173, which spawns in the Light Containment Zone. Their goal is simple: to kill any human they come across. As long as one SCP is alive and no Class-D Personnel or handcuffed Scientists escape, they will always have a chance at winning. All they need is to be the last group standing...or, under the right circumstances, for the Chaos Insurgency to have killed everyone else without rescuing anyone. It can be a bit difficult to talk about the SCPs, as they’re the most diverse of the teams. I’ll list them off one by one here. SCP Classes: SCP-173 - A living statue that only moves when not being looked at. It can force humans to blink and snaps their necks if it gets too close. SCP-106 - A shadowy humanoid that can walk through walls and teleport by setting down portals. Its attacks send players to a dark dimension where they need to blindly walk through a corridor that may or may not kill them. SCP-049 - A plague doctor that can kill humans with a single attack. It can interact with recently-deceased corpses to revive humans as SCP-049-2, a zombie SCP that attacks by clawing at people. SCP-096 - A tall, pale man that walks around sobbing. Looking at its face will send it into a rage, boosting its speed and marking anyone that saw it as a target to be instantly killed. Can also open larger doors that require keycards and break open paths to certain locked rooms. SCP-939 - A red dog-like SCP that can speak to humans using voice chat to trick or taunt them. It attacks with a powerful bite that can kill most players in just two hits. SCP-079 - A sentient computer that takes control of the facility’s cameras. It doesn’t attack directly, but it can relay information to other SCPs, lock doors, and activate traps to assist in slaying humans. These are the various classes you’ll face - or potentially play as - when joining a game of SCP: Secret Laboratory. If you need a refresher in-game, pressing and holding F1 will bring up an information screen about whatever class you’re playing as. It will also list key bindings, which you can modify to your leisure in the game’s settings menu. This is most of what you need to know before starting a game of SCP: Secret Laboratory for the first time. But when the Parabellum update makes its full release, there are some major changes to the game you’ll want to keep an eye out for. SCP: Secret Laboratory Parabellum - New Mechanics and Major Differences As part of the Parabellum update, SCP-173 ended up getting a new model and new mechanics. It can now give itself a speed boost to move faster through the facility, as well as leave behind puddles that slow down players crossing its path. 173 also has way more control over making players blink, allowing it to plan its position and eliminate humans with much greater efficiency. The changes go a little further than that too. SCP-079 was made easier for humans to take out, many of the guns have been reworked with new models and attachments, and players are able to handcuff multiple people at a time. Further information about the new additions to the update can be found , so feel free to check that out at your leisure. on the game’s official website Final Thoughts Even in the beta, some servers are modded to introduce their own changes to the game. Some of them simply change the health values of some SCPs, while others go as far as making it impossible to end the round unless the SCPs kill the Chaos Insurgency (or vice versa). You can usually see what changes were made to each server by clicking them on the server list - some text will appear detailing additional information. SCP: Secret Laboratory has a pretty complex gameplay structure, but it’s not actually as deep as you might think. Currently, there’s no ranking system, no major progress to keep track of, and no larger incentive to keep playing outside of just having fun. This article will likely help new players get into the game much more easily, but no one should feel pressured over needing to be as good as they can. In the end, SCP: Secret Laboratory is simply a fun multiplayer horror game. And, seeing as it’s free, it’s one game I would highly recommend to anyone, especially with its newest update on the verge of a full release. Disclaimer: Most images taken from the game's Steam page and its official wiki

SCP: Secret Laboratory Parabellum - A Beginner's Guide

Part 23 of where I interview my heroes. The series Index to “Interviews with ML Heroes” Today, I’m super excited to be talking to Dr. Leslie Smith. I’m sure Leslie needs no introduction to our friends from the community. For our readers from outside of fast.ai: fast.ai Leslie is currently working as a Senior Research Scientist at the Naval Center for Applied Research in AI, United States Naval Research Laboratory. His past research works include includes deep neural networks and reinforcement applied to robotics research. Prior to that, he has worked in the Maritime Surveillance Section. learning He has a background in Chemistry, he has done his Ph.D. in the Quantum Chemistry domain. His Research Objectives are to perform innovative scientific research and algorithm development in the areas of computer vision, machine learning, robotics, and sparse representations. About the Series: I have very recently started making some progress with my . But to be honest, it wouldn’t be possible at all without the amazing community online and the great people that have helped me. Self-Taught Machine Learning Journey In this Series of Blog Posts, I talk with people that have really inspired me and whom I look up to as my role-models. The motivation behind doing this is that you might see some patterns and hopefully, you’d be able to learn from the amazing people that I have had the chance to learn from. **Sanyam Bhutani:**​ Hello Leslie, Thank you for taking the time to do this. I am honored that you thought to invite me for this interview Dr. Leslie Smith: **Sanyam Bhutani:**​ You’re currently performing scientific research and algorithm development in the areas of computer vision, machine learning, robotics, and sparse representations. You’ve been working at the Naval Research Centre for more than 15 years now. Could you tell the readers about how you got started with Machine Learning? What got you interested in Machine/Deep Learning at first? For the first several years that I worked at the Naval Research Laboratory, I was working in computer vision and related areas. I took notice that neural networks won the 2012 ImageNet Challenge by a wide margin and Google’s “Cats” paper “Building high-level features using large scale unsupervised learning”. In 2013 I started trying a few things and found I was fascinated by the field. Over the next year, I shifted my focus to deep learning and I think most all the other researchers in computer vision did the same in the following couple of years. Dr. Leslie Smith: You had a background as a Business founder before switching to research. Sanyam Bhutani: What made you pick research as a career path? Actually, I picked research as a career when I was 10 years old. I was interested in science and a family trip to the New York World’s fair in 1964 was fascinating to me. I picked to focus on Chemistry but in university, I found physics more interesting. So my Ph.D. is in Chemical Physics. Unfortunately, my postdoc at Princeton University didn’t meet my expectation so I left to work in the industry. After 8 years in the industry, I thought I could create my own business. It took me a decade to realize running a business was not right for me. I joke that I couldn’t sell $20 bills for $5. After some soul searching I realized that doing research is what I find most satisfying. That set me on the road to where I am. Now I really love what I do. Dr. Leslie Smith: Can you tell us more about what you’re currently working on? Sanyam Bhutani: Are you secretly building our Robot AI Overlords? :D Of course. Isn’t everyone! :-) Dr. Leslie Smith: I am working on lots of ideas, which is part of the fun. Fortunately, I am creative and I have many more ideas than I could possibly have time to work on them all. I joke that it is hard to get everything done in a 40 hour day! But I try. Some of the ideas that I am working on include being able to train networks without weight decay (one less hyper-parameter), online batch selection via importance labeling of data, combining novelty detection with few-shot learning, dynamic data augmentation, automatic data labeling, and label cleaning, and defending against adversarial examples. I can’t work on all of these in a single day so each day I pick one topic to focus on. You’ve been working as a researcher for quite a few years now. What has been your favorite project during these years? Sanyam Bhutani: It is always the next one. I get super excited by my ideas. It gives me the motivation to work hard at it. Dr. Leslie Smith: For the readers that are curious about what does a day in the life of a researcher look like, can you give us an insight? Sanyam Bhutani: How much time do you spend on Experimenting Vs Exploring new ideas? I don’t know about other researchers but a majority of my time is spent on reading, experimenting, writing, email and talking with people. Reading and experimenting are the catalysts for most of my ideas. Dr. Leslie Smith: Could you tell us a bit about how do you decide to start a new experiment? What kinds of problems or questions pique your interest? Sanyam Bhutani: Well, if I am reading a paper and it leads to an idea, a big factor is if the authors made their code available. If so, I’ll download it and run it to replicate their experiments. Then I can quickly try out my own idea to see if it makes sense. Quick prototyping is important to me here. Also, their code provides a baseline. Dr. Leslie Smith: Another important factor is my confidence in how likely the idea is to work. If I am confident, I am more motivated to try it than if not. Once you’ve finally found an idea that you want to explore. What are your go-to techniques? How do you approach a question or idea when getting started? Sanyam Bhutani: Start simple. As I said, code on GitHub is a good start and provides a baseline. For example, when I started with few-shot learning, the codes for prototypical networks and MAML were available for a start. Then I try tweaking everything, just to be sure my intuition about how I think it should work matches reality. Dr. Leslie Smith: A part of Research is to know when to put an end to the experiment or continue experimenting. Sanyam Bhutani: For our readers who might get disheartened if their idea isn’t working first or for those who might have been obsessing over some idea for a long while even when things aren’t working, Can you tell us how do you decide between these two? When an idea is not working out, I must understand why. Is it a mistake I made in the code? Or was I wrong in my thinking. If I was wrong, I learn from this and update my intuition. In this way failed experiments are learning experiences, which are valuable. Dr. Leslie Smith: For the readers and the beginners who are dreaming of doing research in this domain, what would be your best advice? Sanyam Bhutani: Learn from the best and learn from everything. I admire Yoshua Bengio and read most of his papers — and there are lots of them. Also, don’t be afraid to fail or ask stupid questions, Everything is a learning experience. On the other hand, don’t make the same mistake twice. I write everything down in my lab notebook and I review it regularly, I will often take time to quietly think about items to see where my thoughts lead me. Dr. Leslie Smith: Many people are of the opinion that making significant contributions to the field requires one to be a post-grad or have research experience. Sanyam Bhutani: For the readers who want to take up Machine Learning as a Career path, do you feel having research experience is a necessity? I believe deep learning is transitioning from mostly research to mostly engineering. There is a world of new potential applications that are yet to be created. There are new factors that will become important, such as verification, reproducibility, explainability, integrity, etc. Real world data is substantially different than the research benchmarks like MNIST and ImageNet. Hence research experience is not necessary to do many of these things. Dr. Leslie Smith: Another opinion that is a “mental barrier” to many is that doing “Machine Learning” or Machine Learning Research requires you to have a cluster of GPU servers and expensive hardware in order to make significant contributions. Sanyam Bhutani: What are your thoughts on this opinion? As I was just saying, data is more important than hardware. Google researchers have access to thousands of GPUs and TPUs but I don’t. It doesn’t prevent me from doing what I can. I’d say my message is “find your own niche”. Do something no one else thought to do. Dr. Leslie Smith: Given the explosive growth rates in research, How do you stay up to date with the cutting edge? Sanyam Bhutani: I spend many, many hours every week keeping up to date but that is just me. On Mondays and Wednesdays, I look through all the new papers on arXiv.org to find ones that might be relevant. This has become a ritual for me. The ones that look relevant, I read the abstract and skim the paper. If it catches my interest, I print it. I spend most of my weekend reading these papers. This takes hours but I stay up to date and reading is a great source of intuition and new ideas. Dr. Leslie Smith: Do you feel Machine Learning has been overhyped? Sanyam Bhutani: Of course it is. I like to paraphrase the quote “All Machine Learning is wrong but sometimes it is useful”. It has proven useful in computer vision, machine translation, and speech recognition, to name a few. I recommend finding new ways to make it useful. Dr. Leslie Smith: I think the complete fastai community is grateful to you for your research on the “1 cycle learning policy”. Sanyam Bhutani: What are your thoughts about the fast.ai course and the community? I hold Jeremy Howard in high regard. He saw the need for practical deep learning and generously provided a wonderful solution. The community that he created is vibrant and strong. Dr. Leslie Smith: Before we conclude, any tips for the beginners who are afraid to get started because of the idea that Deep Learning is an advanced field? Sanyam Bhutani: Beginnings are always the hardest part. As a scientist, I view life as a series of experiments. Make a deal with yourself to try it for a time that is long enough to know if it is right for you. If it is not, stop and go onto something else. Knowing it is an experiment and not a commitment makes it a lot easier to try things. Dr. Leslie Smith: Thank you so much for doing this interview. Sanyam Bhutani: If you found this interesting and would like to be a part of My Learning Path , you can find me on Twitter here . If you’re interested in reading about Deep Learning and Computer Vision news, you can check out my newsletter here .

Interview with Senior Research Scientist at the US Naval Research Laboratory: Dr. Leslie Smith

Ensuring Reproducibility in AI Research: Code and Pre-trained Weights Open-Sourced

Authors:
(1) Yuwei Guo, The Chinese University of Hong Kong;
(2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author;
(3) Anyi Rao, Stanford University;
(4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory;
(5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory;
(6) Yu Qiao, Shanghai Artificial Intelligence Laboratory;
(7) Maneesh Agrawala, Stanford University;
(8) Dahua Lin, Shanghai Artificial Intelligence Laboratory;
(9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Table of Links Abstract and 1 Introduction 2 Work Related 3 Preliminary AnimateDiff 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.2 Learn Motion Priors with Motion Module 4.3 Adapt to New Motion Patterns with MotionLora 4.4 AnimateDiff in Practice 5 Experiments and 5.1 Qualitative Results 5.2 Qualitative Comparison 5.3 Ablative Study 5.4 Controllable Generation 6 Conclusion 7 Ethics Statement 8 Reproducibility Statement, Acknowledgement and References 7 ETHICS STATEMENT We strongly condemn the misuse of generative AI to create content that harms individuals or spreads misinformation. However, we acknowledge the potential for our method to be misused since it primarily focuses on animation and can generate human-related content. It is also important to highlight that our method incorporates personalized text-to-image models developed by other artists. These models may contain inappropriate content and can be used with our method. To address these concerns, we uphold the highest ethical standards in our research, including adhering to legal frameworks, respecting privacy rights, and encouraging the generation of positive content. Furthermore, we believe that introducing an additional content safety checker, similar to that in Stable Diffusion (Rombach et al., 2022), could potentially resolve this issue. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Yuwei Guo, The Chinese University of Hong Kong; (2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author; (3) Anyi Rao, Stanford University; (4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory; (5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory; (6) Yu Qiao, Shanghai Artificial Intelligence Laboratory; (7) Maneesh Agrawala, Stanford University; (8) Dahua Lin, Shanghai Artificial Intelligence Laboratory; (9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Authors: Authors: (1) Yuwei Guo, The Chinese University of Hong Kong; (2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author; (3) Anyi Rao, Stanford University; (4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory; (5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory; (6) Yu Qiao, Shanghai Artificial Intelligence Laboratory; (7) Maneesh Agrawala, Stanford University; (8) Dahua Lin, Shanghai Artificial Intelligence Laboratory; (9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Work Related 2 Work Related 3 Preliminary 3 Preliminary AnimateDiff AnimateDiff 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.2 Learn Motion Priors with Motion Module 4.2 Learn Motion Priors with Motion Module 4.3 Adapt to New Motion Patterns with MotionLora 4.3 Adapt to New Motion Patterns with MotionLora 4.4 AnimateDiff in Practice 4.4 AnimateDiff in Practice 5 Experiments and 5.1 Qualitative Results 5 Experiments and 5.1 Qualitative Results 5.2 Qualitative Comparison 5.2 Qualitative Comparison 5.3 Ablative Study 5.3 Ablative Study 5.4 Controllable Generation 5.4 Controllable Generation 6 Conclusion 6 Conclusion 7 Ethics Statement 7 Ethics Statement 8 Reproducibility Statement, Acknowledgement and References 8 Reproducibility Statement, Acknowledgement and References 7 ETHICS STATEMENT We strongly condemn the misuse of generative AI to create content that harms individuals or spreads misinformation. However, we acknowledge the potential for our method to be misused since it primarily focuses on animation and can generate human-related content. It is also important to highlight that our method incorporates personalized text-to-image models developed by other artists. These models may contain inappropriate content and can be used with our method. To address these concerns, we uphold the highest ethical standards in our research, including adhering to legal frameworks, respecting privacy rights, and encouraging the generation of positive content. Furthermore, we believe that introducing an additional content safety checker, similar to that in Stable Diffusion (Rombach et al., 2022), could potentially resolve this issue. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

AnimateDiff Ethics Statement: Ensuring Responsible Use of Generative AI for Animation

Authors:
(1) Yuwei Guo, The Chinese University of Hong Kong;
(2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author;
(3) Anyi Rao, Stanford University;
(4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory;
(5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory;
(6) Yu Qiao, Shanghai Artificial Intelligence Laboratory;
(7) Maneesh Agrawala, Stanford University;
(8) Dahua Lin, Shanghai Artificial Intelligence Laboratory;
(9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Table of Links Abstract and 1 Introduction 2 Work Related 3 Preliminary AnimateDiff 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.2 Learn Motion Priors with Motion Module 4.3 Adapt to New Motion Patterns with MotionLora 4.4 AnimateDiff in Practice 5 Experiments and 5.1 Qualitative Results 5.2 Qualitative Comparison 5.3 Ablative Study 5.4 Controllable Generation 6 Conclusion 7 Ethics Statement 8 Reproducibility Statement, Acknowledgement and References 6 CONCLUSION In this paper, we present AnimateDiff, a practical pipeline directly turning personalized text-toimage (T2I) models for animation generation once and for all, without compromising quality or losing pre-learned domain knowledge. To accomplish this, we design three component modules in AnimateDiff to learn meaningful motion priors while alleviating visual quality degradation and enabling motion personalization with a lightweight fine-tuning technique named MotionLoRA. Once trained, our motion module can be integrated into other personalized T2Is to generate animated images with natural and coherent motions while remaining faithful to the personalized domain. Extensive evaluation with various personalized T2I models also validates the effectiveness and generalizability of our AnimateDiff and MotionLoRA. Furthermore, we demonstrate the compatibility of our method with existing content-controlling approaches, enabling controllable generation without incurring additional training costs. Overall, AnimateDiff provides an effective baseline for personalized animation and holds significant potential for a wide range of applications. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Yuwei Guo, The Chinese University of Hong Kong; (2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author; (3) Anyi Rao, Stanford University; (4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory; (5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory; (6) Yu Qiao, Shanghai Artificial Intelligence Laboratory; (7) Maneesh Agrawala, Stanford University; (8) Dahua Lin, Shanghai Artificial Intelligence Laboratory; (9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Authors: Authors: (1) Yuwei Guo, The Chinese University of Hong Kong; (2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author; (3) Anyi Rao, Stanford University; (4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory; (5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory; (6) Yu Qiao, Shanghai Artificial Intelligence Laboratory; (7) Maneesh Agrawala, Stanford University; (8) Dahua Lin, Shanghai Artificial Intelligence Laboratory; (9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Work Related 2 Work Related 3 Preliminary 3 Preliminary AnimateDiff AnimateDiff 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.2 Learn Motion Priors with Motion Module 4.2 Learn Motion Priors with Motion Module 4.3 Adapt to New Motion Patterns with MotionLora 4.3 Adapt to New Motion Patterns with MotionLora 4.4 AnimateDiff in Practice 4.4 AnimateDiff in Practice 5 Experiments and 5.1 Qualitative Results 5 Experiments and 5.1 Qualitative Results 5.2 Qualitative Comparison 5.2 Qualitative Comparison 5.3 Ablative Study 5.3 Ablative Study 5.4 Controllable Generation 5.4 Controllable Generation 6 Conclusion 6 Conclusion 7 Ethics Statement 7 Ethics Statement 8 Reproducibility Statement, Acknowledgement and References 8 Reproducibility Statement, Acknowledgement and References 6 CONCLUSION In this paper, we present AnimateDiff, a practical pipeline directly turning personalized text-toimage (T2I) models for animation generation once and for all, without compromising quality or losing pre-learned domain knowledge. To accomplish this, we design three component modules in AnimateDiff to learn meaningful motion priors while alleviating visual quality degradation and enabling motion personalization with a lightweight fine-tuning technique named MotionLoRA. Once trained, our motion module can be integrated into other personalized T2Is to generate animated images with natural and coherent motions while remaining faithful to the personalized domain. Extensive evaluation with various personalized T2I models also validates the effectiveness and generalizability of our AnimateDiff and MotionLoRA. Furthermore, we demonstrate the compatibility of our method with existing content-controlling approaches, enabling controllable generation without incurring additional training costs. Overall, AnimateDiff provides an effective baseline for personalized animation and holds significant potential for a wide range of applications. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

How AnimateDiff Transforms T2I Models into High-Quality Animation Generators with MotionLORA

Nature + Biomimicry + Bitcoin: Mutually Beneficial Partnership

Naturaleza + Biomimetismo + Bitcoin: una alianza mutuamente beneficiosa

Luonto + Biomimikri + Bitcoin: molemminpuolisesti hyödyllinen kumppanuus

प्रकृति + बायोमिमिक्री + बिटकॉइन: पारस्परिक रूप से लाभकारी साझेदारी

Nature + Biomimicry + Bitcoin. փոխշահավետ գործընկերություն

自然 + バイオミミクリー + ビットコイン: 相互に利益のあるパートナーシップ

Nature + Biomimicry + Bitcoin: ორმხრივად მომგებიანი პარტნიორობა

ធម្មជាតិ + Biomimicry + Bitcoin៖ ភាពជាដៃគូដែលមានអត្ថប្រយោជន៍ទៅវិញទៅមក

Natiora + Biomimicry + Bitcoin: Fiaraha-miombon'antoka

Masikirwo + Biomimicry + Bitcoin: Kudyidzana Kunobatsira Kudyidzana

Природа + биомимикрија + биткоин: обострано корисно партнерство

இயற்கை + பயோமிமிக்ரி + பிட்காயின்: பரஸ்பர நன்மை பயக்கும் கூட்டாண்மை

Tabiat + Biomimicry + Bitcoin: O'zaro manfaatli hamkorlik

Authors:
(1) Yuwei Guo, The Chinese University of Hong Kong;
(2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author;
(3) Anyi Rao, Stanford University;
(4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory;
(5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory;
(6) Yu Qiao, Shanghai Artificial Intelligence Laboratory;
(7) Maneesh Agrawala, Stanford University;
(8) Dahua Lin, Shanghai Artificial Intelligence Laboratory;
(9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Table of Links Abstract and 1 Introduction 2 Work Related 3 Preliminary AnimateDiff 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.2 Learn Motion Priors with Motion Module 4.3 Adapt to New Motion Patterns with MotionLora 4.4 AnimateDiff in Practice 5 Experiments and 5.1 Qualitative Results 5.2 Qualitative Comparison 5.3 Ablative Study 5.4 Controllable Generation 6 Conclusion 7 Ethics Statement 8 Reproducibility Statement, Acknowledgement and References 5.4 CONTROLLABLE GENERATION. The separated learning of visual content and motion priors in AnimateDiff enables the direct application of existing content control approaches for controllable generation. To demonstrate this capability, we combined AnimateDiff with ControlNet (Zhang et al., 2023) to control the generation with extracted depth map sequence. In contrast to recent video editing techniques (Ceylan et al., 2023; Wang et al., 2023a) that employ DDIM (Song et al., 2020) inversion to obtain smoothed latent sequences, we generate animations from randomly sampled noise. As illustrated in Figure 8, our results exhibit meticulous motion details (such as hair and facial expressions) and high visual quality. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Yuwei Guo, The Chinese University of Hong Kong; (2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author; (3) Anyi Rao, Stanford University; (4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory; (5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory; (6) Yu Qiao, Shanghai Artificial Intelligence Laboratory; (7) Maneesh Agrawala, Stanford University; (8) Dahua Lin, Shanghai Artificial Intelligence Laboratory; (9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Authors: Authors: (1) Yuwei Guo, The Chinese University of Hong Kong; (2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author; (3) Anyi Rao, Stanford University; (4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory; (5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory; (6) Yu Qiao, Shanghai Artificial Intelligence Laboratory; (7) Maneesh Agrawala, Stanford University; (8) Dahua Lin, Shanghai Artificial Intelligence Laboratory; (9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Work Related 2 Work Related 3 Preliminary 3 Preliminary AnimateDiff AnimateDiff 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.2 Learn Motion Priors with Motion Module 4.2 Learn Motion Priors with Motion Module 4.3 Adapt to New Motion Patterns with MotionLora 4.3 Adapt to New Motion Patterns with MotionLora 4.4 AnimateDiff in Practice 4.4 AnimateDiff in Practice 5 Experiments and 5.1 Qualitative Results 5 Experiments and 5.1 Qualitative Results 5.2 Qualitative Comparison 5.2 Qualitative Comparison 5.3 Ablative Study 5.3 Ablative Study 5.4 Controllable Generation 5.4 Controllable Generation 6 Conclusion 6 Conclusion 7 Ethics Statement 7 Ethics Statement 8 Reproducibility Statement, Acknowledgement and References 8 Reproducibility Statement, Acknowledgement and References 5.4 CONTROLLABLE GENERATION. The separated learning of visual content and motion priors in AnimateDiff enables the direct application of existing content control approaches for controllable generation. To demonstrate this capability, we combined AnimateDiff with ControlNet (Zhang et al., 2023) to control the generation with extracted depth map sequence. In contrast to recent video editing techniques (Ceylan et al., 2023; Wang et al., 2023a) that employ DDIM (Song et al., 2020) inversion to obtain smoothed latent sequences, we generate animations from randomly sampled noise. As illustrated in Figure 8, our results exhibit meticulous motion details (such as hair and facial expressions) and high visual quality. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

AnimateDiff Combines with ControlNet for Precise Motion Control and High-Quality Video Generation

Authors:
(1) Yuwei Guo, The Chinese University of Hong Kong;
(2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author;
(3) Anyi Rao, Stanford University;
(4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory;
(5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory;
(6) Yu Qiao, Shanghai Artificial Intelligence Laboratory;
(7) Maneesh Agrawala, Stanford University;
(8) Dahua Lin, Shanghai Artificial Intelligence Laboratory;
(9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Table of Links Abstract and 1 Introduction 2 Work Related 3 Preliminary AnimateDiff 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.2 Learn Motion Priors with Motion Module 4.3 Adapt to New Motion Patterns with MotionLora 4.4 AnimateDiff in Practice 5 Experiments and 5.1 Qualitative Results 5.2 Qualitative Comparison 5.3 Ablative Study 5.4 Controllable Generation 6 Conclusion 7 Ethics Statement 8 Reproducibility Statement, Acknowledgement and References 5.3 ABLATIVE STUDY Domain adapter. To investigate the impact of the domain adapter in AnimateDiff, we conducted a study by adjusting the scaler in the adapter layers during inference, ranging from 1 (full impact) to 0 (complete removal). As illustrated in Figure 6, as the scaler of the adapter decreases, there is an improvement in overall visual quality, accompanied by a reduction in the visual content distribution learned from the video dataset (the watermark in the case of WebVid (Bain et al., 2021)). These results indicate the successful role of the domain adapter in enhancing the visual quality of AnimateDiff by alleviating the motion module from learning the visual distribution gap. Motion module design. We compare our motion module design of the temporal Transformer with its full convolution counterpart, which is motivated by the fact that both designs are widely employed in recent works on video generation. We replace the temporal attention with 1D temporal convolution and ensured that the two model parameters were closely aligned. As depicted in supplementary materials, the convolutional motion module aligns all frames to be identical but does not incorporate any motion compared to the Transformer architecture. Efficiency of MotionLoRA. The efficiency of MotionLoRA in AnimateDiff was examined in terms of parameter efficiency and data efficiency. Parameter efficiency is crucial for efficient model training and sharing among users, while data efficiency is essential for real-world applications where collecting an adequate number of reference videos for specific motion patterns may be challenging. To investigate these aspects, we trained multiple MotionLoRA models with varying parameter scales and reference video quantities. In Fig. 7, the first two samples demonstrate that MotionLoRA is capable of learning new camera motions (e.g., zoom-in) with a small parameter scale while maintaining comparable motion quality. Furthermore, even with a modest number of reference videos (e.g., N = 50), the model successfully learns the desired motion patterns. However, when the number of reference videos is excessively limited (e.g., N = 5), significant degradation in quality is observed, suggesting that MotionLoRA encounters difficulties in learning shared motion patterns and instead relies on capturing texture information from the reference videos. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Yuwei Guo, The Chinese University of Hong Kong; (2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author; (3) Anyi Rao, Stanford University; (4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory; (5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory; (6) Yu Qiao, Shanghai Artificial Intelligence Laboratory; (7) Maneesh Agrawala, Stanford University; (8) Dahua Lin, Shanghai Artificial Intelligence Laboratory; (9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Authors: Authors: (1) Yuwei Guo, The Chinese University of Hong Kong; (2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author; (3) Anyi Rao, Stanford University; (4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory; (5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory; (6) Yu Qiao, Shanghai Artificial Intelligence Laboratory; (7) Maneesh Agrawala, Stanford University; (8) Dahua Lin, Shanghai Artificial Intelligence Laboratory; (9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Work Related 2 Work Related 3 Preliminary 3 Preliminary AnimateDiff AnimateDiff 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.2 Learn Motion Priors with Motion Module 4.2 Learn Motion Priors with Motion Module 4.3 Adapt to New Motion Patterns with MotionLora 4.3 Adapt to New Motion Patterns with MotionLora 4.4 AnimateDiff in Practice 4.4 AnimateDiff in Practice 5 Experiments and 5.1 Qualitative Results 5 Experiments and 5.1 Qualitative Results 5.2 Qualitative Comparison 5.2 Qualitative Comparison 5.3 Ablative Study 5.3 Ablative Study 5.4 Controllable Generation 5.4 Controllable Generation 6 Conclusion 6 Conclusion 7 Ethics Statement 7 Ethics Statement 8 Reproducibility Statement, Acknowledgement and References 8 Reproducibility Statement, Acknowledgement and References 5.3 ABLATIVE STUDY Domain adapter. To investigate the impact of the domain adapter in AnimateDiff, we conducted a study by adjusting the scaler in the adapter layers during inference, ranging from 1 (full impact) to 0 (complete removal). As illustrated in Figure 6, as the scaler of the adapter decreases, there is an improvement in overall visual quality, accompanied by a reduction in the visual content distribution learned from the video dataset (the watermark in the case of WebVid (Bain et al., 2021)). These results indicate the successful role of the domain adapter in enhancing the visual quality of AnimateDiff by alleviating the motion module from learning the visual distribution gap. Domain adapter. Motion module design. We compare our motion module design of the temporal Transformer with its full convolution counterpart, which is motivated by the fact that both designs are widely employed in recent works on video generation. We replace the temporal attention with 1D temporal convolution and ensured that the two model parameters were closely aligned. As depicted in supplementary materials, the convolutional motion module aligns all frames to be identical but does not incorporate any motion compared to the Transformer architecture. Motion module design. Efficiency of MotionLoRA. The efficiency of MotionLoRA in AnimateDiff was examined in terms of parameter efficiency and data efficiency. Parameter efficiency is crucial for efficient model training and sharing among users, while data efficiency is essential for real-world applications where collecting an adequate number of reference videos for specific motion patterns may be challenging. To investigate these aspects, we trained multiple MotionLoRA models with varying parameter scales and reference video quantities. In Fig. 7, the first two samples demonstrate that MotionLoRA is capable of learning new camera motions (e.g., zoom-in) with a small parameter scale while maintaining comparable motion quality. Furthermore, even with a modest number of reference videos (e.g., N = 50), the model successfully learns the desired motion patterns. However, when the number of reference videos is excessively limited (e.g., N = 5), significant degradation in quality is observed, suggesting that MotionLoRA encounters difficulties in learning shared motion patterns and instead relies on capturing texture information from the reference videos. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Stories, News & Mentions about LAWORATORY

Laworatory’s Stories on HackerNoon