DINGG

dingg

Larian Studios is perhaps the only company with a vast RPG history that still produces genuinely old-school RPGs. Behind this name are fantastic worlds, hours-long gameplay experiences, classic game design, and, of course, humor. The company has distinct canons and rules that give its games a recognizable touch. But at the same time, these also create limitations. Today, we'll discuss Baldur's Gate 3, Larian Studios' most popular project. It introduced the general public to something new - something long known to genre enthusiasts. game The founder of the studio, Swen Vincke, loves board games and has always aimed to transform that passion into a computer game format. This became the foundation for every game design element in the studio's projects. The direction for realizing their ideas was set by a game called Ultima VII. A direction that, according to Vincke, was never continued by anyone else. We're talking about a huge open world, many interactive items and their combinations, various ways to approach a single situation, and thrilling world exploration. These are the very pillars on which Larian Studios' game design stands. From a , the development of every gameplay situation in their games is built on a "+1" model. This means that developers simulate every possible outcome for a gameplay situation and then add... one more on top. This is not only about the progression of the main plot but about interactions with the world. Every barrel in Baldur's Gate 3 can be broken, exploded, or moved. Barrels can block a building entrance or be stacked into a staircase or tower to elevate an archer. One can even put something or someone inside a barrel and carry it around. And that's just the barrel, only one of the many interactive objects in the game. game design perspective Due to this interactivity, players encounter countless unexpected events and twists in their gameplay. Some passage options are always hidden from the player. Access to them might depend, for instance, on a skill to talk to animals or the dead, a random encounter, completing a companion's quest, or performing a specific action. Every chat or minor in-game event can spawn many new scenarios based on the "+1" idea. This makes the game world grow immensely, making even small areas and starting quests feel incredibly expansive. The desire to make the entire environment interactive has been with the studio since its earliest projects. But for me, the Divinity Original Sin series remains the most exemplary and innovative. Its ideas are what Baldur's Gate 3 primarily continues. In Original Sin, players could perform some utterly insane actions. The idea of interaction with "unremarkable" objects was executed brilliantly there. This world here is divided into several components: the regular world inhabited by various humanoids, the world of animals, the world of objects, and the world of living objects. Here, you could mediate disputes between a dog and a cat, accept an intriguing quest from a chicken, argue with a chest, and wear buckets on your head. In this world, objects aren't insignificant, as in most games. Any unliving object in the game might be more important than an NPC. The animals possess a set of qualities, emotions, and motivations that rival those of humans. For instance, in Baldur's Gate 3, there's the story of the "dark urge," obsessed with a desire for blood, paralleled by an emotionally similar dramatic tale of the "Strange Ox." The popular scene from Baldur's Gate 3 trailer, where one has intercourse with a druid in bear form, might be shocking and innovative for the general audience but certainly not for the studio. Compare this with the possibility in Original Sin for elves to peek into the past by eating a corpse. This feature is also continued in Baldur's Gate 3, where players can activate brains and converse with their former owners by inserting them into a skull. The idea of high interactivity brings the element many of us love in games – randomness. In some places, the stakes of this randomness are high, and you wish to encounter it as rarely as possible, like in From Software games. When rolling, the planks beneath you break, and you plunge to death, losing souls. But these random moments are nice in other cases, such as in Baldur's Gate. Both types entertain the player and make their journey emotionally rich. In the discussion on randomness, I'd like to highlight another crucial mechanic in Baldur's Gate 3 – turn-based battles. I won't go into deep details; they're complex and worth a separate discussion. But I'll mention the role of surfaces and how they interact in battles, which makes combat amazing for me. You can set oil, explode barrels to create fire zones, electrify water with lightning, or use gravity to push enemies off the map. This is just a glimpse of the possible interactions. These can be advantageous or dangerous - you might harm teammates or face cunning enemies who use the same tactics. The game's AI is smart, often outthinking the player. So, if an enemy immediately pushes your top character off a ledge, it's intentional, not random. Surfaces play into the primary element of combat – strategic mechanics, but they often emerge from randomness and accompany it until the end. Each player's battle is a unique combination of characters and their builds, abilities, strategies, and the player's reactions to surprises. Developers craft confrontations such that every fight has a vexing element for the player. An archer perched atop a staircase, a necromancer summoning skeletons each turn, or an approaching massive troll. Players strategize to neutralize these threats by prioritizing them and assessing the situation. They are given many opportunities and allowed to combine them as they please. This is why the battles are turn-based. Players have time to ponder the situation, fantasize, and decide. In the first act, you always feel you're given a platform to shine, showcase your intellect, and find an intriguing solution. In the first act, you get to know the game's classes and characters, form your team, and discover how they work together. Every battle becomes an opportunity to try out new skills and strategies. This new game installment nails this aspect, making it feel like you're in . The magic amplifies when you play cooperatively. Imagine exploring a cave while your friend tackles city quests and then meeting to exchange stories at a tavern. Larian Studios has outdone itself, creating a cooperative experience that stands out among games. Their previous titles hinted at this brilliance. It's hard to put into words, but this is what top-notch multiplayer feels like a game where players can engage in rich dialogues, approach quests differently, and even follow their secret goals. a real DnD session A genuinely unique aspect of Baldur's Gate 3 is the DnD mechanic tied to dice rolls. In real life, DnD lets participants role-play any character and do almost anything, guided only by their moral compass and imagined character traits. Lifting restrictions on variability was the developers' most significant challenge when porting this mechanic into the game. They achieved this with minor plot influences from a plethora of small actions. The success of almost every action depends on a dice roll. If the result surpasses a certain value, the action succeeds, and vice versa. It's sometimes hard to resist reloading the game to achieve the desired roll. The most "correct" approach is to accept what happens, like in real DnD, and enjoy a unique storyline with successes and failures that shape your path. Thanks to thousands of these small checks, Baldur's Gate 3 allows each player a unique playthrough. If you play honestly, you can be certain that the likelihood of someone having a playthrough similar to yours has never been so low. Such a scale is impressive. However, it leads to an exponential increase in complexity. The more content and gameplay situations in the game, the more tangled the system becomes, making its realization harder. Hence, the peak number of bugs is concentrated in the third act. And here, I move to the critique. Firstly, all the mentioned mechanics were excellently implemented in Original Sin 2, the studio's previous game. While Baldur's Gate 3 handles them with care, it doesn't bring anything new to the table. Secondly, the vast world and its reliance on constant interactivity mean that after a perfect first act, there's always a second one with less variety. Items might be unique, but they function similarly to the first act's items; enemies become less surprising, and by this point, a character's abilities allow the player to establish a repetitive battle pattern. The game becomes duller. It still entertains with its narrative and nuances, but holistically, the player starts to feel burned out. Statistics show that most players drop off during the second act. Typically, only about 30% make it to its end. By the third act, the situation gets even gloomier. It's usually riddled with even more bugs. Characters, leveled up to god-like statuses, can easily make up to five attacks each turn and land critical hits every time. Gameplay variations start to narrow, and choices that appear different often lead to the same results. The game drags on as the story wraps up its loose ends, leaving the ending a bit diluted. Speaking of the narrative, it's always secondary in the studio's games. Baldur's Gate 3 is no exception. The spotlight is always on gameplay. And while this is good for gameplay, it's not cool for storytelling. The narrative is filled with funny jokes and cool side quests; the companions' stories and the dark urge's plot are enjoyable. But the main plot leans on clichés and templates from two decades ago, now feeling more like post-irony than a well-thought-out script. The mere 12 general endings don't intrigue and blatantly overlook most of the player’s choices. There's no satisfaction from the finale. The pleasure is hidden in the process. Only that process isn't all smooth, either. In Baldur's Gate 3 and all previous Larian games, I start to feel bored by the third act. I don't understand why they always structure the game using the same pattern every time, limiting themselves to the same path. Each new game so closely follows the template of its predecessor that it loses its uniqueness, feeling like an endless update. The glorious Act I, among other things, is conditioned by close interaction with the audience. It was in early access for years, and the developers received tons of feedback during that time. As a result, the game's beginning turned out to be as staggering as possible. Naturally, maintaining that initial benchmark becomes very challenging with such expansive world dimensions focusing on interactive interactions. There's always a shortage of time somewhere, and compromises must be made. My playthrough took 80 hours, but my overall impressions were mixed. I spent most of that time on the first half of the game, where, in my opinion, the most exciting discoveries and beautiful solutions are. I trudged through the second half forcibly, kept taking breaks, and finished two other games before I wrapped up Baldur's Gate. The six-year development of the franchise, born in the 90s, culminated in a significant and well-deserved success for the studio. The increased attention brought hordes of new fans to Larian Studios, discussing a genre they were unfamiliar with just yesterday as if it were a 21st-century revelation. Someone could tell that Baldur's Gate 3 is the best game in history. While the game is one of the year's standout releases and opens up a new genre for many players, an extra year or two in alpha development could have made it even better.

Digging Into The Game Design of Baldur's Gate 3. 

Recent advancements in led to the emergence of AI Agents. In plain terms, an agent is a mechanism that equips LLMs with a set of external tools (functions). These tools can be invoked by the model (once or multiple times in a row) based on the user’s input. Large Language Models Originally, in order to integrate a function into an LLM, it was needed to carefully construct the prompt in order to describe the function. Additionally, the output had to be parsed to detect whether the model intended to use this function. The latest release of GPT introduced function-calling capabilities that simplify the process of integrating the functions. First of all, it defines a clear and fixed structure for describing the functions. Moreover, the models themselves have been further fine-tuned to both detect when a function should be called (based on the input) and to respond with that adheres to the function signature. However, despite these improvements, the integration process still remains tedious and requires a lot of overhead. JSON In this article, we will explore how to build a simple conversational AI agent using the Agent Dingo library. The key feature of Dingo is that it allows the integration of external functions into ChatGPT by adding just a single line of code. In addition, it serves the agent with the required capabilities on an OpenAI-compatible web server. Quick Start To give a quick glimpse into the core functionality of Dingo, we will build a simple agent that is able to provide information about the current weather by querying the API. OpenWeather As the first step, the library has to be installed. pip install agent-dingo Next, we need to set the OpenAI API key. export OPENAI_API_KEY=<YOUR_KEY> Then, our Dingo agent can be instantiated as follows: from agent_dingo import AgentDingo

agent = AgentDingo() Let’s say that for getting the current weather via OpenWeather API we have a function. In order to integrate it into ChatGPT we need to add — the decorator . get_temperature a single line of code @agent.function @agent.function
def get_temperature(city: str) -> str:
    base_url = "https://api.openweathermap.org/data/2.5/weather"
    params = {"q": city, "appid": openweathermap_api_key, "units": "metric"}
    response = requests.get(base_url, params=params)
    data = response.json()
    print("get_temperature function output :", data["main"])
    return str(data) To get the output we need to call the method. chat agent.chat("What is the current weather in Linz?") As you can see the process of function integration is very intuitive and simple. Behind the scenes, we are asking ChatGPT for the current weather in Linz. The model has access to the function ( ) that was integrated by using decorator. The model decides to call function and passes city Linz as an argument. The response is parsed and we receive the following output. get_temperature agent.function get_temperature The current temperature in Linz is 30°C. As you might have noticed, we did not have to provide any descriptions of the function. This is achieved by the automatic docstring generation during the registration process (when decorator is called). agent.function In addition, it is possible to serve the agent on an OpenAI-compatible web server: from agent_dingo.wrapper import DingoWrapper
DingoWrapper(agent).serve() The server can be accessed using the python package: openai import openai

openai.api_base = "http://localhost:8080"

r = openai.ChatCompletion.create(
    model = "gpt-3.5-turbo",
    messages = [{"role": "user", "content": "What is the current weather in Linz?"}],
    temperature=0.0,
) Conclusion In this article, we explored how streamlines the process of integrating Python functions into ChatGPT in order to build a conversational AI agent. With Dingo, you no longer have to worry about manually integrating the functions or dealing with intermediate function calls. The framework is crafted to automate these tasks, allowing you to focus on writing the core functionality of your app. Agent Dingo Also published here.

Dingo: A Microframework for Building Conversational AI Agents

DINGO'S SAGACITY

The Island of Doctor Moreau, by H. G. Wells, is part of the HackerNoon Books Series. You can jump to any chapter in this book here . I. IN THE DINGEY OF THE “LADY VAIN.” I. IN THE DINGEY OF THE “LADY VAIN.” I do not propose to add anything to what has already been written concerning the loss of the Lady Vain. As everyone knows, she collided with a derelict when ten days out from Callao. The longboat, with seven of the crew, was picked up eighteen days after by H. M. gunboat Myrtle, and the story of their terrible privations has become quite as well known as the far more horrible Medusa case. But I have to add to the published story of the Lady Vain another, possibly as horrible and far stranger. It has hitherto been supposed that the four men who were in the dingey perished, but this is incorrect. I have the best of evidence for this assertion: I was one of the four men. But in the first place I must state that there never were four men in the dingey,—the number was three. Constans, who was “seen by the captain to jump into the gig,” luckily for us and unluckily for himself did not reach us. He came down out of the tangle of ropes under the stays of the smashed bowsprit, some small rope caught his heel as he let go, and he hung for a moment head downward, and then fell and struck a block or spar floating in the water. We pulled towards him, but he never came up. Daily News, March 17, 1887. I say luckily for us he did not reach us, and I might almost say luckily for himself; for we had only a small beaker of water and some soddened ship’s biscuits with us, so sudden had been the alarm, so unprepared the ship for any disaster. We thought the people on the launch would be better provisioned (though it seems they were not), and we tried to hail them. They could not have heard us, and the next morning when the drizzle cleared,—which was not until past midday,—we could see nothing of them. We could not stand up to look about us, because of the pitching of the boat. The two other men who had escaped so far with me were a man named Helmar, a passenger like myself, and a seaman whose name I don’t know,—a short sturdy man, with a stammer. We drifted famishing, and, after our water had come to an end, tormented by an intolerable thirst, for eight days altogether. After the second day the sea subsided slowly to a glassy calm. It is quite impossible for the ordinary reader to imagine those eight days. He has not, luckily for himself, anything in his memory to imagine with. After the first day we said little to one another, and lay in our places in the boat and stared at the horizon, or watched, with eyes that grew larger and more haggard every day, the misery and weakness gaining upon our companions. The sun became pitiless. The water ended on the fourth day, and we were already thinking strange things and saying them with our eyes; but it was, I think, the sixth before Helmar gave voice to the thing we had all been thinking. I remember our voices were dry and thin, so that we bent towards one another and spared our words. I stood out against it with all my might, was rather for scuttling the boat and perishing together among the sharks that followed us; but when Helmar said that if his proposal was accepted we should have drink, the sailor came round to him. I would not draw lots however, and in the night the sailor whispered to Helmar again and again, and I sat in the bows with my clasp-knife in my hand, though I doubt if I had the stuff in me to fight; and in the morning I agreed to Helmar’s proposal, and we handed halfpence to find the odd man. The lot fell upon the sailor; but he was the strongest of us and would not abide by it, and attacked Helmar with his hands. They grappled together and almost stood up. I crawled along the boat to them, intending to help Helmar by grasping the sailor’s leg; but the sailor stumbled with the swaying of the boat, and the two fell upon the gunwale and rolled overboard together. They sank like stones. I remember laughing at that, and wondering why I laughed. The laugh caught me suddenly like a thing from without. I lay across one of the thwarts for I know not how long, thinking that if I had the strength I would drink sea-water and madden myself to die quickly. And even as I lay there I saw, with no more interest than if it had been a picture, a sail come up towards me over the sky-line. My mind must have been wandering, and yet I remember all that happened, quite distinctly. I remember how my head swayed with the seas, and the horizon with the sail above it danced up and down; but I also remember as distinctly that I had a persuasion that I was dead, and that I thought what a jest it was that they should come too late by such a little to catch me in my body. For an endless period, as it seemed to me, I lay with my head on the thwart watching the schooner (she was a little ship, schooner-rigged fore and aft) come up out of the sea. She kept tacking to and fro in a widening compass, for she was sailing dead into the wind. It never entered my head to attempt to attract attention, and I do not remember anything distinctly after the sight of her side until I found myself in a little cabin aft. There’s a dim half-memory of being lifted up to the gangway, and of a big round countenance covered with freckles and surrounded with red hair staring at me over the bulwarks. I also had a disconnected impression of a dark face, with extraordinary eyes, close to mine; but that I thought was a nightmare, until I met it again. I fancy I recollect some stuff being poured in between my teeth; and that is all. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. H. G. Wells (1994). The Island of Doctor Moreau. Urbana, Illinois: Project Gutenberg. Retrieved October 2022, from https://www.gutenberg.org/files/159/159-h/159-h.htm This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org , located at https://www.gutenberg.org/policy/license.html.

The Island of Doctor Moreau: I. IN THE DINGEY OF THE “LADY VAIN.”

I have always been a proud proponent of Telegram and what it promises in terms of functionality and user privacy but in light of recent events, I may be forced to rethink that position. On August 19, it was reported that Telegram had carried out an expansive sweep of its user base and revoked inactive usernames. Hundreds of public usernames were seized and disabled, without any prior notification to the owners.  Most of the affected channels either had little or no content or were inactive for over a year. But, that's not all that surprising. While there were innocent people caught in between, username squatters were mainly affected. I'm not against that. What rubbed me (and a ton of other people) the wrong way was what followed. Just a few days after this event, Pavel Durov, Telegram's founder, posted on : his channel I'm really impressed by the success of the auction TON recently conducted for their domain/wallet names. Wallet.ton was sold for 215,250 Toncoin (~$260000) while casino.ton was sold for ~$244000. If TON has been able to achieve these results, imagine how successful Telegram with its 700 million users could be if we put reserved @ usernames, group and channel links for auction. In addition to millions of catchy t.me addresses like @storm or @royal, all four-letter usernames could be made available for sale (@bank, @club, @game, @gift etc). This would create a new platform where username holders could transfer them to interested parties in protected deals – with ownership secured on the blockchain via NFT-like smart-contracts. Other elements of the Telegram ecosystem, including channels, stickers or emoji, could later also become part of this marketplace. When it comes to scalability and speed, TON probably has the best technology to host such decentralized sales. Our team can write bullet-proof smart contracts for TON (since it was us who invented its smart-contract language), so we are inclined to try out TON as the underlying blockchain for our future marketplace. Let's see if we can add a little bit of Web 3.0 to Telegram in the coming weeks. Durov's post would not have seemed out of the ordinary, if not for the extremely odd timing, coming after the widespread revocation of public usernames. As expected, people were not too delighted to find out their channel names were going to be auctioned after being taken away. The post was followed by an overwhelming backlash from Telegram users. It had about 30k dislikes, merely hours after the post. The negative response was so much that Durov turned off reactions in his channel. A previously suggested feature to introduce NFTs as profile pictures was also met with heavy backlash, at the time. It seems Telegram users are heavily opposed to Web 3.0 technologies and that may be for good reason. Also, we all thought the gains from would be enough. It remains to be seen whether Durov will be willing to go against his users’ wishes to make more profit. Regardless, it is clear that Telegram, in its current state, stands in contrast to what it stood for in the past and the values it claimed to uphold. Telegram Premium I truly believe this could signal Telegram's downfall if Durov doesn't begin to retrace his steps.

Is Telegram Digging Its Own Grave?

Building Scalable E-commerce Infrastructure on Magento

Ensuring Reproducibility in AI Research: Code and Pre-trained Weights Open-Sourced

Authors:
(1) Yuwei Guo, The Chinese University of Hong Kong;
(2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author;
(3) Anyi Rao, Stanford University;
(4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory;
(5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory;
(6) Yu Qiao, Shanghai Artificial Intelligence Laboratory;
(7) Maneesh Agrawala, Stanford University;
(8) Dahua Lin, Shanghai Artificial Intelligence Laboratory;
(9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Table of Links Abstract and 1 Introduction 2 Work Related 3 Preliminary AnimateDiff 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.2 Learn Motion Priors with Motion Module 4.3 Adapt to New Motion Patterns with MotionLora 4.4 AnimateDiff in Practice 5 Experiments and 5.1 Qualitative Results 5.2 Qualitative Comparison 5.3 Ablative Study 5.4 Controllable Generation 6 Conclusion 7 Ethics Statement 8 Reproducibility Statement, Acknowledgement and References8 Reproducibility Statement, Acknowledgement and References 2 RELATED WORK Text-to-image diffusion models. Diffusion models (Ho et al., 2020; Dhariwal & Nichol, 2021; Song et al., 2020) for text-to-image (T2I) generation (Gu et al., 2022; Mokady et al., 2023; Podell et al., 2023; Ding et al., 2021; Zhou et al., 2022b; Ramesh et al., 2021; Li et al., 2022) have gained significant attention in both academic and non-academic communities recently. GLIDE (Nichol et al., 2021) introduced text conditions and demonstrated that incorporating classifier guidance leads to more pleasing results. DALL-E2 (Ramesh et al., 2022) improves text-image alignment by leveraging the CLIP (Radford et al., 2021) joint feature space. Imagen (Saharia et al., 2022) incorporates a large language model (Raffel et al., 2020) and a cascade architecture to achieve photorealistic results. Latent Diffusion Model (Rombach et al., 2022), also known as Stable Diffusion, moves the diffusion process to the latent space of an auto-encoder to enhance efficiency. eDiff-I (Balaji et al., 2022) employs an ensemble of diffusion models specialized for different generation stages. Personalizing T2I models. To facilitate the creation with pre-trained T2Is, many works focus on efficient model personalization (Shi et al., 2023; Lu et al., 2023; Dong et al., 2022; Kumari et al., 2023), i.e., introducing concepts or styles to the base T2I using reference images. The most straightforward approach to achieve this is complete fine-tuning of the model. Despite its potential to significantly enhance overall quality, this practice can lead to catastrophic forgetting (Kirkpatrick et al., 2017; French, 1999) when the reference image set is small. Instead, DreamBooth (Ruiz et al., 2023) fine-tunes the entire network with preservation loss and uses only a few images. Textual Inversion (Gal et al., 2022) optimize a token embedding for each new concept. Low-Rank Adaptation (LoRA) (Hu et al., 2021) facilitates the above fine-tuning process by introducing additional LoRA layers to the base T2I and optimizing only the weight residuals. There are also encoder-based approaches that address the personalization problem (Gal et al., 2023; Jia et al., 2023). In our work, we focus on tuning-based methods, including overall fine-tuning, DreamBooth (Ruiz et al., 2023), and LoRA (Hu et al., 2021), as they preserve the original feature space of the base T2I. Animating personalized T2Is. There are not many existing works regarding animating personalized T2Is. Text2Cinemagraph (Mahapatra et al., 2023) proposed to generate cinematography via flow prediction. In the field of video generation, it is common to extend a pre-trained T2I with temporal structures. Existing works (Esser et al., 2023; Zhou et al., 2022a; Singer et al., 2022; Ho et al., 2022b,a; Ruan et al., 2023; Luo et al., 2023; Yin et al., 2023b,a; Wang et al., 2023b; Hong et al., 2022; Luo et al., 2023) mostly update all parameters and modify the feature space of the original T2I and is not compatible with personalized ones. Align-Your-Latents (Blattmann et al., 2023) shows that the frozen image layers in a general video generator can be personalized. Recently, some video generation approaches have shown promising results in animating a personalized T2I model. Tune-a-Video (Wu et al., 2023) fine-tune a small number of parameters on a single video. Text2Video-Zero (Khachatryan et al., 2023) introduces a training-free method to animate a pre-trained T2I via latent wrapping based on a pre-defined affine matrix. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Yuwei Guo, The Chinese University of Hong Kong; (2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author; (3) Anyi Rao, Stanford University; (4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory; (5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory; (6) Yu Qiao, Shanghai Artificial Intelligence Laboratory; (7) Maneesh Agrawala, Stanford University; (8) Dahua Lin, Shanghai Artificial Intelligence Laboratory; (9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Authors: Authors: (1) Yuwei Guo, The Chinese University of Hong Kong; (2) Ceyuan Yang, Shanghai Artificial Intelligence Laboratory with Corresponding Author; (3) Anyi Rao, Stanford University; (4) Zhengyang Liang, Shanghai Artificial Intelligence Laboratory; (5) Yaohui Wang, Shanghai Artificial Intelligence Laboratory; (6) Yu Qiao, Shanghai Artificial Intelligence Laboratory; (7) Maneesh Agrawala, Stanford University; (8) Dahua Lin, Shanghai Artificial Intelligence Laboratory; (9) Bo Dai, The Chinese University of Hong Kong and The Chinese University of Hong Kong. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Work Related 2 Work Related 3 Preliminary 3 Preliminary AnimateDiff AnimateDiff 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.1 Alleviate Negative Effects from Training Data with Domain Adapter 4.2 Learn Motion Priors with Motion Module 4.2 Learn Motion Priors with Motion Module 4.3 Adapt to New Motion Patterns with MotionLora 4.3 Adapt to New Motion Patterns with MotionLora 4.4 AnimateDiff in Practice 4.4 AnimateDiff in Practice 5 Experiments and 5.1 Qualitative Results 5 Experiments and 5.1 Qualitative Results 5.2 Qualitative Comparison 5.2 Qualitative Comparison 5.3 Ablative Study 5.3 Ablative Study 5.4 Controllable Generation 5.4 Controllable Generation 6 Conclusion 6 Conclusion 7 Ethics Statement 7 Ethics Statement 8 Reproducibility Statement, Acknowledgement and References8 Reproducibility Statement, Acknowledgement and References 8 Reproducibility Statement, Acknowledgement and References8 Reproducibility Statement, Acknowledgement and References 2 RELATED WORK Text-to-image diffusion models. Diffusion models (Ho et al., 2020; Dhariwal & Nichol, 2021; Song et al., 2020) for text-to-image (T2I) generation (Gu et al., 2022; Mokady et al., 2023; Podell et al., 2023; Ding et al., 2021; Zhou et al., 2022b; Ramesh et al., 2021; Li et al., 2022) have gained significant attention in both academic and non-academic communities recently. GLIDE (Nichol et al., 2021) introduced text conditions and demonstrated that incorporating classifier guidance leads to more pleasing results. DALL-E2 (Ramesh et al., 2022) improves text-image alignment by leveraging the CLIP (Radford et al., 2021) joint feature space. Imagen (Saharia et al., 2022) incorporates a large language model (Raffel et al., 2020) and a cascade architecture to achieve photorealistic results. Latent Diffusion Model (Rombach et al., 2022), also known as Stable Diffusion, moves the diffusion process to the latent space of an auto-encoder to enhance efficiency. eDiff-I (Balaji et al., 2022) employs an ensemble of diffusion models specialized for different generation stages. Text-to-image diffusion models. Personalizing T2I models. To facilitate the creation with pre-trained T2Is, many works focus on efficient model personalization (Shi et al., 2023; Lu et al., 2023; Dong et al., 2022; Kumari et al., 2023), i.e., introducing concepts or styles to the base T2I using reference images. The most straightforward approach to achieve this is complete fine-tuning of the model. Despite its potential to significantly enhance overall quality, this practice can lead to catastrophic forgetting (Kirkpatrick et al., 2017; French, 1999) when the reference image set is small. Instead, DreamBooth (Ruiz et al., 2023) fine-tunes the entire network with preservation loss and uses only a few images. Textual Inversion (Gal et al., 2022) optimize a token embedding for each new concept. Low-Rank Adaptation (LoRA) (Hu et al., 2021) facilitates the above fine-tuning process by introducing additional LoRA layers to the base T2I and optimizing only the weight residuals. There are also encoder-based approaches that address the personalization problem (Gal et al., 2023; Jia et al., 2023). In our work, we focus on tuning-based methods, including overall fine-tuning, DreamBooth (Ruiz et al., 2023), and LoRA (Hu et al., 2021), as they preserve the original feature space of the base T2I. Personalizing T2I models. Animating personalized T2Is. There are not many existing works regarding animating personalized T2Is. Text2Cinemagraph (Mahapatra et al., 2023) proposed to generate cinematography via flow prediction. In the field of video generation, it is common to extend a pre-trained T2I with temporal structures. Existing works (Esser et al., 2023; Zhou et al., 2022a; Singer et al., 2022; Ho et al., 2022b,a; Ruan et al., 2023; Luo et al., 2023; Yin et al., 2023b,a; Wang et al., 2023b; Hong et al., 2022; Luo et al., 2023) mostly update all parameters and modify the feature space of the original T2I and is not compatible with personalized ones. Align-Your-Latents (Blattmann et al., 2023) shows that the frozen image layers in a general video generator can be personalized. Recently, some video generation approaches have shown promising results in animating a personalized T2I model. Tune-a-Video (Wu et al., 2023) fine-tune a small number of parameters on a single video. Text2Video-Zero (Khachatryan et al., 2023) introduces a training-free method to animate a pre-trained T2I via latent wrapping based on a pre-defined affine matrix. Animating personalized T2Is. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Text-to-Image Diffusion Models and Personalized Animation Techniques

Uni-OVSeg: A Step Towards Efficient and Bias-Resilient Vision Systems

Polymarket Explained: How Blockchain Prediction Markets Are Shaping the Future of Forecasting

Polymarket обяснява: Как пазарите за прогнозиране на блокчейн оформят бъдещето на прогнозирането

Polymarket vysvětlil: Jak trhy s predikcí blockchainu utvářejí budoucnost prognózování

Polymarket explicado: cómo los mercados de predicción de blockchain están dando forma al futuro de la previsión

Polymarket توضیح داد: چگونه بازارهای پیش بینی بلاک چین آینده پیش بینی را شکل می دهند

Պոլիմարկետը բացատրեց. ինչպես են բլոկչեյն կանխատեսման շուկաները ձևավորում կանխատեսումների ապագան

פולימרקט הסביר: כיצד שווקי חיזוי בלוקצ'יין מעצבים את עתיד החיזוי

ポリマーケットの解説: ブロックチェーン予測市場が予測の未来をどう形作るのか

Полимаркет тайлбарлав: Блокчейн таамаглалын зах зээл урьдчилан таамаглах ирээдүйг хэрхэн бүрдүүлж байна

Polymarket Explained: Cum piețele de predicție Blockchain modelează viitorul prognozei

Polymarket ayaa Sharaxay: Sida Blockchain Suuqyada Saadaasha ay u qaabeynayaan Mustaqbalka Saadaasha

Полимаркет објаснио: како тржишта предвиђања блокчејна обликују будућност предвиђања

Полимаркет фаҳмонд: Чӣ гуна бозорҳои пешгӯии блокчейн ояндаи пешгӯиро ташаккул медиҳанд

Stories, News & Mentions about DINGG

Dingg’s Stories on HackerNoon