ElevenLabsis in the AI Infrastructure Industry

ElevenLabs

elevenlabs

Do you know what generative voices are capable of? What vast possibilities does they open up, and how many areas can benefit from them? Maybe you need an app for voicing a video blog or book. Perhaps you want to bring a game character to life with a professional voiceover. How about creating an application for learning foreign languages? Today, has prepared a tutorial for you that will help you get to know AI voice technologies better! lablab.ai Let’s dive in! Introduction It  is one of the most exciting times for software development, what with the emergence of various " " tools in the market. Just name it, cover letter generation? Check! E-mail generation? Check! Automatic code comment generation? Check! Even outside coding and software development, the use case possibilities are enormous. generative AI Now, we can generate images with text prompts with various image generation models. Thus, it makes it possible for us to incorporate generated assets in our various products. The next question is: how about voices? The trend of user experiences in the past few years mentioned "voice command" as one of the emerging trend. It is only natural that the software we build will incorporate voices as one of the features. Which is why, in this tutorial, we will showcase the " " feature offered by in a simple app, which generates random words and have it spell it. To build the UI for this , we will use , a new UI library to share data science projects. Speech Synthesis ElevenLabs Python-based app Streamlit Introduction to ElevenLabs is a voice technology research company which offers solution. With easy to use , it allows developers to generate high-quality speeches using AI. It is made possible by the AI model which has been trained on a vast amount of audiobooks and also podcasts. The training allows the AI to deliver predictable and high-quality results in speech generation. ElevenLabs speech synthesis API There are two main features that has to offer, the first one is VoiceLab, where users can clone voices from recorded audio and/or existing pre-made voices, and also "design" voices based on gender, ages, ethnicities and races. Once users have the voices to work with, they can move on to the next feature, Speech Synthesis, where they can generate speeches using their designed voices or just using the pre-made ones. ElevenLabs Introduction to Model Anthropic's Claude is the latest AI model developed by , an AI research organization which focuses on improving the interoperability, robustness and safety of artificial intelligence systems. Claude Anthropic The model is designed to generate human-like responses, making it a powerful tool for a wide range of applications, from content creation, legal, to customer service. Just like any other in the market, is also trained on a diverse range of internet text. However, unlike most AI models, it has focus on "safety", which makes it possible to refuse outputs that it considers "harmful" or "untruthful" for the users. Claude AI models Claude Introduction to Streamlit is an open-source library that makes it easy and possible for developers and data scientists to create and share visually appealing and customized web apps. Developers can use to build and deploy fully featured data science apps in minutes. It is made possible by the simple and intuitive API that can be used to turn data scripts into UI components. Streamlit Python Streamlit Prerequisites Basic knowledge of and using Python UI development Streamlit Access to Anthropic API Access to ElevenLabs API Outline Initializing our Streamlit Project Adding Word Generation Feature using Model Claude Adding Speech Generation Feature using ElevenLabs API Testing the Word Generator App Discussion There are at least four steps that we will get through in this tutorial. First we need to initialize the Streamlit-based project, to get a general feel of developing user interfaces using Streamlit. Next, we start adding more features, beginning with engineering prompt to get model to give us a randomized word that is commonly misspelled. After that, we'll add text-to-voice generation provided by to demonstrate how the multilingual model spell the words. Finally, we're going to test the simple app. Claude ElevenLabs Initializing our Streamlit Project Let's get into the coding action! First, let's make a directory for our project and enter it! mkdir randomwords
cd randomwords Next, we're going to use this directory as the basis of our project. Because a project is essentially a project, we need to do some steps to initialize our project, such as defining and activating our virtual environment. Streamlit Streamlit Python Python # Creating the virtual environment
python -m venv env

# Activate the virtual environment
# On Linux/Mac
source env/bin/activate

# On Windows:
.\env\Scripts\activate Once activated, the output of our terminal should show the name of the virtual environment (env), like so: Next, it's time to install the libraries we need for this project! let's use the command to install the , , and library. Note that we also install a version-locked library to prevent a Pydantic-related error in one of the function. pip streamlit anthropic elevenlabs pydantic elevenlabs pip install streamlit anthropic elevenlabs "pydantic==1.*" With all the project's requirements out of the way, now let's dive into the coding part! Create a new file inside our project directory, let's call it . randomwords_app.py touch randomwords_app.py After the file is created, let's open the file in our favorite code editor or integrated development environment (IDE). For the starter, let's build our simple app from the simplest parts possible, a title and a text for the caption! import streamlit as st

st.title("Random Words Generator")

st.text("Hello, this is a random words generator app") To wrap up our project initialization part, let's try test running the app. Make sure that our current working directory is still inside our project and our virtual environment is already activated. When everything is ready, use the to run the app. streamlit run <app-name> streamlit run randomwords_app.py The app should open automatically in our default browsers! it should show the title and text for now. Next, we're going to add random word generation feature using model. Anthropic's Claude One last thing though, we'll have to provide our app with the API keys for the services that we're going to use, namely model and feature. As these keys are considered sensitive, we should keep them in a safe and isolated place. Anthropic's Claude ElevenLabs' Speech Synthesis However, this time we don't store them in a file. This is because Streamlit deal with environment variables differently. According to the , we need to create a secret configuration file inside a directory. We can create the directory inside our project and then create the file. .env documentation .streamlit mkdir .streamlit
touch .streamlit/secrets.toml Let's edit the TOML file we created, note that TOML file uses different formatting from the usual file. .env xi_api_key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
claude_key = "sk-ant-xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" Adding Word Generation Feature using Claude Model In this step, we will add a button that will generate the random word, the heading element to show the generated word and the subheading to show the meaning of the word. However, coming from a webdev background, I strongly believe that UI elements should be placed and arranged inside containers. So, we'll do exactly that. Import the necessary libraries First of all, let's add some import statements. We're going to import the library to generate our random words. anthropic import streamlit as st
import anthropic Then, before we get to the UI part, let's create our word generation function first. Defining the word generation function def generate_word():
    prompt = (f"{anthropic.HUMAN_PROMPT} Give me one non-English word that's commonly misspelled and the meaning. Please strictly follow the format! example: Word: Schadenfreude; Meaning: joy at other's expenses."
              f"{anthropic.AI_PROMPT} Word: Karaoke; Meaning: a form of entertainment where people sing popular songs over pre-recorded backing tracks."
              f"{anthropic.HUMAN_PROMPT} Great! just like that. Remember, only respond following the pattern.")

    c = anthropic.Anthropic(api_key=st.secrets["claude_key"])
    resp = c.completions.create(
        prompt=f"{prompt} {anthropic.AI_PROMPT}",
        stop_sequences=[anthropic.HUMAN_PROMPT],
        model="claude-v1.3-100k",
        max_tokens_to_sample=900,
    )

    print(resp.completion)
    return resp.completion In this function, the most heavy lifting is done by ** **l (Thanks, ! 😉). However, our part in this function is how to make Claude return the exact format consistently. So we need to both instruct to "strictly follow the format" and give it an example response by adding it after our initial prompt. Anthropic's Claude mode Claude Claude Finally, we make sure that Claude comply with our agreements by ask it to "Remember to only respond following the pattern". The function ends by returning the response from . Claude Next, let's get back to editing the UI! Updating the UI st.title("Random Words Generator")

with st.container():
    st.header("Random Word")
    random_word = st.subheader("-")
    word_meaning = st.text("Meaning: -")

    st.write("Click the `Generate` button to generate new word")
    if st.button("Generate"):
        result = generate_word()
        # Split the string on the semicolon
        split_string = result.split(";")

        # Split the first part on ": " to get the word
        word = split_string[0].split(": ")[1]

        # Split the second part on ": " to get the meaning
        meaning = split_string[1].split(": ")[1]

        print(f"word result: {word}")
        random_word.subheader(word)
        word_meaning.text(f"Meaning: {meaning}") This time, we added a container with some elements inside it. The header, subheader for displaying the random word, and the text element to show the meaning of the word. We also have a text to show the hint on how to use the app, as well as a button beneath it. In , we can declare click event handler by using a conditional statement, where it returns when the button is clicked. In this code, we invoke the function which returns the generated word and the meaning, and split the result into separate variables for the word and the meaning, respectively. Finally, we update the subheader and the text element to display the word and the meaning. Streamlit True generate_word() Final form Let's double check our code once again! It should contains the import statements, the function for generating the random word, and the updated UI which contains subheader, and text elements as well as button that generate the word by invoking the function. generate_word() import streamlit as st
import anthropic

def generate_word():
    prompt = (f"{anthropic.HUMAN_PROMPT} Give me one non-English word that's commonly misspelled and the meaning. Please strictly follow the format! example: Word: Schadenfreude; Meaning: joy at other's expenses."
              f"{anthropic.AI_PROMPT} Word: Karaoke; Meaning: a form of entertainment where people sing popular songs over pre-recorded backing tracks."
              f"{anthropic.HUMAN_PROMPT} Great! just like that. Remember, only respond following the pattern.")

    c = anthropic.Anthropic(api_key=st.secrets["claude_key"])
    resp = c.completions.create(
        prompt=f"{prompt} {anthropic.AI_PROMPT}",
        stop_sequences=[anthropic.HUMAN_PROMPT],
        model="claude-v1.3-100k",
        max_tokens_to_sample=900,
    )

    print(resp.completion)
    return resp.completion


st.title("Random Words Generator")

with st.container():
    st.header("Random Word")
    random_word = st.subheader("-")
    word_meaning = st.text("Meaning: -")

    st.write("Click the `Generate` button to generate new word")
    if st.button("Generate"):
        result = generate_word()
        # Split the string on the semicolon
        split_string = result.split(";")

        # Split the first part on ": " to get the word
        word = split_string[0].split(": ")[1]

        # Split the second part on ": " to get the meaning
        meaning = split_string[1].split(": ")[1]

        print(f"word result: {word}")
        random_word.subheader(word)
        word_meaning.text(f"Meaning: {meaning}") Testing the Word Generation Function Let's run the app once again with the same command. We can also just rerun the app by clicking the upper right menu and click " " if we've had the app running before. Rerun It should show this updated user interface. Now, let's try clicking the button! Generate One of the sweet things about Streamlit is that it handled loading and provided the loading indicator out of the box. We should see the indicator in the upper-right corner, as well as the option to " " the operation. Neat, huh? stop After a few seconds, the result should be showed in the UI. Perfect! notice that the app correctly split the generated text from the model into word and the meaning. However, if the result doesn't come out according to the expected format, we can always click the button again. Claude Generate The next step is the main feature of this app, to incorporate speech generation into our random word generator. Besides demonstrating how to generate the audio file using the API provided by ElevenLabs, this step also serve to demonstrate the capabilities of ElevenLabs' multilingual model. Adding Speech Generation Feature using API ElevenLabs The first step of this section is, as you've probably guessed, is to add more import statement! So, let's add some functions from that we'll use for the speech generation feature. elevenlabs import streamlit as st
import anthropic
++ from elevenlabs import generate, set_api_key Next, we're going to define the function to handle the speech generation. def generate_speech(word):
    set_api_key(st.secrets['xi_api_key'])
    audio = generate(
        text=word,
        voice="Bella",
        model='eleven_multilingual_v1'
    )

    return audio Thanks to the simplicity and readability of , and also easy-to-use API, we can generate the speech by using this code alone! The function accepts the random word which we use to generate the speech. We also specifically use "eleven_multilingual_v1" model which is a multilingual model, perfect for our use case to demonstrate the spelling and pronounciation of foreign and commonly misspelled words! Finally, we use the " " voice for this tutorial, which is one of the pre-made voice provided by . Python ElevenLabs Bella ElevenLabs Next, we'll add an audio player to play the generated speech. print(f"word result: {word}")
    random_word.subheader(word)
    word_meaning.text(f"Meaning: {meaning}")
++     speech = generate_speech(word)
++     st.audio(speech, format='audio/mpeg') Just below our latest code from earlier, we add the variable to store the generated speech, and run the speech using audio player provided by function from Streamlit. At this point, our app should work as expected, only showing the audio player when there is a random word available to "read". st.audio Curious how it works out? me too! let's test the app now! Testing the Word Spelling Feature Let's run the app again using or just rerun the app if we have it running already. It should look exactly the same as the last time we left it. However, let's try to click the "Generate" button this time! streamlit run Amazing! this time, the app also shows an audio player! Let's try playing it. Using the multilingual model, the speech generated should use the accent and intonation which is close to the origin language of the word. For example, "entrepreneur" should be pronounced in French accent. Conclusion In this short tutorial, hopefully we've explored the capabilities of speech generation technology offered by . With the multilingual model, it's easy to generate speeches that is intended for non-English listener. In our use case, we need multilingual model to aid us in finding the correct way to pronounce and spell non-English words that are commonly misspelled. ElevenLabs With so many ideas, we invite developers to join us in creating the future! , in where you can create your own voice AI app with models! (Additionally, you can leverage other AI models such as large language models, image and video generative models, etc., as long as they are not in direct competition with the hackathon technology). On July 28 lablab.ai launching a challenge ElevenLabs *Your final submission should consist of a ready-to-play working prototype of your idea, a video pitch, and a presentation showcasing your solution. You can find more tutorials and you can other hackathons to build with cutting-edge technologies! HERE JOIN And big thanks to the - the Author of this article. 💚 Septian Adi Nugraha

Building a Simple Word Spelling App with ElevenLabs, Streamlit, and Claude

ElevenLabs, Streamlit, এবং Claude সহ একটি সহজ শব্দ বানান অ্যাপ তৈরি করা

Erstellen einer einfachen Wortbuchstabier-App mit ElevenLabs, Streamlit und Claude

Creación de una aplicación de ortografía de palabras simple con ElevenLabs, Streamlit y Claude

Créer une application d'orthographe simple avec ElevenLabs, Streamlit et Claude

इलेवनलैब्स, स्ट्रीमलिट और क्लाउड के साथ एक सरल शब्द वर्तनी ऐप बनाना

イレブンラボ、Streamlit、Claude を使用してシンプルな単語スペル アプリを構築する

ElevenLabs, Streamlit 및 Claude를 사용하여 간단한 단어 맞춤법 앱 구축

Construindo um aplicativo simples de ortografia de palavras com ElevenLabs, Streamlit e Claude

ElevenLabs, Streamlit ve Claude ile Basit Bir Kelime Yazımı Uygulaması Oluşturma

Xây dựng ứng dụng đánh vần từ đơn giản với ElevenLabs, Streamlit và Claude

使用 ElevenLabs、Streamlit 和 Claude 构建简单的单词拼写应用程序

In this article, we examine the potential disruption of the radio host/live-stream industry through the use of artificial intelligence technology. By analyzing the average radio host salary, the number of spots per hour, the total words spoken per hour, and other factors such as music licensing fees, we demonstrate that it is possible to create a fully automated radio station using AI technology at a fraction of the cost of hiring human talent. We showcase a live demonstration of one such solution currently active under Avalon Star Streams brand, where after running songs from a creative commons music stream, the AI generates new radio content during breaks between those songs, even allowing the random selection of new songs if desired. Finally, we provide details on the tech stack utilized including NodeJS docker image which allows us to control ffmpeg streams and manage playlists efficiently. Running Example At the time of this writing, you can find the example live-streaming at and . Twitch.TV YouTube If for any reason the demo is not live streaming feel free to checkout this YouTube video for an example. https://youtu.be/Jnb4bMJc7x8?embedable=true the YouTube video show cases 2 auto-generated scripts with the AI Voice Actor Antoni Starr. The first is a call for donations, and the second is a random ad play with announcing the next song. NOTE: Cost analysis The key driver behind the potential disruption of the radio host industry lies in the significant reduction in labor costs. Utilizing current pricing models, the yearly cost of AI voice services required to run a full radio station amounts only to approximately $4100 compared to a national average salary of radio hosts estimated around $42k [0]. With an average radio spot duration of 16 minutes and an average speaking rate of 140 words per minute [1][2], each radio spot consists of approximately 2,240 words. AI-powered content generation, such as ChatGPT, comes at a cost of around $0.002 per 750 words. Thus, the total cost of using GPT for one hour of content amounts to approximately $0.006. With radio hosts spending around 45.5 hours per month on air [1], the yearly cost of GPT usage is estimated to be approximately $55. Furthermore, integrating Eleven Labs’ AI voice technology, priced at $330 per month for 40 hours of usage [3], incurs an annual cost of around $4,000. Considering the combined expenses of GPT and Eleven Labs, the total cost of implementing AI for a radio host is approximately $4,100 per year. This represents a significant cost reduction compared to traditional production methods and opens up new possibilities for radio/live-stream hosts with limited budgets. Further considerations While some might argue that AI generated content lacks emotional depth and personal touch provided by human talent, recent advancements in natural language processing have shown otherwise. With deep learning algorithms, AI systems can now analyze vast troves of linguistic data and learn nuances in context, tone and cadence of speech. When trained properly, these systems are able to mimic human-like qualities while still maintaining accuracy and efficiency. In fact, many industries ranging from customer service to journalism have already seen initial success with implementing chat-bots and machine-generated content due to economic advantages, even in sectors thought immune to technological takeover. Ultimately, it seems reasonable to assume a similar future for broadcasting markets like radio hosting. Examples Under our Avalon Star Stream brand, we set up a proof of concept showcasing the efficiency of AI assisted broadcasting. Leveraging open-source tools like ffmpeg and integrated within our custom NodeJS application framework managed through Docker, we were able to achieve a functioning automated live-stream setup complete with real-time generation capabilities for its intermission radio jockey. The system, under default settings, will play 3 songs before attempting a song-break. During the song-break our model analyzes prompts received online during the previous three musical sets and produces its own original written material targeting the donators thanking them, before reading an ad for an imaginary product and continuing. Our model is told take the persona of a Radio Host living within the by the name of “Antoni Starr”. Fallout 4 Universe Due to budget constraints, Antoni employs a cost-saving strategy. While generating content, there is a 10% chance, limited to once per hour, that his system dynamically pulls and generates a new song-break. This method adds an element of surprise and uniqueness to the show while optimizing production costs for the purpose of this tech demo. All other ad-reads will come from a previously generated grab-bag created during testing. Additionally, due to the channel being so new, we are unable to turn on the subscribers/memberships to be used during announcements. Tech Stack While I have not decided to release my code for this yet, I have decided to talk about the tech stack. As seen from the above image, the tool leveraging various technologies (FFmpeg, WebDAV, ChatGPT, EleventLabs, MongoDB) and the application combines them into a platform for live-stream generation. WebDAV + MongoDB This piece of the tech-stack is here to help record generated content and act as a file-store. The WebxDAV aspect allows us to remotely store the music files and download them at instantiation for the stream. ChatGPT + Eleven Labs These are the workhorses of the generative content. When time to generate a new ad-break we leverage ChatGPT API with our custom prompt to get the next script. Our prompt will be pre-seeded with name/information from stream donators and a random fake product to ad-read. FFmpeg The workhorse of streaming. FFmpeg is responsible for all audio/visual you see on the stream. from the static image overlay, to the encoded video playing on the TV and the audio you hear. FFmpeg is the magic behind it all. HTML/CSS/JavaScript Not seen in the diagram above, there is a management interface to tweak params of the running stream. This allows for the admin to force ad-breaks, tweak ad-break rate algorithm and more. Additionally, as mentioned previously, the entire platform is running off of NodeJS Conclusion We examined the possibility of replacing radio hosts with artificial intelligence and concluded that, given certain conditions, it may indeed be possible to do so. Our findings suggest that AI-powered radio stations would have clear financial advantages over their human counterparts and be capable of producing high-quality content equal to or surpassing that of human DJs. Further consideration should be done to ensure that listeners remain engaged and attuned to the program’s offerings amidst such developments. Overall, while the idea of a completely automated radio station or live-stream, with a dynamic voiced personality, may initially seem farfetched, the reality is that emerging technologies are quickly making the notion feasible and practical. Thus, business leaders must recognize the changing landscape and adapt accordingly before they risk being left behind in an ever evolving marketplace. Live-stream Links and Twitch.TV Youtube If you want to see more of tool itself please don’t hesitate to reach out.

The Radio Host and Live-Stream Industry: Poised for GPT Disruption

রেডিও হোস্ট এবং লাইভ-স্ট্রিম শিল্প: GPT ব্যাঘাতের জন্য প্রস্তুত

Die Radiomoderator- und Live-Stream-Branche: Bereit für GPT-Störungen

La industria de los presentadores de radio y la transmisión en vivo: preparados para la interrupción de GPT

L'industrie de l'hôte radio et de la diffusion en direct : prête pour la perturbation des GPT

रेडियो होस्ट और लाइव-स्ट्रीम उद्योग: GPT व्यवधान के लिए तैयार

ラジオホストとライブストリーム業界: GPT 破壊の準備が整っている

라디오 진행자와 라이브 스트리밍 산업: GPT 혼란에 대비

O apresentador de rádio e a indústria de transmissão ao vivo: preparados para a interrupção do GPT

Radyo Sunucusu ve Canlı Yayın Sektörü: GPT Kesintisine Hazır

Ngành công nghiệp máy chủ radio và phát trực tiếp: Đã sẵn sàng cho sự gián đoạn của GPT

电台主持人和直播行业：为 GPT 颠覆做好准备

For those of you who have been following me for a while, you know that I'm passionate about artificial intelligence (AI) and all the potential it holds. I've written several articles on it, ranging from to and . I'm constantly reading and experimenting in this domain. From small Python applications to help with article creation. ChatGPT Wordtune 174 AI Tools To Try Lately, I've noticed a lot of content about how to create an animated avatar using AI. So, I thought I'd give it a shot! Your Avatar To get your avatar made, use a drawing AI app like or DALL-E 2 Midjourney The image below was created with Midjourney: , you will require Discord. If you don't have an account, make one first. To register for Midjourney Then, click the Midjourney logo on the top left to access the 'newbie' channels. From there type ‘/imagine’ Proceed with the prompt. Here you can indicate what Midjourney should create for you. This is the prompt I used: Midjourney then asks you to accept the Terms of Service before giving you 4 images to choose from. You can edit those as well. I picked the one top-right. Pick the one that looks best to you and save it! Script for the Spoken Word I used ChatGPT to create a script. I entered a prompt and it delivered this text below. Now, this piece of text has to transform into a spoken word audio file. Spoken Word Fear not, as I've got the solution - audibly record my text? No way! I'm not a fan of hearing my own voice, so that's off the table. But don't worry, is here to save the day and make my life easier. ElevenLabs 333 or 10,000 If you don’t register with ElevenLabs, you will have 333 characters at your disposal to convert into speech. I registered, so I have 10,000 characters to convert into speech. As you can see, there's a bunch of voices to pick from. I chose the first one, and it read my text out loud. Copy what ChatGPT gave you into the text box at ElevenLabs and press that 'Generate' button. Once you do, you'll see a play icon on the bottom part of your screen. Just hit it, and you hear the text read out loud, and you can experiment with different voices! Have fun! If you're thrilled with the text and the accompanying vocalizations, click that download button! Then, there's just one more step to complete this tale - getting the animation. Create the Animation For this, I used . This application is very easy to use. D-ID In the left bar, click on ‘Create Video’ In the next screen, you will add your avatar, the one you created with Midjourney, as a Presenter. You do this by clicking ‘Add’ under the heading ‘Choose a presenter’ You choose the file you want to upload and you’re done with the presenter. We’ll keep on clicking. On the right side of your screen, you’ll see the option to upload Voice Audio. Click on ‘Upload your own voice’ and load the audio file you saved at the ElevenLabs step earlier. If that went well, you can click on ‘Generate Video’ at the top right. It takes some time to generate your video. Your Own Animated AI Avatar https://youtu.be/b6fIbN2Ao4I?embedable=true Sources (& Tips) How to Make an AI Animated Avatar A Complete Tutorial Create Your Own AI Animated Avatar: A Step-by-Step Guide Bring Your MidJourney AI Avatar To Life (AI Animation) How to Create Your Own AI Animated Avatar How to Create AI Avatar with AI: A Step-by-Step Guide Create Your Own AI Avatar in Under 5 Min Also Published Here

Your Own Animated AI Avatar

Using Free AI Tools to Create a 100% Automated Youtube Shorts Channel

In the recent past with the genesis of large language models, one thing that we critically debate around is whether like AI art, AI videos, etc will destroy the creativity of content creators or can enhance the quality and assist them. In my , I talked about how AI code-generation tools can help add value to the software development cycle, in this post I will mainly be highlighting how by using AI tools creating video content is not only getting easy and fast but also creative. AI-generated content previous blog Before understanding how one can leverage current video and image models in generating creative and engaging content, it is important to understand what the current state looks like, we can layer the current AI video generation landscape as below: into their workflow like Adobe, Canva, etc. Existing video editors trying to integrate AI like Fliki.ai, unscreen.com, synthesia.ai, hourone.ai, etc. AI-based new-age video editing tools dedicated to single use case in video generation workflow, eg helps in creating realistic images for videos, provides a platform to convert image to video or image to image, helps in adding animation to image, and so on. Abstraction layer Midjourney RunwayML Did which forms the base of the entire landscape, software teams can leverage this layer to customize for their use cases. Model Layer As we move above in layer flexibility to customize decreases while ease of use increases, for the context of this article we will be diving deep to understand how we can use the in generating creative videos since this layer falls in the middle of flexibility and ease of use, and as an artist one needs the best of both worlds. Abstraction layer Building text-to-video pipeline we learn how we can create a pipeline to generate using the tools in the abstraction layer it is important to chalk down what will be steps to generate the video and what tools can be used in each layer. Before text-to-video AI text-to-video generation involves the following steps: 1. Generating scenes from the script using GPT prompts The first step I did was to few-shot-prompt GPT to give out the Hindi script in the form of dialogues between the characters involved in every scene. Input Output 2. Generating images from the scenes This is the crucial step and involves creating the images for the scene which was broken down from the script, it all boils down to how creatively we can express ourselves using the prompt guidelines of , the example below mentions the prompt given to Midjourney to generate an image for a scene. Midjourney A cartoon of scene where Indian old hindu saint is asking for a help with Lord Cloud; Lord cloud is personified and have happiness on his face, the environment around is full of trees with dark clouds and lightening all around 3. Adding animation to the image In case you need to add animation to the image you can use or to add character motion and scene animation. DiD RunwayML 4. Generating AI voice for the scene narration In this step, you can generate the AI voice for the narration using eleven labs, generally, these are Text to speech narration models using behind which may sound a bit robotic but solve the purpose of generating voice, one can make it more expressive and realistic from eleven labs paid version, for this story I needed hindi voice narration for which narration does a great job. Ai4Bharat Text to speech 5. Stitching the video clips and syncing the voice This is the last and simplest step to add the images in a video editor and sync the voice as per scene and narration timeline, tools like Canva, and Adobe Express do a great job here. Rough Cost of Video Production Above is the simplest breakdown of how you can quickly generate video from text using a few basic tools, for my example, I generated an almost ~ 3-minute video with 16 unique scenes, interesting would be to see the time and money I paid to generate this video: Midjourney cost ~ $0.05/image - 16*0.05 = $0.8 RunwayML ~ $0.02/image - 16*0.02 = 0.32 Canva ~ Free of cost since not used their premium artifacts Total Cost ~ $1 /video Comparing it with the new-age AI video editors like which charges almost $28/month for 180 minutes of creation, which would for a video length mentioned above. Fliki cost ~ $0.5 Need to bundle the offering Although the final cost of generating the video in the AI-based video editors seems less compared to the total cost incurred by using tools like Midjourney, RunwayML, etc, with added cost these tools provide flexibility and creativity to a video content creator and can help in generating some amazing videos which can be comparable to an amazing scene of Hollywood movie, it seems that if these AI tools can be bundled and integrated with the workflow of video agency or video production houses they can produce maximum value in video production, as Justine Moore, Partner in thread also reflects the same. @a16z this

AI Stack for Text-to-Video Generation

টেক্সট-টু-ভিডিও জেনারেশনের জন্য এআই স্ট্যাক

KI-Stack für die Text-zu-Video-Generierung

Pila de IA para la generación de texto a vídeo

AI Stack pour la génération de texte en vidéo

टेक्स्ट-टू-वीडियो जेनरेशन के लिए एआई स्टैक

テキストからビデオへの生成のための AI スタック

텍스트-비디오 생성을 위한 AI 스택

AI Stack para geração de texto para vídeo

Стек AI для преобразования текста в видео

Metinden Videoya Üretim için AI Yığını

Ngăn xếp AI để tạo văn bản thành video

用于生成文本到视频的 AI 堆栈

Let's learn about via these 59 free stories. They are ordered by most time reading created on HackerNoon. Visit the to find the most read stories about any technology. Video /Learn Repo 1. How Cryptocurrency Can Help Deliver High-quality Video to Your Home The article is about how you can use a crypto project to deliver high-quality video to the end consumer faster and cheaper. 2. Ten Important Factors to Consider When Choosing a Streaming Service Things to consider when choosing a streaming service 3. Video Production Could Increase Your Company's Marketing Efforts Video is no longer just one part of your overall marketing plan. It's central to your outreach and campaign aims, particularly your social plan. 4. What is Adaptive Bitrate Streaming? A Brief Intro to ABR Adaptive bitrate streaming, or ABR, is the capability of a video player to adjust the video quality dynamically to match the viewer's network. 5. Beginners can make Money from Video Content Creation- Here's How Master how to make money with videos online with this practical guide for beginners! Monetize your video creation skills and earn extra income for your business 6. How to Use Secure Reliable Transport (SRT) for Your Live Video Secure Reliable Transport (SRT) is an open source video transport protocol that excels at providing stable streaming at relatively low latency. 7. "Do You Think You Know What it Takes to Tell The User It's Their Birthday?" 8. Top 5 Video Chat Apps of 2020 The world has been evolving ever since its inception. There is no doubt that Generation-Z (1997-2012) has witnessed many changes. These changes were related especially with the advancement of science and technology. The concept of video calling dates back to the year 1910, and back then it was termed as video telephony. So, the concept is pretty antiquated, but with time innovation played its part and kept it looking contemporary. Having said that, we’ll now look at the options we have when it comes to video calling. 9. How we Rendered 3 Years of Video Using JavaScript Lumen5's AI-powered technology helps you create video content in minutes. Here's how we rendered 3 years of video using JavaScript in just one year 10. Three Reasons Behind Ludicrous Spending in Streaming Media Netflix spent $16 billion in 2019 on producing and acquiring new content. Amazon pursued its originals journey with $6 billion. Quibi, a yet to launch streaming service raised a billion dollars to create a catalog for millennials. 11. How to Compete with Netflix: Beyond the Streaming Experience Streaming is all about creating a great viewing experience 12. How to Grow your Video Business with Data TV watching used to be a family affair a decade ago, but today in most households, content watching has become a personal activity. 13. Your Own Animated AI Avatar Using Midjourney I created an avatar. I had ChatGPT create a script. ElevenLabs for text into audio and with D-ID for the video 14. How to Implement a Simple Video Calling Feature Using the ZEGOCLOUD Video API This article describes what is the video API? How does the video API ensure smooth transmission of audio and video？ 15. An Underground Interview with John McAfee 13 Months Before his Death John McAfee talks about making more money than God on the Hacker Noon podcast hosted by David Smooke. 16. "cannot build multi-billion $ org unless you are clear on what instinct or organ you are targeting" 17. Next-Level Communication with WebRTC WebRTC has fast become the de facto framework used for building applications to facilitate audio and video communication over the web. 18. What Business Live Videos are People Actually Watching? Live video is changing the way people watch video as a whole and can be used to build your business. Eight out of every 10 people would watch video that is live rather than read a blog post or social media post. This is an important opportunity to showcase your business in a neglected medium for which there is already an established audience. Creating live video for your business isn’t difficult, and chances are you already have everything you need to get started. Why aren’t you live streaming yet? 19. Three Great Alternatives to YouTube for Businesses Ever since Google's YouTube platform took off in the mid-aughts, it has been the go-to destination for web video of all kinds. Today, it hosts entertainment content, instructional content, and no shortage of cat videos. It's also the preferred video platform of digital marketers around the world. 20. WTF is a Codec? Codec comprises an integral part of our lives like the water we drink. They are continuously in action all around us without us realizing it. 21. How to Download Videos from Reddit If you need to download Reddit videos for offline playback , you will need to visit the Viddit portal and get the links there. 22. 6 Ways Emotion AI is Changing the Healthcare Industry Pairing Emotion AI with Live Video calling can bring a dramatic improvement in healthcare delivery whether its doctor-patient communication or mental wellbeing. 23. A Chance for Emerging Video-Sharing Apps: What Happens If TikTok Is Banned in the US? So far, 800 million people worldwide use the TikTok video streaming app for entertainment and connection, and 12.5% of them, 100 million, are Americans. After President Donald Trump said on July 31 that he planned to bar the Chinese-owned social media app from operating in the US, he reversed course, saying he would allow TikTok to keep operating if it were sold to an American owner. Microsoft is one of the companies most likely to buy the app. The possible acquisition will allow the software company to compete with Facebook and Google-owned YouTube and attract a younger audience. 24. Zoom 5.0: Major 5 Security Updates You Need to Know From yoga classes to worldwide webinars, Zoom community has been expanding all around the world after the global lockdown due to COVID-19. 25. 8 Best YouTube Alternatives for Video Creators and Brands If you feel the need to switch and are looking for channels like YouTube, here is our list which is a blend of free and premium platforms 26. Video on Demand: Comparing the Monetization Models The infrastructure needed to stream videos for your website can be very expensive to build and maintain. These are three monetization strategies that get used. 27. WTF is Transcoding? Transcoding is the process of converting audio or video files from one encoding format to another in order to increase the number of compatible target devices. 28. How to set up a Podcast Studio for Live Streaming A summary of the video and audio setup I have honed over the past 18 months. 29. How Technology Shapes the Future of the UK Video Streaming Industry Technology appears to be developing more quickly every day. With innumerable movies, TV shows, and songs being streamed practically every second, streaming has become one of the most widely used digital entertainment technologies not only in the UK but worldwide. 30. 5 Easy Ways to Get Video Testimonials from Your Customers Customers' video testimonials help in creating brand trust, provide more leads & better higher customer retention rate. This post helps in getting testimonials. 31. "Before School, During School,  (and)  Couple Times Each Class" 32. How to Fix VLC Media Player Not Playing MP4 Videos? What Causes the VLC Player Not Playing Video Files Issue? Missing MP4 codecs, outdated version of Windows OS or VLC player, or the MP4 file may be corrupted. 33. How to Resize a Video in 5 Easy Steps Resizing video is changing the dimensions or video clip frame size to fit within certain parameters like upload limit, output size, and other constraints. 34. How to Implement Sound Waves in iOS by Using ZEGOCLOUD SDK This article explains how to achieve the sound wave effect in live broadcast, voice chat, call and other scenarios 35. Why Can't Live Video Be Better? A look at the emerging trends of live video, and how it can look to become improved in today's market. 36. Reasons To Use Custom VOD Platform A growing number of companies are opting for a custom VOD to distribute their video content on their own terms. Should your company make the switch too? 37. How To Nail Video Email Marketing With Seasonal Email Campaigns Ah, so you've decided to use video marketing. A wise choice indeed, seeing as, according to Google, "6 out of 10 people prefer online video platforms to live TV". 38. How To Create An Online TV Channel For Your TV Station? Learn how to create an online TV channel right from scratch and essential things need to start an online tv broadcasting platform. 39. 5 Reasons Why Video Content Is Imperative for Your Website’s Success With the future of video content looking clear, organizations are reimagination reach as it relates to consumer attention. In this world, Video is king. 40. The Best Video Game Music of 2022 A great soundtrack is not only memorable but also enhances the experience of playing. Here are some of the 2022 games with the best music. 41. Why Choosing The Right Video Platform Is Essential For Your Business The amount of video being created today, as opposed to pre-pandemic, has skyrocketed. 42. How to Get Started with Amazon IVS and Amazon Personalize in Retail This blog will walk through a detailed example of how to configure and deploy live streaming and personalization into an ecommerce site. 43. What Is the Right AWS Live Streaming Solution for Your Use Case In July last year, AWS launched Amazon IVS, a managed live streaming platform for creating low latency interactive streaming services. 44. What Holds People Back from Recording a Video at Work and How Guidde Solved It The purpose of Guidde is to help people, like you, record their videos. They may have numerous reasons why they haven’t started yet, and that’s where we come in 45. How AI Can Help You Make Your Videos Amazing AI is taking over the video editing world by transforming the editing process, allowing organizations to cut time, decrease costs, and increase output. 46. Comparing On-Demand Video (VoD) Platforms For Business Use Cases YouTube is the go-to option for streaming on-demand videos (VoD), but it's not the best for business use cases. Learn more about alternative platforms to use. 47. 5 Tips for Becoming an Amazing Youtuber in 2019 Every digital native out there dreamed of becoming a successful YouTuber at least once in life. Maybe you just hope to make a living out of making interesting videos, want to spread your important message, or tell the rest of the world about your passions. Whatever the reason may be, making YouTube videos as a job seems a really cool and fun way to live. And it is, indeed. 48. Making Promotional Videos for Crypto Startups: Types, Strategies, Expectations, and Costs 49. An Intro to Colorspaces and FFmpeg with InVideo The rendering process of a .mp4 video created on InVideo involves piping a sequence of virtual framebuffers, containing the information of each frame. 50. Top Drone Video Editing Software in 2021: Put Your eye in the sky If you are planning to start on your filming-with-drones journey, check out this list of video editing software. Next-level videos need a next-level edit. 51. Clipchamp Gains 5 million Users as Video Creators Surge More people are looking for easy tools to get into video creation and Clipchamp has grown 40% meeting their needs. 52. How to Use Appsmith, Airtable, and Notion to Build a Video Sorting Tool According to Forbes, 82% of content generated this year is likely to be video. 53. 5 Proven Hacks for Crafting Video Content Your Audience Will Love Whether you're just starting with video marketing or looking to take your efforts to the next level, these hacks will up your game. 54. How Video Compression Works: A Deep Dive into AV1 This blog post provides an overview of why video compression is important and will go over some of the tools that are used in AV1: a royalty-free codec 55. How to Set Perfect YouTube Video Dimensions The article tells about video specifications that should be used for YouTube. And how to change the settings of your video if it does not meet YouTube guideline 56. How High-Quality Videos Can Help You to Convert Leads Currently, video markets get about 66 percent more qualified leads every year. if you want to sell a product online, you can’t possible ignore numbers like these. 57. Image Automation and Assisted Video Annotation Learn how assisted video annotation is providing deeper insights into labeled datasets. 58. How to Look and Sound Better in Video Conference Now that we all spend so much time in video conferencing either for work or life, it is critical to make yourself look and sound the best to your colleagues, family and friends. It is not only about self image, but also about having the “real” you projecting the maximum presence remotely. In this article, I would like to introduce a few simple steps to improve your look and sound in video conferencing tools. 59. A Self-supervised Attention Mechanism To Help With Dense Optical Flow Estimation Multi-object Tracking using self-supervised deep learning Thank you for checking out the 59 most read stories about Video on HackerNoon. to find the most read stories about any technology. Visit the /Learn Repo

59 Stories To Learn About Video

In March 2022, I started sharing my side projects. I used to think my ideas were so special that others might steal them. But that's not true. People noticed when I started sharing my plans. They saw me as a doer - someone who acts, not just talks. My first project was . It was like a for startups and investors. I wanted to learn about investing and make a useful tool. After three months, I put it on . At the time, I didn't know much about marketing. Even so, 26 people signed up. That felt like a big win. Tinvesta Tinder ProductHunt After a break, I made a second app, . I made this because I needed it. I had a list of over 1000 new words written down in my while working for an Israel client. I needed a better way to learn them. NotionLingo was my answer. I also changed how I did marketing. I started writing about how I made my app. This worked much better. Over 100 users signed up. NotionLingo Notion These experiences of publishing two products have been extremely enlightening. Now, I've got a wealth of knowledge to share, with the hope that they can save you some time. Enjoy! An Idea Without Action is Just an Idea So many people believe they have a unique idea. But in reality, an idea without action isn't worth much. My mindset changed after I joined a group for Polish CEOs and angel investors, led by Bartek Pucek from . Within this group, wealthy individuals exchanged ideas to examine both the benefits and drawbacks. Legal professionals were also present to delve into the legal dimensions of these concepts. ElevenLabs This changed everything for me. I was finally around people who thought like me. We all understood that time and energy are valuable. But remember, money isn't everything. Sometimes, you can make a free app and build a small community. We all have to start somewhere. Don't Waste Time and Energy If your project doesn't have any other purpose, like testing something technical or learning a new framework, spending more than a month on it may be unnecessary. If you want to validate your idea or make some money, you need to see how many users are going to sign up for your app. For non-developers looking to validate an idea quickly, No-code tools are an excellent resource. No-code Tools are Awesome For simpler ideas, no-code tools can be invaluable, regardless of whether you're a developer. Services like , , and automation tools like allow for rapid prototyping and deployment. Recently, Zapier has even incorporated AI to write necessary code for you. Webflow Bubble Zapier These tools can save a huge amount of time and energy. They allow you to test and validate your ideas swiftly without having to write a single line of code. However, keep in mind that while no-code tools are incredibly handy, they have limitations. More complex projects may still require coding skills. Solo Foundership is Feasible; AI Can Assist You don't need a team or lots of money. You just need time and willpower. AI can do many things for you. For example, Copy.ai can write for you. It can make ads or social media posts. can help with many tasks too. These tools let you do more things by yourself. You work smarter and make the most of your project. ChatGPT Code Quality is Not Paramount You might write great code. Your backend might have many services, all done perfectly. But does anyone care? You can make a basic, messy project in a week with PHP or C. What really matters is the value your product gives to people. Just use a programming language that makes you fast and happy. It could be any language, even Cobol. If you're successful, no one will care what language you used. Master of One or Jack of All Trades Being great at one thing, like coding, is good. But who cares if you can code without marketing skills? I'd rather be good at many things. I like storytelling, video editing, writing articles, and marketing. These skills make me ready for anything. If I don't know how to do something, I find someone who does. It's often better to be good at many things rather than a master of just one area. Even if you are going to have your own company it may better understand many things, because it may help you understand what’s going on in your company. This approach may also help you in life because you may easily find what you really love to do.

What I Learned From Open-Sourcing 2 of My Side Projects

125 Stories To Learn How To Do X

While ChatGPT is still basking in all the attention it has received over the last few months, AI agents birthed from the tool are giving it close competition already. And the latest member to join this league is AutoGPT – an AI agent that generates prompts by itself and operates independently! Taking You Back to the Start Who knew something that started as a fun side project would become the talk of the digital world? I am fairly confident that even the inventor, who goes by the name Significant Gravitas, would not have anticipated a response so overwhelming when he first shared AutoGPT on GitHub. The torchbearer of AI agents, AutoGPT utilizes while also accessing the internet to execute commands – everything on its own. GPT-4 According to Significant Gravitas, the key features of AutoGPT include: Can search the internet for gathering information Long-and-short-term memory management Text generation using GPT-4 Can access a variety of popular websites and platforms File storage and summarization powered by GPT-3.5 The How AutoGPT categorizes actions as , , and . thoughts reasoning criticism Since it is a GitHub project, using AutoGPT is certainly not as easy as accessing a website (well, some of you hackers are of course an exception). For the unversed, in order to get the most out of AutoGPT, you will need certain things: Python 3.8 or a more recent version An OpenAI API key A Pinecone API key (optional) An ElevenLabs API (only if you want to enable the text-to-speech feature) You can find more instructions, links to these requirements, the setup, and other critical information . on Significant Gravitas' GitHub Capabilities As you might have guessed already, AutoGPT opens doors to a million new possibilities with how efficient it is. With GPT-4 having established its credibility and expertise already, leveraging the tool to get the most out of AI only makes more sense. Here is a list of some more things you can get done faster, and more efficiently using AutoGPT. Create custom marketing tools for copywriting by instructing them to match your brand voice in every scenario. Seamlessly create and embed AI quizzes and lead magnets for lead generation. These can also be integrated with other marketing tools for enhanced visitor experience. Create, deploy, and monetize your favorite prompts by converting them into AI tools dedicated to solving specific use cases. The tool can also be operated in a GPT-3.5 only mode in case you don’t have access to GPT-4 yet. (We both know it is a bit too pricey at the moment, and my sources have informed me OpenAI is working on bringing the costs down. Let’s wait and watch, and until then, GPT-3.5 is your best friend.) AutoGPT also supports image generation using Dall-E. If you want to show off your skills at generating art using Stable Diffusion, you will need a HuggingFace API token. The wise people of the internet (read Twitter) are using AutoGPT to do amazing things. From instructing it to complete the tasks on a to-do list to get it to do market research, it is safe to say we are only at the tip of the iceberg. More examples here: Found, for an app on its own. downloaded, and installed the missing requirements Can for you book flights and order food The Do Anything Machine Well, there can be a few downsides too... The mind behind AutoGPT, , has clarified that the tool is just an experiment to showcase the potential of GPT-4, and I am sure even he must be surprised at how fast things escalated. Significant Gravitas The limitations of AutoGPT, as shared by the inventor, include: It is not yet a polished product, and thus, may not perform well in complex, real-world business scenarios. (If you’ve cracked the code, make sure you inform them too.) Since it can be rather expensive to run, make sure you’ve set and are monitoring your API key limits. To Sum it Up AutoGPT, even in its unpolished self, is a revolutionary tool, to say the least. One thing is for sure – it is here to stay and will only get better as OpenAI continues to release newer versions for GPT. It will be interesting to watch how early adopters leverage it to create great things, automate the mundane, and enhance their overall productivity! What do you think of it?

Introducing AutoGPT – The Newest AI Agent Under the Spotlight 

Stories, News & Mentions about ELEVENLABS

Elevenlabs’s Stories on HackerNoon