: Uvii offers analytic software for product and white label SaaS with group communication across silos

Uvii

Introduction is a foundational class in UIKit and . Everything we see on the screen is constructed from `s and their subclasses. UIView iOS development UIView One topic often considered basic is the geometry of . It's easy to grasp initially, but complete understanding is often elusive, even for very experienced iOS engineers. In this article, we will dive deeply into the world of geometry. We will explain the main concepts, such as , , , , , how they relate to each other, and answer more advanced questions about computing the and Autolayout specifics. UIView UIView frame bounds center anchorPoint transform frame Playground To better understand the geometry, I've also developed a small iOS app: . https://github.com/psharanda/UIViewGeometry In the app, you can experiment with different properties in real time by adjusting sliders. Such an interactive approach is really helpful for grasping the nuances. UIView frame The definition of in Apple documentation is: UIView.frame The frame rectangle, which describes the view’s location and size in its superview’s coordinate system. Most of the time, we operate just with frames, either during manual layout or when printing the view's debug description. The designated initializer for also uses the frame as the only param. UIView This seems simple and straightforward. However, the reality is more complex. The frame is actually a computed property of . It is not the definitive source of the view's geometry. So, what is it exactly? UIView bounds Apparently, has many more properties related to geometry, and we’re now going to check the real, non-computed ones. Let’s examine the first of them: . UIView bounds The definition of is: UIView.bounds The bounds rectangle, which describes the view’s location and size in its own coordinate system. Usually, both and are simply equal to , and is the same as (however, it is not always true). bounds.origin.x bounds.origin.y 0.0 bounds.size frame.size What happens if is not equal to ? In that case, bounds basically act as a viewport, defining which part of the view coordinate system is visible to the outside world (superview). can also be seen as an additional translation (with a minus sign) for all subviews and view’s content rendered using . bounds.origin .zero bounds.origin drawRect This might remind you of . It is for a reason: the property of a is directly linked to its . UIScrollView contentOffset UIScrollView bounds.origin It's important to note that border, background, and shadow are rendered without considering . bounds.origin Previously, we mentioned that sometimes cannot be equal to . It may happen if transform is not equal to . frame.size bounds.size .identity According to the documentation, setting to any value when is not is considered undefined behavior (getting is still fine). However, setting when is not is valid.  In that case, describes the rectangle that fits the view after all transformations in its superview’s coordinate system. frame transform .identity bounds transform .identity frame center The definition of : UIView.center The center point of the view's frame rectangle in its superview’s coordinate system. However, the term "center" can be a bit misleading. It only represents the actual center of the view if the (more on this in the next section) is positioned at the center, specifically at ( , ). A more accurate definition for would be: view.anchorPoint 0.5 0.5 center The coordinates of the view’s anchor point in its superview’s coordinate system. UIView’s is exactly the same as . center CALayer.position The combination of and can fully replace the use of bounds center frame transform The transform property in iOS development is a fundamental concept that enables developers to apply geometric transformations to views. UIView The transform property of is of the type , which represents a matrix used for affine transformations. Affine transformations include: UIView CGAffineTransform Translation To move a view, you set the transform property with . For instance, moves the view 50 points to the right and 75 points down. CGAffineTransform(translationX:y:) view.transform = CGAffineTransform(translationX: 50, y: 75) Scaling To scale a view, use . For example, halves the size of the view in both width and height. CGAffineTransform(scaleX:y:) view.transform = CGAffineTransform(scaleX: 0.5, y: 0.5) Rotation To rotate a view, apply . The angle is in radians, so to rotate 30 degrees, you would use . CGAffineTransform(rotationAngle:) view.transform = CGAffineTransform(rotationAngle: .pi / 6) Combining Transforms One of the powerful aspects of the transform property is the ability to combine transformations. You can combine scaling, rotation, and translation into a single transform using concatenation, for example: var t = CGAffineTransform.identity // 0
t = t.concatenating(CGAffineTransform(scaleX: 0.5, y: 0.5)) //1
t = t.concatenating(CGAffineTransform(rotationAngle: .pi / 6)) //2
t = t.concatenating(CGAffineTransform(translationX: 50, y: 150)) //3
view.transform = t To reset a transformation, set the property to , which is also the default for a newly created . transform CGAffineTransform.identity UIView Perspective ’s transform is baked by which has the type and can be used for even more advanced transformations like changing the perspective UIView CALayer.transform CATransform3D var transform = CATransform3DIdentity;
transform.m34 = 1.0 / 500.0;
view.layer.transform = CATransform3DRotate(transform, .pi / 4, 0, 1, 0) anchorPoint The anchor point defines how a view behaves during transformations. It's specified using the unit coordinate space, where ( , ) is the top-left corner of the view's bounds rectangle, and ( , ) is the bottom-right corner. By default, the anchor point of a is set to ( , ), which is the center of the view’s bounds rectangle. 0 0 1 1 UIView 0.5 0.5 Interestingly, the property was only added to in iOS 16, but it has always been accessible through . anchorPoint UIView view.layer.anchorPoint Geometric manipulations, like rotation or scaling, occur around the anchor point. For example, if you apply a rotation transform to a view with the default anchor point, the view will rotate around its center. If you change the anchor point to a different location, the view will rotate around this new point. How to calculate the frame? We mentioned in the beginning that is a computed property. Now, with enough knowledge, let’s compile all the observations into actual code that computes the frame. frame First, let’s follow the positioning logic that is used during the render process. We start with a rectangle, with set to and set to . is not involved in calculations since it only affects subviews and the output. origin .zero size bounds.size bounds.origin drawRect Before applying the transform, we must translate the view according to its . We apply the transform only when the view, with its , is aligned with the starting point coordinates ( , ). anchorPoint anchorPoint 0 0 Now, we can apply the transform. The last step is to translate the view by its value, positioning it correctly. center These transformations can be concatenated into a single transform (i.e., absolute one), which is then applied to zero-origin bounds by using function. CGRectApplyAffineTransform Here is the final function which computes the frame: func computeFrame(bounds: CGRect,
                  center: CGPoint,
                  anchorPoint: CGPoint,
                  tranform: CGAffineTransform) -> CGRect {

    // set the initial rectangle to bounds with origin set to (0, 0)
    let zeroOriginBounds = CGRect(origin: .zero, size: bounds.size) // 1

    // combine transformations
    let absoluteTransform = CGAffineTransform(
            translationX: -anchorPoint.x * bounds.width,
            y: -anchorPoint.y * bounds.height
        ) // 2
        .concatenating(tranform) // 3
        .concatenating(CGAffineTransform(translationX: center.x, y: center.y)) // 4

    // apply transform to the initial rectangle
    return CGRectApplyAffineTransform(zeroOriginBounds, absoluteTransform)
} How can this knowledge be useful for us? Apart from just satisfying curiosity and gaining a better understanding of the internals, this algorithm has some real-world applications. A classic example is an image editor. Imagine you are building an application that creates custom stories. You want to place stickers on top of a photo, move, scale, and rotate them with your finger. For the editing mode, you use the actual and layout it by modifying its properties like center, bounds, and transform. Eventually, you’d like to render your story project into a bitmap and share it in high resolution, and that’s where understanding how geometry works becomes really useful. The image rendering code can look like the following: UIImageView UIView let renderer = UIGraphicsImageRenderer(size: imageSize)
let image = renderer.image { context in
    let cgContext = context.cgContext
    // render photo…
    // render other things…
    
    // render our sticker
    let absoluteTransform = CGAffineTransform(
        translationX: -sticker.anchorPoint.x * sticker.bounds.width,
        y: -sticker.anchorPoint.y * sticker.bounds.height
    )
        .concatenating(sticker.transform)
        .concatenating(
            CGAffineTransform(translationX: sticker.center.x, y: sticker.center.y)
        )
    
    cgContext.saveGState()
    cgContext.concatenate(absoluteTransform)
    sticker.image.draw(in: CGRect(origin: .zero, size: sticker.bounds.size))
    cgContext.restoreGState()
} What about Autolayout? Autolayout doesn't work with . It also ignores the and properties during its calculations. For each view, Autolayout calculates and sets only the and . The and are still used, but only for the final render on screen, and are applied afterward. frame transform anchorPoint bounds center transform anchorPoint To reiterate, in the Autolayout world, , , and don't exist. The Autolayout process is: constraints in, and out. frame transform anchorPoint bounds center Additional Resources UIView Documentation – https://developer.apple.com/documentation/uikit/uiview View Programming Guide – https://developer.apple.com/library/archive/documentation/WindowsViews/Conceptual/ViewPG_iPhoneOS/WindowsandViews/WindowsandViews.html#//apple_ref/doc/uid/TP40009503-CH2-SW5 Core Animation Basics – https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CoreAnimation_guide/CoreAnimationBasics/CoreAnimationBasics.html#//apple_ref/doc/uid/TP40004514-CH2-SW3 Understanding UIKit Rendering – https://www.wwdcnotes.com/notes/wwdc11/121/ Understanding UIKit: CALayer Anchor Point – https://ikyle.me/blog/2022/understanding-uikit-calayer-anchor-point Understanding UIScrollView – https://oleb.net/blog/2014/04/understanding-uiscrollview/

Mastering UIView Geometry: A Dive into UIKit's Foundational Class and Geometry Concepts for iOS

FastAPI Got Me an OpenAPI Spec Really... Fast

ফাস্টএপিআই আমাকে একটি ওপেনএপিআই স্পেক দিয়েছে... দ্রুত

FastAPI hat mir wirklich ... schnell eine OpenAPI-Spezifikation besorgt

FastAPI me consiguió una especificación OpenAPI realmente... rápido

FastAPI m'a offert une spécification OpenAPI vraiment... Rapide

FastAPI ने मुझे एक OpenAPI विनिर्देशन वास्तव में... तेजी से दिलाया

FastAPI で OpenAPI 仕様を本当に早く入手できました

FastAPI를 통해 OpenAPI 사양이 정말... 빨라졌습니다.

FastAPI me deu uma especificação OpenAPI muito... rápida

FastAPI дал мне спецификацию OpenAPI действительно... быстро

FastAPI Bana Gerçekten Bir OpenAPI Özelliği Sağladı... Hızlı

FastAPI đã cung cấp cho tôi thông số OpenAPI thực sự... Nhanh

FastAPI 让我快速获得了 OpenAPI 规范

Automatic Brand Sentiment Analysis with Generative AI

Having the correct data to support your use case is essential to a successful implementation of LLMs in any business. While most out-of-the-box LLMs are great at tasks, they can struggle with specific business problems. They didn’t train on the data for your business problem, so they don’t have adequate context to solve it. general Businesses often have a treasure trove of internal data and documents that could meet this need for specific context. But, here’s the question: How do we integrate all this useful data (context) into the LLM without doing resource-intensive and time-consuming retraining or fine-tuning the LLM? The answer is (RAG), a technique that enhances LLMs with just-in-time retrieval of close context information. retrieval-augmented generation In this post, we’ll walk through how to use and to implement the storage and retrieval of this contextual data for an LLM to use. We’ll solve a context-specific problem with RAG by using LlamaIndex, and then we’ll deploy our solution easily to Heroku. LlamaIndex LangChain Before we start coding, let’s quickly cover the core concepts. A brief introduction to RAG and LlamaIndex When you ask your LLM a question that requires context to answer, RAG retrieves that contextual data to help the LLM give a more accurate and specific response. It’s like having a sous chef quickly run out to the farmer’s market to get the freshest ingredients that weren’t in the pantry so that the executive chef can cook up the perfect dish with all the necessary ingredients. One key to how the RAG workflow can provide context is using vector databases and vector search indexes. Let’s break down some of the core concepts and what this all entails. A is a set of coded numbers that represent the meaning and context of a piece of text (such as a word, phrase, sentence, or even an entire document). vector An is the actual numerical values in a vector; but most people tend to use the terms “vector” and “embedding” interchangeably. embedding An has been trained on documents so that it can convert newly input text into vectors. Not all texts talk about the same things or in the same way — consider a set of academic research papers versus a set of marketing materials. So, we have different embedding models — each trained on specific datasets with specific goals in mind. embedding model Using an embedding model, we can create embeddings from documents, breaking the text in those documents down to their coded numbers. Creating an embedding might involve a strategy like , which splits large documents into smaller, manageable pieces. From there, each chunk is converted into an embedding. document chunking When you query a , your question is turned into an embedding and compared against all of the other embeddings stored in a vector database. When you build a , you can perform very fast and accurate vector searches (also called similarity searches). Using vector databases lets you perform fast and accurate searches — not merely to match the presence of a specific string like in traditional databases, but to match for documents that are similar in with the words you used. vector database vector search index meaning Within the context of RAG, we use our original prompt to perform a vector search against all of the documents in a vector database. Then, those matching documents are sent as context to the LLM application. The LLM now has a set of detailed notes to refer to when crafting its answer to the original prompt. LlamaIndex is a crucial framework that simplifies the process of integrating, organizing, and retrieving private or specialized data. It will help us with creating the document embeddings and vector search index. Then, we’ll lean on LangChain to piece it all together, executing the similarity search and sending the results to our LLM to fetch a response. Together, LlamaIndex and LangChain provide a secure and dependable solution for handling the RAG workflow. Are you ready to build something? Here we go. An introduction to our demo project The best way to learn about RAG with LlamaIndex and Heroku is to build a small sample application. For our purposes, let’s pretend we work with , a library of over 70,000 free ebooks. We want to build an LLM-based chatbot that can answer specific questions about the free books in the project. Project Gutenberg This is the perfect use case for using RAG, and we can use LlamaIndex to take advantage of the large corpus of book texts available to us. To keep our project simple, we’ll use , a popular book from AD 401. The Confessions of St. Augustine The codebase for our completed project can be found in this . If you prefer, you can clone the repo and deploy the application to Heroku. Or, you can follow along step by step to see how we got the code that we did. GitHub repository We’ll follow these general steps: Set up our project. Load the data. Build the index. Store the index. Integrate LangChain. Deploy to Heroku. Step 1: Set up our project Create a new folder for your Python project. Then, activate a and install the initial dependencies that we will need. venv (venv) ~/project$ pip install llama-index langchain langchain-openai Next, we’ll load the data to be indexed. Step 2: Load the data When building an index of internal data for use with RAG, you must gather all your data (text) into a single place. In our example case, that's the text of the book we mentioned above. This is the context that we will soon convert to a vector index of embeddings using LlamaIndex. In the typical use case, your context data will be a large corpus of text that fits the business problem you're trying to solve. For our mini demo, we'll create a subfolder called , and then we'll download the book as a single file in that folder. data (venv) ~/project$ mkdir data

(venv) ~/project$ curl https://www.gutenberg.org/cache/epub/3296/pg3296.txt -o data/confessions.txt

(venv) ~/project$ ls data
confessions.txt Step 3: Build the index After gathering all your data in a single directory, it’s time to build the index. We’ll write a simple Python application that will use LlamaIndex to index the data, and then we’ll query our index. For this to work, you will need an OpenAI account and API key. This is because LlamaIndex uses as the . (Changing those defaults is outside the scope of this article.) OpenAI’s text-embedding-3-small default embedding model In our project root folder, we create a file called . The initial contents look like this: index.py # index.py

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
import os

if os.environ.get('OPENAI_API_KEY') is None:
  exit('You must provide an OPENAI_API_KEY env var.')

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine()
response = query_engine.query("In which city is Saint Augustine the Bishop?")
print(response) We run our file and receive the expected response: (venv) ~/project$ OPENAI_API_KEY=sk-******** python index.py
Hippo Of course, we can double-check our data. Looking at the first few lines of the book, we see: THE CONFESSIONS OF SAINT AUGUSTINE

By Saint Augustine

Bishop of Hippo As you can see, LlamaIndex did its job. Our Python application completed our question exactly as we would expect from the vector-indexed data. Step 4: Store the index It’s important to note that, in the above example, we only store our indexed data in memory and not on a disk. Our index — now a series of vector embeddings in memory — will be lost completely after we make our call to the OpenAI model and finish the workflow. Creating a vector index (embeddings) for our text isn’t , so we don’t want to have to recompute those results every time we call the model. It’s best to have a separate workflow where we persist the index to disk. Then, we can reference it at any time later. free A common approach to doing this is to store embeddings in a PostgreSQL database and use to perform similarity search. To keep our demonstration simple, we’ll just store our indexed data as flat files. pgvector So, we add this simple step to our file: index.py PERSIST_DIR='./my_vector_indexes/gutenberg/'
index.storage_context.persist(persist_dir=PERSIST_DIR) Now, after we run our file, we can check for our stored index. (venv) ~/project$ OPENAI_API_KEY=sk-******** python index.py
Hippo

(venv) ~/project$ tree
.
├── data
│   └── confessions.txt
├── index.py
└── my_vector_indexes
    └── gutenberg
        ├── default__vector_store.json
        ├── docstore.json
        ├── graph_store.json
        ├── image__vector_store.json
        └── index_store.json

3 directories, 7 files Step 5: Integrate LangChain We’ve learned the basics of what a vector index store is and how easy it is to build one. But, to actually build an end-to-end application that chains it all together, we use . This way, we can deploy our solution as an API. Let’s rewrite our index.py code to be a bit more production-worthy. LangChain We’ll show the code below and then explain what we’re doing after. It might look like a lot of code, but we are only adding a few new steps. # index.py

import os
from langchain_openai import ChatOpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.memory import ConversationBufferWindowMemory
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from langchain_community.retrievers import LlamaIndexRetriever
from fastapi import FastAPI
from pydantic import BaseModel

if os.environ.get('OPENAI_API_KEY') is None:
  exit('You must provide an OPENAI_API_KEY env var.')

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)

# For this demo, we will not persist the index.

retriever = LlamaIndexRetriever(index=index.as_query_engine())

llm = ChatOpenAI(model_name="gpt-3.5-turbo", max_tokens=2048)

memory = ConversationBufferWindowMemory(
  memory_key='chat_history',
  return_messages=True,
  k=3
)

conversation = ConversationalRetrievalChain.from_llm(
  llm=llm, 
  retriever=retriever,
  memory=memory,
  max_tokens_limit=1536  
)

class Prompt(BaseModel):
  question: str

app = FastAPI()

@app.post("/prompt")
async def query_chatbot(prompt: Prompt):
  response = conversation.invoke({'question': prompt.question})
  return response['answer']

if __name__=='__main__':
  import uvicorn
  uvicorn.run(app, host="localhost", port=8000) First, notice that we are now directly using LangChain and OpenAI. We set up our along with some so that our can be “remembered” across subsequent queries. Now, we have an actual ChatBot which we can interact with. llm memory conversation From there, we use to create an API server that listens for requests on the endpoint. Requests to that endpoint are expected to have a request body with a question, which is then passed (along with context from our vector index) to the LLM. FastAPI POST /prompt We use to spin up our server on port 8000. uvicorn Before we can start up our server, let’s add those new Python dependencies: (venv) ~/project$ pip install fastapi pydantic uvicorn Now, it’s time to test. We start by spinning up our server. (venv) ~/project$ OPENAI_API_KEY=sk-******** python index.py
INFO:     Started server process [1101807]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8000 (Press CTRL+C to quit) In another terminal, we send a curl request to our endpoint. $ curl -X POST \
       --header "Content-type:application/json" \
       --data '{"question":"Who is Ambrose?"}' \
       http://localhost:8000/prompt

"Ambrose is a person mentioned in the text provided. He is described as a
respected and celibate man who was esteemed by the author. Ambrose is depicted
as a figure of great honor and excellence, particularly known for his
dedication to reading and studying." Success! Our vector index appears to be up and running, and our chatbot is fully functional. It’s time to deploy. Step 6: Deploy to Heroku After completing the major legwork, we only need to take a few simple steps to deploy our application to Heroku. Create the file with Python dependencies requirements.txt Heroku will need to know what Python dependencies to install when building our project. It looks for this list in a file called . We can easily generate one with this command: requirements.txt (venv) ~/project$ pip freeze > requirements.txt Create the Procfile We also need to tell Heroku how to spin up our Python application. We do this in a file called . Procfile (venv) ~/project$ echo \
   'web: uvicorn index:app --host=0.0.0.0 --port=${PORT}' > Procfile Create the file runtime.txt Lastly, will tell Heroku which Python runtime version we would like to use. runtime.txt (venv) ~/project$ echo 'python-3.11.8' > runtime.txt Those are all the files we need. This is what our project folder structure should look like (we’ve removed the persisted vector index): ~/project$ tree
.
├── data
│   └── confessions.txt
├── index.py
├── Procfile
├── requirements.txt
└── runtime.txt

1 directory, 5 files If you’re working from scratch and didn’t clone the for this demo project, then commit these files to your own Git repository. GitHub repo Create a Heroku app After downloading and installing the , run the following commands. You can choose any name for your app, and you’ll need to provide your unique OpenAI API key. Heroku CLI ~/project$ heroku login

~/project$ heroku apps:create my-llamaindex-app

~/project$ heroku git:remote -a my-llamaindex-app

~/project$ heroku config:add OPENAI_API_KEY=replaceme -a my-llamaindex-app

~/project$ git push heroku main
…
remote: -----> Building on the Heroku-22 stack
remote: -----> Determining which buildpack to use for this app
remote: -----> Python app detected
remote: -----> Using Python version specified in runtime.txt
…
remote: -----> Launching...
remote:        Released v4
remote:        https://my-llamaindex-app-6b48faa3ee6a.herokuapp.com/ deployed to Heroku With our application deployed, we test by sending a curl request to our API server: $ curl -X POST \
       --header "Content-type:application/json" \
       --data '{"question":"Who is Ambrose?"}' \
       https://my-llamaindex-app-6b48faa3ee6a.herokuapp.com/prompt

"Ambrose is a significant figure in the text provided. He is being described
as a respected and happy man, known for his celibacy and his dedication to
reading and studying. He is referred to as a holy oracle and a person of
great influence and wisdom." Keep in mind that the curl call above uses the unique Heroku app URL for our deployment. Yours will be different from what is shown here. We are up and running on Heroku! Conclusion We’ve clearly seen the power of LlamaIndex and the important role it plays when building RAG apps to interact with LLMs. When we can easily add specific data sources as context for an LLM without expensive model retraining — that’s a huge win. It’s a win for companies and developers looking to take their LLM workflows a step further. Combining LlamaIndex with other LangChain toolsets is also seamless and straightforward. Building a chatbot requires only a few lines of additional code. And finally, being able to quickly and easily deploy our solution to Heroku makes our application immediately accessible without any fuss. Simple deployments like this free up developers to focus on the more complicated and important task of building their LLM-based solutions. Happy coding!

Streamlining LLM Implementation: How to Enhance Specific Business Solutions with RAG

Building and deploying LLM-based applications has never been easier. Since the launch and wide adoption of ChatGPT near the end of 2022, we’ve seen a storm of news about tools, products, and innovations stemming from large language models (LLMs) and generative AI (GenAI). While many tech fads come and go within a few years, it’s clear that are here to stay. LLMs and GenAI Do you ever wonder about all the tooling going on in the background behind many of these new tools and products? In addition, you might even ask yourself how these tools — leveraged by both developer and end users — are run in production. When you peel back the layers for many of these tools and applications, you’re likely to come across . LangChain, Python, and Heroku These are the pieces that we’re going to play around with in this article. We’ll look at a practical example of how AI/ML developers use them to build and easily deploy complex LLM pipeline components. Demystifying LLM Workflows and Pipelines Machine learning pipelines and workflows can seem like a black box for those new to the AI world. This is even more the case with LLMs and their related tools, as they’re such (relatively) new technologies. Working with LLMs can be challenging, especially as you’re looking to create engineering-hardened and production-ready pipelines, workflows, and deployments. With new tools, rapidly changing documentation, and limited instructions, knowing where to start or what to use can be hard. So, let’s start with the basics of LangChain and Heroku. The tells us this: “ ” documentation for LangChain LangChain is a framework for developing applications powered by language models. Meanwhile, this way: “ ” Heroku describes itself Heroku is a cloud platform that lets companies build, deliver, monitor, and scale apps — we’re the fastest way to go from idea to URL, bypassing all those infrastructure headaches. If we put this in the context of building an LLM application, then LangChain and Heroku are a match made in heaven. We need a well-tested and easy-to-use framework (LangChain) to build our LLM application, and then we need a way to deploy and host that application (Heroku). Let’s look into each of these technologies in more detail. Diving into LangChain Let’s briefly discuss how LangChain is used. LangChain is a framework that assists developers in building applications based on LLM models and use cases. It has support for , , and . For example, let’s say we were building a tool that generates reports based on user input or automates customer support response. LangChain acts as the scaffolding for our project, providing the tools and structure to efficiently integrate language models into our solution. Python JavaScript TypeScript Within LangChain, we have several key components: : The agent is the component that interacts with the language model to perform tasks based on our requirements. This is the brain of our application, using the capabilities of language models to understand and generate text. Agent : These are sequences of actions or processes that our agent follows to accomplish a task. For example, if we were automating customer support, a chain might include accepting a customer query, finding relevant information, and then crafting a response. Chains : Templates provide a way to structure the outputs from the language model. For example, if our application generates reports, then we would leverage a template that helps format these reports consistently based on the model’s output. Templates : This enables developers to up LangChain applications as a REST API. LangServe deploy and serve : This tool helps developers the interactions in their language model applications to get them ready for production. LangSmith evaluate, test, and refine LangChain is a widely adopted framework for building AI and LLM applications, and it’s easy to see why. LangChain provides the functionality to build and deploy products end-to-end. Diving into Heroku Heroku is best known as a cloud platform as a service (PaaS) which makes it incredibly simple to deploy applications to the cloud. Developers often want to focus solely on code and implementation. When you’re already dealing with complex data pipelines and LLM-based applications, you likely don’t have the resources or expertise to deal with infrastructure concerns like servers, networks, and persistent storage. With the ability to easily deploy your apps through Heroku, the major hurdle of production-izing your projects is handled effortlessly. Building with LangChain For a better understanding of how LangChain is used in an LLM application, let’s work through some example problems to make the process clear. In general, we would chain together the following pieces to form a single workflow for an LLM chain: Start with a to generate a prompt based on parameters from the user. prompt template Add a to the chain to retrieve data that the language model was not originally trained on (for example, from a database of documents). retriever Add a to include chat history so that the language model has context for formulating a better response. conversation retrieval chain Add an agent to interact with an actual LLM. LangChain lets us “chain” together the processes that form the base of an LLM application. This makes our implementation easy and approachable. Let’s work with a simple example. In our example, we’ll work with OpenAI. We’ll craft our prompt this way: Tell OpenAI to take on the persona of an encouraging fitness trainer. Input a question from the end user. To keep it nice and simple, we won’t worry about chaining in the retrieval of external data or chat history. Once you get the hang of LangChain, adding other capabilities to your chain is straightforward. On our local machine, we activate a virtual environment. Then, we install the packages we need: (venv) $ pip install langchain langchain_openai We’ll create a new file called . Our basic Python code looks like this: main.py import os
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

my_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a friendly and encouraging fitness trainer."),
    ("user", "{input}")
])

llm = ChatOpenAI(openai_api_key=os.getenv("OPENAI_API_KEY"))

chain = my_prompt | llm That’s it! In this basic example, we’ve used LangChain to chain together a prompt template and our OpenAI agent. To use this from the command line, we would add the following code: user_input = input("Ask me a question related to your fitness goals.\n")
response = chain.invoke({
  "input": user_input
})

print(response) Let’s test out our application from the command line. (venv) $ OPENAI_API_KEY=insert-key-here python3 main.py 
Ask me a question related to your fitness goals.
How do I progress toward holding a plank for 60 seconds?
content="That's a great goal to work towards! To progress towards holding a plank for 
60 seconds, it's important to start with proper form and gradually increase the 
duration of your plank holds. Here are some tips to help you progress:\n\n1. Start 
with shorter durations: Begin by holding a plank for as long as you can with good 
form, even if it's just for a few seconds. Gradually increase the time as you get 
stronger.\n\n2. Focus on proper form: Make sure your body is in a straight line from 
head to heels, engage your core muscles, and keep your shoulders directly over your 
elbows.\n\n3. Practice regularly: Aim to include planks in your workout routine a few 
times a week. Consistency is key to building strength and endurance.\n\n4. Mix it up: 
Try different variations of planks, such as side planks or plank with leg lifts, to 
work different muscle groups and keep your workouts challenging.\n\n5. Listen to your 
body: It's important to push yourself, but also know your limits. If you feel any 
pain or discomfort, stop and rest.\n\nRemember, progress takes time and patience. 
Celebrate each milestone along the way, whether it's holding a plank for a few extra 
seconds or mastering a new plank variation. You've got this!" (I’ve added line breaks above for readability.) That’s a great start. But it would be nice if the output was formatted to be a bit more human-readable. To do that, we simply need to add an to our chain. We’ll use . output parser StrOutputParser import os
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser

my_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a friendly and encouraging fitness trainer."),
    ("user", "{input}")
])

llm = ChatOpenAI(openai_api_key=os.getenv("OPENAI_API_KEY"))

chain = my_prompt | llm | output_parser

user_input = input("Ask me a question related to your fitness goals.\n")
response = chain.invoke({
  "input": user_input
})

print(response) Now, at the command line, our application looks like this: (venv) $ OPENAI_API_KEY=insert-key-here python3 main.py 
Ask me a question related to your fitness goals.
How do I learn how to do a pistol squat?
That's a great goal to work towards! Pistol squats can be challenging but with practice and patience, you can definitely learn how to do them.

Here are some steps you can follow to progress towards a pistol squat:

1. Start by improving your lower body strength with exercises like squats, lunges, and step-ups.
2. Work on your balance and stability by practicing single-leg balance exercises.
3. Practice partial pistol squats by lowering yourself down onto a bench or chair until you can eventually perform a full pistol squat.
4. Use a support like a TRX band or a pole to assist you with balance and lowering yourself down until you build enough strength to do it unassisted.

Remember to always warm up before attempting pistol squats and listen to your body to avoid injury. And most importantly, stay positive and patient with yourself as you work towards mastering this challenging exercise. You've got this! The LLM response is now formatted for improved readability. Our chains would be much more complex than this for building powerful LLM applications. But that’s the power and simplicity of LangChain. The framework allows for the modularity of logic specific to your needs so you can easily chain together complex workflows. Now that we have a simple LLM application built, we still need the ability to deploy, host, and serve our application to make it useful. As a developer focused on app building rather than infrastructure, we turn to LangServe and Heroku. Serving with LangServe LangServe helps us interact with a LangChain chain through a REST API. To write the serving portion of a LangChain LLM application, we need three key components: A valid chain ( ) like what we built above An API application framework (such as ) FastAPI Route definitions (just as we would have for building any sort of REST API) The provide some helpful examples of how to get up and running. For our example, we just need to use FastAPI to start up an API server and call from LangServe to make our chain accessible via API endpoints. LangServe docs add_routes() Along with this, we’ll need to make some minor modifications to our existing code. We’ll remove the use of the . This will give callers of our API flexibility in how they want to format and use the output. StrOutputParser We won’t prompt for user input from the command line. The API call request will provide the user’s input. We won’t call because LangServe will make this part of handling the API request. chain.invoke() We make sure to add the FastAPI and LangServe packages to our project: (venv) $ pip install langserve fastapi Our final file looks like this: main.py import os
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from fastapi import FastAPI
from langserve import add_routes

my_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a friendly and encouraging fitness trainer."),
    ("user", "{input}")
])

llm = ChatOpenAI(openai_api_key=os.getenv("OPENAI_API_KEY"))

chain = my_prompt | llm

app = FastAPI(title="Fitness Trainer"")

add_routes(app, chain)

if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app, host="localhost", port=8000) On my local machine (Ubuntu 20.04.6 LTS) running Python 3.8.10, I also needed to install some additional packages to get rid of some warnings. You might not need to do this on your machine. (venv) $ pip install sse_starlette pydantic==1.10.13 Now, we start up our server: (venv) $ OPENAI_API_KEY=insert-key-here python3 main.py 

INFO:     Started server process [629848]
INFO:     Waiting for application startup.

LANGSERVE: Playground for chain "/" is live at:
LANGSERVE:  │
LANGSERVE:  └──> /playground/
LANGSERVE:
LANGSERVE: See all available routes at /docs/

INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8000 (Press CTRL+C to quit) Ooooh… nice! In the browser, we can go to . This is what we see: http://localhost:8000/docs LangServe serves up an API docs page that uses a Swagger UI! These are the endpoints now available to us through LangServe. We send a request to the endpoint. But LangServe also gives us a endpoint with a web interface to work with our chain directly. could POST invoke/ playground/ We provide input and click . Here’s the result: Start It’s important to stress the importance of having APIs in the context of LLM application workflows. If you think about it, most use cases of LLMs and applications built on top of them can’t rely on models and resources for inference. This neither makes sense nor scales well. local The real power of LLM applications is the ability to abstract away the complex workflow we’ve described so far. We want to put everything we’ve done behind an API so the use case can scale and others can integrate it. This is only possible if we have an easy option to host and serve these APIs. And that’s where Heroku comes in. Deploying to Heroku Heroku is the key, final part of our LLM application implementation. We have LangChain piece together our workflow, and LangServe will serve it up as a useful REST API. Now, instead of manually setting up complex resources to host and serve traffic, we turn to Heroku for the simple deployment of our application. After , we’re nearly ready to deploy. Let’s walk through the steps. setting up a Heroku account Create a new Heroku app Using the Heroku CLI, we log in and create a new app. $ heroku login
$ heroku create my-langchain-app Set config variables Next, we need to set the environment variable in our Heroku app environment. OPENAI_API_KEY $ heroku config:set OPENAI_API_KEY=replace-with-your-openai-api-key Create config files for Python application deployment To let Heroku know what we need for our Python application to run, we need to create three simple files: : Declares what command Heroku should execute to start our app. Procfile : Specifies the Python package dependencies that Heroku will need to install. requirements.txt : Specifies the exact version of the Python runtime we want to use for our app. runtime.txt These files are quick and easy to create. Each one goes into the project’s root folder. To create the , we run this command: Procfile $ echo 'web: uvicorn main:app --host=0.0.0.0 --port=${PORT}' > Procfile This tells Heroku to run , which is a web server implementation in Python. uvicorn For , we can use the command to output the list of installed packages. requirements.txt pip freeze $ pip freeze > requirements.txt Lastly, for , we will use Python 3.11.8. runtime.txt $ echo 'python-3.11.8' > runtime.txt With these files in place, our project root folder should look like this: $ tree
.
├── main.py
├── Procfile
├── requirements.txt
└── runtime.txt

0 directories, 4 files We commit all of these files to the GitHub repository. Connect Heroku to GitHub repo The last thing to do is for our GitHub repo and then push our code to the remote. Heroku will detect the push of new code and then deploy that code to our application. create a Heroku remote $ heroku git:remote -a my-langchain-app
$ git push heroku main When our code is pushed to the Heroku remote, Heroku builds the application, installs dependencies, and then runs the command in our . The final result of our command looks like this: Procfile git push …

remote: -----> Discovering process types
remote:        Procfile declares types -> web
remote: 
remote: -----> Compressing...
remote:        Done: 71.8M
remote: -----> Launching...
remote:        Released v4
remote:        https://my-langchain-app-ea95419b2750.herokuapp.com/ deployed to Heroku
remote: 
remote: Verifying deploy... done. This shows the URL for our Heroku app. In our browser, we visit . https://my-langchain-app-ea95419b2750.herokuapp.com/playground We also check out our Swagger UI docs page at . https://my-langchain-app-ea95419b2750.herokuapp.com/docs And just like that, we’re up and running! This process is the best way to reduce developer time and overhead when working on large, complex LLM pipelines with LangChain. The ability to take APIs built with LangChain and seamlessly deploy to Heroku with a few simple command line arguments is what makes the pairing of LangChain and Heroku a no-brainer. Conclusion Businesses and developers today are right to ride the wave of AI and LLMs. There’s so much room for innovation and new development in these areas. However, the difference between the successes and failures will depend a lot on the toolchain they use to build and deploy these applications. Using the LangChain framework makes the process of building LLM-based applications approachable and repeatable. But, implementation is only half the battle. Once your application is built, you need the ability to easily and quickly deploy those application APIs into the cloud. That’s where you’ll have the advantage of faster iteration and development, and Heroku is a great way to get you there. Also published . here

Streamlining LLM Application Development and Deployment with LangChain, Heroku, and Python

Table of Contents A Deep Dive into Voice Cloning with SoftVC VITS and Bert-VITS2 Prepare Dataset Extract from a Song UVR Workflows Preparation for vocal recording Cheapskate’s Audio Equipment Audacity workflows audio-slicer Cleaning dataset Match loudness so-vits-svc Set up environment Initialization Download pre-trained models Dataset Preparation Edit Configs Training Inference Audio Editing so-vits-svc-fork Installation Preparation Training Inference DDSP-SVC Preparation Training Inference Bert-vits2-V2.3 Initialization Preparation Transcription Training and Inference vits-simple-api Tweaks Share models In the , I tried a little bit of and found it interesting. So, I decided to train a usable model with my own voice. previous post TTS Generation WebUI This voice cloning project explores both SVC for Voice Changing and for Text-to-Speech. There is no one tool that does all jobs. VITS I have tested several tools for this project. Many of the good guides, like , , and , are in Chinese. So, I thought it would be useful to post my notes in English. this this this Although has been archived for a few months, probably due to oppression, it is still the tool for the best result. so-vits-svc Other related tools such as , , , and provide either faster/liter optimization, more features, or better interfaces. so-vits-svc-fork so-vits-svc-5.0 DDSP-SVC RVC But with enough time and resources, none of these alternatives can compete with the superior result generated by the original so-vits-svc. For TTS, a new tool called works fantastically and has already matured with its final release last month. It has some very different use cases, for example, audio content creation. Bert-VITS2 Prepare Dataset The audio files of the dataset should be in WAV format, 44100 Hz, 16bit, mono, 1-2 hours ideally. Extract From a Song is the easiest tool for this job. There is a that explains everything in detail. Ultimate Vocal Remover thread UVR Workflows Remove and extract Instrumental Model: VR - UVR(4_HP-Vocal-UVR) Settings: 512 - 10 - GPU Output and unclean vocal Instrumental Remove and extract background vocals. Model: VR - UVR(5_HP-Karaoke-UVR) Settings: 512 - 10 - GPU Output and unclean main vocal background vocal Remove reverb and noise. Model: VR - UVR-DeEcho-DeReverb & UVR-DeNoise Settings: 512 - 10 - GPU - No Other Only Output clean main vocal (Optional) Using RipX (non-free) to perform a manual fine-cleaning. Preparation for Vocal Recording It’s better to record in a treated room with a condenser microphone, otherwise, use a directional or dynamic microphone to reduce noise. Cheapskate’s Audio Equipment The very first time I got into music was during high school, with the blue Sennheiser MX500 and Koss Porta Pro. I still remember the first time I was recording a song that was on a Sony VAIO with Cool Edit Pro. Nowadays, I still resist spending a lot of money on audio hardware as an amateur because it is literally a money-sucking black hole. Nonetheless, I really appreciate the reliability of those cheap production equipment. The core part of my setup is a Behringer UCA202, and it’s perfect for my use cases. I bought it for $10 during a price drop. It is a so-called “Audio Interface” but basically just a sound card with multiple ports. I used RCA to 3.5mm TRS cables for my headphones, a semi-open K240s for regular output, and a closed-back HD669/MDR7506 for monitor output. All three mentioned headphones are under $100 for the normal price. And there are clones from Samson, Tascam, Knox Gear, and more out there for less than $50. For the input device, I’m using a dynamic microphone for the sake of my environmental noises. It is an SM58 copy (Pyle) + a Tascam DR-05 recorder (as an amplifier). Other clones such as SL84c or wm58 would do it too. I use an XLR to 3.5mm TRS cable to connect the microphone to the MIC/External-input of the recorder, and then use an AUX cable to connect between the line-out of the recorder and the input of the UCA202. It’s not recommended to buy an “audio interface” and a dedicated amplifier to replicate my setup. A $10 c-media USB sound card should be good enough. The Syba model that I owned is capable of “pre-amping” dynamic microphones directly and even some lower-end phantom-powered microphones. The setup can go extremely cheap ($40~60), but with UCA202 and DR-05, the sound is much cleaner. And I really like the physical controls, versatility, and portability of my old good digital recorder. Audacity Workflows Although when I was getting paid as a designer, I was pretty happy with Audition. But for personal use on a fun project, Audacity is the way to avoid the chaotic evil of Adobe. Noise Reduction Dereverb Truncate Silence Normalize audio-slicer Use or to slice the audio file into small pieces for later use. audio-slicer audio-slicer (gui) The default setting works great. Cleaning Dataset Remove those very short ones and re-slice those that are still over 10 seconds. In case of a large dataset, remove all that are less than 4 sec. In the case of a small dataset, remove those only under 2 sec. If necessary, perform a manual inspection for every single file. Match Loudness Use Audacity again with ; 0db should do it. Loudness Normalization so-vits-svc Set Up the Environment Virtual environment is essential to run multiple Python tools inside one system. I used to use VMs and Docker, but now, I found that is way quicker and handier than the others. anaconda Create a new environment for so-vits-svc, and activate it. conda create -n so-vits-svc python=3.8
conda activate so-vits-svc Then, install requirements. git clone https://github.com/svc-develop-team/so-vits-svc
cd so-vits-svc

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

#for linux
pip install -r requirements.txt

#for windows
pip install -r requirements_win.txt
pip install --upgrade fastapi==0.84.0
pip install --upgrade gradio==3.41.2
pip install --upgrade pydantic==1.10.12
pip install fastapi uvicorn Initialization Download pre-trained models pre-train wget https://huggingface.co/WitchHuntTV/checkpoint_best_legacy_500.pt/resolve/main/checkpoint_best_legacy_500.pt wget https://huggingface.co/lj1995/VoiceConversionWebUI/resolve/main/rmvpe.pt logs/44k wget https://huggingface.co/datasets/ms903/sovits4.0-768vec-layer12/resolve/main/sovits_768l12_pre_large_320k/clean_D_320000.pth wget https://huggingface.co/datasets/ms903/sovits4.0-768vec-layer12/resolve/main/sovits_768l12_pre_large_320k/clean_G_320000.pth logs/44k/diffusion wget https://huggingface.co/datasets/ms903/Diff-SVC-refactor-pre-trained-model/resolve/main/fix_pitch_add_vctk_600k/model_0.pt (Alternative) wget https://huggingface.co/datasets/ms903/DDSP-SVC-4.0/resolve/main/pre-trained-model/model_0.pt (Alternative) wget https://huggingface.co/datasets/ms903/Diff-SVC-refactor-pre-trained-model/blob/main/hubertsoft_fix_pitch_add_vctk_500k/model_0.pt pre-train/nsf_hifigan wget -P pretrain/ https://github.com/openvpi/vocoders/releases/download/nsf-hifigan-v1/nsf_hifigan_20221211.zip unzip -od pretrain/nsf_hifigan pretrain/nsf_hifigan_20221211.zip Dataset Preparation Put all Prepared audio.wav files into dataset_raw/character cd so-vits-svc
python resample.py --skip_loudnorm
python preprocess_flist_config.py --speech_encoder vec768l12 --vol_aug
python preprocess_hubert_f0.py --use_diff Edit Configs The file is located at configs/config.json : the frequency of printing log : the frequency of saving checkpoints : total steps : numbers of saved checkpoints, 0 for unlimited. : fp32 In my case, : the smaller the faster (rougher), the larger the slower (better). log interval eval interval epochs keep ckpts half_type batch_size Recommended batch_size per VRAM: 4=6G；6=8G；10=12G；14=16G；20=24G Keep default for configs/diffusion.yaml Training python cluster/train_cluster.py --gpu
python train_index.py -c configs/config.json
python train.py -c configs/config.json -m 44k
python train_diff.py -c configs/diffusion.yaml On training steps: Use to train the main model; usually, 20k-30k would be usable, and 50k and up would be good enough. This can take a few days depending on the GPU speed. train.py Feel free to stop it by , and it will be continued training by re-run anytime. ctrl+c python train.py -c configs/config.json -m 44k Use to train the diffusion model; training steps is recommended at 1/3 of the main model. train_diff.py Be aware of over-training. Use to monitor the plots to see if it goes flat. tensorboard --logdir=./logs/44k Change the from 0.0001 to 0.00005 if necessary. learning rate When done, share/transport these files for inference. config/ config.json diffusion.yaml logs/44k feature_and_index.pkl kmeans_10000.pt model_0.pt G_xxxxx.pt Inference It’s time to try out the trained model. I’d prefer Webui for the convenience of tweaking the parameters. But before firing it up, edit the following lines in for LAN access: webUI.py os.system("start http://localhost:7860")
app.launch(server_name="0.0.0.0", server_port=7860) Run ; then access its from a web browser. python webUI.py ipaddress:7860 The webui has no English localization, but would be helpful. Immersive Translate Most parameters would work well with the default value. Refer to and to make changes. this this Upload these 5 files: and its main model.pt config.json and its diffusion model.pt diffusion.yaml Either cluster model for speaking or feature retrieval for singing. kmeans_10000.pt feature_and_index.pkl is for speaking only, not for singing. Recommend when using. F0 predictor RMVPE is useful when singing a feminine song using a model with a masculine voice, or vice versa. Pitch change is the way of controlling the tone. Use to get the clearest speech, and use to get the closest tone to the model. Clustering model/feature retrieval mixing ratio 0.1 0.9 should be set around , it enhances the result at steps. shallow diffusion steps 50 30-100 Audio Editing This procedure is optional. Just for the production of a better song. I won’t go into details about this since the audio editing software, or so-called DAW (digital audio workstation), that I’m using is non-free. I have no intention to advocate proprietary software even though the entire industry is paywalled and closed-source. Audacity supports multitrack, effects, and a lot more. It does load some advanced VST plugins as well. It’s not hard to find tutorials on mastering songs with Audacity. Typically, the mastering process should be mixing/balancing, EQ/compressing, reverb, and imaging. The more advanced the tool is, the easier the process will be. I’ll definitely spend more time on adopting Audacity for my mastering process in the future, and I recommend everyone do so. so-vits-svc-fork This is a so-vits-svc with real-time support, and the models are compatible. Easier to use but does not support the Diffusion model. For dedicated real-time voice changing, a is recommended. fork voice-changer Installation conda create -n so-vits-svc-fork python=3.10 pip
conda activate so-vits-svc-fork

git clone https://github.com/voicepaw/so-vits-svc-fork
cd so-vits-svc-fork

python -m pip install -U pip setuptools wheel
pip install -U torch torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install -U so-vits-svc-fork
pip install click
sudo apt-get install libportaudio2 Preparation Put dataset .wav files into so-vits-svc-fork/dataset_raw svc pre-resample
svc pre-config Edit a in . This fork is a larger size than the original. batch_size configs/44k/config.json Training svc pre-hubert
svc train -t
svc train-cluster Inference Use GUI with . This requires a local desktop environment. svcg Or use CLI with for real time and for generating. svc vc svc infer -m "logs/44k/xxxxx.pth" -c "configs/config.json" raw/xxx.wav DDSP-SVC requires fewer hardware resources and runs faster than so-vits-svc. It supports both real-time and diffusion models (Diff-SVC). DDSP-SVC conda create -n DDSP-SVC python=3.8
conda activate DDSP-SVC

git clone https://github.com/yxlllc/DDSP-SVC
cd DDSP-SVC

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt Refer to the for the two files: Initialization section pretrain/rmvpe/model.pt
pretrain/contentvec/checkpoint_best_legacy_500.pt Preparation python draw.py
python preprocess.py -c configs/combsub.yaml
python preprocess.py -c configs/diffusion-new.yaml Edit configs/ batch_size: 32  (16 for diffusion)
cache_all_data: false
cache_device: 'cuda'
cache_fp16: false Training conda activate DDSP-SVC
python train.py -c configs/combsub.yaml
python train_diff.py -c configs/diffusion-new.yaml

tensorboard --logdir=exp Inference It’s recommended to use since it includes both DDSP and diffusion model. main_diff.py python main_diff.py -i "input.wav" -diff "model_xxxxxx.pt" -o "output.wav" Real-time GUI for voice cloning: python gui_diff.py Bert-vits2-V2.3 This is a TTS tool that is completely different from everything above. By using it, I have already created several audiobooks with my voice for my parents, and they really enjoy it. Instead of using the , I used the fork by for an easier setup. original v3u Initialization conda create -n bert-vits2 python=3.9
conda activate bert-vits2

git clone https://github.com/v3ucn/Bert-vits2-V2.3.git
cd Bert-vits2-V2.3

pip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 torchaudio==2.0.2+cu118 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt Download pre-trained models (includes Chinese, Japanese, and English): wget -P slm/wavlm-base-plus/ https://huggingface.co/microsoft/wavlm-base-plus/resolve/main/pytorch_model.bin
wget -P emotional/clap-htsat-fused/ https://huggingface.co/laion/clap-htsat-fused/resolve/main/pytorch_model.bin
wget -P emotional/wav2vec2-large-robust-12-ft-emotion-msp-dim/ https://huggingface.co/audeering/wav2vec2-large-robust-12-ft-emotion-msp-dim/resolve/main/pytorch_model.bin
wget -P bert/chinese-roberta-wwm-ext-large/ https://huggingface.co/hfl/chinese-roberta-wwm-ext-large/resolve/main/pytorch_model.bin
wget -P bert/bert-base-japanese-v3/ https://huggingface.co/cl-tohoku/bert-base-japanese-v3/resolve/main/pytorch_model.bin
wget -P bert/deberta-v3-large/ https://huggingface.co/microsoft/deberta-v3-large/resolve/main/pytorch_model.bin
wget -P bert/deberta-v3-large/ https://huggingface.co/microsoft/deberta-v3-large/resolve/main/pytorch_model.generator.bin
wget -P bert/deberta-v2-large-japanese/ https://huggingface.co/ku-nlp/deberta-v2-large-japanese/resolve/main/pytorch_model.bin Create a character model folder mkdir -p Data/xxx/models/ Download base models: !wget -P Data/xxx/models/ https://huggingface.co/OedoSoldier/Bert-VITS2-2.3/resolve/main/DUR_0.pth
!wget -P Data/xxx/models/ https://huggingface.co/OedoSoldier/Bert-VITS2-2.3/resolve/main/D_0.pth
!wget -P Data/xxx/models/ https://huggingface.co/OedoSoldier/Bert-VITS2-2.3/resolve/main/G_0.pth
!wget -P Data/xxx/models/ https://huggingface.co/OedoSoldier/Bert-VITS2-2.3/resolve/main/WD_0.pth

#More options
https://openi.pcl.ac.cn/Stardust_minus/Bert-VITS2/modelmanage/model_filelist_tmpl?name=Bert-VITS2_2.3%E5%BA%95%E6%A8%A1
https://huggingface.co/Erythrocyte/bert-vits2_base_model/tree/main
https://huggingface.co/OedoSoldier/Bert-VITS2-2.3/tree/main Edit by replacing all to train_ms.py bfloat16 float16 Edit for LAN access: webui.py webbrowser.open(f"start http://localhost:7860")
app.launch(server_name="0.0.0.0", server_port=7860) Edit for and Data/xxx/config.json batch_size spk2id Preparation Similar workflow as in the . previous section Remove noise and silence, normalization, then put the un-sliced WAV file into . Data/xxx/raw Edit for , , and . config.yml dataset_path num_workers keep_ckpts Run to slice the WAV file. python3 audio_slicer.py Clean the dataset ( ) by removing small files that are under 2 sec. Data/xxx/raw Transcription Install whisper pip install git+https://github.com/openai/whisper.git To turn off language auto-detection, set it to English only, and use model; edit as below: large short_audio_transcribe.py # set the spoken language to english
    print('language: en')
    lang = 'en'
    options = whisper.DecodingOptions(language='en')
    result = whisper.decode(model, mel, options)
	
    # set to use large model
    parser.add_argument("--whisper_size", default="large")

    #Solve error "Given groups=1, weight of size [1280, 128, 3], expected input[1, 80, 3000] to have 128 channels, but got 80 channels instead" while using large model
    mel = whisper.log_mel_spectrogram(audio,n_mels = 128).to(model.device) Run to start the transcription. python3 short_audio_transcribe.py Re-sample the sliced dataset: python3 resample.py --sr 44100 --in_dir ./Data/zizek/raw/ --out_dir ./Data/zizek/wavs/ Preprocess transcription: python3 preprocess_text.py --transcription-path ./Data/zizek/esd.list Generate BERT feature config: python3 bert_gen.py --config-path ./Data/zizek/configs/config.json Training and Inference Run to start training python3 train_ms.py Edit for model path: config.yml model: "models/G_20900.pth" Run to start webui for inference python3 webui.py vits-simple-api is a web frontend for using trained models. I use this mainly for its long text support which the original project doesn’t have. vits-simple-api git clone https://github.com/Artrajz/vits-simple-api
git pull https://github.com/Artrajz/vits-simple-api
cd vits-simple-api

conda create -n vits-simple-api python=3.10 pip
conda activate vits-simple-api && 

pip install -r requirements.txt (Optional) Copy pre-trained model files from to Bert-vits2-V2.3/ vits-simple-api/bert_vits2/ Copy and to Bert-vits2-V2.3/Data/xxx/models/G_xxxxx.pth Bert-vits2-V2.3/Data/xxx/config.json vits-simple-api/Model/xxx/ Edit for and as preferred config.py MODEL_LIST Default parameter Edit as below: Model/xxx/config.json "data": {
    "training_files": "Data/train.list",
    "validation_files": "Data/val.list",
	
  "version": "2.3" Check/Edit in as model_list config.yml [xxx/G_xxxxx.pth, xxx/config.json] Run python app.py Tweaks for tone, for randomness, for pronunciation, and for speed. and are self-explanatory SDP Ratio Noise Noise_W Length emotion style Share Models In its , there are a lot of VITS models shared by others. You can try it out first, and then download the desired models from . Hugging Face repo Files The is widely used in some content creation communities because of its high quality. It contains hundreds of characters, although only Chinese and Japanese are supported. Genshin model In , there are a lot of Bert-vits2 models that are made from popular Chinese streamers and VTubers. another repo There are already projects making AI Vtubers like and . I’m looking forward to how this technology can change the industry in the near future. this this https://techshinobi.org/posts/voice-vits/https://techshinobi.org/posts/voice-vits/

How to Voice Clone With SoftVC VITS and Bert-VITS2: A Deep Dive

SoftVC VITS এবং Bert-VITS2 দিয়ে কীভাবে ভয়েস ক্লোন করবেন: একটি গভীর ডুব

So klonen Sie Stimmen mit SoftVC VITS und Bert-VITS2: Ein tiefer Einblick

Cómo clonar por voz con SoftVC VITS y Bert-VITS2: una inmersión profunda

Comment cloner la voix avec SoftVC VITS et Bert-VITS2 : une plongée approfondie

SoftVC VITS और Bert-VITS2 के साथ वॉयस क्लोन कैसे करें: एक गहन जानकारी

SoftVC VITS および Bert-VITS2 を使用して音声クローンを作成する方法: 詳細

SoftVC VITS 및 Bert-VITS2를 사용한 음성 복제 방법: 심층 분석

Como clonar voz com SoftVC VITS e Bert-VITS2: um mergulho profundo

Как озвучить клон с помощью SoftVC VITS и Bert-VITS2: глубокое погружение

SoftVC VITS ve Bert-VITS2 ile Ses Klonlama: Derinlemesine Bir İnceleme

Cách sao chép giọng nói với SoftVC VITS và Bert-VITS2: Tìm hiểu sâu

如何使用 SoftVC VITS 和 Bert-VITS2 进行语音克隆：深入探讨

Asynchronous programming has become a cornerstone of modern . In the iOS ecosystem, introduced native support for asynchronous programming with Swift, including the powerful async/await pattern. In this article, we'll delve into the usage of async/await in iOS development app development Apple Understanding async/await Async/await is a programming paradigm that streamlines the execution of asynchronous code, making it appear more synchronous and readable. It eliminates the complexities of managing callbacks or completion handlers, allowing developers to write asynchronous code that same as synchronous code. With Swift's async/await, developers can efficiently handle tasks such as network requests, database queries, or file I/O without blocking the main thread Basic Usage of async/await in iOS: To create an asynchronous function, simply use the keyword in the function declaration. This signifies that the function will execute asynchronous tasks. Define an async Function: async func fetchData() async -> Data {
    // async operations
} Within an async function, utilize the keyword to call other asynchronous functions or tasks. It suspends the current task until the awaited task is completed. Use await Inside an async Function: await func fetchImageData() async -> Data {
    let url = URL(string: "https://example.com/image.jpg")!
    do {
        let (data, _) = try await URLSession.shared.data(from: url)
        return data
    } catch {
        // Handle errors
    }
} When calling an async function from another function, mark the calling function as async as well. This facilitates the use of for calling async functions. Mark the Caller as async: await func displayImage() async {
    let imageData = await fetchImageData()
    let image = UIImage(data: imageData)
    // Update UI with the image
} Error handling with async/await is straightforward. Utilize and blocks to handle errors thrown within async functions. Error Handling: try catch func fetchImageDataWithThrowing(stringURL: URL) async throws -> Data {
    guard let url = URL(string: stringURL)
    else { throw NSError(domain: "Bad URL", code: 0) }
    
    let (data, _) = try await URLSession.shared.data(from: url)
    return data
}

func displayData() async {
    let url = "https://example.com/image.jpg"
    do {
        let data = try await fetchImageDataWithThrowing(stringURL: url)
        // Process data
    } catch {
        // Handle errors
    }
} To update the UI on the main thread within async code, use to dispatch code to the main queue. Running async Code on the Main Thread: await Task {
    await MainActor.run {
        // Update UI
    }
} or Task { @MainActor in
    // Update UI
} Advanced Use Cases: Utilize the structure to denote the scope of an asynchronous call within synchronous code. Calling an Asynchronous Function in Synchronous Code: Task func upadeImg() {
    Task {
        let image = await fetchImageData()
        MainActor.run {
            // Update UI
        }
    }
} To create an asynchronous function, simply use the construct like or . This may help you if you want to use async/await at the network layer with iOS before version 15 and after 13. Converting a Function with a Callback to Asynchronous: withCheckedContinuation withCheckedThrowingContinuation extension URLSession {
    func asyncDataRequest(url: URL) async throws -> Data {
        try await withCheckedThrowingContinuation { next in
            dataTask(with: url, completionHandler: { data, response, error in
                if let error {
                    next.resume(throwing: error)
                } else if let data {
                    next.resume(returning: data)
                } else {
                    next.resume(throwing: NSError(domain: "Empty data", code: 0))
                }
            }).resume()
        }
    }
} Use constructs like or or to execute multiple asynchronous methods concurrently, reducing lines of code and improving efficiency. Running Multiple Queries in Parallel: async let withTaskGroup withThrowingTaskGroup func fetchUser() async -> User {
    async let imageData = await fetchImageData()
    async let fullName = await fetchUserFullName()
    
    return await User(avatar: imageData, fullName: fullName)
} or withTaskGroup func fetchImages(imageID: [String]) async -> [Data] {
    await withTaskGroup(of: Data.self) { group in
        imageID.forEach { id in
            group.addTask {
                await fetchImageData(byID: id)
            }
        }
        
        return await group.reduce(into: [Data]()) { tmpResult, imageData in
            tmpResult.append(imageData)
        }
    }
} Conclusion Async/await in Swift presents a powerful and readable approach to asynchronous programming in iOS applications. By following the outlined steps, developers can simplify their asynchronous code, improve app responsiveness, and leverage Swift's robust asynchronous programming capabilities to create more efficient and responsive iOS apps

Tricks for Using Async/Await in iOS Development

In this article, we will explore the basics of setting up an extension and how to use Flutter in it. has become a very popular choice for mobile developers; its style of UI code and the fact that it renders the same on both platforms makes it a strong choice. One of the limitations Flutter did have was with iOS app extensions. Flutter If you are not aware, are a way for you to “extend” your functionality outside of your own app and allow users to interact with it through other apps on their devices. This could be the browser, notes app, or pretty much any other app. One problem with Flutter was that you would still need to develop the functionality and the UI of your extension using native code while the majority of your app was in Flutter code. This may not be a problem for some of us, but I think we can all admit that it is annoying to still have to maintain this native code. iOS app extensions Well, Flutter 3.16 changed all this, and you can now have your extension UI built with Flutter! What we will be building Our goal in this article will be rather simple: Users will select some text from anywhere on their device and share it Our app should be displayed in the list of choices in the share sheet the OS shows When the user selects our app, they should be shown a simple screen that displays the text they selected Adding a Share Extension to your Flutter app Alright, so this part is the same whether you were using Flutter or just plain iOS, so I won’t go into too much detail but just cover the overall steps you need to take: Open the file from inside your folder. This should launch Xcode. Use from Xcode’s menu, and you should be presented with something like this: Runner.xcworkspace PROJECT/ios File -> New -> Target Either find or search for and press next. Share Extension Set the product name to what you want the extension to be called and press “Finish.” After this point, you should see a new target in Xcode. Xcode will generate some boilerplate for your extension, and at this point, you should be able to test it out by running it in a simulator (or device) and trying to share something. A few gotchas If you see an error similar to when running your app navigate to your main target’s build phases and make sure the step is immediately after the step. Error (Xcode): Cycle inside Runner Embed Foundation Extensions Copy Bundle Resources If Xcode prompts you to “Activate” your extension after you create it, opt in to activate it. Opting out can result in errors that are unclear. Using Flutter UI in the iOS app extension Now that we have our extension boilerplate setup, we need to make it use Flutter for its UI instead of the Storyboard that it currently uses. First, we need to copy the Flutter framework into our extension. If you navigate to you will find a package. Copy this into the frameworks for your extension in Xcode, and make sure that the option is set to . FLUTTER_SDK_LOCATION/bin/cache/artifacts/engine/ios/extension_safe Flutter.xcframework Embed Embed & Sign Now modify the file for your extension to look similar to: info.plist tells the OS which class to use as the root UI controller for this extension. It uses the format . Note that by default there should be an entry for the main storyboard file, you will need to remove that entry to make this work. NSExtensionPrincipalClass ExtensionName.ClassName informs the OS about when this extension should be included in the share sheet. Setting it to the string makes it always be included, but you can filter this further to only be included for specific types (for example, images only). To learn more about how to configure this, visit the . NSExtensionActivationRule TRUEPREDICATE official documentation Configuring the View Controller to use Flutter Now let’s modify our to not use the boilerplate code and use Flutter instead. Replace all of its contents with this: ShareViewController import UIKit
import Flutter
import UniformTypeIdentifiers

class ShareViewController: UIViewController {
    override func viewDidLoad() {
         super.viewDidLoad()
         showFlutter()
     }

     func showFlutter() {
         let flutterViewController = FlutterViewController(
            project: nil, 
            initialRoute: "/share", 
            nibName: nil, 
            bundle: nil
          )         
         addChild(flutterViewController)
         view.addSubview(flutterViewController.view)
         flutterViewController.view.frame = view.bounds
     }
} is now a simple that loads a as its child. The property let’s us configure what route the Flutter app should open on launch, without this it would default to whatever the home or main route you have configured for your app. ShareViewController UIViewController FlutterViewController initialRoute Now let's modify our Flutter app to account for this and modify the main function to return something similar to this: return MaterialApp(
  // ...
  routes: {
    '/': (context) => const MyHomePage(title: 'Flutter Demo Home Page'),
    "/share": (_) => const ShareWidget(),
  },
); I have omitted the boilerplate that Flutter adds during app creation, but we added a route for the /share route that we configured on the Swift side of things. This will make sure that when the extension launches the Flutter engine, the user sees our share widget instead of the app's landing screen. For now, our share widget is a simple Widget with some text in the center. import 'package:flutter/material.dart';

class ShareWidget extends StatefulWidget {
  const ShareWidget({super.key});

  @override
  State<ShareWidget> createState() => _ShareWidgetState();
}

class _ShareWidgetState extends State<ShareWidget> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: Text("Hello!"),
      ),
    );
  }
} Displaying the text from the share extension in Flutter Next, we want to take the text the user shared and display it in our . First, we need to read the text in our Swift class. ShareWidget Reading shared text inside the share extension view controller if let item: NSExtensionItem = self.extensionContext?.inputItems[0] as? NSExtensionItem, let itemProvider: NSItemProvider = item.attachments?[0] {
     if itemProvider.hasItemConformingToTypeIdentifier(UTType.text.identifier) {
         itemProvider.loadItem(forTypeIdentifier: UTType.text.identifier) {result,error in
             if error == nil {
                // TODO: Send data to Flutter
             }
         }
     }
 } This code gets the text data from the and, for now, does nothing. Eventually, we will add logic here to send it to the Flutter side of things. extensionContext Note: This piece of code works for this example, but it is not the safest way of writing this. Ideally, you want to iterate all items in instead of assuming it’s always just one item. self.extensionContext?.inputItems Sending data from share extension view controller to Flutter If you have worked with passing data between Flutter and Native you know that one of the go-to methods is to use a . Let’s start by setting it up on the Swift side: MethodChannel func showFlutter() {
     let flutterViewController = FlutterViewController(project: nil, initialRoute: "/share", nibName: nil, bundle: nil)
     
     let methodChannel = FlutterMethodChannel(name: "examples.share", binaryMessenger: flutterViewController.binaryMessenger)
     
     if let item: NSExtensionItem = self.extensionContext?.inputItems[0] as? NSExtensionItem, let itemProvider: NSItemProvider = item.attachments?[0] {
         if itemProvider.hasItemConformingToTypeIdentifier(UTType.text.identifier) {
             itemProvider.loadItem(forTypeIdentifier: UTType.text.identifier) {result,error in
                 if error == nil {
                     methodChannel.invokeMethod("setData", arguments: result as! String)
                 }
             }
         }
     }
     
     addChild(flutterViewController)
     view.addSubview(flutterViewController.view)
     flutterViewController.view.frame = view.bounds
 } We create a with the name , this name is important because its value needs to match exactly with the name configured on the channel in Flutter. We then use to call a function and send the text to it as an argument. FlutterMethodChannel examples.share invokeMethod setData Next, lets modify our in Flutter: ShareWidget import 'package:flutter/material.dart';
import 'package:flutter/services.dart';

class ShareWidget extends StatefulWidget {
  // ...
}

class _ShareWidgetState extends State<ShareWidget> {
  var methodChannel = const MethodChannel('examples.share');
  var text = "";

  @override
  void initState() {
    methodChannel.setMethodCallHandler((call) async {
      if (call.method == "setData") {
        setState(() {
          text = call.arguments as String;
        });
      }
    });
    super.initState();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: Text(text),
      ),
    );
  }
} We introduced a variable in the state so we can render the text being sent from Swift. We create a method channel with the name and then we add a method call handler. This handler will get invoked every time native tries to call a method on the , but in this example, we only need to worry about the function. To keep this example simple, we just take the arguments (we know there is only one in this case) and set the value of to it. text examples.share MethodChannel setData text And that’s it! You now have a share extension set on iOS that uses Flutter for its UI instead of Swift. If you like this article, be sure to check out my other work. Also, it always helps if you share this with other people. You can also follow me on or check out . Twitter my website

Use Flutter UI inside iOS App Extensions

UVII

uviiapp.com

-- employees

Since n.d.

COMPANY RANKING

UVII

EVERGREEN INDEX #11145

Representing this company? Click here to claim and customize this page!

Uvii WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

UVII

uviiapp.com

-- employees

Since n.d.

COMPANY RANKING

UVII

EVERGREEN INDEX #11145

RELATED COMPANIESRANK

STORIES ABOUT UVII

Representing this company? Click here to claim and customize this page!

Uvii WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!