Difoosion - A Simple Web-Interface for Stable Diffusion Models

With Stable Diffusion 3 freshly released, I thought it would be nice to make a simple Web-Interface for it. About a month ago, me and a couple of friends released a library for making websites with Python, and this just seemed like too good of an opportunity to pass up.

The interface includes support for the classic SD1, SDXL, and the new SD3. Inference is done using the diffusers library from Huggingface, and the interface is built with Rio. Given how new SD3 is in diffusers, you'll spot the odd issue here and there, but I'm sure that'll improve now with thousands of people using it.

This is a simple Web-Interface for Stable Diffusion, based on our new Rio framework. We've gotten a lot of requests for larger sample projects, so given the highly anticipated release of Stable Diffusion 3, I figured this would be a great example. With all of that out of the way, let's get started! Install at least one model from the list below to begin using Difoosion. Rio: https://github.com/rio-labs/rio

This makes use of Stable Diffusion, which is a sizable AI model. While it can technically run on the CPU, an Nvidia GPU with at least 8GB of VRAM is highly recommended. You can find installation instructions in the project's README: https://gitlab.com/mad-moo/difoosion

Get Started

1. Clone this repository

git clone [email protected]:Vivern/difoosion.git

2. Install the project and its dependencies (Use a venv if you like)

cd difoosion
python -m pip install -e .

This will download some large packages like diffusers and torch. Give it a few minutes to complete.

3. Run the interface

python -m rio run --release

After a few seconds, this will display a URL in the console. Open this URL in your browser to start using the interface.

4. Install Models

When you first connect, you won't have any models installed yet. The site will automatically allow you to download one of the built-in ones.

diffusers doesn't seem to offer any way to track the download progress of models. That's why you won't be able to see how far the download has progressed in the interface. Check the terminal to see updates.

5. Get cracking!

Type a prompt of what you'd like to see, click "Generate," and wait for your masterpiece to be created :)

Tech Stack

• All models are run with the diffusers library from huggingface. There is a thin wrapper around the library to workaround some issues and make it easier to use.

• The web interface is built with Rio, a new web framework that is designed to be 100% Python, and as easy to use as possible. I'm one of the Rio developers, and this project was meant to both test Rio out in a bigger project, as well as be a showcase & reference to other Rio developers.

• Dependencies are managed with rye.

Caveats

Stable Diffusion 3 is still very new, and support in diffusers immature. This has led to a couple of known problems:

• Generating non-square images can lead to errors from pytorch. Stick to aspect ratios of 1:1 for now.

• Freeing up memory is very unreliable. On some of my devices, the model keeps gobbling up more and more RAM as time goes on, while on others, it seems to free up just fine. Keep a close eye on your memory usage.