To asses the generation quality of SDXL we perform a user study against the state of the art text-toimage generation platform Midjourney[1]. As the source for image captions we use the PartiPrompts (P2) benchmark [53], that was introduced to compare large text-to-image model on various challenging prompts.

For our study, we choose five random prompts from each category, and generate four 1024 × 1024 images by both Midjourney (v5.1, with a set seed of 2) and SDXL for each prompt. These images were then presented to the AWS GroundTruth taskforce, who voted based on adherence to the prompt. The results of these votes are illustrated in Fig. 9. Overall, there is a slight preferance for SDXL over Midjourney in terms of prompt adherence.

E.2 Category & challenge comparisons on PartiPrompts (P2)

Each prompt from the P2 benchmark is organized into a category and a challenge, each focus on different difficult aspects of the generation process. We show the comparisons for each category (Fig. 10) and challenge (Fig. 11) of P2 below. In four out of six categories SDXL outperforms Midjourney, and in seven out of ten challenges there is no significant difference between both models or SDXL outperforms Midjourney.

Figure 10: User preference comparison of SDXL (without refinement model) and Midjourney V5.1 across particular text categories. SDXL outperforms Midjourney V5.1 in all but two categories.

Figure 11: Preference comparisons of SDXL (with refinement model) to Midjourney V5.1 on complex prompts. SDXL either outperforms or is statistically equal to Midjourney V5.1 in 7 out of 10 categories.

This paper is available on arxiv under CC BY 4.0 DEED license.

[1] We compare against v5.1 since that was the best version available at that time.

Comparison to Midjourney v5.1

About Author

Topics

Around The Web

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps