118 reads

A Comprehensive Evaluation of 26 State-of-the-Art Text-to-Image Models

by Auto Encoder: How to Ignore the Signal NoiseOctober 12th, 2024

Too Long; Didn't Read

We evaluate 26 recent text-to-image models, spanning diffusion, autoregressive, and GAN types, with sizes from 0.4B to 13B parameters. The models are compared based on their organizations, accessibility (open or closed), and default inference configurations from APIs, GitHub, or Hugging Face repositories. Table 4 summarizes key model properties for a clear comparison.

featured image - A Comprehensive Evaluation of 26 State-of-the-Art Text-to-Image Models

Authors:

(1) Tony Lee, Stanford with Equal contribution;

(2) Michihiro Yasunaga, Stanford with Equal contribution;

(3) Chenlin Meng, Stanford with Equal contribution;

(4) Yifan Mai, Stanford;

(5) Joon Sung Park, Stanford;

(6) Agrim Gupta, Stanford;

(7) Yunzhi Zhang, Stanford;

(8) Deepak Narayanan, Microsoft;

(9) Hannah Benita Teufel, Aleph Alpha;

(10) Marco Bellagente, Aleph Alpha;

(11) Minguk Kang, POSTECH;

(12) Taesung Park, Adobe;

(13) Jure Leskovec, Stanford;

(14) Jun-Yan Zhu, CMU;

(15) Li Fei-Fei, Stanford;

(16) Jiajun Wu, Stanford;

(17) Stefano Ermon, Stanford;

(18) Percy Liang, Stanford.

Table of Links

Abstract and 1 Introduction

7 Experiments and results

8 Related work

9 Conclusion

10 Limitations

Author contributions, Acknowledgments and References

E Human evaluation procedure

6 Models

We evaluate 26 recent text-to-image models, encompassing various types (e.g., diffusion, autoregressive, GAN), sizes (ranging from 0.4B to 13B parameters), organizations, and accessibility (open or closed). Table 4 presents an overview of the models and their corresponding properties. In our evaluation, we employ the default inference configurations provided in the respective model’s API, GitHub, or Hugging Face repositories.