![Search icon](https://hackernoon.imgix.net/search-new.png?w=19&h=19)
Authors:
(1) Wenxuan Wang, The Chinese University of Hong Kong, Hong Kong, China;
(2) Haonan Bai, The Chinese University of Hong Kong, Hong Kong, China
(3) Jen-tse Huang, The Chinese University of Hong Kong, Hong Kong, China;
(4) Yuxuan Wan, The Chinese University of Hong Kong, Hong Kong, China;
(5) Youliang Yuan, The Chinese University of Hong Kong, Shenzhen Shenzhen, China
(6) Haoyi Qiu University of California, Los Angeles, Los Angeles, USA;
(7) Nanyun Peng, University of California, Los Angeles, Los Angeles, USA
(8) Michael Lyu, The Chinese University of Hong Kong, Hong Kong, China.
3.1 Seed Image Collection and 3.2 Neutral Prompt List Collection
3.3 Image Generation and 3.4 Properties Assessment
4.2 RQ1: Effectiveness of BiasPainter
4.3 RQ2 - Validity of Identified Biases
7 Conclusion, Data Availability, and References
To validate the effectiveness of BiasPainter and get more insights on the bias in image generation models, we use BiasPainter to test 5 commercial image generation software products and research models. In this section, we detail the evaluation process and empirically explore the following three research questions (RQs).
• RQ1: Can BiasPainter effectively measure social bias in image generation models?
• RQ2: Are the social bias found by BiasPainter valid?
• RQ3: Can BiasPainter help mitigate the bias in image generation models?
In RQ1, our goal is to investigate the effectiveness of BiasPainter in identifying and measuring social bias in image generation models. In other words, we evaluate the capability of BiasPainter in measuring the biased extent of different systems. In addition, to the best of our knowledge, BiasPainter is the first approach to reveal hidden associations between social groups and biases properties in image generation models. Therefore, we also analyze whether the results generated by BiasPainter can provide an intuitive and constructive impression of social bias in the tested systems. Since BiasPainter adopts diverse computer vision methods, which are generally imperfect (i.e., the methods may produce false positives and true negatives) [14, 26], in RQ2, we evaluate the validity of the identified bias through manual inspection. Finally, measuring the bias is the first step in mitigating the bias. Hence, in RQ3, we illustrate how BiasPainter be helpful for mitigating the bias in image generation models.
This paper is available on arxiv under CC0 1.0 DEED license.