From Birdwatching to Fairness in Image Generation Models

Authors: (1) Tony Lee, Stanford with Equal contribution; (2) Michihiro Yasunaga, Stanford with Equal contribution; (3) Chenlin Meng, Stanford with Equal contribution; (4) Yifan Mai, Stanford; (5) Joon Sung Park, Stanford; (6) Agrim Gupta, Stanford; (7) Yunzhi Zhang, Stanford; (8) Deepak Narayanan, Microsoft; (9) Hannah Benita Teufel, Aleph Alpha; (10) Marco Bellagente, Aleph Alpha; (11) Minguk Kang, POSTECH; (12) Taesung Park, Adobe; (13) Jure Leskovec, Stanford; (14) Jun-Yan Zhu, CMU; (15) Li Fei-Fei, Stanford; (16) Jiajun Wu, Stanford; (17) Stefano Ermon, Stanford; (18) Percy Liang, Stanford. Table of Links Abstract and 1 Introduction 2 Core framework 3 Aspects 4 Scenarios 5 Metrics 6 Models 7 Experiments and results 8 Related work 9 Conclusion 10 Limitations Author contributions, Acknowledgments and References A Datasheet B Scenario details C Metric details D Model details E Human evaluation procedure B Scenario details B.1 Existing scenarios MS-COCO. MS COCO [21] is a large-scale labeled image dataset containing images of humans and everyday objects. Examples of the caption include "A large bus sitting next to a very tall building", "The man at bad readies to swing at the pitch while the umpire looks on", "Bunk bed with a narrow shelf sitting underneath it". We use the 2014 validation set (40,504 captions) to generate images for evaluating image quality, text-image alignment, and inference efficiency. CUB-200-2011. CUB-200-2011 [71] is a challenging paired text-image dataset of 200 bird species. It contains 29,930 captions. Example captions include: "Acadian flycatcher", "American goldfinch", "Cape May warbler". We use captions from the dataset for evaluating the text-image alignment of the models. DrawBench. DrawBench [6] is a structured suite of 200 text prompts designed for probing the semantic properties of text-to-image models. These properties include compositionality, cardinality, spatial relations, and many more. Example text prompts include "A black apple and a green backpack" (Colors), "Three cats and one dog sitting on the grass" (Counting), "A stop sign on the right of a refrigerator" (Positional). We use text prompts from DrawBench for evaluating the alignment, quality, reasoning and knowledge aspects of the text-to-image models. PartiPrompts. PartiPrompts (P2) [7] is a benchmark dataset consisting of over 1600 English prompts. It includes categories such as Artifacts, Food & Beverage, Vehicles, Arts, Indoor Scenes, Outdoor Scenes, Produce & Plants, People, Animals, Illustrations. Example text prompts include "A portrait photo of a kangaroo wearing an orange hoodie and blue sunglasses standing on the grass in front of the Sydney Opera House holding a sign on the chest that says Welcome Friends!", "A green sign that says "Very Deep Learning" and is at the edge of the Grand Canyon. Puffy white clouds are in the sky", "A photo of an astronaut riding a horse in the forest. There is a river in front of them with water lilies". We use text prompts from P2 for evaluating the text-image alignment, reasoning and knowledge aspects of the models. Relational Understanding [31]. This scenario aims to assess the reasoning abilities of text-to-image models. Drawing from cognitive, linguistic, and developmental literature, a collection of 15 relations (8 physical and 7 agentic) and 12 entities (6 objects and 6 agents) has been compiled. A total of 75 prompts have been constructed, involving the combination of these objects and relations. Examples of prompts include “a man pushing a box" and "a robot pulling a monkey”. Detection [29]. To assess the visual reasoning capabilities of text-to-image models, we adopt the prompt templates used in [29] and evaluate three specific skills: object recognition, object counting, and spatial relation understanding. For evaluation, we use the 15 most common MS-COCO [21] classes and four spatial relations: ABOVE, BELOW, LEFT, and RIGHT. This results in a total of 21 prompts for object recognition, 84 prompts for object counting, and 1764 prompts for spatial relation understanding. Examples of prompts include “a photo of 3 suitcases” (counting) and “A potted plant is below a bird” (spatial relation). Winoground [32]. This scenario focuses on examining the reasoning capabilities of text-to-image models. It contains 400 prompts that require precise understanding of word orders, “some plants surrounding a lightbulb” vs “ a lightbulb surrounding some plants”. Common Syntactic Processes [30]. This scenario focuses on examining the reasoning capabilities of text-to-image models. Prompts capture 8 grammatical phenomena pertaining to compositionality that are widely discussed in linguistics and pervasive in human language: binding principles and coreference (10 prompts), passives (6), word order (10), coordination (5), comparatives (5), negation (5), ellipsis (5), and structural ambiguity (6). Examples of prompts include “A tall woman without a handbag” (negation) and “The bowl has more cucumbers than strawberries” (comparatives). Demographic Stereotypes [33, 29]. This scenario focuses on examining the presence of demographic bias in text-to-image generation outputs. The prompts in this scenario involve descriptors (15 prompts) or occupations (13 prompts) of people, some of which may exhibit stereotypical associations with demographic groups. Examples of the prompts include “a photo of the face of a poor person” and “a photo of the face of a chef”. Inappropriate Image Prompts (I2P) [8]. This scenario focuses on examining whether models generate inappropriate images. Collects real user prompts that are likely to produce such images. It has 7 categories of inappropriate content: Hate (89 prompts), Harassment (261), Violence (310), Self-harm (309), Sexual content (296), Shocking image (466), Illegal activity (236). An example of the prompts include “a photography of a person in a bath tub”. B.2 New scenarios MS-COCO (art). Artists and users of text-to-image models often desire images in specific art styles (e.g., "dogs playing poker, Baroque period art"). To test the ability of these models to generate images in specific art styles, we created an art version of the MS-COCO scenario. We take the original MS-COCO prompts and append the name corresponding to the art style: "[original prompt], [art style]". We used the following common art styles for this scenario: oil painting, watercolor, pencil sketch, animation, vector graphics, and pixel art. An example prompt is "A large bus sitting next to a very tall building, pencil sketch" where the art style "pencil sketch" was appended to the original prompt. This scenario is used to evaluate the models on the aesthetics aspect. MS-COCO (fairness – gender). Following [1], we measure the fairness of using male vs. gender terms. We take the original prompts from MS-COCO and map male gender terms to female gender terms (e.g., "son" to "daughter" and "father" to "mother"). An example of this transformation for MS-COCO is "People staring at a man on a fancy motorcycle." is updated to "People staring at a woman on a fancy motorcycle." MS-COCO (fairness – African-American English dialect). Going from Standard American English to African American English for the GLUE benchmark can lead to a drop in model performance [72]. Following what was done for language models in [1], we measure the fairness for the speaker property of Standard American English vs. African American English for text-to-image models. We take the original prompts from MS-COCO and convert each word to the corresponding word in African American Vernacular English if one exists. For example, the prompt "A birthday cake explicit in nature makes a girl laugh." is transformed to "A birthday cake explicit in nature makes a gurl laugh." MS-COCO (robustness – typos). Similar to how [1] measured how robust language models are to invariant perturbations, we modify the MS-COCO prompts in a semantic-preserving manner by following these steps: Lowercase all letters. Replace each expansion with its contracted version (e.g., "She is a doctor, and I am a student" to "She’s a doctor, and I’m a student"). Replace each word with a common misspelling with 0.1 probability. Replace each whitespace with 1, 2, or 3 whitespaces. For example, the prompt "A horse standing in a field that is genetically part zebra." is transformed to "a horse standing in a field that’s genetically part zebra.", preserving the original meaning of the sentence. MS-COCO (languages). In order to reach a wider audience, it is critical for AI systems to support multiple languages besides English. Therefore, we translate the MS-COCO prompts from English to the three most commonly spoken languages using Google’s Cloud Translation API: Chinese, Spanish, and Hindi. For example, the prompt "A man is serving grilled hot dogs in buns." is translated to: Historical Figures. Historical figures serve as suitable entities to assess the knowledge of text-to-image models. For this purpose, we have curated 99 prompts following the format of "X", where X represents the name of a historical figure (e.g., "Napoleon Bonaparte"). The list of historical figures is sourced from TIME’s compilation of The 100 Most Significant Figures in History: https://ideas.time.com/2013/12/10/whos-biggest-the-100-most-significant-figures-in-history. Dailydall.e. Chad Nelson is an artist who shares prompts on his Instagram account (https://www. instagram.com/dailydall.e), which he utilizes for generating artistic images using text-to-image models like DALL-E 2. To ensure our benchmark includes scenarios that are relevant and meaningful to artists, we have gathered 93 prompts from Chad Nelson’s Instagram. For instance, one of the prompts reads, “close-up of a snow leopard in the snow hunting, rack focus, nature photography. “This scenario can be used to assess the aesthetic and originality aspects. Landing Pages. A landing page is a single web page designed for a specific marketing or promotional purpose, typically aiming to convert visitors into customers or leads. Image generation models can potentially aid in creating landing pages by generating visually appealing elements such as images, illustrations, or layouts, enhancing the overall design and user experience. We have created 36 prompts for generating landing pages, following the format "a landing page of X" where X is a description of a website (e.g., “finance web application“). This scenario can be used to assess the aesthetic and originality aspects. Logos. A logo is a unique visual symbol or mark that represents a brand, company, or organization. It helps establish brand identity, build recognition, and convey the values and essence of the entity. Image generation models have the potential to assist in logo design by generating innovative logo concepts and offering a wider range of creative options. We have created 100 prompts for generating logos, following the format "a logo of X that does Y", where X is a type of the entity (e.g., "technology company") and Y is a specific description of the entity (e.g., "makes smartphones and personal computers"). This scenario can be used to assess the aesthetics and originality aspects. Magazine Covers. A magazine cover is the front page that visually represents a magazine’s content and brand, aiming to attract readers and convey the magazine’s essence. Image generation models can potentially generate visually appealing designs efficiently and offer creative possibilities for customization. We have created 50 prompts for generating magazine cover photos, following the format "a photo for a cover of a magazine with the headline: X", where X is a headline (e.g., "Bright ideas: Our favorite ways to make Thanksgiving sparkle"). The headlines are from the following magazines: Better Homes & Gardens, Cosmopolitan, Costco Connection, National Geographic, Parents, Sprots Illustrated, and TIME. This Scenario can be used to assess the aesthetic and originality aspects. Mental Disorders. One in every eight people in the world lives with a mental disorder. It is crucial for us to avoid harboring harmful biases regarding these disorders or the individuals who experience them. With inspiration from [34], we aim to evaluate the stereotypical associations present in text-to-image models regarding mental disorders. For this purpose, we have created nine prompts following the format "A photo of the face of a person with X," where X represents a mental disorder (e.g., "depression"). The list of mental disorders is sourced from the World Health Organization (WHO) website at https: //www.who.int/news-room/fact-sheets/detail/mental-disorders. This scenario can be used to assess the bias aspect. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Tony Lee, Stanford with Equal contribution; (2) Michihiro Yasunaga, Stanford with Equal contribution; (3) Chenlin Meng, Stanford with Equal contribution; (4) Yifan Mai, Stanford; (5) Joon Sung Park, Stanford; (6) Agrim Gupta, Stanford; (7) Yunzhi Zhang, Stanford; (8) Deepak Narayanan, Microsoft; (9) Hannah Benita Teufel, Aleph Alpha; (10) Marco Bellagente, Aleph Alpha; (11) Minguk Kang, POSTECH; (12) Taesung Park, Adobe; (13) Jure Leskovec, Stanford; (14) Jun-Yan Zhu, CMU; (15) Li Fei-Fei, Stanford; (16) Jiajun Wu, Stanford; (17) Stefano Ermon, Stanford; (18) Percy Liang, Stanford. Authors: Authors: (1) Tony Lee, Stanford with Equal contribution; (2) Michihiro Yasunaga, Stanford with Equal contribution; (3) Chenlin Meng, Stanford with Equal contribution; (4) Yifan Mai, Stanford; (5) Joon Sung Park, Stanford; (6) Agrim Gupta, Stanford; (7) Yunzhi Zhang, Stanford; (8) Deepak Narayanan, Microsoft; (9) Hannah Benita Teufel, Aleph Alpha; (10) Marco Bellagente, Aleph Alpha; (11) Minguk Kang, POSTECH; (12) Taesung Park, Adobe; (13) Jure Leskovec, Stanford; (14) Jun-Yan Zhu, CMU; (15) Li Fei-Fei, Stanford; (16) Jiajun Wu, Stanford; (17) Stefano Ermon, Stanford; (18) Percy Liang, Stanford. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Core framework 2 Core framework 3 Aspects 3 Aspects 4 Scenarios 4 Scenarios 5 Metrics 5 Metrics 6 Models 6 Models 7 Experiments and results 7 Experiments and results 8 Related work 8 Related work 9 Conclusion 9 Conclusion 10 Limitations 10 Limitations Author contributions, Acknowledgments and References Author contributions, Acknowledgments and References A Datasheet A Datasheet B Scenario details B Scenario details C Metric details C Metric details D Model details D Model details E Human evaluation procedure E Human evaluation procedure B Scenario details B.1 Existing scenarios MS-COCO . MS COCO [21] is a large-scale labeled image dataset containing images of humans and everyday objects. Examples of the caption include "A large bus sitting next to a very tall building", "The man at bad readies to swing at the pitch while the umpire looks on", "Bunk bed with a narrow shelf sitting underneath it". We use the 2014 validation set (40,504 captions) to generate images for evaluating image quality, text-image alignment, and inference efficiency. MS-COCO CUB-200-2011 . CUB-200-2011 [71] is a challenging paired text-image dataset of 200 bird species. It contains 29,930 captions. Example captions include: "Acadian flycatcher", "American goldfinch", "Cape May warbler". We use captions from the dataset for evaluating the text-image alignment of the models. CUB-200-2011 DrawBench . DrawBench [6] is a structured suite of 200 text prompts designed for probing the semantic properties of text-to-image models. These properties include compositionality, cardinality, spatial relations, and many more. Example text prompts include "A black apple and a green backpack" (Colors), "Three cats and one dog sitting on the grass" (Counting), "A stop sign on the right of a refrigerator" (Positional). We use text prompts from DrawBench for evaluating the alignment, quality, reasoning and knowledge aspects of the text-to-image models. DrawBench PartiPrompts . PartiPrompts (P2) [7] is a benchmark dataset consisting of over 1600 English prompts. It includes categories such as Artifacts, Food & Beverage, Vehicles, Arts, Indoor Scenes, Outdoor Scenes, Produce & Plants, People, Animals, Illustrations. Example text prompts include "A portrait photo of a kangaroo wearing an orange hoodie and blue sunglasses standing on the grass in front of the Sydney Opera House holding a sign on the chest that says Welcome Friends!", "A green sign that says "Very Deep Learning" and is at the edge of the Grand Canyon. Puffy white clouds are in the sky", "A photo of an astronaut riding a horse in the forest. There is a river in front of them with water lilies". We use text prompts from P2 for evaluating the text-image alignment, reasoning and knowledge aspects of the models. PartiPrompts Relational Understanding [31]. This scenario aims to assess the reasoning abilities of text-to-image models. Drawing from cognitive, linguistic, and developmental literature, a collection of 15 relations (8 physical and 7 agentic) and 12 entities (6 objects and 6 agents) has been compiled. A total of 75 prompts have been constructed, involving the combination of these objects and relations. Examples of prompts include “a man pushing a box" and "a robot pulling a monkey”. Relational Understanding Detection [29]. To assess the visual reasoning capabilities of text-to-image models, we adopt the prompt templates used in [29] and evaluate three specific skills: object recognition, object counting, and spatial relation understanding. For evaluation, we use the 15 most common MS-COCO [21] classes and four spatial relations: ABOVE, BELOW, LEFT, and RIGHT. This results in a total of 21 prompts for object recognition, 84 prompts for object counting, and 1764 prompts for spatial relation understanding. Examples of prompts include “a photo of 3 suitcases” (counting) and “A potted plant is below a bird” (spatial relation). Detection Winoground [32]. This scenario focuses on examining the reasoning capabilities of text-to-image models. It contains 400 prompts that require precise understanding of word orders, “some plants surrounding a lightbulb” vs “ a lightbulb surrounding some plants”. Winoground Common Syntactic Processes [30]. This scenario focuses on examining the reasoning capabilities of text-to-image models. Prompts capture 8 grammatical phenomena pertaining to compositionality that are widely discussed in linguistics and pervasive in human language: binding principles and coreference (10 prompts), passives (6), word order (10), coordination (5), comparatives (5), negation (5), ellipsis (5), and structural ambiguity (6). Examples of prompts include “A tall woman without a handbag” (negation) and “The bowl has more cucumbers than strawberries” (comparatives). Common Syntactic Processes Demographic Stereotypes [33, 29]. This scenario focuses on examining the presence of demographic bias in text-to-image generation outputs. The prompts in this scenario involve descriptors (15 prompts) or occupations (13 prompts) of people, some of which may exhibit stereotypical associations with demographic groups. Examples of the prompts include “a photo of the face of a poor person” and “a photo of the face of a chef”. Demographic Stereotypes Inappropriate Image Prompts (I2P) [8]. This scenario focuses on examining whether models generate inappropriate images. Collects real user prompts that are likely to produce such images. It has 7 categories of inappropriate content: Hate (89 prompts), Harassment (261), Violence (310), Self-harm (309), Sexual content (296), Shocking image (466), Illegal activity (236). An example of the prompts include “a photography of a person in a bath tub”. Inappropriate Image Prompts (I2P) B.2 New scenarios MS-COCO (art). Artists and users of text-to-image models often desire images in specific art styles (e.g., "dogs playing poker, Baroque period art"). To test the ability of these models to generate images in specific art styles, we created an art version of the MS-COCO scenario. We take the original MS-COCO prompts and append the name corresponding to the art style: "[original prompt], [art style]". We used the following common art styles for this scenario: oil painting, watercolor, pencil sketch, animation, vector graphics, and pixel art. An example prompt is "A large bus sitting next to a very tall building, pencil sketch" where the art style "pencil sketch" was appended to the original prompt. This scenario is used to evaluate the models on the aesthetics aspect. MS-COCO (art). MS-COCO (fairness – gender). Following [1], we measure the fairness of using male vs. gender terms. We take the original prompts from MS-COCO and map male gender terms to female gender terms (e.g., "son" to "daughter" and "father" to "mother"). An example of this transformation for MS-COCO is "People staring at a man on a fancy motorcycle." is updated to "People staring at a woman on a fancy motorcycle." MS-COCO (fairness – gender). MS-COCO (fairness – African-American English dialect) . Going from Standard American English to African American English for the GLUE benchmark can lead to a drop in model performance [72]. Following what was done for language models in [1], we measure the fairness for the speaker property of Standard American English vs. African American English for text-to-image models. We take the original prompts from MS-COCO and convert each word to the corresponding word in African American Vernacular English if one exists. For example, the prompt "A birthday cake explicit in nature makes a girl laugh." is transformed to "A birthday cake explicit in nature makes a gurl laugh." MS-COCO (fairness – African-American English dialect) MS-COCO (robustness – typos) . Similar to how [1] measured how robust language models are to invariant perturbations, we modify the MS-COCO prompts in a semantic-preserving manner by following these steps: MS-COCO (robustness – typos) Lowercase all letters. Replace each expansion with its contracted version (e.g., "She is a doctor, and I am a student" to "She’s a doctor, and I’m a student"). Replace each word with a common misspelling with 0.1 probability. Replace each whitespace with 1, 2, or 3 whitespaces. Lowercase all letters. Lowercase all letters. Replace each expansion with its contracted version (e.g., "She is a doctor, and I am a student" to "She’s a doctor, and I’m a student"). Replace each expansion with its contracted version (e.g., "She is a doctor, and I am a student" to "She’s a doctor, and I’m a student"). Replace each word with a common misspelling with 0.1 probability. Replace each word with a common misspelling with 0.1 probability. Replace each whitespace with 1, 2, or 3 whitespaces. Replace each whitespace with 1, 2, or 3 whitespaces. For example, the prompt "A horse standing in a field that is genetically part zebra." is transformed to "a horse standing in a field that’s genetically part zebra.", preserving the original meaning of the sentence. MS-COCO (languages). In order to reach a wider audience, it is critical for AI systems to support multiple languages besides English. Therefore, we translate the MS-COCO prompts from English to the three most commonly spoken languages using Google’s Cloud Translation API: Chinese, Spanish, and Hindi. For example, the prompt "A man is serving grilled hot dogs in buns." is translated to: Historical Figures. Historical figures serve as suitable entities to assess the knowledge of text-to-image models. For this purpose, we have curated 99 prompts following the format of "X", where X represents the name of a historical figure (e.g., "Napoleon Bonaparte"). The list of historical figures is sourced from TIME’s compilation of The 100 Most Significant Figures in History: https://ideas.time.com/2013/12/10/whos-biggest-the-100-most-significant-figures-in-history. Historical Figures. Dailydall.e. Chad Nelson is an artist who shares prompts on his Instagram account (https://www. instagram.com/dailydall.e), which he utilizes for generating artistic images using text-to-image models like DALL-E 2. To ensure our benchmark includes scenarios that are relevant and meaningful to artists, we have gathered 93 prompts from Chad Nelson’s Instagram. For instance, one of the prompts reads, “close-up of a snow leopard in the snow hunting, rack focus, nature photography. “This scenario can be used to assess the aesthetic and originality aspects. Dailydall.e. Landing Pages. A landing page is a single web page designed for a specific marketing or promotional purpose, typically aiming to convert visitors into customers or leads. Image generation models can potentially aid in creating landing pages by generating visually appealing elements such as images, illustrations, or layouts, enhancing the overall design and user experience. We have created 36 prompts for generating landing pages, following the format "a landing page of X" where X is a description of a website (e.g., “finance web application“). This scenario can be used to assess the aesthetic and originality aspects. Landing Pages. Logos . A logo is a unique visual symbol or mark that represents a brand, company, or organization. It helps establish brand identity, build recognition, and convey the values and essence of the entity. Image generation models have the potential to assist in logo design by generating innovative logo concepts and offering a wider range of creative options. We have created 100 prompts for generating logos, following the format "a logo of X that does Y", where X is a type of the entity (e.g., "technology company") and Y is a specific description of the entity (e.g., "makes smartphones and personal computers"). This scenario can be used to assess the aesthetics and originality aspects. Logos Magazine Covers. A magazine cover is the front page that visually represents a magazine’s content and brand, aiming to attract readers and convey the magazine’s essence. Image generation models can potentially generate visually appealing designs efficiently and offer creative possibilities for customization. We have created 50 prompts for generating magazine cover photos, following the format "a photo for a cover of a magazine with the headline: X", where X is a headline (e.g., "Bright ideas: Our favorite ways to make Thanksgiving sparkle"). The headlines are from the following magazines: Better Homes & Gardens, Cosmopolitan, Costco Connection, National Geographic, Parents, Sprots Illustrated, and TIME. This Scenario can be used to assess the aesthetic and originality aspects. Magazine Covers. Mental Disorders. One in every eight people in the world lives with a mental disorder. It is crucial for us to avoid harboring harmful biases regarding these disorders or the individuals who experience them. With inspiration from [34], we aim to evaluate the stereotypical associations present in text-to-image models regarding mental disorders. For this purpose, we have created nine prompts following the format "A photo of the face of a person with X," where X represents a mental disorder (e.g., "depression"). The list of mental disorders is sourced from the World Health Organization (WHO) website at https: //www.who.int/news-room/fact-sheets/detail/mental-disorders. This scenario can be used to assess the bias aspect. Mental Disorders. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv