Authors:
(1) Evan Shieh, Young Data Scientists League (evan.shieh@youngdatascientists.org);
(2) Faye-Marie Vassel, Stanford University;
(3) Cassidy Sugimoto, School of Public Policy, Georgia Institute of Technology;
(4) Thema Monroe-White, Schar School of Policy and Government & Department of Computer Science, George Mason University (tmonroew@gmu.edu). Table of Links Abstract and 1 Introduction 1.1 Related Work and Contributions 2 Methods and Data Collection 2.1 Textual Identity Proxies and Socio-Psychological Harms 2.2 Modeling Gender, Sexual Orientation, and Race 3 Analysis 3.1 Harms of Omission 3.2 Harms of Subordination 3.3 Harms of Stereotyping 4 Discussion, Acknowledgements, and References SUPPLEMENTAL MATERIALS A OPERATIONALIZING POWER AND INTERSECTIONALITY B EXTENDED TECHNICAL DETAILS B.1 Modeling Gender and Sexual Orientation B.2 Modeling Race B.3 Automated Data Mining of Textual Cues B.4 Representation Ratio B.5 Subordination Ratio B.6 Median Racialized Subordination Ratio B.7 Extended Cues for Stereotype Analysis B.8 Statistical Methods C ADDITIONAL EXAMPLES C.1 Most Common Names Generated by LM per Race C.2 Additional Selected Examples of Full Synthetic Texts D DATASHEET AND PUBLIC USE DISCLOSURES D.1 Datasheet for Laissez-Faire Prompts Dataset B.5 Subordination Ratio For power-laden prompts, we define the subordination ratio as the proportion of a demographic observed in the subordinate role compared to the dominant role: This allows us to focus on relative differences in the portrayal of characters when power-laden prompts are introduced. If the subordination ratio is less than 1, we observe dominance; if the subordination ratio is greater than 1, we observe subordination; and if the subordination ratio is 1, then the demographic is neutral (independent from power dynamics). B.6 Median Racialized Subordination Ratio Characters generated by the language models bear names with low racial likelihood for all races except White (as shown in Fig. 3a). Therefore, overall subordination will be predominantly influenced by the large volume of Anglicized names. We are more interested in examining how the subordination ratio changes as the threshold varies. If no subordination effect exists, the null hypothesis states that thresholding would not have an impact and the ratio should concentrate around 1 given sufficient sample size. However, setting a “one-size-fits-all” threshold is inherently subjective, and would fail to take into account real differences that affect name distributions between racial groups, including historical differences in periods of migration and assimilation (voluntary, involuntary, or restricted) [33, 37]. To address this, we introduce the median racialized subordination ratio to quantify subordination across a range of possible racial thresholds. First, we control for possible confounding effects of textual cues beyond name by conditioning on gender references (pronouns, titles, etc.). Then, for each intersection of race and gender we take the median of all subordination ratios for names above a variable likelihood threshold t as defined below: With sufficiently granular t, this statistic measures subordination while taking the spectrum of racial likelihoods into account. For our experiments, we set t ∈ [1, 2, … 100]. Using the median controls for possible extremes; however, we nevertheless observe astonishingly high rates of subordination (see Fig. 3c) even though this approach conservatively underestimates perceived subordination (or domination) for racial groups with a high proportion of Anglicized names (e.g., Black names [55]). This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Evan Shieh, Young Data Scientists League (evan.shieh@youngdatascientists.org); (2) Faye-Marie Vassel, Stanford University; (3) Cassidy Sugimoto, School of Public Policy, Georgia Institute of Technology; (4) Thema Monroe-White, Schar School of Policy and Government & Department of Computer Science, George Mason University (tmonroew@gmu.edu). Authors: Authors: (1) Evan Shieh, Young Data Scientists League (evan.shieh@youngdatascientists.org); (2) Faye-Marie Vassel, Stanford University; (3) Cassidy Sugimoto, School of Public Policy, Georgia Institute of Technology; (4) Thema Monroe-White, Schar School of Policy and Government & Department of Computer Science, George Mason University (tmonroew@gmu.edu). Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 1.1 Related Work and Contributions 1.1 Related Work and Contributions 2 Methods and Data Collection 2 Methods and Data Collection 2.1 Textual Identity Proxies and Socio-Psychological Harms 2.1 Textual Identity Proxies and Socio-Psychological Harms 2.2 Modeling Gender, Sexual Orientation, and Race 2.2 Modeling Gender, Sexual Orientation, and Race 3 Analysis 3.1 Harms of Omission 3.1 Harms of Omission 3.2 Harms of Subordination 3.2 Harms of Subordination 3.3 Harms of Stereotyping 3.3 Harms of Stereotyping 4 Discussion, Acknowledgements, and References 4 Discussion, Acknowledgements, and References SUPPLEMENTAL MATERIALS SUPPLEMENTAL MATERIALS A OPERATIONALIZING POWER AND INTERSECTIONALITY A OPERATIONALIZING POWER AND INTERSECTIONALITY B EXTENDED TECHNICAL DETAILS B.1 Modeling Gender and Sexual Orientation B.1 Modeling Gender and Sexual Orientation B.2 Modeling Race B.2 Modeling Race B.3 Automated Data Mining of Textual Cues B.3 Automated Data Mining of Textual Cues B.4 Representation Ratio B.4 Representation Ratio B.5 Subordination Ratio B.5 Subordination Ratio B.6 Median Racialized Subordination Ratio B.6 Median Racialized Subordination Ratio B.7 Extended Cues for Stereotype Analysis B.7 Extended Cues for Stereotype Analysis B.8 Statistical Methods B.8 Statistical Methods C ADDITIONAL EXAMPLES C.1 Most Common Names Generated by LM per Race C.1 Most Common Names Generated by LM per Race C.2 Additional Selected Examples of Full Synthetic Texts C.2 Additional Selected Examples of Full Synthetic Texts D DATASHEET AND PUBLIC USE DISCLOSURES D.1 Datasheet for Laissez-Faire Prompts Dataset D.1 Datasheet for Laissez-Faire Prompts Dataset B.5 Subordination Ratio For power-laden prompts, we define the subordination ratio as the proportion of a demographic observed in the subordinate role compared to the dominant role: subordination ratio This allows us to focus on relative differences in the portrayal of characters when power-laden prompts are introduced. If the subordination ratio is less than 1, we observe dominance; if the subordination ratio is greater than 1, we observe subordination; and if the subordination ratio is 1, then the demographic is neutral (independent from power dynamics). B.6 Median Racialized Subordination Ratio Characters generated by the language models bear names with low racial likelihood for all races except White (as shown in Fig. 3a). Therefore, overall subordination will be predominantly influenced by the large volume of Anglicized names. We are more interested in examining how the subordination ratio changes as the threshold varies. If no subordination effect exists, the null hypothesis states that thresholding would not have an impact and the ratio should concentrate around 1 given sufficient sample size. However, setting a “one-size-fits-all” threshold is inherently subjective, and would fail to take into account real differences that affect name distributions between racial groups, including historical differences in periods of migration and assimilation (voluntary, involuntary, or restricted) [33, 37]. To address this, we introduce the median racialized subordination ratio to quantify subordination across a range of possible racial thresholds. First, we control for possible confounding effects of textual cues beyond name by conditioning on gender references (pronouns, titles, etc.). Then, for each intersection of race and gender we take the median of all subordination ratios for names above a variable likelihood threshold t as defined below: median racialized subordination ratio With sufficiently granular t, this statistic measures subordination while taking the spectrum of racial likelihoods into account. For our experiments, we set t ∈ [1, 2, … 100]. Using the median controls for possible extremes; however, we nevertheless observe astonishingly high rates of subordination (see Fig. 3c) even though this approach conservatively underestimates perceived subordination (or domination) for racial groups with a high proportion of Anglicized names (e.g., Black names [55]). This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

How Do AI Models Portray Power Dynamics?

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A Guide to Exploiting Unknown Strategies From a Known Bias Set

New Study Reveals Subtle but Widespread AI Discrimination

Exploring Social Bias in AI Through 50 Real-World Scenarios

How AI Texts Reinforce Harmful Stereotypes—Even Without Asking

The Omission Bias in LMs Is Worse Than You Think

How AI Assigns Power Based on Race and Gender

A Guide to Exploiting Unknown Strategies From a Known Bias Set

New Study Reveals Subtle but Widespread AI Discrimination

Exploring Social Bias in AI Through 50 Real-World Scenarios

How AI Texts Reinforce Harmful Stereotypes—Even Without Asking

The Omission Bias in LMs Is Worse Than You Think

How AI Assigns Power Based on Race and Gender

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps