From Bias to Hate: Why AI Governance Must Address Digital Violence Against Women of Color

Artificial intelligence has become remarkably good at generating images. In a matter of seconds, users can create portraits, advertisements, fantasy worlds, historical scenes, and increasingly, representations of people and communities. These images circulate across social media platforms, search engines, educational settings, workplaces, and political discourse, shaping how individuals understand themselves and others. As generative AI becomes embedded in everyday digital life, questions of representation have become questions of power. Growing concerns have emerged about the social consequences of algorithmic systems, particularly for communities that have long experienced unequal forms of representation and discrimination.

Yet despite growing awareness of algorithmic bias, I am increasingly concerned that contemporary discussions of AI governance remain too narrowly focused on technical performance. Public conversations often ask whether a system is accurate, transparent, explainable, or fair. These questions are important, but they do not fully capture how algorithmic harms are experienced by those most affected by them. 

In this essay, I argue that effective AI governance must move beyond the identification of bias and grapple more directly with its consequences. While scholars have extensively documented racial and gender disparities in AI systems, governance frameworks have been slower to address how those disparities contribute to broader forms of digital violence against women of color. Understanding this relationship is essential if we are to develop approaches to AI that prioritize not only fairness and accountability, but also dignity, safety, and justice.

What Decades of Scholarship Have Already Taught Us About AI Bias

The contemporary conversation about AI governance did not emerge in a vacuum. Long before generative AI entered mainstream public discourse, scholars in critical race studies, feminist media studies, science and technology studies, and digital humanities were documenting how technological systems reproduce existing social inequalities. Their work challenged the persistent belief that computational systems operate independently of human values and social power.

Among the most influential contributions is Safiya Umoja Noble's Algorithms of Oppression (2018), which demonstrated how search engines can reinforce racist and sexist hierarchies through seemingly neutral processes of ranking and visibility. Noble showed that digital platforms do not simply organize information; they participate in the production of knowledge itself, shaping how communities are represented and understood. Her work was an important reminder that technological infrastructures are also cultural infrastructures.

Similarly, in Race After Technology: Abolitionist Tools for the New Jim Code, Ruha Benjamin argues that technologies frequently reproduce racial inequality under the guise of objectivity and innovation. Through the concept of the "New Jim Code," she demonstrates how automated systems can reinforce historical forms of exclusion while appearing neutral, efficient, or scientifically grounded. Empirical studies have further confirmed these concerns. In their landmark Gender Shades project, Joy Buolamwini and Timnit Gebru (2018) found significant disparities in the performance of commercial facial recognition systems, with the highest error rates occurring for darker-skinned women. Their findings became a defining example of how race and gender intersect within algorithmic systems and helped bring questions of AI accountability into public policy discussions.

In Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor, Virginia Eubanks demonstrates how automated decision-making systems can disproportionately harm marginalized communities through public institutions, particularly in areas such as welfare administration, housing, and social services. She shows that algorithmic inequality often emerges not from isolated technical failures but from broader political and institutional structures. 

Together, this scholarship has fundamentally transformed how we understand artificial intelligence. The central question is no longer whether AI systems can reproduce racial and gender inequalities. The evidence is overwhelming that they can and often do. What remains less fully explored, however, is how these biased outputs become part of broader ecosystems of digital harm. While scholars have convincingly demonstrated that AI systems reproduce inequality, we must also examine how these representations circulate across digital platforms, interact with existing cultures of harassment and discrimination, and contribute to the everyday experiences of women of color online.

It is here that the concept of digital violence becomes particularly important. Bias may describe how a system operates, but it does not fully capture how those operations are experienced by the individuals and communities most affected by them. 

When Bias Becomes Digital Violence

One limitation of contemporary AI governance is that it often treats bias as a problem contained within the system itself. The goal becomes identifying disparities, improving datasets, increasing accuracy, or reducing measurable forms of discrimination. While these interventions are important, they can obscure a larger question: What happens after biased outputs enter the world?

This question is particularly important for women of color. The harms associated with AI are often discussed as technical failures, but they are experienced socially. They shape who is visible, who is desirable, who is credible, who is threatening, and ultimately who is afforded dignity within digital spaces. The consequences of algorithmic bias therefore cannot be separated from the broader social environments in which these systems operate.

 As a scholar of race, gender, and visual and digital culture, I am especially interested in how AI systems participate in the production of representation. Generative AI models do not simply retrieve information. They generate images, stories, voices, and narratives that increasingly circulate across social media platforms, news environments, classrooms, workplaces, and public discourse. In doing so, they help shape collective understandings of identity and difference. The question is not merely whether these representations are accurate. The question is what forms of power they reproduce.

 For women of color, this issue is particularly urgent because digital representation has never been politically neutral. In Misogynoir Transformed: Black Women's Digital Resistance, Moya Bailey demonstrates how racism and sexism intersect to produce distinct forms of hostility toward Black women online. These harms are not simply additive. They emerge through the specific ways Black women are stereotyped, hypersexualized, surveilled, and targeted within digital culture.Similar dynamics affect Indigenous women, Latina women, Asian women, and Muslim women whose identities are frequently reduced to racialized and gendered stereotypes.

Recent developments in generative AI demonstrate how these harms can be amplified at unprecedented scale. One of the clearest examples is the proliferation of AI-generated sexual deepfakes. As reported by Reuters, British Member of Parliament Jess Asato filed a lawsuit against xAI in 2026 after users allegedly generated sexualized images of her using Grok, arguing that the platform facilitated a form of gendered abuse that extended beyond questions of technical performance. The issue was not whether the system had accurately represented reality. The images were understood to be fabricated. The harm emerged through their circulation and their capacity to humiliate, intimidate, and undermine women's participation in public life.

 Similar concerns have emerged around so-called AI "undressing" applications, which use generative AI to create nude images from ordinary photographs without consent. As reported by The Verge, the City of San Francisco filed lawsuits against several of these platforms in 2024, arguing that they facilitated large-scale sexual exploitation and harassment. These systems attracted millions of users despite widespread concerns about their impact on women and girls. Again, the central issue was not algorithmic accuracy. It was the use of AI technologies to produce and distribute gendered harm.

 Race is equally central to these conversations. In recent years, journalists and scholars have documented the rise of AI-generated Black influencers and virtual personas used to market products across social media platforms. While often presented as innovative forms of digital entrepreneurship, these synthetic identities raise troubling questions about the commodification of Blackness, the extraction of cultural aesthetics, and the replacement of human creators with algorithmically generated representations. These developments echo Lisa Nakamura's arguments in Digitizing Race: Visual Cultures of the Internet, which demonstrates how digital technologies often reproduce, rather than transcend, racial formations. As reported by The Verge, the growing use of AI-generated Black influencers illustrates how these concerns are re-emerging in the era of generative AI.

 Taken together, these examples suggest that the harms facing women of color cannot be fully understood through the language of bias alone. Bias describes a condition within a system. Digital violence describes how that condition is experienced within social life. When AI-generated representations reinforce stereotypes, facilitate harassment, enable sexual exploitation, or commodify racial identities, the resulting harm extends beyond technical malfunction. These technologies become participants in broader systems of inequality.

From this perspective, the challenge for governance is not simply reducing disparities in algorithmic performance. It is confronting the ways AI systems can amplify existing structures of racism and sexism while creating new avenues through which those harms circulate. If women of color continue to experience AI primarily through misrepresentation, harassment, and exclusion, then governance frameworks that focus solely on fairness metrics will remain inadequate to the realities they seek to address.

 From Fairness to Justice

 The next challenge for AI governance is no longer demonstrating that algorithmic bias exists. The more urgent question is how governance frameworks can respond to the social consequences of AI systems that shape representation, visibility, participation, and safety for women of color. If governance remains focused primarily on technical performance, it will continue to overlook the broader structures through which digital violence is produced and sustained.

 As generative AI becomes increasingly integrated into social media platforms, search engines, workplaces, schools, healthcare systems, and government institutions, the stakes of these questions will only continue to grow. AI systems are helping produce culture. They influence how identities are represented, how communities are perceived, and whose voices are amplified or marginalized within digital environments. Governance frameworks that focus exclusively on accuracy, transparency, or fairness metrics risk overlooking these broader social consequences.

This is particularly concerning because women of color often occupy positions where multiple forms of inequality intersect. Their experiences reveal how algorithmic harms operate not only through technical systems but through existing structures of racism, sexism, economic inequality, and platform power. Governance frameworks that ignore these intersecting realities risk protecting technical systems while leaving intact the social conditions that enable digital violence to persist.

 What would it mean to take these harms seriously? First, AI governance frameworks must expand beyond technical audits of accuracy, transparency, and fairness to include systematic assessments of representational harm and digital violence. Governments, regulatory agencies, standards organizations, and independent auditors should evaluate not only whether AI systems produce statistically biased outcomes, but also how their outputs shape public narratives, reinforce harmful stereotypes, facilitate harassment, or normalize discriminatory representations. Incorporating these broader social impact assessments would allow governance to better reflect the lived experiences of communities most affected by algorithmic systems rather than limiting accountability to technical performance alone. 

 Second, women of color and other communities disproportionately affected by algorithmic systems should be recognized as active participants in AI governance rather than passive subjects of evaluation. Technology companies, public institutions, and policymakers should create mechanisms for sustained community consultation throughout the design, deployment, and oversight of AI systems. Including those with lived experience of algorithmic harm would strengthen governance by ensuring that policies respond not only to measurable technical risks but also to the social realities of exclusion, surveillance, and digital violence.  

 Finally, policymakers and technology companies must recognize that AI governance is fundamentally a question of democratic accountability and human rights, not solely one of technological innovation. Regulatory frameworks should therefore establish clear standards for transparency, meaningful avenues for redress when harms occur, and stronger protections against the production and circulation of AI-enabled gendered and racialized abuse. Reframing governance in this way shifts the objective from simply reducing technical errors to creating digital environments that protect dignity, participation, and justice for all communities. The future of AI governance should be measured not only by increasingly accurate systems, but also by their ability to contribute to more equitable digital environments.

 For women of color, the ultimate goal is creating technological futures where safety, representation, and human dignity are embedded as fundamental design principles. Moving from bias to justice requires recognizing that algorithmic systems do not operate apart from society but actively participate in shaping it. If AI governance is to meet the challenges posed by generative technologies, it must confront not only biased systems but also the broader structures of digital violence through which inequality is reproduced. Only then can governance frameworks begin to protect not simply fairness in computation but justice in the lived experiences of those most affected by AI.

Funding Disclosure

No external funding was received for the preparation of this manuscript.

Conflicts of Interest

The author declares no conflicts of interest.

Relevant Institutional or Advisory Roles
The author serves as a Faculty Associate within the Center for Excellence in Teaching, Learning, and Online Education (CETLOE) at Georgia State University, where she supports faculty development in the use of artificial intelligence in higher education. This role did not influence the preparation or conclusions of this manuscript.

AI/LLM Disclosure

Large language model (LLM) tools were used solely to identify open access scholarly sources to improve accessibility. No AI tools were used to generate the analysis, arguments, or conclusions presented in this manuscript.