AI Bias: Intelligence Without Imagination
I’ve avoided writing this article for two reasons. First, AI bias is a sensitive topic requiring earnest contemplation. If you’ve read my other articles, you know that’s not always my style. Second, it feels slightly hypocritical to write an article about bias when your last name is “Whiteman.”
Why write one now? It’s partly because I’ve struggled to find a satisfying perspective on AI bias. Conventional wisdom seems to be that “AI training data is biased, and we risk perpetuating those biases if we deploy AI into society.” That’s not how I think about the problem.
However, my primary motivation is that companies are plowing ahead despite the risks. A McKinsey survey found that 72 percent of organizations had adopted AI in at least one business function. Generative AI has been a shot in the arm after years of stagnation.
Source: McKinsey
It hasn’t all been smooth sailing. The issue du jour is hallucinations — also referred to as “inaccuracy.” I’ve written about that topic already. I view AI bias as the other side of the hallucination coin. It’s misguided to talk about one without considering the implications for the other.
Accuracy and bias used to go hand-in-hand. Leaders wanted AI systems to be reliable AND fair. That’s still true of accuracy. McKinsey found that 38 percent of organizations are working to mitigate inaccuracy, up from 32 percent in 2023. Meanwhile, only 12 percent are working on equity and fairness, down from 16 percent a year earlier.
Source: McKinsey
That must mean the AI bias problem is solved, right? Let’s try a simple experiment to be sure. I’ll ask ChatGPT to generate five images of an S&P 500 CEO and five of a nurse. I can’t wait to see the equity and fairness advances AI companies have made.
Source: ChatGPT (prompt: “Create a portrait of a [S&P500 CEO / Nurse]”)
Well, this is awkward. Those people even have the same hairstyle. So why are companies accelerating AI adoption with little regard for bias?
You know the answer. These are the same organizations that have struggled to overcome human biases for years. Accuracy is imperative, but bias isn’t a deal breaker. Leaders may not realize it, but their relentless pursuit of accuracy may exacerbate the bias problem.
Uncomfortable Truths
Why hasn’t there been more progress? Can’t AI companies scrub training data and fine-tune models to produce more equitable and fair outputs?
It’s not that simple. When you ask ChatGPT to create an image, the model optimizes for accuracy. If you request a golden retriever, you expect a fluffy dog with four legs. The training data likely includes golden retrievers with three legs, but most people would consider that output a hallucination.
Now you ask ChatGPT for an S&P 500 CEO. The training data includes far fewer images of female CEOs than male CEOs. Would you consider an image of a female CEO a hallucination? What about a male nurse?
Source: Qualtrics and NurseJournal
Most people don’t classify images of female CEOs and male nurses as hallucinations. They welcome the diversity. Unfortunately, from the AI’s perspective, it isn’t easy to distinguish those outputs from a three-legged golden retriever. Why is one accurate and the other a hallucination?
Creating a more equitable and fair world requires imagination. It means breaking with the past and hallucinating our way to a better future. We risk destroying AI imagination by pursuing accuracy at all costs.
Stumbling through Time
We experience the world differently than AI. Time and place information are woven into our neural circuitry. With the addition of image modalities to large language models, AI is progressing on the “place” front. Time is a different story.
Humans are not passive observers in the world. We generate predictions and work hard to make those predictions come true. Dreams, hopes, and aspirations are hallucinations that motivate us to action.
In 1990, there were zero female CEOs in the 50 largest companies. In 2000, there was one. In 2023, there were eight. It’s easy to spot the trend. Women have been leading more companies over time.
That trend is evident to you but not necessarily to AI. A large language model can spot the trend if you upload the data. However, trends aren’t a natural part of the model's architecture.
AI training happens all at once. There’s time information in the training data, but not much. For example, fiction books often follow a timeline. AI has little problem generating bedtime stories.
Most data used to train AI models doesn’t follow a timeline. Imagine trying to understand the passage of time if all you had access to was an archived copy of Reddit. The past, present, and future are one jumbled mess in AI training data. Models can fake an understanding of time, but they seem to memorize more than reason.
The good news is that this problem should resolve itself. The latest models train on videos, which contain context-specific time information. We also see more AI models embedded in physical robots that experience the world more like us.
Will an enhanced understanding of time produce less biased AI? I don’t know, but that seems like a prerequisite for imagination.
Can we do anything in the meantime?
No Filter
Picture yourself at a party surrounded by people you don’t know. A man approaches you and asks what you thought of the presidential debate. What do you say?
If you’re like me, you probably respond with something innocuous like, “It was certainly entertaining.” I don’t go to parties to argue with strangers about politics. Before I tell this guy what I think, I want to know where he stands on the political spectrum.
We filter our words and actions. People who say precisely what’s on their mind and do what they please without regard for others aren’t authentic — they’re narcissistic. Humans are social animals, and mental filters bind us together.
Can we add filters to large language models? Of course, Google tried — it didn’t go well. Telling an AI to embrace diversity works until you require historical accuracy. Diverse CEOs? How inclusive! Diverse founding fathers? Perhaps not that inclusive.
The results for “generate an image of the Founding Fathers” | Source: The Verge
Google’s approach failed because it hardcoded filters that assumed users always cared about equity and fairness. Most of us care about equity and fairness when asking for a stock image of a CEO. We usually don’t when requesting a portrait of the founding fathers.
For filters to work, they must be context-specific. Most of us don’t provide enough context when interacting with AI. We submit short requests and expect the AI to respond appropriately. To AI, we’re that annoying stranger at the party. We’ve told the AI almost nothing about ourselves and expect it to engage like we’re lifelong friends. That’s not how relationships work.
A little context goes a long way. Adding “I care deeply about gender equity” or “I care deeply about racial equity” to the CEO portrait prompts results in substantially more diverse outputs. A picture of a white man may be the most likely statistical choice, but the AI understands you’re unlikely to consider that output accurate in the context of your prompt.
Source: ChatGPT (prompt: “I care deeply about [gender / racial] equity. Create a portrait of a CEO.”)
AI companies use RLHF (reinforcement learning with human feedback) to gather contextual information during model training. They also collect user feedback with those thumbs-up and thumbs-down buttons we seldom click.
The latest source of contextual data is our devices. “Apple Intelligence” is little more than ChatGPT with Apple-engineered prompts and controlled access to user data. We don’t know how much data each prompt includes, but Apple has plenty of context available if they choose to use it.
Google’s heart was in the right place. They designed the initial release of Gemini to filter responses through a lens of equity and fairness. That wasn’t what users expected or wanted, but it’s better than reinforcing historical biases because AI models lack imagination.
A Simpler Problem
Why did I go through the trouble of redefining the AI bias problem? What’s wrong with saying, “AI training data is biased, and we risk perpetuating those biases if we deploy AI into society.”
That framing perpetuates the status quo. It assumes today's human biases are less harmful than tomorrow’s biased AI. It lacks imagination.
In Artificially Human, I tell the story of a healthcare company that built an applicant screening algorithm. The prototype accelerated hiring, reduced attrition, and improved diversity. It was a clear win.
Unfortunately, the AI didn’t work in production. Diverse candidates were less likely to make it through the interview process. They were also less likely to stay more than a year.
The company did everything right. They omitted features like age, race, and gender from the training data. They rigorously cross-validated the prototype. They took every step you can imagine to mitigate bias.
The problem wasn’t the algorithm. It was the human biases present in the organization. The algorithm suggested qualified, diverse candidates who struggled to impress hiring managers. When they did, candidates often found themselves on teams where they felt isolated and alone.
Rather than address the human biases that the algorithm surfaced, the company pulled the plug. As I said earlier, accuracy is imperative, but bias isn’t a deal breaker.
With a bit of imagination, AI can be a force for good. If you agree with my framing of the problem, here is how we could tackle AI bias in the short, medium, and long term:
Short Term: Provide users with pre-built filters to tailor AI outputs (e.g., “Be aware of historical biases and the importance of equity and fairness when responding to this request”). Don’t force people to use the filters, but give them the ability (and responsibility) to filter outputs in context-appropriate ways.
Medium Term: Engineer models to promote curiosity. The latest OpenAI model uses a chain-of-thought technique to reason through problems. Part of that reasoning should be, “How important is the context in responding to this request, and do I have enough information to know what the user expects?” If the answer is “no,” the AI should ask for more information before proceeding (e.g., “Tell me more about the CEO you want me to portray in the image”).
Long Term: Make the digital world where AI lives look more like the analog world we inhabit. Adjust social media algorithms to promote moderate rather than inflammatory content. Provide AI companies with access to videos and other time-based data. Remove stale data from training sets when it reflects historical biases we’ve overcome.
These steps won’t eliminate AI bias but are more likely to yield progress than the status quo. We know what’s behind AI bias and how to lessen it over time. Mitigating human bias isn’t nearly as straightforward. Shouldn’t we simplify the problem?
I don’t want to ignore or diminish the negative impact AI bias can have on people. There are harmful systems in use today (e.g., law enforcement facial recognition trained primarily on images of white or lighter-skinned individuals). Those applications deserve the scrutiny and criticism they receive.
However, we should also embrace AI as an ally in the fight for equity and fairness. The world would be less biased if that company implemented its applicant screening algorithm. It would also put the company on a path to finally having enough workers to staff its facilities.
Biases are a drain on society. They cause us to make worse decisions, miss opportunities, and undervalue contributions. Companies that insist on AI accuracy at all costs are missing the point. Investing in equity and fairness isn’t only the right thing to do — it’s good business.
Let me make one final point if that’s not enough to motivate you. You may not be part of a marginalized community today, but there’s no guarantee you won’t be in the future. When the AI overlords run the show, we better hope they value equity and fairness as much as accuracy.