What is AI Bias?

It would be easy to assume that machines, being unaffected by human traits such as emotion, would be incapable of bias. If this were true, resume-scanning AI models could help to even the playing field for job seekers from marginalised groups, but the very opposite can (and does) happen.

AI models, including Large Language Models (LLMs), are not without their flaws. One significant issue is bias, which can lead to unfair or prejudiced outputs. Here, we'll explore AI bias, its causes, and how you, as a user, can mitigate the risks.

‍

What is AI bias?

Bias in AI occurs when AI models produce content that reflects systematic errors or distortions. These biases often mirror human biases present in the data used to train the AI. This can result in skewed or unfair outcomes, such as a resume-scanning AI favouring male candidates over equally qualified female candidates.

‍

What causes bias?

AI's reliance on its training data means that biases in the data can easily be absorbed and even amplified. Several factors can contribute to AI bias:

• Biased training data: The data used to train AI models may contain historical biases, stereotypes, or incomplete information. If the data reflects existing societal biases, the AI will likely reproduce these biases.
• Limited data: If the training data doesn't adequately represent all demographics or perspectives, the AI may perform poorly or unfairly for underrepresented groups.
• Algorithmic bias: The design of the AI algorithm itself can introduce bias. For example, certain algorithms might be more sensitive to specific features, leading to skewed results.
  ‍

Real-world examples

Bias in AI happens frequently and can greatly impact people's lives, potentially affecting their employment, health, and more. Here are some examples:

1. Amazon's resume scanner that penalised women: In 2015, Amazon discovered a bias against women in the algorithm it used for hiring employees. This happened because the model was trained on resumes submitted to the company over the course of a decade, which mainly came from men due to male dominance in the tech industry. Recognising this as a pattern, the model penalised applications that included the word "women's", for example, "women's chess club captain".

2. A US healthcare risk algorithm that presented racial bias: By equating health needs with costs, a widely used healthcare risk-prediction algorithm underestimated the health needs of Black patients, who tend to have lower healthcare spending compared to white patients with similar conditions. This bias resulted in Black patients being less likely to be recommended for high-risk care management programs, potentially missing out on critical care.

3. A UK welfare fraud detection system that disproportionately flags certain demographic groups: Designed to identify fraudulent claims for universal credit, the AI model used by the UK government was found to exhibit bias based on factors such as age, disability, marital status, and nationality. However, details such as which demographics are more or less likely to be wrongly singled out are unclear due to officials wanting to prevent fraudsters from gaming the system.

‍

How to mitigate the risk as a user

While developers and researchers are primarily responsible for addressing AI bias at its source, users can take steps to protect themselves from its negative effects. Here's how:

• Be aware and critical: Approach AI-generated content with a critical eye. Recognise that AI is not neutral and can produce biased results, so don't unquestioningly accept the information provided.
• Verify important information: Always double-check crucial information generated by AI. Use reliable sources to confirm facts, statistics, and claims.
• Consider multiple perspectives: If you're using AI for research or decision-making, seek out diverse sources and perspectives to counter potential biases in the AI's output.
• Understand the AI's limitations: Be aware of the AI's intended purpose and limitations. AI models are not a substitute for human judgment and should be used to assist, not replace, human decision-making.
• Use AI as a starting point: Instead of relying solely on AI-generated content, use it as a starting point for further exploration and research. This can help you identify potential biases and develop a more well-rounded understanding of the topic.
• Use multiple LLMs: Comparing the outputs from different models helps identify inconsistencies and inaccuracies. Multiple LLMs can also verify each other's output, reducing the risk of both bias and hallucinations.

‍

AI offers incredible potential, but it's essential to be aware of its limitations, including the risk of bias. Understanding what causes AI bias – and taking proactive steps to mitigate its effects – helps you use AI more responsibly and effectively.

Don't rely on just one LLM: With Narus, you can prompt and manage multiple LLMs in one place.