Chapter 8: AI Ethics and Values

Learning Outcomes

By the end of this chapter, students will be able to:

• Demonstrate an understanding of the fundamental principles of ethics and gain insight into ethical considerations related to AI technologies
• Develop an understanding of AI bias, its sources, and its real-world implications, as well as the ethical considerations
• Identify and apply strategies for mitigating bias in AI systems to promote fairness and transparency in technology
• Recognize the significance of AI policies in promoting responsible, safe, and ethical use of AI technologies

Theory

Ethics in Artificial Intelligence

What is Ethics? Ethics is the branch of philosophy that deals with questions of right and wrong, morality, and proper conduct. In the context of AI, ethics refers to the moral principles that guide the development, deployment, and use of AI systems.

Why AI Ethics Matters:

• AI systems make decisions that affect human lives
• AI can perpetuate or amplify existing biases
• AI systems can be opaque and difficult to understand
• AI raises questions about privacy, autonomy, and human dignity
• AI development is outpacing regulatory frameworks

Key Questions in AI Ethics:

1. Should AI systems make decisions that affect human lives?
2. Who is responsible when AI systems cause harm?
3. How do we ensure AI treats all people fairly?
4. How do we maintain human oversight of AI?
5. How do we balance innovation with safety?

The Five Pillars of AI Ethics

1. Fairness

Definition: AI systems should treat all individuals and groups equitably, without discrimination.

Considerations:

• Equal treatment across demographic groups
• Avoiding discriminatory outcomes
• Ensuring equal access to AI benefits
• Addressing historical biases in data

Example: An AI hiring system should evaluate all candidates based on relevant qualifications, not demographic characteristics.

2. Accountability

Definition: Clear responsibility for AI systems’ development, deployment, and outcomes.

Considerations:

• Identifying who is responsible for AI decisions
• Creating mechanisms for redress when harm occurs
• Documenting AI development processes
• Establishing governance frameworks

Example: A hospital using AI for diagnosis should have clear protocols for human oversight and responsibility for outcomes.

3. Transparency

Definition: AI systems should be understandable but also provide clarity about their limitations, functioning, and purpose.

Considerations:

• Explainability of AI decisions
• Clear communication of AI capabilities and limitations
• Disclosure when AI is being used
• Open development processes where appropriate

Example: Users should be informed when they’re interacting with a chatbot rather than a human.

4. Privacy

Definition: Protecting personal information and ensuring individuals have control over their data.

Considerations:

• Data collection minimization
• Secure data storage and processing
• User consent for data use
• Right to be forgotten
• Protection against surveillance

Example: AI-powered health apps should protect sensitive medical data and give users control over what is shared.

5. Safety and Security

Definition: AI systems should be safe, reliable, and protected against malicious use.

Considerations:

• Robust testing and validation
• Fail-safe mechanisms
• Protection against adversarial attacks
• Cybersecurity measures
• Prevention of harmful applications

Example: Autonomous vehicles must have multiple safety systems to prevent accidents.

Bias in AI Systems

What is AI Bias? AI bias occurs when an AI system produces results that systematically favor or disadvantage certain groups. Bias can lead to unfair, discriminatory, or harmful outcomes.

Types of Bias:

Bias Awareness

Recognizing Bias: Bias can appear at multiple stages of AI development:

1. Data Collection Stage:

   • Who is included/excluded in data?
   • How was data collected?
   • What historical biases exist in data?

2. Model Development Stage:

   • What features are used?
   • How are labels defined?
   • What assumptions are made?

3. Deployment Stage:

   • How is the model being used?
   • Who has access to the technology?
   • What decisions are being made?

4. Evaluation Stage:

   • How is performance measured?
   • Is performance equal across groups?
   • What metrics are prioritized?

Sources of Bias

1. Training Data Bias

• Historical discrimination reflected in data
• Underrepresentation of certain groups
• Incorrect or biased labels
• Selection bias in data collection

2. Algorithmic Bias

• Choice of features that correlate with protected characteristics
• Optimization for biased objectives
• Model architecture limitations

3. Human Bias

• Developers’ unconscious biases
• Biased assumptions in problem framing
• Biased interpretation of results

4. Societal Bias

• Existing social inequalities
• Institutional discrimination
• Cultural stereotypes

Real-World Examples of AI Bias

Example 1: Hiring Algorithms A major tech company’s AI hiring tool was found to discriminate against women because it was trained on historical hiring data that reflected past gender biases.

Example 2: Facial Recognition Studies have shown that some facial recognition systems have significantly higher error rates for people with darker skin tones, particularly for women of color.

Example 3: Criminal Justice Risk assessment algorithms used in criminal justice have been found to disproportionately label Black defendants as higher risk compared to white defendants with similar profiles.

Example 4: Healthcare An AI system used to allocate healthcare resources was found to systematically underestimate needs of Black patients because it used healthcare spending as a proxy for health needs.

Example 5: Language Models Large language models have been shown to associate certain professions with specific genders and exhibit other stereotypical biases present in their training data.

Mitigating Bias in AI Systems

Pre-Processing Strategies

Data-level interventions:

• Collect more diverse and representative data
• Balance datasets across demographic groups
• Remove or modify biased features
• Use data augmentation techniques

In-Processing Strategies

Algorithm-level interventions:

• Use fairness constraints during training
• Apply adversarial debiasing techniques
• Implement fair representation learning
• Use ensemble methods with diverse models

Post-Processing Strategies

Output-level interventions:

• Adjust decision thresholds for different groups
• Apply calibration techniques
• Use fairness-aware evaluation metrics
• Implement human review for edge cases

Best Practices for Bias Mitigation

Developing AI Policies

What are AI Policies? AI policies are guidelines, regulations, and governance frameworks that guide the development and use of AI technologies.

Why AI Policies are Important:

• Protect individuals from harm
• Ensure fair and equitable AI use
• Build public trust in AI
• Guide responsible innovation
• Address legal and liability issues

Key Components of AI Policies:

1. Principles and Values

   • Core ethical principles
   • Organizational values
   • Alignment with human rights

2. Governance Structure

   • Oversight mechanisms
   • Decision-making processes
   • Roles and responsibilities

3. Risk Assessment

   • Impact evaluation procedures
   • Risk classification
   • Mitigation requirements

4. Accountability Mechanisms

   • Documentation requirements
   • Audit processes
   • Redress procedures

5. Transparency Requirements

   • Disclosure obligations
   • Explainability standards
   • Public reporting

Global AI Policy Landscape

Notable AI Policies and Guidelines:

India’s AI Initiatives:

• NITI Aayog’s National Strategy for AI
• Responsible AI initiatives
• AI ethics guidelines for various sectors
• Focus on “AI for All” approach

Understanding Ethical Dilemmas

The Trolley Problem and AI: The classic trolley problem has new relevance for AI:

• How should autonomous vehicles make life-and-death decisions?
• Who decides the ethical principles programmed into AI?
• How do we balance competing values?

Key Ethical Dilemmas in AI:

1. Privacy vs. Benefit

   • More data improves AI performance
   • But data collection raises privacy concerns

2. Automation vs. Employment

   • AI can increase efficiency
   • But may displace workers

3. Personalization vs. Manipulation

   • AI can customize experiences
   • But may exploit psychological vulnerabilities

4. Innovation vs. Safety

   • Rapid development drives progress
   • But may introduce unforeseen risks

Practical Activities

Activity 1: Moral Machine Game

Visit the Moral Machine website (moralmachine.mit.edu) to explore ethical dilemmas faced by autonomous vehicles.

Instructions:

1. Complete the moral machine scenarios
2. Note your choices and reasoning
3. Compare your results with others
4. Reflect on how AI should make these decisions

Reflection Questions:

• What factors influenced your decisions?
• Were some decisions harder than others? Why?
• Should AI systems make these decisions? How?

Activity 2: Survival of the Best Fit Game

Play the “Survival of the Best Fit” game (survivalofthebestfit.com) to understand hiring bias.

Instructions:

1. Complete the game scenarios
2. Observe how bias enters the system
3. Note when problems become apparent
4. Reflect on real-world implications

Reflection Questions:

• How did bias enter the AI system?
• What were the consequences of the biased AI?
• How could the bias have been prevented?

Activity 3: Video Analysis - “Humans Need Not Apply”

Watch and summarize the video “Humans Need Not Apply” (available on YouTube).

Template for Summary:

1. Main Argument: What is the video’s main point?
2. Key Examples: What examples support the argument?
3. Implications: What are the consequences discussed?
4. Personal Response: What do you think about the claims?
5. Ethical Considerations: What ethical issues are raised?

Activity 4: Role Play - Biased AI Systems

Conduct a role-play activity exploring perspectives on biased AI.

Roles:

• AI Developer who created the system
• User negatively affected by bias
• Company executive defending the AI
• Policy maker considering regulations
• AI ethics researcher

Scenario: An AI system used for loan approvals has been found to have racial bias.

Activity 5: Comparative Study of AI Policies

Research and compare AI policies from different organizations.

Template:

Competency-Based Questions

Example Questions

1. Explain the five pillars of AI ethics. (5 marks)
2. Discuss the sources of bias in AI systems and provide examples. (6 marks)
3. Describe strategies for mitigating bias in AI systems. (5 marks)
4. Compare AI policies from two different organizations. (6 marks)

Answers to Example Questions

1. Answer: The five pillars of AI ethics:

   • Fairness: AI should treat all individuals equitably without discrimination
   • Accountability: Clear responsibility for AI decisions and outcomes
   • Transparency: Openness about AI capabilities, limitations, and decision-making
   • Privacy: Protecting personal data and giving users control over their information
   • Safety: Ensuring AI systems are reliable and secure from harm

2. Answer: Sources of AI bias:

   • Training Data Bias: Historical hiring data excluding women → AI discriminates against female candidates
   • Algorithmic Bias: Features that correlate with race used in loan decisions
   • Human Bias: Developers’ unconscious biases reflected in design choices
   • Representation Bias: Facial recognition trained mostly on light-skinned faces → poor performance on darker skin
   • Measurement Bias: Using arrest rates as proxy for crime rates → racial disparities amplified

3. Answer: Bias mitigation strategies:

   • Pre-processing: Collect diverse data, balance datasets, remove biased features
   • In-processing: Apply fairness constraints during training, use adversarial debiasing
   • Post-processing: Adjust decision thresholds, calibrate outputs across groups
   • Governance: Diverse teams, bias audits, regular monitoring
   • Testing: Evaluate performance across demographic groups

4. Answer:

   Aspect	EU AI Act	OECD AI Principles
   Approach	Risk-based regulation	Voluntary guidelines
   Scope	Legal requirements	Recommendations
   Enforcement	Fines and penalties	Self-governance
   Focus	Consumer protection	Innovation balance
   Key Element	Transparency obligations	Human-centered AI

Official Sample Paper Questions

1. What is AI bias and why is it a concern? (2 marks)
2. List the five pillars of AI ethics. (3 marks)
3. Explain the role of transparency in ethical AI development. (4 marks)
4. Discuss the importance of AI policies for responsible AI development. (5 marks)

Answers to Official Sample Paper Questions

1. Answer: AI bias occurs when AI systems produce unfair outcomes that systematically favor or disadvantage certain groups. It’s a concern because it can perpetuate discrimination, harm vulnerable populations, erode trust in AI, and have legal implications.

2. Answer: The five pillars are:

   1. Fairness
   2. Accountability
   3. Transparency
   4. Privacy
   5. Safety and Security

3. Answer: Transparency in ethical AI:

   • Explainability: Users understand why AI makes decisions
   • Disclosure: Clear when AI is being used
   • Limitations: Honest about what AI can and cannot do
   • Documentation: Record of how AI was developed and tested
   • Trust Building: Openness creates confidence in AI systems
   • Accountability: Enables identification of problems

4. Answer: AI policies are important because they:

   • Protect individuals from AI-related harms
   • Ensure fairness through standards and requirements
   • Build public trust in AI technology
   • Guide innovation while maintaining safety
   • Address liability questions when AI causes harm
   • Promote responsible development practices
   • Harmonize approaches across organizations and countries

Practice Questions

Multiple Choice Questions

1. Which is NOT one of the five pillars of AI ethics? a) Fairness b) Accountability c) Profitability d) Transparency

2. What is historical bias in AI? a) Bias that increases over time b) Bias reflecting past discrimination in training data c) Bias in historical AI systems d) Bias in historical research

3. Which strategy addresses bias at the data level? a) Post-processing b) Pre-processing c) In-processing d) Algorithmic adjustment

4. What is the Moral Machine experiment about? a) Testing AI intelligence b) Exploring ethical dilemmas in autonomous vehicles c) Evaluating machine learning accuracy d) Measuring AI processing speed

5. What is the main purpose of AI policies? a) To slow down AI development b) To increase AI profits c) To guide responsible AI development and use d) To eliminate all AI systems

Short Answer Questions

1. What is AI bias and why should we be concerned about it?
2. Explain the difference between fairness and accountability in AI ethics.
3. How can training data contribute to biased AI systems?
4. Why is transparency important in AI systems?

Long Answer Questions

1. Discuss the five pillars of AI ethics with examples of how each applies to real-world AI systems.
2. Explain the various sources of bias in AI systems and strategies to mitigate them.
3. Analyze the role of AI policies in ensuring ethical AI development and compare approaches from different regions.

Summary

Key Points

• AI ethics concerns the moral principles guiding AI development and use
• The five pillars of AI ethics: Fairness, Accountability, Transparency, Privacy, Safety
• AI bias can arise from data, algorithms, human factors, and society
• Bias can be mitigated through pre-processing, in-processing, and post-processing strategies
• AI policies provide governance frameworks for responsible AI development
• Ethical dilemmas in AI require balancing competing values and interests
• Understanding AI ethics is crucial for developing beneficial AI systems

Important Terminologies

• AI Ethics: Moral principles guiding AI development and use
• Bias: Systematic errors that favor or disadvantage certain groups
• Fairness: Equitable treatment of all individuals by AI systems
• Accountability: Clear responsibility for AI outcomes
• Transparency: Openness about AI capabilities and limitations
• Explainability: Ability to understand how AI makes decisions
• AI Governance: Frameworks for managing AI development and deployment
• Algorithmic Discrimination: Unfair outcomes from AI algorithms

Solutions to Practice Questions

Multiple Choice Answers

1. c) Profitability
2. b) Bias reflecting past discrimination in training data
3. b) Pre-processing
4. b) Exploring ethical dilemmas in autonomous vehicles
5. c) To guide responsible AI development and use

Short Answer Model Answers

1. AI bias occurs when AI systems produce outcomes that systematically favor or disadvantage certain groups. It is concerning because biased AI can perpetuate discrimination, cause harm to vulnerable populations, and undermine trust in AI technology.
2. Fairness focuses on equitable treatment and outcomes for all individuals, while accountability ensures clear responsibility for AI decisions and consequences. Both are essential but address different aspects of ethical AI.
3. Training data can contribute to bias through historical discrimination reflected in the data, underrepresentation of certain groups, biased labeling, or selection bias in data collection methods.
4. Transparency is important because it allows users to understand how AI makes decisions, enables identification of errors or biases, builds trust, and supports accountability when problems occur.

Long Answer Model Answers

1. The five pillars ensure comprehensive ethical AI:

   • Fairness: Hiring AI evaluating candidates equally regardless of demographics
   • Accountability: Clear lines of responsibility when AI healthcare diagnoses are wrong
   • Transparency: Disclosing when customers interact with chatbots
   • Privacy: Healthcare AI protecting patient data
   • Safety: Autonomous vehicles with multiple safety systems

2. Sources of bias:

   • Data: Historical discrimination, underrepresentation
   • Algorithms: Biased features, optimization objectives
   • Human: Developer biases, assumptions
   • Society: Existing inequalities

   Mitigation strategies:

   • Pre-processing: Diverse data, balanced datasets
   • In-processing: Fairness constraints, adversarial debiasing
   • Post-processing: Threshold adjustment, calibration

3. AI policies provide necessary governance through principles, accountability mechanisms, risk assessment, and transparency requirements. The EU takes a risk-based approach, while the US focuses on individual rights. Both aim to balance innovation with protection but differ in implementation approaches.

IBM Skills Build Integration

Complete the IBM Skills Build - AI Ethics course to:

• Understand fundamental AI ethics principles
• Learn about bias detection and mitigation
• Explore real-world ethical scenarios
• Develop skills in responsible AI development
• Earn a certification in AI ethics

References

• CBSE Artificial Intelligence Curriculum for Class XI (2025-2026)
• IBM Skills Build - AI Ethics
• MIT Moral Machine Experiment
• “Weapons of Math Destruction” by Cathy O’Neil
• “The Alignment Problem” by Brian Christian
• IEEE Ethically Aligned Design Guidelines
• EU AI Act
• UNESCO Recommendation on AI Ethics