A4.4.1 · Ethical considerations
Ethical Implications of Machine Learning
Machine learning models don't just process data, they make decisions that affect real people, and those decisions can go wrong in ways a simple accuracy score never captures. This statement asks you to discuss those consequences directly, grounded in things that have actually happened.
What principles and approaches should be considered to ensure machine learning models produce accurate results ethically? This statement is the topic's most direct answer: the nine issues below are exactly the principles that need weighing before, during and after a model is deployed.
Explainer
An ethical implication is a real consequence, positive or negative, that deploying a machine learning system can have on people's lives, rights or wellbeing, separate from whether the system is technically accurate.
Worked example
The nine ethical issues
Click each one to see what it actually means and a real, documented case where it played out.
Explainer
Why training-data bias happens
Bias in training data isn't usually the result of anyone deliberately programming unfairness in. It creeps in through the data itself, and three patterns explain most of it:
Historical data encodes historical decisions. If past hiring, lending or sentencing decisions were themselves unequal, a model trained on records of those decisions learns to repeat the same pattern, treating yesterday's bias as today's "correct answer." Under-representation compounds this: if a group barely appears in the training data, the model has little evidence to learn accurately for that group at all. Feedback loops make it worse over time: a biased model's own outputs can become tomorrow's training data, reinforcing the original bias rather than correcting it. The recruitment tool and recidivism examples above are both real cases where exactly this happened.
Reference
Ethics in online communication
The same nine issues take on a specific shape once machine learning is applied to online platforms and communication. The syllabus names five concerns worth knowing on their own:
| Concern | What it looks like online |
|---|---|
| Misinformation | Generative models can produce convincing fake images, audio or video (deepfakes) of real people appearing to say or do things they never did, spreading false claims faster than they can be corrected. |
| Bias | Recommendation algorithms tuned to maximise engagement tend to keep showing people more of what they already agree with, creating echo chambers, well documented on major social platforms' own news-feed and video-recommendation systems. |
| Online harassment | Automated bot accounts, powered by the same language-generation techniques as legitimate chatbots, can flood an individual with abuse at a scale no group of human harassers could match alone. |
| Anonymity | Users assume anonymised data can't be traced back to them, but researchers famously cross-referenced an "anonymised" public movie-ratings data set with another public site's reviews and re-identified individual users from it. |
| Privacy | Behavioural data collected for one purpose (browsing, purchases) can let an algorithm infer sensitive details, like a pregnancy, a health condition, or a life change, that a user never actually disclosed to anyone. |
Check your understanding
A hiring algorithm is trained mostly on resumes from past male applicants and, as a result, consistently ranks equally qualified female applicants lower, even though the model is making accurate predictions based on the patterns it was actually trained on. Which ethical issue is this?
Common mistake
RecurringA genuinely documented pattern: candidates use the word "bias" without separating two genuinely different ideas. Technical bias is a statistical deviation that makes a model's predictions less accurate. Ethical (or social) bias is prejudice baked into the data that produces unfair outcomes for a real group of people, even when the model's predictions are perfectly accurate by its own statistical measures.
Treats "biased" as meaning "inaccurate": "The hiring algorithm is biased because its predictions about who will succeed in the role are wrong." This describes technical bias, a statistical accuracy problem, not the actual ethical issue in play.
Names the actual ethical bias: "The hiring algorithm is ethically biased because it produces unfair outcomes for female applicants, a real prejudice inherited from the historical hiring data, even though its predictions are statistically accurate by the model's own measures." Correctly separates statistical accuracy from fairness to real people.
Revision questions
Past-paper style. Write an answer, then reveal the marking points and mark yourself.
[2]Define the term ethical implication in the context of machine learning.
Marking points
评分要点
- An ethical implication is a consequence of a machine learning system for people's rights, fairness or wellbeing that raises a question of right and wrong.
- It goes beyond whether the model is accurate: it concerns how the system affects the people subject to its decisions.
- 伦理影响是机器学习系统对人们的权利、公平或福祉造成的、引出对错问题的后果。
- 它超越模型是否准确:它关乎系统如何影响受其决策约束的人。
[4]Distinguish between technical bias and ethical bias, using a named real-world example to illustrate the difference.
Marking points
评分要点
- Technical bias is a statistical error: the model is systematically wrong, for example because the training sample was unrepresentative.
- Ethical bias is unfairness: the model's outputs disadvantage a particular group of people, even if the model is statistically accurate on its data.
- A named example, for example a recruitment screening tool that learned from past hiring data and downgraded applications from women, or facial recognition that is less accurate for darker skin tones.
- The point: a model can be accurate and still be ethically biased, and can be unbiased ethically yet technically wrong.
- 技术偏差是一种统计误差:模型系统性地出错,例如因为训练样本不具代表性。
- 伦理偏见是不公:模型的输出使某个特定群体处于劣势,即便模型在其数据上统计准确。
- 一个具名例子,如一个从过去招聘数据中学习并降低女性申请评分的招聘筛选工具,或对深肤色准确度较低的人脸识别。
- 要点:模型可以准确却仍有伦理偏见,也可以没有伦理偏见却在技术上出错。
[8]Discuss the ethical implications of a school using a machine learning system to predict which students are at risk of failing their final exams, referring to at least three named ethical issues.
Marking points
评分要点
- Bias and fairness: the model learns from records of past exam outcomes, which may reflect existing disadvantage, so it could systematically flag students from particular backgrounds.
- Transparency and explainability: students and parents should be able to learn why a student was flagged and challenge it, which many models make hard.
- Accountability: the school, not the model, remains responsible for how a flag is acted on; automation does not remove that duty.
- Privacy: predicting risk uses sensitive personal and family data, which must be collected and stored with consent and safeguards.
- Self-fulfilling prophecy and labelling: a student told they are at risk, or treated differently by teachers, may disengage, so the prediction changes the outcome.
- Consent and autonomy: students may have no say in being profiled this way.
- Balance: used carefully as a prompt for extra support, with human oversight and transparency, it can help; used to sort or limit students it causes clear harm.
- A strong answer names at least three of these issues and weighs benefit against harm.
- 偏见与公平:模型从过去考试结果的记录中学习,而这些记录可能反映现有的不利处境,所以它可能系统性地标记特定背景的学生。
- 透明性与可解释性:学生和家长应能了解某学生为何被标记并提出异议,而许多模型使这变得困难。
- 问责:承担责任的仍是学校而非模型;自动化不免除这一责任。
- 隐私:预测风险使用敏感的个人和家庭数据,必须在同意和保障措施下收集和存储。
- 自我实现的预言与贴标签:被告知有风险、或被老师区别对待的学生可能因此放弃,于是预测改变了结果。
- 同意与自主:学生可能对被这样画像没有发言权。
- 权衡:作为提供额外支持的提示、在人的监督和透明度下谨慎使用,它可以帮助人;用来对学生分类或设限则造成明显伤害。
- 好的答案会点出其中至少三个问题,并权衡收益与伤害。
Flashcards
Active recall for this statement’s key terms.
Keep this statement’s practice with you
Free account: unlock every revision question and flashcard for A4.4.1, and every other IB CS 2027 statement.