MENU
An AI model was trained on historical loan data. A loan officer has noticed that the model disproportionately suggests to refuse loans to people who live in a particular area. What is the type of bias described in the scenario?
The scenario describes Algorithmic bias, which occurs when an AI system reflects and potentially amplifies the prejudices or inequalities present in the historical data it was trained on. In this case, if historical lending practices were discriminatory toward specific neighborhoods (a practice known as 'redlining'), the AI model treats the resulting 'denial' patterns as a mathematical rule. It learns that living in a certain zip code is a predictor of loan failure, even if the individual applicants are creditworthy.
This is a major ethical concern in prompt engineering and AI deployment because the 'bias' is not a glitch in the code, but a reflection of systemic human bias encoded into the model's logic. It differs from 'Sampling bias' (which would occur if the model only looked at one city) or 'Measurement bias' (which involves faulty sensors). Algorithmic bias is particularly insidious because it can give discriminatory decisions a 'veneer of objectivity,' making it harder for human operators to spot the unfairness. Addressing this requires rigorous data auditing and the use of 'fairness constraints' to ensure that the AI does not penalize individuals based on protected characteristics or proxy variables like geography.