AI Bias in HR Glossary: Fairness Terms & Definitions

AI bias in HR is not an edge case — it is the default outcome when recruitment algorithms are trained on historically skewed data and deployed without ongoing disparity monitoring. Every term in this glossary maps to a real failure mode that has produced discriminatory outcomes in production hiring systems. If you are implementing AI screening, parsing, or ranking tools, these definitions are your operational vocabulary for building systems that hold up to legal and ethical scrutiny.

This glossary is a companion resource to the broader AI in recruiting strategy guide for HR leaders. It defines the specific fairness and bias concepts that determine whether an AI hiring system is legally defensible and equitably designed.

Algorithmic Bias

Algorithmic bias is a systematic, repeatable error in an AI system that produces unfair outcomes — consistently favoring or penalizing candidates based on characteristics unrelated to job performance.

In HR, algorithmic bias is the mechanism by which historical human prejudice gets encoded into automated decisions and then executed at scale. When a resume screener is trained on a decade of hiring decisions made by humans who had their own blind spots, the model does not neutralize those patterns — it learns them, amplifies them, and applies them to every candidate it touches.

The key word is systematic. Random errors affect individual candidates inconsistently. Algorithmic bias affects entire demographic groups consistently, in the same direction, every time the model runs. That consistency is what creates both the statistical detectability and the legal exposure.

Gartner research consistently identifies bias risk as the top governance concern among HR leaders evaluating AI hiring tools — outranking accuracy and cost concerns in enterprise organizations.

Why it matters operationally: Automation is not neutral by design. It is neutral only if the data it was trained on is neutral. Because no historical HR dataset is perfectly neutral, bias mitigation must be an active, ongoing process — not a one-time procurement evaluation.

See also: fair design principles for unbiased AI resume parsers for specific architectural controls that reduce algorithmic bias at the model design stage.

Data Bias

Data bias occurs when the information used to train an AI model is unrepresentative of the real-world population it is intended to evaluate, causing the model to produce skewed predictions.

In recruiting, data bias is almost always a historical problem. If a company trained its hiring prediction model on ten years of promotion decisions, and during those ten years women were systematically promoted at lower rates than equally qualified men, the model will learn that being male is a predictive feature for promotion potential. It will carry that signal forward, even after the organization’s promotion practices have changed.

Data bias is distinct from the bias held by any individual recruiter. Individual bias is inconsistent and varies by person and context. Data bias is baked into the model’s weights — it fires every time the model runs, at identical intensity, regardless of who is using it.

How to address it: Correcting data bias requires auditing training datasets for demographic representation, removing or reweighting records that reflect documented discriminatory decisions, and augmenting training data with records from underrepresented candidate pools. This is not a one-time cleanse — it requires governance procedures that prevent new biased data from re-corrupting the training set over time.

Selection Bias

Selection bias is an error in how candidates or records were chosen to enter a training dataset, resulting in a sample that does not represent the full candidate population the model will encounter in production.

Selection bias is a sampling problem, not a content problem. The data may be accurate, but the records included were not chosen randomly or representatively. A common example in HR: a company builds a “successful hire” model using only the records of employees who stayed more than two years. This excludes candidates who were strong performers but left for better opportunities — a non-random exclusion that skews the model’s definition of “success.”

Another common version: an AI trained exclusively on candidates sourced from four universities the company has historically recruited from. The model performs well on candidates from those schools and poorly on everyone else — not because those candidates are less qualified, but because they are outside the model’s training distribution.

The fix: Broaden data sourcing deliberately. Include successful hire profiles from diverse educational backgrounds, sourcing channels, and tenure patterns. Document the sourcing decisions made during dataset construction so they can be audited later.

Representation Bias

Representation bias occurs when specific demographic groups are underrepresented or overrepresented in training data, causing the model to perform inconsistently — and usually worse — for underrepresented groups.

Representation bias and selection bias are related but distinct. Selection bias is about the sampling process. Representation bias is about the demographic composition of whatever dataset resulted from that process. A dataset assembled through a perfectly randomized process can still have representation bias if the underlying population it sampled from was itself non-representative.

In practice, representation bias shows up in model performance disparities: the AI performs well at identifying qualified candidates from the majority demographic and poorly at identifying equally qualified candidates from minority demographics. This is measurable — and measuring it is the first step toward fixing it.

McKinsey research on AI risk has consistently highlighted representation gaps in training data as one of the primary causes of downstream discrimination in algorithmic decision systems.

Practical implication: Before deploying any AI screening tool, demand performance-disaggregated accuracy data from your vendor. A model that is 92% accurate overall but 74% accurate for one demographic group is not a 92% accurate model for your entire candidate pool.

Disparate Impact

Disparate impact is a legal doctrine establishing that an employment practice is unlawful if it disproportionately excludes members of a protected class — regardless of whether discriminatory intent exists.

This is the most consequential fairness concept for HR leaders to understand, because it does not require proof of intent. An AI screener that was designed in good faith, by people with no discriminatory motive, can still produce disparate impact if its outputs systematically filter out candidates from a protected group at a higher rate than others.

The EEOC’s Uniform Guidelines on Employee Selection Procedures apply to AI-driven selection tools the same way they apply to written tests and interviews. Employers bear the burden of demonstrating that selection tools with demonstrated adverse impact are job-related and consistent with business necessity.

The practical measurement standard is the 80% rule (four-fifths rule): if the selection rate for any protected group is less than 80% of the selection rate for the most-selected group, adverse impact is indicated and the practice warrants examination. Applied to AI screening: if your tool advances 60% of male applicants and 42% of female applicants, the ratio is 70% — below the threshold — and that disparity is a regulatory exposure regardless of the model’s design intent.

For a deeper treatment of how these legal obligations interact with AI tool selection, see protecting your business from AI hiring legal risks.

Fairness Metrics

Fairness metrics are quantitative measures used to evaluate how equitably an AI system performs across different demographic groups — producing numbers that can be audited, compared, and acted on.

There is no single universal definition of “fairness” in machine learning. Different metrics operationalize different values, and in most real-world scenarios they cannot all be simultaneously maximized. Understanding which metrics matter for recruiting — and the trade-offs between them — is an operational skill, not an academic one.

Demographic Parity

Demographic parity requires that the AI selects candidates from each demographic group at equal rates, regardless of their qualifications. This is the most straightforward metric but the most controversial: it does not account for differences in the underlying qualification distribution across groups, which may themselves be artifacts of historical systemic barriers.

Equal Opportunity

Equal opportunity requires that among candidates who are qualified, the AI correctly identifies them at equal rates across demographic groups. This metric focuses on ensuring the model does not miss qualified candidates from underrepresented groups — arguably the most practically important metric for recruiting, where false negatives (missing strong candidates) have direct business costs.

Predictive Equality

Predictive equality requires that the model’s error rates — both false positives and false negatives — are equal across groups. This is a stricter standard than equal opportunity and is particularly relevant when AI recommendations drive high-stakes downstream decisions like interview invitations or offer extension.

Calibration

Calibration measures whether a model’s confidence scores mean the same thing across demographic groups. A well-calibrated model that scores a candidate at 85% “fit probability” should have the same actual fit rate for that score, whether the candidate is from Group A or Group B. Miscalibration is a subtle form of bias that is easy to miss without group-disaggregated score analysis.

Harvard Business Review has documented the operational tension between these metrics — maximizing one often comes at measurable cost to another — making transparent trade-off documentation a governance requirement, not optional.

Proxy Discrimination

Proxy discrimination occurs when an AI model uses a seemingly neutral variable as a stand-in for a protected characteristic, producing discriminatory outcomes without explicitly referencing protected attributes.

This is the hardest form of bias to detect and the most common in production HR systems. The model never sees race, gender, or age. But it heavily weights zip code — which correlates with race due to residential segregation. Or it weights university prestige tier — which correlates with socioeconomic status, which correlates with race. Or it penalizes employment gaps — which disproportionately affect women who took caregiving leave.

The variables themselves may have some legitimate predictive value. That is what makes proxy discrimination technically defensible in a vacuum and practically discriminatory at scale. Identifying proxy variables requires feature-importance analysis: pull the top predictors driving your model’s scores and explicitly test each for demographic correlation. If a top-weighted feature correlates with a protected class, its inclusion requires documented business necessity justification.

This is one reason NLP-driven resume analysis requires bias-specific evaluation beyond keyword accuracy — language models can encode proxy variables through association patterns invisible to standard performance benchmarks.

Counterfactual Fairness

Counterfactual fairness asks whether an AI’s decision about a candidate would change if only that candidate’s protected attribute were different, holding all other factors constant.

If an AI ranks a candidate as “not a fit” and that decision would reverse if the candidate’s gender — but not their qualifications — were changed, the model is not counterfactually fair. Counterfactual fairness is one of the most rigorous available tests because it directly interrogates the causal role of protected attributes in model outputs.

In practice, testing for counterfactual fairness requires generating synthetic candidate profiles that are identical except for the protected attribute, running both through the model, and comparing outputs. This is not a standard feature of commercial HR AI products — it requires deliberate evaluation effort or third-party auditing.

Emerging AI regulatory frameworks in the EU and U.S. state-level legislation (notably New York City Local Law 144) are beginning to reference counterfactual-style analysis as an audit requirement, making this an increasingly operational concern rather than a theoretical one.

Model Explainability

Model explainability is the ability to articulate, in plain language, why an AI system produced a specific output — why a candidate was ranked first, filtered out, or flagged for review.

Explainability is a fairness prerequisite. If you cannot explain why the model ranked a candidate out, you cannot defend that decision in an EEOC inquiry, a candidate grievance, or a regulatory audit. “The algorithm decided” is not a legally defensible answer — it is an admission that the decision-making process is opaque and unauditable.

There are two operationally distinct types of explainability in AI systems:

• Global explainability: Understanding which features drive the model’s decisions overall — the feature-importance analysis discussed under proxy discrimination.
• Local explainability: Understanding why the model made a specific decision about a specific candidate. Local explainability is what matters most for individual candidate appeals and EEOC inquiries.

SHRM guidance on AI hiring tools consistently recommends that HR leaders require vendors to demonstrate both levels of explainability before deployment, and to contractually secure the right to request candidate-specific decision explanations.

For a practical framework on combining AI explainability with human oversight in hiring decisions, see blending AI and human judgment in hiring decisions.

Algorithmic Auditing

Algorithmic auditing is a structured evaluation process that tests an AI system’s outputs for bias, disparate impact, and fairness metric violations — conducted independently of the system’s developers or vendors.

The independence criterion is non-negotiable. A vendor auditing their own model for bias has a structural conflict of interest. An internal team that championed the purchase of a tool faces the same conflict. Credible algorithmic audits are conducted by parties with no stake in the outcome.

Deloitte research on responsible AI governance identifies auditing cadence as one of the top differentiators between organizations that successfully maintain AI fairness and those that experience compliance failures after initial deployment. The failure mode is consistent: rigorous pre-deployment testing, then no subsequent monitoring as the model drifts on new data.

Minimum audit cadence for HR AI tools:

• At deployment: baseline disparity analysis across all active requisition categories
• After any model update or retraining event
• Annually at minimum; quarterly for tools used in high-volume screening (>500 applicants/month)
• After any significant change in sourcing channels or candidate demographics

Vendor attestations provided at purchase reflect performance on benchmark datasets — not your candidate pool. They are a starting point, not a substitute for ongoing monitoring.

Related Terms

Training Data

The historical dataset used to build and calibrate an AI model. In HR, training data typically consists of past applications, screening decisions, interview outcomes, hiring decisions, and performance records. The quality, representativeness, and demographic composition of training data directly determines whether the model will perform equitably.

Ground Truth Labels

In supervised machine learning, ground truth labels are the “correct” answers the model is trained to predict — in HR, typically “hired” vs. “not hired” or “high performer” vs. “low performer.” If historical ground truth labels reflect biased human decisions, the model will learn to replicate those decisions. Garbage-in-garbage-out applies directly to fairness.

Adverse Action Notice

A legally required notification informing a candidate that an AI-assisted tool contributed to a negative employment decision. Emerging regulations in multiple jurisdictions require employers to disclose when AI tools influence hiring decisions and to provide candidates a means to request human review. HR leaders should confirm whether their jurisdiction mandates adverse action notices for AI-assisted screening.

Bias Bounty

A structured program — borrowed from cybersecurity bug bounty models — in which an organization invites external researchers to identify bias vulnerabilities in its AI systems in exchange for recognition or compensation. Increasingly referenced in AI governance literature as a proactive fairness mechanism.

Intersectionality

The principle that individuals hold multiple overlapping identities — race, gender, disability status, age — and that bias often operates at the intersection of those identities rather than along any single dimension. An AI tool may show acceptable fairness metrics for women overall and for Black candidates overall, while still producing severely biased outcomes for Black women specifically. Fairness analysis should include intersectional subgroup evaluation, not just single-attribute analysis.

Common Misconceptions

Misconception: AI removes human bias from hiring

AI does not remove human bias — it encodes it, systematizes it, and scales it. Every AI hiring tool is built by humans on data generated by humans making decisions. The biases embedded in that data and those design choices propagate through the model. The accurate framing is that AI can be designed and audited to reduce certain forms of bias — but it does not achieve neutrality automatically or by default.

Misconception: Fairness and accuracy are in tension

This is the most damaging misconception in AI fairness discourse. Models that perform poorly for underrepresented groups are inaccurate for those groups by definition. Correcting representation bias in training data routinely improves model accuracy across the full candidate population. The “fairness-accuracy trade-off” framing typically arises when accuracy is benchmarked against biased historical outcomes — a circular argument that defines fairness as deviation from past discrimination.

Misconception: Removing demographic data from the model eliminates bias

This approach — called “fairness through unawareness” — is demonstrably ineffective. Removing race or gender from the model’s inputs does not prevent the model from learning proxy variables that correlate with those attributes. The model finds the signal through other channels. True bias mitigation requires active identification and treatment of proxy variables, not just input suppression.

Misconception: One audit at deployment is sufficient

AI models are not static. They drift as candidate pools change, as labor markets shift, and as the model retrains on new data. A model that passes a bias audit at deployment can develop significant disparities within months. Ongoing monitoring — not point-in-time certification — is the operational standard for compliant AI hiring.

Building an Audit-Ready AI Hiring Stack

Understanding these terms is the foundation. Operationalizing them requires embedding fairness governance into every layer of your AI hiring infrastructure — from tool selection to requisition design to ongoing monitoring.

The essential AI resume parser features checklist covers the specific technical capabilities to require from vendors before deployment. The workforce diversity implementation guide translates these fairness concepts into measurable DEI outcomes. And the HR data privacy glossary covers the compliance layer — GDPR, CCPA, and the data handling obligations that intersect with bias auditing requirements.

Fairness is not a feature you procure. It is a system property you build, monitor, and defend continuously. The vocabulary in this glossary is where that work begins.