Auditing your AI hiring system for bias in 7 steps takes 4 hours per month and prevents the regulatory, reputational, and human cost of deploying an AI system that systematically disadvantages protected groups — a cost that scales with every hiring decision the biased system makes and cannot be retrospectively corrected without re-reviewing every affected candidate. Here is the complete audit process. See the XAI Fair Hiring guide for the explainability tools that make each audit step efficient.

Steps 1–2: Data Preparation and Demographic Classification

Step 1: Pull the decision dataset. Export all AI screening decisions from the prior 30 days from your ATS: candidate ID, role type, AI score, decision (advance/screen-out), and application date. This is your audit dataset. Step 2: Classify candidates by protected group. If you collect voluntary EEO data at application, join it to the decision dataset by candidate ID. If not, use name-based demographic inference (with documented limitations) or focus the audit on gender (inferrable from pronouns in most ATS systems) as a minimum. A full adverse impact analysis requires data on race/ethnicity, gender, age group, and disability status. Document the classification methodology and its limitations in your audit report.

Steps 3–4: Adverse Impact Calculation and Dimension Decomposition

Step 3: Calculate adverse impact ratios. For each protected group, calculate: (candidates advanced) / (total candidates from that group). Divide each group’s rate by the highest group’s rate. Flag any ratio below 0.80 (EEOC 4/5ths threshold). Record all ratios in your audit log. Step 4: Decompose by rubric dimension. For groups showing adverse impact, extract the score distribution for each rubric dimension separated by protected group. The dimension where the cross-group score variance is largest is the primary bias contributor. Calculate the Pearson correlation between dimension scores and protected group classification — a correlation above 0.15 indicates the dimension is functioning as a proxy for the protected characteristic.

Steps 5–7: Remediation, Validation, and Documentation

Step 5: Implement rubric remediation. For the identified bias-contributing dimension, evaluate three options: remove the dimension entirely if it lacks validated job-performance predictivity; substitute a less biased operationalization (e.g., replace “university tier” with “relevant coursework” to reduce HBCU disadvantage); or apply disparate impact correction weights to neutralize the demographic correlation while preserving the dimension’s predictive content. Step 6: Validate remediation impact. Re-run the adverse impact calculation on the prior 90 days’ decisions using the modified rubric weights to project the post-remediation disparity ratio. If projected ratio is above 0.80, deploy the change; if below, continue modifying until the projected ratio clears the threshold. Step 7: Document and retain. The audit report must include: audit date, data period, sample sizes, adverse impact ratios by group, identified bias sources, remediation actions, and projected post-remediation ratios. Retain for minimum 3 years.

Expert Take — Jeff Arnold, 4Spot Consulting™

Step 4 — dimension decomposition — is where audits stop being a checkbox exercise and start being useful. Most HR leaders know their overall pass rate by demographic group, but they do not know which rubric dimension is driving it. When you find that your “years of uninterrupted experience” dimension has a 0.23 correlation with gender, you have discovered something actionable. That is the finding that changes your rubric and your outcomes. Do not stop the audit at Step 3.

Key Takeaways

• Step 1–2: Pull 30-day decision dataset; classify by protected group using EEO data or documented inference methodology.
• Step 3: 4/5ths rule — any group ratio below 0.80 requires steps 4–7.
• Step 4: Pearson correlation above 0.15 between a dimension score and protected group classification indicates the dimension functions as a demographic proxy.
• Step 5: Three remediation options — remove dimension, substitute operationalization, or apply correction weights.
• Step 6: Project post-remediation ratio on 90 days of historical data before deploying rubric changes.

Frequently Asked Questions

What statistical test should you use for AI bias auditing beyond the 4/5ths rule?

The Fisher Exact Test assesses whether observed selection rate differences are statistically significant given the sample size. The Chi-square test applies to larger samples (above 100 per group). Use these tests alongside the 4/5ths rule — a ratio below 0.80 that is not statistically significant at p<0.05 indicates small sample size rather than systematic bias. Both the ratio and the statistical significance level belong in your audit report.

How do you audit AI bias in interview scheduling, not just resume screening?

Apply the same framework: collect scheduling decision data (who received interview invitations), classify by protected group, calculate the adverse impact ratio for interview invitations versus screening advances. Scheduling bias is less common than screening bias but equally actionable when found. A system that screens fairly but schedules interviews preferentially for certain groups has adverse impact at the interview stage that a screening-only audit would miss.

Should you audit AI bias for internal promotion decisions differently than external hiring?

Yes. For internal promotions, you have more complete demographic data and longer employment records — use them. The audit dataset should include: promotion rate by protected group, AI assessment score distribution by protected group, and performance rating distribution by protected group. A promotion AI that correlates with demographic factors not captured in resume data (which external hiring audits must infer) can be audited with direct demographic data from your HRIS.