A seven-step AI hiring bias audit identifies, documents, and remediates bias in automated screening systems before regulatory inquiry — protecting candidates from discriminatory outcomes and HR teams from EEOC enforcement actions that average $250,000 in settlement costs for AI-related discrimination findings. Run this audit quarterly on any AI screening system in production. Here is each step.

Step 1: How Do You Collect the Data Required for a Bias Audit?

Export three datasets from your ATS for the audit period (minimum 90 days): all applications received with self-reported EEO data, all AI screening decisions (advance/hold/screen-out) with scores, and all final hire outcomes. Remove personally identifying information except the EEO fields required for the analysis. If your ATS does not capture AI decisions separately, export the custom field data written by your Make.com™ or n8n screening scenario. Minimum viable sample: 500 applications per demographic group being analyzed.

Step 2: How Do You Run a Four-Fifths (80%) Adverse Impact Analysis?

For each protected class (gender, race/ethnicity, age 40+), calculate the pass rate through AI screening. Divide the pass rate for the lowest-passing group by the pass rate for the highest-passing group. If the ratio is below 0.80 (80%), adverse impact exists for that protected class. Example: if white applicants pass AI screening at 72% and Black applicants pass at 52%, the ratio is 0.72 — adverse impact is present and requires investigation. Document every result, including results above 0.80. See the XAI Fair Hiring guide for the explainability framework used in Step 5.

Step 3: What Causes Adverse Impact in AI Hiring Systems?

The three most common causes: (1) proxy variables — scoring dimensions that correlate with protected class (graduation institution, employment gap, address) even though the dimension appears neutral; (2) training data bias — models trained on historical hire data encode the demographic profile of past hires; (3) rubric calibration — threshold scores set without testing their demographic impact. Identify which cause is present by comparing pass rates at each scoring dimension separately. The dimension with the highest demographic disparity is the likely root cause.

Step 4: How Do You Remediate Adverse Impact Found in the Audit?

Remediation depends on root cause. For proxy variables: remove the dimension from the rubric entirely and recalibrate the remaining weights to 100%. For training data bias: retrain or replace the model; document the bias finding in your vendor SLA review. For rubric calibration: lower the threshold score and back-test whether the adjusted threshold produces a four-fifths ratio above 0.80 while maintaining hire quality. Never adjust thresholds upward for a disadvantaged group without adjusting the rubric itself — threshold manipulation is not remediation.

Step 5: How Do You Document AI Decisions for EEOC and OFCCP Compliance?

Maintain a decision log for every AI screening outcome including: the factors contributing to the score, the weight of each factor, the threshold applied, and the final decision. This explainability requirement aligns with the EU AI Act’s high-risk AI transparency mandate and the EEOC’s 2023 technical assistance on AI employment tools. Store decision logs for 3 years minimum — the EEOC’s standard charge filing period. David’s manufacturing team automated decision logging to a Teamwork™ compliance folder via Make.com™ on every screening event.

Step 6: How Do You Communicate Bias Audit Results to HR Leadership?

Present audit results in a one-page executive summary covering: audit period, total applications screened, adverse impact findings by protected class (pass/fail per four-fifths rule), root causes identified, and remediation actions taken or planned. Separate the technical appendix (full statistical tables) from the executive summary. Schedule the presentation before the results are needed for a regulatory response — proactive disclosure of an internal audit demonstrates good faith compliance posture that regulators reward in enforcement decisions.

Step 7: How Do You Build a Continuous Bias Monitoring System?

Quarterly manual audits are the baseline. Build continuous monitoring by automating the four-fifths calculation as a monthly Make.com™ scenario that queries your ATS data, computes adverse impact ratios by protected class, and emails an alert to HR leadership if any ratio drops below 0.85 (a 0.05 buffer above the 0.80 regulatory threshold). The alert triggers an off-cycle investigation before the quarterly audit. Nick’s staffing firm has run 18 consecutive months of automated monitoring without a reportable adverse impact finding.

Expert Take — Jeff Arnold, 4Spot Consulting™

The firms I see getting surprised by EEOC AI enforcement actions are almost always the ones running bias audits annually — or not at all. Adverse impact in AI systems builds up incrementally: a model update here, a rubric change there, and six months later a reportable disparity exists that no one noticed because no one was looking monthly. Automate the monitoring. Run it monthly. The 15 minutes of Make.com™ setup prevents the 18 months of regulatory response that follows a finding.

Key Takeaways

• Export applications, AI decisions, and hire outcomes — minimum 500 applications per demographic group for a valid analysis.
• The four-fifths rule: any protected class with a pass rate below 80% of the highest-passing group triggers adverse impact investigation.
• Three root causes: proxy variables, training data bias, rubric calibration — identify which is present before remediating.
• Remediate by removing proxy dimensions, retraining biased models, or recalibrating thresholds — never by adjusting thresholds for individual groups.
• Store decision logs with contributing factors and weights for 3 years minimum.
• Present a one-page executive summary proactively — before regulatory response is required.
• Automate monthly four-fifths monitoring with a 0.85 alert threshold as a buffer above the 0.80 regulatory line.

Frequently Asked Questions

Is a bias audit legally required before deploying AI hiring tools?

Federal law does not currently mandate pre-deployment bias audits for private employers. New York City Local Law 144 requires annual independent bias audits for AI employment decision tools used on NYC residents. Illinois and Colorado have similar requirements. Regardless of jurisdiction, a documented bias audit constitutes evidence of good-faith compliance that significantly reduces enforcement risk.

Who should conduct an AI hiring bias audit — internal HR or an external firm?

Internal HR teams handle operational quarterly audits using the methodology above. External independent audits are required for NYC Local Law 144 compliance and are recommended before deploying any new AI screening system. External audits also provide legal privilege protection if conducted at the direction of legal counsel.

What statistical significance threshold should you use in a bias audit?

The EEOC uses the four-fifths rule (a practical significance test) as its primary criterion, not statistical significance. For large datasets (1,000+ applications), also run a chi-square test with a p-value threshold of 0.05. Present both metrics in your audit documentation — the four-fifths rule for regulatory compliance, the statistical test for technical rigor.