Post: Case Study: How AI Candidate Screening Cut Time-to-First-Interview by 68%
Client Profile and Starting Point
Sarah’s healthcare HR team managed recruiting for a 6-location regional healthcare system. They processed 800-1,200 applications per quarter for clinical and administrative roles. Their existing process: applications arrived in Greenhouse, recruiters manually reviewed each resume, advanced qualified candidates to a phone screen, and emailed rejections individually. At average 8-12 minutes per manual resume review, the team spent 18-24 recruiter hours per week on first-pass screening alone.
Their OpsMap™ audit identified three problems: excessive time on first-pass screening, inconsistent evaluation criteria across four recruiters, and a 8.3-day average lag from application receipt to phone screen scheduling.
The Automation Architecture
The solution had four components deployed in sequence over six weeks:
Week 1-2: Make.com webhook integration to receive applications from Greenhouse and the careers site. AI parsing API to extract and structure 22 candidate data fields. Automated creation of consistent ATS records — solving the data quality problem as a foundation for scoring.
Week 3-4: Scoring model development. The team used 18 months of historical hire data to identify which candidate attributes correlated with successful 90-day retention. The model scored candidates on 8 weighted criteria: years of relevant experience, certification status, geographic distance, gap analysis, role match score, application completeness, tenure patterns, and keyword alignment with the specific job description.
Week 5: Parallel validation. The AI scores ran alongside manual review for 30 days. Recruiters reviewed the top 30% and bottom 20% of AI-scored candidates to validate calibration. Achieved 91% agreement rate with recruiter advancement decisions before full automation.
Week 6: Full deployment. Applications scoring above threshold automatically advanced to phone screen with scheduling invitation. Applications below threshold received a polished, role-specific rejection email within 24 hours. Mid-range scores triggered a 48-hour recruiter review flag.
90-Day Results
| Metric | Before | After 90 Days | Change |
|---|---|---|---|
| Time to first interview | 8.3 days | 2.7 days | -68% |
| Applications reviewed/recruiter/week | 62 | 193 | +211% |
| Recruiter hours on first-pass screening | 21 hrs/week | 5 hrs/week | -76% |
| Phone screen to offer conversion | 31% | 38% | +7pp |
| Adverse impact (gender, pass-through) | Baseline established | Within 3pp of baseline | Compliant |
| Candidate rejection response time | 5.1 days average | 22 hours | -78% |
Lessons Learned
The 30-day parallel validation phase was the most valuable investment. Without it, the scoring model would have been deployed with miscalibrated thresholds. The validation also built recruiter trust in the system — they saw their own judgment reflected in the AI scores, which reduced resistance to the automation.
The team also learned to review the middle-score band weekly. Candidates in the 40th-60th percentile of AI scoring had the highest rate of recruiter overrides — and those overrides often led to strong hires. Building structured human review for the ambiguous middle created a better system than pure AI automation would have.
- Time-to-first-interview dropped 68% — this is the most visible metric to hiring managers and candidates
- 30-day parallel validation before full deployment is not optional — it builds accuracy and team trust simultaneously
- Adverse impact analysis from day one is both a compliance requirement and a quality control signal
- AI screening improves phone-screen-to-offer conversion because recruiters spend their screening time on the most qualified candidates
- The middle-score band requires structured human review — full AI automation of ambiguous candidates produces lower-quality outcomes
Frequently Asked Questions
What AI tools were used for candidate screening in this case study?
The screening stack included Make.com for orchestration, an AI parsing API for resume extraction, a custom scoring model built on historical hire data, and Greenhouse ATS for stage management. The entire system was built without proprietary AI vendor lock-in.
How was AI screening accuracy validated?
The team ran the AI scoring model in parallel with manual recruiter review for 30 days before fully automating. They compared AI advancement decisions against recruiter decisions on the same candidate pool and achieved 91% agreement rate before going live.
Did AI screening reduce diversity in the candidate pipeline?
No. The team ran quarterly adverse impact analysis from day one. At 90 days post-deployment, pass-through rates by gender and ethnicity were within 3 percentage points of the pre-automation baseline — well within acceptable statistical variance.
For the complete guide to building HR workflow automation, see our pillar resource: Automated Offer Letters with Make.com: Transforming Talent Acquisition.
RELATED POST
