Your interview scorecard is where signal collapse becomes visible — or stays hidden. If scorecards don’t capture comparable, outcome-linked judgment, you never discover that your top-screened candidates underperform. These seven mistakes quietly hide a broken funnel, and fixing them turns the scorecard into the feedback loop your hiring lacks. The full context lives in the AI resume screening pillar guide.

Quick Comparison

1. The Scorecard Never Links to Screening Rank

The most damaging mistake is keeping interview scores in a system that never connects back to where the candidate ranked at screening. Without that join, the feedback loop can’t form and filter failure stays invisible. Here is the failure made concrete: your screen ranked a candidate seventh out of forty, but the interview panel raved and the hire turned into your best performer of the year. That single data point is a quiet indictment of your screen — except nobody ever sees it, because the screening rank lives in the ATS and the interview score lives in a separate scorecard, and the two are never put side by side. Make that join and a pattern emerges; skip it and you keep trusting a filter that buries your best people.

• Record each candidate’s original screening rank on the scorecard so the comparison is always one query away.
• Compare interview performance to screen rank routinely, watching for strong interviews from low-ranked candidates.
• Feed results into the screening-to-hire audit to turn anecdotes into evidence.

Verdict: Fix this first; it’s the difference between seeing and not seeing the problem.

2. Rating Scales Are Vague

A 1-to-5 scale with no anchors means every interviewer scores differently, and the data becomes noise. Anchored ratings — concrete descriptions of what a 3 versus a 5 looks like — make scores comparable. One interviewer’s 4 is a generous “seemed fine,” another’s 4 is a demanding “would trust with a hard problem,” and when you average those numbers you are averaging two different languages. The fix is to write down what each point on the scale actually means in behavior: a 5 is “reasoned through an unfamiliar problem without prompting,” a 3 is “answered correctly but needed the path pointed out.” Now a score carries the same meaning regardless of who entered it, and the comparison across candidates becomes real instead of imaginary.

• Define each point on the scale with an example of the behavior that earns it.
• Calibrate interviewers against the anchors so a 4 means the same thing across the panel.

Verdict: Comparable data requires anchored scales.

3. Interviewers Score Resume Polish

When interviewers carry the polished resume into the room, they score presentation a second time instead of evaluating the conversation. The scorecard should direct attention to reasoning, not the document. The trap is subtle: an interviewer reads an impressive resume, walks in already convinced, and then unconsciously scores the conversation to match the impression the document created. They are not evaluating the candidate’s thinking — they are confirming the resume’s marketing. Since the resume is now the most gameable artifact in the whole process, scoring it twice means scoring the gaming twice. A scorecard that asks only about specific decisions, tradeoffs, and reasoning forces the interviewer’s attention onto the one thing the candidate cannot have generated in advance: how they actually think out loud.

• Keep the resume out of the scoring fields so the document cannot anchor the score.
• Score specific decisions and tradeoffs — see behavioral questions AI can’t coach for what to ask.

Verdict: Score the person in the room, not the document on the desk.

4. No Field for Follow-Up Resilience

Fabrication shows up when fluency drops on the second or third follow-up. If the scorecard has no place to record that, your most reliable signal goes uncaptured. A candidate can deliver a rehearsed first answer flawlessly — anyone can memorize one strong story. The truth surfaces on the follow-up: “what would you have done differently,” “who disagreed with that call,” “walk me through the part that went wrong.” A real memory deepens under that pressure because there is always more to tell; a borrowed or invented answer thins out and goes vague because there was never anything underneath the first sentence. That contrast is your strongest read on authenticity, and a scorecard with no field for it throws the signal away the moment the interview ends.

• Add a field for how answers held under follow-up — did the candidate deepen or deflect?
• Train interviewers to probe before they score, treating the follow-up as the real test.

Verdict: The follow-up is the test — capture it.

5. High Assessment Scores Create a Halo

When interviewers see a perfect assessment score, they unconsciously inflate interview ratings. But a perfect score is a yellow flag in the AI era, not a green light. The interview must be scored on its own. The halo effect is well documented: one strong number bleeds into every adjacent judgment, so the interviewer who knows a candidate aced the assessment hears their answers as smarter than they are. The cruel twist is that a flawless assessment score is now the easiest thing in the world to produce with a tool in another tab, which means the halo is built on exactly the signal that has stopped being trustworthy. The defense is structural: the interviewer scores the conversation before they ever see the assessment number, so the two judgments stay independent and the halo has nothing to attach to.

• Withhold assessment scores until after interview scoring so the number cannot color the conversation.
• Treat perfection as a prompt to probe — see automated scoring vs human screens.

Verdict: Independent scoring prevents the halo.

6. No Record of What the Candidate Actually Said

Numeric scores with no supporting notes can’t be audited or learned from. Brief evidence — the specific example a candidate gave — makes the score defensible and the loop analyzable. A bare “4” tells you nothing six months later when you are trying to understand why a confident hire is struggling. A “4 — described rebuilding the deployment pipeline after it failed in production, named the specific tradeoff they accepted” lets you go back and check whether your read was right. Evidence is what makes a scorecard a learning instrument rather than a gut feeling encoded as a digit. It also protects the decision: a score backed by a recorded example is defensible to a hiring manager, to a skeptical panel, and to a legal review, where a naked number is not.

• Require one line of evidence per rating so every score points back to something the candidate said.
• Store it where the audit can read it, so the loop can connect words to outcomes later.

Verdict: Evidence turns scores into learning.

7. Scorecards Aren’t Reviewed Against Outcomes

Even a good scorecard is wasted if no one checks whether high interview scores predicted strong performance. The review closes the loop and exposes whether your whole funnel measures the right things. You can build perfect anchored scales with evidence on every line, and it still teaches you nothing if the scorecards go into a drawer the day the offer is signed. The review is where the funnel finally faces reality: pull the people you hired a year ago, line their interview scores up against how they actually performed, and see whether the scoring predicted anything at all. When it turns out your highest-scored interviews produced middling performers and a few low scores became stars, you have learned that one of your questions is measuring the wrong thing — and now you can fix it. Without the review, you repeat the same flawed scoring forever, confident and wrong.

• Review scorecards against on-the-job outcomes quarterly so the scoring is tested against reality.
• Adjust the funnel based on what predicted performance, retiring questions that turned out to measure nothing.

Verdict: The review is where the funnel finally learns.

How We Evaluated

Each mistake was rated on how thoroughly it obscures the screening-to-performance relationship and on how cheap the fix is. The highest-priority fixes restore the feedback loop. For how to build that loop, see how to audit your screening-to-hire correlation and the pillar guide.