The Interview Scorecard: How to Score Candidates on the Same Yardstick
Mercedes Gordo runs delivery and customer success at Nova and spent two years at LinkedIn in Dublin. Her rule for debriefs: if two interviewers can score the same answer three points apart, the problem isn't the candidate.
Updated 7 min read
In this article
Three people interview the same candidate. In the debrief, one says she was impressive, one says she was fine, and one says he isn’t sure. Nobody is lying and nobody can explain the gap, so the loudest opinion in the room wins.
That isn’t an interviewing problem. It’s a measurement problem, and it has a boring fix.
A scorecard is a form that makes everyone score the same things on the same scale with evidence attached. Here’s how to build one that people actually use, including the part most teams get wrong.
Start with four to six competencies, and six is the ceiling
Pick the things that genuinely predict performance in this specific role, split into two blocks: technical, what they have to know how to do, and behavioral, how they work and decide.
Six is a hard ceiling. Past six nobody holds the rubric in their head during a debrief, scoring drags, and people quietly fall back on overall impressions, which is the exact thing the scorecard exists to replace. If the role needs four, use four. A longer list isn’t more rigorous; it’s more room for things that don’t predict anything.
That’s also where bias gets in wearing a suit. Padding the rubric with “presence” or “energy” or “fits with the team” feels thorough and measures whether the candidate reminds you of yourself.
Give each one a weight, and make them sum to 100
Then assign a percentage to each competency so the total is exactly 100.
That constraint is doing all the work. Everyone agrees everything is important; almost nobody has decided what matters most. Making the numbers add up forces the conversation you’d otherwise have at the end, when there are two finalists and no agreed basis for choosing. Do it before the first interview and the ranking falls out of the arithmetic.
Each candidate’s score is the sum of score times weight, which gives a number out of 4. That number ranks people. It doesn’t decide anything, which matters later.
Mark as critical the one or two competencies where falling short is genuinely disqualifying, and write down the minimum you’ll accept for each. Usually that’s a 3.
Write anchors that describe behavior, not quality
For each competency, describe what a 1, a 2, a 3 and a 4 look like.
- 1, below the bar. Doesn’t reach what the job requires.
- 2, partial. Shows something, with clear gaps.
- 3, solid. The level the role actually needs.
- 4, standout. Clearly above what you expected.
The rule that makes anchors work: describe observable behavior, never an adjective. “Communicates well” is an opinion, and two people will score it three points apart without either of them being wrong. “Explains what they specifically did, why they made that decision and what happened next” is something you either heard or didn’t.
The test is simple. An interviewer reads the anchor and recognizes it in what they actually heard, or the anchor is badly written.
And calibrate to seniority. A 3 for a junior engineer is not a 3 for a head of engineering, and using the same anchors for both is how a scorecard ends up rewarding whoever has been working longest.
One more rule worth building in: anchor on the outcome, not on the style. For anything in the neighborhood of communication or confidence, score clarity, structure, evidence and listening. Not volume, not extroversion, not accent. Those are differences in how people present, and penalizing them has nothing to do with whether somebody can do the job.
Don’t call anything “culture fit”
Culture fit is the most reliable route for affinity bias into a structured process, because what it usually measures is resemblance. The candidate reminds you of people who’ve done well here, so they score well, so the team gets more like itself.
If you want to assess it, and there’s a real thing worth assessing, turn it into behavior. “Operates with high autonomy when the brief is ambiguous.” “Gives and asks for direct feedback.” “Works across functions without needing a manager to broker it.” Those you can anchor, observe and evidence.
Same move for the criteria that show up in briefings and shouldn’t: someone young is seniority and adaptability, good presence is clarity when explaining and how they handle stakeholders, fits in with the crew is a ways-of-working competency. Score the legitimate thing underneath.
The second axis, which is the part most scorecards miss
Here’s the situation. One candidate mentions, unprompted, that she led a product recall under regulatory pressure. It’s genuinely impressive and it’s nowhere in your rubric.
The instinct is to bump her score. Don’t.
Nobody else was asked about it, so adding it rewards her for a conversational accident and breaks the comparison you built the scorecard to protect. Put it on a second axis instead: extra potential, recorded separately and never added to the weighted score. Each item gets the concrete capability, the evidence, and a tag for how much it matters: minor, notable, or differential.
Then use it for what it’s good for. The rubric score ranks. Extra potential breaks ties between candidates who are close, and tells you something about runway when the role is a growth one. It never erases a flag and it never outranks the rubric.
The guardrail: it has to be a specific capability you observed. “I had a good feeling” is not extra potential.
Two small rules that prevent most of the damage
A competency you couldn’t explore is N/A, not a 1. A 1 means you saw something below the bar. Scoring absence as failure penalizes the candidate for how the interviewer managed the time. Mark it N/A, recalculate over the competencies you did assess, or flag the evaluation as incomplete and take the gap to the next round.
A critical competency below its minimum raises a flag, it doesn’t reject. The flag doesn’t change the score or the ranking. It forces a person to look at the case before anyone decides. This is worth being deliberate about: the scorecard exists to make judgment consistent, not to remove the judgment. A number should never be the last thing that happens to a candidate.
How to read the result
The weighted score gives you a number out of 4, and the bands are indicative rather than thresholds:
- Around 3.0 or above with no flags: solid.
- Between 2.0 and 3.0: worth exploring in the next round, with the specific gaps named.
- Below 2.0, or any critical flag: look at it properly before going further.
Neither the number nor the potential decides on its own. They go into the decision conversation, and what the scorecard has done is make that conversation about evidence instead of about who speaks with most conviction.
Where this fits
A scorecard tells you how to evaluate the people in front of you. It says nothing about whether the right people got in front of you, and a rigorous process applied to a weak shortlist produces a well-documented bad hire.
That earlier half is what a merit-based shortlist is for, and it’s what Nova Recruiter does: ranking 800M+ public profiles by a Talent Score built on 150,000+ candidate evaluations, rather than by how closely a profile echoes the words in your brief.
Once you’ve got the scorecard, the natural next piece is the questions that feed it. We wrote that one too: the interview guide, with what a strong answer sounds like and the red flag underneath each question.
And if you’d rather not build the rubric by hand, the method above is one of eleven free AI skills our customer success team packaged up. The Interviews pack has four: this scoring model, the candidate evaluation, the write-up and a coach for the interviewer.
Frequently asked questions
What is an interview scorecard?
A form every interviewer fills in after talking to a candidate, scoring the same four to six competencies on the same 1 to 4 scale, with a line of evidence for each score. It sits on top of a rubric that defines what each number means for each competency. The point isn't the number at the end, it's that two people who interviewed the same person can compare notes on the same terms.
How many competencies should a scorecard have?
Four to six, and six is the ceiling. Past that nobody holds the rubric in their head during a debrief, scoring slows down, and people drift back to overall impressions, which is exactly what the scorecard exists to replace. If the role genuinely needs four, use four. Padding the list with competencies that don't predict performance makes the evaluation worse, not more thorough.
How do you weight the competencies?
Assign each one a percentage and make them sum to 100. That constraint is the whole value: it forces you to decide what actually matters most for this role instead of declaring everything important. Each candidate's score is the sum of score times weight, which gives a number out of 4 you can rank by. If the weights don't add up after you adjust them, redistribute before writing anything else.
What are behavioral anchors?
A description of what a 1, a 2, a 3 and a 4 look like in each competency, written as observable behavior rather than as an adjective. The test is whether an interviewer can read the anchor and recognize it in what they actually heard. They also have to be calibrated to seniority: what counts as a 3 for a junior isn't a 3 for a director.
Can a scorecard reject a candidate automatically?
It shouldn't, and it's worth being deliberate about this. A critical competency scored below its minimum raises a flag and forces a human to look at the case before deciding. It doesn't change the score and it doesn't reject anyone. The scorecard ranks and surfaces; the decision about a person stays with the people in the room.
How do you handle culture fit in a scorecard?
Not as a competency called culture fit. Raw culture fit is the single biggest route for affinity bias, because what it usually measures is resemblance to the people already there. If you want to assess it, turn it into observable behaviors about how the team actually works, like operating with high autonomy in ambiguous situations, or giving and asking for direct feedback. Those you can anchor and evidence. "Would I get a beer with them" you cannot.
Sources and method
The method described here is the one behind Nova's Scoring Model skill, built by the customer success team from live hiring processes. Nova Recruiter figures are Nova's own, as published on novarecruiter.ai.