Interview Rating Scales: 1-5 vs Behaviorally Anchored
Interview Rating Scales: 1-5 vs Behaviorally Anchored
Most hiring teams reach for a 1-5 interview rating scale because it's fast to set up and everybody already knows what a 4 feels like. That last part is the problem. A bare 1-5 scale invites two predictable failures: central tendency bias, where nervous raters park everyone at a 3, and inflation, where a likable candidate floats to a 4 nobody can defend. Behaviorally anchored rating scales (BARS) fix this by writing down what each level looks like in observable terms before the interview starts. The catch is that BARS costs real time to build, and not every role is worth that bill.
So the honest comparison isn't "numeric scale bad, anchored scale good." It's a question of the defensibility tax: how much will it cost you, in build effort and interviewer discipline, to make a score survive scrutiny later? A reqs that closes once a year can run a cheap scale. A role you hire for fifty times across four teams, where a rejected candidate might ask why, needs scores that hold up. This guide compares the two scales head to head, shows you how to build a BARS when you need one, and tells you when the cheaper option is the right call.
Why a Bare 1-5 Scale Drifts
A bare 1-5 scale fails because it stores a private judgment as if it were a public fact, and the defensibility tax comes due the moment two people compare numbers. When the scale labels only the endpoints, or labels nothing at all, each interviewer fills the gap with their own standard. One person's 4 is another's 3, and neither can explain the difference because the difference was never written down. The number looks like data. It's a feeling wearing a numeric costume.
That feeling has two well-documented directions. The first is central tendency bias, which one statistics reference defines as raters "disproportionately [assigning] scores that cluster around the middle" of a scale while avoiding the extreme ends (Psychological Statistics). A panel full of 3s tells you nothing, because the scale's range collapsed before anyone interpreted a single answer. The second direction is leniency, where likability and recency push scores up, so the strong-on-paper candidate everyone enjoyed talking to lands a 4 the transcript can't back. Both errors trace to the same gap: the scale never said what a 3 or a 4 actually requires.
That missing definition is exactly what an anchored scale supplies, which is why the comparison comes down to whether your roles can afford to define it.
Numeric vs Behaviorally Anchored: The Head-to-Head
Here's the tradeoff laid out. Read it as a defensibility-tax ledger: the bare scale is cheap to deploy and expensive to defend, the BARS is expensive to build and cheap to defend.
| Dimension | Bare 1-5 numeric scale | Behaviorally anchored scale (BARS) |
|---|---|---|
| What a level means | Endpoints labeled at best; each rater decides privately | Each level has a written, observable behavior example |
| Build cost | Minutes. Pick a scale, ship it | Days to weeks. Job analysis, critical incidents, retranslation |
| Interrater agreement | Low. One rater's 4 is another's 3 | Higher. Raters match an answer to a description, not a mood |
| Central tendency / inflation | High exposure to both | Reduced, because levels are concrete |
| Defensibility in a debrief or audit | Weak. Hard to explain the number | Strong. The score points to a defined behavior |
| Honest limitation | Drifts fast without anchors | Still imperfect; leniency survives, and it's costly to maintain |
| Best when | Low-volume, lower-stakes roles; early pilots | High-volume, repeated, or legally sensitive roles |
The table makes the contrarian point concrete: BARS wins on every quality dimension and loses on cost, so the decision isn't about which scale is better in the abstract. It's about whether a given role generates enough hiring volume or enough scrutiny to repay the build. Before you commit to that build, it helps to see what the build actually involves.
How to Build a Behaviorally Anchored Scale
If a role clears the defensibility-tax threshold, BARS construction follows a defined sequence rather than a brainstorm. The AIHR BARS guide lays out the core steps, and the order matters because each one feeds the next.
- Collect critical incidents. Ask people who do or supervise the job for specific examples of effective and ineffective behavior on the competency you're scoring. Aim for concrete moments, not adjectives. "Walked the panel through a tradeoff they'd missed" beats "good communicator."
- Group incidents into performance dimensions. Sort the raw examples into the handful of dimensions the question actually tests, so you're not anchoring five overlapping things at once.
- Retranslate to confirm placement. Have a separate group of reviewers independently assign each incident back to a dimension and a level. Keep only the incidents they agree on. This retranslation step is what separates a real BARS from a wish list.
- Assign scale values. Place the surviving incidents at their level on the 1-5 range, so each number now has an observable example attached.
- Write the final scale and pressure-test it. Edit the anchors for clarity, then dry-run them on a few recorded answers to confirm two raters land within a point of each other.
When we pulled apart the scorecards in-house teams brought into Asked, we found that most "anchored" scales had skipped step three entirely: someone wrote level descriptions solo and called it a day. The retranslation step is the unglamorous middle that makes anchors hold, and it's the first thing teams cut when they're rushing. For the rule that keeps each anchor observable rather than a quality, the Interview Rubric Template: Score Candidates Consistently lays it out with a copyable row. Skip it and you've paid the BARS build cost without buying the BARS reliability.
A Quick Decision Checklist
Run a role through this before you decide which scale to invest in:
- We hire for this role often enough that the build cost amortizes across many interviews.
- Scores from this role get compared across interviewers or teams who never talk.
- A rejected candidate could plausibly ask us to justify the decision.
- We have access to people who do the job and can supply real behavior examples.
- We're willing to run the retranslation step, not just write level labels solo.
- If most boxes are unchecked, ship a lightly anchored 1-5 (one written line per level plus a required evidence note) and revisit later.
For the full structure that holds these ratings, see our pillar on the Interview Scorecard: How to Build One That Actually Gets Used, which covers the evidence line that makes any scale honest.
What BARS Won't Fix
BARS is a sharper instrument, not a clean one, and pretending otherwise is how teams get complacent. The honest read comes straight from the people who study it. As the AIHR guide puts it, "a BARS removes or reduces many bias errors, but it doesn't eliminate leniency errors." A motivated rater can still nudge a likable candidate up a level. Worse, BARS may "still suffer from unreliability, leniency bias and lack of discriminant validity between performance dimensions" (Behaviorally anchored rating scales), meaning your carefully built dimensions can blur into each other if the job analysis was thin.
There's a deeper finding here that should change how you spend your effort. A critical review cited in that same source concluded that the strength of BARS "may lie primarily in the performance dimensions which are gathered rather than the distinction between behavioral and numerical scale anchors." Put bluntly: the value isn't the anchor wording, it's the job analysis that forced you to define what good looks like. That's why a lightly anchored 1-5, built on the same honest dimensions and paired with a required evidence note, captures most of the benefit at a fraction of the defensibility tax. To see what those role-specific dimensions look like in practice, the Interview Scorecard Examples: 5 Real Formats by Role Type anchor the same 1-5 differently for engineering, sales, support, manager, and exec hires.
Frequently Asked Questions
What's the difference between a numeric scale and a behaviorally anchored one?
A numeric 1-5 scale gives raters numbers and trusts them to share a definition. A behaviorally anchored scale attaches a written, observable behavior to each number, so the definition is on the page instead of in someone's head. The anchors are what stop one interviewer's 4 from meaning another's 3.
Is a 1-5 scale or a 1-4 scale better for interviews?
An even-numbered scale (like 1-4) removes the neutral midpoint, which forces raters off the fence and cuts central tendency bias. That's its main advantage over an odd-numbered 1-5. The scale length matters far less than whether each level is defined and tied to evidence from the transcript.
Are BARS worth the cost for a small team?
Usually not for a role you hire once or twice a year. BARS is time-consuming and expensive to build, so the cost only amortizes across high-volume or high-scrutiny roles. Smaller teams get most of the benefit from a lightly anchored 1-5 with a required evidence note.
Do This Next
Pick one high-volume role where interviewer scores currently disagree the most. Build a lightly anchored 1-5 for its top three competencies, writing one observable line per level instead of a full BARS. Score your next two candidates against it, then compare interviewer numbers and see where they diverge. Use that gap to decide whether the role earns a full behaviorally anchored build or whether the lighter scale already holds. Start today: try Asked free and let the interview agent attach the transcript evidence to each rating, so your scale measures what was said, not how it felt.