Building Your Interview Process From Scratch

If you are building an interview process for the first time, the good news is that a structured interview process is not complicated to set up. The bad news is that most companies skip the setup entirely and end up with an interview process that is really just “whoever is available asks whatever they feel like.” That approach feels natural and conversational, but it is also roughly half as predictive of job performance as a structured process. Building your interview process from scratch gives you the opportunity to do it right from the beginning, before bad habits calcify.

This is the third post in our First-Time Hiring Guide series. The first post covered the end-to-end hiring playbook, and the second post covered writing effective job descriptions. Now we get into the mechanics of the interview itself.

Why Structure Matters More Than Interviewing Skill

Here is a fact that surprises most people: structured interviews are approximately twice as predictive of job performance as unstructured interviews, according to meta-analyses published in the Journal of Applied Psychology. The interviewer's experience level matters far less than whether the interview is structured.

This means a first-time hiring manager using a structured process will make better hiring decisions than a veteran interviewer winging it. Structure compensates for inexperience, reduces bias, and creates a defensible record of how and why you made your decision.

Structure means three things:

1. Every candidate for the same role gets the same questions
2. Every candidate is evaluated on the same criteria using the same scoring system
3. Interviewers score independently before discussing impressions

That is the entire principle. Everything else in this guide is about how to implement it practically.

Step 1: Design Your Scorecard

The scorecard is the foundation of your interview process. It defines what you are looking for and how you will measure it. Build your scorecard before you write a single interview question.

What Goes on the Scorecard

A good scorecard has 6-10 criteria across three categories:

1. Technical competencies (2-4 criteria): The skills and knowledge directly required for the role. For a software developer: code quality, system design, debugging approach. For a sales rep: prospecting methodology, objection handling, pipeline management. These should map directly to the responsibilities in your job description.
2. Behavioral competencies (2-4 criteria): The soft skills and working style that predict success. These are role-specific, not generic. “Communication skills” is too vague. “Ability to explain technical concepts to non-technical stakeholders” is actionable and measurable.
3. Values and working style (1-2 criteria): How the person approaches work, collaborates with others, and aligns with your team's standards. Define these in behavioral terms: “takes ownership of problems rather than escalating prematurely” rather than “good culture fit.”

Building the Scoring Rubric

For each criterion, define what a 1 through 5 score looks like:

• 1 — Does not meet expectations. No evidence of the competency. Candidate cannot articulate relevant experience or approach.
• 2 — Below expectations. Limited evidence. Candidate shows basic awareness but lacks depth or practical experience.
• 3 — Meets expectations. Solid evidence of competency. Candidate can describe relevant experience with specific examples and clear results.
• 4 — Exceeds expectations. Strong evidence. Candidate demonstrates deep expertise, nuanced understanding, and a track record of applying the competency effectively.
• 5 — Exceptional. Outstanding evidence. Candidate clearly operates at a level above what the role requires, with examples that demonstrate leadership or innovation in this area.

Writing role-specific descriptions for each score level takes time upfront but pays off massively. When interviewers know what a “3” looks like vs a “4,” scoring becomes consistent and calibrated, even across different interviewers.

Step 2: Write Your Interview Questions

With your scorecard defined, every interview question should map to at least one criterion. No orphan questions that sound interesting but do not help you evaluate what matters.

Types of Questions That Work

Behavioral Questions

Format: “Tell me about a time when...”

These are the backbone of a structured interview. They ask candidates to describe specific past experiences, which is the strongest predictor of future behavior. The key is specificity — you want a real example with context, action, and result, not a hypothetical or generalization.

Examples:

• “Tell me about a time you had to learn something new quickly to complete a project. What was the situation, what did you do, and what was the outcome?”
• “Describe a situation where you disagreed with a colleague about the approach to a project. How did you handle it?”
• “Tell me about the most complex problem you solved in your last role. Walk me through your approach.”

Situational Questions

Format: “How would you handle...”

These are useful for scenarios specific to your role that a candidate may not have encountered before. They test problem-solving approach and judgment.

Examples:

• “You receive a customer complaint about a feature that is working as designed but does not meet their expectations. How would you handle this?”
• “You are halfway through a project when the priorities shift and your manager tells you to pivot to something new. How do you approach the transition?”

Role-Specific Technical Questions

Format varies: could be a walkthrough, a problem to solve, or a portfolio review.

For technical roles, include at least one question that requires the candidate to demonstrate the skill, not just describe it. A coding challenge, a design critique, a writing sample review, or a walkthrough of a relevant project. The complexity should match the role — do not give a senior-level problem to a junior candidate or vice versa.

Questions to Avoid

• “What is your greatest weakness?” Generates rehearsed non-answers. Instead: “Tell me about a piece of feedback you received that changed how you work.”
• “Where do you see yourself in five years?” Tests the candidate's ability to manufacture an answer, not anything useful. Instead: “What kind of work do you find most energizing? Least energizing?”
• Brain teasers and trick questions. “How many golf balls fit in a school bus” tests the candidate's ability to answer brain teasers, which has zero correlation with job performance. Google famously abandoned these questions after internal analysis showed they were useless.
• Questions about protected characteristics. Questions about age, family status, religion, health, national origin, or any protected category are illegal, even if asked casually.

Step 3: Structure the Interview Flow

A well-structured interview has a rhythm that helps the candidate perform their best and helps you collect the data you need. Here is a framework for a 45-60 minute interview:

1. Opening (5 minutes): Welcome the candidate. Brief introduction to yourself and the interview format. Set expectations: “I will ask you several questions about your experience. I will take notes as we go. There will be time at the end for your questions.” This reduces anxiety and signals professionalism.
2. Core questions (30-40 minutes): Ask your prepared questions in the same order for every candidate. Listen for specific examples with context, action, and results. Follow up when answers are vague: “Can you give me a specific example?” or “What was the result of that approach?”
3. Candidate questions (10-15 minutes): Let the candidate ask their questions. This is not a formality — the quality of their questions tells you a lot about their engagement, preparation, and priorities. Candidates who ask about the work, the team dynamics, or the challenges of the role are generally more engaged than those who only ask about benefits and time off.
4. Closing (2-3 minutes): Thank the candidate. Tell them the next steps and timeline: “We are interviewing through the end of this week and will have a decision by [date]. You will hear from us either way.” Then follow through.

Step 4: Build Your Scoring Discipline

Scoring is where most interview processes fall apart. Interviewers conduct good interviews, form vague impressions, and then try to remember and compare those impressions days later. By then, the recency effect has taken over — the last candidate interviewed is remembered most clearly, and everyone else blurs together.

Score During the Interview

After each core question (or at natural transition points), score the candidate's response against the relevant criterion on your rubric. This takes practice — you are listening, engaging, and evaluating simultaneously. But even rough in-the-moment scores are more accurate than post-interview reconstructions.

Score Before Discussing

If multiple people interview the same candidate, each interviewer must complete their scorecard independently before any group discussion. This is non-negotiable. The moment one interviewer shares their opinion, it anchors everyone else. Independent scoring preserves the value of having multiple perspectives.

Use the Full Range

Most interviewers default to 3s and 4s for everything. Push yourself to use the full 1-5 range. If a candidate gave a vague, generic answer with no specific examples, that is a 2, not a 3. If a candidate gave a deeply thoughtful answer with a nuanced example and clear results, that is a 5, not a 4. Differentiated scores produce differentiated comparisons.

Step 5: Design Your Interview Panels

For small teams, the “panel” might be just you and one other person. That is fine. Even two independent perspectives are better than one. Here are principles for designing who interviews whom:

• The hiring manager interviews every candidate. This is the person who will work most closely with the new hire. Their assessment is the most important.
• A peer interviews if possible. Someone who does similar work or who will collaborate closely with the new hire. They evaluate different things than the manager — particularly technical competency and working style compatibility.
• Divide the scorecard. If you have two interviewers, each one should focus on different criteria. This prevents redundant questions and ensures full coverage of your scorecard.
• Use the same panel for all candidates. If Candidate A is interviewed by the CEO and a senior developer, and Candidate B is interviewed by the CEO and a junior developer, the comparisons are unreliable. Consistency in panels matters as much as consistency in questions.

Step 6: Run the Debrief

The debrief is where individual interview data gets synthesized into a hiring decision. A good debrief is structured, data-driven, and decisive. A bad debrief is a free-form conversation dominated by whoever has the strongest personality.

The Debrief Framework

1. Share scores first, opinions second. Start by having each interviewer read out their scores for each criterion. No commentary yet — just numbers. This forces the conversation to begin with data rather than impressions.
2. Discuss areas of disagreement. Where interviewers scored differently, discuss the specific evidence behind each score. One interviewer may have probed a criterion more deeply and surfaced information the other did not. The goal is not to negotiate scores to agreement but to understand what evidence each person saw.
3. Compare candidates on the scorecard. Once all candidates have been debriefed, compare their aggregate scores. Look at both total scores and patterns — a candidate with moderate scores across the board may be less interesting than one with high scores on your most important criteria and lower scores on secondary criteria.
4. Make the decision. The scorecard data should inform the decision, not dictate it. There may be legitimate reasons to choose a candidate who did not score highest — cultural factors, team composition, potential for growth. But those reasons should be articulated and recorded, not used as a license to override data with gut feel.

For teams using assessment data alongside interviews, platforms like PersonaScore can aggregate personality assessment results, interview scores, and team fit data into a structured comparison that makes the debrief dramatically more productive. Instead of comparing notes from memory, you are comparing structured data side by side.

The Multi-Round Interview Structure

For most roles, a single interview is not enough. Here is a common multi-round structure that works for small businesses:

Round 1: Phone Screen (15-20 Minutes)

Purpose: logistics and basic qualification verification. Confirm salary expectations, availability, location/remote requirements, and top-line experience. This round eliminates candidates who are not viable for practical reasons, saving interview time for substantive evaluation.

Round 2: Assessment (Async, 30-60 Minutes)

Purpose: objective data collection before the interview. This could be a skills assessment, a personality assessment, a work sample, or a combination. The assessment results inform the interview questions, making the conversation more targeted and productive.

Round 3: Structured Interview (45-60 Minutes)

Purpose: deep evaluation against scorecard criteria. This is the main event. Use the question framework described above, informed by assessment results where available.

Round 4: Final Conversation (30 Minutes, Optional)

Purpose: addressing remaining questions from either side. This might be a conversation with a different team member, a deeper dive into a specific competency area, or an informal meeting to give the candidate a feel for the team culture. Not every role needs this round; use it when the decision is close and additional data would be genuinely useful.

Common Interview Process Mistakes

1. Too many rounds. Every round you add costs the candidate time and patience. If you need more than four rounds (including phone screen) for a non-executive role, your process is too long. Top candidates will drop out before you finish.
2. No timeline communicated to candidates. Candidates who do not know where they stand or when they will hear back start accepting other offers. Communicate your timeline at every stage and stick to it.
3. Different questions for different candidates. This is the single most common structural error. When each candidate gets different questions, you cannot compare them meaningfully. The comparison becomes: “I liked Candidate A's answer about teamwork and Candidate B's answer about leadership.” Those are not comparable data points.
4. Not taking notes. Memory is unreliable. Write down key quotes, examples, and observations during the interview. Your notes are the evidence behind your scores.
5. Letting one interview override data. A charismatic candidate can dazzle in conversation but score poorly on competencies that matter. Trust the scorecard. Charm is not a criterion unless the role specifically requires it.

Building the Process Once, Using It Forever

The upfront investment in building a structured interview process pays off with every subsequent hire. Once you have a scorecard template, a question bank organized by competency, a scoring rubric, and a debrief framework, each new hire requires adaptation rather than creation from scratch. Modify the scorecard for the specific role, select relevant questions from your bank, and run the same proven process.

After each hire, spend 30 minutes reviewing what worked and what did not. Did the scorecard capture the right criteria? Were there questions that consistently produced useful answers and others that fell flat? Did the debrief surface the right information? This iterative improvement is what separates a good process from a great one.

The Bottom Line

An interview process is not a conversation. It is a measurement tool. The better designed the tool, the better the measurements, and the better your hiring decisions. You do not need to be a professional interviewer to run a great interview process — you need a scorecard, consistent questions, disciplined scoring, and a structured debrief. Build that system once, and every hire you make from this point forward will be measurably better.

For the complete context, see the other posts in the First-Time Hiring Guide series: How to Hire When You've Never Hired Before, How to Write a Job Description That Actually Attracts the Right People, and next up: Making the Offer and Not Losing the Candidate. And for the full structured hiring framework, see our complete guide to structured hiring for small businesses.