Back to Play 10 Resources
Play 10: Hiring Screening

Play 10 Complete Implementation Guide

Full walkthrough: intake setup, screening criteria, LLM processing, summary delivery, threshold calibration.

Play 10 Complete Implementation Guide

Intake Setup

Your intake form determines what data you can screen on. Build it to capture structured, machine-readable information.

Required Fields

Configure your ATS or intake form with these exact fields:

  1. Full name (text field)
  2. Email address (validated email field)
  3. Phone number (formatted phone field)
  4. Resume/CV (PDF or DOCX upload, 5MB max)
  5. LinkedIn profile URL (optional but recommended)
  6. Years of relevant experience (dropdown: 0-2, 3-5, 6-10, 10+)
  7. Current location (text field with city/state)
  8. Work authorization status (dropdown: US Citizen, Green Card, H1B, Requires Sponsorship)

Screening Criteria Declaration

Add a visible section at the top of your intake form that states:

"This role requires: [list 3-5 hard requirements]. Applications will be screened using automated analysis. Incomplete submissions will not be reviewed."

This sets expectations and reduces unqualified applications by 30-40%.

Data Processing Consent

Include a checkbox (required) with this exact language:

"I consent to automated processing of my application materials, including analysis by AI systems, for screening purposes. I understand my data will be retained for [X months] per company policy."

Store consent timestamps in your database. This protects you legally and satisfies GDPR/CCPA requirements.

Screening Criteria Definition

Write your criteria as if you're programming a filter. Vague language produces vague results.

Minimum Qualifications (Hard Filters)

Document these as pass/fail requirements:

Education:

  • Bachelor's degree in [specific fields] OR equivalent experience
  • If no degree: 6+ years professional experience in [domain]

Experience:

  • Minimum 3 years in [specific role type]
  • Must include experience with [specific tools/systems]
  • At least 1 year managing [specific responsibility]

Technical Skills:

  • Proficiency in [Tool A, Tool B, Tool C] (must mention at least 2)
  • Experience with [specific methodology or framework]
  • Demonstrated ability to [specific measurable outcome]

Regulatory/Compliance:

  • Valid [license/certification] in [jurisdiction]
  • Clean background check (if applicable)
  • Authorized to work in [country] without sponsorship (if applicable)

Weighted Competencies (Scored Attributes)

Assign point values to desired traits. Total should equal 100 points.

Communication (25 points):

  • Clear, concise writing in cover letter
  • Professional email correspondence
  • Evidence of client-facing work

Problem-Solving (25 points):

  • Examples of complex projects completed
  • Demonstrated analytical thinking
  • Process improvement initiatives

Cultural Fit (20 points):

  • Values alignment based on application responses
  • Team collaboration examples
  • Adaptability indicators

Growth Potential (15 points):

  • Continuous learning evidence (courses, certifications)
  • Career progression trajectory
  • Initiative and self-direction

Domain Expertise (15 points):

  • Industry-specific knowledge
  • Specialized certifications
  • Published work or thought leadership

Automatic Disqualifiers

List these explicitly in your screening configuration:

  • Felony conviction within past 7 years (if role-relevant)
  • Falsified credentials or employment history
  • Requires visa sponsorship (if not offered)
  • Salary expectations exceed budget by 30%+
  • Geographic location incompatible with role requirements

LLM
Processing

Use Claude 3.5 Sonnet or GPT-4 for resume analysis. Do not use GPT-3.5 or older models.

System Prompt Configuration

Create a system prompt file (screening_prompt.txt) with this structure:

You are an expert recruiter screening candidates for a [ROLE TITLE] position at a [FIRM TYPE].

MINIMUM QUALIFICATIONS (must meet ALL):
- [Qualification 1]
- [Qualification 2]
- [Qualification 3]

WEIGHTED COMPETENCIES (score 0-100):
- Communication: [criteria]
- Problem-Solving: [criteria]
- Cultural Fit: [criteria]
- Growth Potential: [criteria]
- Domain Expertise: [criteria]

AUTOMATIC DISQUALIFIERS:
- [Disqualifier 1]
- [Disqualifier 2]

ANALYSIS INSTRUCTIONS:
1. Extract candidate name, contact info, and current role
2. Check each minimum qualification (YES/NO for each)
3. Score each competency (0-100 with brief justification)
4. Flag any disqualifiers
5. Calculate total weighted score
6. Provide 3-sentence summary: background, strengths, concerns

OUTPUT FORMAT:
Return JSON with fields: name, email, phone, qualifications_met (boolean), competency_scores (object), disqualifiers (array), total_score (number), summary (string)

API Integration

Use this Python implementation pattern:

import anthropic
import json

def screen_candidate(resume_text, cover_letter_text, system_prompt):
    client = anthropic.Anthropic(api_key="your_api_key")
    
    message = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=2000,
        system=system_prompt,
        messages=[{
            "role": "user",
            "content": f"RESUME:\n{resume_text}\n\nCOVER LETTER:\n{cover_letter_text}"
        }]
    )
    
    return json.loads(message.content[0].text)

Process resumes in batches of 10-20 to manage API

costs. Expected cost: $0.15-0.30 per candidate.

Quality Control

Run the first 50 candidates through both LLM

screening and manual review. Compare results. If agreement rate is below 85%, revise your system prompt to be more specific.

Summary Delivery

Your hiring team needs digestible information, not raw LLM

output.

Candidate Dashboard Build

Use Airtable, Notion, or a custom web app. Required views:

Top Candidates View:

  • Filter: Total score ≥ 75, all qualifications met
  • Sort: Total score descending
  • Columns: Name, Score, Top 2 Strengths, Application Date, Status

Review Queue View:

  • Filter: Total score 60-74, all qualifications met
  • Sort: Application date ascending
  • Columns: Name, Score, Concerns, Reviewer Assigned, Notes

Disqualified View:

  • Filter: Any disqualifier flagged OR qualifications not met
  • Columns: Name, Disqualification Reason, Application Date

Automated Email Notifications

Set up three notification triggers:

Daily Digest (8am):

  • Send to hiring manager
  • Include: New applications count, top 3 candidates, candidates awaiting review
  • Template: "5 new applications. Top candidate: [Name] (Score: 87). 3 candidates need review."

High-Score Alert (immediate):

  • Trigger: Candidate scores ≥ 85
  • Send to hiring manager and recruiter
  • Template: "Exceptional candidate: [Name] scored 87/100. Review resume: [link]"

Weekly Summary (Friday 4pm):

  • Send to hiring team
  • Include: Total applications, conversion rates, pipeline status
  • Attach: CSV export of all candidates

Collaboration Features

Add these fields to your dashboard:

  • Reviewer (dropdown of team members)
  • Interview Stage (dropdown: Screening, Phone Screen, First Interview, Final Interview, Offer, Rejected)
  • Internal Notes (long text field, visible only to hiring team)
  • Rating Override (number field, allows manual score adjustment with justification required)

Threshold Calibration

Your initial thresholds will be wrong. Plan to adjust them after every 25-50 candidates.

Initial Threshold Setting

Start with these baseline thresholds:

  • Auto-Advance: Score ≥ 80, all qualifications met
  • Manual Review: Score 65-79, all qualifications met
  • Auto-Reject: Score < 65 OR any qualification not met OR any disqualifier present

These are intentionally conservative. You'll loosen them as you gain confidence.

Performance Tracking

Create a calibration spreadsheet with these columns:

  • Candidate Name
  • LLM
    Score
  • Manual Review Decision (Advance/Reject)
  • Interview Performance (if interviewed, scale 1-5)
  • Hire Decision (Yes/No)
  • Hire Performance (if hired, 90-day review score)

Track these metrics weekly:

  • False Positive Rate: Candidates who scored high but performed poorly in interviews
  • False Negative Rate: Candidates who scored low but you advanced anyway and they performed well
  • Score-to-Performance Correlation: Do high scorers actually perform better?

Adjustment Protocol

After every 50 candidates, run this analysis:

  1. Calculate average score of candidates who received offers (target: 75-85)
  2. Identify score range where interview performance is strongest
  3. Check if any disqualifiers were overridden (and why)
  4. Review competency weights - are some predictive, others not?

Adjust thresholds by 5-point increments. Document every change with rationale.

Stakeholder Feedback Loop

Schedule a 30-minute calibration meeting every 2 weeks with:

  • Hiring manager
  • Lead recruiter
  • 1-2 interviewers

Agenda:

  1. Review candidates who scored 75-85 (the "maybe" zone)
  2. Discuss any surprising interview outcomes
  3. Identify screening criteria that need refinement
  4. Vote on threshold adjustments

Document decisions in your screening prompt file with version numbers and dates.

Implementation Checklist

Use this to track your rollout:

  • [ ] Intake form configured with all required fields
  • [ ] Screening criteria documented with point values
  • [ ] System prompt written and tested on 10 sample resumes
  • [ ] API
    integration built and tested
  • [ ] Candidate dashboard created with all three views
  • [ ] Email notifications configured and tested
  • [ ] Calibration spreadsheet created
  • [ ] Initial thresholds set
  • [ ] Hiring team trained on dashboard use
  • [ ] First 50 candidates processed with dual review
  • [ ] First calibration meeting scheduled

This system typically takes 2-3 weeks to build and 4-6 weeks to calibrate properly. Budget 10-15 hours of initial setup time and 2-3 hours per week for ongoing calibration.

Revenue Institute

Reviewed by Revenue Institute

This guide is actively maintained and reviewed by the implementation experts at Revenue Institute. As the creators of The AI Workforce Playbook, we test and deploy these exact frameworks for professional services firms scaling without new headcount.

Revenue Institute

Need help turning this guide into reality? Revenue Institute builds and implements the AI workforce for professional services firms.

RevenueInstitute.com