Book Errata & Corrections

This document tracks corrections, clarifications, and updates to The AI Workforce Playbook. When we find errors or when technology changes materially affect the guidance, we update this page. Bookmark it.

Last updated: [Current Date]

Chapter 2: Evaluating AI Capabilities

Page 47, "Assessing Model Accuracy" section

The accuracy formula was incomplete. Here's the correct version:

Accuracy = (True Positives + True Negatives) / Total Predictions

Where Total Predictions = TP + TN + FP + FN

Why this matters: The original formula omitted True Negatives, which would give you artificially low accuracy scores for models that correctly identify negative cases. If you're evaluating a contract review AI that flags risky clauses, you need to count both the clauses it correctly flags AND the safe clauses it correctly ignores.

Action required: If you built evaluation spreadsheets using the old formula, recalculate your accuracy scores. The corrected formula is now in the online version of Chapter 2.

Chapter 5: Deploying AI in Production

Page 112, "Monitoring Model Performance" section

The original text didn't provide specific threshold guidance. Here's what you actually need:

Set thresholds based on error cost, not arbitrary percentages.

For client-facing work where errors damage relationships:

• Document review AI: 98%+ precision (minimize false positives that waste attorney time)
• Invoice coding AI: 95%+ recall (catch every billable item, even if you flag some non-billables)
• Client intake chatbots: 90%+ intent accuracy (misrouting a prospect is expensive)

For internal efficiency tools where errors are recoverable:

• Meeting summarization: 85%+ accuracy (humans review anyway)
• Email categorization: 80%+ precision (misfiled emails get found eventually)
• Research assistance: 90%+ source accuracy (attorneys verify citations)

How to set your thresholds:

1. Calculate the cost of a false positive (wasted time reviewing a non-issue)
2. Calculate the cost of a false negative (missing a real issue)
3. Set your precision/recall balance based on which error costs more
4. Test with a pilot group and adjust based on their tolerance for errors

Don't use "99% accuracy" as a blanket requirement. A 95% accurate tool that saves 10 hours per week beats a 99% accurate tool that saves 2 hours.

Appendix A: AI Vendor Evaluation Checklist

New section added: Responsible AI Practices

Add these questions to your vendor evaluation scorecard:

Bias Testing & Mitigation

• Do they test for demographic bias across protected classes? (Ask for their testing methodology, not just "yes")
• Can they show you bias audit results from their last three model updates?
• What's their process when bias is detected? (Acceptable answer: retrain with balanced data, adjust decision thresholds, add human review. Unacceptable: "Our AI doesn't have bias.")

Explainability

• Can the system show which input factors drove each decision?
• Will you get explanations in plain language, not just feature importance scores?
• Can you export explanation data for your own audit trail?

Data Handling

• Where is your data stored during training? (US-based servers matter for compliance)
• Do they train their general models on your data? (This should be "no" for professional services firms)
• Can you request complete data deletion? How long does it take?

Human Oversight

• Can you configure mandatory human review for high-stakes decisions?
• Does the system flag low-confidence predictions for review?
• Can you override AI decisions and feed that back into the model?

Scoring: Require satisfactory answers to all four categories before shortlisting a vendor. One weak area (especially data handling) disqualifies them.

Chapter 3: Building the AI-Powered Workforce

Page 68, "Upskilling Existing Employees" section

The original text undersold the importance of continuous learning. One training session doesn't create AI-capable employees.

What works: Structured, ongoing skill development

Month 1-2: Baseline Assessment

Use this three-part evaluation:

1. Technical Skills Test: 30-minute online assessment covering:

   • Prompt engineering basics (can they write a clear instruction?)
   • Data literacy (can they spot bad data in a spreadsheet?)
   • Tool familiarity (have they used ChatGPT, Claude, or similar?)
2. Practical Exercise: Give them a real work task:

   • "Use AI to summarize these five client emails and draft responses"
   • "Create a project timeline from these meeting notes using AI assistance"
   • Evaluate output quality and their process, not just the final result
3. Role-Specific Competency Mapping:

   • Partners: AI strategy, vendor evaluation, risk assessment
   • Senior associates: Advanced prompting, workflow automation, quality control
   • Junior staff: Tool proficiency, data preparation, output verification

Score each person as Beginner/Intermediate/Advanced in each competency. This tells you who needs what training.

Month 3-6: Intensive Upskilling

Run weekly 60-minute sessions:

• Week 1: Prompt engineering fundamentals (with live practice)
• Week 2: Document analysis automation (using your actual documents)
• Week 3: Research acceleration techniques (with your research databases)
• Week 4: Quality control and verification (catching AI errors)

Assign homework: "Use this week's technique on a real client project. Report results in Slack."

Month 7-12: Embedded Learning

• Bi-weekly "AI Office Hours" where people bring real problems
• Monthly "AI Wins" showcase where teams demo successful implementations
• Quarterly skills re-assessment to track progress

Identify and empower AI champions:

Pick 2-3 people per department who score Advanced in the baseline assessment. Give them:

• 4 hours per week dedicated to AI experimentation
• Budget to attend AI conferences or take advanced courses
• Responsibility to run the weekly training sessions
• Recognition (title, bonus, or promotion consideration)

Measuring success:

Track these metrics monthly:

• Percentage of employees using AI tools weekly (target: 80%+ by month 6)
• Average time saved per person (survey monthly, target: 3+ hours/week)
• Number of AI-enhanced projects completed (target: 2+ per person per quarter)
• Employee confidence scores (1-5 scale survey, target: 4+ average)

If you're not hitting these targets by month 6, your training program needs redesign.

Chapter 7: Governing the AI Lifecycle

Page 156, "Establishing AI Governance Policies" section

The original guidance was too conceptual. Here's the operational framework you need.

AI Risk Assessment Framework

Use this scoring rubric for every AI project:

Impact Severity (if the AI makes an error):

• 1 point: Minor inconvenience (email misfiled, meeting notes incomplete)
• 3 points: Moderate business impact (client deliverable needs rework, billing error)
• 5 points: Major consequences (regulatory violation, client relationship damage, revenue loss)

Data Sensitivity:

• 1 point: Public information only
• 3 points: Internal business data
• 5 points: Client confidential data, PII, or regulated data

Bias Risk:

• 1 point: No human-related decisions (document formatting, scheduling)
• 3 points: Indirect human impact (workload distribution, project assignments)
• 5 points: Direct human impact (hiring, performance evaluation, client selection)

Explainability Requirement:

• 1 point: Black box acceptable (spell check, grammar suggestions)
• 3 points: General explanation needed (why this document was flagged)
• 5 points: Detailed justification required (why this candidate was rejected)

Total Score Determines Governance Level:

• 4-8 points: Low risk (manager approval, quarterly review)
• 9-14 points: Medium risk (director approval, monthly monitoring, bias audit every 6 months)
• 15-20 points: High risk (governance board approval, weekly monitoring, quarterly bias audit, mandatory human review)

AI Approval Workflow

For Low-Risk Projects (4-8 points):

1. Department manager reviews one-page proposal
2. IT confirms technical feasibility and security
3. Approval granted within 5 business days
4. Quarterly performance review

For Medium-Risk Projects (9-14 points):

1. Project sponsor submits detailed proposal (use template in Appendix B)
2. Technical review by IT (security, integration, performance)
3. Legal review (compliance, contracts, liability)
4. Risk assessment by governance coordinator
5. Director approval required
6. Monthly performance dashboard review
7. Bias audit every 6 months

Timeline: 15 business days

For High-Risk Projects (15-20 points):

1. Full business case with ROI analysis
2. Technical architecture review (IT + external consultant if needed)
3. Legal and compliance deep dive (include outside counsel for regulated work)
4. Ethics review (partner-level discussion of potential harms)
5. Pilot program required (minimum 30 days, 10+ users)
6. Governance board presentation and approval
7. Phased rollout with weekly monitoring
8. Quarterly bias audit and annual third-party audit

Timeline: 45-60 business days

Ongoing Monitoring Requirements

Build these dashboards (update weekly):

Performance Dashboard:

• Accuracy, precision, recall vs. thresholds
• Error rate trend (should decrease over time)
• User satisfaction score (monthly survey)
• Time saved per user (tracked automatically)
• Cost per transaction (AI cost vs. human cost)

Bias Dashboard:

• Prediction distribution across demographic groups (if applicable)
• Error rate by group (flag if any group has 10%+ higher error rate)
• User feedback by group (survey quarterly)
• Manual override rate by group (if humans frequently override AI for one group, investigate)

Compliance Dashboard:

• Data access logs (who accessed what data when)
• Retention compliance (is old data being deleted on schedule?)
• Vendor SLA performance (uptime, response time, support tickets)
• Security incidents (even minor ones)

Set up automatic alerts:

• Performance drops below threshold (email to project owner immediately)
• Bias metric exceeds 10% variance (email to governance board within 24 hours)
• Security incident detected (email to CTO and governance board immediately)
• User satisfaction drops below 3.5/5 (email to project owner weekly)

Quarterly Governance Review Agenda:

1. Review all active AI projects (15 min per project)
2. Discuss flagged bias or performance issues (30 min)
3. Approve new high-risk projects (45 min)
4. Update risk assessment rubric based on lessons learned (30 min)
5. Plan next quarter's AI initiatives (30 min)

Total meeting time: 2-3 hours quarterly

Annual Third-Party Audit (for high-risk systems):

Hire an external AI auditor to review:

• Model performance on held-out test data
• Bias testing across protected classes
• Data handling and security practices
• Compliance with stated policies
• Comparison to industry benchmarks

Budget: $15,000-$50,000 depending on system complexity

This governance framework scales with risk. Low-risk tools move fast. High-risk tools get the scrutiny they deserve.