Play 11 Workflow Diagram (Visual)

This workflow transforms unstructured questions into cited, verifiable answers while continuously improving your knowledge base. Each stage has specific inputs, outputs, and decision points.

Stage 1: Question Intake

User submits question via Slack, email, web form, or API endpoint.

Capture these fields:

• Full question text (minimum 10 characters)
• User ID and department
• Timestamp
• Source channel
• Urgency flag (optional)

Route to processing queue. Set SLA: 95% of questions analyzed within 30 seconds.

Stage 2: Question Analysis

NLP engine extracts structured data from raw question text.

Process:

1. Tokenize question into semantic chunks
2. Extract named entities (client names, practice areas, document types)
3. Classify intent (factual lookup, procedural guidance, troubleshooting)
4. Generate embedding vector (768-dimension for BERT-based models)
5. Assign confidence score (0-100)

Output: Structured question object with topic tags, intent classification, and vector representation.

Threshold rule: Questions with confidence below 60 trigger human review before proceeding.

Stage 3: Knowledge Base Lookup

Vector search retrieves top 10 candidate articles from knowledge base.

Ranking algorithm:

• Semantic similarity (60% weight): Cosine distance between question vector and article vectors
• Metadata match (25% weight): Overlap in tags, categories, practice areas
• Recency bonus (10% weight): Articles updated in last 90 days get +5 points
• Usage signal (5% weight): Articles with high click-through rates get +3 points

Return ranked list with relevance scores. Articles scoring below 0.65 similarity are excluded.

Decision point: If top result scores above 0.85, proceed to automated answer. If top result scores 0.65-0.85, route to curation. If all results below 0.65, flag as knowledge gap.

Stage 4: Answer Curation

Subject matter expert reviews top 3 articles and constructs response.

Curator checklist:

• Does the top article directly answer the question? (Yes/No)
• What information is missing from existing articles?
• Does the answer require firm-specific context not in the knowledge base?
• Should this become a new standalone article?

Curator actions:

• Select 1-3 source articles
• Extract relevant passages (quote exactly, no paraphrasing)
• Add connecting commentary (2-3 sentences maximum)
• Insert citations with article IDs
• Flag knowledge gaps in tracking system

Time budget: 5 minutes per question. If curation exceeds 10 minutes, escalate to practice group leader.

Stage 5: Answer Delivery

Format and send response through original intake channel.

Response template:

[Direct answer in 1-2 sentences]

[Supporting detail from Article 1, with citation]

[Supporting detail from Article 2, with citation]

Sources:
- [Article Title 1] (ID: KB-1234)
- [Article Title 2] (ID: KB-5678)

Was this helpful? [Yes] [No] [Needs more detail]

Delivery SLA: 80% of curated answers delivered within 15 minutes of question submission.

Track delivery metrics: Open rate, click-through on source articles, feedback response rate.

Stage 6: Knowledge Base Update

Update knowledge base based on question patterns and gaps.

Daily review process:

• Identify questions that scored below 0.65 in lookup (knowledge gaps)
• Group similar questions by topic cluster
• Prioritize gaps by frequency (questions asked 3+ times in 7 days get immediate attention)

Weekly content sprint:

• Create new articles for top 5 knowledge gaps
• Enhance existing articles that required heavy curation (3+ curator edits)
• Retire articles with zero matches in 90 days

Article creation workflow:

1. Draft article using curated response as foundation
2. Add 2-3 examples or scenarios
3. Include decision tree or step-by-step process where applicable
4. Tag with relevant metadata (practice area, document type, jurisdiction)
5. Generate embedding vector and add to search index
6. Assign owner for quarterly review

Feedback integration:

• "Not helpful" responses trigger article review within 48 hours
• "Needs more detail" responses added to enhancement backlog
• "Helpful" responses validate article quality (no action needed)

┌─────────────────┐
│ Question Intake │
│  (Slack/Email)  │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ NLP Analysis    │
│ Extract: Topic, │
│ Intent, Vector  │
└────────┬────────┘
         │
         ▼
    Confidence?
         │
    ┌────┴────┐
    │         │
  <60%      ≥60%
    │         │
    ▼         ▼
┌───────┐  ┌──────────────┐
│Human  │  │Vector Search │
│Review │  │Top 10 Results│
└───────┘  └──────┬───────┘
                  │
                  ▼
            Top Score?
                  │
         ┌────────┼────────┐
         │        │        │
       <0.65   0.65-0.85  >0.85
         │        │        │
         ▼        ▼        ▼
    ┌────────┐ ┌────────┐ ┌──────────┐
    │Knowledge│ │Curator │ │Automated │
    │Gap Flag│ │Review  │ │Answer    │
    └────────┘ └───┬────┘ └────┬─────┘
                   │           │
                   └─────┬─────┘
                         │
                         ▼
                  ┌─────────────┐
                  │Format &     │
                  │Deliver      │
                  │with Sources │
                  └──────┬──────┘
                         │
                         ▼
                  ┌─────────────┐
                  │User Feedback│
                  │Yes/No/More  │
                  └──────┬──────┘
                         │
                         ▼
                  ┌─────────────┐
                  │Update KB:   │
                  │New Articles │
                  │Enhance Old  │
                  └─────────────┘

Question Intake Layer

• Slack app with slash command (/ask) or message listener
• Email parser (Zapier, Make, or custom IMAP handler)
• Web form with API

  API

  Click to read the full definition in our AI & Automation Glossary.

  endpoint (POST to /api/questions)
• Minimum fields: question text, user ID, timestamp

NLP Processing

• Embedding model: sentence-transformers/all-MiniLM-L6-v2 (384-dim, fast) or BAAI/bge-large-en-v1.5 (1024-dim, accurate)
• Entity extraction: spaCy with custom legal/accounting entity recognizer
• Intent classifier: Fine-tuned BERT model on 500+ labeled firm questions
• Hosting: Modal, Replicate, or self-hosted on GPU instance

Knowledge Base Platform

• Vector database

  Vector database

  Click to read the full definition in our AI & Automation Glossary.

  : Pinecone (managed), Weaviate (self-hosted), or Qdrant (hybrid)
• Content storage: Notion, Confluence, or custom PostgreSQL with full-text search
• Minimum 1,000 articles to achieve 0.70+ average match scores
• Update frequency: Real-time vector index updates on article publish

Curation Interface

• Queue management: Airtable, Linear, or custom React dashboard
• Side-by-side view: Question on left, top 3 articles on right
• One-click citation insertion
• Keyboard shortcuts for common actions (approve, edit, escalate)

Delivery System

• Slack: Post formatted message with Block Kit
• Email: HTML template with inline citations
• Web: Render Markdown with syntax highlighting
• Track opens via unique pixel or link parameter

Analytics Dashboard

• Metabase, Grafana, or custom dashboard
• Key metrics: Questions per day, average match score, curation rate, feedback distribution
• Alerts: Match score drops below 0.60 for 3+ consecutive questions

Taxonomy Structure

Three-level hierarchy:

1. Practice area (Tax, Audit, Litigation, Corporate)
2. Document type (Memo, Checklist, Template, FAQ)
3. Jurisdiction or specialty (Federal, State, Industry-specific)

Tag every article with at least one tag from each level.

Authoring Standards

Required article components:

• Title: Question format ("How do I...?") or declarative ("Filing Requirements for...")
• Summary: 1-2 sentence answer (appears in search results)
• Body: 200-800 words with headers, bullets, numbered steps
• Examples: Minimum 1 concrete scenario
• Last updated: Auto-populated timestamp
• Owner: Assigned subject matter expert

Prohibited content:

• Client names or confidential information
• Outdated regulatory references (flag for review if >12 months old)
• Vague advice without specific steps

Review Cycle

Quarterly review for all articles:

• Owner confirms accuracy and relevance
• Update "Last reviewed" timestamp even if no changes
• Archive articles with zero matches in 180 days

Immediate review triggers:

• Regulatory change affecting article content
• Three "not helpful" feedback responses
• Curator notes knowledge gap in related area

Approval Workflow

New articles:

1. Author drafts in staging environment
2. Practice group leader reviews (48-hour SLA)
3. Knowledge manager checks formatting and tags
4. Publish to production and generate vector embedding

Article updates:

• Minor edits (typos, formatting): Author publishes directly
• Content changes: Practice group leader approval required
• Major rewrites: Full approval workflow

Target Metrics (90-Day Baseline)

Question volume:

• Weeks 1-4: 50-100 questions (onboarding phase)
• Weeks 5-8: 100-200 questions (adoption phase)
• Weeks 9-12: 200+ questions (steady state)

Match quality:

• Average top result score: 0.72 or higher
• Percentage of questions with >0.85 match: 35% or higher
• Knowledge gap rate: Below 15%

Curation efficiency:

• Average curation time: 4 minutes
• Percentage requiring curation: 40-60%
• Escalation rate: Below 5%

User satisfaction:

• "Helpful" feedback rate: 75% or higher
• "Not helpful" rate: Below 10%
• Repeat question rate: Below 20%

Monitoring Dashboard

Daily view:

• Questions received (bar chart by hour)
• Average match score (line chart, 7-day rolling average)
• Curation queue depth (current count)

Weekly view:

• Top 10 knowledge gaps (questions with no good match)
• Most-cited articles (usage leaderboard)
• Curator performance (average time, feedback scores)

Monthly view:

• Question volume trend (month-over-month growth)
• Knowledge base coverage (percentage of questions with >0.70 match)
• ROI calculation (curator hours saved vs. manual Q&A)

Optimization Triggers

If average match score drops below 0.65 for 7 consecutive days:

• Audit recent questions for new topic clusters
• Review article tagging accuracy
• Consider retraining embedding model on firm-specific corpus

If curation time exceeds 6 minutes average for 14 days:

• Simplify curation interface
• Provide curator training on efficient article selection
• Evaluate whether articles are too granular or too broad

If "not helpful" rate exceeds 15% for 30 days:

• Conduct user interviews to identify disconnect
• Review answer formatting and citation style
• Test alternative response templates

This workflow converts your knowledge base from a static document repository into a self-improving Q&A system. The key is the feedback loop in Stage 6: every question that doesn't match well becomes a prompt to create better content. Firms running this workflow see curation time drop by 60% within six months as the knowledge base fills gaps and match scores improve. Start with 500 well-tagged articles, assign dedicated curators, and commit to weekly content sprints. You'll reach 0.75+ average match scores within 90 days.