What Is a Vector Database

Vector Database

Click to read the full definition in our AI & Automation Glossary.

? (Plain English)

Your firm's knowledge base contains 10,000 documents. An associate asks: "What's our standard approach to indemnification clauses in SaaS contracts?" Traditional search returns 47 documents containing the word "indemnification." Vector search returns the three documents that actually answer the question.

That's the difference.

An embedding converts text into a list of numbers that represents its meaning. OpenAI's text-embedding-3-small model, for example, converts any text into 1,536 numbers. Similar concepts produce similar number patterns.

Here's what happens when you embed three sentences:

• "The client wants to terminate the agreement" → [0.23, -0.41, 0.67, ...]
• "The customer wishes to end the contract" → [0.25, -0.39, 0.65, ...]
• "We need more coffee in the break room" → [-0.82, 0.15, -0.34, ...]

The first two sentences produce nearly identical number patterns despite using different words. The third sentence produces a completely different pattern. The model understands synonyms, context, and intent.

This matters because keyword search fails constantly in professional services. A partner searches "client offboarding" but the relevant document uses "engagement closure procedures." Vector search finds it anyway.

Standard databases store text and retrieve exact matches. Vector databases

store embeddings and retrieve semantic matches.

Step 1: Ingestion and Embedding

You feed documents into the system. The database chunks each document (typically 500-1000 tokens per chunk), generates an embedding for each chunk, and stores both the embedding and the original text.

A 50-page engagement letter becomes 40 chunks, each with its own 1,536-number embedding.

Step 2: Indexing for Speed

The database builds an index using algorithms like HNSW (Hierarchical Navigable Small World) or IVF (Inverted File Index). These indexes let the system search millions of embeddings in milliseconds instead of hours.

Without indexing, finding similar embeddings requires comparing your query against every stored embedding. With 100,000 chunks, that's 100,000 comparisons per search. HNSW reduces this to roughly 200 comparisons with minimal accuracy loss.

Step 3: Query and Retrieval

A user asks: "What are our standard payment terms for fixed-fee engagements?"

The system embeds this question into the same 1,536-number format, searches the index for the closest matching embeddings, and returns the original text chunks. You typically retrieve the top 3-5 matches.

The returned chunks might come from your engagement letter template, your finance policy manual, and a memo about billing practices. All relevant, none containing the exact phrase "standard payment terms for fixed-fee engagements."

Knowledge Base Q&A That Actually Works

Your associates stop asking the same questions repeatedly because they can find answers themselves. "How do we handle conflicts checks for subsidiaries?" returns your actual conflicts policy, not 30 documents that mention the word "conflicts."

Implementation: Pinecone or Qdrant for the vector database

, OpenAI embeddings, and a simple Python script to chunk and ingest your documents. Total setup time: 4-6 hours for a 1,000-document knowledge base.

Client Intake Automation

When a prospect submits an RFP, vector search instantly surfaces your three most similar past proposals, the relevant subject matter experts, and potential conflicts. What took 2 hours now takes 2 minutes.

Precedent and Template Matching

An attorney needs a non-compete clause for a senior executive in the healthcare industry. Vector search finds the five most similar clauses you've drafted previously, ranked by relevance. No more scrolling through 200 saved documents hoping to spot the right one.

Expertise Location

"Who knows about R&D tax credits for SaaS companies?" Vector search checks every document, email, and project note in your system and identifies the three people who've worked on similar matters most recently.

Week 1: Choose Your Stack

For firms under 50 people: Pinecone (managed service, $70/month to start) For firms over 50 people: Qdrant (self-hosted, more control, free) For embedding model: OpenAI text-embedding-3-small ($0.02 per 1M tokens)

Week 2: Prepare Your Data

Export your knowledge base to markdown or plain text. Remove formatting artifacts. Split documents into logical sections (one section = one chunk). Aim for 300-800 words per chunk.

Bad chunking: Splitting mid-sentence or mid-paragraph Good chunking: One complete policy section, one complete FAQ answer, one complete template with its explanation

Week 3: Ingest and Test

Use LangChain or LlamaIndex to handle chunking and embedding automatically. Ingest 100 documents first. Test with 20 real questions your team has asked recently. Adjust chunk size if results are poor.

If answers are too vague: Reduce chunk size to 200-400 words If answers lack context: Increase chunk size to 800-1200 words

Week 4: Build the Interface

Create a Slack bot or simple web form that accepts questions, queries your vector database

, and returns the top 3 matching chunks. Use GPT-4 to synthesize the chunks into a coherent answer with source citations.

Total cost for 10,000 queries per month: $50-100 (embedding costs) + $70 (Pinecone) + $200 (GPT-4 synthesis) = $320/month.

Mistake 1: Embedding Everything at Once

Start with your 200 most-accessed documents. Prove value before ingesting your entire document archive from 1987.

Mistake 2: No Metadata Filtering

Store metadata with each chunk (document type, practice area, date created). Let users filter: "Find indemnification clauses, but only from technology contracts drafted after 2022."

Mistake 3: Ignoring Chunk Overlap

Use 10-20% overlap between chunks. If a key concept spans two chunks, overlap ensures it appears in at least one complete chunk.

Mistake 4: Skipping Evaluation

Track which searches return useful results and which don't. After 100 queries, you'll see patterns. Adjust your chunking strategy, add metadata, or improve your document formatting.

Vector databases

turn your knowledge base from a document graveyard into a system that answers questions. The technology is production-ready, the costs are reasonable, and the implementation is straightforward.

Start with one high-value use case. Prove it works. Expand from there.

Your associates will stop interrupting partners with questions they could answer themselves. Your partners will stop recreating work that already exists somewhere in the system. Your firm will actually use the knowledge it's spent 20 years accumulating.

That's worth the four hours it takes to set up.