"Apple Vision Pro is the most advanced personal electronics device ever created — and it marks the beginning of the spatial computing era." — Tim Cook, Apple WWDC, June 5, 2023
If you've been following AI developments this year, you've probably started seeing "vector database" mentioned more and more. Pinecone, Weaviate, Chroma, Qdrant — a whole new category of infrastructure is suddenly relevant.
Here's the thing: most explanations of vector databases start with the math and lose most readers by paragraph two. This isn't that explanation. This is the business-first version: what problem they solve, how they work at a conceptual level, and why they matter for any serious AI implementation.
The Problem: Language Models Have No Memory of Your Business
A large language model like GPT-4 is trained on an enormous amount of text from the internet. It knows a tremendous amount about the world. What it does not know is anything specific to your business.
It doesn't know your product documentation. It doesn't know your internal policies. It doesn't know what your past projects looked like or how your team solved problems last quarter. It knows nothing about your customers, your contracts, or your competitive landscape.
If you want an AI system that can actually be useful in a business context — answering questions about your products, helping onboard new employees, summarising your sales data — you need a way to give the model access to your specific information at the moment it's answering a question.
This is the problem vector databases solve.
The Solution: Semantic Search, Not Keyword Search
Traditional databases store structured data — rows and columns — and search it with exact-match or pattern-matching queries. Google "invoice" in your document management system, and it finds documents containing the word "invoice." Simple. Limited.
The problem with keyword search for AI applications: when a user asks "what payment terms do we offer clients?", they might use completely different words than appear in your actual documents. The relevant information might be in a document that talks about "net-30 terms" and "billing cycles" — no mention of "payment terms" at all.
Vector databases work differently. Instead of storing and searching text directly, they store numerical representations of meaning — called embeddings or vectors. An embedding model (like OpenAI's text-embedding-ada-002) converts a piece of text into a list of ~1,500 numbers that represents the semantic meaning of that text in mathematical space.
The magic: texts with similar meanings end up with similar vectors, even if they use completely different words. "Payment terms" and "billing cycles and net-30" would be near each other in vector space.
So when a user asks a question, you:
- Convert the question to a vector using the same embedding model
- Search the vector database for the stored chunks of text with the most similar vectors
- Return the top results — the most semantically relevant pieces of your documents
- Include those results in the prompt you send to the language model
- The model answers based on both its general knowledge and your specific retrieved context
This is Retrieval-Augmented Generation (RAG), and the vector database is the retrieval engine that makes it work.
How Documents Get Into the Vector Database
Before you can retrieve anything, you need to process your documents:
1. Collect and clean your documents. PDFs, Word files, Notion pages, database exports — whatever your knowledge lives in. This often requires more work than people expect. Scanned PDFs need OCR. Proprietary formats need conversion. Inconsistent formatting needs normalisation. 2. Chunk the documents. You split each document into smaller pieces — typically 200-500 words each, with some overlap between chunks so context isn't lost at boundaries. The right chunk size depends on your use case. 3. Embed each chunk. You pass each chunk through an embedding model, which converts it to a vector. For a thousand-page documentation library, this might generate tens of thousands of embeddings. 4. Store the embeddings. The vectors and the original text chunks go into your vector database, indexed for fast retrieval.This processing step is a one-time cost (plus incremental updates when documents change). The ongoing cost is fast, scalable retrieval at query time.
Choosing a Vector Database
The options as of mid-2023:
Pinecone — Managed cloud service. Fast, reliable, good documentation. Starter tier is around $70/month. Best for businesses that want a robust managed solution without infrastructure headaches. Chroma — Open source, can run locally or self-hosted. Free. Good for development and smaller deployments where you don't want to pay for managed services. Weaviate — Open source with managed cloud option. More feature-rich than Chroma, supports hybrid search (combining vector and keyword search). Good for complex retrieval needs. Qdrant — Another open-source option with a managed cloud tier. Good performance, Rust-based so very fast. Strong filtering capabilities.For most SMB projects we're building right now, Pinecone is the default managed choice and Chroma is the default for development environments. As the projects mature and volumes grow, we revisit.
What This Actually Enables
With a vector database and a RAG architecture, you can build:
- Internal knowledge assistants — ask questions in plain language and get answers from your actual documentation
- Customer support systems — agents that can access product specs, FAQs, and troubleshooting guides semantically
- Compliance and policy tools — HR or legal systems that can find relevant policy sections for any question
- Sales enablement — tools that surface the most relevant case studies, specs, or competitive info for any prospect conversation
The common thread: your AI stops being a generic internet model and starts behaving like an expert in your specific domain, trained on your specific knowledge.
That's the memory layer. And without it, most business AI applications are significantly less useful than they could be.
Want to implement AI in your business? Book a 30-minute architecture call at blok-ai.com.