11 — Embedding Models and Vector Databases: Storing Meaning at Scale

The gap between "embed my documents" and "search them fast enough" is where my first vector project stalled. The split that fixed it: an embedding model produces the vectors; a vector database stores and searches them at scale; and the search is approximate on purpose. [1][2] The two pieces are separate — I can swap the embedding model without touching the database, and swap the database without touching the model — and the "approximate" part is not a flaw but the trick that makes any of this fast enough to use.

The framing that finally landed is a pipeline with a deliberate tradeoff at the end. Text goes into an embedding model and comes out as a vector. That vector goes into a vector database alongside the original content. At query time, the query is embedded the same way, and the database finds the stored vectors closest to the query vector. The deliberate tradeoff: instead of comparing the query to every stored vector one by one (exact but slow), the database uses an approximate nearest neighbor (ANN) index that finds _mostly_ the closest vectors in a tiny fraction of the time [1]. Approximation is what lets the whole thing scale to millions of vectors.

Embedding models: the producers

The first piece is the embedding model itself — the thing that turns raw data into vectors [2]. This is separate from the chat/LLM models; an embedding model is trained specifically to produce vectors where semantic similarity maps to geometric closeness. The model choice determines the vector dimensionality, the languages and modalities supported, and the quality of the similarity signal. Everything downstream is capped by this choice, which is why picking the embedding model is the most consequential early decision in any retrieval system.

The provider landscape mirrors the chat-model landscape. OpenAI's Embeddings API turns text into vectors through a hosted service, abstracting away model training and management [3]. Google's Gemini Embedding offers text and multimodal embedding through the Gemini API [4]. Cohere's embeddings are tuned for enterprise NLP — search, clustering, retrieval [5]. Jina AI produces high-performance multilingual and multimodal embedding models with long-context support, including specialized versions for text-to-image and code retrieval [6]. On the open side, Sentence Transformers is the family of models (built on BERT/RoBERTa) that generates high-quality sentence-level embeddings and is the workhorse of most open-source retrieval work [7], and the Hugging Face Hub hosts thousands of pretrained embedding models accessible through transformers or sentence-transformers [8].

The rule I use: pick the embedding model based on the task and the constraints. Hosted (OpenAI, Gemini, Cohere) for speed of integration; open (Sentence Transformers, Jina, Hugging Face models) for cost control, local execution, or domain-specific needs. And benchmark on my own data — embedding quality varies more than the marketing suggests.

Vector databases: the stores

The second piece is the vector database — a system specialized in storing, indexing, and retrieving high-dimensional vectors [1]. Unlike a traditional database optimized for exact matches on structured fields, a vector database is optimized for _similarity_ queries: given a query vector, return the closest stored vectors. The index that makes this fast is the approximate nearest neighbor (ANN) algorithm, which trades a small amount of accuracy for a massive speedup over brute-force comparison.

The landscape splits into a few shapes, and the choice depends on how much I want to operate myself:

• Managed, purpose-built — Pinecone is a managed vector database designed for efficient similarity search at scale, handling indexing and low-latency queries as a service [9]. I reach for it when I don't want to run infrastructure.
• Open-source, self-hostable — Chroma is an open-source, AI-native vector store popular for development and smaller deployments [10]. Weaviate is an open-source vector database that supports integrating external data sources and schemas, combining structured and unstructured data [11]. Qdrant is an open-source vector database with advanced filtering, scalable indexing, and real-time updates [12].
• Library, not service — FAISS (Facebook AI Similarity Search) is a library for efficient similarity search and clustering of dense vectors, optimized to scale to billions of vectors, but it's a library I embed in my own process rather than a standalone service [13]. LanceDB is a vector database designed for scalable storage and retrieval, integrating well with ML workflows [14].
• Extensions to existing databases — Supabase Vector uses PostgreSQL's pgvector extension to provide vector storage and similarity search inside Postgres, so vector data sits alongside regular relational data [15]. MongoDB Atlas adds vector search to its document database, letting me query high-dimensional vectors alongside document data [16].

The pattern that matters most: the vector-database choice is increasingly _not_ a new-system decision. If I already run Postgres, pgvector (Supabase Vector) lets me add vector search without a new datastore. If I already use MongoDB, Atlas vector search does the same. The dedicated vector databases earn their place at scale or when I need features the extensions lack, but for most features the extension-inside-my-existing-DB path is the simpler start.

The purpose, restated

The reason this whole layer exists is one phrase: fast similarity search over unstructured data [1]. Traditional databases handle structured fields — exact matches, ranges, joins. Vector databases handle the unstructured — "find documents about the same idea as this query," "find images similar to this one," "find products a user would like based on what they liked." That capability is what makes RAG, semantic search, and recommendation systems tractable, and it's why the embedding model plus vector database combination is the backbone of almost every production AI feature that touches private data.

How I use this

Two habits fell out of this. First, I pick the embedding model before I pick the database, because the model fixes the vector dimensionality and the similarity quality, and the database has to support whatever the model produces. Second, I start with a vector extension in a database I already run (pgvector in Postgres, Atlas vector search in Mongo) and only move to a purpose-built vector database when I hit a concrete limit — scale, latency, or a feature — that the extension can't meet. The "approximate" in ANN stopped scaring me once I measured it: on real retrieval workloads the recall is high enough that users can't tell, and the speed gain is what makes the feature feel instant. Treating embedding model and vector database as separate, swappable pieces is what keeps the architecture honest.

References

[1] Cloudflare, "Vector Databases," 2024. [Online]. Available: https://developers.cloudflare.com/vectorize/reference/what-is-a-vector-database/

[2] Couchbase, "What are Embedding Models? An Overview," 2024. [Online]. Available: https://www.couchbase.com/blog/embedding-models/

[3] OpenAI, "OpenAI Embeddings API," 2024. [Online]. Available: https://platform.openai.com/docs/api-reference/embeddings/create

[4] Google, "Embeddings," Gemini API, 2024. [Online]. Available: https://ai.google.dev/gemini-api/docs/embeddings

[5] Cohere, "Introduction to Embeddings at Cohere," 2024. [Online]. Available: https://docs.cohere.com/docs/embeddings

[6] Jina AI, "Jina Embeddings," 2024. [Online]. Available: https://jina.ai/en-US/embeddings/

[7] SBERT, "SentenceTransformers Documentation," 2024. [Online]. Available: https://sbert.net/

[8] Hugging Face, "Hugging Face Embedding Models," 2024. [Online]. Available: https://huggingface.co/models?pipeline_tag=feature-extraction

[9] Pinecone, 2024. [Online]. Available: https://www.pinecone.io

[10] Chroma, 2024. [Online]. Available: https://www.trychroma.com/

[11] Weaviate, 2024. [Online]. Available: https://weaviate.io/

[12] Qdrant, 2024. [Online]. Available: https://qdrant.tech/

[13] Meta AI, "FAISS," 2024. [Online]. Available: https://ai.meta.com/tools/faiss/

[14] LanceDB, 2024. [Online]. Available: https://lancedb.com/

[15] Supabase, "Supabase Vector," 2024. [Online]. Available: https://supabase.com/docs/guides/ai

[16] MongoDB, "Vector Search in MongoDB Atlas," 2024. [Online]. Available: https://www.mongodb.com/products/platform/atlas-vector-search