10 — What Are Embeddings? Meaning as Position in Vector Space

"Just vectors for search" was how I waved away embeddings, and it hid the real idea: an embedding is a learned coordinate for a piece of data, where distance in the space encodes semantic similarity — nearby points mean similar things. [1] Once I saw it as coordinates on a map of meaning rather than an opaque array of numbers, every downstream use (search, classification, recommendation, anomaly detection) became the same operation: _measure distance_.

The framing that finally landed is a map. An embedding model takes text (or an image, or audio) and outputs a fixed-length vector — a list of a few hundred or thousand numbers. That vector is the _coordinate_ of the input in a high-dimensional space the model learned during training. The model's training organized the space so that inputs a human would call "similar" land close together, and inputs we'd call "different" land far apart. The geometry _is_ the meaning.

The core mechanic: similarity as distance

Strip the geometry away and the operation is always the same. To compare two things, I compute the distance between their embedding vectors — usually cosine similarity (the angle between them) or Euclidean distance (the straight-line distance). Small distance, or high cosine similarity, means the model considers them semantically close. This works even when the inputs share no words at all: "kitty" and "feline" can land near "cat" without any token overlap, because the model learned the relationships from how words are used in context during training [1][2].

That single mechanic — _measure distance_ — is what makes embeddings so broadly useful. Once data has coordinates, every "find similar things" question becomes a nearest-neighbor lookup.

Semantic search: matching meaning, not keywords

The most direct application is semantic search. Traditional keyword search matches on exact terms — if the document doesn't contain the word I searched for, it won't be returned, even if it's about exactly the same thing. Semantic search embeds both the query and every document, then returns the documents whose vectors are nearest the query vector [3]. A search for "feline friend" can find a document about a "cat" because their embeddings are close, even with zero word overlap.

The practical win is that semantic search handles synonyms, paraphrase, and vocabulary mismatch that keyword search cannot. The cost is that I need an embedding model and a place to store and search the vectors — which is exactly what vector databases (covered next) are for.

Classification: labeling by proximity

Once data has coordinates, classification becomes a question of where new points land relative to labeled examples. A classifier — a logistic regression, a small neural network, even a simple nearest-neighbor rule — can be trained on the embeddings to sort inputs into categories [4]. The advantage of doing classification on embeddings rather than raw text is that the embeddings already capture the underlying relationships; the classifier doesn't have to rediscover meaning from scratch, it just has to draw boundaries in a space where meaning is already geometry.

Recommendation: "users like you, items like this"

Recommendation systems use embeddings to capture similarity between items (and between users). By embedding items and user preferences into the same space, the system can measure how closely related two items are by vector proximity, and recommend items close to ones a user already liked [5]. "Because you watched X" is, under the hood, "items whose embeddings are near X's embedding." This scales far better than hand-coded rules about genres and categories, because the model learns the actual co-occurrence structure.

Anomaly detection: the points that don't fit

The flip side of similarity is anomaly detection. If similar items cluster together, then items far from any cluster — points that deviate significantly from the typical distribution — stand out as anomalies [6]. This works for fraud detection (a transaction whose embedding is far from the user's normal pattern), network security (traffic that doesn't match learned patterns), and quality control. The same embedding that powers "find similar" powers "find weird," which is a quietly elegant reuse.

The open-weight model families

The roadmap threads several open-weight model families through this section, and they're worth naming as concrete examples of what gets embedded. Meta Llama is Meta's open-source LLM family, with different versions and sizes, designed as a capable base for NLP tasks [7]. DeepSeek is a family of efficient, open-weight models for text generation, translation, and QA [8]. Qwen is Alibaba's open-source series, coming in different sizes for flexibility across compute budgets [9]. Gemma is Google's lightweight, high-performing open family, sized so developers can pick the right model for their resource constraints [10].

The reason these matter for embeddings specifically: most have companion embedding models (or can produce embeddings as a side effect), and they're all open-weight — meaning I can run the embedding step locally, on my own infrastructure, without per-call API cost. For a feature that embeds a large corpus once and queries it many times, that's a meaningful economic difference.

How I use this

The way of thinking pays off as a single habit: whenever I'm facing a "find similar" or "categorize" or "rank by relevance" problem, I ask whether embeddings would do the job better than rules or keywords. Usually they do, because the embedding already encodes the relationships I'd otherwise have to hand-code. I reach for semantic search when vocabulary mismatch is killing keyword search, for embedding-based classification when raw text is too high-dimensional for a simple model, and for anomaly detection when "normal" is hard to describe but easy to cluster. The discipline is to remember this is one mechanic — distance in a learned space — wearing several hats, and that the quality of everything downstream depends on the quality of the embedding model I chose up front.

References

[1] Cloudflare, "What are Embeddings in Machine Learning?," 2024. [Online]. Available: https://www.cloudflare.com/en-gb/learning/ai/what-are-embeddings/

[2] IBM, "What is Embedding?," 2024. [Online]. Available: https://www.ibm.com/topics/embedding

[3] Elastic, "What is Semantic Search?," 2024. [Online]. Available: https://www.elastic.co/what-is/semantic-search

[4] Palo Alto Networks, "What Is Data Classification?," 2024. [Online]. Available: https://www.paloaltonetworks.com/cyberpedia/data-classification

[5] IBM, "What is a Recommendation Engine?," 2024. [Online]. Available: https://www.ibm.com/think/topics/recommendation-engine

[6] Towards Data Science, "Boosting Your Anomaly Detection With LLMs," 2024. [Online]. Available: https://towardsdatascience.com/boosting-your-anomaly-detection-with-llms/

[7] Meta, "Llama," 2024. [Online]. Available: https://www.llama.com/

[8] DeepSeek, 2024. [Online]. Available: https://www.deepseek.com/en/

[9] Alibaba, "Qwen," 2024. [Online]. Available: https://chat.qwen.ai/

[10] Google DeepMind, "Gemma," 2024. [Online]. Available: https://deepmind.google/models/gemma/