--- title: "10 — What Are Embeddings? Meaning as Position in Vector Space" uid: what-are-embeddings tags: ["similarity", "semantic-search", "vector-space", "roadmap:ai-engineer", "embeddings", "fundamentals"] excerpt: "An embedding is a learned coordinate for a piece of data — nearby points mean similar things. Once that clicked, search, classification, and recommendation all became one operation." date: 2026-08-13T03:28:37+0000 source: https://www.aveshina.my.id/en/blog/what-are-embeddings --- "Just vectors for search" was how I waved away embeddings, and it hid the real idea: **an embedding is a learned coordinate for a piece of data, where distance in the space encodes semantic similarity — nearby points mean similar things.** [1] Once I saw it as coordinates on a map of meaning rather than an opaque array of numbers, every downstream use (search, classification, recommendation, anomaly detection) became the same operation: _measure distance_. The framing that finally landed is a map. An embedding model takes text (or an image, or audio) and outputs a fixed-length vector — a list of a few hundred or thousand numbers. That vector is the _coordinate_ of the input in a high-dimensional space the model learned during training. The model's training organized the space so that inputs a human would call "similar" land close together, and inputs we'd call "different" land far apart. The geometry _is_ the meaning. ```figure ``` ## The core mechanic: similarity as distance Strip the geometry away and the operation is always the same. To compare two things, I compute the distance between their embedding vectors — usually **cosine similarity** (the angle between them) or **Euclidean distance** (the straight-line distance). Small distance, or high cosine similarity, means the model considers them semantically close. This works even when the inputs share no words at all: "kitty" and "feline" can land near "cat" without any token overlap, because the model learned the relationships from how words are used in context during training [1][2]. That single mechanic — _measure distance_ — is what makes embeddings so broadly useful. Once data has coordinates, every "find similar things" question becomes a nearest-neighbor lookup. ## Semantic search: matching meaning, not keywords The most direct application is **semantic search**. Traditional keyword search matches on exact terms — if the document doesn't contain the word I searched for, it won't be returned, even if it's about exactly the same thing. Semantic search embeds both the query and every document, then returns the documents whose vectors are nearest the query vector [3]. A search for "feline friend" can find a document about a "cat" because their embeddings are close, even with zero word overlap. The practical win is that semantic search handles synonyms, paraphrase, and vocabulary mismatch that keyword search cannot. The cost is that I need an embedding model and a place to store and search the vectors — which is exactly what vector databases (covered next) are for. ## Classification: labeling by proximity Once data has coordinates, **classification** becomes a question of where new points land relative to labeled examples. A classifier — a logistic regression, a small neural network, even a simple nearest-neighbor rule — can be trained on the embeddings to sort inputs into categories [4]. The advantage of doing classification on embeddings rather than raw text is that the embeddings already capture the underlying relationships; the classifier doesn't have to rediscover meaning from scratch, it just has to draw boundaries in a space where meaning is already geometry. ## Recommendation: "users like you, items like this" **Recommendation systems** use embeddings to capture similarity between items (and between users). By embedding items and user preferences into the same space, the system can measure how closely related two items are by vector proximity, and recommend items close to ones a user already liked [5]. "Because you watched X" is, under the hood, "items whose embeddings are near X's embedding." This scales far better than hand-coded rules about genres and categories, because the model learns the actual co-occurrence structure. ## Anomaly detection: the points that don't fit The flip side of similarity is **anomaly detection**. If similar items cluster together, then items far from any cluster — points that deviate significantly from the typical distribution — stand out as anomalies [6]. This works for fraud detection (a transaction whose embedding is far from the user's normal pattern), network security (traffic that doesn't match learned patterns), and quality control. The same embedding that powers "find similar" powers "find weird," which is a quietly elegant reuse. ## The open-weight model families The roadmap threads several open-weight model families through this section, and they're worth naming as concrete examples of what gets embedded. **Meta Llama** is Meta's open-source LLM family, with different versions and sizes, designed as a capable base for NLP tasks [7]. **DeepSeek** is a family of efficient, open-weight models for text generation, translation, and QA [8]. **Qwen** is Alibaba's open-source series, coming in different sizes for flexibility across compute budgets [9]. **Gemma** is Google's lightweight, high-performing open family, sized so developers can pick the right model for their resource constraints [10]. The reason these matter for embeddings specifically: most have companion embedding models (or can produce embeddings as a side effect), and they're all open-weight — meaning I can run the embedding step locally, on my own infrastructure, without per-call API cost. For a feature that embeds a large corpus once and queries it many times, that's a meaningful economic difference. ## How I use this The way of thinking pays off as a single habit: whenever I'm facing a "find similar" or "categorize" or "rank by relevance" problem, I ask whether embeddings would do the job better than rules or keywords. Usually they do, because the embedding already encodes the relationships I'd otherwise have to hand-code. I reach for semantic search when vocabulary mismatch is killing keyword search, for embedding-based classification when raw text is too high-dimensional for a simple model, and for anomaly detection when "normal" is hard to describe but easy to cluster. The discipline is to remember this is one mechanic — distance in a learned space — wearing several hats, and that the quality of everything downstream depends on the quality of the embedding model I chose up front. ## References [1] Cloudflare, "What are Embeddings in Machine Learning?," 2024. [Online]. Available: [https://www.cloudflare.com/en-gb/learning/ai/what-are-embeddings/](https://www.cloudflare.com/en-gb/learning/ai/what-are-embeddings/) [2] IBM, "What is Embedding?," 2024. [Online]. Available: [https://www.ibm.com/topics/embedding](https://www.ibm.com/topics/embedding) [3] Elastic, "What is Semantic Search?," 2024. [Online]. Available: [https://www.elastic.co/what-is/semantic-search](https://www.elastic.co/what-is/semantic-search) [4] Palo Alto Networks, "What Is Data Classification?," 2024. [Online]. Available: [https://www.paloaltonetworks.com/cyberpedia/data-classification](https://www.paloaltonetworks.com/cyberpedia/data-classification) [5] IBM, "What is a Recommendation Engine?," 2024. [Online]. Available: [https://www.ibm.com/think/topics/recommendation-engine](https://www.ibm.com/think/topics/recommendation-engine) [6] Towards Data Science, "Boosting Your Anomaly Detection With LLMs," 2024. [Online]. Available: [https://towardsdatascience.com/boosting-your-anomaly-detection-with-llms/](https://towardsdatascience.com/boosting-your-anomaly-detection-with-llms/) [7] Meta, "Llama," 2024. [Online]. Available: [https://www.llama.com/](https://www.llama.com/) [8] DeepSeek, 2024. [Online]. Available: [https://www.deepseek.com/en/](https://www.deepseek.com/en/) [9] Alibaba, "Qwen," 2024. [Online]. Available: [https://chat.qwen.ai/](https://chat.qwen.ai/) [10] Google DeepMind, "Gemma," 2024. [Online]. Available: [https://deepmind.google/models/gemma/](https://deepmind.google/models/gemma/) ```quiz Q: An embedding is best understood as… - an opaque array of random numbers - a learned coordinate for a piece of data, where distance encodes semantic similarity correct: 1 explain: The embedding model organizes a high-dimensional space so that similar inputs land close together. The geometry is the meaning. Q: Semantic search can find a document about "cat" for the query "feline friend" because… - it normalizes plural forms - the embeddings of similar-meaning text land near each other, even with no word overlap correct: 1 explain: Embeddings capture relationships learned from context. Distance, not token matching, drives retrieval. Q: Which operation underlies search, classification, recommendation, AND anomaly detection with embeddings? - sorting by token count - measuring distance (or cosine similarity) between vectors correct: 1 explain: Every one of these tasks reduces to nearest-neighbor-style lookup or deviation from a cluster in the embedding space. Q: Why does using open-weight model families (Llama, Qwen, Gemma, DeepSeek) matter for an embedding-heavy feature? - they produce better embeddings than any closed model - you can run the embedding step locally, avoiding per-call API cost when embedding a large corpus correct: 1 explain: For a corpus embedded once and queried many times, local open-weight embedding is a meaningful economic advantage. Quality varies per model, not per openness. Q: Anomaly detection with embeddings works because… - anomalies always have more tokens - similar items cluster, so items far from any cluster deviate from the typical distribution correct: 1 explain: The same similarity geometry that powers "find similar" powers "find weird" — outliers are points far from learned clusters. ```