If you’ve used ChatGPT, Perplexity, or any modern AI-powered search engine recently, you've experienced vector search even if you didn’t realize it. Unlike traditional keyword-based search, vector search understands meaning. You can type: “How do I reduce memory usage in Python apps?” and it will return content that doesn’t even contain those exact words, but still answers your question. This magic is powered by embeddings and approximate nearest neighbor (ANN) algorithms. In this tutorial, we’ll build a vector search engine from scratch using: Sentence Transformers - to generate embeddingsFAISS - Facebook AI Similarity Search, for ultra-fast nearest neighbor lookupsPython - because simplicity matters Sentence Transformers - to generate embeddings FAISS - Facebook AI Similarity Search, for ultra-fast nearest neighbor lookups Python - because simplicity matters By the end, you’ll understand: What vector embeddings really areHow semantic similarity worksHow FAISS indexes millions of vectors efficientlyHow to build your own semantic search engineHow this powers modern LLM apps, RAG systems, and chatbots What vector embeddings really are How semantic similarity works How FAISS indexes millions of vectors efficiently How to build your own semantic search engine How this powers modern LLM apps, RAG systems, and chatbots Why Vector Search Matters Traditional search uses lexical matching. Lets say if your document contains: Python memory profiling techniques And your query is: “How to reduce RAM usage?” A keyword engine may fail. Vector search works differently: Text is converted into vectors (lists of numbers).Similar meanings - similar vectors.Search becomes a geometric problem: “Which vectors are closest?” Text is converted into vectors (lists of numbers). Similar meanings - similar vectors. Search becomes a geometric problem: “Which vectors are closest?” This allows: Semantic searchQuestion answeringRecommendation systemsRetrieval-Augmented Generation (RAG) Semantic search Question answering Recommendation systems Retrieval-Augmented Generation (RAG) This is the backbone of modern AI systems. Step 1: Installing Dependencies Let’s install what we need: pip install sentence-transformers faiss-cpu numpy pip install sentence-transformers faiss-cpu numpy If you have a GPU, you can use: pip install faiss-gpu pip install faiss-gpu Step 2: Understanding Embeddings An embedding is a fixed-length vector that represents the meaning of a piece of text. For example: "I love programming" => [0.021, -0.334, 0.876, ...]
"I enjoy writing code" => [0.019, -0.331, 0.880, ...] "I love programming" => [0.021, -0.334, 0.876, ...]
"I enjoy writing code" => [0.019, -0.331, 0.880, ...] These vectors will be close in space. We’ll use Sentence Transformers, which provides pretrained models specifically optimized for semantic similarity. Step 3: Generating Embeddings Let’s embed some example documents. from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

documents = [
    "Python is a programming language",
    "I love writing code",
    "Dogs are great pets",
    "Cats are independent animals",
    "Machine learning is fascinating",
    "I enjoy building AI applications",
]

embeddings = model.encode(documents)
print(embeddings.shape) from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

documents = [
    "Python is a programming language",
    "I love writing code",
    "Dogs are great pets",
    "Cats are independent animals",
    "Machine learning is fascinating",
    "I enjoy building AI applications",
]

embeddings = model.encode(documents)
print(embeddings.shape) Output: (6, 384) (6, 384) Each sentence is now a 384-dimensional vector. Step 4: Similarity: The Heart of Vector Search The most common similarity measures: Cosine Similarity: Measures the angle between vectors.Dot Product: Measures alignment.Euclidean Distance (L2): Measures raw distance. Cosine Similarity: Measures the angle between vectors. Dot Product: Measures alignment. Euclidean Distance (L2): Measures raw distance. FAISS primarily works with L2 distance or inner product. Step 5: Introducing FAISS FAISS is a library for fast similarity search over large vector collections. Why FAISS? Handles millions or billions of vectorsGPU accelerationMany index types (flat, IVF, HNSW, PQ)Memory-efficientBattle-tested Handles millions or billions of vectors GPU acceleration Many index types (flat, IVF, HNSW, PQ) Memory-efficient Battle-tested Let’s build the simplest index first. Step 6: Building a Flat Index A Flat Index does brute-force search: compares your query to every vector. import faiss
import numpy as np

dimension = embeddings.shape[1]
index = faiss.IndexFlatL2(dimension)

index.add(np.array(embeddings))
print("Total vectors indexed:", index.ntotal) import faiss
import numpy as np

dimension = embeddings.shape[1]
index = faiss.IndexFlatL2(dimension)

index.add(np.array(embeddings))
print("Total vectors indexed:", index.ntotal) Step 7: Searching Now let’s perform a semantic search. def search(query, k=3):
    query_embedding = model.encode([query])
    distances, indices = index.search(np.array(query_embedding), k)
    return indices[0], distances[0] def search(query, k=3):
    query_embedding = model.encode([query])
    distances, indices = index.search(np.array(query_embedding), k)
    return indices[0], distances[0] Test it: results, scores = search("I like programming")
for idx, score in zip(results, scores):
    print(documents[idx], " | score:", score) results, scores = search("I like programming")
for idx, score in zip(results, scores):
    print(documents[idx], " | score:", score) You’ll see results that understand meaning, not just keywords. Step 8: Wrapping It Into a Mini Search Engine Let’s make it cleaner. class VectorSearchEngine:
    def __init__(self, documents):
        self.model = SentenceTransformer("all-MiniLM-L6-v2")
        self.documents = documents
        self.embeddings = self.model.encode(documents)
        
        dim = self.embeddings.shape[1]
        self.index = faiss.IndexFlatL2(dim)
        self.index.add(np.array(self.embeddings))

    def search(self, query, k=3):
        q_emb = self.model.encode([query])
        distances, indices = self.index.search(np.array(q_emb), k)
        return [(self.documents[i], distances[0][j]) for j, i in enumerate(indices[0])] class VectorSearchEngine:
    def __init__(self, documents):
        self.model = SentenceTransformer("all-MiniLM-L6-v2")
        self.documents = documents
        self.embeddings = self.model.encode(documents)
        
        dim = self.embeddings.shape[1]
        self.index = faiss.IndexFlatL2(dim)
        self.index.add(np.array(self.embeddings))

    def search(self, query, k=3):
        q_emb = self.model.encode([query])
        distances, indices = self.index.search(np.array(q_emb), k)
        return [(self.documents[i], distances[0][j]) for j, i in enumerate(indices[0])] Usage: engine = VectorSearchEngine(documents)
results = engine.search("AI projects")

for text, score in results:
    print(text, "| score:", score) engine = VectorSearchEngine(documents)
results = engine.search("AI projects")

for text, score in results:
    print(text, "| score:", score) Step 9: Scaling Beyond Brute Force Flat indexes don’t scale. If you have: 1M vectors - slow100M vectors - impossible 1M vectors - slow 100M vectors - impossible This is where Approximate Nearest Neighbor (ANN) comes in. FAISS provides several index types: Index TypeUse CaseIndexFlat ExactslowIVFClustering-basedHNSWGraph-basedPQMemory compressionOPQOptimized PQ Index TypeUse CaseIndexFlat ExactslowIVFClustering-basedHNSWGraph-basedPQMemory compressionOPQOptimized PQ Index TypeUse CaseIndexFlat ExactslowIVFClustering-basedHNSWGraph-basedPQMemory compressionOPQOptimized PQ Index TypeUse Case Index Type Use Case IndexFlat Exactslow IndexFlat Exact slow IVFClustering-based IVF Clustering-based HNSWGraph-based HNSW Graph-based PQMemory compression PQ Memory compression OPQOptimized PQ OPQ Optimized PQ Step 10: IVF Index Example IVF is Inverted File Index and the idea here is Cluster vectors into buckets.Search only relevant buckets. Cluster vectors into buckets. Search only relevant buckets. nlist = 50  # number of clusters
quantizer = faiss.IndexFlatL2(dimension)
index_ivf = faiss.IndexIVFFlat(quantizer, dimension, nlist, faiss.METRIC_L2)

index_ivf.train(np.array(embeddings))
index_ivf.add(np.array(embeddings)) nlist = 50  # number of clusters
quantizer = faiss.IndexFlatL2(dimension)
index_ivf = faiss.IndexIVFFlat(quantizer, dimension, nlist, faiss.METRIC_L2)

index_ivf.train(np.array(embeddings))
index_ivf.add(np.array(embeddings)) Searching: index_ivf.nprobe = 5  # how many clusters to search
distances, indices = index_ivf.search(np.array(query_embedding), k) index_ivf.nprobe = 5  # how many clusters to search
distances, indices = index_ivf.search(np.array(query_embedding), k) More nprobe = better accuracy, slower speed. Step 11: Real-World Example - Searching Technical Articles Let’s build a more realistic example. articles = [
    "Understanding Python memory management",
    "A guide to building REST APIs with FastAPI",
    "Introduction to machine learning pipelines",
    "How to optimize SQL queries",
    "Deep dive into transformers and attention",
    "Scaling microservices with Kubernetes",
]

engine = VectorSearchEngine(articles)
engine.search("How does attention work in neural networks?") articles = [
    "Understanding Python memory management",
    "A guide to building REST APIs with FastAPI",
    "Introduction to machine learning pipelines",
    "How to optimize SQL queries",
    "Deep dive into transformers and attention",
    "Scaling microservices with Kubernetes",
]

engine = VectorSearchEngine(articles)
engine.search("How does attention work in neural networks?") You’ll see it return the transformer-related article, even if the words don’t match. Step 12: Persisting the Index FAISS allows you to save and load indexes. faiss.write_index(engine.index, "articles.index") faiss.write_index(engine.index, "articles.index") Later: index = faiss.read_index("articles.index") index = faiss.read_index("articles.index") This is essential for production. Step 13: Metadata Mapping FAISS stores only vectors. You must maintain your own ID -> document mapping. Example: id_to_doc = {i: doc for i, doc in enumerate(documents)} id_to_doc = {i: doc for i, doc in enumerate(documents)} When FAISS returns [3, 1, 5], you look them up. Step 14: How This Powers RAG Systems Retrieval-Augmented Generation (RAG): User asks a question.Convert it to an embedding.Retrieve relevant documents via vector search.Send them to the LLM as context.Generate grounded responses. User asks a question. Convert it to an embedding. Retrieve relevant documents via vector search. Send them to the LLM as context. Generate grounded responses. This avoids hallucinations. Step 15: Common Mistakes ❌  Using the wrong embedding model Use similarity-optimized models like:all-MiniLM-L6-v2multi-qa-MiniLM-L6-cos-v1 Use similarity-optimized models like: all-MiniLM-L6-v2 multi-qa-MiniLM-L6-cos-v1 ❌  Mixing distance metrics Cosine vs L2 matters. Cosine vs L2 matters. ❌ Forgetting normalization Some indexes require normalized vectors. Some indexes require normalized vectors. Step 16: Production Considerations 1. Sharding: Split indexes across machines. 2. Caching: Cache frequent queries. 3. Incremental Updates: Use index.add() for streaming ingestion. 4. Reindexing: ANN structures degrade over time. Step 17: Performance Benchmarking FAISS can do: • 1M vectors -> sub-10ms search • GPU -> microseconds This is why it’s used by a lot of tech companies Step 18: Why Not Just Use Pinecone or Weaviate? Managed vector DBs are great. But building from scratch teaches you: How similarity really worksTradeoffsIndex internalsLatency tuningMemory behavior How similarity really works Tradeoffs Index internals Latency tuning Memory behavior Final Thoughts Vector search is not a feature, it’s an infrastructure primitive. It powers: ChatbotsSemantic searchRecommendationsRAGAI copilotsKnowledge engines Chatbots Semantic search Recommendations RAG AI copilots Knowledge engines And in this tutorial, you built one from scratch. You now understand: EmbeddingsANNFAISSIndexing strategiesReal-world tradeoffs Embeddings ANN FAISS Indexing strategies Real-world tradeoffs And most importantly, you can now reason about these systems, not just use them.

This story contains AI-generated text. The author has used AI either for research, to generate outlines, or write the text itself. 

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

Build a Vector Search Engine in Python with FAISS and Sentence Transformers

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

How to Build an LLM-Powered CLI Tool in Python

NEW ChatGPT Search vs Perplexity AI: Which One Should You Use?

How to Search Haster With These 3 AI Search Engines

Search Won’t Be One Thing Anymore

I Churned Out 15 AI Features in 6 Months and It Nearly Broke Us. Here's Our Story

Why I Left Big Tech Design to Build AI Infra for Founders

How to Build an LLM-Powered CLI Tool in Python

NEW ChatGPT Search vs Perplexity AI: Which One Should You Use?

How to Search Haster With These 3 AI Search Engines

Search Won’t Be One Thing Anymore

I Churned Out 15 AI Features in 6 Months and It Nearly Broke Us. Here's Our Story

Why I Left Big Tech Design to Build AI Infra for Founders

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps