Building a k-NN Similarity Search Engine using Amazon Elasticsearch and SageMaker

Too Long; Didn't Read

Building a k-NN Similarity Search Engine using Amazon Elasticsearch and SageMaker. 1,104 reads for k-nearest neighbor search. Using Amazon Sagemaker and Elasticsearch, I’ll show you how to build a scalable similarity questions search API. We use light weight DistilBERT model to generate sentence embeddings in this example. The following diagram shows the architecture of above steps:Deploying and running Sagemaker notebook instance in VPC. Mount EFS to model directory, mounting EFS. Mount targets and Sagemaker are in same VPC. Transform questions text to vectors, and index all vectors to Elasticsearch.