If you're reading this blog chances are you really interested in Elasticsearch and the solutions that it provides. This blog will introduce you to Elasticsearch and explain how to get started with implementing a fast search for your app in less than 10 minutes. Of course, we're not going to code up a full-blown production-ready search solution here. But, the below-mentioned concepts will help you get up to speed quickly. So, without further ado, let's start! What is Elasticsearch? Elasticsearch is a distributed search and analytics engine. It provides near real-time search and analytics for all types of data. Whether you have structured or unstructured text, numerical data, or geospatial data. One of the key specialties of Elasticsearch is that it can efficiently store and index it in a way that supports fast searches. You can go far beyond simple data retrieval and aggregate information to discover trends and patterns in your data. Why do you need it? Because Elasticsearch is built on top of , it excels at full-text search. Elasticsearch is also a near real-time search platform, meaning the latency from the time a document is indexed until it becomes searchable is very short — typically one second. As a result, Elasticsearch is well suited for time-sensitive use cases such as security analytics and infrastructure monitoring. Elasticsearch is fast. Lucene The documents stored in Elasticsearch are distributed across different containers known as , which are duplicated to provide redundant copies of the data in case of hardware failure. The distributed nature of Elasticsearch allows it to scale out to hundreds (or even thousands) of servers and handle petabytes of data. Elasticsearch is distributed by nature. shards The speed and scalability of Elasticsearch and its ability to index many types of content mean that it can be used for a number of use cases: Application search Website search Enterprise search Logging and log analytics And many more... We at are building a feature for the upcoming v5 release where we'll use Elasticsearch to perform a super-fast search in our core apps like Page builder, File manager, and Headless CMS. Please check out our to learn more about it. Webiny Github repo Getting started with Elasticsearch Setup Elasticsearch cluster You can create a hosted deployment or set up an Elasticsearch cluster on your local machine. For the purpose of this blog, we'll assume that we have an Elasticsearch cluster running at localhost:9200. If you want to go with a local setup please check out this . guide Setup Elasticsearch Node.js client We'll be going to use the official Node.js client for Elasticsearch. You can create a new Node.js project or use this . example project To install the latest version of the client, run the following command: install @elastic/elasticsearch npm Using the client is straightforward, it supports all the public APIs of Elasticsearch, and every method exposes the same signature. Configure the client The client is designed to be easily configured for your needs. In the example mentioned below, you can see how easy it is to configure it with basic options. { Client } = ( ); client = Client({ node: , maxRetries: , requestTimeout: ,
}); const require "@elastic/elasticsearch" const new // The Elasticsearch endpoint to use. "http://localhost:9200" // Max number of retries for each request. 5 // Max request timeout in milliseconds for each request. 60000 As we previously mentioned we're assuming that we have an Elasticsearch cluster running locally on Note: localhost:9200 Elasticsearch in action Before jumping into the core topic of this blog i.e , we'll need to create the and add few documents to it. Create an index search index Create an index Let's create an index inside our Elasticsearch cluster. You can use the create index API to add a new index to an Elasticsearch cluster. When creating an index, you can specify the following: Settings for the index (optional) Mappings for fields in the index (optional) Index aliases (optional) client.indices.create({ index: ,
}); await // Name of the index you wish to create. "products" We'll be using a dynamic mapping that's why didn't add the and in the body here. But, if needed we could have something like this: settings mappings client.indices.create({ index: , body: { : { : ,
    }, : { : { : { : },
      },
    },
  },
}); await // Name of the index you wish to create. "products" // If you want to add "settings" & "mappings" settings number_of_shards 1 mappings properties field1 type "text" Index documents Now that we have created the index, let's add a few documents so that we can perform a search on those later. There are basically two ways you can do this depending upon the use case. product Index a single document.Index multiple documents in bulk. We'll cover both of these use cases in a moment. Index a single document Here we're going to use the create method on the client that we created earlier. Let's take a look at the code: client.create({ id: , : , index: , : { : , : , : , : ,
  },
}); await // Unique identifier for the document. // To automatically generate a document ID omit this parameter. 1 type "doc" // The name of the index. "products" body id 1 name "iPhone 12" price 699 description "Blast past fast" We can index a new document with the or resource. Using _create guarantees that the document is only indexed if it does not already exist. To update an existing document, you must use the resource. JSON _doc _create _doc Index multiple documents at once This is all good. But, sometimes we want to index multiple documents at once. For example, in our case wouldn't it be better if we can index all brand new iPhones at once? Right? We can use the bulk method for this exact use case. Let's take a look at the code: dataset = [
  { : , : , : , : ,
  },
  { : , : , : , : ,
  },
  { : , : , : , : ,
  },
]; body = dataset.flatMap( [{ : { : } }, doc]); { : bulkResponse } = client.bulk({ : , body }); (bulkResponse.errors) { erroredDocuments = []; bulkResponse.items.forEach( { operation = .keys(action)[ ]; (action[operation].error) {
      erroredDocuments.push({ status: action[operation].status, : action[operation].error, : body[i * ], : body[i * + ],
      });
    }
  }); .log(erroredDocuments);
} const id 2 name "iPhone 12 mini" description "Blast past fast." price 599 id 3 name "iPhone 12 Pro" description "It's a leap year." price 999 id 4 name "iPhone 12 Pro max" description "It's a leap year." price 1199 const => doc index _index "products" const body await refresh true if const // The items array has the same order of the dataset we just indexed. // The presence of the `error` key indicates that the operation // that we did for the document has failed. ( ) => action, i const Object 0 if // If the status is 429 it means that you can retry the document, // otherwise it's very likely a mapping error, and you should // fix the document before to try it again. error operation 2 document 2 1 // Do something useful with it. console The method provides a way to perform multiple , , and actions in a single request. Here we are using the index action but you can use the other actions as per your needs. bulk index create delete update TIP performs multiple indexing or delete operations in a single API call. This reduces overhead and can greatly increase indexing speed. bulk Update an existing document We often need to update an existing document. We'll use the update method for the same. It enables you to script document updates. The script can update, delete, or skip modifying the document. To increment the price, you can call the update method with the following script: client.update({ index: , id: , : { : { : , : { : ,
      },
    },
  },
}); await // The name of the index. "products" // Document ID. -1 body script source "ctx._source.price += params.price_diff" params price_diff 99 The API also supports passing a partial document, which is merged into the existing document. Let's use it to update the of the product with : update description id = -1 client.update({ index: , id: , : { : { : ,
    },
  },
}); await // The name of the index. "products" // Document ID. -1 body doc description "Fast enough!" Delete an existing document It's a no-brainer that we also need to remove existing documents at some point. We'll use the method to remove a document from an index. For that, we must specify the index name and document ID. Let's take a look at an example: delete client.delete({ index: , id: ,
}); await // The name of the index. "products" // Document ID. -1 The Search The API allows us to execute a search query and get back search hits that match the query. search Let's start with a simple query. { body } = client.search({ index: , : { query: { : { : ,
      },
    },
  },
}); // Let's search! const await // The name of the index. "products" body // Defines the search definition using the Query DSL. match description "blast" This query will return all the documents whose field matches with description "blast" INFO: Learn more about Query DSL . here Nice and simple right. But, that's not all! We can go for even more specific queries. Let's look at some examples: Search for exact text like the name of a product { body } = client.search({ index: , : { query: { : {
        title.keyword: { : }
      }
    }
  }
}); // Let's search for products with name "iPhone 12 Pro" ! const await // The name of the index. "products" body // Defines the search definition using the Query DSL. term value "iPhone 12 Pro" Search for a range of values like products between a price range { body } = client.search({ index: , : { query: { : { : { : , : ,
        },
      },
    },
  },
}); // Let's search for products ranging between 500 and 1000! const await // The name of the index. "products" body // Defines the search definition using the Query DSL. range price gte 500 lte 1000 Search using multiple conditions { body } = client.search({ index: , : { query: { bool: { should: [
          { : { : { : , : ,
              },
            },
          },
          { : { : ,
            },
          },
        ],
      },
    },
  },
}); // Let's search for products that are either ranging between 500 and 1000 // or description matching "stunning" const await // The name of the index. "products" body // Defines the search definition using the Query DSL. // Return result for which this nested condition is TRUE. // Acts like an OR operator. // Returns TRUE even if one of these conditions is met range price gte 500 lte 1000 match description "stunning" If you need a search query where all the conditions must be matched then you should use the operator inside the It acts like an AND operator and returns TRUE only if all the conditions met. Inside , there are also other operators and that you can use as per your needs. must bool bool must_not should_not These are just a few examples of search queries, you can perform even more specific and power search queries. INFO: Learn more about search using Query DSL . here Sort search results Elasticsearch allows us to add one or more sorts of specific fields. Each sort can be reversed as well. The sort is defined on a per-field level, with a special field name for to sort by score, and to sort by index order. _score _doc The order defaults to "desc" when sorting on the and defaults to when sorting on anything else. _score "asc" Let's take a look at the following example: { body } = client.search({ index: , : { query: { : { must: [
          { : { : { : , : ,
              },
            },
          },
          { : { : ,
            },
          },
        ],
      },
    }, sort: [
      { : { : ,
        },
      },
    ],
  },
}); // Let's sort the search results! const await // The name of the index. "products" body // Defines the search definition using the Query DSL. bool // Acts like an AND operator. // Returns TRUE only if all of these conditions are met. range price gte 500 lte 1100 match name "iPhone" // Sort the search result by "price" price order "asc" Here we've sorted the search result by in order. price "asc" Paginate search results Pagination is a must-have feature for every decent real-world app. And Elasticsearch helps us with this as well. Let's see how? 🙂 By default, the search method returns the top 10 matching documents. To paginate through a larger set of results, you can use the search API’s size and from parameters. The size parameter is the number of matching documents to return. The from parameter is a zero-indexed offset from the beginning of the complete result set that indicates the document you want to start with. For example, the following search method call sets the from offset to 15, meaning the request offsets, or skips, the first fifteen matching documents. The parameter is 15, meaning the request can return up to 15 documents, starting at the offset. size { body } = client.search({ index: , : { : , size: , query: { : { : ,
      },
    },
  },
}); // Let's paginate the search results! const await // The name of the index. "products" body // Starting offset (default: 0) from 15 // Number of hits to return (default: 10) 15 // Defines the search definition using the Query DSL. match description "blast" Conclusion If you're looking to implement a fast search mechanism for your app or website. I would recommend you to consider Elasticsearch as a solution to that. And if you're interested in building full-stack serverless web applications I would highly recommend you try out The Easiest Way To Adopt Serverless. We've along with baked-in for super-fast search in our core apps like Page builder, File manager, and Headless CMS. Webiny Elasticsearch DynamoDB I hope this blog will help you in your web development journey, but, of course, if you have any further questions, concerns, or ideas, feel free to ping me 💬 over or even directly via our . Twitter community Slack Thanks for reading this blog! My name is Ashutosh, and I work as a full-stack developer at . If you have any questions, comments, or just wanna say hi, feel free to reach out to me via . You can also subscribe 🍿 to our where we post knowledge sharing every week. Webiny Twitter YouTube channel Previously published at https://www.webiny.com/blog/lighting-fast-search-with-elasticsearch?utm_source=Hackernoon&utm_medium=webiny-blog&utm_campaign=webiny-cross-promotion-nov-16&utm_content=webiny-blog-lighting-fast-search-with-elasticsearch&utm_term=W00384

Slack

Super

Twitter

YouTube

2022 - HackerNoon Contributor of the Year - Aws

2022 - HackerNoon Contributor of the Year - Cms

2022 - HackerNoon Contributor of the Year - Software Testing

Learn more  > Webiny.com

Nominated for 2022 - HackerNoon Contributor of the Year - Cms

Nominated for 2022 - HackerNoon Contributor of the Year - Aws

Nominated for 2022 - HackerNoon Contributor of the Year - Software Testing

Too Long; Didn't Read

Cassandra Forward a free Cassandra-focused community event

An Introduction to Elasticsearch: Lightning Fast Search Solutions

An Introduction to Elasticsearch: Lightning Fast Search Solutions

Too Long; Didn't Read

Companies Mentioned

@webiny

RELATED STORIES