Amazon’s New OpenSource Service: AWS Elasticsearch

Introduction

Developed in Java, Elasticsearch is a distributed, open-source search and analytics engine based on Apache Lucene. It began as a scalable version of the Lucene open-source search platform and later included support for scaling Lucene indices horizontally. With the help of Elasticsearch, you can store, search, and analyze massive amounts of data fast and in close to real-time, with results arriving in milliseconds. Because it searches an index rather than the text itself, it can produce quick search results. It has extensive REST APIs for storing and searching the data and employs a structure based on documents rather than tables and schemas. Elasticsearch can be conceptualized as a server that can handle JSON queries and return JSON data at its core.

Logical Concepts of Elasticsearch

Documents

Elasticsearch can index basic informational units defined in JSON, the universal internet data interchange format, called documents. A document can be compared to a row in a relational database, which represents the entity you’re looking for. A document in Elasticsearch can be anything that has structured data that has been encoded in JSON, not only text. Data can be in the form of numbers, strings, or dates. Each document has a special ID and a certain data type that identifies the type of entity it is. A document might be a log entry from a web server or an article from an encyclopedia, for instance.

Indices

A collection of documents with comparable traits is called an index. In Elasticsearch, an index is a highest-level entity that can be searched against. The index can be compared to a relational database schema in terms of structure. An index usually contains documents that are all logically related. You can have an index for Customers, one for Products, one for Orders, and so on in the context of an e-commerce website. An index is recognized by a name that is used to refer to the index when actions such as indexing, searching, updating, and deleting documents from it are being carried out.

Index Inversion

The method by which all search engines operate is known as an “inverted index,” which is what Elasticsearch uses as its index. A mapping from content, such as words or numbers, to their places in a document or series of documents, is stored in this type of data structure. In essence, it is a data structure that resembles a hashmap and guides you from a word to a document. Instead of storing strings directly, an inverted index breaks down each document into individual search phrases (i.e., each word) and then associates each search term with the documents in which it appears.

Cluster of Backend Components

A collection of one or more connected node instances makes up an Elasticsearch cluster. The distribution of jobs, such as searching and indexing, among all cluster nodes, is what gives an Elasticsearch cluster its power.

Node

A single server that is a component of a cluster is known as a node. A node participates in the cluster’s indexing and search processes while storing data. Elasticsearch nodes can be set up in a variety of ways:

• Master Node: Elasticsearch’s master node oversees all cluster-wide activities, such as adding and removing nodes and generating and deleting indexes.
• Data Node: Executes data-related activities like search and aggregation and stores data.
• Client Node: Sends data-related requests to data nodes and cluster requests to the master node.

Shards

Elasticsearch has the option to break the index into numerous sections known as shards. Each shard functions as a complete, independent “index” in and of itself and can be hosted on any cluster node. Elasticsearch can provide redundancy, which both guards against hardware failures and boosts query capacity when nodes are added to a cluster, by distributing the documents in an index across several shards and distributing those shards across numerous nodes.

Replicas

Replica shards, often known as “replicas,” are copies of your index’s shards that Elasticsearch lets you create in any number. A replica shard is essentially a duplicate of a primary shard. Each primary shard in an index contains one document per shard. Replicas offer redundant copies of your data to safeguard against hardware failure and expand capacity to handle read requests like document retrieval or search.

Conclusion

About CloudThat