Getting Started with Amazon Elasticsearch Service and Kibana

It is important to have a search engine for your website or applications. When it comes to automation of these great features such as load balancing and scalability of websites, Amazon provides it own managed service known as Amazon Elasticsearch.

In this tutorial you will learn about what is Amazon Elasticsearch and how to create a Amazon Elasticsearch domain using Amazon Management console and then search the data using Kibana.

Table of contents

  1. What Is Amazon Elasticsearch Service?
  2. Creating the Amazon Elasticsearch Service domain
  3. Upload data to Amazon Elasticsearch for indexing
  4. Search documents using Kibana in Amazon Elasticsearch
  5. Conclusion

What Is Amazon Elasticsearch Service?

Amazon Elasticsearch Service is a managed service which deploys and scale the Elasticsearch clusters in the cloud. Elasticsearch is an open source analytical and search engine which is used to perform real time application monitoring and log analytics.

Amazon Elasticsearch service provisions all resources for Elasticsearch clusters and launches it. It also replaces the failed Elasticsearch nodes in the cluster automatically.

Features of Amazon Elasticsearch Service

  • It can scale up to 3 PB of attached storage
  • It works with various instance types.
  • It easily integrates with other services such as IAM for security for ,VPC , AWS S3 for loading data , AWS Cloud Watch for monitoring and AWS SNS for alerts notifications.

Creating the Amazon Elasticsearch Service domain

In this tutorial you will see how to create Elasticsearch cluster using Amazon Management console. Lets start.

  • Open your favorite web browser and navigate to the AWS Management Console and log in.
  • While in the Console, click on the search bar at the top, search for ‘Elasticsearch’, and click on the Elasticsearch menu item.
  • Now, one thing to note here is the name of Amazon Elasticsearch domain is same as that of Elasticsearch cluster that means domains are clusters with the settings, instance types, instance counts, and storage resources that you specify.
  • Now, click on Create a new domain.
  • Select the deployment type as development and testing domain
  • Now Under Configure domain provide the Elasticsearch domain name as “firstdomain” . A domain is the collection of resources needed to run Elasticsearch. The domain name will be part of your domain endpoint.
  • Under Data nodes, choose the t3.small.elasticsearch and ignore rest of the settings and click on NEXT
  • Under Network configuration, choose Public access. For Fine-grained access control, choose Create master user. Provide a user name as user and password as Admin@123. Fine-grained access control keeps your data safe.
  • For Domain access policy, choose Allow open access to the domain. Access policies control whether a request is accepted or rejected when it reaches the Amazon Elasticsearch Service domain
  • Now click on NEXT till end and create the domain. It takes few minutes for Domain to get Launched.
  • Click on the firstdomain Elasticsearch domain

Upload data to Amazon Elasticsearch for indexing

  • You can load streaming data into your Amazon Elasticsearch Service (Amazon ES) domain from many different sources. Some sources, like Amazon Kinesis Data Firehose and Amazon Cloud Watch Logs, have built-in support for Amazon ES. Others, like Amazon S3, Amazon Kinesis Data Streams, and Amazon DynamoDB, use AWS Lambda functions as event handlers
  • In this tutorial we will directly use a sample data to upload the data.
  • Click on the Kibana link as shown in above snapshot using the username user and password Admin@123 and then click on Add data
  • As this is just the demonstration , lets use sample data and Add e-commerce Orders.

Search documents using Kibana in Amazon Elasticsearch

Kibana is a popular open source visualization tool which works with AWS Elasticsearch service. It provides interface to monitor and search the indexes. Lets use Kibana to search the sample data which you just uploaded in AWS ES.

  • Click on Discover option from the main menu to search the data.
  • Now you will notice that Kibana will search the data and populate for you. You can modify the timelines and many other fields accordingly.

Kibana did provide the data when we searched in the dashboard using sample data which you uploaded.

Conclusion

In this tutorial you learnt about what is Amazon Elasticsearch and how to create a Amazon Elasticsearch domain using Amazon Management console. Also we learnt how to upload the sample data in AWS ES although this can be done using various ways such as S3 , Dynamo DB etc.

So, Now you have a strong fundamental understanding of AWS Elasticsearch , which site are you going to implement it in ?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s