Login
August 14, 2023Voiced by Amazon Polly |
Introduction
The powerful and fully managed Amazon OpenSearch service enables developers to integrate search and analytics features into their applications seamlessly. This service is derived from the open-source Elasticsearch search and analytics engine and ensures a secure and scalable user search environment.
Features of Amazon OpenSearch
- Scalability – Amazon OpenSearch offers the flexibility to expand or shrink its resources based on data volume and query load. It can handle vast amounts of data and numerous queries per second.
- Open-source foundation – Built upon the open source Elasticsearch and Kibana technologies, Amazon OpenSearch benefits from a thriving community of contributors and users. This collaborative environment ensures continuous enhancements and feature additions.
- Simplified Setup and Management – With complete management provided by AWS, the complexities of infrastructure and administrative tasks are abstracted. This empowers users to prioritize their data and applications without the burden of server upkeep or updates.
- Real-time Data Analysis – Amazon OpenSearch facilitates the real-time interpretation of data, enabling the prompt identification of emerging trends and irregularities as they unfold.
- Data Visualization – Including Kibana within Amazon OpenSearch furnishes users with a potent instrument for crafting personalized dashboards and generating insightful reports.
- Robust Security – Comprehensive security features are an integral part of Amazon These include data encryption during transit and storage, meticulous access controls, and seamless integration with AWS Identity and Access Management (IAM).
Pioneers in Cloud Consulting & Migration Services
- Reduced infrastructural costs
- Accelerated application deployment
Steps to Stream logs to Amazon OpenSearch
Step 1 – Create an Amazon OpenSearch cluster with the configuration required
Step 2 – Go to Amazon CloudWatch in the AWS console and select log groups
Create a log group if required, or if having an existing log group move to step 4.
Create a log stream and in the log stream, select log events and create a log event.
Step 3 – Go to AWS IAM roles and create a role for AWS Lambda execution
For the created role, attach Cloudwatchlogsreadonlyaccess, AWS Lambda execute and create inline policy for opensearchservice write.
Step 4 – Inside the log groups, create a subscription filter for Amazon OpenSearch
Select the Amazon OpenSearch cluster you want to stream and select the AWS Lambda execution role created in Step 3
Create the filter after entering the name of the filter.
Step 5 – Copy the OpenSearch domain URL from the AWS console and enter the master username and password given when creating the OpenSearch cluster
Go to security, select all_access from the given roles, enter the role arn created in step 3 under backend roles, and map the role arn.
Step 6 – Go to AWS Lambda in the console and select the function created by the subscription filter.
Create a test event to check if the logs are getting streamed to the OpenSearch cluster.
Once tested and AWS Lambda logs shows success. Go to OpenSearch Dashboard
Step 7 – In OpenSearch, go to index management and check for the logs getting streamed under the specific date.
If Amazon CloudWatch already has applications or services logs, it will directly get streamed to OpenSearch.
Note: Try creating a log event in log streams for the data ingested in OpenSearch.
Example: # 2023-07-25T12:34:56.789Z This is a sample log message.
Step 8 – In the OpenSearch dashboard, go to stack management and create the index pattern you want to see as a graph in the Discover dashboard. Once created, go to Discover and see the required index created.
Conclusion
Amazon Web Services (AWS) offers a robust and scalable search and analytics solution called Amazon OpenSearch. OpenSearch, a development of Amazon’s well-liked Elasticsearch service, offers users looking to create and maintain search applications in the cloud advanced capabilities and enhanced security. Amazon OpenSearch provides developers and business leaders with the necessary resources to swiftly and effortlessly develop powerful search and analytics applications, leveraging the robust infrastructure and extensive ecosystem offered by AWS.
Drop a query if you have any questions regarding Amazon OpenSearch and we will get back to you quickly.
Making IT Networks Enterprise-ready – Cloud Management Services
- Accelerated cloud migration
- End-to-end view of the cloud environment
About CloudThat
CloudThat is an official AWS (Amazon Web Services) Advanced Consulting Partner and Training partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, Amazon QuickSight Service Delivery Partner, AWS EKS Service Delivery Partner, and Microsoft Gold Partner, helping people develop knowledge of the cloud and help their businesses aim for higher goals using best-in-industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.
To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.
FAQs
1. What is Amazon OpenSearch, and how does it differ from Amazon Elasticsearch?
ANS: – Elasticsearch and Kibana-based applications can be created, deployed, and scaled using AWS OpenSearch, a managed search and analytics service. It is the replacement for Amazon Elasticsearch Service. The primary distinction between AWS OpenSearch and Amazon ES is that OpenSearch is a completely compatible open-source distribution of Elasticsearch and Kibana that is based on an earlier version of open-source Elasticsearch, whereas Amazon ES was built on a more recent version of open-source Elasticsearch.
2. How is data indexing and querying handled in Amazon OpenSearch?
ANS: – Data indexing in Amazon OpenSearch entails feeding data into the search cluster to make data available for search and analytics. The procedure entails building an index and specifying its structure, associating information fields with the appropriate data types.
3. How is data secured in Amazon OpenSearch?
ANS: –
- Encryption – Data in transit can be encrypted using HTTPS, and data at rest can be encrypted using AWS Key Management Service (KMS) keys.
- Access Control – AWS OpenSearch integrates with AWS IAM, allowing you to control the service’s access and resources based on IAM policies.
WRITTEN BY Swapnil Kumbar
Swapnil Kumbar is a Research Associate - DevOps. He knows various cloud platforms and has working experience on AWS, GCP, and azure. Enthusiast about leading technology in cloud and automation. He is also passionate about tailoring existing architecture.
PREV
Comments