A Guide to Integrate Amazon CloudSearch with Amazon DynamoDB

Overview

Amazon CloudSearch is a fully managed cloud service that is easy to set up. It can search massive collections of information such as web pages, documents, and forum posts without worrying about setting up hardware or maintenance. It also scales automatically if your application experiences a change in traffic.

It expands automatically to tackle the volume of data search queries to provide quick and accurate results. It provides automatic indexing and scalability. Every searchable item on Amazon CloudSearch has its ID, which is unique compared to the other items in the same domain, ID is important when the Document is modified or deleted later.

Steps to set up a search solution with Amazon CloudSearch

Create and configure a search domain. A search domain contains instances that handle your search requests and searchable data.
Upload the data you want to search to your domain. Amazon CloudSearch deploys the search index to one or more search instances and automatically indexes your data.
Search your domain. You send a search query to the search endpoint of your domain as a HTTP/HTTPS GET request.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Features of Amazon CloudSearch

Auto Complete
Scalable
Reliable
Fully Managed
Query parser
High Performance

Steps to Setup Amazon CloudSearch Domain

Select create a new search Domain.

Give your Amazon CloudSearch domain a suitable name, select your desired Instance type, and click continue.

step1

Select the Amazon DynamoDB table you want to use in the Amazon CloudSearch Domain and click continue.

step2

Amazon CloudSearch does not support list attributes, so instead of a list, use String set in Amazon DynamoDB tables Item.

step3

Amazon CloudSearch does the indexing Automatically. Review the indexing and click continue.

step4

Review all details and select Confirm.

step5

After confirming, it will say, “You have successfully created the domain <your domain name here>.” After that, it will take approximately 10-15 minutes to become active, and only then can any actions be performed on the domain.

Wait for it to become Active.

Initially, it will not contain any searchable documents. Once it is active, we must upload the documents to it to appear in search results. Click on Upload Documents.

step6

Select the Amazon DynamoDB table and click continue.

step7

Click on Finish.

Now, it will show the number of Searchable Documents in the Amazon DynamoDB table.

step8

step8b

Working with Endpoints

Search Endpoint

It is used to submit a search request to the Amazon CloudSearch. It can be used on a browser, postman or we can make a GET API request from the code as well. The format of using the endpoint is:

https://<search-endpoint>/2013-01-01/search?q=<search-item-name>

You can also view all the items available for search in Amazon CloudSearch domain by adding “matchall&q.parser=structured” in the query .

https://<search-endpoint>/2013-01-01/search?q=matchall&q.parser=structured

If the search text is incomplete, a “*” can be added at the last to show all the results associated with that word.

Document Endpoint

Document Endpoint is used to add, replace, or Delete Documents from the Amazon CloudSearch Domain.

AWS Lambda can dynamically add and delete the Documents from Amazon CloudSearch. As soon as any item is added or deleted from the Amazon DynamoDB table, AWS Lambda will be triggered and perform the desired function.

Data must be uploaded to the domain in JSON format. A POST request will be made, requiring a document endpoint URL, headers, and data as parameters.

There is no option to delete the documents on the Amazon CloudSearch console, so if the data is removed from Amazon DynamoDB, it will still appear in search results. Deleting the documents from the Amazon CloudSearch domain with the help of Document Endpoint is important.

Format of Document Endpoint is:

https://<document-endpoint>/2013-01-01/documents/batch

To add Documents from Document Endpoint:

url = "<your-document-endpoint>/2013-01-01/documents/batch"
input = [
                    {“type”: "add”,
                  "id": “789456”,
                  "fields”: {
                    "unique_id":"789456",
                    "description":"this is demo code for adding documents to the cloudsearch domain",
                    "list_example":["aws","cloudsearch"]
                  } } ]
input = json.dumps(input)
headers = {'Content-Type': 'application/json'}
response = requests.request( "POST", url, headers=headers, data=input)

url = "<your-document-endpoint>/2013-01-01/documents/batch"

input = [

{“type”: "add”,

"id": “789456”,

"fields”: {

"unique_id":"789456",

"description":"this is demo code for adding documents to the cloudsearch domain",

"list_example":["aws","cloudsearch"]

} } ]

input = json.dumps(input)

headers = {'Content-Type': 'application/json'}

response = requests.request( "POST", url, headers=headers, data=input)

Deleting a Document from Document Endpoint

url = "<you-document-endpoint>/2013-01-01/documents/batch"
input = [
                    {"type":"delete",
                  "id": “789456”
                  } ]
input = json.dumps( input )
headers = {'Content-Type':'application/json'}
response = requests.request( "POST" , url , headers = headers , data = input )

url = "<you-document-endpoint>/2013-01-01/documents/batch"

input = [

{"type":"delete",

"id": “789456”

} ]

input = json.dumps( input )

headers = {'Content-Type':'application/json'}

response = requests.request( "POST" , url , headers = headers , data = input )

Conclusion

Amazon CloudSearch provides low-cost search operations to the website without trying to monitor, index, and provision. It supports HTTPS and integrates with Identity and Access Management (IAM) to manage the Search Domain.

Drop a query if you have any questions regarding Amazon CloudSearch and we will get back to you quickly.

Making IT Networks Enterprise-ready – Cloud Management Services

Accelerated cloud migration
End-to-end view of the cloud environment

Get Started

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, and many more.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.

FAQs

1. How does Amazon CloudSearch work with Amazon DynamoDB?

ANS: – Amazon CloudSearch can be integrated with Amazon DynamoDB to enable powerful search capabilities on the data stored in DynamoDB tables. This integration allows for seamless indexing and querying of Amazon DynamoDB data using Amazon CloudSearch.

2. What are the key features of Amazon CloudSearch?

ANS: – Amazon CloudSearch is supported in various regions, including Europe (Ireland), Asia Pacific (Singapore, Sydney, Seoul, Tokyo), US East (N. Virginia), US West (N. California, Oregon), South America (São Paulo), and Europe (Frankfurt).

3. Can I use AWS Lambda with Amazon CloudSearch and Amazon DynamoDB integration?

ANS: – Yes, AWS Lambda can be utilized to dynamically add, update, or delete documents in Amazon CloudSearch based on events occurring in Amazon DynamoDB. AWS Lambda functions can be triggered to perform actions in response to Amazon DynamoDB table changes.

WRITTEN BY Aniket Kumar Ambasta

Aniket Kumar Ambasta works as a Research associate- TC - Infra, Security, and Migration at CloudThat. He is AWS Solutions Architect- Associate certified and has completed his Bachelor's in Computer Applications. He has good experience in Cloud technologies. Apart from professional interests, he loves exploring and learning new technologies.