A Guide to Integrate Amazon CloudSearch with Amazon DynamoDB

Overview

Amazon CloudSearch is a fully managed cloud service that is easy to set up. It can search massive collections of information such as web pages, documents, and forum posts without worrying about setting up hardware or maintenance. It also scales automatically if your application experiences a change in traffic.

It expands automatically to tackle the volume of data search queries to provide quick and accurate results. It provides automatic indexing and scalability. Every searchable item on Amazon CloudSearch has its ID, which is unique compared to the other items in the same domain, ID is important when the Document is modified or deleted later.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Steps to set up a search solution with Amazon CloudSearch

Create and configure a search domain. A search domain contains instances that handle your search requests and searchable data.
Upload the data you want to search to your domain. Amazon CloudSearch deploys the search index to one or more search instances and automatically indexes your data.
Search your domain. You send a search query to the search endpoint of your domain as a HTTP/HTTPS GET request.

Features of Amazon CloudSearch

Auto Complete
Scalable
Reliable
Fully Managed
Query parser
High Performance

Steps to Setup Amazon CloudSearch Domain

Select create a new search Domain.

Give your Amazon CloudSearch domain a suitable name, select your desired Instance type, and click continue.

step1

Select the Amazon DynamoDB table you want to use in the Amazon CloudSearch Domain and click continue.

step2

Amazon CloudSearch does not support list attributes, so instead of a list, use String set in Amazon DynamoDB tables Item.

step3

Amazon CloudSearch does the indexing Automatically. Review the indexing and click continue.

step4

Review all details and select Confirm.

step5

After confirming, it will say, “You have successfully created the domain <your domain name here>.” After that, it will take approximately 10-15 minutes to become active, and only then can any actions be performed on the domain.

Wait for it to become Active.

Initially, it will not contain any searchable documents. Once it is active, we must upload the documents to it to appear in search results. Click on Upload Documents.

step6

Select the Amazon DynamoDB table and click continue.

step7

Click on Finish.

Now, it will show the number of Searchable Documents in the Amazon DynamoDB table.

step8

step8b

Working with Endpoints

Search Endpoint

It is used to submit a search request to the Amazon CloudSearch. It can be used on a browser, postman or we can make a GET API request from the code as well. The format of using the endpoint is:

https://<search-endpoint>/2013-01-01/search?q=<search-item-name>

You can also view all the items available for search in Amazon CloudSearch domain by adding “matchall&q.parser=structured” in the query .

https://<search-endpoint>/2013-01-01/search?q=matchall&q.parser=structured

If the search text is incomplete, a “*” can be added at the last to show all the results associated with that word.

Document Endpoint

Document Endpoint is used to add, replace, or Delete Documents from the Amazon CloudSearch Domain.

AWS Lambda can dynamically add and delete the Documents from Amazon CloudSearch. As soon as any item is added or deleted from the Amazon DynamoDB table, AWS Lambda will be triggered and perform the desired function.

Data must be uploaded to the domain in JSON format. A POST request will be made, requiring a document endpoint URL, headers, and data as parameters.

There is no option to delete the documents on the Amazon CloudSearch console, so if the data is removed from Amazon DynamoDB, it will still appear in search results. Deleting the documents from the Amazon CloudSearch domain with the help of Document Endpoint is important.

Format of Document Endpoint is:

https://<document-endpoint>/2013-01-01/documents/batch

To add Documents from Document Endpoint:

url = "<your-document-endpoint>/2013-01-01/documents/batch"
input = [
                    {“type”: "add”,
                  "id": “789456”,
                  "fields”: {
                    "unique_id":"789456",
                    "description":"this is demo code for adding documents to the cloudsearch domain",
                    "list_example":["aws","cloudsearch"]
                  } } ]
input = json.dumps(input)
headers = {'Content-Type': 'application/json'}
response = requests.request( "POST", url, headers=headers, data=input)

url = "<your-document-endpoint>/2013-01-01/documents/batch"

input = [

{“type”: "add”,

"id": “789456”,

"fields”: {

"unique_id":"789456",

"description":"this is demo code for adding documents to the cloudsearch domain",

"list_example":["aws","cloudsearch"]

} } ]

input = json.dumps(input)

headers = {'Content-Type': 'application/json'}

response = requests.request( "POST", url, headers=headers, data=input)

Deleting a Document from Document Endpoint

url = "<you-document-endpoint>/2013-01-01/documents/batch"
input = [
                    {"type":"delete",
                  "id": “789456”
                  } ]
input = json.dumps( input )
headers = {'Content-Type':'application/json'}
response = requests.request( "POST" , url , headers = headers , data = input )

url = "<you-document-endpoint>/2013-01-01/documents/batch"

input = [

{"type":"delete",

"id": “789456”

} ]

input = json.dumps( input )

headers = {'Content-Type':'application/json'}

response = requests.request( "POST" , url , headers = headers , data = input )

Conclusion

Amazon CloudSearch provides low-cost search operations to the website without trying to monitor, index, and provision. It supports HTTPS and integrates with Identity and Access Management (IAM) to manage the Search Domain.

Drop a query if you have any questions regarding Amazon CloudSearch and we will get back to you quickly.

Making IT Networks Enterprise-ready – Cloud Management Services

Accelerated cloud migration
End-to-end view of the cloud environment

Get Started

About CloudThat

CloudThat is an award-winning company and the first in India to offer cloud training and consulting services worldwide. As a Microsoft Solutions Partner, AWS Advanced Tier Training Partner, and Google Cloud Platform Partner, CloudThat has empowered over 850,000 professionals through 600+ cloud certifications winning global recognition for its training excellence including 20 MCT Trainers in Microsoft’s Global Top 100 and an impressive 12 awards in the last 8 years. CloudThat specializes in Cloud Migration, Data Platforms, DevOps, IoT, and cutting-edge technologies like Gen AI & AI/ML. It has delivered over 500 consulting projects for 250+ organizations in 30+ countries as it continues to empower professionals and enterprises to thrive in the digital-first world.

FAQs

1. How does Amazon CloudSearch work with Amazon DynamoDB?

ANS: – Amazon CloudSearch can be integrated with Amazon DynamoDB to enable powerful search capabilities on the data stored in DynamoDB tables. This integration allows for seamless indexing and querying of Amazon DynamoDB data using Amazon CloudSearch.

2. What are the key features of Amazon CloudSearch?

ANS: – Amazon CloudSearch is supported in various regions, including Europe (Ireland), Asia Pacific (Singapore, Sydney, Seoul, Tokyo), US East (N. Virginia), US West (N. California, Oregon), South America (São Paulo), and Europe (Frankfurt).

3. Can I use AWS Lambda with Amazon CloudSearch and Amazon DynamoDB integration?

ANS: – Yes, AWS Lambda can be utilized to dynamically add, update, or delete documents in Amazon CloudSearch based on events occurring in Amazon DynamoDB. AWS Lambda functions can be triggered to perform actions in response to Amazon DynamoDB table changes.

WRITTEN BY Aniket Kumar Ambasta

Aniket Kumar Ambasta works as a Research associate- TC - Infra, Security, and Migration at CloudThat. He is AWS Solutions Architect- Associate certified and has completed his Bachelor's in Computer Applications. He has good experience in Cloud technologies. Apart from professional interests, he loves exploring and learning new technologies.