Voiced by Amazon Polly |
Introduction
Data synchronization is syncing data across two or more devices and automatically updating changes between them to ensure system consistency.
While the massive amount of data stored in the cloud poses issues, it also makes it the ideal option for big data. Today’s data solutions provide quick and straightforward tools for avoiding repetitive activities, resulting in data in sync throughout the system.
Customized Cloud Solutions to Drive your Business Success
- Cloud Migration
- Devops
- AIML & IoT
Agent creation
Use the CLI command below to acquire the most recent DataSync Amazon Machine Image (AMI) ID for the selected AWS Region.
- aws ssm get-parameter –name /aws/service/datasync/ami –region $region
Launch the agent using your AMI from the Amazon EC2 launch wizard from the AWS account where the source file system is stored. To start the AMI, go to the following URL: https://console.aws.amazon.com/ec2/v2/home?region=source-file-system-region#LaunchInstanceWizard:ami=ami-id
Note:- Please make sure that you are launching a machine having RAM of 16 GiB or more
Here I am launching t2.xlarge for my agent with public IP and opening an HTTP port.
Once your agent creating is done now, let us go to DataSync.
Step by Step Guide for Configuring DataSync
- Open the DataSync service from the AWS Console and select the below option for the data transfer option. Click on Get Started.
- Select the Hypervisor as AWS EC2, select the Endpoint type as Public service Endpoints, and provide the public IP of the agent in the agent address as below
- Once the activation key retrieval is successful you can provide the agent with a name and click on create button then your agent will be created and shows Agent status as Online.
- Before creating a task for the DataSync, create an EC2 (t2.micro) which will act as an On-premise system.
- Please open the RDP and NFS ports in the On-premise security group
- Now install the NFS server On-Premise such that it will sync the local files to the cloud through NFS.
- Create a folder and text file inside it. In my case, I have created a folder with the name test and a text file with the name sampletext.txt, which we want to sync. Go to the properties of the folder. Select the NFS sharing option and click on Manage NFS sharing and check the share this folder as shown below.
- Click Apply and then OK to save the configuration over the folder
- Now go to the DataSync console and create a task and choose the configuration details as below:
a. Location as the new location
b. The location type should be Network File System (NFS)
c. Select the agent which we created in the steps
d. Provide the Ip address in the NFS server domain
e. Provide the mount path in my case as /test
f. The destination location is S3, and select the S3 bucket to where these files need to be sync - Leave the remaining fields as default and click on create
- Select the autogenerate for creating the IAM in the provided fields
- Once the previous step is completed, wait until your task’s status becomes available.
- Now start the task and wait for a while; this starts syncing your on-premise data to AWS S3
Conclusion
We investigated and highlighted the relevance of data synchronization in this blog. When just the changed data is transferred, data synchronization will work smoothly. As a result, each synchronization procedure uses a marker to determine the most up-to-date information.
Get your new hires billable within 1-60 days. Experience our Capability Development Framework today.
- Cloud Training
- Customized Training
- Experiential Learning
About CloudThat
CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.
CloudThat is the first Indian Company to win the prestigious Microsoft Partner 2024 Award and is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 850k+ professionals in 600+ cloud certifications and completed 500+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, AWS GenAI Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, Amazon ECS Service Delivery Partner, AWS Glue Service Delivery Partner, Amazon Redshift Service Delivery Partner, AWS Control Tower Service Delivery Partner, AWS WAF Service Delivery Partner, Amazon CloudFront Service Delivery Partner, Amazon OpenSearch Service Delivery Partner, AWS DMS Service Delivery Partner, AWS Systems Manager Service Delivery Partner, Amazon RDS Service Delivery Partner, AWS CloudFormation Service Delivery Partner, AWS Config, Amazon EMR and many more.
FAQs
1. Where can I move data to and from?
ANS: – DataSync supports the following storage location types: Network File System (NFS) shares, Server Message Block (SMB) shares, Hadoop Distributed File Systems (HDFS), self-managed object storage, Google Cloud Storage, Azure Files, AWS Snowcone, Amazon Simple Storage Service (Amazon S3), Amazon Elastic File System (Amazon EFS) file systems, Amazon FSx for Windows File Server file systems, Amazon FSx for Lustre file systems, and Amazon FSx for OpenZFS file systems.
2. Can I use AWS DataSync to copy data from other public clouds to AWS?
ANS: – Yes. Using AWS DataSync, you can copy data from Google Cloud Storage using the S3 API or Azure Files using the SMB protocol. Deploy the DataSync agent in your cloud environment or on Amazon EC2, create your source and destination locations, and then start your task to begin copying data. Learn more about using DataSync to copy data from Google Cloud Storage or Azure Files.
3. Does AWS DataSync preserve the directory structure when copying files?
ANS: – Yes. When transferring files, AWS DataSync creates the same directory structure as the source location’s structure on the destination.
WRITTEN BY Shaik Munwar Basha
Comments