Voiced by Amazon Polly |
Introduction
As organizations modernize their infrastructure and migrate databases to the cloud, data security and migration accuracy remain at the forefront of operational concerns. Amazon Web Services (AWS) continues to innovate in this space through enhancements in its Database Migration Service (DMS). With the release of AWS DMS version 3.5.4, two impactful features have been introduced: column-level data masking and enhanced data validation performance.
These additions are especially valuable for enterprises handling sensitive information and large-scale migrations, providing tools to safeguard data while ensuring platform accuracy. In this blog, we dive into the details of these new capabilities, walk through the step-by-step process of configuring data masking, and explore how they contribute to more secure and efficient migrations.
Pioneers in Cloud Consulting & Migration Services
- Reduced infrastructural costs
- Accelerated application deployment
Data Masking
Data masking allows organizations to protect confidential information by transforming it into anonymized or obfuscated formats during migration. This is particularly important when dealing with personally identifiable information (PII), financial records, or health data, where unauthorized exposure can lead to compliance violations or reputational damage.
Key Data Masking Techniques Introduced
AWS DMS 3.5.4 introduces three masking techniques that can be applied on a per-column basis:
- Mask Digits
This replaces all numerical digits in the selected column with a specified character (e.g., X, #), preserving the data’s structural integrity while concealing the actual values. - Randomize Digits
Digits in the data are replaced with random numbers. This technique maintains the same format but prevents pattern recognition or reverse engineering of original values. - Hash Value
Applies a one-way hash function to the data, converting it into a consistent but irreversible string. This is useful for anonymizing data while retaining uniqueness (e.g., for joins or comparisons).
These transformations can be configured directly within the AWS DMS task settings, making integrating security into the migration workflow easier without custom coding or external tools.
Step-by-Step Guide to Implement Data Masking in AWS DMS
Follow these steps to apply data masking to your AWS DMS migration tasks:
Step 1: Create or Modify a Replication Task
Navigate to the AWS DMS console and create a new replication task or edit an existing one. Ensure your source and target endpoints (e.g., Oracle to PostgreSQL) are correctly configured.
Step 2: Go to Table Mappings
Within the task settings, locate the Table mappings section. This is where you will define the rules for selecting tables, columns, and how the data should be transformed.
Step 3: Add a Transformation Rule
Click “Add transformation rule” and choose transformation-type as mask-column.
- Define the schema, table, and column names where the masking should be applied.
Step 4: Choose the Masking Technique
In the transformation options:
- Select the masking type: mask-digits, random digits, or hash-value.
- Specify a character (e.g., X) that should replace the digits for mask digits.
Step 5: Save and Review
Once the rules are added, review the table mapping settings to confirm that the correct columns apply the appropriate transformation logic.
Step 6: Run the Replication Task
Start the task. AWS DMS will automatically apply the selected data masking rules during migration, ensuring that the destination database receives protected data without manual intervention.
Boosting Performance
Apart from security, AWS DMS 3.5.4 also brings significant performance improvements to data validation, which is essential to ensure data accuracy between the source and target databases.
Benefits of Improved Validation
- Faster processing of large datasets
- Reduced migration lag during full load or change data capture (CDC) operations
- Lower resource utilization on validation tasks
This enhancement currently supports several key migration paths, including:
- Oracle → PostgreSQL
- SQL Server → PostgreSQL
- Oracle → Oracle
- SQL Server → SQL Server
By optimizing the validation engine, AWS DMS ensures high fidelity and consistency in data movement, even for enterprise-scale databases.
Conclusion
The new features in AWS DMS 3.5.4 represent a major stride forward in secure and scalable cloud migration. With the ability to mask sensitive data on the fly and validate large datasets more efficiently, organizations can now meet regulatory requirements and operational demands more confidently.
Drop a query if you have any questions regarding AWS DMS and we will get back to you quickly.
Empowering organizations to become ‘data driven’ enterprises with our Cloud experts.
- Reduced infrastructure costs
- Timely data-driven decisions
About CloudThat
CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.
CloudThat is the first Indian Company to win the prestigious Microsoft Partner 2024 Award and is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 850k+ professionals in 600+ cloud certifications and completed 500+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, AWS GenAI Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, Amazon ECS Service Delivery Partner, AWS Glue Service Delivery Partner, Amazon Redshift Service Delivery Partner, AWS Control Tower Service Delivery Partner, AWS WAF Service Delivery Partner, Amazon CloudFront Service Delivery Partner, Amazon OpenSearch Service Delivery Partner, AWS DMS Service Delivery Partner, AWS Systems Manager Service Delivery Partner, Amazon RDS Service Delivery Partner, AWS CloudFormation Service Delivery Partner, AWS Config, Amazon EMR and many more.
FAQs
1. Can I apply different masking types to different columns in the same table?
ANS: – Yes. AWS DMS allows you to define transformation rules for each column independently so that you can mix mask digits, randomize digits, or hash values based on the sensitivity and use case of each field.
2. Will the masked data affect referential integrity or joins in the target database?
ANS: – It depends. Using a consistent masking technique like hashing, you can preserve joins if the hash output remains the same across records. However, randomize-digits or mask-digits may break relational consistency since they alter values differently.
WRITTEN BY Lakshmi P Vardhini
Comments