Simplifying AWS Cost Reporting with AWS Data Exports

Introduction

As organizations scale their AWS environments, managing cost and usage data across multiple accounts and services becomes complex. While AWS Cost and Usage Reports (CUR) provide detailed billing information, teams often need only specific data for reporting, chargeback, or auditing.

Traditionally, organizations exported the complete dataset to Amazon S3 and used services such as AWS Lambda, AWS Glue, or Amazon Athena for filtering, adding processing overhead.

AWS Data Exports simplifies this process with SQL-based row filtering in CUR 2.0. Teams can select required columns and filter records before delivery to Amazon S3, enabling focused cost exports and simpler FinOps workflows.

Key Features of AWS Data Exports

• SQL-Based Row Filtering – Filter CUR 2.0 records using WHERE clauses based on account, service, Region, or other supported criteria.
• Custom Column Selection – Select only the required columns to create focused and consistent datasets.
• Automated Amazon S3 Delivery – Automatically delivers filtered exports to Amazon S3 for analytics and reporting.
• Flexible Querying – Supports SELECT, WHERE, AND, OR, IN, comparison operators, and LIMIT for targeted exports.
• Infrastructure as Code – Data Exports can be managed using AWS APIs, CLI, SDKs, and CloudFormation for repeatable deployments.

Benefits of Filtered Cost and Usage Exports

• Reduced Data Processing – Filtering during export eliminates the need for separate downstream filtering.
• Reduced Data Volume – Selecting required rows and columns reduces unnecessary data delivery and processing.
• Simplified Architecture – Reduces dependency on separate Lambda, Glue, Athena, or ETL-based filtering pipelines.
• Improved Data Governance – Provides focused exports for specific customers, departments, projects, or reporting needs.
• Automated Reporting – Recurring exports automatically deliver cost and usage data to Amazon S3.
• Consistent Data Schema – Controlled column selection provides a predictable dataset for downstream systems.

Use Cases

• Partner Billing – Generate cost and usage exports for specific customer accounts to support billing and transparency.
• Chargeback and Cost Allocation – Isolate costs by account, department, or program for internal chargeback and showback.
• Audit Reporting – Create focused datasets containing only the cost information required for audits.
• MSP Reporting – Create customer-specific exports for managed AWS environments.
• Service-Level Cost Analysis – Filter CUR 2.0 data to analyze spending for selected AWS services.
• Regional Cost Analysis – Create exports focused on specific AWS Regions for targeted cost analysis.

Technical Implementation and Architecture

The solution uses AWS Data Exports to query CUR 2.0 and deliver filtered results to Amazon S3.

The implementation includes four main components:

• accounts.csv – Stores the AWS account IDs to include in the export, making filter updates easier.
• query.sql – Defines the SQL SELECT statement and required CUR 2.0 columns.
• deploy.sh – Automates deployment and generates the required filtering condition from the account list.
• template.yaml – Defines the Data Export configuration and Amazon S3 bucket policy.

The overall architecture can be represented as:

Account Configuration → SQL Query → AWS Data Exports → Filtered CUR 2.0 Data → Amazon S3

AWS Data Exports executes the SQL query against the CUR 2.0 table and delivers the resulting dataset to the configured S3 destination. Data Exports supports SQL column selection and row filtering, while table configurations can control settings such as time granularity and resource-level data.

Deployment Workflow

The solution follows a simple automated deployment process:

Step 1: Configure Account IDs
Add the required AWS account IDs to accounts.csv to define the export scope.

Step 2: Define the Query
Use query.sql to select the required CUR 2.0 columns and filtering conditions.

Step 3: Generate the Filter
The deployment process reads the account IDs and generates the required SQL filter.

Step 4: Deploy Resources
CloudFormation deploys the Data Export configuration and required Amazon S3 permissions.

Step 5: Deliver Data
AWS Data Exports delivers the filtered dataset to Amazon S3 for downstream analytics and reporting.

Initial delivery can take up to 24 hours. Once active, billing and cost management exports are refreshed at least daily, although the exact timing may vary.

Challenges and Optimizations

• Managing Multiple Accounts – Maintaining account IDs separately in a configuration file makes updates easier and keeps SQL queries manageable.
• Large Datasets – Selecting only required rows and columns reduces unnecessary data processing and movement.
• Complex Filtering – SQL-based filtering supports different combinations of account, service, Region, and other criteria.
• Operational Complexity – Filtering within Data Exports reduces the need for separate ETL pipelines and their maintenance.
• Schema Management – Explicit column selection helps maintain a consistent and predictable output structure.

Extending the Solution

The filtered export can become the foundation for broader FinOps and analytics workflows.

For example:

AWS Data Exports → Amazon S3 → Amazon Athena → Amazon QuickSight

Amazon S3 stores the exported data, while Athena and QuickSight can query and visualize it for cost analysis.

Organizations can create separate export configurations for different needs, such as customer-specific, account-level, or service-level reporting.

Since filtering is defined within Data Exports, configurations can be easily updated as requirements change. AWS Data Exports also supports additional datasets, including cost optimization recommendations, FOCUS data, and carbon emissions data.

Conclusion

Managing AWS cost and usage data across multiple accounts and services can be complex. AWS Data Exports simplifies this by allowing organizations to select required columns and filter CUR 2.0 data using SQL before delivery to Amazon S3.

This reduces unnecessary processing, simplifies architecture, and supports efficient FinOps workflows. By combining SQL-based filtering, automated Amazon S3 delivery, and Infrastructure as Code, organizations can build scalable, maintainable cost-reporting solutions tailored to stakeholder needs.