Skip to content

Commit

Permalink
docs: add example query
Browse files Browse the repository at this point in the history
  • Loading branch information
patheard committed Nov 19, 2024
1 parent e9a4855 commit 4302057
Showing 1 changed file with 9 additions and 1 deletion.
10 changes: 9 additions & 1 deletion docs/data/pipelines/operations/aws/cost-and-usage-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@
## Description
The AWS [Cost and Usage Report (CUR) 2.0](https://docs.aws.amazon.com/cur/latest/userguide/what-is-cur.html) provides detailed billing data exports in [Parquet format](https://parquet.apache.org/). It contains line items for all AWS services usage with resource tags, pricing, and cost allocation data. The data is partitioned by time period and account ID, and updated daily.

This data pipeline creates the Glue data catalog table `cost_usage_report_by_account` in the `operations_aws_production` database. It can be queried in Superset as follows:

```sql
SELECT *
FROM "operations_aws_production"."cost_usage_report_by_account"
LIMIT 10;
```

## Data pipeline
A high level view of the data pipeline is shown below:

Expand Down Expand Up @@ -74,4 +82,4 @@ cds-data-lake-transformed-production/operations/aws/cost-usage-report/data/billi

Additionally, a data catalog table is created in the [`operations_aws_production` database](https://github.com/cds-snc/data-lake/blob/468142031c7bdd1a2720def7d5ebb4e07fff4bef/terragrunt/aws/glue/databases.tf#L1-L4):

- `cost_usage_report_by_account`: CUR 2.0 export data joined with the AWS account tags. This table is made available to users for analysis.
- `cost_usage_report_by_account`: CUR 2.0 export data joined with the AWS account tags. This table is made available to users for analysis in Superset.

0 comments on commit 4302057

Please sign in to comment.