Skip to content

Commit

Permalink
docs: update graph to show source data
Browse files Browse the repository at this point in the history
  • Loading branch information
patheard committed Nov 19, 2024
1 parent 0751d91 commit 745fede
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions docs/data/pipelines/operations/aws/cost-and-usage-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@ LIMIT 10;
```

## Data pipeline
A high level view of the data pipeline is shown below:
A high level view is shown below with more details about each step following the diagram.

```mermaid
graph TD
%% Source Systems
CUR[AWS CUR 2.0 Export]
Lambda[Account Tags Lambda]
Lambda["`**Lambda**<br>Extract account tags`"]
%% Storage
OrgS3["`**Organization S3 Bucket**<br/>cds-cost-usage-report`"]
OrgS3["`**S3 Bucket**<br/>cds-cost-usage-report`"]
RawS3["`**Raw S3 Bucket**<br/>cds-data-lake-raw-production`"]
TransS3["`**Transformed S3 Bucket**<br/>cds-data-lake-transformed-production`"]
Expand All @@ -33,15 +33,15 @@ graph TD
ETL["ETL Job (Daily)"]
%% Flow
subgraph org[Organization]
subgraph org[AWS Organization account]
CUR --> OrgS3
Lambda
end
Lambda --> |account-tags.json|RawS3
OrgS3 --> |S3 Replication|RawS3
subgraph datalake[Data Lake]
subgraph datalake[Data Lake account]
RawS3 --> Crawlers
Crawlers --> |Updates Schema|CatalogRaw
CatalogRaw --> ETL
Expand Down

0 comments on commit 745fede

Please sign in to comment.