Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add Athena workgroup #21

Merged
merged 2 commits into from
Nov 13, 2024
Merged

feat: add Athena workgroup #21

merged 2 commits into from
Nov 13, 2024

Conversation

patheard
Copy link
Member

Summary

Add an Athena workgroup and query result bucket. This will allow queries to be run against the data lake buckets.

Related

@patheard patheard self-assigned this Nov 13, 2024
Add an Athena workgroup and query result bucket.  This will allow queries to
be run against the data lake buckets.
Copy link
Contributor

Production: buckets 🪣

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

Plan: 2 to add, 0 to change, 0 to destroy
Show summary
CHANGE NAME
add module.athena_bucket.aws_s3_bucket.this
module.athena_bucket.aws_s3_bucket_public_access_block.this
Show plan
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.athena_bucket.aws_s3_bucket.this will be created
  + resource "aws_s3_bucket" "this" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "cds-data-lake-athena-production"
      + bucket_domain_name          = (known after apply)
      + bucket_prefix               = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + object_lock_enabled         = (known after apply)
      + policy                      = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags                        = {
          + "CostCentre" = "PlatformDataLake"
          + "Critical"   = "false"
          + "Terraform"  = "true"
        }
      + tags_all                    = {
          + "CostCentre" = "PlatformDataLake"
          + "Critical"   = "false"
          + "Terraform"  = "true"
        }
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + cors_rule (known after apply)

      + grant (known after apply)

      + lifecycle_rule {
          + enabled = true
          + id      = "expire_all"

          + expiration {
              + days = 30
            }
        }
      + lifecycle_rule {
          + abort_incomplete_multipart_upload_days = 7
          + enabled                                = true
          + id                                     = "remove_noncurrent_versions"

          + noncurrent_version_expiration {
              + days = 30
            }
        }

      + logging {
          + target_bucket = "cds-data-lake-bucket-logs-production"
          + target_prefix = "athena/"
        }

      + object_lock_configuration (known after apply)

      + replication_configuration (known after apply)

      + server_side_encryption_configuration {
          + rule {
              + bucket_key_enabled = false

              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

      + versioning {
          + enabled    = true
          + mfa_delete = false
        }

      + website (known after apply)
    }

  # module.athena_bucket.aws_s3_bucket_public_access_block.this will be created
  + resource "aws_s3_bucket_public_access_block" "this" {
      + block_public_acls       = true
      + block_public_policy     = true
      + bucket                  = (known after apply)
      + id                      = (known after apply)
      + ignore_public_acls      = true
      + restrict_public_buckets = true
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + athena_bucket_arn       = (known after apply)
  + athena_bucket_name      = (known after apply)

Warning: Argument is deprecated

  with module.athena_bucket.aws_s3_bucket.this,
  on .terraform/modules/athena_bucket/S3/main.tf line 8, in resource "aws_s3_bucket" "this":
   8: resource "aws_s3_bucket" "this" {

Use the aws_s3_bucket_lifecycle_configuration resource instead

(and 22 more similar warnings elsewhere)

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"
Show Conftest results
20 tests, 20 passed, 0 warnings, 0 failures, 0 exceptions

Copy link
Contributor

Production: athena 🦉

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

Plan: 1 to add, 0 to change, 0 to destroy
Show summary
CHANGE NAME
add aws_athena_workgroup.data_lake
Show plan
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_athena_workgroup.data_lake will be created
  + resource "aws_athena_workgroup" "data_lake" {
      + arn           = (known after apply)
      + force_destroy = false
      + id            = (known after apply)
      + name          = "data-lake-production"
      + state         = "ENABLED"
      + tags_all      = {
          + "CostCentre" = "PlatformDataLake"
          + "Terraform"  = "true"
        }

      + configuration {
          + enforce_workgroup_configuration    = true
          + publish_cloudwatch_metrics_enabled = true
          + requester_pays_enabled             = false

          + result_configuration {
              + output_location = "s3://mock-athena-bucket/data-lake/"

              + encryption_configuration {
                  + encryption_option = "SSE_S3"
                }
            }
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"
Show Conftest results
WARN - plan.json - main - Missing Common Tags: ["aws_athena_workgroup.data_lake"]

20 tests, 19 passed, 1 warning, 0 failures, 0 exceptions

@patheard patheard requested a review from wmoussa-gc November 13, 2024 14:30
@patheard patheard merged commit d1283a1 into main Nov 13, 2024
4 checks passed
@patheard patheard deleted the feat/athena branch November 13, 2024 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants