Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Swithcover for MySQL and PostgreSQL #12646

Merged
merged 7 commits into from
Jan 11, 2025

Conversation

lychgoogle
Copy link
Contributor

@lychgoogle lychgoogle commented Dec 24, 2024

Release Note Template for Downstream PRs (will be copied)

sql: added `replication_cluster` field to `google_sql_database_instance` resource
sql: added support of switchover for MySQL and PostgreSQL in `google_sql_database_instance` resource

@github-actions github-actions bot requested a review from zli82016 December 24, 2024 02:54
Copy link

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

@zli82016, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 1188 insertions(+), 22 deletions(-))
google-beta provider: Diff ( 4 files changed, 1188 insertions(+), 22 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 106
Passed tests: 61
Skipped tests: 15
Affected tests: 30

Click here to see the affected service packages
  • sql

Action taken

Found 30 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccSQLDatabaseInstance_sqlMysqlDataCacheConfig
  • TestAccSQLDatabaseInstance_sqlPostgresDataCacheConfig
  • TestAccSqlDatabaseInstance_Edition_Downgrade
  • TestAccSqlDatabaseInstance_EnableGoogleDataplexIntegration
  • TestAccSqlDatabaseInstance_EnableGoogleMlIntegration
  • TestAccSqlDatabaseInstance_MysqlSwitchoverSuccess
  • TestAccSqlDatabaseInstance_Mysql_Edition_Upgrade
  • TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled
  • TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer
  • TestAccSqlDatabaseInstance_PostgresSwitchoverSuccess
  • TestAccSqlDatabaseInstance_Postgres_Edition_Upgrade
  • TestAccSqlDatabaseInstance_Smt
  • TestAccSqlDatabaseInstance_SqlServerTimezoneUpdate
  • TestAccSqlDatabaseInstance_SwitchoverSuccess
  • TestAccSqlDatabaseInstance_activationPolicy
  • TestAccSqlDatabaseInstance_authNets
  • TestAccSqlDatabaseInstance_basicMSSQL
  • TestAccSqlDatabaseInstance_basic_with_user_labels
  • TestAccSqlDatabaseInstance_deleteDefaultUserBeforeSubsequentApiCalls
  • TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable
  • TestAccSqlDatabaseInstance_settings_deletionProtection
  • TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled
  • TestAccSqlDatabaseInstance_settings_upgrade
  • TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled
  • TestAccSqlDatabaseInstance_updateSslOptionsForPostgreSQL
  • TestAccSqlDatabaseInstance_withPSCEnabled_thenAddAllowedConsumerProjects_thenRemoveAllowedConsumerProject
  • TestAccSqlDatabaseInstance_withPSCEnabled_thenAddPscAutoConnections_thenRemovePscAutoConnections
  • TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange
  • TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange
  • TestAccSqlDatabase_instanceWithActivationPolicy

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🟢 Tests passed during RECORDING mode:
TestAccSQLDatabaseInstance_sqlMysqlDataCacheConfig [Debug log]
TestAccSQLDatabaseInstance_sqlPostgresDataCacheConfig [Debug log]
TestAccSqlDatabaseInstance_Edition_Downgrade [Debug log]
TestAccSqlDatabaseInstance_EnableGoogleDataplexIntegration [Debug log]
TestAccSqlDatabaseInstance_EnableGoogleMlIntegration [Debug log]
TestAccSqlDatabaseInstance_MysqlSwitchoverSuccess [Debug log]
TestAccSqlDatabaseInstance_Mysql_Edition_Upgrade [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer [Debug log]
TestAccSqlDatabaseInstance_PostgresSwitchoverSuccess [Debug log]
TestAccSqlDatabaseInstance_Postgres_Edition_Upgrade [Debug log]
TestAccSqlDatabaseInstance_Smt [Debug log]
TestAccSqlDatabaseInstance_SqlServerTimezoneUpdate [Debug log]
TestAccSqlDatabaseInstance_SwitchoverSuccess [Debug log]
TestAccSqlDatabaseInstance_activationPolicy [Debug log]
TestAccSqlDatabaseInstance_authNets [Debug log]
TestAccSqlDatabaseInstance_basicMSSQL [Debug log]
TestAccSqlDatabaseInstance_basic_with_user_labels [Debug log]
TestAccSqlDatabaseInstance_deleteDefaultUserBeforeSubsequentApiCalls [Debug log]
TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable [Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtection [Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled [Debug log]
TestAccSqlDatabaseInstance_settings_upgrade [Debug log]
TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled [Debug log]
TestAccSqlDatabaseInstance_updateSslOptionsForPostgreSQL [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_thenAddAllowedConsumerProjects_thenRemoveAllowedConsumerProject [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_thenAddPscAutoConnections_thenRemovePscAutoConnections [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange [Debug log]
TestAccSqlDatabase_instanceWithActivationPolicy [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🟢 All tests passed!

View the build log or the debug log for each test

Copy link

@zli82016 This PR has been waiting for review for 3 weekdays. Please take a look! Use the label disable-review-reminders to disable these notifications.

@zli82016
Copy link
Member

It looks like the switchover method is not called from Terraform provider. Is this method called in API side to complete the switchover operation?

@lychgoogle
Copy link
Contributor Author

It looks like the switchover method is not called from Terraform provider. Is this method called in API side to complete the switchover operation?

I'm not familiar with terraform (still), but the API is called from the provider code

retryFunc = func() (rerr error) {
op, rerr = config.NewSqlAdminClient(userAgent).Instances.Switchover(project, d.Get("name").(string)).Do()
return rerr

@github-actions github-actions bot requested a review from zli82016 December 27, 2024 19:00
@lychgoogle
Copy link
Contributor Author

@zli82016 I briefly replied to all of your comment. PTAL

Copy link
Member

@zli82016 zli82016 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some comments. Let me know if you have questions.

@lychgoogle
Copy link
Contributor Author

I added some comments. Let me know if you have questions.

@zli82016 I briefly replied to all of your comment. PTAL

I'm currently fixing test. As side affect of setting replication_cluster as Computed: false, I have to set ExpectNonEmptyPlan: true for many test steps. This is because replication_cluster (and many other fields) of googld_sql_database_instance can be changed without updating it directly, but by updating another googld_sql_database_instance.

If ExpectNonEmptyPlan: true is OK, then I'll continue work on tests. (and also public docs)

@github-actions github-actions bot requested a review from zli82016 January 3, 2025 19:32
@zli82016
Copy link
Member

zli82016 commented Jan 3, 2025

replication_cluster

Sorry to hear that. Do you have the logs for the test that the plan is not empty?

@lychgoogle
Copy link
Contributor Author

replication_cluster

Sorry to hear that. Do you have the logs for the test that the plan is not empty?

I'm still not familiar with terraform (this is first big change I'm making), the first (and probably the biggest) problem I'm having is following: (All manual tests are done after reflecting your suggestion)

  1. We have main.tf as follows (and corresponding resource in cloud SQL backend).
resource "google_sql_database_instance" "original-primary" {
  provider            = google-beta
  project             = "lych-dev-plg-1"
  name                = "tf-test-2025-01-03-prim-1"
  region              = "us-east1"
  database_version    = "MYSQL_8_0"
  instance_type       = "CLOUD_SQL_INSTANCE"
  deletion_protection = false

  replication_cluster {
    failover_dr_replica_name = "lych-dev-plg-1:tf-test-2025-01-03-prim-1-rep-1"
  }

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
    backup_configuration {
      enabled            = true
      binary_log_enabled = true
    }
  }
}

resource "google_sql_database_instance" "original-replica" {
  provider             = google-beta
  project              = "lych-dev-plg-1"
  name                 = "tf-test-2025-01-03-prim-1-rep-1"
  region               = "us-west2"
  database_version     = "MYSQL_8_0"
  instance_type        = "READ_REPLICA_INSTANCE"
  master_instance_name = google_sql_database_instance.original-primary.name
  deletion_protection  = false

  replication_cluster {}

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
  }
}
  1. We make following change to "original primary", and issue "terraform apply"
resource "google_sql_database_instance" "original-primary" {
  provider            = google-beta
  project             = "lych-dev-plg-1"
  name                = "tf-test-2025-01-03-prim-1"
  region              = "us-east1"
  database_version    = "MYSQL_8_0"
  instance_type       = "CLOUD_SQL_INSTANCE"
  deletion_protection = false

-  replication_cluster {
-    failover_dr_replica_name = "lych-dev-plg-1:tf-test-2025-01-03-prim-1-rep-1"
-  }

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
    backup_configuration {
      enabled            = true
      binary_log_enabled = true
    }
  }
}

resource "google_sql_database_instance" "original-replica" {
  provider             = google-beta
  project              = "lych-dev-plg-1"
  name                 = "tf-test-2025-01-03-prim-1-rep-1"
  region               = "us-west2"
  database_version     = "MYSQL_8_0"
  instance_type        = "READ_REPLICA_INSTANCE"
  master_instance_name = google_sql_database_instance.original-primary.name
  deletion_protection  = false

  replication_cluster {}

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
  }
}
  1. But update operation of "original-primary" removes "original-replica"'s replication_cluster (it is intended in cloud SQL), so I want to delete that in main.tf also.
resource "google_sql_database_instance" "original-primary" {
  provider            = google-beta
  project             = "lych-dev-plg-1"
  name                = "tf-test-2025-01-03-prim-1"
  region              = "us-east1"
  database_version    = "MYSQL_8_0"
  instance_type       = "CLOUD_SQL_INSTANCE"
  deletion_protection = false

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
    backup_configuration {
      enabled            = true
      binary_log_enabled = true
    }
  }
}

resource "google_sql_database_instance" "original-replica" {
  provider             = google-beta
  project              = "lych-dev-plg-1"
  name                 = "tf-test-2025-01-03-prim-1-rep-1"
  region               = "us-west2"
  database_version     = "MYSQL_8_0"
  instance_type        = "READ_REPLICA_INSTANCE"
  master_instance_name = google_sql_database_instance.original-primary.name
  deletion_protection  = false

-  replication_cluster {}

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
  }
}

However terrafrom plan shows following "1 to change"

Terraform will perform the following actions:

  # google_sql_database_instance.original-replica will be updated in-place
  ~ resource "google_sql_database_instance" "original-replica" {
        id                             = "tf-test-2025-01-03-prim-1-rep-1"
        name                           = "tf-test-2025-01-03-prim-1-rep-1"
        # (19 unchanged attributes hidden)

      - replication_cluster {
          - dr_replica               = true -> null
            # (1 unchanged attribute hidden)
        }

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

probably related warning message

2025-01-03T15:14:17.285-0800 [WARN]  Provider "registry.terraform.io/hashicorp/google-beta" produced an invalid plan for google_sql_database_instance.original-replica, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      # irrelevant details omitted...
      - .replica_configuration: block count in plan (1) disagrees with count in config (0)

From my understanding, fixing "original-replica" should be done by modifying main.tf, and running "terraform plan" and check that there's no change needed. (I'm following #12241 which is really similar work as I'm doing now)

@zli82016
Copy link
Member

zli82016 commented Jan 3, 2025

replication_cluster

Sorry to hear that. Do you have the logs for the test that the plan is not empty?

I'm still not familiar with terraform (this is first big change I'm making), the first (and probably the biggest) problem I'm having is following: (All manual tests are done after reflecting your suggestion)

  1. We have main.tf as follows (and corresponding resource in cloud SQL backend).
resource "google_sql_database_instance" "original-primary" {
  provider            = google-beta
  project             = "lych-dev-plg-1"
  name                = "tf-test-2025-01-03-prim-1"
  region              = "us-east1"
  database_version    = "MYSQL_8_0"
  instance_type       = "CLOUD_SQL_INSTANCE"
  deletion_protection = false

  replication_cluster {
    failover_dr_replica_name = "lych-dev-plg-1:tf-test-2025-01-03-prim-1-rep-1"
  }

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
    backup_configuration {
      enabled            = true
      binary_log_enabled = true
    }
  }
}

resource "google_sql_database_instance" "original-replica" {
  provider             = google-beta
  project              = "lych-dev-plg-1"
  name                 = "tf-test-2025-01-03-prim-1-rep-1"
  region               = "us-west2"
  database_version     = "MYSQL_8_0"
  instance_type        = "READ_REPLICA_INSTANCE"
  master_instance_name = google_sql_database_instance.original-primary.name
  deletion_protection  = false

  replication_cluster {}

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
  }
}
  1. We make following change to "original primary", and issue "terraform apply"
resource "google_sql_database_instance" "original-primary" {
  provider            = google-beta
  project             = "lych-dev-plg-1"
  name                = "tf-test-2025-01-03-prim-1"
  region              = "us-east1"
  database_version    = "MYSQL_8_0"
  instance_type       = "CLOUD_SQL_INSTANCE"
  deletion_protection = false

-  replication_cluster {
-    failover_dr_replica_name = "lych-dev-plg-1:tf-test-2025-01-03-prim-1-rep-1"
-  }

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
    backup_configuration {
      enabled            = true
      binary_log_enabled = true
    }
  }
}

resource "google_sql_database_instance" "original-replica" {
  provider             = google-beta
  project              = "lych-dev-plg-1"
  name                 = "tf-test-2025-01-03-prim-1-rep-1"
  region               = "us-west2"
  database_version     = "MYSQL_8_0"
  instance_type        = "READ_REPLICA_INSTANCE"
  master_instance_name = google_sql_database_instance.original-primary.name
  deletion_protection  = false

  replication_cluster {}

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
  }
}
  1. But update operation of "original-primary" removes "original-replica"'s replication_cluster (it is intended in cloud SQL), so I want to delete that in main.tf also.
resource "google_sql_database_instance" "original-primary" {
  provider            = google-beta
  project             = "lych-dev-plg-1"
  name                = "tf-test-2025-01-03-prim-1"
  region              = "us-east1"
  database_version    = "MYSQL_8_0"
  instance_type       = "CLOUD_SQL_INSTANCE"
  deletion_protection = false

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
    backup_configuration {
      enabled            = true
      binary_log_enabled = true
    }
  }
}

resource "google_sql_database_instance" "original-replica" {
  provider             = google-beta
  project              = "lych-dev-plg-1"
  name                 = "tf-test-2025-01-03-prim-1-rep-1"
  region               = "us-west2"
  database_version     = "MYSQL_8_0"
  instance_type        = "READ_REPLICA_INSTANCE"
  master_instance_name = google_sql_database_instance.original-primary.name
  deletion_protection  = false

-  replication_cluster {}

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
  }
}

However terrafrom plan shows following "1 to change"

Terraform will perform the following actions:

  # google_sql_database_instance.original-replica will be updated in-place
  ~ resource "google_sql_database_instance" "original-replica" {
        id                             = "tf-test-2025-01-03-prim-1-rep-1"
        name                           = "tf-test-2025-01-03-prim-1-rep-1"
        # (19 unchanged attributes hidden)

      - replication_cluster {
          - dr_replica               = true -> null
            # (1 unchanged attribute hidden)
        }

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

probably related warning message

2025-01-03T15:14:17.285-0800 [WARN]  Provider "registry.terraform.io/hashicorp/google-beta" produced an invalid plan for google_sql_database_instance.original-replica, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      # irrelevant details omitted...
      - .replica_configuration: block count in plan (1) disagrees with count in config (0)

From my understanding, fixing "original-replica" should be done by modifying main.tf, and running "terraform plan" and check that there's no change needed. (I'm following #12241 which is really similar work as I'm doing now)

Thank you for providing the information. It looks like update operation of original-primary doesn't remove original-replica's replication_cluster in step 3. The original-replica's replication_cluster has dr_replica = true. Do you have an idea what is going on here? I doubt if the API returns the value of dr_replica. Is it expected?

@lychgoogle
Copy link
Contributor Author

lychgoogle commented Jan 4, 2025

replication_cluster

Sorry to hear that. Do you have the logs for the test that the plan is not empty?

I'm still not familiar with terraform (this is first big change I'm making), the first (and probably the biggest) problem I'm having is following: (All manual tests are done after reflecting your suggestion)

  1. We have main.tf as follows (and corresponding resource in cloud SQL backend).
resource "google_sql_database_instance" "original-primary" {
  provider            = google-beta
  project             = "lych-dev-plg-1"
  name                = "tf-test-2025-01-03-prim-1"
  region              = "us-east1"
  database_version    = "MYSQL_8_0"
  instance_type       = "CLOUD_SQL_INSTANCE"
  deletion_protection = false

  replication_cluster {
    failover_dr_replica_name = "lych-dev-plg-1:tf-test-2025-01-03-prim-1-rep-1"
  }

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
    backup_configuration {
      enabled            = true
      binary_log_enabled = true
    }
  }
}

resource "google_sql_database_instance" "original-replica" {
  provider             = google-beta
  project              = "lych-dev-plg-1"
  name                 = "tf-test-2025-01-03-prim-1-rep-1"
  region               = "us-west2"
  database_version     = "MYSQL_8_0"
  instance_type        = "READ_REPLICA_INSTANCE"
  master_instance_name = google_sql_database_instance.original-primary.name
  deletion_protection  = false

  replication_cluster {}

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
  }
}
  1. We make following change to "original primary", and issue "terraform apply"
resource "google_sql_database_instance" "original-primary" {
  provider            = google-beta
  project             = "lych-dev-plg-1"
  name                = "tf-test-2025-01-03-prim-1"
  region              = "us-east1"
  database_version    = "MYSQL_8_0"
  instance_type       = "CLOUD_SQL_INSTANCE"
  deletion_protection = false

-  replication_cluster {
-    failover_dr_replica_name = "lych-dev-plg-1:tf-test-2025-01-03-prim-1-rep-1"
-  }

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
    backup_configuration {
      enabled            = true
      binary_log_enabled = true
    }
  }
}

resource "google_sql_database_instance" "original-replica" {
  provider             = google-beta
  project              = "lych-dev-plg-1"
  name                 = "tf-test-2025-01-03-prim-1-rep-1"
  region               = "us-west2"
  database_version     = "MYSQL_8_0"
  instance_type        = "READ_REPLICA_INSTANCE"
  master_instance_name = google_sql_database_instance.original-primary.name
  deletion_protection  = false

  replication_cluster {}

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
  }
}
  1. But update operation of "original-primary" removes "original-replica"'s replication_cluster (it is intended in cloud SQL), so I want to delete that in main.tf also.
resource "google_sql_database_instance" "original-primary" {
  provider            = google-beta
  project             = "lych-dev-plg-1"
  name                = "tf-test-2025-01-03-prim-1"
  region              = "us-east1"
  database_version    = "MYSQL_8_0"
  instance_type       = "CLOUD_SQL_INSTANCE"
  deletion_protection = false

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
    backup_configuration {
      enabled            = true
      binary_log_enabled = true
    }
  }
}

resource "google_sql_database_instance" "original-replica" {
  provider             = google-beta
  project              = "lych-dev-plg-1"
  name                 = "tf-test-2025-01-03-prim-1-rep-1"
  region               = "us-west2"
  database_version     = "MYSQL_8_0"
  instance_type        = "READ_REPLICA_INSTANCE"
  master_instance_name = google_sql_database_instance.original-primary.name
  deletion_protection  = false

-  replication_cluster {}

  settings {
    tier    = "db-perf-optimized-N-2"
    edition = "ENTERPRISE_PLUS"
  }
}

However terrafrom plan shows following "1 to change"

Terraform will perform the following actions:

  # google_sql_database_instance.original-replica will be updated in-place
  ~ resource "google_sql_database_instance" "original-replica" {
        id                             = "tf-test-2025-01-03-prim-1-rep-1"
        name                           = "tf-test-2025-01-03-prim-1-rep-1"
        # (19 unchanged attributes hidden)

      - replication_cluster {
          - dr_replica               = true -> null
            # (1 unchanged attribute hidden)
        }

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

probably related warning message

2025-01-03T15:14:17.285-0800 [WARN]  Provider "registry.terraform.io/hashicorp/google-beta" produced an invalid plan for google_sql_database_instance.original-replica, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      # irrelevant details omitted...
      - .replica_configuration: block count in plan (1) disagrees with count in config (0)

From my understanding, fixing "original-replica" should be done by modifying main.tf, and running "terraform plan" and check that there's no change needed. (I'm following #12241 which is really similar work as I'm doing now)

Thank you for providing the information. It looks like update operation of original-primary doesn't remove original-replica's replication_cluster in step 3. The original-replica's replication_cluster has dr_replica = true. Do you have an idea what is going on here? I doubt if the API returns the value of dr_replica. Is it expected?

(Again my answer is probably wrong about terraform)

for reference, I published unfinished commit that I'm working on bd4c2df

  1. To me the problem is that step 3 triggers the update operation, where it shouldn't at all. My expectation is that terraform plan results in No changes. Your infrastructure matches the configuration. .

  2. From purely backend logic, following happens at the time of execution of 2.

cloud SQL's update operation on "original-primary" changes both "original-primary" and "original-replica"

"original-primary".replication_cluster:

before:

replication_cluster {
  failover_dr_replica_name = "lych-dev-plg-1:tf-test-2025-01-03-prim-1-rep-1"
  dr_replica = false
}

after: non-present

"original-replica".replication_cluster:

before:

replication_cluster {
  dr_replica = true
}

after: non-present

So after step 2, change on main.tf should be a no-op, just reflecting change happened on step 2 (at least this is what I want)

  1. for the field dr_replica, this is always read-only value, which cloud SQL backend computes, and my implementation is also in that way. Following is the code I'm experimenting right now.
func flattenReplicationCluster(replicationCluster *sqladmin.ReplicationCluster, d *schema.ResourceData) []map[string]interface{} {
        data := make(map[string]interface{})
        if replicationCluster.FailoverDrReplicaName != "" {
                data["failover_dr_replica_name"] = replicationCluster.FailoverDrReplicaName
        }
        data["dr_replica"] = replicationCluster.DrReplica
        return []map[string]interface{}{data}
}

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 1242 insertions(+), 22 deletions(-))
google-beta provider: Diff ( 4 files changed, 1242 insertions(+), 22 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 106
Passed tests: 61
Skipped tests: 15
Affected tests: 30

Click here to see the affected service packages
  • sql

Action taken

Found 30 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccSQLDatabaseInstance_sqlMysqlDataCacheConfig
  • TestAccSQLDatabaseInstance_sqlPostgresDataCacheConfig
  • TestAccSqlDatabaseInstance_Edition_Downgrade
  • TestAccSqlDatabaseInstance_EnableGoogleDataplexIntegration
  • TestAccSqlDatabaseInstance_EnableGoogleMlIntegration
  • TestAccSqlDatabaseInstance_MysqlSwitchoverSuccess
  • TestAccSqlDatabaseInstance_Mysql_Edition_Upgrade
  • TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled
  • TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer
  • TestAccSqlDatabaseInstance_PostgresSwitchoverSuccess
  • TestAccSqlDatabaseInstance_Postgres_Edition_Upgrade
  • TestAccSqlDatabaseInstance_Smt
  • TestAccSqlDatabaseInstance_SqlServerTimezoneUpdate
  • TestAccSqlDatabaseInstance_SwitchoverSuccess
  • TestAccSqlDatabaseInstance_activationPolicy
  • TestAccSqlDatabaseInstance_authNets
  • TestAccSqlDatabaseInstance_basicMSSQL
  • TestAccSqlDatabaseInstance_basic_with_user_labels
  • TestAccSqlDatabaseInstance_deleteDefaultUserBeforeSubsequentApiCalls
  • TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable
  • TestAccSqlDatabaseInstance_settings_deletionProtection
  • TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled
  • TestAccSqlDatabaseInstance_settings_upgrade
  • TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled
  • TestAccSqlDatabaseInstance_updateSslOptionsForPostgreSQL
  • TestAccSqlDatabaseInstance_withPSCEnabled_thenAddAllowedConsumerProjects_thenRemoveAllowedConsumerProject
  • TestAccSqlDatabaseInstance_withPSCEnabled_thenAddPscAutoConnections_thenRemovePscAutoConnections
  • TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange
  • TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange
  • TestAccSqlDatabase_instanceWithActivationPolicy

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🟢 Tests passed during RECORDING mode:
TestAccSQLDatabaseInstance_sqlMysqlDataCacheConfig [Debug log]
TestAccSQLDatabaseInstance_sqlPostgresDataCacheConfig [Debug log]
TestAccSqlDatabaseInstance_Edition_Downgrade [Debug log]
TestAccSqlDatabaseInstance_EnableGoogleDataplexIntegration [Debug log]
TestAccSqlDatabaseInstance_EnableGoogleMlIntegration [Debug log]
TestAccSqlDatabaseInstance_MysqlSwitchoverSuccess [Debug log]
TestAccSqlDatabaseInstance_Mysql_Edition_Upgrade [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer [Debug log]
TestAccSqlDatabaseInstance_PostgresSwitchoverSuccess [Debug log]
TestAccSqlDatabaseInstance_Postgres_Edition_Upgrade [Debug log]
TestAccSqlDatabaseInstance_Smt [Debug log]
TestAccSqlDatabaseInstance_SqlServerTimezoneUpdate [Debug log]
TestAccSqlDatabaseInstance_SwitchoverSuccess [Debug log]
TestAccSqlDatabaseInstance_activationPolicy [Debug log]
TestAccSqlDatabaseInstance_authNets [Debug log]
TestAccSqlDatabaseInstance_basicMSSQL [Debug log]
TestAccSqlDatabaseInstance_basic_with_user_labels [Debug log]
TestAccSqlDatabaseInstance_deleteDefaultUserBeforeSubsequentApiCalls [Debug log]
TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable [Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtection [Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled [Debug log]
TestAccSqlDatabaseInstance_settings_upgrade [Debug log]
TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled [Debug log]
TestAccSqlDatabaseInstance_updateSslOptionsForPostgreSQL [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_thenAddAllowedConsumerProjects_thenRemoveAllowedConsumerProject [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_thenAddPscAutoConnections_thenRemovePscAutoConnections [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange [Debug log]
TestAccSqlDatabase_instanceWithActivationPolicy [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🟢 All tests passed!

View the build log or the debug log for each test

@lychgoogle lychgoogle force-pushed the switchover-2024-12-18 branch from bd4c2df to a0f4d0b Compare January 10, 2025 04:38
@github-actions github-actions bot requested a review from zli82016 January 10, 2025 04:39
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 1188 insertions(+), 22 deletions(-))
google-beta provider: Diff ( 4 files changed, 1188 insertions(+), 22 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 106
Passed tests: 61
Skipped tests: 15
Affected tests: 30

Click here to see the affected service packages
  • sql

Action taken

Found 30 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccSQLDatabaseInstance_sqlMysqlDataCacheConfig
  • TestAccSQLDatabaseInstance_sqlPostgresDataCacheConfig
  • TestAccSqlDatabaseInstance_Edition_Downgrade
  • TestAccSqlDatabaseInstance_EnableGoogleDataplexIntegration
  • TestAccSqlDatabaseInstance_EnableGoogleMlIntegration
  • TestAccSqlDatabaseInstance_MysqlSwitchoverSuccess
  • TestAccSqlDatabaseInstance_Mysql_Edition_Upgrade
  • TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled
  • TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer
  • TestAccSqlDatabaseInstance_PostgresSwitchoverSuccess
  • TestAccSqlDatabaseInstance_Postgres_Edition_Upgrade
  • TestAccSqlDatabaseInstance_Smt
  • TestAccSqlDatabaseInstance_SqlServerTimezoneUpdate
  • TestAccSqlDatabaseInstance_SwitchoverSuccess
  • TestAccSqlDatabaseInstance_activationPolicy
  • TestAccSqlDatabaseInstance_authNets
  • TestAccSqlDatabaseInstance_basicMSSQL
  • TestAccSqlDatabaseInstance_basic_with_user_labels
  • TestAccSqlDatabaseInstance_deleteDefaultUserBeforeSubsequentApiCalls
  • TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable
  • TestAccSqlDatabaseInstance_settings_deletionProtection
  • TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled
  • TestAccSqlDatabaseInstance_settings_upgrade
  • TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled
  • TestAccSqlDatabaseInstance_updateSslOptionsForPostgreSQL
  • TestAccSqlDatabaseInstance_withPSCEnabled_thenAddAllowedConsumerProjects_thenRemoveAllowedConsumerProject
  • TestAccSqlDatabaseInstance_withPSCEnabled_thenAddPscAutoConnections_thenRemovePscAutoConnections
  • TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange
  • TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange
  • TestAccSqlDatabase_instanceWithActivationPolicy

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🟢 Tests passed during RECORDING mode:
TestAccSQLDatabaseInstance_sqlMysqlDataCacheConfig [Debug log]
TestAccSQLDatabaseInstance_sqlPostgresDataCacheConfig [Debug log]
TestAccSqlDatabaseInstance_Edition_Downgrade [Debug log]
TestAccSqlDatabaseInstance_EnableGoogleDataplexIntegration [Debug log]
TestAccSqlDatabaseInstance_EnableGoogleMlIntegration [Debug log]
TestAccSqlDatabaseInstance_MysqlSwitchoverSuccess [Debug log]
TestAccSqlDatabaseInstance_Mysql_Edition_Upgrade [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer [Debug log]
TestAccSqlDatabaseInstance_PostgresSwitchoverSuccess [Debug log]
TestAccSqlDatabaseInstance_Postgres_Edition_Upgrade [Debug log]
TestAccSqlDatabaseInstance_Smt [Debug log]
TestAccSqlDatabaseInstance_SqlServerTimezoneUpdate [Debug log]
TestAccSqlDatabaseInstance_SwitchoverSuccess [Debug log]
TestAccSqlDatabaseInstance_activationPolicy [Debug log]
TestAccSqlDatabaseInstance_authNets [Debug log]
TestAccSqlDatabaseInstance_basicMSSQL [Debug log]
TestAccSqlDatabaseInstance_basic_with_user_labels [Debug log]
TestAccSqlDatabaseInstance_deleteDefaultUserBeforeSubsequentApiCalls [Debug log]
TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable [Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtection [Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled [Debug log]
TestAccSqlDatabaseInstance_settings_upgrade [Debug log]
TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled [Debug log]
TestAccSqlDatabaseInstance_updateSslOptionsForPostgreSQL [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_thenAddAllowedConsumerProjects_thenRemoveAllowedConsumerProject [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_thenAddPscAutoConnections_thenRemovePscAutoConnections [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange [Debug log]
TestAccSqlDatabase_instanceWithActivationPolicy [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🟢 All tests passed!

View the build log or the debug log for each test

@lychgoogle
Copy link
Contributor Author

Hello @zli82016 I applied all changes we discussed in offline. PTAL

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 1182 insertions(+), 22 deletions(-))
google-beta provider: Diff ( 4 files changed, 1182 insertions(+), 22 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 106
Passed tests: 57
Skipped tests: 15
Affected tests: 34

Click here to see the affected service packages
  • sql

Action taken

Found 34 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataSourceSqlDatabaseInstances_basic
  • TestAccDataSourceSqlDatabaseInstances_databaseVersionFilter
  • TestAccDataSourceSqlDatabaseInstances_regionFilter
  • TestAccDataSourceSqlDatabaseInstances_tierFilter
  • TestAccSQLDatabaseInstance_sqlMysqlDataCacheConfig
  • TestAccSQLDatabaseInstance_sqlPostgresDataCacheConfig
  • TestAccSqlDatabaseInstance_Edition_Downgrade
  • TestAccSqlDatabaseInstance_EnableGoogleDataplexIntegration
  • TestAccSqlDatabaseInstance_EnableGoogleMlIntegration
  • TestAccSqlDatabaseInstance_MysqlSwitchoverSuccess
  • TestAccSqlDatabaseInstance_Mysql_Edition_Upgrade
  • TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled
  • TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer
  • TestAccSqlDatabaseInstance_PostgresSwitchoverSuccess
  • TestAccSqlDatabaseInstance_Postgres_Edition_Upgrade
  • TestAccSqlDatabaseInstance_Smt
  • TestAccSqlDatabaseInstance_SqlServerTimezoneUpdate
  • TestAccSqlDatabaseInstance_SwitchoverSuccess
  • TestAccSqlDatabaseInstance_activationPolicy
  • TestAccSqlDatabaseInstance_authNets
  • TestAccSqlDatabaseInstance_basicMSSQL
  • TestAccSqlDatabaseInstance_basic_with_user_labels
  • TestAccSqlDatabaseInstance_deleteDefaultUserBeforeSubsequentApiCalls
  • TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable
  • TestAccSqlDatabaseInstance_settings_deletionProtection
  • TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled
  • TestAccSqlDatabaseInstance_settings_upgrade
  • TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled
  • TestAccSqlDatabaseInstance_updateSslOptionsForPostgreSQL
  • TestAccSqlDatabaseInstance_withPSCEnabled_thenAddAllowedConsumerProjects_thenRemoveAllowedConsumerProject
  • TestAccSqlDatabaseInstance_withPSCEnabled_thenAddPscAutoConnections_thenRemovePscAutoConnections
  • TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange
  • TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange
  • TestAccSqlDatabase_instanceWithActivationPolicy

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🟢 Tests passed during RECORDING mode:
TestAccSQLDatabaseInstance_sqlMysqlDataCacheConfig [Debug log]
TestAccSQLDatabaseInstance_sqlPostgresDataCacheConfig [Debug log]
TestAccSqlDatabaseInstance_Edition_Downgrade [Debug log]
TestAccSqlDatabaseInstance_EnableGoogleDataplexIntegration [Debug log]
TestAccSqlDatabaseInstance_EnableGoogleMlIntegration [Debug log]
TestAccSqlDatabaseInstance_MysqlSwitchoverSuccess [Debug log]
TestAccSqlDatabaseInstance_Mysql_Edition_Upgrade [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabledForSqlServer [Debug log]
TestAccSqlDatabaseInstance_PostgresSwitchoverSuccess [Debug log]
TestAccSqlDatabaseInstance_Postgres_Edition_Upgrade [Debug log]
TestAccSqlDatabaseInstance_Smt [Debug log]
TestAccSqlDatabaseInstance_SqlServerTimezoneUpdate [Debug log]
TestAccSqlDatabaseInstance_SwitchoverSuccess [Debug log]
TestAccSqlDatabaseInstance_activationPolicy [Debug log]
TestAccSqlDatabaseInstance_authNets [Debug log]
TestAccSqlDatabaseInstance_basicMSSQL [Debug log]
TestAccSqlDatabaseInstance_basic_with_user_labels [Debug log]
TestAccSqlDatabaseInstance_deleteDefaultUserBeforeSubsequentApiCalls [Debug log]
TestAccSqlDatabaseInstance_rootPasswordShouldBeUpdatable [Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtection [Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled [Debug log]
TestAccSqlDatabaseInstance_settings_upgrade [Debug log]
TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled [Debug log]
TestAccSqlDatabaseInstance_updateSslOptionsForPostgreSQL [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_thenAddAllowedConsumerProjects_thenRemoveAllowedConsumerProject [Debug log]
TestAccSqlDatabaseInstance_withPSCEnabled_thenAddPscAutoConnections_thenRemovePscAutoConnections [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange [Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange [Debug log]
TestAccSqlDatabase_instanceWithActivationPolicy [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🔴 Tests failed during RECORDING mode:
TestAccDataSourceSqlDatabaseInstances_basic [Error message] [Debug log]
TestAccDataSourceSqlDatabaseInstances_databaseVersionFilter [Error message] [Debug log]
TestAccDataSourceSqlDatabaseInstances_regionFilter [Error message] [Debug log]
TestAccDataSourceSqlDatabaseInstances_tierFilter [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@github-actions github-actions bot requested a review from zli82016 January 10, 2025 21:18
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 1174 insertions(+), 22 deletions(-))
google-beta provider: Diff ( 4 files changed, 1174 insertions(+), 22 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 106
Passed tests: 87
Skipped tests: 15
Affected tests: 4

Click here to see the affected service packages
  • sql

Action taken

Found 4 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataSourceSqlDatabaseInstances_basic
  • TestAccDataSourceSqlDatabaseInstances_databaseVersionFilter
  • TestAccDataSourceSqlDatabaseInstances_regionFilter
  • TestAccDataSourceSqlDatabaseInstances_tierFilter

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🔴 Tests failed during RECORDING mode:
TestAccDataSourceSqlDatabaseInstances_basic [Error message] [Debug log]
TestAccDataSourceSqlDatabaseInstances_databaseVersionFilter [Error message] [Debug log]
TestAccDataSourceSqlDatabaseInstances_regionFilter [Error message] [Debug log]
TestAccDataSourceSqlDatabaseInstances_tierFilter [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 5 files changed, 1191 insertions(+), 22 deletions(-))
google-beta provider: Diff ( 5 files changed, 1191 insertions(+), 22 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 106
Passed tests: 91
Skipped tests: 15
Affected tests: 0

Click here to see the affected service packages
  • sql

🟢 All tests passed!

View the build log

Copy link
Member

@zli82016 zli82016 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants