Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[yugabyte] SQL Migrations #1138

Merged
merged 10 commits into from
Dec 13, 2024
Merged

Conversation

barroco
Copy link
Contributor

@barroco barroco commented Dec 2, 2024

This PR adds migration scripts for yugabyte and adapts the migration CLI accordingly.

Migration

Yugabyte does not support the exact same data types and declaration syntax as CockroachDB, therefore the following adaptations have been made:

Data Types (CockroachDB -> Yugabyte):

  1. INT64 -> BIGINT
  2. STRING -> TEXT

Indexes:

  1. Inverted indexes are implemented using ybgin

Syntax required changes:

  1. INDEX can't be declared in the CREATE TABLE block.
  2. Keyword STORING is replaced by INCLUDE.
  3. Index names must be unique.

Additional notes

  • It was agreed to squash the migration to reflect rid v4.0.0 and scd v3.2.0 of the crdb schema.

CI and Tests

  • A job in the CI has been added to cover changes dedicated to Yugabyte migration.
  • Migrations up and down have been successfully tested locally.

@barroco barroco marked this pull request as ready for review December 3, 2024 10:52
Copy link
Contributor

@mickmis mickmis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial review of everything except SQL migrations: LGTM, only minor comments for clarity / control flow.

Question about the SQL migrations: Since with yugabyte DBs we will be starting from scratch anyway, we may have an opportunity here to simplify the definitions by consolidating them into a single migration. Have you considered that?

cmds/db-manager/migration/migrate.go Show resolved Hide resolved
cmds/db-manager/migration/migrate.go Outdated Show resolved Hide resolved
cmds/db-manager/migration/migrate.go Outdated Show resolved Hide resolved
cmds/db-manager/migration/migrate.go Outdated Show resolved Hide resolved
@barroco
Copy link
Contributor Author

barroco commented Dec 5, 2024

@mickmis migrations have been flattened as per discussion during contributors sync.

@barroco barroco requested a review from mickmis December 5, 2024 10:36
Copy link
Contributor

@mickmis mickmis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full review, LGTM, just minor comments/questions to address before merging.

@BenjaminPelletier
Copy link
Member

Just to document my understanding of the reasoning behind "It was agreed to squash the migration to reflect rid v4.0.0 and scd v3.2.0 of the crdb schema.":

We have no intent to support the ability to users to switch between database backends at any time, and we further expect any potential migration of data between database backends to be a one-time occurrence. Therefore, the process of switching to YugabyteDB from an earlier schema version simply includes the step of migrating to rid v4.0.0 and scd v3.2.0 as the first part of the process. YugabyteDB schemas will be on an entirely separate track from CRDB schemas and therefore there is no need to synchronize schema versions. Instead, any particular DSS release will have a minimum and maximum CRDB schema version supported (for the duration that CRDB is still supported) and a separate minimum and maximum YugabyteDB schema version supported.

@barroco barroco merged commit 842680f into interuss:master Dec 13, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants