Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sample Issue Tests against YB Versions #2033

Merged
merged 12 commits into from
Dec 6, 2024
Merged

Conversation

makalaaneesh
Copy link
Collaborator

@makalaaneesh makalaaneesh commented Dec 4, 2024

Describe the changes in this pull request

Adding unit tests for issues.

Workflow that will help us detect when issues are fixed in YB:

  1. Whenever we introduce a new issue, write a unit test asserting that it fails.
  2. The unit test will be run against all the latest YB versions in every series that we support (2.18,2.20,2.23,2024.1,2024.2 for example)
  3. YB fixes an issue in 2025.1.0.0.
  4. As part of qualifying voyager for 2025.1 release, all unit tests are also run against 2025.1. Since the issue was fixed, "assert that it fails" will fail.
  5. This prompts us to a. Add MinimumVersionsFixedIn[2025.1] = 2025.1.0.0 to the issue. b. Write a unit test that asserts that parserDetector.GetAllIssues(..., 2025.1) does not return an issue of that type.
  6. The failing unit test will now start passing because it takes into account MinimumVersionsFixedIn[2025.1]. (this part is not implemented yet)

Describe if there are any user-facing changes

N/A

How was this pull request tested?

N/A. Adding unit tests.

Does your PR have changes that can cause upgrade issues?

Component Breaking changes?
MetaDB No
Name registry json No
Data File Descriptor Json No
Export Snapshot Status Json No
Import Data State No
Export Status Json No
Data .sql files of tables No
Export and import data queue No
Schema Dump No
AssessmentDB No
Sizing DB No
Migration Assessment Report Json No
Callhome Json No
YugabyteD Tables No
TargetDB Metadata Tables No

@makalaaneesh makalaaneesh marked this pull request as ready for review December 5, 2024 05:28
Copy link
Collaborator

@sanyamsinghal sanyamsinghal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two approaches to figure the supporting versions for an issue:

  1. For every issue check the support on all version till date.
  2. For every issue check the support on the latest version, if it is supported then we can look at some other versions(or last few series)

Is that correct?

Our code can have some stale issues also, which be fixed long time back or not very recently.

So how do we plan to ensure that for an issue if it works on latest, then what was the version yugabytedb started supporting it.

success = t.Run(fmt.Sprintf("%s-%s", "unlogged table", ybVersion), testUnloggedTableIssue)
assert.True(t, success)

yugabytedbContainer.Terminate(context.Background())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you already have defer terminate

@makalaaneesh
Copy link
Collaborator Author

Our code can have some stale issues also, which be fixed long time back or not very recently.
So how do we plan to ensure that for an issue if it works on latest, then what was the version yugabytedb started supporting it.

(Not sure I understand your question completely, we can discuss on call)
These unit tests will only help in detecting cases where issues are fixed going forward. For the ones that are already supported, we will have to do the manual work of figuring out which version was support added in . (Although I don't think there are a lot of cases)

Copy link
Contributor

@priyanshi-yb priyanshi-yb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


- name: Test YugabyteDB connection
run: |
psql "postgresql://yugabyte:@127.0.0.1:5433/yugabyte" -c "SELECT version();"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: psql $YB_CONN_STR -c "SELECT version();"

@makalaaneesh makalaaneesh merged commit f3b63ce into main Dec 6, 2024
33 checks passed
@makalaaneesh makalaaneesh deleted the aneesh/issue-tests branch December 6, 2024 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants