Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using dbt-meshify remove table level data_tests that were configured in YML #216

Open
1 of 5 tasks
marcilj opened this issue Aug 14, 2024 · 0 comments
Open
1 of 5 tasks
Labels
bug Something isn't working triage Tis issue or pull request must be triaged by a project maintainer

Comments

@marcilj
Copy link

marcilj commented Aug 14, 2024

Describe the bug

I runned dbt-meshify group data_engineers --owner-name [email protected] --exclude group:data_science and it removed data_tests that were configured before.

It only removed the data_tests at the table level. All the ones defined at the column level were kept intact.

Example :

version: 2

models:
  - name: my_model_name
    access: protected
    group: data_engineers

    columns:
      - name: date
        data_tests:
          - dbt_expectations.expect_column_values_to_be_of_type:
              column_type: date
          - dbt_expectations.expect_row_values_to_have_recent_data:
              datepart: day
              interval: 2
          - not_null

    data_tests:
      -my_dbt_change_check
      - dbt_expectations.expect_table_column_count_to_equal:
          value: 26

got converted to

version: 2

models:
  - name: my_model_name
    access: protected
    group: data_engineers

    columns:
      - name: date
        data_tests:
          - dbt_expectations.expect_column_values_to_be_of_type:
              column_type: date
          - dbt_expectations.expect_row_values_to_have_recent_data:
              datepart: day
              interval: 2
          - not_null

Steps to reproduce

Genera YML file with dbt-meshify with data_tests at the table level.

Expected results

The data_tests configuration should be kept as it was before.

Actual results

The data_tests at the table level are removed from the YML file.

Screenshots and log output

System information

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt debug:

12:54:39  Running with dbt=1.8.4
12:54:39  dbt version: 1.8.4
12:54:39  python version: 3.9.19
12:54:39  python path: 
12:54:39  os info: macOS-14.5-x86_64-i386-64bit
12:54:40  Using profiles dir at 
12:54:40  Using profiles.yml file at 
12:54:40  Using dbt_project.yml file at 
12:54:40  adapter type: snowflake
12:54:40  adapter version: 1.8.3
12:54:40  Configuration:
12:54:40    profiles.yml file [OK found and valid]
12:54:40    dbt_project.yml file [OK found and valid]
12:54:40  Required dependencies:
12:54:40   - git [OK found]

12:54:40  Registered adapter: snowflake=1.8.3
12:54:41    Connection test: [OK connection ok]

12:54:41  All checks passed!

The output of dbt --version:

Core:
  - installed: 1.8.4
  - latest:    1.8.5 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - snowflake: 1.8.3 - Up to date!

Additional context

Are you interested in contributing the fix?

@marcilj marcilj added bug Something isn't working triage Tis issue or pull request must be triaged by a project maintainer labels Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage Tis issue or pull request must be triaged by a project maintainer
Projects
None yet
Development

No branches or pull requests

1 participant