Skip to content

Commit

Permalink
Merge pull request #67 from dbt-labs/feature/add-env-project-params
Browse files Browse the repository at this point in the history
Add environment id and project id params
  • Loading branch information
b-per authored Jun 13, 2024
2 parents a9e1094 + e0e2995 commit ccf0e61
Show file tree
Hide file tree
Showing 20 changed files with 878 additions and 514 deletions.
36 changes: 36 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Run pytest on PRs

on:
pull_request:
branches:
- '**'

jobs:
test:
runs-on: ubuntu-latest

strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Install Poetry
run: |
curl -sSL https://install.python-poetry.org | python3 -
export PATH="$HOME/.local/bin:$PATH"
- name: Install dependencies
run: |
poetry install
- name: Run tests
run: |
poetry run pytest
22 changes: 0 additions & 22 deletions .github/workflows/github-actions-demo.yml

This file was deleted.

8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ With this package's approach, people don't need to learn another tool and can co
### Installation

- Create a Python virtual environment and activate it
- Run `pip install git+https://github.com/dbt-labs/dbt-jobs-as-code.git`
- Run `pip install git+https://github.com/dbt-labs/dbt-jobs-as-code.git` (or `pip install git+https://github.com/dbt-labs/[email protected]` to install a specific release)

The CLI is now available as `dbt-jobs-as-code`

Expand Down Expand Up @@ -57,6 +57,9 @@ Command: `dbt-jobs-as-code plan <config_file.yml>`
Returns the list of actions create/update/delete that are required to have dbt Cloud reflecting the configuration file

- this command doesn't modify the dbt Cloud jobs
- this command also accepts a list of project IDs or environments IDs to limit the command for: `dbt-jobs-as-code plan <config_file.yml> -p 1234 -p 2345 -e 4567 -e 5678`
- it is possible to limit for specific projects and/or specific environments
- when both projects and environments are provided, the command will run for the jobs that are both part of the environment ID(s) and the project ID(s) provided

#### `sync`

Expand All @@ -65,6 +68,8 @@ Command: `dbt-jobs-as-code sync <config_file.yml>`
Create/update/delete jobs and env vars overwrites in jobs to align dbt Cloud with the configuration file

- ⚠️ this command will modify your dbt Cloud jobs if the current configuration is different from the YAML file
- this command also accepts a list of project IDs or environments IDs to limit the command for: `dbt-jobs-as-code sync <config_file.yml> -p 1234 -p 2345 -e 4567 -e 5678`
- it is possible to limit for specific projects and/or specific environments

#### `import-jobs`

Expand All @@ -73,6 +78,7 @@ Command: `dbt-jobs-as-code import-jobs --config <config_file.yml>` or `dbt-jobs-
Queries dbt Cloud and provide the YAML definition for those jobs. It includes the env var overwrite at the job level if some have been defined

- it is possible to restrict the list of dbt Cloud Job IDs by adding `... -j 101 -j 123 -j 234`
- this command also accepts a list of project IDs or environments IDs to limit the command for: `dbt-jobs-as-code sync <config_file.yml> -p 1234 -p 2345 -e 4567 -e 5678`
- once the YAML has been retrieved, it is possible to copy/paste it in a local YAML file to create/update the local jobs definition.

To move some ui-jobs to jobs-as-code, perform the following steps:
Expand Down
17 changes: 3 additions & 14 deletions example_cicd/github_action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,22 +20,11 @@ jobs:
with:
python-version: "3.9.x"

- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-in-project: true

# Optional
# You can do a git clone every time or you can just download the repo in your project and commit it
# If you saved the files in your repo, you can comment the 2 lines below
- name: Clone the repo with dbt-jobs-as-code
run: git clone -b v0.2.0 --depth 1 https://github.com/dbt-labs/dbt-jobs-as-code.git

- name: Install dbt-jobs-as-code package
run: poetry install -C dbt-jobs-as-code --no-interaction -vvv
- name: Install dbt-jobs-as-code
run: pip install git+https://github.com/dbt-labs/[email protected]

- name: Run dbt-jobs-as-code
run: poetry run -C dbt-jobs-as-code dbt-jobs-as-code sync jobs/jobs.yml
run: dbt-jobs-as-code sync jobs/jobs.yml
env:
DBT_API_KEY: "${{secrets.DBT_API_KEY}}"
# DBT_BASE_URL is optional
Expand Down
Loading

0 comments on commit ccf0e61

Please sign in to comment.