diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 980522ed7..fb0d68407 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,4 +1,4 @@ -# Guidance on how to contribute +# Guidance on How to Contribute > All contributions to this project will be released to the public domain. > By submitting a pull request or filing a bug, issue, or @@ -11,7 +11,7 @@ There are two primary ways to help: - Changing the code-base. -## Using the issue tracker +# Using the Issue Tracker Use the issue tracker to suggest feature requests, report bugs, and ask questions. This is also a great way to connect with the developers of the project as well @@ -22,10 +22,193 @@ the issue that you will take on that effort, then follow the _Changing the code- guidance below. -## Changing the code-base +# Changing the Code-Base -Generally speaking, code changes required creation of a personal fork of this repository, committing those changes to the fork, and then submitting a pull request. See the document on [Git usage](doc/GIT_USAGE.md) for more details. +[//]: # (TODO: re-write this but still reference the other Git doc) -New Python code should have associated unit tests whenever possible that validate implemented features and the presence or lack of defects. +[//]: # (TODO: update any links to the other Git doc that should point here now) + +[//]: # (TODO: add comment in other Git doc about how that is more technical) + +[//]: # (TODO: add TOC) + +* [Summary](#summary) +* [Getting Started](#getting-started) +* [Developing Changes](#developing-changes) +* [Keeping Forks Up to Date](#keeping-forks-up-to-date) + +Note that this document only discusses aspects of Git usage that should be needed for day-to-day development and contributions. For a more detailed overview of DMOD's use of Git, see the [GIT_USAGE](doc/GIT_USAGE.md) doc. + +## Summary + +To work with the repo and contribute changes, the basic process is as follows: + +- Create your own DMOD fork in Github +- Clone DMOD locally (conventionally from your fork) and [setup your repo on your local development machine](#getting-started) +- Make sure to [keep your fork and your local clone(s) up to date](#keeping-forks-up-to-date) with the upstream OWP DMOD repo, ensuring histories remain consistent by performing [rebasing](#rebasing-development-branches) +- Create feature/fix branches from `master` when you want to contribute +- Write changes you want to contribute, commit to your local feature/fix branch, and push these commits to a branch in your personal Github fork +- Submit pull requests to the OWP DMOD repo's `master` from a branch in your fork when this branch has a collection of changes ready to be incorporated + +## Getting Started + +In order to be able to contribute code changes, you will first need to [create a Github fork](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo) of the official OWP repo. + +Next, set up your authentication mechanism with Github for your command line (or IDE). You can either [create an SSH key pair](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) and [add the public key](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account) to your Github account, or you can set up a [Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#using-a-personal-access-token-on-the-command-line) if you plan to clone the repo locally via HTTPS. + +After that, [clone a local development repo](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository) from your fork, using a command similar to one of the following: + + # SSH-based clone command. Change URL to match your fork as appropriate + git clone git@github.com:your_user/DMOD.git dmod + + # HTTPS-based clone command. Change URL to match your fork as appropriate + git clone https://github.com/your_user/DMOD.git dmod + +You can now change directories into the local repo, which will have the default branch - `master` - checked out. + + # Move into the repo directory "dmod" + cd dmod + + # You can verify the branch by examining the output of ... + git status + +> [!IMPORTANT] +> Git's will add a [Git remote](https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes) named `origin` to the clone's configuration that points to the cloned-from repo. Because of this, the recommended convention is to clone your local repo(s) from your personal fork, thus making `origin` point to your fork. This is assumed to be the case in other parts of the documentation. + +Next, add the upstream OWP DMOD repo as a second remote for the local clone. The standard convention used in this doc and elsewhere is to name that remote `upstream`. Doing the addition will look something like: + + # Add the remote + git remote add upstream https://github.com/NOAA-OWP/DMOD.git + + # Verify + git remote -v + +Set up the user and email in the local repo's configuration. + + cd dmod + git config user.name "John Doe" + git config user.email "john@doe.org" + +Alternatively, one could also set these in the machine's global Git config (or rely upon the global settings if already configured). + + git config --global user.name "John Doe" + git config --global user.email "john@doe.org" + +### Optional: Git Hooks + +While not discussed in depth here, Git hooks are a useful feature for helping maintain code quality in your local development repo. More details on those and DMOD's use [can be found in this document](doc/GIT_USAGE.md#optional-setting-up-hook-scripts). [//]: # (TODO: add section/document on code style) + + +## Developing Changes + +* [Work in a Dedicated Branch](#work-in-a-dedicated-branch) +* [Pushing Incremental Commits](#pushing-incremental-commits) +* [Submitting Pull Requests](#submitting-pull-requests) + +### Work in a Dedicated Branch + +When you want to contribute a fix or new feature, start by creating and checking out a local branch (e.g., `new_branch`) to contain your work. This should be based on `master` (you may need to [sync upstream changes](#getting-upstream-changes) first): + + # Create the new branch "new_branch" based on "master" + git branch new_branch master + + # Check out "new_branch" locally to work in it + git checkout new_branch + +Also, go ahead and push the branch itself to your fork: + + # Assuming the convention of your fork's remote being `origin` + git push -u origin new_branch + +From there begin writing and committing your changes to the branch. While up to you, it is suggested that development work be committed frequently when changes are complete and meaningful. If work requires modifying more than one file in the source, it is recommended to commit the changes independently to help avoid too large of conflicts if/when they occur. + +### Pushing Incremental Commits + +Especially if making more frequent, smaller commits as suggested above, it is a good practice to regularly push these smaller commits to your fork. If the `-u` option was used when initially pushing the branch, it is simple to check if there are local, unpushed commits. + + # The fetch is probably unnecesssary unless you work from multiple local repos + git fetch + + # Assuming your branch of interest is still checked out: + git status + + # And if there are some newer, local changes that haven't been push yet: + git push + +### Submitting Pull Requests + +Once a code contribution is finished, make sure all changes have been pushed to the branch in your fork. Then you can navigate to the OWP repo via Github's web interface and [submit a PR](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) to pull this branch into the OWP `master` branch. Verify that `master` is selected as the recipient branch. You will also need to make sure you are comparing across forks, choosing your appropriate fork and branch to pull from. Complete details on the process can be found in Github's documentation. + +#### PR Review and Requested Revisions + +Once the PR is submitted, it will be reviewed by one or more other repo contributors. Often conversations will be had within the Github PR if reviewers have questions or request revisions be made to the proposed changes. If revisions are requested, you will need to make those in your locally copy of the feature/fix branch, and then re-push that branch (and the updates) to your personal fork. Then, use the PR page in Github to re-request review. + +## Keeping Forks Up to Date + +- [A Rebase Strategy](#a-rebase-strategy) +- [Getting Upstream Changes](#getting-upstream-changes) +- [Rebasing Development Branches](#rebasing-development-branches) +- [Fixing Diverging Development Branches](#fixing-diverging-development-branches) + +### A Rebase Strategy + +The development team for DMOD uses a *rebase* strategy for integrating code changes, rather than a *merge* strategy. More information on rebasing is [available here](https://git-scm.com/book/en/v2/Git-Branching-Rebasing), but the main takeaway is that it is important all changes are integrated into branches using this approach. Deviation from this will likely cause a bit of mess with branch commit histories, which could force rejection of otherwise-good PRs. + +### Getting Upstream Changes + +When it is time to check for or apply updates from the official OWP repo to a personal fork and/or a local repo, check out the `master` branch locally and do fetch-and-rebase, which can be done with `pull` and the `--rebase` option: + + # Checkout local master branch + git checkout master + + # Fetch and rebase changes + git pull --rebase upstream master + +Then, make sure these get pushed to your personal fork. Assuming [the above-described setup](#getting-started-with-your-fork) where the local repo was cloned from the fork, and assuming the local `master` branch is currently checked out, the command for that is just: + + # Note the assumptions mentioned above that are required for this syntax + git push + +Alternatively, you can use the more explicit form: + +`git push :` + +The previous example command is effectively equivalent to running: + + # Cloning a repo from a fork created a remote for the fork named "origin"; see above assumptions + git push origin master:master + +You also can omit `:` (including the colon) and supply just the remote branch name if the appropriate local branch is still checked out. + +#### For `production` Too + +Note that the above steps to get upstream changes from the official OWP repo can be applied to the `production` branch also (just swap `production` in place of `master`). `production` should not be used as the basis for feature/fix branches, but there are other reasons why one might want the latest `production` locally or in a personal fork. + +### Rebasing Development Branches + +When the steps in [Getting Upstream Changes](#getting-upstream-changes) do bring in new commits that update `master`, it is usually a good idea (and often necessary) to rebase any local feature/fix branches were previously created. E.g., + + # If using a development branch named 'faster_dataset_writes' + git checkout faster_dataset_writes + git rebase master + +See documentation on [the "git rebase" command](https://git-scm.com/docs/git-rebase) for more details. + +#### Interactive Rebasing + +It is possible to have more control over rebasing by doing an interactive rebase. E.g.: + + git rebase -i master + +This will open up a text editor allowing for reordering, squashing, dropping, etc., development branch commits prior to rebasing them onto the new base commit from `master`. See the [**Interactive Mode**](https://git-scm.com/docs/git-rebase#_interactive_mode) section on the rebase command for more details. + +### Fixing Diverging Development Branches + +If a local feature/fix branch is already pushed to a remote fork, and then later rebasing the local branch is necessary, doing so will cause the histories to diverge. For simple cases, the fix is to just force-push the rebased local branch. + + # To force-push to fix a divergent branch + git push -f origin feature_branch + +However, extra care is needed if multiple developers may be using the branch in the fork (e.g., a developer is collaborating with someone else on a large set of changes for some new feature). The particular considerations and best ways to go about things in such cases are outside the scope of this document. Consult Git's documentation and Google, or contact another contributor for advice. diff --git a/doc/GIT_USAGE.md b/doc/GIT_USAGE.md index 11f52fb6f..87f671d5c 100644 --- a/doc/GIT_USAGE.md +++ b/doc/GIT_USAGE.md @@ -1,134 +1,171 @@ # Git Strategy -- [Branching Design](#branching-design) -- [Contributing](#contributing) - - [TL;DR](#contributing-tldr) - - [Getting Started With Your Fork](#getting-started-with-your-fork) - - [Fork Consistency Requirements](#fork-consistency-requirements) - - [Fork Setup Suggestions](#fork-setup-suggestions) -- [Keeping Forks Up to Date](#keeping-forks-up-to-date) - - [Getting Upstream Changes](#getting-upstream-changes) - - [Rebasing Development Branches](#rebasing-development-branches) - - [Fixing Diverging Development Branches](#fixing-diverging-development-branches) - -## Branching Design +Note that this document goes into detail on the Git strategy and branching model for DMOD. It is here for openness and transparency, but most contributors and users will not need to be concerned with this level of detail. For information geared toward day-to-day development contributions and Git, see the [CONTRIBUTING](../CONTRIBUTING.md) doc. -- The main DMOD repo has a primary long-term branch called **master** - - There may be other branches for specific purposes, but these should still derive from `master` -- Interaction with DMOD repo is done via pull requests (PRs) to this `master` branch +- [Branching Model](#branching-model) + - [Feature Branches from `master`](#feature-branches-from-master) + - [Relating `production`, `master`, and Release Branches](#relating-production-master-and-release-branches) +- [Contributing](#contributing) +- [Optional: Setting Up Hook Scripts](#optional-setting-up-hook-scripts) + +## Branching Model + +- The DMOD repo uses a branching model based on [Gitflow](https://nvie.com/posts/a-successful-git-branching-model/) that has two primary long-term branches: + - **master**: the main development and integration branch containing the latest completed development work intended for the next released version + - **production**: the branch representing the latest code verified as production-ready and pointing to the most recently release, official version +- Rebasing is used to integrate changes across branches, rather than merge commits + - This allows the repo to maintain a more robust and complete history +- Most interaction with the official OWP DMOD repo is done via pull requests (PRs) to the `master` branch + - Independent branches for features or bug fixes are created off `master` to contain development work that is in progress + - Once work in a feature/fix branch is complete (or at least thought complete), it is used to create a PR + - PRs and their linked branches are reviewed and, once approved, have their changes integrated back into `master` + - Typically feature/fix branches exist in personal clones and personal Github forks, but not in the official OWP repo +- Release branches (e.g., `release-X` for pending version `X`) will be created whenever it is time to officially release a new version + - These effectively are release candidates, with branches created from `master` + - The release branches are managed by the core OWP contributors team + - They do exist in the official OWP repo + - But they are short-lived and removed once the release becomes official + - See the [Release Management](RELEASE_MANAGEMENT.md) doc for more details on the release process + +### Feature Branches from `master` +This illustrates the relationship between feature branches and `master`. They should be created from `master` and independently contain commits from their feature. Once done, the changes will be reintegrated back into `master` via rebasing. + +```mermaid + %%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': { 'showBranches': true, 'showCommitLabel':true, 'mainBranchName': 'master'}}}%% + gitGraph + commit id:"feature1.1" + commit id:"feature1.2" + branch feature-2 + branch feature-3 + checkout feature-2 + commit id:"feature2.1" + commit id:"feature2.2" + checkout master + merge feature-2 + checkout feature-3 + commit id:"feature3.1" + commit id:"feature3.2" + commit id:"feature3.3" + checkout master + merge feature-3 +``` + +The resulting state of `master` after rebasing the two new feature branches would be: + +```mermaid + %%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': { 'showBranches': true, 'showCommitLabel':true, 'mainBranchName': 'master'}}}%% + gitGraph + commit id:"feature1.1" + commit id:"feature1.2" + commit id:"feature2.1" + commit id:"feature2.2" + commit id:"feature3.1" + commit id:"feature3.2" + commit id:"feature3.3" +``` + +### Relating `production`, `master`, and Release Branches + +This illustrates the relationship between `production`, `master`, and `release-v2`. Notice that `production` has already been tagged with version `v1` at the start. Commits for `feature1` and `feature2` at some point are integrated into `master`. When it is time to prepare to release version `v2`, `release-v2` is created. A few bug fix commits were needed in `release-v2`. After that, all the changes in `release-v2` are integrated into `production`, and `production` is tagged `v2`. All the changes are also integrated back into `master`. + + +```mermaid + %%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': { 'showBranches': true, 'showCommitLabel':true, 'mainBranchName': 'master'}}}%% + gitGraph + commit id:"v1-commit" + branch production + checkout production + commit id:"v1-commit" tag: "v1" + checkout master + commit id:"feature1.1" + commit id:"feature1.2" + commit id:"feature2.1" + commit id:"feature2.2" + commit id:"feature2.3" + branch release-v2 + checkout release-v2 + commit id:"fix2.1" + commit id:"fix2.2" + checkout production + merge release-v2 tag:"v2" + checkout master + merge release-v2 + +``` + +The resulting state of `production` is: + +```mermaid + %%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': { 'showBranches': true, 'showCommitLabel':true, 'mainBranchName': 'production'}}}%% + gitGraph + commit id:"v1-commit" tag:"v1" + commit id:"feature1.1" + commit id:"feature1.2" + commit id:"feature2.1" + commit id:"feature2.2" + commit id:"feature2.3" + commit id:"fix2.1" + commit id:"fix2.2" tag:"v2" +``` + +The resulting state of `master` is essentially the same: + +```mermaid + %%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': { 'showBranches': true, 'showCommitLabel':true, 'mainBranchName': 'master'}}}%% + gitGraph + commit id:"v1-commit" + commit id:"feature1.1" + commit id:"feature1.2" + commit id:"feature2.1" + commit id:"feature2.2" + commit id:"feature2.3" + commit id:"fix2.1" + commit id:"fix2.2" +``` ## Contributing -- [TL;DR](#contributing-tldr) -- [Getting Started With Your Fork](#getting-started-with-your-fork) -- [Fork Consistency Requirements](#fork-consistency-requirements) -- [Fork Setup Suggestions](#fork-setup-suggestions) - -### Contributing TL;DR - -To work with the repo and contribute changes, the basic process is as follows: - -- Create your own DMOD fork in Github -- Clone your fork and [setup your repo on your local development machine](#getting-started-with-your-fork) -- Make sure to [keep your fork and your local clone(s) up to date](#keeping-forks-up-to-date) with the upstream DMOD repo, [ensuring histories remain consistent](#fork-consistency-requirements) by performing [rebasing](#rebasing-development-branches) -- Figure out a branching strategy that works for you, with [this strategy offered as a suggestion](#fork-setup-suggestions) -- Make changes you want to contribute, commit them locally, and push them to your Github fork -- Submit pull requests to the upstream repo's `master` from a branch in your fork when you have a collection of changes ready to be incorporated - -### Getting Started With Your Fork - -After creating a fork in Github, clone a local development repo from the fork. This should make the fork a remote for that local repo, typically named **origin**. - -Add the main DMOD repo as a second remote for the local clone. The standard convention, used here and elsewhere, is to name that remote `upstream`. Doing the addition will look something like: - - # Add the remote - git remote add upstream https://github.com/NOAA-OWP/DMOD.git - - # Verify - git remote -v - -Set up a local user and email in the local repo's configuration. - - cd local_repo_directory - git config user.name "John Doe" - git config user.email "john@doe.org" - -Alternatively, one could also set these in the machine's global Git config (or rely upon the global settings if already configured). - - git config --global user.name "John Doe" - git config --global user.email "john@doe.org" - -### Fork Consistency Requirements - -Within a local repo and personal fork, users are mostly free to do whatever branching strategy works for them. However, a branch used for a pull request typically should be (re)based on the `HEAD` commit of the current OWP `upstream/master` branch, to ensure the repo history remains consistent. - -### Fork Setup Suggestions - -Note that while this setup is not strictly required, examples and instructions in this document may assume its use. - -Maintain a personal `master` branch, on any local development clones and within a personal fork, just as [a place to rebase changes from `upstream/master`](#getting-upstream-changes). Do not do any development work or add any commits to these directly. Just keep these as a "clean," current copy of the `upstream/master` branch. - -Use separate feature branches for development work as appropriate. When preparing to make a PR, making sure the branch is both up to date with `upstream/master` and has all the desired local changes. - -A separate, "clean" local `master` should be easy to keep it in sync with `upstream/master`, which in turn will make it relatively easy to rebase local development branches on `master` whenever needed. This simplifies maintaining the base-commit consistency requirement for the branches used for pull requests. - -Clean up above mentioned PR branches regularly (i.e., once their changes get incorporated). This is generally a good practice for other local or fork development branches, to avoid having [diverged histories that need to be fixed](#fixing-diverging-development-branches). - -## Keeping Forks Up to Date - -- [A Rebase Strategy](#a-rebase-strategy) -- [Getting Upstream Changes](#getting-upstream-changes) -- [Rebasing Development Branches](#rebasing-development-branches) -- [Fixing Diverging Development Branches](#fixing-diverging-development-branches) - - -To remain consistent with changes to the official OWP DMOD repo (i.e., the **upstream** repo), a developer will need to regularly synchronize with it. This requires having an `upstream` remote configured, as described in the section on [getting started with your fork and local repo](#getting-started-with-your-fork). - -### A Rebase Strategy - -The development team for DMOD uses a *rebase* strategy for integrating code changes, rather than a *merge* strategy. More information on rebasing is [available here](https://git-scm.com/book/en/v2/Git-Branching-Rebasing), but the main takeaway is that it is important all changes are integrated into branches using this approach. Deviation from this will likely cause a bit of mess with branch commit histories, which could force rejection of otherwise-good PRs. - - -### Getting Upstream Changes - -When it is time to check for or apply updates to a personal fork and/or a local repo, check out the local `master` branch and do fetch-and-rebase, which can be done with `pull` and the `--rebase` option: +More details on the practical processes and requirements for contributing code changes can be found in the [CONTRIBUTING](../CONTRIBUTING.md) doc. In summary: - # Checkout local master branch - git checkout master +- Github Pull Requests (PRs) are required to incorporate changes into the official OWP DMOD repo + - Contributors should generally not be pushing changes directly to branches in the OWP repo +- PRs should be submitted using a feature/fix branch contained in a personal Github fork +- Rebasing is used, rather than merge commits, to integrate changes across branches and keep branches from different repos in sync +- PRs should be configured to pull changes into the `master` branch +- Feature/fix branches should be created from `master` +- Personal forks and local clone(s) should be kept up to date with the upstream OWP DMOD repo regularly to minimize the introduction of merge conflict in PRs - # Fetch and rebase changes - git pull --rebase upstream master -Then, make sure these get pushed to the personal fork. Assuming a typical setup where a developer has cloned from a fork, and still has `master` checked out, that is just: +## Optional: Setting Up Hook Scripts - # Note the assumptions mentioned above that are required for this syntax - git push +_Git_ supports the capability to automatically run various scripts when certain events happen. These are referred to as [_Git_ hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks). Use of these hooks is optional but recommended. However, client-side hooks are not copied as part of cloning a repository. They must be set up either manually or, [as discussed next](#integration-with-_pre-commit_-utility), via some other helper tool. -Depending on individual setup, a developer may want to do this immediately (e.g., if the `master` branch is "clean", as [discussed in the forking suggestions](#fork-setup-suggestions)), or wait until the local `master` is in a state ready to push to the personal fork. +### Integration with _pre-commit_ Utility -### Rebasing Development Branches +The DMOD repo does contain a configuration for a related helper utility named [_pre-commit_](https://pre-commit.com/). This tool can be used to manage and install scripts for use with _Git_'s `pre-commit` hook. The `pre-commit` hook runs automatically when `git commit` is invoked and can be used to perform valuable checks on the files in the repo. -When the steps in [Getting Upstream Changes](#getting-upstream-changes) do bring in new commits that update `master`, rebase any local development branches were previously created. E.g., +#### Installing _pre-commit_ - # If using a development branch named 'dev' - git checkout dev - git rebase master +The _pre-commit_ utility must be installed as a separate step. Installation can be performed using a package manager (e.g. `brew install pre-commit`) or from _pip_. If you are to use _pip_, it is highly recommend to use a virtual environment. -See documentation on [the "git rebase" command](https://git-scm.com/docs/git-rebase) for more details. +#### Provided Configuration -#### Interactive Rebasing +The [`.pre-commit-config.yaml`](./.pre-commit-config.yaml) file in the root of the repo provides a DMOD-maintained configuration of hook scripts that _pre-commit_ (the utility) will add to the `pre-commit` _Git_ hook. This file is version controlled and can be changed like any other source file. -It is possible to have more control over rebasing by doing an interactive rebase. E.g.: +#### Installing Hook Scripts Using _pre-commit_ - git rebase -i master +Once the utility is available, install the _pre-commit_-configured hook scripts into your _Git_ clone by running: -This will open up a text editor allowing for reordering, squashing, dropping, etc., development branch commits prior to rebasing them onto the new base commit from `master`. See the [**Interactive Mode**](https://git-scm.com/docs/git-rebase#_interactive_mode) section on the rebase command for more details. +```shell +pre-commit install +``` -### Fixing Diverging Development Branches +The hook scripts will now run when code is committed. -If a local development branch is already pushed to a remote fork, and then later rebasing the local branch is necessary, doing so will cause the histories to diverge. For simple cases, the fix is to just force-push the rebased local branch. +Alternatively, you can run the hook scripts manually via: - # To force-push to fix a divergent branch - git push -f origin dev +```shell +pre-commit run --all-files +``` -However, extra care is needed if multiple developers may be using the branch in the fork (e.g., a developer is collaborating with someone else on a large set of changes for some new feature). The particular considerations and best ways to go about things in such cases are outside the scope of this document. Consult Git's documentation and Google, or contact another contributor for advice. +For more information, see [_pre-commit_'s documentation](https://pre-commit.com/index.html). \ No newline at end of file diff --git a/doc/RELEASE_MANAGEMENT.md b/doc/RELEASE_MANAGEMENT.md new file mode 100644 index 000000000..8802f4575 --- /dev/null +++ b/doc/RELEASE_MANAGEMENT.md @@ -0,0 +1,63 @@ +# Release Management + +The page discusses the release process for official versions of DMOD. This process is very much interrelated to the repo branching management model, as discussed in detail on the [GIT_USAGE](./GIT_USAGE.md) doc. + +# The Release Process + +## TL;DR + +The release process for DMOD can be summarized fairly simply: +- A version name is finalized +- A release branch is created +- Testing, QA, fixes are done on the release branch +- Once ready the release is tagged and changes are pulled into `production` and `master` + +## Process Steps + + +[comment]: <> (TODO: Document release manual testing and QA procedures) + +1. The next DMOD version name/number is [decided/finalized](#rules-for-version-numbers) +2. A release branch, based on `master`, is created in the official OWP repo + - The name of this branch will be `release-X` for version `X` +3. All necessary testing and quality pre-release tasks are performed using this release branch + - **TODO**: to be documented in more detail +4. (If necessary) Bug fixes, documentation updates, and other acceptable, non-feature changes are applied to the release branch + - Such changes should go through some peer review process before inclusion in the official OWP branch (e.g., PRs, out-of-band code reviews, etc.) + - **TODO**: process to be decided upon and documented +5. Steps 3. and 4. are repeated as needed until testing, quality checks, etc. in Step 3. do not require another iteration of Step 4. + - At this point, the branch is ready for official release +6. All changes in the release branch are incorporated into `production` in the official OWP repo + - Note that **rebasing** should be used to reconcile changes ([see here](../CONTRIBUTING.md#a-rebase-strategy) for more info) +7. The subsequent `HEAD` commit of `production` is tagged with the new version in the official OWP repo +8. All changes in the release branch are incorporated back into `master` in the official OWP repo + - This will include things like bug fixes committed to `release-X` after it was branched from `master` + - As with `production` in Step 6., this should be [done using rebasing](../CONTRIBUTING.md#a-rebase-strategy) +9. The release branch is deleted from the OWP repo (and, ideally, other clones and forks) +10. (If necessary) Any additional tags are applied as needed to the `HEAD` commit of `production` in the official OWP repo + - At this time none are currently needed, but there are plans to consider these in the future for things like specific package versions of contained packages + +# Versions + +The versioning for DMOD is a little complicated. + +DMOD contains the sources of several independently-versioned Python packages; e.g., `dmod-core-0.19.0`. As long as this code remains organized as multiple separate packages, the package versions need to continue to be maintained individually. + +DMOD contains other source code wholly separate from these package, such as helper scripts, Dockerfiles, stack configurations, and other files. These are not contained in some inner organizational unit with its own versioning, and many (if not all) of them are particularly relevant to DMOD deployment. + +As such, DMOD utilizes another, independent versioning scheme for itself as a whole. + +## Rules for Version Numbers + +Version numbers for DMOD should follow a system akin to [Semantic Versioning](https://semver.org/), using the typical `MAJOR.MINOR.PATCH` pattern. Because of DMOD's design and purpose, the rules for incrementing components need to be applied at slightly higher levels: + +1. `MAJOR` increments if incompatible changes are made with respect to job execution, data orchestration, infrastructure capabilities, or the deployment process; examples include (but are not limited to) + * any executable job types are removed + * previously satisfactory data formats to fulfill a job requirement become no longer satisfactory + * a special process is needed to upgrade an existing deployment that goes significantly beyond repeating steps performed when setting up a new deployment +2. `MINOR` increments if significant, backwards compatible changes are made with respect to job execution, data orchestration, infrastructure capabilities, or the deployment process + * a new executable job type is added + * an additional dataset format is added + * a new DMOD service is added + * new iterations of service or worker runtime entities (e.g., Docker images) must be generated, but using the same processes and tools used previously +3. `PATCH` increments if only smaller or less noticeable changes than those above are made diff --git a/gitflow.md b/gitflow.md deleted file mode 100644 index 24502a1db..000000000 --- a/gitflow.md +++ /dev/null @@ -1,133 +0,0 @@ -# MaaS Git-Flow - -## Getting Started - -Begin by making a local copy of the repository -`git clone ` - -Don't forget to ensure identity is set properly, by default, git doesn't configure e-mail address well! You can set your machines global policy using - - git config --global user.name "John Doe" - git config --global user.email "john@doe.org" - -If you only want to make name and e-mail changes for THIS newly cloned repository then use - - cd repo - git config user.name "John Doe" - git config user.email "john@doe.org" - -Local development is left to developer preference, but all development should sync with the upstream development branch frequently. This guide shows the process using local development “feature” branches, but ultimately, changes should end up on development, synced upstream, and then pushed to the development branch upstream with as much commit history as is meaningful (minimal merge commits). - -## Synchronizing - -To sync a single upstream branch and pull all the upstream changes into your working tree: - -`git fetch ; git checkout ; git rebase origin/;` -OR -`git checkout ; git pull --rebase` -OR -`git pull origin --rebase` - -## Development -To get started with some feature development, create a local branch and check it out - - git checkout develop - git branch featureA - git checkout featureA -OR -`git checkout -b featureA develop` - -Development work should commit frequently when changes are complete and meaningful. If work requires modifying more than one file in the source, it is recommended to commit the changes independently to help avoid too large of conflicts if the occur. - -### Setting up `pre-commit` hooks - -[`pre-commit`](https://pre-commit.com/) is used to manage and install `git` pre-commit hooks. -pre-commit hooks run automatically when `git commit` is invoked and perform checks on the code you are committing. -Example checks are removing trailing white spaces, verifying that large files are not included, or running a code linting tool. -Hooks are configured in the [`.pre-commit-config.yaml`](./.pre-commit-config.yaml) file in the root of the repo. - -`pre-commit` can be installed using a package manager (e.g. `brew install pre-commit`) or from `pip`. -If you are to use `pip`, it is highly recommend to use a virtual environment. - -Install pre-commit hooks into your `git` clone by running: - -```shell -pre-commit install -``` - -Hooks will now run when code is committed. -Alternatively, you can run the pre-commit hooks manually via: - -```shell -pre-commit run --all-files -``` - -For more information, see [`pre-commit`'s documentation](https://pre-commit.com/index.html). - -### Commits - -When changes are made to a file and ready for committing, then use these git commands - - git add - git commit -m “meaningful commit message here” - -If multiple files need to be associated with a single commit (this should be rare) then you can add multiple files with -`git add ` - -Use `git status` to see what files are modified but not committed. - -### Rebasing - -When work is ready to be shared upstream, it should be synced with the upstream development branch. This should be done frequently, even if the work isn’t complete. This will help minimize potential conflicts. To prepare for moving work upstream, first make sure the local repository is synced (see above instructions for [Synchronizing](#synchronizing).) - -Next we want to rebase the new work onto the development branch. To rebase local changes: - - git checkout featureA - git rebase develop - -OR -`git rebase develop featureA` - -Once the rebase is complete and conflicts are resolved, the develop branch can be fast forwarded to include all the local changes. This is done by - - git checkout develop - git merge featureA --ff - -### Interactive rebasing - -This is a good time to tidy up and clean up the local development work, and interactive rebasing is a good idea here. Just use `git rebase -i` in place of the `git rebase` commands above (the other arguments stay the same.) - -In interactive rebasing, you have the opportunity to drop commits, squash multiple together into a more logical patch, or even edit commits by splitting them up into multiple commits. For more on the power of interactive rebasing, browse the internet. A great place to get familiar with git flows and rebasing is [https://learngitbranching.js.org](https://learngitbranching.js.org/) - -### Pushing Upstream - -Once local work has been rebased onto the synchronized development branch, push those changes upstream to share the work. -`git push origin develop` - -### Cleanup - -If you have local branches that are no longer needed once they have made their way into the remote tracked development branch, you can remove them with -`git branch -d ` - -## General rebase reference - -A quick synopsis of local rebasing to manage local branches is provided here - -**Add commits from branch B to the end of branch A** -`git rebase A B` -To sync both to same commit, invert the rebase or merge -`git rebase B A` OR `git checkout A; git merge B` - -**sync two branches very explicitly** - - git checkout B - git rebase A -sync both to same commit: - - git checkout A - git rebase B - -OR `git merge B --ff` - - -> Written with [StackEdit](https://stackedit.io/).