-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updates to ci/gha #117
Updates to ci/gha #117
Changes from all commits
2f88083
f7db1aa
7c41b43
77f79d5
77876cc
d706ee8
4e4e376
6676404
934facf
baed83e
51488d9
b5b8c74
ddc7b48
7ca92d3
b3b59da
dc80fd2
39218dd
707907e
2390554
388292a
7650cd7
669896f
ba74434
541d492
aececc2
167f59c
d545078
d85cf3e
3603659
d5533e1
d0da880
461d1e7
0884ee6
e0b20c3
8f71809
bf2d673
53aa05a
b636b82
fbc7841
6fea37b
ed3e6a4
87cab81
7d49de2
59d5d3e
88532b6
497f9ad
be188ee
fb5c471
994a33b
6bea511
7e1012d
32816a9
5fb8468
efea9f4
f82845c
11f75e2
6cb9957
b8a2030
0ae640b
6d81099
edf07a4
c74aebe
2655930
7cb6f4b
b75d325
c8a13bc
1f43c02
53b0d57
1dd2455
fd560d3
54edeb2
067efd0
3129a27
f6155ec
3da2fe9
d399d83
37d9a6e
f4b95cf
71609b4
7f45ea9
f3ea37d
fbb18fd
0006f69
acfdf94
663ae54
780b3ec
e896645
fbfa531
87da3f7
f8f7017
d83a6d3
1284a74
b5d8087
74d7b7d
4f4dc42
b340e07
23ed462
77cd22d
bc32459
f3962e6
43f4853
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -7,10 +7,12 @@ description: | | |||||
StackStorm-Exchange/ci/gha/apt-dependencies. | ||||||
author: StackStorm | ||||||
|
||||||
|
||||||
inputs: | ||||||
python-version: | ||||||
required: true | ||||||
description: Which python version we should install | ||||||
default: "3.6" | ||||||
cache-version: | ||||||
required: false | ||||||
default: "v0" | ||||||
|
@@ -24,28 +26,30 @@ runs: | |||||
using: "composite" | ||||||
steps: | ||||||
|
||||||
- name: 'Set up Python (${{ input.python-version }})' | ||||||
- name: 'Set up Python (${{ inputs.python-version }})' | ||||||
uses: actions/setup-python@v2 | ||||||
with: | ||||||
python-version: '${{ input.python-version }}' | ||||||
python-version: '${{ inputs.python-version }}' | ||||||
|
||||||
- name: Cache Python Dependencies | ||||||
uses: actions/cache@v2 | ||||||
with: | ||||||
path: | | ||||||
~/.cache/pip | ||||||
~/virtualenv | ||||||
key: ${{ runner.os }}-python-${{ input.python-version }}-${{ input.cache-version }}-${{ hashFiles('requirements.txt') }} | ||||||
key: ${{ runner.os }}-python-${{ inputs.python-version }}-${{ inputs.cache-version }}-${{ hashFiles('requirements.txt') }} | ||||||
restore-keys: | | ||||||
${{ runner.os }}-python-${{ input.python-version }}-${{ input.cache-version }}- | ||||||
${{ runner.os }}-python-${{ inputs.python-version }}-${{ inputs.cache-version }}- | ||||||
|
||||||
- name: Install virtualenv | ||||||
# this should run in the st2 checkout | ||||||
working-directory: st2 | ||||||
run: | | ||||||
./scripts/github/install-virtualenv.sh | ||||||
# - name: Install virtualenv | ||||||
# shell: bash | ||||||
# # this should run in the st2 checkout | ||||||
# working-directory: st2 | ||||||
# run: | | ||||||
# ./scripts/github/install-virtualenv.sh | ||||||
|
||||||
- name: Create ~/virtualenv | ||||||
shell: bash | ||||||
id: virtualenv | ||||||
# this should run in the st2 checkout | ||||||
working-directory: st2 | ||||||
|
@@ -55,27 +59,31 @@ runs: | |||||
./scripts/github/install-virtualenv.sh | ||||||
PIP_VERSION=$(grep '^PIP_VERSION' Makefile | awk '{print $3}') | ||||||
echo "::set-output name=pip-version::${PIP_VERSION}" | ||||||
${VIRTUALENV_DIR}/bin/pip install -U "pip==${PIP_VERSION}" setuptools | ||||||
${{env.VIRTUALENV_DIR}}/bin/pip install -U "pip==${PIP_VERSION}" setuptools | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||||||
|
||||||
- name: Install StackStorm requirements | ||||||
# this should run in the st2 checkout | ||||||
shell: bash | ||||||
working-directory: st2 | ||||||
env: | ||||||
VIRTUALENV_DIR: "~/virtualenv" | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK. Since tilde expansion isn't happening like I thought, so maybe we can do this instead:
Suggested change
To do that, we need to add a step at the beginning of the job like this: - name: add HOME to env context
shell: bash
run: echo ::set-env name=HOME::$HOME There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I just pushed this into change to the upstream gha branch. |
||||||
run: | | ||||||
${VIRTUALENV_DIR}/bin/pip install -r requirements.txt | ||||||
${{env.VIRTUALENV_DIR}}/bin/pip install -r requirements.txt | ||||||
|
||||||
- name: Install runners | ||||||
shell: bash | ||||||
env: | ||||||
VIRTUALENV_DIR: "~/virtualenv" | ||||||
ACTION_PATH: ${{ github.action_path }} | ||||||
ST2_REPO_PATH: st2 | ||||||
CI_DIR: ${{ format('{0}{1}', github.action_path, '/../..') }} | ||||||
# TODO: This Makefile has CircleCI assumptions | ||||||
run: | | ||||||
cp ${ACTION_PATH}/../../.circle/Makefile . | ||||||
cp ${{env.ACTION_PATH}}/../../.circle/Makefile . | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It turns out, github already exposes action_path as an env var, so we can remove ACTION_PATH from the env block and just use
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I just pushed this to the upstream gha branch |
||||||
make requirements-ci .install-runners | ||||||
|
||||||
- name: Install pack requirements | ||||||
shell: bash | ||||||
working-directory: pack | ||||||
env: | ||||||
VIRTUALENV_DIR: "~/virtualenv" | ||||||
|
@@ -84,19 +92,20 @@ runs: | |||||
run: | | ||||||
if [[ -f "${PACK_REQUIREMENTS_FILE}" ]]; then | ||||||
echo "Installing pack requirements from ${PACK_REQUIREMENTS_FILE}" | ||||||
${VIRTUALENV_DIR}/bin/pip install -r "${PACK_REQUIREMENTS_FILE}" | ||||||
${{env.VIRTUALENV_DIR}}/bin/pip install -r "${PACK_REQUIREMENTS_FILE}" | ||||||
fi | ||||||
if [[ -f "${PACK_TESTS_REQUIREMENTS_FILE}" ]]; then | ||||||
echo "Installing pack tests requirements from ${PACK_TESTS_REQUIREMENTS_FILE}" | ||||||
${VIRTUALENV_DIR}/bin/pip install -r "${PACK_TESTS_REQUIREMENTS_FILE}" | ||||||
${{env.VIRTUALENV_DIR}}/bin/pip install -r "${PACK_TESTS_REQUIREMENTS_FILE}" | ||||||
fi | ||||||
|
||||||
- name: Print versions | ||||||
shell: bash | ||||||
env: | ||||||
VIRTUALENV_DIR: "~/virtualenv" | ||||||
run: | | ||||||
set -ex | ||||||
source ${VIRTUALENV_DIR}/activate | ||||||
source ${{env.VIRTUALENV_DIR}}/bin/activate | ||||||
python3 --version | ||||||
pip --version | ||||||
virtualenv --version |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -24,27 +24,51 @@ runs: | |
using: "composite" | ||
steps: | ||
|
||
# - name: Get Pack Name (original) | ||
# shell: bash | ||
# id: pack-name | ||
# env: | ||
# VIRTUALENV_DIR: "~/virtualenv" | ||
# ACTION_PATH: ${{ github.action_path }} | ||
# run: | | ||
# env | ||
# export PACK_NAME=$(${{env.VIRTUALENV_DIR}}/bin/python ${ACTION_PATH}/../../.circle/validate.py "${GITHUB_REPOSITORY##*/}" pack.yaml) | ||
# if [[ -z "${PACK_NAME}" ]]; then | ||
# echo "Unable to retrieve pack name." | ||
# exit 1 | ||
# fi | ||
# echo "::set-output name=pack-name::${PACK_NAME}" | ||
|
||
- name: Get Pack Name | ||
shell: bash | ||
id: pack-name | ||
env: | ||
VIRTUALENV_DIR: "~/virtualenv" | ||
ACTION_PATH: ${{ github.action_path }} | ||
run: | | ||
export PACK_NAME=$(${VIRTUALENV_DIR}/bin/python ${ACTION_PATH}/../../.circle/validate.py "${GITHUB_REPOSITORY##*/}" pack.yaml) | ||
echo -e '\n\n/home/runner/work/logicmonitor-stackstorm-pack/ \n\n' | ||
ls /home/runner/work/logicmonitor-stackstorm-pack/ | ||
echo -e '\n\n/home/runner/work/lm-ydubler/logicmonitor-stackstorm-pack \n\n' | ||
ls /home/runner/work/logicmonitor-stackstorm-pack/logicmonitor-stackstorm-pack/ | ||
echo -e '\n\n/home/runner/work/logicmonitor-stackstorm-pack/logicmonitor-stackstorm-pack/pack/ \n\n' | ||
ls /home/runner/work/logicmonitor-stackstorm-pack/logicmonitor-stackstorm-pack/pack/ | ||
echo -e '\n\n' | ||
export PACK_NAME=$(${{env.VIRTUALENV_DIR}}/bin/python ${ACTION_PATH}/../../.circle/validate.py "lm-ydubler/logicmonitor-stackstorm-pack" "/home/runner/work/logicmonitor-stackstorm-pack/logicmonitor-stackstorm-pack/pack/pack.yaml") | ||
if [[ -z "${PACK_NAME}" ]]; then | ||
echo "Unable to retrieve pack name." | ||
exit 1 | ||
fi | ||
echo "::set-output name=pack-name::${PACK_NAME}" | ||
|
||
- name: | ||
- name: Unsure | ||
shell: bash | ||
env: | ||
VIRTUALENV_DIR: "~/virtualenv" | ||
FORCE_CHECK_ALL_FILES: true if on default branch | ||
ROOT_DIR: pack directory | ||
CI_DIR: ${{ github.action_path + "/../.." }} | ||
ROOT_DIR: "/home/runner/work/logicmonitor-stackstorm-pack/logicmonitor-stackstorm-pack/" | ||
CI_DIR: ${{ format('{0}{1}', github.action_path, '/../..') }} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Here is where CI_DIR is populated - I kind of guessed on the relative path though, so feel free to adjust this if it's not right. |
||
ST2_REPO_PATH: st2 directory | ||
ENABLE_COMMON_LIBS: ${{ input.enable-common-libs }} | ||
ENABLE_COMMON_LIBS: ${{ inputs.enable-common-libs }} | ||
# TODO: This Makefile has CircleCI assumptions | ||
run: | | ||
if [[ "true" == "${ENABLE_COMMON_LIBS}" ]]; then | ||
|
@@ -54,6 +78,6 @@ runs: | |
export ST2_CONFIG_FILE=${CI_DIR}/conf/st2.tests.conf | ||
fi | ||
cp ${CI_DIR}/.circle/Makefile . | ||
make -C "${ROOT_DIR}" all-ci | ||
make -C "${{env.ROOT_DIR}}" all-ci | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If CI_DIR is not populated correctly, that is concerning. Hmm.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might have been populating correctly but possibly just had the wrong value. GHA doesn't allow string concatenation using a + so I improvised to use the format() function. So I'll look it over and make sure it all matches up knowing what the correct directory should be as a fail safe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And just so you know, this is a testing PR and has a million commits a second. When it checks out , I'll create the final-draft PR which will have a handful of commits just so that it comes across clean.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. I expected that. But having it in a PR allows for some discussion about the issues you are working on 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm going to consolidate some of your commits and add them to the gha branch of StackStorm-Exchange/ci.