diff --git a/.github/ISSUE_TEMPLATE/release-qa.md b/.github/ISSUE_TEMPLATE/release-qa.md index 0e31bb1b5a52..b29885ce1fe8 100644 --- a/.github/ISSUE_TEMPLATE/release-qa.md +++ b/.github/ISSUE_TEMPLATE/release-qa.md @@ -2,7 +2,7 @@ name: Release QA about: Checklist of required tests prior to release title: 'Release QA:' -labels: '#g-mdm,#g-endpoint-ops,:release' +labels: '#g-mdm,#g-orchestration,#g-software:release' assignees: 'xpkoala,pezhub,jmwatts' --- @@ -27,11 +27,12 @@ Smoke tests are limited to core functionality and serve as a pre-release final r ### Prerequisites -1. `fleetctl preview` is set up and running the desired test version using [`--tag` parameters.](https://github.com/fleetdm/fleet/blob/main/handbook/product.md#manual-qa ) +1. `fleetctl preview` is set up and running the desired test version using [`--tag` parameters.](https://fleetdm.com/handbook/engineering#run-fleet-locally-for-qa-purposes) 2. Unless you are explicitly testing older browser versions, browser is up to date. 3. Certificate & flagfile are in place to create new host. 4. In your browser, clear local storage using devtools. +### Orchestration
Test name | Step instructions | Expected result | pass/fail |
---|---|---|---|
$Name | {what a tester should do} | {what a tester should see when they do that} | pass/fail | pass/fail | -
Query flow | Create, edit, run, and delete queries. | - -1. permissions regarding creating/editing/deleting queries are up to date with documentation -2. syntax errors result in error messaging -3. queries can be run manually - | pass/fail |
Host Flow | Verify a new host can be added and removed following modal instructions using your own device. | - -1. Host is added via command line -2. Host serial number and date added are accurate -3. Host is not visible after it is deleted -4. Warning and informational modals show when expected and make sense - | pass/fail |
Packs flow | Verify management, operation, and loggingΒ of ["2017 packs"](https://fleetdm.com/handbook/company/why-this-way#why-does-fleet-support-query-packs). | 1. Packs successfully run on host machines after migrations @@ -82,16 +69,17 @@ Smoke tests are limited to core functionality and serve as a pre-release final r 2. Software, query, policy, and packs logs are successfully sent to Filesystem log destinations | pass/fail |
OS settings | Verify OS settings functionality | - - | |
My device page | Verify the end user's my device page loads successfully. | - -1. Clicking the Fleet desktop item, then "My device" successfully loads the my device page. -2. The "My device" page is populated correctly and as expected. -3. Styling and padding appears correct. - +1. Verify able to configure Disk encryption (macOS, Windows, & Linux). +2. Verify host enrolled with Disk encryption enforced successfully encrypts. | pass/fail |
Test name | Step instructions | Expected result | pass/fail |
---|---|---|---|
$Name | {what a tester should do} | {what a tester should see when they do that} | pass/fail |
MDM enrollment flow | Verify MDM enrollments, run MDM commands | 1. Erase an ADE-eligible macOS host and verify able to complete automated enrollment flow. @@ -106,45 +94,22 @@ Smoke tests are limited to core functionality and serve as a pre-release final r 3. Turn off MDM on a non ADE-eligible macOS host. 4. On the My device page, follow the "Turn on MDM" instructions and verify that MDM is turned on. | pass/fail |
Scripts | Verify script library and execution | - -1. Verify able to run a script on all host types from CLI. -2. Verify scripts library upload/download/delete. -3. From Host details (macOS, Windows, & Linux) run a script that should PASS, verify. -4. From Host details (macOS, Windows, & Linux) run a script that should FAIL, verify. -5. Verify UI loading state and statuses for scripts. -6. Disable scripts globally and verify unable to run. -7. Verify scripts display correctly in Activity feed. - | pass/fail |
Software | Verify software library and install / download | - -1. Verify software library upload/download/delete. -2. From Host details (macOS, Windows, & Linux) run an install that should PASS, verify. -3. From My Device (macOS, Windows, & Linux) software tab should have self-service items available, verify. -4. Verify UI loading state and statuses for installing software. -6. Verify software installs display correctly in Activity feed. - | pass/fail |
OS settings | Verify OS settings functionality | -1. Verify able to configure Disk encryption (macOS, Windows, & Linux). -2. Verify host enrolled with Disk encryption enforced successfully encrypts. -3. Verify Profiles upload/download/delete (macOS & Windows). -4. Verify Profiles are delivered to host and applied. +1. Verify Profiles upload/download/delete (macOS & Windows). +2. Verify Profiles are delivered to host and applied. | pass/fail |
Setup experience | Verify macOS Setup experience | 1. Configure End user authentication. -2. Upload a Bootstrap package. -3. Add software (FMA, VPP, & Custom pkg) -4. Add a script -5. Enroll an ADE-eligible macOS host and verify successful authentication. -6. Verify Bootstrap package is delivered. -7. Verify SwiftDialogue window displays. -8. Verify software installs and script runs. +3. Upload a Bootstrap package. +4. Add software (FMA, VPP, & Custom pkg) +5. Add a script +6. Enroll an ADE-eligible macOS host and verify successful authentication. +7. Verify Bootstrap package is delivered. +8. Verify SwiftDialogue window displays. +9. Verify software installs and script runs. | pass/fail |
OS updates | Verify OS updates flow | @@ -161,6 +126,7 @@ Smoke tests are limited to core functionality and serve as a pre-release final r 3. Verify Profiles are delivered to host and applied. 4. Verify VPP apps install & display correctly in Activity feed. + | pass/fail |
Certificates Upload | APNs cert and ABM token renewal workflow | 1. Renew APNs Certificate. @@ -168,18 +134,70 @@ Smoke tests are limited to core functionality and serve as a pre-release final r 3. Ensure ADE hosts can enroll. | pass/fail |
Test name | Step instructions | Expected result | pass/fail |
---|---|---|---|
$Name | {what a tester should do} | {what a tester should see when they do that} | pass/fail |
Query flow | Create, edit, run, and delete queries. | + +1. permissions regarding creating/editing/deleting queries are up to date with documentation +2. syntax errors result in error messaging +3. queries can be run manually + | pass/fail |
Host Flow | Verify a new host can be added and removed following modal instructions using your own device. | + +1. Host is added via command line +2. Host serial number and date added are accurate +3. Host is not visible after it is deleted +4. Warning and informational modals show when expected and make sense + | pass/fail |
My device page | Verify the end user's my device page loads successfully. | + +1. Clicking the Fleet desktop item, then "My device" successfully loads the my device page. +2. The "My device" page is populated correctly and as expected. +3. Styling and padding appears correct. + + | pass/fail |
Scripts | Verify script library and execution | + +1. Verify able to run a script on all host types from CLI. +2. Verify scripts library upload/download/delete. +3. From Host details (macOS, Windows, & Linux) run a script that should PASS, verify. +4. From Host details (macOS, Windows, & Linux) run a script that should FAIL, verify. +5. Verify UI loading state and statuses for scripts. +8. Disable scripts globally and verify unable to run. +9. Verify scripts display correctly in Activity feed. + | pass/fail |
Software | Verify software library and install / download | + +1. Verify software library upload/download/delete. +2. From Host details (macOS, Windows, & Linux) run an install that should PASS, verify. +3. From My Device (macOS, Windows, & Linux) software tab should have self-service items available, verify. +4. Verify UI loading state and statuses for installing software. +7. Verify software installs display correctly in Activity feed. + | pass/fail |
Migration Test | Verify Fleet can migrate to the next version with no issues. | Using the migration scripts located in fleet/test/upgrade/ 1. Run the upgrade_test.go script using the most recent stable version of Fleet and `main`. 2. Upgrade test returns an 'OK' response. | pass/fail |
Test name | Step instructions | Expected result | pass/fail |
---|---|---|---|
$Name | {what a tester should do} | {what a tester should see when they do that} | pass/fail |
Release blockers | Verify there are no outstanding release blocking tickets. | 1. Check [this](https://github.com/fleetdm/fleet/labels/~release%20blocker) filter to view all open `~release blocker` tickets. 2. If any are found raise an alarm in the `#help-engineering` and `#g-mdm` (or `#g-endpoint-ops`) channels. - | pass/fail | pass/fail |
{{queryResult}}
+ This group is intended for anyone who supports or manages a group of Macs or iOS devices of any size (from less than a dozen to ten of thousands) for an institution of any stripe (Small/Medium Business, Non-profit, Enterprise, Education) in the Greater Philadelphia area (South Eastern PA, South Jersey, Delaware) to swap ideas and solutions, network, and hang out.
-Discover security misconfigurations and vulnerabilities and prioritize risks that matter to your organization.
+Identify security misconfigurations and vulnerabilities and prioritize risks that matter to your organization.
Continuously scan host filesystems for indicators of compromise (IOC). Import malware signatures from threat intelligence sources.
+Use attack signatures from threat intelligence sources to scan and resolve indicators of compromise (IOC).