Skip to content

Testing iscript

Jump to bottom
Aki Sasaki edited this page Apr 11, 2019 · 31 revisions

Updated 2019.04.10

Manually testing with iscript

Assumptions

This currently assumes we already have a manual testing environment (e.g. partner-repack-1) with an appropriately set up developer Apple account, a keychain with the appropriate pkg- and app-signing certs as well as the notarization account credentials, and python3.

These steps take us from testing manually via individual commands or bash scripts to testing via iscript.

initial setup

  • create a base directory. Currently I'm working in /builds/notarization/aki.
    • the owner of this directory needs passwordless sudo (/etc/sudoers)

Virtualenv

# chdir to base dir
cd /builds/notarization/aki

# clone iscript
git clone https://github.com/escapewindow/scriptworker-scripts

# create a python3 virtualenv.
/tools/python36/bin/python3.6 -mvenv venv
. venv/bin/activate

# install scriptworker_client and iscript via |python setup.py develop|
cd scriptworker-scripts/scriptworker_client
python setup.py develop
cd ../iscript
python setup.py develop
cd ../..

# Now, `git pull` or local changes to these python files will change iscript's behavior.

script_config.yaml

This can be json or yaml, since yaml is a superset of json. I thought it easier to use yaml. Look, comments:

# the work will happen in this dir
work_dir: /tmp/work

# artifacts will be created in this dir. You may want to nuke this dir before running iscript
# manually, or you may have leftover artifacts from previous runs.
artifact_dir: /tmp/artifacts

taskcluster_scope_prefix: "project:releng:signing:"

# Debug logging?
verbose: true

# These are used for multi_account notarization. If there are fewer accounts than apps in
# upstreamArtifacts, then we only notarize len(local_notarization_accounts) at a time.
local_notarization_accounts: ["account1"]

# mac_config, as opposed to a future ios_config
mac_config:

    # Currently dep is the only supported config.
    dep:

        # multi_account or single_zip
        # these two should allow for all 3 setup configs:
        # 1. single daemon, multi_account, multiple apps per task;
        # 2. single daemon, single zip, multiple apps per task;
        # 3. multiple daemons, single zip, single app per task
        notarize_type: multi_account

        # This is the path to the keychain
        signing_keychain: /path/to/dep.keychain

        # base of the notarization bundle id. We add some dots, TASK_ID, and timestamp
        base_bundle_id: org.mozilla.firefox_dep

        # app signing identity
        identity: ...

        # keychain password
        keychain_password: ...

        # pkg signing cert
        pkg_cert_id: ...

        # apple notarization account info
        apple_notarization_account: ...
        apple_notarization_password: ...

        # max time we wait for apple to return `success` after submitting notarization
        # requests.
        notarization_poll_timeout: 900

    # someday we'll support the above, with
    nightly:
        ...

    release:
        ...

We may want to make changes or add more config items to the above; still easy to do. Most likely we want to disable notarization in dep before rollout.

task.json

iscript will read task.json from the path <work_dir>/task.json. It will look like this, minus comments:

{
    # The schema requires this, but we don't care what's in it at the script level
    "dependencies": ["a"],

    # We require a single valid `cert` signing scope
    "scopes": ["project:releng:signing:cert:dep-signing"],
    "payload": {

        # The taskId/path combinations are the apps we will sign/notarize/pkg
        # Currently, the paths after public/build/ need to be unique across all tasks
        # i.e., task1 with public/build/target.tar.gz and task2 with public/build/target.tar.gz
        # would result in one overwriting the other in the artifact dir
        # Currently only .tar.gz tarballs and filenames are supported.
        "upstreamArtifacts": [{
            "taskId": "task1",
            "taskType": "build",
            "paths": ["public/build/1231-1/target.tar.gz", "public/build/0107-1/target-0107-enUS.tar.gz", "public/build/0107-1/target-0107-fr.tar.gz", "public/build/0101-1/target.tar.gz"]
        }, {
            "taskId": "task2",
            "taskType": "build",
            "paths": ["public/build/0101-2/target.tar.gz"]
        }],

        # to be implemented
        "entitlementsUrl": "",

        # this can be one of:
        # mac_notarize: sign, notarize app, create and sign pkg
        # mac_pkg: create and sign pkg from signed app
        # mac_sign: sign app
        # mac_sign_and_pkg: sign app, create and sign pkg
        "behavior": "mac_notarize"
    }
}

directory tree

The artifact dir will contain artifacts after a successful run of iscript.

The work dir will need to be prepopulated with a task.json and cot artifacts, which match the task.json's upstreamArtifacts. In addition, we need a browser.entitlements.txt that is a sibling of the work directory (since work/ is nuked, and we need this file to persist across runs until we have an in-tree solution.)

browser.entitlements.txt
work/cot/task1/public/build/en-US/target.tar.gz
work/cot/task2/public/build/fr/target.tar.gz
work/cot/task2/public/build/de/target-de.tar.gz
work/task.json

running iscript

We want to:

  • nuke the artifact dir to avoid cruft
  • redirect output to a log, ideally dated

So something like this works:

# after activating venv
rm -rf artifacts && iscript script_config.yaml 2>&1 | tee log.`date '+%Y%m%d%H%M'`
Clone this wiki locally