diff --git a/docs/architecture.md b/docs/architecture.md index 1209708a..5d97497d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -160,13 +160,11 @@ To validate commits that could be decades old without being obstructed by expire This module encapsulates the `GitRepository` class, a high-level abstraction over Git operations, designed to interface directly with Git repositories at the filesystem level. The `GitRepository` class serves as an intermediary, enabling programmatic access to Git actions including: creating branches, working with commits, and working with remotes. It leverages [`pygit2`](https://www.pygit2.org/) for some of the interactions with Git. Other interactions use direct shell command execution via subprocess for operations not covered by `pygit2` or where direct command invocation is preferred for efficiency or functionality reasons. -### `taf/repository_tool.py` +### `taf/tuf/repository` -Contains a `Repository` class, which is a wrapper around TUF's repository, making it simple to execute important updates, like -adding new signing keys, updating and signing metadata files and extracting information about roles, keys, -delegations and targets. - -NOTE: Long-term plan is to rework this part of the codebase. This is necessary to transition to the newest version of TUF, since it is relying on parts which no longer exist in newer TUF. +Contains a `MetadataRepository` class, which is an implementation of TUF's `Repository` class for editing metadata. +It simplifies the execution of important updates such as adding new signing keys, updating and signing metadata +files, and extracting information about roles, keys, delegations, and targets. ### `taf/auth_repo.py` diff --git a/docs/developers/repository-classes.md b/docs/developers/repository-classes.md index 19dc95f7..b5c6bc49 100644 --- a/docs/developers/repository-classes.md +++ b/docs/developers/repository-classes.md @@ -1,18 +1,18 @@ # Repositories -As a tool focused on creation and secure update of Git repositories (authentication repositories and their -targets), TAF contains classes and functions which strive to make integration with Git and TUF as easy as possible. -`GitRepository` can be seen as a wrapper around git calls which make it possible to interact with the actual `Git` -repository located on the file system. E.g. to create a new branch, list commits, push to the remote etc. -On the other hand, `Repository` class contained by the `repository_tool` module can instantiate a TUF repository, -provided that the directory passed to it contains metadata files expected to be found in such a repository. It also -implements important TUF concepts, such as adding a new delegated role, determine which role is responsible for which -target file, add TUF targets etc. An authentication repository can be seen as a Git repository which is also a TUF repository - it -contains TUF's metadata and target files and a `.git` folder. TAF's `auth_repo` module's `AuthenticationRepository` -class follows that logic and is derived from the two previously mentioned base classes. Finally, `repositoriesdb` -is a module inspired by TUF's modules like `keysdb`, which deals with instantiation of repositories and stores the -created classes inside a "database" - a dictionary which maps authentication repositories and their commits -to lists of their target repositories at certain revisions. +As a tool focused on the creation and secure update of Git repositories (authentication repositories and their +targets), TAF contains classes and functions that strive to make integration with Git and TUF as simple as possible. +`GitRepository` acts as a wrapper around Git calls, enabling interaction with the actual `Git` repository on the file +system, e.g., creating a new branch, listing, creating, and pushing commits, etc. Conversely, the `MetadataRepository` +class in `tuf/repository.py` extends TUF's `Repository` class, an abstract class for metadata modifying implementations. +It provides implementations of crucial TUF concepts, such as adding a new delegated role, determining which role is +responsible for which target file, and adding TUF targets etc. An authentication repository can be seen as a Git +repository that is also a TUF repository. It contains TUF's metadata and target files and a `.git` folder. TAF's +`auth_repo` module's `AuthenticationRepository` class follows that logic and is derived from the two previously +mentioned base classes. Finally, `repositoriesdb` is a module inspired by TUF's modules like `keysdb`, which deals with +the instantiation of repositories and stores the created classes inside a "database" - a dictionary which maps +authentication repositories and their commits to lists of their target repositories at certain revisions. Note: the +concept of databases has been removed from TUF and removal of `repositoriesdb` is also planned in case of TAF. ## GitRepository @@ -66,44 +66,38 @@ repo.commit_empty('An example message') repo.push() ``` -## Repository tool's `Repository` +## Implementation of TUF's `Repository` class (`tuf/repository/MetadataRepository`) + +This class extends TUF's repository interface, providing features for executing metadata updates, such as +adding new signing keys, updating and signing metadata files, and extracting information about roles, +keys, delegations, and targets. It can be used to create a new TUF repository, retrieve information about +a TUF repository, or update its metadata files. TAF's implementation of the repository class follows the +convention of separating metadata and target files into directories named `metadata` and `target`: -This class can be seen as a wrapper around a TUF repository, making it simple to execute important updates, like -adding new signing keys, updating and signing metadata files and extracting information about roles, keys, -delegations and targets. It is instantiated by passing file system path which corresponds to a directory containing -all files and folders that a TUF repository expects. That means that `metadata` and `targets` folders have to exist -and that a valid `root.json` file needs to be found inside `metadata`. So: ``` - repo_root - metadata - root.json - targets ``` -Optionally, `name` attribute can also be specified during instantiation. It will be used to set name of the TUF's -repository instance. This value is set to `default` if not provided. If more than one repository is to be used -at the same time, it is important to set distinct names. - -TUF repository is instantiated lazily the first time it is needed. This object is not meant to be used directly. -The main purpose of TAF's repository class is to group operations which enable valid update of TUF metadata and acquiring -information like can a key be used to sign a certain metadata file or finding roles that are linked with -the provided public key. To set up a new repository or add a new signing key, it is recommended to use the -`developer_tool` module since it contains full implementations of these complex functionalities. Functionalities -like updating targets and signing metadata or updating a metadata's expiration date are fully covered by repository -class's methods and can be used directly. These include: -- `update_timestamp_keystores`, `update_snapshot_keystores` (`update_rolename_keystores`) and `update_role_keystores` (for delegated roles) --`update_timestamp_yubikeys`, `update_snapshot_yubikeys` (`update_rolename_yubikeys`) and `update_role_yubikeys` (for delegated roles) - -If `added_targets_data` or `removed_targets_data` is passed in when calling these methods (only applicable to -`targets` and delegated target roles), information about target files will be updated and the corresponding metadata -file will be signed. Its expiration date will be updated too. If there is targets data or if the called method -corresponds to a non-targets role, the metadata file's expiration will still be updated and the file will be signed. + +It is instantiated by providing the repository's path. Unlike the previous implementation, which was based on an +older version of TUF, this repository does not have, nor does it need, a name. The class can be instantiated +regardless of whether there are `metadata` files located at `path/metadata`. In fact, it is possible to read the +metadata and target files from mediums other than the local file system. TUF enables such flexibility by allowing +custom implementations of the `StorageBackendInterface`. These implementations can redefine how metadata and target +files are read and written. To instantiate a `MetadataRepository` class with a custom storage interface, use the +`storage` keyword argument. If not specified, TUF's default `FilesystemBackend` will be used. + +This class is used extensively to implement API functions. ## `AuthenticationRepository` -This class is derived from both `GitRepository` and TAF's `Repository`. Authentication repositories are expected -to contain TUF metadata and target files, but are also Git repositories. It is important to note that only files -inside the `targets` folder are tracked and secured by TUF. +This class is derived from `GitRepository`, and indirectly from `MetadataRepository`. Authentication repositories are +expected to contain TUF metadata and target files, but are also Git repositories. It is important to note that only +files inside the `targets` folder are tracked and secured by TUF. + Instances of the `AuthenticationRepository` are created by passing the same arguments as to `GitRepository` (`library_dir`, `name`, `urls`, `custom`, `default_branch`, `allow_unsafe` and `path` which can replace `library_dir` and `name` combination), as well as some optional additional arguments: - `conf_directory_root` - path to the directory where the `last_validated_commit` will be stored. diff --git a/docs/testing/testing_notes.md b/docs/testing/testing_notes.md deleted file mode 100644 index 0ebccc1d..00000000 --- a/docs/testing/testing_notes.md +++ /dev/null @@ -1,57 +0,0 @@ -# Developer tool - -## Setting up repositories - -### Yubikey flow - -1. `taf repo create ./law --keys-description ./data/keys.json --commit-msg "Generated initial metadata"` -2. `taf targets update-repos-from-fs ./law --library-dir ./data/targets --namespace test` -3. `taf targets generate-repositories-json ./law --library-dir ./data/targets --namespace test --custom data/custom_data.json` -4. `taf targets sign ./law` -5. `taf roles add-signing-key ./law --role targets` - -### Yubikey + Keystore flow - -1. `taf repo create ./law --keys-description ./data/keys.json --commit-msg "Generated initial roles" --keystore ./data/keystore/` -1. `taf targets update-repos-from-fs ./law --library-dir ./data/targets --namespace test` -1. `taf targets generate-repositories-json ./law --library-dir ./data/targets --namespace test --custom data/custom_data.json` -1. `taf targets sign --keystore ./data/keystore/` -1. `taf roles add-signing-key ./law --role targets` - -### keys.json - -``` -{ - "roles": { - "root": { - "yubikey": true, - "number": 3, - "length": 2048, - "threshold": 2 - }, - "targets": { - "yubikey": true, - "length": 2048 - }, - "snapshot": {}, - "timestamp": {} - } -} -``` - -### custom_data.json - -``` -{ - "test/law-xml": { - "type": "xml", - "allow-unauthenticated-commits": true - }, - "test/law-xml-codified": { - "type": "xml-codified" - }, - "test/law-html": { - "type": "html" - } -} -``` diff --git a/taf/auth_repo.py b/taf/auth_repo.py index 7ea1ac10..3ee4f045 100644 --- a/taf/auth_repo.py +++ b/taf/auth_repo.py @@ -125,8 +125,9 @@ def conf_dir(self) -> str: return self._conf_dir @property - def certs_dir(self) -> str: - certs_dir = Path(self.path, "certs") + def certs_dir(self): + certs_dir = self.path / "certs" + certs_dir.mkdir(parents=True, exist_ok=True) return str(certs_dir) @property diff --git a/taf/tuf/repository.py b/taf/tuf/repository.py index 39cdf405..ee090f4e 100644 --- a/taf/tuf/repository.py +++ b/taf/tuf/repository.py @@ -108,17 +108,6 @@ class MetadataRepository(Repository): serializer = JSONSerializer(compact=False) - # TODO - what is this? - # @property - # def repo_id(self): - # return GitRepository(path=self.path).initial_commit - - @property - def certs_dir(self): - certs_dir = self.path / "certs" - certs_dir.mkdir(parents=True, exist_ok=True) - return str(certs_dir) - def __init__(self, path: Union[Path, str], *args, **kwargs) -> None: storage_backend = kwargs.pop("storage", None) super().__init__(*args, **kwargs)