Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update deployment documents #77

Merged
merged 1 commit into from
May 13, 2024
Merged

Update deployment documents #77

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 13 additions & 8 deletions deployment/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

CCNP is designed for collecting confidential computing primitives in cloud native environments. It can run as DaemonSet in a Kubernetes cluster or containers in a Docker environment on confidential virtual machines, such as Intel TDX guest(TD).

CCNP deployment supports to deploy on Ubuntu 24.04 and Ubuntu 23.10. The follows will use Ubuntu 24.04. Please see [deployment guide](https://github.com/cc-api/confidential-cloud-native-primitives/blob/v0.4.0/deployment/README.md) for Ubuntu 23.10.

## Build CCNP Kernel

Run [build.sh](../tools/build/build.sh) to build kernel packages for CCNP. It's recommended to run the tool on TDX host mentioned in [Configuration](../README.md#configuration).
Expand All @@ -14,7 +16,7 @@ $ sudo ./build.sh

**NOTE:**
- CCNP kernel patches are at [kernel](../tools/build/kernel/)
- The tool should be run on a Ubuntu 23.10 TDX host with TDX early preview packages installed. Please refer to [here](https://github.com/canonical/tdx)
- The tool should be run on a Ubuntu 24.04 TDX host with TDX early preview packages installed. Please refer to [here](https://github.com/canonical/tdx)
ruomengh marked this conversation as resolved.
Show resolved Hide resolved


## Prepare TDX guest image
Expand All @@ -26,30 +28,34 @@ It's recommended to run the tool on TDX host mentioned in [Configuration](../REA
A quick start is as below.

```
# Refer to TDX early preview to [generate a TD guest image](https://github.com/canonical/tdx/tree/noble-24.04?tab=readme-ov-file#create-a-new-td-guest-image).
# Refer to TDX early preview to [create a TD guest image](https://github.com/canonical/tdx?tab=readme-ov-file#create-a-new-td-guest-image).

# Get cvm image rewriter.
$ git clone https://github.com/cc-api/cvm-image-rewriter.git

# Plugin 98 is an example of setting IMA. It will take a few minutes. Suggest to skip it if not needed.
$ touch cvm-image-rewriter/plugins/98-ima-example/NOT_RUN

# Set file path of the generated output folder above. Plugin 06 will install the kernel in the guest image.
$ export CVM_TDX_GUEST_REPO=<path to above output folder>

# Set image size
# (Optional)Set image size
$ export GUEST_SIZE=<image size>G

# Run CVM image rewriter to configure a TDX guest image for CCNP
$ cd cvm-image-rewriter
$ ./run.sh -i <mantic-server-cloudimg-amd64.img or your initial guest image> -t <timeout in minutes, suggest to set to 15>
$ ./run.sh -i <initial guest image> -t <timeout in minutes, suggest to set to 15>
```

**NOTE:**
- By default all the plugins will be executed. Generate a `NOT_RUN` file under the specific plugin folder if you want to skip it.
- It's required to run [plugin](https://github.com/cc-api/cvm-image-rewriter/tree/main/plugins) 06, 07, 08, 09 for CCNP.
- Plugin 98 is an example of setting IMA. It will take a few minutes. Suggest to skip it if not needed.
- It's required to run [plugin](https://github.com/cc-api/cvm-image-rewriter/tree/main/plugins) 06, 07, 08, 09, 10 for CCNP.


## Create a TD

Start a TD using [qemu-test.sh](../tools/cvm-image-rewriter/qemu-test.sh) or [start-virt.sh](../tools/cvm-image-rewriter/start-virt.sh).
Start a TD using [qemu-test.sh](../tools/cvm-image-rewriter/qemu-test.sh).

- Use `qemu-test.sh`.
```
Expand Down Expand Up @@ -98,8 +104,7 @@ ccnp-webhook <your image tag>

## Setup QGS and PCCS on the Host

Intel Quote Generation Service(QGS) and Provisioning Certification Caching Service(PCCS) should be installed and configured on the host for getting TD Quote. Please refer to Section 4.3.2, 4.3.3 and 4.3.4 of [guide](https://www.intel.com/content/www/us/en/content-details/789198/whitepaper-linux-stacks-for-intel-trust-domain-extensions-1-5.html)
for QGS and PCCS installation.
Intel Quote Generation Service(QGS) and Provisioning Certification Caching Service(PCCS) should be installed and configured on the host for getting TD Quote. Please refer to [PCCS](https://github.com/cc-api/confidential-cloud-native-primitives/blob/v0.5.0/container/pccs/README.md) and [QGS](https://github.com/cc-api/confidential-cloud-native-primitives/blob/v0.5.0/container/qgs/README.md) tp start PCCS and QGS container and register the platform.


## Deploy CCNP in Kubernetes
Expand Down
23 changes: 5 additions & 18 deletions deployment/docker-compose/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The CCNP can be deployed in the confidential VMs using docker compose. In this d
The following scripts can help to generate CCNP images and deploy them in the TD nodes. `build.sh` can run on either host or TD. Other scripts are supposed to run in the TD.

- [build.sh](../../container/build.sh): The tool will build docker images and push them to remote registry if required. Skip it if you already have docker images prepared.
- [prerequisite.sh](./prerequisite.sh): This tool will complete the prerequisites for deploying CCNP on Ubuntu. For other distributions, you can follow the manual steps in [Prerequisite Manually](#optional-ccnp-prerequisite-manual-steps).
- [prerequisite.sh](./prerequisite.sh): This tool will complete the prerequisites for deploying CCNP on Ubuntu.
- [deploy-ccnp.sh](./deploy-ccnp.sh): The tool will deploy CCNP service using docker compose.
- [exec-ccnp-example.sh](./exec-ccnp-example.sh): The tool will create a docker container, getting container event logs, measurement and performing verification using CCNP SDK.

Expand All @@ -29,7 +29,7 @@ Use the script [deploy-ccnp.sh](./depoly-ccnp.sh) to deploy the CCNP services.
# Deploy CCNP with user specified remote registry and image tag
$ sudo ./deploy-ccnp.sh -r <remote registry> -g <tag>
e.g.
$ sudo ./deploy-ccnp.sh -r test-registry.intel.com/test -g 0.3
$ sudo ./deploy-ccnp.sh -r test-registry.intel.com/test -g 0.5
```

This script has some options as below.
Expand All @@ -44,7 +44,7 @@ You will see below container running after the deployment.
```
$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3a9de1a9c7d7 ccnp-server:0.3 "/usr/bin/ccnp_serve…" 36 seconds ago Up 34 seconds ccnp-server-ctr-ccnp-server-1
3a9de1a9c7d7 ccnp-server:0.5 "/usr/bin/ccnp_serve…" 36 seconds ago Up 34 seconds ccnp-server-ctr-ccnp-server-1
```

### Deploy CCNP Usage Example
Expand All @@ -53,7 +53,7 @@ The script [exec-ccnp-example.sh](./exec-ccnp-example.sh) will launch a containe
It will get measurement, event logs and cc_report using CCNP SDK and save the output in `/tmp/docker_ccnp/example.log`.

```
$ sudo ./exec-ccnp-example.sh -r test-registry.intel.com/test -g 0.3
$ sudo ./exec-ccnp-example.sh -r test-registry.intel.com/test -g 0.5
```

This script has some options as below.
Expand All @@ -70,7 +70,7 @@ You will see below container running after the deployment.
```
$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e815b6edafcb ccnp-example:0.3 "tail -f /dev/null" 17 seconds ago Up 15 seconds ccnp-example-ctr-ccnp-example-1
e815b6edafcb ccnp-example:0.5 "tail -f /dev/null" 17 seconds ago Up 15 seconds ccnp-example-ctr-ccnp-example-1
```

### Clean Up
Expand All @@ -80,16 +80,3 @@ The script `cleanup.sh` will help stop three containerized services and remove c
```
$ sudo ./cleanup.sh
```


### (Optional) CCNP Prerequisite Manual Steps
__NOTE: Below are manual Steps of CCNP prerequisite for your reference. They can be skipped if prerequisite.sh is run successfully.__

Basically the `prerequisite.sh` complete below steps to ensure `docker` is installed and set device permission. You can also complete them following below steps manually.

- Install docker on the TD nodes. Please refer to [Get Docker](https://docs.docker.com/get-docker/).
- Change the access privilege of the TDX device.
```
$ chmod 0666 $(find /dev/ -name "tdx*")
```
- Clean up `/tmp/docker_ccnp`.
33 changes: 2 additions & 31 deletions deployment/kubernetes/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ After TDs are started, users can let the TDs join an existing K8S cluster. Pleas
The following scripts can help to generate CCNP images and deploy them in the TD nodes. `build.sh` can run on either host or TD. Other scripts are supposed to run in the TD.

- [build.sh](../../container/build.sh): The tool will build docker images and push them to remote registry if required. Skip it if you already have docker images prepared.
- [prerequisite.sh](../kubernetes/script/prerequisite.sh): This tool will complete the prerequisites for deploying CCNP on Ubuntu. For other distributions, you can follow the manual steps in [Prerequisite Manually](#optional-ccnp-prerequisite-manual-steps).
- [prerequisite.sh](../kubernetes/script/prerequisite.sh): This tool will complete the prerequisites for deploying CCNP on Ubuntu.
- [deploy-ccnp.sh](../kubernetes/script/deploy-ccnp.sh): The tool will deploy CCNP services as DaemonSet on TDs in the K8S cluster.
- [deploy-ccnp-example.sh](../kubernetes/script/deploy-ccnp-example.sh): The tool will deploy an example pod with CCNP SDK installed.
- [exec-ccnp-example.sh](../kubernetes/script/exec-ccnp-example.sh): The tool will show getting event logs, measurement and perform verification using CCNP in the pod.
Expand All @@ -45,7 +45,7 @@ Run below scripts on each TD node.
# Deploy CCNP with user specified remote registry and image tag
$ sudo ./deploy-ccnp.sh -r <remote registry> -g <tag>
e.g.
$ sudo ./deploy-ccnp.sh -r test-registry.intel.com/test -g 0.3
$ sudo ./deploy-ccnp.sh -r test-registry.intel.com/test -g 0.5

# Delete existing CCNP and Deploy CCNP with user specified remote registry and image tag
$ sudo ./deploy-ccnp.sh -r <remote registry> -g <tag> -d
Expand Down Expand Up @@ -94,32 +94,3 @@ $ sudo ./exec-ccnp-example.sh -v

The example output of verification can be found at [sample-output-for-container-measurement.txt](../../docs/sample-output-for-container-measurement.txt) and
[sample-output-for-container-eventlog.txt](../../docs/sample-output-for-container-eventlog.txt).


### (Optional) CCNP Prerequisite Manual Steps
__NOTE: Below are manual Steps of CCNP prerequisite for your reference. They can be skipped if prerequisite.sh is run successfully.__

Basically the `prerequisite.sh` complete below steps to ensure `helm`, `docker` and `pip` are installed and check whether file permission is set correctly.
You can also complete them following below steps manually.
- Install Helm on the TD nodes. Please refer to the [HELM quick start](https://helm.sh/docs/intro/quickstart/).
- Install docker on the TD nodes. Please refer to [Get Docker](https://docs.docker.com/get-docker/).
- Install python3-pip on the TD nodes. Please refer to [pip document](https://pip.pypa.io/en/stable/installation/).
- Set access permission to TD device node and ccnp working directory on the TD nodes.
```
$ sudo mkdir -p /etc/udev/rules.d
$ sudo touch /etc/udev/rules.d/90-tdx.rules
# Check TD device node on TD
$ ls /dev/tdx*

# If above output is "/dev/tdx-guest"
$ sudo bash -c 'echo "SUBSYSTEM==\"misc\",KERNEL==\"tdx-guest\",MODE=\"0666\"">/etc/udev/rules.d/90-tdx.rules'
# If above output is "/dev/tdx_guest"
$ sudo bash -c 'echo "SUBSYSTEM==\"misc\",KERNEL==\"tdx_guest\",MODE=\"0666\"">/etc/udev/rules.d/90-tdx.rules'
# make the udev setup effective
$ sudo udevadm trigger

$ sudo touch /usr/lib/tmpfiles.d/ccnp.conf
$ sudo bash -c 'echo "D /run/ccnp/uds 0757 - - -">/usr/lib/tmpfiles.d/ccnp.conf'
# make the directory setup effective by running below command or restarting the node
$ sudo systemd-tmpfiles --create
```
Loading