Linux 2.22 Open Source Gold Release

Upgraded to OpenSSL 3.0.10. Added interoperable RA-TLS support which follows CCC design. Enhanced Protect File System performance and added additional dependency `libsgx_pthread.a`. Added the Constant Time instruction Decoder (CTD) into the default AEX-Notify mitigation handler in order to prevent the introduction of any additional subtle sidechannel leakages within the default handler. Added Mistletoe 3 mitigations to the IPP Cryptography Library to the AES-ECB, AESGCM, and AES-CMAC algorithms. These have been incorporated transparently into the `sgx_tcrypto` library. Resigned all Intel® SGX Architecture Enclaves. Upgraded Intel SGX Quote Verification Enclave to integrate OpenSSL/SgxSSL 3.0.10. Added Attestation Library support for Intel(R) TDX Migration TD. Added Rust wrapper for low-level Quote Generation APIs. Enabled `SE_TRACE` log in release binary. Updated Rust QVL wrapper to use native Rust structure for quote verification collateral. Added a limitation in the DCAP QVL to only allow the user to set the QvE load policy once. Fixed bugs. Signed-off-by: Li, Xun <[email protected]>
intel · Oct 24, 2023 · 8a22317 · 8a22317
1 parent 8be98b1
commit 8a22317
Show file tree

Hide file tree

Showing 135 changed files with 3,752 additions and 2,316 deletions.
diff --git a/.gitmodules b/.gitmodules
@@ -23,3 +23,6 @@
 [submodule "external/mbedtls/mbedtls_code"]
 	path = external/mbedtls/mbedtls_code
 	url = https://github.com/Mbed-TLS/mbedtls.git
+[submodule "external/cbor/libcbor"]
+	path = external/cbor/libcbor
+	url = https://github.com/PJK/libcbor.git
diff --git a/Makefile b/Makefile
@@ -55,7 +55,10 @@ preparation:
 	cd external/openmp/openmp_code && git apply ../0001-Enable-OpenMP-in-SGX.patch >/dev/null 2>&1 ||  git apply ../0001-Enable-OpenMP-in-SGX.patch --check -R
 	cd external/protobuf/protobuf_code && git apply ../sgx_protobuf.patch >/dev/null 2>&1 ||  git apply ../sgx_protobuf.patch --check -R
 	./external/sgx-emm/create_symlink.sh
-	@# download prebuilt binaries
+	cd external/mbedtls/mbedtls_code && git apply ../sgx_mbedtls.patch >/dev/null 2>&1 || git apply ../sgx_mbedtls.patch --check -R
+	cd external/cbor && cp -r libcbor sgx_libcbor
+	cd external/cbor/libcbor && git apply ../raw_cbor.patch >/dev/null 2>&1 || git apply ../raw_cbor.patch --check -R
+	cd external/cbor/sgx_libcbor && git apply ../sgx_cbor.patch >/dev/null 2>&1 || git apply ../sgx_cbor.patch --check -R
 	./download_prebuilt.sh
 	./external/dcap_source/QuoteGeneration/download_prebuilt.sh
 

diff --git a/README.md b/README.md
@@ -36,13 +36,13 @@ Introduction
 ------------
 Intel(R) Software Guard Extensions (Intel(R) SGX) is an Intel technology for application developers seeking to protect select code and data from disclosure or modification.
 
-The Linux\* Intel(R) SGX software stack is comprised of the Intel(R) SGX driver, the Intel(R) SGX SDK, and the Intel(R) SGX Platform Software (PSW). The Intel(R) SGX SDK and Intel(R) SGX PSW are hosted in the [linux-sgx](https://github.com/01org/linux-sgx) project.
+The Linux\* Intel(R) SGX software stack is comprised of the Intel(R) SGX driver, the Intel(R) SGX SDK, and the Intel(R) SGX Platform Software (PSW). The Intel(R) SGX SDK and Intel(R) SGX PSW are hosted in the [linux-sgx](https://github.com/intel/linux-sgx) project.
 
 The [SGXDataCenterAttestationPrimitives](https://github.com/intel/SGXDataCenterAttestationPrimitives/) project maintains an out-of-tree driver for the Linux\* Intel(R) SGX software stack, which will be used until the driver upstreaming process is complete. It is used on the platforms with *Flexible Launch Control* and *Intel(R) AES New Instructions* support and could support both Elliptic Curve Digital Signature algorithm (ECDSA) based attestation and Enhanced Privacy Identification (EPID) based attestation.
 
 **Note**: Ice Lake Xeon-SP (and the future Xeon-SP platforms) doesn't support EPID attestation.
 
-The [linux-sgx-driver](https://github.com/01org/linux-sgx-driver) project hosts the other out-of-tree driver for the Linux\* Intel(R) SGX software stack, which will be used until the driver upstreaming process is complete. It is used to support Enhanced Privacy Identification (EPID) based attestation on the platforms without *Flexible Launch Control*.
+The [linux-sgx-driver](https://github.com/intel/linux-sgx-driver) project hosts the other out-of-tree driver for the Linux\* Intel(R) SGX software stack, which will be used until the driver upstreaming process is complete. It is used to support Enhanced Privacy Identification (EPID) based attestation on the platforms without *Flexible Launch Control*.
 
 The [intel-device-plugins-for-kubernetes](https://github.com/intel/intel-device-plugins-for-kubernetes) project enables users to run container applications running Intel(R) SGX enclaves in Kubernetes clusters. It also gives instructions how to set up ECDSA based attestation in a cluster.
 
@@ -63,8 +63,8 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for details.
 
 Documentation
 -------------
-- [Intel(R) SGX for Linux\* OS](https://01.org/intel-softwareguard-extensions) project home page on [01.org](https://01.org)
-- [Intel(R) SGX Programming Reference](https://software.intel.com/sites/default/files/managed/7c/f1/332831-sdm-vol-3d.pdf)
+- [Intel(R) SGX for Linux\* OS](https://www.intel.com/content/www/us/en/developer/tools/software-guard-extensions/linux-overview.html) project home page on [Intel Developer Zone](https://www.intel.com/content/www/us/en/developer/overview.html)
+- [Intel(R) SGX Programming Reference](https://www.intel.com/content/dam/develop/external/us/en/documents/329298-002-629101.pdf)
 
 Quick Start with Docker and Docker Compose
 -----------------------------------------
@@ -82,26 +82,24 @@ $ cd linux/installer/docker && ./build_compose_run.sh
 Build and Install the Intel(R) SGX Driver
 -----------------------------------------
 Follow the [README.md](https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/driver/linux/README.md) in the [SGXDataCenterAttestationPrimitives](https://github.com/intel/SGXDataCenterAttestationPrimitives/) project to build and install the Intel(R) SGX driver.
-**NOTE**: The above Intel(R) SGX driver requires *Flexible Launch Control* and *Intel(R) AES New Instructions* support. If your platform doesn't meet the requirement, please follow  the instructions in the [linux-sgx-driver](https://github.com/01org/linux-sgx-driver) project to build and install this version of Intel(R) SGX driver.
+**NOTE**: The above Intel(R) SGX driver requires *Flexible Launch Control* and *Intel(R) AES New Instructions* support. If your platform doesn't meet the requirement, please follow  the instructions in the [linux-sgx-driver](https://github.com/intel/linux-sgx-driver) project to build and install this version of Intel(R) SGX driver.
 
 Build the Intel(R) SGX SDK and Intel(R) SGX PSW Package
 -------------------------------------------------------
 ### Prerequisites:
 - Ensure that you have one of the following required operating systems:
-  * Ubuntu\* 18.04 LTS Desktop 64bits
-  * Ubuntu\* 18.04 LTS Server 64bits
   * Ubuntu\* 20.04 LTS Desktop 64bits
   * Ubuntu\* 20.04 LTS Server 64bits
   * Ubuntu\* 22.04 LTS Server 64bits
-  * Red Hat Enterprise Linux Server release 8.6 64bits
-  * CentOS Stream 8 64bits
+  * Red Hat Enterprise Linux Server release 9.2 64bits
+  * CentOS Stream 9 64bits
   * CentOS 8.3 64bits
   * SUSE Linux Enterprise Server 15.4 64bits
   * Anolis OS 8.6 64bits
   * Debian 10 64bits
 
 - Use the following command(s) to install the required tools to build the Intel(R) SGX SDK:
-  * On Ubuntu 18.04 and Debian 10:
+  * On Debian 10:
   ```
     $ sudo apt-get install build-essential ocaml ocamlbuild automake autoconf libtool wget python3 libssl-dev git cmake perl
     $ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 1
@@ -110,13 +108,17 @@ Build the Intel(R) SGX SDK and Intel(R) SGX PSW Package
   ```
     $ sudo apt-get install build-essential ocaml ocamlbuild automake autoconf libtool wget python-is-python3 libssl-dev git cmake perl
   ```
-  * On Red Hat Enterprise Linux 8.6:
+  * On Red Hat Enterprise Linux 9.2:
   ```
     $ sudo yum groupinstall 'Development Tools'
     $ sudo yum install ocaml ocaml-ocamlbuild wget python3 openssl-devel git cmake perl
-    $ sudo alternatives --set python /usr/bin/python3
   ```
-  * On CentOS Stream 8 and CentOS 8.3:
+  * On CentOS Stream 9:
+  ```
+    $ sudo dnf group install 'Development Tools'
+    $ sudo dnf install ocaml ocaml-ocamlbuild redhat-rpm-config openssl-devel wget rpm-build git cmake perl python3
+  ```  
+  * On CentOS 8.3:
   ```
     $ sudo dnf group install 'Development Tools'
     $ sudo dnf --enablerepo=powertools install ocaml ocaml-ocamlbuild redhat-rpm-config openssl-devel wget rpm-build git cmake perl python3
@@ -145,11 +147,15 @@ Build the Intel(R) SGX SDK and Intel(R) SGX PSW Package
       ```
         $ sudo apt-get install libssl-dev libcurl4-openssl-dev protobuf-compiler libprotobuf-dev debhelper cmake reprepro unzip pkgconf libboost-dev libboost-system-dev libboost-thread-dev lsb-release libsystemd0
       ```
-      * On Red Hat Enterprise Linux 8.6:
+      * On Red Hat Enterprise Linux 9.2:
       ```
         $ sudo yum install openssl-devel libcurl-devel protobuf-devel cmake rpm-build createrepo yum-utils pkgconf boost-devel protobuf-lite-devel systemd-libs
       ```
-      * On CentOS Stream 8 and CentOS 8.3:
+      * On CentOS Stream 9:
+      ```
+        $ sudo dnf install openssl-devel libcurl-devel protobuf-devel cmake rpm-build createrepo yum-utils pkgconf boost-devel protobuf-lite-devel systemd-libs
+      ```      
+      * On CentOS 8.3:
       ```
         $ sudo dnf --enablerepo=powertools install openssl-devel libcurl-devel protobuf-devel cmake rpm-build createrepo yum-utils pkgconf boost-devel protobuf-lite-devel systemd-libs
       ```
@@ -249,7 +255,7 @@ You can find the tools and libraries generated in the `build/linux` directory.
   $ make
 ```
 - To build the Intel(R) SGX PSW installer, enter the following command:
-  * On Ubuntu 18.04, Ubuntu 20.04, Ubuntu 22.04 and Debian 10:
+  * On Ubuntu 20.04, Ubuntu 22.04 and Debian 10:
    ```
   $ make deb_psw_pkg
   ```
@@ -262,7 +268,7 @@ You can find the tools and libraries generated in the `build/linux` directory.
   ```
   $ make deb_psw_pkg DEBUG=1
   ```
-  * On Red Hat Enterprise Linux 8.6, CentOS Stream 8, CentOS 8.3, Anolis OS 8.6 and SUSE Linux Enterprise Server 15.4:
+  * On Red Hat Enterprise Linux 9.2, CentOS Stream 9, CentOS 8.3, Anolis OS 8.6 and SUSE Linux Enterprise Server 15.4:
   ```
   $ make rpm_psw_pkg
   ```
@@ -282,10 +288,6 @@ You can find the tools and libraries generated in the `build/linux` directory.
     **Note**: The above command builds the local package repository. If you want to use it, you need to add it to the system repository configuration. The local package repository is not signed, you need to trust it for the purpose of development.
 
 - To add the local Debian package repository to the system repository configuration, append the following line to /etc/apt/sources.list. You need to replace PATH_TO_LOCAL_REPO with the proper path on your system:
-  * On Ubuntu 18.04:
-  ```
-  deb [trusted=yes arch=amd64] file:/PATH_TO_LOCAL_REPO bionic main
-  ```
   * On Ubuntu 20.04:
   ```
   deb [trusted=yes arch=amd64] file:/PATH_TO_LOCAL_REPO focal main
@@ -312,7 +314,7 @@ You can find the tools and libraries generated in the `build/linux` directory.
   **Note**: The above command builds the local package repository. If you want to use it, you need to add it to the system repository configuration. Since the local package repository is not signed with GPG, you should ignore the gpgcheck when installing the packages.
 
 - To add the local RPM package repository to the system repository configuration, you can use the following command. You need to replace PATH_TO_LOCAL_REPO with the proper path on your system:
-  * On Red Hat Enterprise Linux 8.6, CentOS Stream 8, CentOS 8.3, Anolis OS 8.6:
+  * On Red Hat Enterprise Linux 9.2, CentOS Stream 9, CentOS 8.3, Anolis OS 8.6:
   ```
   $ sudo yum-config-manager --add-repo file://PATH_TO_LOCAL_REPO
   ```
@@ -321,11 +323,11 @@ You can find the tools and libraries generated in the `build/linux` directory.
   $ sudo zypper addrepo PATH_TO_LOCAL_REPO LOCAL_REPO_ALIAS
   ```
 - To ignore the gpgcheck when you install the package, enter the following command:
-  * On Red Hat Enterprise Linux 8.6, CentOS Stream 8, CentOS 8.3, Anolis OS 8.6:
+  * On Red Hat Enterprise Linux 9.2, CentOS Stream 9, CentOS 8.3, Anolis OS 8.6:
   ```
   $ sudo yum --nogpgcheck install <package>
   ```
-  * On SUSE Linux Enterprise Server 15.5:
+  * On SUSE Linux Enterprise Server 15.4:
   ```
   $ sudo zypper --no-gpg-checks install <package>
   ```
@@ -334,28 +336,31 @@ Install the Intel(R) SGX SDK
 ------------------------
 ### Prerequisites
 - Ensure that you have one of the following operating systems:
-  * Ubuntu\* 18.04 LTS Desktop 64bits
-  * Ubuntu\* 18.04 LTS Server 64bits
   * Ubuntu\* 20.04 LTS Desktop 64bits
   * Ubuntu\* 20.04 LTS Server 64bits
   * Ubuntu\* 22.04 LTS Server 64bits
-  * Red Hat Enterprise Linux Server release 8.6 64bits
-  * CentOS Stream 8 64bits
+  * Red Hat Enterprise Linux Server release 9.2 64bits
+  * CentOS Stream 9 64bits
   * CentOS 8.3 64bits
   * SUSE Linux Enterprise Server 15.4 64bits
   * Anolis OS 8.6 64bits
   * Debian 10 64bits
 - Use the following command to install the required tool to use Intel(R) SGX SDK:
-  * On Ubuntu 18.04 and Debian 10:
+  * On Debian 10:
   ```
     $ sudo apt-get install build-essential python3
-    $ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 1
+    $ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3
   ```
    * On Ubuntu 20.04 and Ubuntu 22.04:
   ```
     $ sudo apt-get install build-essential python-is-python3
   ```
-  * On Red Hat Enterprise Linux 8.6, CentOS Stream 8, CentOS 8.3 and Anolis OS 8.6:
+  * On Red Hat Enterprise Linux 9.2 and CentOS Stream 9:
+  ```
+     $ sudo yum groupinstall 'Development Tools'
+     $ sudo yum install python3
+  ```
+  * On CentOS 8.3 and Anolis OS 8.6:
   ```
      $ sudo yum groupinstall 'Development Tools'
      $ sudo yum install python3
@@ -421,13 +426,11 @@ Install the Intel(R) SGX PSW
 ----------------------------
 ### Prerequisites
 - Ensure that you have one of the following operating systems:
-  * Ubuntu\* 18.04 LTS Desktop 64bits
-  * Ubuntu\* 18.04 LTS Server 64bits
   * Ubuntu\* 20.04 LTS Desktop 64bits
   * Ubuntu\* 20.04 LTS Server 64bits
   * Ubuntu\* 22.04 LTS Server 64bits
-  * Red Hat Enterprise Linux Server release 8.6 64bits
-  * CentOS Stream 8 64bits
+  * Red Hat Enterprise Linux Server release 9.2 64bits
+  * CentOS Stream 9 64bits
   * CentOS 8.3 64bits
   * SUSE Linux Enterprise Server 15.4 64bits
   * Anolis OS 8.6 64bits
@@ -437,15 +440,19 @@ Install the Intel(R) SGX PSW
 - Configure the system with the **Intel SGX hardware enabled** option and install Intel(R) SGX driver in advance.
   See the earlier topic, *Build and Install the Intel(R) SGX Driver*, for information on how to install the Intel(R) SGX driver.
 - Install the library using the following command:
-  * On Ubuntu 18.04, Ubuntu 20.04, Ubuntu 22.04 and Debian 10:
+  * On Ubuntu 20.04, Ubuntu 22.04 and Debian 10:
   ```
     $ sudo apt-get install libssl-dev libcurl4-openssl-dev libprotobuf-dev
   ```
-  * On Red Hat Enterprise Linux 8.6:
+  * On Red Hat Enterprise Linux 9.2:
   ```
     $ sudo yum install openssl-devel libcurl-devel protobuf-devel
   ```
-  * On CentOS Stream 8 and CentOS 8.3:
+  * On CentOS Stream 9:
+  ```
+    $ sudo dnf install libcurl-devel protobuf-devel
+  ```
+  * On CentOS 8.3:
   ```
     $ sudo dnf --enablerepo=powertools install libcurl-devel protobuf-devel
   ```
@@ -463,7 +470,7 @@ The SGX PSW provides 3 services: launch, EPID-based attestation, and algorithm a
 
 #### Using the local repo(recommended)
 
-|   |Ubuntu 18.04, Ubuntu 20.04, Ubuntu 22.04 and Debian 10|Red Hat Enterprise Linux 8.6, CentOS Stream 8 and CentOS 8.3| SUSE Linux Enterprise Server 15|
+|   |Ubuntu 20.04, Ubuntu 22.04 and Debian 10|Red Hat Enterprise Linux 9.2, CentOS Stream 9, CentOS 8.3 and Anolis OS 8.6| SUSE Linux Enterprise Server 15|
 | ------------ | ------------ | ------------ | ------------ |
 |launch service |apt-get install libsgx-launch libsgx-urts|yum install libsgx-launch libsgx-urts|zypper install libsgx-launch libsgx-urts|
 |EPID-based attestation service|apt-get install libsgx-epid libsgx-urts|yum install libsgx-epid libsgx-urts|zypper install libsgx-epid libsgx-urts|
@@ -484,11 +491,11 @@ apt-get dist-upgrade -o Dpkg::Options::="--force-overwrite"
 ```
 #### Configure the installation
 Some packages are configured with recommended dependency on other packages that are not required for certain usage. For instance, the background daemon is not required for container usage. It will be installed by default, but you can drop it by using the additional option during the installation.
-* On Ubuntu 18.04, Ubuntu 20.04, Ubuntu 22.04 and Debian 10:
+* On Ubuntu 20.04, Ubuntu 22.04 and Debian 10:
 ```
   --no-install-recommends
 ```
-* On Red Hat Enterprise Linux 8.6, CentOS Stream 8, CentOS 8.3 and Anolis OS 8.6:
+* On Red Hat Enterprise Linux 9.2, CentOS Stream 9, CentOS 8.3 and Anolis OS 8.6:
 ```
   --setopt=install_weak_deps=False
 ```

diff --git a/SampleCode/Cxx17SGXDemo/Enclave/TrustedLibrary/Libcxx.cpp b/SampleCode/Cxx17SGXDemo/Enclave/TrustedLibrary/Libcxx.cpp
@@ -846,7 +846,7 @@ void ecall_cxx17_gcd_lcm() {
 
 void print_map(std::string_view comment, const auto& data)
 {
-    printf("%s", comment);
+    printf("%s", comment.data());
     for (auto [k, v] : data)
         printf("  %d(%c)", k, v);
     printf("\n");
@@ -967,7 +967,7 @@ void ecall_cxx17_string_view() {
 
     for (int y{}, p{}; y != 3; ++y, p = ((p + 1) % 4)) {
         for (int x{}; x != 16; ++x)
-            printf("%s", unicode[p]);
+            printf("%s", unicode[p].data());
         printf("\n");
     }
 }

diff --git a/SampleCode/LocalAttestation/App/UntrustedEnclaveMessageExchange.cpp b/SampleCode/LocalAttestation/App/UntrustedEnclaveMessageExchange.cpp
@@ -108,7 +108,7 @@ ATTESTATION_STATUS send_request_ocall(uint32_t session_id, secure_message_t* req
 ATTESTATION_STATUS end_session_ocall(uint32_t session_id)
 {
 	sgx_status_t ret;
-	uint32_t retcode;
+        uint32_t retcode = (uint32_t) INVALID_SESSION;
 
 	ret = end_session(responder_enclave_id, &retcode, session_id);
 	if (ret != SGX_SUCCESS || retcode != SGX_SUCCESS)

diff --git a/SampleCode/LocalAttestation/AppResponder/CPTask.cpp b/SampleCode/LocalAttestation/AppResponder/CPTask.cpp
@@ -258,6 +258,7 @@ int process_close_req(int clientfd, SESSION_CLOSE_REQ * close_req)
         return -1;
 
     // send back response
+    memset(&close_ack, 0, sizeof(FIFO_MSG));
     close_ack.header.type = FIFO_DH_CLOSE_RESP;
     close_ack.header.size = 0;
 

diff --git a/SampleCode/LocalAttestation/EnclaveInitiator/EnclaveMessageExchange.cpp b/SampleCode/LocalAttestation/EnclaveInitiator/EnclaveMessageExchange.cpp
@@ -317,7 +317,7 @@ ATTESTATION_STATUS send_request_receive_response(dh_session_t *session_info,
 ATTESTATION_STATUS close_session(dh_session_t *session_info)
 {
     sgx_status_t status;
-    uint32_t retstatus;
+    uint32_t retstatus = (uint32_t) ATTESTATION_SE_ERROR;
 
     if(!session_info)
     {

diff --git a/SampleCode/SampleAEXNotify/Enclave/Enclave.cpp b/SampleCode/SampleAEXNotify/Enclave/Enclave.cpp
@@ -90,12 +90,10 @@ void count_powers_of_two_with_aex(uint64_t low, uint64_t high, uint32_t* count,
     sgx_aex_mitigation_node_t node;
 
     sgx_register_aex_handler(&node, my_aex_notify_handler, (const void*)args);
-    sgx_set_ssa_aexnotify(1);
 
    const uint32_t local_count = count_powers_of_two(low,high);
    *count = local_count;
 
-   sgx_set_ssa_aexnotify(0);
    sgx_unregister_aex_handler(my_aex_notify_handler);
 
    *aex_count = g_aex_count;

diff --git a/SampleCode/SampleAttestedTLS/Makefile b/SampleCode/SampleAttestedTLS/Makefile
@@ -51,8 +51,9 @@ build: $(SGXSSL_HEADER_CHECK)
 	$(MAKE) -C client
 	$(MAKE) -C non_enc_client
 
-tdx: $(SGXSSL_HEADER_CHECK)
+tdx:
 	$(MAKE) -C server_tdx
+	$(MAKE) -C non_enc_client
 
 clean:
 	$(MAKE) -C server clean

diff --git a/SampleCode/SampleAttestedTLS/client/enc/openssl_client.cpp b/SampleCode/SampleAttestedTLS/client/enc/openssl_client.cpp
@@ -83,7 +83,8 @@ int communicate_with_server(SSL* ssl)
         if (error == SSL_ERROR_WANT_WRITE)
             continue;
         t_print(TLS_CLIENT "Failed! SSL_write returned %d\n", error);
-        ret = bytes_written;
+        if (bytes_written == 0) ret = -1;
+        else ret = bytes_written;
         goto done;
     }
 
@@ -103,12 +104,16 @@ int communicate_with_server(SSL* ssl)
                 continue;
 
             t_print(TLS_CLIENT "Failed! SSL_read returned error=%d\n", error);
-            ret = bytes_read;
+            if (bytes_read == 0) ret = -1;
+            else ret = bytes_read;
             break;
         }
 
         t_print(TLS_CLIENT " %d bytes read\n", bytes_read);
         // check to to see if received payload is expected
+        // Note that if you just want to use client here but server from other 
+        // applications, you need to ignore this check, SERVER_PAYLOAD_SIZE 
+        // need to be adjusted. 
         if ((bytes_read != SERVER_PAYLOAD_SIZE) ||
             (memcmp(SERVER_PAYLOAD, buf, bytes_read) != 0))
         {
@@ -136,7 +141,7 @@ int communicate_with_server(SSL* ssl)
 int create_socket(char* server_name, char* server_port)
 {
     int sockfd = -1;
-	struct sockaddr_in dest_sock;
+    struct sockaddr_in dest_sock;
     int res = -1;
 
     sockfd = socket(AF_INET, SOCK_STREAM, 0);
@@ -204,14 +209,14 @@ int launch_tls_client(char* server_name, char* server_port)
 
     // specify the verify_callback for custom verification
     SSL_CTX_set_verify(ssl_client_ctx, SSL_VERIFY_PEER, &verify_callback);
-	t_print(TLS_CLIENT "load cert and key\n");
+    t_print(TLS_CLIENT "load cert and key\n");
     if (load_tls_certificates_and_keys(ssl_client_ctx, cert, pkey) != 0)
     {
         t_print(TLS_CLIENT
                " unable to load certificate and private key on the client\n");
         goto done;
     }
-	
+
     if ((ssl_session = SSL_new(ssl_client_ctx)) == nullptr)
     {
         t_print(TLS_CLIENT