Merge pull request #192 from microsoft/feature/developer-documentation

📝. Developer Documentation

README.md

@@ -14,27 +14,27 @@ ElectionGuard is always improving. To keep up with the latest, check our **[offi
 
 ### Documentation
 
-This repository is a living document to help everyone interact with ElectionGuard. The [official ElectionGuard site][election-guard-official-page] is built using the `/docs` folder and [mkdocs][mkdocs-official-site] with [mkdocs-material][material-mkdocs]. Ensure you have the python 3.8 or newer installed and run `make` to install the dependencies and start the site.
+This repository is a living document to help everyone interact with ElectionGuard. The [official ElectionGuard site][election-guard-official-page] is built using the `/docs` folder and [mkdocs][mkdocs-official-site] with [mkdocs-material][material-mkdocs]. Ensure you have the Python 3.8 or newer installed and run `make` to install the dependencies and start the site.
 
 ### Python
 
-A core component of ElectionGuard implemented in python which includes ballot encryption, decryption, key generation, and tallying.
+An ElectionGuard Core implemented in Python which includes ballot encryption, decryption, key generation, and tallying.
 
 [📁 Source][election-guard-python-source] |
 [📦 Package][election-guard-python-package] |
 [📝 Documentation][election-guard-python-documentation]
 
 ### C ++
 
-A subset of the ElectionGuard SDK implemented in C++ to support ballot encryption.
+An ElectionGuard Core implemented in C++ to support ballot encryption.
 
 [📁 Source][election-guard-cpp-source-code] |
 [📦 Package][election-guard-cpp-package] |
 [📝 Documentation][election-guard-cpp-documentation]
 
 ### Web API
 
-A Web API that wraps the python package to perform ballot encryption, casting, spoiling, and tallying.
+A Web API that wraps the Python package to perform ballot encryption, casting, spoiling, and tallying.
 
 [📁 Source][election-guard-web-api-source] | 
 [🐳 Docker][election-guard-web-api-docker] | 

docs/basics/steps/4_Decryption.md

@@ -2,7 +2,7 @@
 
 At the conclusion of voting, all of the cast ballots are published in their encrypted form in the election record together with the proofs that the ballots are well-formed. Additionally, all of the encryptions of each option are homomorphically-combined to form an encryption of the total number of times that each option was selected. The homomorphically-combined encryptions are decrypted to generate the election tally. Individual cast ballots are not decrypted. Individual spoiled ballots are decrypted and the plaintext values are published along with the encrypted representations and the proofs.
 
-In order to decrypt the homomorphically-combined encryption of each selection, each Guardian participating in the decryption must compute a specific Decryption Share of the decryption. It is preferable that all guardians be present for decryption, however in the event that guardians cannot be present, Electionguard includes a mechanism to decrypt with the Quorum of Guardians.
+In order to decrypt the homomorphically-combined encryption of each selection, each Guardian participating in the decryption must compute a specific Decryption Share of the decryption. It is preferable that all guardians be present for decryption, however in the event that guardians cannot be present, ElectionGuard includes a mechanism to decrypt with the Quorum of Guardians.
 
 ![Encrypt][encrypt-image]
 

docs/develop/Applications.md

@@ -0,0 +1,70 @@
+# Applications
+
+There are many different paths for creating applications that work using ElectionGuard. For many of these paths, there are reference implementations that can be forked or used as examples or used as packages. Many reference implementations include packaging or containerization for easy consumption. Below are some examples of paths a developer could take when developing using ElectionGuard. This list is by no means complete and some of these paths can be combined for different applications and use cases. 
+
+If you develop an app for ElectionGuard, we suggest you [utilize our badges][badges] to indicate which version of the specification your application targets. 
+
+## Paths
+
+### ElectionGuard Core
+
+An ElectionGuard Core is an application that implements the base level [features] such as Ballot Encryption by implementing them according to the specification. The internal examples of this are the Python and C++ reference implementations. There is a community example of this with the Java port done by the community. 
+
+### Ballot Marking
+
+Ballot marking is the process of recording a voter's [selections] on a ballot. A ballot marking app displays the contests from the manifest according to an individuals voter's ballot style and language. The voter then votes and the [selections] are recorded on a [plaintext ballot][plaintext-ballot]. 
+
+### Ballot Encryption
+
+Ballot encryption is the process of converting voter [selections] into data such that it cannot be read by unauthorized parties. A ballot encryption app provides this fundamental functionality. The app functions as a tool that encrypts a [plaintext ballot][plaintext-ballot] into a encrypted ballot aka [ciphertext ballot][ciphertext-ballot]. 
+
+Two quick ways to start are by using the ElectionGuard [encryption nuget package][encryption nuget package] or or [python package][python package].
+
+### Ballot Box
+
+A ballot box app takes the encrypted ballot of a voter and allows a user to submit the ballot as [cast][cast-ballot] or [spoil][spoiled-ballot] the ballot. A submitted cast ballot is aggregated into the tally. A submitted spoiled ballot is added to the list of spoiled ballots. 
+
+### Administer Election
+
+Administering an election for ElectionGuard usually requires the following steps. An example of this is available in [electionguard-ui repository][admin app].
+
+1. Setup Election - An election should be setup with a manifest and selected guardians
+2. Key Ceremony - A key ceremony should be run that results in the encryption key that can be used for the ballots.
+3. Tally Ceremony - A tally ceremony when run will decrypt the tally and any spoiled ballots
+4. Election Record - An election record should be retrievable at the end of the election for publication and verification purposes. 
+
+A quick way to start is to use the [electionguard api-client][api client] and the [electionguard-python-api][api].
+
+### Publish Election
+
+The results of election can be displayed after an election is closed. An example of this is available in [electionguard-ui repository][result app]. This application should contain three pieces. 
+
+1. **Results** - Using decrypted tally and manifest, the tallied results should be displayed. 
+2. **Ballot Confirmation** - Confirm a ballot is in the Election Record.
+    - Confirm Cast - Confirm a ballot was cast and included in tally
+    - Challenge Ballot - Confirm a ballot was spoiled and display decrypted spoiled ballot to user
+3. **Download Election Record** - The Election Record should be downloadable for the public. 
+
+### Verify Election
+
+A verifier app verifies an election is true and accurate by using the publicly published information about a completed election. At the close of an election using ElectionGuard, an election record should be created and made public. ElectionGuard encourages third parties to create their own verifiers and help verify election records. A basic example of a verifier can be found within the [electionguard-python repository][verifier], but  verifiers should aim to be as thorough as possible. After consuming the files in an election record, verifiers can validate everything from the key ceremony to the decryption since the election is end to end verifiable. 
+
+<!--Links-->
+[selections]: ../../overview/Glossary/#selection
+[plaintext-ballot]: ../../overview/Glossary/#plaintext-ballot
+[ciphertext-ballot]: ../../overview/Glossary/#ciphertext-ballot
+[cast-ballot]: ../../overview/Glossary/#cast-ballot
+[spoiled-ballot]: ../../overview/Glossary/#spoiled-ballot
+[badges]: ../Badges
+[features]: ../../overview/Features
+[admin app]: https://github.com/microsoft/electionguard-ui/tree/main/packages/admin-app
+[result app ]: https://github.com/microsoft/electionguard-ui/tree/main/packages/result-app
+[api client]: https://github.com/microsoft/electionguard-ui/tree/main/packages/api-client
+[api]:https://github.com/microsoft/electionguard-api-python
+[election record]: Election_Record.md
+[verifier]: https://github.com/microsoft/electionguard-python/tree/main/src/electionguard_verify
+[encryption nuget package]: https://www.nuget.org/packages/ElectionGuard.Encryption/
+[python package]: https://pypi.org/project/electionguard/
+
+
+

docs/develop/Approach.md

@@ -0,0 +1,25 @@
+# Approach
+
+For the ElectionGuard repositories, these are the concepts that guide the approach for developers and writers. 
+
+## ✅ Simplicity
+
+Simplicity is the first and foremost goal of the code. The intent is for others to be able to easily transliterate the code to any other programming language with little more than structures and functions. This simplicity applies to all aspects of the code design, including naming.
+
+## ✅ Transparency
+
+ElectionGuard is open source first. Our code, our issues and even our Continuous Integration (CI) is designed to be visible to the public. This is to allow others to both be able to trust the code, learn, and hopefully help improve ElectionGuard. 
+
+GitHub tools have been our primary tool to maintain transparency. Our cards are all GitHub issues. Our continuous integration uses GitHub Workflows to be out in the open. 
+
+## ✅ Readability
+
+Whether developing or writing for ElectionGuard, the goal is to create easy to read and understand text. We will always be improving in this area, but the goal is to make the system easy to grasp for not only developers, but for the public. This is the primary reason that Python was chosen as the primary reference implementation. Understanding the code and the written word is essential for confidence.
+
+## ✅ Testability
+
+The goal is to maximize code coverage and testing on the reference implementations. Tests are essential to the repositories for both providing proof the code functions but also to provide an understanding of how the code works. This means writing code that can be tested and writing the accompanying tests.
+
+## ✅ Extendable and Interpretable
+
+The reference implementations are intentionally general-purpose to support the different use cases of voting systems. Different projects may wish to use different pieces and parts of each repository. 

docs/develop/Architecture.md

@@ -0,0 +1,81 @@
+# Architecture
+
+There are five official repositories in the **ElectionGuard SDK**.
+
+- [ElectionGuard Specification][election-guard-specification]
+- [ElectionGuard Python][election-guard-python-source]
+- [ElectionGuard C++][election-guard-cpp-source-code]
+- [ElectionGuard Python API][election-guard-web-api-source]
+- [ElectionGuard React UI][election-guard-ui-source]
+
+The **ElectionGuard Specification** repository focuses on explaining ElectionGuard as a whole including documentation and the specification itself. The other repositories are all reference implementations. These reference implementations are designed to demonstrate to developers how to implement a system using the ElectionGuard Specification. In depth descriptions of the repositories are detailed in the [repositories][repositories] page.
+
+## Cores
+
+An **ElectionGuard Core** is a code base that implements fundamental features of the ElectionGuard Specification (such as ballot encryption) to act as a core or central integration. ElectionGuard provides two official reference implementations of Cores: Python and C++. The **Python Core** (`electionguard-python`) is full featured including all elements of the specification to showcase ElectionGuard to its fullest. The **C++ Core** (`electionguard-cpp`) is designed with a focus on encrypting ballots. Its goal is to allow developers to use the ElectionGuard encryption on encryption devices.
+
+| Feature           | `electionguard-python` |  `electionguard-cpp` |
+| ----------------- | :--------------------: | :------------------: |
+| Ballot Encryption | :material-check:       | :material-check:     |
+| Ballot Decryption | :material-check:       |                      |
+| Key Generation    | :material-check:       |                      |
+| Key Ceremony      | :material-check:       |                      |
+| Tally Ceremony    | :material-check:       |                      |
+| Tally Decryption  | :material-check:       |                      |
+
+## Paths
+
+The specification serves as the guide to implement any ElectionGuard Core. For the official reference implementations, there are two main paths within the reference implementations of ElectionGuard. One path starts with the Python Core (`electionguard-python`) and the other starts with C++ Core (`electionguard-cpp`). These paths can used together, but have different feature sets.
+
+```mermaid
+flowchart TD
+    specification["Specification"] -.-> |Python|python["Full Featured Core"];
+    specification -.-> |C++|cpp["Encryption Core"];
+    python --> api["API"];
+    api --> ui["User Interface"];
+```
+
+This chart shows the relationship of the specification to the reference implementations by repository name. This paints the picture of how the repositories are related to each other.  
+
+```mermaid
+flowchart TD
+    spec["electionguard"] -.-> |Python|e-python["electionguard-python"];
+    spec["electionguard"] -.-> |C++|e-cpp["electionguard-cpp"];
+    e-python --> web-api["electionguard-api-python"];
+    web-api --> e-ui["electionguard-ui"];
+```
+
+
+### Python Path
+
+The Python path is full featured including all elements of the specification to showcase ElectionGuard to its fullest. There is a full stack provided for Python so there are many different approaches and entry points a developer could use when starting here. The following suggested approaches can be used to develop apps like those listed in the [applications][application-paths] list.
+
+#### Suggested Approaches
+
+- Build a custom Python app with `electionguard-python` package
+
+- Build a custom web app and use `electionguard-api-python` as an API
+
+- Build a custom web app with `electionguard-ui`'s api client and use `electionguard-api-python` as an API.
+
+- Build a custom web app from fork of `electionguard-ui`'s web apps and use `electionguard-api-python` as an API
+
+### C++ Path
+
+For the C++ path, the included reference implementation of a C++ Core and resulting packages are currently designed with a focus on encrypting ballots. Its goal is to allow developers to use ElectionGuard encryption on encryption devices. The following suggested approaches can be used to develop a ballot encryption app or integration as described in the [applications][application-encryption] list.
+
+#### Suggested Approaches
+
+- Build custom app linking C++ library
+
+- Build custom app and use NuGet Package
+
+
+[repositories]: ../Repositories
+[application-paths]: ../Applications/#paths
+[application-encryption]: ../Applications/#ballot-encryption
+[election-guard-specification]: https://github.com/microsoft/electionguard "Election Guard Github"
+[election-guard-python-source]: https://github.com/microsoft/electionguard-python "Election Guard Python source code"
+[election-guard-cpp-source-code]: https://github.com/microsoft/electionguard-cpp/ "Election Guard C++ source code"
+[election-guard-web-api-source]: https://github.com/microsoft/electionguard-api-python "Election Guard Web API source code"
+[election-guard-ui-source]: https://github.com/microsoft/electionguard-ui "Election Guard UI source code"

docs/spec/Data_Format.md → docs/develop/Data_Format.md

@@ -1,13 +1,15 @@
 # Data Format
 
-!!! warning "Work in Progress"
-    This is a work in progress. Feel free to contribute.
+ElectionGuard expects data to be formatted in a particular way. The expectation is to keep this format versioned alongside the specification. The data format will be defined here to indicate to developers what they should be expecting. 
 
+Keep in mind that not all serialized files should be public. The [election record][election-record] includes the list of all data formats that should be public. [Sample data][sample-data] for the election is also available for developers. This will include both what is in the Election Record, but also some private data only available for testing. 
 
 ## Schema ![Version 0.95.0][shield-green-0.95.0]
 
-These are [JSON schema][json-schema] files that can be used to verify the schema of the setup files like the manifest pre-election and the election record files post-election.
+Currently, JSON is used as the primary serialization format of the artifacts from an election. These are [JSON schema][json-schema] files that can be used to verify the schema of the setup files like the manifest pre-election and the election record files post-election. This represents the full list of items that can be serialized / deserialized, but not all elections will use every file type. 
 
+!!! warning "Work in Progress"
+    This is a work in progress. Feel free to contribute.
 
 ### Election
 
@@ -23,8 +25,9 @@ These are [JSON schema][json-schema] files that can be used to verify the schema
 ### Ballot
 
 - [Ballot][plaintext_ballot] / `PlaintextBallot` - Plaintext ballot
+- [Compact Ballot][compact_ballot] / `CompactBallot` - Compact ballot
 - [Encrypted Ballot][ciphertext_ballot] / `CiphertextBallot` - Encrypted ballot
-- [Submitted Ballot][submitted_ballot] / `SubmittedBallot` -  Cast or spoiled/challenged ballot
+- [Submitted Ballot][submitted_ballot] / `SubmittedBallot` -  Submitted cast or spoiled ballot
 
 ### Tally
 - [Tally][plaintext_tally] / `PlaintextTally` - Plaintext tally
@@ -35,6 +38,8 @@ These are [JSON schema][json-schema] files that can be used to verify the schema
 [shield-green-0.95.0]: https://img.shields.io/badge/🗳%20ElectionGuard%20Specification-0.95.0-green
 [json-schema]: https://json-schema.org/specification.html "Json Schema Specification"
 
+[election-record]: ../Election_Record
+[election-record]: ../Sample_Data
 [manifest]: https://github.com/microsoft/electionguard/blob/main/data/0.95.0/schema/manifest.schema.json "Manifest Json Schema"
 [ciphertext_election_context]: https://github.com/microsoft/electionguard/blob/main/data/0.95.0/schema/ciphertext_election_context.schema.json "Ciphertext Election Context Json Schema"
 [election_constants]: https://github.com/microsoft/electionguard/blob/main/data/0.95.0/schema/election_constants.schema.json "Election Constants Json Schema"

docs/develop/Election_Record.md

@@ -0,0 +1,44 @@
+### Election Record
+
+The Election Record is the group of files required at the end of the election to verify the election and is intended to be posted publicly. It should not contain any sensitive data (eg: no guardian private keys) in the record. These include the following:
+
+- Encryption Devices Information
+- Guardian Records
+- Decrypted Spoiled Ballots
+- Encrypted Submitted Ballots
+- Lagrange Coefficients Record
+- Election Constants
+- Election Context
+- Encrypted Tally
+- Manifest
+- Decrypted Tally
+
+## Folder Structure
+
+The Election Record should be expected to be a zip folder containing the following information. 
+
+```
+📂 record
+--- 📁 encryption_devices
+------- 📄 device_id_1.json
+------- 📄 device_id_2.json
+------- ...
+--- 📁 guardians
+------- 📄 guardian_id_1.json
+------- 📄 guardian_id_2.json
+------- ...
+--- 📁 spoiled_ballots
+------- 📄 spoiled_ballot_id_3.json
+------- 📄 spoiled_ballot_id_4.json
+------- ...
+--- 📁 submitted_ballots
+------- 📄 submitted_ballot_id_1.json
+------- 📄 submitted_ballot_id_2.json
+------- ...
+--- 📄 coefficients.json
+--- 📄 constants.json
+--- 📄 context.json
+--- 📄 encrypted_tally.json
+--- 📄 manifest.json
+--- 📄 tally.json
+```