Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs]: Update State section #1029

Merged
merged 18 commits into from
Jan 9, 2025
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/architecture/notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ Upon successful verification of the transaction:
1. The Miden operator records the note’s nullifier as “consumed” in the nullifier database.
2. The note’s one-time claim is thus extinguished, preventing reuse.

#### Note recipient - restricting consumption
#### Note recipient restricting consumption
phklive marked this conversation as resolved.
Show resolved Hide resolved

Consumption of a note can be restricted to certain accounts or entities. For instance, the P2ID and P2IDR note scripts target a specific account ID. Alternatively, Miden defines a `RECIPIENT` (represented as a `Word`) computed as:

Expand All @@ -118,7 +118,7 @@ The [transaction prologue](transactions/kernel.md) requires all necessary data t

For a practical example, refer to the [SWAP note script](https://github.com/0xPolygonMiden/miden-base/blob/main/miden-lib/asm/note_scripts/SWAP.masm), where the `RECIPIENT` ensures that only a defined target can consume the swapped asset.

#### Note nullifier - ensuring private consumption
#### Note nullifier ensuring private consumption
phklive marked this conversation as resolved.
Show resolved Hide resolved

The note nullifier, computed as:

Expand Down
112 changes: 66 additions & 46 deletions docs/architecture/state.md
Original file line number Diff line number Diff line change
@@ -1,91 +1,111 @@
Miden rollup state describes the current condition of all accounts and notes; i.e. what is currently the case.
# State

phklive marked this conversation as resolved.
Show resolved Hide resolved
As the state model uses concurrent off-chain state, Polygon Miden aims for private, and parallel transaction execution and state bloat minimization.
## What is the purpose of state?

Miden's goals include:
Miden enables secure, private, and scalable transaction execution. By employing a concurrent state model with local execution and proving, Miden achieves three primary objectives: preserving privacy, supporting parallel transactions, and minimizing on-chain data storage (state bloat).

phklive marked this conversation as resolved.
Show resolved Hide resolved
* Notes and nullifiers to ensure privacy of note consumption.
* Flexible data storage for users who can store their data off-chain or with the network.
* Parallel transactions executed concurrently by distinct actors.
* Concurrent state model that allows block production without knowing the full state.
Miden’s state model focuses on:

Privacy is enforced by a UTXO-like state model consisting of notes and nullifiers combined with off-chain execution using zero-knowledge proofs.
- **Privacy:**
By using notes and nullifiers, Miden ensures that value transfers remain confidential. Zero-knowledge proofs allow users to prove correctness without revealing sensitive information.

phklive marked this conversation as resolved.
Show resolved Hide resolved
phklive marked this conversation as resolved.
Show resolved Hide resolved
State bloat describes the ever growing state stored in blockchain nodes. Polygon Miden addresses this challenge via its state model that enables concurrent off-chain execution and off-chain storage. Simply put, users can store their own data locally which reduces the burden on the network while integrity is ensured using zero-knowledge.
- **Flexible data storage:**
Users can store data privately on their own devices or within the network. This approach reduces reliance on the network for data availability, helps maintain user sovereignty, and minimizes unnecessary on-chain storage.

## State components
- **Concurrency:**
Multiple transactions can be processed concurrently by distinct actors which improves throughput and efficiency.

phklive marked this conversation as resolved.
Show resolved Hide resolved
phklive marked this conversation as resolved.
Show resolved Hide resolved
Miden nodes maintain three databases to describe state:
Miden’s state model supports a private, secure, and high-throughput environment while also addressing the challenges of state bloat commonly associated with continuously growing blockchain states.

phklive marked this conversation as resolved.
Show resolved Hide resolved
phklive marked this conversation as resolved.
Show resolved Hide resolved
1. A database of accounts.
2. A database of notes.
3. A database of nullifiers for already consumed notes.
## What is state?

![Architecture core concepts](../img/architecture/state/state.png)
The state of the Miden rollup describes the current condition of all accounts and notes in the protocol; i.e., the current reality.

phklive marked this conversation as resolved.
Show resolved Hide resolved
## State model components

These databases are represented by authenticated data structures that enable easy proof of items added to or removed from a database. This ensures that the commitment to the database remains very small.
The Miden node maintains three databases to describe state:

Polygon Miden has two databases to capture the note states. The note database is append-only and stores all notes permanently. The nullifier database stores nullifiers that indicate that a note has been previously consumed. Separating note storage into these two databases gives Polygon Miden client-side proving and advanced privacy.
1. Accounts
2. Notes
3. Nullifiers

![Architecture core concepts](../img/architecture/state/state.png)

### Account database

The latest account states - and data for on-chain accounts - are recorded in a sparse Merkle tree which maps account IDs to account hashes, and account data if needed.
The accounts database stores the latest account states for public accounts or state commitments for private accounts.

phklive marked this conversation as resolved.
Show resolved Hide resolved
![Architecture core concepts](../img/architecture/state/account-db.png)

As described in the [accounts section](accounts.md), there are two types of accounts:

* Public accounts where all account data is stored on-chain.
* Private accounts where only the hashes of accounts are stored on-chain.
- **Public accounts:** where all account data is stored on-chain.
- **Private accounts:** where only the hash of the account is stored on-chain.

phklive marked this conversation as resolved.
Show resolved Hide resolved
Private accounts significantly reduce the storage overhead for nodes. A private account contributes only $40$ bytes to the global state ($8$ bytes account ID + $32$ bytes account hash). Or, said another way, 1 billion private accounts takes up only $40$ GB of state.
Private accounts significantly reduce storage overhead. A private account contributes only $40$ bytes to the global state ($8$ bytes for the account ID + $32$ bytes for the account hash). For example, 1 billion private accounts take up only $40$ GB of state.

phklive marked this conversation as resolved.
Show resolved Hide resolved
The storage contribution of a public account depends on the amount of data it stores.

phklive marked this conversation as resolved.
Show resolved Hide resolved
> **Warning**
> Losing the state of a private account means loss of funds in a similar manner as a loss of a private key - as the user won't be able to execute transactions. This problem can be mitigated by storing encrypted account state in the cloud or backing it up somewhere else. Unlike storing private keys in the cloud, this does not compromise privacy or account security.
>
> In the future we hope to enable encrypted accounts where the account data is stored on-chain but in an encrypted format. This is especially interesting for shared accounts like advanced multi-sig wallets.
> - In Miden, when the user is the custodian of their account state (in the case of a private account), losing this state amounts to losing their funds, similar to losing a private key.

### Note database

Notes are recorded in an append-only accumulator, a [Merkle Mountain Range](https://github.com/opentimestamps/opentimestamps-server/blob/master/doc/merkle-mountain-range.md). Each leaf is a block header which contains the commitment to all notes created in that block. The commitment is a Sparse Merkle Tree of all the notes in a block. The size of the Merkle Mountain Range grows logarithmically with the number of items in it.
As described in the [notes section](notes.md), there are two types of notes:
phklive marked this conversation as resolved.
Show resolved Hide resolved

- **Public notes:** where the entire note content is stored on-chain.
- **Private notes:** where only the note’s hash is stored on-chain.

Private notes greatly reduce storage requirements and thus result in lower fees. They add only $64$ bytes to the state ($32$ bytes when produced and $32$ bytes when consumed).

phklive marked this conversation as resolved.
Show resolved Hide resolved
At high throughput (e.g., 1K TPS), the note database could grow by about 1TB/year. However, only unconsumed public notes and enough information to construct membership proofs must be stored explicitly. Private notes, as well as consumed public notes, can be discarded. This solves the issue of infinitely growing note databases.

![Architecture core concepts](../img/architecture/state/note-db.png)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This picture is outdated: we don't actually have a separate notes MMR. Instead, we have the blockchain MMR where every leaf is a block header. Then, each block header contains a commitment to the notes created in that block.

As described in [the notes section](notes.md), there are two types of notes:
### Nullifier database

phklive marked this conversation as resolved.
Show resolved Hide resolved
* Public notes where the entire note content is recorded in the state.
* Private notes where only a note's hash is recorded in the state.
Each [note](notes.md) has an associated nullifier which enables the tracking of whether it's associated note has been consumed or not, preventing double-spending.

As with accounts, there is a strong incentive to use private notes as they result in lower fees. This is also beneficial to the network as a private note adds only $64$ bytes to the state ($32$ bytes when it is produced, and $32$ bytes when it is consumed).
To prove that a note has not been consumed, the operator must provide a Merkle path to the corresponding node and show that the node’s value is `0`. Since nullifiers are $32$ bytes each, the Sparse Merkle Tree height must be sufficient to represent all possible nullifiers. Operators must maintain the entire nullifier set to compute the new tree root after inserting new nullifiers.

phklive marked this conversation as resolved.
Show resolved Hide resolved
Using a Merkle Mountain Range (append-only accumulator) is important for two reasons:
> **Note**
> - Nullifiers in Miden break linkability between privately stored notes and their consumption details. To know the [note’s nullifier](notes.md#note-nullifier-ensuring-private-consumption), one must know the note’s data.

1. Membership witnesses (that a note exists in the database) against such an accumulator needs to be updated very infrequently.
2. Old membership witnesses can be extended to a new accumulator value, but this extension does not need to be done by the original witness holder.
![Architecture core concepts](../img/architecture/state/nullifier-db.png)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this diagram outdated? It states that nullifiers values are either 0/1 but they are the block number in which the note was consumed.
It also mentions the nullifier tree is a tiered sparse merkle tree, though right now it is just an SMT. Not sure right now if the plan is for it to be a tiered SMT in the future to support dropping older epochs.

Both of these properties are needed for supporting local transactions using client-side proofs and privacy. In an append-only data structure, witness data does not become stale when the data structure is updated. That means users can generate valid proofs even if they don’t have the latest state of this database; so there is no need to query the operator on a constantly changing state.
## Additional information
phklive marked this conversation as resolved.
Show resolved Hide resolved

However, the size of the note database does not grow indefinitely. Theoretically, at high tps, it grows very quickly: at $1$K TPS there are about $1$TB/year added to the database. But, only the unconsumed public notes, and enough info to construct membership proofs against them, need to be stored explicitly. Private notes, as well as public notes which have already been consumed, can be safely discarded. Such notes would still remain in the accumulator, but there is no need to store them explicitly as the append-only accumulator can be updated without knowing about all the items stored in it. This reduces actual storage requirements to a fraction of the database's nominal size.
### Public shared state

phklive marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of this section but I think it should be a part of a bigger section the describes how different users work with the state. Specifically, we could come up with different types of state:

  1. Self-custodied state - state managed by a single user.
  2. Federated state - state managed by a small number of users (e.g., under 100).
  3. Network-managed state - state managed by the network itself (this would be the "public shared state").
  4. Hybrids between the above (this could be a really cool thing).

Would be good to describe how these work and what are the tradeoffs between them.

### Nullifier database
In most blockchains, most smart contracts and decentralized applications (e.g., AAVE, Uniswap) need public shared state. Public shared state is also available on Miden and can be represented by the following example:

phklive marked this conversation as resolved.
Show resolved Hide resolved
Nullifiers are stored in a sparse Merkle tree, which maps [note nullifiers](notes.md#note-nullifier-to-ensure-private-consumption) to block numbers at which the nullifiers are inserted into the chain (or to `0` for nullifiers which haven't been recorded yet). Nullifiers provide information on whether a specific note has been consumed. The database allows proving that a given nullifier is not in the database.
![Public shared state](../img/architecture/state/public-shared-state.png)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The explanation of the diagram is duplicated: Once within the diagram itself and once below in the text. I think we should remove the explanation from the diagram. This way it can also be enlarged a little bit as some text (like the "tx" with numbers) is fairly small.
  • I would also remove the extra heading "Handling shared state" in the diagram. Other diagrams don't have this.
  • Can we write out "Acc" as "Account" and "Uni" as "Uniswap"?
  • Can we align the "note x" text with the bubble surrounding it and capitalize it? Ideally so that note and number are on the same line. Some of them are underlined with red markers. Would be great to remove that.

![Architecture core concepts](../img/architecture/state/nullifier-db.png)
In this diagram, multiple participants interact with a common, publicly accessible state. The figure illustrates how notes are created and consumed:

To prove that a note has not been consumed previously, the operator needs to provide a Merkle path to its node, and then show that the value in that node is `0`. In our case nullifiers are $32$ bytes each, and thus, the height of the Sparse Merkle Tree needs to be $256$.
1. **Independent Transactions Creating Notes (tx1 & tx2):**
Two separate users (Acc1 and Acc2) execute transactions independently:
- `tx1` produces **note 1**
- `tx2` produces **note 2**

To add new nullifiers to the database, operators need to maintain the entire nullifier set. Otherwise, they would not be able to compute the new root of the tree.
These transactions occur in parallel and do not rely on each other, allowing concurrent processing without contention.

> **Note**
> Nullifiers as constructed in Miden break linkability of privately stored notes and the information about the note's consumption. To know the [note's nullifier](notes.md#note-nullifier-to-ensure-private-consumption) one must know the note's data.
2. **Sequencing and Consuming Notes (tx3):**
The Miden node executes `tx3` against the shared account, consuming **notes 1 & 2** and producing **notes 3 & 4**. `tx3` is a network transaction executed by the sequencer. It merges independent contributions into a unified state update.

3. **Further Independent Transactions (tx4 & tx5):**
After the shared state is updated:
- `tx4` consumes **note 4**
- `tx5` consumes **note 5**

Both users can now interact with notes generated by the public account, continuing the cycle of state evolution.

In the future, when the network experiences a large number of transactions per second (TPS), there will be one tree per epoch (~3 months), and Miden nodes always store trees for at least two epochs. However, the roots of the old trees are still stored. If a user wants to consume a note that is more than $6$ months old, there must be a merkle path provided to the Miden Node for verification.
### State bloat minimization

## State bloat minimization
Miden nodes do not need to know the entire state to verify or produce new blocks. Rather than storing the full state data with the nodes, users keep their data locally, and the rollup stores only commitments to that data. While some contracts must remain publicly visible, this approach minimizes state bloat and preserves privacy, since nodes and other users only see hashes of user data.

phklive marked this conversation as resolved.
Show resolved Hide resolved
Operators don’t need to know the entire state to verify or produce a new block. No operator is required to store the entire state.
This ensures that the account and note databases remain manageable, even under sustained high usage.

At its core, the idea is simple: Instead of storing the full state data with the operators, the users store their data, and the rollup only keeps track of commitments to the data. At least for private accounts, some smart contracts need to be publicly visible. This minimizes state bloat—as the operator doesn’t need to store an ever-growing database, and provides privacy because all other users and the operator only see a hash of other users’ data.
## Conclusion

That way the account and note databases remain manageable, even at high usage for extended periods of time.
Miden’s state model lays the foundation for a scalable, privacy-preserving, and user-centric environment. By combining parallelizable execution, flexible data storage, and Zero-knowledge proofs that ensure integrity and confidentiality, Miden addresses many of the challenges of traditional blockchains. As a result, the network can handle high throughput, maintain manageable state sizes, and support a wide range of applications.
Binary file modified docs/img/architecture/state/account-db.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading