feat: delayed note authentication including notes metadata (#772)

CHANGELOG.md

@@ -4,28 +4,28 @@
 
 ### Enhancements
 
-- Created `get_serial_number` procedure to get the serial num of the currently processed note(#760)
-- Generalized `build_recipient_hash` procedure to build recipient hash for custom notes (#706)
-- Added ability for users to set the aux field when creating a note (#752)
-- Updated and improved [.pre-commit-config.yaml](.pre-commit-config.yaml) file (#748)
-- Fixed documentation and added `make doc` CI job (#746)
-- Made `DataStore` conditionally async using `winter-maybe-async` (#725)
-- Add `Option<NoteTag>`to `NoteFile` (#741).
 - Replaced `cargo-make` with just `make` for running tasks (#696).
 - [BREAKING] Introduce `OutputNote::Partial` variant (#698).
 - [BREAKING] Split `Account` struct constructor into `new()` and `from_parts()` (#699).
+- Generalized `build_recipient_hash` procedure to build recipient hash for custom notes (#706).
 - [BREAKING] Changed the encoding of inputs notes in the advice map for consumed notes (#707).
 - Created additional `emit` events for kernel related `.masm` procedures (#708).
 - Implemented `build_recipient_hash` procedure to build recipient hash for custom notes (#710).
 - Removed the `mock` crate in favor of having mock code behind the `testing` flag in remaining crates (#711).
 - [BREAKING] Created `auth` module for `TransactionAuthenticator` and other related objects (#714).
 - Added validation for the output stack to make sure it was properly cleaned (#717).
 - Added new `NoteFile` object to represent serialized notes (#721).
-- [BREAKING] Added support for delegated authenticated notes (#724).
+- Made `DataStore` conditionally async using `winter-maybe-async` (#725).
+- Changed note pointer from Memory `note_ptr` to `note_index` (#728).
 - [BREAKING] Changed rng to mutable reference in note creation functions (#733).
 - Added transaction IDs to the `Block` struct (#734).
 - [BREAKING] Replaced `ToNullifier` trait with `ToInputNoteCommitments`, which includes the `note_id` for delayed note authentication (#732).
-- Changed note pointer from Memory `note_ptr` to `note_index` (#728)
+- Added `Option<NoteTag>`to `NoteFile` (#741).
+- Fixed documentation and added `make doc` CI job (#746).
+- Updated and improved [.pre-commit-config.yaml](.pre-commit-config.yaml) file (#748).
+- Added ability for users to set the aux field when creating a note (#752).
+- Created `get_serial_number` procedure to get the serial num of the currently processed note (#760).
+- [BREAKING] Added support for input notes with delayed verification of inclusion proofs (#724, #732, #759, #770, #772).
 
 ## 0.3.1 (2024-06-12)
 * Replaced `cargo-make` with just `make` for running tasks (#696).

docs/architecture/transactions/procedures.md

@@ -50,9 +50,9 @@ To import the transaction procedures set `use.miden::tx` at the beginning of the
 | Procedure name           | Inputs           | Outputs     | Context | Description                                                                                                                                                                  |
 |--------------------------|------------------|-------------|---------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | `get_block_number`       | `[]`             | `[num]`     | account, note | <ul> <li>Returns the block number `num` of the last known block at the time of transaction execution. |
-| `get_block_hash`         | `[]`             | `[H]`       |  account, note | <ul> <li>Returns the block hash `H` of the last known block at the time of transaction execution.</li> </ul> |
-| `get_input_notes_hash`   | `[]`             | `[COM]`     |  account, note | <ul> <li>Returns the input notes hash `COM`. </li><li>This is computed as a sequential hash of (nullifier, script_root) tuples over all input notes. </li> </ul> |
-| `get_output_notes_hash`  | `[0, 0, 0, 0]`   | `[COM]`     |  account, note | <ul> <li>Returns the output notes hash `COM`. </li><li>This is computed as a sequential hash of (note_hash, note_metadata) tuples over all output notes. </li> </ul> |
+| `get_block_hash`         | `[]`             | `[H]`       | account, note | <ul> <li>Returns the block hash `H` of the last known block at the time of transaction execution.</li> </ul> |
+| `get_input_notes_hash`   | `[]`             | `[COM]`     | account, note | <ul> <li>Returns the input notes hash `COM`. </li><li>This is computed as a sequential hash of (nullifier, empty_word_or_note_hash) tuples over all input notes. The `empty_word_or_notes_hash` functions as a flag, if the value is set to zero, then the notes are authenticated by the transaction kernel. If the value is non-zero, then note authentication will be delayed to the batch/block kernel. The delayed authentication allows a transaction to consume a public note that is not yet included to a block. </li> </ul> |
+| `get_output_notes_hash`  | `[0, 0, 0, 0]`   | `[COM]`     | account, note | <ul> <li>Returns the output notes hash `COM`. </li><li>This is computed as a sequential hash of (note_id, note_metadata) tuples over all output notes. </li> </ul> |
 | `create_note`            | `[ASSET, tag, RECIPIENT]` | `[ptr]` | account | <ul> <li>Creates a new note and returns a pointer to the memory address at which the note is stored.</li><li> `ASSET` is the asset to be included in the note. </li><li>`tag` is the tag to be included in the note. `RECIPIENT` is the recipient of the note.</li><li> `ptr` is the pointer to the memory address at which the note is stored.</li> </ul> |
 
 

miden-lib/asm/kernels/transaction/api.masm

@@ -465,8 +465,8 @@ end
 
 #! Returns the input notes commitment.
 #!
-#! This is computed as a sequential hash of `(NULLIFIER, EMPTY_WORD_OR_NOTEID)` over all input notes. The
-#! data `EMPTY_WORD_OR_NOTEID` functions as a flag, if the value is set to zero, then the notes are
+#! This is computed as a sequential hash of `(NULLIFIER, EMPTY_WORD_OR_NOTE_HASH)` over all input notes. The
+#! data `EMPTY_WORD_OR_NOTE_HASH` functions as a flag, if the value is set to zero, then the notes are
 #! authenticated by the transaction kernel. If the value is non-zero, then note authentication will
 #! be delayed to the batch/block kernel. The delayed authentication allows a transaction to consume a
 #! public note that is not yet included to a block.
@@ -484,7 +484,7 @@ export.get_input_notes_commitment
     # => [COM]
 end
 
-#! Returns the output notes hash. This is computed as a sequential hash of (note_hash, note_metadata)
+#! Returns the output notes hash. This is computed as a sequential hash of (note_id, note_metadata)
 #! tuples over all output notes.
 #!
 #! Inputs: [0, 0, 0, 0]

miden-lib/asm/miden/kernels/tx/memory.masm

@@ -133,7 +133,7 @@ const.CONSUMED_NOTE_DATA_SECTION_OFFSET=1064960
 const.CONSUMED_NOTE_NUM_PTR=1048576
 
 # The offsets at which data of a consumed note is stored relative to the start of its data segment
-const.CONSUMED_NOTE_HASH_OFFSET=0
+const.CONSUMED_NOTE_ID_OFFSET=0
 const.CONSUMED_NOTE_CORE_DATA_OFFSET=1
 const.CONSUMED_NOTE_SERIAL_NUM_OFFSET=1
 const.CONSUMED_NOTE_SCRIPT_ROOT_OFFSET=2
@@ -151,7 +151,7 @@ const.CONSUMED_NOTE_ASSETS_OFFSET=8
 const.CREATED_NOTE_SECTION_OFFSET=4194304
 
 # The offsets at which data of a created note is stored relative to the start of its data segment.
-const.CREATED_NOTE_HASH_OFFSET=0
+const.CREATED_NOTE_ID_OFFSET=0
 const.CREATED_NOTE_METADATA_OFFSET=1
 const.CREATED_NOTE_RECIPIENT_OFFSET=2
 const.CREATED_NOTE_ASSETS_HASH_OFFSET=3
@@ -918,15 +918,15 @@ end
 
 #! Sets the metadata for a consumed note located at the specified memory address.
 #!
-#! Stack: [note_ptr, METADATA]
-#! Output: []
+#! Stack: [note_ptr, NOTE_METADATA]
+#! Output: [NOTE_METADATA]
 #!
 #! Where:
 #! - note_ptr, the memory address at which the consumed note data begins.
-#! - METADATA, the metadata of the consumed note.
+#! - NOTE_METADATA, the metadata of the consumed note.
 export.set_consumed_note_metadata
     push.CONSUMED_NOTE_METADATA_OFFSET add
-    mem_storew dropw
+    mem_storew
 end
 
 #! Returns the note's args.

miden-lib/asm/miden/kernels/tx/note.masm

@@ -22,7 +22,7 @@ const.ERR_NOTE_TOO_MANY_ASSETS=0x0002002A
 # =================================================================================================
 
 # The diff between the memory address after first mem_stream operation and the next target when
-# generating the consumed notes commitment. Must be NOTE_MEM_SIZE - 2;
+# generating the output notes commitment. Must be NOTE_MEM_SIZE - 2;
 const.OUTPUT_NOTE_HASHING_MEM_DIFF=510
 
 # INPUT NOTE PROCEDURES
@@ -219,20 +219,20 @@ proc.compute_output_note_assets_hash
     # => []
 end
 
-#! Computes the hash of an output note located at the specified memory address.
+#! Computes the ID of an output note located at the specified memory address.
 #!
-#! The hash is computed as follows:
+#! The note ID is computed as follows:
 #! - we define, recipient =
 #!       hash(hash(hash(serial_num, [0; 4]), script_hash), input_hash)
-#! - we then compute the output note hash as:
+#! - we then compute the output note ID as:
 #!       hash(recipient, assets_hash)
 #!
 #! Stack: [note_data_ptr]
-#! Output: [OUTPUT_NOTE_HASH]
+#! Output: [NOTE_ID]
 #!
 #! - note_data_ptr is a pointer to the data section of the output note.
-#! - OUTPUT_NOTE_HASH is the hash of the output note located at note_data_ptr.
-proc.compute_output_note_hash
+#! - NOTE_ID is the ID of the output note located at note_data_ptr.
+proc.compute_output_note_id
     # pad capacity elements of hasher
     padw
 
@@ -250,7 +250,7 @@ proc.compute_output_note_hash
 end
 
 #! Computes a commitment to the output notes. This is computed as a sequential hash of
-#! (note_hash, note_metadata) tuples.
+#! (note_id, note_metadata) tuples.
 #!
 #! Stack: []
 #! Output: [OUTPUT_NOTES_COMMITMENT]
@@ -279,15 +279,15 @@ export.compute_output_notes_commitment
 
     # loop and hash output notes
     while.true
-        # compute output note hash (this also computes the note's asset hash)
-        dup.12 exec.compute_output_note_hash
-        # => [NOTE_HASH, PERM, PERM, PERM, note_ptr, end_ptr, ...]
+        # compute and save output note ID to memory (this also computes the note's asset hash)
+        dup.12 exec.compute_output_note_id
+        # => [NOTE_ID, PERM, PERM, PERM, note_ptr, end_ptr, ...]
 
-        # drop output note hash from stack
+        # drop output note ID from stack (it will be read from memory by the next instruction)
         dropw
         # => [PERM, PERM, PERM, note_ptr, end_ptr, ...]
 
-        # permute over (note_hash, note_metadata)
+        # permute over (note_id, note_metadata)
         mem_stream hperm
         # => [PERM, PERM, PERM, note_ptr + 2, end_ptr, ...]
 

miden-lib/asm/miden/kernels/tx/prologue.masm

@@ -125,7 +125,7 @@ proc.process_block_data
     exec.memory::get_block_data_ptr
     # => [block_data_ptr]
 
-    # read block data and compute its digest
+    # read block data and compute its subhash. See `Advice stack` above for details.
     padw padw padw
     adv_pipe hperm
     adv_pipe hperm
@@ -214,9 +214,9 @@ proc.ingest_acct_storage_types
 
     # prepare the stack for reading storage slot types
     padw padw padw
-    # => [EMPTY_WORD, EMPTY_WORD, EMPTY_WORD, slot_type_data_ptr, TYPES_COM]
+    # => [PERM, PERM, PERM, slot_type_data_ptr, TYPES_COM]
 
-    # read the storage slot types from the advice provider and store in memory
+    # read slot types and compute its digest. See `Advice Map` above for details.
     repeat.32
         adv_pipe hperm
         # => [PERM, PERM, PERM, slot_type_data_ptr', TYPES_COM]
@@ -412,7 +412,7 @@ proc.process_account_data
     exec.memory::get_acct_data_ptr
     # => [acct_data_ptr]
 
-    # read account data and compute its digest
+    # read account details and compute its digest. See `Advice stack` above for details.
     padw padw padw
     adv_pipe hperm
     adv_pipe hperm
@@ -581,7 +581,7 @@ proc.process_input_note_details
     exec.memory::get_consumed_note_core_ptr
     # => [note_data_ptr]
 
-    # read note details (SERIAL_NUMBER, SCRIPT_ROOT, INPUTS_HASH, ASSETS_HASH)
+    # read input note's data and compute its digest. See `Advice stack` above for details.
     padw padw padw
     adv_pipe hperm
     adv_pipe hperm
@@ -594,27 +594,32 @@ end
 
 #! Copies the note's metadata and args from the advice stack to memory.
 #!
-#! Note: The note's metadata is validated when the note is authenticated, since
-#! the authentication is `hash(NOTE_ID || NOTE_METADATA)`. This is either done
-#! by the procedure `authenticate_note` for or delayed to another kernel.
+#! # Notes
 #!
-#! Stack: [note_ptr]
-#! Advice stack: [
-#!      METADATA,
-#!      ARGS,
-#! ]
-#! Output: []
+#! - The note's ARGS are not authenticated, these are optional arguments the user can provide when
+#!   consuming the note.
+#! - The note's metadata is authenticated, so the data is returned in the stack. The value is used
+#!   to compute the NOTE_HASH as `hash(NOTE_ID || NOTE_METADATA)`, which is the leaf value of the note's
+#!   tree in the contained in the block header. The NOTE_HASH is either verified by this kernel, or
+#!   delayed to be verified by another kernel (e.g. block or batch kernels).
+#!
+#! Inputs:
+#!   Stack: [note_ptr]
+#!   Advice stack: [NOTE_ARGS, NOTE_METADATA]
+#! Outputs:
+#!   Stack: [NOTE_METADATA]
+#!   Advice stack: []
 #!
 #! Where:
 #! - note_ptr, memory location for the input note.
-#! - METADATA, note's metadata.
-#! - ARGS, user arguments passed to the note.
-proc.process_note_metadata
-    padw adv_loadw dup.4 exec.memory::set_consumed_note_metadata
+#! - NOTE_ARGS, user arguments passed to the note.
+#! - NOTE_METADATA, note's metadata.
+proc.process_note_args_and_metadata
+    padw adv_loadw dup.4 exec.memory::set_consumed_note_args
     # => [note_ptr]
 
-    padw adv_loadw movup.4 exec.memory::set_consumed_note_args
-    # => []
+    padw adv_loadw movup.4 exec.memory::set_consumed_note_metadata
+    # => [NOTE_METADATA]
 end
 
 #! Copies the note's assets the advice stack to memory and verifies the commitment.
@@ -674,7 +679,7 @@ proc.process_note_assets
 
     # loop and read assets from the advice provider
     while.true
-        # read assets (ASSET_n, ASSET_n+1)
+        # read data and compute its digest. See `Advice stack` above for details.
         adv_pipe hperm
         # => [PERM, PERM, PERM, assets_ptr+2, note_ptr, counter, rounded_num_assets]
 
@@ -788,8 +793,8 @@ end
 #!      SCRIPT_ROOT,
 #!      INPUTS_HASH,
 #!      ASSETS_HASH,
-#!      METADATA,
 #!      ARGS,
+#!      NOTE_METADATA,
 #!      assets_count,
 #!      ASSET_0, ..., ASSET_N,
 #!      is_authenticated,
@@ -808,7 +813,7 @@ end
 #! - SCRIPT_ROOT, note's script root.
 #! - INPUTS_HASH, sequential hash of the padded note's inputs.
 #! - ASSETS_HASH, sequential hash of the padded note's assets.
-#! - METADATA, note's metadata.
+#! - NOTE_METADATA, note's metadata.
 #! - ARGS, user arguments passed to the note.
 #! - assets_count, note's assets count.
 #! - ASSET_0, ..., ASSET_N, padded note's assets.
@@ -830,7 +835,7 @@ proc.process_input_note
     exec.process_input_note_details
     # => [NULLIFIER, note_ptr, idx, HASHER_CAPACITY]
 
-    # save nullifier to memory
+    # save NULLIFIER to memory
     movup.5 exec.memory::get_consumed_note_nullifier_ptr mem_storew
     # => [NULLIFIER, note_ptr, HASHER_CAPACITY]
 
@@ -840,54 +845,49 @@ proc.process_input_note
     movup.4
     # => [note_ptr, NULLIFIER, HASHER_CAPACITY]
 
-    dup exec.process_note_metadata
-    # => [note_ptr, NULLIFIER, HASHER_CAPACITY]
+    dup exec.process_note_args_and_metadata
+    # => [NOTE_METADATA, note_ptr, NULLIFIER, HASHER_CAPACITY]
+
+    movup.4
+    # => [note_ptr, NOTE_METADATA, NULLIFIER, HASHER_CAPACITY]
 
     # note assets
     # ---------------------------------------------------------------------------------------------
 
     dup exec.process_note_assets
     dup exec.add_input_note_assets_to_vault
-    # => [note_ptr, NULLIFIER, HASHER_CAPACITY]
+    # => [note_ptr, NOTE_METADATA, NULLIFIER, HASHER_CAPACITY]
 
     # note id
     # ---------------------------------------------------------------------------------------------
 
     dup exec.compute_note_id
-    # => [NOTE_ID, note_ptr, NULLIFIER, HASHER_CAPACITY]
+    # => [NOTE_ID, note_ptr, NOTE_METADATA, NULLIFIER, HASHER_CAPACITY]
 
     # save note id to memory
-    dup.4 exec.memory::set_consumed_note_id
-    # => [NOTE_ID, note_ptr, NULLIFIER, HASHER_CAPACITY]
+    movup.4 exec.memory::set_consumed_note_id
+    # => [NOTE_ID, NOTE_METADATA, NULLIFIER, HASHER_CAPACITY]
 
     # note authentication
     # ---------------------------------------------------------------------------------------------
 
+    # NOTE_HASH: `hash(NOTE_ID || NOTE_METADATA)`
+    swapw hmerge
+    # => [NOTE_HASH, NULLIFIER, HASHER_CAPACITY]
+
     adv_push.1
-    # => [authenticated, NOTE_ID, note_ptr, NULLIFIER, HASHER_CAPACITY]
+    # => [is_authenticated, NOTE_HASH, NULLIFIER, HASHER_CAPACITY]
 
     if.true
-        # if the note is authenticated
-        # -----------------------------------------------------------------------------------------
-
-        # NOTE_HASH: `hash(NOTE_ID || NOTE_METADATA)`
-        movup.4 exec.memory::get_consumed_note_metadata hmerge
         # => [NOTE_HASH, NULLIFIER, HASHER_CAPACITY]
 
         exec.authenticate_note
         # => [NULLIFIER, HASHER_CAPACITY]
 
         padw
         # => [EMPTY_WORD, NULLIFIER, HASHER_CAPACITY]
-
-    else
-        # if the note authentication is delayed
-        # -----------------------------------------------------------------------------------------
-
-        movup.4 drop
-        # => [NOTE_ID, NULLIFIER, HASHER_CAPACITY]
     end
-    # => [EMPTY_WORD_OR_NOTEID, NULLIFIER, HASHER_CAPACITY]
+    # => [EMPTY_WORD_OR_NOTE_HASH, NULLIFIER, HASHER_CAPACITY]
 
     # update the input note commitment
     hperm
@@ -897,7 +897,7 @@ end
 #! Process the input notes data provided via the advice provider. This involves reading the data
 #! from the advice provider and storing it at the appropriate memory addresses. As each note is
 #! processed its hash and nullifier are computed. The transaction input notes commitment is
-#! computed via a sequential hash of all (nullifier, NOTEID_OR_EMPTY_WORD) pairs for all input
+#! computed via a sequential hash of all (NULLIFIER, EMPTY_WORD_NOTE_HASH) pairs for all input
 #! notes.
 #!
 #! Stack: []
@@ -946,7 +946,7 @@ proc.process_input_notes_data
     #      processed.
     # - The num_notes is kept at position 13, because dup.13 is cheap.
     # - The [idx, num_notes] pair is kept in a word boundary, so that its word can be swapped with a
-    #   cheap mov intructions to update the `idx` counter.
+    #   cheap swapw.3 intruction to update the `idx` counter.
 
     push.0 padw padw padw
     # => [PERM, PERM, PERM, idx, num_notes]

miden-lib/asm/miden/kernels/tx/tx.masm

@@ -99,7 +99,7 @@ export.memory::get_blk_num->get_block_number
 #! - INPUT_NOTES_COMMITMENT is the input notes commitment hash.
 export.memory::get_input_notes_commitment
 
-#! Returns the output notes hash. This is computed as a sequential hash of (note_hash, note_metadata)
+#! Returns the output notes hash. This is computed as a sequential hash of (note_id, note_metadata)
 #! tuples over all output notes.
 #!
 #! Inputs: []