feat: import NoteFile in client (#375)

* Update import in cli to use `NoteFile` * Update CHANGELOG * Update cli export to use `NoteFile` * Fix build errors with file tag * Rename `import_input_note` to `import_note` * Implement `From<InputNoteRecord>` for `NoteFile` * Change `import_note` parameter to `NoteFile` * Fix integration tests * Implement `metadata` for `NoteDetails` * Add `get_notes_without_block_header` method to store * Add `ignored` and `imported_tag` to `InputNoteRecord` * Update `import_note` with new functionality * Fix integration tests * Update `ignored` field on note tag add * Update `export` to accept type of note file * Update MMR data for committed notes on sync * Update ignored notes on sync * Fix after rebase * Update tracked note when importing `NoteFile::NoteId` * Remove `NoteFile` conversion from `InputNoteRecord` * Check ephemeral tags when importing * Fix rebase * Fix integration tests * Remove tag from export cli command * Remove `--no-verify` flag from import cli command * Remove `InputNoteRecord` conversion on cli export * Update doc comments for `import_note` * Change suggestions * Add `maybe_await` * Update note when importing `NoteId` even if private * Only update ignored notes if specified in `sync` command * Change suggestions * Update `cli-reference` * Change suggestions * Add tests for ignored notes * Improve doc for `get_notes_without_block_header` * Change suggestions * Fix authentication hash calls for note * Temporarily change `NODE_BRANCH` in Makefile * force CI reset --------- Co-authored-by: igamigo <[email protected]>
0xPolygonMiden · Jun 28, 2024 · 847ba6f · 847ba6f
1 parent 3726411
commit 847ba6f
Show file tree

Hide file tree

Showing 26 changed files with 800 additions and 205 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,6 @@
 # Changelog
 
+* Note importing in client now uses the `NoteFile` type (#375).
 * Added build script to import Miden node protobuf files to generate types for `tonic_client` and removed `miden-node-proto` dependency (#395).
 * Implemented retrieval of executed transaction info (id, commit height, account_id) from sync state RPC endpoint (#387).
 * Renamed "pending" notes to "expected" notes (#373).

diff --git a/Makefile b/Makefile
@@ -10,7 +10,7 @@ FEATURES_INTEGRATION_TESTING="integration"
 FEATURES_CLI="testing, executable, concurrent"
 NODE_FEATURES_TESTING="testing"
 WARNINGS=RUSTDOCFLAGS="-D warnings"
-NODE_BRANCH="next"
+NODE_BRANCH="polydez-future-notes"
 
 # --- Linting -------------------------------------------------------------------------------------
 
@@ -63,7 +63,7 @@ test: ## Run tests
 
 .PHONY: integration-test
 integration-test: ## Run integration tests
-	cargo nextest run --no-capture --release --test=integration --features $(FEATURES_INTEGRATION_TESTING) --no-default-features
+	cargo nextest run --release --test=integration --features $(FEATURES_INTEGRATION_TESTING) --no-default-features
 
 .PHONY: integration-test-full
 integration-test-full: ## Run the integration test binary with ignored tests included

diff --git a/docs/cli-reference.md b/docs/cli-reference.md
@@ -25,7 +25,7 @@ Note that the debug flag overrides the `MIDEN_DEBUG` environment variable.
 
 ## Commands
 
-### `init` 
+### `init`
 
 Creates a configuration file for the client in the current directory.
 
@@ -133,6 +133,10 @@ miden notes --show 0x70b7ec
 
 Sync the client with the latest state of the Miden network. Shows a brief summary at the end.
 
+| Flags             | Description                                                 | Short Flag |
+|-------------------|-------------------------------------------------------------|------------|
+|`--update-ignored` | Update ignored notes in sync                                | `-u`       |
+
 ### `tags`
 
 View and add tags.
@@ -233,12 +237,24 @@ Continue with proving and submission? Changes will be irreversible once the proo
 
 This confirmation can be skipped in non-interactive environments by providing the `--force` flag (`miden send --force ...`):
 
-### `import`
+### Importing and exporting
+
+#### `export`
 
-Import entities managed by the client, such as accounts and notes. The type of entitie is inferred.
+Export input note data to a binary file .
 
-When importing notes the CLI verifies that they exist on chain. The user can add an optional flag `--no-verify` that skips this verification.
+| Flag                           | Description                                    | Aliases |
+|--------------------------------|------------------------------------------------|---------|
+| `--filename <FILENAME>`        | Desired filename for the binary file.          | `-f`    |
+| `--export-type <EXPORT_TYPE>`  | Exported note type.                            | `-e`    |
 
-### `export`
+##### Export type
 
-Export input note data to a binary file .
+The user needs to specify how the note should be exported via the `--export-type` flag. The following options are available:
+- `id`: Only the note ID is exported. When importing, if the note ID is already tracked by the client, the note will be updated with missing information fetched from the node, this works for both public and private notes. If the note isn't tracked and the note is public, the whole note is fetched from the node and start being tracked.
+- `full`: The note is exported with all of its information (metadata and inclusion proof). When importing, the note is considered committed. The note may not be consumed directly after importing as it's block header will not be stored in the client. The block header will be fetched during the next sync.
+- `partial`: The note is exported with minimal information and may be imported even if the note is not committed on chain. When importing, the note will be considered to be "expected" and will be updated after a sync when the original note is committed.
+
+#### `import`
+
+Import entities managed by the client, such as accounts and notes. The type of entities is inferred.
diff --git a/src/cli/export.rs b/src/cli/export.rs
@@ -1,11 +1,7 @@
 use std::{fs::File, io::Write, path::PathBuf};
 
-use miden_client::{
-    rpc::NodeRpcClient,
-    store::{InputNoteRecord, Store},
-    Client,
-};
-use miden_objects::crypto::rand::FeltRng;
+use miden_client::{rpc::NodeRpcClient, store::Store, Client};
+use miden_objects::{crypto::rand::FeltRng, notes::NoteFile};
 use miden_tx::{auth::TransactionAuthenticator, utils::Serializable};
 use tracing::info;
 
@@ -22,14 +18,25 @@ pub struct ExportCmd {
     /// Desired filename for the binary file. Defaults to the note ID if not provided
     #[clap(short, long)]
     filename: Option<PathBuf>,
+
+    /// Exported note type
+    #[clap(short, long, value_enum)]
+    export_type: ExportType,
+}
+
+#[derive(clap::ValueEnum, Clone, Debug)]
+pub enum ExportType {
+    Id,
+    Full,
+    Partial,
 }
 
 impl ExportCmd {
     pub fn execute<N: NodeRpcClient, R: FeltRng, S: Store, A: TransactionAuthenticator>(
         &self,
         client: Client<N, R, S, A>,
     ) -> Result<(), String> {
-        export_note(&client, self.id.as_str(), self.filename.clone())?;
+        export_note(&client, self.id.as_str(), self.filename.clone(), self.export_type.clone())?;
         Ok(())
     }
 }
@@ -41,6 +48,7 @@ pub fn export_note<N: NodeRpcClient, R: FeltRng, S: Store, A: TransactionAuthent
     client: &Client<N, R, S, A>,
     note_id: &str,
     filename: Option<PathBuf>,
+    export_type: ExportType,
 ) -> Result<File, String> {
     let note_id = get_output_note_with_id_prefix(client, note_id)
         .map_err(|err| err.to_string())?
@@ -51,10 +59,19 @@ pub fn export_note<N: NodeRpcClient, R: FeltRng, S: Store, A: TransactionAuthent
         .pop()
         .expect("should have an output note");
 
-    // Convert output note into InputNoteRecord before exporting
-    let input_note: InputNoteRecord = output_note
-        .try_into()
-        .map_err(|_err| format!("Can't export note with ID {}", note_id.to_hex()))?;
+    let note_file = match export_type {
+        ExportType::Id => NoteFile::NoteId(output_note.id()),
+        ExportType::Full => match output_note.inclusion_proof() {
+            Some(inclusion_proof) => {
+                NoteFile::NoteWithProof(output_note.clone().try_into()?, inclusion_proof.clone())
+            },
+            None => return Err("Note does not have inclusion proofx".to_string()),
+        },
+        ExportType::Partial => NoteFile::NoteDetails(
+            output_note.clone().try_into()?,
+            Some(output_note.metadata().tag()),
+        ),
+    };
 
     let file_path = if let Some(filename) = filename {
         filename
@@ -65,7 +82,7 @@ pub fn export_note<N: NodeRpcClient, R: FeltRng, S: Store, A: TransactionAuthent
 
     info!("Writing file to {}", file_path.to_string_lossy());
     let mut file = File::create(file_path).map_err(|err| err.to_string())?;
-    file.write_all(&input_note.to_bytes()).map_err(|err| err.to_string())?;
+    file.write_all(&note_file.to_bytes()).map_err(|err| err.to_string())?;
 
     println!("Succesfully exported note {}", note_id);
     Ok(file)

diff --git a/src/cli/import.rs b/src/cli/import.rs
@@ -4,15 +4,11 @@ use std::{
     path::PathBuf,
 };
 
-use miden_client::{
-    rpc::NodeRpcClient,
-    store::{InputNoteRecord, Store},
-    Client,
-};
+use miden_client::{rpc::NodeRpcClient, store::Store, Client};
 use miden_objects::{
     accounts::{AccountData, AccountId},
     crypto::rand::FeltRng,
-    notes::NoteId,
+    notes::{NoteFile, NoteId},
     utils::Deserializable,
 };
 use miden_tx::auth::TransactionAuthenticator;
@@ -27,9 +23,6 @@ pub struct ImportCmd {
     /// Paths to the files that contains the account/note data
     #[arg()]
     filenames: Vec<PathBuf>,
-    /// Skip verification of note's existence in the chain (Only when importing notes)
-    #[clap(short, long, default_value = "false")]
-    no_verify: bool,
 }
 
 impl ImportCmd {
@@ -40,17 +33,18 @@ impl ImportCmd {
         validate_paths(&self.filenames)?;
         let (mut current_config, _) = load_config_file()?;
         for filename in &self.filenames {
-            let note_id = import_note(&mut client, filename.clone(), !self.no_verify).await;
-            if note_id.is_ok() {
-                println!("Succesfully imported note {}", note_id.unwrap().inner());
-                continue;
-            }
-            let account_id = import_account(&mut client, filename)
-                .map_err(|_| format!("Failed to parse file {}", filename.to_string_lossy()))?;
-            println!("Succesfully imported account {}", account_id);
-
-            if account_id.is_regular_account() {
-                maybe_set_default_account(&mut current_config, account_id)?;
+            let note_id = import_note(&mut client, filename.clone()).await;
+
+            if let Ok(note_id) = note_id {
+                println!("Succesfully imported note {}", note_id.inner());
+            } else {
+                let account_id = import_account(&mut client, filename)
+                    .map_err(|_| format!("Failed to parse file {}", filename.to_string_lossy()))?;
+                println!("Succesfully imported account {}", account_id);
+
+                if account_id.is_regular_account() {
+                    maybe_set_default_account(&mut current_config, account_id)?;
+                }
             }
         }
         Ok(())
@@ -84,23 +78,15 @@ fn import_account<N: NodeRpcClient, R: FeltRng, S: Store, A: TransactionAuthenti
 pub async fn import_note<N: NodeRpcClient, R: FeltRng, S: Store, A: TransactionAuthenticator>(
     client: &mut Client<N, R, S, A>,
     filename: PathBuf,
-    verify: bool,
 ) -> Result<NoteId, String> {
     let mut contents = vec![];
     let mut _file = File::open(filename)
         .and_then(|mut f| f.read_to_end(&mut contents))
         .map_err(|err| err.to_string());
 
-    let input_note_record =
-        InputNoteRecord::read_from_bytes(&contents).map_err(|err| err.to_string())?;
-
-    let note_id = input_note_record.id();
-    client
-        .import_input_note(input_note_record, verify)
-        .await
-        .map_err(|err| err.to_string())?;
+    let note_file = NoteFile::read_from_bytes(&contents).map_err(|err| err.to_string())?;
 
-    Ok(note_id)
+    client.import_note(note_file).await.map_err(|err| err.to_string())
 }
 
 // HELPERS

diff --git a/src/cli/mod.rs b/src/cli/mod.rs
@@ -18,6 +18,7 @@ use miden_objects::{
 };
 use miden_tx::auth::TransactionAuthenticator;
 use rand::Rng;
+use sync::SyncCmd;
 use tracing::info;
 use transactions::TransactionCmd;
 
@@ -76,8 +77,7 @@ pub enum Command {
     Export(ExportCmd),
     Init(InitCmd),
     Notes(NotesCmd),
-    /// Sync this client with the latest state of the Miden network.
-    Sync,
+    Sync(SyncCmd),
     /// View a summary of the current client state
     Info,
     Tags(TagsCmd),
@@ -139,7 +139,7 @@ impl Cli {
             Command::Init(_) => Ok(()),
             Command::Info => info::print_client_info(&client, &cli_config),
             Command::Notes(notes) => notes.execute(client).await,
-            Command::Sync => sync::sync_state(client).await,
+            Command::Sync(sync) => sync.execute(client).await,
             Command::Tags(tags) => tags.execute(client).await,
             Command::Transaction(transaction) => transaction.execute(client).await,
             Command::Export(cmd) => cmd.execute(client),

diff --git a/src/cli/sync.rs b/src/cli/sync.rs
@@ -1,16 +1,32 @@
+use clap::Parser;
 use miden_client::{rpc::NodeRpcClient, store::Store, Client};
 use miden_objects::crypto::rand::FeltRng;
 use miden_tx::auth::TransactionAuthenticator;
 
-pub async fn sync_state<N: NodeRpcClient, R: FeltRng, S: Store, A: TransactionAuthenticator>(
-    mut client: Client<N, R, S, A>,
-) -> Result<(), String> {
-    let new_details = client.sync_state().await?;
-    println!("State synced to block {}", new_details.block_num);
-    println!("New public notes: {}", new_details.new_notes);
-    println!("Tracked notes updated: {}", new_details.new_inclusion_proofs);
-    println!("Tracked notes consumed: {}", new_details.new_nullifiers);
-    println!("Tracked accounts updated: {}", new_details.updated_onchain_accounts);
-    println!("Commited transactions: {}", new_details.commited_transactions);
-    Ok(())
+#[derive(Debug, Parser, Clone)]
+#[clap(about = "Sync this client with the latest state of the Miden network.")]
+pub struct SyncCmd {
+    /// If enabled, the ignored notes will also be updated by fetching them directly from the node.
+    #[clap(short, long)]
+    update_ignored: bool,
+}
+
+impl SyncCmd {
+    pub async fn execute<N: NodeRpcClient, R: FeltRng, S: Store, A: TransactionAuthenticator>(
+        &self,
+        mut client: Client<N, R, S, A>,
+    ) -> Result<(), String> {
+        let mut new_details = client.sync_state().await?;
+        if self.update_ignored {
+            new_details.combine_with(&client.update_ignored_notes().await?);
+        }
+
+        println!("State synced to block {}", new_details.block_num);
+        println!("New public notes: {}", new_details.new_notes);
+        println!("Tracked notes updated: {}", new_details.new_inclusion_proofs);
+        println!("Tracked notes consumed: {}", new_details.new_nullifiers);
+        println!("Tracked accounts updated: {}", new_details.updated_onchain_accounts);
+        println!("Commited transactions: {}", new_details.commited_transactions);
+        Ok(())
+    }
 }