feat: add StateSync component
#650
Conversation
tomyrd
force-pushed
the
tomyrd-sync-component-alt
branch
from
8731b2b to
d54e186
Compare
|
I need to tidy up the code and update the documentation, but the basic idea with this updated version is that we have callback functions (defined in These callbacks could also work as a sort of event reaction, as the user could redefine them and they would be called on each sync with all the new and updated node information. |
tomyrd
force-pushed
the
tomyrd-sync-component-alt
branch
from
6baeafa to
50938fb
Compare
tomyrd
force-pushed
the
tomyrd-sync-component-alt
branch
from
50938fb to
2f26007
Compare
| for (account_id, digest) in account_updates.mismatched_private_accounts() { | ||
| // Mismatched digests may be due to stale network data. If the mismatched digest is | ||
| // tracked in the db and corresponds to the mismatched account, it means we | ||
| // got a past update and shouldn't lock the account. | ||
| if let Some(account) = self.get_account_header_by_hash(*digest).await? { | ||
| if account.id() == *account_id { | ||
| continue; | ||
| } | ||
| } |
There was a problem hiding this comment.
Could this not be done before calling the store? Then it would be deduplicated from the codebase, or am I missing something?
There was a problem hiding this comment.
Could this not be done before calling the store?
Not with the current structure, because the SyncStateUpdate is built inside the StateSync component which doesn't have access to the store. But the code duplication isn't ideal, I'll add it to the follow ups. If we want to access the store inside the process, we should add a callback (maybe we can re-add this one).
| .updated_input_notes() | ||
| .iter() | ||
| .chain(committed_notes.new_input_notes().iter()) | ||
| .chain(note_updates.new_input_notes().iter()) | ||
| .filter(|note| note.is_committed()) |
There was a problem hiding this comment.
Why is this change needed?
There was a problem hiding this comment.
Before this PR, the check_block_relevance only received committed note updates, now it receives every note update, and some of those input note records can't be converted to Note (if the note was consumed externally it doesn't store its metadata).
igamigo
changed the title
StateSync component (alternative)StateSync component
|
I've fixed some of the comments that I had on my pending review and refactored the code a little bit. I've also added a comment on the follow-up issue (#663). One thing to note is that the |
| pub async fn sync_state_step( | ||
| &self, | ||
| current_block: BlockHeader, | ||
| current_block_has_relevant_notes: bool, | ||
| current_partial_mmr: PartialMmr, | ||
| accounts: Vec<AccountHeader>, | ||
| note_tags: Vec<NoteTag>, | ||
| unspent_nullifiers: Vec<Nullifier>, | ||
| ) -> Result<Option<SyncStatus>, ClientError> { |
There was a problem hiding this comment.
Question: maybe I missed this, but I thought we were going to do a full sync (not just one step) and then return the result of the entire sync from this function. Then, the result would be inserted into the store.
What is the motivation for exposing step-wise syncing?
There was a problem hiding this comment.
Hmm, not entirely sure what the original motivation here was. One thing that comes to mind is that we get the new state from the store each time we do a new step. This involves the block number, account headers and note tags. I think it shouldn't be too difficult to make it so that we can expose the full sync instead though there is probably some logic that we need to add in order to make it keep a consistent state within the component. If everything else looks OK I could make another PR with these changes (unless you prefer to have them in this PR instead)
This PR is the second iteration of the
StateSynccomponent, based on this comment. Although the PR is still in draft, most of the core changes are already made so it can be reviewed, at least partially.I like this new version a lot more than the previous. It's already working and the rust unit and integration tests are passing.
General changes
rpc_apiin the client was changed fromBoxtoArcso it can be shared with the component. This also means that I had to change the base struct to give it interior mutability.crates/rust-client/src/sync/mod.rsto the newcrates/rust-client/src/sync/state_sync.rs. The moved code isn't a 1 to 1 replica, since I was moving the code, I went ahead and refactored it a bit to be more clear and added some extra documentation. I belive this new version of the sync process is more clear and easier to read/understand.TODOs
apply_mmr_changespart inside the component. I feel like it could be better documented and it should fit nicely inside thenote_state_synclogic (it's just a different note state change).