Refactor CallStack::Synthesize to produce consistent dummy outputs for faster nested import deployments #2598

kpandl · 2025-01-17T14:51:17Z

Summary

This PR refactors the way CallStack::Synthesize is run, decoupling the Synthesize mode from the old execute_function approach. It produces consistent dummy outputs (inspired by CheckDeployment) more rapidly and skips sub-circuit construction for nested import calls. The result is significantly faster deployments for deeply nested imports, while still generating cryptographically coherent record nonces.

Motivation

Previously, Authorize and Synthesize modes were lumped together. As such, calls in CallStack::Synthesize were forced to run a real sub-circuit via execute_function. This is slow for nested imports and has led to an exponential growth in deployment time w.r.t. the import depth.

Thus, this PR refactors the Synthesize branch to skip full circuit execution and produce “dummy” outputs with consistent record nonces.

Note

This PR changes expected future IDs in 3 of the execute_and_finalize tests. The reason is that previously, Synthesize always ran real sub-circuits, consuming RNG draws. With this PR, certain sub-circuits are skipped by producing dummy outputs. As a result, the RNG is executed less times, leading to different transaction commitments and future IDs.

Test Plan

Tested the deployment and execution of programs with deep imports in a local devnet.

Related PRs

vicsn · 2025-01-20T18:39:57Z

NOTE: this PR will also significantly speed up tests using nested deployments, e.g. test_deep_nested_execution_cost.

vicsn · 2025-01-21T14:58:24Z

synthesizer/process/src/stack/call/mod.rs

@@ -45,7 +45,7 @@ pub trait CallTrait<N: Network> {
    /// Executes the instruction.
    fn execute<A: circuit::Aleo<Network = N>, R: CryptoRng + Rng>(
        &self,
-        stack: &(impl StackEvaluate<N> + StackExecute<N> + StackMatches<N> + StackProgram<N>),
+        stack: &Stack<N>,


Why change the interface? Can we keep it the way it was? Same for the other function.

Addressed here: 774efae

vicsn · 2025-01-21T14:58:57Z

synthesizer/process/src/stack/mod.rs

@@ -316,6 +316,12 @@ impl<N: Network> StackProgram<N> for Stack<N> {
            .ok_or_else(|| anyhow!("Function '{function_name}' does not exist"))
    }

+    /// Returns true if the proving key for the given function name already exists in this stack.
+    #[inline]
+    fn has_proving_key(&self, function_name: &Identifier<N>) -> bool {


Why add this extra wrapper?

I think if we don't want to have this wrapper, I need to revert 774efae.
I guess the cleanest solution is if I go ahead and to that, so we can directly call contains_proving_key?

Ah I see. Hmm I don't know the reasoning behind the choice of StackProgram<N> trait existing. But let's try to keep that unchanged.

One approach which could be somewhat consistent, is to create a new trait StackKeys. And these particular functions (contains/get/insert proving/verifying keys) could impl<N> StackKeys<N>.

addressed here: 6ebe642

vicsn · 2025-01-21T14:59:39Z

synthesizer/process/src/stack/call/mod.rs

+                        // Return the request and response.
+                        (request, response)
+                    }
+                    CallStack::Synthesize(_, private_key, ..) => {


Missing comment like you added to the other synthesize and authorize cases

Is there any difference to the contents of CallStack::Synthesize and CallStack::CheckDeployment? Perhaps they can be merged into the same case.

Addressed both here: 815a4df
Synthesize and CheckDeployment are indeed the same!

vicsn · 2025-01-21T15:02:13Z

synthesizer/process/src/stack/call/mod.rs

+                                    // Compute the randomizer.
+                                    let randomizer = N::hash_to_scalar_psd2(&[*request.tvk(), index])?;
+                                    // Construct the record nonce from that randomizer.
+                                    let record_nonce = N::g_scalar_multiply(&randomizer);
+                                    // Sample the record with that nonce.


Could this be made a part of sample_record, so that callers of sample_record don't have to think about it anymore and all are guaranteed correct usage?

Created a new function sample_record_using_tvk here: 0503e5c

vicsn · 2025-01-22T15:46:19Z

synthesizer/process/src/stack/call/mod.rs

@@ -265,22 +267,24 @@ impl<N: Network> CallTrait<N> for Call<N> {
                            inputs.iter(),
                            &function.input_types(),
                            root_tvk,
-                            is_root,
+                            false,


Why use false?

addressed here: d7070d9

vicsn · 2025-01-22T15:46:42Z

synthesizer/process/src/stack/call/mod.rs

@@ -289,38 +293,38 @@ impl<N: Network> CallTrait<N> for Call<N> {
                            inputs.iter(),
                            &function.input_types(),
                            root_tvk,
-                            is_root,
+                            false,


Why use false?

addressed here: d7070d9

kpandl · 2025-01-23T16:26:16Z

note: the failing snarkvm CI test is a CI/CDN issue

optimize Synthesize

e58ad8d

fix proving key for Synthesize mode

37cd8ed

vicsn requested changes Jan 21, 2025

View reviewed changes

kpandl added 3 commits January 21, 2025 16:35

revert interface change

774efae

clean up match arms

815a4df

sample_record_using_tvk

0503e5c

kpandl requested a review from vicsn January 21, 2025 17:16

vicsn reviewed Jan 22, 2025

View reviewed changes

kpandl added 3 commits January 23, 2025 13:19

update test expectations

a076216

StackKeys

6ebe642

is_root instead of false

d7070d9

kpandl requested a review from vicsn January 23, 2025 16:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor CallStack::Synthesize to produce consistent dummy outputs for faster nested import deployments #2598

Refactor CallStack::Synthesize to produce consistent dummy outputs for faster nested import deployments #2598

kpandl commented Jan 17, 2025 •

edited

Loading

vicsn commented Jan 20, 2025

vicsn Jan 21, 2025

kpandl Jan 21, 2025

vicsn Jan 21, 2025

kpandl Jan 21, 2025

vicsn Jan 22, 2025

kpandl Jan 23, 2025

vicsn Jan 21, 2025

vicsn Jan 21, 2025

kpandl Jan 21, 2025 •

edited

Loading

vicsn Jan 21, 2025

kpandl Jan 21, 2025

vicsn Jan 22, 2025

kpandl Jan 23, 2025

vicsn Jan 22, 2025

kpandl Jan 23, 2025

kpandl commented Jan 23, 2025

Refactor CallStack::Synthesize to produce consistent dummy outputs for faster nested import deployments #2598

Are you sure you want to change the base?

Refactor CallStack::Synthesize to produce consistent dummy outputs for faster nested import deployments #2598

Conversation

kpandl commented Jan 17, 2025 • edited Loading

Summary

Motivation

Note

Test Plan

Related PRs

vicsn commented Jan 20, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kpandl Jan 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kpandl commented Jan 23, 2025

kpandl commented Jan 17, 2025 •

edited

Loading

kpandl Jan 21, 2025 •

edited

Loading