warmup

[FilipAndersson245]

Hello, it would be nice if it would be possible to add warmup runs, in my benchmark case the first run is always slower as some data has to be fetched into memory, and it always drags down the "real" lowest benchmark lowest as this is done only once.
I would suggest #[divan::bench(warmup=10) or something simular with the default being either zero or a 1.

[nvzqz]

Are you suggesting in this case that Divan would discard the first 10 samples after it finds a proper sample size?

Perhaps we could have something like:

#[divan::bench(
    // Discard at least the first 10 samples.
    warmup_sample_count = 10,
    // Discard samples until 1 second has elapsed.
    // This would use `IntoDuration` like `min_time`.
    warmup_min_time = 1,
)

---

In the meantime, I recommend moving out the benchmark closure and doing your own custom warmup.

#[divan::bench]
fn my_bench(bencher: divan::Bencher) {
    let work_fn = || {
        // Benchmarked work in here.
    };

    // Warmup:
    for _ in 0..1000 {
        work_fn();
    }

    // Benchmark:
    bencher.bench(work_fn);
}

[FilipAndersson245]

Are you suggesting in this case that Divan would discard the first 10 samples after it finds a proper sample size?

Perhaps we could have something like:

#[divan::bench(
    // Discard at least the first 10 samples.
    warmup_sample_count = 10,
    // Discard samples until 1 second has elapsed.
    // This would use `IntoDuration` like `min_time`.
    warmup_min_time = 1,
)

Yee something like that would be very nice.

In the meantime, I recommend moving out the benchmark closure and doing your own custom warmup.

#[divan::bench]
fn my_bench(bencher: divan::Bencher) {
    let work_fn = || {
        // Benchmarked work in here.
    };

    // Warmup:
    for _ in 0..1000 {
        work_fn();
    }

    // Benchmark:
    bencher.bench(work_fn);
}

that would work for me, Thanks 👍

[dvdplm]

It'd probably be a good default to have a warmup. One thing users coming to divan from criterion will notice is that the output is noisier with divan.

[nanoqsh]

+1

I noticed a performance degradation in a test after renaming (so the test ended up at the top of the list). This is very confusing