[ASM][RCM] Read waf actions per file and remove per file #6072

anna-git · 2024-09-24T15:06:35Z

Summary of changes

Read waf actions per file and remove per file

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging · 2024-09-24T15:27:54Z

Datadog Report

Branch report: anna/asm/actions-per-file
Commit report: e0807d5
Test service: dd-trace-dotnet

✅ 0 Failed, 364493 Passed, 2083 Skipped, 16h 6m 8.83s Total Time

andrewlock · 2024-09-24T15:39:13Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6072) - mean (71ms)  : 68, 74
     .   : milestone, 71,
    master - mean (70ms)  : 67, 74
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (6072) - mean (1,110ms)  : 1090, 1130
     .   : milestone, 1110,
    master - mean (1,105ms)  : 1083, 1126
     .   : milestone, 1105,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6072) - mean (110ms)  : 106, 114
     .   : milestone, 110,
    master - mean (109ms)  : 107, 112
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (6072) - mean (771ms)  : 755, 787
     .   : milestone, 771,
    master - mean (772ms)  : 759, 785
     .   : milestone, 772,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6072) - mean (93ms)  : 90, 97
     .   : milestone, 93,
    master - mean (93ms)  : 90, 95
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (6072) - mean (730ms)  : 710, 750
     .   : milestone, 730,
    master - mean (727ms)  : 713, 742
     .   : milestone, 727,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6072) - mean (190ms)  : 187, 193
     .   : milestone, 190,
    master - mean (190ms)  : 187, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6072) - mean (1,195ms)  : 1174, 1216
     .   : milestone, 1195,
    master - mean (1,194ms)  : 1172, 1217
     .   : milestone, 1194,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6072) - mean (277ms)  : 271, 283
     .   : milestone, 277,
    master - mean (276ms)  : 272, 280
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (6072) - mean (938ms)  : 919, 958
     .   : milestone, 938,
    master - mean (940ms)  : 924, 957
     .   : milestone, 940,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6072) - mean (265ms)  : 261, 268
     .   : milestone, 265,
    master - mean (265ms)  : 261, 268
     .   : milestone, 265,

    section CallTarget+Inlining+NGEN
    This PR (6072) - mean (923ms)  : 905, 942
     .   : milestone, 923,
    master - mean (922ms)  : 908, 937
     .   : milestone, 922,

andrewlock · 2024-09-24T16:04:48Z

Benchmarks Report for appsec 🐌

Benchmarks for #6072 compared to master:

1 benchmarks are slower, with geometric mean 1.162
1 benchmarks have fewer allocations
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	72.7μs	86.4ns	334ns	0.0728	0	6 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	62.7μs	98.8ns	383ns	0.0937	0	6.95 KB
master	`AllCycleSimpleBody`	net472	49.8μs	72.9ns	282ns	1.3	0	8.33 KB
master	`AllCycleMoreComplexBody`	net6.0	78.3μs	84.7ns	328ns	0.117	0	9.51 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	69.4μs	99.9ns	387ns	0.14	0	10.37 KB
master	`AllCycleMoreComplexBody`	net472	55.9μs	146ns	567ns	1.87	0.0283	11.85 KB
master	`ObjectExtractorSimpleBody`	net6.0	142ns	0.136ns	0.51ns	0.00394	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	197ns	0.309ns	1.16ns	0.0037	0	272 B
master	`ObjectExtractorSimpleBody`	net472	163ns	0.178ns	0.687ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	3.1μs	2.4ns	9ns	0.0528	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.08μs	1.48ns	5.35ns	0.051	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	3.81μs	5.37ns	20.8ns	0.601	0.00571	3.8 KB
#6072	`AllCycleSimpleBody`	net6.0	73.2μs	87.5ns	339ns	0.0733	0	6.01 KB
#6072	`AllCycleSimpleBody`	netcoreapp3.1	63μs	92ns	344ns	0.0943	0	6.95 KB
#6072	`AllCycleSimpleBody`	net472	49.2μs	40.9ns	158ns	1.32	0	8.34 KB
#6072	`AllCycleMoreComplexBody`	net6.0	78μs	179ns	691ns	0.116	0	9.51 KB
#6072	`AllCycleMoreComplexBody`	netcoreapp3.1	68.8μs	89.3ns	334ns	0.137	0	10.37 KB
#6072	`AllCycleMoreComplexBody`	net472	55.8μs	113ns	436ns	1.87	0.0283	11.85 KB
#6072	`ObjectExtractorSimpleBody`	net6.0	140ns	0.213ns	0.825ns	0.00397	0	280 B
#6072	`ObjectExtractorSimpleBody`	netcoreapp3.1	199ns	0.544ns	2.03ns	0.00363	0	272 B
#6072	`ObjectExtractorSimpleBody`	net472	163ns	0.23ns	0.889ns	0.0446	0	281 B
#6072	`ObjectExtractorMoreComplexBody`	net6.0	3.05μs	1.52ns	5.5ns	0.0534	0	3.78 KB
#6072	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.23μs	1.75ns	6.78ns	0.0508	0	3.69 KB
#6072	`ObjectExtractorMoreComplexBody`	net472	3.87μs	3.98ns	15.4ns	0.602	0.00583	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6072

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs‑net6.0	1.162	70,520.92	81,974.52

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EncodeArgs`	net6.0	39.1μs	21.7ns	81.3ns	0.468	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	54.6μs	30.2ns	117ns	0.434	0	32.4 KB
master	`EncodeArgs`	net472	66.6μs	98.1ns	367ns	5.15	0.0661	32.5 KB
master	`EncodeLegacyArgs`	net6.0	70.4μs	122ns	471ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	107μs	38.9ns	145ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	net472	150μs	88.8ns	344ns	0.297	0	2.15 KB
#6072	`EncodeArgs`	net6.0	38.9μs	18.6ns	72.1ns	0.443	0	32.4 KB
#6072	`EncodeArgs`	netcoreapp3.1	54.2μs	11.9ns	46.2ns	0.433	0	32.4 KB
#6072	`EncodeArgs`	net472	66.3μs	34ns	127ns	5.13	0.0667	32.5 KB
#6072	`EncodeLegacyArgs`	net6.0	80.8μs	447ns	2.79μs	0	0	2.14 KB
#6072	`EncodeLegacyArgs`	netcoreapp3.1	106μs	283ns	1.1μs	0	0	2.14 KB
#6072	`EncodeLegacyArgs`	net472	150μs	157ns	610ns	0.3	0	2.15 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunWafRealisticBenchmark`	net6.0	185μs	80.5ns	312ns	0	2.44 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	197μs	143ns	536ns	0	2.39 KB
master	`RunWafRealisticBenchmark`	net472	208μs	86.5ns	335ns	0.312	2.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	122μs	69.2ns	250ns	0	1.47 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	131μs	114ns	442ns	0	1.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	139μs	36.2ns	136ns	0.209	1.49 KB
#6072	`RunWafRealisticBenchmark`	net6.0	184μs	79.7ns	298ns	0	2.44 KB
#6072	`RunWafRealisticBenchmark`	netcoreapp3.1	197μs	63.4ns	246ns	0	2.39 KB
#6072	`RunWafRealisticBenchmark`	net472	209μs	43.7ns	169ns	0.313	2.46 KB
#6072	`RunWafRealisticBenchmarkWithAttack`	net6.0	122μs	109ns	407ns	0	1.47 KB
#6072	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	132μs	124ns	464ns	0	1.46 KB
#6072	`RunWafRealisticBenchmarkWithAttack`	net472	139μs	29.6ns	115ns	0.208	1.49 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6072

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	255 KB	262.74 KB	7.74 KB	3.04%

Fewer allocations 🎉 in #6072

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	59.51 KB	59.07 KB	-440 B	-0.74%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	62.9μs	892ns	8.78μs	43.44 KB
master	`StringConcatBenchmark`	netcoreapp3.1	61.8μs	822ns	8.01μs	42.64 KB
master	`StringConcatBenchmark`	net472	37.1μs	88.1ns	318ns	59.51 KB
master	`StringConcatAspectBenchmark`	net6.0	307μs	1.43μs	5.71μs	255 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	307μs	5.83μs	57.7μs	254.65 KB
master	`StringConcatAspectBenchmark`	net472	268μs	5.76μs	54.7μs	278.53 KB
#6072	`StringConcatBenchmark`	net6.0	61μs	707ns	7.07μs	43.44 KB
#6072	`StringConcatBenchmark`	netcoreapp3.1	62.3μs	929ns	9.25μs	42.64 KB
#6072	`StringConcatBenchmark`	net472	37.1μs	103ns	385ns	59.07 KB
#6072	`StringConcatAspectBenchmark`	net6.0	307μs	1.44μs	8.54μs	262.74 KB
#6072	`StringConcatAspectBenchmark`	netcoreapp3.1	341μs	1.93μs	13.1μs	253.52 KB
#6072	`StringConcatAspectBenchmark`	net472	272μs	5.84μs	56.1μs	278.53 KB

andrewlock · 2024-09-24T22:26:40Z

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6072) (11.127M)   : 0, 11127314
    master (10.930M)   : 0, 10929935
    benchmarks/2.9.0 (11.081M)   : 0, 11080577

    section Automatic
    This PR (6072) (7.215M)   : 0, 7215230
    master (7.314M)   : 0, 7313759
    benchmarks/2.9.0 (7.732M)   : 0, 7732233

    section Trace stats
    master (7.732M)   : 0, 7732033

    section Manual
    master (10.861M)   : 0, 10861024

    section Manual + Automatic
    This PR (6072) (6.722M)   : 0, 6722369
    master (6.769M)   : 0, 6769200

    section DD_TRACE_ENABLED=0
    master (10.060M)   : 0, 10059907

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6072) (9.555M)   : 0, 9554813
    master (9.514M)   : 0, 9514093
    benchmarks/2.9.0 (9.798M)   : 0, 9798067

    section Automatic
    This PR (6072) (6.343M)   : 0, 6343441
    master (6.435M)   : 0, 6435454

    section Trace stats
    master (6.873M)   : 0, 6873207

    section Manual
    master (9.470M)   : 0, 9470296

    section Manual + Automatic
    This PR (6072) (6.182M)   : 0, 6181663
    master (6.063M)   : 0, 6063264

    section DD_TRACE_ENABLED=0
    master (8.925M)   : 0, 8925381

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6072) (10.133M)   : 0, 10132522
    master (10.100M)   : 0, 10100102
    benchmarks/2.9.0 (10.067M)   : 0, 10067315

    section Automatic
    This PR (6072) (6.865M)   : 0, 6864790
    master (6.811M)   : 0, 6810644
    benchmarks/2.9.0 (7.552M)   : 0, 7552193

    section Trace stats
    master (7.461M)   : 0, 7461037

    section Manual
    master (10.234M)   : 0, 10233774

    section Manual + Automatic
    This PR (6072) (6.414M)   : 0, 6413915
    master (6.269M)   : 0, 6268679

    section DD_TRACE_ENABLED=0
    master (9.440M)   : 0, 9440381

andrewlock · 2024-09-24T23:38:03Z

Benchmarks Report for tracer 🐌

Benchmarks for #6072 compared to master:

2 benchmarks are faster, with geometric mean 1.180
2 benchmarks are slower, with geometric mean 1.160
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.83μs	44.1ns	296ns	0.0153	0.00765	0	5.43 KB
master	`StartStopWithChild`	netcoreapp3.1	9.8μs	55ns	348ns	0.0187	0.00935	0	5.62 KB
master	`StartStopWithChild`	net472	16.1μs	52.5ns	203ns	1.02	0.303	0.0958	6.06 KB
#6072	`StartStopWithChild`	net6.0	7.61μs	41.6ns	250ns	0.0153	0.00764	0	5.42 KB
#6072	`StartStopWithChild`	netcoreapp3.1	9.96μs	56.4ns	391ns	0.015	0.005	0	5.61 KB
#6072	`StartStopWithChild`	net472	16μs	44.9ns	174ns	1.01	0.298	0.0885	6.06 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	469μs	287ns	1.11μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	644μs	210ns	812ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	831μs	578ns	2.16μs	0.417	3.3 KB
#6072	`WriteAndFlushEnrichedTraces`	net6.0	483μs	251ns	940ns	0	2.7 KB
#6072	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	657μs	317ns	1.23μs	0	2.7 KB
#6072	`WriteAndFlushEnrichedTraces`	net472	832μs	839ns	3.25μs	0.414	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	193μs	1.05μs	6.04μs	0.199	18.45 KB
master	`SendRequest`	netcoreapp3.1	216μs	1.24μs	10.1μs	0.202	20.61 KB
master	`SendRequest`	net472	0.00598ns	0.00129ns	0.005ns	0	0 b
#6072	`SendRequest`	net6.0	194μs	1.07μs	7.74μs	0.193	18.45 KB
#6072	`SendRequest`	netcoreapp3.1	217μs	1.21μs	7.97μs	0.217	20.61 KB
#6072	`SendRequest`	net472	0.000854ns	0.000443ns	0.00166ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	554μs	2.21μs	8.55μs	0.548	0	0	41.61 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	684μs	3.45μs	15.8μs	0.332	0	0	41.74 KB
master	`WriteAndFlushEnrichedTraces`	net472	844μs	3.99μs	15.4μs	8.17	2.45	0.408	53.33 KB
#6072	`WriteAndFlushEnrichedTraces`	net6.0	571μs	1.3μs	4.87μs	0.573	0	0	41.62 KB
#6072	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	688μs	3.15μs	13.7μs	0.363	0	0	41.66 KB
#6072	`WriteAndFlushEnrichedTraces`	net472	828μs	2.34μs	8.75μs	8.39	2.52	0.419	53.3 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.27μs	1.42ns	5.51ns	0.014	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.77μs	1.08ns	4.17ns	0.0133	1.02 KB
master	`ExecuteNonQuery`	net472	2.06μs	2.17ns	8.41ns	0.156	987 B
#6072	`ExecuteNonQuery`	net6.0	1.32μs	0.752ns	2.81ns	0.0145	1.02 KB
#6072	`ExecuteNonQuery`	netcoreapp3.1	1.8μs	1.95ns	7.54ns	0.0135	1.02 KB
#6072	`ExecuteNonQuery`	net472	2.19μs	2.45ns	9.47ns	0.156	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.12μs	0.569ns	2.21ns	0.0136	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.53μs	4.27ns	16.5ns	0.013	976 B
master	`CallElasticsearch`	net472	2.61μs	2.16ns	8.1ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.27μs	1.03ns	3.84ns	0.0135	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.7μs	1.61ns	6.01ns	0.0133	1.02 KB
master	`CallElasticsearchAsync`	net472	2.53μs	1.11ns	4.3ns	0.167	1.05 KB
#6072	`CallElasticsearch`	net6.0	1.2μs	0.935ns	3.5ns	0.0134	976 B
#6072	`CallElasticsearch`	netcoreapp3.1	1.51μs	1.09ns	4.24ns	0.0128	976 B
#6072	`CallElasticsearch`	net472	2.47μs	2.32ns	9ns	0.158	995 B
#6072	`CallElasticsearchAsync`	net6.0	1.22μs	0.874ns	3.27ns	0.0134	952 B
#6072	`CallElasticsearchAsync`	netcoreapp3.1	1.58μs	0.62ns	2.32ns	0.0134	1.02 KB
#6072	`CallElasticsearchAsync`	net472	2.61μs	1.07ns	3.86ns	0.167	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6072

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0	1.169	1,357.70	1,161.29

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.36μs	0.919ns	3.44ns	0.0136	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.57μs	0.542ns	1.88ns	0.0125	952 B
master	`ExecuteAsync`	net472	1.9μs	1.16ns	4.51ns	0.145	915 B
#6072	`ExecuteAsync`	net6.0	1.16μs	0.965ns	3.74ns	0.0134	952 B
#6072	`ExecuteAsync`	netcoreapp3.1	1.61μs	0.709ns	2.65ns	0.012	952 B
#6072	`ExecuteAsync`	net472	1.85μs	0.864ns	3.34ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.15μs	1.19ns	4.45ns	0.0312	2.22 KB
master	`SendAsync`	netcoreapp3.1	4.97μs	1.57ns	5.89ns	0.0375	2.76 KB
master	`SendAsync`	net472	7.85μs	2.62ns	10.1ns	0.499	3.15 KB
#6072	`SendAsync`	net6.0	4.35μs	0.911ns	3.41ns	0.0303	2.22 KB
#6072	`SendAsync`	netcoreapp3.1	5.08μs	1.96ns	7.08ns	0.038	2.76 KB
#6072	`SendAsync`	net472	7.69μs	2.88ns	11.2ns	0.498	3.15 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.52μs	1.02ns	3.93ns	0.0229	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.25μs	1.03ns	3.84ns	0.0226	1.64 KB
master	`EnrichedLog`	net472	2.46μs	1.52ns	5.87ns	0.25	1.57 KB
#6072	`EnrichedLog`	net6.0	1.45μs	0.867ns	3.36ns	0.0232	1.64 KB
#6072	`EnrichedLog`	netcoreapp3.1	2.19μs	1.75ns	6.8ns	0.022	1.64 KB
#6072	`EnrichedLog`	net472	2.62μs	0.832ns	3.11ns	0.25	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	114μs	203ns	787ns	0.0567	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	119μs	167ns	647ns	0	0	4.28 KB
master	`EnrichedLog`	net472	148μs	192ns	743ns	0.661	0.22	4.46 KB
#6072	`EnrichedLog`	net6.0	115μs	128ns	479ns	0.0574	0	4.28 KB
#6072	`EnrichedLog`	netcoreapp3.1	118μs	166ns	597ns	0.0585	0	4.28 KB
#6072	`EnrichedLog`	net472	148μs	202ns	783ns	0.667	0.222	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.94μs	1.22ns	4.41ns	0.0311	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.03μs	1.21ns	4.7ns	0.03	2.2 KB
master	`EnrichedLog`	net472	5.02μs	1.86ns	7.19ns	0.319	2.02 KB
#6072	`EnrichedLog`	net6.0	3.06μs	2.67ns	10.3ns	0.0298	2.2 KB
#6072	`EnrichedLog`	netcoreapp3.1	4.08μs	3.41ns	13.2ns	0.0284	2.2 KB
#6072	`EnrichedLog`	net472	4.86μs	1.46ns	5.66ns	0.319	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`SendReceive`	net6.0	1.42μs	1.25ns	4.86ns	0.0163	0	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.71μs	0.83ns	3.11ns	0.0153	0	1.14 KB
master	`SendReceive`	net472	2.06μs	0.875ns	3.39ns	0.184	0	1.16 KB
#6072	`SendReceive`	net6.0	1.38μs	0.627ns	2.43ns	0.0159	0	1.14 KB
#6072	`SendReceive`	netcoreapp3.1	1.68μs	1.87ns	7ns	0.0151	0	1.14 KB
#6072	`SendReceive`	net472	2.17μs	1.97ns	7.37ns	0.183	0.00108	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.69μs	0.917ns	3.55ns	0.0215	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	4μs	1.58ns	5.91ns	0.0219	1.65 KB
master	`EnrichedLog`	net472	4.36μs	1.75ns	6.54ns	0.323	2.04 KB
#6072	`EnrichedLog`	net6.0	2.82μs	1.49ns	5.58ns	0.0227	1.6 KB
#6072	`EnrichedLog`	netcoreapp3.1	3.95μs	1.79ns	6.7ns	0.0217	1.65 KB
#6072	`EnrichedLog`	net472	4.32μs	1.4ns	5.23ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6072

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1	1.165	544.65	634.34

Faster 🎉 in #6072

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0	1.192	558.11	468.33

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	393ns	0.165ns	0.619ns	0.00804	576 B
master	`StartFinishSpan`	netcoreapp3.1	545ns	0.217ns	0.783ns	0.00768	576 B
master	`StartFinishSpan`	net472	680ns	0.509ns	1.97ns	0.0916	578 B
master	`StartFinishScope`	net6.0	558ns	0.214ns	0.829ns	0.00978	696 B
master	`StartFinishScope`	netcoreapp3.1	657ns	0.477ns	1.85ns	0.0093	696 B
master	`StartFinishScope`	net472	835ns	0.834ns	3.12ns	0.104	658 B
#6072	`StartFinishSpan`	net6.0	403ns	0.227ns	0.879ns	0.00808	576 B
#6072	`StartFinishSpan`	netcoreapp3.1	634ns	0.445ns	1.72ns	0.00776	576 B
#6072	`StartFinishSpan`	net472	671ns	0.788ns	3.05ns	0.0915	578 B
#6072	`StartFinishScope`	net6.0	468ns	0.273ns	1.06ns	0.00975	696 B
#6072	`StartFinishScope`	netcoreapp3.1	668ns	0.574ns	2.15ns	0.00972	696 B
#6072	`StartFinishScope`	net472	791ns	0.744ns	2.68ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6072

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1	1.154	890.51	1,027.95

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	572ns	0.289ns	1.12ns	0.00984	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	888ns	1.9ns	7.36ns	0.00947	696 B
master	`RunOnMethodBegin`	net472	1.12μs	1.09ns	4.21ns	0.104	658 B
#6072	`RunOnMethodBegin`	net6.0	596ns	0.458ns	1.71ns	0.00966	696 B
#6072	`RunOnMethodBegin`	netcoreapp3.1	1.03μs	1.07ns	4.02ns	0.00923	696 B
#6072	`RunOnMethodBegin`	net472	1.16μs	0.67ns	2.51ns	0.104	658 B

fix:: must be an array to waf

e-n-0

LGTM 😄

NachoEchevarria

LGTM!

## Summary of changes [Read waf actions](https://datadoghq.atlassian.net/browse/APPSEC-54989) per file and remove per file ## Reason for change ## Implementation details ## Test coverage ## Other details

anna-git force-pushed the anna/asm/actions-per-file branch 2 times, most recently from 5ef0d02 to 66b9998 Compare September 30, 2024 15:42

anna-git added 6 commits October 1, 2024 11:10

change mechanism

4b5f83c

fix unit tests

29218d2

Change the way we respond to action duplicates and store them by files

a4100ba

Improve update result

a37bb29

fix:: must be an array to waf

in the end dont handle duplicates in tracer

8098a61

it's a fact

e0807d5

anna-git force-pushed the anna/asm/actions-per-file branch from 2ed6be7 to e0807d5 Compare October 1, 2024 09:10

anna-git changed the title ~~change mechanism~~ [ASM][RCM] Read waf actions per file and remove per file Oct 1, 2024

anna-git marked this pull request as ready for review October 1, 2024 14:56

anna-git requested a review from a team as a code owner October 1, 2024 14:56

e-n-0 approved these changes Oct 1, 2024

View reviewed changes

NachoEchevarria approved these changes Oct 1, 2024

View reviewed changes

daniel-romano-DD approved these changes Oct 1, 2024

View reviewed changes

anna-git merged commit 255e567 into master Oct 2, 2024
75 checks passed

anna-git deleted the anna/asm/actions-per-file branch October 2, 2024 08:07

github-actions bot added this to the vNext-v3 milestone Oct 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ASM][RCM] Read waf actions per file and remove per file #6072

[ASM][RCM] Read waf actions per file and remove per file #6072

anna-git commented Sep 24, 2024 •

edited

Loading

datadog-ddstaging bot commented Sep 24, 2024 •

edited

Loading

andrewlock commented Sep 24, 2024 •

edited

Loading

andrewlock commented Sep 24, 2024 •

edited

Loading

Raw results

Slower ⚠️ in #6072

Raw results

Raw results

More allocations ⚠️ in #6072

Fewer allocations 🎉 in #6072

Raw results

andrewlock commented Sep 24, 2024 •

edited

Loading

andrewlock commented Sep 24, 2024 •

edited

Loading

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6072

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6072

Faster 🎉 in #6072

Raw results

Slower ⚠️ in #6072

Raw results

e-n-0 left a comment

NachoEchevarria left a comment

[ASM][RCM] Read waf actions per file and remove per file #6072

[ASM][RCM] Read waf actions per file and remove per file #6072

Conversation

anna-git commented Sep 24, 2024 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Sep 24, 2024 • edited Loading

Datadog Report

andrewlock commented Sep 24, 2024 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock commented Sep 24, 2024 • edited Loading

Benchmarks Report for appsec 🐌

Benchmark details

Raw results

Slower ⚠️ in #6072

Raw results

Raw results

More allocations ⚠️ in #6072

Fewer allocations 🎉 in #6072

Raw results

andrewlock commented Sep 24, 2024 • edited Loading

Throughput/Crank Report ⚡

andrewlock commented Sep 24, 2024 • edited Loading

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6072

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6072

Faster 🎉 in #6072

Raw results

Slower ⚠️ in #6072

Raw results

e-n-0 left a comment

Choose a reason for hiding this comment

NachoEchevarria left a comment

Choose a reason for hiding this comment

anna-git commented Sep 24, 2024 •

edited

Loading

datadog-ddstaging bot commented Sep 24, 2024 •

edited

Loading

andrewlock commented Sep 24, 2024 •

edited

Loading

andrewlock commented Sep 24, 2024 •

edited

Loading

andrewlock commented Sep 24, 2024 •

edited

Loading

andrewlock commented Sep 24, 2024 •

edited

Loading