-
Notifications
You must be signed in to change notification settings - Fork 145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Google Protobuf Instrumentation #6166
base: master
Are you sure you want to change the base?
Conversation
Datadog ReportBranch report: ✅ 0 Failed, 558462 Passed, 4621 Skipped, 46h 26m 49.25s Total Time |
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6166) - mean (69ms) : 66, 71
. : milestone, 69,
master - mean (69ms) : 65, 73
. : milestone, 69,
section CallTarget+Inlining+NGEN
This PR (6166) - mean (982ms) : 961, 1003
. : milestone, 982,
master - mean (979ms) : 952, 1006
. : milestone, 979,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6166) - mean (108ms) : 105, 110
. : milestone, 108,
master - mean (107ms) : 104, 110
. : milestone, 107,
section CallTarget+Inlining+NGEN
This PR (6166) - mean (679ms) : 663, 694
. : milestone, 679,
master - mean (678ms) : 663, 693
. : milestone, 678,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6166) - mean (91ms) : 90, 93
. : milestone, 91,
master - mean (91ms) : 89, 93
. : milestone, 91,
section CallTarget+Inlining+NGEN
This PR (6166) - mean (630ms) : 613, 647
. : milestone, 630,
master - mean (633ms) : 616, 650
. : milestone, 633,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6166) - mean (191ms) : 186, 196
. : milestone, 191,
master - mean (189ms) : 185, 193
. : milestone, 189,
section CallTarget+Inlining+NGEN
This PR (6166) - mean (1,096ms) : 1062, 1129
. : milestone, 1096,
master - mean (1,091ms) : 1058, 1124
. : milestone, 1091,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6166) - mean (276ms) : 271, 281
. : milestone, 276,
master - mean (275ms) : 270, 279
. : milestone, 275,
section CallTarget+Inlining+NGEN
This PR (6166) - mean (870ms) : 837, 902
. : milestone, 870,
master - mean (868ms) : 838, 898
. : milestone, 868,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6166) - mean (265ms) : 260, 269
. : milestone, 265,
master - mean (264ms) : 259, 270
. : milestone, 264,
section CallTarget+Inlining+NGEN
This PR (6166) - mean (845ms) : 808, 883
. : milestone, 845,
master - mean (846ms) : 817, 874
. : milestone, 846,
|
Throughput/Crank Report ⚡Throughput results for AspNetCoreSimpleController comparing the following branches/commits: Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red. Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards! gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6166) (11.119M) : 0, 11119281
master (11.080M) : 0, 11079759
benchmarks/2.9.0 (11.033M) : 0, 11032866
section Automatic
This PR (6166) (7.310M) : 0, 7310019
master (7.154M) : 0, 7154253
benchmarks/2.9.0 (7.786M) : 0, 7785853
section Trace stats
master (7.512M) : 0, 7512246
section Manual
master (11.035M) : 0, 11035311
section Manual + Automatic
This PR (6166) (6.699M) : 0, 6698601
master (6.611M) : 0, 6611049
section DD_TRACE_ENABLED=0
master (10.155M) : 0, 10155447
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6166) (9.620M) : 0, 9619604
master (9.567M) : 0, 9566967
benchmarks/2.9.0 (9.495M) : 0, 9494821
section Automatic
This PR (6166) (6.542M) : 0, 6541592
master (6.361M) : 0, 6360592
section Trace stats
master (6.540M) : 0, 6540289
section Manual
master (9.643M) : 0, 9642857
section Manual + Automatic
This PR (6166) (5.903M) : 0, 5902855
master (5.877M) : 0, 5877320
section DD_TRACE_ENABLED=0
master (8.928M) : 0, 8928120
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6166) (9.937M) : 0, 9937361
master (9.916M) : 0, 9915888
benchmarks/2.9.0 (10.020M) : 0, 10019592
section Automatic
This PR (6166) (6.436M) : 0, 6436371
master (6.337M) : 0, 6336520
benchmarks/2.9.0 (7.255M) : 0, 7255257
section Trace stats
master (6.963M) : 0, 6963468
section Manual
master (9.842M) : 0, 9842464
section Manual + Automatic
This PR (6166) (6.123M) : 0, 6122994
master (6.172M) : 0, 6171910
section DD_TRACE_ENABLED=0
master (9.319M) : 0, 9318507
|
Benchmarks Report for tracer 🐌Benchmarks for #6166 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ElasticsearchBenchmark - Slower
|
Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 | 1.150 | 1,177.74 | 1,354.03 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | CallElasticsearch |
net6.0 | 1.18μs | 0.454ns | 1.76ns | 0.0136 | 0 | 0 | 976 B |
master | CallElasticsearch |
netcoreapp3.1 | 1.53μs | 0.69ns | 2.58ns | 0.013 | 0 | 0 | 976 B |
master | CallElasticsearch |
net472 | 2.54μs | 1.82ns | 7.06ns | 0.157 | 0 | 0 | 995 B |
master | CallElasticsearchAsync |
net6.0 | 1.25μs | 0.747ns | 2.89ns | 0.0131 | 0 | 0 | 952 B |
master | CallElasticsearchAsync |
netcoreapp3.1 | 1.6μs | 1.34ns | 5.02ns | 0.0136 | 0 | 0 | 1.02 KB |
master | CallElasticsearchAsync |
net472 | 2.63μs | 1.55ns | 6ns | 0.166 | 0 | 0 | 1.05 KB |
#6166 | CallElasticsearch |
net6.0 | 1.35μs | 0.766ns | 2.97ns | 0.0136 | 0 | 0 | 976 B |
#6166 | CallElasticsearch |
netcoreapp3.1 | 1.48μs | 0.954ns | 3.57ns | 0.0128 | 0 | 0 | 976 B |
#6166 | CallElasticsearch |
net472 | 2.49μs | 1.83ns | 7.09ns | 0.157 | 0 | 0 | 995 B |
#6166 | CallElasticsearchAsync |
net6.0 | 1.18μs | 0.792ns | 2.96ns | 0.013 | 0 | 0 | 952 B |
#6166 | CallElasticsearchAsync |
netcoreapp3.1 | 1.69μs | 3.11ns | 12ns | 0.0143 | 0 | 0 | 1.02 KB |
#6166 | CallElasticsearchAsync |
net472 | 2.75μs | 1.81ns | 7.02ns | 0.167 | 0 | 0 | 1.05 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | ExecuteAsync |
net6.0 | 1.24μs | 0.389ns | 1.46ns | 0.0132 | 0 | 0 | 952 B |
master | ExecuteAsync |
netcoreapp3.1 | 1.63μs | 0.932ns | 3.49ns | 0.0131 | 0 | 0 | 952 B |
master | ExecuteAsync |
net472 | 1.79μs | 0.583ns | 2.26ns | 0.145 | 0 | 0 | 915 B |
#6166 | ExecuteAsync |
net6.0 | 1.18μs | 3.49ns | 13.5ns | 0.0134 | 0 | 0 | 952 B |
#6166 | ExecuteAsync |
netcoreapp3.1 | 1.71μs | 1.22ns | 4.71ns | 0.0127 | 0 | 0 | 952 B |
#6166 | ExecuteAsync |
net472 | 1.78μs | 0.521ns | 2.02ns | 0.145 | 0 | 0 | 915 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | SendAsync |
net6.0 | 4.34μs | 1.23ns | 4.43ns | 0.0324 | 0 | 0 | 2.31 KB |
master | SendAsync |
netcoreapp3.1 | 5.29μs | 4.28ns | 16.6ns | 0.0371 | 0 | 0 | 2.85 KB |
master | SendAsync |
net472 | 7.48μs | 1.35ns | 5.24ns | 0.495 | 0 | 0 | 3.12 KB |
#6166 | SendAsync |
net6.0 | 4.4μs | 1.08ns | 3.91ns | 0.0331 | 0 | 0 | 2.31 KB |
#6166 | SendAsync |
netcoreapp3.1 | 5.18μs | 1.78ns | 6.41ns | 0.0364 | 0 | 0 | 2.85 KB |
#6166 | SendAsync |
net472 | 7.38μs | 1.72ns | 6.2ns | 0.496 | 0 | 0 | 3.12 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 1.59μs | 0.988ns | 3.7ns | 0.023 | 0 | 0 | 1.64 KB |
master | EnrichedLog |
netcoreapp3.1 | 2.26μs | 1.77ns | 6.61ns | 0.0225 | 0 | 0 | 1.64 KB |
master | EnrichedLog |
net472 | 2.62μs | 2.25ns | 8.71ns | 0.25 | 0 | 0 | 1.57 KB |
#6166 | EnrichedLog |
net6.0 | 1.48μs | 0.922ns | 3.45ns | 0.023 | 0 | 0 | 1.64 KB |
#6166 | EnrichedLog |
netcoreapp3.1 | 2.27μs | 2.09ns | 7.83ns | 0.0216 | 0 | 0 | 1.64 KB |
#6166 | EnrichedLog |
net472 | 2.75μs | 2.6ns | 10.1ns | 0.249 | 0 | 0 | 1.57 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 116μs | 150ns | 582ns | 0 | 0 | 0 | 4.28 KB |
master | EnrichedLog |
netcoreapp3.1 | 120μs | 90.1ns | 337ns | 0.0602 | 0 | 0 | 4.28 KB |
master | EnrichedLog |
net472 | 150μs | 102ns | 393ns | 0.674 | 0.225 | 0 | 4.46 KB |
#6166 | EnrichedLog |
net6.0 | 117μs | 163ns | 632ns | 0.0578 | 0 | 0 | 4.28 KB |
#6166 | EnrichedLog |
netcoreapp3.1 | 120μs | 119ns | 445ns | 0 | 0 | 0 | 4.28 KB |
#6166 | EnrichedLog |
net472 | 151μs | 120ns | 463ns | 0.683 | 0.228 | 0 | 4.46 KB |
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 2.99μs | 1.1ns | 4.28ns | 0.03 | 0 | 0 | 2.2 KB |
master | EnrichedLog |
netcoreapp3.1 | 4.1μs | 1.98ns | 7.4ns | 0.0286 | 0 | 0 | 2.2 KB |
master | EnrichedLog |
net472 | 4.87μs | 1.25ns | 4.85ns | 0.32 | 0 | 0 | 2.02 KB |
#6166 | EnrichedLog |
net6.0 | 2.98μs | 1.57ns | 6.07ns | 0.0312 | 0 | 0 | 2.2 KB |
#6166 | EnrichedLog |
netcoreapp3.1 | 4.12μs | 8.29ns | 31ns | 0.0286 | 0 | 0 | 2.2 KB |
#6166 | EnrichedLog |
net472 | 5.11μs | 1.39ns | 5.37ns | 0.32 | 0 | 0 | 2.02 KB |
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | SendReceive |
net6.0 | 1.31μs | 1.06ns | 4.11ns | 0.0164 | 0 | 0 | 1.14 KB |
master | SendReceive |
netcoreapp3.1 | 1.86μs | 1.15ns | 4.47ns | 0.0149 | 0 | 0 | 1.14 KB |
master | SendReceive |
net472 | 2.04μs | 1.07ns | 4.14ns | 0.183 | 0 | 0 | 1.16 KB |
#6166 | SendReceive |
net6.0 | 1.39μs | 0.968ns | 3.75ns | 0.0159 | 0 | 0 | 1.14 KB |
#6166 | SendReceive |
netcoreapp3.1 | 1.76μs | 1.51ns | 5.64ns | 0.0151 | 0 | 0 | 1.14 KB |
#6166 | SendReceive |
net472 | 2.1μs | 0.546ns | 2.04ns | 0.183 | 0 | 0 | 1.16 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 2.74μs | 0.925ns | 3.58ns | 0.0219 | 0 | 0 | 1.6 KB |
master | EnrichedLog |
netcoreapp3.1 | 4.02μs | 1.23ns | 4.44ns | 0.0219 | 0 | 0 | 1.65 KB |
master | EnrichedLog |
net472 | 4.35μs | 3.57ns | 13.8ns | 0.324 | 0 | 0 | 2.04 KB |
#6166 | EnrichedLog |
net6.0 | 2.74μs | 0.838ns | 3.25ns | 0.0219 | 0 | 0 | 1.6 KB |
#6166 | EnrichedLog |
netcoreapp3.1 | 3.94μs | 16.5ns | 63.8ns | 0.0213 | 0 | 0 | 1.65 KB |
#6166 | EnrichedLog |
net472 | 4.34μs | 3.39ns | 13.1ns | 0.322 | 0 | 0 | 2.04 KB |
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | StartFinishSpan |
net6.0 | 408ns | 0.249ns | 0.964ns | 0.00798 | 0 | 0 | 576 B |
master | StartFinishSpan |
netcoreapp3.1 | 550ns | 0.262ns | 0.979ns | 0.00771 | 0 | 0 | 576 B |
master | StartFinishSpan |
net472 | 684ns | 0.411ns | 1.59ns | 0.0916 | 0 | 0 | 578 B |
master | StartFinishScope |
net6.0 | 484ns | 0.263ns | 1.02ns | 0.0097 | 0 | 0 | 696 B |
master | StartFinishScope |
netcoreapp3.1 | 647ns | 0.412ns | 1.43ns | 0.00941 | 0 | 0 | 696 B |
master | StartFinishScope |
net472 | 844ns | 1.56ns | 6.06ns | 0.104 | 0 | 0 | 658 B |
#6166 | StartFinishSpan |
net6.0 | 453ns | 0.29ns | 1.12ns | 0.00815 | 0 | 0 | 576 B |
#6166 | StartFinishSpan |
netcoreapp3.1 | 585ns | 0.328ns | 1.27ns | 0.00788 | 0 | 0 | 576 B |
#6166 | StartFinishSpan |
net472 | 680ns | 0.625ns | 2.42ns | 0.0915 | 0 | 0 | 578 B |
#6166 | StartFinishScope |
net6.0 | 486ns | 0.144ns | 0.557ns | 0.00981 | 0 | 0 | 696 B |
#6166 | StartFinishScope |
netcoreapp3.1 | 719ns | 0.472ns | 1.83ns | 0.00959 | 0 | 0 | 696 B |
#6166 | StartFinishScope |
net472 | 884ns | 0.694ns | 2.69ns | 0.104 | 0 | 0 | 658 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️
Slower ⚠️ in #6166
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0
1.148
591.88
679.33
Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 | 1.148 | 591.88 | 679.33 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | RunOnMethodBegin |
net6.0 | 592ns | 0.304ns | 1.18ns | 0.00985 | 0 | 0 | 696 B |
master | RunOnMethodBegin |
netcoreapp3.1 | 1.02μs | 0.561ns | 2.17ns | 0.00963 | 0 | 0 | 696 B |
master | RunOnMethodBegin |
net472 | 1.1μs | 0.312ns | 1.21ns | 0.104 | 0 | 0 | 658 B |
#6166 | RunOnMethodBegin |
net6.0 | 679ns | 0.158ns | 0.568ns | 0.00987 | 0 | 0 | 696 B |
#6166 | RunOnMethodBegin |
netcoreapp3.1 | 962ns | 2.84ns | 11ns | 0.00908 | 0 | 0 | 696 B |
#6166 | RunOnMethodBegin |
net472 | 1.16μs | 0.552ns | 2.14ns | 0.104 | 0 | 0 | 658 B |
40cb1af
to
daf650f
Compare
41f0d81
to
209f25c
Compare
## Summary of changes added the code of Microsoft.OpenApi to the tracer, removed/changed the parts that require more recent versions of the language/fwk than we have, and updated generated files. ## Reason for change It's going to be needed for the protobuf instrumentation in #6166, and I'm doing it in a separate PR to limit the size of the diff in the actual change. ## Implementation details @bouwkast ran the vendoring tool to add the files, since it's not working on Mac apparently ## Test coverage ## Other details When looking at the size of artefacts published on the CI, it looks like this is increasing the size of the dll by ~180KiB <!--⚠️ Note: where possible, please obtain 2 approvals prior to merging. Unless CODEOWNERS specifies otherwise, for external teams it is typically best to have one review from a team member, and one review from apm-dotnet. Trivial changes do not require 2 reviews. --> --------- Co-authored-by: Steven Bouwkamp <[email protected]>
72fd7b2
to
164139c
Compare
164139c
to
d2f7ece
Compare
…the items property
Summary of changes
Adds a new instrumentation for Google Protobuf (not https://github.com/protobuf-net/protobuf-net)
The instrumentation doesn't create any new spans, we are just adding tags on existing ones. The main one being a tag containing the full protobuf schema as an OpenAPI json. Since extracting the schema is costly, we only do it once every 30 seconds (hardcoded for now).
Reason for change
On request from DSM team, as an equivalent of its java counterpart: https://github.com/DataDog/dd-trace-java/tree/master/dd-java-agent/instrumentation/protobuf
Implementation details
DbConnectionCache
, where it's using a cache for small cardinality, but if there are too many different values, it stops caching.Test coverage
Added a sample app that serializes and deserializes a protobuf message. Spans are created manually since this integration doesn't create one.
Other details