Allow 16KB Request header field size for application gateway to align with Kestrel.

[rpetersson]

The current configuration for HTTP/2 header field sizes between Kestrel and Azure Application Gateway presents a discrepancy that may mislead developers. Specifically, Kestrel's HTTP/2 default maximum header size is set to 16 KB, while Azure Application Gateway enforces an 8 KB limit.

https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.server.kestrel.core.http2limits.maxrequestheaderfieldsize?view=aspnetcore-8.0#microsoft-aspnetcore-server-kestrel-core-http2limits-maxrequestheaderfieldsize

https://github.com/MicrosoftDocs/azure-docs/blob/main/includes/application-gateway-limits.md

This mismatch can lead to unforeseen issues during application deployment, as developers might design their applications around a 16 KB limit, only to encounter errors when deploying behind an Azure Application Gateway.

Expected Behavior: To avoid configuration conflicts, it would be beneficial to either:

Update Azure Application Gateway to support the default 16 KB header size of Kestrel for HTTP/2, or
Explicitly document this limitation prominently, so developers can adjust Kestrel settings accordingly during application setup or make the option available to adjust the HTTP/2 header field size in application gateway.

Impact: This discrepancy may result in:

Increased troubleshooting time during production deployments.
Misalignment of expectations, where developers believe they have a 16 KB limit when using Azure Application Gateway, only to face operational issues.

Additional configuration complexity, requiring manual intervention to align header size limits between Kestrel and Azure infrastructure.

Suggested Solution:

Align Configuration: Consider updating Azure Application Gateway to match Kestrel's 16 KB header size limit for HTTP/2.

Enhanced Documentation: If an alignment is not feasible, provide clear documentation on the header size limitation for Azure Application Gateway, ideally within both the Application Gateway and Kestrel configuration guidance.

Additional Context: For developers utilizing Azure Application Gateway as a primary ingress point, it is critical to ensure that any default
configurations align closely with underlying application frameworks, such as Kestrel, to minimize potential runtime issues and avoid unnecessary configuration adjustments.