Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds synchronous and asynchronous OpenTelemetry metrics #43

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

jahough
Copy link

@jahough jahough commented Nov 26, 2024

Problem Statement:

otelpgx, as of today, is only responsible for tracing observability.
Clients of pgx and otelpgx are forced to manage their own metric observability which can lead to:

  • Redundancy
  • Lesser granularity
  • Potentially varying degrees of non-uniformity across a system using pgx because of implementation redundancy

Currently, PR #18 exists to add PGX Stats as Asynchronous Observable OpenTelemetry metrics.
This PR, while also instruments the same asynchronous metrics, aims to do so with a few changes and alongside additional features:

This PR also implicitly resolves issue #40.


PR #18 seems to have left off awaiting information for how to have OTLP metrics scraped by Prometheus.
Those instructions, at least to me, seem outside the scope of otelpgx since that is more infrastructure-setup specifics, which OpenTelemetry-ecosystem tends to be agnostic to, and could become a chore to maintain long-term.
Because of that, I haven't included any specifics in the README as part of this PR; though if felt strongly otherwise, I can of course do so.

In my own environment (Application -> Collector <- Prometheus), I am able to verify metrics being reported.
See attached screenshots:

Screenshots

Screenshot 2024-11-22 at 4 32 30 PM

Screenshot 2024-11-22 at 4 35 32 PM

@jahough jahough marked this pull request as ready for review November 26, 2024 00:41
Copy link
Member

@obitech obitech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this comprehensive MR 🥳
In general it looks good, just a couple nits.

meter.go Outdated Show resolved Hide resolved
meter.go Show resolved Hide resolved
@obitech obitech requested review from costela and obitech December 10, 2024 08:51
Copy link
Member

@obitech obitech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks a lot for the great work here 🙏

@costela some final thoughts?

Copy link
Member

@costela costela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the great work! Just a few minor questions/suggestions 🙏

// WithAttributes specifies additional attributes to be added to the span.
func WithAttributes(attrs ...attribute.KeyValue) Option {
// WithTracerAttributes specifies additional attributes to be added to spans.
func WithTracerAttributes(attrs ...attribute.KeyValue) Option {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change, right? Can we maybe add a wrapper function with the old name and a Deprecated marker to avoid it?

pgxPoolTotalConnections = "pgxpool.total_connections"
)

// RecordStats records database statistics for provided pgxpool.Pool at the provided interval.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// RecordStats records database statistics for provided pgxpool.Pool at the provided interval.
// RecordStats records database statistics for provided pgxpool.Pool.

the interval is now AFAICT handled via WithMinimumReadDBStatsInterval?

}

// WithStatsMeterProvider sets meter provider to use for pgx stat metric collection.
func WithStatsMeterProvider(provider metric.MeterProvider) StatsOption {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, I'm a bit confused by this vs WithMeterProvider: is it worth it to have two ways of setting it? 🤔

// recordSpanError handles all error handling to be applied on the provided span.
// The provided error must be non-nil and not a sql.ErrNoRows error.
// Otherwise, recordSpanError will be a no-op.
func (t *Tracer) recordSpanError(span trace.Span, err error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICT we're not using the Tracer here? So why make it a method? What am I missing?

Comment on lines +62 to +83
err error

// Asynchronous Observable Metrics
acquireCount metric.Int64ObservableCounter
acquireDuration metric.Int64ObservableCounter
acquiredConns metric.Int64ObservableUpDownCounter
cancelledAcquires metric.Int64ObservableCounter
constructingConns metric.Int64ObservableUpDownCounter
emptyAcquires metric.Int64ObservableCounter
idleConns metric.Int64ObservableUpDownCounter
maxConns metric.Int64ObservableGauge
maxIdleDestroyCount metric.Int64ObservableCounter
maxLifetimeDestroyCount metric.Int64ObservableCounter
newConnsCount metric.Int64ObservableCounter
totalConns metric.Int64ObservableUpDownCounter

observeOptions []metric.ObserveOption
serverAddress = semconv.ServerAddress(db.Config().ConnConfig.Host)
serverPort = semconv.ServerPort(int(db.Config().ConnConfig.Port))
dbNamespace = semconv.DBNamespace(db.Config().ConnConfig.Database)
poolName = fmt.Sprintf("%s:%d/%s", serverAddress.Value.AsString(), serverPort.Value.AsInt64(), dbNamespace.Value.AsString())
dbClientConnectionPoolName = semconv.DBClientConnectionPoolName(poolName)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we just refactor the instantiations below into := and get rid of this big block? This smells a bit like C 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants