Add dashboards for Alertmanager's mixin

[ArthurSens]

Feature request

It would be lovely if Alertmanager's mixins could include two dashboards:

1. Alertmanager monitoring: To monitor Alertmanager itself, it would include metrics that are exposed by Alertmanager.
2. Alertmanager alerts: An informational dashboard that tells us what alerts are currently at the firing state, split by severity. For this one, I think it would be necessary to have Alertmanager's Grafana Datasource plugin, so I'm not sure how viable it could be.

[beorn7]

Yeah, not sure either, if we should rely on a specific plugin in this mixin. The same is true for all other dashboards provided by mixins. Perhaps let's defer that to a possible native support for displaying alerts in Grafana.

About the general dashboards: Sure, that's something we should have eventually.

[ArthurSens]

Perhaps let's defer that to a possible native support for displaying alerts in Grafana.

Sounds good to me :)

Regarding the dashboard that monitors Alertmanager itself, could we start with a plain JSON? Or do you prefer that we should already start using jsonnet+grafonnet?

[beorn7]

The reason why I'm marked as the maintainer of so many mixins is mostly because I took the task of shepherding the many open mixin PRs into merging. I'm not really a stakeholders. Those are more a bunch of colleagues of mine at Grafana Labs, but also folks at Red Hat and elsewhere.

I'm saying that so that you can take my opinion with a grain of salt.

I'd say that just providing plain JSON is a good first step, but it's not much different from providing dashboards via https://grafana.com/grafana/dashboards . If you use grafonnet, you make the dashboards configurable. One might argue if jsonnet is the right way to do that, but since the alerts are already provided in jsonnet, I would do the same with dashboards, at least for now.

[ArthurSens]

I've practiced a little bit with grafonnet-lib this last week. I think I'll have no problem creating the dashboard with jsonnet instead of plain JSON :)

One might argue if jsonnet is the right way to do that, but since the alerts are already provided in jsonnet, I would do the same with dashboards, at least for now.

Well, the mixin design document does suggest jsonnet above all other options, I don't see why not use jsonnet 🤷

[xkilian]

As long as it is explained how someone not using jssonnet can also benefit from these dashboards, this is all good. (Not everyone uses it.)
I would also like to point out that there has not been a release of alertmanager in alsmot a year and the most useful feature "time based muting" has finaly been merged.
Is there something blocking release?

[paulfantom]

@xkilian when #2340 is finished dashboards from mixin will be available as JSON at https://monitoring.mixins.dev/alertmanager/#dashboards :)

[xkilian]

Nice, thank you. Doesn't this kind of overlap with what is in the grafana website for dashboards. Would be nice to see the grafan website permit publishing mixins and the individual elements.