Result formatting for archiving

[mfeit-internet2]

Rayid Tartir at UGA asked if there was a way to dump individual test results for measurements seen in Grafana. I replied:

The data that forms the result of a measurement is a blob of JSON that gets sent through the archiving process and Logstash/OpenSearch/Grafana. So it would be possible to pull those records out of OpenSearch, but all you’d have is the JSON that isn’t very human-readable. pScheduler has the ability to format JSON results into plain text (which you see when you run a test with the CLI) or HTML (which we don’t currently use for anything). To make it quickly-accessible, we’d either have to produce a formatted result with each measurement or have a way for Grafana to ask a pScheduler system to format them on demand. The former would eat up some compute time and storage during the result’s lifetime and the latter would have some complications surrounding which pScheduler server gets asked to do that and whether or not the plugins required are installed on it.

A good compromise might be to have a parameter that tells the archiver what formats to produce, which would happen on the first archive attempt and get stored for future ones so it doesn't have to repeat it. E.g.:

{
    "archiver": "xyz",
    "data": {  ...  },
    "ttl": "PT1H",
    "format": [ "text/plain", "text/html" ]
}

We'd have to figure out how to get Grafana to show it if it's available, but that's not for pScheduler to worry about.