From a tech perspective I get *why* #Prometheus throws away metric metadata like "this is a gauge." (what do you do if the metric changes kinds later?)
But there's a lot of critical stuff you *do* with the data that depends on the kind of the metric. Using a `rate` on a gauge is (often) wrong, so you have to use naming conventions like "http_request_duration_total" to remind yourself that the metric is a cumulative counter. That sort of reliance on implicit / hinted information is a problem.
@joeblubaugh I'm hoping we can help improve the situation on #Prometheus Metadata by keeping track of them on a per-series basis in the WAL and informing remore_writw with something like
https://github.com/prometheus/prometheus/pull/11640
Do you think that would improve the situation, and/or would you be interested in giving some feedback?
@tpaschalis I suppose native histograms are the first kind of series where “there’s no going back”, huh? I guess that isn’t true for other measurements.
Metadata in the WAL is one thing, but it doesn’t really help me on the read/query side much, right?. I want to be able to do something like the Grafana query builder’s “kind” hints in a principled way. Right now it just depends on the metric name.