Metrics / Reading
Reading
read gives series as they were written: each with its name, labels and
samples, every value bit for bit. latest gives each series' newest sample.
names, labelNames and labelValues list what a store holds, for the
menus of a dashboard, without reading a sample.
const series = await store.metrics.read({ name: 'cpu', where: { host: 'web-1' }, since: '1h' })
for (const { labels, times, values } of series) chart.line(labels.host, times, values)series = await store.metrics.read(name="cpu", where={"host": "web-1"}, since="1h")
for s in series:
chart.line(s.labels["host"], s.times, s.values)A series comes back with its samples as two columns: times, unix
milliseconds in a Float64Array, which a chart and new Date take, and
values, a Float64Array whose bytes keep every value's bits. Python gives
the same two arrays; Go and Rust a slice of samples. A NaN in values is a
value that was stored, never a missing one.
Conditions
const failing = await store.metrics.read({
name: 'http_requests_total',
since: '1h',
where: { status: ['500', '502'], env: { not: 'dev' }, host: { prefix: 'api-' } },
})
const host = await store.metrics.read({ where: { host: 'web-1' }, since: '10m' }) // every name of one hostfailing = await store.metrics.read(
name="http_requests_total",
since="1h",
where={"status": ["500", "502"], "env": {"not": "dev"}, "host": {"prefix": "api-"}},
)
host = await store.metrics.read(where={"host": "web-1"}, since="10m")failing, err := metrics.Read(ctx, store, metrics.Query{Name: "http_requests_total", Since: time.Hour,
Where: metrics.Where{"status": metrics.OneOf("500", "502"), "env": metrics.Not("dev"), "host": metrics.Prefix("api-")}})
host, err := metrics.Read(ctx, store, metrics.Query{Where: metrics.Where{"host": "web-1"}, Since: 10 * time.Minute})| In a query | Finds |
|---|---|
name | the series of this name; a timer's name, all it writes |
{ host: 'a' } | a label equal to this value |
['500', '502'] | equal to one of these |
{ not: 'dev' } | equal to none of these, or absent |
{ prefix: 'api-' } | starting with this text |
since | the span before now, '1h'; or from and to |
- Every condition holds. A query needs a
name, or a label's value, list or prefix, to find its series:notalone isinvalid, since it would read every series. There are no regular expressions. - A number in
whereis its text:{ status: 200 }finds"200". - A condition given as
undefinedisinvalid, never left out. - A query is data: the same object is kept in a dashboard's settings, sent over the wire, and read the same in every language.
The newest value
latest gives one sample a series, the newest in the range. A series without
one is left out, so the range bounds how stale an answer may be: a host that
went down leaves the answer rather than showing its last value for ever. It
reads one small piece of each series, however long the range.
What a store holds
const page = await store.metrics.names({ prefix: 'http_' })
// { names: [{ name, kind, seriesCount, unit, help, keep, retainedFrom, newest }, …], next }
const { labels } = await store.metrics.labelNames('http_requests_total') // ['route', 'status']
const { values } = await store.metrics.labelValues('http_requests_total', 'route', { prefix: '/api' })
await store.metrics.describe('cpu', { unit: '%', help: 'Busy time of a core.' })page = await store.metrics.names(prefix="http_")
labels = (await store.metrics.label_names("http_requests_total")).labels
values = (await store.metrics.label_values("http_requests_total", "route", prefix="/api")).values
await store.metrics.describe("cpu", unit="%", help="Busy time of a core.")page, err := metrics.Names(ctx, store, metrics.Prefix("http_"))
labels, err := metrics.LabelNames(ctx, store, "http_requests_total")
values, err := metrics.LabelValues(ctx, store, "http_requests_total", "route", metrics.Prefix("/api"))
err = metrics.Describe(ctx, store, "cpu", metrics.Description{Unit: "%", Help: "Busy time of a core."})let page = stats.names().prefix("http_").page()?;
let labels = stats.label_names("http_requests_total").page()?.labels;
let routes = stats.label_values("http_requests_total", "route").prefix("/api").page()?.values;
stats.describe("cpu", Description { unit: "%".into(), help: "Busy time of a core.".into() })?;None of them reads a sample: the file keeps what they give beside the series,
so a dashboard's menus stay fast however much the store holds. Pages are
1,000 entries unless limit says; after: page.next reads the next.
Cost before it runs
const plan = await store.metrics.explain({ name: 'cpu', since: '30d', every: '1h', op: 'avg' })
// { series: 40, bytes: 1310720, decoded: 960 }A query is bounded before it starts: series, bytes read and samples decoded
are separate budgets, and a query past one is limit, naming it, never a
smaller answer. explain says what a read or an aggregate would spend of
each, and in exceeds the bound it would stop at.
| Bound | A query's limit |
|---|---|
| series | 1,000 |
| bytes read | 16 MiB |
| samples decoded | 1,048,576 |
| an answer | 100,000 samples or buckets |
Removing a series
await store.metrics.delete({ name: 'cpu', labels: { host: 'web-1' } }) // true if it held that serieslet held = stats.delete("cpu", [("host", "web-1")])?;delete removes the one series of exactly these labels and all its samples.
The labels are always given, so a delete never reads as one of every series of
a name. A series whose bytes changed under it reads corrupt, stops taking
samples, and is listed by damaged; delete removes it.
See also
- Metrics: the engine's overview.
- Aggregates: rates, averages and percentiles by bucket.