Under the hood / One budget of memory
One budget of memory
A store runs inside your program, or beside it on the same machine. It must not take the machine's memory when a report reads a year of logs, and it must say so when it cannot answer, instead of answering less. This chapter explains the bounds every call has, the budget a store can be given, and what a limit error tells you.
Every call is bounded
Every call that could hold a lot of data has a bound of its own, whatever else is going on:
| Call | Bound |
|---|---|
SQL all, columns | 64 MiB of values; read more with each or a page |
a page of list | 1,000 keys, files, jobs or events |
| an aggregate of metrics | 100,000 buckets, and its own budgets of samples and bytes |
top and distinct of events | 64 MiB of different values |
| a value in kv, a job, a step | 1 MiB |
| an upload | a few buffers of 64 KiB to 1 MiB, whatever the file's size |
| a channel's messages | 64 MiB of the store's memory for all channels |
A call past its bound fails with a limit error. It never returns a smaller
answer that looks complete: a list cut at its bound with no word would show a
user's dashboard with data missing, and nobody would know.
A budget for the whole store
A bound per call does not stop ten calls at once from each taking their 64 MiB. A store can be given one budget of bytes that all its work shares:
let store = Store::open("data", Options { memory: Some(512 << 20), ..Options::default() })?;tinystore serve ./data --memory 512MiBWith a budget, work reserves the bytes it will hold before it allocates them,
and gives them back when it is done. A call whose reservation does not fit
waits up to ten seconds for other work to give memory back, then fails with
limit. A server that listens on the network has a budget of 1 GiB unless
--memory says otherwise, because its callers are not under your control.
Without a budget, as in an embedded store or the sidecar by default, every
call still has its own bound.
The budget counts what the store's work holds: the rows of an answer, a block being decoded, an upload's buffers. It is not a limit on the process's resident memory, which also holds your program and SQLite's page caches.
What a limit error says
A limit error names which bound it met, what the call wanted and what the
bound was:
The error's limit | It means | What to do |
|---|---|---|
bytes of a call's rows | one call's rows passed 64 MiB | read a page, or each |
store memory | one call alone wanted more than the whole budget | read a page, or raise the budget |
store memory, now | the call did not fit beside what others hold now | try again, or do less at once |
The last one is the only one that a retry can fix. The first two say the call is too big, and will be too big every time.
Why this design?
Why refuse instead of answering less. An answer cut silently is the worst kind of bug: tests with small data pass, and production quietly shows wrong numbers. A refusal is loud, names its bound, and says what to do.
Why reserve before allocating. If work allocated first and checked later, the check would come after the memory was already gone. Reserving first means a call that does not fit fails before it takes anything, and the store's other work goes on.
Why work that holds a writer never waits for memory. A commit that waited for memory while other work waited for that commit would deadlock. So work that holds a file's writer takes its memory or fails at once, and only reads wait. A large reservation is never passed over forever by small ones: waiters are served in order (contract STORE-05).
See also
- SQLite as TinyStore runs it: readers that close when idle, which also keeps memory down.
- Time and errors: every kind of error,
limitamong them.