Benchmarks

How TinyStore compares with the services it replaces: PostgreSQL, Redis, MinIO, VictoriaMetrics and Loki. The numbers come from one machine and one round of measurements of the 0.1.0 release. This page shows where TinyStore wins, where it loses, and what each test did.

TinyStore is not faster than each of these services at its own job. It replaces five processes with one, and an application on it uses a fraction of the memory. Under load it answers more requests, because a call does not cross a network. A single client that writes waits longer, because a write that touches two engines syncs two files.

#At rest

Each stack opens everything a small application uses, writes to each engine once, and then rests for a minute.

The memory is TinyStore's server, tinystore serve. With the core loaded into the program instead, open(dir, { embedded: true }), a Bun program holds 17 MiB more than Bun alone, which holds 38. Valkey alone holds 9 MiB at rest and Prometheus 101.

#An application

A request reads a user's session and a note, logs a line and counts itself. One request in ten also updates the note and queues a job to index it, and one in fifty uploads a 16 KiB attachment. A worker in the same program runs the jobs. Every write is on the disk before the request returns.

With one to eight clients the services answer more requests. A request that writes updates the note in app.db and queues its job in jobs.db, so it waits for two disk syncs, one per file. PostgreSQL writes the note and the job in one transaction and syncs once. Under load, TinyStore groups the writes of many requests into one sync per file, and wins. Every side kept every job and every log line in every test.

#One engine against its service

KV against Redis and Valkey, 128-byte values over 100,000 keys:

A read in process takes 6 µs, and through Redis 43 µs. A synced write waits for the disk on both sides: TinyStore's 'full' against Redis's appendfsync always. An 'os' write returns before the sync: TinyStore's 'os' against Redis's appendfsync everysec. Both survive a crash of the program, and Redis writes six times as many. With a million keys, Redis holds 211 MiB in memory at rest and TinyStore's server 18 MiB, with its 162 MiB file in the operating system's page cache.

SQL against PostgreSQL, one table of 100,000 rows:

Files against MinIO, through Bun's S3 client:

#Logs and metrics on disk

Logs and metrics are measured on the Rust core, because a JavaScript client would hide the engine behind its own encoding.

Metrics against VictoriaMetrics and Prometheus, TSBS DevOps: 1,010 series over three days, 26 million samples.

Logs against Loki, 13 datasets of LogHub, 2.6 GiB of text, each line at the time its own text spells:

Loki's queries cost more as their time range grows, and TinyStore's as the data grows: LogHub's BGL covers seven months, and HDFS's 11 million lines two days. zstd compresses the whole file at once, and nothing can be found in it without reading all of it.

#Where TinyStore loses

• A few clients that write. With one to eight clients the services answer twice the requests, because a write that touches two engines syncs two files. Keep a job in the same database as its rows, db.queue, to commit both in one file.
• Writes that do not wait for the disk. With 'os', kv writes stop at about 30,000 per second, where Redis reaches 190,000.
• Large files. MinIO reads a 1 MiB file 2.7 times as fast.
• Metrics on disk. VictoriaMetrics keeps a sample in 0.71 bytes, against TinyStore's 1.00.
• A search over many lines in a short range. Loki counts a word over HDFS's 11 million lines in half the time, because it reads on every core.

#How it was measured

Every test ran in one Linux container on an AMD Ryzen 7 7700 (8 cores, 16 threads) under Docker Desktop on WSL2, one stack at a time. Each figure is the median of three passes, or two for logs and metrics. The programs are written in Bun 1.4.2 and reach TinyStore through @tinyshed/tinystore@0.1.0 from npm. They reach the services through Bun's own Redis, PostgreSQL and S3 clients, the fastest a Bun program has. Logs and metrics use the Rust crate tinystore 0.1.0.

• Memory is the proportional set size (PSS) of every process of a stack, read ten times a second. PSS counts a page that several processes share once, so PostgreSQL's backends are not counted twice.
• A write that waits for the disk is compared only with a write that also waits: TinyStore's 'full' with Redis's appendfsync always and PostgreSQL's default. TinyStore's 'os' is compared with Redis's appendfsync everysec: both survive a crash of the program, and neither survives a power loss for the last moments.
• Each service runs as it ships, except where a test needs the same durability from both sides or the service would refuse the load. The report lists every change.
• The disk syncs in about 3 ms, slower than a server's disk. Every synced rate is this disk's, and the comparisons between contenders are what carries over.

The full method, every pass, the raw data and the commands that reproduce the round are in the round's report.

#See also

• Why TinyStore: what TinyStore is for, and what it is not.
• SQLite as TinyStore runs it: one writer per file, and why a write syncs once for many callers.
• How logs and metrics pack: what the bytes on disk are made of.
• Limits and defaults: every bound, durability among them.