Skip to content
llm-speed

Privacy

Privacy & data handling

What we collect, what we don't, and how to run fully anonymously.

Last updated

llm-speed Privacy

Every byte the CLI uploads is listed below. Preview the exact JSON for any run before it ships:

llm-speed bench --quick --dry-run --print-payload

1. What we collect by default

Each upload contains exactly these fields. No others.

Hardware fingerprint (bucketed / coarse)

FieldExampleWhy
os_nameDarwin, Linux, Windowsleaderboards filter by OS family
os_version26, 13, 6 (major version only)major-version-level perf differences are real; patch level is identifying
cpu_modelApple M3 Pro, AMD Ryzen 9 7950X3Dthe canonical CPU label users compare on
cpu_cores12CPU-bound workloads scale here
ram_gb40 (rounded to nearest 8 GB)memory headroom matters; serial-number-grade precision doesn't
gpus[].nameRTX 4090, M3 Pro, MI300Xthe canonical GPU label
gpus[].kindnvidia, amd, apple, intel, cputells the leaderboard which family
gpus[].memory_gb24 (rounded to nearest 8 GB)VRAM bucket determines what models fit
accelerator_summaryM3 Pro (18-core GPU) + 36GB unifiedhuman-readable headline for the run page
fingerprint_hasha1b2c3d4e5f60708 (16 hex chars)groups runs of the same hardware class for outlier detection. Hashed over only the bucketed fields above — two physically-different machines with the same SoC + RAM bucket + major OS produce the SAME hash. NOT user-level identity. Omitted entirely in --strict-anon mode.
extras.backends{"llama.cpp": "...", "metal": "Metal 3"}so we can correlate decode tps to backend version

Workload results (the actual numbers)

For each workload (W1..W5):

FieldWhy
workloadwhich W1..W5
suite_versionwhich methodology version (suite-v1, ...)
backend / backend_versionllama.cpp / ollama / vllm / mlx / hosted-api + version
model.{name,size,quant,digest}the (model x quant) tuple is the leaderboard axis
ttft_ms, prefill_tps, decode_tps, decode_p50_latency_ms, decode_p95_latency_msthe benchmark metrics
prompt_tokens, output_tokens, batch_size, context_tokensworkload params for replay
wall_msend-to-end timing
prefix_cache_hit_ratewhen backend exposes it
error, flagscaptured failure mode + thermal/battery flags
extrassmall backend-specific telemetry; capped at 256 chars per string field by the client-side privacy invariant (see below)
extras.failure_kind (0.0.8)enum describing why a workload failed (oom, unsupported_arch, ...); failure-only runs upload only with --upload-failures
extras.peak_vram_mib, extras.peak_vram_source (0.0.8)peak GPU / MLX memory during the workload, rounded to 16 MiB (no total VRAM, which stays bucketed)
extras.time_to_first_answer_ms, extras.reasoning_tokens_est, extras.answer_started (0.0.8)timing of the first non-reasoning token; no text is uploaded
extras.llama_cpp_build, extras.llama_cpp_commit, extras.llama_cpp_build_source (0.0.8)exact llama.cpp build the numbers came from
extras.llama_server_args (0.0.8; 0.0.10 adds --no-cache-prompt, --cache-prompt, --cache-ram, --temp, --seed, --top-k, --top-p, --min-p, --reasoning, --kv-unified; the next release adds -lm/--load-mode, -lzm/--lazy-mode)allowlisted llama-server flags and values (e.g. -c 4096 --n-cpu-moe 20 --temp 0 --seed 42 --reasoning off); the 0.0.10 flags keep only numeric or on/off/auto values; --load-mode keeps only auto, none, mmap, mlock, mmap+mlock or dio, and --lazy-mode only on, auto or off; other flags and path-like or free-text values are redacted
extras.n_ctx_slot, extras.kv_unified (0.0.22; 0.0.23 keeps them on the row only)concurrent-decode only: the slot size llama-server reported (/slots or /props) and whether that server is using a unified KV cache (true, false, or null when it did not say). Integers and a boolean. No paths. Not repeated on each curve point
extras.concurrency_curve[].prefill_tps, prefill_tps_source, ttft_ms, ttft_max_ms (0.0.23)per batch: mean prefill tok/s and its basis (server_prompt_per_second or prompt_over_ttft, joined with + when a batch mixes them), the p50 time to first token across lanes, and the slowest lane, in milliseconds. Floats are rounded to 2 decimal places. A number or a short label, or null. No text. Only concurrent-decode uses the server prompt rate. The row-level prefill_tps and ttft_ms copy the successful 1-lane point. extras.row_metrics_basis is 1-lane, 1-lane point ran but failed, or no 1-lane point. At most 6 curve points per run
extras.decode_tps_stream (0.0.22)chunk-gap decode tok/s, (text chunks − 1) / (seconds from the first text chunk to the last). Always the stream-chunk rate. When llama-server timings are missing, decode_tps is this same rate and decode_tps_source is stream_chunks. When timings are used, this number is kept beside decode_tps so the two can be compared. A number, or null. No text
extras.decode_tps_source (0.0.22)which formula produced decode_tps: server_timings, server_predicted_n_ms, or stream_chunks. The same field is on each concurrent-decode point. A batch whose streams used more than one method joins those names with +. A short label, or null. No text. It rides the existing extras object
extras.sampling, extras.sampling_ignored (0.0.10)sampling settings the CLI sent with each request (temperature, seed, top_k, top_p, min_p; numbers only) and the names of requested keys the backend could not take; absent when none were set (presets default to temperature 0, seed 42)
extras.model_file, extras.model_file_bytes (0.0.8; 0.0.20)GGUF file basename (already sent as model.identifier) and its size. A split GGUF puts every shard basename in the same model_file string, comma-separated, when that list fits in 200 characters; otherwise the first shard, whose -00001-of-000NN name identifies the rest. model_file_bytes is the sum of the shards. No paths
extras.ssd_stream, extras.cpu_tensors, extras.peak_rss_bytes, extras.disk_read_bytes (0.0.13)only for a preset entry marked ssd_stream (weights streamed from disk): true; the exact allowlisted tensor names pinned to CPU (only per_layer_token_embd.weight; the field is dropped whole if any other value appears); the llama-server process's peak resident memory and bytes read from disk during the workload (integers). No paths, device or mount names

Fit attempts (--preset only, 0.0.8)

llm-speed bench --preset also keeps one row per attempt (model file, quant, context) in fit_attempts.json, so failures count as evidence ("does not fit at 16K"). The row holds: preset sweep id, model slug, GGUF basename (never a path), hardware slug (e.g. rtx-5090, m3-ultra-96gb; never the hostname), GPU/chip name, quant, context, GPU layers, --n-cpu-moe, status (ok / oom / failed / skipped) and error type, a short error line (path-scrubbed, max 200 characters), decode/prefill tok/s, time to first token and to first answer, reasoning-token estimate, total and peak GPU memory, memory in use before the model loaded (and by other processes at start), file size, backend + llama.cpp build/commit, NVIDIA driver major or macOS major, and CLI version. A manual llm-speed bench --preset run keeps these rows on disk only (there is no public fit-attempts upload route). Only machines running llm-speed agent (enrolled with a pairing code the queue operator approves, plus a one-time consent) send them, inside the job result to the queue they enrolled with; the official queue stores fit rows only from machines it approved as its own (community runners are switched off).

Provenance

FieldWhy
cli_versionso backend-version-specific anomalies are diagnosable
started_at, finished_atrun-duration sanity check
signature (ed25519)proves the bytes weren't edited after signing
public_key (ed25519, base64 32B raw)server verifies signature with this; persistent in default mode, ephemeral per run in --strict-anon

Contributor credit (opt-in, off by default)

If you choose to, you can add your X or Reddit handle to the runs you upload. Nothing is collected unless you type it in yourself.

FieldExampleWhy
credit.platformx or redditwhich site the handle belongs to
credit.handlename (from @name or u/name)shown next to your uploaded runs so you get credit
  • It is public. The handle is shown on llm-speed.com next to each run you upload with it, and anyone can see it.
  • It is unverified. We don't check that the account is yours, so the site labels it "unverified". A credited run is still a community report, never one of our own measured results.
  • It links your runs. Every run that carries the same handle can be grouped together by anyone, including runs from different machines.
  • How you give it. The CLI asks once, right after you agree to uploads, and only at an interactive terminal. Press Enter to stay anonymous. You can also set it with llm-speed config credit @name (or u/name), or for a single run with --credit. The CLI never reads it from git, your hostname, your username, environment variables or anywhere else.
  • When it is never sent. Never with --strict-anon or --anon, never from llm-speed agent jobs, and never with --no-credit.
  • Where it is stored. On your machine in ~/.config/llm-speed/config.json (mode 0600). It is sent in the upload request next to the signed run; it is not added to the signed run or to your local run files.
  • How to change or remove it. llm-speed config credit u/name changes it for future uploads, and llm-speed config credit --clear removes it, so future uploads are anonymous. Runs you already uploaded keep the handle until you ask us to remove it: email [email protected] with the handle (and, if you have them, the run ids) and we remove it from those runs within 7 days.

Storage drive (opt-in, off by default, 0.0.13)

Only if you turn it on, and only on runs that stream weights from disk (a preset entry with ssd_stream: true, recorded as extras.ssd_stream: true), the run adds one block, fingerprint.storage, describing the drive that holds the model file. When a model is too big for your GPU and RAM, its speed depends mostly on that drive. Every other run is never probed and never carries the block, even with the setting on.

FieldExampleWhy
storage.busnvme, sata, usb, virtual, otherhow the drive is connected
storage.pcie_gen4 (NVMe only, else null)PCIe generation of the drive's current link
storage.pcie_width4 (NVMe only, else null)PCIe lanes of the current link
storage.fsext4, xfs, btrfs, apfs, ntfs, 9p, otherfilesystem type the model file sits on
storage.wsl_9ptrue / falsethe model sits on a WSL /mnt/<drive> share (much slower reads)
  • No drive model name. The drive's model or friendly name is never read, sent or accepted (a user can rename a drive to anything, and model strings can carry serials).
  • On the server: if a run carries fingerprint.storage, it must have exactly the five keys above with the exact values listed and come from a run that streams weights from disk (a result with extras.ssd_stream: true); otherwise the API refuses the whole upload (HTTP 400, invalid fingerprint.storage) and stores nothing. Any other key, model included, is refused. The CLI only ever sends that exact shape. Nothing on llm-speed.com displays the block today.
  • Never collected: drive model or name, capacity, serial numbers, WWN/EUI, device paths, mount points, partition or volume ids, volume labels. Anything a probe can't read is null.
  • How you turn it on. The CLI asks once, right after the contributor-credit question, only at an interactive terminal; Enter means no. Or llm-speed config storage-info on|off, or bench --share-storage-info / --no-share-storage-info for one run. Saved as share_storage_info in ~/.config/llm-speed/config.json (mode 0600); only the JSON value true counts as on.
  • When it is never sent: on any run that doesn't stream weights from disk, with --strict-anon (even with --share-storage-info), from llm-speed agent jobs, in CI, or when the setting is unset. When it is off the CLI doesn't look at the drive at all.
  • How it is read. Linux: sysfs (which kind of device holds the file, and the NVMe controller's PCIe current_link_speed / current_link_width). macOS: diskutil info and system_profiler SPNVMeDataType SPSerialATADataType -json, reading only the device name to match the disk and the link fields (never the model or serial). Windows and WSL2: Windows' own powershell.exe by its fixed path (%SystemRoot%\System32\WindowsPowerShell\v1.0, or /mnt/c/Windows/System32/WindowsPowerShell/v1.0 from WSL; never one found on PATH or in the current folder), with Get-PhysicalDisk BusType and the controller's PCIe link; the script prints nothing else.
  • Presets. A bench --preset step that streams from disk tells its bench run which model file to look at through a local environment variable (LLM_SPEED_MODEL_PATH_LOCAL); the path itself is never uploaded or logged.
  • Local files. A run saved while it is on contains the same block (local runs are the same signed payload), only for runs that stream weights from disk.

Agent job uploads only: the job's upload nonce

Only runs uploaded by an llm-speed agent job (a machine you enrolled with a queue and consented for) carry one more field inside the signed run:

FieldExampleWhy
agent_job.assignment_idasg_0123456789abcdefthe queue's id for this job assignment
agent_job.upload_noncelsn_ + 32 hex charactersa random value the queue gave the machine for this one job
  • What it links. It lets the queue match an uploaded run to the job it gave out. The queue already knows which machine took that job and receives the run ids in the job result, so it adds no new identity; it does mean an agent job's --strict-anon run is tied to that job on the server. It contains nothing about your machine or you.
  • It is private. The server stores it privately (runs.agent_nonce) and never shows it on the site, in an API response or in a dataset export.
  • When it is sent. Only by agent jobs whose queue issued a nonce, only to that queue, and only while that job runs. Never by llm-speed bench run by hand, bench --resume, or any other upload. There is no setting for it.
  • Asking for it. The agent's job claims say it can use the nonce, with the fixed value "features": ["upload_nonce"] (the same for every install of this version; nothing about your machine). llm-speed bench sends no claim.
  • On your machine. Kept in memory for the job only: not in the job log, run files, status files or the job result, and never printed.

2. What we don't collect

The CLI does not send any of the following. They never leave your machine:

  • PCI bus IDs (pci.bus_id from nvidia-smi)
  • GPU driver build numbers (full driver_version strings)
  • GPU vBIOS versions
  • Kernel patch versions (6.8.0-31-generic → just 6)
  • OS patch level (26.3.1 → just 26)
  • macOS marketing minor/patch (13.5.7 → just 13)
  • Linux distro patchlevel (Ubuntu 24.04.1 LTS is dropped entirely)
  • Locale
  • Disk serial numbers, WWN/EUI, capacity, device paths, mount points, partition or volume ids and labels
  • Disk model or name (never, even with storage-drive info on)
  • Disk connection and filesystem, unless you opt in to storage-drive info (section 1)
  • Hostname / username
  • Power source / thermal state (captured locally for run flags, never sent)
  • Prompt text — the workload prompts are part of the open-source suite, not user data
  • Model output text — never captured at all
  • Per-token raw timings (raw_timings_ms) — kept locally for replay/dispute, only included in upload if you opt in via include_raw_timings=True

Privacy invariant (client-side)

Before any upload, the CLI walks the payload tree and refuses to send if any non-whitelisted string field exceeds 256 characters. The whitelist: accelerator_summary, error, fingerprint_hash, public_key, signature, suite_version, cli_version, model.{identifier,name,size,quant,digest}, workload, backend, backend_version. Anything else aborts the upload with upload payload contains unexpected long string at <path>; refusing to upload (privacy invariant).


Update check (0.0.14)

Once a day the CLI asks https://pypi.org/pypi/llm-speed/json whether a newer llm-speed exists, and installs it with your installer (pipx, uv, Homebrew or pip). This request goes to PyPI, not to llm-speed: it sends nothing to llm-speed servers and carries no run data, fingerprint or key. PyPI (and, for Homebrew installs, GitHub and Homebrew's servers, which brew upgrade contacts) see an ordinary download request from your IP address. Turn it off with LLM_SPEED_DISABLE_AUTOUPDATE=1 or llm-speed config set auto_update false; it is already off in CI and without a terminal. --anon and --strict-anon don't turn it off: they cover what a benchmark uploads, not this check. When pip, pipx or uv install the update they send PyPI their usual user agent (installer and Python versions, OS and similar platform details), as on any manual install; the CLI's own JSON check sends httpx's default user agent. Details: docs/CLI.md "Auto-update".

3. Three modes

Capabilitydefault--anon--strict-anon
Hardware bucket (OS major, CPU, RAM bucket, GPU name + VRAM bucket)yesyesyes
accelerator_summaryyesyesyes
fingerprint_hash (class-level, bucketed)yesyesno
public_key (persistent across your runs)yesyesno — ephemeral, rotated per run
User-Agent: llm-speed-cli/<version>sentsentnot sent (default httpx UA)
Authorization: Bearer <api-key>if providednot sentnot sent
X-LLM-Speed-Anon: 1 headernoyesno (would be a beacon)
Server-side IP loggingnot persistednot persistednot persisted
Cross-run correlation by the serverpossible via public_key and fingerprint_hashpossible via public_keynot possible
Contributor credit handle (opt-in)if you set onenever sentnever sent
Storage-drive block (opt-in; only on runs that stream weights from disk)if you turn it onif you turn it onnever sent

--anon vs --strict-anon

--anon is "soft anonymous": the persistent ed25519 keypair (and therefore public_key / fingerprint_hash) still ships, so the server can group your runs together. The Authorization: Bearer <api-key> header is dropped — your contributor account is not linked to the upload. Use --anon if you don't want the leaderboard to attribute the run to you but you're fine with the cross-run-correlation property the persistent keypair provides.

--strict-anon is "no identity at all": ephemeral keypair per run, no fingerprint hash, default User-Agent.

--strict-anon in detail

  • A fresh ed25519 keypair is generated in memory for each run. The persistent keypair on disk (~/.config/llm-speed/keys/ed25519.key) is never read or written.
  • No fingerprint_hash in the payload. Two consecutive --strict-anon runs from the same machine are unlinkable from the server's perspective.
  • The signature still proves the bytes weren't tampered with after signing — it just doesn't tie the run to a long-term identity.

--no-upload

  • No network call. The result is saved to ~/.cache/llm-speed/runs/<isoformat>-<fp_hash>.json.
  • Re-upload later with llm-speed submit <path> or llm-speed bench --resume <path> (both verify the signature locally and post the saved file unchanged). llm-speed submit with no path lists saved runs that have no local upload marker and uses the same consent prompt. The marker is a local sidecar only; it is not part of the upload, and the saved run is not modified.

4. Server-side

The fields the server stores are documented at /privacy.json (machine-readable, versioned).

  • We don't persist client IPs. Not in the database, not in application logs. Inbound request metadata (User-Agent, request path, response code) is logged at the Cloudflare-platform level per Cloudflare's privacy policy. We don't export or keep those logs. Each day our scripts ask Cloudflare for total request counts by page and by browser or app type, so we can count AI crawlers and visits from AI assistants. We keep only the counts and never look up IP addresses.
  • Rate-limiting uses an ephemeral hashed-IP counter. The hash is salted with a value that rotates daily and is held in memory only — never written to storage, never returned in any response. Cross-day correlation by IP is not possible.
  • Agent pairing keeps a keyed IP hash for about 70 minutes. Only for POST /v1/agent/pair (machines enrolling with a queue): to cap open pairings per address, the pairing row stores HMACs of your IP (IPv6 reduced to its /64, /56 and /48) under a key that changes every UTC day and is derived from a server secret. Never the IP itself, never returned in any response. It is deleted with the pairing row, which is removed by the first pairing or queue request after it is an hour past its 10-minute lifetime (there is no background job, so on a quiet day it can sit a little longer; the daily key change stops it matching any later request). Pairings made with an operator-issued enrolment token store no IP hash at all.
  • The server has no access to anything the CLI didn't put in the payload. Driver builds, PCI IDs, kernel patches, etc. are physically absent.

Assistant apps: ChatGPT app, Claude connector and Custom GPT

llm-speed offers read-only lookups for AI assistants: an MCP server at https://api.llm-speed.com/mcp (used by the llm-speed app in ChatGPT and by Claude connectors) and the same lookups as plain web requests (/v1/can-it-run, /v1/speed, /v1/vram-fit, described at /openapi.json) for the llm-speed Custom GPT.

AspectValue
What the assistant sends usonly the tool inputs: a model name, a GPU or Mac name, and optionally a quant, a memory size, a context size and which surface is calling (app or gpt)
What we never receiveyour conversation, your prompt text, your name, email or account, or any OpenAI/Anthropic user identifier. The assistant decides what to send; our tools accept no other fields and reject unknown ones
Accounts / sign-innone. No OAuth, no API key, no cookies
What we storenothing. The lookups don't write to our database and we don't log tool inputs ourselves
Rate limitingthe same daily-salted, in-memory hashed-IP counter as above (§4), held for one minute; the requester is normally OpenAI's or Anthropic's server, not your device
Platform logsCloudflare, which runs our servers, logs basic request details on its side under its own privacy policy. Our own server keeps no request logs; we only see total request and error counts.
Links in answerslinks back to llm-speed.com carry utm_source=chatgpt-app or utm_source=chatgpt-gpt so our cookieless analytics (§8) can count visits from each surface. They identify the surface, never you
What the answers containonly already-published benchmark data: measured runs (labelled as measured by llm-speed or community-submitted) and fit tests. No hardware fingerprints, public keys or file paths
The assistant providerOpenAI and Anthropic handle your conversation under their own privacy policies. We have no access to it

Run requests and email updates

Run requests. When you press "Request this run" we store the model and hardware you picked (and optional quantization or runtime), a vote count, and when the request was created and last voted on. We don't store your IP address: abuse limits use a short-lived, daily-salted hash of it held in Worker memory only, as described above. Requests may be protected by Cloudflare Turnstile.

Email is always optional. You can request a run or download the dataset without giving one.

Every address is confirmed first (double opt-in). Whichever form you use, we first email a confirmation link (valid for 7 days, usable once) and send nothing else to that address until you press Confirm. If you don't confirm, the sign-up is deleted after 7 days. Sign-up forms are protected by Cloudflare Turnstile and per-network rate limits; to enforce the hourly limit we keep a daily-rotating keyed hash of your IPv4 address or IPv6 /64 network (never the address itself) on an unconfirmed sign-up, and delete it when you confirm.

What we collect, and why:

  • "Get an email when this run is live." If you enter an address on a request, we store it with the model and hardware you requested so we can send you a notice when that run is published, once you've confirmed.
  • "Also send me new benchmark results (about monthly, unsubscribe anytime)" on the request form, or "Get dataset updates" on the data page. This box is unticked by default. Only if you tick it do we record that you opted in, when, and the version of the exact wording you agreed to, and it counts only after you confirm. Nobody gets updates without that tick and confirmation.
  • "Email me once when a run of … is posted" on a "measurement wanted" page (/requests/<model>/<hardware>). We store your address, the model and hardware you're watching, when you signed up, a hashed confirmation code and whether you confirmed. Once confirmed, we send one email when a matching run is posted and delete the address. Addresses are never shown on the site or returned by the API.

To limit abuse we also keep, for two days, a log of mail sent: a one-way hash of the address (not the address) and its domain, so one inbox gets at most three confirmation emails a day.

Addresses entered before sign-ups had this confirmation step are kept unconfirmed and are never emailed, not even to ask for confirmation. You can still have them deleted (see below).

Retention: unconfirmed sign-ups are deleted after 7 days; notice-only addresses are deleted after the notice is sent. Update list addresses are kept until you unsubscribe. Every email we send is plain text and carries a personal unsubscribe link that contains neither your address nor any code from the confirmation email; using it deletes your address from the list and from any pending notices straight away. You can also ask for deletion at [email protected].

We never sell, rent or share your email address, and we use it for nothing other than the purposes above. The sending provider we use (when sending is switched on) processes addresses only to deliver our mail.

5. How to inspect what's sent

llm-speed bench --quick --dry-run --print-payload

--dry-run runs the workloads but skips both upload and local save. --print-payload prints the exact JSON that would be uploaded to stdout.

You can also save without uploading and pretty-print the file:

llm-speed bench --quick --no-upload --json /tmp/run.json
cat /tmp/run.json | jq

The bytes signed by the CLI are exactly the bytes between the outer braces of that JSON, modulo the signature and public_key fields (which are excluded from canonical hashing). The server verifies against the same canonicalization.


6. How to delete your data

  • Self-serve consent revocation: rm ~/.config/llm-speed/consent.json — the next upload will re-prompt.
  • Contributor credit: llm-speed config credit --clear stops sending your handle. To remove it from runs you already uploaded, email [email protected] with the handle; we remove it within 7 days.
  • Email [email protected] with the id (the r_xxxxxxxx slug) of any result you want removed. We remove it within 7 days.
  • Public dispute thread: every result page has a dispute link. File a challenge; if it stands, the result is withdrawn.

--strict-anon submissions can't be linked to your machine after the fact (by design — there's no persistent identity to match against), so deletion of those runs requires the run id from your local cache.


6a. GDPR / EU data protection

For visitors and submitters in the EU, EEA, and UK:

  • Data controller: meadow-kun, the maintainer of llm-speed.com, reachable at [email protected] or via GitHub issues at meadow-kun/llm-speed. This is a single-maintainer open-source project; there is no separate legal entity behind the domain.

  • Legal basis (Article 6): legitimate interest under Art. 6(1)(f) for the benchmark payload (necessary to operate a public benchmark database whose value depends on aggregating submitted runs). First-run consent is also captured locally in ~/.config/llm-speed/consent.json before any upload, so the same processing also has consent under Art. 6(1)(a).

  • What's processed and retained:

    • Benchmark payloads (the fields enumerated in §1): retained indefinitely so historical leaderboards remain meaningful. You can request deletion of any specific run at any time by emailing the address above with the run id.
    • Edge access logs (Cloudflare's platform-level logs covering IP + User-Agent + path + response code): governed by Cloudflare's privacy policy; we do not export, query, or retain those logs ourselves.
    • Rate-limit hashed-IP counter: in-memory only, dropped after one minute; daily salt rotated and never persisted (see §4).
    • Agent pairing IP hash: keyed, daily-rotating HMAC on the pairing row, deleted with the pairing row, normally about 70 minutes after pairing (see §4).
    • Site analytics (Cloudflare Web Analytics, see §8): aggregated pageview counts, no per-visitor identifiers, no cookies.
  • Your rights under the GDPR:

    • Access (Art. 15) — request a copy of any benchmark payload tied to a public key you control.
    • Rectification (Art. 16) — file a public dispute on the run page.
    • Erasure (Art. 17) — request deletion via [email protected] with the run id.
    • Portability (Art. 20) — every benchmark payload is already public JSON at https://api.llm-speed.com/v1/results/<id> and signed by your own keypair.
    • Right to lodge a complaint with a supervisory authority (Art. 77).
  • International transfers: benchmark payloads live on Cloudflare D1 (SQLite-on-edge), replicated globally across Cloudflare's network for read performance. No primary copy in any specific jurisdiction; serving is from the closest edge.

  • No automated decision-making. Submitted runs are aggregated and ranked; no individual visitor is profiled or scored.

  • Children: this service is not directed at children under 16. We do not knowingly process data from children.

If your submission was made on a work machine and you need a sentence your DPO can point at: "The processing has a legitimate-interest legal basis (Art. 6(1)(f) GDPR), the data is technical-benchmark-only with no personal identifiers in the payload (see §1 and §2), retention is indefinite for the run record but immediate-deletion-on-request is supported, and the controller can be reached at [email protected]."


7. First-run consent

The first time you run llm-speed bench against the public API in a mode that will upload (i.e. not --no-upload, not --dry-run, not --json, not --strict-anon — strict-anon is implicit consent because it's the most-private option), the CLI prints the consent text and asks [Y/n]. The choice is saved to ~/.config/llm-speed/consent.json (mode 0600) along with a timestamp, the CLI version, and the trimmed fingerprint_hash. Delete that file to revoke.

Right after a yes, the CLI asks one optional question: whether you want credit by X or Reddit handle (see "Contributor credit" in section 1). Press Enter to stay anonymous. It is asked once (people who agreed with an earlier version are asked once on their next interactive upload), only at an interactive terminal, and never in CI, with --yes, --strict-anon or --anon, or in llm-speed agent jobs.

Right after that (0.0.13), one more optional question, default No: whether to add the storage-drive block, only on runs that stream weights from disk (see "Storage drive" in section 1). Asked once, only at an interactive terminal; never in CI, with --yes, --strict-anon, --share-storage-info / --no-share-storage-info, or in llm-speed agent jobs. Unset means off.


8. Site analytics

The website (llm-speed.com) uses Cloudflare Web Analytics to count pageviews. A small first-party shopping-interest counter is described below.

AspectValue
ProviderCloudflare, Inc. (the same edge that already terminates TLS for the site)
Cookiesnone
localStorage / sessionStoragenone (see Browser storage)
Client-side fingerprintingnone
Persistent client identifiernone
Client IP storagenot persisted (Cloudflare aggregates at the edge; the unaltered edge-log retention applies as already documented in §4)
Beacona small, cookieless script (static.cloudflareinsights.com/beacon.min.js, ~1 KB) loads in your browser and reports a pageview + page-load timing. It sets no cookies and stores no identifier. (Cloudflare Pages does not auto-inject it on this project, so it is wired explicitly in web/app/layout.tsx.)
Data fields collectedrequest path, response code, referring host (not full URL), country, browser family, page-load timing
Data NOT collectedsearch query terms, full referrer URL, screen resolution, mouse movements, session replay, cross-site identity
RetentionCloudflare aggregates, ~6 months on the free tier; docs/metrics/<iso>-analytics.json snapshots in this repo retain 7-day aggregates indefinitely as a public log
API token scopeAccount Analytics: Read, account-scoped to llm-speed only, used by tools/metrics_analytics.py for the daily snapshot
Opt-outblock static.cloudflareinsights.com (the beacon script) and/or cloudflareinsights.com (where it POSTs). Blocking either stops all pageview reporting; the rest of the site keeps working. The beacon never sees identifiable data.

The beacon also reports basic web vitals (LCP / CLS / INP) as part of the same cookieless request. If we ever change what analytics the site loads, this document and the CSP get updated in the same commit. We don't ship analytics features without disclosing them here.

Browser storage

The site saves a few preferences in your browser, like your theme and your recent model and GPU picks. They never leave your browser and aren't used for tracking. Clearing site data removes them.

Shopping-interest trial

The RTX 5090/M3 Ultra comparison article contains an ordinary Amazon US link. It has no affiliate tag and earns us no commission. We count when at least half of that shopping card becomes visible and when its link is activated, at most once each per page load. These are event counts, not unique visitors or purchases.

The browser sends only an event type (view or click), a random event receipt and a trial/QA label to /api/hardware-interest. Receipts are created separately for each event, are not saved in cookies or browser storage, and do not identify you across pages. Our existing Cloudflare storage retains the UTC date, event type and receipt for at most 45 days. We retain daily aggregate counts for review. The counter does not store IP addresses, page URLs, referrers, email addresses or merchant activity. Existing Cloudflare platform request handling in §4 still applies.

Do Not Track (1) and Global Privacy Control disable this counter. You can also block /api/hardware-interest; the retailer link continues to work. QA counts are kept separate. Visiting the retailer is your choice and its own privacy policy applies once you follow the link.

Retailer links

Some pages link to stores like Newegg or Amazon. These are plain links with no affiliate tag, and we earn nothing from them. We count how many times each link is clicked per day, and that's all. Nothing about you is saved.


9. Open source

The CLI has a public source repository. Public entry points include:

Website and API code, including the shopping-interest counter, are maintained in a separate internal repository. Their exact source is not currently available through the public CLI repository.

File an issue if anything in this document doesn't match the code.