RiffDB Handbook

RiffDB is a standalone, contract-first operational database. Applications declare entities, commands, invariants, outcomes, queries, and least-authority roles. They mutate state only by invoking compiled commands; RiffDB owns the concurrency checks, idempotency record, durable outcome, provenance, events, and projection progress.

RiffDB is currently a proof of concept supporting single-node operation and single-primary asynchronous follower replication. It is suitable for evaluation and local agent-built applications, not production workloads. Read Known Limitations, Compatibility, and Security before choosing an environment.

Choose a path

The application boundary

The transports do not define separate semantics. CLI, generated clients, gRPC, and MCP all enter the same application service and authorization boundary. MCP never opens storage directly.

First successful loop

From a source checkout:

cargo run -p riffdb-cli -- new item-desk
cd item-desk
../target/debug/riffdb dev --seed --run

The scaffold contains the contract, a bounded query, seed commands, exact lock, and generated clients. Continue with Your First Application for the review boundaries and installed workflow.

What the database proves

• Business invariants are evaluated by the database under concurrency.
• A retry keeps one idempotency identity and recovers the committed outcome.
• A commit atomically records mutations, outcome, events, provenance, and its commit-log entry.
• Queries execute against one authoritative snapshot with compiler-proven bounds.
• Projections expose frontiers so callers can request read-after-commit consistency explicitly.
• Credentials expose named operations and scopes, not ambient storage access.

What RiffDB Is

RiffDB moves operational state rules out of scattered application code and into a checked contract. The application asks for a named operation such as CreateTicket or AllocateBudget; it does not assemble arbitrary mutations.

Contract-first state

A contract declares:

• entities and exact value types;
• aggregates and the partition that owns a mutation;
• commands, inputs, preconditions, mutations, and closed outcomes;
• invariants and commit-time checks;
• durable events and outbox intents;
• indexes and projections; and
• symbolic roles that name allowed commands and queries.

The compiler resolves symbols, assigns internal identities, proves bounds, and produces an executable plan. Application authors review a lock containing those derived identities; they do not hand-author them.

Why commands instead of SQL writes

An application-level SQL statement can be individually valid while a workflow using it is unsafe: a read/check/write sequence can race, a retry can duplicate an effect, or two services can implement an invariant differently. RiffDB admits only compiled command plans and makes those properties part of database execution.

This does not make PostgreSQL unsafe. PostgreSQL can implement the same rules with careful transactions, constraints, locking, retry design, and application discipline. RiffDB’s claim is narrower: its public application surface removes the unchecked write path that permits common mistakes. See PostgreSQL Safety Comparison.

Reads and derived state

RiffQL provides bounded, symbolic operational queries. Named queries are compiled into immutable modules and execute in one snapshot. General SQL, unbounded scans, and analytical joins are outside the POC.

Entity state and the commit log are authoritative. Projections are derived and rebuildable. A projection frontier identifies the greatest commit incorporated, so a caller can wait for a known commit without treating projection state as the source of truth.

One semantic path

Generated Rust, Go, TypeScript, and Python clients, the CLI, public gRPC, and native MCP all use the shared application service. Authentication and authorization run before command execution. The deterministic runtime cannot use the network, filesystem, operating-system clock, process-global mutation, or untracked randomness.

POC boundary

The current implementation supports local and verified direct-TLS deployments, single-primary asynchronous follower replication, and explicit incarnation-fenced primary promotion. It deliberately excludes Raft or quorum consensus, automatic failover, multi-primary writes, distributed transactions, SQL compatibility, and built-in end-user OAuth or identity providers. Review Known Limitations for the complete current boundary.

Why RiffDB Exists

RiffDB is the data layer for the average multi-tenant SaaS application: safe transactions, schema migrations, event subscribers, analytics, vectors, and full-text search as opinionated native parts of one database, so that the patterns which normally require complex, risky external wiring are simply present, correct, and bounded. Its primary builders are expected to be AI agents, and every surface is designed to be learned from documentation and error messages at run time: typed outcomes, finite grammars, deploy-time rejection over runtime surprise.

The seams argument

The obvious objection: capable agents will happily wire Postgres, Kafka, Elasticsearch, and a vector store together, so an integrated database saves labor that is about to be free.

The objection mistakes what the wiring costs. Agent labor makes assembly free. It does not make seams correct. A five-service stack has no transaction across its parts, no freshness contract between its source of truth and its indexes, and a dual-write problem in every seam that no amount of careful glue can remove — it is removed by architecture (a changelog and a single total order) or it is not removed. Glue code produced quickly and confidently fails in the invisible ways: lost updates under concurrency, replay storms, search results that are wrong until a customer notices. And the operational bill remains: five failure domains, five upgrade cycles, five things to secure and monitor, and no coherent backup of the whole.

RiffDB’s answer is that the seams do not exist. Committed means indexed means deliverable, inside one total order, behind one typed freshness vocabulary, in one system with one backup unit. That is a lot of architecture for most projects to buy any other way — and most projects are exactly who this is for. In a world of capable agents, the stacks they wire together mass-produce the pathology this design removes by construction.

Scale is a placement change, not a semantics change

One process is how RiffDB runs today and is what the proof of concept proves. It is not the ceiling the design is drawn to.

The unit of ownership is already the partition route. A command declares the route it writes, every binding must derive the same route, and a write that crosses routes is refused at compile time rather than quietly becoming a distributed transaction. Conflict keys are functions of the aggregate root key inside that route, so two writers to one aggregate always serialize against each other and never against the rest of the database. That is the same shape a shard-per-core, elastically scheduled engine uses to place data — with the difference that here the boundary is declared in the contract and proven by the compiler rather than discovered under load.

So scaling out is meant to be a change in where a route lives, not a change in what a transaction means. A route keeps one owner. The total order that matters is the one inside it. Cross-route work stays exactly what it is today, which is not a transaction, and no amount of distribution turns it into one.

What RiffDB will not buy scale with is the usual currency. Tunable quorums, last-writer-wins conflict resolution, and read repair make a database elastic by making its guarantees probabilistic, and everything above this section depends on those guarantees being absolute. If a distributed mode cannot keep one owner per route and proven invariants inside it, the answer is a smaller distributed mode, not a weaker promise.

Single-primary asynchronous follower replication and explicit incarnation-fenced primary promotion are implemented today, while multi-node partitioned write placement and distributed consensus remain outside the proof of concept. The standing design test on every decision record — does this assume co-located storage or single-node memory — is what keeps distributed route placement open without requiring a semantic redesign.

Two boundaries that keep it honest

Opinions live only in the data layer. RiffDB’s capability model governs data authority — what an application principal may read and write. It does not and will not do end-user identity, sessions, sign-in flows, or OAuth; that is application land, and applications keep their own frameworks, languages, and auth providers. Platforms whose data layer is a commodity must expand upward to differentiate. RiffDB differentiates inside the data layer, so it can afford to stay narrow — and narrowness is load-bearing: a finite surface is a provable surface, and a provable surface is one an agent can be taught completely in context.

Everything is à la carte, and no one is trapped. Use the transactions and ignore the projections; use the events and ignore the search. When an application outgrows an opinionated pillar, the changelog is the graduation path: RiffDB remains the system of record and feeds the specialist system, rather than holding data hostage to its own feature ceiling. Export is a supported operation, not a negotiation.

What safety means here

Unsafety is unexpressible at the public surface, not discouraged (AGENTS.md, non-negotiable boundaries): no interactive transactions to leave half-open, no raw query language, no omittable tenant scoping, no way to opt out of durability on an acknowledged write. Internals may use unsafe machinery freely so long as the boundary holds. Every architecture decision record answers two standing design tests — can a developer or agent express an unsafe operation through this surface, and does this assume co-located storage or single-node memory — so the boundaries survive the people and agents who wrote them.

Claims here are meant to be checked, not believed: the decision records in adr/, the requirement inventory in SPEC.md, and the crash, recovery, and falsifiability evidence attached to each work package are the primary documents. This page is only the argument that connects them.

The same discipline applies inside the engine: a guarantee may be proven per datum or per process generation, never paid again per operation unless the operation itself is what is being guaranteed. Internal lanes pass bounded move-only proofs of already-validated material; the receiving lane rechecks only facts that can have changed, such as current authority, dependencies, conflicts, sequence, and durability. Architecture tests reject hot paths that decode, normalize, hash, or validate already-proven bytes a second time merely because ownership crossed an internal boundary.

Installation

For application-language package names and the common four-step workflow, see Driver and CLI Packages.

RiffDB’s alpha deployment target is Linux. Protected key and bearer loaders depend on Linux /proc/self/status and exact owner/mode checks. The convenience installer retains its loopback development profile; controlled remote application deployments can opt into verified direct TLS through explicit server configuration.

RiffDB is available under either the MIT License or the Apache License, Version 2.0, at your option. A verified release bundle includes the ownership notice and both license texts.

Copyright © 2026 Kevin O’Shea and O’Shea & Sons, LLC.

Source Checkout Convenience Install

Run the convenience commands from the repository root so Cargo reads .cargo/config.toml and its riffdb alias. Rust 1.98.1, Linux, systemd, Git, jq, a util-linux uuidgen with UUIDv7 support, and the ordinary utilities checked by the installer are required.

Use a clean, reviewed checkout for an attributable installation. When the checkout is clean, the installer embeds its exact Git revision. A dirty checkout is allowed for development, but its binaries are labeled development-unversioned and are not release evidence or attributable to a specific revision.

For a service owned by the current user:

cargo riffdb install --user
cargo riffdb bootstrap --user

To register the generic MCP developer capability with Codex during bootstrap:

cargo riffdb bootstrap --user --register-codex

To create several isolated databases under one service, provide every alias on the first install and then bootstrap them separately:

cargo riffdb install --user --database ea --database orders
cargo riffdb bootstrap --user --database ea
cargo riffdb bootstrap --user --database orders

To add a database to an existing installer-owned user or system installation, use the offline migration command:

cargo riffdb install --user --add-database ea
cargo riffdb bootstrap --user --database ea

The command stops an active service, preserves a legacy database as alias default, moves the legacy backup contents to backups/default, creates the new sibling root backups/ea, atomically publishes named configuration, and starts the service again. Its private phase journal makes the operation resumable after process or machine interruption. --no-start requires the service to already be inactive and leaves it inactive. The command accepts only an exact installer-generated configuration; it refuses customized TOML without changing it.

--database is repeatable on install, accepts the canonical [a-z][a-z0-9_-]{0,63} alias grammar, and is bounded to 32 unique installer-managed aliases (riffdbd itself supports up to 512 [databases.<alias>] tables with online configuration reload and optional idle_hibernate_seconds; see Multiple Databases). Repeating installation with explicit aliases requires the exact retained alias list. Only --add-database changes an existing installer-owned registry. The installer never edits custom server configuration.

Named bootstrap writes separate private files such as client-ea.toml, operator-ea.credential, mcp-ea.toml, and mcp-ea.credential. With --register-codex, it registers riffdb_ea. The unsuffixed filenames and registration name remain the compatibility form for alias default.

For a machine-wide service, select the same scope in both phases:

cargo riffdb install --system
cargo riffdb bootstrap --system

System scope is an administrator-facing POC path. Repository automation validates the shared release service assets but does not execute this source installer’s privileged account, /etc, or /usr/local flow. Perform the first privileged installation and service-start acceptance on the intended disposable or staging host, review every sudo prompt, and require authenticated RiffDB Health to report the exact deployment-required state before creating the first application.

Always invoke these as the intended unprivileged operator. Never use sudo cargo riffdb ...; the installer rejects a root caller. The system installer builds all three binaries as the operator and invokes sudo internally to inspect protected destination state, publish root-owned files, create the two service accounts and their directories, verify the unit, and control the system service. System bootstrap still runs as the operator and uses the public loopback API. The helper is mutable source-checkout code and is not suitable for a narrow sudoers allowlist. System installation assumes the operator already has ordinary administrative authorization for the displayed sudo operations. Privileged publication opens only root-owned destination paths; the unprivileged process opens and streams each checkout or build source.

Installed Paths

Bootstrap always writes the current operator’s private client files beneath ${XDG_CONFIG_HOME:-$HOME/.config}/riffdb/ and recovery state beneath ${XDG_STATE_HOME:-$HOME/.local/state}/riffdb/, including for a --system database. In particular, client.toml, operator.credential, mcp.toml, and mcp.credential are user-owned mode-0600 files. The same private state root retains operator-capability.id and mcp-capability.id for deliberate server-side revocation. System scope does not turn an operator credential into a root-owned secret.

User binaries follow the XDG-recommended ~/.local/bin convention. Add that directory to PATH for direct riffdb and riffdb-mcp invocations when the login environment does not already include it. The cargo riffdb bootstrap command uses the installed absolute paths and does not depend on that PATH entry.

Start Control and User Managers

Installation normally verifies, enables, and starts the selected service. To publish and verify files without reloading, enabling, starting, or restarting the service:

cargo riffdb install --user --no-start
cargo riffdb install --system --no-start

Bootstrap requires the selected service to be running and reachable. After a user --no-start installation, start it deliberately with:

systemctl --user daemon-reload
systemctl --user enable --now riffdbd.service

Every user-scope installation requires a valid existing XDG_RUNTIME_DIR owned by the current user with mode 0700; systemd-analyze --user verify uses it even with --no-start. Automatic user-service start or restart also requires a functioning per-user systemd manager, normally supplied by a PAM login session. install --user --no-start verifies the unit without contacting the live user bus. The installer does not call loginctl, enable lingering, or fabricate a runtime directory or user bus. On a headless or restricted SSH host, arrange the runtime directory, user manager, and linger policy explicitly, retain --no-start until the manager exists, or use the system scope. For system --no-start, an administrator later runs:

sudo systemctl daemon-reload
sudo systemctl enable --now riffdbd.service

Explicit Bootstrap Semantics

Install creates filesystem state and, unless --no-start is selected, starts an unbootstrapped server. It does not silently create database authority. The separate bootstrap command:

1. submits the one-time first-human bootstrap through loopback gRPC;
2. durably retains uncertainty-recovery material before submission;
3. writes private operator CLI configuration and credential files;
4. leaves the active contract catalog empty;
5. creates a separate generic developer capability and config for riffdb-mcp, limited to contract validation, contract catalog reads, contract deployment, and authenticated health; and
6. with --register-codex, runs codex mcp add riffdb -- ... only after the database and MCP credential are ready.

Successful bootstrap checks the exact empty-catalog health state, including a healthy primary replication component with no registered followers. Projection and outbox workers may still report degraded readiness during startup.

These are authoritative database operations, not local file setup. A failure after owner bootstrap may leave the database successfully bootstrapped even when later MCP-capability creation or Codex registration failed. Rerun the same bootstrap command: it preserves retained bootstrap material and pending capability identity so uncertainty is resolved rather than assigned a new identity. It refuses to replace changed credential/config files or a conflicting Codex MCP entry named riffdb (or riffdb_<alias>), while treating an identical existing registration as an idempotent success. Bootstrap serializes --register-codex across RiffDB callers with an exclusive lock on ${XDG_CONFIG_HOME:-$HOME/.config}/riffdb/.codex-mcp.lock; avoid concurrent non-RiffDB edits to that user’s Codex MCP configuration.

The MCP developer can author, validate, and deploy an application contract without installation choosing an application. Because application permissions name exact contract lineage and stable IDs, the pre-deployment capability does not grant wildcard command, entity, index, projection, commit, provenance, or outbox access. After deployment, use the administrator CLI config to bind a compiled application role or issue an exact scoped capability, then configure the application or a separate MCP identity with that credential.

For an exact locked application, the canonical installed workflow is:

riffdb application lock --write
riffdb application generate --locked
riffdb application deploy \
  --provision-role EaApplication \
  --tenant organization_acme \
  --seed

Deployment refuses unlocked or drifted sources. Contract and query deployment are idempotently resumable, role creation is explicit, and seed files run as ordinary idempotent commands under the resulting application credential. Private lock- and database-bound progress, the application credential, and generated application-only client.toml and mcp.toml live beneath .riffdb/deployments/<database>/; this state records identities but is never an authority source. The journal records the exact lock hash, contract bundle hash, query-module names, versions and hashes, compiled role identity, and capability ID. A deploy compares those locked identities with every server response; it never treats an operation name or a locally completed stage as proof that the active server identity matches.

The deploy command is safe to rerun after interruption at contract deployment, query-module deployment, role binding, or any individual seed command. Remote success is reconciled with the retained request and capability identity before the next stage is published locally. Do not delete .riffdb/deployments/<database>/ to recover from an uncertain deploy: rerun the same command with the same exact lock.

The lower-level, caller-identified installation campaign start and observation surface is documented in Application Installation Campaigns. It reports durable partial progress and never treats running as installed.

Python applications

The Python application distribution is riffdb-application and supports CPython 3.13 and 3.14 with manylinux 2.28 wheels on x86_64 and aarch64. Release bundles carry the matching limited-API wheel under public/python, allowing the installed CLI to create a fully locked, offline scaffold:

riffdb new my-app --language python
cd my-app
uv sync --locked

Direct pip and uv wheel installation, source-distribution builds, explicit application-format migration, sync and async client usage, and the release artifact commands are documented in Python Application Driver.

Changing the application lock is a successor deployment, not a resume. When a prior application role exists, RiffDB refuses to mutate the server until the operator supplies both the role and the explicit replacement acknowledgement:

riffdb application deploy \
  --provision-role EaApplication \
  --replace-role-credential \
  --seed

That path deploys the exact successor and starts a resumable least-authority credential rotation. RiffDB retains the old capability while it creates and proves the exact successor, atomically publishes the new protected client and MCP configuration, and then explicitly revokes the predecessor. A retry continues from the retained phase; it does not revoke the only proven secret. A module compare-and-swap race fails with the database alias plus the lock, locked-module, expected-active, and actual-active hashes. Never copy a hash from an error into source or bypass the lock; rerun deployment after reconciling the competing deployment.

For a successor, run riffdb application lock --write while the intended database alias is selected. Unlike deployment, this is an authorized read-only preview: the server compiles against the active parent and the CLI pins the canonical result in generated/riffdb.contract.bundle. Deployment later sends the exact parent version/hash and candidate hash from that artifact. Parent or candidate drift is rejected before catalog admission and the error reports the lock, expected parent, actual active, and compiled candidate hashes.

Role check, bind, and provisioning consume that pinned parent-aware bundle; they do not recompile successor source as a genesis contract. If the symbolic role widened, use --provision-role <role> --replace-role-credential so the new role is proven and published before the retained predecessor capability is explicitly revoked. The legacy --replace-expired-credential spelling remains an alias.

Do not use contract deploy as a probe. A successful direct deployment changes the active pointer and the POC has no rollback RPC. Use contract validate or the successor lock preview to inspect a candidate. Recovery from an unwanted activation is an offline backup restore (or a destructive disposable-alpha reset), performed only while the service is stopped.

Bootstrap defaults to http://127.0.0.1:7443. To select a deliberately reconfigured loopback listener, set an exact endpoint for that invocation:

RIFFDB_ENDPOINT=http://127.0.0.1:7553 \
  cargo riffdb bootstrap --user

Treat that value as an authority selection: verify it identifies the intended unbootstrapped database, and do not leave a stale override in the environment. User and system installs both default to port 7443, so choose one default scope. Running both on one host requires distinct server listen ports and separate operator XDG config/state roots; otherwise their service listeners and private client filenames collide.

The generated owner capability requests 30 days and the MCP developer capability requests less than 30 days. The POC does not renew either in the background; run cargo riffdb bootstrap (--user|--system) [--database ALIAS] --renew or riffdb capability rotate <CAPABILITY_UUIDV7> --credential-output <PATH> --revoke-old before the current administrative capability expires.

Rerun and Update Limits

Re-running install validates ownership, modes, non-symlink paths, and existing key documents, then republishes the three binaries. It preserves existing digest keys, server configuration, service-unit files, and system sysusers/tmpfiles definitions byte-for-byte rather than treating them as upgrade templates. During first-time key creation it publishes a private, create-only marker before either key. A rerun creates a missing typed counterpart only when that exact marker is still present, server configuration and the service unit have not been published, and the data, backup, and state directories remain empty. Partial key state without the marker, or any missing key beside later-phase configuration, unit, data, backup, or state evidence, fails closed as possible key loss. It refuses linked or malformed key state, unsafe file types, unexpected ownership, or modes.

Each binary replacement is staged on its destination filesystem and backed up in .riffdb-binary-rollback-v1 until all three binaries are published and unit verification succeeds. If installation fails or is interrupted during binary publication, the installer restores the previous binary set on exit or at the start of the next cargo riffdb install invocation.

Consequently, the convenience command is not a general configuration migration, key-rotation, data-deletion, or compatibility-aware upgrade tool (cargo riffdb uninstall removes only binaries and unit files; see Upgrade and Removal).

Pre-Alpha Database Reset

A reset is an offline, destructive operator action for disposable pre-alpha data. It is not a schema migration, is never performed by contract deployment, and has no runtime API. Prefer an additive successor whenever the compatibility rules permit it.

Resetting one alias creates a new database identity. Its old operator, MCP, and application credentials no longer authorize the new database. Retained bootstrap material, capability IDs, generated client configuration, and application deployment progress for that alias must be archived with the old database and recreated.

For an installer-owned user database named ea, first close every client that could restart the service, then stop the one server process that owns all configured aliases:

systemctl --user stop riffdbd.service
systemctl --user is-active riffdbd.service

Require the second command to report inactive. Set the XDG roots, create a private timestamped archive, and move the selected database plus its database-bound local authority and recovery files into it:

config_root="${XDG_CONFIG_HOME:-$HOME/.config}/riffdb"
data_root="${XDG_DATA_HOME:-$HOME/.local/share}/riffdb"
state_root="${XDG_STATE_HOME:-$HOME/.local/state}/riffdb"
reset_archive="$data_root/manual-reset/ea-$(date -u +%Y%m%dT%H%M%SZ)"
install -d -m 0700 "$reset_archive/config" "$reset_archive/state"

mv -- "$data_root/data/ea.redb" "$reset_archive/ea.redb"
for path in \
  "$config_root/operator-ea.credential" \
  "$config_root/client-ea.toml" \
  "$config_root/mcp-ea.credential" \
  "$config_root/mcp-ea.toml" \
  "$state_root/bootstrap-ea.credential" \
  "$state_root/bootstrap-local-owner-ea.json" \
  "$state_root/mcp-local-developer-ea.json" \
  "$state_root/operator-capability-ea.id" \
  "$state_root/mcp-capability-ea.pending-id" \
  "$state_root/mcp-capability-ea.id"
do
  if [[ -e "$path" || -L "$path" ]]; then
    case "$path" in
      "$config_root"/*) mv -- "$path" "$reset_archive/config/" ;;
      "$state_root"/*) mv -- "$path" "$reset_archive/state/" ;;
    esac
  fi
done

Before executing this procedure, confirm the selected [databases.ea].path in the active riffdbd.toml; custom installations may use another path. Backups under backups/ea are deliberately retained. Do not move another alias, the shared capability or idempotency digest keys, or the server configuration.

Start the service, bootstrap only the new empty alias, and deploy a complete genesis application:

systemctl --user start riffdbd.service
cargo riffdb bootstrap --user --database ea

riffdb application lock --write
riffdb application generate --locked
riffdb --config "$config_root/client-ea.toml" application deploy \
  --provision-role EaApplication \
  --tenant organization_acme \
  --seed

Archive or remove the application’s old .riffdb/deployments/ea directory before the final command so the application does not reuse database-bound deployment identities. If Codex or another MCP host retained the old riffdb_ea registration, replace it only after the new bootstrap has produced the new mcp-ea.toml. A reset is complete only after authenticated health, active-contract inspection, and an application query succeed against alias ea. Review Upgrade and Removal and compatibility policy before changing revisions. Use --no-start when replacing binaries should not immediately restart the existing service.

--from-binaries DIR is an advanced staging/test input. DIR must contain exactly named executable regular files riffdbd, riffdb, and riffdb-mcp. This mode skips the locked Cargo build and performs no revision, checksum, signature, or provenance verification; use only a directory whose complete contents were independently verified. Normal source installs should omit it.

After bootstrap succeeds, the convenience flow is complete. The procedures below are advanced alternatives; do not manually reprovision keys or overwrite configuration created by the convenience installer.

Advanced: Install a Verified Release Bundle

Use this procedure for an extracted archive produced by scripts/release-poc --verify. Keep the archive, its adjacent checksum, and the extracted directory together until verification completes:

sha256sum -c riffdb-0.1.0-<target>.tar.gz.sha256
tar -xzf riffdb-0.1.0-<target>.tar.gz
cd riffdb-0.1.0-<target>
sha256sum -c SHA256SUMS

The extracted directory is the bundle root. Install exactly its three product binaries and service definitions:

sudo install -o root -g root -m 0755 \
  bin/riffdbd bin/riffdb bin/riffdb-mcp /usr/bin/
sudo install -o root -g root -m 0644 sysusers.d/riffdb.conf \
  /usr/lib/sysusers.d/riffdb.conf
sudo install -o root -g root -m 0644 tmpfiles.d/riffdb.conf \
  /usr/lib/tmpfiles.d/riffdb.conf
sudo systemd-sysusers
sudo systemd-tmpfiles --create /usr/lib/tmpfiles.d/riffdb.conf
sudo install -o root -g riffdb -m 0640 \
  release/config/riffdbd.toml.example /etc/riffdb/riffdbd.toml
sudo install -o root -g root -m 0644 systemd/riffdbd.service \
  systemd/riffdb-mcp.socket systemd/riffdb-mcp@.service \
  /usr/lib/systemd/system/

Install the reviewed operational documentation, examples, and bundle demo without changing their relative asset layout:

sudo install -o root -g root -m 0755 -d \
  /usr/share/doc/riffdb/docs \
  /usr/share/doc/riffdb/release/config \
  /usr/share/doc/riffdb/examples/contracts
sudo install -o root -g root -m 0644 \
  README.md COPYRIGHT LICENSE-MIT LICENSE-APACHE /usr/share/doc/riffdb/
sudo install -o root -g root -m 0755 demo /usr/share/doc/riffdb/demo
sudo install -o root -g root -m 0644 docs/*.md \
  /usr/share/doc/riffdb/docs/
sudo install -o root -g root -m 0644 \
  release/README.md release/RELEASE_NOTES.md \
  /usr/share/doc/riffdb/release/
sudo install -o root -g root -m 0644 release/config/* \
  /usr/share/doc/riffdb/release/config/
sudo install -o root -g root -m 0644 examples/contracts/budget.riff \
  /usr/share/doc/riffdb/examples/contracts/budget.riff

The release verifier installs the staged bundle into a private root, compares every required asset and mode, checks the extracted layout, and runs the demo from the private installed layout before creating the archive.

Advanced: Manual Build and Install From Source

Install rustup, let rust-toolchain.toml select Rust 1.98.1, and build the three product binaries:

cargo build --locked --release \
  -p riffdb-server --bin riffdbd \
  -p riffdb-cli --bin riffdb \
  -p riffdb-mcp-stdio --bin riffdb-mcp

Install the binaries and service definitions from the source tree:

sudo install -o root -g root -m 0755 \
  target/release/riffdbd target/release/riffdb target/release/riffdb-mcp \
  /usr/bin/
sudo install -o root -g root -m 0644 release/sysusers.d/riffdb.conf \
  /usr/lib/sysusers.d/riffdb.conf
sudo install -o root -g root -m 0644 release/tmpfiles.d/riffdb.conf \
  /usr/lib/tmpfiles.d/riffdb.conf
sudo systemd-sysusers
sudo systemd-tmpfiles --create /usr/lib/tmpfiles.d/riffdb.conf
sudo install -o root -g riffdb -m 0640 release/config/riffdbd.toml.example \
  /etc/riffdb/riffdbd.toml
sudo install -o root -g root -m 0644 \
  release/systemd/riffdbd.service \
  release/systemd/riffdb-mcp.socket \
  release/systemd/riffdb-mcp@.service \
  /usr/lib/systemd/system/

Install the documentation assets using the same installed layout as the release bundle:

sudo install -o root -g root -m 0755 -d \
  /usr/share/doc/riffdb/docs \
  /usr/share/doc/riffdb/release/config \
  /usr/share/doc/riffdb/examples/contracts
sudo install -o root -g root -m 0644 \
  README.md COPYRIGHT LICENSE-MIT LICENSE-APACHE /usr/share/doc/riffdb/
sudo install -o root -g root -m 0755 release/demo \
  /usr/share/doc/riffdb/demo
sudo install -o root -g root -m 0644 docs/*.md \
  /usr/share/doc/riffdb/docs/
sudo install -o root -g root -m 0644 \
  release/README.md release/RELEASE_NOTES.md \
  /usr/share/doc/riffdb/release/
sudo install -o root -g root -m 0644 release/config/* \
  /usr/share/doc/riffdb/release/config/
sudo install -o root -g root -m 0644 contracts/examples/budget.riff \
  /usr/share/doc/riffdb/examples/contracts/budget.riff

The remaining examples use the installed asset root:

asset_root=/usr/share/doc/riffdb

You may instead set asset_root to an extracted release-bundle root.

Provision Digest Keys

The service requires two distinct protected key documents. Each file must be a regular file owned by the riffdb effective user with no group or other mode bits. The first entry is the current write key; up to seven later entries are readable retired keys.

The following uses OpenSSL only as an operator-side random-byte source:

sudo -u riffdb sh -eu -c '
umask 077
capability=$(openssl rand -hex 32)
idempotency=$(openssl rand -hex 32)
test "$capability" != "$idempotency"
printf "riffdb-capability-digest-keys-v1\n1:%s\n" "$capability" \
  > /var/lib/riffdb/capability.keys.new
printf "riffdb-idempotency-digest-keys-v1\n1:%s\n" "$idempotency" \
  > /var/lib/riffdb/idempotency.keys.new
'
sudo install -o riffdb -g riffdb -m 0600 \
  /var/lib/riffdb/capability.keys.new /etc/riffdb/capability.keys
sudo install -o riffdb -g riffdb -m 0600 \
  /var/lib/riffdb/idempotency.keys.new /etc/riffdb/idempotency.keys
sudo rm /var/lib/riffdb/capability.keys.new \
  /var/lib/riffdb/idempotency.keys.new

Do not reuse material between the documents. Back them up through a separate secret-management procedure. /etc/riffdb is administrator-owned, while the key files remain owned by the riffdb effective user because the protected loader requires exact owner matching. The database backup root deliberately does not contain them.

Start riffdbd

riffdbd.service selects the strict /etc/riffdb/riffdbd.toml configuration. Review its state, backup, key, loopback, audience, and optional hosted-MCP settings before enabling it. Unknown or malformed configuration fails startup.

The RiffDB service account must be the only writer to /var/lib/riffdb/data. Never start two riffdbd processes against the same database and never grant a backup tool, CLI user, or bridge direct write access to the data directory. The packaged unit and tmpfiles rules enforce this ownership boundary for the standard layout.

Validate the unit syntax, reload systemd, and start the service:

systemd-analyze verify /usr/lib/systemd/system/riffdbd.service
sudo systemctl daemon-reload
sudo systemctl enable --now riffdbd.service
sudo journalctl -u riffdbd.service -n 50 --no-pager

Type=exec means systemd’s active state is a process state, not RiffDB readiness. The server writes a bounded line beginning with riffdbd-ready-v1<TAB> after its startup proof. Once bootstrap exists, use the authenticated riffdb server health operation for database readiness.

The unit closes stdin. riffdbd ignores clean stdin EOF and stops on SIGTERM through its bounded graceful drain. TimeoutStopSec=80s exceeds the server’s 35-second transport drain plus runtime shutdown allowance.

Bootstrap the First Operator

Create private operator directories and a request from the installed example:

umask 077
mkdir -p "$HOME/.config/riffdb" "$HOME/.local/state/riffdb"
install -m 0600 \
  "$asset_root/release/config/bootstrap-operator.json.example" \
  "$HOME/.config/riffdb/bootstrap-operator.json"

Generate and durably retain both the one-time bootstrap document and its bearer presentation while submitting the bootstrap request:

riffdb --endpoint http://127.0.0.1:7443 --output json \
  capability bootstrap \
  --request "$HOME/.config/riffdb/bootstrap-operator.json" \
  --generate "$HOME/.local/state/riffdb/bootstrap.credential" \
  --bearer-output "$HOME/.config/riffdb/operator.credential"

If the response is uncertain, retry with --bootstrap-file and the retained bootstrap document. Do not generate a second identity:

riffdb --endpoint http://127.0.0.1:7443 --output json \
  capability bootstrap \
  --request "$HOME/.config/riffdb/bootstrap-operator.json" \
  --bootstrap-file "$HOME/.local/state/riffdb/bootstrap.credential" \
  --bearer-output "$HOME/.config/riffdb/operator-recovered.credential"

Bearer output paths are create-only. Choose a new absent output path for an uncertain retry. After a terminal bootstrap result and successful authenticated health call, retain the exact owner capability ID before securely retiring the bootstrap document:

mapfile -t bootstrap_lines \
  < "$HOME/.local/state/riffdb/bootstrap.credential"
[[ "${#bootstrap_lines[@]}" -eq 3 \
   && "${bootstrap_lines[0]}" == "riffdb-bootstrap-credential-v1" \
   && "${bootstrap_lines[1]}" =~ ^capability-id:([0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})$ ]] \
  || exit 1
operator_capability_id="${BASH_REMATCH[1]}"
operator_id_file="$HOME/.local/state/riffdb/operator-capability.id"
[[ ! -e "$operator_id_file" && ! -L "$operator_id_file" ]] || exit 1
(set -o noclobber; printf '%s\n' "$operator_capability_id" > "$operator_id_file") \
  || exit 1
chmod 0600 "$operator_id_file"

Retain every later capability ID through the same private operational inventory; operators can also inspect issued and revoked capability metadata offline with riffdb capability list and riffdb capability show. Retain the normal operator credential at mode 0600.

Install client configuration:

install -m 0600 "$asset_root/release/config/client.toml.example" \
  "$HOME/.config/riffdb/client.toml"
sed -i "s|/home/operator|$HOME|" "$HOME/.config/riffdb/client.toml"
riffdb --config "$HOME/.config/riffdb/client.toml" server health

The CLI performs no implicit configuration discovery. Every later example therefore passes --config explicitly.

Deploy the example contract through the public application service:

contract_path="$asset_root/examples/contracts/budget.riff"
riffdb --config "$HOME/.config/riffdb/client.toml" \
  contract validate "$contract_path"
riffdb --config "$HOME/.config/riffdb/client.toml" \
  contract deploy "$contract_path"

Run the Installed-Binary Demo

The packaged demo starts a disposable server with closed stdin, bootstraps it, deploys the bundled contract, commits CreateBudget and AllocateBudget, queries the resulting commit record, and stops the server with SIGTERM. It uses only the three installed binaries and packaged assets; it does not invoke Cargo, repository tests, or Git:

"$asset_root/demo" --assert

This is an installation smoke, not the complete POC-exit evidence run. The revision-specific acceptance demo remains ./scripts/demo --assert in the matching full source checkout.

Optional MCP Bridge

The normal deployment model is for an MCP client to spawn:

riffdb-mcp --config /path/to/mcp.toml

This gives the client a direct stdio child and is the most widely supported form. The bridge is still an ordinary public gRPC client and has no storage access.

For clients that can speak MCP stdio framing over a local Unix stream, the release includes an optional socket-activated systemd pair. Install the MCP configuration and a normal restricted capability:

sudo install -o riffdb-mcp -g riffdb-mcp -m 0600 \
  /secure/staging/riffdb-mcp.credential \
  /var/lib/riffdb-mcp/credential
sudo install -o root -g riffdb-mcp -m 0640 \
  "$asset_root/release/config/mcp.toml.example" \
  /etc/riffdb-mcp/config.toml
sudo systemctl enable --now riffdb-mcp.socket

The socket is /run/riffdb-mcp.sock, mode 0660, group riffdb-mcp. Adding a user to that group gives the user the effective authority of the bridge’s shared capability. Use a narrowly scoped agent capability, review group membership, and prefer one spawned bridge per client when principals must remain separate. Never give the bridge access to /var/lib/riffdb.

Driver and CLI Packages

RiffDB’s alpha distribution tells the same application story in Rust, Go, TypeScript, and Python. The language packages are delivery vessels around the first-party Rust driver boundary; they do not reimplement TLS, credentials, retry, cancellation, public-error validation, or read-after-commit semantics.

Supported alpha platforms

The signed 0.1.0 package set supports:

• x86_64-unknown-linux-gnu
• aarch64-unknown-linux-gnu

Package resolution fails on other operating systems and architectures. This is an explicit alpha limitation, not an invitation to run an unverified binary.

Install the CLI and runtime

The Python riffdb package is an exact install alias for the normative riffdb-application==0.1.0 implementation package. Generated Python modules therefore continue to import riffdb_application. Go and TypeScript generated modules use their published runtimes; Rust uses riffdb-client-rust. The Rust CLI distribution includes its generator templates inside the packaged query-module crate, so compiling the installed CLI needs no RiffDB source checkout.

The signed native binary installer also places the one architecture-matching riffdb-application wheel under its installation prefix at public/python. Consequently, the installed native CLI can run riffdb new --language python without RIFFDB_APPLICATION_WHEEL; the scaffold embeds that already verified wheel and pins it in uv.lock. A source-tree CLI still requires the explicit environment variable because it has no signed installation prefix.

Local development registries

Adapter and application packages can be developed in separate repositories before the RiffDB drivers are publicly released. The local development registry derives one immutable development publication from the ordinary unsigned driver distribution and serves it through native loopback registries. This facility contains no application-specific integration code: a Better Auth adapter, for example, belongs in its own repository and consumes @riffdb/client like any other npm package.

Build and serve one publication with a positive, never-reused build number:

mkdir -p "$HOME/tmp/riffdb-development-registry"
./scripts/build-driver-development-publication \
  --distribution /absolute/path/to/wp601-distribution \
  --registry-root "$HOME/tmp/riffdb-development-registry" \
  --build-number 42
./scripts/serve-driver-development-registries \
  --publication "$HOME/tmp/riffdb-development-registry/builds/42" \
  --state "$HOME/tmp/riffdb-development-registry/state-42"

The server binds only 127.0.0.1. It exposes Verdaccio at port 4873 and the Python Simple, Go proxy, and Cargo sparse views at port 4874. Verdaccio 6.9.2 is pinned; set RIFFDB_VERDACCIO_BIN to an absolute executable installed by your tooling, or allow the lifecycle command to obtain that exact version through npm exec. Registry state and its local publishing credential remain under the explicit --state directory. No user-global npm, pip, Go, or Cargo configuration is changed.

Build 42 maps to these exact versions and project-local settings:

Use repository-local configuration or command flags in consumers. Stop the foreground lifecycle command to close both ports. The publication under builds/42 remains immutable and may be served again with the same state; reusing build 42 for different bytes fails closed. Use a new build number after any driver change.

Run the complete clean-consumer proof with:

./scripts/driver-development-registry-acceptance --all-ecosystems

The acceptance creates four directories outside the repository, installs or resolves through the four native registry protocols, imports or compiles each runtime, checks exact lock metadata, runs both npm install and a clean npm ci, and rejects repository paths. An npm development publication includes only the native CLI packages present in its source distribution. The publisher removes absent first-party platform packages from @riffdb/cli’s optional dependencies, while missing required or peer dependencies fail publication; therefore a generated lock file never names an unpublished platform package. Rust’s third-party dependencies remain explicitly sourced from crates.io in the sparse metadata; all riffdb-* transitive crates remain on riffdb-dev. Development publications are unsigned prereleases and cannot be passed back into the production distribution or signing workflow.

For TypeScript consumers, a successful named query against a domain-empty database reports applicationHead: 0n. The application frontier is a non-negative u64; commit sequences, contract versions, and read-after-commit inputs remain strictly positive. The clean npm-consumer cell exercises this zero-frontier boundary through the installed package on every development publication.

One four-step workflow

With a RiffDB service running and an ordinary application credential available:

riffdb init inventory --generator typescript
riffdb push
riffdb generate
npm start

Replace the generator and final command for the selected language. init creates only bounded RiffDB project files, push performs the existing exact check/lock/install ceremony, and generate writes only the selected typed binding. A changed compiler-owned identity still requires explicit acceptance; an incompatible successor still routes to riffdb migrate.

The exact four-step contract lives in fixtures/driver/quickstart-v1.json and is checked across all four languages.

Cross-driver qualification

One generated conformance application is locked once and exercised from the signed runtime packages in all four languages against the same remote TLS service. Rust and Python carry native plan identities. TypeScript carries both the plan and retained-driver operation identity. Go carries the exact manifest, catalog, and operation identity through the retained Rust driver host. These are different transport representations of the same compiler-owned contract, not language-specific semantics.

The package matrix pins the bundle, plan, generated-artifact, manifest, catalog, operation, input-schema, outcome, replay, freshness, and public-error observations. It also pins the onboarding story at four steps. Run:

./scripts/check-driver-package-matrix
./scripts/driver-package-conformance-acceptance

The second command builds or accepts one signed distribution, installs every runtime from package artifacts, generates from the installed CLI, and runs the shared corpus. A source-tree runtime, handwritten wire adapter, numeric compiler identity, divergent outcome, or extra onboarding step fails.

Sealed package-first evaluation bundles additionally include an exact TypeScript compiler, Node type declarations, and their closed transitive type dependencies. The installed riffdb dev --run workflow places that tooling on the TypeScript child build path automatically; applications do not need an ambient global tsc or a hand-authored typeRoots override.

The generated Rust Cargo.lock is package-shaped rather than workspace-shaped: with the sealed bundle’s CARGO_HOME, cargo check --offline --locked must not rewrite it. Do not copy a machine-wide Cargo configuration into the project; ambient compiler wrappers and linker settings are outside the signed package environment.

For a local package-first repository that has no running service or ordinary credential yet, use riffdb dev --seed --run (or riffdb dev --run without seed inputs). The four-step init / push / generate / invoke workflow is the remote-service path: push deliberately requires both the service and its credential and does not bootstrap either one.

Sparse V7 packages do not acquire undeclared generated surfaces during this workflow. When a Go or TypeScript package omits mcp, the installed CLI derives the local driver’s exact operation catalog once into protected disposable state; no generated/mcp/tools.json file or MCP lock artifact is required.

For a Go or TypeScript HTTP application, --run remains attached while the application serves. Wait for the application’s ready marker, exercise the page, and then terminate the development process. A healthy serving process is not expected to exit on its own.

Verify a standalone binary bundle

Release bundles contain checksums.sha256 and its detached SSH signature. The release identity and allowed-signers file must come from a separately trusted channel:

cd /absolute/path/to/distribution
ssh-keygen -Y verify \
  -f /protected/riffdb-allowed-signers \
  -I riffdb-release \
  -n riffdb-driver-distribution-v1 \
  -s checksums.sha256.sig < checksums.sha256
sha256sum --strict --check checksums.sha256

/absolute/path/to/distribution/install/riffdb-install \
  --distribution /absolute/path/to/distribution \
  --target x86_64-unknown-linux-gnu \
  --prefix "$HOME/.local"

Run both verification commands from the distribution root. They use only the separately trusted signer file and operating-system tools, so verification does not bootstrap through executable code from the untrusted bundle. The installer then rechecks the selected binary against that signed inventory before atomically publishing it. Each published runtime package (riffdb-client-rust, riffdb.dev/application, @riffdb/client, and riffdb) and the distribution’s conformance/ directory also carry the language’s exact DRV-014 conformance-manifest-v1.json. Registry credentials and signing keys are never included in a distribution or receipt.

Sealed and offline installations

Normal package-arrival projects use registry imports. Existing vendored @riffdb/application, riffdb-application, riffdb.dev/application, and the source-contained Rust SDK remain supported only for an explicitly sealed or offline bundle. Offline mode does not weaken application identities or server authorization.

Alpha compatibility

These packages carry the durable-format policy described in Compatibility. Physical downgrade is unsupported. A future incompatible storage epoch must refuse before mutation and use the documented symbolic export/reimport path; it must never silently reset data.

Your First Application

This page documents the self-contained application scaffold. To add RiffDB to an existing project with riffdb.toml, an empty schema, database-shaped verbs, and selected SDK targets, use the Database-Shaped Project Workflow.

This walkthrough creates the small generated Item application, runs it on a disposable RiffDB database, and shows where source intent becomes an exact reviewed application.

Prerequisites

Install the binaries with the source checkout convenience installer, or build the workspace once:

cargo build --locked -p riffdb-cli -p riffdb-server

Scaffold

From the directory that should contain the new project:

riffdb new item-desk
cd item-desk

The generated repository separates three things:

Inspect the contract before accepting a changed lock. The starter command creates an item through a declared command; no generated client contains a raw entity mutation.

Check, lock, and generate

riffdb application check
riffdb application lock --write
riffdb application lock --check
riffdb application generate --locked

check is read-only. lock --write is the explicit point at which new compiler-owned identities are accepted. generate --locked reproduces clients only for the exact lock. Deployment performs all local source, role, query, and generated-artifact preflight before its first remote mutation.

Run a disposable server

riffdb dev --seed --run

The command starts an isolated loopback server, bootstraps through the normal capability path, deploys the exact application, binds the requested role, runs seed JSONL as ordinary idempotent commands, and starts the generated example. It removes the disposable database and credentials when the process exits.

Use this mode while authoring. For a persistent user service, follow Installation and deploy against the configured database alias.

Make a change

Add a field, command, or bounded query using the Contract Authoring Reference and RiffQL Language. Then repeat the check and lock sequence. A compatible successor must increment the contract version and preserves the exact active parent identity.

Schema evolution supports the documented additive subset and reviewed .riffm contract migrations (see Contract Migrations). If disposable local development data needs a destructive reset, pass riffdb dev --reset or use the explicit offline reset procedure in Upgrade, Reset, and Removal; never delete an active database behind a running server.

Choose a client

• Rust uses the stable Rust application facade and generated operation module.
• Go uses generated facades over the retained Rust driver host.
• TypeScript uses the checked generated package.
• Python uses the Rust-backed sync or async driver.
• MCP exposes only tools authorized for the bound role and selected database.

All five preserve the same operation identities, values, closed outcomes, idempotency behavior, and public errors.

Database-Shaped Project Workflow

Use the project verbs when RiffDB is one data dependency inside an existing Rust, Go, TypeScript, or Python project. These commands do not create or own the application’s source tree, package manifest, web framework, identity system, or process lifecycle.

The older riffdb application ... interface remains available for self-contained application packages. Project verbs are an additive orchestration layer over the same compiler, lock, authorization, deployment, and migration paths.

Initialize an existing project

From the project root:

riffdb init inventory --generator rust --generator typescript

Initialization creates only:

• riffdb.toml, containing client selection and project settings;
• riffdb.application.json, the symbolic application source; and
• riffdb/contract.riff, a domain-empty version-1 contract.

It creates no application code, package manifest, credential, lock, generated binding, entity, command, query, or role. The application source retains one zero-query structural module so subsequent generation has a stable exact module identity; it grants no authority and exposes no operation.

Initialization is idempotent when every target byte is already exact. It preflights the complete bounded file set and refuses before writing if any existing target differs or is unsafe.

The generated configuration has this closed shape:

[client]
endpoint = "http://127.0.0.1:7443"
database = "default"

[project]
schema = "riffdb.application.json"
generators = ["rust", "typescript"]

schema is relative to the configuration file’s directory. Generator names are limited to rust, go, typescript, python, and mcp; the list must contain one through five unique values. riffdb init writes Application Source V7 and declares exactly this selected set, so a Go-only initialization has only a Go generation path and an MCP-only initialization has only an MCP catalog path. Project commands discover riffdb.toml in the current directory unless --config or RIFFDB_CONFIG selects another file. General client flags and environment values retain their documented fieldwise precedence.

Author and push

Edit the contract and application source using the Contract Authoring Reference, Application Source and Exact Lock, and RiffQL Language. Add an explicit application role before expecting application credentials or MCP tools; an empty project deliberately has none.

With the intended database running and an ordinary authorized capability credential selected:

riffdb push

push composes symbolic checking, parent-aware lock compilation, exact local publication, and the existing resumable application deployment path.

• A fresh project needs no acceptance because there is no previous identity to compare.
• A no-op exact push needs no acceptance and safely resumes deployment.
• A changed compiler-owned lock stops before any local or remote mutation and reports the proposed lower-hex lock hash. After review, accept exactly that proposal:

riffdb push --accept-lock <proposed-lock-hash>
riffdb push --generate

A different or malformed value fails closed. Passing --generate refreshes the configured generated SDK bindings in the same step after deployment. Contract changes classified as requiring migration or incompatible are never deployed by push; after the same exact acceptance boundary, the command retains the candidate artifacts and points to riffdb migrate plan.

Generate configured SDKs

riffdb generate

riffdb generate works completely offline after riffdb init or local .riff/.riffq edits: when riffdb.application.lock.json is absent or stale, riffdb generate compiles and refreshes the local lock and materializes the configured SDK bindings without requiring a running server. During local prototyping, riffdb dev --reset safely wipes <project_root>/.riffdb/dev and regenerates all declared SDK surfaces in a polyglot repository.

For Application Source V7, the exact lock binds deterministic artifacts for exactly the declared SDK and MCP surfaces. riffdb.toml selects which of those declared artifacts are materialized in this checkout; selecting an undeclared target fails with a typed identity/configuration refusal. The exact manifest, contract bundle, reactive modules, and migration artifacts remain mandatory under their owning formats and are not optional SDK surfaces. An already-present unselected declared surface must remain exact, so local selection cannot hide stale generated code.

generate writes only selected declared targets and never deletes an unselected or undeclared file. Changing only generators within the declared set neither changes database state nor requires push acceptance. Adding or removing a source declaration changes exact identity and follows the normal review and push-acceptance ceremony.

Observe local and installed identity

riffdb status
riffdb diff

Both commands are read-only locally and remotely. They verify the selected local materialization and compare its exact contract and structural query-module identity with one authenticated catalog observation from the configured database. Results distinguish not_installed, exact, and different and contain bounded symbolic names and lower-hex identities, never schema source or credentials.

Run a staged migration

The project wrappers preserve the existing migration gates:

riffdb migrate plan
riffdb migrate check --operation-id <uuid-v7>
riffdb migrate apply \
  --operation-id <same-uuid-v7> \
  --confirm-apply <exact-migration-hash>
riffdb migrate operation <same-uuid-v7>

plan is local and read-only. check is a read-only server preflight. apply requires the exact locked migration hash and preserves caller-stable outcome recovery. operation observes the retained operation after interruption or database restart. See Contract Migrations for the migration source and compatibility model.

POC limitations

The POC expects a running RiffDB service and an already provisioned ordinary capability credential for authenticated project operations. Run riffdb agent init to install the generated repository skill, managed AGENTS section, and project MCP entry; the command deliberately does not provision or embed that credential. Public ecosystem package placement is documented in Package Installation.

Agent Database Quickstart

This page is generated from the same bounded source as the repository skill, managed AGENTS section, and llms.txt distillation.

Install the project rails

From a project containing riffdb.toml:

riffdb agent init

The command installs .agents/skills/riffdb/SKILL.md, appends one managed section to root AGENTS.md, and merges mcpServers.riffdb into root .mcp.json. It retains unrelated configuration, is byte-idempotent, and fails before writing when a managed target conflicts. It never embeds a credential.

Safe working loop

1. Edit the symbolic schema and bounded named queries.
2. Run riffdb push and review an exact proposed lock identity when one is reported.
3. Run riffdb generate to materialize the selected typed SDKs from that exact lock.
4. Call only generated operations and use explicit freshness or staleness surfaces.
5. Run riffdb status or riffdb diff to compare local and installed identities.

Core rules

• Treat riffdb.toml and symbolic RiffDB source as the author-owned database interface.
• Use generated application operations; never hand-author protocol identities, storage access, or transaction machinery.
• Resolve uncertain mutations with the same idempotency key before taking another action.
• Use MCP discovery and riffdb://application/guide for the deployed, authorization-filtered application surface.

The application-specific MCP guide is riffdb://application/guide. Its content is generated at read time from the current, authorization-filtered deployed catalog, so it does not teach hidden operations or stale contract identities.

POC limitations

The POC requires an already running RiffDB service and an ordinary authorized capability credential supplied through protected environment or configuration. Agent initialization never creates or embeds a credential.

Agent application quickstart

The normal RiffDB application path is contract and RiffQL text compiled into generated application operations. The kernel gRPC protocol is not an application-development API.

Before replacing the sample contract, read the repository-local AUTHORING.md and complete its transaction-route worksheet: list each command’s complete atomic mutation set, assign that set one aggregate root and route parameter, then list the route for every page. Define entity keys and indexes only after those decisions. This prevents the most expensive authoring mistake—modeling records as independent roots and later discovering that one command must change them atomically.

$ riffdb new order-desk
created application `order-desk` at order-desk
next: read order-desk/AUTHORING.md, replace the sample domain, then run `riffdb application check --source-only`

$ cd order-desk
$ riffdb dev --seed --run
generated exact application bindings
application boundary check passed
riffdb-dev-seed-v1  1  application-manifest
riffdb-dev-context-v1 database=default role=OrderDeskApplication
riffdb-dev-ready-v1 http://127.0.0.1:... ... OrderDeskApplication

The context line names values that application clients need but that must not be guessed from the role symbol. Pass database=default as the generated client’s database alias. The readiness line remains positional for existing launch supervisors; the context line is labelled for humans and agents.

riffdb new compiles before it writes and refuses to overwrite an existing file or non-empty directory. It accepts either a new child directory or an already-created empty directory:

mkdir my-empty-repository
cd my-empty-repository
riffdb new order-desk --directory .

The destination must be a regular, writable, empty directory. A destination symlink, file, populated directory, or missing parent fails before publication. The repository is compiled in a bounded sibling staging directory. A new child is published as one directory generation. An existing directory retains its inode; known top-level entries are moved in and the compiler lock is published last. Interrupted staging is therefore never treated as a valid application lock, and the retained sibling staging directory is recoverable evidence rather than silently discarded.

The scaffold creates:

riffdb.application.json
riffdb.application.lock.json
riffdb/
  contract.riff
  queries/item_page.riffq
  seed/01-CreateItem.jsonl
generated/
  riffdb.application.exact.json
  rust/client.rs
  go/client.go
  typescript/client.ts
  python/client.py
  mcp/tools.json
src/

The Rust scaffold additionally includes Cargo.toml and an exact Cargo.lock; its first cargo check --locked is reproducible without a separate resolver step. TypeScript and Python scaffolds likewise include their native exact locks.

riffdb.application.json is author-owned symbolic source. It contains names and paths, never compiler hashes, numeric IDs, masks, plans, or encoded keys. riffdb.application.lock.json is compiler-owned and pins the exact contract, query plans, role authority definitions, compiler formats, and generated artifact hashes. The compatible V1 exact manifest under generated/ is also compiler-owned.

Commands that receive a manifest or exact-manifest path discover its application workspace from the nearest ancestor containing the configured contract source. An unrelated outer repository or sibling application cannot make a nested application ambiguous. The nearest workspace still has to pass the ordinary exact identity checks; proximity never substitutes authority.

The explicit authoring operations are:

riffdb application check --source-only
riffdb application check --write
riffdb application lock --write
riffdb application check
riffdb application generate --locked

check --source-only performs no writes and compiles the complete author-owned application and roles without comparing an intentionally stale lock during an edit cycle. Exact check performs no writes and verifies source, lock, and every generated artifact; it cannot report a false green over drift. check --write and lock --write are the only operations that accept new derived identities; both compile symbolic sources, stage generated files, and publish the lock last in one offline step without starting a server. generate --locked reproduces bindings only when the current symbolic sources compile to the exact reviewed lock. A stale, substituted, interrupted, or partially generated application therefore fails before role binding, authorization, deployment, or application execution.

Every successful check also prints the configured seed-input count and the exact development command. Use riffdb dev --seed --run only with one or more ordered seed_inputs; use riffdb dev --run for an intentionally seedless application. After a breaking local contract edit, pass riffdb dev --reset to discard local development state before redeploying. The seeded form fails before startup when the list is empty and names up to eight unreferenced JSONL files as an actionable correction.

The source-tree development runner uses Cargo’s effective target directory for all RiffDB binaries. A workspace Cargo target-dir setting or an inherited CARGO_TARGET_DIR therefore applies consistently to riffdb, riffdbd, the driver host, and development-only benchmark binaries; no unset or repository- local target-directory workaround is required.

For a Python application, the source-tree runner also builds the current riffdb_application wheel with Maturin and adds it as an ephemeral uv run overlay. The application’s exact lock remains unchanged, while the disposable loopback cannot mix a current server and generated client with an older registry-pinned native runtime. This source workflow requires python3, uv, and the maturin module; installed release runners continue to use the matching wheel shipped with the release.

Generated query calls pin the compiler-owned plan hash as well as the contract and module. Successful Rust, Go, TypeScript, and Python results expose the verified contract and operation identity. Native transports verify the server tuple directly; driver-host transports first bind the exact manifest and catalog handshake, then return the matching compiler-owned result identity. A missing or different identity rejects the response; application code never assembles an identity from numeric IDs or unreviewed server state.

For a finite order family, that public operation identity is the complete family hash pinned by the generated catalog. Continuation state additionally binds the selected immutable order member, so changing the enum choice rejects the cursor without making successful responses appear to be a different named operation.

Complete public references:

• contract language and bounds
• command and invariant cookbook
• application source schema
• symbolic inspection
• RiffQL v1
• authoring diagnostics

Every new scaffold also includes AUTHORING.md, a local compact guide to the safe aggregate, relationship, invariant, indexed-page, dependent-key batch, and generated read-after-write patterns. Start there while replacing the sample; use the complete references when a diagnostic points to a narrower rule.

The sealed kit also provides riffdb-builder-mcp --workspace <path>. It is a credential-less local MCP server with reference resources and only six authoring tools: describe, check, diagnostic explanation, lock preview, explicit lock write, and exact generation. It cannot deploy, bind a role, execute a query or command, read storage, or obtain a credential. The mutating lock/generation tools remain explicit and are marked accordingly in MCP.

riffdb dev starts one local server, waits for explicit readiness, deploys the exact contract and module, compiles and binds the symbolic application role, refreshes the lock only for its product-owned ephemeral database, regenerates all bindings, checks the application boundary, executes seed JSONL through ordinary idempotent commands, and shuts down its child on failure or interrupt. Credentials live only in a mode-protected temporary directory. If role compilation or binding fails, the installed runner prints the public role bind error before cleaning up its temporary service state. A failed development run must not collapse that diagnostic into a silent non-zero exit.

With --run, the same product-owned workflow starts the repository’s generated application after readiness. A repository must contain exactly one supported runner: Cargo.toml for Rust, go.mod for Go, package.json for TypeScript, or pyproject.toml for Python. Every generated SDK exposes a zero-ceremony environment connection and a causal read-after-write handle (GeneratedApplicationClient::from_env().await? and client.causal(&write_result) in Rust, connectGeneratedApplicationFromEnv() and client.causal(writeResult) in TypeScript, GeneratedApplicationClient.from_env() / AsyncGeneratedApplicationClient.from_env() and client.causal(write_result) in Python, and generated.ConnectFromEnv(ctx) and client.Causal(writeResult) in Go) alongside the explicit constructors. Rust and Python receive the protected application endpoint, credential path, and database alias owned by their first-party runtime. They receive the values both as the existing three positional arguments and as RIFFDB_ENDPOINT, RIFFDB_CREDENTIAL_FILE, and RIFFDB_DATABASE; observing both forms with different values is an error. Go and TypeScript receive only a private driver-host socket plus compiler-owned handshake identity (RIFFDB_DRIVER_SOCKET and RIFFDB_DRIVER_HANDSHAKE or the positional arguments). The Go and TypeScript positional ABI is exactly:

SOCKET MANIFEST_HASH CATALOG_HASH DATABASE ROLE ROLE_HASH REMOTE_HASH LINEAGE VERSION BUNDLE_HASH

Application code must use ConnectFromEnv / connectGeneratedApplicationFromEnv or copy the closed decoding in the generated starter rather than infer another ordering. The application does not need to parse the database-readiness line or keep a separate bootstrap process alive. Rust, Go, and Python one-shot runners exit normally. A TypeScript web runner stays in the foreground. Leave it running after the application prints riffdb-app-ready-v1<TAB>PORT, then issue the application HTTP request from another terminal; the generated starter is exercised with curl http://127.0.0.1:PORT/item. The foreground command has not completed merely because the ready marker appeared. Stopping it shuts down the scoped driver and development daemon. Application stdout is caller-owned output and is streamed directly, not copied into RiffDB diagnostics or logs.

riffdb dev supports roles declaring either "tenant_scope": "global" or "tenant_scope": "tenant". When the selected role declares "tenant_scope": "tenant", riffdb dev binds the local development credential to the tenant supplied by --tenant <tenant_id> or RIFFDB_TENANT (defaulting to "dev-tenant" when omitted). Passing --tenant for a global role is rejected with an explicit diagnostic so role scope remains unambiguous.

A release installation places the reviewed riffdb-dev workflow beside the riffdb and riffdbd binaries. That installed workflow takes precedence over an application-local script, so opening an unfamiliar repository cannot replace the development control plane. Source checkouts retain the application-local fallback for first-party development. The release workflow uses only installed binaries and public packages; it never rebuilds or reads RiffDB implementation source.

The application boundary

Handwritten application code may use the stable application facade and compiler-generated operations. It may not:

• depend on kernel, storage, service, protobuf, gRPC, Tonic, or Prost packages;
• construct raw entity/index requests, field masks, numeric schema IDs, or encoded keys;
• pack or decode named-query records by hand;
• create handwritten RiffDB transport wrappers;
• place handwritten code under generated/.

Run the mandatory check with:

scripts/check-application-boundary .

Generated bindings contain a compiler marker and are the only files allowed to contain protocol adaptation. That exception is narrow: application authors cannot obtain a trusted marker by moving handwritten code into the generated tree. The check examines application manifests and source, while ignoring the standard package/build output trees target/, node_modules/, and .venv/. Those trees may contain generated transport internals supplied by reviewed driver packages; handwritten application source remains checked regardless of which supported language it uses.

Administrative and kernel credentials remain separate from the generated application role. The normal scaffold has no kernel dependency, feature, role, credential, MCP tool, or documentation path. Kernel access is an explicit source-repository operational workflow and must not reuse an application credential.

Whole-query bounds

Plain Limit means a caller may request any positive page size through 499. The compiler therefore charges a take $limit binding as 499 rows, even when the parameter has a smaller default. Use Limit<100> (or another reviewed maximum from 1 through 65,534) when callers need a runtime page size but the role should authorize a narrower bound. Index-scan charges from every binding in one named query are added before a role is accepted.

For a page with several independently bounded collections, use reviewed fixed limits or bounded runtime limits whose aggregate index-scan maximum fits the role:

many comments from Comment
    where site_id == $site_id && post_id == post.post_id
    order by created_at asc, comment_id asc
    take 100 after $comments_after

many post_tags from PostTag
    where site_id == $site_id && post_id == post.post_id
    order by tag_id asc
    take 50

riffdb application check performs this role derivation before writing a lock or starting a server. RDB-AR007 names the role and query when its aggregate worst-case scan exceeds the stable bound. RiffDB never replaces this rejection with a hidden scan, partial result, or larger ambient grant.

Product safety rule

Application authors express only symbolic intent. The compiler owns every derived identity, access plan, schema, capability requirement, visibility set, cost bound, cursor codec, and transport adaptation.

If an application can hand-author or override one of those derived values, the surface is a kernel or administrative API, not an application API. Normal templates, generated clients, CLI commands, MCP tools, and documentation must not expose it. RiffDB rejects the whole operation on identity, authorization, cardinality, boundedness, locality, or generated-artifact drift; it does not silently omit fields, choose an ambient version, accept a broader role, or continue with partially updated output.

Build a symbolic application in ten minutes

RiffDB applications read with RiffQL and mutate with compiled contract commands. The supported kernel gRPC protocol remains available for low-level tools and conformance tests, but application code does not need entity IDs, field IDs, index IDs, masks, encoded keys, or protobuf record construction.

From the repository root, start the complete disposable TicketDesk environment:

cargo run -p riffdb-cli -- dev --seed

riffdb dev:

1. starts riffdbd on a random loopback port;
2. bootstraps an operator through the production capability path;
3. deploys the TicketDesk contract and immutable query module;
4. creates a named-only capability from the default ticketdesk-application role preset;
5. creates 276 rows through compiled commands; and
6. prints the endpoint and protected application credential path.

Nothing in this path bypasses authorization, command semantics, audit, the commit coordinator, or durable redb storage. The disposable directory and credentials are removed when the process exits.

Use --watch to redeploy changed contract and .riffq sources. Query-only changes increment the query-module version without requiring client regeneration:

cargo run -p riffdb-cli -- dev --watch

Use --run instead when the application repository should execute against the disposable server:

riffdb dev --seed --run

The workflow accepts exactly one repository runner (Cargo.toml, go.mod, package.json, or pyproject.toml), supplies only the protected language-appropriate application connection, streams its stdout, and tears down the server when the runner exits or is interrupted. Go and TypeScript use the private first-party driver-host socket; no runner receives operator or kernel authority.

For a Go library module whose executable is a nested main package, select it explicitly with --go-runner-package cmd/server. The path is repository-confined and is validated before the local driver or credentials are created; the module-root default remains . after the runner detects a Go repository. When the option is omitted, the CLI forwards no Go-only configuration: the runner detects package.json applications and executes npm run build followed by npm run start as usual.

When the selected development role declares "tenant_scope": "tenant", riffdb dev binds the disposable role credential to --tenant <TENANT_ID> (defaulting to "dev-tenant" or RIFFDB_TENANT when --tenant is omitted), records the bound tenant in .riffdb/dev/session.json, and exports RIFFDB_TENANT to child application runners and generated SDK transports.

When authoring new contract commands or adding .riffq files under riffdb/queries/, pass --sync-manifest to riffdb application check, riffdb application lock, riffdb generate, or riffdb dev to discover unregistered riffdb/queries/*.riffq files and grant newly compiled commands and queries to the sole application role in canonical sorted order before refreshing the lock and generated clients. Multi-role manifests require --sync-role <ROLE> alongside --sync-manifest and fail closed with RDB-AR012 when --sync-role is omitted.

Use --role ticketdesk-agent for the separately named agent allowlist. It still receives no ad-hoc query or kernel authority. Use --role ticketdesk-kernel only for low-level diagnosis; it receives no application command or RiffQL authority and cannot seed. There is no combined preset. See Safe application profiles.

Use a directory of JSONL command batches with bounded concurrency. Each file contains complete inputs for one exact command and is named NN-CommandName.jsonl so dependency stages sort explicitly:

cargo run -p riffdb-cli -- dev \
  --seed-dir ./my-seed \
  --seed-concurrency 4

The development concurrency bound is 1..8. Every line is submitted through the resumable riffdb command batch scheduler as an ordinary symbolic command; seed files are not storage imports or collection transactions. See Resumable command batches.

Write a read

Save a bounded query as queries/my_page.riffq:

query MyPage(
    $organization_id: Organization.organization_id,
    $ticket_id: Ticket.ticket_id,
) {
    one ticket from Ticket
        where organization_id == $organization_id
            && ticket_id == $ticket_id
        else NotFound

    return Found {
        ticket: ticket {
            ticket_id
            title
            status
        }
    }

    outcomes Found | NotFound
}

Check and explain before execution:

riffdb query check queries/my_page.riffq
riffdb query explain queries/my_page.riffq

Deploy a directory and run by name:

riffdb query deploy queries/ \
  --module-name my_app \
  --module-version 1 \
  --contract-lineage TicketDesk \
  --contract-version 1

riffdb query run-named MyPage \
  --contract-lineage TicketDesk \
  --contract-version 1 \
  --parameters parameters.json

While riffdb dev is running in a workspace, it writes bounded local session metadata to .riffdb/dev/session.json and .riffdb/dev/profile.credential (mode 0600) and removes them on exit. Read-only inspection commands (riffdb query run, riffdb query run-named, and riffdb entity get) automatically discover --endpoint, --database, --credential-file, --contract-lineage, and --contract-version from .riffdb/dev/session.json when those flags are omitted, and support --inline <RIFFQL>, repeatable --param <KEY=VALUE>, and --params-json <JSON_OBJECT>:

riffdb query run \
  --inline 'query Adhoc($ticket_id: Ticket.ticket_id) { one t from Ticket where ticket_id == $ticket_id else Missing return Found { ticket: t { ticket_id title } } outcomes Found | Missing }' \
  --param 'ticket_id={"$uuid":"535bbe7a-3f53-4792-97bb-5d9a692be0ef"}'

The server resolves symbols, type-checks parameters, derives the complete authorization request, verifies one partition and bounded access, chooses the index plan, executes the page in one authoritative snapshot, and returns name-addressed typed records.

Write a mutation

Application writes remain compiled commands:

riffdb command run CreateTicket --input create-ticket.json

The input object uses contract field names. Idempotency, declared outcomes, provenance, events, and the commit record retain their existing atomic semantics.

Generated application clients

Stable named operations can generate Rust, Go, TypeScript, Python, and MCP artifacts:

./scripts/generate-query-clients
./scripts/generate-query-clients --check

Generated query requests pin the exact contract lineage, contract version, contract bundle hash, query-module hash, and query name. The facade owns parameter encoding, result decoding, typed outcomes, cursors, read fences, public errors, and command uncertainty recovery. A command retry retains its original idempotency identity and uses outcome resolution when the commit result is uncertain.

Rust application code calls only generated names and types:

use riffdb_client_rust::{AttemptBudget, CallMetadata, QueryOptions};
use riffdb_ticketdesk::{TicketDeskClient, TicketPageParams};

async fn example(
    application: riffdb_client_rust::StableApplicationClient,
    metadata: CallMetadata,
) -> Result<(), riffdb_client_rust::ApplicationClientError> {
let mut db = TicketDeskClient::new(
    application,
    metadata,
    AttemptBudget::new(3).expect("positive configured attempt budget"),
);
let page = db
    .ticket_page_with_options(
        TicketPageParams {
            organization_id: "c93bf186-4901-4cb6-8af2-fec65eb928e6".to_owned(),
            ticket_id: "535bbe7a-3f53-4792-97bb-5d9a692be0ef".to_owned(),
            comments_after: None,
        },
        QueryOptions::new()
            .read_after_commit(1843)
            .at_least_admission_head(),
    )
    .await?;
let _ = page;
Ok(())
}

read_after_commit is the exact causal choice when the caller has a commit sequence. at_least_admission_head is the stronger server-resolved choice when the caller has only a consistency preference. If both are present, RiffDB uses the greater floor. The latter may return a typed freshness failure rather than serve a provider epoch behind the head observed after admission.

TypeScript exposes the same named operations and observations, including bigint application heads and commit sequences and the closed consistency: "admissionHead" query option. MCP query and command schemas are generated from the same registry and carry the same exact identities.

Code generation is optional for exploration: ad-hoc RiffQL and named CLI/MCP execution remain first-class. It is the normal boundary for stable application code. See Application manifest.

See RiffQL language, planning, and query modules.

Symbolic application inspection

Inspection explains compiler-owned consequences; it does not ask application authors to copy those consequences into source.

Check source without writing

riffdb application check
riffdb --output json application check

The human and JSON forms contain the same stable stage, code, local source path, byte span, symbolic path, cause, suggested fixes, file disposition, and retry classification. They do not include submitted values, credentials, hidden schema, or arbitrary internal messages.

Review the exact generation

riffdb application lock --write
git diff -- riffdb.application.lock.json generated/
riffdb application lock --check
riffdb application generate --locked

lock --write is the explicit authority/plan acceptance point. The lock records exact source, contract, query plans, symbolic role definitions, compiler format versions, and generated artifact hashes. Edit the symbolic source to change intent; never edit a hash or generated file to force a match.

Inspect roles

riffdb role check riffdb.application.json --role MuseumApplication
riffdb role describe riffdb.application.json --role MuseumApplication

For a tenant-scoped role, add --tenant <tenant>. The description names queries, commands, fields, bounds, contract lineage, and exact derived authority. It grants nothing. Binding is a separate authenticated operation.

Inspect generated application operations

The Rust, Go, TypeScript, and Python generated clients expose typed parameter structures, result/outcome unions, cursor types, retry-safe commands, and checked public errors. generated/mcp/tools.json uses the least-sufficient V2 through V5 profile when it has no command and V6 whenever its retained command registry is nonempty. V6 records the exact predecessor profile and adds the hosted MCP name, compiler input/outcome identities, service envelope identity, composed output identity, and closed discovery descriptor to each command. Its visible named queries carry the same closed hosted descriptor. The catalog remains generated expectation evidence, never dispatch or authorization authority.

Regenerating an older command-bearing package rotates the MCP artifact hash and therefore its enclosing application-lock identity without advancing the lock schema. Review riffdb application preview, accept with application lock --write, and verify with application lock --check. Existing exact V2 through V5 catalogs remain readable but make no hosted-descriptor parity claim. Application code imports those operations, not Protobuf or kernel requests.

The exact compatibility manifest at generated/riffdb.application.exact.json is compiler-owned deployment input. It is useful for auditing, but its hashes and IDs are never author inputs.

Explain a failure

Use the diagnostic code and symbolic path:

1. Correct the referenced author-owned source.
2. Run riffdb application check.
3. Review riffdb application lock --write.
4. Verify with riffdb application lock --check.

Do not broaden a role, switch to a kernel API, omit a requested field, or replace a bounded query with client-side RPC composition merely to make the failure disappear.

Vector Search

RiffDB vector search is a named, bounded RiffQL read over authoritative application-supplied embeddings. The application computes embeddings; RiffDB validates and stores them through compiled commands, tracks when they become stale, and maintains a derived per-organization search projection.

The application never selects an index implementation. A contract may declare an approximate-search threshold and recall floor; RiffDB then uses exact search for small admitted partitions. Above the threshold it can reuse a matching first-party HNSW graph; a cold or unavailable graph falls back to exact search.

The scan ceiling also bounds materialization of the organization’s merged candidate rows, before policy admission and distance calculations. Oversized partitions return a typed scan-budget refusal. Scalar range filters exclude null cells; null does not satisfy an upper-bound comparison.

Declare the vector field

entity Document {
    key (organization_id: uuid, document_id: uuid)
    field title: string<256>
    field body: string<4096>
    vector_field embedding(1536, cosine, (title, body), staleness_slo 60,
        model "text-embedding-v1", current_version "2026-08-21",
        replay_age_seconds 86400, replay_bytes 1073741824,
        replay_backlog 100000,
        ann_threshold 256, recall_target_bps 9500)
}

ann_threshold and recall_target_bps are an atomic pair. In this example, search is exact at 256 or fewer admitted rows in the requested organization and may use a matching warm graph above 256. Cold queries remain exact while a bounded background build may prepare reuse. The declared 9,500 basis points means recall@K must be at least 0.95 against exact search at the same frontier. Query text and parameters cannot lower that target or force a tier.

The source fields (title, body) declare what makes the embedding stale. The model identity and current version are also contract facts; a caller cannot write a differently identified embedding and claim it is current.

Write embeddings through a command

command SetDocumentEmbedding {
    input request_id: string<128>
    input organization_id: uuid
    input document_id: uuid
    input embedding: vector<1536>
    input submitted_model: string<256>
    input submitted_version: string<256>

    idempotency_key request_id
    mutate Document(organization_id, document_id) as document else Missing {}
    embed document.embedding = embedding
        from (submitted_model, submitted_version)
    return Embedded { document: document }
}

The embed effect commits the vector and its model/version/write-sequence evidence atomically with the command outcome, provenance, and idempotency record. Generated clients provide a field-specific constructor using the contract-sealed model identity and version. Generic set cannot mutate a vector field.

Changing a declared source field without a later embedding command makes that row stale. Generated clients expose bounded symbolic stale-row and model-version inspection when the application role declares the corresponding authority.

Read through a named RiffQL operation

query SimilarDocuments(
    $organization_id: Document.organization_id,
    $query_vector: Document.embedding,
    $k: Limit<499>,
) {
    source projected Document.embedding
    freshness causal inherit_session_commit true max_wait_ms 500

    many documents from Document
        where organization_id == $organization_id
        nearest(embedding, $query_vector, $k)

    return Found {
        documents: documents {
            document_id
            title
        }
    }
    outcomes Found
}

The organization equality is mandatory, K is positive and compiler-bounded, and the source must name the same vector field used by nearest. Row policy is applied on every query to the complete candidate set before vector validation, graph lookup or construction, scoring, ranking, or K selection. Another organization and a policy-denied row cannot shape graph statistics or results.

freshness causal is the normal read-after-write choice. available permits a currently published frontier, while duration-bounded freshness uses trusted commit timestamps. Every successful projected response reports its frontier.

Correct typed failures

• RDB-QP002 means the query lacks a provable single partition. Add the declared organization equality; do not fetch broadly and filter in the app.
• RDB-PROJECTION-0104 means the named query lacks the exact compiler-owned source projected Entity.vector_field identity. Refresh and deploy the query module rather than selecting a storage index manually.
• RDB-PROJECTION-0103 means no published generation satisfies the declared freshness floor. Retry according to the typed outcome; do not silently weaken freshness.
• A vector dimension or model/version mismatch is an input error naming the symbolic field or input path. Regenerate or use the field-specific generated constructor; do not encode vectors as bytes.
• Building, rebuilding, or detached projection state is unavailable until a complete checkpoint is published. RiffDB never serves a partial graph or an old generation as if it were current.

POC bounds

Production nearest reads admit at most 500 rows from one organization when ann_threshold is omitted, or up to 65,536 rows per organization when ann_threshold and recall_target_bps are declared. Search is exact at or below ann_threshold; above it, both primary and follower providers can reuse warm in-memory HNSW graphs for an unchanged admitted population at the same generation and frontier, falling back to exact search while a graph is cold. Model, policy, or capability changes cannot reuse a mismatched graph. Graphs reopen cold and are not part of backup identity; persistent incremental graphs remain deferred. The current projected-vector endpoint does not support continuation cursors.

For the full declarations and query grammar, see Contract Authoring and RiffQL.

TicketDesk End to End

TicketDesk is the main symbolic application example. It demonstrates scoped multi-tenant entities, bounded page queries, dependent reads, named commands, generated clients, role-derived authority, and same-workload PostgreSQL measurement.

Run the disposable application

From the RiffDB source checkout:

cargo run --locked -p riffdb-cli -- dev --seed

The command prints the loopback endpoint and scoped credential after the server is ready. It creates 276 rows by invoking compiled commands; it does not import storage files or bypass authorization.

To execute the checked application runner as well:

cargo run --locked -p riffdb-cli -- dev --seed --run

Read one page

TicketDesk’s named page query resolves a ticket and its bounded dependent data in one server request and one authoritative snapshot. The query compiler owns the index selection, dependent-key proof, cardinality limits, result schema, and cursor codec.

Inspect the plan without executing it:

cargo run --locked -p riffdb-cli -- query explain \
  queries/ticketdesk/ticket_page.riffq

For an installed generated application, invoke named queries through its generated Rust, Go, TypeScript, Python, or MCP operation. Do not copy internal IDs from the explain output into application code.

Mutate through commands

TicketDesk mutations name commands such as create, assign, comment, or status transition. Each call carries an idempotency key and returns one declared outcome. The role used by the application contains exact named commands and queries; it has no kernel entity or index authority.

Verify the boundary

./scripts/check-ticketdesk-symbolic-boundary
./scripts/check-application-bindings

These checks reject raw Protobuf construction, storage dependencies, numeric schema identities, handwritten transport adapters, and generated-artifact drift in the application path.

Explore further

• Build a symbolic application
• RiffQL planning and bounds
• Safe application profiles

PostgreSQL Safety Comparison

RiffDB includes equivalent LegalSpend and TicketDesk workloads for PostgreSQL and RiffDB. The comparison is a semantic demonstration first and a benchmark second: it identifies mistakes an ordinary PostgreSQL application can make and the corresponding write path RiffDB does not expose.

Run the public comparison

The PostgreSQL side requires a disposable database URL. Never point the runner at a database containing valuable data.

export RIFFDB_COMPARISON_POSTGRES_URL='postgres://localhost/riffdb_comparison'
cargo test --locked --manifest-path examples/budget-comparison/Cargo.toml \
  --test public_comparison -- --nocapture

The TicketDesk performance runner is under examples/app-baseline. Its README records the exact setup, workload, and measurement rules. Benchmark numbers are environment-specific; the checked semantic assertions are the durable result. benchmarks/run-app-baseline-python, benchmarks/run-app-baseline-typescript, and benchmarks/run-app-baseline-golang run that same mix from Python, TypeScript, or Go so the language-runtime cost is visible: PostgreSQL still implements the postgres_safe_app obligations in that language’s SQL, while the RiffDB path uses the generated TicketDesk client with no application-language safety code (riffdbd enforces authorization, idempotency, audit, events, and outbox). TypeScript and Go reach riffdbd through riffdb-driverd. These runners are not release evidence.

Patterns under test

PostgreSQL can implement each safe pattern. The negative controls intentionally show what happens when the application does not. RiffDB’s value is that its normal application protocol cannot express the unsafe alternative.

What the comparison does not claim

• It is not a general database performance ranking.
• It does not compare analytical SQL or ad-hoc joins; RiffDB does not provide them in the POC.
• It does not claim agents can skip contract review. Unsafe semantics in an accepted contract remain unsafe semantics.
• It does not place PostgreSQL on RiffDB’s critical path. RiffDB owns its standalone storage and commit semantics.

Review the source in examples/budget-comparison and examples/app-baseline before interpreting measurements.

Contract-First State

In RiffDB, a contract is the source of executable operational semantics rather than only a record schema. It defines what state means, which operations may change it, what each operation can observe, and which results callers must handle.

Source, IR, and plan

The compiler boundary has three layers:

1. The source AST preserves author syntax and spans for diagnostics.
2. Typed IR contains resolved symbols, types, bounds, and canonical identities.
3. An executable command plan contains only operations the deterministic runtime and commit evaluator can execute.

Applications interact with source names and generated types. Numeric IDs, storage keys, masks, dependency encodings, and plan hashes are compiler-owned. The exact application lock records the derived result for review and reproduction.

Closed mutation surface

Every application mutation names a compiled command. A command declares:

• its input types and locality;
• the aggregate that owns each mutation;
• observations and predicate dependencies;
• preconditions, invariants, and commit checks;
• entity and index changes;
• durable events and outbox intents; and
• every business outcome.

A business rejection is a declared outcome, not an unstructured exception. Invalid input, authorization failure, resource exhaustion, and unavailable service state remain typed public errors.

Static intent, current truth

Compilation makes the dependency shape explicit; execution still checks the current database. The runtime evaluates against a snapshot and returns a CommitIntent. The commit coordinator alone enters the authoritative write transaction, revalidates dependencies and commit checks, assigns a sequence, and applies the atomic command graph.

This separation prevents a long-lived storage write transaction from being held while application expressions execute.

Continue reading

• Contract language overview
• Command and invariant cookbook
• Command lifecycle
• Application source and exact lock

Command Lifecycle

One command follows the same path whether it arrived through a generated client, CLI, gRPC, or MCP.

1. Select and validate

The request selects a database before authentication. The service authenticates the database-bound credential, resolves the exact active contract and command, materializes values under that schema, enforces request limits, and authorizes the named operation and derived scope.

The idempotency identity is the principal, command identity, caller-owned key, and canonical input hash. Replaying the same logical command returns the stored outcome. Reusing the key with different input fails closed.

2. Acquire and evaluate

Compiler-derived conflict keys are canonicalized and acquired in a stable order. Cancellation releases these in-memory capabilities. The runtime reads an authoritative snapshot, tracks every influential entity, index epoch, and predicate dependency, and evaluates the executable plan deterministically.

The runtime produces an internal CommitIntent; it does not mutate storage or assign a commit sequence.

3. Revalidate and commit

The commit coordinator opens the authoritative write transaction, revalidates read dependencies against transaction-current state, reevaluates bounded commit checks, verifies capacity, and then assigns the next sequence.

The encoded reservation includes command-prefix evidence: intermediate entity and index values needed when a later command in the same physical group overwrites or deletes them. These bytes share the existing size limits. A command whose complete reservation exceeds the limit is refused before sequence assignment; an archive sink is not part of admission or acknowledgement.

For a successful command, the following become durable atomically:

• entity and index mutations;
• the terminal typed outcome;
• durable events and outbox intents;
• provenance; and
• the commit record and its command-prefix evidence.

No response is reported as committed before the configured durability boundary is satisfied.

4. Recover uncertainty

A transport failure after submission does not prove failure. The caller keeps the original idempotency key and either retries the same canonical input or resolves the stored outcome. Creating a new key would create a new logical command.

Derived projection workers consume committed events and advance explicit frontiers. Their state is rebuildable; command acknowledgement does not depend on a projection pretending to be authoritative.

Consistency and Recovery

RiffDB exposes the boundaries that application code usually has to infer from separate transaction, queue, and projection systems.

Authoritative state

Entity state, terminal outcomes, durable events, provenance, and the commit log are authoritative. One successful command publishes them as one atomic graph. If restart recovery observes incomplete or contradictory durable evidence, the database fails closed rather than guessing.

Idempotent uncertainty recovery

An idempotency key identifies one caller-intended command. The canonical input hash prevents that identity from being reused for different input. A terminal outcome survives restart, so a caller can distinguish a known committed result from OutcomeUnknown after retry capacity is exhausted.

Provenance

Each mutation records the authenticated actor, ingress, request and idempotency identities, contract and plan identities, logical inputs, and the resulting commit. Public responses and logs are redacted before emission; internal sources are correlated with bounded incident IDs rather than exposed.

Durable effects

Commands record effect intent in the same atomic commit as state. Connectors deliver from the outbox after commit and must tolerate duplicate external delivery. The database does not claim that an unreliable external system is inside its transaction.

Projection frontiers

A projection frontier is the greatest contiguous authoritative commit applied to a projection generation. Callers that need read-after-commit behavior pass a known commit sequence and wait within a bounded deadline. A projection can be rebuilt from authoritative history without changing command truth.

Aggregate projection catch-up groups up to 64 consecutive commits within the existing total row and byte bounds. Each commit, including one with no relevant event, retains its equality marker. Rows, all markers and the final frontier become durable together, so intermediate positions in a batch are not separately visible. A retry checks every marker, and an uncertain write is reconciled before notification. Command acknowledgement and durability are unchanged.

Projected-query responses carry opaque V2 commit tokens and frontiers bound to the database identity, history incarnation and applied position. Carry the returned token unchanged for a causal projected read. A follower refuses a token from another database or incarnation with RDB-HISTORY-0101, even when the numerical commit sequences match. Available and Bounded reads report the follower’s local applied frontier.

Legacy V1 tokens remain decodable and accepted by the primary compatibility path, but contain no database identity and cannot prove lineage to a follower. Followers refuse them. Upgrade clients before consuming V2 projected responses; V1-only readers reject the new version. No stored database, projection artifact or Protobuf field changes as part of this opaque-token version change.

Discovery catalog fences carry the observed history incarnation too. A supplied nonzero incarnation that differs from the current database is refused as RDB-HISTORY-0101. A changed server process at the same incarnation instead invalidates the catalog cache hint and forces freshly authorized discovery. Opaque continuation cursors belong to their issuing process; foreign or obsolete handles return RDB-CURSOR-0101 and never restart a page internally.

Admission-head reads

When a caller needs a stronger read but does not possess a commit sequence, it can request AdmissionHead. After initial authorization, RiffDB captures the authoritative application head once and serves only a snapshot or provider epoch at or beyond that floor. A later concurrent commit is not required, so this is a precise fresh-through-admission guarantee rather than linearizability.

The wait is bounded by the request and server freshness limits. Lag, rebuild, retirement, cancellation, and saturation return typed failures; RiffDB never falls back to a stale result. The first page stores the consistency class and floor in opaque server cursor state. Continuations reuse that state without capturing a moving head, and a non-fenced cursor cannot be upgraded in place. An explicit read-after-commit fence still applies, with the effective floor set to the maximum of all applicable fences.

Concurrency

Logical conflict capabilities reduce avoidable races but do not replace transaction-current validation. Exact entity versions, index epochs, and predicate dependencies are rechecked immediately before commit. This is what protects observations that influenced an outcome even when the observed row was not mutated.

Safety by construction

Status: implemented application-surface rule.

RiffDB’s application product is not a safer collection of table operations. It is a closed set of compiled domain operations whose unsafe alternatives are not available to ordinary application credentials.

Interaction profiles

Permissions do not flow upward between these profiles. In particular, ReadEntity and ScanIndex do not authorize RiffQL, and application-query permissions do not authorize the corresponding kernel RPCs.

The checked development presets are:

riffdb dev issues one selected credential. It has no full, all, or combined preset.

Product rules

1. Application writes invoke only named compiled commands.
2. Stable application reads invoke only exact named deployed queries.
3. Agent ad-hoc RiffQL is a separate, explicit capability.
4. Raw kernel reads require an explicit kernel/admin capability and are absent from normal generated clients and MCP catalogs.
5. Relationships and same-partition uniqueness are contract declarations whose enforcement is proven by the compiler and authoritative transaction.
6. Every query carries one compiler-proven, policy-authorized whole-request cost vector enforced again as execution fuel.
7. Missing fields, rows, relationships, indexes, authority, or budget reject the operation; they do not silently weaken the requested semantics.

For compatibility, the current capability’s max_scan_rows is applied once to the complete query’s scanned-row, point-read, dependent-key, and intermediate- row totals. It is not reset for each binding. The compiler also hashes bounded step, projected-value, and encoded-result-byte maxima into the plan. The executor decrements matching fuel and treats inconsistent backend work reports as a closed failure. Fuel exhaustion cannot publish a partial result or cursor.

Required same-partition relationships

A required relationship is declared on the referencing entity using stored fields and one complete target primary key:

reference project
  (organization_id, project_id)
  -> Project(organization_id, project_id)

This is an integrity declaration, not query shorthand. Compilation rejects the declaration unless the target list is the complete primary key in canonical order, source and target types match exactly, every source component is required, and both entities have the same aggregate-scoped partition identity. Names, matching field spellings, equal runtime bytes, and a shared tenant UUID do not establish colocation.

A command that creates a referencing record, or sets any relationship component, must contain an earlier source-declared read of the exact resulting target key. Target expressions are compared structurally after type resolution. A partial key, different input, later read, mutate binding, or runtime equality does not count. The read’s else branch is the declared missing-target business outcome.

The compiler records this proof as a RelationshipCheckPlan naming the relationship, source binding, and target read. Command explain renders the link. No hidden read is injected: the ordinary read observation remains in the dependency set and the commit coordinator rechecks it before authoritative mutation. Missing evidence, a changed target, or a plan mismatch fails closed without a commit sequence.

Only required same-partition relationships are supported. Optional, cross-partition, polymorphic, cascading, deferred, and inferred relationships are rejected rather than approximated.

Declared same-partition uniqueness

Scoped uniqueness is declared on an entity and backed by an authoritative index:

unique user_email (organization_id, email)

The declaration is accepted only when every component is required and key-compatible and the key begins with the complete aggregate-root route. RiffDB does not infer uniqueness from names, application checks, a currently empty scan, or a globally repeated field. Global, collation-dependent, cross-partition, optional, and undeclared uniqueness are unsupported.

For every create or mutation that can change the value, the compiler derives the complete resulting tuple from validated command inputs. That tuple becomes an additional logical conflict capability. If the result depends on an entity read, omitted assignment, clock, or other runtime value, compilation rejects the command; the application cannot substitute a check-then-write sequence.

After semantic validation and while the conflict capability is still held, the commit transaction reads the exact complete unique prefix. Vacant or same-entity ownership may proceed. Ownership by another entity produces the durable, retry-replayable UniqueConflict execution result before capacity or commit-sequence assignment. Entity mutation, removal of an old unique entry, creation of the new entry, outcome, provenance, and commit record remain one atomic write.

Startup readiness recomputes each declared unique entry from the authoritative entity post-image and validates both directions in one immutable recovery view. A missing, duplicate, stale, malformed, or orphan unique-index row is authoritative corruption. Recovery withholds operational ports and never silently repairs or chooses a winner.

Bug classes made unavailable

The stable application profile must prevent:

• generic insert/update/delete and host-language transaction callbacks;
• commits based on invalidated command dependencies or violating declared invariants;
• duplicate mutation after uncertain retry;
• partial mutation/outcome/event/provenance/outbox commits;
• raw N+1 and multi-snapshot page assembly;
• unreviewed or string-constructed query execution;
• writes that establish a declared relationship without proving the target;
• races that violate a declared same-partition unique key;
• cross-partition or unindexed query fallback;
• ambiguous result cardinality; and
• cumulative query work above the admitted budget.

Claim boundary

RiffDB validates transaction-current dependencies. This does not establish freshness of caller intent: two correctly serialized replacement commands can still overwrite an edit. Per-assignment evidence distinguishes arithmetic, explicitly associated business checks, and caller-version fences. Only a fence checks a supplied caller revision, and its effectiveness depends on the application maintaining that revision protocol. See the write evidence cookbook.

RiffDB proves declared semantics. It cannot infer a business rule absent from the contract, infer that several separately named commands were intended as one business action, or stop a process from calling an external service outside RiffDB. A workflow that requires atomic database state and an external effect must commit durable event/outbox intent in one command.

The safety claim therefore applies to supported RiffDB application interactions, not arbitrary surrounding application behavior. Every committed intermediate state must still satisfy all declared invariants.

Required negative acceptance

The application-platform gate includes small intentionally bad programs. A gate passes only when each program fails before protected output or mutation:

• application credential calls GetEntity or ScanIndex;
• named-only credential submits ad-hoc RiffQL;
• caller substitutes another module hash, query name, plan, partition, or capability revision;
• command creates a dangling declared relationship;
• concurrent commands attempt the same declared unique value;
• repeated query steps amplify a capability’s whole-request budget;
• dependent batch exceeds field, row, partition, cursor, or revocation scope;
• TicketDesk attempts a public N+1 or multi-snapshot detail page; and
• generated stable-application code attempts to construct raw query source, field masks, encoded keys, or kernel requests.

Positive acceptance must also prove that the same stable application capability can execute its exact generated commands and named queries through gRPC, CLI wrappers, and generated MCP tools without gaining any rejected authority.

Run the complete checked negative corpus with:

./scripts/safety-by-construction-acceptance

The corpus registry is tests/authorization/fixtures/bad-application-corpus.json. It maps every negative-acceptance case to executable compiler, policy, service, transaction, fuel, or application-boundary evidence. riffdb-dev-acceptance additionally proves against live riffdbd that the default credential can execute the exact TicketDesk workload but cannot submit ad-hoc RiffQL.

Migration from pre-cutover development credentials

Capabilities issued before the named-authority cutover are not upgraded or narrowed in place. Create one replacement credential for the intended profile, move the caller to it, verify its exact named operations, and revoke the old credential. Do not reuse an old capability containing ReadEntity or ScanIndex as an application credential.

Stable Rust application code should accept StableApplicationClient, which has no entity/index or administrative methods. Kernel tooling deliberately uses RiffDbClient and a separate kernel/admin credential. Generated Go, TypeScript, Python, and MCP artifacts contain exact named operations and immutable module hashes; raw fixed MCP tools are hidden by current-policy discovery for application and agent credentials.

The current lower-level JSON capability documents still encode compiler-derived field visibility and command IDs. They are implementation fixtures, not the application authoring format. Symbolic role compilation replaces those documents without changing the authority separation established here.

Compiled row policies

RiffDB row policies are closed contract programs attached symbolically to application roles. They are intended to make a forgotten application-side row check impossible: a request carries ordinary operation values only, while the compiler and authorization path own the exact policy proof.

The row-policy compiler and runtime freeze authoring, IR, identities, and capability-fact delegation, and apply the shared pure evaluator across authoritative, projected, search, live, event, contextual, workflow, scheduler, and bulk safe points. It denies missing rules, facts, rows, and exact relationship evidence; evaluates read/create/delete against the required row; evaluates update against both transaction-current and successor state; and issues a move-only proof bound to the exact capability revision and row hashes. The same evaluator filters rows before limits and aggregates, and its final-safe-point recheck rejects capability or row drift.

Boolean constants, fields and declared principal facts can be used directly as predicates or under logical operators. The evaluator’s former incorrect denial of these compiled policies is fixed; after upgrade, allow create when true can allow creation subject to every existing authority check. Missing or mistyped values are still denied, and required relationship evidence is checked even when another Boolean branch is true. See contract authoring.

Protected operations remain unavailable from the ordinary unbound role grant. Capability Record V4 persists the exact application role, canonical principal facts, selected policy per protected entity, and closed operation classes. Trusted role provisioning type-checks the complete fact set before returning otherwise-withheld permissions in that V4 grant.

Authoritative named and ad hoc RiffQL reads now consume that V4 authority in the query-executor-owned snapshot over API-neutral storage readers. Point reads become indistinguishable absence when denied; dependent reads filter before cardinality and hydration; index scans filter each authoritative candidate before it can consume the visible page limit or cursor; and operational aggregates see only the resulting authorized rows. The service resolves the selected policy against the exact active bundle and reauthorizes the capability immediately before response release. A missing binding, stale policy name, wrong operation class, invalid relationship plan, or adapter without the policy-aware execution port fails closed.

Compiled commands now carry the same move-only V4 authority through admission and deterministic evaluation. Before sequence assignment, the authoritative write transaction reloads the exact capability revision and every bounded, compiler-derived relationship lookup. Create checks the proposed row, update checks both transaction-current and successor rows, and delete checks the transaction-current row. Any stale capability, missing relationship evidence, or denied transition rolls back without mutation and returns only the ordinary authorization-denied class.

Live named queries execute their initial snapshot and every subsequent re-evaluation through the same policy-aware RiffQL path. Protected watches conservatively re-evaluate after any commit in the routed partition, including ACL-only relationship changes, so a narrowing emits removals or a reset before the next value is delivered. Capability changes close the watch instead of reusing its prior authority.

Contextual hydration has a policy-aware grouped-query path that executes every declared hydration for one work item inside one shared authoritative snapshot. The service derives the protected entity union only from exact deployed reactive and query modules, resolves it against current V4 authority, and reauthorizes before cursor publication, hydration, and release. The trigger event is independently checked through its compiler-owned current-row anchor before it can be leased. Acknowledgement and negative acknowledgement repeat that event check in the same authoritative transaction as consumer mutation.

Contextual reaction helpers validate the sealed causation token and exact live lease, then repeat current capability, trigger-row, relationship, and event policy evaluation in one storage safe point before invoking the ordinary compiled-command service. They cannot submit an evaluated mutation or bypass the transaction-current command verifier; a denied event, current row, or successor row commits nothing. Old events without an anchor remain hidden.

Native projected queries now derive the complete candidate-key set from one organization partition and one immutable projection snapshot. The configured authoritative adapter reloads every current row and bounded indexed relationship observation in one read snapshot, evaluates the same V4 policy, and returns a move-only admission proof bound to the entity and exact candidate set. The columnar engine verifies that proof and excludes denied rows before request predicates, scan charging, limits, grouping, aggregates, or ranking. The operation refuses a candidate set above the fixed 100,000-row admission ceiling; it never post-filters or returns a partial aggregate.

Event replay/tail, durable streams, contextual trigger delivery and reactions, consumer status, protected opaque seek, and acknowledgement now consume the same current anchor authority. Filtering occurs before visible limits, leases, hydration, cursor release, and reaction command admission. Protected replay returns a typed bounded_progress result after at most 1,024 physical candidates so hidden history cannot force unbounded work or disclose its cardinality. Cursors bind capability revision and application-role policy identity. The untargeted global reactive-wakeup resource remains denied for V4 credentials because it has no subscription identity with which to suppress hidden commit timing.

Export additionally requires a distinct current Capability V5 grant; V4 row policy alone never implies it. Event policy is never guessed from payload field names. An event without the compiler-owned source entity/key and policy anchor is indistinguishable hidden history for ordinary protected consumers. Provisioning rejects missing, extra, or mistyped facts as RDB-AR010; it does not silently drop an unknown fact or substitute a default value. A role whose selected policy reads principal.id also rejects a principal that is not canonical lowercase UUID text before capability creation.

Contract declarations

A principal fact has a symbolic name and either a supported scalar type or a bounded scalar set. A row policy names one entity and defines at most one rule for each operation class:

principal fact team_ids: list<uuid, 32>

row policy DocumentAccess on Document {
  allow read when visibility == Public
    || owner_id == principal.id
    || team_id in principal.fact.team_ids
  allow create when owner_id == principal.id
  allow update when owner_id == principal.id
  allow delete when owner_id == principal.id
}

The closed expression language supports:

• fields of the protected row;
• principal.id, principal.kind, and declared principal.fact.<name>;
• typed constants and enum variants;
• equality, inequality, null/existence, bounded membership, &&, and ||;
• one complete indexed, partition-local relationship-existence probe.

It does not support request-provided predicates, callbacks, arbitrary functions, dynamic field names, recursion, nested relationship traversal, unindexed scans, cross-partition probes, clocks, randomness, filesystem or network access, or a client-side evaluator. Unsupported forms fail during contract compilation with a source span.

Exact limits

Each relationship probe must name a declared index completely. Its target partition prefix and the source row route must have the same symbolic field and type. There is no fallback scan.

Roles and exact identity

Application Source V6 and V7 make row_policies a required array on each role. A role compiling a protected read or mutation must select a policy containing the corresponding read, create, or update rule. Unknown policies, multiple policies for the same protected entity, and missing operation coverage fail closed.

The compiler emits Application Manifest V4 and V5, Application Lock V7 and V8, and role definition format V3. The lock safely names the selected policy, entity, and operation classes. The contract bundle hash covers the complete executable policy IR, and the role hash covers its policy selection and required public fact schemas. Existing V1–V6 fixtures remain byte-exact; the current rotation receipt is fixtures/application-locks/row-policy-rotation-v1.json.

Catalogs may expose a visible policy name, entity name, operation classes, fact name, and symbolic fact type. They do not expose policy bytecode, stable numeric IDs, hidden branches, row values, or a capability’s fact values.

Principal facts and delegation

Facts are authorization state, never request claims. The V1 fact set:

• sorts fact names and set members canonically;
• rejects duplicate names and members;
• supports only checked scalars or one bounded scalar set;
• has a strict versioned decoder that rejects non-canonical bytes; and
• redacts values from Debug output.

An authorization binding ties the set to one capability ID and nonzero revision, database, environment, principal ID/kind, canonical audience set, tenant scope, issue time, and exclusive expiry. Delegation inherits database and environment and may only reduce audiences, reduce global scope to one tenant (or retain the same tenant), shorten the lifetime, remove facts, retain an equal scalar, or reduce a set-valued fact to a subset. Adding or changing a scalar fact, adding a set member, changing a tenant, widening an audience, or extending expiry is rejected.

Capability Record V4 is an additive successor: V1, migration-only V2, and installation-only V3 remain byte-exact. V4 is selected only for a checked row-policy extension, and its decoder rejects missing extensions, noncanonical fact or binding order, duplicate entity selection, unknown operation tags, and role-identity substitution. Current authorization reconstructs facts only from the transaction-current retained record. Authoritative RiffQL, native projected queries, compiled commands, live named RiffQL watches, compiler-anchored event replay/streams, protected consumer state, and contextual subscriptions/reactions are enabled protected surfaces. The generic database-wide reactive wakeup remains unavailable because it cannot prove subscription-specific visibility. Other protected surfaces remain unavailable until their shared policy and final-safe-point enforcement lands.

Binding a protected role

Principal facts are supplied only by an operator while binding the exact compiled application role. They never appear in a generated query or command method and cannot be overridden by an application request. Write one bounded JSON object whose keys exactly match riffdb role describe’s principal_fact_schemas, then bind it explicitly:

{
  "team_ids": [
    { "$uuid": "00000000-0000-0000-0000-000000000008" }
  ]
}

riffdb role bind riffdb.application.json \
  --role DocumentReader \
  --principal 00000000-0000-0000-0000-000000000007 \
  --actor-kind service \
  --audience document-service \
  --principal-facts principal-facts.json \
  --credential-output .riffdb/document-reader.credential

The facts document uses the same natural symbolic JSON values as application inputs. Missing, extra, wrongly typed, duplicate, or over-bound facts fail before capability creation with application_role_principal_facts_invalid. Enum facts use variant names; the compiler resolves them against that fact’s declared enum, for example { "$enum": "Internal" }, so operator files never contain stable numeric enum IDs. Lists of enum facts use the same tagged form for each member. When a selected policy reads principal.id, the bound principal must be a canonical lowercase UUID. Diagnostics may name the required fact and public type but never render its value.

Secret Field Classification

A stored entity field may be declared secret:

entity Session {
  key (org_id: uuid, session_id: uuid)
  field secret token_hash: string<256>
  field secret refresh_secret: string<256>
  field expires_at: timestamp
  unique token (org_id, token_hash)
}

The classification marks values whose accidental display would be a security incident — session tokens, verification tokens, credential hashes, provider refresh secrets. It is declared once, in the contract, and every present and future display surface inherits it.

secret is a contextual modifier: it is an ordinary identifier everywhere else, so a field named secret stays valid, and adopting the keyword is a compatible contract evolution. Only stored entity fields accept it — key fields, event fields, and command inputs cannot carry the classification by construction. A contract that never uses the keyword keeps its exact prior encoding and bundle hash.

The promise

A secret-classified field’s value does not appear in:

• tracing, log, metric, or health output,
• typed public errors and diagnostics (which are value-free by construction),
• MCP tool output text,
• CLI rendering of records,
• provenance and audit summaries,
• generated example and documentation output.

Where a record rendering would have shown the value, surfaces emit the one stable marker instead:

[redacted:token_hash]

Redaction is structural, not reviewed-for: the service’s single release point wraps every secret value in a type whose display and debug forms emit only the marker and which no serializer can consume; the only way a value leaves the wrapper is a named reveal method whose call sites an architecture test enumerates. A new surface cannot obtain an unredacted rendering by accident, because it never receives the value at all.

Field visibility defaults deny

Capability field-visibility grants never reveal a secret field through their ordinary field list — enumerating every field (the shape every role default produces) is inert for secrets. Revealing one requires naming it explicitly through the grant’s dedicated secret-field naming, and:

• requesting or projecting an unrevealed secret fails with a typed field-visibility denial, never a silent narrowing;
• a delegated capability can only name secrets its parent names;
• scanning an index whose entries embed a secret field’s bytes requires the same explicit naming, because released index keys are a projection of the value.

Everything that uses the value without returning it works with no visibility at all: predicates on the field, uniqueness enforcement, and index participation run inside the command and query engines against full records.

Revealing a secret explicitly

The escape hatch is a capability whose field-visibility entry names the secret field in its dedicated secret_field_ids list (distinct from the ordinary field_ids, which never reveals). The naming survives storage at full fidelity (capability record V6) and delegation can only narrow it. A read that explicitly selects the named field under such a capability returns the value; the same read one grant short fails with a typed field-visibility denial. When creating a capability through the CLI or the admin API, add secret_field_ids to the field-visibility entry:

{
  "contract_lineage": "myapp",
  "entity_type_id": 3,
  "field_ids": [1, 2],
  "secret_field_ids": [7]
}

What is stored

Durable records, backups, exports, and changelog frames carry secret fields at full fidelity. A backup that silently dropped or masked secret fields would restore a broken application; what an auth workload stores is already the appropriate at-rest form (hashes, opaque tokens). The classification governs display surfaces, not storage.

What secret does not promise

secret is a display-and-visibility classification, not a cryptographic promise. It does not encrypt the value at rest or in transit, does not manage keys, and does not hash anything for you. Store secrets in an appropriate at-rest form — hash session tokens, keep provider refresh secrets opaque — exactly as you would without the keyword. The classification itself is not a secret: diagnostics may name a field as secret-classified.

See the unsafe-pattern entry in Unsafe Patterns and Corrections for the misuse this non-promise guards against.

The classification is sticky

The compiler follows a secret-classified value through expressions. Copying one into a non-secret stored field, durable event field, or command outcome is a compile error unless the exact flow site names every secret source with reveals:

set session.public_token = session.token_hash reveals session.token_hash

emit TokenIssued {
  token: session.token_hash reveals session.token_hash
}

return Issued {
  token: session.token_hash reveals session.token_hash
}

A value derived from multiple secrets names each source separately:

return Issued {
  score: row.secret_score + row.secret_offset
    reveals row.secret_score
    reveals row.secret_offset
}

The annotation is static contract identity, not an application permission. It is checked into source, carried in the executable plan and bundle hash, shown by command explain/catalog surfaces, and cannot be supplied by a caller at runtime. It does not grant read visibility to either source field. Copying a secret into another secret-classified stored field remains sticky and requires no disclosure annotation.

Named RiffQL reads use the same explicit-review principle at a returned leaf:

return Found {
  session: session {
    token_hash reveals session.token_hash
  }
}

Unlike a command-flow annotation, this declaration also becomes an exact application-role authority atom when—and only when—the role selects that immutable named query. The compiler places the stable field ID in the grant’s dedicated secret list; callers cannot add it through parameters or ad-hoc RiffQL. A missing or stale role returns RDB-AUTH-0214 with no partial result. MCP and reactive catalogs omit the query entirely; typed application SDK and gRPC execution remain available under the exact role. Generated Rust, Go, TypeScript, and Python clients publish the exact symbolic query/entity/field metadata. Their generated debug, string, representation, and redaction helpers do not print the returned secret value. Authorized application code can still read the typed value after release; metadata is for structural diagnostics and does not replace runtime authorization.

RDB-C046 rejects missing, wrong, duplicate, or excess declarations. Its diagnostic identifies both the destination flow site and the secret source field declaration, so an author can either remove the copy or make the intentional disclosure reviewable. Once disclosed into an event or outcome, the destination is ordinary visible data; reveals does not encrypt it or redact it after delivery.

Current alpha limitations

• Remote gRPC responses carry secret fields by omission (fail-closed): the wire record simply lacks them, and the remote CLI therefore shows absence, never the [redacted:…] marker. The marker appears on the MCP surface, which renders in-process from the service’s views. Wire-level marker carriage is follow-up work.
• TypeScript values are ordinary structural objects after authorized release; use the generated redact<Query>Result helper in diagnostics rather than logging the result object directly.

Symbolic application source and exact lock

The normal author-owned input is riffdb.application.json. New project-shaped repositories use riffdb.application-source/v7; existing V1 through V6 documents retain their exact writers and decoders. The source names paths, operations, roles, and output targets. It never asks an author or agent to discover, copy, or maintain a compiler-derived identity.

The compiler writes riffdb.application.lock.json using the lock generation corresponding to the source schema (V1 through V8 today). That exact lock covers the normalized symbolic source; contract source, bundle, plan-root, lineage, version, and compiler formats; query module, source, and plan identities; tenant-unbound role definitions and their exact operation authority; and every generated artifact hash. It contains no clock, host path, credential, active pointer, tenant value, or runtime-selected identity.

author source: riffdb.application-source/v1
compiler lock: riffdb.application-lock/v1

Source and lock V1 are permanent compatibility formats. They contain Rust, TypeScript, and MCP generation targets and are never upgraded implicitly. A Python application uses the explicit V2 successor, which adds exactly one required generation member:

{
  "generation": {
    "mcp": "generated/ticketdesk.mcp.json",
    "python": "src/ticketdesk/generated.py",
    "rust": "src/generated/riffdb.rs",
    "typescript": "src/generated/riffdb.ts"
  },
  "schema": "riffdb.application-source/v2"
}

Preview migration locally with riffdb application migrate --to v2. The --write form atomically changes only riffdb.application.json; it does not write a lock, generate code, deploy, bind a role, seed data, or contact a server. Run riffdb application lock --write separately after reviewing the canonical source change. Lock V2 then covers the exact generated Python path, bytes, and digest alongside the existing three artifacts.

Application Source V3 adds an exact migrations array while retaining the V2 Python target. Each entry names one .riffm source and one retained canonical parent bundle. Lock V4 pins those sources, parent identities, generated migration bundles, and the one canonical successor bundle. Source V3 is for contract-data migration; application migrate --to v2 remains only the source-format V1-to-V2 helper and does not create V3.

{
  "schema": "riffdb.application-source/v3",
  "migrations": [
    {
      "parent_bundle": "retained/ticketdesk-v1.riffdb.contract.bundle",
      "source": "riffdb/migrations/ticketdesk-v1-to-v2.riffm"
    }
  ]
}

See Contract Migrations for the complete identity model, supported Gate A changes, and read-only planning workflow.

Application Source V4 adds declared .riffr reactive modules and three closed role allowlists: event_streams, watch_queries, and agent_subscriptions. The compiler emits exact Application Manifest V2 and Lock V5 artifacts. Lock V5 pins separate reactive source, operation, and module hashes and retains the V4 migration and canonical contract-bundle closure.

{
  "schema": "riffdb.application-source/v4",
  "reactive_modules": [
    {
      "name": "TicketActivity",
      "source": "riffdb/reactive/ticket_activity.riffr",
      "version": 1
    }
  ],
  "roles": [
    {
      "agent_subscriptions": ["TicketAgent"],
      "commands": [],
      "environment": "development",
      "event_streams": ["TicketEvents"],
      "name": "TicketDeskAgent",
      "queries": [],
      "tenant_scope": "tenant",
      "watch_queries": ["TicketWatch"]
    }
  ]
}

These members are required arrays in V4, including when a particular role has no operations of that kind. Consume authority never implies stream seek authority. See Reactive Modules for grammar and fixed bounds.

Application Source V5 adds one exact Go generation target. It emits Application Manifest V3 and Lock V6; the lock pins Go, Rust, TypeScript, Python, MCP, contract-bundle, migration, and reactive artifacts in one closure. The reactive_modules member remains required but may be empty—an application with named reads and commands does not invent a dummy stream.

{
  "schema": "riffdb.application-source/v5",
  "generation": {
    "go": "generated/go/client.go",
    "mcp": "generated/mcp/tools.json",
    "python": "generated/python/client.py",
    "rust": "generated/rust/client.rs",
    "typescript": "generated/typescript/client.ts"
  },
  "migrations": [],
  "reactive_modules": []
}

riffdb new --language rust|go|typescript|python uses this same V5/V3/V6 identity chain for every language. Selecting a language changes the runnable starter, not the compiled application identity or operation schemas.

Application Source V6 adds a required symbolic row_policies allowlist to every role. It emits exact Application Manifest V4 and Lock V7. The lock’s V3 role-definition receipt names each selected policy, protected entity, and operation class; executable policy bytecode remains covered by the contract bundle hash and is never copied into an application request or generated client.

{
  "schema": "riffdb.application-source/v6",
  "roles": [
    {
      "agent_subscriptions": [],
      "commands": [],
      "environment": "development",
      "event_streams": [],
      "name": "DocumentReader",
      "queries": ["GetDocument"],
      "row_policies": ["DocumentAccess"],
      "tenant_scope": "tenant",
      "watch_queries": []
    }
  ]
}

The manifest schema exposes this structure for compiler review and exact-lock generation. Protected queries, commands, projections, live views, event delivery, and contextual reactions now use the shared transaction-current evaluator. A database-wide reactive wakeup remains unavailable to protected roles because it has no compiler-owned stream and partition identity; use the protected consumer’s bounded next/long-poll operation instead. Application middleware is not an accepted substitute. See Compiled Row Policies.

Application Source V7 keeps V6 command, query, reactive, migration, role, and row-policy semantics, but replaces the complete generation object with one nonempty closed subset of rust, go, typescript, python, and mcp. It emits Application Manifest V5 and Lock V8. Each present member is generated in memory and hashed into the exact lock; an absent member is not generated, hashed, checked, materialized, or used to discover a language toolchain.

{
  "generation": {
    "go": "generated/go/client.go"
  },
  "schema": "riffdb.application-source/v7"
}

The five target names and their artifact kinds come from one compiler-owned closed registry. Paths must be unique. Empty maps, aliases, unknown targets, unsafe paths, missing declared artifacts, and extra undeclared SDK or MCP lock artifacts fail closed. Removing a target changes source, manifest, and lock identity; RiffDB leaves any old undeclared file in place as unowned content until the user removes it explicitly.

riffdb.toml may request only targets declared by V7. It may materialize a subset in one checkout, but it cannot add a target to the lock or change the authoritative identity. Omitting mcp suppresses only the generated catalog file; it does not disable or change the hosted MCP service, authorization, or operation visibility. A local generated driver still needs a closed dispatch catalog. riffdb dev --run recompiles that catalog once from the exact source and lock into its private temporary directory, verifies every operation against the locked role, and passes its compiler-domain hash through the local handshake. The temporary runtime catalog is not an application artifact, does not appear in Manifest V5 or Lock V8, and is removed with the disposable development process.

The source document is closed JSON with exactly these top-level members:

{
  "application": "ticketdesk",
  "contract": {
    "lineage": "TicketDesk",
    "source": "riffdb/contract.riff",
    "version": 1
  },
  "generation": {
    "mcp": "generated/ticketdesk.mcp.json",
    "rust": "src/generated/riffdb.rs",
    "typescript": "src/generated/riffdb.ts"
  },
  "query_modules": [
    {
      "name": "ticketdesk",
      "queries": [
        {
          "name": "TicketPage",
          "source": "riffdb/queries/ticket_page.riffq"
        }
      ],
      "version": 1
    }
  ],
  "roles": [
    {
      "commands": ["CreateTicket"],
      "environment": "development",
      "name": "TicketDeskAgent",
      "queries": ["TicketPage"],
      "tenant_scope": "tenant"
    }
  ],
  "schema": "riffdb.application-source/v1",
  "seed_inputs": ["riffdb/seed/dev.jsonl"]
}

All paths are normalized, workspace-relative paths. Absolute paths, parent components, empty components, duplicate paths, duplicate names, undeclared query references, and unknown members are rejected. Versions are positive. Names and collections are bounded; the source document may not exceed one MiB. No hash field is valid in author source.

The manifest format also permits the domain-empty bootstrap emitted by riffdb init: it contains one named structural query module whose queries array is empty and an empty roles array. The module still receives a canonical identity for generation and deployment, but it exposes no query, command, tool, or authority. Authors add explicit queries and roles as the domain grows; no placeholder operation is required.

Each role declares an exact environment and either global scope or tenant scope. A tenant-scoped role must receive one concrete tenant at binding time; a global role rejects a tenant argument. The role author names only commands and named queries. Field visibility, stable IDs, indexes, partitions, result shape, and scan ceilings are compiler-private consequences of those names. If a selected .riffq named query declares an exact secret returned leaf, the compiler also derives its query/entity/field authority atom; application source has no field-authority list or wildcard.

RiffDB sorts every set-like collection, serializes the validated source as compact canonical JSON with a final line feed, and hashes those bytes under the source version’s domain. The compiler exposes byte spans for source-defined symbols and paths so diagnostics can point to the reviewed input. Reformatting or reordering set-like members does not change identity; changing a semantic value does.

Lock compilation resolves all symbolic names against exact compiled inputs and privately derives role authority. Locked generation byte-compares the recompiled lock before writing. It never silently selects an ambient active version, trusts an author-provided hash, widens a role, or regenerates against a different identity.

The older riffdb.application-manifest/v1 exact document remains a supported compatibility artifact for deployments and stable V1 tooling. It includes bundle and module hashes, but is now emitted under generated/riffdb.application.exact.json; application authors do not edit it. The checked-in TicketDesk compatibility fixture is fixtures/application-manifests/ticketdesk-v1.json in the repository root.

Run source, lock, generation, and drift checks with:

riffdb application check
riffdb application lock --write
riffdb application lock --check
riffdb application generate --locked
./scripts/check-application-bindings

Generated Rust and TypeScript facades are present in V1; V2 also requires the generated Python facade. V3 and V4 retain all four generation targets. V5 adds the generated Go facade and requires all five targets; V6 retains that complete set. V7 declares any nonempty subset and binds exactly that subset in Manifest V5 and Lock V8. The language facades own parameter serialization, response decoding, exact identity checks, opaque cursors, read-after-commit fences, typed command outcomes, and retry-safe uncertainty recovery. Generated MCP schemas come from the same operation registry but omit secret-output queries. Application code uses the generated facades for those authorized reads; it does not turn the query into a reveal-shaped command. text; numeric IDs, field masks, protobuf records, and raw RPC wrappers remain generated or internal implementation details.

Check, inspect, bind, and revoke roles symbolically:

riffdb role check riffdb.application.json --role TicketDeskAgent --tenant acme
riffdb role describe riffdb.application.json --role TicketDeskAgent --tenant acme
riffdb role bind riffdb.application.json \
  --role TicketDeskAgent \
  --tenant acme \
  --principal app:ticketdesk \
  --actor-kind service \
  --audience ticketdesk \
  --credential-output .riffdb/ticketdesk.credential
riffdb role revoke <capability-uuidv7> --reason replaced

If the role selects a row policy with declared principal facts, add one operator-owned --principal-facts <JSON_OBJECT_PATH> document. Its object keys must exactly match the symbolic principal_fact_schemas reported by riffdb role describe; values use the natural JSON forms documented in Row policies. Generated application clients never receive the document, policy bytecode, or a policy-bypass parameter.

tenant_scope is exact author intent. A role declaring "tenant" requires --tenant <tenant-id> on check, describe, bind, development binding, and provisioning. A role declaring "global" requires that --tenant be omitted. The CLI reports application_role_tenant_required or application_role_tenant_forbidden with that precise recovery action before role compilation; it never silently widens a tenant role or invents a tenant.

The compiled role has a domain-separated identity covering the manifest, contract, immutable query modules, environment, tenant binding, named operations, compiler-derived visibility, and resource ceiling. That identity is retained as a non-authorizing capability marker for audit and substitution detection. The marker cannot execute any operation.

One identity chain at deployment

riffdb application deploy is the normal installed path for contract, query module, application role, generated client/MCP configuration, and seed data. It binds these steps into one checked chain:

application lock
  -> pinned canonical contract bundle
  -> exact parent version + bundle hash
  -> contract bundle hash
  -> query module version + hash
  -> reactive module version + hash + exact query dependencies
  -> compiled role identity
  -> capability ID + private credential

The private deployment journal records each exact identity after remote verification. Query-module publication uses the observed active module hash as its compare-and-swap precondition. A concurrent change reports the selected database and the lock, locked-module, expected-active, and actual-active hashes instead of degrading to invalid input. A process interruption before local publication is recovered by rerunning the same deploy; retained request and capability identities make replay deterministic.

For a V4 application, deployment then publishes each .riffr module against that exact contract and the canonical set of locked query-module identities. The server recompiles source and the CLI accepts the result only when the returned module name, version, content hash, and contract identity equal the lock. Immutable replay is accepted; a same-name/same-version content change, missing contract, or missing query-module dependency fails closed. The private deployment journal records reactive module identities separately, so a process interruption after remote publication resumes by verifying and replaying the same immutable module rather than inventing a new identity.

A changed lock is deliberately not resumed under an old role. If a role is already retained, deploy requires --provision-role <name> together with --replace-role-credential. The replacement is a resumable rotation: RiffDB retains the predecessor, creates and proves the exact successor, atomically switches the protected generated client and MCP configuration, and only then revokes the predecessor. A retry resumes from the retained phase; it never revokes the only proven credential. Application code and scripts should read identity from the generated client or lock and should never hardcode a contract version or module hash.

Role compilation for lock V3 always consumes generated/riffdb.contract.bundle and the exact locked query modules. Both application bind-dev-role and standalone role check|describe|bind discover that lock from either riffdb.application.json or the generated exact manifest. A successor is never reinterpreted as genesis during role compilation. The legacy --replace-expired-credential spelling remains an alias for --replace-role-credential.

Lock V3 makes the contract bundle part of this same chain. Genesis application lock --write remains local. For a successor, the command performs an authorized, read-only server preview against the active parent and writes generated/riffdb.contract.bundle; it does not deploy. Offline application lock --check and application generate --locked decode that pinned bundle instead of recompiling the successor as genesis.

When a V3 lock already pins byte-identical contract source, application lock --write refreshes query, role, and generated-client artifacts locally from that checked bundle. This is the supported path after installing a corrected code generator: it rewrites compiler-owned artifact hashes without treating the already-active contract as another successor. Any contract-source change still requires the authorized server preview against the active parent, and a damaged or substituted pinned bundle fails closed instead of falling back to the network path.

Lock V4 retains the V3 contract-bundle pin and adds exact direct-parent migration artifacts. riffdb migration plan --application riffdb.application.json validates and prints that local plan. This command is inspection only; it cannot mutate or activate a database.

Before deployment, the server compares the lock’s expected parent version and bundle hash and its candidate bundle hash. A disagreement returns the lock, expected-parent, actual-active, and server-compiled candidate identities and changes no catalog, module, role, seed, or deployment-journal state. If the exact candidate is already active, retry is an idempotent success.

Contract migrations

RiffDB distinguishes a compatible contract successor from one that requires existing authoritative rows or derived state to be checked or transformed. The latter is RequiresMigration: the candidate is valid, but deployment does not activate it without an exact migration artifact for the active parent.

The current implementation compiles and locks Gate A additive, Gate B structural, and Gate C key/ownership migration artifacts, checks and applies them through the public administration service, and drives the internal redb execution and startup-recovery path: complete preflight, an immutable automatic backup, bounded staged transforms, projection rebuild, complete validation, atomic publication, and automatic post-publication rollback. riffdb migration plan remains a local read-only inspection. The retained TicketDesk Gate-A, StructuralRows Gate-B, and KeyOwnershipRows Gate-C fixtures freeze the additive, structural, and key/ownership artifact boundaries.

Semantic execution boundary

The catalog rechecks the exact parent, candidate, migration hash, direct compatibility, and complete gate-specific proof before producing a validated plan. Migration expressions run as pure row-local evaluations. Apply repeats the whole read-only preflight before it constructs any mutation.

The commit-owned migration coordinator is the only first-party caller of the stage mutation and cutover ports. It scans at most 256 rows per page and applies at most 64 row/index transitions per atomic journal step. Every transition rechecks the exact target, entity version, schema binding, canonical bytes, and canonical row hash. Required projection generations are tied to the frozen application frontier before final cutover.

Final validation still scans every row and validates the private stage and its immutable history. The redb migration validator checks newly added unique constraints with at most two entries per complete canonical unique prefix. Every row must have exactly its independently derived owner, schema, partition and covered values in the rebuilt index. Missing entries cannot hide duplicates. The observation borrows the exclusive frozen stage and cannot survive a mutation or restart; adapters without this observation retain independent full scans. Preflight and cutover atomicity are unchanged.

Gate-A required-field transforms advance each changed entity version exactly once and bind that post-image to the successor. Index-only and validation-only work does not advance entity versions. Migration never assigns an application commit sequence or rewrites command, outcome, event, provenance, idempotency, outbox, or retained history bytes. Only successful final cutover assigns one administration sequence in the reference model.

Every transformed row archives its exact displaced predecessor envelope under the migration operation identity in the successor database. Startup accepts a version transition without an application commit only when that archive row, the permanent migration record, the migration hash, the predecessor chain tip, and the successor schema binding all agree. Missing or inconsistent evidence fails closed. These archives preserve historical continuity; they are not a second writable entity store.

These semantics are not a hidden storage-edit API. The memory stage remains the reference model. The redb implementation realizes the same sealed ports; only the commit-owned coordinator can construct batches or cutover, and neither the server recovery controller nor a storage adapter interprets migration IR.

Durable redb lifecycle

An accepted operation durably binds a UUIDv7 operation ID, canonical input hash, exact parent/candidate/migration hashes, exact protected artifact files, and restart-stable admission evidence. Same operation ID with different input fails before drain. The receipt uses a closed monotonic phase graph and contains no row value, credential, token, or absolute path.

After the selected database drains, apply repeats the complete read-only preflight before allocating disk or creating a backup. RiffDB reserves a conservative amount of space, creates and verifies the immutable normal backup pre-migration-<operation-id>, and materializes a protected sibling stage on the target filesystem. It never stages through /tmp, accepts a caller-selected path, follows a symlink, or publishes across filesystems.

A deterministic artifact, predecessor-data, pending-admission, capacity, or disk preflight rejection records a value-free FailedClosed reason, creates no backup, and reopens the freshly validated predecessor. Invalid predecessor data must then be repaired through ordinary predecessor commands before a new migration operation is attempted.

The stage journal advances atomically with each bounded mutation transaction. The coordinator scans at most 256 rows per page and writes at most 64 mutations per batch. Required projections build in fresh generations through the frozen application frontier. A complete startup-equivalent validation of the private stage is required before atomic rename and parent-directory synchronization.

Cutover atomically activates the exact successor, installs the predecessor write fence, stores permanent migration evidence and terminal service audit, and consumes one administration sequence. It consumes no application sequence and does not rewrite command history. The automatic backup is retained after success.

Restart and rollback behavior

Startup reconciles the checksummed external receipt, protected artifacts, backup manifest, sibling stage, published target, in-database journal, permanent migration record, and predecessor-write retirement. It never guesses between inconsistent evidence.

• Before publication, the predecessor remains authoritative. Recovery resumes the exact journaled stage or recreates it from the verified backup.
• At the publication boundary, recovery determines whether the exact predecessor or exact successor is installed and continues from that state.
• After publication, the successor is not ready until a fresh complete startup validation succeeds.
• If published validation fails, recovery restores and freshly validates the exact automatic backup before readiness, then records FailedRolledBack.
• A target, receipt, stage, backup, journal, retirement, or permanent-record combination outside the closed reconciliation table fails closed.

Migration maintenance owns only the selected database files. Database identity checks and path confinement prevent a stage, backup, or receipt from being rebound to a sibling database. Apply drains only the database selected by the client’s --database or client configuration. Other configured databases keep serving, while one process-wide migration exclusion prevents a second check or apply from starting concurrently.

The durable V1 compatibility vectors live under fixtures/migrations/durable/v1. Protobuf schema hashes and record bounds live under fixtures/proto. Run ./scripts/generate-migration-durable-fixtures --check after changing a receipt codec, migration table key, or migration record encoding.

Exact identity model

One application release has one canonical successor contract bundle. Each supported predecessor gets a separate direct migration:

exact parent bundle + .riffm source + canonical successor bundle
    -> canonical MigrationBundleV1

A supported parent must have the same lineage and a lower version. It does not have to be the successor bundle’s immediate compilation parent. This permits a release to support several retained predecessor versions without chaining migrations or producing several successor identities. Source version numbers alone are insufficient: the lock pins both endpoint bundle hashes.

At most 32 parent migrations may be retained in one Application Source V3 through V7. Every source path, parent artifact path, parent version, and generated migration artifact path must be unique.

Author a migration

Retain the canonical parent bundle before changing the contract. Add the successor contract source, bump its version, and declare the exact parent and migration source in riffdb.application.json:

{
  "schema": "riffdb.application-source/v3",
  "contract": {
    "lineage": "TicketDesk",
    "source": "riffdb/contract.riff",
    "version": 2
  },
  "migrations": [
    {
      "parent_bundle": "retained/ticketdesk-v1.riffdb.contract.bundle",
      "source": "riffdb/migrations/ticketdesk-v1-to-v2.riffm"
    }
  ]
}

The abbreviated object above shows only migration-related members. Source V3 also requires all normal source members and a Python generation target. See the canonical fixture at fixtures/application-manifests/ticketdesk-migration-v3.json.

For a required field, provide one deterministic row-local expression:

migration TicketDesk from 1 to 2 {
  transform Ticket {
    set normalized_priority = old.priority + 1
  }
}

Migration expressions may inspect constants and fields on old. They cannot read another row, access the network or filesystem, observe wall-clock time, or use ambient randomness. Input size, nesting, declarations, expressions, steps, output bytes, and retained parents are bounded.

For these Gate A changes, the compiler derives the step and no handwritten clause is needed:

• add an index to an existing entity;
• add a relationship or unique constraint to existing entities;
• add an entity or aggregate invariant over existing rows; and
• add a projection that must be rebuilt from the frozen frontier.

An empty body is valid when all necessary work is compiler-derived:

migration TicketDesk from 1 to 2 {}

Missing, duplicate, stale, or unnecessary proofs fail with bounded source-span diagnostics.

Structural Gate B

A semantic rename keeps the original numeric stable ID only when the exact type and semantic role remain unchanged. The predecessor name becomes a permanent lineage alias and can never be allocated again. Use the fully qualified owner for scoped identities:

migration StructuralRows from 1 to 2 {
  rename entity Row to Record
  rename field Row.status to workflow_status
}

Renaming an entity, field, enum variant, command, event, outcome, or projection uses the migration-aware successor compiler. Ordinary successor compilation does not infer a rename and continues to reject stable-ID reuse. The generated application lock pins the migration-aware successor, so lock checks and deploy preview agree on one exact bundle identity.

Changing a field’s type or meaning is a replacement, not a rename. The old identity is tombstoned, the replacement receives a fresh field ID, and every predecessor row is checked before staging begins:

migration StructuralRows from 1 to 2 {
  transform Row {
    require old.value >= 0
    replace value with amount using checked_i64_to_u64
  }
}

The closed Gate-B conversion names are:

• identity and wrap_optional;
• assert_unwrap_optional;
• checked_i64_to_u64 and checked_u64_to_i64;
• decimal_exact;
• assert_bounded_narrow for strings, bytes, and lists;
• list_elements for the closed element-conversion set; and
• uuid_to_string and string_to_uuid using lowercase hyphenated UUID text.

The complete migration fails preflight on a failed require, null optional unwrap, integer overflow or sign error, inexact decimal rescale, bound violation, invalid list element, or noncanonical UUID string. It never rounds, truncates, clamps, substitutes a fallback, or skips a row.

Enum changes require one exhaustive mapping entry for every predecessor variant. Mapping targets must exist in the successor enum:

map enum WorkflowStatus {
  Open -> Open
  Closed -> Archived
}

retire logically removes an entity, command, event, outcome, enum variant, or projection identity. Entity rows move to the migration archive; historical bundles and committed events, outcomes, provenance, and idempotency evidence remain byte-exact. Retiring a command or entity also closes its owned identity tree. There is no physical purge. A replacement projection uses a fresh projection identity; retire the old projection and let the compiler derive the new projection rebuild through the frozen frontier.

migration Surface from 4 to 5 {
  retire command LegacyWrite
  retire projection OldTotals
}

Gate C supports primary-key replacement, rekeying, repartitioning, aggregate-membership changes, relationship-target changes, and conflict-domain changes. A rekey supplies every successor primary-key component as a row-local expression over the complete predecessor row:

migration Accounts from 2 to 3 {
  transform Account {
    set region = "global"
    rekey (old.tenant_id, "global", old.account_id)
  }
  acknowledge repartition Accounts
  acknowledge conflict Accounts
}

The compiler derives all affected authoritative indexes, relationships, uniqueness checks, aggregate ownership, partition materialization, and conflict meaning from the exact parent and successor. An acknowledgement authorizes only the corresponding compiler-derived aggregate change; it cannot supply a key or override the candidate schema.

Preflight evaluates every row before backup, proves complete successor-target uniqueness, resolves relationships against the complete candidate target set, and validates successor unique prefixes. A target already occupied in the predecessor keyspace fails closed, even when another row would later vacate it. Consequently Gate C does not support key swaps or predecessor-occupied rekey chains; those require a future two-phase key namespace rather than inferred ordering. Each admitted move archives the exact old target and row, advances the entity version once, removes every old index suffix, and installs the successor row and complete index set in the same journaled stage transaction. The private stage remains invisible until final validation and cutover retire predecessor writes.

Lock and inspect

For a successor, lock creation asks the selected server for an authorized, read-only parent-aware candidate preview. It then compiles every declared migration locally and writes Application Lock V4 through V8 plus generated migration bundles:

riffdb --config "$HOME/.config/riffdb/client.toml" \
  application lock --write
riffdb application lock --check
riffdb --output json migration plan \
  --application riffdb.application.json \
  --lock riffdb.application.lock.json

The plan reports the canonical successor, each supported parent and source hash, migration bundle identity, stable step categories, step count, and fixed resource bounds. Planning rechecks every locked file and fails on drift. It does not contact storage or mutate a database.

Application Lock V4 through V8 pin the normal generated artifacts, canonical successor bundle, exact .riffm source hash, retained parent bundle version and hash, and generated MigrationBundleV1 hash. Do not edit generated bundles or lock identities by hand.

Contract migrations are operator-only. They are not generated as application commands, application-role permissions, MCP tools, or MCP resources.

Authorize a migration operator

Migration authority is a dedicated global, database/environment-bound, exact lineage permission. Deploy, backup, restore, capability administration, MCP, and application-role authority do not imply it. Create a separate short-lived operator capability through an existing capability administrator:

{
  "principal_id": "migration-operator",
  "actor_kind": "human",
  "requested_lifetime_seconds": 3600,
  "audiences": ["riffdb-grpc-loopback"],
  "grant": {
    "tenant_scope": { "type": "global" },
    "partition_scope": { "type": "all" },
    "permissions": [
      { "type": "migrate_contract", "contract_lineage": "TicketDesk" }
    ],
    "field_visibility": [],
    "max_scan_rows": 1,
    "approval_required": []
  }
}

riffdb --database ticketdesk \
  --credential-file "$HOME/.config/riffdb/operator.credential" \
  capability create \
  --request migration-operator.json \
  --credential-output "$HOME/.config/riffdb/migration.credential"

Adding "migrate_contract" to approval_required denies migration until an approval integration can satisfy that obligation. The POC does not provide such an integration, so leave it absent for an intentionally authorized local operator. Retain the returned capability ID so it can be revoked.

Check, apply, and observe

First inspect the lock and retain the exact lowercase migration hash:

riffdb migration plan \
  --application riffdb.application.json \
  --lock riffdb.application.lock.json

Use a fresh canonical UUIDv7 for check. A lock with multiple retained parents also requires --migration-hash; omitting it is accepted only when exactly one migration is locked.

riffdb --database ticketdesk \
  --credential-file "$HOME/.config/riffdb/migration.credential" \
  migration check \
  --operation-id "$check_operation_uuidv7" \
  --migration-hash "$migration_hash"

Check is durable and read-only. Retry an uncertain check with the same operation ID, artifacts, and migration hash. Apply is a different semantic operation and therefore requires a different UUIDv7. Its confirmation is both the exact selector and the destructive confirmation:

riffdb --database ticketdesk \
  --credential-file "$HOME/.config/riffdb/migration.credential" \
  migration apply \
  --operation-id "$apply_operation_uuidv7" \
  --confirm-apply "$migration_hash"

After durable acceptance, client cancellation does not cancel apply. The selected database rejects ordinary traffic while draining, offline, staging, publishing, and validating; configured siblings remain available. Poll after the selected database reopens:

riffdb --database ticketdesk \
  --credential-file "$HOME/.config/riffdb/migration.credential" \
  migration operation "$apply_operation_uuidv7"

Responses contain only the closed phase/failure classes, exact artifact hashes, and protected backup identity. They never contain row values, credentials, absolute paths, or internal error sources. Same operation ID with different input fails before drain. A successful repeat apply can resolve as already_applied only when the permanent migration edge and retained terminal receipt match exactly.

There remains no supported storage-level or manual file procedure for starting a migration. TypeScript, Python, generated application clients, and MCP expose no migration authority or operation; use the CLI, public gRPC administration API, or the stable Rust client facade.

Workflow Transitions and Fenced Leases

RiffDB workflow operations are compiled commands, not a lock API. They use the same authorization, idempotency, provenance, commit, and recovery path as every other application mutation. There is no generic compare-and-swap, entity editor, caller clock, or scheduler bypass.

For the bounded worker protocol that safely composes these commands with named queries or durable events, see Bounded Workflow Schedulers.

Declare the workflow state

enum WorkState { Queued, Running, Complete }

entity WorkItem {
  key (organization_id: uuid, work_id: uuid)
  field state: WorkState
  field lease_owner: optional<uuid>
  field lease_expires_at: optional<timestamp>
  field lease_fence: u64
  field lease_attempts: u64
}

workflow WorkLifecycle {
  entity WorkItem
  state state
  initial Queued
  transition Start from (Queued) to Running
  transition Finish from (Running) to Complete
  lease execution {
    owner lease_owner
    expires_at lease_expires_at
    fencing_token lease_fence
    attempts lease_attempts
    duration_seconds (5, 900)
  }
}

initial is the only ordinary-create path for a workflow state. The compiler injects the named enum value into every ordinary or bounded-collection create; the command has no input or set operation that can choose a different state. Without initial, an ordinary create cannot construct the required workflow state field. Portable restoration is a separate, operator-authorized operation and does not widen normal application commands.

A declared transition may return to the same state. This is useful for revision-checked operations such as rotating a session token while the session remains Active. A self-transition still requires the exact observed revision, checks the declared source state, returns the declared stale/illegal outcomes, and advances the entity revision only with the atomic mutation.

The owner and expiry fields must be optional<uuid> and optional<timestamp>. The fence is u64; the optional attempt field is also u64. Lease fields must be distinct non-key fields. Duration bounds are inclusive, nonzero seconds and cannot exceed 86,400 seconds.

New rows receive their workflow state from initial. They initialize the optional owner and expiry to null and must explicitly initialize the fence and attempt counters. The first successful claim advances the fence from zero to one.

Claim work

command ClaimWork {
  input request_key: string<128>
  input organization_id: uuid
  input work_id: uuid
  input owner_id: uuid
  input duration: u64
  input expected_revision: u64
  idempotency_key request_key

  mutate WorkItem(organization_id, work_id) as work else Missing {}
  lease claim execution on work owner owner_id duration_seconds duration revision expected_revision
    stale StaleRevision {}
    unavailable AlreadyClaimed {}
    invalid InvalidDuration {}
    exhausted LeaseCounterExhausted {}

  return Claimed { work: work }
}

Claim requires an exact observed entity revision. It succeeds only for an unowned lease or one whose expiry is at or before the admitted transaction time. Success increments the nonzero fence and, when declared, the attempt counter. Counter or timestamp overflow returns the declared exhausted outcome without mutation.

The owner, duration, revision, and fencing-token expressions used by lease operations must be direct, typed command inputs. RiffDB rejects computed or implicitly substituted values. Generated application bindings therefore make the exact concurrency evidence required method parameters.

On the declared success outcome, generated clients also return a bounded workflow_revisions collection. Each item names the source binding and its exact successor entity revision. Callers feed that revision into the next transition, renewal, release, expiration, or fenced business command; they do not re-read the entity or guess from a commit sequence. Non-success outcomes return an empty collection. Same-key replay returns the identical successor revision because the accepted input and persisted outcome are identical.

For example, a successful ClaimWork called with expected_revision = 7 returns { binding: "work", revision: 8 } alongside Claimed. Revision overflow fails closed as an invalid generated response and never wraps.

Renew, release, expire, and protect work

The remaining operations form a closed family:

lease renew execution on work owner owner_id fencing_token token duration_seconds duration revision expected_revision
  stale RenewStale {} invalid WrongHolder {} expired LeaseExpired {} exhausted LeaseCounterExhausted {}

lease release execution on work owner owner_id fencing_token token revision expected_revision
  stale ReleaseStale {} invalid WrongHolder {}

lease expire execution on work revision expected_revision
  stale ExpireStale {} active LeaseStillActive {}

lease fence execution on work owner owner_id fencing_token token revision expected_revision
  stale WorkStale {} invalid WrongHolder {} expired LeaseExpired {}

• renew requires the current owner, fence, revision, and an unexpired lease; it computes a new expiry from transaction time and the checked duration.
• release requires the current owner, fence, and revision; it clears owner and expiry but never rolls the fence backward.
• expire requires the exact revision and succeeds only after the recorded expiry. Time passing alone never mutates authoritative state.
• fence performs no lease-field mutation. It proves that ordinary protected work is still held by the exact owner, fence, and revision before the other command effects can commit.

Every command that mutates an entity with a declared lease must execute a lease operation for that binding, either once at the top level or once inside every apply arm of a decide statement on that binding. Omitting the fence is a compiler error. This prevents a generated caller from accidentally writing around the lease. Creation remains possible so the initially unowned row can be established.

Time, retries, and authority

Lease expiry uses only the service-observed transaction time sealed at command admission. The deterministic runtime never reads the operating-system clock or randomness. Retry and recovery reuse the persisted observation, so the same idempotency identity cannot acquire a different expiry or service-owned value.

A lease is state, not authority. Possessing an owner ID or fencing token never grants command permission. Every claim, renewal, release, expiration, and protected action is authenticated and authorized under current capability facts. Avoid placing lease evidence in logs or public diagnostics.

Aggregate lease fences are also distinct from database-history incarnation and future replication leadership epochs. They are never interchangeable and a future promoted leader must validate both routing/leadership evidence and the aggregate’s current fence.

Failure model

All business failures are declared outcomes and commit with zero mutations. Concurrent changes are transaction-current dependencies: RiffDB either commits the exact checked operation, reevaluates within its bounded retry policy, or returns the declared stale outcome. It never silently substitutes a newer revision or token for the caller.

Bounded Workflow Schedulers

RiffDB’s scheduler is an application worker, not a privileged database subsystem. It discovers eligible work through one exact named RiffQL query or durable event stream and performs every mutation through generated compiled commands. It has no storage handle, entity editor, arbitrary query text, generic transaction callback, or authority derived from a lease.

The POC provides the transport-neutral riffdb-scheduler orchestration crate and generated workflow observations for Rust, Go, TypeScript, and Python. A deployment still owns the small adapter that maps the closed scheduler port to its generated application client. There is not yet a standalone scheduler daemon.

Least-authority role

A scheduler role contains only its selection operation and the exact commands for its workflow. For example, the MLflow-shaped acceptance role grants:

query DueRun
command ClaimRun
command CompleteRun
command ReleaseRun

The Woodpecker-shaped role consumes the exact PipelineTransitions durable stream and grants only ClaimPipeline, StartPipeline, and ReleasePipeline. Neither role grants kernel reads, raw field access, administrative operations, or generic command execution. Server authorization is rechecked on every operation. A current owner ID and fencing token are workflow state, never credentials.

One worker turn

For one bounded work item, the worker performs this sequence:

1. Select at most one item from the configured named query or durable stream.
2. Resolve the stable business-command idempotency identity.
3. Acquire a fresh fenced claim for the current delivery attempt.
4. Execute the exact business command only when no persisted outcome exists.
5. Release the current fence using the generated successor revision.
6. Acknowledge the durable item only after business state and lease cleanup are resolved.

Resolving the business outcome before claiming matters after a crash. The business attempt identity stays stable across redelivery, while claim and release identities include the nonzero durable delivery-attempt ordinal. A redelivery therefore cannot replay an obsolete fence, and it cannot execute the business mutation twice. If the previous process died after the business commit, the next process acquires and releases a fresh fence before advancing the checkpoint.

Attempt identities and bounds

The v1 attempt hash is domain-separated and binds:

• schedule name and exact query or reactive module identity;
• logical due timestamp from application state or the event;
• explicit partition component and bounded target-key suffix; and
• attempt kind, including delivery ordinal for claim and release.

The resulting sched_<digest> value is suitable for a generated command’s idempotency input. It contains no plaintext tenant or target data. The current POC accepts at most eight target components, 1,024 canonical target bytes, eight in-flight items, ten delivery attempts, a 30-second selection wait, a 5–900 second claim duration, and a one-hour retry-delay cap.

Wakeup hints contain only an opaque generation. They carry no event payload, cursor, lease, capability, or business key. The worker always obtains the authorized bounded work item through its named application operation.

Delivery guarantee

The scheduler composes durable at-least-once delivery with command idempotency. A process may receive an item again after death, lease expiry, revocation, cancellation, or a lost acknowledgement. Repeating the stable business identity resolves the stored command outcome; it does not duplicate business state or its atomically emitted domain events.

RiffDB does not claim exactly-once external effects. External calls belong after the command commit and should consume durable events or outbox intent with their own idempotent destination identity.

Generated workflow evidence

Successful generated workflow commands expose a bounded collection of symbolic binding names and successor revisions:

WorkflowSuccessorRevision { binding: "run", revision: 42 }

Declared failure outcomes expose an empty collection. Callers pass the exact successor revision to the next fenced command rather than re-reading, guessing from commit sequence, or silently substituting a newer revision. Rust, Go, TypeScript, and Python bindings implement the same rule and reject revision overflow.

The checked fixtures under fixtures/adapters/mlflow and fixtures/adapters/woodpecker demonstrate exact role compilation and generated bindings. ./scripts/adapter-workflow-acceptance --all-languages checks their locks and compiles or type-checks all four language surfaces. The process-level scheduler recovery test exercises both shapes at every selection, claim, business/event, release, and checkpoint crash boundary, plus duplicate wakeup, stale worker, revocation, lease expiry, and restart.

Domain events

Application-facing consumption is defined with bounded .riffr reactive modules, not raw commit-log access. See Reactive Modules for partition proofs, selected payload fields, exact identities, and role permissions.

RiffDB domain events are immutable facts committed atomically with the command’s state changes, declared outcome, idempotency result, provenance, and commit record. They are not a second event store and do not require reconstructing entity state from events.

Declaring an application-streamable event

An event is available to symbolic replay only when it declares the ordered payload fields that derive its application partition:

event TicketCreated {
  partition_by (organization_id)
  policy_anchor current Ticket(
    organization_id: organization_id,
    ticket_id: ticket_id,
  )

  organization_id: uuid
  ticket_id: uuid
  created_at: timestamp
}

policy_anchor current is required before an event may be released to a row-policy-protected role. It is compiler-owned: the declaration names exactly one source entity and maps every partition/key component to a required, exactly-typed payload field. Partial, optional, cross-partition, unindexed, or multiple anchors fail compilation. RiffDB persists the resolved entity key and policy identity with the event; clients never provide either one.

At every protected replay, tail, stream lease, acknowledgement, contextual delivery, and contextual reaction safe point, RiffDB reloads the current capability, current source row, and bounded relationship evidence together. Missing/deleted rows, old events without an anchor, revoked facts, and denied current policy are indistinguishable absence. Event payload field names are never interpreted as authority. Consequently, a retained event can become invisible after an ACL change and visible again when explicitly replayed from an earlier retained cursor after authority is restored.

Every command that emits TicketCreated must supply the exact command partition expression under the same aggregate-namespaced canonical key schema. The compiler rejects a merely equal-looking value, a different expression, a different aggregate namespace, a missing field, or a cross-partition emit. There is no runtime fallback to a global scan.

Events without partition_by remain valid sources for projections and outbox delivery, but the application event catalog marks them as unavailable for replay and streaming.

Outbox delivery remains retryable after commit. If a delivery attempt fails or its completion races with a status change, the dispatcher retains its position before the interrupted item. Remaining items in that page are available on the next scan; delivery is still at least once and consumers must deduplicate by the stable event ID. Operators can configure the POC’s Durable Local Outbox Archive; applications retain the same compiled event emission and consumption interfaces.

Evolution

Events use the contract version and stable event symbol as their only evolution clock. Adding an optional payload field is compatible and historical events are materialized with null by the catalog. Changing a field’s meaning requires a new event symbol. Once an event has a partition proof, removing or changing that proof is incompatible.

Adding a partition declaration to a previously unpartitioned event is allowed only when the compiler proves every retained writer shape. Historical routing is rebuilt from authoritative commits, and every replayed payload must derive the same partition as its enclosing commit before it can cross the application boundary.

Replay safety

Application replay is symbolic and single-partition. Callers select an event name, exact partition fields, and an explicit bounded payload field set. RiffDB then joins each route to the immutable event, enclosing commit, historical writer plan, and active lineage materialization proof. Missing or contradictory evidence fails the complete page.

The normal envelope may include:

• stable event identity and symbolic event name;
• writer contract version and plan identity;
• commit sequence/ordinal and logical occurrence time;
• symbolic command name and command request correlation;
• safe actor kind and opaque provenance locator;
• history incarnation, opaque cursor, and explicitly selected payload fields.

It never includes raw principal or agent-session identity, partition or conflict keys, unselected payload, credentials, or process-local trace data.

Replay order is increasing EventId within exactly one logical partition. This does not promise global order, time-based retention, raw CDC, exactly-once delivery, or event-sourced reconstruction.

Inspecting events

RiffDB provides an operator-oriented event catalog over the shared application service. event describe requires ReadContract; event replay and event tail require ReadCommit. These permissions are intentionally broader than the least-authority named-stream permissions generated for applications in Reactive Modules.

Describe the active declaration before constructing a replay request:

riffdb --database ticketdesk event describe TicketCreated

Replay requires every ordered partition component and at least one explicitly selected payload field. CLI partition values use the same tagged canonical JSON shape as other kernel commands:

riffdb --database ticketdesk event replay TicketCreated \
  --partition 'organization_id={"type":"uuid","value":"018f6f50-6f31-7d62-9a7e-4f8b913d2f11"}' \
  --field ticket_id \
  --field created_at \
  --limit 50

next_cursor is opaque, process-local, principal-bound, and valid for five minutes. It may be present on an empty page because the route scan advanced past other event types in the same partition. Resume such a page with --cursor; do not infer completion from an empty items array alone.

Every page also carries disposition. page is an ordinary visible page or exact end. bounded_progress means the protected scan reached its fixed 1,024-candidate work ceiling and returns an opaque cursor without revealing how many candidates were hidden. Resume it exactly like any other cursor. For a protected cursor, capability revision and application-role policy identity are part of the server-side binding; rotation or narrowing invalidates it rather than splicing old and new authority.

Tail performs a race-free catch-up before waiting for a commit wakeup and then replays authoritative routes again. It waits for at most 30 seconds and reports wait_timed_out: true only when that interval expires without a selected event:

riffdb --database ticketdesk event tail TicketCreated \
  --partition 'organization_id={"type":"uuid","value":"018f6f50-6f31-7d62-9a7e-4f8b913d2f11"}' \
  --field ticket_id \
  --after 41:0 \
  --wait-nanos 30000000000

Both replay and tail return the current history_incarnation. Supplying it on a later call detects restore and fails closed instead of interpreting a stale position against replacement history. A tail request does not accept a replay cursor; use the last returned event ID as its next --after position.

The singleton global reactive-wakeup resource has no stream, consumer, or partition identity, so it cannot distinguish visible from hidden commits. RiffDB therefore denies that resource to row-policy-protected roles. Such workers use the protected consumer pull/long-poll operation; a wakeup is never authority to disclose or act on an event.

Reactive modules

Reactive module grammar V1 is the compile-time application surface for named event streams, query watches, and contextual subscriptions. Authors place the bounded source in a .riffr file and declare it in Application Source V4. RiffDB resolves every name against one exact contract and immutable query modules before it creates any deployable artifact.

reactive TicketActivity version 1 {
  stream TicketEvents($organization_id: Ticket.organization_id) {
    partition (organization_id = $organization_id);
    event TicketChanged select (organization_id, ticket_id, status);
  }
  watch TicketWatch(
    $organization_id: Ticket.organization_id,
    $ticket_id: Ticket.ticket_id
  ) query GetTicket updates patch;
  subscription TicketAgent($organization_id: Ticket.organization_id) {
    stream TicketEvents(organization_id = $organization_id);
    hydrate ticket query GetTicket(
      organization_id = $organization_id,
      ticket_id = event.ticket_id
    );
    reaction assign command AssignTicket;
    limits { batch 4; in_flight 4; lease_seconds 60; }
  }
}

Static guarantees

• A stream has one complete partition binding shared by every selected event.
• Event types and payload fields are explicit. Predicates can use only selected fields and typed parameters.
• A patch watch is accepted only when its query returns a complete authorized primary key. Other bounded queries use reset updates.
• A subscription binds one exact stream, no more than 16 bounded hydration queries, no more than 32 named command reactions, and fixed delivery limits.
• Aggregate query work remains within 500 rows and encoded output remains within four MiB.
• Batch and in-flight limits are each 1 through 8. Leases are 5 through 900 seconds.

Reactive source, each compiled operation, and the complete module have separate domain-separated hashes. Lock V5 records the grammar and IR versions, all source/module/operation hashes, exact query dependencies, V2 role definitions, and generated artifact hashes.

Least-authority roles

Application Source V4 roles name event_streams, watch_queries, and agent_subscriptions. The compiler lowers those names to exact permissions bound to contract lineage, reactive module hash, and operation name:

SeekEventStreamConsumer is separate administrative authority. Normal role compilation never grants it as a consequence of consuming a stream. Reactive permissions also do not imply raw commit-log, entity, index, or ad-hoc query access.

Publication and durable consumption

riffdb application deploy publishes every locked reactive module after its exact contract and immutable query modules are available. Publication compiles the reviewed .riffr source on the server, verifies its name, version, module hash, contract identity, and query dependencies against the application lock, then retains the module atomically through the shared authorized control plane. Publishing the same immutable module again is idempotent. Reusing a name and version for different content is a closed version conflict.

riffdb dev uses this same locked publication path before it binds the selected development role, so generated watches and contextual subscriptions are available when the readiness line is emitted.

A durable consumer is identified by the complete tuple of reactive module hash, operation name, canonical typed parameters, and consumer name. It is therefore not silently rebound when a module changes. Within one consumer, eligible events are leased in partition order with at-least-once delivery. An event is advanced only by an exact acknowledgement carrying the event ID, lease token, and current history incarnation. Negative acknowledgement schedules a bounded retry; lease expiry does the same during recovery. The tenth failed attempt is retained as a dead letter so later eligible events can continue without pretending the failed event was acknowledged.

Use the exact reactive module hash from the application lock or deployment state. Parameters use NAME=JSON_VALUE, so string values include JSON quotes:

riffdb event consume \
  --module-hash <64-hex-module-hash> \
  --operation TicketEvents \
  --parameter 'organization_id={"type":"uuid","value":"018f6f50-6f31-7d62-9a7e-4f8b913d2f11"}' \
  --consumer-name ticket-indexer \
  --batch-limit 4 \
  --lease-seconds 60

riffdb event ack \
  --module-hash <64-hex-module-hash> \
  --operation TicketEvents \
  --parameter 'organization_id={"type":"uuid","value":"018f6f50-6f31-7d62-9a7e-4f8b913d2f11"}' \
  --consumer-name ticket-indexer \
  --event-id <commit-sequence:event-ordinal> \
  --lease-token <64-hex-lease-token> \
  --history-incarnation <incarnation>

riffdb event status \
  --module-hash <64-hex-module-hash> \
  --operation TicketEvents \
  --parameter 'organization_id={"type":"uuid","value":"018f6f50-6f31-7d62-9a7e-4f8b913d2f11"}' \
  --consumer-name ticket-indexer

event nack accepts the same lease identity plus an optional retry delay. event seek is an administrative checkpoint move and requires separately granted seek authority. event retire permanently retires that exact consumer and releases its retention fence. An active consumer’s checkpoint prevents history pruning past the commit that still contains its first required event.

Backup and restore preserve consumer state but publish a new history incarnation. Outstanding pre-restore lease tokens then fail closed, and the consumer resumes from its restored checkpoint. Consumers in sibling named databases remain isolated.

Delivery boundary

The public surface includes immutable publication, durable pull and gRPC stream consumption, leases, acknowledgement, retry, dead-letter, recovery, seek, retire, status, and retention fencing. Live queries use compiler-derived invalidation, checked cursor reconnect, and the update protocol described in Live Named Queries. Generated Rust, Go, TypeScript, Python, MCP, CLI, and application-owned browser relay helpers adapt those same operations as described in Reactive Application Clients.

Contextual subscriptions combine one leased event with freshly authorized, same-snapshot hydration and currently authorized declared reactions. A normal event-consumer permission does not imply contextual, query, or command authority. MCP uses the same service boundary and its resource notifications contain no event or context payload. See Contextual Agent Subscriptions.

See Domain Events, Immutable Query Modules, and Application Source and Exact Lock.

Reactive application clients

Application Source V4 through V7 generate reactive operations for Rust, Go, TypeScript, Python, and MCP from the same exact reactive-module identity. The CLI exposes the same API-neutral service operations. Generated code owns operation names, module hashes, parameter types, event variants, and live-query result variants; application code supplies only values, a durable consumer name, and bounded delivery options.

Regenerate after reviewing the source and exact lock:

riffdb application generate --locked
riffdb application lock --check

Do not hand-copy a module hash or recreate an event or live-update union in application code. A changed generated reactive diff is an application API change and must be reviewed with the source and lock that produced it.

Durable event consumers

A consumer identity includes the selected database, immutable reactive-module hash, operation name, canonical parameters, and consumer name. Reuse the same consumer name and parameters to resume its checkpoint. Changing any identity component selects a different consumer; it does not rename or inherit the old one.

Delivery is at least once and partition ordered. For each delivery:

1. Validate and handle the generated closed event variant.
2. Commit any external side effect or idempotent RiffDB reaction.
3. Acknowledge with the exact delivery object or its retained lease evidence.
4. Negative-acknowledge with a bounded retry delay when work should be retried.

An event ID is stable, but its lease token is attempt-specific. A stale, expired, or already-resolved lease returns a closed mutation result and never grants authority by possession. Do not acknowledge before the work it protects is durable. Seek is a separately authorized administrative operation and is not implicitly granted to a normal consumer role.

The generated Rust, Go, and Python clients expose a typed delivery batch, next_<stream>, acknowledge, negative-acknowledge, seek, and status methods. Use the bounded next_<stream> pull when an empty batch is meaningful, such as scheduled or serverless work. Python’s generated method accepts the batch, in-flight, lease, and maximum-wait bounds as keyword arguments and defaults to a zero-wait pull. The stable Rust facade also exposes a checked streaming response for workers that keep one gRPC stream open. TypeScript and Python additionally expose generated async iterators; reconnecting an iterator uses the same durable consumer identity.

Contextual agent subscriptions

Generated contextual clients expose one operation-specific next, ack, nack, status, and reaction helper. A pull returns zero or one work item. The item contains the typed event, one authoritative context_head, bounded named-query hydrations, exact lease evidence, and only the declared reactions currently authorized for the caller.

Pass the returned work item to generated reaction and acknowledgement methods; do not reconstruct its token fields. RiffDB derives the reaction command’s idempotency input from the immutable subscription identity, event ID, command, and reaction name. Caller-provided idempotency input is replaced before command validation. A retry after an uncertain response therefore resolves the same outcome. Acknowledge only after the reaction outcome or intended external work is durable.

Context is recomputed on redelivery from one shared snapshot at or beyond the event sequence. It is not persisted in the consumer record. Available means the command is authorized at delivery time, not that its business preconditions will succeed. Reaction execution performs fresh authorization and validates the sealed token, live lease, restore incarnation, database, partition, principal, and target command.

Live named queries

Generated watch methods return the closed Snapshot, Patch, Reset, Checkpoint, and Terminal update vocabulary. Apply updates in delivery order:

• Replace local state with a Snapshot or Reset result.
• Apply every operation in one Patch atomically to the named result field.
• Persist the cursor after the corresponding state update is durable.
• Advance only cursor and frontier on Checkpoint.
• Clear all retained result state and stop on Terminal.

Cursors are opaque evidence, not authority. Rust persists their bytes without interpreting them. Go, TypeScript, Python, CLI, and MCP use standard padded Base64. Reconnect with the last applied cursor and the same exact operation and parameters. RiffDB reauthorizes the watch and returns a complete snapshot or a typed reset when continuation evidence is no longer valid.

The TypeScript generator also emits a framework-neutral live store. Its generated SSE relay accepts an application-owned authorized() callback and an already-authorized update iterator. Run that relay in the application server. Browser JavaScript receives application-authenticated SSE data but never a RiffDB endpoint credential, capability, lease token, or direct database connection.

CLI operations

Use the exact module hash from the application lock:

riffdb event consume \
  --module-hash <64-hex-module-hash> \
  --operation TicketEvents \
  --parameter 'organization_id={"type":"uuid","value":"018f6f50-6f31-7d62-9a7e-4f8b913d2f11"}' \
  --consumer-name ticket-worker \
  --batch-limit 4 \
  --lease-seconds 60

riffdb event status \
  --module-hash <64-hex-module-hash> \
  --operation TicketEvents \
  --parameter 'organization_id={"type":"uuid","value":"018f6f50-6f31-7d62-9a7e-4f8b913d2f11"}' \
  --consumer-name ticket-worker

riffdb query watch \
  --module-hash <64-hex-module-hash> \
  --parameter 'organization_id={"type":"uuid","value":"018f6f50-6f31-7d62-9a7e-4f8b913d2f11"}' \
  TicketWatch

event ack, event nack, and event seek use the exact identity printed by the consume or status result. query watch returns one update and cursor per invocation so shell programs can durably apply the update before reconnecting. See the CLI reference for every bound and option.

Contextual workers use the same generated module, operation, parameters, and consumer identity:

riffdb contextual next \
  --module-hash <64-hex-module-hash> \
  --operation TriageTicket \
  --parameter 'organization_id={"type":"uuid","value":"018f6f50-6f31-7d62-9a7e-4f8b913d2f11"}' \
  --consumer-name triage-worker \
  --wait-nanos 30000000000

riffdb contextual react \
  --module-hash <64-hex-module-hash> \
  --operation TriageTicket \
  --parameter 'organization_id={"type":"uuid","value":"018f6f50-6f31-7d62-9a7e-4f8b913d2f11"}' \
  --consumer-name triage-worker \
  --reaction comment \
  --causation-token <token-from-next> \
  --command-name CreateComment \
  --input comment.json \
  --expected-version 1

contextual ack and contextual nack require the exact event ID, lease token, and history incarnation returned by contextual next; contextual status uses only the stable consumer identity. Prefer the generated SDK helpers in application workers because they carry this evidence without reconstructing it. The raw CLI is intended for process integrations and diagnosis, and still uses the same authorization and application-service path as generated clients.

MCP operations and wakeups

MCP exposes the fixed underscore-only tools riffdb_event_next, riffdb_event_ack, riffdb_event_nack, riffdb_event_seek, riffdb_event_status, and riffdb_query_watch. Generated application MCP artifacts additionally contain exact operation-specific underscore-only names and JSON Schemas. Every call resolves current authorization and uses the same application service as gRPC, CLI, and the SDKs.

Contextual roles additionally expose riffdb_contextual_next, riffdb_contextual_ack, riffdb_contextual_nack, riffdb_contextual_status, and riffdb_contextual_react, plus generated operation-specific tools. The reaction call forwards the causation token from the work item; possession of that token never bypasses current policy or lease validation.

An MCP notifications/resources/updated message is only a payload-free wakeup hint. Subscribe to the exact riffdb://reactive/wakeup URI. The resource is discoverable only when the role has at least one watch, event-consumer, or contextual-consumer permission that does not require approval. Its body is an opaque generation and must not be interpreted as a commit sequence or work count. A change may be caused by unrelated application work, so clients should coalesce refreshes. Fetch current work immediately after subscribing because a subscription reports later changes and does not assert that the current queue is empty.

The notification parameters contain one authorized resource URI and no event, parameters, context, field values, or lease evidence. The client must retrieve work with riffdb_event_next or refresh a watch with riffdb_query_watch; reading a notification is never an acknowledgement. Notifications may coalesce, so correctness must depend on the durable consumer checkpoint or live cursor rather than notification count.

A successful resources/subscribe retains only the bounded fingerprint from its fresh authorized resource read as the observer baseline. This is the subscription linearization point: a later resource change is compared against that baseline and cannot be absorbed by the first five-second observer pass.

See Reactive Modules, Live Named Queries, and MCP for Agents.

Live named queries

A live named query watches one exact watch operation from a published reactive module. It is not an ad-hoc query, a raw commit subscription, or a storage change feed. The watch resolves an immutable query module and plan, executes that bounded RiffQL query in one authoritative snapshot, and then uses authoritative commits only as invalidation signals.

Define a watch

The query must already be part of an immutable query module. A reactive module binds its parameters and update mode:

reactive TicketActivity version 1 {
  watch OpenTickets(
    $organization_id: Ticket.organization_id,
    $project_id: Project.project_id
  ) query OpenTicketsPage updates patch;
}

Use updates patch only when the query has one top-level many result and explicitly selects every primary-key field. Otherwise use updates reset. Paginated queries with an after cursor are not watchable because they expose only a partial result.

The caller needs the exact WatchNamedQuery permission for the contract lineage, reactive module hash, and operation name. Query parameters and field visibility remain subject to the same application-query policy used by normal named RiffQL execution. A cursor does not grant authority.

Update protocol

QueryService.WatchNamedQuery is a server-streaming gRPC operation. Every stream starts with one of these complete views:

• Snapshot: a fresh authorized result, frontier, and resume cursor.
• Reset: a fresh complete result when supplied cursor evidence cannot be continued, with a typed reason.

Later messages use the closed update vocabulary:

• Patch: ordered insert, remove, replace, or move operations for one keyed top-level collection.
• Reset: a complete result after an outcome change or a diff that exceeds the patch bound.
• Checkpoint: the result is unchanged, but the authoritative frontier and cursor advanced.
• Terminal: the watch is closed and carries no result payload.

A patch key contains only the complete explicitly selected public key fields. Internal entity keys, stable field IDs, commit change records, and unselected fields never cross the public boundary.

Frontiers and reconnect

The initial query reads at application head S. RiffDB then subscribes to authoritative commits strictly after S; durable catch-up closes the race between snapshot completion and subscription registration. Notifications are wakeup hints. Every relevant change is re-read through authoritative storage and the bounded query is re-executed.

Retain the cursor from the last applied update. On reconnect, submit the same reactive module hash, operation name, canonical parameters, and that cursor. RiffDB freshly authenticates and authorizes the request and returns a complete snapshot or typed reset. A cursor binds:

• history incarnation;
• exact contract, reactive operation, query module, and query plan;
• canonical parameter hash and partition set;
• application frontier and expiry.

Restore, expiry, or identity drift therefore cannot become an ambiguous continuation. Reconnecting from an older delivered cursor is safe because the new watch executes a fresh authoritative snapshot.

Limits and closure

One database admits at most 128 live watches. A watch lives for at most 15 minutes, retains no more than the bounded stream state, emits at most 500 patch operations in four MiB, and uses cursors no larger than 4,096 bytes. Burst processing may skip transient intermediate views, but the delivered state must converge to a fresh execution of the named query.

Typed terminal reasons include authorization change, buffer pressure, lifetime expiry, service unavailability, integrity failure, and definition change. Authorization is revalidated before delivery. Closing for revocation discards the service-owned retained result before releasing the subscription.

Rust transport client

The current low-level Rust transport accepts a checked riffdb_proto::v1::WatchNamedQueryRequest through RiffDbClient::watch_named_query. Read messages with LiveQueryUpdateStream::message; strict public decoding rejects unknown, missing, oversized, noncanonical, or internally identified fields.

Generated application-specific Rust, Go, TypeScript, Python, CLI, MCP, and browser relay surfaces adapt this same checked service boundary. Every generated client persists the last applied opaque cursor for reconnect and clears retained result state when Terminal is received. See Reactive Application Clients for the operation-specific surfaces and relay boundary.

See Reactive Modules, RiffQL Planning, and Consistency and Recovery.

Contextual agent subscriptions

A contextual subscription turns a durable domain event into bounded agent work without running an agent, connector, or callback inside the originating transaction. Its compiled definition binds one exact event stream, up to 16 named hydration queries, up to 32 declared command reactions, canonical parameters, and fixed delivery limits.

Work item semantics

One pull leases at most one event. RiffDB executes every hydration in one shared read snapshot whose application head is at least the event’s commit sequence. The returned item contains:

• the selected typed event and stable event ID;
• attempt-specific lease token, attempt, expiry, and history incarnation;
• one shared context_head and bounded named hydration results;
• only reactions that are both declared and currently authorized; and
• an opaque server-sealed causation token for each available reaction.

Hydrated values are never stored in consumer state. Redelivery recomputes them under current authorization, so an agent must treat each returned item as a fresh view. Available reactions can still return declared business-rejection outcomes when command preconditions do not hold.

Reaction safety

Reaction execution uses the ordinary application command service and commit coordinator. The contextual permission is not command authority. Before a new commit, RiffDB reauthorizes the target command and verifies the token’s database, history incarnation, reactive identity, canonical parameter and consumer identity, event and lease attempt, principal and capability revision, target command, and expiry against durable lease truth.

For a protected role, lease truth alone is insufficient. Immediately before a reaction command begins, the storage safe point also reloads the exact current capability revision, the trigger event’s compiler-owned policy anchor, its current source row, and bounded relationship evidence in one authoritative snapshot. A revoked capability, deleted or newly hidden row, changed role, old unanchored event, or stale lease all produce the same authorization denial and no new command admission. The nested command then performs its own ordinary transaction-current authorization and commit verification.

RiffDB derives the direct UUID or bounded-string idempotency input from the reactive module hash, contextual operation hash, event ID, command ID, and declared reaction name. It replaces caller input at the service boundary. The command’s mutations, outcome, events, causing event, inherited root request, provenance, and commit record are then atomic.

The reaction operation and its nested command are separate audited service operations. RiffDB derives a deterministic child request ID for the nested command so their audit lifecycles cannot collide, while the durable causation record continues to carry the original root request ID. Applications must not construct or substitute either identity.

If the reaction commits but its response or the following acknowledgement is lost, the event is redelivered. Repeating the reaction resolves the persisted outcome and cannot duplicate authoritative state. An expired token may resolve that exact prior outcome but cannot authorize a new commit.

Worker order

1. Pull one contextual work item.
2. Inspect its typed event, fresh hydration, and available reactions.
3. Execute one generated reaction helper or perform separately idempotent work.
4. Persist the successful result or external effect.
5. Acknowledge with the exact work item evidence.
6. Negative-acknowledge with a bounded delay when retry is appropriate.

Never acknowledge before protected work is durable. Never reconstruct a lease or causation token. On restore, the new history incarnation invalidates old lease evidence and the durable consumer resumes from its restored checkpoint.

Public surfaces

The five API-neutral operations are consume, acknowledge, negative acknowledge, status, and execute reaction. gRPC exposes them as bounded unary methods. Generated Rust, Go, TypeScript, Python, and MCP artifacts bind the exact reactive identity. MCP fixed tools use underscore-only names:

• riffdb_contextual_next
• riffdb_contextual_ack
• riffdb_contextual_nack
• riffdb_contextual_status
• riffdb_contextual_react

MCP uses the same authorization and service path as every other transport. A resource update is only a coalescible, payload-free wakeup; durable consumer truth remains the source of correctness.

The singleton database-wide reactive wakeup is unavailable to protected roles because it would expose hidden commit timing without a subscription identity. Protected agents should long-poll riffdb_contextual_next; reconnecting or a spurious host wakeup always returns to that durable, policy-revalidated path.

See Reactive Modules, Reactive Application Clients, and MCP for Agents.

Safe application profiles

RiffDB deliberately does not have one development credential that can do everything. Choose the smallest interaction profile:

# Default: exact generated commands and named queries only
cargo run -p riffdb-cli -- dev --role ticketdesk-application --seed

# Agent runtime: a separate exact named-operation role
cargo run -p riffdb-cli -- dev --role ticketdesk-agent --seed

# Low-level diagnosis: raw entity/index access, no application execution
cargo run -p riffdb-cli -- dev --role ticketdesk-kernel

Each invocation issues one protected credential for only the selected profile. The kernel profile cannot seed or run the application acceptance workload. The stable profile cannot check, explain, or execute ad-hoc source and cannot call raw entity/index operations. The agent profile is also compiled from its symbolic manifest allowlist; it does not gain ad-hoc RiffQL merely because its principal is an agent.

Why the split exists

An exact named query permission binds contract lineage, immutable query-module hash, and query name. The compiler-derived plan, partition, fields, indexes, result shape, and whole-request cost remain private execution requirements. They cannot be reused to construct GetEntity or ScanIndex.

A kernel permission is the opposite: it authorizes one compatible low-level operation but does not authorize RiffQL. This preserves the kernel protocol for conformance and diagnosis without making it the application programming model.

MCP discovery is filtered from the same current capability. Application and agent catalogs omit raw entity/index and ad-hoc-source tools. Kernel catalogs do not silently acquire named commands or queries.

Moving an existing development deployment

There is no automatic widening or reinterpretation:

1. Deploy the current contract and query module.
2. Issue a replacement application, agent, or kernel credential.
3. Verify the exact named application workload with the new credential.
4. Revoke the old combined credential.
5. Keep kernel credentials out of application configuration.

Do not add ReadEntity or ScanIndex to make a named query pass. A named-query denial means the exact module/query authority, field visibility, partition scope, current capability revision, or whole-query budget is wrong.

Evidence and limits

The original independent TicketDesk build rated the storage-shaped application path 4/10 and identified numeric IDs, manual masks, N+1 reads, and multi-snapshot page composition. Its follow-up build stayed on symbolic commands and named RiffQL; the one junction-to-entity collection gap it found was closed by bounded dependent-key batch queries. The checked TicketDesk application now uses one request per page and an application-only client facade.

This is not a claim that RiffDB infers undeclared business rules or controls external side effects. It proves supported, declared RiffDB semantics. Use durable events/outbox intent when an external effect must be coupled to a command, and declare every relationship or uniqueness rule that RiffDB must enforce.

Run:

./scripts/safety-by-construction-acceptance
./scripts/riffdb-dev-acceptance
./scripts/check-ticketdesk-symbolic-boundary

Compiled framework profiles

RiffDB does not implement an external framework’s generic CRUD store or transaction callback. An integration repository resolves the framework’s configured models and plugins at generation time, emits an ordinary RiffDB contract with named commands and queries, and dispatches only to that compiled surface. Unknown models, hooks, plugins, and transaction shapes fail during generation instead of degrading into sequential writes at runtime.

The adapter remains application code. No framework dependency, identity protocol, session logic, or privileged storage path enters RiffDB. A framework “transaction” is supported only when one generated command describes its complete atomic graph. Bounded collection commands cover finite repeated graphs; workflow commands cover revision-checked session transitions. An adapter must never run several commands and report that they were one atomic transaction.

The framework-agnostic acceptance contract at fixtures/adapters/framework-profile/riffdb/contract.riff demonstrates the generic capabilities:

• transactional unique identities and provider/account links;
• one bounded atomic user/account/session create graph;
• compiler-owned session initialization plus exact-revision refresh/revoke;
• secret-classified session and verification-token digests;
• consume-once verification with transaction-time expiry refusal; and
• server-observed transaction time and UUID values.

It is a workload shape, not a bundled framework adapter.

Repository ownership boundary

Framework-specific contracts, profiles, generated SDKs, route hosts, and route- level compatibility tests belong to the integration repository that owns the adapter. That repository pins an immutable RiffDB release and records its framework version, enabled plugin/model surface, generated identities, and acceptance observations. RiffDB does not copy those artifacts into its own source tree.

The RiffDB repository retains framework-neutral fixtures for the compiler, providers, authorization, generated transports, and cross-language semantics. For exact result sets, the generic Document corpus proves contains, starts- with, ends-with, typed optional filtering, deterministic orders, bounded direct offset, and complete exact total. An adapter can depend on that public capability, but neither its route vocabulary nor its userland policy becomes a RiffDB feature.

External adapters can pin fixtures/riffql/operational-query-capability-v5.json. V5 preserves the immutable V1 through V4 receipts and extends the neutral operational-access corpus with binary UTF-8 interval/complement alongside shared equality, membership, prefix, canonical range/complement, nullable, cursor, policy, and dependent-key shapes. It binds the exact application lock and generated Rust, Go, TypeScript, Python, and MCP artifacts.

V5 also binds one value-free external tuple-changelog receipt demonstrating that one bytewise text-key index can supply strict lower/upper selection and order, and that its exact package may declare only a generated Go surface. That receipt contains hashes and typed verification classes only; it contains no external schema, route, adapter, generated profile, or stored value. It explicitly does not claim external runtime, full external application, or framework conformance.

The distinct terminal fixtures/riffql/wp775-external-binary-text-interval-execution-v1.json receipt now proves one narrow owning-repository runtime case: two invocations of the same generated strict-lower/strict-upper query returned original logical strings in bytewise ascending order, and the consumer reused a returned string as its own next lower token. The run used one existing index, one partition, two bounded result pages, and no client filter, sort, page walk, duplicate index, or token conversion. This is not evidence that a framework route, the complete adapter surface, or a full external application conforms; those wider claims still require their own owning-repository acceptance.

Better Auth support boundary

The Better Auth integration supports only the exact materialized profile its owning repository generates, configures, and tests. The framework’s generic facade is a profile-bound dispatcher to exact generated commands and named queries; it is not generic RiffDB CRUD or a public transaction surface. The profile fixes its models, fields, plugins, schemas, atomic command shapes, queries, roles, and generated identities before deployment.

Additional fields or plugins require generation and review of a new materialized profile, followed by its generated operations, installation review, and conformance evidence. Unsupported configuration fails while the profile is generated or configured. Unknown runtime calls refuse instead of synthesizing an operation, omitting a field, falling back to a generic store, or splitting one framework transaction across several RiffDB commands. The RiffDB-owned neutral fixture proves underlying symbolic capabilities but does not widen the supported external profile or establish a complete Better Auth upstream-suite claim.

The complete ten-record classification and the separately scoped Better Auth admin and lifecycle evidence are listed in Adapter evidence classes.

Keyless upstream retries

When an upstream framework supplies no idempotency key, the adapter owns the logical request identity:

1. Mint one bounded RiffDB command idempotency value before the first dispatch.
2. Freeze the complete generated command input containing that value.
3. Reuse that exact input for every transport retry and uncertainty-resolution attempt until a terminal declared outcome is known.
4. Allow the driver to mint a fresh transport request ID for each attempt; the transport ID is not the command’s idempotency identity.

If retry must survive adapter process loss, retain the mapping from a stable upstream operation identity to the minted command identity before dispatch. If the framework provides no stable operation identity and the adapter did not retain one, it cannot honestly claim process-loss idempotency and must surface that limitation. Never mint a new key after an uncertain result, and never derive the key from a plaintext token or other secret.

The Rust driver test adapter_minted_idempotency_survives_driver_retries_with_fresh_transport_ids proves that bounded retries preserve the byte-identical command input while using fresh transport request IDs. On the admission side, equal_key_commands_commit_once_and_replay_exactly proves the property the minted identity rides: equal-identity submissions commit exactly once, the replay returns the byte-equal stored outcome under one commit sequence and one provenance record, and different input under the same identity is refused as an input mismatch.

Acceptance evidence

Run the focused profile gate:

./scripts/adapter-framework-profile-acceptance

It compiles and round-trips the profile, exercises the atomic graph and token/ workflow refusal paths, runs the real-redb concurrent uniqueness schedule, and checks retry identity preservation. The full deployable-alpha gate invokes the same focused phase.

Adapter evidence classes

RiffDB classifies each adapter-related claim at the strength of its retained evidence. The canonical machine-readable inventory is fixtures/adapters/adapter-evidence-v1.json; the records below are a handbook view of that closed ADR-0201 inventory. Classification metadata is not execution evidence and cannot replace an installation, conformance, export, restore, or endurance observation.

Only alpha.openfga.general has complete_upstream_suite: true. General-alpha and capability-specific records remain separate: a general matrix cannot erase a narrower external or live receipt, and a partial receipt cannot establish complete framework compatibility.

General-alpha records

The MLflow and Woodpecker rows prove that RiffDB’s bounded symbolic interfaces can express and preserve their declared data and workflow shapes. They do not show an upstream MLflow or Woodpecker release operating against RiffDB. Better Auth’s row covers the exact generated materialized-profile matrix, not the framework’s complete upstream suite.

Capability-specific and post-alpha records

These narrower records keep their own scope and custody. In particular, the external MLflow search receipts are not collapsed into the RiffDB-authored general matrix, the 100-tag MLflow Run remains a RiffDB-custodied live loopback, and Payload remains a post-alpha language-expressiveness shape.

Better Auth support boundary

An external Better Auth integration may present the framework’s generic facade only as a profile-bound dispatcher to the exact generated commands and named queries selected for its materialized profile. Models, fields, plugins, schemas, and atomic transaction shapes are fixed while that profile is generated and configured. Additional fields or plugins require a newly generated and reviewed profile plus its generated operations and conformance evidence. Unsupported configuration and unknown calls refuse; there is no generic RiffDB CRUD, dynamic schema, transaction callback, or runtime fallback.

Repository check

Run the bounded offline guard after changing any classification or release-gate wording:

./scripts/check-adapter-evidence-claims
./scripts/check-adapter-evidence-claims --self-test

The first command checks the exact ten records, source locators, normative SPEC claims, handbook language, and gate wiring. The self-test proves that missing, duplicate, reordered, widened, unknown, collapsed, or promoted claims and the two unsafe legacy wordings are rejected.

Change note

Package: WP-724

Tier: surface

Behavior added or changed: the release gate and handbook now report each of the ten ADR-0201 claims at its exact scope, class, custody, and upstream-suite strength; the checked JSON inventory is the repository source of truth.

Checks run: package acceptance, the exact and adversarial claim guards, the four-subject/four-language adapter conformance gate, and local sibling compile checks (recorded before the checker was retired) against riffdb-openfga, riffdb-better-auth, and riffdb-mlflow with ./scripts/downstream-adapter-check --repo-root /home/user/dev. Those local workstation checks are not pinned CI evidence.

Compatibility: classification and wording only. Runtime behavior, public and durable formats, protocols, storage, authorization, transactions, evidence bytes and results, thresholds, and retained receipts are unchanged.

Hazards and follow-ups: any changed record, locator, requirement wording, or release claim requires human review. WP-578 still owns the separate 72-hour execution and classification metadata cannot satisfy it. WP-723 assigns compatibility checks to the adapter repositories. They pin RiffDB revisions and track upstream changes independently; core CI does not require those sample applications.

Authoring diagnostics

Every local application compiler path returns the same bounded riffdb-authoring-diagnostic/v1 semantics. Human CLI output and --output json are renderings of one checked value; builder MCP uses the JSON form unchanged.

Each diagnostic contains:

• a stable stage and code;
• a workspace-relative source path and half-open UTF-8 byte span when the originating compiler has one;
• a bounded symbolic path, never a submitted runtime value;
• one closed cause and closed corrective-action codes;
• what happened to local files; and
• whether to correct source, review and write a lock, retry the same inputs, or contact an operator.

Example:

RDB-QP003 [query_plan] query has no bounded index access plan
  --> riffdb/queries/list_items.riffq:104..179
  symbol: ListItems.Item
  cause: missing_index
  fixes: add_index
  files: no_files_changed; retry: correct_source

The byte range points into the caller-owned local file. RiffDB does not copy source snippets, literals, command inputs, credentials, application values, hidden schema, engine errors, or arbitrary library prose into the diagnostic, logs, evidence, or MCP output.

The main correction codes are deliberately executable concepts:

• use_language_reference: correct syntax using the bundled grammar.
• correct_symbol / correct_type: resolve the named contract operation or make exact types agree.
• supply_partition_route: add the complete same-partition route.
• prove_relationship_target: read the complete relationship target before the mutable binding, declare its missing-target outcome, and reuse the same target-key input expressions when storing the relationship fields.
• add_index: add the bounded index identified by query explain.
• add_bound: declare explicit positive cardinality/work bounds.
• reduce_input: reduce a complete worst-case input or query-work bound. For RDB-AR007, replace plain Limit parameters on a multi-collection page with fixed take limits or reviewed Limit<MAX> declarations whose aggregate index-scan bound fits the role.
• narrow_role: remove or correct unsafe symbolic authority.
• write_lock: review the symbolic/authority diff, then run riffdb application lock --write.
• generate_locked: restore compiler-owned output with riffdb application generate --locked.

RDB-C017 reports supply_partition_route and covers one unsafe shape: bindings whose route expressions are not provably identical. A command MAY create and mutate records owned by different aggregates, provided every binding derives the same partition route; the command’s conflict ownership is then the union of the conflict keys those bindings derive, so each aggregate keeps its own key. Start from the command’s complete mutation set and make every binding derive one route. A write that genuinely spans partitions must still be split into independently idempotent commands, because RiffDB does not turn it into an implicit distributed transaction.

RDB-C024 rejects a relationship change whose exact target proof is missing or has expression drift. The complete target read must appear before the relevant create or mutate, must declare what happens when the target is absent, and must use the same key input expressions as the stored relationship fields. For example, after read Author(site_id, author_id) as author, store post.author_id = author_id. In the contract grammar, storing post.author_id = author.author_id is value-equivalent after the read but is not the same structural expression and therefore does not satisfy the proof. The database does not infer that equivalence or weaken the declared relationship.

application check always reports no_files_changed. A staged write failure reports whether staging was discarded, the previous generation remains accepted, or generated files may be partial while the exact lock was not published. No diagnostic turns partial output into an accepted application.

During iterative source editing, application check --source-only compiles the complete author-owned contract, queries, and roles without comparing the old compiler-owned generation. Success explicitly says no lock or generated artifact was checked. This mode never writes, deploys, or grants authority.

When the default lock exists, exact application check verifies the source, pinned contract bundle, exact lock, and every generated artifact. It cannot report a source-only success over a stale lock. With no lock present, success explicitly says that only symbolic sources compiled and that no lock or generated artifact was checked.

Source, lock, or generated-artifact drift is always the structured RDB-AL008 diagnostic with identity_drift and write_lock. This includes an interrupted or substituted generated file; the CLI, JSON output, and builder MCP never replace it with unstructured process text. Review the symbolic diff before writing a new lock. If source was not intentionally changed, restore it and regenerate from the existing exact lock instead.

RDB-AR003 describes a disagreement between a role’s exact manifest contract and the contract bundle used to derive its grant; it is not a report about an older capability still stored by the server. With lock V3, role operations use the pinned parent-aware bundle, so an exact application check and an RDB-AR003 from the same unchanged workspace would be an implementation defect, not a reason to rewrite the same lock repeatedly. A deliberately widened role instead produces a new role definition hash and requires --replace-role-credential; replacement diagnostics include the retained and locked role hashes in JSON when both are known.

RDB-AR011 reports a compiled command or named query that is not granted to any role in a non-empty riffdb.application.json manifest, together with a fix hint showing how to add the operation to roles[].commands or roles[].queries so generated SDK role clients and MCP roles can call it.

Use JSON when an agent or editor needs deterministic fields:

riffdb --output json application check

Malformed, unknown, oversized, or path-escaping diagnostic inputs fail closed; they cannot introduce a new fix, retry instruction, source path, or prose field.

Contract and RiffQL editor tooling

The signed RiffDB CLI includes a compiler-backed Language Server Protocol service:

riffdb lsp

The command speaks LSP over standard input and output. Editors should launch it directly with no endpoint, credential, or database configuration. It supports full-document synchronization for .riff and .riffq files, compiler diagnostics, hover types, go-to-definition, and schema-fed completion.

Diagnostics are not an editor approximation. Contract diagnostics pass through the same compiler and authoring-diagnostic conversion used by riffdb push; RiffQL diagnostics use the authoritative RiffQL parser and, when the local contract is available, the query-module compiler. The primary code, safe message, and UTF-8 source span therefore agree with command-line checks. LSP positions are translated to UTF-16 only at the protocol boundary.

The service is deliberately local and authority-free. It does not load a credential, connect to a daemon, mutate files, deploy contracts, or run an application operation. It accepts at most 1 MiB per frame or source document, 32 open documents, 32 compiler diagnostics, and 256 schema symbols. Malformed frames fail closed; malformed JSON receives the standard bounded JSON-RPC parse error. Full-document changes are the only accepted synchronization shape.

Highlighting grammars

CLI packages also carry tree-sitter-riff and tree-sitter-riffql. These grammars provide highlighting for .riff and .riffq in Tree-sitter-capable editors and source forges. They are intentionally non-authoritative: a highlighter accepting a token sequence never makes that sequence a valid RiffDB contract or query.

Their keyword inventories are generated from the first-party lexer/parser. The checked-in highlight snapshot corpus covers the bounded public contract and RiffQL handbook examples. Run:

./scripts/generate-editor-grammars --check

to verify both the inventories and snapshots. The canonical package copies live under crates/riffdb-cli/assets/editor/.

Installed asset locations are deterministic:

VS Code and other editors

clients/editor/vscode-riffdb is a thin extension scaffold. It registers the two file types and launches riffdb lsp; it contains no parser, compiler, authorization logic, credentials, or application transport.

Other editors should use the same launch contract. Configure the language server command as riffdb lsp, associate .riff and .riffq, and install the corresponding Tree-sitter grammar if the editor supports Tree-sitter-based highlighting.

The alpha grammar packages include JavaScript grammar sources, highlight queries, exact keyword inventories, and Tree-sitter metadata. Generated native parser bindings are editor-distribution concerns and are not linked into the RiffDB runtime.

Application errors

RiffDB application operations fail with one versioned semantic error object. The object is the same checked domain value on gRPC, Rust, Go, TypeScript, Python, CLI, and MCP. Presentation casing may follow the host language, but codes, categories, recovery actions, fixes, operation identity, and optional context have identical meaning.

Application code should branch on code, recovery_action, or fixes. It must not parse the human message.

{
  "type": "application",
  "code": "RDB-AUTH-0214",
  "message": "application operation is not authorized",
  "category": "authorization",
  "recovery_action": "obtain_permission",
  "operation": "ExecuteQuery",
  "contract_lineage": "ticketdesk",
  "contract_version": "18",
  "operation_symbol": "TicketPage",
  "fixes": ["bind_application_role"],
  "trace_id": "019bf6aa-a640-7de6-89c9-8a7f70bbbd23"
}

The message and remediation text are selected from a closed registry. The complete v1 registry is frozen in fixtures/application-errors/registry-v1.tsv.

RDB-AUTH-0215 means the presented credential reciprocally matched a retained capability that has been irreversibly revoked. Generated clients can therefore distinguish replacement/role-rebinding work from an ordinary RDB-AUTH-0214 policy denial. Unknown, malformed, expired, or boundary-mismatched credentials remain the generic authentication/authorization class; the revoked code is not an oracle for credentials that did not establish an exact retained match.

Exact result sets and projected vector reads use four projection codes. RDB-PROJECTION-0101 reports that the declared providers cannot prove one common epoch; RDB-PROJECTION-0102 reports that a requested snapshot has retired; and RDB-PROJECTION-0103 reports that no retained snapshot satisfies the requested freshness floor. RDB-PROJECTION-0104 reports that a retained nearest-query artifact predates the required compiler-owned projected source; regenerate, deploy, and pin the current query module. The first and third permit a bounded retry. The retired- snapshot response requires a new first-page/current-snapshot request. None of these codes authorizes client-side filtering, page walking, or weaker consistency.

Safety boundary

An application error may contain:

• the closed operation, code, category, recovery action, and fix codes;
• exact contract lineage and version already visible to the caller;
• a query, command, module, or role symbol the caller submitted or was authorized to discover;
• a bounded path of similarly visible symbols;
• a bounded source span into source the caller submitted;
• the request ID as an opaque trace ID; and
• an opaque incident ID for operator correlation.

It can never contain submitted application values, credentials, principals, tenant secrets, raw entity/field/index IDs, capability masks, storage errors, server source locations, arbitrary server prose, or a symbol inferred from a context-free infrastructure failure.

This restriction is intentional. For example, a denial of TicketPage may name TicketPage because the caller invoked it. It may name Ticket.description only when that symbol is already safe to release after authorization filtering. RiffDB omits unavailable context rather than fabricating a more detailed but potentially disclosive explanation.

Wire compatibility

riffdb.app.v1 RPCs carry riffdb.app.v1.ApplicationError directly as their bounded gRPC details payload. Application clients reject a kernel error payload, unknown tag, unknown enum, inconsistent redundant field, oversized context, status-code mismatch, or message mismatch.

Kernel and administrative RPCs continue to carry the existing riffdb.v1.PublicError bytes unchanged. This preserves low-level protocol compatibility while making the normal application surface symbolic and actionable.

A matched revoked credential receives a structured refusal on both boundaries. Symbolic application RPCs report RDB-AUTH-0215. Kernel command RPCs encode the existing authorization-denied PublicError; generated command facades therefore report RDB-AUTH-0214. The kernel encoding does not retain the application-only revoked code. Clients still reject absent or malformed error details rather than infer a refusal from the status message.

The sole exception is emergency internal containment when the server cannot obtain a real incident ID. That response remains the accepted details-free internal error; RiffDB never invents a trace or incident identity.

Retry behavior

Recovery actions are guidance constrained by the operation:

• correct_request: change the rejected input before another submission;
• retry: use an explicit bounded caller retry policy;
• resolve_with_same_idempotency_key: never invent a new mutation identity;
• obtain_permission: bind the declared symbolic application role;
• refresh_contract: refresh exact contract/module metadata;
• contact_operator: report only the opaque trace and incident IDs; and
• none: do not automatically resubmit.

Generated command helpers retain RiffDB’s uncertainty rules. Translating a kernel command failure into application context does not make an unsafe retry safe.

Resumable command batches

riffdb command batch is a bounded client-side scheduler for ordinary symbolic commands. It is not a bulk-write API and it does not create a transaction covering the input collection.

Each non-empty JSONL line is the complete named-command input. The selected idempotency field must be a non-empty string and must be unique within the file:

{"idempotency_key":"seed:ticket:1","organization_id":{"$uuid":"01900000-0000-7000-8000-000000000001"},"status":{"$enum":"Open"},"title":"Cannot sign in"}
{"idempotency_key":"seed:ticket:2","organization_id":{"$uuid":"01900000-0000-7000-8000-000000000001"},"status":{"$enum":"Closed"},"title":"Export is slow"}

For riffdb dev --seed, RiffDB selects the command’s compiler-declared idempotency input automatically. A command may call it request_id, request_key, or another valid symbolic name; seed files use that exact name. riffdb command batch --application <manifest> uses the same compiled lookup, and rejects an explicit --idempotency-field that disagrees with the command. Without --application, the standalone operation retains its explicit option and defaults an omitted value to idempotency_key for compatibility.

Run or resume it with:

riffdb command batch CreateOrganization fixtures/batches/01-CreateOrganization.jsonl \
  --checkpoint .riffdb/create-organization.batch.json \
  --concurrency 8 \
  --progress

The application boundary deliberately has these semantics:

• one exact command name and optional exact contract version per input file;
• one separately authenticated public command request per line;
• one stable idempotency key, typed outcome, provenance record, and commit decision per item;
• at most 32 in-flight items, 4,096 items, 64 MiB of source, and an 8 MiB command frame per line (with the decoded input still limited to 4 MiB and the exact compiler-proved command maximum);
• no collection atomicity, generic entity mutation, storage batch, or authorization shortcut;
• a checksummed checkpoint containing only source/item digests and safe public results, never command inputs, capability tokens, or idempotency keys;
• automatic resume only when the exact source bytes, command identity, expected contract version, checkpoint checksum, and every completed item digest still agree.

The client parses and validates the entire bounded source before submitting the first item. Malformed input, duplicate idempotency keys, an invalid checkpoint, or a changed source therefore fails closed without creating partial work. JSON strings remain strings even when they resemble another scalar type. UUIDs use {"$uuid":"canonical-uuid"} and enum variants use {"$enum":"VariantName"}. RiffDB never guesses a type from string contents, and neither form contains a compiler-allocated numeric ID.

Exact decimals use a string tag, never a JSON floating-point number:

{"unit_price":{"$decimal":"12.34"},"total":{"$money":{"currency":"USD","amount":"12.34"}}}

The decimal text accepts canonical base-10 notation with an optional leading minus and optional fractional digits. Exponents, a leading plus, leading zeroes such as 01.25, and a trailing decimal point are rejected. The selected command schema supplies the declared precision; the text supplies the exact coefficient and scale. The older explicit coefficient_twos_complement/scale object remains accepted for compatible machine-generated inputs.

Successful and declared business outcomes are terminal per item. Add --error-outcome OutcomeName to classify a durable declared outcome as a recorded item failure for import policy purposes. Authorization, contract, validation, and idempotency-reuse failures are also terminal and resumable. Transport uncertainty, cancellation, deadlines, and storage unavailability remain pending; a later invocation resubmits the identical command input and key through normal idempotency recovery. Response-size rejection is a terminal recorded item failure under the CLI path (one ordinary Execute per line).

When any item is rejected, --output json includes at most 16 rejected_items, each containing only its one-based ordinal, optional symbolic outcome, public error_code, and public error_message. rejected_items_truncated reports how many additional rejections were omitted. The summary never copies command input, idempotency keys, credentials, or business values. Use the ordinal to correct the corresponding JSONL line, then start a reviewed new batch or resume only items still marked pending.

The checkpoint is also the stable receipt. To avoid turning one small seed into hundreds of checkpoint fsync operations, the CLI replaces it atomically after each bounded wave of 16 observed terminal results and once more before a normal or interrupted return. A process kill may therefore lose receipt knowledge for at most one transport wave, never the server’s idempotency identity or durable outcome. Resume replays those same command inputs and keys, so committed items recover their stored results without duplicating business state. An application deploy scopes its generated seed-checkpoint filename to the locked contract version. A successor therefore starts a distinct checkpoint without deleting the predecessor receipt. A manually selected checkpoint from another version fails with batch_checkpoint_contract_version_mismatch, naming the checkpoint file and both versions; resume it against its original version or archive it after reviewing the prior outcomes.

Generated Rust, Go, TypeScript, and Python clients expose the same bounded policy for every symbolic command. Rust emits create_ticket_batch(inputs, options) and returns input-ordered independent item results plus a checkpoint. Go emits CreateTicketBatch; TypeScript emits createTicketBatch(inputs, { concurrency, checkpoint }); Python emits paired sync and async create_ticket_batch methods. Each uses a bounded worker or transport pool and records either the typed result or the public error for each item. All four reject zero items, more than 4,096 items, and concurrency outside 1..=384 before submitting work. This SDK ceiling is distinct from the CLI’s operator-facing 1..=32 bound documented above.

Both bindings report a monotonically increasing completed count and the largest contiguous input checkpoint. Completion may arrive out of order, but a reported checkpoint advances only after every earlier item has an independent result. Persist that checkpoint and pass it back when resuming; skipped inputs are never resubmitted, while any uncertain later item still retains its original idempotency key.

Batch methods do not introduce a second write semantic. The Rust SDK and CLI coalesce independent items into the public bounded transport batch (ExecuteBatch, at most 16 items) while preserving each item’s idempotency key, typed outcome, and uncertainty classification. A whole-RPC failure or a retryable per-item error re-enters only the affected ordinary commands through the same-key recovery path. The server may also physically group compatible durable transitions; callers still receive independent per-item results. At SDK concurrency 384, Rust opens at most 24 simultaneous transport exchanges of at most 16 items each; the setting never creates a 384-item RPC or an application-visible transaction. The coordinator remains independently bounded at 512 queued messages and 32 MiB, and one physical transaction remains bounded at 256 commands and 16 MiB.

Per-item results and recovery

Current servers return an always-populated, input-ordered item list on the batch response. Each item is either the ordinary command response or a typed application error that names the batch operation. Sibling successes stay intact when one item fails. The Rust SDK classifies each item error by its registry-derived recovery action: Retry and ResolveWithSameIdempotencyKey re-enter that item through ordinary same-idempotency-key recovery (with the same attempt budget and Overloaded backoff as a single command); other recovery actions surface immediately as certain failures with zero re-entry. Service-control failures (cancellation, deadline, response size, emergency containment) still fail the whole transport RPC. Older servers that omit the item list keep the historical whole-RPC recovery path.

A batch checkpoint is the largest contiguous input prefix that already has an independent terminal result (success or non-reentered failure after the item’s attempt budget). Resume skips that prefix and never resubmits those indices; retryable items are re-entered inside the original batch call before the checkpoint advances past them.

Why batching is not a bulk-write API

Bounded transport and client concurrency amortize admission without creating a second write semantic. Every item still has its own authorization, idempotency, provenance, commit sequencing, crash recovery, and typed outcome — exactly the single-command definitions.

Generated operational queries

Generated MCP tools for a paginated named query return the business outcome under result and the sole nullable opaque continuation under page.next_cursor. Reuse a non-null cursor only through the compiler-declared input cursor property; the service retains snapshot, order, expiry, and authorization authority. Hosted and stdio MCP use this same envelope. Unpaged tools retain their flat generated result, and generated SDK query options keep their existing typed cursor behavior.

Operational RiffQL gives an application a finite set of compiler-owned read plans without accepting a predicate tree, field name, operator, order clause, or index from the caller. A deployed named query may expose typed optional values, null/existence tests, binary prefix lookup, stable cursors, and bounded exact aggregates. Every possible optional-value presence combination is compiled, authorized, costed, and indexed before deployment.

Generate bindings from the reviewed exact lock:

riffdb application lock --write riffdb.application.json

The generated Rust, Go, TypeScript, and Python methods accept only the values declared by the query. For example, a relation filter is an optional string, not a generic condition:

let page = db.list_fga_tuples(ListFgaTuplesParams {
    store_id,
    relation: Some("viewer".to_owned()),
    after: None,
}).await?;

const page = await db.listFgaTuples({
  store_id: storeId,
  relation: "viewer",
});

Go and TypeScript use the long-lived Rust driver host; Python and Rust may use the verified-TLS application transport directly. In every case the result is a closed generated outcome with bounded arrays, optional values, opaque cursors, and exact integer/decimal carriage. Aggregate sums that do not assert a wire precision are decoded under the generated result schema; a conflicting wire precision or scale is rejected.

Runtime predicate checks compare same-type decimal coefficients numerically and money amounts numerically within the same currency and decimal type. Byte values use lexicographic byte order and Boolean values order false before true. These checks apply to compiler-admitted access paths; they do not add new physical range families to the closed access matrix below.

Cursor rule

Persist the opaque next_cursor returned with a page and submit it only to the same exact generated operation, contract, module, invariant parameters, principal, and page semantics. Do not decode it. For an ordinary ordered take $limit after $cursor binding, a resumed invocation may choose a different valid runtime Limit or Limit<MAX> value; page cardinality does not identify the keyset position. The plan and declared maximum remain immutable, and nearest/vector K, filters, partition, ordering, authority, snapshot, and provider epoch remain bound. An invalid, expired, cross-principal, or stale cursor is a typed failure; clients must not silently restart from the first page.

When a named query declares Cursor or Cursor?, generated Rust, Go, TypeScript, and Python methods route a present value to the protected query options cursor. They omit it from the symbolic parameter record. Absence means the first page for Cursor?; an exact Cursor remains a required method parameter. Supplying both the generated cursor parameter and the existing low-level cursor option fails locally instead of choosing one. Cursor bytes and server validation are unchanged. Generated methods issue exactly one bounded request. They do not expose an iterator that walks pages to implement count, offset, filtering, sorting, or any other query semantic.

Binary text-key ordering

Use a declared text_key(field, binary_utf8_v1) index component when a named query must traverse text in exact UTF-8 byte order. It can supply the bounded order by suffix without requiring a prefix predicate on that field. For example:

index by_title (organization_id, title, document_id) text_key(title, binary_utf8_v1)

proves order by title asc, document_id asc after the required organization equality, or the wholly reversed descending order. In that order doc-3 precedes doc6. A plain string index has canonical length-first ordering, so declare the text-key profile when bytewise interoperability matters. RiffDB still requires the complete deterministic index order, bounded take, and an opaque snapshot-bound cursor; callers cannot select the collation or index at runtime.

The same component also supports exact equality as a complete index-prefix term. For example, an index ordered by (organization_id, relation, user, document_id) with binary_utf8_v1 text keys on relation and user can back both order by relation, user, document_id and relation == $relation followed by order by user, document_id. This is one maintained index: the compiler must prove each named shape independently, and the runtime does not walk pages or filter after take.

It also supports one bounded membership branch when the text component is the first remaining order term:

where organization_id == $organization_id
  && object_id in $object_ids
order by object_id asc, relation asc, user asc
take 25 after $after

Declare object_id as text_key(object_id, binary_utf8_v1) in that shared index. RiffDB canonicalizes the submitted set once, lowers each distinct member to an exact byte prefix once, orders the prefixes physically, and applies one global page, plus-one probe, scan budget, and cursor across the union. Empty sets return an exact empty page. Forward and reverse traversal both retain bytewise order, so length-divergent values such as doc-3 and doc6 do not fall back to canonical string ordering.

Closed ordinary access matrix

For an order-preserving canonical i64, u64, timestamp, date, uuid, or enum component, or a bounded string declared with either text-key profile, <, <=, >, and >= compile to inclusive/exclusive physical endpoints. Binary text uses exact valid UTF-8 byte order without normalization, collation, case folding, or token parsing. unicode_fold_v1 uses the frozen Unicode 17.0.0 NFKC plus full non-Turkic case fold and compares the resulting bytes. Index maintenance and submitted predicates share that exact transform, while results retain the original authorized string. One lower and one upper bound may form an interval; != forms two disjoint complement intervals. The selected component must supply the first remaining order term. Memory and redb traverse the same normalized half-open schedule, including reverse continuation across an interval boundary, under one page, plus-one probe, scan, hydration, cursor, and output budget. Contradictory runtime bounds produce an exact empty page without a scan.

Canonical length-prefixed strings are not logical text-order ranges; use an explicit binary_utf8_v1 component for bytewise string intervals or unicode_fold_v1 for folded-byte intervals. Multiple branching dimensions, overlapping unions, joins, provider bridges, caller-selected indexes, and general client page walking remain unsupported. An older module whose range was never physically proved is refused as query unavailable with refresh_contract guidance; recompile and redeploy rather than filtering a returned page.

The repository audit found no active unsafe ordinary fixture: Ticketdesk’s canonical membership, the presence and binary-prefix conformance queries, and dependent complete-key batches all constrain storage before page selection. The richer directory filtering fixture uses the separately governed exact result-set provider. Historical external ordinary range queries require recompilation so the compiler can prove their physical interval shape; unsupported scalar or multi-branch forms remain refused.

Extending ordinary access paths

Ordinary access support is a closed compiler matrix, not a collection of backend conveniences. Adding an encoding, comparison profile, role, or role combination requires all of the following in one reviewed change:

1. Freeze the logical equality/order meaning and the exact compatible component roles.
2. Prove one complete partition prefix, branching or interval component, total-order suffix, unique tie-breaker, authority set, and static bounds.
3. Lower typed values once per step and request into the shared immutable range schedule; never validate profiles or construct ranges per row.
4. Pass the same forward/reverse, exact-end/plus-one, cursor, policy, fuel, and cancellation matrix in memory and redb.
5. Add a real-consumer amendment for any new relationship shape and separately bound driver rows, fan-out, intermediates, probes, bytes, output, missing behavior, policy, and identity.

The neutral application at fixtures/riffql/operational-access-corpus-v1 freezes equality, canonical and binary-text membership, prefix, typed canonical and binary-text interval and complement, nullable state and order, forward/reverse cursor, row-policy authority, and dependent complete-key shapes. Its exact lock and Rust, Go, TypeScript, Python, and MCP artifacts are checked by scripts/generate-operational-access-corpus --check.

fixtures/riffql/operational-query-capability-v5.json binds that corpus, the unchanged V4 capability receipt, and a value-free external tuple-changelog source-compilation and Go-only package receipt. The external receipt proves only the named access shapes it lists. It deliberately does not copy an external contract, schema, route, adapter, generated profile, or stored value into RiffDB, and it does not claim external runtime or full framework conformance.

The separate terminal receipt at fixtures/riffql/wp775-external-binary-text-interval-execution-v1.json records one fresh owning-repository execution. A consumer pinned to the exact recorded RiffDB lock and generated Go artifact invoked one named strict-lower/ strict-upper query twice over one partition and reused an original logical string returned by the first invocation as the second invocation’s consumer-owned lower token. The receipt retains only exact identity hashes, bounded counts and duration, and closed verification facts. It proves this narrow delegation and bytewise-order case only—not a framework route, full adapter, or full-application conformance—and it neither imports consumer vocabulary or values nor turns that logical token into a RiffDB cursor. The receipt’s riffdb_revision is the exact runtime revision that performed the run; the later repository revision that carries the immutable receipt is not reinterpreted as an execution revision.

Feature preflight

The authorized application catalog returns the closed riffdb.application-catalog/v1 feature registry for one exact contract. Rust applications can call StableApplicationClient::preflight_application_features. The checked result contains only the exact contract identity, active module hashes, and the six closed feature states; it contains no raw catalog symbols, numeric compiler identities, capability record, IR, or storage key.

unavailable is a prohibition, not a request for client emulation. In the current alpha, binary UTF-8 and unicode_fold_v1 prefix lookup are available. Applications submit only logical strings; the compiler-selected profile owns the exact transform and physical range. Application code must not replace an unavailable feature with client filtering, raw reads, N+1 requests, or handwritten query text.

Exact count and numeric offset

An exact-result named query may combine a bounded indexed text predicate, take $limit offset $offset, and exact_count(). Generated Rust, Go, TypeScript, and Python methods carry limit and offset as ordinary typed parameters and decode both the bounded row collection and { value: u64 } total. The CLI and generated MCP tool use the same operation and schema.

The server chooses one authorized provider epoch and returns the page and total from that epoch. For a protected query, bounded background activation applies the current row policy before it constructs count or ordinal state; the capability identity and revision select a separate derived slot. Denied rows therefore cannot influence the released total or order. The request path seeks the indexed ordinal directly; it never scans, walks cursor pages, filters in the client, or counts only the page. A fresh invocation is a new current- snapshot observation, so numeric offset is intended for admin-style windows rather than stable traversal across concurrent writes. Use ordinary cursor queries when snapshot-bound continuation is required.

The initial request waits for a fixed, bounded readiness window while the derived provider builds in the background. It may still return RDB-QUERY-0102 when that window expires. RDB-PROJECTION-0101 means the required providers cannot prove one common epoch, RDB-PROJECTION-0102 means the requested snapshot retired, and RDB-PROJECTION-0103 means no provider snapshot yet satisfies the request’s current or read-after-commit freshness. Generated clients expose these as the same typed application error codes on every transport. A host may retry these lifecycle outcomes with a bounded attempt budget; it must not emulate a scan, retain a stale page, or weaken freshness.

The narrow exact-result form accepts one partition equality, one exact text predicate, and at most one compiler-declared optional typed equality filter. The richer V6 form accepts the closed exact-predicate vocabulary documented in Exact Predicate and Order Families: comparisons, bounded typed sets, state tests, binary text operations, capped Boolean shape, and compiler-enumerated optional guards. It also permits a total order over fields independent from the search fields. Any unsupported or unindexed family member rejects the complete source with a span; it is never a request-time filter or ignored condition.

Set parameters accept at most 64 submitted values. RiffDB sorts and deduplicates them canonically before indexed execution; empty in and not_in sets retain their documented exact semantics. Plain Limit is positive and at most 499; Limit<MAX> enforces its compiler-declared maximum (1 through 65,534) and is charged at that maximum. Offset is bounded by the compiled family. Every declared predicate and order field contributes provider storage and rebuild work, and every optional guard multiplies the finite member count. Keep named operations small and task-shaped rather than building one combinatorial search endpoint.

Framework-neutral exact-result proof

The retained operational fixture exposes a generic Document query family for binary UTF-8 contains, starts-with, and ends-with matching. The family combines one typed optional document filter, deterministic ascending and descending orders, bounded limit and numeric offset, and a complete exact total. Its generated Rust, Go, TypeScript, Python, CLI, and MCP surfaces prove the RiffDB semantics without embedding an external framework’s schema, route, role, or application policy in this repository. Its generic DirectoryUser family additionally proves bounded sets, comparison ranges, null/existence state, optional-presence members, Unicode values, and search-by-one-field/order-by- another through the same six public surfaces.

Framework integrations own their generated profile and route-level acceptance in their own repositories. They pin a RiffDB build and prove that their public API delegates to generated named methods without client-side filtering, sorting, counting, raw query construction, or storage access. An owning external adapter may perform narrow exact-page translation when its authoritative interface requires more rows than one compiled RiffDB page: it requests min(remaining, MAX), follows one unchanged cursor chain, appends every row exactly once in order, and returns the final cursor unchanged or none on true exhaustion. It may not discard, reshape, restart, or release partial success after any cancellation or error. RiffDB keeps the generic compiler, provider, policy, transport, and conformance surface; it does not ship a Better Auth admin route or generated Better Auth admin SDK.

CLI and MCP

The same named query can be invoked by CLI or its generated MCP tool. CLI parameters use the public tagged JSON value format; MCP input and output JSON Schemas contain only symbolic parameters and bounded result fields. Neither surface accepts an index, plan, field ID, or arbitrary predicate.

The retained acceptance corpus is:

./scripts/adapter-operational-query-acceptance --all-languages

It deploys one exact application over verified TLS, seeds through compiled commands, and checks OpenFGA tuple filtering, an MLflow exact metric dashboard, Payload binary-prefix and null pages, a Woodpecker state queue, a typed session lookup, and the framework-neutral exact-result family through Rust, Go, TypeScript, Python, CLI, and generated MCP schemas. The exact-result phase proves all three binary-text operators, both declared order directions, direct offset boundaries, complete totals, optional-filter null/present forms, and the V6 set/range/state/independent-order family. The corpus also proves optional first-page and generated continuation calls in all four languages, that a malformed generated cursor fails closed, and that each response stays within its declared page bound. It scans the application runners for kernel, storage, numeric-ID, raw-transaction, and client-filter escape hatches. External adapter repositories own their separate route-level compatibility matrices.

Partition queries buffer at most 64 rows per stream refill within their existing compiled fuel. Speculative reads use only unused row allowance; requests at the maximum bound retain lazy stream heads until remaining output is guaranteed. Mixed-direction queries sort each incoming page, merge ordered runs, and retain the physical-prefix boundary needed for safe cursors. These changes preserve query limits, authorization, snapshot epochs and continuation formats.

Rust Applications

Rust applications use the public riffdb-client-rust facade plus a generated module for the exact application lock. They do not depend on server, storage, policy, compiler, Tonic, Prost, or raw Protobuf crates.

For riffdb dev --run, the runner supplies the protected local values as both RIFFDB_ENDPOINT, RIFFDB_CREDENTIAL_FILE, and RIFFDB_DATABASE and the legacy positional arguments ENDPOINT CREDENTIAL_FILE DATABASE. Generated starters accept either form and fail closed if both are present but disagree. Ambient values cannot redirect the runner-owned child process.

For a remote daemon, construct the closed riffdb_config::TlsClientConfig and call RiffDbClient::connect_verified_tls. The client loads either an explicit protected CA root (TlsClientConfig::new) or the platform system CA bundle (TlsClientConfig::system_trust / from_pem_or_system), verifies the exact DNS/IP identity, and has no trust-all, cleartext-fallback, client-certificate, cipher-suite, or provider selector. Long-lived hosts can retain VerifiedTlsConnector; it atomically adopts only a complete valid trust-root replacement for new connections and retains the last valid snapshot otherwise. See Remote and Local Application Ingress.

StableApplicationClient::connect_verified_tls constructs the application-only facade directly from the same closed TLS configuration. The first-party Driver Host uses that path to centralize trust, pooling, retry, cancellation, uncertainty, and reactive resumption for target languages without exposing the kernel client.

Generated boundary

Generate from the reviewed lock:

riffdb application generate --locked

The generated module contains exact contract, query-module, command-plan, and artifact identities; checked input and result types; declared outcomes; and the operation-specific client methods. The stable facade owns transport validation, retry classification, fresh request IDs, and uncertain-outcome recovery.

When the compiler proves that a named page is fully served by an explicit covering index, the generated Rust operation accepts a compact positional wire result and decodes it directly into the same public result type. The field layout is compiled into the operation and is never a caller option. Any identity, field-order, cardinality, width, enum, bound, or value mismatch fails closed; the same decoder still accepts the legacy named-record response from an older compatible server.

A compiler-typed Cursor? generated parameter is Option<String>; None requests the first page and a present opaque cursor resumes that exact method. An exact Cursor remains a required String. Generated code routes the value through QueryOptions without encoding it as symbolic input and rejects a simultaneous low-level cursor option. Each method performs one bounded request; no generated helper walks pages for count, offset, filtering, or sorting.

let outcome = client
    .create_item(CreateItemInput {
        item_id,
        title: "Review contract".to_owned(),
    })
    .await?;

match outcome {
    CreateItemOutcome::Created(value) => println!("{}", value.item_id),
    CreateItemOutcome::AlreadyExists(_) => {}
}

The exact generated names come from your contract and lock. Treat a changed generated diff like an API change: review the source change and new lock before accepting it.

Generated role clients also embed their compiled role and lock identity constants, provide connect_env for one-line environment-backed setup with a default AttemptBudget::new(3), generate a fresh IdempotencyKey per command invocation when omitted, track causal read-your-writes watermarks by default on each client handle (causal() / with_default_read_after_commit), expose direct .outcome_tag(), .unwrap_found(), .or_none(), and .items() result helpers, emit contract enum types in query parameter and row structs, and provide bounded <query>_pages / <query>_items pagination helpers with a default 64-page ceiling.

Optional bounded application session

Rust generated clients expose open_bounded_session as an explicit alpha transport candidate. Unary application RPCs remain the default. The call takes the SHA-256 identity of the exact canonical riffdb.application.lock.json that produced the generated client; the generated module supplies its compiled contract and query-module identities itself.

let application_lock_hash = load_reviewed_application_lock_hash()?;
client.open_bounded_session(application_lock_hash).await?;
assert!(client.bounded_session_enabled());
// Before a controlled colocated-server shutdown:
client.close_bounded_session();

The session is transport multiplexing only. Every command and query is still independently authenticated, authorized, bounded, and executed through the ordinary application service. Stream order grants no transaction, snapshot, freshness, or command ordering: use generated *_after_commit methods for causal reads exactly as on unary transport. Command retries retain the original idempotency input, and stream loss retains the existing OutcomeUnknown classification when the durable result cannot be resolved.

One session is bound to one database, credential presentation, exact contract, and finite set of query modules. Metadata or application-identity drift fails closed. The alpha ceiling is 128 independently in-flight operations; response state, cancellation state, total lifetime work, and output stalls are bounded. There is no automatic fallback after a session has opened. If establishment is unavailable, the application may deliberately keep the unchanged unary client; diagnostics and benchmarks must report which transport was selected. close_bounded_session explicitly ends the request stream, releases pending operations under the ordinary uncertainty rules, and returns the facade to unary transport. Drop the client or call it before waiting for a colocated server’s graceful shutdown.

Operational applications can preflight the exact authorized server feature registry with StableApplicationClient::preflight_application_features before constructing the generated facade. An unavailable feature never authorizes a client-side scan or filter. See Generated Operational Queries.

Generated entity and outcome decoders consume every wire field exactly once. An optional<T> field maps a wire null to None and a present value to Some(T) through the same typed decoder used for required fields. This applies to command outcomes as well as query results; application code never probes or removes a field by its compiler ID.

Application Source V4 also generates typed durable event consumers and live named queries. Event deliveries retain exact lease evidence for generated acknowledge and negative-acknowledge methods. Live updates use a closed union; persist each applied cursor and clear retained results on Terminal. See Reactive Application Clients.

Generated command batch methods accept GeneratedBatchOptions concurrency from 1 through 384 and at most 4,096 inputs. The setting bounds independent in-flight items; each public ExecuteBatch request still carries at most 16 ordinary commands, with separate identities, outcomes, and recovery. At the maximum, Rust opens at most 24 bounded transport exchanges; it does not create one 384-command RPC or application transaction.

Retry rule

Use one caller-owned idempotency key for one logical command and retain it until the outcome is known. The stable client may retry only according to its bounded attempt policy. OutcomeUnknown means the server result could not be resolved; it does not mean the command failed.

Migration administration

Contract migration is a kernel administration surface, not generated application API. The stable facade exposes CheckContractMigration, ApplyContractMigration, generate_contract_migration_operation_id, and the three migration client methods. Construct one immutable submission and reuse it across bounded retries; only the outer request ID changes.

let check_id = generate_contract_migration_operation_id()?;
let check = CheckContractMigration::new(
    check_id,
    candidate_bundle.clone(),
    migration_bundle.clone(),
)?;
let checked = client
    .check_contract_migration_with_retry(&check, attempts, &metadata)
    .await?;

let apply_id = generate_contract_migration_operation_id()?;
let apply = ApplyContractMigration::new(
    apply_id,
    candidate_bundle,
    migration_bundle,
    reviewed_migration_hash,
)?;
let accepted = client
    .apply_contract_migration_with_retry(&apply, attempts, &metadata)
    .await?;

Check and apply require different operation IDs because their canonical input identities differ. After an uncertain apply, retain the apply ID and call get_contract_migration_operation after the selected database reopens. The current Go, TypeScript, and Python facades intentionally do not expose this administrative surface.

Public API

The built handbook publishes Rustdoc only for riffdb-client-rust. See the Rust API reference. Internal workspace crates are implementation boundaries, not supported application APIs.

Driver Host

riffdb-driverd is the long-lived Rust transport owner for generated server-side TypeScript and Go bindings. It connects to one remote RiffDB application over verified TLS, retains the protected capability credential, pools HTTP/2 connections, and exposes only the intersection of one exact generated application catalog and the selected symbolic role over a private Unix socket. An operation generated for a different role is absent rather than being sent remotely and rejected after the fact.

The host is not a generic gRPC proxy. Its local protocol cannot express kernel entity/index reads, administration, contract deployment, ad-hoc RiffQL, raw Protobuf, credentials, remote endpoints, numeric compiler IDs, field masks, or arbitrary methods.

Configuration

Pass exactly one protected TOML path to riffdb-driverd. The file must be a regular, non-symlink file with no group or other permissions. Every configured path is absolute and normalized.

[driver]
socket = "/run/myapp/riffdb-driver.sock"

[application]
lock_file = "/srv/myapp/riffdb.application.lock.json"
manifest_file = "/srv/myapp/generated/riffdb.application.exact.json"
operation_catalog_file = "/srv/myapp/generated/mcp/tools.json"
database = "ea"
role = "EaApplication"

[remote]
endpoint = "https://riffdb.internal:7443"
tls_trust_root = "/run/secrets/riffdb-ca.pem"
tls_server_name = "riffdb.internal"
credential_file = "/run/secrets/ea-application.credential"
pool_connections = 4
streams_per_connection = 64

The socket parent directory must already exist, be a real directory, and have no group or other permission bits. The socket is created as mode 0600. Another live owner, a symlink, or a regular file at the path stops startup. A private same-owner stale socket can be reclaimed. Linux peer credentials are checked before a request is read.

The complete socket pathname must fit the operating system’s Unix socket address, measured in filesystem bytes. Linux permits at most 107 bytes; other Unix platforms may impose a smaller limit. Use a short protected runtime directory. Native preflight rejects an unrepresentable address before reading application artifacts, opening remote connections or changing the socket path. The same platform check applies to the operator driver.

The endpoint is always HTTPS. There is no cleartext fallback, native-root mode, trust-all mode, client-certificate setting, cipher-suite setting, or TLS provider setting on this surface.

Exact handshake

Before binding the socket, the host authenticates across every pooled TLS channel and verifies the database alias, active contract lineage/version, and bundle hash against the exact lock. Every local connection then begins with protocol V2 handshake. Both sides prove:

• protocol version and driver build identity;
• application manifest, active contract, generated module/operation catalog, and per-invocation schema hashes;
• selected database alias and symbolic role-definition hash;
• application value and structured-error registry hashes; and
• a digest of the host-configured verified remote endpoint identity.

The local protocol never carries the endpoint or bearer credential itself. A mismatch closes the application session before any remote operation runs.

generated/mcp/tools.json retains the least-sufficient V2 through V5 profile when no generated command is present. A nonempty command registry selects the strict V6 successor and records that predecessor profile. V6 command entries bind the compiler’s command ID and underscore MCP name, exact input and outcome schemas, the service-owned operation envelope, and the mechanically composed hosted output schema. Command and visible-query mcp_descriptor objects contain only name, inputSchema, outputSchema, and the four standard safety hints. The catalog is a generated expectation for parity checking; the live policy grant still decides visibility and the service still owns dispatch.

Moving an exact package to V6 changes its MCP artifact hash and enclosing lock identity but not the application-lock schema. Preview and review that hash change before application lock --write; old V2 through V5 catalogs remain readable and carry no V6 parity claim. Reactive entries retain their closed operation kind and action, and contextual reactions still pin the declared reaction and target command identity.

Calls, batches, and backpressure

An invoke selects one generated operation and repeats the exact generated input-schema hash. Values use the shared tagged application-value registry; integers and exact decimals never pass through floating point. Fixed-dimension vectors cross this local JSON boundary as bounded arrays of exact IEEE-754 binary32 component bits; the host reconstructs and revalidates the canonical finite vector before application dispatch. Query calls may carry an opaque cursor, a read-after-commit frontier, and the one closed admission_head consistency value. Deadline and retry bounds are explicit. Driver protocol V4 adds that stronger value; V1 through V3 remain readable but fail closed rather than accept or downgrade it.

For a compiler-proven covering named query, a generated client also advertises that it accepts the V2 compact result arm. The host returns the compiler-sealed entity symbol, field order once, and bounded positional rows. The generated operation-specific decoder checks that exact outcome, result field, entity, layout, row width, value type, and bound before constructing typed results directly. Callers cannot request a physical index or supply a layout. Queries without a complete safe cover, and older clients that do not opt in, continue to receive the legacy named-record result. CLI and MCP also retain that legacy shape.

A batch selects one generated command, at most 4,096 independently idempotent inputs, concurrency from 1 through 384, and a contiguous resume checkpoint. Each item remains an ordinary command with its own durable outcome and error. The batch is not one atomic transaction and cannot submit arbitrary writes.

Pool connections, HTTP/2 stream admission, local connection count, per-connection operations, response queues, and frame sizes are bounded. Saturation returns the typed RDB-CAPACITY-0101 result rather than creating an unbounded queue. A peer has five minutes to complete its initial handshake. After an exact handshake succeeds, the session remains attached until the caller closes it or the host drains. This matters for generated applications whose least-authority role sessions may legitimately remain quiet for more than five minutes; the 256-connection host ceiling bounds that retained resource.

Cancellation and uncertainty

Local requests and batches have explicit identities and may be cancelled from the same or another authenticated local session. Batch cancellation propagates to each in-flight ordinary command. Cancelling a read stops waiting without a commit claim. Cancelling or timing out a command after submission reports outcome_uncertain: true and prescribes resolution with the same idempotency identity. It never reports that the command failed merely because the local caller stopped waiting.

Rust task cancellation also releases the local in-flight request registration and pool admission. Dropping a batch releases its parent registration and aborts its owned child tasks, whose registrations retire when their cancellation is polled. A completed or aborted request does not reserve its local request ID for the host’s lifetime. This cleanup makes no claim about the durable outcome of a submitted command; recovery still uses the original idempotency identity.

On SIGINT/SIGTERM integration, the host stops accepting work, asks established connections to drain, allows a bounded 30-second completion interval, then closes the private socket. Remote command uncertainty remains represented by the same structured result.

Reactive operations

The same invoke envelope supports generated durable stream pulls, acknowledgements, negative acknowledgements, seeks, status, contextual work items and reactions, plus resumable live-query updates. Cursors, lease tokens, and causation tokens remain opaque. A live-query invocation returns one typed snapshot, patch, reset, checkpoint, or terminal update; the caller reconnects with the returned cursor.

Structured failures preserve the stable code, category, retryability, recovery action, checked operation and symbol path, authorized contract identity, trace identity, incident identity, and command uncertainty flag. Local parsing never classifies failures from prose.

Reimport operator host

Application reimport uses a different process, socket, protocol, capability, and target-language binding: riffdb-operator-driverd. It is not a mode of riffdb-driverd, and the normal generated application client has no reimport method. The Rust process owns verified TLS, the Capability V7 credential, the exact portability artifacts, and the campaign identity. A target-language operator sees only the protected socket and the public handshake identity.

The operator configuration is a protected regular TOML file (mode 0600), and every referenced path is absolute, normalized, non-symlinked, and non-writable by group or other users:

[driver]
socket = "/run/riffdb/reimport-018f.sock"

[campaign]
database = "restored"
campaign_id = "018f2f85-3c20-7a31-8f11-112233445566"
contract_lineage = "orders"
scope = "whole_application"
portability_manifest = "/srv/orders/riffdb.portability.json"
export_manifest = "/srv/orders/export-manifest.json"
export_receipt = "/srv/orders/export-receipt.json"

[remote]
endpoint = "https://riffdb.internal:7443"
tls_trust_root = "/run/secrets/riffdb-ca.pem"
tls_server_name = "riffdb.internal"
credential_file = "/run/secrets/orders-reimport.credential"

Start the protected host with exactly that path:

riffdb-operator-driverd /run/riffdb/orders-reimport.toml

The V1 local protocol admits only start, apply_page, status, and cancel. start must repeat the exact manifest and receipt already loaded by the Rust host; apply_page carries one bounded canonical export page and its exact hash. The host rejects alternate campaign, database, manifest, receipt, page, cursor, or completion identities before invoking the server. It cannot express an application command, raw row write, caller-selected idempotency, remote endpoint, bearer credential, or storage operation.

The language bindings are deliberately separate from generated application facades:

session, err := operator.ConnectOperator(ctx, socketPath, operator.OperatorIdentity{
    Database: "restored", CampaignID: campaignID,
    PortabilityManifestHash: portabilityHash,
})

import { OperatorTransport } from "@riffdb/application/operator";

const operator = await OperatorTransport.connect(socketPath, {
  database: "restored", campaignId, portabilityManifestHash,
});

from riffdb_application.operator import OperatorIdentity, OperatorTransport

operator = OperatorTransport.connect(
    socket_path,
    OperatorIdentity("restored", campaign_id, portability_manifest_hash),
)

One operator session is serial and bounded because it advances one durable campaign. Closing the local connection does not cancel or roll back the campaign; reconnect with the same identity and call status. Target-language bindings decode structured public errors and never receive credentials or secret diagnostic values.

Cross-language conformance

The alpha freezes one shared corpus under fixtures/driver. Rust, Go, TypeScript, and Python each create and replay a command, perform a read-after-commit query, and observe the same typed idempotency-reuse failure against one verified-TLS daemon, database, contract, and symbolic role. The same gate revokes that role and proves the retained host no longer supplies authority. Every language observes the same typed RDB-AUTH-0215 capability revocation rather than a generic authorization denial. Local installed-artifact cells cover bounded cancellation, uncertainty, malformed frames, reactive cursor/lease handling, and shutdown.

Run the complete proof with:

./scripts/driver-conformance --all-languages --remote

fixtures/driver/manifests/*-v1.json states each language’s exact transport, platform claims, feature availability, and corpus digest. An unsupported cell is named explicitly; it is not filled by handwritten application glue.

The POC driver host is Linux-only. Generated Go and long-lived TypeScript bindings speak this protocol through one retained local session. Their public runtimes expose only the protected socket and public exact-handshake identity; they have no endpoint, TLS, bearer-credential, gRPC, remote retry classifier, kernel, or administration surface. Application authors must not hand-write protocol frames. See Go Applications and TypeScript Applications.

Go applications

RiffDB Go applications use a generated, typed facade over one retained local session to riffdb-driverd:

generated Go operations
        |
        v
riffdb.dev/application
        |
        v
private Unix socket
        |
        v
riffdb-driverd -> verified remote RiffDB

The Go process never receives the remote endpoint, TLS configuration, or capability credential. The Rust host owns connection pooling, authentication, retries, uncertainty resolution, read-after-commit behavior, reactive cursor resumption, and overload. The Go runtime owns only bounded local framing, language values, context cancellation, and typed facade assembly.

Create an offline-buildable starter with:

riffdb new orders --language go
cd orders
GOPROXY=off go test ./...

The generated repository contains:

go.mod
main.go
generated/go/client.go
third_party/riffdb-application/
riffdb.application.json
riffdb.application.lock.json

generated/go/client.go contains exact operation and schema identities, typed query parameters/results, closed command outcomes, per-item batch results, exact decimals/money, reactive iterators, lease operations, and live query cursors. context.Context cancellation propagates to the Rust host; a cancelled command is never reported as failed when its outcome is uncertain.

Eligible covering named queries use the negotiated driver V2 compact result. The generated method validates its exact compiler-owned positional layout and constructs the same typed result directly, without a per-row map or exposed ordinal. The runtime retains legacy named-record decoding for compatible servers and rejects malformed or mixed response shapes.

For a named query with Cursor?, pass the returned cursor through the same generated parameter on the next call; leaving it nil requests the first page. An exact Cursor remains a required string. The generated method moves that opaque value into QueryOptions.Cursor and never includes it in the symbolic input. Do not also set the low-level option: the generated method rejects the collision locally. It makes one bounded request and does not walk pages to implement count, offset, filtering, or sorting.

The starter connects with the driver socket plus the host’s public handshake identity. Those hashes are substitution checks, not authority. Do not put a credential, remote endpoint, raw method name, numeric compiler ID, field mask, or Protobuf value in Go application code. Generated role clients also embed their compiled role and lock identity constants (Connect<Role>Client), automatically generate an IdempotencyKey per command invocation when omitted from the input struct, track causal read-your-writes watermarks by default on each client handle (Causal()), and expose .OutcomeTag(), .UnwrapFound(), .OrNil(), .Items(), and bounded <Query>Pages / <Query>Items pagination helpers (defaulting maxPages to 64).

For a repository created with riffdb init --generator go, author main.go against the generated client and riffdb.dev/application, then run the whole local stack with:

riffdb dev --seed --run

The development workflow starts riffdb-driverd and passes the private socket plus the exact compiler-owned handshake identity as the program’s ten positional arguments. Their order is a closed development-runner ABI:

SOCKET MANIFEST_HASH CATALOG_HASH DATABASE ROLE ROLE_HASH REMOTE_HASH LINEAGE VERSION BUNDLE_HASH

VERSION is the ninth value after the program name. The generated riffdb new --language go starter demonstrates the complete checked decoding; copy that connection prelude when authoring an application from riffdb init. Do not guess or reorder the values, and do not derive, persist, or replace them. They are substitution evidence used only to open the generated application session. The workflow supports go.mod as a first-class runner manifest and runs with the caller’s selected offline module source.

A V7 Go-only application does not declare or retain an MCP artifact. Before starting the driver, the development workflow derives its operation catalog once from the exact application source and V8 lock in a private temporary directory. The catalog remains compiler-checked and role-bounded; it is not a hidden generated application surface and disappears when the workflow exits.

A Go library module can keep its executable in one dedicated package:

riffdb dev --seed --run --go-runner-package cmd/server

For a detected Go repository, the default remains .; omitting the option does not classify a TypeScript, Rust, or Python repository as Go. The option accepts only one canonical, repository-relative directory inside the root module and requires that package to declare package main. Absolute paths, parent traversal, symlink escapes, nested modules, non-main packages, and flag-like executable selection fail before the driver starts or credentials exist. RiffDB invokes go run directly with the checked package and prints it in the bounded runner summary. This is development process configuration only: it changes no contract, query module, role, application lock, plan, or production identity.

Current limitation: the Go/driver-host path is Linux-only in the alpha. The Go runtime has no pure-Go remote fallback.

TypeScript applications

RiffDB TypeScript applications use two compiler- and product-owned layers:

generated named operations
        |
        v
@riffdb/application
        |
        v
private driver socket
        |
        v
riffdb-driverd -> verified public RiffDB application surface

Server-side application code opens one retained DriverApplicationTransport, wraps it in DriverGeneratedApplicationTransport, and passes that transport to the generated contract client. It then calls only generated methods:

const driver = await DriverApplicationTransport.connect({
  socketPath,
  identity: publicDriverIdentity,
});
const transport = new DriverGeneratedApplicationTransport(driver);
const db = new AgentAlphaClient(transport, 3);

const created = await db.createItem({
  idempotency_key: requestId,
  item_id: itemId,
  title,
});
const page = await db.itemPage(
  { item_id: itemId },
  created.commitSequence === undefined
    ? {}
    : { readAfterCommit: created.commitSequence },
);

await driver.shutdown();

The generated request carries the exact contract/module/plan identities and compiler-owned schemas. The runtime rejects a response before returning it if its identity, cardinality, outcome, symbolic fields, wire field identities, or value types differ.

For a named page fully proven by an explicit covering index, the generated request negotiates driver V2 compact carriage. Its operation-specific decoder checks the exact compiler-owned entity and field order, then constructs typed rows directly from bounded positional values. Application code cannot select the index, schema, or ordinal and receives the same result type when a legacy named-record response is used.

Exact decimals and money never pass through a JavaScript number. Use the first-party constructors instead of manually encoding coefficient bytes:

import { exactDecimal, exactMoney } from "@riffdb/application";

const unitPrice = exactMoney("USD", "25.00");       // money<USD>
const taxRate = exactDecimal("0.0825", 8, 4);       // decimal<8,4>

The constructors reject exponents, excess scale or precision, invalid currencies, and floating-point inputs. The current CLI preserves the compiled precision in tagged results. Generated decoding also restores it from the exact result schema when a compatible older tagged value omits the redundant field, while still rejecting a conflicting precision, scale, or currency.

Generated clients are valid under standard strict TypeScript projects with both noUnusedLocals and noUnusedParameters enabled. Compact-carriage decoder helpers are emitted by reachability from the compiled result layouts: a client contains only the string, integer, timestamp, and shared payload helpers its generated decoders actually call. Do not copy unused decoder support into the generated file or disable an unused-symbol check to accommodate generated output.

Generated command batch methods accept concurrency from 1 through 384 and at most 4,096 inputs. They use a bounded worker pool over ordinary generated commands; every item keeps its own identity and result, and the collection is not atomic.

The protected application-role credential belongs only to riffdb-driverd. The TypeScript process receives a private socket path and public exact-handshake identity, never the credential, remote endpoint, TLS configuration, or a gRPC client. Do not expose the driver socket or generated server client to browser JavaScript. CliApplicationTransport remains a compatibility/debug adapter; new generated repositories use the retained host path.

AbortSignal cancels query, command, batch, stream, contextual, and live-query waits through the local protocol. Stream batch, in-flight, lease, wait, and negative-acknowledgement delay bounds are transmitted exactly rather than being advisory client values. Async iterators retain cursors and stop on typed terminal results. Call shutdown() during server drain.

Generated offline web repository

riffdb new my-app --language typescript emits a complete server-side HTTP starter:

package.json
package-lock.json
tsconfig.json
vendor/riffdb-application/
node_modules/              # materialized by the sealed installation
generated/typescript/client.ts
src/main.ts

The sealed installation copies its exact TypeScript 7 compiler, Node type definitions, and product runtime into the staged repository. No registry access or manual package edit is required:

npm_config_offline=true npm run check
npm_config_offline=true npm run build
npm start -- <socket> <manifest-hash> <catalog-hash> <database> <role> \
  <role-hash> <remote-identity-hash> <lineage> <version> <bundle-hash>

The starter serves /item, executes one generated idempotent command, performs one generated page-shaped read with the returned read-after-commit fence, and returns only the typed observation. Browser code never receives the credential.

For the canonical local loop, run riffdb dev --seed --run. RiffDB starts a verified loopback-TLS daemon and the first-party riffdb-driverd, provisions the exact application role, and supplies the starter only the protected socket plus its public handshake identity. The sealed alpha bundle carries a static development-only loopback certificate and key for this workflow; they are not production credentials and the daemon never binds them to a non-loopback address. Remote and production deployments must supply their own TLS identity through the documented driver-host configuration.

The command stays in the foreground for a web application. Once it prints riffdb-app-ready-v1<TAB>PORT, leave that terminal running and exercise the generated route from another terminal:

curl http://127.0.0.1:PORT/item

The ready marker is an invitation to send requests, not command completion. Interrupting the foreground command closes the application, scoped driver, and development daemon together.

Application Source V4 additionally generates typed event async iterators, live query update unions, a framework-neutral live store, and an application-server SSE relay. The relay retains the RiffDB credential server-side and requires an application-owned authorization callback. See Reactive Application Clients.

Python Application Driver

RiffDB’s Python package is riffdb-application; application code imports riffdb_application. Wheels use the stable cp313-abi3 interface, so installing a wheel does not require Rust. Building the source distribution requires Rust 1.98.1.

The alpha release matrix is deliberately finite:

Unsupported combinations fail installation; there is no pure-Python transport fallback. The machine-readable source of truth, including the reason for each unsupported family, is release/python/platform-matrix-v1.json in the source distribution repository.

The package is application-only. It exposes generated named queries and typed commands through SyncApplicationTransport and AsyncApplicationTransport. It does not expose Protobuf stubs, kernel requests, administrative RPCs, SQL, or the private native module. Both transports delegate gRPC status validation, request identity, bounded retry, and uncertain-outcome behavior to the stable Rust application client.

The wheel’s in-process PyO3 transport and the local-socket driver use one first-party Rust protocol core. That core alone admits values, applies nesting, collection, string, and frame bounds, constructs typed query and command requests, and classifies public failures. Python keeps its self-contained wheel and does not require riffdb-driverd; it also cannot accept a value the socket driver would reject. One collection is limited to 4,096 values, one string to 262,144 UTF-8 bytes, and enum names use the closed generated-symbol alphabet. Compiled operation schemas may impose stricter limits.

Generated named queries that the compiler proves fully covered negotiate a compact result from the native transport. The generated synchronous and asynchronous decoders validate the exact plan-owned entity, field order, cardinality, row width, enum values, and bounds before constructing the same dataclasses directly. No ordinal or physical-index choice is exposed to Python, and legacy named-record results remain accepted.

AsyncApplicationTransport also backs Application Source V5 generated event iterators and live named-query iterators. Generated delivery types retain the attempt-specific acknowledgement evidence, and live cursors are returned as standard padded Base64 for persistence. See Reactive Application Clients.

Generated synchronous and asynchronous command batch methods accept CommandBatchOptions(concurrency=...) from 1 through 384 and at most 4,096 inputs. Each item remains an ordinary independently authorized and idempotent command; the collection is not one transaction.

Create an application

Build or obtain the matching RiffDB wheel, then create a source-layout project:

export RIFFDB_APPLICATION_WHEEL=/path/to/riffdb_application-0.1.0-cp313-abi3-linux_x86_64.whl
riffdb new my-app --language python
cd my-app
uv sync --locked
PYTHONPATH=src uv run --locked python -m unittest discover -s tests

Installed release bundles place the architecture-matching wheel under public/python; the environment variable is only needed by a source-tree CLI that has no installed bundle. The scaffold copies the wheel into vendor/, pins its SHA-256 in uv.lock, and performs no registry lookup during uv sync --locked.

Run the generated example after deploying and binding its application role:

PYTHONPATH=src uv run --locked python -m my_app \
  http://127.0.0.1:50051 \
  .riffdb/development/credentials/MyAppApplication.credential \
  my_database

riffdb dev --run supplies those same three values as both positional arguments and the runner-owned RIFFDB_ENDPOINT, RIFFDB_CREDENTIAL_FILE, and RIFFDB_DATABASE environment. A runner that accepts both forms must reject a mismatch. Go and TypeScript do not receive these variables because their application process is confined to the private driver-host protocol.

Credentials are Rust-owned, redacted, nonextractable, non-pickleable objects. Use BearerCredential.from_protected_file() for a mode-0600 Linux credential. Use CallMetadata.with_database(DatabaseAlias("my_database")) to select one database before authentication.

For an authenticated remote listener, configure explicit private-CA trust; Python never falls back to native roots or a trust-all mode:

from riffdb_application import SyncApplicationTransport, VerifiedTlsConfig

tls = VerifiedTlsConfig(
    endpoint="https://riffdb.internal:7443",
    trust_root="/run/secrets/riffdb-ca.pem",
    server_name="riffdb.internal",
    pool_connections=4,
    streams_per_connection=64,
)
with SyncApplicationTransport.connect_verified_tls(tls, metadata) as transport:
    client = MyAppClient(transport, AttemptBudget(3))

The asynchronous transport exposes the same verified-TLS constructor. Pool connections are bounded from 1 through 16 and streams per connection from 1 through 256 before native transport work begins.

Existing applications

Application source v1 remains exact and does not gain Python implicitly. Preview the explicit source-only migration:

riffdb application migrate --to v2

Write only the source manifest after review:

riffdb application migrate --to v2 --write
riffdb application lock --write
riffdb application lock --check

The migration command never writes a lock or generated artifact and never contacts a server. Source v2 requires generation.python; lock v2 contains one Python artifact path and digest. Normal lock review remains the only operation that publishes compiler-derived identities and generated bytes.

Direct installation

Install a released wheel with either tool:

python3 -m pip install ./riffdb_application-0.1.0-cp313-abi3-manylinux_2_28_x86_64.whl
uv pip install ./riffdb_application-0.1.0-cp313-abi3-manylinux_2_28_x86_64.whl

Source installation is conventional but compiles the bundled Rust dependency graph:

RUSTUP_TOOLCHAIN=1.98.1 python3 -m pip install ./riffdb_application-0.1.0.tar.gz

Publishing to PyPI is intentionally a separate maintainer action. The POC release pipeline produces and verifies artifacts but holds no upload token.

Maintainers build both wheels and the self-contained source distribution with:

scripts/build-python-distribution dist/python

The build also writes riffdb-python-artifacts-v1.json, a deterministic receipt containing the platform-matrix digest and the name, SHA-256, and byte length of each artifact. The sdist contains only the pinned Rust application-client closure (including the shared driver host and operation registry), required persistent Protobuf fixtures, licenses, and Python package; it does not copy the repository or depend on a checkout. The release checks install the wheel with Rust absent from PATH and install the sdist with Cargo network access disabled.

That command defaults to manylinux_2_28. Local development hosts that are not manylinux build environments can set RIFFDB_PYTHON_COMPATIBILITY=linux for a non-release smoke artifact. The pinned GitHub workflow builds and installs the x86_64 and aarch64 cp313-abi3 wheels on CPython 3.13 and 3.14; it never publishes them. Maintainers can run the same focused checks with:

./scripts/build-python-distributions --check
./scripts/test-python-platform-matrix

RiffDB MCP Agent Cookbook

RiffDB exposes policy-filtered database operations through MCP. Use tools/list after initialization to discover the commands authorized for the current credential and database. Tool names contain lowercase letters, digits, and underscores only.

Deployed Application Guide

When an active contract is visible, resources/list also exposes riffdb://application/guide. Reading it performs fresh authorization and renders the exact same-fence application catalog: active contract identity, visible generated operation names, schema and command links, and visible consistency surfaces. It is non-subscribable and never contains credentials, source text, hidden-operation counts, or inferred permissions. Hosted HTTP and stdio use the same renderer.

Values

• Send signed or unsigned integers as ordinary JSON integers when the generated schema says integer; RiffDB resolves signedness after selecting the operation schema. Tagged {"$i64":"2"} and {"$u64":"2"} remain accepted compatibility forms.
• Send fixed-scale decimals with the generated exact decimal object. Do not send a JSON floating-point number.
• Send UUID values as canonical lowercase strings, for example "01900000-0000-7000-8000-000000000001".
• Send enum variants as their declared strings, for example "Linear".
• Follow each tool’s inputSchema; undeclared properties are rejected.

Mutating Commands

Choose a fresh caller-owned idempotency_key for each logical command. Preserve that key until the result is known. Never generate a new key merely because a connection was interrupted.

A declared business rejection is a successful MCP tool call with a typed outcome. Branch on its declared outcome field rather than parsing text or treating it as a protocol failure.

If a connection fails after submission, the command may have committed. Retry the same command with the same input and idempotency key, or resolve the returned outcome URI. Reusing a key with different canonical input fails closed.

Discovery and Documentation

tools/list is the authorized catalog for the selected database and current credential. A missing command may be absent, stale, or unauthorized; do not infer which.

Use riffdb_application_catalog when an agent needs bounded symbolic introspection beyond the immediately callable tool list. It returns only authorized names, public types, source-relative spans, and closed feature availability. Page with its opaque lowercase cursor; do not decode it or retry an invalid cursor as a first-page request. The catalog intentionally never returns numeric compiler IDs, raw plans, capability details, storage keys, or hidden totals.

For a command resource:

• riffdb://command/<lineage>/<command-id>/docs is the invocation guide with inputs, a complete MCP call example, declared outcomes, and retry guidance.
• riffdb://command/<lineage>/<command-id>/plan is the compiler-oriented executable plan and generated schema detail.

Use /docs to call a command. Use /plan when inspecting exact compiler or dependency behavior.

Input Errors

Invalid tool arguments return one redacted riffdb.mcp.input-error/v1 diagnostic. Its path is a JSON Pointer, code classifies the first deterministic violation, and expected describes public schema shape. The error never echoes the submitted value.

Database Selection

Each MCP connection is bound to one configured database alias. Stdio selects it in mcp.toml; hosted MCP uses the riffdb-database header. Credentials are database-bound and cannot authorize another database.

Authenticated server_health and active-contract responses include the selected database alias; health also includes the configured authentication audience. Check these before deploying or mutating. To validate an stdio configuration without starting an MCP session, run:

riffdb-mcp doctor --config ~/.config/riffdb/mcp-ea.toml

doctor first uses authenticated health when the role permits it. A narrow application role need not gain health authority: the generated application MCP config retains the expected audience, and doctor instead proves the credential and database route through capability-filtered tool discovery.

The bootstrap MCP credential is an authoring identity: validation, catalog, deployment, and health. A separately bound application-role credential is the runtime identity for named queries and commands. Register a second MCP process with that application credential rather than widening the bootstrap identity.

Application-tool discovery is deliberately not an execution proof. It shows a compiled command only for an exact InvokeCommand permission and a named query only for an exact contract-lineage, query-module-hash, and query-name ExecuteNamedQuery permission. Tenant and partition scoping do not hide the operation because its actual partition is derived from invocation inputs. Each call resolves the tool again and performs fresh current-policy authorization against those inputs; revocation, scope, module substitution, and query-name substitution therefore fail closed after an earlier successful tool listing.

Deployed named-query tools use the compiler-owned <module_snake>_<query_snake> name, advertise the generated parameter and result schemas, and execute one named query in one snapshot. They never submit ad-hoc RiffQL or gain raw entity/index authority.

A named query with a compiler-declared Cursor or Cursor? uses the generated pagination result profile:

{"result":{"outcome":"Found","items":[]},"page":{"next_cursor":null}}

Read business fields only below result. When page.next_cursor is a string, pass that exact opaque 22-character value back in the cursor parameter named by the tool’s input schema (commonly after). A null cursor is the final page. Do not decode, modify, log, or substitute this value, and do not confuse it with the lowercase 32-character cursors used by MCP discovery and resources. Generated catalog V5 identifies this result profile. Older exact V2–V4 catalogs keep their unpaged flat results byte-for-byte; callers that select V5 must understand the envelope and fail locally on version skew. Invalid, stale, expired, mismatched, or denied continuations fail without structured content and without echoing the cursor.

Reactive operations

Reactive application roles may discover riffdb_event_next, riffdb_event_ack, riffdb_event_nack, riffdb_event_seek, riffdb_event_status, and riffdb_query_watch, filtered by their exact permissions. Tool names contain underscores rather than dots for broad MCP host compatibility.

Reactive parameters use the exact tagged scalar objects advertised by each tool schema. This prevents JSON number precision loss and preserves durable type identity. For example, UUID and signed-integer parameters are written as:

{
  "parameters": {
    "workspace_id": {
      "type": "uuid",
      "value": "01900000-0000-7000-8000-000000000001"
    },
    "after_sequence": { "type": "i64", "value": "42" }
  },
  "consumer_name": "Worker_1"
}

Decimals include their fixed precision and scale; enums include the compiler-pinned type_id, variant_id, and display name. Use the generated schema rather than guessing these identities.

Treat event delivery as at least once. Perform the intended idempotent work, then acknowledge with the exact event ID, lease token, and history incarnation returned by riffdb_event_next. A lease token is not authority and may be stale after retry, restore, or expiry. Seek requires separate administrative authority.

Persist the cursor from each applied riffdb_query_watch update. Snapshot and reset replace local state; patch applies its ordered closed operations; checkpoint advances the cursor without changing state; terminal requires the agent or application to clear retained query data and stop.

Subscribe to riffdb://reactive/wakeup for reactive work hints. Its resource body contains one opaque generation and notifications/resources/updated contains only that URI. Both are coalescible hints, never an event delivery, work count, commit sequence, or acknowledgement. Retrieve work through an authorized reactive tool and rely on the durable checkpoint or cursor for correctness. Fetch once immediately after subscribing; the first notification represents a later change, not the current queue state. See Reactive Application Clients.

Contextual work and reactions

Use riffdb_contextual_next for an agent subscription. One call returns at most one leased event, fresh named-query hydrations sharing one context_head, and the currently authorized declared reactions. Use only a reaction returned in that item.

Call riffdb_contextual_react with the exact reaction name and causation token from the item. RiffDB derives and binds the command idempotency value, validates the live lease and current authority, and records the causing event with the command provenance. If the response is uncertain, repeat the same reaction; do not invent a replacement command identity. An expired token can resolve an already committed exact reaction but cannot admit a new command.

After durable success, call riffdb_contextual_ack with the event ID, lease token, and history incarnation. Use riffdb_contextual_nack for a bounded retry and riffdb_contextual_status to inspect durable progress. A redelivered item has freshly recomputed context and may expose a different authorized reaction set. MCP notifications remain payload-free hints; they never consume or acknowledge contextual work.

RiffDB contract language

The complete parser reference is generated directly from the parser grammar. The crate-local and public copies are checked byte-for-byte in CI.

A contract is compiled before deployment. Parsing alone grants nothing and does not create an executable plan. Compilation must prove:

• every symbol and expression has one exact static type;
• every entity belongs to one aggregate and has a complete key;
• every command is confined to one statically derivable partition;
• mutating commands have one direct bounded-string idempotency input;
• relationship and uniqueness changes are validated transaction-current;
• required fields are definitely initialized and each mutation target is legal;
• every declared outcome and emitted event has one consistent typed shape; and
• all source, nesting, collection, schema, key, row, and plan bounds hold.

If any proof is missing, compilation fails with a stable source-spanned diagnostic. There is no fallback to a callback, SQL statement, unrestricted transaction, cross-partition write, or runtime-only integrity convention.

Initialized exact-key transitions

Use init_or_mutate when one named command must apply the same checked effects to an exact entity key whether the row is absent or present:

init_or_mutate Counter(organization_id, counter_id) as counter initialize {
    enabled: false,
    revision: 0,
}
set counter.enabled = true
set counter.revision = counter.revision + 1

For an absent row, RiffDB constructs a provisional record from the declared key and initializer, runs the common command suffix, and stages exactly one Create. For a present row, it ignores initializer values, runs that same suffix against the transaction-current preimage, and stages exactly one revision-checked Replace. Revalidation discards and reevaluates a candidate if concurrent work changes which state was observed. The command has one ordinary closed outcome surface; callers cannot choose the state path, conflict behavior, or overwrite policy.

Initializer expressions may use constants, command inputs, service-owned values, deterministic transaction context, and the current element of a bounded bulk command. They cannot read entity state, query results, storage, the operating-system clock, or randomness. An initializer may be partial or empty, but the compiler proves that every absent-path read is dominated by an initial value or common assignment and that every successful create postimage is complete.

The compiler requires the union of create and update authority and both row policy operation families before the command can be invoked. Static index and graph accounting charges the maximum of the possible create and replace work, never their sum; runtime charges only the selected mutation. Collection count, aggregate-byte, one-partition, duplicate-target, mutation, and index-work ceilings remain unchanged. This is a sealed exact-key transition, not a generic upsert: there are no caller-selected conflict targets, conditional arms, callbacks, raw predicates, or last-write-wins behavior.

Compiler-sealed decisions

Use observe_or_initialize (or a mutate or init_or_mutate binding) plus an immediately following decide when the same named command must atomically choose between applying a finite graph, accepting an exact no-effect transition, or rejecting the whole command:

observe_or_initialize TupleState(organization_id, tuple_key) as state
  initialize { active: false, revision: 0 }
decide state {
  when state.active == expected_active => apply {
    set state.active = requested_active
    set state.revision = state.revision + 1
  }
  when state.active == requested_active => no_effect
  else => reject StateConflict { tuple_key: tuple_key }
}

Arms are ordered, the first true when wins, and the mandatory else makes selection total. An apply finalizes the deferred row as exactly one create or revision-checked replace and executes only that arm’s compiler-known suffix (including any branch-local workflow transition or lease instructions). no_effect emits no entity, index, event, outbox, workflow, or changelog effect. reject discards every provisional effect from the ordinary or bulk command and persists its declared whole-command outcome.

The caller cannot select or observe an arm or absence/presence origin. RiffDB authorizes the union of all arms before reading state, then enforces the exact selected create/update row policy transaction-current. One through eight when arms are allowed; nested decisions, callbacks, dynamic targets, caller-selected predicates, and general branching are not. Bulk decisions use the existing element, aggregate-byte, graph, index-work, retry, and single-partition limits. Static effect cost is the maximum arm plus real union overhead, while runtime constructs and charges only the selected graph.

Source bounds

The authoritative constants live in the parser and compiler. The generated language reference is checked against the parser grammar in CI.

Command index-work bounds

Mutating plans prove secondary-index work before plan hashing. The compiler keeps physical index-entry removals and additions at 4,096, permits at most 65,535 exact affected prefix epochs and complete validation positions, and also requires their correlated charge to fit:

index-entry deltas + affected prefix epochs + validation positions <= 65,535

Affected targets and their current epoch observations must independently fit the 16 MiB command read-state ceiling. A bounded collection shares its proved partition prefix across elements, but the compiler assumes no equality among caller values, entity keys, other index fields, or different elements. When a collection declares aggregate_bytes, the same checked aggregate bound may limit copies of element-sourced bytes in index prefixes; count and work charges are never reduced by byte correlation.

An exceeded known ceiling is reported as RDB-C020 with a closed resource identity and the checked actual and maximum integers. These diagnostics contain compiler-owned plan metadata only—never submitted values, keys, stored rows, or backend details. There is no application-selectable work budget, runtime fallback, partial command, or automatic command split.

Author workflow

edit riffdb/contract.riff
riffdb application check
riffdb application lock --write
riffdb application lock --check

Authors use names. Stable IDs, hashes, command plans, capability requirements, field visibility, and generated transport code belong to the compiler-owned lock and generated tree.

See First application authoring for reserved identifiers, namespace rules, supported comments, natural CLI/MCP JSON values, and installed deployment inputs.

See the command and invariant cookbook for safe patterns and the Museum example for a complete multi-entity application.

Contract and application authoring reference

RiffDB contract and RiffQL source accept // line comments. Block comments are not supported.

The following lowercase words are reserved by the contract lexer and cannot be used as field or declaration identifiers:

active aggregate allow approval as attempts binary_utf8_v1 bool bytes capability
checked_by child claim columnar_projection command conflict_key contract count create date decimal
default delete duration_seconds else emit entity enum event
exhausted exists expire expired expires_at fact false fence fencing_token field
from frontier i64 idempotency_key illegal import in include index input invalid
initialize init_or_mutate observe_or_initialize decide apply no_effect reject invariant is key lease list measure module money mutate not null on optional
overwrite owner partition_by policy presence principal projection query read reference
release renew require return revision root row service set source stale
state state_machine string sum text_index text_key timestamp to
transaction_time transactionally_ordered transition true u64 unavailable
unicode_fold_v1 unique update uuid uuid_v7 vector_field version when where workflow

For example, use origin, origin_label, or source_label instead of the reserved field name source.

The vector search words cosine, euclidean, dot_product, staleness_slo, ann_threshold, and recall_target_bps are contextual, not reserved: they are meaningful only inside a vector_field declaration and remain usable as ordinary field and declaration identifiers everywhere else. The same is true of nearest in RiffQL — a contract field named nearest stays queryable.

Names are unique in their semantic namespace, not globally across an entire contract. Enum variants belong to their enum, command outcomes belong to their command, and indexes belong to their owning entity. A duplicate diagnostic identifies the current declaration and includes the related span of the first declaration.

Each set assignment needs initialization, a narrow same-field arithmetic update, an explicit checked_by requirement_name association, or a caller-version fence. Reading another field on the binding is insufficient. Use overwrite for deliberate replacement without those proofs. The write evidence cookbook explains the distinctions and limits of each proof.

Command declaration order and relationship proofs

Grammar-v1 command clauses follow one visible order: inputs and service values, the idempotency declaration, every read/mutate/create binding, requirements, effects such as set and emit, then return. Bind everything a requirement can observe before writing the requirement:

command AddComment {
    input request_key: string<128>
    input site_id: uuid
    input post_id: uuid
    input comment_id: uuid
    service created_at: transaction_time
    idempotency_key request_key

    read Post(site_id, post_id) as post else PostMissing {}
    create Comment(site_id, post_id, comment_id) as comment
        else CommentExists {}
    require PublishedPost: post.state == PostState.Published
        else PostNotPublished {}
    set comment.created_at = created_at
    return Created { comment: comment }
}

Every required relationship affected by a create or mutation needs a dominating exact read of that relationship’s complete target key. Reading one aggregate root does not prove a separate author, route, project, or other target exists. This is intentional: the compiled command proves all reference integrity in the same transaction instead of relying on a racy application preflight.

Choosing a partition route

partition_by and conflict_key are separate decisions and are routinely confused. Getting the distinction wrong produces a contract that compiles and is quietly unusable, so it is worth stating plainly.

conflict_key is what writers contend on. Two commands whose conflict keys differ do not serialise against each other, whatever partition they are in.

partition_by is the locality and visibility unit. It decides three things:

• what one index scan or bounded query can see, because a scan is scoped to one partition;
• which entities a single command may write, since every create/mutate binding must derive the same route; and
• what a capability may be scoped to.

The practical consequence is that a partition should be as wide as the widest thing you need to search or write atomically, and the conflict key should then be as narrow as the thing you actually update. Widening a partition does not cost concurrency; narrowing a conflict key is what buys it.

Two failure modes follow directly, and both are silent:

• A route that identifies a single row makes that entity’s indexes useless. If Item keys (item_id) and the aggregate declares partition_by item_id, every item is alone in its partition, so index by_created (...) can only ever return that one item. There is no way to list items. If a collection has to be searched or ordered as a whole, partition it by the scope it is searched within — a tenant, an organization, or a fixed scope constant for a genuinely global collection — and keep the per-row identifier in the conflict key.
• A unique key is enforced inside one partition, which is why RDB-C025 requires it to begin with the partition route. Prefixing the route makes a value unique per route, which is vacuous when the route already identifies one row. A value that must be unique across a wider scope needs the aggregate partitioned by that scope; see the entry under “Uniqueness” below.

A fixed scope constant is a legitimate route. An entity partitioned by scope: string<32> always set to "global" is one searchable collection whose writers still do not contend, provided the conflict key carries the row identifier.

Partition-local cross-aggregate writes

A command may write roots owned by different aggregates when every binding derives the same complete partition route. Keep the aggregates’ conflict keys independent; RiffDB acquires the union in its compiler-owned order and commits the mutations atomically. For example, inventory and an order may remain separate aggregates while one checkout writes both, provided both aggregate declarations say partition_by tenant_id and both bindings use the same tenant_id input.

Do not copy the partition value through a different expression or omit part of a composite route. RDB-C017 means the compiler could not prove the routes identical. Make each create and mutate key derive the complete same route, or split a genuinely cross-partition operation into independently idempotent commands. A cross-aggregate command does not make cross-partition transactions available.

Uniqueness

unique is enforced by a durable occupancy row inside the writing command’s partition, transaction-current, in the same commit. That makes it exact and race-free, and it is why the key must start with the partition route.

To make a value unique across every row of a collection, partition the aggregate by the scope the value is unique in and include that scope in the unique key:

entity Experiment {
  key (scope: string<32>, experiment_id: u64)
  field name: string<500>
  unique by_name (scope, name)
}
aggregate Experiments {
  root Experiment
  partition_by scope
  conflict_key (scope, experiment_id)
}

Creates and renames are then both checked atomically by the engine. Concurrent writes to different experiments do not contend, because the conflict key carries experiment_id.

Boolean row policies

A row-policy predicate can use a Boolean constant, row field, or declared principal fact directly, including under !, &&, and ||. For example, allow create when true permits creation when the other capability, scope, and command checks allow it. An update must satisfy its policy for both the current and proposed row. Missing fields, mistyped facts, and missing required relationship evidence still deny the operation; values are never coerced to Booleans.

Upgrade note: the row-policy evaluator corrects an earlier defect that denied direct Boolean predicates even though they compiled. Such policies now apply their declared true/false result and may admit rows that the defective evaluator denied.

Delete policy

Every deletable entity declares what happens to inbound relationships. Choose the policy from the relationship graph, not from the command that happens to delete the row:

• delete_policy no_inbound is valid only when no declared relationship can target the entity.
• delete_policy restrict Child.by_parent names the complete partition-local reverse index used to prove there are no referencing rows. The delete command also declares restrict Referenced {} so a reference produces a typed zero-mutation outcome.
• delete_policy cascade { relationship Child.parent using Child.by_parent maximum 32 } enumerates every direct inbound relationship and its fixed fan-out maximum. Cascade is one bounded hop; recursive discovery is never substituted.

The reverse index starts with the target’s partition and key fields. Child entities reached by a cascade declare no_inbound, and the command must use the bounded collection-delete surface described in Bounded Commands. RDB-C045 means the declared policy is incomplete or inconsistent with that graph; add the missing reverse-index proof or choose the policy that matches the actual inbound relationships.

Composing two indexed predicates

RiffQL has no general join, but one ordered dependent batch composes two bounded index reads into an intersection: read the first predicate, then read the second restricted to the first’s keys. A candidate with no matching second row simply produces no row, which is the intersection.

The planner accepts this only in an exact shape, and RDB-QP007 reports the whole shape rather than the part that failed. The conditions most often missed:

• Every comparison in the dependent binding must be on key fields. A predicate over a non-key field is a residual filter and is refused. If you need to match on a value that is not in the key, put a fixed-size digest of it in the key and compare that.
• The dependent binding needs an absence outcome (else <Outcome>). For a many binding it never fires — an empty match is zero rows, not an outcome — but the planner requires it to be declared.
• Both bindings order ascending by exactly the joining field, and the dependent binding’s take must not carry an after cursor and must be no larger than the source’s.

Operational query indexes

Indexes used by null/existence and text-prefix RiffQL predicates declare their physical semantics in the contract:

entity Document {
    key (organization_id: uuid, document_id: uuid)
    field deleted_at: optional<timestamp>
    field title: string<200>

    index by_deleted (organization_id, deleted_at, document_id)
        presence(deleted_at)
    index by_title (organization_id, title, document_id)
        text_key(title, binary_utf8_v1)
}

presence is valid only for an optional authoritative key scalar and preserves missing, explicit null, and non-null as distinct index states. text_key is valid only for a bounded string field. binary_utf8_v1 compares exact UTF-8 bytes; it is case-sensitive and performs no Unicode normalization. The unicode_fold_v1 profile applies the frozen Unicode 17.0.0 NFKC plus full non-Turkic case fold for folded-byte equality, membership, range, prefix, and ordering. Adding or changing either encoding changes durable index identity and requires the contract migration/rebuild path; RiffDB never silently changes a text profile during a software upgrade.

An optional field is not an ordinary index key. If a query filters or orders on an optional field, declare presence(field) as shown above so missing, explicit-null, and present values remain distinct and the compiler can prove the access shape.

Tokenized text indexes

An entity may declare a bounded, weighted tokenized-text index over required or optional bounded string fields:

entity Document {
    key (organization_id: uuid, document_id: uuid)
    field title: string<256>
    field body: optional<string<65536>>

    text_index search((title weight 4, body weight 1),
        analyzer standard_v1,
        staleness_slo 60,
        replay_age_seconds 86400,
        replay_bytes 1073741824,
        replay_backlog 100000,
        result boolean_v1,
        max_terms 16,
        max_candidates 10000,
        max_results 1000)
}

Every clause is required and compiler owned. Source names must be distinct; weights and all budgets must be positive and within the fixed format maxima; max_results cannot exceed max_candidates. Callers cannot replace these values, name a field or analyzer, provide boosts, request a scan, or choose a consistency fallback at runtime.

The v1 analyzer set is closed:

• keyword_v1 emits the complete source value as exactly one term, including one empty term for an empty value. It performs no normalization.
• standard_v1 first applies Unicode 17.0.0 UAX 29 word segmentation, assigns monotonically increasing positions, and then applies the deterministic unicode_fold_v1 compatibility fold to each term. Analyzer behavior is pinned by checked-in conformance tests rather than changing with the host.

Two consequences of UAX 29 segmentation are worth knowing before you choose standard_v1, because both are pinned behaviour rather than defects:

• Scripts without spacing segment per character. UAX 29 carries no dictionary, so 東京大学 analyzes to four single-character terms rather than to words. Search over such text is therefore effectively per-character. If a field is predominantly CJK and you need whole-value matching, keyword_v1 is the honest choice until a dictionary-backed analyzer is declared.
• A value can analyze to no terms at all. A value made only of punctuation, whitespace, or emoji produces an empty term list, so that row cannot be found through this index by any query. It is still authoritative and still readable by key; it is simply absent from the tokenized index.

The declaration selects boolean_v1; public named tokenized queries compile bounded matching(...) predicates (conjunction, disjunction, phrase, and ordered proximity) and optional order by relevance desc using the fixed riff_bm25_v1 ranking model (caller-supplied boosts and raw floating-point scores are not exposed). The stale-entity and replay ceilings are durable contract identity for the tokenized provider stage. Changing an analyzer or field weight changes the contract bundle identity. See Tokenized Text Search.

Explicit covering indexes

A new index identity may declare bounded authoritative fields that are stored atomically with each index entry:

index by_board_project_status
    (organization_id, project_id, status, ticket_id)
    cover (title, reporter_id, assignee_id)

The compiler may use this index for a named query only when its key plus cover contains every predicate, ordering, and selected field and the complete result has one statically bounded positional layout. The application still names only the query; it cannot select the index, request a partial cover, provide field ordinals, or disable validation. Covered bytes are derived from the same authoritative post-image as the index key and commit atomically with it.

cover (...) is intentionally explicit and ordered. Adding it to an existing index is not an in-place optimization: add a new index identity and use the normal migration/rebuild path. Missing, stale, malformed, duplicate, or non-derivable covered values fail closed; the executor never falls back to entity reads after selecting a covered plan. Unbounded, secret, unsupported, or policy-incompatible layouts remain on the ordinary checked result path.

Vector fields (alpha)

An entity may declare one or more fixed-dimension embedding fields:

entity Document {
    key (org_id: uuid, doc_id: uuid)
    field title: string<256>
    field body: string<65536>
    vector_field embedding(1536, cosine, (title, body), staleness_slo 60,
        model "text-embedding-3-small", current_version "2026-08-21",
        replay_age_seconds 86400, replay_bytes 1073741824,
        replay_backlog 100000,
        ann_threshold 256, recall_target_bps 9500)
}

The declaration carries the search configuration into the compiled contract bundle, where it is part of the contract’s durable identity:

• The dimension is a positive integer of at most 4,096.
• The metric is exactly one of cosine, euclidean, or dot_product.
• The source fields name 1 to 1,024 distinct existing fields on the same entity — the fields whose edits make a stored embedding stale. A repeated name or the vector field itself is a compile error at its span.
• staleness_slo is the positive v1 stale-entity count threshold. A staleness observer breaches the SLO only when stale_count > staleness_slo; equality does not breach it. Duration-based or clock-based staleness is a named future amendment, not a v1 interpretation of this declaration.
• The production clause is atomic: model, current_version, replay_age_seconds, replay_bytes, and replay_backlog must all appear in that order or all be absent. Model strings are nonempty and at most 256 bytes. Replay limits are positive and compiler-bounded (one year, one TiB, and 100,000,000 sequences respectively). They are projection retention ceilings, not request parameters.
• Omitting the production clause preserves legacy exact-vector bundle bytes, but that field cannot back a production nearest query. A production clause selects contract IR V15; earlier vector and ANN records remain byte-frozen.
• The optional ANN clause is atomic: ann_threshold and recall_target_bps must appear together in that order. The threshold is 1 to 65,536 rows per organization. Search remains exact at or below the threshold. Above it, a matching warm graph may engage; a cold or unavailable graph uses the exact path.
• recall_target_bps is 1 to 10,000 basis points. For example, 9500 declares recall@K of at least 0.95 against exact scan at the same projection frontier. Application query inputs cannot lower or omit this projection-owned target.

The approximate graph is derived, bounded by the query’s organization partition and scan budget, and rebuilt deterministically from the visible snapshot in a bounded background build. Primary and follower providers may reuse an exact population match at the same generation and frontier. Graph entry points and links are computed only after scalar filters and principal-policy admission, so another organization or a denied row cannot shape traversal or reported graph statistics. Every cache hit repeats current admission. Exact KNN remains the reference and cold fallback path.

Production vector fields are written only through the compiler-sealed embed effect; generic set is rejected at the target field:

command SetDocumentEmbedding {
    input request_id: string<128>
    input org_id: uuid
    input doc_id: uuid
    input embedding: vector<1536>
    input submitted_model: string<256>
    input submitted_version: string<256>

    idempotency_key request_id
    mutate Document(org_id, doc_id) as document else Missing {}

    embed document.embedding = embedding
        from (submitted_model, submitted_version)

    return Embedded { document: document }
}

The vector dimension is part of the input type. Model identity and version are distinct direct command inputs and must exactly match the field’s compiled production declaration; a mismatch is rejected before admission with the symbolic input path. The entity post-image and its authoritative embedding evidence (model identity, model version, and commit sequence) are derived from the same checked command and persist atomically with its outcome, provenance, and idempotency record. An application cannot submit evidence separately or use generic field mutation to bypass it.

Typed vector values cross the native, Protobuf, gRPC, hosted MCP, CLI, and generated Rust, Go, TypeScript, and Python conversion boundaries with finite-component and exact-dimension checks. Each generated embedding command also has a field-specific constructor that fills the contract-sealed model identity and version; generated constants and accessors keep that evidence inspectable without asking application code to duplicate it. Direct input construction remains checked by the same server authority. The production application surface also exposes bounded symbolic stale/outdated inspection for the exact vector field when its role declares that authority. The authoritative observer maintains the distinct vector_staleness health component without scanning entities during a probe. Production exact-nearest queries use an explicit projected source and freshness declaration. The running server synchronizes compiler-owned vector sources after checked contract activation, including the first deployment into an empty database; no daemon restart is required. A successor that would reuse incompatible derived state allocates a distinct durable generation and fails closed until its rebuild publishes a complete replacement. Rebuild captures one bounded authoritative entity snapshot, applies the retained commit tail, checkpoints the candidate, and only then publishes Ready; a partial snapshot is never queryable. The compiler-owned replay age, byte, and backlog ceilings detach an over-budget generation from retention and trigger the same snapshot-plus-tail path. Building, RebuildRequired, and Rebuilding are typed temporary states, not permission to serve the previous generation. Only Ready controls hold commit-log retention. A crash or restart resumes from the durable control record without claiming a frontier beyond the exact checkpoint. See the RiffQL language reference.

Revision-checked workflow transitions

A workflow declaration closes the legal state graph for one aggregate-owned entity. A transition effect must name a mutable binding and an exact revision supplied by the caller:

workflow WorkLifecycle {
    entity WorkItem
    state state
    transition Start from (Queued) to Running
}

command StartWork {
    input request_key: string<128>
    input organization_id: uuid
    input work_id: uuid
    input expected_revision: u64
    idempotency_key request_key

    mutate WorkItem(organization_id, work_id) as work else Missing {}
    transition Start on work revision expected_revision
        stale StaleRevision {}
        illegal IllegalState {}
    return Started { work: work }
}

RiffDB compares expected_revision with the observed entity revision before it checks the source state. A mismatch returns the declared StaleRevision outcome; a matching revision in any state outside Start’s source set returns IllegalState. Both are ordinary idempotent zero-mutation command outcomes. The legal path assigns the declared destination and retains the exact entity version as a transaction-current read dependency, so a concurrent replacement cannot turn the checked result into a lost update at commit.

There is no unconditional workflow update, automatic substitution of a newer revision, generic compare-and-swap, or caller-supplied transition name. The current executable surface covers revision-checked declared transitions plus sealed service-owned values:

command StartWork {
    input organization_id: uuid
    input work_id: uuid
    input expected_revision: u64
    service execution_id: uuid_v7
    service started_at: transaction_time
    idempotency_key request_key

    mutate WorkItem(organization_id, work_id) as work else Missing {}
    transition Start on work revision expected_revision
        stale StaleRevision {}
        illegal IllegalState {}
    set work.execution_id = execution_id
    set work.started_at = started_at
    return Started { work: work }
}

transaction_time is the logical time sampled for durable admission; uuid_v7 is generated by the service before deterministic evaluation. Neither value is a caller input, and the runtime receives the values without clock, randomness, filesystem, or network authority. RiffDB stores the canonical service-value record with pending and terminal command evidence and retains it inside the authoritative command segment. A retry, uncertain-outcome recovery, or restart therefore returns and reconstructs the originally sealed values rather than sampling replacements. Service values consequently require a durably idempotent mutation command; unjournaled read-only commands cannot declare them.

For fenced workflow leases (lease claim, renew, release, expire, and fence), see Workflow Transitions and Fenced Leases.

CLI application JSON

The application CLI accepts inline JSON, @path, a legacy bare path, or - for stdin:

riffdb command run CreateItem --input \
  '{"item_id":"01900000-0000-7000-8000-000000000001","priority":2,"mode":"Linear"}'

riffdb command run CreateItem --input @fixtures/item.json
riffdb command run CreateItem --input fixtures/item.json
riffdb command run CreateItem --input - < fixtures/item.json

Natural schema-directed values are:

The explicit $i64, $u64, $uuid, $enum, $decimal, $money, $bytes, $date, and $timestamp objects remain compatibility forms for lossless transport or fixture work. Public type failures name the operation and symbolic path when that context is available. They never release numeric field IDs through the application error envelope; numeric IDs remain confined to the explicitly low-level kernel interface.

Successor contract evolution

Every changed deployment must declare a contract version greater than the active version and must compare against the exact expected active version. Version numbers alone do not establish compatibility; the compiler compares the checked successor with the active bundle.

For application source trees, application lock --write obtains that exact parent through the authorized read-only candidate preview. The resulting lock pins both the parent identity and generated/riffdb.contract.bundle. Never manufacture a successor lock with an offline genesis compile or copy a hash from an error.

The pre-alpha compatibility classes are:

The complete initial definition of a newly added entity or aggregate is treated as one addition. A new aggregate cannot take ownership of an existing entity. These restrictions ensure that a compatible successor does not scan, rewrite, or reinterpret an existing durable row.

Use the application workflow when deploying a locked source tree:

riffdb application check
riffdb application lock --write
riffdb application generate --locked
riffdb application deploy

For a direct contract deployment, validate before deploying and supply the active version expected by that CLI version:

riffdb --config "$HOME/.config/riffdb/client-ea.toml" \
  contract validate riffdb/contract.riff
riffdb --config "$HOME/.config/riffdb/client-ea.toml" \
  contract deploy --expected-version 3 riffdb/contract.riff

An invalid source returns bounded syntax or semantic diagnostics. A checked but incompatible successor returns incompatible_candidate with its parent, overall compatibility class, and stable compatibility-code counts. Neither result activates the candidate. Do not work around incompatibility by reusing stable IDs or editing the active database file.

A valid successor that needs existing state to be checked or transformed returns migration_required. Use Application Source V3, a parent-specific .riffm proof, Application Lock V4, and the read-only plan described in Contract Migrations. Offline check, apply, and status operations activate that successor through the migration CLI and operator service.

contract deploy is a mutating operator command, not a compatibility probe. There is no contract rollback RPC. Use contract validate for source-only validation or application lock --write for an exact read-only successor preview, then review the lock before application deploy. If a direct deploy activates an unwanted successor, stop the service and follow the documented offline backup/restore or disposable pre-alpha reset procedure; do not deploy another probe hoping to undo it.

RiffDB Contract Language Reference v26

This is the complete grammar-version-26 parser source. Grammar v26 adds contextual mutate first selection under statically proved pool ownership. Normative semantics, bounds, compatibility gates, and the prohibition on general loops and raw transactions are defined by ADR-0002 as amended by ADR-0015, ADR-0107, ADR-0109, ADR-0133, ADR-0136, ADR-0153, ADR-0173, ADR-0256, ADR-0262, and ADR-0264. Semantic actions below construct only the source-oriented AST; they are not typed IR, an executable plan, or a durable encoding.

Complete Grammar

use crate::ast::*;
use crate::diagnostic::SyntaxDiagnostic;
use crate::lexer::Token;
use crate::parser::{self, ParsedExpression};
use crate::span::Spanned;

grammar;

pub ContractDocument: ContractDocument = {
    <contract:ContractDeclaration> => ContractDocument { contract },
};

ContractDeclaration: Spanned<Contract> = {
    <lo:@L> "contract" <name:Identifier> "version" <version:UnsignedInteger>
        "{" <declarations:Declaration*> "}" <hi:@R>
        => parser::spanned(Contract { name, version, declarations }, lo, hi),
};

Declaration: Spanned<Declaration> = {
    <lo:@L> <entity:EntityDeclaration> <hi:@R>
        => parser::spanned(Declaration::Entity(entity), lo, hi),
    <lo:@L> <event:EventDeclaration> <hi:@R>
        => parser::spanned(Declaration::Event(event), lo, hi),
    <lo:@L> <enumeration:EnumDeclaration> <hi:@R>
        => parser::spanned(Declaration::Enum(enumeration), lo, hi),
    <lo:@L> <aggregate:AggregateDeclaration> <hi:@R>
        => parser::spanned(Declaration::Aggregate(aggregate), lo, hi),
    <lo:@L> <command:CommandDeclaration> <hi:@R>
        => parser::spanned(Declaration::Command(command), lo, hi),
    <lo:@L> <workflow:WorkflowDeclaration> <hi:@R>
        => parser::spanned(Declaration::Workflow(workflow), lo, hi),
    <lo:@L> <projection:ProjectionDeclaration> <hi:@R>
        => parser::spanned(Declaration::Projection(projection), lo, hi),
    <lo:@L> <columnar:ColumnarProjectionDeclaration> <hi:@R>
        => parser::spanned(Declaration::ColumnarProjection(columnar), lo, hi),
    <lo:@L> <fact:PrincipalFactDeclaration> <hi:@R>
        => parser::spanned(Declaration::PrincipalFact(fact), lo, hi),
    <lo:@L> <policy:RowPolicyDeclaration> <hi:@R>
        => parser::spanned(Declaration::RowPolicy(policy), lo, hi),
};

PrincipalFactDeclaration: PrincipalFactDeclaration = {
    "principal" "fact" <name:Identifier> ":" <ty:TypeExpression>
        => PrincipalFactDeclaration { name, ty },
};

RowPolicyDeclaration: RowPolicyDeclaration = {
    "row" "policy" <name:Identifier> "on" <entity:Identifier>
        "{" <rules:RowPolicyRule+> "}"
        => RowPolicyDeclaration { name, entity, rules },
};

RowPolicyRule: Spanned<RowPolicyRule> = {
    <lo:@L> "allow" <operation:RowPolicyOperation> "when"
        <expression:RowPolicyExpression> <hi:@R>
        => parser::spanned(RowPolicyRule { operation, expression: expression.into_syntax() }, lo, hi),
};

RowPolicyOperation: Spanned<RowPolicyOperation> = {
    <lo:@L> "read" <hi:@R> => parser::spanned(RowPolicyOperation::Read, lo, hi),
    <lo:@L> "create" <hi:@R> => parser::spanned(RowPolicyOperation::Create, lo, hi),
    <lo:@L> "update" <hi:@R> => parser::spanned(RowPolicyOperation::Update, lo, hi),
    <lo:@L> "delete" <hi:@R> => parser::spanned(RowPolicyOperation::Delete, lo, hi),
};

RowPolicyExpression: parser::ParsedRowPolicyExpression = RowPolicyOrExpression;

RowPolicyOrExpression: parser::ParsedRowPolicyExpression = {
    <left:RowPolicyAndExpression> <rest:(<OrOperator> <RowPolicyAndExpression>)*>
        =>? parser::grammar_result(parser::row_policy_binary_chain(left, rest)),
};

RowPolicyAndExpression: parser::ParsedRowPolicyExpression = {
    <left:RowPolicyPredicate> <rest:(<AndOperator> <RowPolicyPredicate>)*>
        =>? parser::grammar_result(parser::row_policy_binary_chain(left, rest)),
};

RowPolicyPredicate: parser::ParsedRowPolicyExpression = {
    <left:RowPolicyUnary> <operator:ComparisonOperator> <right:RowPolicyUnary>
        =>? parser::grammar_result(parser::row_policy_binary(left, operator, right)),
    <needle:RowPolicyUnary> "in" <haystack:RowPolicyUnary>
        =>? parser::grammar_result(parser::row_policy_in(needle, haystack)),
    <value:RowPolicyUnary> "is" <negated:("not")?> "null"
        =>? parser::grammar_result(parser::row_policy_is_null(value, negated.is_some())),
    RowPolicyUnary,
};

RowPolicyUnary: parser::ParsedRowPolicyExpression = {
    <lo:@L> "!" <value:RowPolicyUnary> <hi:@R>
        =>? parser::grammar_result(parser::row_policy_not(value, parser::span(lo, hi))),
    RowPolicyPrimary,
};

RowPolicyPrimary: parser::ParsedRowPolicyExpression = {
    <literal:Literal> => parser::row_policy_literal(literal),
    <path:RowPolicyPath> => parser::row_policy_path(path),
    <lo:@L> "(" <expression:RowPolicyExpression> ")" <hi:@R>
        =>? parser::grammar_result(parser::row_policy_parenthesized(expression, parser::span(lo, hi))),
    <lo:@L> "exists" <entity:Identifier> "." <index:Identifier>
        "(" <arguments:RowPolicyArgumentList> ")" <hi:@R>
        =>? parser::grammar_result(parser::row_policy_exists(entity, index, arguments, parser::span(lo, hi))),
};

RowPolicyArgumentList: Vec<parser::ParsedRowPolicyExpression> = {
    <head:RowPolicyExpression> <tail:("," <RowPolicyExpression>)*> <trailing:Comma?> => {
        let _ = trailing;
        let mut values = vec![head];
        values.extend(tail);
        values
    },
};

RowPolicyPath: Spanned<Path> = {
    <lo:@L> <head:RowPolicyPathSegment> <tail:("." <RowPolicyPathSegment>)*> <hi:@R> => {
        let mut segments = vec![head];
        segments.extend(tail);
        parser::spanned(Path { segments }, lo, hi)
    },
};

RowPolicyPathSegment: Spanned<String> = {
    Identifier,
};

WorkflowDeclaration: WorkflowDeclaration = {
    "workflow" <name:Identifier> "{"
        "entity" <entity:Identifier>
        "state" <state_field:Identifier>
        <initial_state:("initial" <Identifier>)?>
        <transitions:WorkflowTransitionDeclaration+>
        <lease:WorkflowLeaseDeclaration?>
    "}" => WorkflowDeclaration { name, entity, state_field, initial_state, transitions, lease },
};

WorkflowTransitionDeclaration: Spanned<WorkflowTransitionDeclaration> = {
    <lo:@L> "transition" <name:Identifier> "from" "(" <source_states:IdentifierList> ")"
        "to" <destination:Identifier> <hi:@R>
        => parser::spanned(WorkflowTransitionDeclaration {
            name,
            source_states,
            destination,
        }, lo, hi),
};

WorkflowLeaseDeclaration: Spanned<WorkflowLeaseDeclaration> = {
    <lo:@L> "lease" <name:Identifier> "{"
        "owner" <owner_field:Identifier>
        "expires_at" <expiry_field:Identifier>
        "fencing_token" <fencing_token_field:Identifier>
        <attempt_field:WorkflowLeaseAttempts?>
        "duration_seconds" "(" <minimum_duration_seconds:UnsignedInteger> ","
            <maximum_duration_seconds:UnsignedInteger> ")"
    "}" <hi:@R> => parser::spanned(WorkflowLeaseDeclaration {
        name,
        owner_field,
        expiry_field,
        fencing_token_field,
        attempt_field,
        minimum_duration_seconds,
        maximum_duration_seconds,
    }, lo, hi),
};

WorkflowLeaseAttempts: Spanned<String> = {
    "attempts" <field:Identifier> => field,
};

EntityDeclaration: EntityDeclaration = {
    "entity" <name:Identifier> "{" <items:EntityItem*> "}"
        => EntityDeclaration { name, items },
};

EntityItem: Spanned<EntityItem> = {
    <lo:@L> "key" "(" <fields:TypedFieldList> ")" <hi:@R>
        => parser::spanned(EntityItem::Key(KeyDeclaration { fields }), lo, hi),
    <lo:@L> "field" <field:TypedField> <hi:@R>
        => parser::spanned(EntityItem::Field(FieldDeclaration {
            secret: None,
            name: field.name,
            ty: field.ty,
        }), lo, hi),
    // Contextual: the classification modifier is an ordinary identifier
    // everywhere except between `field` and the field name, so `field
    // secret: ...` stays a valid field named `secret` (ADR-0118 additive
    // compatibility).
    <lo:@L> "field" <modifier:Identifier> <field:TypedField> <hi:@R>
        =>? parser::grammar_result(parser::classified_field_item(modifier, field, lo, hi)),
    <lo:@L> <invariant:InvariantDeclaration> <hi:@R>
        => parser::spanned(EntityItem::Invariant(invariant), lo, hi),
    <lo:@L> "index" <name:Identifier> "(" <fields:IdentifierList> ")"
        <options:IndexOption*> <hi:@R>
        => parser::spanned(EntityItem::Index(IndexDeclaration { name, fields, options }), lo, hi),
    <lo:@L> "unique" <name:Identifier> "(" <fields:IdentifierList> ")" <hi:@R>
        => parser::spanned(EntityItem::Unique(UniqueDeclaration { name, fields }), lo, hi),
    <lo:@L> "reference" <name:Identifier> "(" <source_fields:IdentifierList> ")"
        "->" <target_entity:Identifier> "(" <target_fields:IdentifierList> ")" <hi:@R>
        => parser::spanned(EntityItem::Reference(ReferenceDeclaration {
            name,
            source_fields,
            target_entity,
            target_fields,
        }), lo, hi),
    <lo:@L> "vector_field" <name:Identifier> "(" <dimension:UnsignedInteger> ","
        <metric:VectorMetric> "," "(" <source_fields:IdentifierList> ")" ","
        <slo_keyword:Identifier> <staleness_slo:UnsignedInteger>
        <production:VectorProductionClause?>
        <ann:VectorAnnClause?> ")" <hi:@R>
        =>? parser::grammar_result(parser::vector_field_item(
            name,
            dimension,
            metric,
            source_fields,
            slo_keyword,
            staleness_slo,
            production,
            ann,
            lo,
            hi,
        )),
    <lo:@L> "text_index" <name:Identifier> "(" "(" <source_fields:TextIndexSourceList> ")"
        "," <analyzer_keyword:Identifier> <analyzer:Identifier>
        "," <slo_keyword:Identifier> <staleness_slo:UnsignedInteger>
        "," <age_keyword:Identifier> <replay_age_seconds:UnsignedInteger>
        "," <bytes_keyword:Identifier> <replay_bytes:UnsignedInteger>
        "," <backlog_keyword:Identifier> <replay_backlog:UnsignedInteger>
        "," <result_keyword:Identifier> <result_model:Identifier>
        "," <terms_keyword:Identifier> <max_terms:UnsignedInteger>
        "," <candidates_keyword:Identifier> <max_candidates:UnsignedInteger>
        "," <results_keyword:Identifier> <max_results:UnsignedInteger>
        ")" <hi:@R>
        =>? parser::grammar_result(parser::text_index_item(
            name,
            source_fields,
            analyzer_keyword,
            analyzer,
            slo_keyword,
            staleness_slo,
            age_keyword,
            replay_age_seconds,
            bytes_keyword,
            replay_bytes,
            backlog_keyword,
            replay_backlog,
            result_keyword,
            result_model,
            terms_keyword,
            max_terms,
            candidates_keyword,
            max_candidates,
            results_keyword,
            max_results,
            lo,
            hi,
        )),
    <lo:@L> "pattern_index" <name:Identifier> "(" <field:Identifier>
        "," <profile_keyword:Identifier> <profile:Identifier>
        "," <operators_keyword:Identifier> "(" <operators:IdentifierList> ")"
        "," <bounds:LongPatternBounds> ")" <hi:@R>
        =>? parser::grammar_result(parser::long_pattern_item(
            name,
            field,
            profile_keyword,
            profile,
            operators_keyword,
            operators,
            bounds,
            lo,
            hi,
        )),
    <lo:@L> "delete_policy" "no_inbound" <hi:@R>
        => parser::spanned(EntityItem::DeletePolicy(DeletePolicyDeclaration::NoInbound), lo, hi),
    <lo:@L> "delete_policy" "restrict" <source_entity:Identifier> "." <index:Identifier> <hi:@R>
        => parser::spanned(EntityItem::DeletePolicy(DeletePolicyDeclaration::Restrict {
            source_entity,
            index,
        }), lo, hi),
    <lo:@L> "delete_policy" "cascade" "{" <relationships:CascadeRelationship+> "}" <hi:@R>
        => parser::spanned(EntityItem::DeletePolicy(DeletePolicyDeclaration::Cascade {
            relationships,
        }), lo, hi),
};

LongPatternBounds: Vec<LongPatternBoundDeclaration> = {
    <head:LongPatternBound> <tail:("," <LongPatternBound>)*> => {
        let mut values = vec![head];
        values.extend(tail);
        values
    },
};

LongPatternBound: LongPatternBoundDeclaration = {
    <name:Identifier> <value:UnsignedInteger> => LongPatternBoundDeclaration { name, value },
};

TextIndexSourceList: Vec<Spanned<TextIndexSourceDeclaration>> = {
    => Vec::new(),
    <head:TextIndexSource> <tail:("," <TextIndexSource>)*> <_trailing:Comma?> => {
        let mut values = vec![head];
        values.extend(tail);
        values
    },
};

TextIndexSource: Spanned<TextIndexSourceDeclaration> = {
    <lo:@L> <field:Identifier> <weight_keyword:Identifier> <weight:UnsignedInteger> <hi:@R>
        => parser::spanned(TextIndexSourceDeclaration { field, weight_keyword, weight }, lo, hi),
};

VectorProductionClause: VectorProductionDeclaration = {
    "," <model_keyword:Identifier> <model_identity:StringLiteral>
        "," <version_keyword:Identifier> <current_model_version:StringLiteral>
        "," <age_keyword:Identifier> <replay_age_seconds:UnsignedInteger>
        "," <bytes_keyword:Identifier> <replay_bytes:UnsignedInteger>
        "," <backlog_keyword:Identifier> <replay_backlog:UnsignedInteger>
        =>? parser::grammar_result(parser::vector_production_declaration(
            model_keyword,
            model_identity,
            version_keyword,
            current_model_version,
            age_keyword,
            replay_age_seconds,
            bytes_keyword,
            replay_bytes,
            backlog_keyword,
            replay_backlog,
        )),
};

CascadeRelationship: Spanned<CascadeRelationshipDeclaration> = {
    <lo:@L> "relationship" <source_entity:Identifier> "." <relationship:Identifier>
        "using" <index_entity:Identifier> "." <index:Identifier>
        "maximum" <maximum:UnsignedInteger> <hi:@R>
        => parser::spanned(CascadeRelationshipDeclaration {
            source_entity,
            relationship,
            index_entity,
            index,
            maximum,
        }, lo, hi),
};

VectorAnnClause: VectorAnnDeclaration = {
    "," <threshold_keyword:Identifier> <row_threshold:UnsignedInteger>
        "," <recall_keyword:Identifier> <recall_target_bps:UnsignedInteger>
        =>? parser::grammar_result(parser::vector_ann_declaration(
            threshold_keyword,
            row_threshold,
            recall_keyword,
            recall_target_bps,
        )),
};

IndexOption: Spanned<IndexOption> = {
    <lo:@L> "cover" "(" <fields:IdentifierList> ")" <hi:@R>
        => parser::spanned(IndexOption::Cover { fields }, lo, hi),
    <lo:@L> "presence" "(" <field:Identifier> ")" <hi:@R>
        => parser::spanned(IndexOption::Presence { field }, lo, hi),
    <lo:@L> "text_key" "(" <field:Identifier> "," <profile:TextKeyProfile> ")" <hi:@R>
        => parser::spanned(IndexOption::TextKey { field, profile }, lo, hi),
};

TextKeyProfile: Spanned<TextKeyProfile> = {
    <lo:@L> "binary_utf8_v1" <hi:@R>
        => parser::spanned(TextKeyProfile::BinaryUtf8V1, lo, hi),
    <lo:@L> "unicode_fold_v1" <hi:@R>
        => parser::spanned(TextKeyProfile::UnicodeFoldV1, lo, hi),
};

// Contextual: the metric names and `staleness_slo` are ordinary identifiers
// everywhere except inside a vector_field declaration, so `field cosine: ...`
// stays valid (the WP-591 hard keywords broke that additively-claimed
// compatibility).
VectorMetric: Spanned<VectorMetricKeyword> = {
    <metric:Identifier> =>? parser::grammar_result(parser::vector_metric(metric)),
};

EventDeclaration: EventDeclaration = {
    "event" <name:Identifier> "{" <partition_by:EventPartition?>
        <policy_anchor:EventPolicyAnchor?> <fields:SpannedTypedField*> "}"
        => EventDeclaration { name, partition_by, policy_anchor, fields },
};

EventPartition: Spanned<Vec<Spanned<String>>> = {
    <lo:@L> "partition_by" "(" <fields:IdentifierList> ")" <hi:@R>
        => parser::spanned(fields, lo, hi),
};

EventPolicyAnchor: Spanned<EventPolicyAnchorDeclaration> = {
    <lo:@L> "policy_anchor" <current:Identifier> <entity:Identifier>
        "(" <fields:EventPolicyAnchorFieldList> ")" <hi:@R> =>? {
            parser::grammar_result(parser::event_policy_anchor(current, entity, fields, parser::span(lo, hi)))
        },
};

EventPolicyAnchorFieldList: Vec<Spanned<EventPolicyAnchorField>> = {
    <head:EventPolicyAnchorField> <tail:("," <EventPolicyAnchorField>)*> <trailing:Comma?> => {
        let _ = trailing;
        let mut values = vec![head];
        values.extend(tail);
        values
    },
};

EventPolicyAnchorField: Spanned<EventPolicyAnchorField> = {
    <lo:@L> <entity_field:Identifier> ":" <payload_field:Identifier> <hi:@R>
        => parser::spanned(EventPolicyAnchorField { entity_field, payload_field }, lo, hi),
};

EnumDeclaration: EnumDeclaration = {
    "enum" <name:Identifier> "{" <variants:IdentifierList> "}"
        => EnumDeclaration { name, variants },
};

AggregateDeclaration: AggregateDeclaration = {
    "aggregate" <name:Identifier> "{" <items:AggregateItem*> "}"
        => AggregateDeclaration { name, items },
};

AggregateItem: Spanned<AggregateItem> = {
    <lo:@L> "root" <name:Identifier> <hi:@R>
        => parser::spanned(AggregateItem::Root(name), lo, hi),
    <lo:@L> "child" <name:Identifier> <hi:@R>
        => parser::spanned(AggregateItem::Child(name), lo, hi),
    <lo:@L> "partition_by" <expression:Expression> <hi:@R>
        => parser::spanned(AggregateItem::PartitionBy(expression.into_syntax()), lo, hi),
    <lo:@L> "conflict_key" "(" <expressions:ExpressionList> ")" <hi:@R>
        => parser::spanned(AggregateItem::ConflictKey(expressions), lo, hi),
    <lo:@L> <invariant:InvariantDeclaration> <hi:@R>
        => parser::spanned(AggregateItem::Invariant(invariant), lo, hi),
};

InvariantDeclaration: InvariantDeclaration = {
    "invariant" <name:Identifier> ":" <expression:Expression>
        => InvariantDeclaration { name, expression: expression.into_syntax() },
};

TypedField: TypedField = {
    <name:Identifier> ":" <ty:TypeExpression> => TypedField { name, ty },
};

SpannedTypedField: Spanned<TypedField> = {
    <lo:@L> <field:TypedField> <hi:@R> => parser::spanned(field, lo, hi),
};

TypedFieldList: Vec<Spanned<TypedField>> = {
    <head:SpannedTypedField> <tail:("," <SpannedTypedField>)*> <trailing:Comma?> => {
        let _ = trailing;
        let mut values = vec![head];
        values.extend(tail);
        values
    },
};

IdentifierList: Vec<Spanned<String>> = {
    <head:Identifier> <tail:("," <Identifier>)*> <trailing:Comma?> => {
        let _ = trailing;
        let mut values = vec![head];
        values.extend(tail);
        values
    },
};

TypeExpression: Spanned<TypeExpression> = {
    <lo:@L> "bool" <hi:@R> => parser::spanned(TypeExpression::Bool, lo, hi),
    <lo:@L> "i64" <hi:@R> => parser::spanned(TypeExpression::I64, lo, hi),
    <lo:@L> "u64" <hi:@R> => parser::spanned(TypeExpression::U64, lo, hi),
    <lo:@L> "timestamp" <hi:@R> => parser::spanned(TypeExpression::Timestamp, lo, hi),
    <lo:@L> "date" <hi:@R> => parser::spanned(TypeExpression::Date, lo, hi),
    <lo:@L> "uuid" <hi:@R> => parser::spanned(TypeExpression::Uuid, lo, hi),
    <lo:@L> "decimal" "<" <precision:UnsignedInteger> "," <scale:UnsignedInteger> ">" <hi:@R>
        => parser::spanned(TypeExpression::Decimal { precision, scale }, lo, hi),
    <lo:@L> "money" "<" <currency:Identifier> ">" <hi:@R>
        =>? parser::grammar_result(parser::currency_type(currency, parser::span(lo, hi))),
    <lo:@L> "string" "<" <maximum:UnsignedInteger> ">" <hi:@R>
        => parser::spanned(TypeExpression::String { maximum }, lo, hi),
    <lo:@L> "bytes" "<" <maximum:UnsignedInteger> ">" <hi:@R>
        => parser::spanned(TypeExpression::Bytes { maximum }, lo, hi),
    <lo:@L> "vector" "<" <dimension:UnsignedInteger> ">" <hi:@R>
        => parser::spanned(TypeExpression::Vector { dimension }, lo, hi),
    <lo:@L> "optional" "<" <inner:TypeExpression> ">" <hi:@R>
        => parser::spanned(TypeExpression::Optional(Box::new(inner)), lo, hi),
    <lo:@L> "list" "<" <element:TypeExpression> "," <bound:ListBound> ">" <hi:@R>
        => parser::spanned(TypeExpression::List {
            element: Box::new(element),
            minimum: bound.0,
            maximum: bound.1,
        }, lo, hi),
    <lo:@L> <name:Identifier> <hi:@R>
        => parser::spanned(TypeExpression::Named(name), lo, hi),
};

ListBound: (Option<Spanned<String>>, Spanned<String>) = {
    <maximum:UnsignedInteger> => (None, maximum),
    <lo:@L> <range:"unsigned integer range"> <hi:@R> => {
        let (minimum, maximum) = range;
        (Some(parser::spanned(minimum, lo, hi)), parser::spanned(maximum, lo, hi))
    },
};

CommandDeclaration: CommandDeclaration = {
    <kind:ApplicationCommandKind> <name:Identifier> "{"
        <inputs:InputDeclaration*>
        <service_values:ServiceValueDeclaration*>
        <idempotency:IdempotencyClause?>
        <bindings:Binding*>
        <decisions:CommandDecision*>
        <bulk_iteration:BulkIteration?>
        <requirements:Requirement*>
        <effects:Effect*>
        <return_clause:ReturnClause>
    "}" => CommandDeclaration {
        kind,
        name,
        inputs,
        service_values,
        idempotency,
        bindings,
        decisions,
        bulk_iteration,
        reconstitution: None,
        requirements,
        effects,
        return_clause,
    },
    "reimport" "command" <name:Identifier> "{"
        <inputs:InputDeclaration+>
        <reconstitution:ReconstitutionClause>
        <return_clause:ReturnClause>
    "}" => CommandDeclaration {
        kind: CommandKind::Reimport,
        name,
        inputs,
        service_values: Vec::new(),
        idempotency: None,
        bindings: Vec::new(),
        decisions: Vec::new(),
        bulk_iteration: None,
        reconstitution: Some(reconstitution),
        requirements: Vec::new(),
        effects: Vec::new(),
        return_clause,
    },
};

ApplicationCommandKind: CommandKind = {
    "command" => CommandKind::Ordinary,
    "bulk" "command" => CommandKind::Bulk,
};

ReconstitutionClause: Spanned<ReconstitutionClause> = {
    <lo:@L> "reconstitute" <entity:Identifier> "from" <source:Identifier>
        "else" <failure:OutcomeExpression> <hi:@R>
        => parser::spanned(ReconstitutionClause { entity, source, failure }, lo, hi),
};

BulkIteration: Spanned<BulkIteration> = {
    <lo:@L> "for" <element:Identifier> "in" <collection:Identifier> "{"
        <bindings:Binding*>
        <decisions:CommandDecision*>
        <requirements:Requirement*>
        <effects:Effect*>
    "}" <hi:@R> => parser::spanned(BulkIteration {
        element,
        collection,
        bindings,
        decisions,
        requirements,
        effects,
    }, lo, hi),
};

CommandDecision: Spanned<CommandDecision> = {
    <lo:@L> "observe_or_initialize" <binding:InitializedEntityBinding>
        "decide" <subject:Identifier> "{"
            <when_arms:DecisionWhenArm+>
            <else_action:DecisionElseArm>
        "}" <hi:@R>
        => parser::spanned(CommandDecision {
            binding,
            subject,
            when_arms,
            else_action,
        }, lo, hi),
};

DecisionWhenArm: Spanned<DecisionArm> = {
    <lo:@L> "when" <predicate:Expression> "=>" <action:DecisionAction> <hi:@R>
        => parser::spanned(DecisionArm {
            predicate: predicate.into_syntax(),
            action,
        }, lo, hi),
};

DecisionElseArm: Spanned<DecisionAction> = {
    "else" "=>" <action:DecisionAction> => action,
};

DecisionAction: Spanned<DecisionAction> = {
    <lo:@L> "apply" "{" <bindings:Binding*> <effects:Effect*> "}" <hi:@R>
        => parser::spanned(DecisionAction::Apply { bindings, effects }, lo, hi),
    <lo:@L> "no_effect" <hi:@R>
        => parser::spanned(DecisionAction::NoEffect, lo, hi),
    <lo:@L> "reject" <outcome:OutcomeExpression> <hi:@R>
        => parser::spanned(DecisionAction::Reject(outcome), lo, hi),
};

ServiceValueDeclaration: Spanned<ServiceValueDeclaration> = {
    <lo:@L> "service" <name:Identifier> ":" <kind:ServiceValueKind> <hi:@R>
        => parser::spanned(ServiceValueDeclaration { name, kind }, lo, hi),
};

ServiceValueKind: Spanned<ServiceValueKind> = {
    <lo:@L> "uuid_v7" <hi:@R> => parser::spanned(ServiceValueKind::UuidV7, lo, hi),
    <lo:@L> "transaction_time" <hi:@R>
        => parser::spanned(ServiceValueKind::TransactionTime, lo, hi),
};

InputDeclaration: Spanned<InputDeclaration> = {
    <lo:@L> "input" <field:TypedField>
        <aggregate_bytes:("aggregate_bytes" "<=" <UnsignedInteger>)?> <hi:@R>
        => parser::spanned(InputDeclaration { field, aggregate_bytes }, lo, hi),
};

IdempotencyClause: Spanned<IdempotencyClause> = {
    <lo:@L> "idempotency_key" <expression:Expression> <hi:@R>
        => parser::spanned(IdempotencyClause { expression: expression.into_syntax() }, lo, hi),
};

Binding: Spanned<Binding> = {
    <lo:@L> "mutate" <modifier:Identifier> <entity:Identifier> "." <index:Identifier>
        "(" <arguments:ExpressionList> ")" "as" <binding:Identifier> "else" <failure:OutcomeExpression> <hi:@R>
        =>? parser::grammar_result(parser::selected_binding(modifier, EntityBinding {
            selection_index: Some(index), entity, arguments, binding, failure,
            restriction_failure: None, cascade_failure: None,
        }, lo, hi)),
    <lo:@L> "read" <binding:EntityBinding> <hi:@R>
        => parser::spanned(Binding::Read(binding), lo, hi),
    <lo:@L> "mutate" <binding:EntityBinding> <hi:@R>
        => parser::spanned(Binding::Mutate(binding), lo, hi),
    <lo:@L> "create" <binding:EntityBinding> <hi:@R>
        => parser::spanned(Binding::Create(binding), lo, hi),
    <lo:@L> "init_or_mutate" <binding:InitializedEntityBinding> <hi:@R>
        => parser::spanned(Binding::InitOrMutate(binding), lo, hi),
    <lo:@L> "delete" <binding:DeleteEntityBinding> <hi:@R>
        => parser::spanned(Binding::Delete(binding), lo, hi),
};

InitializedEntityBinding: InitializedEntityBinding = {
    <entity:Identifier> "(" <arguments:ExpressionList> ")"
        "as" <binding:Identifier> "initialize" <initializer:ObjectLiteral>
        => InitializedEntityBinding {
            entity,
            arguments,
            binding,
            initializer,
        },
};

EntityBinding: EntityBinding = {
    <entity:Identifier> "(" <arguments:ExpressionList> ")"
        "as" <binding:Identifier> "else" <failure:OutcomeExpression>
        => EntityBinding {
            selection_index: None,
            entity,
            arguments,
            binding,
            failure,
            restriction_failure: None,
            cascade_failure: None,
        },
};

DeleteEntityBinding: EntityBinding = {
    <entity:Identifier> "(" <arguments:ExpressionList> ")"
        "as" <binding:Identifier> "else" <failure:OutcomeExpression>
        <restriction_failure:("restrict" <OutcomeExpression>)?>
        <cascade_failure:("cascade" <OutcomeExpression>)?>
        => EntityBinding {
            selection_index: None,
            entity,
            arguments,
            binding,
            failure,
            restriction_failure,
            cascade_failure,
        },
};

Requirement: Spanned<Requirement> = {
    <lo:@L> "require" <name:Identifier> ":" <condition:Expression>
        "else" <rejection:OutcomeExpression> <hi:@R>
        => parser::spanned(Requirement {
            name,
            condition: condition.into_syntax(),
            rejection,
        }, lo, hi),
};

Effect: Spanned<Effect> = {
    <lo:@L> "set" <target:Path> "=" <value:Expression> <checked_by:("checked_by" <Identifier>)?> <reveals:RevealAnnotation*> <hi:@R>
        => parser::spanned(Effect::Set(SetEffect {
            spelling: WriteSpelling::Set,
            checked_by,
            target,
            value: value.into_syntax(),
            reveals,
        }), lo, hi),
    <lo:@L> "overwrite" <target:Path> "=" <value:Expression> <checked_by:("checked_by" <Identifier>)?> <reveals:RevealAnnotation*> <hi:@R>
        => parser::spanned(Effect::Set(SetEffect {
            spelling: WriteSpelling::Overwrite,
            checked_by,
            target,
            value: value.into_syntax(),
            reveals,
        }), lo, hi),
    <lo:@L> "embed" <target:Path> "=" <value:Expression>
        "from" "(" <model_identity:Expression> "," <model_version:Expression> ")" <hi:@R>
        => parser::spanned(Effect::Embed(EmbeddingEffect {
            target,
            value: value.into_syntax(),
            model_identity: model_identity.into_syntax(),
            model_version: model_version.into_syntax(),
        }), lo, hi),
    <lo:@L> "emit" <event:Identifier> <payload:ObjectLiteral> <hi:@R>
        => parser::spanned(Effect::Emit(EmitEffect { event, payload }), lo, hi),
    <lo:@L> "transition" <transition:Identifier> "on" <binding:Identifier>
        "revision" <expected_revision:Expression>
        "stale" <stale:OutcomeExpression>
        "illegal" <illegal:OutcomeExpression> <hi:@R>
        => parser::spanned(Effect::WorkflowTransition(WorkflowTransitionEffect {
            transition,
            binding,
            expected_revision: expected_revision.into_syntax(),
            stale,
            illegal,
        }), lo, hi),
    <lo:@L> "lease" "claim" <lease:Identifier> "on" <binding:Identifier>
        "owner" <owner:Expression>
        "duration_seconds" <duration_seconds:Expression>
        "revision" <expected_revision:Expression>
        "stale" <stale:OutcomeExpression>
        "unavailable" <unavailable:OutcomeExpression>
        "invalid" <invalid:OutcomeExpression>
        "exhausted" <exhausted:OutcomeExpression> <hi:@R>
        => parser::spanned(Effect::WorkflowLease(Box::new(WorkflowLeaseEffect {
            lease,
            binding,
            operation: WorkflowLeaseOperation::Claim {
                owner: owner.into_syntax(),
                duration_seconds: duration_seconds.into_syntax(),
                expected_revision: expected_revision.into_syntax(),
                stale,
                unavailable,
                invalid,
                exhausted,
            },
        })), lo, hi),
    <lo:@L> "lease" "renew" <lease:Identifier> "on" <binding:Identifier>
        "owner" <owner:Expression>
        "fencing_token" <fencing_token:Expression>
        "duration_seconds" <duration_seconds:Expression>
        "revision" <expected_revision:Expression>
        "stale" <stale:OutcomeExpression>
        "invalid" <invalid:OutcomeExpression>
        "expired" <expired:OutcomeExpression>
        "exhausted" <exhausted:OutcomeExpression> <hi:@R>
        => parser::spanned(Effect::WorkflowLease(Box::new(WorkflowLeaseEffect {
            lease,
            binding,
            operation: WorkflowLeaseOperation::Renew {
                owner: owner.into_syntax(),
                fencing_token: fencing_token.into_syntax(),
                duration_seconds: duration_seconds.into_syntax(),
                expected_revision: expected_revision.into_syntax(),
                stale,
                invalid,
                expired,
                exhausted,
            },
        })), lo, hi),
    <lo:@L> "lease" "release" <lease:Identifier> "on" <binding:Identifier>
        "owner" <owner:Expression>
        "fencing_token" <fencing_token:Expression>
        "revision" <expected_revision:Expression>
        "stale" <stale:OutcomeExpression>
        "invalid" <invalid:OutcomeExpression> <hi:@R>
        => parser::spanned(Effect::WorkflowLease(Box::new(WorkflowLeaseEffect {
            lease,
            binding,
            operation: WorkflowLeaseOperation::Release {
                owner: owner.into_syntax(),
                fencing_token: fencing_token.into_syntax(),
                expected_revision: expected_revision.into_syntax(),
                stale,
                invalid,
            },
        })), lo, hi),
    <lo:@L> "lease" "expire" <lease:Identifier> "on" <binding:Identifier>
        "revision" <expected_revision:Expression>
        "stale" <stale:OutcomeExpression>
        "active" <active:OutcomeExpression> <hi:@R>
        => parser::spanned(Effect::WorkflowLease(Box::new(WorkflowLeaseEffect {
            lease,
            binding,
            operation: WorkflowLeaseOperation::Expire {
                expected_revision: expected_revision.into_syntax(),
                stale,
                active,
            },
        })), lo, hi),
    <lo:@L> "lease" "fence" <lease:Identifier> "on" <binding:Identifier>
        "owner" <owner:Expression>
        "fencing_token" <fencing_token:Expression>
        "revision" <expected_revision:Expression>
        "stale" <stale:OutcomeExpression>
        "invalid" <invalid:OutcomeExpression>
        "expired" <expired:OutcomeExpression> <hi:@R>
        => parser::spanned(Effect::WorkflowLease(Box::new(WorkflowLeaseEffect {
            lease,
            binding,
            operation: WorkflowLeaseOperation::Fence {
                owner: owner.into_syntax(),
                fencing_token: fencing_token.into_syntax(),
                expected_revision: expected_revision.into_syntax(),
                stale,
                invalid,
                expired,
            },
        })), lo, hi),
};

ReturnClause: Spanned<ReturnClause> = {
    <lo:@L> "return" <outcome:OutcomeExpression> <hi:@R>
        => parser::spanned(ReturnClause { outcome }, lo, hi),
};

OutcomeExpression: Spanned<OutcomeExpression> = {
    <lo:@L> <name:Identifier> <payload:ObjectLiteral> <hi:@R>
        => parser::spanned(OutcomeExpression { name, payload }, lo, hi),
};

ObjectLiteral: Spanned<ObjectLiteral> = {
    <lo:@L> "{" <fields:ObjectFieldList?> "}" <hi:@R>
        => parser::spanned(ObjectLiteral { fields: fields.unwrap_or_default() }, lo, hi),
};

ObjectFieldList: Vec<Spanned<ObjectField>> = {
    <head:ObjectField> <tail:("," <ObjectField>)*> <trailing:Comma?> => {
        let _ = trailing;
        let mut values = vec![head];
        values.extend(tail);
        values
    },
};

ObjectField: Spanned<ObjectField> = {
    <lo:@L> <name:Identifier> ":" <value:Expression> <reveals:RevealAnnotation*> <hi:@R>
        => parser::spanned(ObjectField { name, value: value.into_syntax(), reveals }, lo, hi),
};

RevealAnnotation: Spanned<Path> = {
    "reveals" <source:Path> => source,
};

ProjectionDeclaration: ProjectionDeclaration = {
    "projection" <name:Identifier> "{"
        "source" "event" <source_event:Identifier>
        <filter:WhereClause?>
        "key" "(" <key:ProjectionKeyList> ")"
        <measures:Measure+>
        <frontier:FrontierClause>
    "}" => ProjectionDeclaration {
        name,
        source_event,
        filter,
        key,
        measures,
        frontier,
    },
};

ColumnarProjectionDeclaration: ColumnarProjectionDeclaration = {
    "columnar_projection" <name:Identifier> "{"
        "source" "entity" <source_entity:Identifier>
        <fields_keyword:Identifier> "(" <fields:IdentifierList> ")"
    "}" =>? parser::grammar_result(parser::columnar_projection_declaration(
        name,
        source_entity,
        fields_keyword,
        fields,
    )),
};

ProjectionKeyList: Vec<Spanned<Expression>> = {
    <head:ProjectionKeyExpression> <tail:("," <ProjectionKeyExpression>)*> <trailing:Comma?> => {
        let _ = trailing;
        let mut values = vec![head.into_syntax()];
        values.extend(tail.into_iter().map(ParsedExpression::into_syntax));
        values
    },
};

ProjectionKeyExpression: ParsedExpression = {
    <lo:@L> <callee:Identifier> "(" <operand:Expression> ")" <hi:@R>
        =>? parser::grammar_result(parser::temporal_bucket_expression(callee, operand, parser::span(lo, hi))),
    Expression,
};

WhereClause: Spanned<Expression> = {
    "where" <expression:Expression> => expression.into_syntax(),
};

Measure: Spanned<Measure> = {
    <lo:@L> "measure" <name:Identifier> "=" <aggregation:Aggregation> <hi:@R>
        => parser::spanned(Measure { name, aggregation }, lo, hi),
};

Aggregation: Spanned<Aggregation> = {
    <lo:@L> "count" "(" ")" <hi:@R>
        => parser::spanned(Aggregation::Count, lo, hi),
    <lo:@L> "sum" "(" <expression:Expression> ")" <hi:@R>
        => parser::spanned(Aggregation::Sum(expression.into_syntax()), lo, hi),
    <lo:@L> <callee:Identifier> "(" <expression:Expression> ")" <hi:@R>
        =>? parser::grammar_result(parser::named_aggregation(callee, expression, lo, hi)),
};

FrontierClause: Spanned<Frontier> = {
    <lo:@L> "frontier" "transactionally_ordered" <hi:@R>
        => parser::spanned(Frontier::TransactionallyOrdered, lo, hi),
};

ExpressionList: Vec<Spanned<Expression>> = {
    <head:Expression> <tail:("," <Expression>)*> <trailing:Comma?> => {
        let _ = trailing;
        let mut values = vec![head.into_syntax()];
        values.extend(tail.into_iter().map(ParsedExpression::into_syntax));
        values
    },
};

Expression: ParsedExpression = OrExpression;

OrExpression: ParsedExpression = {
    <left:AndExpression> <rest:(<OrOperator> <AndExpression>)*>
        =>? parser::grammar_result(parser::binary_chain(left, rest)),
};

AndExpression: ParsedExpression = {
    <left:ComparisonExpression> <rest:(<AndOperator> <ComparisonExpression>)*>
        =>? parser::grammar_result(parser::binary_chain(left, rest)),
};

ComparisonExpression: ParsedExpression = {
    <left:AdditiveExpression> <rest:(<ComparisonOperator> <AdditiveExpression>)*>
        =>? parser::grammar_result(parser::binary_chain(left, rest)),
};

AdditiveExpression: ParsedExpression = {
    <left:MultiplicativeExpression> <rest:(<AdditiveOperator> <MultiplicativeExpression>)*>
        =>? parser::grammar_result(parser::binary_chain(left, rest)),
};

MultiplicativeExpression: ParsedExpression = {
    <left:UnaryExpression> <rest:(<MultiplicativeOperator> <UnaryExpression>)*>
        =>? parser::grammar_result(parser::binary_chain(left, rest)),
};

UnaryExpression: ParsedExpression = {
    <lo:@L> <operator:UnaryOperator> <operand:UnaryExpression> <hi:@R>
        =>? parser::grammar_result(parser::unary_expression(operator, operand, parser::span(lo, hi))),
    PrimaryExpression,
};

PrimaryExpression: ParsedExpression = {
    <literal:Literal> => parser::literal_expression(literal),
    <path:Path> => parser::path_expression(path),
    <lo:@L> "(" <expression:Expression> ")" <hi:@R>
        =>? parser::grammar_result(parser::parenthesized_expression(expression, parser::span(lo, hi))),
};

Literal: Spanned<Literal> = {
    <lo:@L> "true" <hi:@R> => parser::spanned(Literal::Bool(true), lo, hi),
    <lo:@L> "false" <hi:@R> => parser::spanned(Literal::Bool(false), lo, hi),
    <lo:@L> "null" <hi:@R> => parser::spanned(Literal::Null, lo, hi),
    <lo:@L> <value:"unsigned integer literal"> <hi:@R>
        => parser::spanned(Literal::UInt(value), lo, hi),
    <lo:@L> <value:"fixed decimal literal"> <hi:@R>
        => parser::spanned(Literal::FixedDecimal(value), lo, hi),
    <lo:@L> <value:"string literal"> <hi:@R>
        => parser::spanned(Literal::String(value), lo, hi),
};

Path: Spanned<Path> = {
    <lo:@L> <head:Identifier> <tail:("." <PathSegment>)*> <hi:@R> => {
        let mut segments = vec![head];
        segments.extend(tail);
        parser::spanned(Path { segments }, lo, hi)
    },
};

PathSegment: Spanned<String> = {
    Identifier,
    <lo:@L> "date" <hi:@R> => parser::spanned("date".to_owned(), lo, hi),
};

UnaryOperator: Spanned<UnaryOperator> = {
    <lo:@L> "!" <hi:@R> => parser::spanned(UnaryOperator::Not, lo, hi),
    <lo:@L> "-" <hi:@R> => parser::spanned(UnaryOperator::Negate, lo, hi),
};

MultiplicativeOperator: Spanned<BinaryOperator> = {
    <lo:@L> "*" <hi:@R> => parser::spanned(BinaryOperator::Multiply, lo, hi),
    <lo:@L> "/" <hi:@R> => parser::spanned(BinaryOperator::Divide, lo, hi),
};

AdditiveOperator: Spanned<BinaryOperator> = {
    <lo:@L> "+" <hi:@R> => parser::spanned(BinaryOperator::Add, lo, hi),
    <lo:@L> "-" <hi:@R> => parser::spanned(BinaryOperator::Subtract, lo, hi),
};

ComparisonOperator: Spanned<BinaryOperator> = {
    <lo:@L> "==" <hi:@R> => parser::spanned(BinaryOperator::Equal, lo, hi),
    <lo:@L> "!=" <hi:@R> => parser::spanned(BinaryOperator::NotEqual, lo, hi),
    <lo:@L> "<" <hi:@R> => parser::spanned(BinaryOperator::Less, lo, hi),
    <lo:@L> "<=" <hi:@R> => parser::spanned(BinaryOperator::LessEqual, lo, hi),
    <lo:@L> ">" <hi:@R> => parser::spanned(BinaryOperator::Greater, lo, hi),
    <lo:@L> ">=" <hi:@R> => parser::spanned(BinaryOperator::GreaterEqual, lo, hi),
};

AndOperator: Spanned<BinaryOperator> = {
    <lo:@L> "&&" <hi:@R> => parser::spanned(BinaryOperator::And, lo, hi),
};

OrOperator: Spanned<BinaryOperator> = {
    <lo:@L> "||" <hi:@R> => parser::spanned(BinaryOperator::Or, lo, hi),
};

Identifier: Spanned<String> = {
    <lo:@L> <value:"identifier"> <hi:@R> => parser::spanned(value, lo, hi),
    <lo:@L> "cover" <hi:@R> => parser::spanned("cover".to_owned(), lo, hi),
    <lo:@L> "presence" <hi:@R> => parser::spanned("presence".to_owned(), lo, hi),
    <lo:@L> "text_key" <hi:@R> => parser::spanned("text_key".to_owned(), lo, hi),
    <lo:@L> "binary_utf8_v1" <hi:@R>
        => parser::spanned("binary_utf8_v1".to_owned(), lo, hi),
    <lo:@L> "unicode_fold_v1" <hi:@R>
        => parser::spanned("unicode_fold_v1".to_owned(), lo, hi),
    <lo:@L> "cascade" <hi:@R> => parser::spanned("cascade".to_owned(), lo, hi),
    <lo:@L> "relationship" <hi:@R>
        => parser::spanned("relationship".to_owned(), lo, hi),
    <lo:@L> "using" <hi:@R> => parser::spanned("using".to_owned(), lo, hi),
    <lo:@L> "maximum" <hi:@R> => parser::spanned("maximum".to_owned(), lo, hi),
    <lo:@L> "idempotency_key" <hi:@R>
        => parser::spanned("idempotency_key".to_owned(), lo, hi),
    <lo:@L> "workflow" <hi:@R> => parser::spanned("workflow".to_owned(), lo, hi),
    <lo:@L> "state" <hi:@R> => parser::spanned("state".to_owned(), lo, hi),
    <lo:@L> "initial" <hi:@R> => parser::spanned("initial".to_owned(), lo, hi),
    <lo:@L> "transition" <hi:@R> => parser::spanned("transition".to_owned(), lo, hi),
    <lo:@L> "from" <hi:@R> => parser::spanned("from".to_owned(), lo, hi),
    <lo:@L> "to" <hi:@R> => parser::spanned("to".to_owned(), lo, hi),
    <lo:@L> "lease" <hi:@R> => parser::spanned("lease".to_owned(), lo, hi),
    <lo:@L> "claim" <hi:@R> => parser::spanned("claim".to_owned(), lo, hi),
    <lo:@L> "renew" <hi:@R> => parser::spanned("renew".to_owned(), lo, hi),
    <lo:@L> "release" <hi:@R> => parser::spanned("release".to_owned(), lo, hi),
    <lo:@L> "expire" <hi:@R> => parser::spanned("expire".to_owned(), lo, hi),
    <lo:@L> "fence" <hi:@R> => parser::spanned("fence".to_owned(), lo, hi),
    <lo:@L> "owner" <hi:@R> => parser::spanned("owner".to_owned(), lo, hi),
    <lo:@L> "expires_at" <hi:@R> => parser::spanned("expires_at".to_owned(), lo, hi),
    <lo:@L> "fencing_token" <hi:@R> => parser::spanned("fencing_token".to_owned(), lo, hi),
    <lo:@L> "attempts" <hi:@R> => parser::spanned("attempts".to_owned(), lo, hi),
    <lo:@L> "duration_seconds" <hi:@R>
        => parser::spanned("duration_seconds".to_owned(), lo, hi),
    <lo:@L> "service" <hi:@R> => parser::spanned("service".to_owned(), lo, hi),
    <lo:@L> "uuid_v7" <hi:@R> => parser::spanned("uuid_v7".to_owned(), lo, hi),
    <lo:@L> "transaction_time" <hi:@R>
        => parser::spanned("transaction_time".to_owned(), lo, hi),
    <lo:@L> "on" <hi:@R> => parser::spanned("on".to_owned(), lo, hi),
    <lo:@L> "revision" <hi:@R> => parser::spanned("revision".to_owned(), lo, hi),
    <lo:@L> "stale" <hi:@R> => parser::spanned("stale".to_owned(), lo, hi),
    <lo:@L> "illegal" <hi:@R> => parser::spanned("illegal".to_owned(), lo, hi),
    <lo:@L> "unavailable" <hi:@R> => parser::spanned("unavailable".to_owned(), lo, hi),
    <lo:@L> "invalid" <hi:@R> => parser::spanned("invalid".to_owned(), lo, hi),
    <lo:@L> "exhausted" <hi:@R> => parser::spanned("exhausted".to_owned(), lo, hi),
    <lo:@L> "expired" <hi:@R> => parser::spanned("expired".to_owned(), lo, hi),
    <lo:@L> "active" <hi:@R> => parser::spanned("active".to_owned(), lo, hi),
    <lo:@L> "principal" <hi:@R> => parser::spanned("principal".to_owned(), lo, hi),
    <lo:@L> "fact" <hi:@R> => parser::spanned("fact".to_owned(), lo, hi),
    <lo:@L> "row" <hi:@R> => parser::spanned("row".to_owned(), lo, hi),
    <lo:@L> "policy" <hi:@R> => parser::spanned("policy".to_owned(), lo, hi),
    <lo:@L> "allow" <hi:@R> => parser::spanned("allow".to_owned(), lo, hi),
    <lo:@L> "when" <hi:@R> => parser::spanned("when".to_owned(), lo, hi),
    <lo:@L> "update" <hi:@R> => parser::spanned("update".to_owned(), lo, hi),
    <lo:@L> "delete" <hi:@R> => parser::spanned("delete".to_owned(), lo, hi),
    <lo:@L> "in" <hi:@R> => parser::spanned("in".to_owned(), lo, hi),
    <lo:@L> "is" <hi:@R> => parser::spanned("is".to_owned(), lo, hi),
    <lo:@L> "not" <hi:@R> => parser::spanned("not".to_owned(), lo, hi),
    <lo:@L> "reveals" <hi:@R> => parser::spanned("reveals".to_owned(), lo, hi),
};

UnsignedInteger: Spanned<String> = {
    <lo:@L> <value:"unsigned integer literal"> <hi:@R>
        => parser::spanned(value, lo, hi),
};

StringLiteral: Spanned<String> = {
    <lo:@L> <value:"string literal"> <hi:@R>
        => parser::spanned(value, lo, hi),
};

Comma: () = { "," => () };

extern {
    type Location = usize;
    type Error = SyntaxDiagnostic;

    enum Token {
        "contract" => Token::Contract,
        "version" => Token::Version,
        "entity" => Token::Entity,
        "key" => Token::Key,
        "field" => Token::Field,
        "invariant" => Token::Invariant,
        "index" => Token::Index,
        "cover" => Token::Cover,
        "presence" => Token::Presence,
        "text_key" => Token::TextKey,
        "binary_utf8_v1" => Token::BinaryUtf8V1,
        "unicode_fold_v1" => Token::UnicodeFoldV1,
        "unique" => Token::Unique,
        "reference" => Token::Reference,
        "vector_field" => Token::VectorField,
        "text_index" => Token::TextIndex,
        "pattern_index" => Token::LongPattern,
        "delete_policy" => Token::DeletePolicy,
        "no_inbound" => Token::NoInbound,
        "restrict" => Token::Restrict,
        "cascade" => Token::Cascade,
        "relationship" => Token::Relationship,
        "using" => Token::Using,
        "maximum" => Token::Maximum,
        "event" => Token::Event,
        "enum" => Token::Enum,
        "aggregate" => Token::Aggregate,
        "root" => Token::Root,
        "child" => Token::Child,
        "partition_by" => Token::PartitionBy,
        "policy_anchor" => Token::PolicyAnchor,
        "conflict_key" => Token::ConflictKey,
        "projection" => Token::Projection,
        "columnar_projection" => Token::ColumnarProjection,
        "source" => Token::Source,
        "where" => Token::Where,
        "measure" => Token::Measure,
        "count" => Token::Count,
        "sum" => Token::Sum,
        "frontier" => Token::Frontier,
        "transactionally_ordered" => Token::TransactionallyOrdered,
        "command" => Token::Command,
        "bulk" => Token::Bulk,
        "reimport" => Token::Reimport,
        "reconstitute" => Token::Reconstitute,
        "for" => Token::For,
        "workflow" => Token::Workflow,
        "state" => Token::State,
        "initial" => Token::Initial,
        "transition" => Token::Transition,
        "from" => Token::From,
        "to" => Token::To,
        "lease" => Token::Lease,
        "claim" => Token::Claim,
        "renew" => Token::Renew,
        "release" => Token::Release,
        "expire" => Token::Expire,
        "fence" => Token::Fence,
        "owner" => Token::Owner,
        "expires_at" => Token::ExpiresAt,
        "fencing_token" => Token::FencingToken,
        "attempts" => Token::Attempts,
        "duration_seconds" => Token::DurationSeconds,
        "service" => Token::Service,
        "uuid_v7" => Token::UuidV7,
        "transaction_time" => Token::TransactionTime,
        "on" => Token::On,
        "revision" => Token::Revision,
        "stale" => Token::Stale,
        "illegal" => Token::Illegal,
        "unavailable" => Token::Unavailable,
        "invalid" => Token::Invalid,
        "exhausted" => Token::Exhausted,
        "expired" => Token::Expired,
        "active" => Token::Active,
        "input" => Token::Input,
        "aggregate_bytes" => Token::AggregateBytes,
        "idempotency_key" => Token::IdempotencyKey,
        "read" => Token::Read,
        "mutate" => Token::Mutate,
        "create" => Token::Create,
        "init_or_mutate" => Token::InitOrMutate,
        "observe_or_initialize" => Token::ObserveOrInitialize,
        "decide" => Token::Decide,
        "apply" => Token::Apply,
        "no_effect" => Token::NoEffect,
        "reject" => Token::Reject,
        "initialize" => Token::Initialize,
        "as" => Token::As,
        "else" => Token::Else,
        "require" => Token::Require,
        "set" => Token::Set,
        "overwrite" => Token::Overwrite,
        "checked_by" => Token::CheckedBy,
        "embed" => Token::Embed,
        "emit" => Token::Emit,
        "return" => Token::Return,
        "reveals" => Token::Reveals,
        "principal" => Token::Principal,
        "fact" => Token::Fact,
        "row" => Token::Row,
        "policy" => Token::Policy,
        "allow" => Token::Allow,
        "when" => Token::When,
        "update" => Token::Update,
        "delete" => Token::Delete,
        "in" => Token::In,
        "is" => Token::Is,
        "not" => Token::Not,
        "exists" => Token::Exists,
        "bool" => Token::Bool,
        "i64" => Token::I64,
        "u64" => Token::U64,
        "timestamp" => Token::Timestamp,
        "date" => Token::Date,
        "uuid" => Token::Uuid,
        "decimal" => Token::Decimal,
        "money" => Token::Money,
        "string" => Token::String,
        "bytes" => Token::Bytes,
        "vector" => Token::Vector,
        "optional" => Token::Optional,
        "list" => Token::List,
        "true" => Token::True,
        "false" => Token::False,
        "null" => Token::Null,
        "{" => Token::LeftBrace,
        "}" => Token::RightBrace,
        "(" => Token::LeftParen,
        ")" => Token::RightParen,
        "<=" => Token::LessEqual,
        ">=" => Token::GreaterEqual,
        "->" => Token::Arrow,
        "=>" => Token::FatArrow,
        "==" => Token::EqualEqual,
        "!=" => Token::BangEqual,
        "&&" => Token::AndAnd,
        "||" => Token::OrOr,
        "<" => Token::Less,
        ">" => Token::Greater,
        "," => Token::Comma,
        ":" => Token::Colon,
        "." => Token::Dot,
        "=" => Token::Equal,
        "!" => Token::Bang,
        "-" => Token::Minus,
        "*" => Token::Star,
        "/" => Token::Slash,
        "+" => Token::Plus,
        "fixed decimal literal" => Token::FixedDecimalLiteral(<String>),
        "unsigned integer literal" => Token::UIntLiteral(<String>),
        "unsigned integer range" => Token::UIntRange(<(String, String)>),
        "string literal" => Token::StringLiteral(<String>),
        "identifier" => Token::Identifier(<String>),
    }
}

Bounded Delete Commands

Alpha surface: grammar, executable IR v16, atomic runtime execution, and generated Rust, Go, TypeScript, Python, and MCP bindings are implemented. Collection commands use the ordinary symbolic command service; they do not introduce a generic transaction API. Compiler-bounded one-hop cascade grammar, executable IR V13, and the command/runtime/storage path are implemented. Generated Rust, Go, TypeScript, and Python cascade facades expose the declared typed success and overflow outcomes; generated MCP deliberately omits secret-bearing query output.

Atomic unary delete and preimage return

An ordinary idempotent command may delete exactly one complete-primary-key, partition-local entity whose structural policy is no_inbound. The delete binding denotes the immutable transaction-current preimage: requirements, explicit events, and the declared outcome all read the same record and version carried as the delete mutation’s prior image.

command ConsumeToken {
    input request_id: uuid
    input organization_id: uuid
    input token_id: uuid
    idempotency_key request_id

    delete OneTimeToken(organization_id, token_id)
        as token
        else TokenMissing {}

    return TokenConsumed {
        identifier: token.identifier,
        value: token.value reveals token.value
    }
}

Absence selects TokenMissing with no mutation. If another command updates or deletes the row after evaluation, transaction-current validation discards the entire evaluated graph and performs bounded whole-command reevaluation. Therefore an update winner supplies the new preimage, while a delete winner supplies the missing outcome; a stale preimage is never persisted or released. Idempotent replay returns the already-persisted consumed or missing outcome without another read or delete attempt.

All returned, required, and event-bound preimage fields are compiler-declared access dependencies. Complete-record use remains explicit and schema-bounded; forward-compatible unknown stored fields remain in the mutation’s prior-image evidence but do not become public output. Secret fields require an explicit reveals declaration and independent release-time field authority—delete permission does not grant reveal permission. Sets, a second alias for the target, multiple ordinary deletes, restrict, cascade, and caller-selected targets receive source diagnostics. Those broader deletion policies remain available only through the compiler-bounded collection forms below.

Generated Rust, Go, TypeScript, and Python clients expose the same closed consumed/missing outcome union and publish exact symbolic metadata for every secret outcome field. Generated MCP command schemas carry that same value-free metadata. TypeScript also generates a structured-redaction helper, while Rust, Go, and Python redact the default diagnostic representation. MCP never gains a raw delete or transaction method. CLI and transport uncertainty recovery retain the original idempotency identity, so a response lost after commit resolves to the persisted outcome rather than executing another delete.

The value-free external acceptance receipt at fixtures/unary-delete-preimage/external-consumer-receipt-v1.json binds the generic consume plans and generated artifacts to a Rust, Go, TypeScript, and Python loopback over one real TLS service. The harness creates consumer workspaces outside the repository; no framework schema, adapter, route, generated profile, or authentication policy is retained by RiffDB.

Compiler-bounded collection commands

RiffDB collection writes are compiled commands, not caller-defined transactions. A bulk command may expand exactly one bounded list, once, with no nesting or data-dependent iteration. The compiler proves the complete partition, aggregate, conflict, read, write, index, and capacity sets before deployment. The baseline tier admits at most 256 list elements and 256 statically possible authoritative mutation instances. The high-cardinality tier admits at most 1,024 list elements and 4,096 statically possible authoritative mutation instances. In both tiers, the complete input and mutation graph remains subject to the 16 MiB command bound.

Mutation-instance accounting is conservative and closed: every non-read binding outside the loop is charged once, every non-read binding inside the loop is charged once per declared maximum element, and checked cascade fan-out is charged at its declared maximum. A command crosses into the high-cardinality tier if either baseline ceiling is exceeded. The compiler then selects contract, executable-IR, and bundle V23 automatically; callers cannot opt in, select a larger limit, or split one invocation inside RiffDB. Commands within both baseline ceilings keep their least-sufficient pre-V23 identity. Checked cascade deletion retains its independent 256-instance ceiling.

High cardinality changes only the statically admitted envelope. It does not relax the one-list, one-partition, conflict, index-maintenance, aggregate-byte, graph-byte, request-byte, durability, idempotency, or atomic-commit guarantees. A legal invocation therefore commits its complete authoritative mutation set and outcome once, or commits none of it; there is no partial-success or application-visible batching surface. A conflict derivation that is exactly the proved partition route is charged once because all elements share that partition; any other element-dependent conflict derivation is charged once per declared maximum element and must remain within the unchanged 256-key conflict ceiling.

entity Row {
    key (organization_id: uuid, row_id: uuid)
    delete_policy no_inbound
}

aggregate Rows {
    root Row
    partition_by organization_id
    conflict_key (organization_id, row_id)
}

bulk command DeleteRows {
    input request_id: uuid
    input organization_id: uuid
    input row_ids: list<uuid, 1..64>
    idempotency_key request_id

    for row_id in row_ids {
        delete Row(organization_id, row_id) as row else Missing {}
    }

    return Deleted {}
}

Every accepted delete names a policy on the entity:

• delete_policy no_inbound is legal only when no declared relationship targets the entity.
• delete_policy restrict Child.by_parent requires every inbound relationship to be covered by that exact, canonical, partition-local reverse-index prefix. The command plan records a transaction-current empty-prefix check; a runtime scan or unindexed relationship discovery is never substituted.

An indexed-restrict delete must name the business result separately from a missing row:

delete Parent(tenant_id, parent_id) as parent
    else Missing {}
    restrict Referenced {}

If the reverse index is nonempty, RiffDB persists Referenced as the command’s typed zero-mutation outcome. If a reference races with the delete, transaction-current validation reevaluates the whole bounded command and selects Referenced; it does not spin on an infrastructure retry. Grammar/IR v6 permits one indexed-restrict delete template per command.

Set-null, orphaning, cross-partition deletion, physical history removal, nested loops, and caller-supplied callbacks are not part of this surface. A bounded delete may target an entity with declared unique keys. RiffDB derives and removes each exact old index entry from the checked predecessor inside the authoritative write transaction; application input does not supply or guess a release conflict for a value visible only in that predecessor. Ordered execution is intentional: a create submitted before the delete still receives the typed unique conflict, while a create admitted after the committed delete may reuse the released value. RDB-C044, the former feature seal for this audited path, is retired; unsafe deletion policy remains the source-spanned RDB-C045 refusal. Ordinary commands support only the single exact-key no_inbound form described above; indexed restrict and cascade deletion remain compiler-bounded collection commands.

Compiler-bounded one-hop cascade (V13)

An exhaustive cascade policy may name direct inbound relationships whose source rows share the target aggregate, partition derivation, and aggregate conflict key. Every source entity must have delete_policy no_inbound; recursive and multi-level deletion remain unavailable.

entity User {
    key (organization_id: uuid, user_id: uuid)
    delete_policy cascade {
        relationship Account.account_user using Account.by_user maximum 32
        relationship Session.session_user using Session.by_user maximum 32
    }
}

bulk command DeleteUsers {
    input request_id: uuid
    input organization_id: uuid
    input user_ids: list<uuid, 1..3>
    idempotency_key request_id

    for user_id in user_ids {
        delete User(organization_id, user_id) as user
            else UserMissing {}
            cascade CascadeLimitExceeded {}
    }

    return UserDeleted {}
}

The compiler requires every direct inbound relationship exactly once, an exact canonical reverse index prefix, a positive maximum, at most 32 entries, and at most one cascade delete template per command. It proves root maximum * (1 + sum(relationship maxima)) <= 256 and charges the complete possible predecessor graph against 16 MiB. Callers provide only root keys and ordinary inputs; they cannot choose relationships, indexes, maxima, child keys, or traversal depth.

Cascade contracts emit the least-sufficient bundle, grammar, and executable IR identity V13. Contracts without cascade continue to emit V1 through V12 as appropriate. This compiler package is deployable by a V13-capable runtime. After acquiring the command’s canonical aggregate conflict, the coordinator reads each named reverse-index prefix only through its compiler-declared maximum + 1 bound. If any prefix returns the extra row, it durably records the declared cascade outcome with no entity mutation. The root and every present candidate in that bounded maximum-plus-one set are nevertheless materialized and checked against the transaction-current capability revision and row policy before the outcome is released. An unauthorized caller therefore cannot distinguish an absent, within-limit, or over-limit protected relationship by observing the cascade result. Otherwise the runtime rereads the roots, discovered predecessors, and range epochs in one consistent view and evaluates one children-before-parent graph.

The persisted entity mutations remain canonically ordered in the existing command envelope. This is a physical encoding detail: the validated semantic graph proves every child precedes its parent before the commit coordinator derives the ordinary tombstones, index removals, provenance, changelog, and commit records. Crash recovery, idempotent replay, backup/restore, and follower application therefore use the same paths as any other bounded collection delete.

Duplicate collection keys reject the whole command. All expanded effects, the typed outcome, events, provenance, idempotency record, and commit record are one atomic command result. A crash or retry can expose only the complete persisted result or complete absence; an element-level partial result is not part of the protocol.

External adapters can exercise this surface through generated Rust, Go, TypeScript, and Python clients while retaining their own identity, session, and application models. RiffDB supplies only the generic compiled data command, authorization, and durability guarantees.

Correlated aggregate element bytes (V16)

When a list’s individual element maximum multiplied by its cardinality would reject a useful atomic command, the expanded input may declare one smaller aggregate bound:

bulk command WritePolicyMutations {
    input request_id: uuid
    input mutations: list<PolicyMutation, 1..100>
        aggregate_bytes <= 900000
    idempotency_key request_id

    for mutation in mutations {
        create PolicyMutation(mutation.organization_id, mutation.mutation_id) as stored
            else Exists {}
        set stored.context = mutation.context
    }

    return Written {}
}

The number is compiler-owned, positive, and applies only to the single list expanded by the bulk command. It is the sum of the complete canonical binary documents for the submitted elements; it is not JSON, Protobuf, compressed, or in-memory size. Every element still obeys its individual type and byte bounds, and the list still obeys its count bound.

The compiler derives how many times element-variable bytes can appear in the canonical command input, entity mutations, and events. It proves both the command’s least-sufficient root request envelope and fixed bytes + aggregate bytes * derived copy coefficient <= 16 MiB. Unsupported flows, arithmetic overflow, or either excessive result fails at the source clause. The coefficient is stored in executable IR V16 and cannot be selected by an application or caller.

Generated Rust, Go, TypeScript, and Python methods compute the exact canonical sum before opening the transport. MCP publishes x-riffdb-aggregateCanonicalElementBytes on the list schema. The service recomputes the sum authoritatively before effects, and the deterministic runtime retains the complete graph check. No layer splits, truncates, retries a subset, or returns partial element outcomes.

Client and CLI preflight

Generated Rust, Go, TypeScript, and Python methods carry the compiler’s exact inclusive list minimum and maximum. They reject an out-of-range collection before invoking the transport. The generated MCP JSON Schema carries the same minItems, maxItems, and any compiled aggregate canonical-element-byte extension. Canonical command encoding also enforces the fixed document-byte ceiling before a request is sent; no client silently splits one atomic collection command into smaller writes.

Large atomic command inputs

Most commands retain the original 1 MiB complete input ceiling. When the compiler proves that a command’s typed maximum is greater than 1 MiB but no greater than 4 MiB, that command selects the additive V22 contract identity and carries its exact maximum in the executable plan. There is no source flag or caller option for this behavior.

Only the complete root command-input record receives this headroom. Every string, bytes value, list, nested record, entity, event, and outcome retains the ordinary 1 MiB canonical document ceiling, and the complete input-and-write graph retains its independent 16 MiB ceiling. The service materializes the exact canonical input and enforces the plan maximum before evaluation or effects. Small command inputs keep byte-identical canonical encoding and idempotency hashes.

Command-bearing gRPC, application-session, contextual-reaction, MCP, CLI, command-batch, and local driver messages have a fixed 8 MiB frame ceiling to hold bounded Protobuf, JSON, field-name, and base64 overhead. The frame is not decoded command capacity: ordinary services still accept 1 MiB, and large commands still stop at both 4 MiB and their lower compiler-proved plan maximum.

For example, one MLflow-shaped command may create a Run, up to 100 indexed tag children whose values are individually bounded to 8,000 bytes, and one packed hydration field atomically. RiffDB does not split the command, reduce the legal tag count, omit search rows, or make the adapter coordinate multiple writes.

High-cardinality atomic collections

Collection commands retain their original 256-element and 256-possible-mutation tier unless the compiler proves that the source needs the additive V23 identity. V23 permits one declared list of at most 1,024 elements and at most 4,096 possible authoritative mutations. The mutation proof counts each fixed non-read binding once and each collection-local non-read binding once per declared element, using checked arithmetic. The independent 256-row cascade ceiling is unchanged.

This is count capacity, not a larger transaction escape hatch. The compiler still proves one partition and one mutation aggregate, and the existing conflict, observation, dependency, index, event, 4 MiB input, 8 MiB frame, 16 MiB graph, cancellation, and deadline bounds all apply independently. A command may therefore fit the 1,024-element tier and still be rejected by another closed resource. V1 through V22 retain their exact 256/256 interpretation and bytes.

Generated Rust, Go, TypeScript, and Python bindings expose the source-declared maximum rather than the process ceiling. CLI, MCP, local-driver, and remote gRPC submissions use that same sealed plan; none can choose V23, split the collection, retry a subset, or stage a partial result. The checked adapter corpus executes one 1,000-metric operation plus its fixed summary mutation through a generated binding and redb, replays the same outcome, and proves a forced business failure leaves neither a visible row prefix nor an extra summary revision.

The CLI accepts inline JSON, @file, a file path, or - for stdin. Nested collection elements are ordinary symbolic JSON records; scalar tags remain explicit where JSON has no lossless native form:

{
  "request_id": {"$uuid":"018f0f8b-7c6d-7e31-8a4f-000000000001"},
  "tuples": [{
    "store_id": {"$uuid":"018f0f8b-7c6d-7e31-8a4f-000000000002"},
    "tuple_id": {"$uuid":"018f0f8b-7c6d-7e31-8a4f-000000000003"},
    "object": "document:roadmap",
    "relation": "viewer",
    "subject": "user:agent"
  }]
}

Pass the exact application source to make the CLI load its checked lock and reject the compiled collection count before connecting:

riffdb command run WriteTuples \
  --application riffdb.application.json \
  --input @write-tuples.json

Without --application, local byte and structural bounds still apply and the server remains the authoritative fail-closed validator, but the ad-hoc CLI cannot know a command-specific list range.

Atomic indexed selection

An allocation command can choose and change the first available indexed row in one transaction. The caller supplies the pool and business inputs; RiffDB owns the bounded selection and contention retry. This removes the application-side query/choose/claim loop for this supported shape.

entity Slot {
  key (pool_id: uuid, slot_id: uuid)
  field available: bool
  field holder: uuid
  index by_available (pool_id, available, slot_id)
}
aggregate Slots {
  root Slot
  partition_by pool_id
  conflict_key (pool_id)
}
command Allocate {
  input request: uuid
  input pool_id: uuid
  input holder: uuid
  idempotency_key request
  mutate first Slot.by_available(pool_id, true) as slot else NoCapacity {}
  require available: slot.available else NoCapacity {}
  set slot.available = false checked_by available
  set slot.holder = holder checked_by available
  return Allocated { slot: slot }
}

Place these declarations inside a versioned contract. first is contextual; existing entities or inputs named first keep their meaning.

What selection guarantees

The compiler proves that the equality prefix fixes one partition and that every conflict key is computable before reading state. The example uses one pool conflict key. A conflict key including the unknown slot_id cannot support this selection. Companions in other aggregates are allowed when their exact targets come from input and they share the proved partition route. Every written aggregate retains its own invariants.

Selection seeks the prefix and retains its first complete index key in ascending canonical order, including the entity-key tie breaker. It reads at most one candidate per evaluation. The index entry, row, and required aggregate roots come from one consistent storage view. There is no application callback inside the transaction.

Before committing any outcome, including NoCapacity, RiffDB revalidates the partition/index generation and exact row/root observations. Changed evidence causes complete reevaluation under the same input, logical context, and idempotency identity. The existing limit is three evaluations, subject to the request deadline and cancellation. Exhaustion returns the existing typed retryable failure. Selection promises no fairness or starvation bound.

A requirement failing on the first candidate returns its declared failure. It does not search further. Row-policy denial likewise returns the redacted denial; it never skips a candidate or reports it as an empty pool. The usual command, capability, relationship, and secret-flow controls still apply.

Selection alone supplies no assignment evidence. Keep explicit business checks, caller-version fences, or state-derived arithmetic as described in the write cookbook.

Retry and uncertainty

An empty result is a durable business result. Repeating that request identity returns the original NoCapacity, even after another command releases a slot. Start a new business attempt with a new identity to try newly available capacity. For an unknown commit status, resolve the original identity before making another allocation. The generated command APIs retain their ordinary uncertainty rules.

Choosing a pool

A pool conflict key makes selection safe by serializing writes in that pool. Larger pools can concentrate contention; separate pools can reduce conflict-lock sharing. Index generations remain conservative per partition and index: an index change in another pool of the same partition can still invalidate an evaluation. There is no automatic repartitioning or finer-grained generation guarantee.

POC bounds and compatibility

• One selector in the ordinary binding phase; no collection or decision-arm selector, cascade/restrict deletion, or second explicit binding to the selected entity type in the same command.
• Prefix arguments use only validated inputs and constants and canonical index field encodings. They must prove partition and complete conflict ownership. An index containing a secret field cannot drive selection: its membership or ordering would disclose that field without an explicit reveal site.
• No residual predicate, cursor, offset, reverse or random order, candidate page, skip-locked behavior, or caller-selected scan limit.
• A selected value cannot determine another explicit binding target. Required invariant-root reads are compiler-owned and may follow the selected root key.
• Cross-partition writes, cross-aggregate invariants, projection-based write decisions, and durable multi-command orchestration remain unavailable.

Selector contracts use grammar/IR/bundle V26 and require a compatible server. Contracts without selectors keep their historical least-sufficient bytes and identities. Deploy the exact bundle and regenerate its ordinary bindings and lock artifacts; SDK, CLI, gRPC, and MCP do not expose a separate raw scan API. Command explain describes order, prefix expressions, ownership, generation validation, and bounds without exposing row values.

Selection retains one live candidate after the bounded checkpoint/journal merge. That bound does not imply constant disk I/O, parallel throughput, or fairness.

Command and invariant cookbook

These patterns preserve RiffDB’s application safety rule: writes are accepted only through compiled commands whose complete data dependencies, conflicts, invariants, outcomes, and partition route are known before runtime.

A contract does not pick a grammar version. The compiler assigns the least sufficient one for the constructs the contract actually uses, so a contract built only from the patterns below keeps the V1 identity, and one that reaches for a newer construct — partition-local cross-aggregate writes, high-cardinality atomic collections — selects the version that admits it. The current maximum is V26. SPEC.md §7 defines the base surface and its changelog records what each later version added.

Create an aggregate root

command CreateCollection {
  input idempotency_key: string<128>
  input collection_id: uuid
  input name: string<96>
  idempotency_key idempotency_key
  create Collection(collection_id) as collection
    else CollectionExists { collection_id: collection_id }
  set collection.name = name
  set collection.created_at = tx.time
  return Created { collection: collection }
}

The idempotency input is caller-stable across retries. create ... else declares duplicate handling as a typed business outcome. Every required non-key field is assigned exactly once.

Create a related child safely

command AddArtifact {
  input idempotency_key: string<128>
  input collection_id: uuid
  input artifact_id: uuid
  input title: string<128>
  idempotency_key idempotency_key
  read Collection(collection_id) as collection
    else CollectionMissing { collection_id: collection_id }
  create Artifact(collection_id, artifact_id) as artifact
    else ArtifactExists { artifact_id: artifact_id }
  set artifact.title = title
  set artifact.created_at = tx.time
  return Created { artifact: artifact }
}

The dominating exact read proves the required relationship target exists in the same partition. Do not replace it with an application preflight read: check-then-write races are exactly what the compiled command prevents.

Enforce state rules in the command

require StillOpen: collection.closed == false
  else CollectionClosed { collection_id: collection_id }

Place a requirement after the binding that supplies its fields. The condition is evaluated transaction-current inside the command. The rejection is a declared typed outcome, not an exception inferred by client code.

Entity or aggregate invariants use:

invariant NonnegativeQuantity: quantity >= 0

An invariant is part of compilation and mutation validation. Do not reproduce it with a prior RiffQL read or a generated-client branch.

Change a relationship

Read the complete new target key before mutating relationship fields:

read Shelf(collection_id, new_shelf_id) as shelf
  else ShelfMissing { shelf_id: new_shelf_id }
mutate Artifact(collection_id, artifact_id) as artifact
  else ArtifactMissing { artifact_id: artifact_id }
overwrite artifact.shelf_id = new_shelf_id

The compiler rejects a relationship-changing command without the dominating target read. Required references use required fields; optional relationships must use a contract-supported optional shape.

Declare the evidence for each write

set and overwrite are checked per assignment. Reading another stored field, incrementing revision, or taking a decision branch does not authorize an unrelated replacement set.

For arithmetic, this POC recognizes only target + delta, delta + target, and target - delta, using existing checked numeric types. Delta may read inputs, constants, and keys but no stored state. old * 0 + replacement and reading a different target field do not qualify.

Associate a business transition explicitly:

mutate Slot(tenant_id, slot_id) as slot else SlotMissing {}
require is_free: slot.free else SlotTaken {}
set slot.free = false checked_by is_free
set slot.held_by = holder checked_by is_free

Removing is_free makes these associations fail compilation. The compiler checks the requirement’s scope, execution order, and stored-state dependency. The author asserts its relevance: even a weak predicate must never be read as a proof of caller freshness. An invalid association fails even if another proof would admit the write. Use one name after the value and before any reveals annotation.

Use an expected revision for replacement edits:

mutate Parent(tenant_id, parent_id) as parent else ParentMissing {}
mutate Item(tenant_id, parent_id, item_id) as item else ItemMissing {}
require current: parent.revision == expected_revision else Stale {}
set parent.revision = parent.revision + 1
set item.value = value

The same fence covers replacement fields on parent itself. Merely incrementing parent.revision does not. The comparison and increment must be unconditional; revision and input must both be i64 or both u64, and the increment is exactly one, with either operand order. Children must belong to the same aggregate and match every root-key argument. Overflow refuses the command. Relevant writers must advance this revision, and callers must send the revision they observed; the compiler does not enforce that protocol across all commands.

A deliberate replacement without these proofs uses overwrite. It can coexist with a revision increment or an independent business check. Creation uses set; initialized bindings are checked for their existing-row case. An invariant, an unassociated requirement, and a decision predicate do not count as per-write evidence. Current decision requirements execute after arm effects, so they cannot be associated as dominating checks. Bulk outer requirements execute after element effects; they cannot justify those element assignments.

riffdb contract explain reports every assignment occurrence and its evidence, including decision-arm locations and requirement/fence references. Multiple proofs are preserved. RDB-C048 identifies a set without evidence, RDB-C049 a contradictory overwrite, and RDB-C050 an invalid business association.

Newly compiled assignment-bearing plans use grammar, IR, and bundle V25. Their hashes and application locks change. Review each affected contract, then use the normal lock-writing workflow. Existing deployed artifacts retain their behavior; explain labels their classification as legacy dependency evidence. They acquire the new checks only when their source is migrated and recompiled.

Mutually exclusive decide arms (when ... => { ... } and else => { ... }) may also declare branch-local workflow transition and lease instructions on mutate, init_or_mutate, or observe_or_initialize bindings, using the same expected revision, allowed source state, lease holder/epoch fences, and zero-mutation declared rejection outcomes as top-level instructions.

Unique values

Declare uniqueness on the entity:

unique collection_name (collection_id, name)

The complete partition prefix must lead the unique key. Commands that create or change the tuple receive compiler-derived transaction-current occupancy and conflict checks. A query followed by a write is not a uniqueness mechanism.

Safe retry rule

Reuse the exact command name, input, role, contract identity, and idempotency key until RiffDB returns the stored terminal outcome. Never generate a new key to escape an uncertain response. Generated command helpers own this behavior.

Patterns intentionally unavailable

• raw insert, update, delete, or SQL;
• arbitrary transaction callbacks;
• network, filesystem, wall-clock, or process-global access in commands;
• client-side check-then-write integrity;
• cross-partition mutation;
• undeclared business exceptions;
• unbounded loops, recursion, scans, or collections; and
• manually supplied conflict keys, plans, IDs, field masks, or capability grants.

An unavailable pattern is not an invitation to work around the compiler. Model the invariant, relationship, aggregate, command, or declared outcome so RiffDB can prove it.

Allocate an unspecified free slot

Use atomic indexed selection when an equality prefix fixes the partition and every conflict key. mutate first selects one row and carries its index generation through commit validation. Keep the availability requirement and explicit write associations: selection itself does not authorize replacement. Empty results are durable and replay under the same request identity; a new attempt after capacity changes needs a new identity.

Negative examples and safe corrections

These are intentionally rejected application patterns. Diagnostic messages do not echo source values; the code and span identify the symbolic defect.

Mutation without idempotency

Removing idempotency_key idempotency_key from AddArtifact produces RDB-C010. Restore one direct required string<1..=128> input and use it only as the command idempotency declaration. Do not generate a new key on retry.

Missing relationship proof

Removing the exact read Collection(collection_id) from AddArtifact while creating an Artifact produces RDB-C024. Restore the dominating same- partition target read and its declared missing-target outcome. Do not preflight with a separate query.

Unindexed page ordering

Changing CollectionPage to order artifacts by title produces RDB-QP003 because the contract has no matching bounded access path. Add a declared index whose key supports the complete predicate and ordering, or keep the supported artifact_id order. RiffDB will not fetch and sort the collection in client code.

Unbounded collection

Removing take $artifact_limit produces RDB-QS009. Restore an explicit positive literal or typed Limit parameter. There is no unlimited application collection.

Role names an undeclared query

Adding "SearchEverything" to a role produces RDB-AS007. Declare and compile the named query or remove it from the role. Do not substitute raw entity/scan permissions.

Editing generated output

Changing any file under generated/ makes riffdb application lock --check fail. Restore it with riffdb application generate --locked; if symbolic intent changed, review and write a new lock instead.

Treating secret as encryption

Declaring field secret token: string<256> and then storing a plaintext bearer token treats a display-and-visibility classification as a cryptographic control it never was. secret withholds the value from display surfaces and field-visibility grants (surfaces show [redacted:token]); the durable record, backups, exports, and changelog frames still carry the value at full fidelity by design. Store the at-rest form you would store anyway — a hash for verification secrets, an opaque provider token where recovery is required — and keep the secret classification on that stored field. See Secret Field Classification.

RiffQL language versions

RiffQL is RiffDB’s formal, symbolic, read-only application language. It is an inspectable compiler input, not natural-language execution and not SQL.

A named query declares typed $parameters, then ordered one, maybe, or many bindings. Every binding has a where predicate. one declares an absence outcome; every many has an explicit positive take bound. Ordering is explicit and cursors are optional typed parameters.

For application pages, start with a fixed bound such as take 25 or take 50. Use Limit<MAX> when callers need to choose a smaller page at runtime while the compiler must prove a reviewed maximum. MAX may be as large as 65,534; the compiler still refuses a query whose independently bounded result bytes, cost, authority, provider work, or transport envelope cannot support it.

one and maybe are partition-local point or unique-key reads: their predicates must constrain either the target entity’s complete primary key or every field of a declared same-partition unique key index with == (and no order by). When a covering index on those same fields is also declared, the compiler prefers it automatically. For a singular lookup across partitions, declare a small route entity keyed by that identifier, point-read the route, and then point-read the target entity through its stored key.

Two bounded collection dependencies are available for operational reads. First, an earlier bounded many field in canonical ascending key order may be consumed directly by in for junction reads:

many ticket_labels from TicketLabel
    where organization_id == $organization_id
      && ticket_id == ticket.ticket_id
    order by label_id asc
    take 50

many labels from Label
    where organization_id == $organization_id
      && label_id in ticket_labels.label_id
    order by label_id asc
    take 50
    else IntegrityFailure

Second, when hydrating parent or foreign-key records from an earlier bounded many binding whose order by is chosen for display (such as updated_at desc) or whose foreign-key field may contain duplicates or null values, wrap the source path in distinct(...):

many tickets from Ticket
    where organization_id == $organization_id
    order by updated_at desc, ticket_id desc
    take 50

many assignees from User
    where organization_id == $organization_id
      && user_id in distinct(tickets.assignee_id)
    take 50

In both forms, the target bound cannot exceed the source bound and both accesses must remain in one partition. Direct in binding.field requires the source collection to be ordered by the consumed field ascending and requires every target to exist (selecting the declared else outcome if missing). in distinct(binding.field) skips null values on optional foreign keys, sorts and deduplicates present keys in canonical ascending target-key order before executing the bounded point batch, and makes else and target order by optional. Collection-as-scalar use, non-key fan-out, nested per-row collections, and Cartesian products are rejected.

query OpenTickets($tenant: TenantId, $limit: Limit<499> = 25) {
    many tickets from Ticket
        where tenant_id == $tenant && status == TicketStatus.Open
        order by updated_at desc, ticket_id desc
        take $limit

    return Found {
        tickets: tickets {
            ticket_id
            title
            priority
        }
    }

    outcomes Found
}

Multiple declared outcomes use |, never commas or whitespace alone:

outcomes Found | NotFound | IntegrityFailure

Every many binding must have a declared contract index whose equality prefix and ordering match the query. When one is absent, add the compiler-suggested bounded index to the contract rather than filtering or sorting in application code.

The ordinary operational expression set is ==, !=, <, <=, >, >=, bounded in, &&, and ||. Exact-result language V6 additionally recognizes bounded not_in, starts_with, ends_with, and contains while compiling a provider-independent semantic family. See Exact Predicate and Order Families for its stricter shape and current activation status. Tokenized-text language V10 adds four compiler-sealed matching shapes and the optional fixed riff_bm25_v1 order; see Tokenized Text Search. Candidate language V11 adds complete-before-order set sources, including declared long-value exact pattern providers; see Long-Value Pattern Search. RiffQL has no mutation, SQL escape, arbitrary function call, recursion, loop, callback, clock, randomness, network, filesystem, unrestricted scan, or arbitrary join.

The parser accepts at most 1 MiB of UTF-8 source, 131,072 tokens/AST nodes, 32 nesting levels, 4,096 bindings, and 1,024 items in any local collection. Diagnostics are value-free and use stable RDB-QS001 through RDB-QS009 codes.

Exact secret outputs (language V3)

A newly compiled named query may return a stored secret field only when the result leaf declares the exact source once:

return Found {
    session: session {
        token_hash reveals session.token_hash
    }
}

reveals is contextual; contracts, bindings, aliases, and fields may still be named reveals. The declaration is legal only on a direct stored scalar leaf, must match the projected binding and field exactly, and must name a secret-classified field. Missing, duplicate, ordinary-field, mismatched, record, aggregate, and excess declarations fail compilation with RDB-QR007. There is no wildcard, whole-record, expression, cursor, or parameter form, and one query is limited to 1,024 declared secret leaves.

When a result field itself is the bare contextual identifier reveals, keep its comma delimiter before a following field. The canonical formatter emits that delimiter automatically so parse-format-parse cannot reinterpret the field as another secret-output clause.

The declaration records review intent in RiffQL language V3 and query IR V4; it is not caller-supplied authority. Selecting the immutable query in an application role derives only its exact dedicated secret-field visibility. Without that current role authority, execution returns RDB-AUTH-0214 before releasing any result. Secret-bearing named queries remain available through typed SDK and gRPC execution, but are omitted from MCP tool generation and cannot be referenced by reactive watches, live queries, hydrations, or contextual subscriptions.

Compiler-declared bounded runtime limits (language V9 and V11)

Limit<MAX> is a query-only page-size refinement with an implicit minimum of one and an inclusive maximum from 1 through 65,534:

query OpenTickets(
    $tenant: TenantId,
    $limit: Limit<100> = 25,
    $after: Cursor?,
) {
    many tickets from Ticket
        where tenant_id == $tenant && status == TicketStatus.Open
        order by updated_at desc, ticket_id desc
        take $limit after $after
    return Found { tickets: tickets { ticket_id title priority } }
    outcomes Found
}

MAX is a canonical unsigned decimal literal. Zero, leading zeros, a maximum above 65,534, optional or set wrappers, and defaults outside 1..=MAX are compile-time errors. At runtime, an omitted parameter uses its compiled default; otherwise zero, a wrong type, or a value above MAX is rejected before any provider or storage work.

The compiler charges every binding at its declared MAX, never its default or submitted value. Charges remain cumulative when a query has several bindings. Bounds through 499 retain language V9, query IR V12, and module V12. Bounds from 500 through 65,534 use the additive language V11, query IR V14, and module V14 identities. The maximum is also part of generated input schemas, roles, plans, locks, and cursor-bound operation identity, so changing it requires the normal immutable query deployment rotation.

Fixed take literals retain their exact bound. Limit<MAX> is also accepted as an existing nearest K parameter, but it narrows only returned page work; a projection provider’s separately declared candidate or full-population work remains unchanged. This is not a general integer refinement and cannot be used for offsets, byte budgets, or contract fields.

For an ordinary ordered take $limit after $cursor page, including the final root page after a complete candidate pipeline, the submitted limit controls only that invocation’s cardinality. A continuation may be resumed with another valid value in the same Limit or Limit<MAX> domain. The cursor still binds the immutable plan—including the declared maximum—and every invariant parameter, predicate, order, authority, snapshot, physical profile, and provider epoch. A limit used by nearest, by an unpaged binding, by a candidate source, or by any mixed semantic shape remains cursor-identity-bearing. There is no source annotation for this distinction; the compiler derives it from the closed plan.

For an explicit finite set of aggregate partitions, use Set<Entity.partition_field, MAX> and route the binding with partition_field in $parameter. MAX is 1 through 65,535; the empty submitted set is valid. The database executes one sealed local plan at one snapshot and applies global order, limit, and cursor selection after merging the selected partitions. See Bounded partition-set queries for the complete locality, scale, authorization, candidate, and cursor rules.

Complete candidate sets before root ordering (language V11)

A query may declare one compiler-owned, non-output candidate binding before its ordinary root binding. The binding projects one type-exact root-key component from up to eight declared, same-partition indexes and applies exactly one of single-source deduplication, intersection, union, or authorized difference:

candidates matching: Experiment.experiment_id
    from intersect {
        ExperimentTag.experiment_id using by_tag_digest
            where scope == $scope && tag_key == $first_key && value_digest == $first_digest,
        ExperimentTag.experiment_id using by_tag_digest
            where scope == $scope && tag_key == $second_key && value_digest == $second_digest,
    }
    within 65535
    else IntegrityFailure

many experiments from Experiment
    where scope == $scope && experiment_id in matching
    order by last_update_time desc, experiment_id asc
    take $limit after $after
    else IntegrityFailure

Every source completes and deduplicates independently, then the set operation completes, before any root entity is hydrated, policy-checked, ordered, paged, or assigned a cursor. Exceeding a source, distinct-key, key-byte, hydration, sort, work, or output bound returns only the declared refusal; no partial set, row, hidden count, or cursor is observable. A difference expression must name the complete policy-filtered root-key universe first, followed by its negative sources separated by ;, so denied rows cannot be inferred through complement.

Authorization retains each compiler-declared candidate source as an independent scan ceiling. A role must cover every source individually, and the authorizer checks their bounded sum against the sealed plan cost; one source cannot borrow unused authority from another. Ordinary non-candidate queries continue to use one whole-request scan ceiling.

The compiler proves the relationship mapping, common partition parameter, declared source index, exact key type, finite source count, and exactly one root in consumer. Ordinary index-backed candidate sources accept exact equality as well as the closed operational predicate set (!=, <, <=, >, >=, bounded in, prefix over text_key(...), and is null, is not null, or exists over presence(...)) when the declared source index prefix proves the predicate before projecting the root key. Candidate names, sources, operators, indexes, and budgets do not appear in SDK or MCP schemas. They are immutable module/plan/role identity, not request data. This is bounded same-partition existence filtering, not a general join, recursive query, caller-selected plan, or runtime optimizer.

Bounded one-to-many expansion (language V14)

A later many binding may declare one earlier bounded many binding as its driver and one per-driver target bound:

many comments from Comment
    for each ticket in tickets
    where organization_id == $organization_id
        && ticket_id == ticket.ticket_id
    order by created_at asc, comment_id asc
    take 8 per ticket

The target must use one declared same-partition index whose prefix is bound by the common partition route and driver key. The compiler rejects an absent or later driver, a driver that is itself expanded, a mismatched per name, an independent target cursor or offset, a non-prefix index shape, a cross-partition shape, or a driver-maximum times per-driver-maximum product above 65,535. The return block must nest the target binding directly below its driver binding.

Expansion structure is immutable compiler identity. Generated clients, MCP tools, and application callers still submit only the declared business parameters. Memory and redb execute every target access inside the driver’s one authorized snapshot and release a result only after every driver’s bounded target set is complete. Row and field policy run before a target can affect membership, order, count, or nesting. Generated Rust, Go, TypeScript, and Python clients expose the nested list as a typed field; gRPC, MCP, and CLI use the same name-addressed list-of-record carriage. A target has no independent cursor: paging remains on the driver binding.

Root relationship existence (language V14)

A bounded root collection may filter through one declared same-partition junction relationship without exposing a join plan:

many experiments from Experiment
    where scope == $scope
    exists ExperimentTag using by_tag_digest
        where scope == $scope && tag_key == $key && value_digest == $digest
    order by last_update_time desc, experiment_id asc
    take $limit
    else IntegrityFailure

Write not exists ExperimentTag using ... for the authorized complement. The positive form lowers to an intersection of the route-complete authorized root universe and the junction’s projected root keys. The negated form lowers to the same root universe minus those keys. Both use the existing candidate algebra, its 65,535-key completion ceiling, the root binding’s refusal outcome, and its policy-before-observation behavior; they add no executor step, provider, cursor family, request field, or authority surface.

The junction must declare exactly one relationship to the root’s complete scalar key, preserve the root partition, and name an ordinary index that carries the projected key after a complete predicate prefix. The root must be the first bounded many binding, carry an exact partition parameter, select a route-complete ordered index, and declare else. Unknown or ambiguous relationships, cross-partition mappings, incomplete index prefixes, a missing root-key projection, generated-name collisions, and existence on a dependent binding are source-spanned compilation refusals. The equivalent explicit candidates spelling compiles to identical V18 plan bytes.

A candidate source may instead name a compatible contract pattern_index and one compiled exact pattern predicate. That source is supplied by one ready, policy-aligned provider epoch and is completed under the same participant proof as ordinary sources before root work. Pattern postings only choose candidates; the provider verifies complete retained matched values. See Long-Value Pattern Search for declaration, matching, bounds, freshness, and negation rules.

Finite root-order families (language V12)

When one public operation needs several caller-selected orders, the query may bind a contract enum to a finite family of complete immutable orders:

$order: ExperimentOrder,

many experiments from Experiment
    where scope == $scope && experiment_id in matching
    order by $order {
        NameAsc: name asc, experiment_id asc;
        NameDesc: name desc, experiment_id asc;
        CreatedDesc: creation_time desc, experiment_id asc;
        UpdatedDesc: last_update_time desc, experiment_id asc;
    }
    take $limit after $after
    else IntegrityFailure

The family must declare every variant of its contract enum exactly once and is limited to 32 total compiled members (2^k * |variants| <= 32). Order families may compose with up to eight when $param { ... } optional predicates; the compiler expands every (presence_mask, order_variant) combination before deployment and checks its fields, directions, null placement, unique tie-breaker, authority, cost, and candidate/root semantics independently. The service resolves the submitted presence bits and enum variant to one compiled member before storage work. Generated Rust, Go, Python, TypeScript, MCP, CLI, gRPC, local, and remote paths carry only the declared optional parameters and closed enum; there is no public field list, expression, index hint, collation, direction, or tie-breaker input.

Order families use additive RiffQL V12, query-family IR V15, and module V15. A successful result reports the complete finite family’s compiler identity, which is the identity pinned by generated clients and application catalogs. The selected member plan and the enum parameter are additionally cursor-bound, so a cursor cannot be resumed under another order. Candidate families always capture one admission head; all sources complete at the resulting snapshot before root policy, total sorting, paging, and cursor formation.

Nearest-neighbor bindings (alpha)

A many binding over an entity that declares a contract vector_field may replace its ordering and bound with a nearest clause:

query SimilarDocuments(
    $org_id: Document.org_id,
    $query_vec: Document.embedding,
    $k: Limit<499>,
) {
    source projected Document.embedding
    freshness causal inherit_session_commit true max_wait_ms 500

    many results from Document
        where org_id == $org_id
        nearest(embedding, $query_vec, $k)
    return Found { results: results { title } }
    outcomes Found
}

• nearest is valid only on many bindings. It replaces order by (rows come back in ascending distance order) and take (K is the binding’s checked page bound).
• K is either a positive integer literal, Limit, or Limit<MAX> parameter. All share the 65,534-row structural page-take ceiling; Limit<MAX> also enforces its declared maximum. A provider’s smaller compiled candidate or partition ceiling remains authoritative. Invalid values are rejected before nearest provider or storage work begins.
• The binding still requires the exact partition (organization) equality predicate that routes all query access; a nearest query without it does not compile (RDB-QP002).
• The static plan cost charges the provider’s complete compiled partition-scan ceiling, not K: exact nearest search examines the whole admitted partition.
• Production execution requires the exact compiler-owned source projected Entity.vector_field declaration. The source must match the nearest binding and a production-capable contract vector field; it is never selected from a request or inferred from whichever projection happens to exist.
• freshness available serves the currently published generation and reports its frontier. freshness causal inherit_session_commit true max_wait_ms N waits for the generated client’s scoped commit fence for at most the compiler-bounded duration. freshness bounded max_lag_ms N compares trusted logical timestamps at the authoritative head and projection frontier; it never estimates elapsed time from sequence distance or process wall time.
• A vector source is queryable only while its durable generation is Ready and exactly matches the compiler-owned definition. Initial build, successor rebuild, replay-budget detachment, and restart recovery return typed building/rebuilding failures; they never silently serve the previous or a partially rebuilt generation. A successful result’s frontier is the exact durable checkpoint frontier of that generation.
• The first production form is intentionally one nearest many binding with equality-parameter scalar predicates and fields from that entity. Mixed authoritative/projected bindings, dependent relations, aggregates, and multiple vector sources fail at compile time rather than mixing snapshots.
• Current row policy filters the complete bounded candidate set before scalar predicates, distance calculation, ranking, K, counts, or output. The same capability and policy identity is revalidated before release.
• nearest is a contextual word, not reserved — a contract field named nearest remains fully queryable.

The production path uses the automatically registered Entity.field columnar projection and the shared 500-row default admission ceiling (or up to 65,536 rows per organization when ann_threshold is declared). If the contract declares ann_threshold and recall_target_bps, the server uses exact search at or below that per-organization threshold. Above it, an exactly matched warm HNSW graph may serve the query; cold or unavailable graphs use exact execution. Current organization, model and policy admission still run on every query. The choice and recall floor are compiler-owned; neither is a request option. See Vector Search and Known Limitations.

Event and scalar columnar projected queries

source projected <Target> also compiles named queries over contract-declared event projections (projection <Name>) and scalar columnar projections (columnar_projection <Name>):

1. Event-driven projections (source projected <ProjectionName>):
   • Requires freshness available or freshness causal inherit_session_commit true max_wait_ms N.
   • Binds one, maybe, or many over <ProjectionName>, requiring an exact equality predicate on the leading organization-scope key component, optional leading key equalities, and an optional trailing key range bound (>=, >, <=, <).
   • Group-key components and declared measures (count, sum, min, max, mean) are projected as ordinary typed RiffQL fields.
2. Scalar columnar projections (source projected <ColumnarProjectionName>):
   • Requires freshness available, freshness bounded max_lag_ms N, or freshness causal inherit_session_commit true max_wait_ms N.
   • Binds over the backing <Entity> narrowed to its organization-scope field, primary-key fields, and the fields listed in columnar_projection <Name>, supporting equality/range predicates, order by, take, and aggregate / group by blocks (including UTC temporal bucket keys, quantiles, stddev, and measure-ordered group by).

Accepted application-profile target

The following boundary defines the safe application query profile:

• Stable applications execute exact named deployed queries with typed parameters. They do not submit source text.
• Scoped agents may check, explain, or execute ad-hoc RiffQL only through separately granted permissions.
• Raw GetEntity and ScanIndex remain kernel/administrative operations. Their permissions do not authorize RiffQL, and RiffQL permissions do not authorize them.
• Generated stable-application clients expose neither raw field/index IDs nor constructors for source text, field masks, encoded keys, or kernel requests.

Required same-partition relationships and same-partition unique keys are contract declarations rather than RiffQL predicates. RiffQL may navigate a declared relationship only when it still lowers to the bounded complete-key accesses described above. A query never creates a missing integrity guarantee by convention or inference.

The symbolic catalog exposes each relationship by source name, ordered source fields, target entity, and complete ordered target key. That metadata grants no new join operator. Current RiffQL must still spell an accepted point read or bounded dependent batch; future navigation syntax must compile to those same plans.

Declared uniqueness is not implemented as a RiffQL exists or count preflight. Commands that establish or change a unique tuple carry compiler-derived conflict and transaction-current occupancy plans; a read query cannot acquire write authority or make a later write safe.

Application-safety product rule

An application operation is accepted only when RiffDB can prove its complete typed access program before any data access. The proof must identify the exact contract and operation, authorize every returned field and access path, route every access to one partition, select declared indexes or complete keys, bound all work and output, establish cardinality, and execute the read against one engine-owned snapshot. If any proof is missing, the operation is rejected with a source-spanned symbolic diagnostic.

There is no application fallback to an unrestricted scan, client-side join or sort, partial field redaction, N+1 public requests, cross-partition access, string-built predicate, or optimizer-dependent plan. This is the key difference from treating RiffQL as a smaller SQL dialect: the rejected forms are product safety properties, not optimizer preferences.

Independent Blog/CMS and Orders/Inventory corpora exercise this rule. Both completed without a new grammar, IR node, hash input, access operator, cursor, or result semantic. The positive corpus proves:

• primary-key page and slug-route reads;
• declared-index feeds, moderation queues, histories, and dashboards;
• singular foreign-key reads from prior singular bindings;
• bounded junction-to-entity and line-to-product batches;
• numeric inventory invariants and command-only reservation transitions; and
• identical generated Rust, Go, TypeScript, Python, and MCP operation schemas.

The retained negative corpus rejects an unindexed title ordering (RDB-QP003), a cross-partition author scan (RDB-QP002), collection-as-scalar fan-out (RDB-QP007), and a collection without take (RDB-QS009). Exact spans, classifications, and smallest remedies are generated in fixtures/agent-alpha/gap-report-v1.json.

The original application evidence did not justify computed-field, fragment, or general-join syntax. Those constructs remain outside the language. Operational evidence has since justified the closed version-2 predicate and aggregate declarations below; this does not introduce a general query-expression or optimizer surface.

Operational optional predicates (language version 2)

Named deployed queries may declare up to eight optional predicate parameters and guard a top-level conjunct with the matching parameter:

query SearchTickets(
    $organization_id: Ticket.organization_id,
    $status: Ticket.status?
) {
    many tickets from Ticket
        where organization_id == $organization_id
          && when $status { status == $status }
        order by updated_at asc, ticket_id asc
        take 25

    return Found { tickets: tickets { ticket_id status updated_at } }
    outcomes Found
}

The compiler enumerates the complete presence family at deployment: at most eight parameters and 256 ordinary bounded plans. Every member must independently prove the same partition route, result schema, locality, declared access path, and ordinary RiffQL safety properties. One missing index or unsafe member rejects the whole query with a source-spanned diagnostic; there is no runtime planner, scan fallback, predicate object, or caller-selected field/operator.

At invocation, a value selects the present member; omission or explicit null selects the absent member. Generated bindings expose the parameter as an optional language value. Authorization covers the union of every family member and policy charges the component-wise maximum cost before execution. The query service then selects the exact compiler-owned member by presence and executes it against one snapshot. Values never influence plan selection.

An operational source has a version-2 language and query-module identity. Ordinary version-1 sources retain their original canonical bytes and identities. The ad-hoc source path and version-1 reactive/live-query modules reject operational families until those surfaces receive their own versioned presence-selection contracts.

Version 2 executes is null, is not null, exists, and binary UTF-8 prefix predicates only through explicitly declared operational indexes. For example:

query SearchDocuments(
    $organization_id: Document.organization_id,
    $title_prefix: Document.title,
) {
    many documents from Document
        where organization_id == $organization_id
          && title prefix $title_prefix
        order by title asc, document_id asc
        take 25
    return Found { documents: documents { document_id title } }
    outcomes Found
}

is null matches only an explicitly stored null. is not null matches only a present non-null value. exists matches both explicit null and present non-null values; it does not match a field missing from an older compatible record. The compiler requires presence(field) on the selected index so these three states cannot collapse during planning or execution.

Binary prefix lookup compares the exact UTF-8 bytes of the original string and requires text_key(field, binary_utf8_v1). It performs no case folding, normalization, locale collation, tokenization, or relevance scoring. text_key(field, unicode_fold_v1) instead applies the frozen Unicode 17.0.0 NFKC plus full non-Turkic case fold to stored and submitted strings through one shared implementation. Equality, membership, range, complement, prefix, and order compare folded bytes, while returned fields retain the original authorized string. Neither profile uses a runtime locale or caller-supplied normalizer.

That declared text-key component may also provide ordinary bounded ordering without a prefix predicate. For example, an index (organization_id, title, document_id) text_key(title, binary_utf8_v1) proves order by title asc, document_id asc after the partition equality, and proves the wholly reversed descending order as well. Ordering is by the original canonical UTF-8 bytes: doc-3 sorts before doc6. A plain string index retains RiffDB’s canonical length-first value order and is not reinterpreted as a text key. With unicode_fold_v1, ordering uses the frozen folded bytes instead. The compiler still requires the complete index suffix and deterministic key tie-breaker, and an opaque continuation remains bound to the exact plan, direction, text profile, index epoch, authorization, and snapshot.

Exact equality consumes one complete declared text-key component as an index prefix. This permits progressively narrower compiled queries to reuse one declared index: (organization_id, relation, user, document_id) with text keys on relation and user proves both order by relation, user, document_id and, after relation == $relation, order by user, document_id. Binary equality compares exact UTF-8; Unicode-fold equality compares the frozen folded bytes.

Bounded in may consume one declared index component either when that component is the first remaining order term or when it is a non-partition prefix component followed by the remaining index order by suffix (for example, status in $statuses followed by order by updated_at desc, ticket_id desc on (organization_id, status, updated_at, ticket_id)). When in precedes trailing order columns, the executor performs a bounded k-way merge across the distinct canonical set values with one shared page limit, scan budget, and opaque cursor. Single-partition bindings may also declare mixed asc/desc directions across the ordered index suffix, executing a bounded in-memory group merge within the Whole-request scan ceiling. Submitted sets are sorted and deduplicated once, and an empty set returns an exact empty page without storage work. Multiple non-partition in dimensions in one binding and an in predicate followed by another filtered index component remain rejected rather than forming a Cartesian prefix product or a residual filter.

The ordinary executable component matrix is closed: canonical components support exact equality, bounded membership at the first remaining order term, canonical order, and typed intervals for order-preserving i64, u64, timestamp, date, uuid, and enum components; presence components support state selection and explicit state placement; binary_utf8_v1 supports equality, bounded membership, leading-byte prefix, bytewise interval and complement predicates, and bytewise order; unicode_fold_v1 supports the same roles over its frozen folded bytes. A lower and upper comparison form one half-open physical interval. != forms at most two disjoint complement intervals. Inclusive source bounds are normalized around the complete encoded component before storage traversal, and the interval component must be the first remaining total-order term.

All intervals in one query share a single page limit, continuation probe, scan budget, cursor, and output budget. Reverse traversal reverses both interval order and the rows inside each interval. Contradictory submitted bounds return an exact empty page without scanning. Canonical length-prefixed strings do not preserve logical text comparison and therefore cannot prove a range; declare text_key(field, binary_utf8_v1) for exact valid UTF-8 bytewise interval semantics, or text_key(field, unicode_fold_v1) for folded-byte interval semantics. Physical ordered bytes never replace the returned logical string, and the database performs no token parsing, normalization, or collation. Multiple interval dimensions, overlapping unions, intersections, skip scans, and post-page residual predicates remain unavailable.

Exact indexed result sets (language version 4)

An application that needs an exact whole-population total and numeric offset may declare the closed exact-result shape. It is available only for a bounded binary_utf8_v1 text index, partition-aligned or bounded-row-policy admission, and one compiler-proved total order with the entity key as the unique tie-breaker:

query SearchUsers(
    $organization_id: User.organization_id,
    $needle: User.name,
    $active: User.active?,
    $limit: Limit<499> = 50,
    $offset: u64 = 0
) {
    many users from User
        where organization_id == $organization_id
          && when $active { active == $active }
          && name contains $needle
        order by name asc, user_id asc
        take $limit offset $offset

    aggregate total from users { exact_count() as value }

    return Found {
        users: users { user_id name }
        total: total { value }
    }
    outcomes Found
}

The same provider epoch supplies users and total. exact_count() counts the complete authorized matching population, not the returned page or a candidate cap. offset is a direct indexed ordinal: zero selects the first row, exact-end and beyond-end return an empty page with the unchanged exact total, and no earlier row or cursor page is read or discarded. Offset stability is scoped to the selected snapshot; a later current-snapshot request may see intervening writes.

When the selected role carries a row predicate, the background provider first evaluates one bounded authoritative candidate set under the current capability revision. Only admitted rows enter its count, order, and ordinal structures. This is not request-time filtering: each capability revision selects separate derived state, and a revision change requires fresh activation before release.

Healthy binary-text and exact-predicate providers replay retained V3 receipt post-images in bounded passes of at most 64 application commits and one source byte page. Missing receipt history, oversized indivisible groups, and changes to indexed policy dependencies can require a complete snapshot rebuild. Attached followers currently use that rebuild path because they retain applied progress without the primary’s receipt history. Provider checkpoints keep their existing complete encoding and validation.

Background rebuilds read rows, catalog evidence, and row-policy evidence from one captured authoritative view. A healthy prior provider stays selected while its replacement is prepared, and the replacement checkpoint is reopened and validated before selection. The provider reports its actual captured frontier; Latest requests still require that frontier to equal their captured application head and may receive a typed freshness refusal while it lags.

The closed predicates are ==, starts_with, ends_with, and contains. They compare exact UTF-8 bytes; missing and null values do not match, and an empty needle is rejected. The compiler refuses absent indexes, unsupported policy shapes, non-total orders, excessive bounds, and any combination the provider cannot prove. There is no scan, page-walk, client-filter, approximate count, or fallback execution path.

The current public source shape admits the complete partition equality, one exact-text predicate, and at most one top-level optional typed equality guard. The guard must use the same optional parameter on both sides, target a distinct entity field, and compile in every presence member against declared indexes. When absent, the V3 provider uses its complete admitted posting family; when present, it selects the disjoint canonical-value posting family before count and ordinal selection. Additional, required, disjunctive, or otherwise unimplemented predicates are rejected at their source span and are never retained only as metadata or silently omitted by execution.

Ascending and descending text orders are separate compiler-fixed named plan members; callers pass no field name or sort expression. Both append the complete ascending entity-key tie breaker. Applications that need several operator/order choices declare several named queries, preserving a finite generated surface rather than accepting an arbitrary query structure.

Exact-result requests currently start a fresh one-page result and therefore do not accept an ordinary query cursor. A causal minimum may be supplied through the normal generated read-after-commit option. Rebuild and capacity return temporary query unavailability; divergence, retired snapshots, and an unsatisfied freshness floor return the typed RDB-PROJECTION-0101 through RDB-PROJECTION-0103 application errors.

Bounded exact aggregates (language versions 2 and 8)

Version 2 also reserves a closed aggregate declaration over one earlier, bounded collection binding:

query TicketSummary($organization_id: Ticket.organization_id) {
    many tickets from Ticket
        where organization_id == $organization_id
        order by ticket_id asc
        take 50

    aggregate summary from tickets {
        group by status, priority
        count() as ticket_count
        sum(story_points) as total_points
        min(created_at) as earliest
        max(updated_at) as latest
    }

    return Found {
        summary: summary {
            status
            priority
            ticket_count
            total_points
            earliest
            latest
        }
    }
    outcomes Found
}

The grammar is deliberately closed: a query may declare at most 16 aggregate results, each with at most eight grouping keys and 16 measures. Version 2 provides count(), sum(field), min(field), and max(field). Additive language version 8 is selected when a query uses any of:

• count_present(field), which excludes canonical NoValue;
• count_distinct(field), which includes NoValue as one exact typed value;
• count_distinct_present(field), which excludes NoValue;
• mean(field), which returns ExactMeanV1 { total, count } without division, rounding, or floating point;
• any(field) and all(field) over required or optional Boolean fields (bool and bool?, skipping NoValue);
• sum(field) and mean(field) over optional numeric fields (i64?, u64?, decimal<P,S>?) and fixed-currency money<C> / money<C>? fields (widened to decimal<39,2> at fixed scale 2, skipping NoValue);
• conditional per-measure where <predicate> filters (count() where status == TicketStatus.Open as open_count, sum(amount) where ... as ...);
• deterministic UTC temporal grouping buckets hour(ts), day(ts), week(ts) (ISO Monday 00:00:00 UTC), month(ts), and year(ts) in group by with optional as <alias>, plus as <alias> on plain group keys and aggregate return field selections;
• post-aggregation having <predicate> filters over group keys and measures, order by <key_or_measure> asc|desc [nulls first|last] sorting, and top-N take <N> truncation when order by is declared;
• exact nearest-rank order statistics median(field) (or p50(field)), p90(field), p95(field), and p99(field) over non-null ordered scalars;
• source-order positional selectors first(field) and last(field) over non-null scalars;
• exact fixed-scale population standard deviation stddev(field) (or parser alias variance(field)) over numeric and money<C> fields via checked integer square root (u128::isqrt) without floating point;
• bounded scalar collections collect(field) and collect_distinct(field) returning List<T> capped by the 256-element per-measure budget; or
• per-parent nested rollups aggregate <name> from <child_binding> per <parent> folding a for each <item> in <parent> expanded binding per parent row and nesting the rollup inside <parent> { ... } in return.

Function names, input fields, aliases, grouping keys, and the source binding are source declarations; callers cannot submit any of them at runtime.

This syntax and its version-3/version-11 query-module representations are executable for named queries. Finite-family compilation resolves every grouping, filter, having, order by, and measure field, derives result types, includes all source fields in the authorization union, clamps grouped output to the source row limit, and seals the descriptors and cost ceiling into module identity. The ordinary version-1 compiler still rejects aggregate declarations with RDB-QP008.

The selected source access and every aggregate fold run inside the same authoritative read snapshot. count and sum return zero for an empty whole set (or an empty per-parent rollup); min, max, median/p50..p99, first, and last return absence; collect and collect_distinct return an empty list. Integer, fixed-decimal, and money<C> sums and stddev use checked 128-bit integer arithmetic and cross the public boundary as an exact decimal with the source scale (2 for money<C>) and no floating-point approximation. Overflow, group-bound exhaustion, fuel exhaustion, or a malformed backend row withholds the complete query result—there is no partial aggregate response.

The exact aggregate core independently bounds admitted rows, groups, distinct values per measure/group, partial-state bytes, checked arithmetic operations, scans, result cells, and encoded output. Empty distinct and present counts are zero; empty mean is { total: 0, count: 0 }; empty any is false; and empty all is true. A bound or arithmetic failure refuses the whole query.

Grouped rows are ordered by the declared order by clause (with canonical group-key tie-breaking) or by the canonical encoded group key when order by is omitted. Their count is clamped by the source take value (including a submitted Limit), by the product of the grouping keys’ finite enum/bool/optional domains when the source take exceeds 256, or by an explicit group by ... take <N> cap. When group by is paired with order by, take <N> truncates the sorted groups (after having) to the top N; without order by, exceeding an explicit group by ... take <N> cap at runtime withholds the complete aggregate result rather than silently truncating unordered groups. Aggregate live watches remain a separate future surface: current reactive compilation accepts only an ordinary single access program and cannot substitute one operational-family member.

Symbolic catalog schema

The versioned application-catalog response model is frozen as riffdb.application-catalog/v1 with at most 100 authorized symbols per page. It can represent contracts, enums, entities, fields, relationships, indexes, commands and outcomes, events, query modules and queries, roles, generated operations, and a closed feature-preflight registry. Paths contain names only; types and source-relative spans are bounded.

The type deliberately has no hidden or total count, per-symbol visibility bit, numeric compiler ID, capability record, raw IR, storage key, source text, or filesystem path. Unauthorized and nonexistent symbols therefore have the same representation: absence from the authorized page. The exact registry and its forbidden-field ledger are frozen in fixtures/riffql/application-catalog-schema-v1.json.

The catalog is available through the public ApplicationQueryService.GetApplicationCatalog RPC, the Rust client’s get_application_catalog method, and the riffdb_application_catalog MCP tool. Each request selects an active or exact contract, requests between 1 and 100 visible symbols, and may carry the preceding opaque cursor. MCP uses the same canonical 32-character lowercase cursor spelling as its other bounded discovery tools.

Catalog access requires contract-read authority. Command and named-query symbols are then filtered by the caller’s exact operation grants before the page is formed. A continuation is bound to the principal, exact contract, active query-module identity, requested limit, and prior visibility. A later authorization change may only narrow subsequent pages. Invalid, expired, cross-principal, or identity-stale cursors fail as invalid input; RiffDB never falls back to a first page or discloses which hidden symbol changed.

The feature list is a closed preflight view of the selected application surface. available means the exact compiler/runtime path implements the feature; unavailable is an explicit result and must not be interpreted as permission to emulate the feature with kernel reads or client-side scans.

Bounded partition-set queries

RiffQL can execute one compiler-sealed operational query across an explicit, finite set of aggregate partitions while preserving database-owned ordering, authorization, snapshots, and cursor semantics. Use the additive bounded set form on the entity’s declared partition field:

query SearchRuns(
    $experiment_ids: Set<Run.experiment_id, 1000>,
    $limit: Limit<5000> = 1000,
    $after: Cursor?,
) {
    many runs from Run
        where experiment_id in $experiment_ids
        order by start_time desc, run_id desc
        take $limit after $after
    return Found { runs: runs { experiment_id run_id start_time } }
    outcomes Found
}

MAX is a canonical integer from 1 through 65,535. Generated Rust, Go, TypeScript, Python, gRPC, CLI, and MCP inputs expose one bounded list. Submitted values are type-checked, sorted by canonical encoding, and deduplicated before authorization or execution. An empty list is valid and returns the declared empty result without storage or provider work.

The compiler accepts only partition_field in $bounded_set as the route for every access in the query. It selects one partition-prefixed declared index, replicates the same immutable local plan, proves one global total order, and multiplies authority and work from the declared maximum—not the submitted count. Values discovered by another binding, an index, a provider, or a relationship cannot become routes. Cross-partition joins, writes, aggregates, caller-selected plans, and partial per-partition results remain unavailable.

Invariant exact filters may occupy index components between the partition and order suffix. For example, MLflow lifecycle filtering uses (experiment_id, lifecycle_stage, start_time, run_id) with experiment_id in $experiment_ids, lifecycle_stage == $lifecycle_stage, and order by start_time desc, run_id asc. RiffDB applies lifecycle membership inside every partition scan before the global merge. Enum constants such as LifecycleStage.Active have the same semantics. The filter value is bound into the plan or cursor parameter identity; the adapter never fetches and discards the opposite lifecycle.

Execution opens one storage read view for the complete operation. Uniform index orders use a bounded k-way heap: one initial row is observed from each selected partition, then only the winning stream is advanced. Mixed-direction orders use the compiler-bounded complete-group path. Candidate indexes and long-pattern providers complete independently in every selected partition. Partition providers may rebuild independently. The merged result binds every selected generation and requires one common servable commit epoch. Its candidate identity is the pair (partition, key) before root hydration and global ordering. Row policy is applied below each stream and no denial releases partition counts or partial output.

One opaque continuation represents the global page. It binds the normalized route set, every invariant filter, the selected plan and role, the last global order/key/partition marker, and a domain-separated digest of all observed partition or provider epochs. Page size may change on resume within its declared Limit<MAX> domain. Route changes, epoch drift, policy drift, tampering, expiry, or observation loss fail closed as a typed cursor error; the cursor never exposes one token or epoch per partition.

The structural page ceiling remains 65,534 rows, not the predecessor 499-row limit. Query-specific result-byte, route-byte, partition, scan, policy, candidate, provider, merge, and response ceilings remain independent. Uniform heap execution therefore supports large pages without work proportional to partitions × page size. Mixed orders use a separate 67,107,840-physical-row whole-operation ceiling, enough to retain the full 65,535-row local scan domain through 1,024 declared partitions; larger declared route sets receive a proportionally smaller per-partition allowance.

This surface uses RiffQL V13. A route followed directly by its order suffix uses query IR/module V16; a route followed by invariant exact index-prefix predicates uses additive query IR/module V17. Existing V16 and scalar-route queries and their modules, locks, roles, generated bindings, and cursors retain their prior exact bytes. Adopting either bounded partition-set shape requires normal query recompilation, module deployment, role binding, and binding regeneration; authoritative stored data does not migrate.

Long-value pattern search

long_pattern_v1 is RiffDB’s exact, bounded pattern facility for strings that are too large for ordinary operational text keys. It participates in a named RiffQL candidate binding before final root authorization, ordering, limit, and cursor selection. It is not tokenized search, SQL, a regular-expression API, or a request-time fallback scan.

Declare the provider

A contract attaches one provider to one bounded string field and freezes every operator, matching profile, lifecycle setting, and resource ceiling:

pattern_index by_name_pattern(
  name,
  profile unicode_fold_v1,
  operators (equals, starts_with, ends_with, contains, like, ilike,
    not_like, not_ilike),
  max_source_bytes 8000,
  max_matched_bytes 144000,
  max_rows 65535,
  max_total_matched_bytes 268435456,
  max_grams_per_row 8000,
  max_distinct_grams 65535,
  max_postings 1048576,
  max_postings_bytes 67108864,
  max_pattern_bytes 8000,
  max_pattern_atoms 8000,
  max_candidates 65535,
  max_verification_bytes 268435456,
  max_results 65534,
  staleness_slo 60,
  replay_age_seconds 86400,
  replay_bytes 1073741824,
  replay_backlog 100000,
  retained_generations 8
)

All values are checked compiler input and provider identity, not caller cost controls. Source strings may be at most 8,000 UTF-8 bytes. Unicode folding can expand the retained matched form, so its independent maximum may be as high as 144,000 bytes. The provider does not widen RiffDB’s 4,096-byte storage-key ceiling and does not copy quadratic substring keys.

Use it before root ordering

The provider is named only in compiled query source. Generated clients submit the typed pattern, ordinary invariant parameters, page limit, and opaque cursor; they cannot choose a provider, gram, matcher, scan, sort field, or budget.

Declare the pattern parameter with the provider field’s type, for example $name_pattern: Experiment.name. An explicit bounded string with the same bound, such as $name_pattern: string<500>, is equivalent. RiffDB resolves both forms to the exact contract value type and rejects an unknown field or a mismatched bound at the parameter or predicate source span.

candidates matching_experiments: Experiment.experiment_id
    from intersect {
        ExperimentTag.experiment_id using by_tag_digest
            where scope == $scope
                && tag_key == $tag_key
                && value_digest == $tag_digest,
        Experiment.experiment_id using by_name_pattern
            where scope == $scope
                && name ilike $name_pattern,
    }
    within 65535
    else IntegrityFailure

many experiments from Experiment
    where scope == $scope
        && experiment_id in matching_experiments
    order by last_update_time desc, experiment_id asc
    take $limit after $after
    else IntegrityFailure

Every source completes and deduplicates before the set intersection. RiffDB then hydrates the complete root candidate population in one authoritative snapshot, reapplies root policy, derives every total-order key, and sorts before selecting the page or minting a cursor. It never filters a pre-paginated root page or stops when it has found enough rows for the requested limit.

Exact matching semantics

binary_utf8_v1 compares canonical UTF-8 bytes. unicode_fold_v1 applies the pinned Unicode fold before matching. Equality and literal starts_with, ends_with, and contains do not interpret wildcard characters.

For like and ilike, % matches zero or more matched-form Unicode scalar values and _ matches exactly one. Backslash escapes only %, _, and backslash; an incomplete or different escape is refused. Matching uses bounded linear-space state.

Digests and mandatory three-byte literal grams are candidate filters only. Every candidate’s complete retained matched value is verified before membership, so hash collisions and gram false positives cannot alter results. A short or wildcard-only pattern with no mandatory gram performs a compiler-bounded scan of retained provider values, never an authoritative entity scan.

Negated membership is expressed as candidate difference from a complete, policy-authorized positive universe. A direct not_like or not_ilike provider source is refused because complementing only provider-visible rows could reveal hidden membership.

Lifecycle and bounds

Background workers build policy-aligned provider state from one exact authoritative snapshot and atomically activate a checksummed generation. A request uses only a ready generation whose descriptor, schema, policy shape, plan, history incarnation, frontier, and work observation match the compiled query. On first use or catch-up, RiffDB registers every compiler-bounded provider participant together and waits within the request deadline and its finite server-owned readiness ceiling for the background worker to publish a sufficient generation. Current authority is revalidated after every wait. Callers cannot select the worker, cadence, ceiling, provider, or fallback, and the request path never performs the rebuild itself.

If building, rebuilding, or freshness work does not finish inside that bound, or if capacity exhaustion, corruption, stale state, or generation retirement is observed, the query returns the corresponding typed refusal. It never reports provider activation as generic storage unavailability and never serves a lower frontier or partial candidate population.

The provider and ordinary candidate sources share one admission-head-fenced participant proof. For a finite partition set, that proof binds every constituent generation even when partitions rebuild independently. A cursor binds that participant set and the complete root order. If its authoritative snapshot or provider generation can no longer be served, continuation fails typed instead of switching to current state.

A live continuation retains the actual immutable provider values for each compiled step and selected partition, independently of the discovery ring. Resolved requests retain their own references. Expiry, successful replacement, and cancellation release only their respective ownership; they preserve an already captured request or the prior cursor when replacement fails. This does not relax authoritative-snapshot availability or freshness checks.

Candidate, verification, result, replay, checkpoint, retention, root hydration, policy, sort, and output work are independent. The structural page maximum is 65,534 rows with one reserved continuation probe, but the compiled query’s own maximum and the unchanged 4 MiB encoded-result and transport ceilings can make the practical page smaller for wide rows.

Tokenized text search

RiffDB tokenized search is a finite family of compiled named queries over a declared text_index. It is separate from exact-text starts_with, ends_with, and contains: tokenized search matches analyzed terms, while exact text compares the stored value’s bytes.

Declare the index

One entity declaration names the indexed fields, their positive integer weights, one analyzer, provider lifecycle bounds, and query budgets:

text_index search(
  (title weight 4, body weight 1),
  analyzer standard_v1,
  staleness_slo 60,
  replay_age_seconds 86400,
  replay_bytes 1073741824,
  replay_backlog 100000,
  result boolean_v1,
  max_terms 16,
  max_candidates 10000,
  max_results 1000
)

keyword_v1 preserves the complete value as one term. It is intended for tags, identifiers, and enum-like strings. standard_v1 applies Unicode 17 word segmentation and then the pinned Unicode fold. Neither analyzer stems words or uses a language-specific stop-word list. Analyzer identity and field weights are part of the contract and provider identity; changing either rebuilds the derived index.

The three query budgets are whole-operation ceilings. An empty analyzed query, too many terms, too many candidates, or too many results returns a typed refusal. RiffDB never truncates an oversized match set and never falls back to an entity scan.

Compile a named operation

RiffQL V10 has exactly four match shapes. The source chooses the index, shape, and—only for proximity—the positive distance:

query search_docs(
    $org: Document.organization_id,
    $query: Document.title,
    $limit: Limit<20>,
    $cursor: Cursor,
) {
    many docs from Document where docs.organization_id == $org
        matching(search, conjunction, $query, riff_bm25_v1)
        order by docs.doc_id asc
        take $limit after $cursor
    return { doc_id: docs.doc_id title: docs.title }
}

Replace conjunction with disjunction, phrase, or proximity, 5 to compile the other shapes. Conjunction requires every analyzed term. Disjunction requires at least one. Phrase requires consecutive positions in one indexed field. Proximity requires the terms in order in one indexed field, with each adjacent distance no greater than the compiled bound. Repeated intermediate terms are considered at every reachable position: an earlier occurrence cannot hide a later occurrence that completes the match.

Omitting riff_bm25_v1 gives Boolean execution and canonical entity-key order. Adding it selects the only ranked order. Applications cannot submit a field, analyzer, shape, distance, weight, score function, boost, tie-breaker, provider, budget, cost hint, scan, or fallback at runtime. Generated Rust, Go, TypeScript, Python, gRPC, CLI, and MCP operations expose only the query’s typed values and the ordinary opaque cursor/options carriage. Scores are not public result fields.

Deterministic ranking

Ranking never changes membership. It orders the exact Boolean match set over the caller’s complete authorized corpus at one admission-head-fenced snapshot. Unauthorized rows contribute to neither results nor document-frequency and field-length statistics.

riff_bm25_v1 uses checked unsigned integer arithmetic and scale S = 1,000,000. For authorized document count N, term document frequency df, term frequency tf, document field length dl, total authorized token length L for the field, and its contract weight:

idf_scaled = floor(S * (N - df + 1) / (df + 1))
denominator = tf*S*L + 1200*(250*L + 750*dl*N)
tf_scaled = floor(S * tf * 2200 * 1000 * L / denominator)
term_contribution = floor(weight * idf_scaled * tf_scaled / S)
score = checked_sum(term_contribution)

Every division rounds toward zero. Equal scores use canonical entity-key bytes ascending. Arithmetic that cannot fit the frozen checked representation is a typed whole-result refusal, not saturation.

Freshness, rebuilds, and cursors

The production request path uses only a ready, policy-aligned posting segment. A background worker builds and checkpoints it from one bounded authoritative snapshot. Initial build, rebuild, capacity exhaustion, stale state, corruption, and an unavailable epoch are typed provider lifecycle outcomes. There is no request-time authoritative scan or stale-generation fallback.

A ranked first page captures admission-head consistency and binds its causal floor, provider epoch and generation, authorized statistics identity, immutable plan, invariant parameters, current capability identity and revision, history incarnation, total order, and result ceiling to the opaque cursor. Continuations reuse that snapshot and never recapture statistics. The production POC keeps eight epochs in each registered query shape’s discovery ring. A live ranked cursor additionally retains its actual immutable epoch, even after that epoch leaves the ring. Its native state stays charged until the cursor and any captured requests release their references. Cursor expiry, replacement, and cancellation release only their own references. If the bound epoch becomes unavailable—or the cursor expires, authority changes, or history is restored— the continuation fails typed rather than rescoring against newer data.

Boolean results do not expose a tokenized continuation. They use the compiled bounded numeric offset and canonical entity-key order; separate requests do not freeze the corpus across writes. Ranked pages use the ordinary generated cursor option and may request a different valid page size without changing the frozen operation identity.

Deliberate exclusions

V1 has no wildcard, regular expression, fuzzy matching, stemming, language packs, highlighting, snippets, caller-selected relevance controls, score output, hybrid vector/text ranking, general joins, or ad hoc search endpoint. Applications export their changelog or authoritative data when a different search product is the better fit.

Ranked execution computes each distinct query term’s document frequency and each field’s corpus length once for the captured provider snapshot. Scores and statistics identity share those values. Repeated query terms still contribute repeatedly to the frozen fixed-point score; they do not alter corpus identity.

Exact predicate and order families

RiffQL language V6 freezes a provider-independent semantic model for bounded, exact operational result sets. It is a compiler surface, not a public query object. Applications still invoke named generated operations; callers never submit fields, operators, Boolean trees, order expressions, provider choices, cost hints, or fallback flags.

The compiler accepts the following closed predicate vocabulary inside one partition-routed many binding:

• ==, !=, <, <=, >, and >= over one exact scalar type;
• bounded typed in and not_in sets;
• is null, is not null, and exists state tests;
• binary UTF-8 prefix, starts_with, ends_with, and contains;
• conjunction and compiler-capped disjunction; and
• when $optional { ... } guards whose complete presence family is compiled before deployment.

Value predicates are two-valued. Missing and explicit-null fields satisfy none of the value or complement operators. Use a state test when state is relevant. An empty in set matches nothing; an empty not_in set matches every present, non-null value. Submitted sets are sorted and deduplicated canonically before they enter execution or identity. Text uses exact binary UTF-8 semantics; no locale, regex, wildcard, or implicit normalization exists.

An exact family also declares a nonempty total order independent from its search fields. The order ends with the complete canonical entity key in ascending direction. Order fields must be present and non-null for the whole installed lineage. The query combines that order with exact_count() and a bounded take $limit offset $offset window:

query SearchUsers(
  $organization_id: User.organization_id,
  $needle: User.email,
  $states: Set<User.state>,
  $after: User.created_at?,
  $limit: Limit<499>,
  $offset: u64
) {
  many users from User
    where organization_id == $organization_id
      && email contains $needle
      && state not_in $states
      && when $after { created_at >= $after }
    order by created_at desc, user_id asc
    take $limit offset $offset

  aggregate totals from users { exact_count() as total }
  return Found {
    users: users { user_id email state created_at }
    totals: totals { total }
  }
  outcomes Found
}

Nullable order placement

RiffQL V7 adds explicit placement for an order field that may be missing in an installed schema version or may contain null:

order by subtitle asc nulls last, created_at desc nulls first, record_id asc

Only nulls first and nulls last are accepted. They are part of the named query and its plan/module identity; an application request cannot choose or override them. Omission is valid only when the compiler proves every admitted row and installed schema version has a present, non-null value. The complete ascending entity-key suffix is always present-only and cannot carry a null placement.

For ordering, a missing field and an explicit null are one NoValue class. They compare equal for that term, so later order terms and finally the complete entity key resolve the tie. Direction changes only value-versus-value order: desc nulls last still places every NoValue after all present values. RiffDB does not inherit SQL, storage-engine, locale, or host-language null defaults and does not substitute a sentinel scalar.

The compiler makes the V7 source and its provider-independent V10 semantic/module artifacts compilable. Rebuildable provider-state V5 activates its internal exact-count/direct-ordinal execution lane and connects that lane through the shared service, gRPC, CLI, MCP, and generated Rust, Go, TypeScript, and Python methods. There is no scan, request-time sort, page walk, or older- provider fallback.

Declare every nullable placement in the checked-in named .riffq document, then regenerate the application lock and typed clients. Generated callers pass only the operation’s domain parameters, limit, and offset; there is no sort descriptor to construct. The framework-neutral InventoryRecord corpus in fixtures/adapters/operational-conformance demonstrates optional text and timestamp orders in both directions and placements, later-term NoValue ties, exact totals, direct ordinal boundaries, and missing/null/value transitions.

Every predicate and order field requires a declared partition-routed index. One missing index, invalid type, incomplete key tie-breaker, excessive Boolean shape, or unsupported family member rejects the complete named query with a source-spanned diagnostic. There is no partial family and no runtime scan.

Current activation status

RiffQL V6 source, semantic IR V9, and query-module V9 are compilable and reproducible. Rebuildable provider-state format V4 is active through the shared application service, gRPC, CLI, MCP, and generated Rust, Go, TypeScript, and Python methods. The compiler seals optional-family members, comparison profiles, policy mode, exact-count and ordinal requirements, independent order layouts, and static work/state bounds into its provider identity. Existing V1–V8 source, plan, module, and V1 through V3 provider identities remain byte-exact.

Explicit nullable placement selects the additive V7/V10/V10 identities even when the current field is required. Sources without placement continue to emit V6/V9/V9 when the complete present/non-null proof succeeds; earlier bytes and hashes are not reinterpreted.

For a successor contract, compilation that omits placement must receive the complete contiguous lineage beginning at version 1. A current bundle by itself, an out-of-order lineage, or a lineage with a version gap is not proof and fails closed with a diagnostic suggesting nulls first or nulls last.

V4 builds one policy-aligned partition in the background. Equality, range, set, state, and exact-text postings combine inside that provider. Each declared order has its own bounded ordinal index; exact count and offset selection use bitset rank/select rather than walking skipped rows or pages. Bounded row-level policy admission happens before a row can enter the provider universe, so a denied row cannot affect complements, counts, ordering, ordinals, diagnostics, or lifecycle selection.

V5 uses the same partition-scoped lifecycle and admission boundary for nullable orders. It stores missing and explicit null in one NoValue state class, with the compiler-selected first/last rank, and stores present values under the existing canonical scalar comparison. Optional text and timestamp orders are covered by the same state-aware index. Every V5 checkpoint binds the compiled program, descriptor, policy shape, partition, history incarnation, generation, frontier, and checksum; V4 refuses V5 bytes rather than reinterpreting them. Descriptor/layout validation is paid when a provider generation is bound, and the epoch proof is paid once for the opened result set—not once per row, term, probe, or returned item.

Callers submit only the generated typed values for one named operation. They do not submit a predicate, field, operator, order, provider, plan member, policy, or cost. The service materializes and type-checks values before authorization, selects the compiler-enumerated optional-presence member, and reauthorizes both before provider execution and before releasing values. A V6 query never degrades to an authoritative request-time scan, client-side filter or sort, page walk, materialized full population, cross-provider identifier transfer, or a nearby older provider.

The initial request for a new plan/policy/partition slot waits for a fixed, bounded readiness window while background construction remains the sole owner of authoritative scans and provider publication. It can still return RDB-QUERY-0102 when construction does not finish within that window. The same typed lifecycle, freshness (RDB-PROJECTION-0103), retired-snapshot, divergence, and revocation failures cross every public transport. The application may use a bounded retry budget; it may not emulate the result while the provider is unavailable.

Healthy binary-text and exact-predicate providers replay retained V3 receipt post-images in bounded passes of at most 64 application commits and one source byte page. Missing receipt history, oversized indivisible groups, and changes to indexed policy dependencies can require a complete snapshot rebuild. Attached followers currently use that rebuild path because they retain applied progress without the primary’s receipt history. Provider checkpoints keep their existing complete encoding and validation.

Background rebuilds read rows, catalog evidence, and row-policy evidence from one captured authoritative view. A healthy prior provider stays selected while its replacement is prepared, and the replacement checkpoint is reopened and validated before selection. The provider reports its actual captured frontier; Latest requests still require that frontier to equal their captured application head and may receive a typed freshness refusal while it lags.

Numeric offsets are bounded direct ordinal selections inside one provider epoch. Use generated cursor pagination instead when a user journey spans multiple requests and must retain snapshot identity across concurrent writes; do not turn successive numeric offsets into a client-side page walk.

The immutable generic capability receipt is fixtures/riffql/operational-query-capability-v5.json. It preserves V1 through V4 byte-for-byte, so the V6/V7 exact-result plans, generated-language artifacts, and strict TypeScript helper-reachability gate retain their prior evidence. V5 adds binary UTF-8 interval/complement operations to the separately governed ordinary access-path corpus and binds one value-free external source- compilation and Go-only package receipt. It does not claim an external runtime, framework route, or full application has passed: that evidence remains the responsibility of the repository owning the adapter.

The separately checksum-bound fixtures/riffql/wp775-external-binary-text-interval-execution-v1.json is terminal evidence for one much narrower external execution. It records that the owning consumer invoked one compiler-generated strict-lower/strict-upper binary-text query twice, received original logical strings in bytewise ascending order, and reused one returned string as its consumer-owned next lower endpoint. Its closed value-free facts exclude client shaping, a duplicate index, database-owned token conversion, and framework or full-application claims. It does not alter or advance V5, and neither external values nor consumer vocabulary enter this repository.

RiffQL v1 planning

riffdb-query-compiler compiles one parsed query against one exact immutable contract catalog. Successful compilation produces a QueryAccessProgramV1; execution does not receive source text or perform access-path discovery.

The v1 planner accepts only:

• one exact partition parameter shared by every binding;
• complete primary-key point reads for one and maybe;
• bounded many reads whose equality/membership prefix and complete ordering match one declared index;
• a bounded dependent primary-key batch where one earlier many field supplies exactly one complete-key component through in, the target bound does not exceed the source bound, and a missing-target outcome is declared;
• one wholly forward or wholly reverse traversal direction;
• explicit application-visible row bounds no greater than the 65,534-row page ceiling (the 65,535-row physical scan ceiling reserves one continuation probe).

Operational version-2 members may additionally use null/existence or binary prefix predicates when the selected contract index carries the matching sealed physical encoding. presence(field) expands one logical optional field into a missing/null/value discriminator and a typed payload component. The executor forms only the finite discriminator prefixes required by the source predicate; it never fetches an unbounded candidate set to answer existence. A text_key(field, binary_utf8_v1) component uses a zero-escaped ordered-byte codec, allowing an exact leading-byte range while returning the original field value. The same component may provide an ordinary bounded order by field suffix even when the query has no prefix predicate on that field. Its order is the exact UTF-8 byte order frozen by binary_utf8_v1, not the canonical length-first string order. Exact equality on that field consumes the complete text-key component and advances the ordered suffix, so one index can serve a wider order by relation, user query and a narrower relation == $relation plus order by user query. Runtime converts the typed equality string to the same physical profile bytes only while constructing the index prefix; residual predicate evaluation still compares the original typed value. The remaining index suffix must equal the complete declared order, including its deterministic entity-key tie-breaker; forward and reverse cursor traversal use the same encoded key bytes. Compiler, command index maintenance, and both storage readers consume the same versioned key schema.

The compiler’s closed component registry treats equality, membership, interval/complement, state, prefix, order, and tie-breaking as composable roles of one declared physical component. A component is not assigned one exclusive purpose: for example, binary_utf8_v1 may consume bounded membership or one bytewise interval/complement and produce the first remaining order term in the same plan. The caller cannot select any role, index, encoding, comparator, or fallback.

After typed parameter validation, the executor forms one bounded ordered range schedule per access step and request. Canonical set normalization, text-profile encoding, typed endpoint construction, range sorting, deduplication, and shape validation happen once. One page, continuation probe, scan/fuel allowance, policy/hydration allowance, cursor, and output budget covers the entire schedule. Memory and redb consume that semantic schedule; neither backend may apply a logical predicate after page selection or invent a storage-local profile transform.

The program contains ordered accesses, dependency edges, cardinality and row bounds, and the complete entity/field/index authorization requirement. The application supplies names; stable numeric IDs remain compiler-internal. Authorization is fail-closed before execution and never silently removes a selected field.

Canonical program bytes are hashed in the accepted riffdb.query-plan/v1 domain. Explain output is a bounded, deterministic, name-only view of the same program. An absent compatible index produces RDB-QP003 with a source span and a suggested symbolic index declaration. Every predicate and the complete total order must be represented by that one selected index. An order-compatible index cannot leave a predicate for post-scan filtering: doing so could apply take, an ordinal offset, or a continuation before the predicate and silently omit matches.

RiffQL v1 does not perform an unbounded fallback scan, client-side sort, cross-partition join, or optimizer-dependent plan choice. The additive finite partition-set route is not a join: it replicates one compiler-sealed local index plan over caller-submitted bounded routes in one read view. The same bounded multi-stream operator also executes single-partition non-partition field in $set prefixes followed by trailing order by index columns and single-partition mixed asc/desc order suffixes. Uniform orders use a bounded k-way heap; mixed physical directions use a bounded complete-group path. Both produce one globally ordered page and one opaque cursor only after whole-set policy and epoch observation succeeds. Compiler-proved exact predicates may consume index components between the partition route and order suffix. Those values form each local physical prefix and are applied before merge, limit, and cursor selection; an absent compatible covering index remains a typed RDB-QP003 refusal rather than a residual filter.

When a declared index includes cover (...) fields and covers every predicate, order, dependency, and selected or aggregated field of an ordinary Index step (across single-step or multi-binding queries, singular one/maybe lookups, or operational aggregate source bindings) on an entity without row-read policy, the compiler seals a per-step CoveredResultLayoutV1. Covered steps reconstruct rows directly from index entries (scan_covered) without primary-key point reads and charge zero point reads in the whole-query cost vector.

One-level expansion repeats a compiler-selected same-partition index access for each row of an earlier bounded driver. Planning proves the driver maximum, per-driver maximum, product ceiling, index-prefix equality, driver-first then index order, and direct result nesting before emitting the V18 access tag. An expanded binding cannot drive another expansion and has no independent cursor. Rejected operational shapes additionally carry an anonymized class containing only operator, cardinality, and partition categories; names, identifiers, and submitted values are excluded.

Candidate-set plans

Language V11 candidate bindings compile to complete internal source steps plus one root-hydration step. Each ordinary source uses the exact declared index, partition equality, predicate prefix, projected root key, and independent within ceiling. Source steps have no page or cursor semantics: a backend continuation means the complete source exceeded its bound and the whole query refuses before root work.

The executor canonicalizes and deduplicates every complete source, evaluates the sealed single/intersection/union/difference operator, then batch-hydrates every surviving root key in the same read view. Missing and policy-hidden roots are indistinguishable and are removed before the compiler-declared mixed root order is evaluated. Only after the complete authorized root population has been sorted does the executor apply the requested bounded page and one continuation marker. Resume repeats candidate completion at the cursor-bound snapshot and epochs, locates the complete root-key marker, and continues in the same total order; it does not filter or sort a pre-paginated index page.

The plan and role retain source indexes, relationship proof, operator, bounds, root order, cost, and authority. Aggregate query fuel charges all complete sources plus hydration. Role format V6 records that candidate authority is present; its max_scan_rows remains the per-access ceiling representable by the public u16 grant, while the sealed whole-query cost independently limits the sum across sources. There is no scan fallback, intermediate output, or application-supplied set.

A pattern_index candidate step is a provider participant rather than an ordinary row-store scan. The service obtains its complete, exactly verified candidate batch before opening final root execution, validates the compiler descriptor, policy shape, plan, generation, frontier, and observed work, and negotiates one result-set epoch proof for the whole query. Final root hydration, policy, mixed total ordering, page selection, and cursor formation still occur inside the authoritative storage snapshot. The request path cannot scan entities to decide pattern membership; only the bounded background rebuild worker may populate provider state from an exact authoritative snapshot.

Finite operational plan families

Language-version-2 optional predicates compile at deployment into a closed family of ordinary access programs. Presence parameters are canonically ordered and capped at eight, so the family contains exactly 2^n members and no more than 256. Every member has the same contract, query name, partition parameter, and result schema. Each member must independently pass the ordinary index, locality, cardinality, and bounded-cost planner.

The family has its own canonical identity over the resolved query surface, ordered presence domain, every member program, complete authorization union, and component-wise maximum cost. Policy therefore authorizes the whole deployed family rather than only the member selected by one request. Runtime selection is an exact presence-bit lookup; it performs no parsing or access-path planning. The selected member drives physical execution while responses and generated bindings retain the stable family identity.

Language-version-2 aggregate declarations compile only through the finite operational-family path. The ordinary single-plan compiler returns source-spanned RDB-QP008, preventing an aggregate declaration from being ignored. The operational family seals exact aggregate descriptors, source fields, authorization union, cost, and the source-row-limit group clamp. At runtime the selected member access and all folds execute in one engine-owned snapshot; no storage scan fallback or client-side fold exists.

Language-version-8 aggregate queries use query-IR/module version 11. The plan seals exact distinct cardinality, partial-state byte, and arithmetic-operation ceilings independently from its source row, group, scan, cell, and output bounds. The reference and columnar evaluators consume those resources across the whole result, apply row policy before contribution, and withhold every aggregate cell if any dimension is exhausted.

When an operational aggregate declares per-measure where filters, UTC temporal grouping buckets (hour, day, week, month, year), having post-aggregation predicates, order by sorting with optional top-N take, exact quantiles (median/p50, p90, p95, p99), first/last, fixed-scale stddev (variance alias), bounded collect/collect_distinct lists, or a per <parent> nested rollup over a for each expanded binding, the compiler includes every referenced source, filter, and grouping field in the step’s selected fields and whole-query authorization union, scales projected cells and encoded result bytes by the parent driver maximum for per <parent> rollups and by the 256-element distinct ceiling for collect/collect_distinct, and encodes all additional clauses as backward-compatible trailing tags so existing version-2 and version-8 plan hashes remain unchanged.

Declared relationship metadata may justify symbolic navigation only when it lowers to the target’s complete primary-key point read or an already bounded dependent-key batch. It cannot infer colocation, omit a partition predicate, introduce a hidden scan, or accept an otherwise unsupported join.

Dependent batches execute as ordered point reads inside the same engine-owned snapshot as the source scan. They do not become public N+1 requests. Null, duplicate, noncanonical, over-bound, or missing dependent keys fail closed before a partial result can be released.

explain remains value-free and names a dependent collection as dependent primary-key batch from source.binding_field. The complete target/source mapping, dependencies, and absence outcome are already sealed in canonical plan bytes; changing any of them changes plan identity. The compiler deliberately keeps existing explain bytes stable because application manifests include that representation.

The base relationship inventory covers three foundational shapes: an exact complete-key point, a singular-key-driven separately bounded index access, and the dependent complete-key point batch. Later language versions extend that foundation with compiler-sealed candidates algebra, correlated exists / not exists probes, and one-level bounded for each one-to-many expansion, each with independent fan-out, intermediate, probe, byte, output, policy, snapshot, and identity bounds. Arbitrary or caller-supplied joins, cross-partition work, Cartesian products, recursion, and runtime join optimization remain unavailable.

The relationship inventory is validated against three independent applications: TicketDesk, agent-blog, and agent-orders. Its checked fixture records every current source/target mapping together with the access shape, driver and target row maxima, shared partition proof, continuation-probe and complete access-key byte maxima, whole-query intermediate/projected-value/result-byte ceilings, missing behavior, cursor eligibility, authority ownership, explain evidence, and plan-identity participation. The current dependent batches are TicketDesk labels (50), blog tags (32), order-line products (100), and inventory products (499). Exact point dependencies have a one-row driver and target. Singular-key-driven index reads retain their own declared take, scan, point, byte, output, and optional opaque-cursor bounds; they do not become a relationship runtime operator.

Required point or batch targets select their declared absence outcome. An optional point produces None, and an empty bounded-index driver produces an empty collection. A row-policy-denied point is intentionally indistinguishable from a missing point, preventing an existence leak. A dependent batch is all-or-nothing: empty input produces an empty collection, while null, duplicate, noncanonical, out-of-order, over-bound, missing, or policy-denied targets release neither partial rows nor a cursor. Memory and redb preserve input position and apply one shared row-policy context while the same engine-owned snapshot remains open.

The closure corpus is fixtures/riffql/operational-access-corpus-v1. It is intentionally domain- neutral and generates an exact cross-language application surface. External consumer evidence is retained only as the value-free receipt fixtures/riffql/wp690-external-tuple-capability-v1.json; external schemas, routes, adapters, and generated profiles remain in their owning repository.

Whole-query authorization and execution fuel

The compiled query runtime enforces the exact application-query authorization and work boundary below.

The service resolves the entire query before data access and presents policy with one application-query request. For a named query this includes the exact contract lineage, immutable module hash, query name, plan hash, partition route, complete entity/field/index/output requirements, principal and capability revision, and one whole-request cost vector. Ad-hoc execution uses a distinct permission and classification.

An allow decision produces a process-local, non-cloneable and nonserializable proof bound to that exact request and current capability revision. The service consumes the proof at the executor boundary with the matching plan and parameters, then reauthorizes before releasing output. It cannot be converted into, or reused to authorize, public GetEntity/ScanIndex requests.

The plan-hashed cost vector includes at least step count, scanned rows, point reads, dependent keys, intermediate rows, projected values, and encoded result bytes. Policy compares the complete vector once. Execution consumes matching fuel and checks backend-reported work; exhaustion returns no partial result or cursor. Per-step bounds remain defense in depth, not the aggregate authority model.

The compiler derives each component conservatively:

• index scans charge their complete declared take maximum;
• point reads include direct reads, every dependent key, and index-result hydration;
• dependent batches charge their complete source-key maximum;
• intermediate rows are summed across all bindings rather than reset per entity or step;
• projected values include every declared result copy and every maximum aggregate group cell; and
• result bytes use contract field byte bounds plus deterministic envelope reserves.

The complete vector is appended to the canonical access-program bytes before the riffdb.query-plan/v1 hash is computed. Changing any component therefore changes plan identity and invalidates substitution.

The predecessor compatible capability record has one max_scan_rows field. For ordinary non-candidate plans it is the whole-query aggregate allowance for index-scan rows; every scan step is summed and no step may independently reuse it. Candidate-aware role V6 instead treats the same bounded field as a per-access ceiling, because candidate plans can contain several independently complete 65,535-row sources; their aggregate is still sealed and enforced by the whole-query cost vector and fuel. Point reads, dependent keys, and retained intermediate rows have separate conservative ceilings derived as that scan bound times the fixed maximum query-step count. Projected values add the existing maximum visible-field multiplier, and encoded bytes retain the fixed 4 MiB service ceiling. The exact plan cost is still authorized once and consumed as execution fuel, so these derived ceilings do not create ambient or unmetered read authority.

Each Limit<MAX> access is charged at its immutable declared MAX; two uses of Limit<100> charge 200 rows even when a request submits smaller values. Fixed take bounds let a multi-collection page divide authority deliberately. Defaults and submitted values never narrow static cost. The separate 65,535-row physical scan ceiling reserves one row for a continuation probe; it is not an application-visible page size. The 4 MiB encoded-result ceiling and all provider-, policy-, hydration-, authority-, and transport-specific ceilings remain independent, so a large structural maximum is not permission to exceed any of them.

An ordinary cursor-paged ordered access treats the submitted runtime limit as invocation cardinality rather than continuation identity. The compiler derives that exclusion only when every use of the parameter is a take on a cursor- paged index access. Mixed use, unpaged limits, and nearest K remain bound. The cursor-specific invariant-parameter hash is paid once per admitted request; the shared query parameter hash and immutable plan identity are unchanged.

The V12 row-limit IR retains the parameter name, positive declared maximum, and optional default. Runtime validates the effective value before opening a provider or storage access. One accepted invocation executes the selected page directly; it does not walk one-row cursors, overfetch and discard, or re-plan per page.

At runtime the executor creates a move-only fuel value from the exact program cost. It decrements fuel for every access step, backend-reported scanned row, point read, dependent key, retained intermediate row, projected value, and encoded result byte. Backend scan reports are reconciled with returned rows. An impossible, under-reported, over-reported, or exhausted execution returns a closed failure before a result or cursor is constructed.

Aggregate grouping uses length-framed canonical value encodings as the ordered key. Whole-set folds always produce one record, including for an empty source; grouped folds produce no record for an empty source. Sum arithmetic is checked over i128; exact decimal scale comes from sealed result type metadata, never from a row or caller. Empty min/max outer absence is distinct from a contributed canonical null for optional inputs: the aggregate record omits the field only for the outer absence, while a present null remains a normal value.

Immutable query modules

A query module is a versioned, content-addressed set of canonically formatted named RiffQL documents. It is pinned to one exact retained contract lineage, version, and bundle hash.

Deployment recompiles every source document at the catalog trust boundary, derives the canonical module hash, and submits an audited compare-and-swap activation through the control-plane coordinator. The module bytes and active pointer are committed atomically. Startup replays and validates the durable module history before readiness.

Named execution always submits the exact module hash emitted by the generated binding or manifest. Selecting the ambient active module is an inspection and development action, not an execution identity: omitting the hash fails closed. Cached plans are bounded and keyed by immutable identity. The server authorizes the complete compiler-derived application-query request for every invocation; a module or generated client conveys no authority.

Ordinary modules retain the original version-1 byte layout and identities. A module containing an operational optional-predicate query uses the additive version-2 representation. It stores the complete finite plan family, its authorization union, maximum cost, and family identity. Generated clients pin that family identity; the service selects one exact member only from parameter presence and returns the family identity on the response. Strict decoding recompiles source against the exact contract and compares canonical bytes.

A module containing bounded exact aggregates uses additive version 3. Its aggregate descriptors, result schemas, authorization fields, group ceiling, cost, source map, explanation, and family identity are all compiler-owned and strictly recompiled on decode. Version-3 aggregate modules are deployable and execute only through the one-snapshot operational evaluator. The service passes the sealed descriptors alongside the selected family member; an adapter that does not implement this exact boundary fails closed with InvalidProgram instead of falling back to the ordinary source-row result.

A module using the additive exact aggregate core uses version 11 paired with RiffQL language version 8 and query IR version 11. Its canonical bytes include the new closed function tags and the independently sealed distinct, state, and arithmetic budgets. Strict decode recompiles all three identities; module versions 1 through 10 retain their existing readers and exact bytes.

Operational families currently execute through the named-query application surface. Version-1 reactive/live-query modules do not substitute a representative member: compilation excludes an operational family until a versioned reactive presence-selection surface exists.

A module containing a compiler-checked secret output uses additive version 4. Its canonical query plan carries each exact query, entity, field, stable field identity, result slot, and declaration span. Strict decode recompiles those requirements against the exact contract. Selecting such a query in a symbolic application role derives a V4 role identity with ordered query/entity/field atoms and dedicated secret-field visibility; selecting an ordinary query does not inherit another query’s secret authority. Modules and roles without these declarations keep their previous V1 through V3 bytes.

Secret-bearing queries are generated for typed Rust, Go, TypeScript, Python, and gRPC application execution. Each language publishes the same exact symbolic secret-output metadata and a language-appropriate redacted diagnostic surface. SDK operation identities are generated directly from the complete named-query module; they are not inferred from the MCP inventory. Secret queries remain deliberately absent from generated MCP tool catalogs and every reactive query catalog.

The additive identity transition is frozen in fixtures/riffql/operational-identity-rotation-v1.json. That generated receipt records the v1/v2/v3 language, IR, plan/family, module, manifest, role, application-lock, and Rust/Go/TypeScript/Python/MCP artifact hashes. It also binds the unchanged ordinary-v1 TicketDesk and agent-alpha closures. The normal generated-client check regenerates the receipt, so a partial rotation or stale example identity fails the repository gate.

Generated artifacts

The compiler emits:

• Rust parameter, result-union, command-input, and operation types;
• Go, TypeScript, and Python equivalents;
• exact per-query query/entity/field secret-output metadata and redacted debug/string/representation helpers when required;
• exact contract and module identity constants;
• response-identity verification helpers; and
• optional module-qualified MCP read tools with domain-shaped JSON Schemas for queries that do not return secret-classified fields.

For TicketDesk, the generated MCP names are:

ticketdesk_list_tickets
ticketdesk_project_members
ticketdesk_project_summary
ticketdesk_ticket_page

Generated MCP operations are read-only and non-destructive. They complement the general riffdb.query operation; they do not replace ad-hoc RiffQL. Mutation tools remain separate compiled-command operations with mutation risk metadata.

The checked fixtures are:

• fixtures/query-modules/ticketdesk.rs
• clients/typescript/ticketdesk/client.ts
• fixtures/query-modules/ticketdesk.mcp.json

scripts/generate-query-clients --check regenerates the checked Rust, Go, TypeScript, Python, and MCP artifacts, compares exact bytes, and validates their language-specific build boundaries.

Symbolic catalog and query surface IR

The RiffQL compiler resolves RiffQL against one exact validated contract bundle. The symbolic catalog indexes entities, fields, enums, variants, and indexes by their exact source names. Internal stable IDs remain compiler data and never appear in parameter schemas, result schemas, source maps, or diagnostics.

The resolved surface artifact begins with RIFFDB-QUERY-SURFACE\0, query-IR version 1, the exact bundle hash, lineage and contract version, canonical RiffQL source, resolved binding identities, and complete name-addressed parameter/result schemas. Lengths and counts are big-endian u32; the complete artifact is bounded to 4 MiB. Source maps are kept separately so whitespace and source offsets do not change canonical semantic bytes.

Finite operational plan families use additive query-IR version 2. They seal every compiler-enumerated presence member, the whole-family authorization union, maximum cost, and stable cursor identity. A family containing bounded exact aggregate descriptors uses additive query-IR version 3 for both its resolved surface and enclosing family. Version 3 adds the symbolic source binding/entity, ordered group keys and result types, ordered measures and result types, and the compiler-proven maximum group count. Ordinary version-1 and non-aggregate version-2 bytes do not rotate.

The additive exact aggregate core uses query-IR version 11. It adds closed function tags for present count, both exact distinct forms, exact mean state, Boolean any, and Boolean all, plus compiler-sealed distinct-value, partial-state-byte, and arithmetic-operation budgets. ExactMeanV1 is a structural result record containing a widened exact decimal total and u64 count; the IR never carries a quotient, rounding mode, or floating-point value. Earlier aggregate surface and family bytes remain version 3.

An exact secret-output declaration selects additive query-IR version 4. The surface records the immutable contract identity plus the query-local binding, symbolic and stable entity/field identities, result branch and nested slot, and declaration source span for every returned secret leaf. These ordered, bounded requirements participate in the surface, plan, module, and role hashes. A declaration-free query retains its earlier exact bytes.

Parameter types are contract enums, Entity.field references, optional wrappers, bounded query sets, Cursor, Limit, or the closed query-only Limit<MAX> refinement. Result schemas contain only source names and closed scalar/optional/record/list shapes. A list repeats its literal bound, plain Limit parameter name, or bounded-limit parameter name and inclusive maximum.

RiffQL V9 selects additive query IR V12. V12 encodes Limit<MAX> with its positive maximum before its optional default in the parameter schema and row limit, and uses the same maximum for result-list bounds and whole-plan cost. Existing V1 through V11 bytes remain unchanged. Changing only MAX changes the surface, plan, module, role, lock, generated schema, and operation hashes.

Cursor-page-cardinality eligibility adds no IR field or version. It is derived from the existing V12 row-limit, cursor, cardinality, and physical access structure: a parameter qualifies only when every use is an ordinary ordered cursor-page limit. The declared MAX remains identity-bearing in the IR and plan; only the validated submitted value is omitted from the separate process-local cursor lookup hash.

Candidate syntax selects additive query IR V14 and query-module V14. The resolved surface canonically records each non-output binding’s name, root entity/key, closed set operator, ordered source entity/key/access triples, positive-source count, distinct-key maximum, and refusal outcome. The access program adds complete candidate-source row limits, candidate predicate references, and one root-hydration access with its maximum, complete key, and finite total-order terms. Source-map tags 11 and 12 identify the declaration and each source access. These fields are plan and role identity only; public parameter and result schemas remain ordinary business values and never expose candidate structure.

Bounded Set<T, MAX> syntax selects RiffQL V13. A query using that value as the exact partition-field membership route selects query IR V16 and query-module V16. The program encodes the parameter name and declared partition maximum, one partition-set index access with its physical direction, complete global order, per-partition scan ceiling, and hidden entity-key tie fields, plus multiplied cost and authority. Canonical cursor parameters retain the normalized complete set. Existing V1 through V15 query plans and modules are unchanged.

When the partition component is followed by one or more invariant exact predicate components before the order suffix, the same RiffQL V13 source selects additive query IR V17 and query-module V17. V17 additionally encodes the fixed prefix width so continuation reconstructs the partition, every exact filter value, and the global order key without placing invariant filter values in the public cursor marker. Exact-filter parameters remain cursor-identity bearing. A partition-set plan without this prefix retains byte-exact V16 plan and module artifacts.

Bounded expansion syntax selects RiffQL V14, query IR V18, and query-module V18. The expansion access encodes the selected declared index, physical order, driver binding and singular item identity, per-driver maximum, and proven whole-expansion product maximum. Its predicate retains an ordinary binding-field dependency on the driver, and its result schema nests the bounded target list under the driver record. Queries without an expansion keep their predecessor surface, plan, module, hash, and cursor bytes. V18 adds no provider descriptor, provider epoch, provider state, service port, generated input, or durable storage format.

Resolution diagnostics are bounded and value-free:

Each diagnostic carries the closed compiler stage, primary UTF-8 byte span, a safe source-name path, static summary, and optional static help. It contains no query parameter, business value, raw source, storage key, or internal error.

Configuration Reference

Application gRPC has a closed listener profile: development-only cleartext on a literal loopback address, verified direct TLS on TCP, or a protected Unix socket. Hosted MCP supports literal-loopback cleartext or verified direct TLS on TCP. There is no insecure remote mode. See Remote and Local Application Ingress.

Project riffdb.toml

The database-shaped init, push, generate, status, diff, and migrate verbs use a closed project document. They select riffdb.toml in the current directory by default; explicit --config and then RIFFDB_CONFIG take precedence. Other CLI commands retain the existing no-discovery behavior.

[client]
endpoint = "http://127.0.0.1:7443"
database = "default"

[project]
schema = "riffdb.application.json"
generators = ["rust"]

The schema path must be bounded, workspace-relative, and free of . or .. components. generators contains one through four unique values from rust, go, typescript, and python. Unknown keys, duplicate targets, symlinked configuration files, absolute or escaping schema paths, and empty target lists reject without fallback. See the Database-Shaped Project Workflow for command semantics.

riffdbd

Every server field resolves independently in this order:

1. an explicit CLI value;
2. its RIFFDB_* environment value;
3. an explicitly selected TOML document; then
4. the built-in safe local default.

A present invalid higher-precedence value rejects instead of falling through. The TOML path is selected by --config, then RIFFDB_CONFIG, then absence. There is no implicit configuration-file discovery.

The legacy database, environment, and backup settings below define one database with alias default. A process may instead define 1 through 512 [databases.<alias>] tables. Mixing the legacy and named forms rejects.

The only TOML tables and keys are:

[server]
database = "/var/lib/riffdb/data/riffdb.redb"
grpc_listen = "127.0.0.1:7443"
environment = "local"
audience = "riffdb-grpc-loopback"
capability_keys = "/etc/riffdb/capability.keys"
idempotency_keys = "/etc/riffdb/idempotency.keys"
# mcp_listen = "127.0.0.1:7444"
# mcp_origins = ["http://127.0.0.1:3000"]
# metrics_listen = "127.0.0.1:9464"
# telemetry_flight_recorder = "/var/lib/riffdb/telemetry/flight.ndjson"
# telemetry_flight_interval_ms = 10000

[maintenance]
backup_root = "/var/lib/riffdb/backups"

Loopback metrics and telemetry flight recorder

When --metrics-listen (RIFFDB_METRICS_LISTEN or server.metrics_listen) is configured, riffdbd binds a literal IPv4 or IPv6 loopback address with a non-zero port and serves read-only HTTP/1 GET /metrics in Prometheus text format (text/plain; version=0.0.4; charset=utf-8) and GET /v1/metrics in OpenTelemetry JSON ExportMetricsServiceRequest format (application/json). Non-loopback addresses, zero ports, non-GET methods, and other request paths fail closed. All exported counters, gauges, and stage histograms use closed compile-time label vocabularies and numeric identifiers; raw tenant, entity, request, and secret values are never emitted.

When --telemetry-flight-recorder <PATH> (RIFFDB_TELEMETRY_FLIGHT_RECORDER or server.telemetry_flight_recorder) is configured, riffdbd appends one bounded riffdb.telemetry.flight-record.v1 NDJSON line at startup, every --telemetry-flight-interval-ms (100..=3600000, default 10000), and after graceful drain on shutdown. Operators can aggregate a recorded flight file offline with riffdb telemetry summarize <PATH> (in human or --output json mode) to inspect throughput deltas, stage latency p50/p95/p99 high-water marks, writer busy saturation, health transitions, online retention state/fence and pruned-sequence progress, and storage/journal byte growth across 24-hour and 72-hour soak runs.

Named archive directories

An operator may bind up to 16 archive names per database in TOML. The legacy single-database form uses [[maintenance.archives]]; the named form uses [[databases.<alias>.archives]]. Each entry requires name, path, and encryption:

[[maintenance.archives]]
name = "daily"
path = "/var/lib/riffdb/archives/daily"
encryption = "operator_managed"

Names are checked identifiers and must be unique within their database. Paths must be bounded absolute directories and disjoint from every database, backup, projection, archive and other reserved process path. Invalid or overlapping entries reject configuration. encryption is either unencrypted or operator_managed; the latter declares external operator-provided encryption, and does not ask RiffDB to encrypt files. Archive paths never come from restore request input.

Add backup = "baseline" to the entry to collect published changelog frames beside the verified full backup named baseline under this database’s backup_root. Without backup, the entry is available for restore only. The backup must already exist when the database starts. Each collector verifies its backup and existing archive before appending; it never rebinds an archive to a different backup or history incarnation. Use a new directory for a new base.

Collection runs independently of command acknowledgement and the write gate. Each collector buffers at most one uncertain frame and retries those exact bytes up to three attempts. A missing or invalid backup, unavailable startup sink, pruned history gap, changed lineage, or exhausted append retries stops that collector with a redacted diagnostic. It does not stop primary writes. Correct the cause and restart the database to revalidate and resume. Shutdown joins all collectors before closing storage. The POC supports local filesystem sinks; operator_managed encryption must be supplied outside RiffDB. See Backup and Restore.

Durable local outbox destination

An optional [maintenance.outbox] section, or [databases.<alias>.outbox] for named databases, configures one durable_local destination with an absolute protected directory and positive quota_encoded_bytes. Configuration is restart-applied and refused on followers. See Durable Local Outbox Archive for custody, delivery, accounting, and offline inspection.

Online retention

riffdbd can compact commit history while it serves. The legacy single-database form uses [maintenance.retention]; the named form uses [databases.<alias>.retention]:

[maintenance.retention]
retained_sequences = 1000000
sequences_per_epoch = 256

Set exactly one of retained_sequences, a positive count of commit sequences to keep below the durable application head, or hold = true. sequences_per_epoch bounds each step, from 1 through 256 (default 256). An absent section is a hold, so a configuration that never names retention prunes nothing online. Targets are sequence counts, never ages. Any other key, both forms together, or an out-of-range value rejects configuration.

With a target, one background lane per database advances the retention watermark in steps. Each step is one administration transition through the database’s single ordered writer. It removes whole command segments, appends the chained tombstone, and advances the watermark in one atomic commit. A journal-backed command waits until its extent is sealed. A sealed extent may span several logical prune steps; each step still covers at most 256 sequences. Logical pruning keeps the extent’s bytes intact. The lane reclaims a whole file only after every command in it has been pruned and the existing source-history protocol has released every receipt reference. The retained source resume anchor, older pinned snapshots, and an active background scrub keep the file alive. The lane never creates a source acknowledgement or discards a receipt to free space. One retired file can supply the next preallocated journal spare; excess eligible files are deleted. File work runs outside the ordered writer gate. Repeated requests for a missing spare share the unfinished preparation job and its completion signal. They do not queue additional waiting sealer threads or create new extent-publication obligations. Each actual extent publication keeps its own completion tracking. A completed failed preparation permits a fresh attempt; it does not certify a failed extent publication. The lane holds no exclusive lease and opens no second writer. A writer waits behind at most the one step in progress. After each attempt that took the writer gate, the lane rests nineteen times the time it held the gate, so it holds the writer at most about five percent of the time and a large backlog compacts gradually rather than at full speed. It plans each step without the writer gate and takes the gate only when it has history to remove. With the Standard durability profile, it prepares retained views and the tombstone digest from a pinned composite snapshot outside the gate. The writer revalidates every precondition and current fence before submitting one V4 prune frame through the existing group-sync journal lane. Durability completes after releasing the gate; watermark publication precedes the corresponding deletion view. It does not drain unrelated command frames just to prune. Checkpoint materialization remains background work. Hardened profile steps retain their direct engine transaction. Once the lane reaches its target or a fence it rechecks about once a second. A single command segment larger than the budget is removed alone, never more than 256 sequences. A crash at any point in a step reopens with the watermark either before or after that step, and the restarted lane resumes.

The lane never passes a fence: projection frontiers, durable event-consumer checkpoints, undelivered outbox events, a staged contract migration, operator holds, and registered followers’ acknowledged frontiers. An unreadable fencing source stops the lane with an internal incident until the next restart; it never prunes past it. No gRPC health member, control RPC, or MCP tool drives the lane, and application principals have no retention operation. While riffdbd serves, /metrics, /v1/metrics, and the periodic NDJSON flight recorder export the closed online retention state (riffdb_online_retention_state / online_retention_state), active binding fence (riffdb_online_retention_fence / online_retention_fence), completed step count (riffdb_online_retention_steps_total), pruned sequence count (riffdb_online_retention_sequences_total), and latest watermark sequence (riffdb_online_retention_watermark_sequence). The shutdown release census line also records its state after graph teardown has joined the lane, the name of the fence that last held it, and its final step and sequence counts. An enabled lane normally ends as cancelled; a refused lane remains refused, and a configured hold that starts no worker remains holding. The graph and process release receipts share that frozen observation. The separate riffdb-retention-extents-v1 line records files deleted or recycled, passes held by references or readers, and off-writer reclamation microseconds. Logical sequence progress does not imply physical space reclamation.

The retention lane also owns source-receipt reclamation across observations. It validates immutable history in pages of at most 256 rows, then yields if a writer is queued, a checkpoint is in flight, or the journal is not drained. It also yields until off-gate extent sealing finishes, before capturing a validation pin. A changed publication discards the candidate. At a later independently durable checkpoint it rechecks consumer and derived-reader holds and removes at most 256 receipts. Source holds retain their separate 4,096-registration bound. Its own reclamation and prefix-refresh receipts cannot create an idle reclamation loop. The separate riffdb-retention-receipts-v1 shutdown line records removed receipts and gate_us; that time participates in the lane’s existing writer-occupancy pacing. Receipt removal does not prove that an extent was deleted or reused.

Eligibility planning and checkpoint licensing resolve receipt points on the same immutable source view that passed native validation, including its acknowledged journal suffix. A newer installed checkpoint does not extend an older captured source’s readable range. Missing coverage beyond that range yields; a missing receipt inside claimed coverage still refuses the lane.

Follower acknowledgement preparation shares this source-proof owner and its maintenance rest deadline. Initial validation runs in bounded pages outside the writer; the closed acknowledgement enters the existing commit coordinator only after preparation. A completed candidate retains custody until admission, yield or cancellation. Ordinary queued writers keep their FIFO order. The 32-frame report cadence and existing request deadlines are unchanged. The internal riffdb-follower-ack-work-v1 diagnostic reports numeric phase (1 preparation, 2 admission), result (0 ready/completed, 1 yielded, 2 refused), off_gate_us, writer_wait_us, gate_held_us, initial_rows and continued_rows. Writer wait and gate residence come from the native FIFO census; off-gate elapsed work excludes both. Durations are elapsed observations, not CPU measurements or a latency promise; the maximum unsigned value means unknown. These observations contain no hold identity or payload and grant no authority to advance a hold.

Held bootstrap manifest/page reads share that same source certificate through a closed native read protocol. The first complete proof advances in bounded pages outside the writer; subsequent checks use the qualified source pin or checked native successors. Each release rechecks current exact custody after bounded artifact I/O. Incomplete validation releases no bytes, and a reader cannot displace a prepared follower acknowledgement. These remain foreground reads under existing source-job capacity and deadlines; they create no control write, reclamation license or extra maintenance budget. Receiver startup and publication continue to use their complete validators. The internal riffdb-bootstrap-custody-work-v1 diagnostic reports numeric result (0 ready, 1 yielded, 2 refused), elapsed_us, initial_rows and continued_rows for actual native proof work. It contains no hold identity, payload or endpoint. Elapsed observations are not CPU measurements; the maximum unsigned value means unknown.

The runtime binds the shared source scheduler even when retention is held. Acknowledgement custody uses the same rest policy. Rebinding cannot reset already charged rest or clock-overflow refusal.

After retention invalidates optional validated-prefix startup evidence, the maintenance lane can rebuild it from a complete structural, catalog-semantic, and retained-history scrub on one pinned snapshot. Each validation page contains at most 64 rows and observes shutdown cancellation. Population validation and checkpoint planning run outside the writer gate. Publication requires the exact unchanged history binding and a drained, sealed checkpoint; otherwise the candidate is discarded. The lane never reuses deleted evidence or forces a journal drain to publish it. Pin capture and publication participate in the existing maintenance pacing. Sustained-load recovery remains subject to the milestone’s first-successful-write rehearsal gate.

Pruned history stays auditable. A pruned command keeps its outcome and provenance, so its idempotent retry answers the original result. Reads of its events or commit, and commit scans that start at or below the watermark, return the typed HistoryPruned error.

Read the retention limits in Known limitations before enabling it. In particular, unless [maintenance.outbox] (or [databases.<alias>.outbox]) is configured so the local outbox worker archives and acknowledges emitted events, a contract that emits events compacts only the history before its first undelivered event. The offline riffdb retention operations remain available on a stopped database; see Backup and Restore.

The legacy server.grpc_listen form always means literal-loopback cleartext. For an explicit closed listener profile, omit grpc_listen and select exactly one tagged table:

[server.application_listener]
mode = "loopback_cleartext"
listen = "127.0.0.1:7443"

[server.application_listener]
mode = "direct_tls"
listen = "0.0.0.0:7443"
public_endpoint = "https://riffdb.internal.example:7443"
certificate_chain = "/etc/riffdb/tls/server-chain.pem"
private_key = "/etc/riffdb/tls/server-key.pem"

[server.application_listener.bounds]
max_connections = 1024
max_streams_per_connection = 128
handshake_timeout_seconds = 10
idle_timeout_seconds = 300
keepalive_interval_seconds = 30
drain_timeout_seconds = 30

[server.application_listener]
mode = "local_socket"
path = "/run/riffdb/application.sock"
access = "owner_only" # or "owner_and_group"

The explicit listener table cannot be mixed with grpc_listen, --listen, or RIFFDB_LISTEN. TLS has no cleartext fallback, trust-all mode, mutual-TLS selector, cipher-suite selector, or provider selector. The certificate must contain the DNS/IP identity in public_endpoint; validation happens before bind. Certificate/key replacements are adopted atomically for new handshakes only after the complete pair validates.

The equivalent multi-database form is:

[server]
grpc_listen = "127.0.0.1:7443"
audience = "riffdb-grpc-loopback"
capability_keys = "/etc/riffdb/capability.keys"
idempotency_keys = "/etc/riffdb/idempotency.keys"

[databases.ea]
path = "/var/lib/riffdb/data/ea.redb"
backup_root = "/var/lib/riffdb/backups/ea"
environment = "local"

[databases.budget_demo]
path = "/var/lib/riffdb/data/budget-demo.redb"
backup_root = "/var/lib/riffdb/backups/budget-demo"
environment = "local"

Aliases match [a-z][a-z0-9_-]{0,63}, sort by canonical bytes, and are unique. All database, backup, projection, source/receiver artifact, and key paths across the process must be pairwise lexically disjoint. The process opens every database and builds every graph before publishing readiness. One structural startup failure stops the process.

When riffdbd runs in primary mode with [databases.<alias>] loaded from a configuration file (--config or RIFFDB_CONFIG), updating that file on disk reconciles added and removed database aliases online without interrupting peer databases:

• Newly added [databases.<alias>] sections are validated, opened, and published onto the gRPC and Hosted MCP routing tables only after startup and disjoint-path checks succeed.
• Removed [databases.<alias>] sections (when at least one database remains configured) are atomically unpublished from the routing tables, drained, closed with a clean-close checkpoint, and released from their redb file locks.
• Any reload that fails TOML validation, introduces overlapping paths, mutates process-global listener/audience/key settings, or changes a retained database’s immutable storage paths or environment fails closed, logs a riffdbd config reload rejected diagnostic, and retains the active inventory unchanged.

Backup roots must be siblings. This is valid:

[databases.default]
path = "/var/lib/riffdb/data/riffdb.redb"
backup_root = "/var/lib/riffdb/backups/default"
environment = "local"

[databases.ea]
path = "/var/lib/riffdb/data/ea.redb"
backup_root = "/var/lib/riffdb/backups/ea"
environment = "local"

This is invalid because one configured root contains another:

[databases.default]
path = "/var/lib/riffdb/data/riffdb.redb"
backup_root = "/var/lib/riffdb/backups"
environment = "local"

[databases.ea]
path = "/var/lib/riffdb/data/ea.redb"
backup_root = "/var/lib/riffdb/backups/ea"
environment = "local"

Startup reports the two conflicting logical roles and aliases, but never echoes configured path values.

The complete document is UTF-8 and at most 524,288 bytes. Unknown tables or keys, duplicates, wrong types, duplicate scalar flags, missing flag values, and empty selected values reject. Paths are nonempty, at most 4,096 platform bytes, and lexically normalized. The database and key paths may be relative to the server working directory; the backup root must be absolute.

The database path, backup root, projection root, source and receiver artifact roots, capability key path, and idempotency key path must be lexically disjoint. A file is not accepted when another configured path is equal to it, contains it, or is contained by it.

Each database reserves a private replication artifact root by appending .riffreplication to its path (for example, data/main.redb.riffreplication). The derived path also has the 4,096-byte ceiling. This directory holds at most four source bootstrap artifacts and an exclusion lock; it cannot overlap any other configured database or root. Source composition creates it with private permissions and refuses symlink or identity substitution.

Each database also reserves .riffreceiver beside its database path, with the same path-length and disjointness checks. Its fixed receiver inventory is separate from source artifacts so promotion cannot mix the two layouts. The managed receiver holds an exclusive inventory lock and creates private transfer and candidate directories. Initial creation uses temporary names until the first durable progress checkpoint exists; crash recovery discards only those known unpublished files and resumes published progress. After authenticated attachment succeeds, the receiver removes candidate scratch links and then transfer evidence, while retaining the live follower’s engine lock. Interrupted cleanup resumes against the validated live follower. See follower mode for daemon configuration and lifecycle behavior.

The packaged systemd unit fixes the authoritative state root at /var/lib/riffdb/data, the backup root at /var/lib/riffdb/backups, and the configuration root at /etc/riffdb. The latter contains the root-owned riffdbd.toml path-only configuration and the two protected digest-key documents. It contains no raw digest key in TOML.

Per-database resource limits

An optional [maintenance.limits] table (or [databases.<alias>.limits] in the multi-database form) bounds per-database storage, request rates, blocking pool concurrency, commit coordinator capacity, journal extent sizing, and idle hibernation:

[maintenance.limits]
max_storage_bytes = 536870912
max_write_rps = 50
write_burst = 100
max_read_rps = 250
read_burst = 500
max_blocking_operations = 4
blocking_worker_threads = 2
max_coordinator_capacity = 32
extent_profile = "compact"
idle_hibernate_seconds = 300

• max_storage_bytes (65_536..): refuses new state-growing commands, contract deployments, migrations, and capability creations with typed Overloaded when the authoritative database file plus journal footprint reaches or exceeds the ceiling. Reads, ResolveCommandOutcome, RevokeCapability, and application exports (StartApplicationExport, GetApplicationExportPage, GetApplicationExport, CancelApplicationExport) remain permitted at the storage ceiling so data and outcomes are never stranded.
• max_write_rps (1..=1_000_000) and optional write_burst (1..=1_000_000, default max_write_rps): monotonic token-bucket governor for mutating commands and administration writes.
• max_read_rps (1..=1_000_000) and optional read_burst (1..=1_000_000, default max_read_rps): monotonic token-bucket governor for reads, queries, and export scans (GetHealth and GetStatistics probes are exempt).
• max_blocking_operations (1..=256), blocking_worker_threads (1..=8), and max_coordinator_capacity (1..=4_096): override the per-database blocking pool and commit coordinator queue bounds.
• extent_profile ("standard" or "compact", default "standard"): selects the per-database journal extent profile. "standard" preallocates 320 MiB active and spare extents; "compact" preallocates an 8 MiB active extent (4 MiB checkpoint rotation threshold) and defers .riffjournal.spare allocation until 2 MiB (25%) active utilization or on-demand checkpoint rotation. Switching profiles across restarts normalizes empty active extents in place and seals extents that contain materialized history stubs before opening the next generation at the newly configured capacity.
• idle_hibernate_seconds (1..=86_400, default absent): enables scale-to-zero idle hibernation for a database in multi-database hosting. When no gRPC or hosted MCP request has touched the database for idle_hibernate_seconds, and the database has no active service jobs, no active commit notification subscribers, and zero pending outbox deliveries, the supervisor cleanly drains the database’s runtime graph, commits a clean-close certificate, and releases its exclusive redb file lock and worker threads while keeping the process and alias routes alive. The next gRPC or hosted MCP request targeting the hibernated database transparently wakes it via single-flight initialization before dispatching the request.

Optional hosted MCP settings:

In TOML, [server.mcp_listener] may also select an explicit loopback_cleartext or direct_tls profile (mutually exclusive with mcp_listen / mcp_origins / --mcp-listen / --mcp-origin):

[server.mcp_listener]
mode = "direct_tls"
listen = "0.0.0.0:7444"
public_endpoint = "https://riffdb.internal.example:7444"
certificate_chain = "/etc/riffdb/tls/server-chain.pem"
private_key = "/etc/riffdb/tls/server-key.pem"
origins = ["https://app.internal.example:8443"]

--mcp-origin without --mcp-listen rejects. With no allowed origins, only requests without an Origin header are accepted. The MCP protected-resource audience is derived as http://<listen-authority>/mcp for loopback_cleartext and <public_endpoint>/mcp for direct_tls; it cannot be configured and must differ from the gRPC audience. Direct-TLS origins must use https://.

RIFFDB_MCP_ORIGINS is one nonempty comma-separated list with no trimming or normalization. TOML uses a string array and CLI uses one flag per origin. Whichever source wins supplies the whole list. At most 16 unique origins and 8,192 aggregate bytes are accepted; each origin is at most 512 visible ASCII bytes and must pass the hosted transport’s origin checks.

Other RIFFDB_* variables are not server settings. There is no storage-engine selector, group-commit tuning, transaction callback, or nondeterministic scheduler setting in the POC configuration surface.

Digest-Key Documents

Capability keys:

riffdb-capability-digest-keys-v1
1:<exactly 64 lowercase hexadecimal characters>

Idempotency keys:

riffdb-idempotency-digest-keys-v1
1:<exactly 64 lowercase hexadecimal characters>

Documents are at most 1,024 bytes, end with LF, contain one through eight entries, use unique nonzero canonical decimal IDs, and contain no duplicate material. The first listed entry is the write key; later entries remain readable for rotation. The two namespaces must never reuse material.

On Linux, each secret source must be a regular non-symlink final component owned by the process effective user with no group or other mode bits. Metadata is checked before and after a no-follow open.

riffdb Client

Configuration is resolved independently for each field, highest precedence first:

The TOML path is selected only by --config, then RIFFDB_CONFIG, then absence. There is no directory discovery. A present invalid higher-precedence value rejects instead of falling through.

The endpoint is either exact lowercase http://<literal-loopback-IP>:<port-1..65535> or canonical https://<DNS-or-IP>:<port-1..65535>. For HTTPS, omitting tls_trust_root selects the platform system CA bundle, and omitting tls_server_name derives the expected peer identity from the endpoint authority host; when supplied, tls_trust_root must be an absolute, normalized path and tls_server_name must match the endpoint authority. Supplying either TLS field for cleartext or requesting a trust-all/downgrade mode rejects. Generated application client configuration retains any explicit non-secret TLS selectors. Unknown, duplicate, or wrong-type input rejects. Output is human or json. Attempts is canonical decimal 1..=10.

Exactly one normal credential source may resolve:

• the protected credential file selected above; or
• RIFFDB_CAPABILITY_TOKEN, an exact 43-byte bearer presentation.

Supplying both rejects. Prefer the protected file. Machine output is one compact JSON object plus LF and no other stdout bytes.

riffdb-mcp Stdio Bridge

The bridge accepts only:

--config PATH
--endpoint LOOPBACK_HTTP_ENDPOINT
--database DATABASE
doctor --config PATH

Configuration path precedence is --config, RIFFDB_MCP_CONFIG, then absent. Endpoint precedence is --endpoint, RIFFDB_MCP_ENDPOINT, [mcp].endpoint, then http://127.0.0.1:7443. Database precedence is --database, RIFFDB_MCP_DATABASE, [mcp].database, then default.

Its TOML contains only:

[mcp]
endpoint = "http://127.0.0.1:7443"
database = "default"
credential_file = "/var/lib/riffdb-mcp/credential"
expected_audience = "riffdb-grpc-loopback"

Credential selection is exactly one of RIFFDB_MCP_CAPABILITY_TOKEN, RIFFDB_MCP_CREDENTIAL_FILE, or [mcp].credential_file. The environment file path takes precedence over the TOML path, while simultaneous raw token and file selection rejects. The credential is loaded once and moved into the ordinary public gRPC client. expected_audience, when present, is bounded target-identity evidence written by application provisioning; it grants no authority and has no flag or environment override.

The bridge accepts a database alias, not a path. It does not accept a storage option, environment, audience authority override, policy, or server-side authority.

Follower mode

--mode follower (or RIFFDB_MODE=follower, or [server] mode = "follower") selects the separate follower startup path before primary initialization or maintenance. The default remains primary. Each configured database must have one complete replication_source block; primary mode rejects such blocks. The mode follows the usual CLI, environment, then file precedence. Source identity and trust settings are file-only and cannot be overridden individually.

For the legacy single-database form, use [server.replication_source]. For a named database, use [databases.<alias>.replication_source]. Required fields are:

Credential files use the existing protected-file loader. TLS always verifies the configured CA and server name. Trust and credential paths cannot overlap database, backup, projection, replication scratch or digest-key paths. All source peers are checked before local construction starts. Receiver scratch is the reserved <database-path>.riffreceiver sibling directory.

Startup bootstraps an absent follower or fully validates and reopens an existing one. A retained complete transfer permits the existing candidate-publication recovery path to check its exact fence and destination before replacement. Tail reception uses the supervised worker’s bounded retry and shutdown rules. A committed promotion is reconciled from local authoritative control and external audit before loading former-source credentials. Once completely validated, it starts as a source even with its existing follower configuration. Unresolved pre-cutover attempts enter restricted promotion-retry hosting after full local validation, without connecting to the former source at boot. Ordinary readiness stays fenced. A freshly authorized exact retry requires that source’s authenticated fence proof before cutover. See follower administration.

After validation, the daemon activates a writer-free application service for each configured alias, binds configured hosted MCP routes, and emits the ordinary application-ready receipt. Standard input accepts shutdown; process signals also stop the daemon. Shutdown closes transport admission, stops replication and drains accepted service reads before releasing the remaining snapshot pins.

Authentication and current policy observe the latest completed follower prefix. A capability or contract installed later on the source becomes visible through replication; followers expose no local capability-bootstrap convenience path. Commands, administration, migration, maintenance and export return the typed follower-mode refusal. The experimental promotion operation has its own external audit owner. Policy-required durable read audit still refuses.

The daemon process tests compare every authoritative namespace under the app-baseline workload and resume across repeated source and follower crashes. The follower replication and projection freshness checks pass their full CI gates, including follower freshness, provider equality and sequence-lag reporting.

Follower named queries can use the existing exact-text, exact-predicate, tokenized-text and long-pattern providers. Their local checkpoints live under the configured projections_root and remain rebuildable. Background workers read only completed follower snapshots; the applied application frontier is their local head for minimum-commit and AdmissionHead checks. A provider that has not caught up returns the existing typed freshness or availability refusal. Configured scalar and vector columnar sources stay cold until demand. One owned worker builds independently validated disposable V2 views under projections_root/follower-columnar; no source artifact is required and no local authoritative control is written. Cold or activating sources return the existing typed Building/unavailable outcome and keep columnar health degraded. Failed materialization remains closed until owned recovery or restart; a request cannot clear failure. Restart rebuilds these disposable views from the completed prefix.

Projected reads retain Causal, Bounded and Available semantics using the applied local head. Replication source-head observations only report lag. Foreign database/history tokens refuse before waiting, and opaque cursors are local to one live process. See sequence-lag health and statistics for both-node reporting.

Exact-provider checkpoint custody

On Linux, exact-text, predicate, nullable-predicate, tokenized-text and long-pattern checkpoint directories are created privately (0700), with private regular checkpoint files (0600) owned by the daemon user. Checkpoint access rejects symlinked path components, foreign ownership, broadened permissions, extra hard links and replaced directory or file identities. Paths are bounded by 4096 encoded bytes and 32 components; relative configured paths remain supported. Unsupported platforms refuse this checkpoint custody.

Publication synchronizes complete pending bytes, publishes without replacing an occupied name, and synchronizes the pinned directory. Restart completes only a verified two-name publication or rebuilds absent derived state. Partial work cannot become authoritative state. Checkpoint envelope formats remain unchanged. An installation with older permissive derived directories or files must stop the daemon and make these owned directories and files private before restarting; RiffDB refuses unsafe custody instead of changing permissions on existing objects. Ownership records retain live directory and file descriptors through cleanup, so a deleted object’s inode cannot be reused while its ownership claim remains. Retirement removes only the recorded selected and pending derived objects after all captures release. A foreign replacement is preserved; the retiring registration continues to consume capacity until its custody can be proved.

Exact-provider retention clock

Primary and follower servers use the same process-local monotonic clock for cursor expiry and exact-provider minimum retention. A failed clock sample, backward time, or an unrepresentable retention deadline returns the existing provider Unavailable error. There is no application or operator override. Each exact-provider family retains its 256-registration limit. Under registration or aggregate admission pressure, the server retires the least-recently-used eligible owner, with canonical family/key tie breaking. Eligibility requires the existing 300-second minimum retention to expire and all query, cursor, worker and publication pins to release. Successful access extends retention; failed capture does not. When every owner is protected, new demand returns typed unavailable. Later access to an evicted registration starts ordinary Building/rebuild admission with a fresh generation. It cannot resume a cursor using a substitute epoch.

One pre-admitted 512-KiB cleanup lane remains reserved for the runtime’s lifetime, allowing cleanup at budget saturation. Cleanup runs outside the discovery lock; contention refuses new admission without changing an owner’s phase. Bounded, admitted worker inventories hold keys only and capture one live owner at a time.

A freshly authorized higher revision of the same capability retires its older registrations across all five families. Existing captures remain charged until released; retirement cannot shorten the 300-second retention obligation. Obsolete workers stop before source observation and at bounded replay, scan, materialization and publication points. Each worker pass attempts at most four eligible retired-owner cleanups, including when no capacity pressure exists; this cleanup does not evict idle active registrations.

New exact-provider registrations reserve owned registration data and shared compiled metadata against one compiled 1-GiB process budget. Shared metadata is charged once and retains its charge through its final data reference, including catalog references. Budget pressure or incomplete ownership evidence returns the existing typed Unavailable result. Servers sharing a process share this pool; there is no budget configuration or application cache control. Queries and workers also reserve their explicit capture pins against this pool; capture pressure returns the same typed unavailable result. Encoded checkpoint read buffers reserve capacity before reading and retain their charge through the last buffer reference. Budget pressure leaves rebuild work eligible for a later retry. Provider source decoding and derived-row normalization reserve construction capacity before allocation. Unchanged rows share their admitted original; materialized fields retain their charge through the final semantic row clone. Private provider row and delta conversion buffers also reserve native capacities before copying, and retain their reservations through destruction. The exact-provider worker emits bounded operator diagnostics on stderr at startup, at most once per 60 seconds while its service clock is healthy, and at worker stop. Lines use trigger=exact_provider_lifecycle and fixed numeric fields: the process limit, fixed owner bytes, owned/reserved bytes, outstanding reservations (reserved_work), live/released accounting allocations, active/retiring owners, unique pinned owners, distinct pin guards, and finally released owners. Cloned references to one pin guard count once. Retiring or pinned state retains its bytes; removing discovery entries cannot report an owner as released. Clock or accounting failure emits available=false without fabricated counters. These lines contain no keys, tenants, capabilities, paths or payloads. Multiple runtimes in a process report the same aggregate pool.

Provider catalog capture borrows an exactly matching catalog already published by the ordinary catalog adapter or completed follower prefix. It reserves shared ownership before capture and refuses when that publication is absent, stale or contended; providers cannot cold-decode a private catalog. Retained physical source descriptors retain their metadata charge through their final capture.

Provider receipt replay and source-pin validation reserve native construction work before decoding, including archived history stubs. Retained cursors and receipts carry their charges through destruction; pressure refuses without advancing a receipt cursor. Provider head checks read the immutable snapshot frontier without decoding a command record.

Provider policy reads, observation copies and evaluation work now reserve capacity on the same captured source; resulting proofs retain their charge through final destruction.

A strictly newer contract can retire eligible provider state for the same named operation and exact partition/authority identity after pins and minimum retention clear. Historical explicit queries can rebuild with a fresh owner.

Durable local outbox archive

A primary database can deliver its committed events to one protected local archive. This POC destination runs in a server-owned worker after command acknowledgement. Applications continue to emit events through compiled commands; they cannot select destinations or write archive records.

Precreate an absolute directory owned by the server’s effective user with mode 0700. Every path component must be a real directory, without symlinks. The directory must be disjoint from database, backup, projection, replication, credential, and other destination paths. It contains only archive.redb; that file must be a private regular file with exactly one hard link.

For a single database, add this restart-applied server configuration:

[maintenance.outbox]
kind = "durable_local"
directory = "/var/lib/riffdb/outbox"
quota_encoded_bytes = 1073741824

For named databases use [databases.<alias>.outbox] instead. Unknown options, relative/overlapping paths, zero quotas, and destination configuration on a follower reject configuration. Omitting this section preserves the existing no-destination behavior: intents remain pending and outbox health degrades.

The worker validates the archive in bounded pages before dispatch. Startup validation, unavailability, and exhausted capacity degrade outbox health while primary command durability remains intact. Known destination refusal pauses before claiming a new delivery. Repair availability or permissions in place, or increase the quota and restart. A changed archive binding, incompatible format, or corrupt content stops delivery and requires operator investigation.

Delivery is at least once. Acceptance requires an Immediate, two-phase archive transaction storing the exact event and its quota charge atomically. A crash between acceptance and source completion can replay the event; equal retries preserve one record, and different bytes under the same identity are refused. Database identity, history incarnation, and event identity form the key. Each event keeps its own archive transaction. The worker claims up to sixteen consecutive pending events of one scan page in one primary transaction: the run ends at the first event that is not yet due, has used every attempt, or would not fit the archive quota together with the earlier members, and that event is handled on its own as before. Quota is checked for the run as a whole, so a full archive pauses delivery without spending attempts. Members are then delivered one at a time; a member whose lease has passed by the time it is reached renews it before delivery. The worker likewise completes up to sixteen already accepted events in one primary transaction, flushing at the bound, page end, or before ordinary failure handling. It adds no wait to fill a group. Source completion of an earlier event can therefore follow a later event’s claim; until completion, every such event remains undelivered for retention. A crash can replay a whole bounded group using the same immutable archive identities and existing recovery rules; a member claimed but not yet delivered when the process stopped counts that claim as one of its attempts. Failures discovered after a claim use the existing bounded retry and dead-letter rules. Dead letters remain undelivered and continue to fence source retention.

Quota charges include canonical keys and complete encoded records. Allocated engine blocks and cache memory are additional resources. There is no automatic archive deletion. Preserve the archive across restart and account for this intentionally retained destination separately from reclaimable source history.

Live delivery counts

The primary’s riffdb server stats response reports pending_outbox_deliveries from the existing commit-coupled undelivered index. It requires a global capability with read_statistics. The count includes initial pending events, scheduled retries, active deliveries and dead letters; successful source completion removes an event from the count. Reading it does not scan history, rebuild an index, drain a journal or change a retention hold.

null means the count is unavailable, including while the index is unbuilt or degraded. It cannot establish an empty backlog. The count is a current index observation, sampled separately from the response’s publication frontier, and does not authorize reclamation. Destination readiness still has its own health classification: an empty source backlog alone cannot prove archive custody.

Offline inspection

Stop the owning server before inspection; the archive requires exclusive engine custody. Supply the expected database identity as 32 hexadecimal characters:

riffdb --output json outbox archive inspect \
  --directory /var/lib/riffdb/outbox \
  --database-id 00000000002a71018101010101010101 \
  --max-pages 16

The command reports format, database binding, record count, encoded bytes, physical allocated bytes, pages examined, and integrity. Each page examines at most 256 records and 4 MiB. --max-pages is 1 through 256, default 1. Integrity is verified only after the complete scan matches durable accounting; incomplete means the supplied bound ended first. Increase the bound for a larger archive. Counts alone are not integrity evidence. The validated library reader can continue bounded pages for campaign reconciliation of larger archives.

Inspection refuses a missing archive and never creates destination records. Errors contain no filesystem paths or event payloads. There is no application RPC or MCP inspection method. This implementation requires Linux; it refuses unsupported platforms.

Helm Operations

The RiffDB Helm chart is the supported Kubernetes packaging surface for the proof of concept. It assembles existing safe binaries and references Secrets that an operator provisions separately. It does not create credentials, capabilities, application installations, or any other authority.

The chart is pre-alpha and is restricted to a controlled design-partner network. Network policy, ingress allowlists, certificate issuance, Secret provisioning, backup export, and storage-class behavior remain the operator’s responsibility.

Install

Create the target namespace and the external server Secret before installing. The Secret must contain server-chain.pem, server-key.pem, capability.keys, idempotency.keys, riffdb-ca.pem, and readiness.credential, all from the same reviewed deployment. Keep Secret values out of values files, rendered manifests, shell history, and logs. The application and operator proof Secrets, when enabled, must be nonempty and pairwise distinct from each other and from the server Secret. The chart refuses an overlapping name and projects only credential and ca.pem into each proof pod.

Review a pinned values file and render it before applying anything:

helm lint release/helm/riffdb
helm template riffdb release/helm/riffdb \
  --namespace riffdb-alpha --values /absolute/path/to/reviewed-values.yaml
helm upgrade --install riffdb release/helm/riffdb \
  --namespace riffdb-alpha --values /absolute/path/to/reviewed-values.yaml

An empty image.tag deliberately resolves to the chart appVersion. A nonempty tag must equal that exact appVersion; the chart refuses any other value. Never use a mutable tag. The data claim belongs to the StatefulSet and the backup claim has Helm’s keep policy, so neither is disposable release state.

Controlled network

Permit only reviewed design-partner sources to reach the TLS listener or the pass-through proxy. Every chart-owned probe and proof Job uses the exact <release>.<namespace>.svc identity; there is no values override for another endpoint, host networking, or a loopback shortcut. Refuse public or untrusted ingress at infrastructure boundaries. RiffDB authentication and authorization remain mandatory inside that restricted network; network location is not authority.

Authenticated readiness

Liveness is the closed, payload-free pre-bootstrap health check and carries no credential or database selector. Readiness uses readiness.credential copied from the server Secret into the size-bounded memory-backed protected volume. Its exec gate exits successfully only for the canonical typed ready response; authenticated not_ready, degraded, malformed, or failed responses keep the pod out of Service routing. A TLS-only or process-alive result does not satisfy readiness.

The optional proof Jobs are disabled by default. When explicitly enabled, the operator must first provision distinct applicationSecretName and operatorSecretName Secrets, each containing credential and ca.pem. The Jobs execute only server health, from separate pods, through service DNS. They do not install an application or mint authority.

Optional pod-local telemetry (server.telemetry.metrics.enabled and server.telemetry.flightRecorder.enabled) is also disabled by default. When enabled, metrics_listen binds strictly to 127.0.0.1:<port> inside the pod for pod-local sidecar or kubectl exec scraping (never exposed on the cluster Service), and telemetry_flight_recorder writes bounded periodic NDJSON snapshots to /var/lib/riffdb/data/telemetry-flight.ndjson inside the writable data volume so riffdb telemetry summarize /var/lib/riffdb/data/telemetry-flight.ndjson can inspect latency percentiles and saturation in place.

Follower deployment

To deploy a follower release that replicates from a primary release, set server.mode: follower and populate server.replicationSource (endpoint, serverName, tlsTrustRootFile, credentialSecretKey, database, databaseId, historyIncarnation, leadershipEpoch, and holdId). The follower’s external server Secret must include the replication-stream bearer token under server.replicationSource.credentialSecretKey (default replication-source.credential), which the protect-server-secrets init container copies into /run/riffdb-protected/ with 0400 permissions. The chart refuses follower rendering if any required replicationSource identity field is omitted. See Follower administration for hold registration, status inspection, and audited failover.

Drain

Stop new application, MCP, and administrative submissions at ingress, then terminate the StatefulSet pod normally. riffdbd stops admission and performs its bounded internal drain on SIGTERM. Require a clean container exit before closed-database work; never use a forced deletion as an upgrade step.

Verified backup

Before the drain, create a named backup through the authenticated public maintenance API and retain its operation ID:

riffdb --config /run/operator/client.toml backup create before-upgrade
riffdb --config /run/operator/client.toml backup operation OPERATION_UUIDV7

Proceed only after terminal success. Preserve the immutable backup and verify its manifest as described in Backup and Restore. Do not readmit writes after taking the upgrade backup. Copy retained backups outside the cluster failure domain according to local recovery policy.

Storage preflight

With every RiffDB pod stopped and the database volume mounted exclusively into a controlled maintenance environment, run the target binary’s source-free decision:

riffdb storage preflight --database-path /var/lib/riffdb/data/riffdb.redb

ready permits target startup. upgrade_required permits only the exact manifest-authorized riffdb storage upgrade action printed by the release, using the verified pre-upgrade backup. Any other result is a refusal. Do not edit a marker, receipt, database, or journal to bypass it.

Chart upgrade

Pin and verify the target chart archive and image, render the target with the reviewed values, compare the objects and Secret/PVC references, take the verified backup, drain, and complete Storage preflight. Only then run:

helm upgrade riffdb /absolute/path/to/riffdb-TARGET.tgz \
  --namespace riffdb-alpha --values /absolute/path/to/reviewed-values.yaml

Require complete startup validation, authenticated readiness, and the bounded application smoke before reopening ingress. The repository-local scripts/check-helm-upgrade-rehearsal packages reserved local identities and renders object transitions only; it does not prove or claim published release artifacts, Kubernetes execution, durable compatibility, or database rollback.

Certificate rotation

Issue a successor certificate for the exact configured service DNS name. Update the external server Secret atomically with the successor chain/key and the needed trust overlap, then perform a bounded rolling restart. Prove TLS and authenticated readiness through service DNS before retiring the predecessor certificate or trust root. Never place certificate private keys in chart values.

Credential rotation

Create a least-authority successor capability through the existing authorized capability surface and write its bearer only to the appropriate external Secret. Use the successor, prove, switch, revoke sequence: prove the successor from its separate workload, switch the Secret consumer, confirm authenticated readiness or application health, and only then revoke the predecessor. Rotate application, operator, and readiness principals independently; the chart never creates their authority.

Restore

Restoration is a destructive history replacement, not a chart rollback. Stop all database pods, select a previously verified immutable backup, and follow Backup and Restore. For a nonempty target the public operation requires --confirm-replace-current-database. Require full startup validation and authenticated readiness before reopening ingress.

Deployment rollback

Before the candidate binary has accessed storage, helm rollback may restore the predecessor deployment objects. After candidate storage access, do not assume that object rollback makes the database compatible. Follow the durable format manifest: retry forward with the target release, or deliberately perform the documented destructive Restore from the verified pre-upgrade backup.

Durable downgrade refusal

Helm history is not a database-format history. An unsupported downgrade remains refused even if Kubernetes can render or start an older container. Never force, reset, hand-edit, or delete durable format evidence. If no manifest-authorized edge exists, preserve the stopped volume and backup and request human review.

Remote and Local Application Ingress

RiffDB application gRPC has three and only three listener profiles:

• loopback_cleartext for local development;
• direct_tls for TCP beyond literal loopback; and
• local_socket for a same-host or same-pod first-party client.

Hosted MCP supports loopback_cleartext and direct_tls. There is no insecure non-loopback profile. Every protected operation still requires a RiffDB bearer capability and the normal database selector; transport identity never becomes application authority.

Direct TLS

Configure a canonical public endpoint and one static certificate/key pair:

[server]
audience = "riffdb-application"
capability_keys = "/etc/riffdb/capability.keys"
idempotency_keys = "/etc/riffdb/idempotency.keys"

[server.application_listener]
mode = "direct_tls"
listen = "0.0.0.0:7443"
public_endpoint = "https://riffdb.internal.example:7443"
certificate_chain = "/etc/riffdb/tls/server-chain.pem"
private_key = "/etc/riffdb/tls/server-key.pem"

The leaf certificate must contain the endpoint’s DNS name or IP address in its subject alternative names. RiffDB validates the complete pair and endpoint identity before opening the listener. The certificate must be a regular, non-symlink file that is not group/world writable. The private key must be a regular, non-symlink file with no group or other permission bits.

Replace certificate and key files using atomic same-directory renames. On the next connection RiffDB reads both files as one snapshot and installs the new identity only if the complete pair validates. Invalid or partially published material retains the previous valid snapshot and emits a path-free diagnostic. Established connections keep their existing identity until bounded drain.

The Rust client supports either an explicit CA bundle or the platform system CA bundle:

use std::num::NonZeroU32;
use std::time::Duration;
use riffdb_client_rust::RiffDbClient;
use riffdb_config::{
    CanonicalHttpsEndpoint, ProtectedFilePath, TlsClientConfig,
    TlsServerIdentity,
};

let tls = TlsClientConfig::new(
    CanonicalHttpsEndpoint::parse("https://riffdb.internal.example:7443")?,
    ProtectedFilePath::new("/etc/my-app/riffdb-ca.pem")?,
    TlsServerIdentity::parse("riffdb.internal.example")?,
    Duration::from_secs(10),
    Duration::from_secs(30),
    NonZeroU32::new(1).unwrap(),
    NonZeroU32::new(128).unwrap(),
)?;
let client = RiffDbClient::connect_verified_tls(&tls).await?;

For publicly trusted endpoints (such as Let’s Encrypt or cloud-managed TLS), use TlsClientConfig::system_trust or omit tls_trust_root and tls_server_name in client configuration to verify against the operating system’s CA bundle and derive the server identity from the endpoint host.

For a long-lived host that creates replacement channels, retain a VerifiedTlsConnector instead. It validates the initial CA bundle, atomically adopts a complete valid file replacement for later handshakes, and retains its last valid snapshot when a replacement is missing, unsafe, or malformed. Existing RiffDbClient channels keep the snapshot used for their handshake. last_reload_status() reports only the closed current/retained disposition and never exposes a path or certificate bytes.

The TLS surface uses rustls with the reviewed ring provider, certificate/key files on the server, and either an explicit CA root or the platform system CA bundle plus verified peer identity on the client. There is no mTLS, trust-all, redirect, downgrade, cipher-suite, protocol-version, or provider configuration.

The CLI accepts the same verified target through its ordinary client file:

[client]
endpoint = "https://riffdb.internal.example:7443"
database = "default"
credential_file = "/run/secrets/riffdb-application.credential"
tls_trust_root = "/run/secrets/riffdb-ca.pem"
tls_server_name = "riffdb.internal.example"

When tls_trust_root is omitted on an https:// endpoint, the client uses the platform system CA bundle; when tls_server_name is omitted, it derives the expected server identity from the endpoint authority host. --tls-trust-root, --tls-server-name, RIFFDB_TLS_TRUST_ROOT, and RIFFDB_TLS_SERVER_NAME are the only CLI and environment overrides for those TLS selectors. The CLI does not accept inline certificate PEM, bearer material, or a verification override on the command line.

Liveness and readiness

riffdb server health without a credential is a payload-free liveness probe. It must also omit a database selector. Its successful response says only that the application protocol process is alive: lifecycle is unspecified, the database alias and authentication audience are empty, and readiness is false. Adding a database selector to an unauthenticated probe rejects rather than confirming whether that alias exists.

Database readiness is a separate authenticated probe. Supply the selected database and a protected credential authorized for health; the response may then include lifecycle, readiness, database alias, audience, and component status. The canonical component names are authoritative_storage, catalog, commit_coordinator, projection, outbox, vector_staleness, and replication; each is healthy, degraded, or unavailable. vector_staleness reports embedding quality state under its own identity and must not be interpreted as projection or vector-index readiness.

On a primary, vector staleness is observed from the active contract’s retained vector health summaries. A breached threshold reports degraded; unavailable or inconsistent observations report unavailable. Contracts without vector fields omit that component. Once authoritative serving is available, a non-healthy projection, outbox, vector-staleness, or replication component makes aggregate authenticated health degraded. Container routing must use authenticated readiness, while process restart may use unauthenticated liveness. Neither probe prints bearer material.

Replication health and statistics

Both node roles publish one replication health component. Its typed counters also appear in authenticated gRPC Statistics. role is primary or follower. Each frontier contains separate application and administration positions, including an explicit before-first position. The counters are observations; they cannot satisfy a read’s freshness policy or authorize promotion.

The continuous follower waits for publication notifications on an open source stream. A clean stream close reconnects after a fixed 250 ms pause and resets failure backoff. Transient source failures back off from 250 ms to at most 30 seconds; a successfully applied frame also resets that backoff. Shutdown interrupts either wait and drains storage work before releasing custody. A complete frame already received is validated and durably applied before shutdown releases the applier. Corrupt received input fails the drain; no new stream is opened after shutdown is requested.

Lag is measured in logical sequences, never elapsed time. Control-only receipts do not increase it. A zero value means the two observed positions match; omission means unknown or not applicable. A primary with no registered followers reports count zero, omits acknowledgement and lag, and has healthy replication. A follower can have an applied prefix without a recorded local acknowledgement after recovery. Older sources may omit source-head metadata; the follower then omits lag instead of reporting zero. MCP and CLI present 64-bit sequences and lag as decimal strings to preserve their full range.

Other replication observations are healthy only when both lag counters are zero and acknowledgement is present; otherwise they are degraded. A primary also reports degradation when a live registration reaches its hold budget or configured sequence expiry, even if both lag counters are zero. This health observation never releases retention by itself. The internal expiry continuation requires a previously persisted degradation point and an atomic audited release; the primary schedules it through its existing coordinator. The observer makes bounded passes and joins before coordinator shutdown. Operators manage these policies through follower registration and retirement. A follower requires an available replication component, catalog, and authoritative read storage for readiness. It has no primary commit-coordinator component. An unavailable replication component makes follower health not_ready.

Under the accepted follower audit boundary, authorized Health/Statistics and authenticated denied reads emit bounded redacted telemetry. They do not append local durable audit. Writes and reads whose policy requires durable audit remain refused with the typed follower-mode outcome.

Application credential rotation

An application deployed with --provision-role retains its exact role hash, capability ID, protected credential path, and resumable deployment state under .riffdb/deployments/<database>/. Rotate it by rerunning the same exact locked deployment with the same role and explicit replacement:

riffdb --config /run/secrets/riffdb-operator.toml application deploy \
  riffdb.application.json \
  --lock riffdb.application.lock.json \
  --provision-role TicketDeskApplication \
  --replace-role-credential

Replacement is a retained campaign, not revoke-then-create. RiffDB records a fresh successor identity, creates a new create-only protected credential file, proves the selected database and audience with authenticated health, proves the successor’s active contract and complete authorized command/query catalog, switches generated application client configuration, then explicitly revokes the predecessor. The predecessor file is removed only after revocation is durably observed. The bearer is never returned in command output or deployment state.

If the process stops at any phase, rerun the identical command with --replace-role-credential. Omitting the flag while a campaign is active fails with a resume instruction. If a crash loses a just-created successor secret before local retention, the retry revokes that exact unusable successor, preserves the predecessor, and asks for one more retry with a fresh identity. A capability-ID conflict is never treated as authority owned by the campaign and is never revoked.

Certificate/trust rotation and capability rotation are independent. Replacing TLS files does not create, widen, or revoke an application role; replacing an application credential does not modify TLS trust.

Protected local socket

For a same-host or same-pod process, configure:

[server.application_listener]
mode = "local_socket"
path = "/run/riffdb/application.sock"
access = "owner_only"

owner_only publishes mode 0600; owner_and_group publishes 0660. The parent directory must already exist and must not be group/world writable. RiffDB refuses an existing pathname instead of deleting a potentially unrelated file. After an unclean stop, the operator must verify and remove a stale socket before restart. Normal shutdown removes only the socket inode the process created.

The socket protects the transport boundary; it does not bypass bearer authentication, database selection, authorization, or shared application service semantics.

Bounds and proxies

The optional [server.application_listener.bounds] table sets bounded maximum connections, HTTP/2 streams per connection, handshake, idle, keepalive, and drain durations. Defaults are documented in the configuration reference. These are resource ceilings, not application authority or durability controls.

Successful reads or writes reset the connection idle deadline. Both directions observe expiry even when separate tasks are waiting to read and write. Cancelling a Rust client call after its response has already been routed leaves the session usable. Cancelling an outstanding call retains bounded correlation state for late responses; command outcome uncertainty still requires the normal idempotency recovery flow.

RiffDB ignores Forwarded, X-Forwarded-*, and proxy-supplied principal, tenant, database, and authorization assertions. A proxy may carry the byte stream, but the ordinary RiffDB credential and current authorization checks are always repeated by the shared service.

Per-database storage ceilings (max_storage_bytes), write/read token-bucket rate limits (max_write_rps, max_read_rps), and blocking pool bounds are configured in [maintenance.limits] or [databases.<alias>.limits]; see the configuration reference. Deploy remote ingress behind external network connection controls as well when operating shared infrastructure.

Compose and Kubernetes

The current checked release surfaces are:

• release/container/compose.yaml, with a direct-TLS database, TCP pass-through proxy, and disjoint one-shot application/operator proof containers; and
• release/helm/riffdb, with a stateful database pod, retained backup volume, authenticated typed-readiness gate, payload-free liveness, pass-through proxy, and exactly two opt-in proof Jobs using separate external Secrets.

release/kubernetes/riffdb.yaml and scripts/remote-kubernetes-render-check are retained historical fixtures. They are not the current NET-010 proof, an operator installation path, or controlled-cluster execution evidence.

The release container normalizes the installed riffdb and riffdbd modes inside the image before switching to its fixed non-root identity. A restrictive host build umask therefore cannot produce an image whose runtime user cannot execute the installed binaries. The acceptance harness also detects a rootless Podman-backed docker command and applies keep-id only to the three RiffDB containers, preserving owner-only config and credential mounts without running the service as container root. Docker Engine uses the checked Compose file without that override.

The application workloads mount no database, backup, digest key, TLS private key, or operator credential. Chart Secret projections select only the required keys and are copied by a bounded init container into a size-bounded, memory-backed owner-only directory because RiffDB deliberately rejects group-readable bearer and private-key files. Chart-owned probes and proof Jobs use the exact release Service DNS identity; the values surface has no endpoint or TLS-name override that can redirect them. Application routing begins only after the authenticated probe returns the canonical typed ready result. Bootstrap remains local-authority-only and outside chart assembly. The Compose ceremony executes that one operation inside the database container’s loopback namespace and publishes the result to a protected operator-only handoff directory. It does not permit bootstrap through the proxy.

Run ./scripts/remote-compose-acceptance for the real container proof, ./scripts/check-helm-operator-package for the parsed chart gate, and ./scripts/check-helm-upgrade-rehearsal for the explicitly offline non-release lifecycle render. The Compose proof builds the release image, starts sibling containers, bootstraps through the TLS proxy, proves separate credentials, rotates certificate files and the application credential, rejects the revoked predecessor, and performs a bounded graceful stop. The pass-through proxy uses the container runtime’s resolver with bounded retries, so replacing or restarting the database container cannot leave it pinned to a stale backend address. Neither surface treats proxy headers or network placement as authority.

Follower registration and retirement

Operators can register and retire a follower through the primary’s gRPC AdminService, the Rust client, or riffdb follower. Both operations use the shared application service and sole commit coordinator. They require a current global AdministerCapabilities capability with all-partition scope. Replication-stream permission alone is insufficient. Application roles, row policies, tenant restrictions and unmet approval requirements refuse these operations. They are not MCP tools.

Registration is available before contract deployment. The primary checks its current database ID, history incarnation and leadership epoch. Followers refuse both operations with the typed follower-mode error.

Register a follower

Use the exact source identity and the same nonzero opaque 16-byte hold ID that the follower’s replication configuration will use. Obtain the source identity from the validated source configuration or authenticated replication observation. The examples use placeholder source identity and protected operator configuration; replace them with the intended source values.

riffdb --config ./operator.toml --output json follower register \
  --database-id 018f22a1-7b3c-7def-8123-456789abcdef \
  --history-incarnation 1 --leadership-epoch 1 \
  --hold-id 71717171717171717171717171717171 \
  --hold-budget-sequences 100000

The sequence budget must be positive. An optional --expires-at-sequence N selects a positive application commit sequence, not a duration or wall-clock deadline. Reaching the budget degrades health and continues to retain history. A configured expiry first persists degradation; a later bounded coordinator turn can commit its audited release. See replication health.

A new registration retains the source frontier while awaiting bootstrap. It can also adopt an existing legacy follower hold without advancing that hold’s fence. Registration does not install a follower, start replication, or prove that it has caught up.

Success returns administration_sequence, registration_generation, and status (applied or replayed). CLI sequences are decimal strings. Keep the registration generation for retirement. The generation identifies this registration; it is not an application commit sequence or an authorization token.

Retry and retire

An exact registration retry uses the same lineage, hold ID, budget and expiry. Each submission uses a fresh request ID; the CLI creates one automatically. The retry returns the original receipt. A changed policy refuses. Retrying after retirement returns historical registration evidence and does not recreate the hold. Use a fresh hold ID for a new registration. Retired identities remain tombstones and count toward the POC’s 4096-entry source-hold bound.

Retire the exact generation returned by registration:

riffdb --config ./operator.toml --output json follower retire \
  --database-id 018f22a1-7b3c-7def-8123-456789abcdef \
  --history-incarnation 1 --leadership-epoch 1 \
  --hold-id 71717171717171717171717171717171 \
  --registration-generation 10

Replace 10 with the returned generation. Retirement releases this follower’s retention fence and matching bootstrap custody. Independent archive holds remain. An exact retirement retry returns the original retirement receipt; a stale generation refuses. A registration already released by configured expiry cannot be reported as a successful explicit retirement.

The closed refusal results are lineage_mismatch, registration_conflict, registration_missing_or_stale, expiry_reached, and capacity_exhausted. Authentication, authorization, transport and uncertain-outcome errors use the existing typed error surface. After an uncertain outcome, retry the identical selection and policy with current authority. Changing identity would select a different operation.

Audit and POC limits

Started and terminal service audit records retain the exact lineage-scoped follower target in ServiceAuditRecordV3. Success links the authoritative registration or retirement receipt, including on replay. The hold, administration record and V3 physical transaction commit atomically. Current authorization is checked before admission, after capacity admission, and against transaction- current facts before the coordinator commits.

Old audit generations retain their bytes. Use matching updated clients for the new RPCs; see compatibility. Automatic registry migration is not provided. The fencing and promotion path below has recovery and under-load proofs, a completed CI battery, and receipted replication acceptance evidence.

Inspecting replication status

Operators can inspect the replication role, frontiers, lag sequences, and registered follower count directly with riffdb follower status:

riffdb --config ./operator.toml --output json follower status

riffdb follower status queries the authenticated Health RPC and extracts the replication component and ReplicationStatistics (role, history_incarnation, source_frontier, applied_frontier, acknowledged_frontier, registered_followers, application_lag_sequences, and administration_lag_sequences).

Primary fencing, promotion, and one-command failover

RiffDB exposes riffdb follower fence-primary, riffdb follower promote, and riffdb follower failover through the same checked Rust client, gRPC service and policy owner. Automatic election, quorum durability, and cascading followers remain outside the POC scope.

Fencing requires the distinct global, all-partition fence_replication_primary permission and a currently attached registration. It drains admitted commands, permanently refuses subsequent authoritative writes as RDB-REP-0102, and survives restart. There is no unfence command. Required audit and replication evidence remain available. A lost response requires an exact retry with the same operation ID, follower and generation.

Promotion requires current global AdministerCapabilities authority. Point operator configuration at the follower; retain the source fence operation and registration selection. The daemon obtains proof through its configured CA/name-verified TLS source connection. The caller cannot supply a new incarnation, epoch, RPO or unauthenticated proof.

Operators can run the two steps individually or orchestrate both steps in a single audited command with riffdb follower failover:

riffdb --config ./follower-operator.toml --output json follower failover \
  --primary-config ./source-operator.toml \
  --database-id 018f22a1-7b3c-7def-8123-456789abcdef \
  --history-incarnation 1 --leadership-epoch 1 \
  --hold-id 71717171717171717171717171717171 \
  --registration-generation 10

When --fence-operation-id or --operation-id are omitted, riffdb follower failover mints fresh UUIDv7 operation IDs before contacting either node, fences the primary first, verifies the applied or replayed fence receipt, and then promotes the follower. If either step returns an uncertain outcome, the CLI error output preserves both fence_operation_id and operation_id so the operator can retry the exact same operation IDs idempotently:

riffdb --config ./source-operator.toml follower fence-primary \
  --database-id <SOURCE_DATABASE_UUIDV7> \
  --history-incarnation <INCARNATION> --leadership-epoch <EPOCH> \
  --hold-id <32_LOWERCASE_HEX_DIGITS> \
  --registration-generation <GENERATION> --operation-id <FENCE_UUIDV7>

riffdb --config ./follower-operator.toml follower promote \
  --database-id <SOURCE_DATABASE_UUIDV7> \
  --history-incarnation <INCARNATION> --leadership-epoch <EPOCH> \
  --hold-id <32_LOWERCASE_HEX_DIGITS> \
  --registration-generation <GENERATION> --operation-id <PROMOTION_UUIDV7> \
  --fence-operation-id <FENCE_UUIDV7>

Success reports the exact applied application sequence, new incarnation and leadership epoch, administration sequence and application-sequence RPO. Scoped causal tokens from the former incarnation return RDB-HISTORY-0101 on the promoted source, including while its local projection is unavailable. Opaque continuations from the former service generation are invalid; restart a query from fresh observations. JSON integers are decimal strings; BeforeFirst is null. Keep both operation IDs and the complete selection unchanged after an uncertain response. An exact retry on a successfully promoted source checks fresh authority, returns the original result and audits the invocation without contacting the former source.

Standby client failover and read-replica routing

Applications using StableApplicationClient or riffdb-driverd can configure up to 4 standby_endpoints and up to 4 read_replica_endpoints in the driver host [remote] section (or via with_standby_clients and with_read_replica_clients in Rust):

• When the active primary returns RDB-REP-0102 (PrimaryFenced / FollowerMode) or transport unavailability during an idempotent command or query, the client probes configured standby endpoints in order, verifies the active contract identity (GetActiveContract), promotes the verified standby channel to active primary, and retries the operation with the original idempotency key.
• When read_replica_endpoints are configured, read queries that do not require AdmissionHead consistency route to healthy follower replicas first and fall back to the active primary if a replica is unavailable or lags behind a causal read_after_commit frontier.

Keep <backup-root>/.maintenance/replication_promotion with the database and maintenance receipts. Missing or contradictory evidence fences readiness. A failed or denied attempt that already froze a selection continues to fence ordinary startup; terminal failure does not release its selected frontier. Only complete reconciliation of an exact successful retry under the current exclusive owner can discharge that selection. Earlier interrupted attempts for that exact operation retain their original audit rows; a successful retry does not fabricate terminal replies for them. Denial before selection does not leave this fence. Never delete receipts to bypass recovery.

After committed cutover, restart validates the exact local control, audit and external receipt before starting as a source, without loading the former source’s credentials. Pre-cutover restart fully validates the local follower and hosts only authenticated PromoteFollower retries. It neither connects to the source nor advances replication at boot, and emits no application-ready receipt. Health, application reads/writes, bootstrap, replication and maintenance remain unavailable. Use the same promotion operation, fence operation, target and registration generation with a fresh request ID and current authority. Other authenticated attempts are durably denied without changing the selection.

An authorized retry requires the configured source to be available for fresh authenticated fence proof. It drains local readers, rechecks current capability facts and expiry, and preserves the selected frontier before cutover. Success is returned only after complete source validation and service activation. If interrupted, retain the same configuration and evidence and retry; shutdown does not imply that an uncertain cutover failed. Follower projection generation and lifecycle remain locally owned derived state; authoritative commit/entity zero-RPO claims do not extend to derived state.

Multiple Databases

One riffdbd process can host up to 512 independently durable databases. Each configured alias has its own redb file, catalog, credentials, commit sequence, projections, outbox, health state, and recovery boundary.

Selection occurs before authentication

Every public connection selects a canonical database alias before its credential is authenticated. A credential is bound to one database audience and cannot authorize another alias. The server does not fall back to a default database when an explicit selection is unknown or malformed.

Public paths select the alias as follows:

Add or remove a database

When riffdbd is started with a configuration file that declares [databases.<alias>] tables, the daemon monitors that configuration file for modifications and reconciles the configured database inventory online without restarting the process or interrupting active peer aliases:

• Online provisioning: Adding a new [databases.<alias>] entry opens or initializes its path, starts its per-database workers (and idle-hibernation supervisor when limits.idle_hibernate_seconds is configured), and registers the alias on the gRPC and hosted MCP routing tables.
• Online deprovisioning: Removing an existing [databases.<alias>] entry removes the alias from gRPC and hosted MCP routing, drains its per-database workers, writes CleanCloseState::Clean, and releases its redb file lock while leaving its durable file on disk.
• Fail-closed validation: If a modified configuration file is malformed, exceeds 512 databases (512 KiB configuration document bound), rebinds an active alias’s path, or mutates process-wide listener, TLS, authentication, or resource settings, the reload is rejected with a warning diagnostic and the previously active database inventory remains unchanged.

For installer-managed single-user deployments, you can also use the database management workflow described in Installation.

Do not copy a credential from one alias into another configuration. Create or bind the application role against the target database so the credential, audience, active contract, and named operations agree.

MCP per application

Create one MCP configuration per database and role. Multiple entries may point to the same riffdb-mcp binary and riffdbd endpoint, but each bridge is bound to exactly one selected alias and credential. This preserves least authority without requiring a separate server instance.

Run before registering a bridge:

riffdb-mcp doctor --config ~/.config/riffdb/mcp-myapp.toml

The doctor verifies routing and credential/tool discovery without widening a narrow application role merely to grant health access.

Isolation and operations

Backup, restore, reset, and removal operate on one database while the service is offline. A fault in one database can make that database unavailable without changing another database’s authoritative state. Process-level resource limits remain shared, so capacity planning must account for the combined configured set.

Startup Integrity and Clean Restarts

RiffDB chooses its startup-validation mode automatically. There is no flag, configuration setting, API, or maintenance shortcut that can force the faster mode or suppress complete validation.

An ordinary primary open synchronizes its parent directory after the engine file exists and before returning a usable store. A failed directory sync refuses the open. Reopening repeats this check, including when a previous attempt left a valid file and format marker behind; journal creation is not required to make the primary filename durable.

The changelog V3 substrate adds an internal first-activation transition. An inactive database must complete structural and catalog validation before one hardened transaction publishes the V3 registry and all required roots together. The exclusive startup lease remains held through that proof join and the DIRTY transition. A database already claiming the current V3 registry refuses missing roots; deleting them cannot request legacy startup or a fresh activation. No operator switch or application API controls this transition.

After a graceful shutdown, RiffDB writes a private clean-close lifecycle record as the final authoritative database mutation. The record binds the database identity and history incarnation, durable-format registry, exact application and administration frontiers, selected journal header, fixed metadata roots, active contract catalog, active query modules, and initial authority roots. The next process may use bounded startup validation only when all of those facts still match and the storage engine reports no repair. Before any writer, worker, maintenance task, or public service becomes active, RiffDB durably replaces that clean state with its next dirty generation. A crash after readiness therefore cannot leave reusable clean evidence.

Missing, dirty, malformed, stale, copied, or contradictory lifecycle evidence does not select a population walk. Such a startup opens the storage engine, checks the durable-format identity, recovers the complete journal suffix, validates the same bounded roots the clean path verifies, admits the current registry through a retained checkpoint or exact Completed binding, consumes the dirty generation, and only then publishes readiness. It does not enumerate entity, index, history, event, audit, provenance, projection, outbox, or idempotency rows, and repeating it creates no command, event, outcome, provenance, audit, projection, outbox, or allocator advance. The state that recovery itself owns is capped at a compiled 64 MiB; exceeding the cap is a typed internal-defect refusal, never a partial start.

After an abrupt stop the storage engine also repairs its own allocator state on the first open, reading the whole database file. That repair is storage-engine recovery, not a RiffDB population walk; startup reports it as an engine repair, and its duration grows with file size. On the reference benchmark host a kill restart reached readiness in about 0.5 seconds at 65,536 rows and about 53 seconds at the production benchmark population (about 1.1 million rows), almost all of it engine repair; the RiffDB startup stages together took well under a second. RiffDB leaves the engine’s quick-repair mode off because it would change commit behavior. These figures are host-specific observations, not guarantees.

The first write after any restart also rebuilds the process’s in-memory command indexes once, because steady-state writes depend on them. At the production benchmark population that rebuild holds writes for tens of seconds; reads and readiness are unaffected, and later writes run at normal speed.

Complete startup validation is still used when those bounded-recovery preconditions do not hold: a database with neither a loadable validated-prefix checkpoint nor an exact Completed registry binding, an inactive or incompatible registry, or a follower start without a valid clean-close certificate. Pending registry metadata provides no shortcut. A mismatch in a bound root fails closed. None of these cases asks the operator to classify the preceding shutdown.

Complete native structural and catalog-semantic validation publishes the private Completed registry binding through the sole ordered writer, atomically with its prefix-completion receipt. It binds the exact database, history incarnation and current registry. Partial validation, cancellation, migration, a bounded restart and background refresh cannot create that witness. An uncertain commit fences writes until normal recovery resolves it.

Restore advances the history incarnation. Copied completion and checkpoint bindings cannot certify complete validation of that successor. An ordinary self-contained restore instead binds fresh bounded readiness to its durable restore receipt, verified backup, database identity, new incarnation and exact application frontier. It checks private physical integrity and recovers the complete journal before publication, then validates bounded roots and consumes the dirty generation on reopen. It creates no complete scrub proof, checkpoint, completion witness or clean certificate. Its first-access checks and background scrub retain the assurance boundary below. Legacy migration, archive restore, follower bootstrap and promotion retain their required complete validation. The copied Completed binding still names the predecessor incarnation. A later dirty restart without matching clean evidence or another eligible checkpoint therefore uses complete validation under the existing startup rules. Bounded restore does not silently renew that binding; a successful restore alone must not be interpreted as a promise of bounded startup after every later crash. When complete validation runs, it can replace the exact predecessor checkpoint bytes with a new proof chain and publish its own completion binding in the same durable receipt. An ordinary checkpoint refresh cannot perform that renewal.

Online retention still deletes the optional validated-prefix checkpoint in its first prune step. Ordinary writes and prune preserve the Completed binding, so its absence does not require another population walk. This binding proves completed format/catalog validation; it grants no source-reclamation authority or evidence that every current row is intact. First-access checks and the complete rate-bounded background scrub remain required.

With sequence retention enabled, maintenance can independently rebuild the deleted optional checkpoint from complete pinned structural, catalog, and retained-history validation. Publication refuses a changed history binding and yields to busy writers or unfinished checkpoints. It does not reuse the invalidated proof or force a journal drain. See retention configuration for the maintenance bounds. The sustainable TicketDesk milestone separately measures the first successful write after restart.

Assurance boundary

A clean or bounded dirty restart is a continuity proof, not a full database scrub. It verifies the bounded roots needed to establish operational readiness without walking every live or retained population row. Rows skipped by startup retain their local envelope, checksum, canonical-key, schema, size, and required relationship checks before they can affect authorization, execution, output, delivery, projection, or mutation. Latent corruption unrelated to the bounded roots can therefore be reported when the affected row is first accessed.

Those checks validate RiffDB records and their relationships. Ordinary redb reads do not verify every traversed engine page checksum. Bounded readiness and the background semantic scrub therefore do not certify the entire physical engine tree; explicit physical verification belongs to the offline scrub.

Databases created by current binaries carry the V2 authoritative state catalog. A V2 database never takes the clean-certificate fast path; each of its restarts that meets the preconditions above uses the bounded recovery path, so a graceful restart and a crash restart pay the same bounded cost. Before readiness, RiffDB point-validates the database’s source admission and, when it is fenced, the referenced fence record and its exact receipt; a missing or contradictory record refuses startup. The exhaustive search for a fence row written outside any transaction belongs to the scrubs described below. V1-catalog databases retain the bounded clean-certificate path exactly as documented above.

Background and offline scrubs

Because no restart walks the population, exhaustive corruption discovery happens in two places, both read-only: a background scrub after every start, and an offline scrub an operator requests. Neither repairs, deletes, or hides a row, and neither is an application, agent, or MCP operation.

After readiness, one server-owned background scrub reads the complete structural and retained-history evidence through a least-authority reader at a fixed rows-per-step rate. Authenticated health reports it as the storage_scrub member, which holds exactly one state: pending, running, complete, failed, or cancelled. Every process start begins at pending, and shutdown cancels a running scrub. A failure records one incident ID in the member and sets the authoritative_storage component to degraded, so aggregate health reports degraded while service continues and the damaged rows keep failing closed when accessed. On a V2 database a retained-history failure, such as a stray fence row, instead withdraws authoritative readiness. Health carries no diagnostic text, row, key, count, path, or hash. The scrub has no rate, schedule, scope, or suppression setting. It is one pass over the whole database that keeps one core busy until it completes, so throughput just after a start is lower until storage_scrub reports complete; benchmark windows start only after that.

riffdb storage scrub requests the complete exact-end validation on demand. It uses the same global AdministerCapabilities authorization and offline maintenance lifecycle as backup and restore: the server stops admission, drains accepted work, closes the database, streams the same complete structural and catalog-semantic validator that staged restore and format upgrade use, and then reopens through ordinary startup. The command takes no arguments. It generates the operation ID and prints it; observe the operation with riffdb backup operation <maintenance-operation-id>. Its phases are accepted, draining, offline, validating, and then succeeded or failed_closed. A scrub observation has kind storage_scrub and no backup name.

Before the complete record walk, offline scrub captures one private verification copy of the primary under exclusive original-file ownership and runs redb’s complete engine-integrity checker on it. The record walk uses that same copy and reads the original native journals and retained extents. Neither pass replays a journal or repairs the original. A repair callback, an unsuccessful integrity result, source substitution, cancellation or unavailable capacity refuses success. A journal suffix needing recovery must first pass ordinary startup recovery; scrub cannot certify a partially materialized frontier.

Reserve additional local disk space for one fully allocated copy of the primary file, plus filesystem overhead. The copy cannot grow beyond the captured primary size; inability to allocate or verify within that capacity fails closed. The verifier uses a private directory beside the primary, releases the copy after success or ordinary failure, and reclaims its single recognized scratch artifact after process exit. Unexpected files, symlinks, an original-file alias or uncertain custody refuse cleanup and verification. No repaired copy is published.

A validation_failed scrub means the pass refused the database: incomplete, contradictory, corrupt, or over-limit evidence all fail closed. The server process then exits. Its next start opens the database through ordinary startup with the terminal receipt retained, and the background scrub reports the same defect. Keep the incident, and restore from a verified backup when directed. A succeeded scrub creates no clean certificate, forces no readiness, and does not suppress the next background scrub.

Only one maintenance operation of any kind may be incomplete at a time. If the process stops during a scrub, the next start reruns the pass from its beginning before ordinary startup; there is no resume point. Retrying the same operation ID returns its stored observation, and reusing that ID for a backup, restore, or retirement is refused as idempotency-key reuse. The scrub’s receipt is a bounded, checksummed version-4 maintenance receipt beside the other receipts in the maintenance directory. Do not edit or delete lifecycle metadata or receipts to simulate a scrub.

Graceful shutdown

Configured columnar sources remain cold while the authoritative graph reaches core readiness. Startup resolves their bounded checked registrations but does not open or decode a generation artifact, build a snapshot, catch up, publish, or checkpoint. Consequently a no-demand graceful close also performs no columnar population work. The projection health component is degraded while any registered columnar source remains cold or is activating; this derived health signal does not withdraw authoritative storage, catalog, or command readiness.

The first semantic projected read requests activation from the single server-owned columnar worker and immediately receives the existing typed Building result with no rows. Concurrent first reads coalesce onto that same activation, and cancelling a requesting read does not cancel the shared worker operation. There is no prewarm, eager-mode, activation, artifact-selection, or validation-bypass setting in the CLI, configuration, SDKs, MCP, or transports.

Only the current selected generation becomes queryable after its complete bounded validation. A corrupt, mismatched, stale, excessive, incompatible, or unavailable selection fails closed and serves no rows; RiffDB never substitutes another generation, layout, source, frontier, or authoritative scan. Repeated requests do not create an artifact reopen loop. Correct the underlying storage or selection problem and restart the process; a separately published vector generation may also be admitted through its normal durable lifecycle. Do not weaken freshness or emulate the query against authoritative rows.

Shutdown first stops admission and drains writers, workers, and the journal. If the columnar worker is catching up, it observes the internal stop only after finishing its current authoritative page of at most 64 commit records. It may discard later unpublished in-memory columnar work at that boundary. Any safe prefix published before the stop remains valid, but the stop itself does not publish, notify, checkpoint, or advance the durable projection frontier. There is no application, agent, configuration, or operator option for choosing this behavior or changing its page size.

After restart, columnar state resumes from its last durable projection frontier and replays the authoritative log idempotently. Projected reads can temporarily return their existing typed building or lagging outcome until replay catches up; this does not mean authoritative entity or commit data was lost. An operator should retain the authoritative history required by the projection and allow normal catch-up rather than editing projection files or weakening a query’s declared freshness policy.

After the durable journal-suffix barrier, RiffDB inspects the optional validated-prefix checkpoint through fixed metadata and table cardinalities. It never rebuilds, repairs, deletes, or rewrites that proof during shutdown. An exact-current checkpoint and every companion proof row remain byte-identical; an absent checkpoint remains absent, and stale or ineligible bytes remain unchanged. Those states do not prevent a clean close because the lifecycle certificate is independent evidence.

Shutdown finishes journal sealing already in progress and validates the empty active journal, but does not preallocate a new spare extent for future writes. Unused journal scratch is removed before the clean lifecycle record is written.

RiffDB then rereads the bounded lifecycle roots and writes the clean lifecycle record with immediate durability as the final database mutation. Failure before that commit leaves DIRTY; commit uncertainty is resolved on reopen as either the complete prior DIRTY state or the complete CLEAN successor. A later dirty open still verifies the retained prefix and exact suffix or performs complete validation. Shutdown never moves that population work into the close path.

The internal shutdown receipt reports only a closed checkpoint disposition, a closed lifecycle outcome, and three saturating stage durations. It contains no database identity, path, frontier, generation, hash, key, value, row count, or engine diagnostic.

Startup and shutdown durations are observations, not correctness deadlines or comparative database benchmarks. Their cost depends on durability mode, storage engine recovery, dataset shape, filesystem, and whether the preceding close produced eligible clean evidence. The emitted lifecycle observations use only closed stage names, bounded durations, success/failure state, and the automatically selected startup mode. They contain no database path, identifier, frontier, table population, hash, key, or application value, and none is an accepted configuration or command input.

See also Compatibility, Backup and Restore, and Troubleshooting.

Command payload cache

Command lookup keeps complete exact locator metadata for retained history, while its disposable decoded payload cache is capped at 64 entries and a conservative 64 MiB ownership charge. This bounds payload retention, not total database memory: exact locator, manifest and event metadata still grow with retained identities. Startup rebuild streams one bounded segment at a time. The payload cache fills lazily from validated reads, so publication does not encode or decode commands again to populate it.

Eviction can add an exact storage read and decode. Both cold and cached reads validate against the caller’s captured storage view; missing or corrupt evidence is an error, never evidence that a committed command is absent. Follower replacement retains the prior manifest and event membership needed to remove old indexes even after payload eviction. These limits are internal and do not add operator tuning or weaken recovery checks.

Offline Backup and Restore

RiffDB’s POC backup and restore operations are offline, exclusive maintenance operations. They are exposed through the public gRPC API, Rust client, and riffdb CLI. They are not MCP tools, and neither the CLI nor a client receives or resolves a server filesystem path.

Configuration

For an already-active changelog V3 database, the offline incarnation stamp validates retained receipts and source fences before atomically installing a new lineage anchor and clearing replication-local progress. A destructive restore cannot continue the old replication tail; a receiver needs a new bootstrap. The public maintenance operation ID and retry protocol below are unchanged.

The offline retention-watermark stamp also validates retained V3 history before returning from an equal-value retry. A changed watermark and its exact V3 receipt commit together in the existing hardened transaction. This adds no commit or flush and changes neither watermark normalization nor the recorded tombstone chain-root binding.

Retention also runs online inside riffdbd when its configuration names a retained-sequence target; see Online retention. The online lane uses the same fencing set, tombstone chain, and watermark as the offline operations below, which remain available on a stopped database.

Offline and online retention both respect each registered follower’s durable acknowledged application frontier. A follower that has acknowledged no application commit pins the watermark at zero; a faster follower cannot release a slower follower’s fence. Retention status reports FollowerLowWater when this is the binding minimum. Unreadable or inconsistent V3 hold evidence refuses collection and prune. Budget exhaustion degrades typed Health and never releases a hold by itself. The internal coordinator supports audited retirement and configured sequence expiry, with durable degradation before expiry release and bounded background scheduling. Follower registration and retirement are available through the primary operator interfaces. Fencing and promotion remain POC-scoped.

riffdbd requires an absolute --backup-root path. The path must be lexically disjoint from the database file, capability digest keys, and idempotency digest keys. Only the server joins a checked backup name beneath this root.

The service account needs permission to:

• read and write the backup root;
• create and synchronize <backup-root>/.maintenance;
• create and retire immutable named backup directories; and
• close, reopen, and, for restore, replace the configured database target.

Do not place the database or secret key files beneath the backup root.

Names and Operation IDs

A backup name contains 1 through 64 ASCII bytes. Its first byte is a lowercase letter or digit. Remaining bytes may also contain - and _. Dots, path separators, drive prefixes, whitespace, and Unicode are rejected.

Each public API start request carries a UUIDv7 maintenance operation ID. SDK callers generate it; the CLI generates it for the operator and returns it in accepted or uncertain output. Keep that ID until the operation reaches a terminal state. An API/SDK retry uses:

• a fresh transport request ID;
• the same maintenance operation ID; and
• the exact same operation kind, backup name, and restore confirmation.

Reusing an operation ID with different semantic input fails with a stable input mismatch and starts no work.

Create a Backup

riffdb --config "$HOME/.config/riffdb/client.toml" \
  backup create before-upgrade

The CLI prints the maintenance operation ID and the current bounded status. Poll an accepted operation with:

riffdb --config "$HOME/.config/riffdb/client.toml" \
  backup operation 01900000-0000-7000-8000-000000000000

The server durably creates or resolves the external maintenance receipt before reporting acceptance. It then stops ordinary admission, joins the local outbox, online retention, and background scrub owners before draining the coordinator, drains accepted work under a fixed deadline, closes the database, publishes the complete immutable backup, and performs the full startup validation again before returning to readiness.

Remote container disaster drill

The release Compose topology can exercise total database-volume loss through the same public maintenance API used by operators:

./scripts/remote-compose-acceptance --backup-restore

The drill deploys and seeds the release-owned four-subject workload corpus, creates and polls a backup from the separate operator container, creates a post-backup authority record, stops the database container, empties only its freshly allocated test data root, starts an empty replacement, and restores the backup through the public TLS endpoint. It then reruns the generated OpenFGA, MLflow, Better Auth, and Woodpecker observations, including a typed Better Auth user/session graph whose secret token is never placed in the observation or receipt. The retained Payload operational-query shape runs in the same corpus as a post-alpha regression and is labeled separately. The drill proves that the backed-up operator and application authorities work and that the post-backup authority disappeared. The operator container receives no database or backup mount and never supplies a server filesystem path.

The complete adapter gate also validates each domain’s immutable conformance manifest and installation/evolution plans before running that destructive corpus:

./scripts/adapter-disaster-recovery-acceptance --all-domains --remote

The drill writes a secret-free JSON receipt beneath target/adapter-disaster-recovery/. The receipt binds both maintenance operation identities and input hashes, the verified backup-manifest checksum, the destroyed post-backup suffix, the exact contract bundle, startup validation, the accepted four-subject inventory, the separately classified Payload regression, and the reconciled adapter-observation digest. The backup manifest is the frontier-bearing durable artifact; the receipt binds its checksum rather than decoding storage-format bytes in application or operator code. Recovery proves the declared lifecycle observation and does not promote any subject’s evidence classification. In particular, the MLflow and Woodpecker matrix results do not become upstream- framework conformance through backup and restore.

An existing named backup is never silently replaced.

Retire a Backup

Published immutable backups must not be deleted directly. Retire one exact checked name through the authorized public surface:

riffdb --config "$HOME/.config/riffdb/client.toml" \
  backup retire before-upgrade

Retirement requires the same global AdministerCapabilities authority as create and restore. The server resolves the exact succeeded create receipt and manifest; the client never submits a path. It then renames the complete four-file inventory into an operation-private maintenance directory, syncs both parents, deletes only those four known regular files, and records terminal success. Poll the returned operation ID with backup operation exactly as for create and restore.

A terminal retirement receipt is the only reason startup accepts a succeeded create receipt whose published artifact is absent. Missing artifacts without that exact receipt pair, symlinks, unknown files, manifest mismatches, and ambiguous staged state fail readiness closed. Retired names remain permanently consumed: neither create nor restore can silently reuse them.

Do not remove a backup directory or any file beneath .maintenance with filesystem tools. Routine retention should invoke backup retire after a newer backup has been fully validated. An interrupted retirement is recovered from its durable phase and exact create/manifest evidence on restart.

Restore a Backup

Restore to a nonempty target requires the exact destructive confirmation:

riffdb --config "$HOME/.config/riffdb/client.toml" \
  backup restore before-upgrade \
  --confirm-replace-current-database

The flag confirms destructive replacement only. It does not authenticate the caller, grant authority, satisfy an approval, or prove the backup is safe.

For a healthy current database, RiffDB authenticates and authorizes the caller before draining. It then validates the selected backup in private staging and freshly authenticates and authorizes the same bearer against the staged database. The current authorization result cannot satisfy staged authorization.

For an empty, unreadable, or corrupt target, the restricted recovery path validates the staged backup and authorizes only against that staged database. It does not claim a trustworthy current authorization state.

Ordinary self-contained restore verifies every immutable artifact checksum, refuses private engine repair, checks physical engine integrity, recovers the complete journal and required history, and validates bounded metadata roots, active catalog, bootstrap lifecycle and allocators before staged authorization. It then stamps and publishes the restored incarnation and performs a fresh, receipt-bound bounded startup before readiness. Success proves authorized materialization and bounded recoverability; it does not certify a complete semantic scrub of every entity, audit, outcome or other population row. First access still validates each record and fails closed on corruption. The ordinary background scrub runs after readiness and degrades Health on failure. Operators requiring complete semantic assurance can request the existing offline scrub. Supported legacy formats that need migration retain complete validation. Archive replay, follower bootstrap and promotion retain their complete proofs. On eligible native empty-target restores, temporary history reads borrow the original backup’s verified read-only sealed-extent descriptors. The backup’s complete manifest and artifact inventory remain pinned and checked; each read has an independent position and refuses changed identities, writes or links. The primary, active journal and marker are still copied and synchronized. Backups requiring an active-extent capacity conversion use complete copying and history hydration before validation. An interrupted private conversion cannot make the replacement ready or publish it; the original backup is unchanged. Complete retained-history validation and the receipt-authorized RestoreAnchor must remove every surviving source reference before temporary readers release. Original backup files remain unchanged, and publication contains exactly the three self-contained database artifacts. Publication rechecks every original backup artifact and independently reads back all three transferred artifacts. A Prepared or Published restart reacquires and fully validates the original backup; a StartupConsumed restart recovers the actual target and its later acknowledged writes. These checks do not grant readiness or replace startup validation. Complete startup and private offline validation inspect audit rows through the same bounded dispatcher: at most four workers, 64 rows and 64 MiB of copied input per batch, all on the original immutable validation snapshot. Every worker joins before the page returns; findings retain physical row order. There is no operator setting to suppress checks. Copying, hashing, physical integrity and journal recovery still scale with retained data; the sustainable TicketDesk recovery-time gate remains unqualified.

Closed backup inventory

New backups contain four fixed regular files:

database.redb
journal.riffextent
format.riffdb
manifest.riffdb

Retained journal history adds one regular file per sealed extent, named journal.sealed.<20-digit-generation>.riffextent. The manifest bounds this inventory and authenticates every sealed file in generation order. The four fixed files remain required even when there are no sealed extents.

The manifest authenticates the redb checkpoint, the complete fixed-size durability-journal extent, and the exact pre-open format marker as one unit. It also declares the inclusive binary format range allowed to restore the physical artifacts. Restore checks that range before discovering, creating, locking, staging, or replacing the target. It then checks every artifact checksum, the journal database identity and selected generation, an empty suffix at the manifest checkpoint, and the marker identity/fixture binding. Each retained sealed extent must also pass complete frame and footer validation. An oversized footer is corruption: restore rejects it before allocating the footer contents or publishing the target. The marker is published last, so startup cannot mistake a partially published restore for current data.

Unknown files, an absent current artifact, a legacy manifest without a physical range, stale or mixed identities, and checksum damage fail closed. Historical two- and three-file backups remain decodable for inspection and recovery with their source release; the current binary does not infer physical compatibility from successful decoding. Restore a legacy artifact with its source release before following docs/compatibility.md’s declared upgrade edge.

Extent creation reserves and physically zero-fills its fixed capacity before the database becomes ready. Insufficient space therefore fails during startup, backup staging, or restore staging—not after a command has been accepted.

The journal format is local recovery media. Replication and changelog records continue to derive from published durable frontiers and do not copy journal bytes.

Interruption and Recovery

Receipts live beneath:

<backup-root>/.maintenance/

This subtree is reserved and cannot be selected by a backup name. A receipt is versioned, checksummed, bounded, atomically replaced, and parent-directory synchronized. It is durable operational state, not database metadata or application history, and it is not included in backup artifacts.

The external ledger also recognizes archive-only receipt V3. It preserves the verified archive selection across retries, shares the existing maintenance ownership and inventory limit, and retains an admitted private replay stage for archive recovery. Private replay can stop at the receipt’s exact selected manifest after the archive advances; an explicitly empty selected suffix remains empty. Ordinary create/restore V1 and retire V2 receipt bytes are unchanged. Restart routes an unfinished V3 receipt through its archive-specific recovery checks. Recognizing a receipt does not validate or publish its replay stage. For an interior command stop, storage can rebuild a separate private artifact from the verified backup and checked command-prefix evidence after full original replay. It retains the original archive selection separately from its actual application/audit frontier. Ordinary source and follower opens refuse this artifact. A separate validation owner now checks its exact construction bytes, full structural/catalog evidence, reciprocal command graph and actual stopped frontier. It can release one pinned snapshot for staged authentication and current policy without local writes. Once every authorization reader closes, storage can seal the validated cut and publish it through the matching durable Offline V3 receipt. Its recorded incarnation authorizes a fresh RestoreAnchor at the actual stopped frontier; only then is the source journal created and the normal format marker restored. Interrupted private stages rebuild from the receipt’s original selection. One storage preparation owner now covers an empty suffix, the exact backup fence, earlier complete receipts and interior command cuts. It first validates the entire selected original suffix, then rebuilds an earlier stop from the same backup and selection. An explicit sequence stops at its first exact boundary; last-archived retains the selected terminal administration frontier. Both paths provide a validated immutable snapshot for staged authorization and use the same receipt-authorized publication owner. The internal driver now drains an admitted V3 receipt, resolves its existing owned archive from operator configuration, and persists its verified selection before replay, authorizes the validated immutable stage, and records the actual frontier and incarnation before publication. Success requires fresh source startup validation and complete authoritative state equality with the retained stage. Before publication is durably recorded, credential-free resumption requires exact staged/target bytes and the receipt-bound RestoreAnchor, marker and empty journal. After publication is recorded, recovery permits engine bookkeeping changes from interrupted startup, but requires the same lineage, RestoreAnchor and dual frontier, permits only empty DirtyActivation receipts from those startups, and compares every authoritative row against the retained published stage before success. It cannot rebuild an unpublished stage without credentials. The storage owner can publish a separately validated and sealed replay only against matching durable Offline V3 evidence and its recorded incarnation. It creates an empty source journal at the actual restored application/administration frontier, validates the new source completely, and then uses the existing replacement and parent-sync boundaries. Pre-receipt cleanup cannot delete an admitted V3 stage. The shared staged-authorization check uses one immutable restored-state snapshot; it can check replayed grants, expiry, approvals and revocations without local follower authority writes.

Archive restore integration status

A ready source server now accepts the distinct RestoreArchivedBackup RPC and Rust-client RestoreArchivedBackup submission. The CLI uses the same service:

riffdb --config "$HOME/.config/riffdb/client.toml" \
  storage restore before-upgrade --archive daily \
  --stop-at-sequence 42 --confirm-replace-current-database

The archive name resolves through server configuration. Omitting --stop-at-sequence selects the last archived frontier when the server first verifies the archive. An explicit sequence must be positive and fall within the verified backup and selected suffix. The operation freezes that selection before replay, and a retry retains the operation ID, archive name, stop choice and replacement confirmation. The client never falls back to an ordinary restore when the archive RPC is unavailable.

Accepted and uncertain output contains the maintenance operation ID. Poll it using backup operation; rerunning storage restore generates a new ID. Archive status includes archive_restore.archive_name, the requested stop, and, once resolved, backup_application_frontier and restored_frontier. The latter has independent application and administration positions. An absent frontier means unresolved; before_first means a known empty history. Ordinary maintenance output is unchanged.

For an interrupted archive restore admitted against a readable current database, restart accepts only the same archive restore operation ID and exact input. That retry authenticates against the current database and rechecks current restore policy before resuming. Ordinary commands, new maintenance operations, and receipt observation remain unavailable on this restricted retry host. The retained bearer must also pass staged authentication and policy before any unpublished target replacement. A different archive, stop, confirmation, or ordinary restore request cannot resume the receipt.

After complete publication, restart can recover without a bearer or archive access. It must verify the receipt-bound stage, exact restored target and full restored authority before readiness. A partially published target does not gain this credential-free authority.

When the current database is unavailable, the restricted recovery host accepts archive restore through the same distinct RPC. It verifies the full backup and selected archive prefix, reconstructs and validates the private candidate, then authenticates the bearer and checks restore policy against that restored state. A new recovery attempt persists its exact selection and admission in V3 only after those checks pass. Denial before admission leaves no receipt or target replacement. Recovery offers no ordinary application operations, backup creation, or receipt polling.

An interrupted restore with an unreadable target uses the same staged checks and its original receipt selection. It cannot choose a newer archive head or change a recorded incarnation. Complete publication evidence permits validation without a bearer or archive access; incomplete publication requires the retained backup and selected archive prefix plus fresh staged authorization. Receipt editing is unsupported.

Archive replay can advance projection controls beyond the local projection rows copied from the full backup. Before validating a replayed candidate, restore uses the same catalog-validated projection rebuild as follower startup. It reconstructs only derived rows and apply markers, keeps source controls and history unchanged, and then runs the complete structural and catalog scrub. An earlier stop repeats this process at its selected boundary; an interior command cut rebuilds its physical predecessor before reconstructing the private command prefix.

Automatic collection is enabled by the optional backup name in a configured archive entry; see Named archive directories. The collector starts from the verified backup fence or the last confirmed archive frame, validates retained history, and writes outside command acknowledgement. Collection failures stop the collector and leave primary writes available; monitor the redacted riffdb-archive-v1 diagnostics. A stopped collector needs a database restart after the cause is corrected. It cannot bridge a history gap already pruned from the source. The live sink-loss acceptance case withdraws the archive directory after confirmed collection, checks that primary writes still succeed after the collector stops, then restores only the confirmed prefix after the directory is returned. This proves write availability and prefix recovery for that failure; acknowledgement latency is measured separately.

Filesystem archive files are immutable. The collector may publish CURRENT as a checked hard link to the durable manifest, retaining the same selector bytes and atomic rename; existing independent selector files remain readable. Do not edit archive artifacts in place.

The Linux acceptance runner ./scripts/check-archive-cli builds the production CLI and runs both stop modes against a real daemon with verified TLS and automatic collection. It checks the generated operation ID, terminal JSON frontiers, restored application data and exact retry through the Rust client. These two process tests are explicitly ignored by ordinary Cargo test runs because they require the separate CLI binary; ci-all runs the dedicated runner.

The storage recovery matrix covers an archive sink refusing a frame and losing confirmation after persisting it. In both Standard and Hardened profiles, the source commits further commands before a process exit; repeated source recovery retains them, while validated archive replay includes only the durable archive prefix. The collector never reports its uncertain frame as confirmed.

Full CI at 71ae34a7 and the later sink-failure matrix checks passed; the repository bundle at release/evidence/wp-749/verification-71ae34a7/README.md records their exact scope. The later combined implementation also passed full integration CI. The same-binary archive comparison completed twenty valid cells and found measurable archive overhead: write-only median throughput fell 8.49% and request P95 rose 4.35%. The later isolated diagnostic measured acknowledgement P99 at 88.35 ms off and 114.93 ms on. Checkpoint-coupled journal durability is the named dominant mechanism, present without archive; roughly 11.7 ms remained unattributed. The checkpoint-coupled journal durability mitigation is complete, with its remaining shared tail accepted. After bounding cursor tail walks, single-pass frame digest computation, and deferring archive disk writes during background spare-extent zero-filling, the post-mitigation direct pair passes host, correctness, and acknowledgement-latency qualification (+0.00% report P50/P95, raw P50 +1.46%, raw P95 +0.94%, raw P99 +0.35%, throughput -1.17%, and zero terminal archive lag). The full replication qualification campaign remains separate, so multi-node replication paths still require their own campaign before production disaster-recovery deployment.

An SDK or direct API caller supplies the maintenance operation ID. After a lost response or process interruption, that caller retries the exact same start request with the same operation ID. It must not choose a new ID merely because the prior response was uncertain.

The CLI start commands generate an operation ID internally and cannot reissue a start request with a caller-selected ID. An accepted or uncertain CLI result includes the generated ID. Resolve it by polling:

riffdb --config "$HOME/.config/riffdb/client.toml" \
  backup operation 01900000-0000-7000-8000-000000000000

Do not rerun backup create, backup restore, or backup retire to resolve CLI uncertainty; that would generate a different operation identity. Startup validates all receipts and reconciles incomplete work only from exact receipt, manifest, inventory, and checksum evidence. It does not infer success from a partial directory or an ambiguous filesystem result.

Unknown receipt versions, invalid checksums, impossible phase transitions, or receipt/artifact mismatches fail readiness closed. Operator repair of receipt bytes is unsupported.

Restore Rewinds History

The POC preserves the DatabaseId stored in the backup. Destructive restore still rewinds sequence allocators and can reuse destroyed sequence suffixes.

A durable history_incarnation fence now makes that rewind detectable. The incarnation is a retained-metadata value bootstrapped to 1 and bumped only on destructive restore. Backups carry the value when present; responses that expose commit-sequence-derived positions include the current incarnation. Sequence-anchored requests may send optional observed_history_incarnation; when present and different from the current value, the server rejects with RDB-HISTORY-0101 before any storage work.

Detection residual risk: clients that omit observed_history_incarnation keep working unvalidated and can still silently bind stale observations. Adopt the field for any long-lived cursor, subscription, or sequence-derived assumption.

A backup may physically contain the source database’s private clean-close lifecycle record, but restore never treats that record as permission to skip validation. Destructive restore changes the history incarnation; the copied lifecycle is therefore stale and is consumed or replaced as dirty before the restored database can activate writers. A later graceful shutdown may produce a new clean record bound to the restored incarnation. Copying backup artifacts or a database file cannot forge clean eligibility.

Ordinary backup restore and staged-only backup recovery verify the copied artifacts before the first private engine open and retain its native exclusion through history hydration, complete physical verification and bounded readiness, the existing storage/catalog/retained-metadata/bootstrap/allocator joins, and fresh staged bearer authentication. Required native compatibility preparation uses the existing transformations; formats requiring migration retain complete validation afterward. Bounded readiness creates no complete semantic-scrub proof, checkpoint, registry-completion witness or clean certificate. No operational source writer or readiness capability is activated for that authorization read. Every immutable reader must close before receipt-authorized stamping. The ordinary maintenance controller retains that unique private engine through the stamp and then closes it once. Its live preparation state cannot publish. Direct closed-stage and archive workflows retain their existing close protocols.

The receipt-authorized incarnation stamp verifies the complete primary, journal and marker content seals under the same native custody and uses the existing immediate two-phase metadata transaction. Its complete retained-history evidence is bound to the same private engine and native mutation generation; stamping checks transaction-current roots before consuming that evidence. Equal-watermark retries reuse the same complete evidence while preserving every restored fencing and tombstone check. A real watermark change discards it and requires fresh physical and complete history validation. This grants no complete semantic-scrub assurance or journal replay permission.

For current ordinary receipt-authorized restore, command values that survive publication are hydrated first. Predecessor HISTORY receipts can remain in their original verified copied extents through complete validation and authorization. The canonical RestoreAnchor removes those references atomically. After checking that no command or receipt stub survives and observing engine close, the owner retires the original extents and verifies the renewed empty journal above the configured target’s extent generation. Original descriptors, content seals and file/inventory continuity remain checked until retirement. Registry completion alone cannot admit deferred history: all three native command locator tables must already exist. Images requiring table preparation retain the complete path; unsupported images refuse before authorization or publication. The owner then binds a fresh full seal of the three publication artifacts. Publication retains immutable-backup revalidation, target exclusion, full artifact checks, marker-last rename and fresh post-publication startup. On Linux, an eligible self-contained ordinary restore into an empty target on the same filesystem transfers the three original sealed files instead of copying them again. A bounded private publication intent records their original identities and seals before any move. Full readback of all three transferred files precedes final publication. Other restore states retain verified copying; an interrupted transfer cannot switch to copying or silently rebuild the target from the backup.

During eligible ordinary materialization, the retained original-backup owner performs the complete content check before first use. At native publication, that same owner revalidates all original artifacts, including sealed history. Bounded manifest-binding reads and continuity checks do not replace either full boundary check. Without the actual retained owner, the complete original-backup fallback check remains mandatory. Private artifact seals and independent post-transfer readback are unchanged.

For an eligible fresh publication in the same process, the actual verified immutable-backup and three-artifact owners now remain alive through startup consumption. Original descriptors, full source seals and completed independent transfer readback remain bound; exact file continuity and closed receipt/intent inventories are checked again before consumption. The one bounded handoff admits one consumer. A failed or abandoned retained owner cannot be replaced by a cold owner within that handoff. An actual process restart reconstructs custody and performs the complete cold checks.

Startup durably consumes that intent before opening the published native engine. If startup is interrupted after consumption, recovery uses the actual published database, performs ordinary native engine crash recovery when needed, and completes current structural, journal and catalog validation before success. It cannot use an obsolete pre-startup primary hash or a clean/checkpoint shortcut to admit changed content. Private stages and a freshly transferred target still refuse engine repair. The server keeps application admission fenced until the ordinary Succeeded receipt is durable and matching intent removal and directory synchronization finish. A crash after success can finish cleanup and normal startup, preserving subsequent writes. A failed ownership or validation check preserves the immutable backup and actual failure evidence. Existing timing gates and resource limits remain unchanged.

Private archive conversion, command-prefix reconstruction, archive validation, promoted-backup validation and archive incarnation stamping also abort engine repair requests. Private archive replay startup and projection preparation use the same refusal policy; published follower crash recovery remains available. A repaired private stage cannot gain publication authority from a subsequent successful integrity check. Before incarnation stamping, the sealed owner also rechecks its primary, journal, and format marker checksums; intended metadata writes cannot replace the seal of a changed artifact. Stamping independently refuses engine repair. Refusal leaves the immutable backup intact, and a retry rebuilds the unpublished stage from that backup.

The private stage retains the native identities of its primary, journal, and format marker through authorization, sealing, stamping, and publication. Replacing any of these files with an identical copy still refuses the operation; matching checksums alone cannot establish continuity. Archive recovery tracks its intentional removal of the empty source journal, receipt-authorized creation of the new RestoreAnchor journal, and rename of the retained marker explicitly. Ordinary and archived backup restore use their owned private stages directly for complete verification. Archive conversion, replay, projection rebuilding and an interior command-prefix reconstruction retain the original native owner. Their final immutable proof includes the complete physical, structural and catalog checks; a private cut additionally proves its exact reconstruction binding and reciprocal command graph. The server runs the ordinary startup joins over that same proof before fresh bearer authentication. All authorization readers must close before sealing. Journal absence and the original marker are sealed alongside the primary; only the receipt-authorized journal creation and marker restoration retire those bindings. The original incarnation transaction is followed by a fresh complete source proof under continuous custody before publication. A witnessed failure permanently invalidates that private candidate. The complete rehearsal remains unqualified. Its saved-population primary-loss restore gate passes; the scheduled-population 45-second release, final-source CI and both measured campaigns remain outstanding. Backup preparation precedes deliberate primary loss and remains separately visible, including its service interruption and affected offered-work windows. Recovery retains one 60-second deadline from primary loss through the first acknowledged write, exact replay and full state reconciliation. Old combined-clock failures cannot be reclassified. After the backup becomes durable, the campaign can continue reads while keeping writes fenced at its included acknowledgement prefix. It pauses and drains those readers after primary loss, charging that work to the same recovery deadline. Original offers, queue and in-flight bounds, preparation interruption and read progress remain part of the evidence; this adds no post-backup write-loss allowance.

The immutable backup artifact preserves the source bytes, including a matching certificate. Private restore staging nevertheless drives the complete structural and catalog-semantic streams through exact end before staged authentication; it never uses those bytes to skip restore validation. Ordinary backup restore consumes the complete native proof for authorization without an operational startup open; copied clean or checkpoint state cannot replace that proof. Eligibility is a property of the complete stopped database, journal, format marker, maintenance state, and retained roots, not of certificate bytes alone. A complete stopped lifecycle-unit copy preserves eligibility; a bare database-file copy does not and takes complete validation. Registry migration, an incompatible internal format, maintenance replacement, retention hold or prune, and history-incarnation rotation likewise invalidate or fail closed. A backup-bound compatible marker-only format upgrade preserves an otherwise unchanged lifecycle unit. An offline integrity scrub deliberately preserves the bytes but does not consume, refresh, or replace them.

A destructive restore removes every observation created after the restored frontier: the full backup’s included frontier for ordinary restore, or the verified selected frontier for archive-suffix restore. The removed application and administration sequence suffixes may later be reused for different records. An idempotency key that existed only in the removed suffix may be accepted again.

After restore, discard all pre-restore:

• outcome, commit, provenance, event, outbox, projection, and audit locators beyond the restored frontier;
• idempotency assumptions about the removed suffix;
• pagination cursors, subscriptions, sessions, and process generations;
• read-after-sequence expectations beyond the restored frontier; and
• capability and authorization facts learned from the removed suffix.

Preserving DatabaseId does not make those observations valid. Archive-suffix restore supports recovery at an exact commit sequence within its verified coverage, and RedbVerifiedArchiveBackup::resolve_archive_stop_at_timestamp resolves a wall-clock target timestamp (Timestamp) against a verified baseline backup and bound RedbArchiveRepository into a deterministic ArchivePitrResolutionV1 (stop_at_sequence when an archive prefix applies, or None when the target falls on the baseline backup fence and no archived commit is <= target). Timestamp resolution refuses pre-fence targets, pre-first-commit targets, and non-monotonic commit timestamps across the target boundary fail closed (ArchivePitrResolutionError). Each confirmed archive frame also exposes an ArchiveFrameSummaryV1 (RedbArchiveRepository::frame_summaries and RedbArchiveRepository::set_confirmed_frame_observer) with its monotonic sequence and commit-timestamp range and SHA-256 digest for external object-store replication hooks. Globally stable pre-restore locators, online backup, built-in backup encryption, and a built-in remote object-store sink remain outside the POC. The archive collector currently uses an operator-configured filesystem directory.

Promotion recovery evidence

Keep the private <backup-root>/.maintenance/replication_promotion ledger with maintenance receipts and the configured database. Terminal failure retains any frozen selection and refuses ordinary startup until an exact successful retry has been completely reconciled under the current owner. Deleting this evidence is not a recovery procedure. A V2 source always receives complete validation, even after CLEAN; V1 clean-start behavior is unchanged. The experimental follower administration path describes restricted pre-cutover retries and the required fresh source fence proof.

Private restore materialization retains the original primary, journal and format-marker file identities before their first engine open. A same-byte replacement of one of those files or its parent directory refuses the private open. This identity check does not substitute for complete integrity validation.

Archive image validation preserves the immutable backup’s exact history fence. It performs complete physical, journal and structural/catalog verification without appending ordinary source compatibility receipts before selected replay. Ordinary backup restore retains its required source compatibility transformations. The final receipt-stamped source proof closes without granting a second staged authorization or source activation; fresh ordinary post-publication startup still owns its existing compatibility and readiness checks.

Offline storage compaction

After offline or online retention pruning deletes historical commit, event, and outbox rows, the freed pages remain inside the redb file for internal reuse. To compact those freed pages and lower the file’s on-disk high-water mark while riffdbd is stopped, run:

riffdb storage compact --database-path /var/lib/riffdb/database.redb

Compaction verifies the database format, journal rebase, retained V3 history, and tombstone chain before and after running redb::Database::compact(), compacts the derived sidecar when present, and reports database_id, bytes_before, bytes_after, reclaimed_bytes, compacted, retention_watermark_sequence, and history_incarnation.

Symbolic Application Export

RiffDB application export is an operator-authorized portability surface. It reads one immutable published snapshot and releases bounded canonical JSONL pages containing symbolic contract names and natural typed values. It never exports redb keys, numeric schema IDs, durable envelopes, journal/changelog bytes, credentials, or hidden schema.

Export is not backup. Use Backup and Restore for an exact disaster-recovery copy within its declared durable-format range. Use application export when data must cross an incompatible alpha format or leave RiffDB through a portable application representation.

Authority and scopes

An export requires an explicit Capability V5 grant for the selected contract lineage and record classes. Entity or event read permission, capability administration, backup authority, and application query roles do not imply export authority.

The two scopes are:

• principal: applies the current application role’s row policy and field visibility before serialization, counting, cursor advancement, and hashing;
• whole: requires a distinct whole-application grant with global tenant and all-partition scope.

Provenance and public audit each require their own grant bit. V5 deliberately does not permit audit-only or provenance-only export: at least entities or events must also be selected.

Every start, page, status, and cancel operation reloads the exact current capability. Revision, revocation, role, field-visibility, or row-policy changes close an in-progress export rather than continuing under stale authority.

Start an immutable export

The CLI generates a UUIDv7 operation identity unless --operation-id is provided. Supplying a stable operation identity makes an uncertain start safe to retry.

riffdb --database ticketdesk export start \
  --lineage TicketDesk \
  --scope whole \
  --entities \
  --events \
  --provenance \
  --lease-seconds 3600 \
  --output json

The response contains the operation identity, exact snapshot binding, and an opaque base64 cursor. The cursor is bound to the operation, snapshot, principal, scope, and export format. Do not decode or edit it.

For a future reimport, start with portability intent and bind the exact adapter-owned manifest while the source database is still available:

riffdb --database ticketdesk export start \
  --lineage TicketDesk \
  --scope whole \
  --entities \
  --portability-manifest riffdb/portability-manifest-v3.json \
  --lease-seconds 3600 \
  --output json

The manifest is strictly decoded, checked against the pinned contract bundle, and retained by hash as part of the operation’s immutable retry identity. An existing general export cannot be replayed with this option or upgraded into portability authority later.

Write bounded JSONL pages

Pass the returned cursor to export page and select a new local output file:

riffdb --database ticketdesk export page \
  --operation-id 018f2f85-3c20-7a31-8f11-112233445566 \
  --cursor 'OPAQUE_BASE64_CURSOR' \
  --max-rows 500 \
  --jsonl export/entities-0001.jsonl \
  --output json

The destination must not already exist. RiffDB creates it with mode 0600, writes one canonical JSON object per line, flushes it, and reports the page hash, class completion, operation completion, and next cursor. Use a distinct file for every page. A response lost after the server advances its durable checkpoint can be retried with the same cursor during the lease; the server replays only the exact page bound to that durable successor.

Pages are bounded to 500 rows, four MiB of canonical JSON, and the operation’s server time budget. The client never accumulates the complete export in memory.

An operation retains at most 4,096 pages within a shared 256 KiB durable-state budget. New operations retain a compact checkpoint and one append-only commitment per page, committed atomically. Space for the final manifest and receipt is reserved before a page is released, so the byte budget can stop an export before the page ceiling. Terminalization verifies the complete retained prefix once; public JSONL, page hashes, manifest order and receipt bytes are unchanged. Existing operations keep their original checkpoint and cursor encoding.

The ledger adds an authoritative storage namespace for export and re-import checkpoints. The release manifest and normal format preflight bind its exact registry/catalog inventory. A database from a different current-build manifest is refused before mutation; use its matching binary. No automatic conversion of an active V3 history or physical downgrade is introduced. Explicitly recognized pre-V3 offline upgrades retain legacy export records and install an empty ledger. Keep the required backup and follow format compatibility before changing binaries.

Observe or cancel

riffdb --database ticketdesk export status \
  --operation-id 018f2f85-3c20-7a31-8f11-112233445566

riffdb --database ticketdesk export cancel \
  --operation-id 018f2f85-3c20-7a31-8f11-112233445566

A completed operation includes a canonical manifest and receipt binding the database/history identity, contract and module identities, row-policy scope, snapshot frontiers, per-class and total counts and bytes, ordered page hashes, omissions, capability identity/revision, and terminal state. Cancellation, expiry, authority change, source failure, and process crash produce or retain an inspectable incomplete operation; they are never reported as complete.

An immutable redb read transaction is process-local. After a process crash, the durable checkpoint remains observable, but its old snapshot cannot be continued. RiffDB returns a typed snapshot-unavailable/restart outcome rather than silently selecting a newer snapshot. Start a new operation and retain the older incomplete receipt.

A portability-intent export additionally inspects every selected workflow row at the same snapshot. Lease owner and expiry must both be null. Each workflow’s checked and quiescent row counts are sealed into the v2 export manifest; the v2 receipt binds the exact portability-manifest hash. An active lease or a partially cleared owner/expiry pair closes the export as workflow_not_quiescent before a completed portability receipt exists. Release or expire the work at the source and start a new export. A general export does not perform or claim this proof.

Reimport boundary

JSONL output is not accepted by a generic insert or transaction API. An application’s conformance manifest must map portable record classes to operator-only reimport command declarations or an accepted migration plan. Reimport therefore re-applies current types, invariants, references, row policies, provenance, declared outcomes, and server-derived command idempotency. Commit sequences and physical identities are not portable identities.

The adapter-owned riffdb.application-portability-manifest/v3 binds the exact adapter manifest, contract lineage, contract version, bundle hash, and query module used by each reconciliation observation. Observation parameters are canonical typed scalar values in symbolic-name order; field IDs, nested records, vectors, and caller-supplied query source are not representable. Each portable entity or event symbol selects only one of these closed strategies:

• a named compiler-owned reimport command whose sole input is a bounded list of the exact exported entity record; or
• one exact migration hash already accepted by the adapter’s evolution manifest.

There is no callback, method path, table name, field ID, transaction handle, or generic entity writer. The manifest has no idempotency-input or field-mapping authority: the server derives identity from the exact export and portability manifests plus the stable entity key. Reconciliation runs declared, bounded named queries and publishes a canonical riffdb.application-reimport-receipt/v2 only when every mapping outcome and observation digest agrees with the portability manifest. The receipt records the new database identity and deliberately does not claim preservation of physical commit sequences.

Frozen v1 and v2 portability manifests and receipts remain readable for compatibility inspection, but neither older format can be emitted as v3 authority and a v1 application-command mapping cannot be compiled as a modern reimport command. OpenFGA, MLflow, Better Auth, and Woodpecker compiler fixtures prove the closed mapping boundary. The MLflow and Woodpecker JSONL fixtures additionally carry noninitial workflow state, nonzero fencing tokens and attempts, and null owner/expiry fields; the same parser used by the service acceptance tests rejects noncanonical or type-inexact variants. Portability- intent export and its source-side quiescence proof are available through the public start/page/status/cancel export campaign. A distinct operator-only destination campaign accepts that exact completed source through reimport start, reimport page, reimport status, and reimport cancel. It remains unavailable to application roles and MCP discovery.

Each general-alpha subject retains its existing adapter evidence classification through this portability evidence; an export result does not turn a matrix or partial profile into upstream-framework conformance. The earlier Payload fixture remains a post-alpha portability regression and is not a general-alpha record.

Reimport an exact completed source

Reimport targets a new, empty database that has not been published ready. The credential must carry one Capability V7 reimport grant bound to the campaign ID, lineage, portability-manifest hash, scope, and compiled application-role identity. Export authority, installation authority, and an ordinary application role do not imply this grant.

Start with all three terminal source documents. Reuse --campaign-id for every uncertain retry:

riffdb --database restored --credential-file operator-reimport.credential \
  reimport start \
  --campaign-id 019f... \
  --lineage TicketDesk \
  --scope whole \
  --portability-manifest riffdb/portability-manifest-v3.json \
  --export-manifest export/manifest.json \
  --export-receipt export/receipt.json

Apply each entity page in its original order. The page arguments come from the corresponding export page result and must not be recomputed or edited:

riffdb --database restored --credential-file operator-reimport.credential \
  reimport page \
  --campaign-id 019f... \
  --export-operation-id 019e... \
  --page-number 1 \
  --jsonl export/entities-0001.jsonl \
  --page-hash 7d6f... \
  --next-cursor RF...

For a terminal page, omit --next-cursor and pass both --class-complete --operation-complete. RiffDB verifies the domain-separated page hash, exact expected page position, source manifest, current authority, and every canonical row before compiler-owned commands can mutate the empty destination. A repeated page is an exact replay; a different page at the same position fails closed. The final page is also the recovery entry point if its commands committed before reconciliation completed: RiffDB replays only commands not covered by the durable page checkpoint, resumes the exact module-bound observations, and returns the already sealed receipt after an uncertain terminal response.

Observe or cancel with the same campaign identity:

riffdb --database restored reimport status --campaign-id 019f...
riffdb --database restored reimport cancel --campaign-id 019f...

Cancellation and partial failure never publish the destination ready. After all pages, the coordinator performs the portability manifest’s bounded observations and publishes a canonical reimport receipt only when every mapping and observation reconciles. The installation campaign may advance to credentials only after that receipt is retained. There is no CLI verb that accepts raw JSONL without its export operation, page number, page hash, completion flags, and cursor evidence.

Application Installation Campaigns

RiffDB installation campaigns retain one caller-stable campaign identity and one immutable, compiler-checked application plan. They are an operator surface, not an application credential, generated-client, or MCP capability.

The campaign is deliberately not described as one transaction. Contract and module publication, offline migration, role reconciliation, credential rotation, driver proof, and ordinary command seeds retain their own durable boundaries. An interrupted campaign reports the exact completed stages and the next safe action; only a sealed receipt is installation success.

Start or resume

The plan must be canonical installation-plan bytes and the campaign ID must be a caller-retained UUIDv7. Ordinary installations continue to use riffdb.application-installation-plan/v1. A portability reimport uses v2, which additionally binds the exact export manifest, completed export receipt, and portability manifest hashes. A role selecting a named query with a compiler-declared secret output uses v3. V3 carries optional reimport identity and complete canonical role authority as closed Operation { kind, name } and QuerySecretOutput { query, entity, field } atoms.

The compiler remains the least-sufficient writer: secret-free plans retain their frozen v1 or v2 bytes. An installer that does not understand v3 refuses before remote mutation. For an existing role, every desired atom absent from the observed role is widening and requires one approval bound to the exact predecessor role hash and complete ordered additions. This includes adding a secret output while its query operation is unchanged, or while another query already reveals the same field. Initial roles show their complete authority in the plan; starting that exact plan is the explicit confirmation, and the terminal receipt binds its plan hash and installed role hashes.

The atom sets contain names only—never field IDs, values, query source, capability bytes, or bearer credentials. Each role is limited to 1,024 operations, 1,024 secret outputs, and 2,048 atoms total. Missing, excess, reordered, stale, or renamed approval data fails before capability creation.

Start or resume the exact plan with:

riffdb --config operator.toml application install \
  --plan installation-plan.json \
  --campaign-id 018f2f85-3c20-7a31-8f11-112233445566

The CLI validates the complete plan locally before making a connection. A transport retry changes only the outer request ID; the campaign ID and plan bytes remain exact. Reusing the campaign ID with another plan or lineage fails closed. If the outcome remains uncertain, the CLI returns the campaign ID and the recovery action observe_application_installation.

Observe retained progress without resubmitting local plan bytes:

riffdb --config operator.toml application installation \
  --campaign-id 018f2f85-3c20-7a31-8f11-112233445566

After every plan-declared driver has performed its exact generated-client or driver-host identity handshake, resume the current driver_proof stage with the complete plan-ordered set:

riffdb --config operator.toml application install \
  --plan installation-plan.json \
  --campaign-id 018f2f85-3c20-7a31-8f11-112233445566 \
  --driver-proof rust \
  --driver-proof typescript

This flag is an explicit installation-authority attestation. It does not run the driver and must not be supplied before the controller has checked the driver’s deployed identity. A subset, duplicate, reordered value, undeclared driver, or proof submitted at another stage fails closed.

After nonempty seed batches have run through ordinary compiled commands, the controller can submit one canonical receipt document:

{"schema":"riffdb.application-installation-seed-receipts/v1","plan_hash":"0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef","seeds":[{"name":"initial-data","content_hash":"abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789","succeeded":5,"replayed":2}]}

The document is compact canonical JSON followed by one newline. plan_hash must identify the submitted installation plan; seed rows must have the exact plan order, names, lowercase content hashes, and item totals. It contains no command inputs, idempotency keys, credentials, or host paths. Submit it only at the current seeds stage:

riffdb --config operator.toml application install \
  --plan installation-plan.json \
  --campaign-id 018f2f85-3c20-7a31-8f11-112233445566 \
  --seed-receipts seed-receipts.json

--driver-proof and --seed-receipts are mutually exclusive because one resume request can complete only the one exact current external stage.

Machine output names only symbolic stages, the plan hash, lineage, phase, next action, typed safe failure, and a redacted terminal receipt. It never returns bearer credentials, host paths, seed values, numeric schema IDs, or durable storage encodings.

Execute the reviewed deployment

application deploy can bind its existing exact deployment executor to the same immutable plan and campaign:

riffdb --config operator.toml application deploy \
  --installation-plan installation-plan.json \
  --installation-campaign-id 018f2f85-3c20-7a31-8f11-112233445566 \
  --provision-role AppRole

Both flags are required together. Before opening a transport, the CLI strictly decodes the source, exact lock, generated artifacts, manifest, query/reactive modules, selected role, seed inputs, target database/environment/lineage, and plan. Any mismatch fails locally and makes no remote application mutation. The campaign is resumed immediately before deployment so conflicting remote state also stops the invocation before the existing deploy executor runs.

Campaign stage order is preflight, contract, migration, query modules, reactive modules, roles, reimport, credentials, driver proof, seeds, and the terminal receipt. A v1 plan records reimport_not_required at the reimport stage. A v2 or v3 reimport plan stops there until the server’s reimport coordinator has verified the exact plan-bound source documents and published a completed reimport receipt. Caller-supplied stage evidence cannot skip that boundary, and credentials are not published while reimport is incomplete.

The reviewed successor capability ID comes from the plan’s credential destination. The deployer never substitutes a freshly generated capability ID for a plan-bound install or rotation. Existing local deployment state must be empty, already name that successor, or name the plan’s exact predecessor with explicit --replace-role-credential.

Driver proof remains an external observation and therefore splits a seeded installation into honest phases:

1. Run plan-bound application deploy without --seed.
2. Exercise every declared generated driver, then submit the complete --driver-proof set with application install.
3. Resume the same plan-bound deploy with --seed. Each seed remains an ordinary resumable command batch; the CLI submits only its value-free terminal counters to the campaign.

The final deploy output includes installation_campaign_id, installation_plan_hash, installation_phase, and installation_next_action. A running campaign is never labeled installed. Only the server-sealed receipt yields installation_phase: installed.

The local artifact inventory uses fixed symbolic singleton names manifest, contract, rust, typescript, go, python, mcp, and migration; query and reactive artifacts use their declared module names. Manifest seed inputs use seed-001, seed-002, and so on in manifest order, with the domain-separated content hash and exact nonempty command count. Host paths never enter the plan.

Durable and authorization behavior

• Campaign state lives inside the selected RiffDB database in a versioned, startup-validated durable record. A separate local journal is not authority.
• Start/resume requires the dedicated installation permission for the exact database, environment, and application lineage. Deploy, migration, or capability-administration permission alone does not imply it.
• Observation first resolves the retained lineage without disclosing campaign contents, then authorizes that exact lineage before returning progress.
• The same campaign and plan are idempotent. Compare-and-swap disagreement is outcome uncertainty, not permission to overwrite another controller’s progress.
• Partial or failed stages never carry a terminal receipt. installed requires every closed stage plus the receipt stage in dependency order.

Newly sealed ordinary-install terminal receipts use riffdb.application-installation-receipt/v2. The canonical, content-addressed document includes the exact nonsecret capability ID for every credential destination, value-free per-seed succeeded/replayed checkpoints, the adapter manifest digest, the installed terminal state, and exact migration and backup receipt references when a migration was required. A reimport installation uses v3 and additionally names the exact export manifest, completed export receipt, portability manifest, and destination reimport-reconciliation receipt. It never includes bearer credentials, seed values, host paths, or hidden schema. Accepted v1 and v2 receipts and v1 campaign states remain readable; resuming one preserves its original receipt identity instead of silently rotating it. New campaign checkpoints use v2 so the reimport stage is represented explicitly; decoding a retained v1 campaign inserts only the closed reimport_not_required evidence appropriate to its v1 plan.

Adapter conformance manifests

An adapter may bind a canonical adapter-conformance document to the installation plan. V1 remains readable as the initial compatibility format. Current V2 (riffdb.adapter-conformance-manifest/v2) requires an explicit required, optional, degraded, or unavailable disposition for every closed alpha feature, preventing omission from becoming a silent fallback. The document names one exact application manifest and lock, contract lineage, generated artifacts, symbolic roles and operations, first-party driver/runtime and platform identities, bounded conformance probes, feature dispositions, and empty/populated evolution cases. It is declarative data: executable hooks, shell commands, numeric storage IDs, caller-defined permissions, version ranges, and silent fallback behavior are not fields in the schema.

Validate a manifest and its exact installation-plan binding before any remote installation mutation:

riffdb application conformance adapter.conformance.json \
  --plan installation-plan.json

The command accepts only canonical encodings. When a plan carries an adapter_manifest_hash, it must also contain the corresponding adapter_manifest artifact content hash. Deployment preflight checks that artifact alongside every generated application artifact; omitting or substituting either identity fails closed. The installed receipt records the same manifest digest, but never bearer credentials, probe values, or host paths.

The current command validates the bounded document and exact plan identity. It does not execute adapter tests supplied by the document; conformance probes are first-party symbolic operation identities and expected observation digests, not executable extension points. The release-owned adapter acceptance runner executes those operations through public clients.

The release corpus owns OpenFGA-, MLflow-, Payload-, and Woodpecker-shaped V2 manifest fixtures. Each binds both an empty-install plan and a populated compatible application-evolution plan. Compatible evolution can retain the exact contract version and bundle while generated artifacts, explicitly approved symbolic role authority, and credentials rotate; it is not permission to substitute a same-version contract bundle. The fixtures include terminal receipts, exact language-neutral operation observations, and immutable typed partial-campaign evidence. Only OpenFGA has complete upstream-suite evidence; MLflow and Woodpecker are language-expressiveness matrices, Payload remains post-alpha, and Better Auth’s general record is limited to its exact materialized-profile matrix. Separately scoped external and live receipts retain their own claims. Run the complete RiffDB-owned corpus with:

./scripts/adapter-conformance --all-domains --all-languages

That gate invokes only public TLS application surfaces and generated Rust, Go, TypeScript, and Python clients. Unsupported required features and driver platforms are product/platform gaps; adapters may not emulate them with source inspection, handwritten transport, shell hooks, or kernel access.

Current POC limit

On every start or resume, the server now reconciles the campaign against its authoritative contract catalog, permanent migration edge, query/reactive module records, compiled-role identity permissions, and successor capability records. Exact already-completed remote work advances durably; absent work remains a symbolic next action, and conflicting identity becomes a typed partial campaign. This makes resuming existing deployment operations safe without accepting caller-asserted remote identities.

The public operator-only reimport coordinator now owns the plan-bound reimport stage. A v2 reimport installation remains safely stopped at reimport_application until the separate Capability V7-authorized campaign has applied every exact page through compiler-owned commands and retained its canonical reconciliation receipt. The installation service observes that receipt from authoritative state; callers cannot submit a completion flag or advance directly to credential publication.

The CLI and Rust operator SDK can resume the exact current driver_proof stage with StartApplicationInstallation::with_driver_proof and the exact current seeds stage with with_seed_receipts. Both forms are checked against the immutable plan before transport and again at service admission. They cannot assert a server-observed stage, skip or replay a stage, carry seed values, or seal the receipt directly. An empty seed plan is completed by the server; a nonempty plan requires one canonically ordered receipt per declared batch, whose successful and replayed counters add to the plan’s exact item count.

These two forms are controller-observed attestations made under the dedicated installation authority. A controller must first run each declared first-party driver identity handshake and each seed through ordinary compiled commands. The attestation does not grant application-write authority and does not turn seed execution into a privileged bulk-write path.

Plan-bound deployment composes the existing exact migration and selected-role deploy shapes. When the campaign reaches apply_migration, deployment submits the plan’s locked migration bundle and confirmation under a deterministic migration operation ID derived from the campaign ID. A running migration returns migration_running with its phase and resume_application_deploy; the invocation does not continue into contract, module, role, or seed work. Rerunning the identical command resumes the retained operation, rechecks its parent, successor, and migration identities, and advances only after the migration reports succeeded. Failed-closed or substituted operations never advance the campaign. The server also requires the campaign-derived migration operation to have a retained succeeded apply receipt with its verified backup manifest before the migration stage can complete, so terminal receipt references cannot name a guessed or unrelated operation.

Every role and credential destination remains present in the immutable plan and is verified by the server; a multi-role application invokes the same plan-bound deploy once per role. Each destination gets a disjoint protected deployment state and credential directory, so one role cannot overwrite or silently replace another. The seed invocation must select a planned role authorized for every declared seed command. Installation never invents migration confirmation, backup policy, or downtime approval: all three are exact, reviewed inputs of the immutable plan and the underlying migration operation retains its existing backup and offline-exclusive owners.

Upgrade and Removal

Upgrade

The POC has no general in-place upgrade or downgrade guarantee. Every release publishes an exact alpha epoch/writer manifest and closed release-edge table. Treat every binary change as a compatibility review; only a manifest-declared edge may run in place.

1. Read release/RELEASE_NOTES.md, docs/compatibility.md, and the accepted ADRs for storage, durable envelope, public protocol, contract IR, and MCP.

2. Run ./scripts/ci-all, ./scripts/demo --assert, and ./scripts/release-poc --verify at the exact source revision.

3. Quiesce application writes, close client-spawned MCP bridges, and stop the optional MCP socket and instances when installed. Keep only the administrative path needed for backup available.

4. Create an offline backup and poll its caller-stable operation ID to a terminal success. Do not readmit application traffic after the backup.

5. Stop riffdbd with SIGTERM and confirm a clean unit exit.

6. Preserve the old binaries, key documents, release bundle, SBOM, and checksums outside the database and backup roots.

7. Install all three binaries from one verified bundle.

8. With riffdbd still stopped, inspect the new binary’s source-free decision:

   riffdb storage preflight --database-path /absolute/path/to/riffdb.redb
   

   If it reports upgrade_required, run only the printed action with the exact verified backup from step 4:

   riffdb storage upgrade \
     --database-path /absolute/path/to/riffdb.redb \
     --backup /absolute/path/to/verified-backup
   

   Keep the database closed throughout. A backup mismatch refuses before an upgrade receipt or format marker is created. Interrupted admitted upgrades are retried with the same database and backup; do not delete or edit the sibling .riffdb-format-upgrade-v1 receipt.

9. Start riffdbd and require the complete startup validation and authenticated health result before admitting application traffic.

10. Re-run the budget smoke, inspect projection/outbox derived health, and only then restart MCP bridges and application clients.

For the optional system bridge, stop activation before draining instances:

sudo systemctl stop riffdb-mcp.socket
sudo systemctl stop 'riffdb-mcp@*.service'

Restart the socket only after authenticated database health succeeds.

Do not rotate digest keys in the same maintenance window as a binary or storage upgrade. A retired key must remain in the readable list until no authoritative or idempotency record needs it.

Downgrade is unsupported. Restoring a pre-upgrade backup is a destructive history rewind, not a downgrade mechanism. It invalidates the destroyed suffix even though DatabaseId is preserved.

Source Convenience Upgrade

For a source convenience installation, perform the compatibility review and steps 1 through 6 above from a clean, reviewed checkout. In particular, close every Codex or other client-spawned riffdb-mcp process before publication; a running process does not change merely because its executable path is replaced. After ./scripts/release-poc --verify succeeds, target/release contains the exact three binaries that the verifier compared, exercised, and placed in the verified bundle. With the selected service stopped and its old assets preserved, publish those same binaries without starting it:

verified_bin_dir="$PWD/target/release"

# User scope
cargo riffdb install --user \
  --from-binaries "$verified_bin_dir" \
  --no-start
systemctl --user start riffdbd.service

# System scope
cargo riffdb install --system \
  --from-binaries "$verified_bin_dir" \
  --no-start
sudo systemctl start riffdbd.service

Require authenticated RiffDB Health and rerun the budget smoke before restarting MCP clients or readmitting traffic. The convenience installer preserves existing configuration, keys, unit, sysusers, and tmpfiles content. If a reviewed revision requires any of those files to change, this is not a supported automatic upgrade: stage and review a package-specific migration instead of forcing the installer to overwrite them.

Removal

cargo riffdb uninstall (--user|--system) [--no-stop] removes only the installed binaries and systemd unit files while preserving databases, backups, digest keys, configuration, and credentials. Before deleting any state, configuration, digest key, or credential, decide whether a retained backup is intended to be restorable and verify it. Capability revocation is part of the authoritative snapshot: revoking a capability after a backup does not revoke the record inside that backup.

Source Convenience Capability Revocation

The convenience bootstrap retains the operator and MCP capability IDs in the operator’s private XDG state root. Treat the owner and MCP capabilities as independent: owner bootstrap can succeed before a deterministic capability request or health failure prevents MCP creation. Resolve each identity whose retained state shows that submission may have begun. Rerun the same cargo riffdb bootstrap invocation when it can resolve that uncertainty; correct a deterministic pre-MCP failure before retrying. Do not delete bootstrap.credential, mcp-capability.pending-id, credential candidates, or related recovery state while a server-side outcome is uncertain.

When mcp-capability.id exists and bootstrap is terminal, revoke that generic MCP developer capability while the server is running and the operator credential is still valid:

config_home="${XDG_CONFIG_HOME:-$HOME/.config}"
state_home="${XDG_STATE_HOME:-$HOME/.local/state}"
riffdb_bin="$HOME/.local/bin/riffdb" # Use /usr/local/bin/riffdb for --system.
"$riffdb_bin" \
  --config "$config_home/riffdb/client.toml" \
  capability revoke \
  "$(<"$state_home/riffdb/mcp-capability.id")" \
  --reason requested

If both mcp-capability.id and mcp-capability.pending-id are absent in intact bootstrap state, and it is certain MCP creation was never submitted, skip only MCP revocation. Continue to preserve or revoke the independently created owner capability as described below. If an expected identity file was lost or its submission outcome is uncertain, stop; local credential deletion is not server-side revocation.

Revocation is a durable database operation. Removing a Codex entry, bearer file, or service unit is not a substitute. Quiesce application writes after unwanted capabilities are revoked, then create and verify the final retained backup while one deliberately retained administrative capability and its bearer are still usable. Keep application writes quiesced from that snapshot through shutdown.

If the database state may be used again, retain that operator capability or create and verify another administrative capability before removing local files. For permanent retirement only, revoke the operator capability after the final backup and immediately before shutdown:

config_home="${XDG_CONFIG_HOME:-$HOME/.config}"
state_home="${XDG_STATE_HOME:-$HOME/.local/state}"
riffdb_bin="$HOME/.local/bin/riffdb" # Use /usr/local/bin/riffdb for --system.
"$riffdb_bin" \
  --config "$config_home/riffdb/client.toml" \
  capability revoke \
  "$(<"$state_home/riffdb/operator-capability.id")" \
  --reason requested

Restoring the final backup restores its retained operator authority even when the live database revoked that operator after the snapshot. Protect the retained bearer as active and revoke or replace it immediately after any restore.

Source Convenience User Scope

If bootstrap registered the MCP server with Codex, remove that client entry after server-side revocation and before deleting local files:

codex mcp remove riffdb

Close the active Codex session and any other client-spawned riffdb-mcp process. Removing the client entry prevents future launches but does not terminate an already-running stdio bridge.

Stop and disable the user service and remove only the installed binaries and unit (pass --no-stop if the service was installed with --no-start and no user manager is running after confirming no riffdbd process is using the database):

cargo riffdb uninstall --user

The uninstaller deliberately leaves the database, backups, keys, server/client configuration, credentials, and recovery state. Archive and verify these roots before any explicit deletion:

• ${XDG_DATA_HOME:-$HOME/.local/share}/riffdb
• ${XDG_CONFIG_HOME:-$HOME/.config}/riffdb
• ${XDG_STATE_HOME:-$HOME/.local/state}/riffdb

After the irreversible decision to destroy them:

config_home="${XDG_CONFIG_HOME:-$HOME/.config}"
data_home="${XDG_DATA_HOME:-$HOME/.local/share}"
state_home="${XDG_STATE_HOME:-$HOME/.local/state}"
rm -rf -- "$data_home/riffdb"
rm -rf -- "$config_home/riffdb"
rm -rf -- "$state_home/riffdb"

Source Convenience System Scope

If used, remove the user-owned Codex entry as the operator after server-side revocation:

codex mcp remove riffdb

Close that Codex session and every other client-spawned riffdb-mcp process before deleting binaries; removing the entry does not terminate an existing stdio bridge. Then stop the system service and remove the source convenience binaries, unit, sysusers, and tmpfiles definitions:

cargo riffdb uninstall --system

The database, backups, digest keys, server configuration, riffdb and riffdb-mcp accounts, and both accounts’ tmpfiles-managed roots remain. The operator’s client.toml, operator.credential, mcp.toml, mcp.credential, capability IDs, and recovery files also remain under that operator’s XDG config/state roots. Archive and verify all of them before deletion. Only after that explicit decision:

sudo rm -rf -- /var/lib/riffdb /var/lib/riffdb-mcp
sudo rm -rf -- /etc/riffdb /etc/riffdb-mcp
sudo userdel riffdb-mcp
sudo userdel riffdb

Remove the operator’s XDG files separately after confirming they are not also used for a user-scope database.

Verified Bundle or Manual System Scope

The convenience capability-ID filenames and /usr/local/bin command do not apply to this scope. Revoke capabilities before shutdown with /usr/bin/riffdb and the IDs deliberately retained when those capabilities were issued. If an ID was not retained, inspect the offline database with riffdb capability list --database-path /var/lib/riffdb/data/riffdb.redb before starting the server to revoke it; deleting its bearer presentation still does not revoke the server-side record.

Before shutdown, quiesce application writes and client-spawned MCP bridges, revoke unwanted capabilities, and create and verify the final retained backup while one deliberately retained administrator remains active in that snapshot. Keep writes quiesced through shutdown. For permanent retirement only, the live database’s last operator may be revoked after the backup; the backup still contains that operator authority, so protect its retained bearer as active and revoke or replace it immediately after any restore.

Stop and disable the optional bridge first, then the database:

sudo systemctl disable --now riffdb-mcp.socket
sudo systemctl stop 'riffdb-mcp@*.service'
sudo systemctl disable --now riffdbd.service

Archive or intentionally destroy the following before removing packages:

• /var/lib/riffdb/data
• /var/lib/riffdb/backups
• /etc/riffdb
• /var/lib/riffdb-mcp
• /etc/riffdb-mcp
• operator and agent credential files
• release checksums, SBOM, configuration, and accepted ADR revision

Remove binaries and unit files:

sudo rm -f /usr/bin/riffdbd /usr/bin/riffdb /usr/bin/riffdb-mcp
sudo rm -f /usr/lib/systemd/system/riffdbd.service
sudo rm -f /usr/lib/systemd/system/riffdb-mcp.socket
sudo rm -f /usr/lib/systemd/system/riffdb-mcp@.service
sudo rm -f /usr/lib/sysusers.d/riffdb.conf
sudo rm -f /usr/lib/tmpfiles.d/riffdb.conf
sudo rm -rf /usr/share/doc/riffdb
sudo systemctl daemon-reload

State, configuration, and the two service accounts are deliberately not deleted by unit removal. After independent backup verification and explicit authorization, remove /var/lib/riffdb, /var/lib/riffdb-mcp, /etc/riffdb, and /etc/riffdb-mcp separately, then remove the riffdb-mcp and riffdb accounts. Destruction of digest keys can make retained databases and idempotency records unreadable; destruction of capability presentations does not revoke their server-side records.

Only after that irreversible decision, remove retained data, configuration, and accounts:

sudo rm -rf -- /var/lib/riffdb /var/lib/riffdb-mcp
sudo rm -rf -- /etc/riffdb /etc/riffdb-mcp
sudo userdel riffdb-mcp
sudo userdel riffdb

Troubleshooting

Start with the process boundary, selected database, credential audience, and active application identity. Do not work around a failure by widening a role or editing a redb file.

Service does not start

systemctl --user status riffdbd.service
journalctl --user -u riffdbd.service --since today

Common causes are a missing protected digest-key file, incorrect permissions, duplicate database aliases, a redb path that is not writable by the service user, or startup integrity validation rejecting incomplete durable evidence. Correct configuration or restore from a verified backup; do not delete records inside the database.

An orderly stop normally permits bounded validation on the next start. A forced kill, incomplete drain, storage repair, restore, migration, or invalid clean-close lifecycle record instead selects complete validation automatically. This can make one restart materially slower without indicating a startup deadline failure. Do not remove or modify private metadata to force either path. See Startup Integrity and Clean Restarts.

If stderr reports RDB-FORMAT-0101, keep the service stopped and inspect the same immutable decision explicitly:

riffdb storage preflight --database-path /absolute/path/to/riffdb.redb

Follow only the returned action. A declared same-epoch transition uses riffdb storage upgrade with the exact verified source backup; an unsupported epoch requires the retained source binary and the documented symbolic export/reimport ceremony. Never remove the marker or upgrade receipt, copy a marker from another database, or look for a force/reset flag.

If a restart reports corruption only when a particular record is used, that is the documented accessed-row integrity boundary, not evidence that validation was disabled. Keep the service stopped if the error affects required authority or active application state, retain the incident ID, and restore from a verified backup when directed. To check the whole database, run riffdb storage scrub and observe the printed operation with riffdb backup operation; it takes the database offline for the pass. Do not edit lifecycle metadata or invoke internal storage APIs as a substitute.

Health reports degraded with a failed storage scrub

The background scrub found a row that fails validation. The storage_scrub member is failed with an incident ID, and authoritative_storage is degraded. Service continues, and any request that touches the damaged row fails closed. Retain the incident ID, avoid the affected data where possible, and restore from a verified backup when directed. A restart does not clear the defect: the next background scrub starts at pending and finds it again. On a V2 database a failure in retained history withdraws authoritative readiness instead; the server then reports not_ready. See Startup Integrity.

CLI connects to the wrong database

Inspect the client configuration and its database alias. Authenticated health and active-contract responses include the selected alias and audience. An unknown alias or a credential issued for another database fails closed.

See Configuration and Multiple Databases.

MCP starts but tools are missing

riffdb-mcp doctor --config ~/.config/riffdb/mcp-myapp.toml

Then check:

1. The MCP host restarted or reloaded after configuration changed.
2. Tool names contain only lowercase letters, digits, and underscores.
3. The bridge selects the intended database alias.
4. The credential is the bound application role, not a bootstrap credential.
5. The active contract and query module match the generated application lock.

tools/list is capability-filtered. A missing operation can be absent, stale, or unauthorized; the server deliberately does not disclose which to an untrusted caller.

Application check reports stale artifacts

Run the explicit sequence from the application repository:

riffdb application check
riffdb application lock --write
riffdb application generate --locked
riffdb application lock --check

Review the lock and generated diff. Do not edit generated files. A successor must compile against the exact active parent; all local role and query preflight must succeed before deployment performs its first remote mutation.

A command response was interrupted

Do not invent a new idempotency key. Retry the identical command with its original key or use the client facade’s outcome resolution. A changed input under the same key is rejected; a new key denotes a new logical command.

Contract deployment fails

Read the structured source-spanned diagnostic. Additive evolution is bounded: renames, removals, type changes, and other incompatible edits require the documented pre-alpha reset path for disposable data. Back up the database before reset and stop the service throughout the operation.

Projection is behind

Inspect health and the reported projection frontier. A read-after-commit query may wait only to its bounded deadline. Projection state is rebuildable; never patch it to make a frontier appear current.

Report a defect

Capture the RiffDB version, public error code, incident ID, database alias, operation name, relevant configuration with secrets removed, and a minimal reproduction. Never attach credential files, digest keys, raw protected logs, or production database files.

Security Posture and Threat Model

RiffDB’s alpha target is a controlled Linux deployment. Application gRPC can use verified direct TLS or a protected Unix socket; hosted MCP supports literal-loopback cleartext or verified direct TLS. The system is not hardened for an untrusted internet-facing or shared multi-tenant service.

The source-build TLS dependency baseline is pinned to rustls 0.23.46 and rustls-webpki 0.103.14 with the existing ring provider. This addresses RUSTSEC-2026-0285, which concerns handshake messages crossing encryption-key boundaries. Peer verification, transport limits and capability authentication remain required; there is no insecure fallback or application-selectable provider.

The 0.23.46 patch also addresses RUSTSEC-2026-0346 through RUSTSEC-2026-0349, with no TLS provider or feature-set change. See the upstream release.

Trust Boundaries

• riffdbd owns authoritative storage, key providers, authentication, authorization, command execution, sequencing, and atomic persistence.
• riffdb, the Rust SDK, hosted MCP, and riffdb-mcp are public clients. None may open redb or construct a privileged service.
• MCP tool discovery and invocation are policy filtered. Maintenance is not an MCP operation.
• Administrative V3 replication requires the explicit ReplicateChangelog permission, global tenant scope and all partitions. Application-role grants and bootstrap convenience capabilities cannot carry it. Each release uses current policy, and the RPC requires direct TLS or a protected local socket. The outbound replication peer requires an explicit protected CA file and exact server-name TLS verification before any credential is sent. It exposes no arbitrary-channel or cleartext constructor. Receiver connections verify the configured source lineage and complete durable transfer before allowing offline materialization; receiving bytes alone never grants readiness. See follower administration.
• Projection and outbox workers consume committed authoritative state. Their failure may degrade service but cannot roll back a commit.
• The deterministic runtime receives checked inputs and reserved logical time; it has no ambient I/O, clock, or randomness.
• Row-policy source, principal-fact schemas, and role selection are compiler inputs. Application requests cannot submit a predicate, fact, evaluator, or bypass. The row-policy reference records the current compiler/runtime boundary.
• Symbolic application export has distinct Capability V5 authority. Entity, event, provenance, public-audit, capability-administration, and backup rights do not imply export. Principal-filtered scope requires the exact current V4 role/policy extension; whole-application scope requires an explicit named lineage under global tenant and all-partition scope.
• Compiler-owned application reimport has distinct Capability V7 authority. It binds one exact destination installation campaign, contract lineage, portability-manifest hash, and principal-filtered or whole-application scope. Every reimport grant also carries an exact compiled application-role identity and row-policy grant; whole-application scope does not disable that policy.

Primary Threats

Deployment Rules

• Keep cleartext application gRPC and hosted MCP on literal loopback. Remote application gRPC and hosted MCP must use the direct_tls profile; same-host/sidecar clients may use a protected local_socket profile for application gRPC.
• A TCP or HTTP/2 proxy may forward verified TLS, but forwarded identity, tenant, database, principal, and authorization assertions are ignored. The normal RiffDB bearer credential and database selector remain mandatory.
• Keep the database file, backup root, capability digest-key file, and idempotency digest-key file lexically disjoint. The packaged layout does so.
• Make the riffdb service account the only writer to the database directory. Never run two riffdbd processes against one database, and never grant the CLI, MCP bridge, backup utilities, or another service direct write access.
• Own secret files by the exact service effective user at mode 0600.
• Protect TLS private keys with no group or other permission bits. Replace the certificate and key atomically; an invalid replacement retains the last valid identity for new handshakes.
• Give the optional MCP bridge a narrowly scoped normal capability. Never use the bootstrap document as its long-lived credential.
• Treat membership in the riffdb-mcp socket group as access to the bridge capability.
• Protect backup artifacts and receipts even though they contain no digest-key documents.
• Inspect only structured, redaction-safe output. Do not add raw business values, tokens, filesystem paths, or arbitrary error strings to telemetry.

The systemd units remove ambient capabilities, deny privilege escalation, restrict writable paths, devices, namespaces, kernel interfaces, address families, and non-loopback IP traffic. The bridge units have no access to the RiffDB state directory. Hardening is defense in depth; application-enforced loopback and authorization rules remain authoritative.

Incident Handling

For suspected bearer compromise, revoke the capability through the public service and replace the credential file. Replacing a file alone does not revoke the durable capability. Operators can list and inspect issued or revoked capabilities without exposing bearer tokens or secret verifiers using riffdb capability list and riffdb capability show, either against a closed database file (--database-path <PATH>) or against the local non-secret operator inventory ledger (--inventory-file <PATH>).

For digest-key compromise, stop admission and obtain architecture review before rotation. Capability and idempotency keys are distinct typed namespaces and must not reuse material. Preserve required readable keys during rotation.

For storage or receipt corruption, stop. RiffDB intentionally fails readiness closed and has no online repair path. Restore only from a fully validated offline backup and account for the destructive rewind limitations.

Capability-create JSON may include a bounded grant.export.applications array. Each entry names one contract lineage, selects principal_filtered or whole_application, and explicitly selects entities, events, provenance, and public audit. At least one of entities or events is required. This authority surface is separate from ordinary read or administrator credentials.

An operator capability for reimport adds grant.reimport alongside its compiler-derived grant.row_policy and matching application_role_identity permission. The reimport object accepts only:

{
  "contract_lineage": "TicketDesk",
  "campaign_id": "01900000-0000-7000-8000-000000000043",
  "portability_manifest_hash": "4343434343434343434343434343434343434343434343434343434343434343",
  "scope": "whole_application"
}

The campaign ID must be the exact destination installation UUIDv7 and the hash must identify the canonical adapter-owned portability manifest. Changing either identity, using the capability against another database, dropping the row-policy grant, or widening a principal-filtered grant fails closed. This authority is not advertised through application command discovery or MCP.

Follower read audit boundary

The accepted follower service audit boundary keeps the follower applier as the sole writer. The follower service component retains current authorization and redaction for reads. Authorized Health/Statistics and authenticated read denials produce bounded redacted operational telemetry, with no durable local service audit guarantee. Denials release no protected data. Telemetry cannot satisfy a policy-required durable audit obligation; standard reads with that obligation return the typed follower-mode refusal before releasing data. Applications requiring durable audit of reads or denials must use the primary. Primary audit behavior and replicated audit records are unchanged.

The daemon and process crash tests exercise this boundary across restarts. See follower administration before planning deployment.

Known Limitations

These limits are part of the POC release posture, not hidden roadmap promises.

Deployment and Operations

• RiffDB is a pre-release proof of concept, and its dual license does not change its security, compatibility, or support limits.
• Linux is the supported deployment platform; protected credential and key-file loading is unavailable on other operating systems.
• The source convenience installer targets Linux systemd, requires a valid private XDG_RUNTIME_DIR for user-scope unit verification, and does not configure login sessions or lingering.
• Automated source-install acceptance exercises user scope only, while system-scope installation requires first-install acceptance on a disposable or staging host.
• Application gRPC supports literal-loopback cleartext, verified direct TLS with static certificate and key paths, and a protected Unix socket, and hosted MCP supports literal-loopback cleartext or verified direct TLS, while the riffdb-mcp stdio bridge’s upstream gRPC hop is loopback-only and OAuth, mTLS, automatic certificate issuance, and untrusted internet-facing operation are unsupported.
• Browser code must use an application-owned backend and never receive a RiffDB credential.
• riffdbd exposes only the documented POC configuration fields, with a loopback-only metrics listener and no storage-engine selector, general tuning surface, or secret values in TOML.
• After generic bootstrap, systemd active means the process started while authenticated Health remains not_ready until the first application contract is deployed.
• The production server uses redb only, and the isolated Fjall comparison workspace is never linked into or shipped as a server binary.
• Benchmark reports are revision-specific evidence rather than a standing performance promise, with no absolute TPS gate and no valid claim without published hardware, filesystem, durability mode, workload, revision, and raw output.
• The sustainable TicketDesk rehearsal remains blocked by continuing-load source-receipt reclamation and final-window tail latency.
• Ordinary redb reads validate RiffDB envelopes, CRCs, schemas, and keys without verifying every engine page checksum, whereas offline physical verification requires extra disk space for a private copy and refuses repair or unrecovered journal suffixes; see startup integrity.
• The shared crash-recovery matrix covers the enumerated POC recovery boundaries rather than providing a general disaster-recovery guarantee.
• The POC authorization model uses opaque local capability tokens rather than a production identity provider or OAuth authorization server.
• Generic bootstrap deploys no application and grants its MCP developer no wildcard application-data authority until an administrator binds a compiled role or lineage-scoped capability.
• Convenience bootstrap capabilities expire after at most 30 days with no background daemon renewal, though operators can rotate capabilities explicitly with riffdb capability rotate <CAPABILITY_UUIDV7> --credential-output <PATH> --revoke-old or cargo riffdb bootstrap (--user|--system) --renew.
• Re-running the source installer updates the three binaries while preserving existing keys, configuration, and unit files, and cargo riffdb uninstall removes only installed binaries and unit files without performing configuration migration, key rotation, data deletion, or format upgrades.
• The source installer stages and backs up riffdbd, riffdb, and riffdb-mcp inside .riffdb-binary-rollback-v1 on the destination filesystem so an interrupted binary publication rolls back to the previous set on exit or at the start of the next install.
• Optional Codex MCP registration serializes bootstrap callers through $XDG_CONFIG_HOME/riffdb/.codex-mcp.lock and treats an identical existing server entry as idempotent, while external writers to the same user’s Codex MCP configuration outside that lockfile remain uncoordinated.
• Metrics are exposed only over an optional literal-loopback HTTP listener (GET /metrics and GET /v1/metrics) or periodic local NDJSON flight-recorder files (riffdb telemetry summarize <PATH>), with no non-loopback network metrics listener or push exporter.
• Neither a clean restart nor a bounded dirty restart performs a full population scrub, and riffdb storage scrub takes the database offline for the entire pass with no partial scope, resume, or repair option.
• Restart after an abrupt stop pays a storage-engine repair proportional to the database file size, and the first write after any restart rebuilds the in-memory command indexes once before admitting writes.
• Outbox delivery supports only the configured protected local archive directory ([maintenance.outbox] / [databases.<alias>.outbox] and riffdb outbox archive), with no external network, webhook, or broker connector.
• The optional systemd MCP socket shares one capability among members of its Unix socket group, so operators must spawn one bridge per client when principal identities must differ.
• Release verification validates systemd unit syntax and hardening directives without starting the units inside the host systemd manager.
• A POC-exit decision requires all POC-001 through POC-010 evidence together with a human architecture review.

Contracts and Writes

• There is no general SQL surface, arbitrary transaction callback, analytical join engine, distributed transaction, automatic failover, or consensus protocol.
• An ordinary command may atomically delete and return the transaction-current preimage of at most one complete-key, partition-local no_inbound entity, whereas multi-row deletes must use the separately compiler-bounded collection deletion forms.
• Reactive applications are partition-local and bounded, with no raw CDC, cross-partition ordering, wall-clock event retention, exactly-once external effects, event-sourced reconstruction, webhooks, or in-process agent inference.
• Protected event replay and reaction validation acquire the storage mutation fence to obtain a transaction-current capability, row, and relationship view, and protected workers use bounded consumer long-poll rather than the global reactive wakeup.
• Contract migration administration is available through the operator service, Rust client, and CLI, and is intentionally absent from MCP, TypeScript, Python, and generated application clients; Gate C rejects key swaps and predecessor-occupied rekey chains.
• The PostgreSQL counterexamples in the repository demonstrate only that unsafe write patterns remain expressible in a general SQL surface and are excluded from RiffDB’s compiled command surface.

Queries and Projections

• Runtime-selectable page sizes use the query-only Limit<MAX> refinement up to 65,534 rows and do not refine offsets, byte budgets, contract integers, aggregate work, or projection candidate budgets.
• A single RiffDB query invocation returns at most its compiled row limit (and never more than 65,534 rows or 4 MiB of encoded results), with no hidden multi-page response or streaming query protocol.
• Named bounded RiffQL aggregates and the ad-hoc projected CLI support exact counts (count, count_present, count_distinct, count_distinct_present), sum, mean (over i64, u64, uniform-scale Decimal, and uniform-currency Money), min, max, Boolean any/all, and bounded single- or multi-column group by with canonical or explicit group-key --order and top-N --limit truncation; compiled RiffQL queries additionally support bounded having and order by over up to four aggregate or group-key items, while ad-hoc projected CLI queries do not expose a having clause or sorting by computed aggregate outputs.
• Candidate algebra supports one compiler-sealed non-output binding across at most eight same-partition declared-index sources, one-level bounded one-to-many expansion, and at most 32 compiler-expanded root orders, without depth-two expansion, cross-partition joins, or caller-supplied sort expressions.
• long_pattern_v1 provides exact binary or Unicode-fold equality, literal affix/substring, and LIKE/ILIKE matching for values up to 8,000 bytes as a rebuildable candidate provider, without general regular expressions or direct unanchored negation.
• Ordinary row-store indexes execute exact equality, one bounded membership or range/prefix branch, and the complete declared order, while canonical string ranges, multiple branching dimensions, residual post-page filters, and runtime join optimization remain compile-time refusals.
• The external tuple-capability fixture is a narrow source-compilation receipt for bytewise equality, membership, and order rather than a full external framework-conformance claim.
• Named vector search selects exact scan at or below the declared per-organization ann_threshold (up to 500 rows when undeclared or 65,536 rows when declared) and first-party in-memory HNSW on primaries and followers above that threshold when a warm graph is available, refusing larger organizations with ScanBudgetExceeded.
• Vector staleness uses v1 count semantics (stale_count > threshold) rather than duration-based thresholds.
• Production nearest() requires one explicit compiler-owned projected source with available, causal, or duration-bounded freshness and refuses multi-source vector queries, vector joins, or row-store fallbacks.
• Columnar artifacts are demand-activated on their first semantic query, returning Building while activation is running, retrying transient control-read unavailability on the next worker pass, and allocating a replacement candidate rebuild when a published artifact fails validation.
• Projection state is derived and rebuildable, so projections may report degraded or building status while authoritative commits remain available.
• Tokenized-text search supports conjunction, capped disjunction, phrase, and bounded ordered proximity with fixed riff_bm25_v1 ranking across eight retained provider epochs, without wildcards, regular expressions, fuzzy matching, stemming, snippets, caller boosts, or hybrid vector/text queries.

Replication, Backup, and Restore

• Asynchronous follower replication, follower reads, operator registration and retirement, and explicit incarnation-fenced primary promotion operate on a single-primary topology without quorum durability, automatic failover, cascading followers, or multi-primary writes; see follower administration.
• Restore publication requires the database directory to be private to the sole configured service identity, and shared write access or concurrent server processes are unsupported.
• Full backup and restore publication are offline maintenance operations, while changelog-derived archive backup and restore operate at commit-sequence granularity (with storage-level wall-clock timestamp resolution via resolve_archive_stop_at_timestamp) without built-in archive encryption; see backup and restore.
• Destructive restore preserves DatabaseId while rewinding sequence allocators and bumping the durable history_incarnation fence (RDB-HISTORY-0101); stateful RiffDbClient and riffdb-mcp sessions automatically track and send observed_history_incarnation across commit and event reads, while newly connected stateless callers that omit a prior observed_history_incarnation adopt the restored incarnation.
• Locators, cursors, sessions, read-after-sequence expectations, authorization observations, and idempotency assumptions from a destroyed history suffix are invalid after a restore.

Retention

Online retention (see Configuration) never passes a fence, and in this POC two fences and one checkpoint interaction bind more than an operator may expect (live state and active binding fence are exposed on /metrics, /v1/metrics, and the periodic NDJSON flight recorder):

• When [maintenance.outbox] (or [databases.<alias>.outbox]) is omitted, no outbox worker is configured, so an emitted event is never marked delivered automatically and holds retention from the first undelivered event onward (online_retention_fence="undelivered_outbox") until local archive delivery is configured.
• A declared columnar projection that has not been queried in the current process holds retention at its last populated frontier (online_retention_fence="projection_durable_frontier", or at the beginning of history if never populated) until a query activates it or an operator detaches it with riffdb retention projection-detach.
• Online retention invalidates the optional validated-prefix startup checkpoint when logical pruning commits and refreshes it off the commit gate once a retention batch drains, while a Completed startup registry binding preserves bounded restart even before that refresh finishes; see Startup integrity.
• Online logical pruning is paced to about five percent of writer time in journaled steps of at most 256 sequences and can fall behind under sustained write saturation.

Storage reclamation

• Sealed journal extents (<database>.riffjournal.sealed.<generation>) retain command and receipt history referenced by compact stubs in the database file and must never be deleted by hand.
• A sealed journal extent is retired only after logical pruning and source-history reclamation release every command and receipt reference with no active reader or scrub holding it.

What a command cannot express

The compiler proves bounded structural properties of application writes; see the write cookbook. The command language has the following structural limits:

• Commands read entities by complete key or through one atomic indexed first-row selection whose equality prefix proves the partition and every conflict key before the read.
• An invariant may span entities inside one aggregate, and a command may mutate multiple aggregates that share a partition route, but no invariant may relate two distinct aggregates.
• A command cannot read a derived projection because write decisions must never depend on an asynchronous projection watermark.
• Allocating an unspecified free row from an unowned pool in a single round trip without root contention is not expressible inside one command.
• General multi-step durable workflows and sagas outside a single compiled command are not implemented in this POC.

Compatibility

RiffDB 0.1.0 is a pre-release POC. Compatibility is checked at explicit boundaries; it is not a blanket promise that arbitrary commits can be mixed.

Storage adapter Rust source compatibility

riffdb_storage_api::DeclaredOutcome shares its immutable canonical value across clones. value() still returns &CanonicalRecord, but it is no longer a const fn. Storage adapters calling it inside a const function must make that caller an ordinary function. Generated application interfaces and stored outcome bytes retain their existing contracts.

Per-write evidence in V25

Grammar, executable IR, and bundle V25 apply to newly compiled commands with field assignments. Every assignment carries validated evidence in its plan hash. Recompilation rejects replacement set writes justified only by unrelated reads or revision increments; migrate to an explicit checked_by business association, a caller-version fence, or an intentional overwrite. Normal application-lock renewal is required, including commands whose execution behavior is unchanged. There is no warning-only compiler opt-out.

Older deployed bundles retain their bytes, identities, validation, and execution. Their historical “guarded” classification is reported as legacy dependency evidence, not retroactively as caller-version protection. See the write cookbook for the proof limits and supported POC shapes.

Boundaries

Generated source, descriptors, schema inventories, wire vectors, contract fixtures, and interface checkpoints are compatibility evidence. The current source release also carries:

RiffQL V9, query IR V12, and query-module V12 are additive identities for the compiler-declared Limit<MAX> page type (and later additive versions through RiffQL V14, query IR V18, and query-module V18 cover relationship algebra, multi-column group by / having / order by, and event/scalar columnar projected queries). Earlier language sources and IR/module artifacts retain their existing writers and strict readers. A bounded maximum is immutable operation identity: changing it requires module/application rotation and invalidates predecessor cursors, but does not migrate entity data or change storage or public Protobuf formats.

The process-local named-query cursor hash domain excludes only compiler-proved ordinary page-cardinality values. Shared parameter hashes, query/IR/module/plan/lock identities, public opaque token bytes, protocols, storage, and durable formats are unchanged. Cursor registries already do not survive process restart, so predecessor tokens are not reinterpreted. A resumed ordinary ordered query may change its valid submitted page size while all invariant parameters, authority, snapshot, order, and provider epoch stay bound.

• release/version-topology-v1.json, the cross-domain map of independently owned reader/writer windows, writer policy, lifecycle, source assertions, and evidence described in Versioning and Retirement;

• release/durable-format-manifest-v1.json, the exact epoch/writer and complete readable/writable record, redb-layout, journal, backup, marker, and named receipt-family inventory plus the minimum/maximum supported source release;

• fixtures/compatibility/release-pairs-v1.json, the closed supported upgrade graph;

• fixtures/compatibility/physical-formats-v1.json, the redb, journal, backup, receipt, history-incarnation, retention, and recovery-validation inventory; and

• fixtures/compatibility/durable-fixture-inventory-v1.txt, whose SHA-256 is bound into the release manifest.

Run ./scripts/check-version-topology to verify all release-significant version domains, ./scripts/check-durable-format-manifest to verify the physical boundary directly, or ./scripts/check-generated to verify both with every generated artifact. A release bundle is rejected if the manifest, upgrade table, fixture digest, export posture, downgrade posture, or known limitations are absent or stale.

The current pre-alpha identity is alpha epoch 1, writer 4. Downgrade is unsupported. A format marker from another epoch or writer is not evidence that its bytes are readable merely because a decoder accepts some records.

RiffDB retains explicit backup-bound offline upgrades from writers 0, 1, 2 and 3. Writers 2 and 3 are accepted only with their exact frozen registry and compatibility inventory; an epoch or writer number alone grants no transition. Use the existing riffdb storage upgrade operation with a verified matching backup, downtime and free space of at least twice the source size. The native upgrade checks this floor before any mutation on initial admission and every retry. It counts logical lengths of the database and its retained journal, extent, marker, receipt and lifecycle working files, excluding the immutable backup and unrelated siblings. Directory discovery is limited to 16,384 entries; missing, ambiguous, nonregular or unreadable source artifacts and arithmetic overflow refuse. An insufficient-space refusal requires provisioning space and retrying with the same immutable backup. Availability is observed for the invoking principal on the target filesystem; it reserves no space, and later I/O failures still use existing fenced recovery. The backup remains immutable. Ordinary startup and older binaries refuse incompatible markers before opening storage; downgrade remains unsupported.

A predecessor physical backup remains valid as immutable upgrade evidence, but current-binary physical restore requires its declared format range to include writer 4. Upgrade the source and create a new verified backup, or use the matching predecessor binary for restoration before its supported offline upgrade. A successor restore refuses an incompatible backup before creating or changing the target. It never promotes or rewrites the frozen backup.

The writer-3 registry adds private StoredRecordRegistryV3 metadata with Pending or Completed state at the existing metadata key. Upgrade reconciles the acknowledged journal and installs Pending through a durable migration receipt. It grants no readiness shortcut. Only complete native structural and catalog-semantic validation can publish a Completed binding for the exact database, history incarnation and current registry, atomically with its validated-prefix completion receipt. Legacy V2 registry bytes remain readable and contain no independent completion witness.

Writer 4 adds logical journal frame V5 and its closed SourceReclamation kind. V2, V3 and V4 remain readable with their frozen semantics; only V5 may encode the new kind. The physical extent format, canonical V3 receipts and application interfaces are unchanged. Online Standard-profile source reclamation publishes a bounded source-prefix cut without first draining the active suffix. The cut preserves application and administration frontiers and remains subject to complete validation, all holds and an independent durable checkpoint. Replay validates its actual preimages and applies it atomically.

The native writer admits frame ownership against the compiled 64-MiB recovery ceiling before publication. Its conservative bound includes nested terminal receipt reconstruction and can refuse work below the 16-MiB wire ceiling with the existing typed limit failure. Recovery checks the same bound before decoding recorded frames or applying mutations. There is no memory-limit override.

A writer-3 source requires a newly verified matching immutable backup for its own upgrade. A completed older upgrade receipt can establish only that exact previous transition; it never substitutes for the new backup or permits a marker-only override. Upgrade preserves acknowledged commands, outcomes and provenance. Writer-3 binaries refuse the successor marker before opening the engine or replaying V5. Downgrade remains unsupported.

Capability records follow a least-successor rule: ordinary grants remain V1, migration authority uses migration-only V2, installation authority uses V3, and a compiler-bound row-policy grant uses V4. A grant with explicit symbolic application-export authority uses V5; V5 is not selected merely because a grant can read entities, events, provenance, audit, or administer capabilities. V5 preserves the exact optional V2 through V4 extensions and adds a bounded, lineage-ordered export extension. Principal-filtered export requires the matching V4 role/policy extension, while whole-application export requires explicit global/all-partition authority. The public export operation uses that authority for its shared-service, gRPC, Rust operator-client, and CLI safe points; it never derives export access from an older permission. Capability V6 adds administrative replication and primary-fencing authority (ReplicateChangelog and FenceReplicationPrimary), and Capability V7 adds compiler-owned application reimport authority (grant.reimport).

The V4 extension binds one exact application-role hash, canonical principal facts, and canonical protected entity/operation selections; omitting the extension under the V4 identity is corruption. V1 through V4 source, descriptor, hash, and wire fixtures remain frozen. V2’s source and schema hash are frozen to the original capability-V2 bytes. A short-lived pre-alpha build accidentally emitted the additive V3 installation payload under V2’s compact identity; current readers recover that exact payload without dropping authority, while every new installation grant is written as V3. No general unknown-field or best-effort durable decoder is enabled by this narrow compatibility repair.

Normal daemon startup performs the source-free format comparison before redb can open the file. An exact mismatch exits with RDB-FORMAT-0101, the retained and binary epoch/writer identities, the sole manifest-authorized action, backup and downtime requirements, and a safe next command. It never falls through to a decoder-shaped error, creates a replacement database, or enters backup recovery merely because the format is incompatible.

For a new path, RiffDB durably publishes current-format initialization intent before creating the redb container. A crash with only that marker (or an empty container) resumes initialization on the next start; it is never mistaken for a predecessor database that needs an offline upgrade.

The current durable-record registry adds the private clean-close lifecycle record through one exact predecessor-to-successor registry migration. That migration does not synthesize clean evidence: the first open performs complete validation and records dirty lifecycle state. A matching clean lifecycle record is eligible only under the exact successor registry digest. Older binaries do not recognize that registry and downgrade remains unsupported.

Clean-close compatibility is exact rather than best effort: