Storage

Tarski separates two storage concerns: blobs (large raw content referenced by observations) and observation history (the append-only log itself, kept forever and tiered by semantics).

Blob storage

Large raw content — uploads, fetched documents, archived reports — is stored as content-addressed blobs. A blob’s identity is sha256:<hex> of its canonical bytes, computed before any backend write; observations and receipts carry digest-only BlobRefs.

Where bytes physically live is deployment configuration with no semantic authority: replaying the same timeline must produce byte-identical digests, receipts, derived facts, and exports on every supported backend. Backends: sqlite (zero-config local default), memory, mock (deterministic test double with scripted faults), and s3 (any S3-compatible endpoint).

[storage]
blob_backend = "s3"
max_blob_bytes = 26214400

[storage.s3]
bucket = "my-app-blobs"
endpoint = "https://s3.example.com"
access_key_env = "MY_S3_ACCESS_KEY"    # credentials by env-var name only
secret_key_env = "MY_S3_SECRET_KEY"    # inline values are rejected at load

Selection priority is flag > env (TARSKI_BLOB_STORE) > file > sqlite, with a --blob-store flag on dev, serve, replay, and verify. A configured S3 target runs a write/read/head/delete probe at startup and fails closed — there is never a silent fallback to another store. Managed Cloud ignores app-level storage configuration entirely; the platform owns the target there.

Apps reach storage through declared blob.put / blob.get capabilities. Typed errors cover the honest failure surface: blob.size_limit_exceeded, blob.quota_exceeded, blob.not_found, blob.corrupt_content, blob.tombstoned, blob.restricted, blob.range_unsatisfiable.

Operate it with:

tarski stats                      # usage, egress, quota pressure
tarski gc --blobs --dry-run       # unreferenced-blob collection, tombstoned
tarski storage migrate --to s3    # copy with digest verification
tarski blob put ./report.pdf      # plus: hash, get, inspect

Quota pressure reports as under, soft_exceeded, or hard_exceeded, with dedup-aware usage. Studio’s Storage surface adds a blob browser, uploads, downloads, and strictly dry-run-gated reclamation.

Semantic tiering

The observation log is canonical truth, so Tarski’s posture is keep everything — at object-storage cost. Semantic tiering makes that economical without a recency heuristic: sealed, hash-chained archive segments carry semantic coverage computed from the same canonical mappers replay uses — which namespaces, entity keys, and strata each segment can affect, under which evaluator digest.

Cold reads then demand history in ontology terms — everything (replay), a namespace reduct (a declared query’s read set), or an entity neighborhood (provenance traversal) — and fetch only matching segments. The rules are conservative by construction:

• Fail-safe pruning. Absent coverage, a foreign evaluator digest, or an overflowing neighborhood index always degrade to fetch-everything. Pruning is an optimization, never a correctness boundary.
• Fail-closed integrity. Digests verify before decoding; tampered bytes fail.
• Tier-invariant identity. Archive replay produces the same revision digest as hot replay; pruned query execution produces the same result digest as full hydration. Every hydration returns a receipt (segments fetched, observations, bytes).

tarski stats folds validated segment manifests into per-lineage archive cost lines — segment count, archived observations, archived bytes, floor bytes-per-observation.