Storage

Tarski separates two storage concerns: blobs (large raw content referenced by observations) and observation history (the append-only log itself, kept forever and tiered by semantics).

Blob storage

Large raw content — uploads, fetched documents, archived reports — is stored as content-addressed blobs. A blob’s identity is sha256:<hex> of its canonical bytes, computed before any backend write; observations and receipts carry digest-only BlobRefs.

Where bytes physically live is deployment configuration with no semantic authority: replaying the same timeline must produce byte-identical digests, receipts, derived facts, and exports on every supported backend. Backends: sqlite (zero-config local default), memory, mock (deterministic test double with scripted faults), and s3 (any S3-compatible endpoint).

[storage]
blob_backend = "s3"
max_blob_bytes = 26214400

[storage.s3]
bucket = "my-app-blobs"
endpoint = "https://s3.example.com"
access_key_env = "MY_S3_ACCESS_KEY"    # credentials by env-var name only
secret_key_env = "MY_S3_SECRET_KEY"    # inline values are rejected at load

Selection priority is flag > env (TARSKI_BLOB_STORE) > file > sqlite, with a --blob-store flag on dev, serve, replay, and verify. A configured S3 target runs a write/read/head/delete probe at startup and fails closed — there is never a silent fallback to another store. Managed Cloud ignores app-level storage configuration entirely; the platform owns the target there.

Apps reach storage through declared blob.put / blob.get capabilities. Typed errors cover the honest failure surface: blob.size_limit_exceeded, blob.quota_exceeded, blob.not_found, blob.corrupt_content, blob.tombstoned, blob.restricted, blob.range_unsatisfiable.

Operate it with:

tarski stats                      # usage, egress, quota pressure
tarski gc --blobs --dry-run       # unreferenced-blob collection, tombstoned
tarski storage migrate --to s3    # copy with digest verification
tarski blob put ./report.pdf      # plus: hash, get, inspect

Client blob contract

@tarski/client 0.8.0 exposes generated upload, inventory, and metadata operations; client.downloadBlob() handles full or ranged raw-byte downloads. Each request is lineage-scoped and uses the same in-memory runtime or verified end-user bearer as other SDK calls.

Blob authorization is not “possession of the digest”. The runtime binds the authenticated organization, tenant, workspace/lineage, active redaction policy, and caller scope before touching storage. A digest from another tenant cannot be used to probe existence. Listing is filtered before pagination, and restricted content may leave its digest visible while withholding bytes.

The default per-blob cap is 256 MiB; deployments may set a smaller cap. The workspace hard quota is dedup-aware: uploading bytes already stored under the same digest is idempotent and does not consume quota again. Hosted transfer and egress quotas can also return typed, retryable throttling with a retry window.

Client-visible failures are stable:

Code	HTTP	Meaning
blob.size_limit_exceeded	413	This object exceeds the configured per-blob cap
blob.quota_exceeded	413	The workspace hard quota would be exceeded
blob.not_found	404	No authorized object resolves for the digest
blob.restricted	403	Redaction/authorization permits metadata but withholds bytes
blob.tombstoned	410	Physical purge is recorded by a durable tombstone
blob.range_unsatisfiable	416	The requested range lies outside the byte length
blob.corrupt_content	502	Stored bytes failed digest verification and were not served

Use capability discovery before showing an upload control. An authority that omits the generated blob operations resolves to unsupported_runtime; the SDK never guesses an older route.

Quota pressure reports as under, soft_exceeded, or hard_exceeded, with dedup-aware usage. Studio’s Storage surface adds a blob browser, uploads, downloads, and strictly dry-run-gated reclamation.

Semantic tiering

The observation log is canonical truth, so Tarski’s posture is keep everything — at object-storage cost. Semantic tiering makes that economical without a recency heuristic: sealed, hash-chained archive segments carry semantic coverage computed from the same canonical mappers replay uses — which namespaces, entity keys, and strata each segment can affect, under which evaluator digest.

Cold reads then demand history in ontology terms — everything (replay), a namespace reduct (a declared query’s read set), or an entity neighborhood (provenance traversal) — and fetch only matching segments. The rules are conservative by construction:

• Fail-safe pruning. Absent coverage, a foreign evaluator digest, or an overflowing neighborhood index always degrade to fetch-everything. Pruning is an optimization, never a correctness boundary.
• Fail-closed integrity. Digests verify before decoding; tampered bytes fail.
• Tier-invariant identity. Archive replay produces the same revision digest as hot replay; pruned query execution produces the same result digest as full hydration. Every hydration returns a receipt (segments fetched, observations, bytes).

tarski stats folds validated segment manifests into per-lineage archive cost lines — segment count, archived observations, archived bytes, floor bytes-per-observation.