Bitspark constellation
proposed source ↗

ADR 0033 — the live plane is substrate-native: subscribe to stele watch for facts; per-service buses publish only rebuildable projections, never authoritative fact logs

  • Status: proposed
  • Date: 2026-06-21
  • Builds on: 0016 (stele is the substrate; arche-runner's process.v0 watch/reconcile loop is "the canonical fact-driven reconcile pattern" — this ADR names the watch side of that loop as the substrate-native live signal and the home of the live plane), 0031 (the fact-derived judgment–realization loop — record → lens → … → observer → record; the loop is level-triggered over the record, so how a consumer learns the record changed is exactly this ADR)
  • Relates: 0030 (decide-once; the substrate, not each consumer, is the read-authorization site), 0010 (atlas coordinates, the record is authoritative — preserved), 0028 (one logical order — no divergent per-service fact logs), 0026 (the runtime desired/observed planes a consumer reacts to), pharos ADR 0008 (pharos's in-process lifecycle event bus — this ADR draws the line between what stays on it and what becomes a watch projection; the repo is private)

The "live plane" — how a consumer learns the record changed and reacts now — is substrate-native for facts, because the stele op-log already is a signed, space-authorized, append-only event log and stele already ships watch. So a consumer of substrate facts builds its live view from stele watch, not from a re-derived per-service event log over those facts — a service-owned fact log strips the substrate's signatures, read-authorization, and resumable cursor and forks one-root coherence (0028) into N divergent logs. But this ratifies the DIRECTION, not stele's current transport guarantees — and an expert review established that several are not yet met. Concretely, today: a raw watch event is an authorized change signal ({Kind, FactID, Cursor}) — it does not carry the fact body, so a consumer reads-then-queries (or uses derived-query watch, which does carry rows); the lagged backpressure resume, the initial-batch handoff, and the post-revocation projection are not yet gap-free / authorization-current (§"Prerequisites"). So the rule for a service bus is not "carries no facts" but: a service may publish a rebuildable, cursor-attributed projection of stele facts (e.g. pharos's dashboard fan-out), but must never be an independently authoritative or replay-defining fact log. pharos ADR-0008's bus stays for genuine non-fact runtime events (vault.state, collection.finished); its deploy.* subset becomes such a projection. The cheap first step (deploy.exp1's poll → watch, single-node) is real; the durable, gap-free end-state is gated on the stele prerequisites below, not assumed.

Context

0031 closed a fact-derived control loop whose canonical shape (from 0016) is watch/reconcile: a consumer watches the record, reconciles level-triggered, asserts results back. That loop presumes a live signal — "the record at this space changed; re-read and react." What was never ratified is where that signal comes from, and the constellation grew two answers at once:

  • stele already has the substrate-native signal — and it is unused. watch is one of stele's reserved wire methods: a WebSocket push (GET /v1/watch), in all three SDKs (Watch() Go/Rust/TS) and the CLI (stele watch <space> [--since N]), marked delivered in the parity ledger. Yet no consumer subscribes: pharos's deploy.exp1 shadow loop polls SignedQuery on a ~10-minute ticker; kosmos and corpus read facts on-demand per request. The op-log is an event log, and everyone pages through it by hand.
  • pharos, needing a live plane for its dashboard, built its own (pharos ADR 0008): an in-process hub off its own SQLite audit log, over WebSocket. Some of what it carries (deploy.*) is substrate facts pharos wrote into stele, now re-derived from a private SQLite shadow. pharos did not re-invent a capability in use; it routed around one never adopted.

The direction — consume the substrate's own change stream rather than maintain a divergent shadow — is sound. But the strength of stele's watch guarantees was overstated in the investigation this ADR crystallizes, and a review corrected it. This ADR therefore ratifies the direction and records the prerequisites, rather than asserting the end-state already exists. What watch provides today, read from the wire and SDK:

  • A raw event is WatchEvent{Kind: appended|retracted|replace|lagged, ID: FactID (0 for lagged), Cursor} (sdk/go/watch.go). It is an authorized invalidation/change signal keyed by fact id — it does not carry the fact body or proof. The fact is obtained by a follow-up query/snapshot, or by using derived-query watch (a live claims_query over watch), whose events do carry rows.
  • Initial batch + live tail, and read-authorization applied to each new event.
  • lagged backpressure (a drop signal) and a monotonic op-count cursor — but with the gap caveats in Prerequisites.

Decision

1. The dividing line: is this datum a substrate fact? A substrate fact is a signed, space-placed, op-log-recorded assertion (deploy.v0.*, bindings, grants, runtime desired-state — 0031); its change stream already exists (the op-log) and its live signal is stele watch. A non-fact runtime event is a service's own process/lifecycle state — no fact-id, unsigned, on no op-log, consumed only by that service's own surface (pharos's vault.state, collection.finished); its live signal is that service's own bus. The test is mechanical.

2. For facts, build the live view from stele watch — as an authorized change signal, not a fact-carrying firehose. The pattern is watch(space, since) then, per event, resolve the fact: a raw event delivers {Kind, FactID, Cursor}, so the consumer either (a) queries the affected fact(s) on the signal (read-then-query — the signal says what changed, the query yields the signed body under the substrate's read-gate), or (b) subscribes via derived-query watch, whose events carry the fact rows directly. Either way the consumer's view derives from the substrate's stream, not from a service-owned event log. The "first subscriber is zero new stele code" claim holds for the single-node / sandbox carrier via the existing SDK — but only as signal-then-query or derived-query, not as "fold the signed fact out of raw events" (raw events do not carry it).

3. A service bus may carry a rebuildable projection of facts — but never an authoritative fact log. The rule is not "service buses carry no facts" (that would forbid the very dashboard projection this ADR endorses). It is:

A service may publish a rebuildable, cursor-attributed projection of stele facts — a fan-out for its own surface that any client could reconstruct from the substrate at the carried cursor. It must never be an independently authoritative source for those facts, nor replay-defining (its cursor/ordering is a restatement of stele's, never a second source of record).

pharos ADR-0008's hub is the right tool for genuine non-fact runtime events and stays for them; its deploy.* subset becomes a projection in this exact sense — pharos subscribes to stele watch, resolves the facts, and fans them to its dashboard, holding no authoritative fact log of its own.

4. The live plane is a projection of facts and cannot be more available than they are — honest, not added, fragility. Coupling the live plane to stele (its Postgres, auth, signing) is correct: a per-service log that stays "up" during a stele outage is up but stale — a fork of reality presented as live. A projection that fails with its source is more honest than a cache that silently diverges. This is 0028's single source of truth and 0031's level-triggered, fail-closed posture applied to the read path.

5. Read-authorization is the substrate's, decided once — but "read-gated per new event" ≠ "authorization-current projection." A watch subscriber authorizes for the space and the substrate re-applies the read-gate to each new event (0030 on the read path) — the decisive reason facts belong on watch rather than a re-authorizing per-service bus. But raw watch does not retract already-delivered facts that a later revocation makes unreadable (the grant-retraction event may itself be filtered once the grant is inactive), so a consumer folding raw events can retain rows no longer in its authorized projection. Keeping a projection authorization-current therefore requires one of: derived-query watch (it re-enumerates and diffs the answer set on grant operations, emitting removals), close-and-resnapshot on authority-changing operations, or constraining raw-watch consumers to stable privileged readers whose grants don't churn. The ADR requires authorization-current projections to use one of these — it does not ratify that plain raw watch keeps a projection current by itself.

6. The durable cross-process carrier: LISTEN/NOTIFY is a wake-up over the durable op-log, not a durable feed itself. stele's watch hub is in-process today (a second node on the same op-log does not see the first's appends as events, because the stele_ops(seq BIGSERIAL, op BYTEA) append emits no change-feed). The substrate-native completion is the durable op-log as the feed, with Postgres LISTEN/NOTIFY as the wake-up: each node retains a consumed-sequence watermark, and on a notify (or on startup/reconnect — Postgres delivers notifications only to currently-listening sessions and has a documented post-LISTEN startup race) it catches up from the op-log past its watermark, deduplicates, and folds the new ops into the existing hub → existing serveWatch fan-out. LISTEN/NOTIFY is purely the nudge; correctness rests on the catch-up-from-the-durable-log, never on the notification being delivered. (Logical replication / wal2json is an alternative carrier to evaluate separately.) This touches only the Postgres store backend — not the wire or SDK.

7. Prerequisites in stele before the durable, gap-free end-state is ratified (not assumed). The direction is ratified now; these are stele's to close, recorded here as gates rather than guarantees, because the review showed they are not met today:

  • (a) Facts on the stream. Raw events carry only a FactID. Either the read-then-query / derived-query pattern (§2) is the documented contract, or stele extends the wire to carry the signed fact — and the latter is not "zero new stele code." The ADR adopts §2 as the contract; any wire extension is stele's separate decision.
  • (b) Gap-free backpressure resume. Today on overflow the node drops the current update and the lagged event carries the dropped cursor, which AutoResume adopts as since — so the dropped (and any earlier undelivered buffered) op is skipped. The guarantee "lagged resync, not silent drop" requires stele to make lagged carry a cursor definitely delivered to the client (or the SDK to resume from its last normal event), plus an end-to-end node-overflow + SDK-resume test.
  • (c) Resumable initial batch. Every initial-batch row shares one snapshot-bound cursor and the SDK advances its resume position on any cursor-bearing event, so a disconnect mid-batch skips the remaining rows (no batch-complete marker). The reliable contract is an atomic snapshot → watch(since = snapshot_cursor) handoff, or batch-complete framing/acknowledgement. Until one exists, transparent Watch reconnect is not "gapless" for the initial subscription.
  • (d) Authorization-current projection — per §5: removals on revocation via derived-query watch (or resnapshot / stable-reader), not assumed of raw watch.

A consumer adopting watch before (b)/(c) land must either use derived-query watch where it provides the missing property, or accept and document the gap (e.g. a periodic resnapshot as a safety net) — the ADR does not paper over it.

Tracked as stele issues — and the gate is now mostly closed (resolved 2026-06-21). Prereqs (b)–(d) are RESOLVED: the lagged-resume skip (stele#358) → fixed in stele#363 (lagged now carries the last-delivered cursor, tri-core + an end-to-end resume test); the initial-batch handoff (stele#359) → fixed in stele#364 (a wire caught_up initial-handoff marker, tri-core + conformance vectors); the authorization-current projection (stele#360) → resolved in stele#362 (the read-gated-forward contract is now documented; derived-query watch remains the authorization-current path). The one remaining gate is the durable cross-process carrier, stele#361 (§6) — required for the multi-node end-state but not for a single-node / sandbox first subscriber (the in-process hub already works there), so the deploy.exp1 Phase-1 proof is unblocked now. Seam-home note (logos-db extraction, ADR 0034): once logos-db owns the log, the tail/subscribe + durable change-notify surface is a logos-db-seam surface — stele-the-shell re-exposes it rather than owning it, and the #361 carrier plan is an input to the seam's notify primitive (designed once; the frozen seam-types draft's §2b pins the contract: an engine-owned total durable order — linearizable append, contiguous cursor — with proof byte-parity keyed on canonical bytes, never log position). The consumer-facing ruling here — subscribe to the substrate's watch for facts — is unchanged; only the surface's home migrates.

8. Ownership split (cross-cutting frame here; mechanics downstream). This ADR holds the fact/non-fact dividing line, the rebuildable-projection-not-authoritative-log rule, the projection-availability posture, and the staging direction. The mechanics land in the owning repos' own ADRs: stele — prerequisites (a)–(d), the durable wake-up-over-op-log carrier (the consumed-seq + catch-up + dedup), and the delivery-semantics stance (at-least-once + idempotent-by-fact-id today; exactly-once is a tracked open item); pharos — re-scoping its ADR-0008 bus to non-fact events and turning deploy.* into a watch projection, and repointing the deploy.exp1 loop's poll at watch. Adopt watch for a consumer that has fact-derived state it currently polls or caches — not for a decision that needs per-request freshness at the moment of decision (kosmos/corpus authorization checks evaluate the record then; they stay on-demand). Don't subscribe a permission check.

Consequences

  • The watch/reconcile loop gets its watch — once the prerequisites land. Reconcilers can become level-triggered in fact rather than polling on a timer; the cheap proof (deploy.exp1's poll → watch, single-node, signal-then-query or derived-query) needs no stele change, while the durable, gap-free, multi-node end-state is gated on §7.
  • A redundancy is removed without losing a tool, and without over-claiming. pharos's bus stays for vault.state / collection.finished; deploy.* becomes a rebuildable projection of stele watch. No service maintains an authoritative fact log shadowing the op-log.
  • The honest cost is named, not hidden. The cross-process carrier is a real stele build (durable catch-up over LISTEN/NOTIFY, not LISTEN/NOTIFY alone), and three transport guarantees (facts on the stream, gap-free resume, authorization-current projection) are prerequisites, not present capabilities. Ratifying the direction while gating the guarantees is the point of this revision.
  • This is a frame, not a wire format. It ratifies where the live plane lives, the dividing line, the projection rule, and the prerequisites; the LISTEN/NOTIFY+catch-up mechanics, the wire/resume fixes, and the pharos projection are downstream in stele's and pharos's own ADRs.
  • Lineage. Builds on 0016 / 0031 (the watch side + trigger of their loop); preserves 0010 / 0028 (one record, one order); applies 0030's decide-once to the read path; and re-scopes pharos ADR 0008 rather than superseding it. Mints no new layer and takes no prod authority.

Open questions (recorded, not decided)

  • Read-then-query vs wire-carried facts. Whether stele keeps raw watch as a FactID signal (the §2 contract) and consumers query/derived-watch for bodies, or extends the wire so events carry the signed fact — a stele wire decision with a real cost.
  • Delivery semantics. at-least-once + idempotent-by-fact-id today (exactly-once is a tracked stele open item); whether the durable carrier tightens it.
  • Change-feed mechanism. LISTEN/NOTIFY-as-wake-up + op-log catch-up vs logical replication / wal2json for the durable carrier — a stele mechanics decision.
  • seq ↔ op-count contract. The reconciliation between the Postgres seq watermark and the wire op-count cursor (coincident for the single-writer log today).

The Bitspark constellation — how the systems are built and relate.

GitHub