Substratum.cloud

Appearance

Architecture Decision Record (ADR) 28: Receipt Sync Queue and Grantee Access Removal ​

Status: Accepted
Date: 2026-05-19
Last Updated: 2026-05-24

Glossary (acronyms in full, with abbreviation in brackets) ​

  • Architecture Decision Record (ADR)
  • Access Control List (ACL)
  • Authenticated Transfer Protocol (AT Protocol)
  • Content Identifier (CID)
  • Decentralized Identifier (DID)
  • Merkle Search Tree (MST)
  • Namespaced Identifier (NSID)
  • Open Authorization (OAuth)
  • Personal Data Server (PDS)

Context ​

Historical lineage: detached signatures → PDS commits ​

Early design (ADR 11, ADR 21) assumed passport receipts would be signed JSON artifacts verified by the gateway or swarm (for example a detached provenanceSignature field), optionally produced in the browser via WebAssembly or on home-base with keys that never reach the cloud.

Security review and ADR 27: Zero Trust PDS-Based Provenance replaced that model:

  • The gateway must not hold user #atproto signing keys or unilaterally attest uploads.
  • Mesh authorization must follow owner-repo cloud.substratum.passport.receipt records on the PDS, verified via MST commits—not gateway attestation or standalone JSON signatures.

ADR 21 remains useful for cloud.substratum.* lexicons and ref-only records. Its provenanceSignature requirement is superseded; do not implement detached receipt signatures for new work. Trust is PDS repo commits (ADR 27).

The async boundary exposed by ADR 27 ​

Ingress today maintains a catalog in PostgreSQL (Drive entries, drive_entry_access_control, passport index rows) while the retrieval layer enforces object-level ACL from the owner PDS (get_record on cloud.substratum.passport.receipt).

Synchronous issue_receipt couples HTTP handlers to owner OAuth (get_authenticated_client for receipt.owner_did). That creates failures when:

  • The file owner has a gateway session but no stored PDS OAuth session.
  • A grantee legitimately requests self-removal at the HTTP layer but cannot write the owner repo (grantee OAuth only covers the grantee PDS).

ADR 11 addresses HTTP ↔ libp2p concurrency via in-process mpsc actors; it does not address HTTP ↔ PDS durability or multi-process producers.

Producers: gateway and home-base ​

ADR 10 positions apps/gateway (cloud) and apps/home-base (local daemon) as separate binaries sharing core, auth, and retrieval, not necessarily resolution (PostgreSQL). Both must participate in converging catalog intent onto owner-repo receipts.

Decision ​

We adopt a two-plane authorization model with a shared receipt-sync pipeline and a two-phase grantee removal lexicon.

1. Two planes of truth ​

PlaneStoreAuthority forMay lag
CatalogPostgreSQL (gateway); optional local DB (home-base, future)File-explorer UX, HTTP APIs, RLSAhead of PDS during sync
MeshOwner PDS cloud.substratum.passport.receiptGetBlock, pin/replication ACL (ADR 27)Behind catalog until sync completes

Invariant: The swarm never treats Postgres ACL alone as sufficient for mesh reads. The catalog may expose receipt_sync status: pending, synced, or failed.

Operated PDS hosting (volume, Spaces, backups) and AT Protocol plane separation (gateway as product read plane): ADR 41.

2. Receipt sync queue (transactional outbox) ​

crates/passport-sync is implemented for the gateway with:

  • Outbox column event (not operation) holding business domain event discriminants—user/product actions, not SQL or worker verbs (see ADR 30 §3). Serialized receipt or removal intent and idempotency key (owner_did, subject_id, acl_version_or_hash):
    • file.uploaded / file.access_changed — target names for upload finalize vs owner ACL share/patch/inherit; shipped as owner.receipt.published_requested (OwnerReceiptPublishedRequested) until migrated
    • access.removal_acknowledged — target for grantee self-removal owner convergence (updates owner passport.receipt and writes passport.accessRemovalAck in one ReceiptSyncWorker job); shipped as owner.access_removal_acknowledged (OwnerAccessRemovalAcknowledged)
    • access.removal_requested — optional future outbox snapshot only; grantee intent is inline in ingress today (create_access_removal_request). GranteeAccessRemovalRequested / grantee.access_removal_requested is defined in code but not enqueued from handlers.
  • OutboxPort adapters:
    • Gateway: PostgreSQL receipt_sync_outbox rows committed in the same transaction as catalog ACL updates.
    • Home-base: local outbox (for example SQLite) and/or publish to a shared broker when cloud relay is available (ADR 24) — not yet shipped.
  • ReceiptSyncWorker: restores owner OAuth, calls com.atproto.repo.createRecord / updates cloud.substratum.passport.receipt on the owner repo, updates catalog receipt_cid and receipt_sync = synced, with retries and poison handling.

HTTP handlers enqueue instead of blocking on PDS; they return explicit sync state to clients.

Per-owner_did ordering: workers serialize PDS writes per owner repository to avoid out-of-order receipt commits.

Consumer leadership: at most one active worker per owner_did when both gateway and home-base are online, unless idempotent PDS writes are proven sufficient.

ADR 11’s mpsc swarm mailbox remains in-process command routing; the receipt-sync outbox is a durable domain boundary between catalog intent and PDS effects.

3. Grantee access removal — two repos, two roles ​

Do not duplicate the same record bytes on both PDSes. Use complementary record types:

Phase A — Intent (grantee PDS) ​

New lexicon: cloud.substratum.passport.accessRemovalRequest (see libs/lexicons/defs/).

  • Writer: grantee via their own OAuth (createRecord on grantee repo).
  • Meaning: cryptographically attributable request to revoke the grantee’s access to a specific asset under a named owner.
  • Trust: PDS MST on the grantee repo (ADR 27 model). No detached provenanceSignature.

Required fields: version, granteeDid, ownerDid, assetCid, receiptUri (or equivalent at:// pointer to the owner receipt), requestedAt.

Deterministic rkey (for example derived from ownerDid + assetCid) for idempotent writes.

Phase B — Effect (owner PDS) ​

  • Writer: ReceiptSyncWorker using owner OAuth (or home-base when the owner device is active).
  • Action: update cloud.substratum.passport.receipt on the owner repo so access_control no longer includes the grantee.
  • accessRemovalAck: cloud.substratum.passport.accessRemovalAck on the owner repo — written by ReceiptSyncWorker after the owner receipt ACL update when removal_request_uri is present in the outbox payload (ack.rs). Mesh ACL follows the updated receipt only (ack is audit, not a third ACL plane).

Grantee HTTP self-removal (PATCH /api/v1/drives/{drive_id}/nodes/access-control in ingress): validate grantee may only remove themselves → Phase A: write grantee accessRemovalRequest on the grantee PDS when OAuth is available → Phase B: enqueue access.removal_acknowledged (OwnerAccessRemovalAcknowledged) → update catalog with receipt_sync = pending until the worker updates the owner receipt and optional accessRemovalAck.

Mesh behavior (gateway default) ​

Owner receipt (primary): mesh GetBlock / Pin / inbound replication consult cloud.substratum.passport.receipt on the owner repo via LocalPassportAccessControlResolver.

Grantee self-deny (Phase 2b — implemented): when requester_did != owner_did and the owner receipt would allow access, the resolver loads cloud.substratum.passport.accessRemovalRequest on the grantee repo at deterministic rkey (ownerDid + assetCid). If the record matches ownerDid, assetCid, and granteeDid, access is denied even if the owner receipt still lists the grantee. Lookup uses a 2s timeout; skipped when the owner receipt already denies. Errors/timeouts fail open to the owner-receipt decision.

Catalog/UI still updates immediately on grantee PDS write; mesh deny follows grantee intent without waiting for owner receipt convergence.

4. Relationship to prior ADRs ​

ADRRelationship
11WASM/native signing may still publish records from home-base; mesh verification does not use ADR 11 detached checks.
21Keep cloud.substratum.* NSIDs and ref-only records; ignore superseded provenanceSignature rules.
27Owner-repo receipts remain mesh authority; this ADR defines how catalog changes converge on that model.
10Shared worker crate; gateway uses Postgres outbox, home-base uses local/broker adapter.

Consequences ​

Positive ​

  • Decouples who may request ACL changes from who can commit owner-repo proofs.
  • Grantee self-removal no longer requires owner OAuth at click time for intent (grantee PDS write).
  • Gateway and home-base share one event/worker contract; fits modular monolith decomposition.
  • Aligns product UX with honest sync state instead of opaque 403s when owner PDS sessions are cold.

Negative ​

  • Eventual consistency: catalog and mesh ACL can diverge until the worker succeeds; must be visible in UI and documented for support.
  • Operational complexity: outbox monitoring, retries, poison queues, per-owner ordering, and multi-consumer leadership.
  • Grantee-repo mesh checks add one XRPC round-trip for non-owner requesters (2s timeout); skipped when owner receipt already denies.
  • Orphan removal requests if the owner never syncs; requires TTL or policy for stale intents (see below).

Orphan removal requests and catalog cleanup (policy) ​

When a grantee self-removes but the owner never syncs an updated receipt (owner.access_removal_acknowledged jobs retry or poison):

ArtifactImmediate behaviorDocumented TTL / ops policy
Catalog (drive_*_access_control, shared-with-me)Grantee removed on successful HTTP PATCHNo automatic rollback; support uses receipt_sync / outbox metrics
Grantee PDS accessRemovalRequestWritten on Phase A when grantee OAuth is available90 days — eligible for future grantee-repo GC / ops purge of stale intents (not implemented in v1)
Outbox owner.access_removal_acknowledgedRetries with backoff; poison after max_attempts30 days stuck/poison — alert via metrics; manual replay or discard per runbook
MeshMay still allow until owner receipt converges (Phase 1) or grantee-repo deny (Phase 2b)N/A

Ingress Phase 2c limits abuse volume: PATCH …/nodes/access-control is rate-limited per authenticated DID and rejects access_control longer than MAX_PASSPORT_GRANTEES (default 100). See gateway env ACL_PATCH_RATE_LIMIT_MAX / ACL_PATCH_RATE_LIMIT_WINDOW_SECS.

Implementation notes (gateway) ​

  • Ingress: enqueue-only on the HTTP path (receipt_sync_outbox in the same transaction as catalog ACL updates); no inline owner createRecord.
  • Worker: in-process ReceiptSyncWorker in apps/gateway when RECEIPT_SYNC_ENABLED (default on); RECEIPT_SYNC_POLL_MS controls poll interval.

Remediation status (2026-05-20) ​

Gateway Phase A/B pipeline is implemented (Postgres outbox, ReceiptSyncWorker, grantee accessRemovalRequest, owner receipt convergence). Tracked in Share known issues and the share remediation plan.

ItemStatus
Grantee E2E (grantee sees shared)Required gate — not verified green in docs
accessRemovalAck on owner repo (with removal_request_uri in outbox)Implemented in ReceiptSyncWorker
Replication-scoped pin JWT (Phase 2e)Implemented — swarm security gaps §3
Mesh grantee-deny (Phase 2b)Implemented
Ingress share-spam guards (Phase 2c)Planned
receipt_sync on GET /api/v1/shared-with-mePlanned (Phase 1, if E2E needs it)
Home-base outbox, mobile pds_urlDeferred (Phase 3)

Out of scope (this ADR) ​

  • AT Protocol delegation for third-party writes on the owner repo (future ADR).
  • Phase 3 deferred: home-base outbox adapter and broker relay; orphan-removal TTL when owner never syncs; mobile FFI pds_url for home-base PDS routing — see Share known issues § Phase 3.