docs/RemoteProxy.md

@@ -0,0 +1,264 @@
+# Remote Proxy / Satellite Design Review
+
+Design notes for serving configured domains (rp-ids) from a **remote** paskia
+instance instead of the local database, so that latency-sensitive checks
+(`/auth/api/forward`, `/auth/api/validate`) answer in ~1 ms even when the auth
+server is on another continent. Example: `app2.example.com` runs on our local
+host and needs fast local checks, while `app1.example.com` and
+`auth.example.com` run far away — all sharing `example.com` as rp-id.
+
+Client applications that previously used `https://auth.example.com` as their
+auth backend (forward-auth checks) only repoint to the local satellite
+(`http://127.0.0.1:4401`); both remain usable interchangeably, and the
+satellite ultimately uses `auth.example.com`.
+
+Status: **proposal, not implemented.** Revision 2: no `db.data()` magic;
+explicit domain-aware stores; auth host mandatory for remote domains.
+
+## Facts from the architecture review
+
+- The whole DB is already in RAM (msgspec structs, Kanta JSONL for
+  persistence). Reads never touch disk — the remote is fast, so the latency
+  problem is purely network RTT. The job is replicating *read state*, not
+  reimplementing a database.
+- `/auth/api/forward` (the hot path) is a **pure read**:
+  `session_ctx(cookie, host)` → perm/max_age checks → 204 + `Remote-*`
+  headers. No DB write, no cookie mutation.
+- Sessions are host-bound and expire by `validated + SESSION_LIFETIME`;
+  expiry is enforced by background deletion (`db/lifecycle.py`), and
+  `Remote-Session-Expires` is already present in the 204 response headers.
+- `/auth/api/validate` is the only renewal write (throttled to ≥5 min
+  spacing); it can set a fresh cookie.
+- Login/register/remote-auth run over WebSockets with state in process RAM
+  (PoW challenges, `remoteauth.instance`, `authcode` 60 s exchange codes).
+  These cannot be cached or replicated.
+- The sync feed carries no usable secrets: sessions are keyed by
+  `hash_secret` output, credentials carry public keys only, and the OIDC
+  signing key is never exported. Knowing a stored session hash does not let
+  you authenticate.
+- Restricted paths (`/auth/api/admin/`, `/auth/api/user/`, `/auth/ws/`) are
+  already refused off a domain's own auth host
+  (`fastapi/auth_host.py:is_restricted_path`), and UI paths already redirect
+  to the auth host. With an auth host configured for a remote domain, the
+  satellite never serves profile, admin or WS traffic for it — browsers and
+  WebSockets go directly to the remote auth host (the WS dispatch rules in
+  `fastapi/dispatch.py` already allow an origin domain's own auth host).
+
+## Option A — caching HTTP reverse proxy
+
+A thin proxy that caches `/auth/api/forward`, `/check`, `/user-info`,
+`/settings` responses keyed by `(Host, cookie, query)` with
+`TTL = min(configured TTL, Remote-Session-Expires − now)`. Everything else is
+forwarded verbatim; WebSockets are tunneled; `/logout` and session-scoped
+mutations are intercepted to evict cache entries. Optionally
+stale-while-revalidate so remote-side changes propagate within a TTL without
+adding latency to any request.
+
+**Option B** is A plus a small remote addition: a WebSocket change feed
+(session deleted, user/role/permission changed) so the proxy evicts within
+one network RTT instead of waiting for the TTL.
+
+- Remote changes: **none** for A; one additive, RAM-only endpoint for B.
+- Staleness: TTL-bounded for anything that bypasses the proxy (admin edits,
+  logout-all from another device) in A; ~1 RTT in B.
+- The proxy needs no credentials of its own — every cacheable request is
+  authenticated by the end user's cookie, forwarded on a miss.
+
+## Option D — satellite instance with per-domain RAM replica (chosen direction)
+
+The same paskia codebase, with two kinds of domains in the existing
+multi-domain config:
+
+- **Local domains** — exactly as today, backed by the local Kanta DB.
+- **Remote domains** — `DomainConfig` gains `remote: {url, token, cache_ttl,
+  refresh_interval}`. This is the only new persisted state, and it lives in
+  the **satellite's** DB. The remote's DB gets nothing new. A remote domain
+  **must** have an auth host configured (the remote's); this is what makes
+  the simplifications below possible.
+
+For a remote domain the satellite holds a **RAM-only read replica** of the
+remote's tables (users, orgs, roles, permissions, credentials, sessions) —
+another plain `DB` struct instance, never opened by Kanta, rebuilt from the
+remote on startup and kept fresh by the sync channel plus snooping on
+proxied traffic.
+
+### No `db.data()` magic: explicit per-request stores
+
+The first draft proposed making the global `db.data()` accessor return a
+replica based on the `current_domain` contextvar. Rejected: a global
+accessor whose meaning shifts under the caller is exactly the kind of magic
+that breeds bugs in no-request-context code (background jobs, OIDC notify,
+CLI). Instead:
+
+- The runtime `Domain` object (`domains.py`) gets a `store` attribute: a
+  `DB` instance. For local domains it **is** the global `operations._db`;
+  for remote domains it is the replica. Registry build / remote-manager
+  startup attaches it.
+- Endpoints resolve the store once, explicitly, from the request:
+  `request.state.domain.store` (dispatch already puts the domain there), and
+  pass it down as an ordinary parameter. No global accessor changes
+  meaning; local domains observe zero functional change.
+
+Refactor needed to make the hot path store-explicit (small, all in the
+"manageable core"):
+
+- `DB.session_ctx()` (`db/structs.py`) currently leans on struct convenience
+  properties (`s.user`, `user.role`, `role.org`, `s.credential`,
+  `org.permissions`) that internally call the global `db.data()`. Rewrite it
+  to index `self.*` tables directly. This is a pure correctness fix — it
+  makes `session_ctx` honest about which DB it reads.
+- `authsession.session_ctx`, `permutil.session_context`, `authz.verify`,
+  `sessionutil.check_session_age` gain an explicit `store: DB` parameter
+  (or are called with one) instead of reaching the global.
+- `userinfo.build_user_info` takes a `store` and lists
+  `store.sessions`/`store.credentials` filtered by user instead of the
+  `user.sessions` / `user.credentials` convenience properties.
+- `userinfo.build_session_context`, `api._remote_headers`,
+  `authz.auth_error_content` already operate on the `SessionContext` alone —
+  no change.
+- Struct convenience properties (`User.sessions`, `Session.user`, …) keep
+  using the global `db.data()`; they are only used by local-domain code
+  (admin, profile, CLI) and by Kanta maintenance. Remote-domain request
+  paths simply never call them.
+
+### Endpoint categorization for remote domains
+
+**1. True special handling (remote-aware logic):**
+
+| Endpoint | Handling |
+|---|---|
+| `POST /auth/api/validate` | Read from replica; apply the throttled refresh to the replica and write it behind as `session_refresh{key, validated, ip, ua}` over the sync WS; set the renewed cookie locally (same secret, same rules as today) |
+| `POST /auth/api/logout` | Forward to remote with the user's cookie; evict the session from the replica immediately (optimistic); the sync event confirms |
+| `POST /auth/api/set-session` | Forward code redemption to the remote (the 60 s exchange code lives in its RAM); relay the `Set-Cookie`; the new session arrives via sync event (created during the login ceremony) |
+| `GET /auth/api/token-info` | Proxy to remote (reset tokens exist only there) |
+| `/auth/oidc/token`, `/userinfo`, `/keys`, `/backchannel-logout` | Proxy to remote (signing key and OIDC sessions stay there); `/keys` is cacheable |
+| Avatar `GET .../profile.webp` | Proxy to remote (storage may be remote-local); not latency-critical |
+
+**2. Store-explicit, no functional change** (the "alike db magic" group —
+they read the database but only through the store they are handed):
+
+- `GET /auth/api/forward` — the hot path, pure read from `store`.
+- `POST /auth/api/validate` — the read/verify half (category 1 covers the
+  refresh half).
+- `GET /auth/api/check` — reads users/roles/orgs/permissions from `store`.
+- `GET /auth/api/user-info` — `build_user_info(store, …)`.
+
+**3. Untouched — no knowledge of remoting:**
+
+- `GET /auth/api/settings` — domain config only; already reports
+  `auth_host`/`auth_site_url`, which is how frontends learn to open
+  WebSockets on the remote auth host directly.
+- `/.well-known/openid-configuration`, `/.well-known/webauthn` — derived
+  from Host and local domain config.
+- Frontend statics, `/auth/restricted/iframe` HTML, reset-token pages —
+  static files; their API calls land in categories 1–2.
+- `auth_host.redirect_middleware` — already redirects UI to the remote auth
+  host and 404s restricted paths off it; with auth host mandatory this is
+  exactly the desired behavior, unchanged.
+- The entire admin app, user-profile app, `/auth/ws/*`, `/auth/remote-auth/*`
+  — reachable only on the domain's auth host, which for remote domains is
+  the remote itself; the satellite never executes this code for remote
+  domains.
+- Dispatch middleware, background cleanup of the local DB, CLI, bootstrap.
+- All local-domain handling of every endpoint.
+
+### The sync channel (the one remote addition, RAM-only)
+
+New endpoint on the remote, e.g. `wss://…/auth/api/sync/ws`, plus an
+emission hook in `db/operations.py` (publish after each commit; no-op when
+no peer is connected):
+
+- **Remote → satellite**: `hello{generation, protocol_version}` → full
+  snapshot (chunked msgspec; these tables are tiny — 10k sessions ≈ 2 MB) →
+  then `event{seq, op: upsert|delete, table, key, payload}`. The remote
+  keeps a bounded event ring buffer in RAM; on reconnect the satellite sends
+  its last `seq` and either gets a replay or a fresh snapshot.
+- **Satellite → remote**: `session_refresh` write-behind; optionally
+  `session_deleted` as belt-and-braces alongside proxied logout.
+- **Authentication**: the satellite presents a token the remote reads from
+  env/CLI (e.g. `PASKIA_SYNC_TOKENS`) — nothing lands in the remote's
+  database. mTLS client certs are an alternative.
+
+Freshness hierarchy, worst case last:
+
+1. Changes made **through** the satellite: immediate (optimistic eviction on
+   logout, snooped `Set-Cookie`/login results).
+2. Changes made **directly on the remote** (admin on another device,
+   logout-all): WS event, ~1 network RTT.
+3. WS down: per-entry TTL lazy revalidation + periodic full snapshot; on
+   reconnect, replay or re-sync. Fail-open (the replica enforces
+   `validated + EXPIRES` itself, with its own little sweeper mirroring
+   `cleanup_expired`) vs fail-closed is configurable per domain.
+
+Deliberately out of scope:
+
+- **Running WebAuthn ceremonies on the satellite** — duplicates the
+  security-critical verification path and breaks centralized sign-count
+  clone detection; browsers talk to the remote auth host directly anyway.
+- **OIDC signing on the satellite** — key material never leaves the remote.
+- **Admin/profile functionality for remote domains** — deferred to the
+  remote auth host; this is what keeps the satellite's surface small.
+
+## D vs. A(+B): what the read-only local state buys
+
+Semantics, not just speed:
+
+- **Full `SessionContext` locally.** A answers the exact byte-response it
+  once saw; D *computes* the answer. New `perm`/`max_age`/`public` query
+  combinations never seen before are served locally by D, while A must go
+  upstream on every cache-key miss. D caches the *domain model*, so derived
+  answers (effective permissions per host, `max_age` against
+  `credential.last_used`, per-request `Remote-*` header composition) are
+  correct without having been witnessed before.
+- **One invalidation model.** In A, invalidation rules are hand-built per
+  endpoint (query-key mapping, cookie re-keying on renew, 401 variants,
+  `/user-info` vs `/forward` overlap). In D, events mutate the replica
+  (upsert/delete by table+key) and every endpoint becomes consistent at
+  once — including endpoints added in the future.
+- **Degradation behaves like a real instance.** With the remote down, D
+  keeps serving a coherent auth service from the replica (expiry enforced
+  locally). A serves a pile of unrelated cached responses with gaps wherever
+  the cache was cold.
+- **Multi-domain uniformity.** D is a property of a domain in the existing
+  registry: local rp-ids and remote rp-ids coexist, and the same instance
+  genuinely *is* the auth server for its local domains. A is a separate
+  component bolted in front of specific URLs.
+- **User simplicity.** D is configured once as domain config
+  (`remote.url` + token); A needs deployment and key-mapping discipline per
+  frontend application.
+
+## What D costs
+
+- **A store-explicitness refactor in the core.** `DB.session_ctx` must stop
+  leaning on global-accessing convenience properties; the verify/userinfo
+  helper chain gains a `store` parameter. Small and mechanical, but it
+  touches the security-critical read path, so it wants careful review and
+  tests proving local-domain behavior is byte-identical. (The rejected
+  alternative — a contextvar-dependent `db.data()` — avoided this refactor
+  at the price of a global whose meaning shifts under the caller.)
+- **A sync protocol to design and version.** Snapshot + sequenced events +
+  ring-buffer replay + reconnect reconciliation, plus a protocol-version
+  handshake because remote and satellite releases can drift in
+  `SessionContext` semantics. A needs no protocol at all.
+- **A trusted satellite host.** The replica holds everything the remote DB
+  has except the OIDC key (public keys, hashed session keys, user data) in
+  RAM. The host must be as trusted as the remote. A caches only what traffic
+  it saw, in the shape it saw it.
+- **Remote code, albeit additive.** D requires the sync endpoint and commit
+  hooks on the remote (RAM-only, inert when unused, token-gated). A requires
+  zero remote changes; B requires the same endpoint in a thinner form.
+- **Replica housekeeping** — its own expiry sweeper, write-behind ordering,
+  optimistic eviction vs. event confirmation. A's failure modes are dumber
+  and narrower.
+
+## Summary
+
+A (+B) is the right tool when the goal is "make forward-auth fast in front
+of an untouched server" — small, deployable anywhere, byte-compatible. D is
+the right tool when the satellite should *be* a paskia instance for its
+remote domains: one consistency model, correct answers for un-cached query
+shapes, graceful degradation, and per-domain mixing with local rp-ids. With
+the auth host made mandatory for remote domains, D's surface shrinks to four
+locally served read endpoints, a handful of proxied mutations, an explicit
+`store` parameter in the core read path, and one RAM-only sync endpoint on
+the remote.

paskia/authsession.py

@@ -18,14 +18,18 @@ from paskia.db.structs import ResetToken
 from paskia.util import hostutil
 
 if TYPE_CHECKING:
-    from paskia.db import ResetToken
+    from paskia.db import DB, ResetToken
 
 EXPIRES = SESSION_LIFETIME
 
 
-def session_ctx(auth: str, host: str | None = None):
+def session_ctx(auth: str, host: str | None = None, store: DB | None = None):
-    """Get session context with normalized host."""
+    """Get session context with normalized host.
-    return db.data().session_ctx(auth, hostutil.normalize_host(host))
+
+    store defaults to the local database; remote-domain request paths pass
+    their domain's replica explicitly.
+    """
+    return (store or db.data()).session_ctx(auth, hostutil.normalize_host(host))
 
 
 def expires() -> datetime:

paskia/db/structs.py

@@ -728,17 +728,21 @@ class DB(msgspec.Struct, dict=True, omit_defaults=False):
         if s.host != host:
             return None
 
+        # Look up via this instance's own tables: a DB must be
+        # self-contained so that read replicas work unchanged.
         try:
-            user = s.user
+            user = self.users[s.user_uuid]
-            role = user.role
+            role = self.roles[user.role_uuid]
-            org = role.org
+            org = self.orgs[role.org_uuid]
-            credential = s.credential
+            credential = self.credentials[s.credential_uuid]
         except KeyError:
             return None
 
         # Effective permissions: role's permissions that the org can grant,
         # filtered by domain restriction
-        org_perm_uuids = {p.uuid for p in org.permissions}
+        org_perm_uuids = {
+            p.uuid for p in self.permissions.values() if org.uuid in p.orgs
+        }
 
         effective_perms = []
         for perm_uuid in role.permission_set:

paskia/domains.py

@@ -90,6 +90,7 @@ class Domain:
         self.config = config
         self.site_url = site_url
         self.site_path = site_path
+        self._store = None
         self.passkey = Passkey(
             rp_id=rp_id,
             rp_name=config.rp_name,
@@ -97,6 +98,23 @@ class Domain:
             related_origins=[origin_url(k) for k in related],
         )
 
+    @property
+    def store(self):
+        """The DB instance this domain's request paths read.
+
+        Defaults to the local database; remote domains get their read
+        replica attached by the sync client at startup.
+        """
+        if self._store is not None:
+            return self._store
+        from paskia import db
+
+        return db.data()
+
+    @store.setter
+    def store(self, value) -> None:
+        self._store = value
+
     @property
     def rp_name(self) -> str:
         return self.passkey.rp_name

paskia/fastapi/api.py

@@ -97,6 +97,11 @@ def _parse_perm(perm: list[str]) -> list[tuple[str, ...]]:
         raise HTTPException(status_code=400, detail=str(e))
 
 
+def _store(request: Request):
+    """The dispatched domain's data store (local DB or read replica)."""
+    return request.state.domain.store
+
+
 @app.post("/validate")
 async def validate_token(
     request: Request,
@@ -114,6 +119,7 @@ async def validate_token(
             perm_groups,
             host=request.headers.get("host"),
             max_age=max_age,
+            store=_store(request),
         )
     except HTTPException:
         # Global handler will clear cookie if 401
@@ -162,7 +168,7 @@ async def check_user(
 
     No session cookie is read or written. Caller authentication is not required.
     """
-    data = db.data()
+    data = _store(request)
     try:
         u = data.users[user_uuid]
         role = u.role
@@ -267,6 +273,7 @@ async def forward_authentication(
             perm_groups,
             host=request.headers.get("host"),
             max_age=max_age,
+            store=_store(request),
         )
         _set_log_extra(request, forwarded, ctx.session.key)
         remote_headers = _remote_headers(ctx)
@@ -329,7 +336,7 @@ async def api_user_info(
             detail="Authentication required",
             mode="login",
         )
-    ctx = session_ctx(auth, request.headers.get("host"))
+    ctx = session_ctx(auth, request.headers.get("host"), store=_store(request))
     if not ctx:
         raise authz.AuthException(
             status_code=401,
@@ -346,6 +353,7 @@ async def api_user_info(
             session_key=ctx.session.key,
             request_host=request.headers.get("host"),
             ctx=ctx,
+            store=_store(request),
         )
     )
 

paskia/fastapi/authz.py

@@ -62,6 +62,7 @@ async def verify(
     match: Callable | None = None,
     host: str | None = None,
     max_age: str | None = None,
+    store=None,
 ):
     """Validate session token and optional list of required permissions.
 
@@ -69,6 +70,9 @@ async def verify(
     scope patterns (OR semantics within a group). All entries must be
     satisfied (AND semantics).
 
+    store defaults to the local database; remote-domain request paths pass
+    their domain's replica explicitly.
+
     Returns the session context.
 
     Raises AuthException on failure with metadata for UI rendering.
@@ -80,7 +84,7 @@ async def verify(
             mode="login",
         )
 
-    ctx = await permutil.session_context(auth, host)
+    ctx = await permutil.session_context(auth, host, store=store)
     if not ctx:
         raise AuthException(
             status_code=401,

paskia/util/permutil.py

@@ -129,8 +129,8 @@ def has_all_scopes_groups(scopes: set[str], groups: Sequence[Sequence[str]]) ->
     return all(group_satisfied(scopes, g) for g in groups)
 
 
-async def session_context(auth: str | None, host: str | None = None):
+async def session_context(auth: str | None, host: str | None = None, store=None):
     if not auth:
         return None
     normalized_host = normalize_host(host) if host else None
-    return session_ctx(auth, normalized_host)
+    return session_ctx(auth, normalized_host, store=store)

paskia/util/userinfo.py

@@ -41,26 +41,37 @@ async def build_user_info(
     session_key: str,
     request_host: str | None,
     ctx: SessionContext | None = None,
+    store=None,
 ) -> ApiUserDetail:
-    """Build user info struct for authenticated users."""
+    """Build user info struct for authenticated users.
-    user = db.data().users[user_uuid]
+
+    store defaults to the local database; remote-domain request paths pass
+    their domain's replica explicitly.
+    """
+    data = store or db.data()
+    user = data.users[user_uuid]
     normalized_host = hostutil.normalize_host(request_host)
 
+    user_sessions = [s for s in data.sessions.values() if s.user_uuid == user_uuid]
+    user_credentials = [
+        c for c in data.credentials.values() if c.user_uuid == user_uuid
+    ]
+
     sessions = {
         s.key: ApiUserSession.from_db(
             s,
             current_key=session_key,
             normalized_host=normalized_host,
         )
-        for s in user.sessions
+        for s in user_sessions
     }
 
     return ApiUserDetail(
         user=ApiUser.from_db(user, avatar_url=avatar.avatar_browser_url(user.uuid)),
-        credentials={c.uuid: c for c in user.credentials},
+        credentials={c.uuid: c for c in user_credentials},
         aaguid_info={
             k: ApiAaguidInfo(**v)
-            for k, v in aaguid.filter(c.aaguid for c in user.credentials).items()
+            for k, v in aaguid.filter(c.aaguid for c in user_credentials).items()
         },
         sessions=sessions,
         permissions={p.uuid: ApiPermission.from_db(p) for p in ctx.permissions}