# Remote Proxy / Satellite Design Review

Design notes for serving configured domains (rp-ids) from a **remote** paskia
instance instead of the local database, so that latency-sensitive checks
(`/auth/api/forward`, `/auth/api/validate`) answer in ~1 ms even when the auth
server is on another continent. Example: `app2.example.com` runs on our local
host and needs fast local checks, while `app1.example.com` and
`auth.example.com` run far away — all sharing `example.com` as rp-id.

Client applications that previously used `https://auth.example.com` as their
auth backend (forward-auth checks) only repoint to the local satellite
(`http://127.0.0.1:4401`); both remain usable interchangeably, and the
satellite ultimately uses `auth.example.com`.

Status: **proposal, not implemented.** Revision 2: no `db.data()` magic;
explicit domain-aware stores; auth host mandatory for remote domains.

## Facts from the architecture review

- The whole DB is already in RAM (msgspec structs, Kanta JSONL for
  persistence). Reads never touch disk — the remote is fast, so the latency
  problem is purely network RTT. The job is replicating *read state*, not
  reimplementing a database.
- `/auth/api/forward` (the hot path) is a **pure read**:
  `session_ctx(cookie, host)` → perm/max_age checks → 204 + `Remote-*`
  headers. No DB write, no cookie mutation.
- Sessions are host-bound and expire by `validated + SESSION_LIFETIME`;
  expiry is enforced by background deletion (`db/lifecycle.py`), and
  `Remote-Session-Expires` is already present in the 204 response headers.
- `/auth/api/validate` is the only renewal write (throttled to ≥5 min
  spacing); it can set a fresh cookie.
- Login/register/remote-auth run over WebSockets with state in process RAM
  (PoW challenges, `remoteauth.instance`, `authcode` 60 s exchange codes).
  These cannot be cached or replicated.
- The sync feed carries no usable secrets: sessions are keyed by
  `hash_secret` output, credentials carry public keys only, and the OIDC
  signing key is never exported. Knowing a stored session hash does not let
  you authenticate.
- Restricted paths (`/auth/api/admin/`, `/auth/api/user/`, `/auth/ws/`) are
  already refused off a domain's own auth host
  (`fastapi/auth_host.py:is_restricted_path`), and UI paths already redirect
  to the auth host. With an auth host configured for a remote domain, the
  satellite never serves profile, admin or WS traffic for it — browsers and
  WebSockets go directly to the remote auth host (the WS dispatch rules in
  `fastapi/dispatch.py` already allow an origin domain's own auth host).

## Option A — caching HTTP reverse proxy

A thin proxy that caches `/auth/api/forward`, `/check`, `/user-info`,
`/settings` responses keyed by `(Host, cookie, query)` with
`TTL = min(configured TTL, Remote-Session-Expires − now)`. Everything else is
forwarded verbatim; WebSockets are tunneled; `/logout` and session-scoped
mutations are intercepted to evict cache entries. Optionally
stale-while-revalidate so remote-side changes propagate within a TTL without
adding latency to any request.

**Option B** is A plus a small remote addition: a WebSocket change feed
(session deleted, user/role/permission changed) so the proxy evicts within
one network RTT instead of waiting for the TTL.

- Remote changes: **none** for A; one additive, RAM-only endpoint for B.
- Staleness: TTL-bounded for anything that bypasses the proxy (admin edits,
  logout-all from another device) in A; ~1 RTT in B.
- The proxy needs no credentials of its own — every cacheable request is
  authenticated by the end user's cookie, forwarded on a miss.

## Option D — satellite instance with per-domain RAM replica (chosen direction)

The same paskia codebase, with two kinds of domains in the existing
multi-domain config:

- **Local domains** — exactly as today, backed by the local Kanta DB.
- **Remote domains** — `DomainConfig` gains `remote: {url, token, cache_ttl,
  refresh_interval}`. This is the only new persisted state, and it lives in
  the **satellite's** DB. The remote's DB gets nothing new. A remote domain
  **must** have an auth host configured (the remote's); this is what makes
  the simplifications below possible.

For a remote domain the satellite holds a **RAM-only read replica** of the
remote's tables (users, orgs, roles, permissions, credentials, sessions) —
another plain `DB` struct instance, never opened by Kanta, rebuilt from the
remote on startup and kept fresh by the sync channel plus snooping on
proxied traffic.

### No `db.data()` magic: explicit per-request stores

The first draft proposed making the global `db.data()` accessor return a
replica based on the `current_domain` contextvar. Rejected: a global
accessor whose meaning shifts under the caller is exactly the kind of magic
that breeds bugs in no-request-context code (background jobs, OIDC notify,
CLI). Instead:

- The runtime `Domain` object (`domains.py`) gets a `store` attribute: a
  `DB` instance. For local domains it **is** the global `operations._db`;
  for remote domains it is the replica. Registry build / remote-manager
  startup attaches it.
- Endpoints resolve the store once, explicitly, from the request:
  `request.state.domain.store` (dispatch already puts the domain there), and
  pass it down as an ordinary parameter. No global accessor changes
  meaning; local domains observe zero functional change.

Refactor needed to make the hot path store-explicit (small, all in the
"manageable core"):

- `DB.session_ctx()` (`db/structs.py`) currently leans on struct convenience
  properties (`s.user`, `user.role`, `role.org`, `s.credential`,
  `org.permissions`) that internally call the global `db.data()`. Rewrite it
  to index `self.*` tables directly. This is a pure correctness fix — it
  makes `session_ctx` honest about which DB it reads.
- `authsession.session_ctx`, `permutil.session_context`, `authz.verify`,
  `sessionutil.check_session_age` gain an explicit `store: DB` parameter
  (or are called with one) instead of reaching the global.
- `userinfo.build_user_info` takes a `store` and lists
  `store.sessions`/`store.credentials` filtered by user instead of the
  `user.sessions` / `user.credentials` convenience properties.
- `userinfo.build_session_context`, `api._remote_headers`,
  `authz.auth_error_content` already operate on the `SessionContext` alone —
  no change.
- Struct convenience properties (`User.sessions`, `Session.user`, …) keep
  using the global `db.data()`; they are only used by local-domain code
  (admin, profile, CLI) and by Kanta maintenance. Remote-domain request
  paths simply never call them.

### Endpoint categorization for remote domains

**1. True special handling (remote-aware logic):**

| Endpoint | Handling |
|---|---|
| `POST /auth/api/validate` | Read from replica; apply the throttled refresh to the replica and write it behind as `session_refresh{key, validated, ip, ua}` over the sync WS; set the renewed cookie locally (same secret, same rules as today) |
| `POST /auth/api/logout` | Forward to remote with the user's cookie; evict the session from the replica immediately (optimistic); the sync event confirms |
| `POST /auth/api/set-session` | Forward code redemption to the remote (the 60 s exchange code lives in its RAM); relay the `Set-Cookie`; the new session arrives via sync event (created during the login ceremony) |
| `GET /auth/api/token-info` | Proxy to remote (reset tokens exist only there) |
| `/auth/oidc/token`, `/userinfo`, `/keys`, `/backchannel-logout` | Proxy to remote (signing key and OIDC sessions stay there); `/keys` is cacheable |
| Avatar `GET .../profile.webp` | Proxy to remote (storage may be remote-local); not latency-critical |

**2. Store-explicit, no functional change** (the "alike db magic" group —
they read the database but only through the store they are handed):

- `GET /auth/api/forward` — the hot path, pure read from `store`.
- `POST /auth/api/validate` — the read/verify half (category 1 covers the
  refresh half).
- `GET /auth/api/check` — reads users/roles/orgs/permissions from `store`.
- `GET /auth/api/user-info` — `build_user_info(store, …)`.

**3. Untouched — no knowledge of remoting:**

- `GET /auth/api/settings` — domain config only; already reports
  `auth_host`/`auth_site_url`, which is how frontends learn to open
  WebSockets on the remote auth host directly.
- `/.well-known/openid-configuration`, `/.well-known/webauthn` — derived
  from Host and local domain config.
- Frontend statics, `/auth/restricted/iframe` HTML, reset-token pages —
  static files; their API calls land in categories 1–2.
- `auth_host.redirect_middleware` — already redirects UI to the remote auth
  host and 404s restricted paths off it; with auth host mandatory this is
  exactly the desired behavior, unchanged.
- The entire admin app, user-profile app, `/auth/ws/*`, `/auth/remote-auth/*`
  — reachable only on the domain's auth host, which for remote domains is
  the remote itself; the satellite never executes this code for remote
  domains.
- Dispatch middleware, background cleanup of the local DB, CLI, bootstrap.
- All local-domain handling of every endpoint.

### The sync channel (the one remote addition, RAM-only)

New endpoint on the remote, e.g. `wss://…/auth/api/sync/ws`, plus an
emission hook in `db/operations.py` (publish after each commit; no-op when
no peer is connected):

- **Remote → satellite**: `hello{generation, protocol_version}` → full
  snapshot (chunked msgspec; these tables are tiny — 10k sessions ≈ 2 MB) →
  then `event{seq, op: upsert|delete, table, key, payload}`. The remote
  keeps a bounded event ring buffer in RAM; on reconnect the satellite sends
  its last `seq` and either gets a replay or a fresh snapshot.
- **Satellite → remote**: `session_refresh` write-behind; optionally
  `session_deleted` as belt-and-braces alongside proxied logout.
- **Authentication**: the satellite presents a token the remote reads from
  env/CLI (e.g. `PASKIA_SYNC_TOKENS`) — nothing lands in the remote's
  database. mTLS client certs are an alternative.

Freshness hierarchy, worst case last:

1. Changes made **through** the satellite: immediate (optimistic eviction on
   logout, snooped `Set-Cookie`/login results).
2. Changes made **directly on the remote** (admin on another device,
   logout-all): WS event, ~1 network RTT.
3. WS down: per-entry TTL lazy revalidation + periodic full snapshot; on
   reconnect, replay or re-sync. Fail-open (the replica enforces
   `validated + EXPIRES` itself, with its own little sweeper mirroring
   `cleanup_expired`) vs fail-closed is configurable per domain.

Deliberately out of scope:

- **Running WebAuthn ceremonies on the satellite** — duplicates the
  security-critical verification path and breaks centralized sign-count
  clone detection; browsers talk to the remote auth host directly anyway.
- **OIDC signing on the satellite** — key material never leaves the remote.
- **Admin/profile functionality for remote domains** — deferred to the
  remote auth host; this is what keeps the satellite's surface small.

## D vs. A(+B): what the read-only local state buys

Semantics, not just speed:

- **Full `SessionContext` locally.** A answers the exact byte-response it
  once saw; D *computes* the answer. New `perm`/`max_age`/`public` query
  combinations never seen before are served locally by D, while A must go
  upstream on every cache-key miss. D caches the *domain model*, so derived
  answers (effective permissions per host, `max_age` against
  `credential.last_used`, per-request `Remote-*` header composition) are
  correct without having been witnessed before.
- **One invalidation model.** In A, invalidation rules are hand-built per
  endpoint (query-key mapping, cookie re-keying on renew, 401 variants,
  `/user-info` vs `/forward` overlap). In D, events mutate the replica
  (upsert/delete by table+key) and every endpoint becomes consistent at
  once — including endpoints added in the future.
- **Degradation behaves like a real instance.** With the remote down, D
  keeps serving a coherent auth service from the replica (expiry enforced
  locally). A serves a pile of unrelated cached responses with gaps wherever
  the cache was cold.
- **Multi-domain uniformity.** D is a property of a domain in the existing
  registry: local rp-ids and remote rp-ids coexist, and the same instance
  genuinely *is* the auth server for its local domains. A is a separate
  component bolted in front of specific URLs.
- **User simplicity.** D is configured once as domain config
  (`remote.url` + token); A needs deployment and key-mapping discipline per
  frontend application.

## What D costs

- **A store-explicitness refactor in the core.** `DB.session_ctx` must stop
  leaning on global-accessing convenience properties; the verify/userinfo
  helper chain gains a `store` parameter. Small and mechanical, but it
  touches the security-critical read path, so it wants careful review and
  tests proving local-domain behavior is byte-identical. (The rejected
  alternative — a contextvar-dependent `db.data()` — avoided this refactor
  at the price of a global whose meaning shifts under the caller.)
- **A sync protocol to design and version.** Snapshot + sequenced events +
  ring-buffer replay + reconnect reconciliation, plus a protocol-version
  handshake because remote and satellite releases can drift in
  `SessionContext` semantics. A needs no protocol at all.
- **A trusted satellite host.** The replica holds everything the remote DB
  has except the OIDC key (public keys, hashed session keys, user data) in
  RAM. The host must be as trusted as the remote. A caches only what traffic
  it saw, in the shape it saw it.
- **Remote code, albeit additive.** D requires the sync endpoint and commit
  hooks on the remote (RAM-only, inert when unused, token-gated). A requires
  zero remote changes; B requires the same endpoint in a thinner form.
- **Replica housekeeping** — its own expiry sweeper, write-behind ordering,
  optimistic eviction vs. event confirmation. A's failure modes are dumber
  and narrower.

## Summary

A (+B) is the right tool when the goal is "make forward-auth fast in front
of an untouched server" — small, deployable anywhere, byte-compatible. D is
the right tool when the satellite should *be* a paskia instance for its
remote domains: one consistency model, correct answers for un-cached query
shapes, graceful degradation, and per-domain mixing with local rp-ids. With
the auth host made mandatory for remote domains, D's surface shrinks to four
locally served read endpoints, a handful of proxied mutations, an explicit
`store` parameter in the core read path, and one RAM-only sync endpoint on
the remote.