@@ -0,0 +1,264 @@
# Remote Proxy / Satellite Design Review
Design notes for serving configured domains (rp-ids) from a **remote ** paskia
instance instead of the local database, so that latency-sensitive checks
(`/auth/api/forward` , `/auth/api/validate` ) answer in ~1 ms even when the auth
server is on another continent. Example: `app2.example.com` runs on our local
host and needs fast local checks, while `app1.example.com` and
`auth.example.com` run far away — all sharing `example.com` as rp-id.
Client applications that previously used `https://auth.example.com` as their
auth backend (forward-auth checks) only repoint to the local satellite
(`http://127.0.0.1:4401` ); both remain usable interchangeably, and the
satellite ultimately uses `auth.example.com` .
Status: **proposal, not implemented. ** Revision 2: no `db.data()` magic;
explicit domain-aware stores; auth host mandatory for remote domains.
## Facts from the architecture review
- The whole DB is already in RAM (msgspec structs, Kanta JSONL for
persistence). Reads never touch disk — the remote is fast, so the latency
problem is purely network RTT. The job is replicating * read state * , not
reimplementing a database.
- `/auth/api/forward` (the hot path) is a **pure read ** :
`session_ctx(cookie, host)` → perm/max_age checks → 204 + `Remote-*`
headers. No DB write, no cookie mutation.
- Sessions are host-bound and expire by `validated + SESSION_LIFETIME` ;
expiry is enforced by background deletion (`db/lifecycle.py` ), and
`Remote-Session-Expires` is already present in the 204 response headers.
- `/auth/api/validate` is the only renewal write (throttled to ≥5 min
spacing); it can set a fresh cookie.
- Login/register/remote-auth run over WebSockets with state in process RAM
(PoW challenges, `remoteauth.instance` , `authcode` 60 s exchange codes).
These cannot be cached or replicated.
- The sync feed carries no usable secrets: sessions are keyed by
`hash_secret` output, credentials carry public keys only, and the OIDC
signing key is never exported. Knowing a stored session hash does not let
you authenticate.
- Restricted paths (`/auth/api/admin/` , `/auth/api/user/` , `/auth/ws/` ) are
already refused off a domain's own auth host
(`fastapi/auth_host.py:is_restricted_path` ), and UI paths already redirect
to the auth host. With an auth host configured for a remote domain, the
satellite never serves profile, admin or WS traffic for it — browsers and
WebSockets go directly to the remote auth host (the WS dispatch rules in
`fastapi/dispatch.py` already allow an origin domain's own auth host).
## Option A — caching HTTP reverse proxy
A thin proxy that caches `/auth/api/forward` , `/check` , `/user-info` ,
`/settings` responses keyed by `(Host, cookie, query)` with
`TTL = min(configured TTL, Remote-Session-Expires − . Everything else is
forwarded verbatim; WebSockets are tunneled; `/logout` and session-scoped
mutations are intercepted to evict cache entries. Optionally
stale-while-revalidate so remote-side changes propagate within a TTL without
adding latency to any request.
**Option B ** is A plus a small remote addition: a WebSocket change feed
(session deleted, user/role/permission changed) so the proxy evicts within
one network RTT instead of waiting for the TTL.
- Remote changes: **none ** for A; one additive, RAM-only endpoint for B.
- Staleness: TTL-bounded for anything that bypasses the proxy (admin edits,
logout-all from another device) in A; ~1 RTT in B.
- The proxy needs no credentials of its own — every cacheable request is
authenticated by the end user's cookie, forwarded on a miss.
## Option D — satellite instance with per-domain RAM replica (chosen direction)
The same paskia codebase, with two kinds of domains in the existing
multi-domain config:
- **Local domains** — exactly as today, backed by the local Kanta DB.
- **Remote domains** — `DomainConfig` gains `remote: {url, token, cache_ttl,
refresh_interval}`. This is the only new persisted state, and it lives in
the **satellite's** DB. The remote's DB gets nothing new. A remote domain
**must** have an auth host configured (the remote's); this is what makes
the simplifications below possible.
For a remote domain the satellite holds a **RAM-only read replica** of the
remote's tables (users, orgs, roles, permissions, credentials, sessions) —
another plain ` DB` struct instance, never opened by Kanta, rebuilt from the
remote on startup and kept fresh by the sync channel plus snooping on
proxied traffic.
### No ` db.data()` magic: explicit per-request stores
The first draft proposed making the global ` db.data()` accessor return a
replica based on the ` current_domain` contextvar. Rejected: a global
accessor whose meaning shifts under the caller is exactly the kind of magic
that breeds bugs in no-request-context code (background jobs, OIDC notify,
CLI). Instead:
- The runtime ` Domain` object (` domains.py`) gets a ` store` attribute: a
` DB` instance. For local domains it **is** the global ` operations._db`;
for remote domains it is the replica. Registry build / remote-manager
startup attaches it.
- Endpoints resolve the store once, explicitly, from the request:
` request.state.domain.store` (dispatch already puts the domain there), and
pass it down as an ordinary parameter. No global accessor changes
meaning; local domains observe zero functional change.
Refactor needed to make the hot path store-explicit (small, all in the
"manageable core"):
- ` DB.session_ctx()` (` db/structs.py`) currently leans on struct convenience
properties (` s.user`, ` user.role`, ` role.org`, ` s.credential`,
` org.permissions`) that internally call the global ` db.data()`. Rewrite it
to index ` self.*` tables directly. This is a pure correctness fix — it
makes ` session_ctx` honest about which DB it reads.
- ` authsession.session_ctx`, ` permutil.session_context`, ` authz.verify`,
` sessionutil.check_session_age` gain an explicit ` store: DB` parameter
(or are called with one) instead of reaching the global.
- ` userinfo.build_user_info` takes a ` store` and lists
` store.sessions`/` store.credentials` filtered by user instead of the
` user.sessions` / ` user.credentials` convenience properties.
- ` userinfo.build_session_context`, ` api._remote_headers`,
` authz.auth_error_content` already operate on the ` SessionContext` alone —
no change.
- Struct convenience properties (` User.sessions`, ` Session.user`, …) keep
using the global ` db.data()`; they are only used by local-domain code
(admin, profile, CLI) and by Kanta maintenance. Remote-domain request
paths simply never call them.
### Endpoint categorization for remote domains
**1. True special handling (remote-aware logic):**
| Endpoint | Handling |
|---|---|
| ` POST /auth/api/validate` | Read from replica; apply the throttled refresh to the replica and write it behind as ` session_refresh{key, validated, ip, ua}` over the sync WS; set the renewed cookie locally (same secret, same rules as today) |
| ` POST /auth/api/logout` | Forward to remote with the user's cookie; evict the session from the replica immediately (optimistic); the sync event confirms |
| ` POST /auth/api/set-session` | Forward code redemption to the remote (the 60 s exchange code lives in its RAM); relay the ` Set-Cookie`; the new session arrives via sync event (created during the login ceremony) |
| ` GET /auth/api/token-info` | Proxy to remote (reset tokens exist only there) |
| ` /auth/oidc/token`, ` /userinfo`, ` /keys`, ` /backchannel-logout` | Proxy to remote (signing key and OIDC sessions stay there); ` /keys` is cacheable |
| Avatar ` GET .../profile.webp` | Proxy to remote (storage may be remote-local); not latency-critical |
**2. Store-explicit, no functional change** (the "alike db magic" group —
they read the database but only through the store they are handed):
- ` GET /auth/api/forward` — the hot path, pure read from ` store`.
- ` POST /auth/api/validate` — the read/verify half (category 1 covers the
refresh half).
- ` GET /auth/api/check` — reads users/roles/orgs/permissions from ` store`.
- ` GET /auth/api/user-info` — ` build_user_info(store, …)`.
**3. Untouched — no knowledge of remoting:**
- ` GET /auth/api/settings` — domain config only; already reports
` auth_host`/` auth_site_url`, which is how frontends learn to open
WebSockets on the remote auth host directly.
- ` /.well-known/openid-configuration`, ` /.well-known/webauthn` — derived
from Host and local domain config.
- Frontend statics, ` /auth/restricted/iframe` HTML, reset-token pages —
static files; their API calls land in categories 1–
- ` auth_host.redirect_middleware` — already redirects UI to the remote auth
host and 404s restricted paths off it; with auth host mandatory this is
exactly the desired behavior, unchanged.
- The entire admin app, user-profile app, ` /auth/ws/*`, ` /auth/remote-auth/*`
— reachable only on the domain's auth host, which for remote domains is
the remote itself; the satellite never executes this code for remote
domains.
- Dispatch middleware, background cleanup of the local DB, CLI, bootstrap.
- All local-domain handling of every endpoint.
### The sync channel (the one remote addition, RAM-only)
New endpoint on the remote, e.g. ` wss://…/auth/api/sync/ws`, plus an
emission hook in ` db/operations.py` (publish after each commit; no-op when
no peer is connected):
- **Remote → satellite**: ` hello{generation, protocol_version}` → full
snapshot (chunked msgspec; these tables are tiny — 10k sessions ≈ 2 MB) →
then ` event{seq, op: upsert|delete, table, key, payload}`. The remote
keeps a bounded event ring buffer in RAM; on reconnect the satellite sends
its last ` seq` and either gets a replay or a fresh snapshot.
- **Satellite → remote**: ` session_refresh` write-behind; optionally
` session_deleted` as belt-and-braces alongside proxied logout.
- **Authentication**: the satellite presents a token the remote reads from
env/CLI (e.g. ` PASKIA_SYNC_TOKENS`) — nothing lands in the remote's
database. mTLS client certs are an alternative.
Freshness hierarchy, worst case last:
1. Changes made **through** the satellite: immediate (optimistic eviction on
logout, snooped ` Set-Cookie`/login results).
2. Changes made **directly on the remote** (admin on another device,
logout-all): WS event, ~1 network RTT.
3. WS down: per-entry TTL lazy revalidation + periodic full snapshot; on
reconnect, replay or re-sync. Fail-open (the replica enforces
` validated + EXPIRES` itself, with its own little sweeper mirroring
` cleanup_expired`) vs fail-closed is configurable per domain.
Deliberately out of scope:
- **Running WebAuthn ceremonies on the satellite** — duplicates the
security-critical verification path and breaks centralized sign-count
clone detection; browsers talk to the remote auth host directly anyway.
- **OIDC signing on the satellite** — key material never leaves the remote.
- **Admin/profile functionality for remote domains** — deferred to the
remote auth host; this is what keeps the satellite's surface small.
## D vs. A(+B): what the read-only local state buys
Semantics, not just speed:
- **Full ` SessionContext` locally.** A answers the exact byte-response it
once saw; D *computes* the answer. New ` perm`/` max_age`/` public` query
combinations never seen before are served locally by D, while A must go
upstream on every cache-key miss. D caches the *domain model*, so derived
answers (effective permissions per host, ` max_age` against
` credential.last_used`, per-request ` Remote-*` header composition) are
correct without having been witnessed before.
- **One invalidation model.** In A, invalidation rules are hand-built per
endpoint (query-key mapping, cookie re-keying on renew, 401 variants,
` /user-info` vs ` /forward` overlap). In D, events mutate the replica
(upsert/delete by table+key) and every endpoint becomes consistent at
once — including endpoints added in the future.
- **Degradation behaves like a real instance.** With the remote down, D
keeps serving a coherent auth service from the replica (expiry enforced
locally). A serves a pile of unrelated cached responses with gaps wherever
the cache was cold.
- **Multi-domain uniformity.** D is a property of a domain in the existing
registry: local rp-ids and remote rp-ids coexist, and the same instance
genuinely *is* the auth server for its local domains. A is a separate
component bolted in front of specific URLs.
- **User simplicity.** D is configured once as domain config
(` remote.url` + token); A needs deployment and key-mapping discipline per
frontend application.
## What D costs
- **A store-explicitness refactor in the core.** ` DB.session_ctx` must stop
leaning on global-accessing convenience properties; the verify/userinfo
helper chain gains a ` store` parameter. Small and mechanical, but it
touches the security-critical read path, so it wants careful review and
tests proving local-domain behavior is byte-identical. (The rejected
alternative — a contextvar-dependent ` db.data()` — avoided this refactor
at the price of a global whose meaning shifts under the caller.)
- **A sync protocol to design and version.** Snapshot + sequenced events +
ring-buffer replay + reconnect reconciliation, plus a protocol-version
handshake because remote and satellite releases can drift in
` SessionContext` semantics. A needs no protocol at all.
- **A trusted satellite host.** The replica holds everything the remote DB
has except the OIDC key (public keys, hashed session keys, user data) in
RAM. The host must be as trusted as the remote. A caches only what traffic
it saw, in the shape it saw it.
- **Remote code, albeit additive.** D requires the sync endpoint and commit
hooks on the remote (RAM-only, inert when unused, token-gated). A requires
zero remote changes; B requires the same endpoint in a thinner form.
- **Replica housekeeping** — its own expiry sweeper, write-behind ordering,
optimistic eviction vs. event confirmation. A's failure modes are dumber
and narrower.
## Summary
A (+B) is the right tool when the goal is "make forward-auth fast in front
of an untouched server" — small, deployable anywhere, byte-compatible. D is
the right tool when the satellite should *be* a paskia instance for its
remote domains: one consistency model, correct answers for un-cached query
shapes, graceful degradation, and per-domain mixing with local rp-ids. With
the auth host made mandatory for remote domains, D's surface shrinks to four
locally served read endpoints, a handful of proxied mutations, an explicit
` store` parameter in the core read path, and one RAM-only sync endpoint on
the remote.