LeoVasanko
20b145d816
DB.session_ctx now reads only its own tables instead of struct convenience properties that reach the global database, so a DB instance (a read replica) is self-contained. session_ctx/session_context/ authz.verify/build_user_info take an explicit store parameter defaulting to the local database; api.py endpoints resolve it from the dispatched domain (Domain.store).
265 lines
14 KiB
Markdown
265 lines
14 KiB
Markdown
# Remote Proxy / Satellite Design Review
|
||
|
||
Design notes for serving configured domains (rp-ids) from a **remote** paskia
|
||
instance instead of the local database, so that latency-sensitive checks
|
||
(`/auth/api/forward`, `/auth/api/validate`) answer in ~1 ms even when the auth
|
||
server is on another continent. Example: `app2.example.com` runs on our local
|
||
host and needs fast local checks, while `app1.example.com` and
|
||
`auth.example.com` run far away — all sharing `example.com` as rp-id.
|
||
|
||
Client applications that previously used `https://auth.example.com` as their
|
||
auth backend (forward-auth checks) only repoint to the local satellite
|
||
(`http://127.0.0.1:4401`); both remain usable interchangeably, and the
|
||
satellite ultimately uses `auth.example.com`.
|
||
|
||
Status: **proposal, not implemented.** Revision 2: no `db.data()` magic;
|
||
explicit domain-aware stores; auth host mandatory for remote domains.
|
||
|
||
## Facts from the architecture review
|
||
|
||
- The whole DB is already in RAM (msgspec structs, Kanta JSONL for
|
||
persistence). Reads never touch disk — the remote is fast, so the latency
|
||
problem is purely network RTT. The job is replicating *read state*, not
|
||
reimplementing a database.
|
||
- `/auth/api/forward` (the hot path) is a **pure read**:
|
||
`session_ctx(cookie, host)` → perm/max_age checks → 204 + `Remote-*`
|
||
headers. No DB write, no cookie mutation.
|
||
- Sessions are host-bound and expire by `validated + SESSION_LIFETIME`;
|
||
expiry is enforced by background deletion (`db/lifecycle.py`), and
|
||
`Remote-Session-Expires` is already present in the 204 response headers.
|
||
- `/auth/api/validate` is the only renewal write (throttled to ≥5 min
|
||
spacing); it can set a fresh cookie.
|
||
- Login/register/remote-auth run over WebSockets with state in process RAM
|
||
(PoW challenges, `remoteauth.instance`, `authcode` 60 s exchange codes).
|
||
These cannot be cached or replicated.
|
||
- The sync feed carries no usable secrets: sessions are keyed by
|
||
`hash_secret` output, credentials carry public keys only, and the OIDC
|
||
signing key is never exported. Knowing a stored session hash does not let
|
||
you authenticate.
|
||
- Restricted paths (`/auth/api/admin/`, `/auth/api/user/`, `/auth/ws/`) are
|
||
already refused off a domain's own auth host
|
||
(`fastapi/auth_host.py:is_restricted_path`), and UI paths already redirect
|
||
to the auth host. With an auth host configured for a remote domain, the
|
||
satellite never serves profile, admin or WS traffic for it — browsers and
|
||
WebSockets go directly to the remote auth host (the WS dispatch rules in
|
||
`fastapi/dispatch.py` already allow an origin domain's own auth host).
|
||
|
||
## Option A — caching HTTP reverse proxy
|
||
|
||
A thin proxy that caches `/auth/api/forward`, `/check`, `/user-info`,
|
||
`/settings` responses keyed by `(Host, cookie, query)` with
|
||
`TTL = min(configured TTL, Remote-Session-Expires − now)`. Everything else is
|
||
forwarded verbatim; WebSockets are tunneled; `/logout` and session-scoped
|
||
mutations are intercepted to evict cache entries. Optionally
|
||
stale-while-revalidate so remote-side changes propagate within a TTL without
|
||
adding latency to any request.
|
||
|
||
**Option B** is A plus a small remote addition: a WebSocket change feed
|
||
(session deleted, user/role/permission changed) so the proxy evicts within
|
||
one network RTT instead of waiting for the TTL.
|
||
|
||
- Remote changes: **none** for A; one additive, RAM-only endpoint for B.
|
||
- Staleness: TTL-bounded for anything that bypasses the proxy (admin edits,
|
||
logout-all from another device) in A; ~1 RTT in B.
|
||
- The proxy needs no credentials of its own — every cacheable request is
|
||
authenticated by the end user's cookie, forwarded on a miss.
|
||
|
||
## Option D — satellite instance with per-domain RAM replica (chosen direction)
|
||
|
||
The same paskia codebase, with two kinds of domains in the existing
|
||
multi-domain config:
|
||
|
||
- **Local domains** — exactly as today, backed by the local Kanta DB.
|
||
- **Remote domains** — `DomainConfig` gains `remote: {url, token, cache_ttl,
|
||
refresh_interval}`. This is the only new persisted state, and it lives in
|
||
the **satellite's** DB. The remote's DB gets nothing new. A remote domain
|
||
**must** have an auth host configured (the remote's); this is what makes
|
||
the simplifications below possible.
|
||
|
||
For a remote domain the satellite holds a **RAM-only read replica** of the
|
||
remote's tables (users, orgs, roles, permissions, credentials, sessions) —
|
||
another plain `DB` struct instance, never opened by Kanta, rebuilt from the
|
||
remote on startup and kept fresh by the sync channel plus snooping on
|
||
proxied traffic.
|
||
|
||
### No `db.data()` magic: explicit per-request stores
|
||
|
||
The first draft proposed making the global `db.data()` accessor return a
|
||
replica based on the `current_domain` contextvar. Rejected: a global
|
||
accessor whose meaning shifts under the caller is exactly the kind of magic
|
||
that breeds bugs in no-request-context code (background jobs, OIDC notify,
|
||
CLI). Instead:
|
||
|
||
- The runtime `Domain` object (`domains.py`) gets a `store` attribute: a
|
||
`DB` instance. For local domains it **is** the global `operations._db`;
|
||
for remote domains it is the replica. Registry build / remote-manager
|
||
startup attaches it.
|
||
- Endpoints resolve the store once, explicitly, from the request:
|
||
`request.state.domain.store` (dispatch already puts the domain there), and
|
||
pass it down as an ordinary parameter. No global accessor changes
|
||
meaning; local domains observe zero functional change.
|
||
|
||
Refactor needed to make the hot path store-explicit (small, all in the
|
||
"manageable core"):
|
||
|
||
- `DB.session_ctx()` (`db/structs.py`) currently leans on struct convenience
|
||
properties (`s.user`, `user.role`, `role.org`, `s.credential`,
|
||
`org.permissions`) that internally call the global `db.data()`. Rewrite it
|
||
to index `self.*` tables directly. This is a pure correctness fix — it
|
||
makes `session_ctx` honest about which DB it reads.
|
||
- `authsession.session_ctx`, `permutil.session_context`, `authz.verify`,
|
||
`sessionutil.check_session_age` gain an explicit `store: DB` parameter
|
||
(or are called with one) instead of reaching the global.
|
||
- `userinfo.build_user_info` takes a `store` and lists
|
||
`store.sessions`/`store.credentials` filtered by user instead of the
|
||
`user.sessions` / `user.credentials` convenience properties.
|
||
- `userinfo.build_session_context`, `api._remote_headers`,
|
||
`authz.auth_error_content` already operate on the `SessionContext` alone —
|
||
no change.
|
||
- Struct convenience properties (`User.sessions`, `Session.user`, …) keep
|
||
using the global `db.data()`; they are only used by local-domain code
|
||
(admin, profile, CLI) and by Kanta maintenance. Remote-domain request
|
||
paths simply never call them.
|
||
|
||
### Endpoint categorization for remote domains
|
||
|
||
**1. True special handling (remote-aware logic):**
|
||
|
||
| Endpoint | Handling |
|
||
|---|---|
|
||
| `POST /auth/api/validate` | Read from replica; apply the throttled refresh to the replica and write it behind as `session_refresh{key, validated, ip, ua}` over the sync WS; set the renewed cookie locally (same secret, same rules as today) |
|
||
| `POST /auth/api/logout` | Forward to remote with the user's cookie; evict the session from the replica immediately (optimistic); the sync event confirms |
|
||
| `POST /auth/api/set-session` | Forward code redemption to the remote (the 60 s exchange code lives in its RAM); relay the `Set-Cookie`; the new session arrives via sync event (created during the login ceremony) |
|
||
| `GET /auth/api/token-info` | Proxy to remote (reset tokens exist only there) |
|
||
| `/auth/oidc/token`, `/userinfo`, `/keys`, `/backchannel-logout` | Proxy to remote (signing key and OIDC sessions stay there); `/keys` is cacheable |
|
||
| Avatar `GET .../profile.webp` | Proxy to remote (storage may be remote-local); not latency-critical |
|
||
|
||
**2. Store-explicit, no functional change** (the "alike db magic" group —
|
||
they read the database but only through the store they are handed):
|
||
|
||
- `GET /auth/api/forward` — the hot path, pure read from `store`.
|
||
- `POST /auth/api/validate` — the read/verify half (category 1 covers the
|
||
refresh half).
|
||
- `GET /auth/api/check` — reads users/roles/orgs/permissions from `store`.
|
||
- `GET /auth/api/user-info` — `build_user_info(store, …)`.
|
||
|
||
**3. Untouched — no knowledge of remoting:**
|
||
|
||
- `GET /auth/api/settings` — domain config only; already reports
|
||
`auth_host`/`auth_site_url`, which is how frontends learn to open
|
||
WebSockets on the remote auth host directly.
|
||
- `/.well-known/openid-configuration`, `/.well-known/webauthn` — derived
|
||
from Host and local domain config.
|
||
- Frontend statics, `/auth/restricted/iframe` HTML, reset-token pages —
|
||
static files; their API calls land in categories 1–2.
|
||
- `auth_host.redirect_middleware` — already redirects UI to the remote auth
|
||
host and 404s restricted paths off it; with auth host mandatory this is
|
||
exactly the desired behavior, unchanged.
|
||
- The entire admin app, user-profile app, `/auth/ws/*`, `/auth/remote-auth/*`
|
||
— reachable only on the domain's auth host, which for remote domains is
|
||
the remote itself; the satellite never executes this code for remote
|
||
domains.
|
||
- Dispatch middleware, background cleanup of the local DB, CLI, bootstrap.
|
||
- All local-domain handling of every endpoint.
|
||
|
||
### The sync channel (the one remote addition, RAM-only)
|
||
|
||
New endpoint on the remote, e.g. `wss://…/auth/api/sync/ws`, plus an
|
||
emission hook in `db/operations.py` (publish after each commit; no-op when
|
||
no peer is connected):
|
||
|
||
- **Remote → satellite**: `hello{generation, protocol_version}` → full
|
||
snapshot (chunked msgspec; these tables are tiny — 10k sessions ≈ 2 MB) →
|
||
then `event{seq, op: upsert|delete, table, key, payload}`. The remote
|
||
keeps a bounded event ring buffer in RAM; on reconnect the satellite sends
|
||
its last `seq` and either gets a replay or a fresh snapshot.
|
||
- **Satellite → remote**: `session_refresh` write-behind; optionally
|
||
`session_deleted` as belt-and-braces alongside proxied logout.
|
||
- **Authentication**: the satellite presents a token the remote reads from
|
||
env/CLI (e.g. `PASKIA_SYNC_TOKENS`) — nothing lands in the remote's
|
||
database. mTLS client certs are an alternative.
|
||
|
||
Freshness hierarchy, worst case last:
|
||
|
||
1. Changes made **through** the satellite: immediate (optimistic eviction on
|
||
logout, snooped `Set-Cookie`/login results).
|
||
2. Changes made **directly on the remote** (admin on another device,
|
||
logout-all): WS event, ~1 network RTT.
|
||
3. WS down: per-entry TTL lazy revalidation + periodic full snapshot; on
|
||
reconnect, replay or re-sync. Fail-open (the replica enforces
|
||
`validated + EXPIRES` itself, with its own little sweeper mirroring
|
||
`cleanup_expired`) vs fail-closed is configurable per domain.
|
||
|
||
Deliberately out of scope:
|
||
|
||
- **Running WebAuthn ceremonies on the satellite** — duplicates the
|
||
security-critical verification path and breaks centralized sign-count
|
||
clone detection; browsers talk to the remote auth host directly anyway.
|
||
- **OIDC signing on the satellite** — key material never leaves the remote.
|
||
- **Admin/profile functionality for remote domains** — deferred to the
|
||
remote auth host; this is what keeps the satellite's surface small.
|
||
|
||
## D vs. A(+B): what the read-only local state buys
|
||
|
||
Semantics, not just speed:
|
||
|
||
- **Full `SessionContext` locally.** A answers the exact byte-response it
|
||
once saw; D *computes* the answer. New `perm`/`max_age`/`public` query
|
||
combinations never seen before are served locally by D, while A must go
|
||
upstream on every cache-key miss. D caches the *domain model*, so derived
|
||
answers (effective permissions per host, `max_age` against
|
||
`credential.last_used`, per-request `Remote-*` header composition) are
|
||
correct without having been witnessed before.
|
||
- **One invalidation model.** In A, invalidation rules are hand-built per
|
||
endpoint (query-key mapping, cookie re-keying on renew, 401 variants,
|
||
`/user-info` vs `/forward` overlap). In D, events mutate the replica
|
||
(upsert/delete by table+key) and every endpoint becomes consistent at
|
||
once — including endpoints added in the future.
|
||
- **Degradation behaves like a real instance.** With the remote down, D
|
||
keeps serving a coherent auth service from the replica (expiry enforced
|
||
locally). A serves a pile of unrelated cached responses with gaps wherever
|
||
the cache was cold.
|
||
- **Multi-domain uniformity.** D is a property of a domain in the existing
|
||
registry: local rp-ids and remote rp-ids coexist, and the same instance
|
||
genuinely *is* the auth server for its local domains. A is a separate
|
||
component bolted in front of specific URLs.
|
||
- **User simplicity.** D is configured once as domain config
|
||
(`remote.url` + token); A needs deployment and key-mapping discipline per
|
||
frontend application.
|
||
|
||
## What D costs
|
||
|
||
- **A store-explicitness refactor in the core.** `DB.session_ctx` must stop
|
||
leaning on global-accessing convenience properties; the verify/userinfo
|
||
helper chain gains a `store` parameter. Small and mechanical, but it
|
||
touches the security-critical read path, so it wants careful review and
|
||
tests proving local-domain behavior is byte-identical. (The rejected
|
||
alternative — a contextvar-dependent `db.data()` — avoided this refactor
|
||
at the price of a global whose meaning shifts under the caller.)
|
||
- **A sync protocol to design and version.** Snapshot + sequenced events +
|
||
ring-buffer replay + reconnect reconciliation, plus a protocol-version
|
||
handshake because remote and satellite releases can drift in
|
||
`SessionContext` semantics. A needs no protocol at all.
|
||
- **A trusted satellite host.** The replica holds everything the remote DB
|
||
has except the OIDC key (public keys, hashed session keys, user data) in
|
||
RAM. The host must be as trusted as the remote. A caches only what traffic
|
||
it saw, in the shape it saw it.
|
||
- **Remote code, albeit additive.** D requires the sync endpoint and commit
|
||
hooks on the remote (RAM-only, inert when unused, token-gated). A requires
|
||
zero remote changes; B requires the same endpoint in a thinner form.
|
||
- **Replica housekeeping** — its own expiry sweeper, write-behind ordering,
|
||
optimistic eviction vs. event confirmation. A's failure modes are dumber
|
||
and narrower.
|
||
|
||
## Summary
|
||
|
||
A (+B) is the right tool when the goal is "make forward-auth fast in front
|
||
of an untouched server" — small, deployable anywhere, byte-compatible. D is
|
||
the right tool when the satellite should *be* a paskia instance for its
|
||
remote domains: one consistency model, correct answers for un-cached query
|
||
shapes, graceful degradation, and per-domain mixing with local rp-ids. With
|
||
the auth host made mandatory for remote domains, D's surface shrinks to four
|
||
locally served read endpoints, a handful of proxied mutations, an explicit
|
||
`store` parameter in the core read path, and one RAM-only sync endpoint on
|
||
the remote.
|