Make session read path store-explicit
DB.session_ctx now reads only its own tables instead of struct convenience properties that reach the global database, so a DB instance (a read replica) is self-contained. session_ctx/session_context/ authz.verify/build_user_info take an explicit store parameter defaulting to the local database; api.py endpoints resolve it from the dispatched domain (Domain.store).
This commit is contained in:
@@ -0,0 +1,264 @@
|
||||
# Remote Proxy / Satellite Design Review
|
||||
|
||||
Design notes for serving configured domains (rp-ids) from a **remote** paskia
|
||||
instance instead of the local database, so that latency-sensitive checks
|
||||
(`/auth/api/forward`, `/auth/api/validate`) answer in ~1 ms even when the auth
|
||||
server is on another continent. Example: `app2.example.com` runs on our local
|
||||
host and needs fast local checks, while `app1.example.com` and
|
||||
`auth.example.com` run far away — all sharing `example.com` as rp-id.
|
||||
|
||||
Client applications that previously used `https://auth.example.com` as their
|
||||
auth backend (forward-auth checks) only repoint to the local satellite
|
||||
(`http://127.0.0.1:4401`); both remain usable interchangeably, and the
|
||||
satellite ultimately uses `auth.example.com`.
|
||||
|
||||
Status: **proposal, not implemented.** Revision 2: no `db.data()` magic;
|
||||
explicit domain-aware stores; auth host mandatory for remote domains.
|
||||
|
||||
## Facts from the architecture review
|
||||
|
||||
- The whole DB is already in RAM (msgspec structs, Kanta JSONL for
|
||||
persistence). Reads never touch disk — the remote is fast, so the latency
|
||||
problem is purely network RTT. The job is replicating *read state*, not
|
||||
reimplementing a database.
|
||||
- `/auth/api/forward` (the hot path) is a **pure read**:
|
||||
`session_ctx(cookie, host)` → perm/max_age checks → 204 + `Remote-*`
|
||||
headers. No DB write, no cookie mutation.
|
||||
- Sessions are host-bound and expire by `validated + SESSION_LIFETIME`;
|
||||
expiry is enforced by background deletion (`db/lifecycle.py`), and
|
||||
`Remote-Session-Expires` is already present in the 204 response headers.
|
||||
- `/auth/api/validate` is the only renewal write (throttled to ≥5 min
|
||||
spacing); it can set a fresh cookie.
|
||||
- Login/register/remote-auth run over WebSockets with state in process RAM
|
||||
(PoW challenges, `remoteauth.instance`, `authcode` 60 s exchange codes).
|
||||
These cannot be cached or replicated.
|
||||
- The sync feed carries no usable secrets: sessions are keyed by
|
||||
`hash_secret` output, credentials carry public keys only, and the OIDC
|
||||
signing key is never exported. Knowing a stored session hash does not let
|
||||
you authenticate.
|
||||
- Restricted paths (`/auth/api/admin/`, `/auth/api/user/`, `/auth/ws/`) are
|
||||
already refused off a domain's own auth host
|
||||
(`fastapi/auth_host.py:is_restricted_path`), and UI paths already redirect
|
||||
to the auth host. With an auth host configured for a remote domain, the
|
||||
satellite never serves profile, admin or WS traffic for it — browsers and
|
||||
WebSockets go directly to the remote auth host (the WS dispatch rules in
|
||||
`fastapi/dispatch.py` already allow an origin domain's own auth host).
|
||||
|
||||
## Option A — caching HTTP reverse proxy
|
||||
|
||||
A thin proxy that caches `/auth/api/forward`, `/check`, `/user-info`,
|
||||
`/settings` responses keyed by `(Host, cookie, query)` with
|
||||
`TTL = min(configured TTL, Remote-Session-Expires − now)`. Everything else is
|
||||
forwarded verbatim; WebSockets are tunneled; `/logout` and session-scoped
|
||||
mutations are intercepted to evict cache entries. Optionally
|
||||
stale-while-revalidate so remote-side changes propagate within a TTL without
|
||||
adding latency to any request.
|
||||
|
||||
**Option B** is A plus a small remote addition: a WebSocket change feed
|
||||
(session deleted, user/role/permission changed) so the proxy evicts within
|
||||
one network RTT instead of waiting for the TTL.
|
||||
|
||||
- Remote changes: **none** for A; one additive, RAM-only endpoint for B.
|
||||
- Staleness: TTL-bounded for anything that bypasses the proxy (admin edits,
|
||||
logout-all from another device) in A; ~1 RTT in B.
|
||||
- The proxy needs no credentials of its own — every cacheable request is
|
||||
authenticated by the end user's cookie, forwarded on a miss.
|
||||
|
||||
## Option D — satellite instance with per-domain RAM replica (chosen direction)
|
||||
|
||||
The same paskia codebase, with two kinds of domains in the existing
|
||||
multi-domain config:
|
||||
|
||||
- **Local domains** — exactly as today, backed by the local Kanta DB.
|
||||
- **Remote domains** — `DomainConfig` gains `remote: {url, token, cache_ttl,
|
||||
refresh_interval}`. This is the only new persisted state, and it lives in
|
||||
the **satellite's** DB. The remote's DB gets nothing new. A remote domain
|
||||
**must** have an auth host configured (the remote's); this is what makes
|
||||
the simplifications below possible.
|
||||
|
||||
For a remote domain the satellite holds a **RAM-only read replica** of the
|
||||
remote's tables (users, orgs, roles, permissions, credentials, sessions) —
|
||||
another plain `DB` struct instance, never opened by Kanta, rebuilt from the
|
||||
remote on startup and kept fresh by the sync channel plus snooping on
|
||||
proxied traffic.
|
||||
|
||||
### No `db.data()` magic: explicit per-request stores
|
||||
|
||||
The first draft proposed making the global `db.data()` accessor return a
|
||||
replica based on the `current_domain` contextvar. Rejected: a global
|
||||
accessor whose meaning shifts under the caller is exactly the kind of magic
|
||||
that breeds bugs in no-request-context code (background jobs, OIDC notify,
|
||||
CLI). Instead:
|
||||
|
||||
- The runtime `Domain` object (`domains.py`) gets a `store` attribute: a
|
||||
`DB` instance. For local domains it **is** the global `operations._db`;
|
||||
for remote domains it is the replica. Registry build / remote-manager
|
||||
startup attaches it.
|
||||
- Endpoints resolve the store once, explicitly, from the request:
|
||||
`request.state.domain.store` (dispatch already puts the domain there), and
|
||||
pass it down as an ordinary parameter. No global accessor changes
|
||||
meaning; local domains observe zero functional change.
|
||||
|
||||
Refactor needed to make the hot path store-explicit (small, all in the
|
||||
"manageable core"):
|
||||
|
||||
- `DB.session_ctx()` (`db/structs.py`) currently leans on struct convenience
|
||||
properties (`s.user`, `user.role`, `role.org`, `s.credential`,
|
||||
`org.permissions`) that internally call the global `db.data()`. Rewrite it
|
||||
to index `self.*` tables directly. This is a pure correctness fix — it
|
||||
makes `session_ctx` honest about which DB it reads.
|
||||
- `authsession.session_ctx`, `permutil.session_context`, `authz.verify`,
|
||||
`sessionutil.check_session_age` gain an explicit `store: DB` parameter
|
||||
(or are called with one) instead of reaching the global.
|
||||
- `userinfo.build_user_info` takes a `store` and lists
|
||||
`store.sessions`/`store.credentials` filtered by user instead of the
|
||||
`user.sessions` / `user.credentials` convenience properties.
|
||||
- `userinfo.build_session_context`, `api._remote_headers`,
|
||||
`authz.auth_error_content` already operate on the `SessionContext` alone —
|
||||
no change.
|
||||
- Struct convenience properties (`User.sessions`, `Session.user`, …) keep
|
||||
using the global `db.data()`; they are only used by local-domain code
|
||||
(admin, profile, CLI) and by Kanta maintenance. Remote-domain request
|
||||
paths simply never call them.
|
||||
|
||||
### Endpoint categorization for remote domains
|
||||
|
||||
**1. True special handling (remote-aware logic):**
|
||||
|
||||
| Endpoint | Handling |
|
||||
|---|---|
|
||||
| `POST /auth/api/validate` | Read from replica; apply the throttled refresh to the replica and write it behind as `session_refresh{key, validated, ip, ua}` over the sync WS; set the renewed cookie locally (same secret, same rules as today) |
|
||||
| `POST /auth/api/logout` | Forward to remote with the user's cookie; evict the session from the replica immediately (optimistic); the sync event confirms |
|
||||
| `POST /auth/api/set-session` | Forward code redemption to the remote (the 60 s exchange code lives in its RAM); relay the `Set-Cookie`; the new session arrives via sync event (created during the login ceremony) |
|
||||
| `GET /auth/api/token-info` | Proxy to remote (reset tokens exist only there) |
|
||||
| `/auth/oidc/token`, `/userinfo`, `/keys`, `/backchannel-logout` | Proxy to remote (signing key and OIDC sessions stay there); `/keys` is cacheable |
|
||||
| Avatar `GET .../profile.webp` | Proxy to remote (storage may be remote-local); not latency-critical |
|
||||
|
||||
**2. Store-explicit, no functional change** (the "alike db magic" group —
|
||||
they read the database but only through the store they are handed):
|
||||
|
||||
- `GET /auth/api/forward` — the hot path, pure read from `store`.
|
||||
- `POST /auth/api/validate` — the read/verify half (category 1 covers the
|
||||
refresh half).
|
||||
- `GET /auth/api/check` — reads users/roles/orgs/permissions from `store`.
|
||||
- `GET /auth/api/user-info` — `build_user_info(store, …)`.
|
||||
|
||||
**3. Untouched — no knowledge of remoting:**
|
||||
|
||||
- `GET /auth/api/settings` — domain config only; already reports
|
||||
`auth_host`/`auth_site_url`, which is how frontends learn to open
|
||||
WebSockets on the remote auth host directly.
|
||||
- `/.well-known/openid-configuration`, `/.well-known/webauthn` — derived
|
||||
from Host and local domain config.
|
||||
- Frontend statics, `/auth/restricted/iframe` HTML, reset-token pages —
|
||||
static files; their API calls land in categories 1–2.
|
||||
- `auth_host.redirect_middleware` — already redirects UI to the remote auth
|
||||
host and 404s restricted paths off it; with auth host mandatory this is
|
||||
exactly the desired behavior, unchanged.
|
||||
- The entire admin app, user-profile app, `/auth/ws/*`, `/auth/remote-auth/*`
|
||||
— reachable only on the domain's auth host, which for remote domains is
|
||||
the remote itself; the satellite never executes this code for remote
|
||||
domains.
|
||||
- Dispatch middleware, background cleanup of the local DB, CLI, bootstrap.
|
||||
- All local-domain handling of every endpoint.
|
||||
|
||||
### The sync channel (the one remote addition, RAM-only)
|
||||
|
||||
New endpoint on the remote, e.g. `wss://…/auth/api/sync/ws`, plus an
|
||||
emission hook in `db/operations.py` (publish after each commit; no-op when
|
||||
no peer is connected):
|
||||
|
||||
- **Remote → satellite**: `hello{generation, protocol_version}` → full
|
||||
snapshot (chunked msgspec; these tables are tiny — 10k sessions ≈ 2 MB) →
|
||||
then `event{seq, op: upsert|delete, table, key, payload}`. The remote
|
||||
keeps a bounded event ring buffer in RAM; on reconnect the satellite sends
|
||||
its last `seq` and either gets a replay or a fresh snapshot.
|
||||
- **Satellite → remote**: `session_refresh` write-behind; optionally
|
||||
`session_deleted` as belt-and-braces alongside proxied logout.
|
||||
- **Authentication**: the satellite presents a token the remote reads from
|
||||
env/CLI (e.g. `PASKIA_SYNC_TOKENS`) — nothing lands in the remote's
|
||||
database. mTLS client certs are an alternative.
|
||||
|
||||
Freshness hierarchy, worst case last:
|
||||
|
||||
1. Changes made **through** the satellite: immediate (optimistic eviction on
|
||||
logout, snooped `Set-Cookie`/login results).
|
||||
2. Changes made **directly on the remote** (admin on another device,
|
||||
logout-all): WS event, ~1 network RTT.
|
||||
3. WS down: per-entry TTL lazy revalidation + periodic full snapshot; on
|
||||
reconnect, replay or re-sync. Fail-open (the replica enforces
|
||||
`validated + EXPIRES` itself, with its own little sweeper mirroring
|
||||
`cleanup_expired`) vs fail-closed is configurable per domain.
|
||||
|
||||
Deliberately out of scope:
|
||||
|
||||
- **Running WebAuthn ceremonies on the satellite** — duplicates the
|
||||
security-critical verification path and breaks centralized sign-count
|
||||
clone detection; browsers talk to the remote auth host directly anyway.
|
||||
- **OIDC signing on the satellite** — key material never leaves the remote.
|
||||
- **Admin/profile functionality for remote domains** — deferred to the
|
||||
remote auth host; this is what keeps the satellite's surface small.
|
||||
|
||||
## D vs. A(+B): what the read-only local state buys
|
||||
|
||||
Semantics, not just speed:
|
||||
|
||||
- **Full `SessionContext` locally.** A answers the exact byte-response it
|
||||
once saw; D *computes* the answer. New `perm`/`max_age`/`public` query
|
||||
combinations never seen before are served locally by D, while A must go
|
||||
upstream on every cache-key miss. D caches the *domain model*, so derived
|
||||
answers (effective permissions per host, `max_age` against
|
||||
`credential.last_used`, per-request `Remote-*` header composition) are
|
||||
correct without having been witnessed before.
|
||||
- **One invalidation model.** In A, invalidation rules are hand-built per
|
||||
endpoint (query-key mapping, cookie re-keying on renew, 401 variants,
|
||||
`/user-info` vs `/forward` overlap). In D, events mutate the replica
|
||||
(upsert/delete by table+key) and every endpoint becomes consistent at
|
||||
once — including endpoints added in the future.
|
||||
- **Degradation behaves like a real instance.** With the remote down, D
|
||||
keeps serving a coherent auth service from the replica (expiry enforced
|
||||
locally). A serves a pile of unrelated cached responses with gaps wherever
|
||||
the cache was cold.
|
||||
- **Multi-domain uniformity.** D is a property of a domain in the existing
|
||||
registry: local rp-ids and remote rp-ids coexist, and the same instance
|
||||
genuinely *is* the auth server for its local domains. A is a separate
|
||||
component bolted in front of specific URLs.
|
||||
- **User simplicity.** D is configured once as domain config
|
||||
(`remote.url` + token); A needs deployment and key-mapping discipline per
|
||||
frontend application.
|
||||
|
||||
## What D costs
|
||||
|
||||
- **A store-explicitness refactor in the core.** `DB.session_ctx` must stop
|
||||
leaning on global-accessing convenience properties; the verify/userinfo
|
||||
helper chain gains a `store` parameter. Small and mechanical, but it
|
||||
touches the security-critical read path, so it wants careful review and
|
||||
tests proving local-domain behavior is byte-identical. (The rejected
|
||||
alternative — a contextvar-dependent `db.data()` — avoided this refactor
|
||||
at the price of a global whose meaning shifts under the caller.)
|
||||
- **A sync protocol to design and version.** Snapshot + sequenced events +
|
||||
ring-buffer replay + reconnect reconciliation, plus a protocol-version
|
||||
handshake because remote and satellite releases can drift in
|
||||
`SessionContext` semantics. A needs no protocol at all.
|
||||
- **A trusted satellite host.** The replica holds everything the remote DB
|
||||
has except the OIDC key (public keys, hashed session keys, user data) in
|
||||
RAM. The host must be as trusted as the remote. A caches only what traffic
|
||||
it saw, in the shape it saw it.
|
||||
- **Remote code, albeit additive.** D requires the sync endpoint and commit
|
||||
hooks on the remote (RAM-only, inert when unused, token-gated). A requires
|
||||
zero remote changes; B requires the same endpoint in a thinner form.
|
||||
- **Replica housekeeping** — its own expiry sweeper, write-behind ordering,
|
||||
optimistic eviction vs. event confirmation. A's failure modes are dumber
|
||||
and narrower.
|
||||
|
||||
## Summary
|
||||
|
||||
A (+B) is the right tool when the goal is "make forward-auth fast in front
|
||||
of an untouched server" — small, deployable anywhere, byte-compatible. D is
|
||||
the right tool when the satellite should *be* a paskia instance for its
|
||||
remote domains: one consistency model, correct answers for un-cached query
|
||||
shapes, graceful degradation, and per-domain mixing with local rp-ids. With
|
||||
the auth host made mandatory for remote domains, D's surface shrinks to four
|
||||
locally served read endpoints, a handful of proxied mutations, an explicit
|
||||
`store` parameter in the core read path, and one RAM-only sync endpoint on
|
||||
the remote.
|
||||
Reference in New Issue
Block a user