Document the translator WebSocket API and reference client

AGENTS.md

@@ -14,6 +14,7 @@ Pagerite is a CMS. See `docs` for the full design and implementation details. Ke
   - `data.py` — msgspec Structs for the kanta database.
   - `chunks.py` — block-level Markdown chunking and content-hash keys for the chunk stores (docs/migrate.md).
   - `i18n.py` — language selection, translation assembly (chunks + patches).
+  - `translate.py` — translator service protocol (msgspec structs) and pending/store core for the `/_translate/{key}` WebSocket (docs/localization.md).
   - `migrations.py` — kanta migrations (`migrate_vN`); ALL schema/storage upgrades live here (raw state dict before struct decoding), never in the app lifespan: v1 moves legacy in-db file blobs to the on-disk store and rebuilds the legacy flat `pages` as the menu tree, v2 rewrites `/_f/{hash}.ext` image links to the extension-less form, backfills AVIF/WebP/JPEG derivatives on disk and drops the obsolete `version` field.
   - `markdown.py` — markdown-it-py renderer.
   - `views.py` — shared page layout and rendering; theme/user-font resolution across `THEME_DIRS` / `FONT_DIRS` (cwd, site, platform data roots, then built-in `pagerite/themes/`, see `docs/themes-and-assets.md`).
@@ -25,6 +26,7 @@ Pagerite is a CMS. See `docs` for the full design and implementation details. Ke
   - `pagerite.js` — public page entry.
   - `assets/` — base CSS, Pygments styles, fonts.
 - `scripts/devserver.py` — dev server with auto reload (the user mostly uses this; avoid running the server yourself, ask the user to test).
+- `scripts/translator.py` — Seed-X translator service client for the `/_translate/{key}` socket (reference client, runs in its own uv env via PEP 723).
 
 Server run by CLI entry point `uv run pagerite` (no auto reloads, build needed). Dev mode is `scripts/devserver.py` (auto reloads, no build needed).
 
@@ -55,5 +57,5 @@ Server run by CLI entry point `uv run pagerite` (no auto reloads, build needed).
 
 - Keep dependencies minimal; add via `uv add` and mention it.
 - The public URL space belongs to content (pretty slugs at root). Reserve only `/_` for the machinery (`/_api/`, `/_f/`, `/_assets/`), plus `/favicon.ico` from the build. Slugs are lowercase ASCII letters, digits, hyphens and underscores `[a-z0-9_-]` (the site editor filters input live via `slugify.js`, built on the `transliteration` npm package — unicode folds to ASCII, spaces become hyphens; an empty slug on a new page is derived from its title), may not begin with `_` or `.`, and such URLs are never looked up as content.
-- No auth in core code; the SSO/reverse proxy gates all of `/_api` (forward-auth) and owns `/auth/` (login/logout, session validation). Pages render identically for everyone; pagerite.js adds the editing UI only after the auth server validates the session.
+- No auth in core code; the SSO/reverse proxy gates all of `/_api` (forward-auth) and owns `/auth/` (login/logout, session validation). Pages render identically for everyone; pagerite.js adds the editing UI only after the auth server validates the session. The one keyed exception is `/_translate/{key}` (translator service; `Data.translate_key`, see docs/localization.md).
 - Update the relevant MarkDown files when architecture, tooling, or conventions change.

docs/localization.md

@@ -210,14 +210,38 @@ def get_translation(path, lang, data) -> Translation | None:
   updates `Data.chunks` / `node.chunks` — only genuinely new text lands in
   the kanta change diff (see docs/migrate.md).
 
+### Translator service API
+
+An external machine-translation service connects over WebSocket at
+`/_translate/{key}` — deliberately **not** under `/_api`: the SSO
+forward-auth does not cover that route, and the key in the path is the
+access control. The key is `Data.translate_key`, generated once at startup
+and surfaced to the admin in `GET /_api/settings` as `translate_key`. A
+wrong or empty key rejects the handshake (close-before-accept → HTTP 403).
+
+Frames are JSON-encoded tagged msgspec structs (`pagerite/translate.py`;
+`bytes` fields ride as base64):
+
+- `{"type": "hello", "langs": [...]}` — client greeting: the target
+  languages it handles (normalized to base subtags; `en`/empty dropped).
+- `{"type": "job", "lang", "items": [{key, text, path, kind}]}` — server
+  push: pending fragments (article titles and chunks), deduped by key.
+- `{"type": "result", "lang", "items": [{key, text}]}` — client reply:
+  translated fragments, matched to content by chunk key alone.
+
+The model is **push**, not polling: on `hello` the server sends everything
+pending per announced language; afterwards `_invalidate_pages()` (called by
+every content/translation write) schedules a delta push of newly pending
+items. Outstanding-item tracking is per connection, so a reconnecting client
+simply re-receives everything still pending. Results are stored into `trans`
+in one transaction and set `node.langs[lang]` on every article they touch
+(shared chunks make several pages gain a language from one fragment).
+
 ### Explicitly out of scope for phase 2
 
-- The machine translation itself: chunking output goes in, translated chunks
+- The machine translation itself: the API above moves fragments in and out;
-  come back. The **service API exists** — `GET /_api/translate/{lang}` lists
+  the translating is external. `scripts/translator.py` is the reference
-  pending items (`{"key", "text", "path", "kind"}`, key = base64 chunk hash),
+  client (Seed-X-PPO-7B only — its 28 languages are the ceiling).
-  `POST /_api/translate/{lang}` stores a batch (`{"items": [{key, text}]}`)
-  into `trans` and maintains `node.langs`; an external service does the
-  actual translating (gated by the /_api forward-auth like everything else).
 - Garbage collection of orphaned chunks/translations (see docs/migrate.md).
 - sitemap.xml per-language entries; translated UI chrome; per-language
   typographer options; multi-locale date/number formatting.

docs/migrate.md

@@ -52,6 +52,9 @@ class Node(msgspec.Struct, omit_defaults=True):
 
 class Data(msgspec.Struct):
     ...
+    #: API key gating the translator service WebSocket (/_translate/{key});
+    #: generated lazily at startup (see the lifespan in app.py).
+    translate_key: str = ""
     #: All original-language text, content-addressed: blake3(normalized)
     #: digest[:9] -> Markdown chunk. Shared by every article. Keys are
     #: bytes; kanta/msgspec base64-encode them at the JSON level.
@@ -90,13 +93,12 @@ that article rendering, `select_language`'s availability check, and hreflang
 alternate links never enumerate chunks. It is written by whoever writes
 translation data, in the same transaction:
 
-- **Translator job:** after writing `trans[h][lang]` entries for an
+- **Translator service:** the WebSocket API at `/_translate/{key}` (see
-  article's chunks (or its title), set `node.langs[lang] = True`. The
+  docs/localization.md) pushes pending fragments (titles + translatable
-  translation service API does both: `GET /_api/translate/{lang}` lists
+  chunks lacking an entry, deduped by hash) and receives result batches;
-  pending items (titles + translatable chunks lacking an entry, deduped by
+  storing a batch writes `trans[h][lang]` entries, sets
-  hash), `POST /_api/translate/{lang}` stores a batch into `trans`, sets
+  `node.langs[lang] = True` on every article that gained one and
-  `langs` on every article that gained an entry and invalidates the page
+  invalidates the page cache — all in one transaction.
-  cache — all in one transaction.
 - **Translated-view save:** appending the first patch for `f"{path}:{lang}"`
   sets `node.langs[lang] = True` (patches alone make the version exist).
 - **Removals:** deleting a patch or GC'ing translations re-derives the key: