Document the translation dispatcher and capability handshake
This commit is contained in:
+32
-13
@@ -222,20 +222,39 @@ wrong or empty key rejects the handshake (close-before-accept → HTTP 403).
|
||||
Frames are JSON-encoded tagged msgspec structs (`pagerite/translate.py`;
|
||||
`bytes` fields ride as base64):
|
||||
|
||||
- `{"type": "hello", "langs": [...]}` — client greeting: the target
|
||||
languages it handles (normalized to base subtags; `en`/empty dropped).
|
||||
- `{"type": "job", "lang", "items": [{key, text, path, kind}]}` — server
|
||||
push: pending fragments (article titles and chunks), deduped by key.
|
||||
- `{"type": "result", "lang", "items": [{key, text}]}` — client reply:
|
||||
translated fragments, matched to content by chunk key alone.
|
||||
- `{"type": "hello", "langs": [...]}` — client greeting announcing its
|
||||
**capabilities**: the language codes its model can produce (normalized
|
||||
to base subtags; `en`/empty dropped).
|
||||
- `{"type": "job", "lang", "key", "text", "path", "kind"}` — server push:
|
||||
ONE fragment to translate (an article title or a chunk).
|
||||
- `{"type": "result", "lang", "key", "text"}` — client reply: the
|
||||
translation of the connection's current job, matching it by (lang, key).
|
||||
|
||||
The model is **push**, not polling: on `hello` the server sends everything
|
||||
pending per announced language; afterwards `_invalidate_pages()` (called by
|
||||
every content/translation write) schedules a delta push of newly pending
|
||||
items. Outstanding-item tracking is per connection, so a reconnecting client
|
||||
simply re-receives everything still pending. Results are stored into `trans`
|
||||
in one transaction and set `node.langs[lang]` on every article they touch
|
||||
(shared chunks make several pages gain a language from one fragment).
|
||||
Which languages get translated is **server-configured**:
|
||||
`Data.translate_langs` (presence-key dict, read/set via `/_api/settings`
|
||||
as `translate_langs`; no editing UI yet). The dispatcher offers a
|
||||
connection jobs only in `wanted ∩ capable`; a connection without overlap
|
||||
simply stays idle.
|
||||
|
||||
Dispatch semantics (all in app.py):
|
||||
|
||||
- **One job at a time per connection** — the next job is sent only after
|
||||
the current one's result. Clients wanting parallelism open multiple
|
||||
connections (e.g. several `scripts/translator.py` instances).
|
||||
- Pending work is derived from the `trans` store
|
||||
(`translate.pending_items`) minus the items in flight on any connection,
|
||||
so a **disconnect requeues** that connection's in-flight item and it is
|
||||
offered to any free capable connection.
|
||||
- Dispatch re-runs on every relevant event: Hello, result, disconnect and
|
||||
content change (`_invalidate_pages()` schedules it, so the pass runs
|
||||
after the writing transaction commits).
|
||||
- A result with no job in flight, a mismatched (lang, key), a duplicate
|
||||
hello, or any malformed frame closes the socket with a protocol error.
|
||||
|
||||
Results are stored into `trans` in one transaction and set
|
||||
`node.langs[lang]` on every article they touch (shared chunks make several
|
||||
pages gain a language from one fragment). Unknown keys are stored anyway
|
||||
and re-storing overwrites — results are idempotent.
|
||||
|
||||
### Explicitly out of scope for phase 2
|
||||
|
||||
|
||||
Reference in New Issue
Block a user