Masked translation round-trip; named translator keys
- pagerite/masking.py: technical spans (code, URLs, {placeholders}, attrs,
footnote/link labels, container names, HTML tags) become numbered sentinels
for the LLM round trip; results are restored by number and rejected when a
sentinel is mangled (skipped for the rest of the run, stays pending).
Chunks with no prose left after masking are never dispatched.
- Data.translate_key -> translate_keys dict (key -> name); the first key is
generated at bootstrap, result transactions record the key as user=, and
startup logs the service URL(s) via translate.log_service_urls.
- Fix /_translate proxying through the Vite dev server (missing slash).
This commit is contained in:
@@ -13,8 +13,9 @@
|
||||
"""Pagerite translator service: translate site content with Seed-X-PPO-7B.
|
||||
|
||||
Connects to a Pagerite server's translator WebSocket — the full URL
|
||||
including the access key (the admin finds it in the site settings,
|
||||
GET /_api/settings -> ``translate_key``) — and announces the languages the
|
||||
including the access key (printed at server startup; the admin also finds
|
||||
the key in the site settings, GET /_api/settings -> ``translate_keys``) —
|
||||
and announces the languages the
|
||||
model CAN translate (capabilities). The server dispatches one single-item
|
||||
job at a time per connection, offered only in its configured target
|
||||
languages (``Data.translate_langs``) ∩ the announced capabilities; a
|
||||
@@ -55,8 +56,10 @@ SEED_X_TAGS = {
|
||||
}
|
||||
SEED_X_NAMES = {v: k for k, v in SEED_X_TAGS.items()}
|
||||
|
||||
#: The fragments are Markdown; Seed-X has no system prompt, so it goes in-line.
|
||||
NOTE = ", preserving all Markdown formatting, URLs and code exactly unchanged"
|
||||
#: The fragments are masked Markdown (pagerite/masking.py: ⟦N⟧ sentinels
|
||||
#: stand in for code, URLs, placeholders...); Seed-X has no system prompt,
|
||||
#: so the instruction goes in-line.
|
||||
NOTE = ", preserving all Markdown formatting and keeping every ⟦N⟧ token exactly unchanged"
|
||||
|
||||
|
||||
# The wire structs below duplicate pagerite/translate.py: this script runs
|
||||
@@ -76,7 +79,7 @@ class Job(msgspec.Struct, tag="job"):
|
||||
|
||||
lang: str
|
||||
key: bytes #: 9-byte chunk hash (base64 in the JSON frame)
|
||||
text: str
|
||||
text: str #: masked (pagerite/masking.py): the ⟦N⟧ tokens must survive verbatim
|
||||
path: str #: article it came from ("" = front page), no leading slash
|
||||
kind: str #: "chunk" | "title"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user