article mode: inject the page title as # heading, jargon loanword rule

When the render would inject the page title as an h1 (markdown.has_h1 is
False for the body), an article job's text carries the same '# {title}'
line: the title translates in document context and the opening paragraphs
see the heading (works both ways). The heading's pair in the decomposed
result becomes the title fragment — heading text only, pure prose, never
stored as a body chunk; a demoted or merged heading skips the title,
which stays pending for a scoped title job. The job covers the title key
so it is not double-dispatched afterwards.

Prompt: prefer established technical loanwords with English roots over
forced localizations (frontend -> frontti in Finnish, not etupääte).

Verified offline (title extraction, demoted-heading skip, no-injection
path) and live against ollama qwen3.8:27b: the standalone title job gave
'Alkuun pääseminen', the article job then overwrote it with the
in-context 'Aloitus' matching the body's translated heading.
This commit is contained in:
2026-09-21 01:07:54 +00:00
parent 37e9ab2cd2
commit 859d11491d
4 changed files with 76 additions and 19 deletions
+6 -1
View File
@@ -125,7 +125,12 @@ Three job modes, in increasing granularity:
Titles are jobs like any other in all modes (`kind="title"` keeps its
article-opening context rule; in `markdown` mode a title crosses as
plain text, since it carries no markup by construction).
plain text, since it carries no markup by construction). Additionally, an
`article` job carries the page title injected as a `# {title}` line at
the top when the render would inject it (the body has no h1 of its own):
the title translates in document context and the opening paragraphs see
the heading. The heading's pair in the decomposed result becomes the
title fragment (heading text only, never stored as a body chunk).
### Dispatch and validation
+8 -2
View File
@@ -418,13 +418,19 @@ offerable to clients of another approach.
connections and only while a page is *mostly* pending (a new article or
a full refresh; steady-state edit follow-up stays scoped jobs). The
job's key is the page's first chunk; `Job.texts` carries the full
original Markdown. The result is decomposed per chunk
original Markdown — with the page title injected as a `# {title}` line
at the top when the render would inject it (the body has no h1 of its
own), so the title translates in document context and the opening
paragraphs see the heading. The result is decomposed per chunk
(`align_article`): non-translatable blocks (code fences, container
fences, raw HTML — everything `needs_translation` rejects) must appear
verbatim and in order and anchor the alignment; regions between anchors
pair positionally, a region whose block count changed stores nothing
(its chunks stay pending and fall back to scoped jobs), and a paired
block whose destinations/placeholders did not survive likewise.
block whose destinations/placeholders did not survive likewise. An
injected title heading's pair becomes the title fragment (heading text
only — never a body chunk; a demoted or merged heading simply skips it
and the title stays pending for a scoped title job).
`scripts/llm_translator.py` is the reference markdown+article client
(instruct LLMs via an OpenAI Chat Completions endpoint or ollama's native
+60 -15
View File
@@ -42,6 +42,7 @@ from pagerite.chunks import (
needs_translation,
)
from pagerite.data import Data, Node, node_markdown, resolve, sorted_nodes
from pagerite.markdown import has_h1
from pagerite.segments import Span, join, pure_prose, split
logger = logging.getLogger(__name__)
@@ -121,6 +122,11 @@ class Result(msgspec.Struct, tag="result"):
#: Union of the client -> server frames (the "type" tag selects).
ClientMsg = Hello | Result
#: A dispatchable offer: the job, its segment spans (segments mode), the
#: full source text, the (lang, key) pairs it covers, and the page title's
#: chunk key when an article job carries an injected title heading.
_Offer = tuple["Job", list[Span], str, set[tuple[str, bytes]], "bytes | None"]
#: Constructs a translation must preserve verbatim inside a prose block:
#: link/image destinations and {...} placeholders (sorted multisets are
#: compared, so additions and drops both fail validation).
@@ -307,16 +313,21 @@ class _Connection:
self.spans: list[Span] = []
self.original: str = "" # its full source text (splicing / alignment)
self.kind: str = "" # "chunk" | "title" | "article"
#: Article jobs with an injected title heading: the page title's
#: chunk key (its translation is extracted from the result's first
#: block, never stored as a body chunk).
self.title_key: bytes | None = None
def take(self) -> tuple[str, str, str, list[Span]]:
def take(self) -> tuple[str, str, str, list[Span], bytes | None]:
"""Snapshot and clear the in-flight job's working state."""
mode, kind, spans, original = self.mode, self.kind, self.spans, self.original
out = (self.mode, self.kind, self.spans, self.original, self.title_key)
self.inflight = None
self.items = set()
self.mode = self.kind = ""
self.spans = []
self.original = ""
return mode, kind, spans, original
self.title_key = None
return out
class Dispatcher:
@@ -370,11 +381,9 @@ class Dispatcher:
return
asyncio.create_task(self._dispatch())
def _scoped_job(
self, item: TransItem, lang: str, mode: str
) -> tuple[Job, list[Span], str, set[tuple[str, bytes]]] | None:
def _scoped_job(self, item: TransItem, lang: str, mode: str) -> _Offer | None:
"""A title/chunk job for one pending item, in segments or markdown
mode: (job, spans, original, covered (lang, key) pairs)."""
mode."""
if mode == "segments":
spans, texts, contexts = split(item.text)
if not texts:
@@ -405,7 +414,13 @@ class Dispatcher:
mode="markdown",
contexts=contexts,
)
return job, spans if mode == "segments" else [], item.text, {(lang, item.key)}
return (
job,
spans if mode == "segments" else [],
item.text,
{(lang, item.key)},
None,
)
def _block_contexts(self, lang: str, item: TransItem) -> list[str]:
"""The previous and next block of the served hybrid around a pending
@@ -439,10 +454,15 @@ class Dispatcher:
def _article_job(
self, lang: str, items: list[TransItem], inflight: set[tuple[str, bytes]]
) -> tuple[Job, list[Span], str, set[tuple[str, bytes]]] | None:
) -> _Offer | None:
"""A whole-page job for the first page that is mostly pending for
``lang`` (a whole new article or a full refresh; steady-state edits
stay scoped jobs). The job's key is the page's first chunk."""
stay scoped jobs). The job's key is the page's first chunk.
When the render would inject the page title as an h1 (the body has
none of its own), the job text carries the same ``# {title}`` line:
the title translates in document context, and the opening
paragraphs see the heading."""
by_path: dict[str, list[TransItem]] = {}
for item in items:
if item.kind == "chunk":
@@ -470,15 +490,22 @@ class Dispatcher:
if (lang, key, "article") in self.validation_failures:
continue
md = node_markdown(self.data, node) or ""
title_key = None
if node.title and not has_h1(md):
md = f"# {node.title}\n\n{md}"
title_key = chunk_key(node.title)
covered = {(lang, k) for k in pend}
if title_key is not None:
covered.add((lang, title_key))
job = Job(
lang=lang, key=key, texts=[md], path=path, kind="article", mode="article"
)
return job, [], md, {(lang, k) for k in pend}
return job, [], md, covered, title_key
return None
def _pick(
self, state: _Connection, langs: list[str], inflight: set[tuple[str, bytes]]
) -> tuple[Job, list[Span], str, set[tuple[str, bytes]]] | None:
) -> _Offer | None:
"""The next job for a free connection: titles before articles before
chunks — across languages too, so every menu is named before any
article body is worked on (a page's name is its most visible
@@ -530,13 +557,14 @@ class Dispatcher:
offer = self._pick(state, langs, inflight)
if offer is None:
continue
job, spans, original, items = offer
job, spans, original, items, title_key = offer
state.inflight = (job.lang, job.key) # before the await: no double-assign
state.mode = job.mode
state.kind = job.kind
state.spans = spans
state.original = original
state.items = items
state.title_key = title_key
try:
await ws.send_text(msgspec.json.encode(job).decode())
except Exception: # send failed: the receive loop cleans up
@@ -550,6 +578,7 @@ class Dispatcher:
original: str,
spans: list[Span],
texts: list[str],
title_key: bytes | None = None,
) -> list[TransResult] | None:
"""Validate a Result against its in-flight job and turn it into
storable fragments; None when it fails validation (the caller skips
@@ -563,6 +592,22 @@ class Dispatcher:
pairs = align_article(original, texts[0]) if len(texts) == 1 else None
if not pairs:
return None
if title_key is not None:
# The job carried an injected "# {title}" heading: its pair
# becomes the title fragment (heading text only, never a body
# chunk). A demoted/merged heading just skips the title — it
# stays pending for a scoped title job.
heading = chunk_key(chunk_markdown(original)[0])
title = ""
kept = []
for k, t in pairs:
if k == heading and not title:
m = re.fullmatch(r"# (.+)", t)
if m and pure_prose(m.group(1)):
title = m.group(1)
continue
kept.append((k, t))
pairs = ([(title_key, title)] if title else []) + kept
return [TransResult(key=k, text=t) for k, t in pairs]
async def handle_ws(self, ws: WebSocket, clientkey: str) -> None:
@@ -617,9 +662,9 @@ class Dispatcher:
):
await ws.close(code=1002)
return
mode, kind, spans, original = state.take()
mode, kind, spans, original, title_key = state.take()
results = self._results(
mode, kind, msg.key, original, spans, msg.texts
mode, kind, msg.key, original, spans, msg.texts, title_key
)
if results is None:
# The model broke the contract (bad segment count,
+2 -1
View File
@@ -83,7 +83,8 @@ Rules:
- The text uses extended Markdown (container fences ::: name, {...} attributes, task lists, footnotes and more): all of it is formatting syntax and must be preserved exactly — only the human-readable text is translated.
- Newlines are significant: a single newline inside a paragraph renders as an actual line break, so keep the line structure exactly and never join, split or rewrap lines.
- Preserve the block structure exactly: same blocks separated by blank lines, same headings (# levels), lists, code fences, images and links; do not merge, split, add, drop or reorder blocks.
- Never translate or alter URLs, image destinations, code, or {...} placeholders. Image alt texts and link texts ARE translated."""
- Never translate or alter URLs, image destinations, code, or {...} placeholders. Image alt texts and link texts ARE translated.
- Prefer established technical loanwords with English roots over forced localizations — the jargon professionals actually use (in Finnish "frontend" becomes "frontti", not "etupääte")."""
def article_prompt(target: str, doc: str) -> str: