Keep container fence lines out of prose chunks

A closing ::: glued to a paragraph (no blank line before it) rode inside
the prose chunk and crossed to the translator as part of the text run;
when the model dropped it, validation passed and the splice lost the
fence — the rest of the page rendered inside the container (seen in the
Spanish translation). Container fence lines (::: openers and closers
alike) are now always their own prose-free chunk, never reaching the
translator. Affected pages re-chunk on next save and re-translate under
the new hashes, repairing themselves.
This commit is contained in:
2026-09-04 19:04:24 +00:00
parent 3f27a0a292
commit b57b7060ec
2 changed files with 24 additions and 4 deletions
+5 -1
View File
@@ -146,7 +146,11 @@ served Markdown at render time.
`chunk_markdown(markdown)` splits the source into block-level chunks —
blank-line-separated blocks: headings, paragraphs, code fences (kept whole),
list blocks, tables, HTML blocks. A chunk's identity is its **source text**,
list blocks, tables, HTML blocks. Container fence lines (`::: name` openers
and `:::` closers) are always their own chunk, blank lines or not — folded
into a prose chunk the closer would cross to the translator as part of the
text, where the model can drop it (the rest of the page then renders inside
the container). A chunk's identity is its **source text**,
gettext-msgid style:
```python
+19 -3
View File
@@ -17,6 +17,13 @@ from pagerite.segments import has_prose
#: backticks or tildes (CommonMark).
_FENCE_OPEN = re.compile(r"^ {0,3}(`{3,}|~{3,})")
#: A container fence line (mdit-py-plugins container): the "::: aside"
#: opener and the ":::" closer alike. Always its own block, even with no
#: blank line around it: folded into a prose paragraph it would cross to
#: the translator as part of the text run, where the model can drop it —
#: the rest of the page then renders inside the container.
_CONTAINER = re.compile(r"^ {0,3}:{3,}(?:[ \t]|$)")
#: HTML block openers that may span blank lines (CommonMark types 1-5:
#: script/pre/style/textarea, comments, processing instructions,
#: declarations, CDATA) with their closing condition. Other HTML blocks
@@ -54,9 +61,11 @@ def chunk_markdown(markdown: str) -> list[str]:
Blocks are separated by blank lines; fenced code blocks and the
multi-line HTML blocks (comments, script/pre/style, CDATA...) are
kept atomic, even across blank lines, and end at their closing
condition. Chunks carry no surrounding blank lines and no trailing
newline; rejoining with ``join_chunks`` reproduces the source modulo
blank-line normalization.
condition. Container fence lines (:::, open and close alike) are
always their own block, blank lines or not (see _CONTAINER). Chunks
carry no surrounding blank lines and no trailing newline; rejoining
with ``join_chunks`` reproduces the source modulo blank-line
normalization.
"""
chunks: list[str] = []
buf: list[str] = []
@@ -91,6 +100,13 @@ def chunk_markdown(markdown: str) -> list[str]:
fence = m.group(1)
buf.append(line)
continue
if _CONTAINER.match(line):
# Container fence lines (open and close alike) are their own
# block — never part of a prose chunk (see _CONTAINER).
flush()
buf.append(line)
flush()
continue
if not buf:
for open_re, close_re in _HTML_ATOMIC:
if open_re.match(line):