Keep container fence lines out of prose chunks
A closing ::: glued to a paragraph (no blank line before it) rode inside the prose chunk and crossed to the translator as part of the text run; when the model dropped it, validation passed and the splice lost the fence — the rest of the page rendered inside the container (seen in the Spanish translation). Container fence lines (::: openers and closers alike) are now always their own prose-free chunk, never reaching the translator. Affected pages re-chunk on next save and re-translate under the new hashes, repairing themselves.
This commit is contained in:
@@ -146,7 +146,11 @@ served Markdown at render time.
|
||||
|
||||
`chunk_markdown(markdown)` splits the source into block-level chunks —
|
||||
blank-line-separated blocks: headings, paragraphs, code fences (kept whole),
|
||||
list blocks, tables, HTML blocks. A chunk's identity is its **source text**,
|
||||
list blocks, tables, HTML blocks. Container fence lines (`::: name` openers
|
||||
and `:::` closers) are always their own chunk, blank lines or not — folded
|
||||
into a prose chunk the closer would cross to the translator as part of the
|
||||
text, where the model can drop it (the rest of the page then renders inside
|
||||
the container). A chunk's identity is its **source text**,
|
||||
gettext-msgid style:
|
||||
|
||||
```python
|
||||
|
||||
+19
-3
@@ -17,6 +17,13 @@ from pagerite.segments import has_prose
|
||||
#: backticks or tildes (CommonMark).
|
||||
_FENCE_OPEN = re.compile(r"^ {0,3}(`{3,}|~{3,})")
|
||||
|
||||
#: A container fence line (mdit-py-plugins container): the "::: aside"
|
||||
#: opener and the ":::" closer alike. Always its own block, even with no
|
||||
#: blank line around it: folded into a prose paragraph it would cross to
|
||||
#: the translator as part of the text run, where the model can drop it —
|
||||
#: the rest of the page then renders inside the container.
|
||||
_CONTAINER = re.compile(r"^ {0,3}:{3,}(?:[ \t]|$)")
|
||||
|
||||
#: HTML block openers that may span blank lines (CommonMark types 1-5:
|
||||
#: script/pre/style/textarea, comments, processing instructions,
|
||||
#: declarations, CDATA) with their closing condition. Other HTML blocks
|
||||
@@ -54,9 +61,11 @@ def chunk_markdown(markdown: str) -> list[str]:
|
||||
Blocks are separated by blank lines; fenced code blocks and the
|
||||
multi-line HTML blocks (comments, script/pre/style, CDATA...) are
|
||||
kept atomic, even across blank lines, and end at their closing
|
||||
condition. Chunks carry no surrounding blank lines and no trailing
|
||||
newline; rejoining with ``join_chunks`` reproduces the source modulo
|
||||
blank-line normalization.
|
||||
condition. Container fence lines (:::, open and close alike) are
|
||||
always their own block, blank lines or not (see _CONTAINER). Chunks
|
||||
carry no surrounding blank lines and no trailing newline; rejoining
|
||||
with ``join_chunks`` reproduces the source modulo blank-line
|
||||
normalization.
|
||||
"""
|
||||
chunks: list[str] = []
|
||||
buf: list[str] = []
|
||||
@@ -91,6 +100,13 @@ def chunk_markdown(markdown: str) -> list[str]:
|
||||
fence = m.group(1)
|
||||
buf.append(line)
|
||||
continue
|
||||
if _CONTAINER.match(line):
|
||||
# Container fence lines (open and close alike) are their own
|
||||
# block — never part of a prose chunk (see _CONTAINER).
|
||||
flush()
|
||||
buf.append(line)
|
||||
flush()
|
||||
continue
|
||||
if not buf:
|
||||
for open_re, close_re in _HTML_ATOMIC:
|
||||
if open_re.match(line):
|
||||
|
||||
Reference in New Issue
Block a user