LeoVasanko/pagerite
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Chunk keys as 9-byte bytes digests; trans nested by hash -> lang

Browse Source
This commit is contained in:
LeoVasanko
2026-09-02 02:09:27 +00:00
parent f63bb2c482
commit cc750236bc
6 changed files with 55 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/localization.md
+9 -8
View File
@@ -100,7 +100,7 @@ list blocks, tables, HTML blocks. A chunk's identity is its **source text**,
gettext-msgid style:
```python
chunk_key = blake3(normalize(chunk_text), digest_size=16).hex()
chunk_key = blake3(normalize(chunk_text)).digest(9) # bytes; base64 at the JSON level
```
(`normalize`: strip trailing whitespace per line, collapse surrounding blank
@@ -153,10 +153,11 @@ Full storage design and the `migrate_v3` restructuring live in
`docs/migrate.md`. The short version, as it concerns this document:
- Originals **and** translations are content-addressed text chunks in flat
stores: `Data.chunks: dict[hash, str]` and
`Data.trans: dict[f"{chunk_hash}:{lang}", str]` — path-independent, so
repeated paragraphs and menu titles are translated once and article moves
touch nothing. `Node.chunks: list[hash]` gives each article its order.
stores: `Data.chunks: dict[bytes, str]` and
`Data.trans: dict[bytes, dict[str, str]]` (chunk hash → lang → text) —
path-independent, so repeated paragraphs and menu titles are translated
once and article moves touch nothing. `Node.chunks: list[bytes]` gives
each article its order.
- `Node` gains **`language: str = ""`**, inherited down the tree like
`banner` (empty = nearest ancestor, front page last, site default `en`
final). `select_language` and `<html lang>` use the resolved value instead
@@ -177,7 +178,7 @@ def get_translation(path, lang, data) -> Translation | None:
if lang not in node.langs:
return None
hybrid = "\n\n".join(
chunks[h] if h in node.no_trans else trans.get(f"{h}:{lang}", chunks[h])
chunks[h] if h in node.no_trans else trans.get(h, {}).get(lang, chunks[h])
for h in node.chunks
)
for patch in data.patches.get(f"{path}:{lang}", []):
@@ -191,8 +192,8 @@ def get_translation(path, lang, data) -> Translation | None:
hreflang never probe the `trans` store chunk by chunk. A stale key is
benign (the "translation" just renders as the original).
- `titles` for nav/sidebar/cards: each node's translated title is
`trans.get(f"{hash(node.title)}:{lang}")` with per-node fallback — one dict
lookup per nav item at render time.
`trans.get(hash(node.title), {}).get(lang)` with per-node fallback — one
dict lookup per nav item at render time.
- Cache invalidation: writes to `chunks` / `trans` / `patches` (translator,
editor saves) call `_invalidate_pages()`, same as content writes.