Mark phase-2 localization storage implemented; list chunks.py in AGENTS.md

This commit is contained in:
2026-09-02 01:48:47 +00:00
parent 98a1dc8d0f
commit f63bb2c482
3 changed files with 21 additions and 4 deletions
+16 -1
View File
@@ -1,6 +1,6 @@
# migrate_v3: content-addressed chunk storage
Status: **final plan**. `migrate_v3` restructures how article text and
Status: **implemented**. `migrate_v3` restructures how article text and
translations are stored, motivated by the localization model in
`docs/localization.md` (phase 2). Since it is a full migration, it is free to
break the current `Node.content: str | None` layout.
@@ -132,6 +132,21 @@ Chunking must be deterministic and shared with render/save, so
`pagerite/chunks.py`) and are imported by both `migrations.py` and
`views.py`/`app.py`.
## Implementation notes (deviations from the plan above)
- Chunking lives in `pagerite/chunks.py`; hashing uses the `blake3` package
(already a dependency) with a 16-byte digest (`hexdigest(16)`).
- `Translation.titles` stayed keyed by node path (phase-1 shape, views
untouched): `get_translation` builds it by walking the menu with the same
per-title `trans[f"{chunk_key(node.title)}:{lang}"]` lookups.
- Insert hunks anchor on the whole preceding block (not just its tail) —
a stronger, simpler search context.
- `make_patch` diffs with `SequenceMatcher(autojunk=False)` so patches are
deterministic (popular lines like blank separators never become junk).
- Step 3's path normalization is a no-op in practice: the only path-keyed
store (`patches`) starts empty at v3; analytics paths live outside the
kantadb. The code still strips leading slashes defensively.
## Garbage collection (later, manual or idle-time)
Orphaned entries accumulate: chunks no longer referenced by any