Make kanta.diff/patch functions public API.

Validators receive the live data object (and optionally the Kanta
instance) and raise on inconsistency. They run after replay/migrations
during open() and after each transaction before the change is queued;
a failure rolls back the transaction or aborts the open. Multiple
validators run in registration order until the first failure. Sync-only,
since transactions are synchronous.

README.md

@@ -51,100 +51,14 @@ asyncio.run(main())
 3. Let Kanta flush queued changes to disk in the background.
 4. Use snapshots and replay for fast startup and full history.
 
-## Bootstrap and Open Modes
+## Documentation
 
-When `open()` creates a brand-new database, it always writes a single bootstrap
-change record from the initial data object you passed to `Kanta(...)`. The
-simplest bootstrap is therefore the object itself — no extra code is required.
+- [Usage patterns](https://git.zi.fi/LeoVasanko/kanta/src/branch/main/docs/usage.md) — opening, data ownership, and lifecycle patterns
+- [Bootstrap and open modes](https://git.zi.fi/LeoVasanko/kanta/src/branch/main/docs/bootstrap.md) — seeding new databases, strict and read-only opens
+- [Validation](https://git.zi.fi/LeoVasanko/kanta/src/branch/main/docs/validation.md) — `@kanta.validate` integrity checks on open and transactions
+- [Migrations](https://git.zi.fi/LeoVasanko/kanta/src/branch/main/docs/migrations.md) — versioned schema evolution with `migrate_vN`
+- [Retention and rotation](https://git.zi.fi/LeoVasanko/kanta/src/branch/main/docs/rotation.md) — bounding history to a time window
+- [Fatal error handlers](https://git.zi.fi/LeoVasanko/kanta/src/branch/main/docs/fatal-errors.md) — observing background write failures
+- [On-disk format](https://git.zi.fi/LeoVasanko/kanta/src/branch/main/docs/database.md) — record layout and invariants
 
-Bootstrap handlers are optional. Use them only when you need to modify the
-initial state at creation time, for example to seed defaults or perform
-expensive/external setup that should happen exactly once:
-
-```python
-kanta = Kanta("data.kantadb", Data())
-
-@kanta.bootstrap(action="seed", user="system")
-def seed_defaults(data) -> None:
-    data.users["admin"] = User(name="Admin")
-
-await kanta.open()
-```
-
-You can also use `@kanta.bootstrap` with no arguments and async handlers:
-
-```python
-@kanta.bootstrap
-async def bootstrap_async(data) -> None:
-    data.counter = 1
-```
-
-Whether or not handlers are registered, exactly one bootstrap change record is
-written when a new database is created. The record contains the initial object,
-or the state after all bootstrap handlers have run. When handlers are present:
-- they run in registration order,
-- bootstrap metadata (`action`, `user`, `mtime`) is taken from the last
-  registration.
-
-If any bootstrap handler raises, Kanta closes and removes the database file,
-then re-raises the error.
-
-`open()` also supports strict open mode:
-
-```python
-await kanta.open(create=False)
-```
-
-With `create=False`, open fails if the database file does not exist or is
-empty.
-
-Read-only mode opens an existing database without locking it or starting the
-background flush task. This is useful for readers that must not block the
-writer or modify the file:
-
-```python
-await kanta.open(readonly=True)
-```
-
-In read-only mode, records are replayed and migrations are applied in memory,
-but transactions and explicit flushes are rejected and the file is never
-created if missing.
-
-## Fatal Error Handlers
-
-Fatal background write errors can be observed with a decorator:
-
-```python
-import os
-import signal
-
-@kanta.fatal_error
-async def on_fatal(err):
-    os.kill(os.getpid(), signal.SIGTERM)  # Die
-```
-
-Multiple fatal handlers are supported and run in registration order.
-
-## Migrations
-
-Adding or removing a field and other such simple operations are automatic, but when the time comes to really change your data model, implement a `migrate_v1` function that converts your old data to the new form. This works on plain built-in dict and other types, to avoid needing to preserve old versions of your structs.
-
-Pass a module (or import path) containing `migrate_vN` functions:
-
-```python
-kanta = Kanta("data.kantadb", Data(), migrations="myapp.migrations")
-await kanta.open()
-```
-
-Kanta tracks migration version metadata automatically, and fast forwards your database to current version by running all the migrations needed while opening the database.
-
-## On-Disk Format
-
-Kanta in JSON mode (default) stores newline-delimited records. Transaction history is viewable by any simple text editor, and rollbacks to prior state are done by simply removing final lines (one per transaction)
-
-MsgPack mode uses binary records with length and checksum to avoid data corruption.
-
-- Change line: JSON object with metadata + `diff`
-- Snapshot line: `SNAPSHOT { ... full state ... }`
-
-See `docs/database.md` for format details and invariants.
+Kanta in JSON mode (default) stores newline-delimited records, so transaction history is viewable in any text editor; MsgPack mode uses binary records with length and checksum to guard against corruption.

demo/demo.py

@@ -6,9 +6,7 @@ from pathlib import Path
 
 import msgspec
 
-from kanta import Kanta
-from kanta.callbacks import DictPre
-from kanta.logging import configure_logging
+from kanta import Kanta, configure_logging
 
 
 filename = Path(__file__).with_name("demo.kantadb")
@@ -47,11 +45,11 @@ kanta_v1 = Kanta(filename, Data(), migrations=sys.modules[__name__])
 
 
 @kanta_v1.logfmt
-def resolve_user(value: str, path: str, previous: DictPre) -> str | None:
+def resolve_user(value: str, path: str, state: dict) -> str | None:
     """Resolve user ids to names from the database state itself."""
     if path != "$user" and not path.startswith("users."):
         return None
-    return previous.get("users", {}).get(value, {}).get("name")
+    return state.get("users", {}).get(value, {}).get("name")
 
 
 async def main() -> None:
@@ -85,7 +83,9 @@ async def main() -> None:
                 data.total = 99
                 raise ValueError("simulated failure")
         except ValueError:
-            print(f"\nReset rolled back: {data.total=} (we can always read data without tx)\n")
+            print(
+                f"\nReset rolled back: {data.total=} (we can always read data without tx)\n"
+            )
 
         with kanta.transaction(action="delete", user="userid002") as data:
             del data.users["userid001"]

docs/bootstrap.md

@@ -0,0 +1,48 @@
+# Bootstrap and Open Modes
+
+When `open()` creates a brand-new database, it always writes a single bootstrap change record from the initial data object you passed to `Kanta(...)`. The simplest bootstrap is therefore the object itself — no extra code is required.
+
+Bootstrap handlers are optional. Use them only when you need to modify the initial state at creation time, for example to seed defaults or perform expensive/external setup that should happen exactly once:
+
+```python
+kanta = Kanta("data.kantadb", Data())
+
+@kanta.bootstrap(action="seed", user="system")
+def seed_defaults(data) -> None:
+    data.users["admin"] = User(name="Admin")
+
+await kanta.open()
+```
+
+You can also use `@kanta.bootstrap` with no arguments and async handlers:
+
+```python
+@kanta.bootstrap
+async def bootstrap_async(data) -> None:
+    data.counter = 1
+```
+
+Whether or not handlers are registered, exactly one bootstrap change record is written when a new database is created. The record contains the initial object, or the state after all bootstrap handlers have run. When handlers are present:
+
+- they run in registration order,
+- bootstrap metadata (`action`, `user`, `mtime`) is taken from the last registration.
+
+If any bootstrap handler raises, Kanta closes and removes the database file, then re-raises the error.
+
+## Strict open mode
+
+```python
+await kanta.open(create=False)
+```
+
+With `create=False`, open fails if the database file does not exist or is empty.
+
+## Read-only mode
+
+Read-only mode opens an existing database without locking it or starting the background flush task. This is useful for readers that must not block the writer or modify the file:
+
+```python
+await kanta.open(readonly=True)
+```
+
+In read-only mode, records are replayed and migrations are applied in memory, but transactions and explicit flushes are rejected and the file is never created if missing.

docs/database.md

@@ -1,13 +1,13 @@
 # Kanta Database Format and Design Principles
 
-This document describes the on-disk format and design principles of Kanta.
-It is intentionally focused on the current standalone package behavior.
+This document describes the on-disk format and design principles of Kanta. It is intentionally focused on the current standalone package behavior.
 
 ## Core Principles
 
 1. Append-only durability
 - State changes are persisted as appended JSON lines.
 - Existing lines are never edited in place.
+- In JSON mode the history is viewable in any text editor, and a manual rollback to a prior state is possible by removing final lines (one per transaction).
 
 2. Differential persistence
 - Kanta stores diffs (patches), not full state, for normal writes.
@@ -28,8 +28,7 @@ It is intentionally focused on the current standalone package behavior.
 
 ## On-Disk Record Types
 
-Kanta uses a newline-delimited stream where each line is either a change
-record or a snapshot record.
+Kanta uses a newline-delimited stream where each line is either a change record or a snapshot record.
 
 ### Change record
 
@@ -68,29 +67,24 @@ Fields:
 3. Replay subsequent change records in order using patch application.
 4. The final replay state becomes in-memory `kanta.data`.
 
-This model provides fast startup for large logs while retaining append-only
-history.
+This model provides fast startup for large logs while retaining append-only history.
 
 ## Serialization Semantics
 
 - In-memory data is defined by an application `msgspec.Struct` type.
 - Kanta round-trips through plain builtins for persistence and diffing.
 - Dict keys are serialized as strings (`str_keys=True`) for stable JSON form.
-- Normalization changes introduced by struct decode/encode are logged together
-  with migrations as `migrate:vN`, or as `migrate:msgspec` when no migration
-  ran but normalization still produces a diff.
+- Normalization changes introduced by struct decode/encode are logged together with migrations as `migrate:vN`, or as `migrate:msgspec` when no migration ran but normalization still produces a diff.
 
 ## Transaction Semantics
 
 - `kanta.transaction(action=...)` captures a pre-transaction snapshot dict.
-- By default a transaction updates the modification time `m` to the current UTC
-  time.
+- By default a transaction updates the modification time `m` to the current UTC time.
 - `mtime=True|False|datetime` controls the modification time `m`:
   - `True` (default) sets `m` to the current UTC time.
   - `False` omits `m`, leaving the previous modification time in effect.
   - A `datetime` sets `m` to that explicit value.
-- System operations such as `migrate:msgspec` use `mtime=False` so they are not
-  considered modifications and do not advance `m`.
+- System operations such as `migrate:msgspec` use `mtime=False` so they are not considered modifications and do not advance `m`.
 - On success:
   - compute diff between previous builtins and current builtins,
   - queue a `ChangeRecord` if non-empty,
@@ -103,9 +97,7 @@ Nested transactions are rejected.
 
 ## Modification Time
 
-`kanta.mtime` exposes the last modification time carried forward from change
-records. It is updated by normal transactions and preserved across snapshots and
-reloads, while system operations such as migrations leave it unchanged.
+`kanta.mtime` exposes the last modification time carried forward from change records. It is updated by normal transactions and preserved across snapshots and reloads, while system operations such as migrations leave it unchanged.
 
 ## Flush and Lifecycle
 
@@ -120,89 +112,57 @@ reloads, while system operations such as migrations leave it unchanged.
 - `await kanta.open()` (default) creates the database file if missing.
 - `await kanta.open(create=False)` fails when the file is missing or empty.
 - `await kanta.open(readonly=True)` opens an existing database read-only.
-  - The file is opened without acquiring a lock and without a background flush
-    task.
+  - The file is opened without acquiring a lock and without a background flush task.
   - Existing records are replayed and migrations are still applied in memory.
   - Transactions and explicit flushes are rejected.
   - The file is never created if missing.
 
 ### Callbacks
 
-All callbacks are registered via decorators and receive arguments by their
-annotation types.  Parameters without a supported annotation are only allowed
-when they have a default value.
+All callbacks are registered via decorators and receive arguments by their annotation types. Parameters without a supported annotation are only allowed when they have a default value.
 
 #### Bootstrap Callbacks
 
-- When `open()` creates a new database, it always writes a single bootstrap
-  `ChangeRecord`.
-- The simplest bootstrap is the initial data object passed to `Kanta(...)`;
-  bootstrap callbacks are optional and only needed when you want to modify or
-  enrich that object at creation time.
+- When `open()` creates a new database, it always writes a single bootstrap `ChangeRecord`.
+- The simplest bootstrap is the initial data object passed to `Kanta(...)`; bootstrap callbacks are optional and only needed when you want to modify or enrich that object at creation time.
 - Register callbacks via:
   - `@kanta.bootstrap`
   - `@kanta.bootstrap(action=..., user=..., mtime=...)`
-- Bootstrap callbacks may be sync or async.  The live root data object is
-  injected by annotating a parameter with the struct type passed to `Kanta`,
-  and the `Kanta` instance itself can be injected by annotating a parameter
-  with `Kanta`.
+- Bootstrap callbacks may be sync or async. The live root data object is injected by annotating a parameter with the struct type passed to `Kanta`, and the `Kanta` instance itself can be injected by annotating a parameter with `Kanta`.
 - Multiple bootstrap callbacks are supported:
   - callbacks execute in registration order,
   - exactly one bootstrap `ChangeRecord` is queued,
-  - bootstrap metadata (`action`, `user`, `mtime`) is taken from the last
-    callback registration.
-- If no bootstrap callbacks are registered, the bootstrap record still uses
-  `action="bootstrap"` and contains the initial data object.
-- If any bootstrap callback raises, Kanta closes and removes the database file,
-  then re-raises the exception.
+  - bootstrap metadata (`action`, `user`, `mtime`) is taken from the last callback registration.
+- If no bootstrap callbacks are registered, the bootstrap record still uses `action="bootstrap"` and contains the initial data object.
+- If any bootstrap callback raises, Kanta closes and removes the database file, then re-raises the exception.
 
 #### Fatal Error Handlers
 
 - Fatal background persistence errors can be handled with `@kanta.fatal_error`.
-- Handlers may be sync or async.  The `DatabaseError` is injected by annotating
-  a parameter with `DatabaseError`; `Kanta` may also be injected.
-- Multiple handlers are supported and invoked in registration order.  A failing
-  handler is logged and does not prevent subsequent handlers from running.
+- Handlers may be sync or async. The `DatabaseError` is injected by annotating a parameter with `DatabaseError`; `Kanta` may also be injected.
+- Multiple handlers are supported and invoked in registration order. A failing handler is logged and does not prevent subsequent handlers from running.
 
 #### Clock
 
-- `@kanta.clock` registers a callback `() -> datetime` that replaces the
-  default UTC clock.  Its value is used for all record timestamps (`ts`, and
-  `m` when `mtime` is `True`) and for snapshot timestamps.
-- The clock is only read when a timestamp is actually produced; no-op
-  transactions and skipped snapshot checks do not read it.
-- Register before `open()` so that bootstrap and migration records use the
-  custom clock as well.  This is mainly useful for tests and reproducible
-  demos.
+- `@kanta.clock` registers a callback `() -> datetime` that replaces the default UTC clock. Its value is used for all record timestamps (`ts`, and `m` when `mtime` is `True`) and for snapshot timestamps.
+- The clock is only read when a timestamp is actually produced; no-op transactions and skipped snapshot checks do not read it.
+- Register before `open()` so that bootstrap and migration records use the custom clock as well. This is mainly useful for tests and reproducible demos.
 
 #### Transaction Log Formatting
 
-- Logfmt callbacks prettify identifiers in the change log and are registered with
-  `@kanta.logfmt`.
-- A logfmt callback is called for every value Kanta renders: diff values, path
-  components, and the transaction `user`.  It receives the value as its first
-  parameter and optionally a `path: str` parameter with the dot-notation path
-  to the value.  The special path `"$user"` is used when rendering the
-  transaction actor, replacing the old `user_display` parameter.
-- The callback returns `str | None`: a string replaces the default rendering,
-  while `None` means "fall through to the next formatter".
-- State dicts can be injected via `DictPre` (`Annotated[dict, "pre"]`)
-  and `DictPost` (`Annotated[dict, "post"]`); the `Kanta` instance can also be
-  injected.
-- Alternatively, a logfmt callback can be a class inheriting from `LogFmt`; the
-  framework instantiates it with the state dicts and calls its
-  `resolve(value, path) -> str | None` method.
-- Multiple logfmt callbacks are stacked in registration order; the first
-  callback to return a non-`None` result wins.  If none handle a value, Kanta
-  falls back to its default formatting.
+- Logfmt callbacks prettify identifiers in the change log and are registered with `@kanta.logfmt`.
+- A logfmt callback is called for every value Kanta renders: diff values, path components, and the transaction `user`. It receives the value as its first parameter and optionally a `path: str` parameter with the dot-notation path to the value. The special path `"$user"` is used when rendering the transaction actor.
+- The callback returns `str | None`: a string replaces the default rendering, while `None` means "fall through to the next formatter".
+- State dicts are injected by parameter name or annotation tag, which share the same vocabulary: `prev` receives the previous state dict and `state` the current one. Matching by name ignores the annotation entirely. The `DictPrev`/`DictState` aliases (`Annotated[dict, "prev"]` / `Annotated[dict, "state"]`) work under any parameter name, and a tag takes precedence over the name. `DictPre` and `DictPost` are kept as aliases of `DictPrev` and `DictState`. The `Kanta` instance can also be injected.
+- Alternatively, a logfmt callback can be a class inheriting from `LogFmt`; the framework instantiates it with the state dicts and calls its `resolve(value, path) -> str | None` method.
+- Multiple logfmt callbacks are stacked in registration order; the first callback to return a non-`None` result wins. If none handle a value, Kanta falls back to its default formatting.
 
-The decorator accepts an optional ``path`` so the callback only runs for
-values at that exact path:
+The decorator accepts an optional ``path`` so the callback only runs for values at that exact path:
 
 ```python
 @kanta.logfmt(path="$user")
-def resolve_user(value: str, current: DictPost) -> str | None:
-    return current.get("users", {}).get(value, {}).get("name")
+def resolve_user(value: str, state: dict) -> str | None:
+    return state.get("users", {}).get(value, {}).get("name")
 
 @kanta.logfmt(path="users.uuid-1")
 def resolve_user_key(value: str) -> str | None:
@@ -211,63 +171,21 @@ def resolve_user_key(value: str) -> str | None:
 
 #### Transaction Log Headers
 
-- By default a transaction is logged with an `action by user` header followed
-  by the diff lines.  Added paths are colored green, deleted paths red.
-- `kanta.transaction(..., extra=...)` accepts a display-only value that is
-  shown after the action in the header.  Anything other than `None` is
-  printed str-converted (colored by Kanta), unless a custom logemit handler
-  does something else with it; it is never persisted in the `ChangeRecord`.
-- `kanta.transaction(..., logdiff=False)` skips building and printing the diff
-  body and logs only the header, which is useful for large or noisy
-  changesets.  Diff output can also be disabled globally with
-  `configure_logging(diff=False)`; diff lines are emitted on the
-  `kanta.transaction.diff` child logger so applications can route or silence
-  them separately from the headers.
+- By default a transaction is logged with an `action by user` header followed by the diff lines. Added paths are colored green, deleted paths red. ANSI color codes are stripped after formatting when the standard error stream does not support color: `NO_COLOR` disables colors, `FORCE_COLOR` forces them, otherwise a tty check and a journald (`JOURNAL_STREAM`) check decide. The CLI (`python -m kanta`) strips its output the same way.
+- `kanta.transaction(..., extra=...)` accepts a display-only value that is shown after the action in the header. Anything other than `None` is printed str-converted (colored by Kanta), unless a custom logemit handler does something else with it; it is never persisted in the `ChangeRecord`.
+- `kanta.transaction(..., logdiff=False)` skips building and printing the diff body and logs only the header, which is useful for large or noisy changesets. Diff output can also be disabled globally with `configure_logging(diff=False)`; diff lines are emitted on the `kanta.transaction.diff` child logger so applications can route or silence them separately from the headers.
 
 #### Log Emitters
 
-- Every change-related message Kanta emits (transaction/bootstrap/migration
-  changes, file created/opened lines, migration summaries, aborted
-  transactions) is described by a `kanta.logging.LogEvent` and dispatched
-  through
-  `kanta.logging.emit_event`.  Kanta's own output goes through the same
-  mechanism: when no `logemit` callback handles an event,
-  `kanta.logging.default_emit` renders it with the built-in formatting.
-- A `LogEvent` carries the event `kind` (`"change"`, `"created"`,
-  `"opened"`, `"migrated"`, `"aborted"`), the preferred `logger` and `level`,
-  the
-  `kanta` instance, and all relevant state: `action`, `user`, `extra`,
-  `error` (for aborted transactions), `diff`, `previous`/`current` state
-  dicts, the built `logfmt` chain, and version info for migration events.
-  Application-specific context (e.g. a connection id) can be stored in
-  `kanta.ctx` — a user-writable namespace — and read back in callbacks as
-  `event.kanta.ctx`, which also covers creation/bootstrap events.
-- The built-in formatting is assembled from standard blocks that custom
-  emitters can reuse as-is or replace piecemeal:
-  - `event.header` — a lazy property producing the default one-line header
-    for any kind: `<action>[ <extra>][ by <user>]` for changes,
-    `<action>[ <extra>][ by <user>] transaction aborted: <error>` for aborts,
-    and the `🛢️ <file> created|opened|migrated ...` summaries.  It is
-    settable: assign
-    `event.header = ...` and return truthy to restyle the header while
-    keeping the default diff routing.
-  - `event.diff_lines` — a lazy property producing the pretty diff body for
-    change events (built only if accessed).
+Every change-related message Kanta emits (transaction/bootstrap/migration changes, file created/opened lines, migration summaries, aborted transactions) is described by a `kanta.logging.LogEvent` and dispatched through `kanta.logging.emit_event`. Kanta's own output goes through the same mechanism: when no `logemit` callback handles an event, `kanta.logging.default_emit` renders it with the built-in formatting.
+
+- A `LogEvent` carries the event `kind` (`"change"`, `"created"`, `"opened"`, `"migrated"`, `"aborted"`), the preferred `logger` and `level`, the `kanta` instance, and all relevant state: `action`, `user`, `extra`, `error` (for aborted transactions), `diff`, `previous`/`current` state dicts, the built `logfmt` chain, and version info for migration events. Application-specific context (e.g. a connection id) can be stored in `kanta.ctx` — a user-writable namespace — and read back in callbacks as `event.kanta.ctx`, which also covers creation/bootstrap events.
+- The built-in formatting is assembled from standard blocks that custom emitters can reuse as-is or replace piecemeal:
+  - `event.header` — a lazy property producing the default one-line header for any kind: `<action>[ <extra>][ by <user>]` for changes, `<action>[ <extra>][ by <user>] transaction aborted: <error>` for aborts, and the `🛢️ <file> created|opened|migrated ...` summaries. It is settable: assign `event.header = ...` and return truthy to restyle the header while keeping the default diff routing.
+  - `event.diff_lines` — a lazy property producing the pretty diff body for change events (built only if accessed).
   - `default_emit` itself is just `header` plus the `diff_lines` routing.
-- `@kanta.logemit` registers a callback receiving the event.  The callback
-  decides what is logged and where: it may log one or more messages on
-  `event.logger`, log somewhere else, or nothing at all.  A falsy return
-  value marks the event handled and stops the chain; a truthy return value
-  passes the event — possibly modified — to the next registered callback.
-  When all callbacks pass, `default_emit` renders the event; a callback may
-  also call `default_emit(event)` itself to delegate events it does not
-  customize.  Operational diagnostics (integrity errors, background flush
-  failures) do not go through this mechanism.
-- Logging never breaks functionality: a crashing `logemit` callback is
-  reported with `logger.exception` and the event falls back to the built-in
-  formatting; if the built-in formatting itself fails, the error is reported
-  and swallowed.  The same applies to `logfmt` callbacks (a failing one is
-  treated as a fall-through) and `logmigr` callbacks.
+- `@kanta.logemit` registers a callback receiving the event. The callback decides what is logged and where: it may log one or more messages on `event.logger`, log somewhere else, or nothing at all. A falsy return value marks the event handled and stops the chain; a truthy return value passes the event — possibly modified — to the next registered callback. When all callbacks pass, `default_emit` renders the event; a callback may also call `default_emit(event)` itself to delegate events it does not customize. Operational diagnostics (integrity errors, background flush failures) do not go through this mechanism.
+- Logging never breaks functionality: a crashing `logemit` callback is reported with `logger.exception` and the event falls back to the built-in formatting; if the built-in formatting itself fails, the error is reported and swallowed. The same applies to `logfmt` callbacks (a failing one is treated as a fall-through) and `logmigr` callbacks.
 
 ```python
 @kanta.logemit
@@ -282,17 +200,10 @@ def emit(ev: LogEvent):
 #### Terminal Formatting Helpers
 
 - `kanta.tty` provides the building blocks used by Kanta's own rendering:
-  - `colors`: the mutable color palette.  Colors are bare SGR parameter
-    strings (e.g. `"1;34"`, `"38;5;226"`) without escape framing.  Attributes
-    are read at render time, so assignments (`colors.action = "36"`) and
-    additions (`colors.session = "38;5;226"`) take effect immediately.
-  - `Line`: builds a terminal string part by part.  Calling it appends
-    content (`str`-converted); `.<colorname>` arms a palette color for the
-    next call only, and the reset is folded into a single escape sequence
-    with whatever color comes next.  `width=`/`align=` pad by display width;
-    `str(line)` finishes the line and restores default colors.
-  - `strip_ansi`, `displaywidth` (wide chars and emoji count correctly) and
-    `pad` for working with pre-colored strings.
+  - `colors`: the mutable color palette. Colors are bare SGR parameter strings (e.g. `"1;34"`, `"38;5;226"`) without escape framing. Attributes are read at render time, so assignments (`colors.action = "36"`) and additions (`colors.session = "38;5;226"`) take effect immediately.
+  - `Line`: builds a terminal string part by part. Calling it appends content (`str`-converted); `.<colorname>` arms a palette color for the next call only, and the reset is folded into a single escape sequence with whatever color comes next. `width=`/`align=` pad by display width; `str(line)` finishes the line and restores default colors.
+  - `strip_ansi`, `displaywidth` (wide chars and emoji count correctly) and `pad` for working with pre-colored strings.
+  - `use_color(stream)`: the color-support test used by Kanta's own output — honors `NO_COLOR`/`FORCE_COLOR`, then `stream.isatty()`, then the journald `JOURNAL_STREAM` device/inode match.
 
 ## Migrations
 
@@ -304,8 +215,6 @@ def emit(ev: LogEvent):
 
 ## Safety Invariants
 
-- Any detected out-of-transaction mutation is treated as a fatal consistency
-  violation.
+- Any detected out-of-transaction mutation is treated as a fatal consistency violation.
 - Flush failures mark the instance as failed and trigger shutdown behavior.
-- Object identity of `kanta.data` is preserved across rollback when possible,
-  minimizing stale-reference hazards for callers.
+- Object identity of `kanta.data` is preserved across rollback when possible, minimizing stale-reference hazards for callers.

docs/fatal-errors.md

@@ -0,0 +1,14 @@
+# Fatal Error Handlers
+
+Fatal background write errors can be observed with a decorator:
+
+```python
+import os
+import signal
+
+@kanta.fatal_error
+async def on_fatal(err):
+    os.kill(os.getpid(), signal.SIGTERM)  # Die
+```
+
+Multiple fatal handlers are supported and run in registration order.

docs/migrations.md

@@ -0,0 +1,12 @@
+# Migrations
+
+Adding or removing a field and other such simple operations are automatic, but when the time comes to really change your data model, implement a `migrate_v1` function that converts your old data to the new form. This works on plain built-in dict and other types, to avoid needing to preserve old versions of your structs.
+
+Pass a module (or import path) containing `migrate_vN` functions:
+
+```python
+kanta = Kanta("data.kantadb", Data(), migrations="myapp.migrations")
+await kanta.open()
+```
+
+Kanta tracks migration version metadata automatically, and fast forwards your database to current version by running all the migrations needed while opening the database.

docs/rotation.md

@@ -0,0 +1,70 @@
+# Database Rotation
+
+Kanta can bound the on-disk history of a database to a configurable retention window (e.g. the last 30 days) by *rotating* the database file: the aged-out content is copied to a timestamped sibling file and the main file is truncated and rewritten in place with only the retained history plus fresh snapshots. Normal operation stays append-only under the exclusive lock; rotation is the only operation that rewrites the file.
+
+## Configuration
+
+Rotation is enabled with a keyword option on `Kanta(...)`:
+
+- `retention: timedelta | int | None = None` — history window to keep; a plain `int` is interpreted as a number of days. `None` (default) disables rotation entirely.
+
+Rotation uses the same clock as record timestamps, so a custom `@kanta.clock` callback controls it as well (useful in tests).
+
+## When rotation runs
+
+Rotation happens inside `Kanta.open()`, after acquiring the exclusive lock and before replay. At that point the file is quiescent: no background flush loop is running yet and no records are in flight. Rotation therefore never races with the background writer, snapshot requests, or migration snapshot writes.
+
+Consequence: a database that is never reopened never rotates. For long-running services, rotation takes effect on the next restart.
+
+## Rotated file naming
+
+The history that aged out is preserved at:
+
+```
+{stem}@{ISO-8601 timestamp}.kantadb
+```
+
+- `{stem}` is the original filename with its extension stripped (`Path(filename).stem`), so databases named `data`, `data.kantadb`, or `data.db` all rotate to `data@….kantadb`.
+- The timestamp is the `ts` of the last record dropped by the rotation (the leading snapshot of the rewritten main file carries the same `ts`), not the rotation time — the name tells you exactly which point in history the rotated file ends at. It is rendered in ISO 8601 basic format at second precision (e.g. `20260902T143000Z`); the exact microsecond timestamp of the cutoff remains available inside the file as the `ts` of its final line and of the leading snapshot of the new file.
+- Rotated files live in the same directory as the main file.
+- On collision (a rotated file with the same name already exists), an incrementing suffix is inserted before the extension (`data@….1.kantadb`, `data@….2.kantadb`, …) rather than overwriting.
+- Rotated files are never deleted by rotation.
+
+## Rotation algorithm
+
+Let `cutoff = now - retention`. All planning happens on the in-memory bytes of the database file already read by open; no second disk read is needed.
+
+1. **Eligibility.** Rotation is skipped when there is nothing to do: when the file contains no change records older than `cutoff` (the retention window already covers all history), or when the file contains no change records at all (a snapshot-only file is treated as already fully rotated and never re-rotated).
+
+2. **Replay base.** The base is the newest snapshot whose `ts <= cutoff`, falling back to the start of file when no such snapshot exists. Starting at the most recent snapshot would silently drop history that must be retained.
+
+3. **Replay and validate.** The file is replayed from the base forward to the end. Records with `ts < cutoff` are applied to the replay (they are needed to reach the cutoff state) but not retained in the output. At every snapshot encountered after the base, the replayed state is validated against the snapshot state; a mismatch means the history is corrupt, and rotation is aborted with a `DatabaseError`, leaving the original file untouched. The byte offset just after the last record with `ts < cutoff` (frame-boundary aligned) is remembered as `cutoff_end`.
+
+4. **Copy the original aside.** The main file is copied with `shutil.copy2` directly to its final `{stem}@{ts}.kantadb` name. On filesystems with copy-on-write this performs a cheap reflink copy. A failure here aborts rotation with the original file intact.
+
+5. **Rewrite the main file in place.** On the locked file descriptor the content is replaced (seek 0, truncate, write, fsync) with, in order:
+   1. A **snapshot of the state at the cutoff**, stamped with the schema version and modification time in effect at the cutoff. Its `ts` is the `ts` of the last pre-cutoff record — the same timestamp used in the rotated filename. This snapshot is the new replay base and carries the version forward so migrations are not re-run; it is always written.
+   2. The retained change records (`ts >= cutoff`), recreated record by record — no internal snapshots are carried over.
+   3. A **final snapshot** of the state after the last retained record, written only when enough changes were retained to warrant one (the same policy as regular snapshot writes). If no records survived the cutoff, the new file consists of the single leading snapshot and nothing else — the steady state for databases whose history has fully aged out.
+
+6. **Trim the rotated copy.** The rotated file is truncated to `cutoff_end` bytes, so it contains only the dropped history and does not duplicate the records retained in the main file. The cut is at a frame boundary, so the rotated file remains a valid, replayable database on its own. This happens only after step 5's fsync, so until then the rotated file still holds the complete original content as a crash-recovery anchor.
+
+7. **Continue normal open.** Replay and migrations proceed on the same locked file. Because the leading snapshot carries the current version, migrations run exactly as they would have against the old content.
+
+Failure rule: any error before step 5 leaves the main file byte-identical (only an extra copy exists). A crash during step 5 may leave the main file torn, but the rotated copy still holds the complete original content — recovery is copying it back. After step 6 the split is complete and both files are consistent.
+
+## Design notes
+
+**In-place rewrite, not rename-and-recreate.** The writer holds an exclusive `flock` on the file from `open()` until `close()`, and `flock` is attached to the open file description (inode), not the path. Renaming the locked file away and creating a fresh file at the main path would open a race: between the rename and the creation, a second instance could open the missing path with `O_CREAT`, acquire its own lock on the fresh inode, and bootstrap a divergent database. On Windows, renaming the locked file would fail outright (the database is opened with `FILE_SHARE_READ` only). Rotation therefore never renames or unlinks the main file and never releases its lock; a second instance opening the path at any moment gets either the old content or the new, and never its own lock. The only primitive this requires is `LockedFile.replace_content()` (seek 0, truncate, write, fsync).
+
+**Rewrite (re-frame), not verbatim copy**, for all records written to the main file. `BinFramer` checksums are offset-keyed (the checksum includes the absolute `record_offset`), so a verbatim byte copy to a new offset would be unreadable; binary records are re-framed at their new offsets. Rewriting also normalizes encoding drift and lets rotation drop the redundant intermediate snapshots the original file accumulated. The rotated copy is the one place where verbatim bytes are used — a raw `copy2` plus a frame-aligned tail truncation — which is safe precisely because it preserves original offsets: the truncated prefix keeps every frame at its original `record_offset`, so binary checksums stay valid.
+
+## Integrity guarantees
+
+- Rotation runs under the exclusive lock, before the background writer starts.
+- The main path is never renamed, unlinked, or unlocked during rotation; no bootstrap race with a second instance is possible.
+- Replay is validated against every snapshot in range; any validation failure aborts rotation with the original file intact.
+- A leading cutoff snapshot (ts = last pre-cutoff record, schema version at the cutoff) is always written; a final snapshot is written only when warranted by retained changes.
+- The full original content sits at `{stem}@{ts}.kantadb` before the main file is touched, and is only trimmed to the dropped-history prefix after the rewritten main file is fsynced.
+- Rotated files are never deleted by rotation.
+- Files with no change records (already reduced to a snapshot) are never re-rotated.

docs/usage.md

@@ -0,0 +1,96 @@
+# Usage Patterns: Opening and Data Handling
+
+This document covers the basics of opening a database and working with the live data object: who owns it, where initial data comes from, and the lifecycle patterns Kanta supports.
+
+## The data object belongs to you
+
+You pass the root state object to `Kanta(...)`, and Kanta never replaces it with a new instance. Replay, migrations, transaction rollbacks — all restore or mutate the object's contents in place, preserving its identity. This means you may hold external references to the object (or to parts of it) and they stay valid for the lifetime of the `Kanta` instance:
+
+```python
+data = Data()
+kanta = Kanta("data.kantadb", data)
+await kanta.open()
+
+assert kanta.data is data  # always the same object
+```
+
+Access the state as `kanta.data`, via your own reference, or both — they are the same object. Mutations must happen inside a transaction (see below); Kanta treats any detected out-of-transaction mutation as a fatal consistency violation.
+
+Note that replacing the whole object is also possible (`kanta.data = Data()` has a setter), but then previously held references point at the old object — prefer in-place mutation.
+
+## Where initial data comes from
+
+The object passed to `Kanta(...)` seeds the database: when `open()` creates a brand-new (missing or empty) file, it writes a single bootstrap change record containing that object's contents, optionally modified by `@kanta.bootstrap` handlers (see [Bootstrap and open modes](bootstrap.md)).
+
+When the file already exists, the constructor argument is *not* used as state — replay rebuilds the contents of `self.data` in place from the stored records, and the argument only defines the struct type. Consequence: if you close an instance, delete the file, and `open()` again, the new database is bootstrapped from the object's *current* contents — the old database's final state, not its original initial values. For a genuinely fresh start, construct a new data object (or reset the fields in a bootstrap handler).
+
+## Lifecycle patterns
+
+### Context manager (single database, scoped lifetime)
+
+```python
+async with Kanta("data.kantadb", Data()) as kanta:
+    with kanta.transaction(action="create_user") as data:
+        data.users[user_id] = User(name="Alice")
+```
+
+`async with` guarantees `open()`/`close()` pairing: the final flush happens on exit even on errors. Both `as` bindings are pure shortcuts — `async with Kanta(...) as kanta` binds the `Kanta` instance itself, and `with kanta.transaction(...) as data` binds exactly `kanta.data`. Use them or don't:
+
+```python
+async with Kanta("data.kantadb", Data()) as db:
+    with db.transaction(action="rename"):
+        db.data.users[user_id].name = "Bob"  # same object as `data` above
+```
+
+### Module-level global (typical application state)
+
+When the database lives as long as the process, define it once and open/close at application startup and shutdown:
+
+```python
+kanta = Kanta("data.kantadb", Data())
+data = kanta.data  # optional: your own direct reference
+
+
+async def startup() -> None:
+    await kanta.open()
+
+
+async def shutdown() -> None:
+    await kanta.close()
+
+
+async def create_user(name: str) -> None:
+    with kanta.transaction(action="create_user") as d:
+        d.users[uuid7()] = User(name=name)
+```
+
+Transactions are synchronous context managers, so no `await` is needed per operation; the background task flushes queued changes periodically.
+
+### Dynamically created databases (per-project, per-tenant, …)
+
+Nothing requires module-level definitions. An app managing many databases simply constructs instances on demand and tracks them itself:
+
+```python
+class ProjectStore:
+    def __init__(self) -> None:
+        self.projects: dict[str, Kanta[Data]] = {}
+
+    async def get(self, project_id: str) -> Kanta[Data]:
+        kanta = self.projects.get(project_id)
+        if kanta is None:
+            kanta = Kanta(f"projects/{project_id}.kantadb", Data())
+            await kanta.open()
+            self.projects[project_id] = kanta
+        return kanta
+```
+
+Remember that each open writer holds an exclusive lock on its file, so keep one `Kanta` instance per path and close instances you no longer need.
+
+## Transactions in brief
+
+- `with kanta.transaction(action=..., user=...) as data:` yields the live state object for mutation.
+- On success, Kanta computes a diff against the pre-transaction state and queues a change record; on exception, the in-memory data is rolled back and the exception is re-raised.
+- Nested transactions are rejected.
+- A transaction that changes nothing queues no record.
+
+See [On-disk format](database.md) for the full transaction semantics and [Validation](validation.md) for integrity checks that run inside each transaction.

docs/validation.md

@@ -0,0 +1,20 @@
+# Validation
+
+`@kanta.validate` registers an integrity validator for the database state, beyond the structural validation msgspec already performs during decoding.
+
+```python
+@kanta.validate
+def check_users(data: Data) -> None:
+    for user in data.users.values():
+        if not user.name:
+            raise ValueError("user without a name")
+```
+
+Validators receive the live data object (and optionally the `Kanta` instance as a second annotated parameter) and must **raise an exception** when the data is inconsistent. They are not intended to modify or correct the data — only to fail.
+
+Validators run:
+
+- **on open** — after replay, msgspec decoding and migrations, before the database becomes usable; a failure aborts the open and releases the file,
+- **after each transaction** — before the change is committed to history; a failure rolls the transaction back, so invalid state never reaches the log.
+
+Multiple validators may be registered; they run in registration order until the first failure. Validators must be synchronous (transactions are synchronous) — async callbacks are rejected at registration time.

kanta/__init__.py

@@ -1,5 +1,20 @@
+from .callbacks import DictPrev, DictState, LogFmt
+from .diff import diff, patch
+from .exceptions import DatabaseError
 from .kanta import Kanta
+from .logging import LogEvent, configure_logging
+from .migrations import MigrationReport
 
 __all__ = [
     "Kanta",
+    "DatabaseError",
+    "configure_logging",
+    "diff",
+    "patch",
+    # Callback argument types
+    "DictPrev",
+    "DictState",
+    "LogEvent",
+    "LogFmt",
+    "MigrationReport",
 ]

kanta/__main__.py

@@ -0,0 +1,676 @@
+"""Module-level CLI for reading a kantadb file and printing its change log."""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import contextlib
+import importlib
+import importlib.metadata
+import importlib.util
+import logging
+import sys
+import tempfile
+from pathlib import Path
+from typing import Any
+
+import msgspec
+
+from kanta import Kanta
+from kanta.callbacks import InjectionContext
+from kanta.exceptions import DatabaseError, DataIntegrityError, ReplayError
+from kanta.grep import GrepPattern, evaluate
+from kanta.logging import (
+    LogEvent,
+    emit_event,
+    format_action_header,
+    format_diff,
+    migration_logger,
+)
+from kanta.replaylog import (
+    RangeNotFoundError,
+    Selection,
+    SnapshotEvent,
+    _snapshot_lines,
+    end_of_file,
+    record_change_event,
+    record_label,
+    replay_events,
+    scan_events,
+    select,
+)
+from kanta.serialization import Serializer, dict_to_struct, struct_to_dict
+from kanta.structs import ChangeRecord, Snapshot
+from kanta.tty import Line, strip_ansi, use_color
+
+EXIT_SUCCESS = 0
+EXIT_GENERIC = 1
+EXIT_RANGE_ERROR = 2
+EXIT_PARSE_ERROR = 10
+EXIT_MIGRATION_ERROR = 20
+EXIT_VALIDATION_ERROR = 21
+
+_logger = logging.getLogger(__name__)
+
+
+def _print(*args: Any) -> None:
+    """Print to stderr, stripping ANSI codes when the stream has no color support.
+
+    Color detection runs per call so redirected or reassigned ``sys.stderr``
+    (and environment changes) are honored; ANSI codes are stripped after
+    formatting, not by formatting differently.
+    """
+    text = " ".join(str(arg) for arg in args)
+    if not use_color():
+        text = strip_ansi(text)
+    print(text, file=sys.stderr)
+
+
+class _CliError(Exception):
+    """A user-facing error message paired with a process exit code."""
+
+    def __init__(self, message: str, code: int = EXIT_GENERIC) -> None:
+        self.code = code
+        super().__init__(message)
+
+
+def _import_dotted(path: str) -> Any:
+    """Import ``module.submodule.Attr`` or a filesystem path and return the attribute."""
+    if _is_file_path(path):
+        return _import_from_file(path)
+    if "." not in path:
+        raise ValueError(f"dotted path must contain a dot: {path!r}")
+    module_name, attr_name = path.rsplit(".", 1)
+    module = importlib.import_module(module_name)
+    try:
+        return getattr(module, attr_name)
+    except AttributeError as exc:
+        raise ImportError(f"{path!r} not found in {module_name!r}") from exc
+
+
+def _is_file_path(path: str) -> bool:
+    """Return True if *path* looks like a filesystem path rather than a dotted name."""
+    return "/" in path or "\\" in path or ":" in path
+
+
+def _import_from_file(path: str) -> Any:
+    """Import a module or attribute from a filesystem path.
+
+    *path* may be ``path/to/file.py`` (returns the module) or
+    ``path/to/file.py:symbol`` (returns ``symbol`` from the module).
+    """
+    if ":" in path:
+        file_path, symbol = path.rsplit(":", 1)
+    else:
+        file_path, symbol = path, None
+
+    file_path = Path(file_path).resolve()
+    if not file_path.exists():
+        raise ImportError(f"{file_path!r} not found")
+    if not file_path.is_file():
+        raise ImportError(f"{file_path!r} is not a file")
+
+    module_name = f"_kanta_cli_{file_path.stem}_{file_path.stat().st_ino}"
+    spec = importlib.util.spec_from_file_location(module_name, file_path)
+    if spec is None or spec.loader is None:
+        raise ImportError(f"cannot load {file_path!r}")
+    module = importlib.util.module_from_spec(spec)
+
+    file_dir = str(file_path.parent)
+    added_dir = False
+    if file_dir not in sys.path:
+        sys.path.insert(0, file_dir)
+        added_dir = True
+    try:
+        sys.modules[module_name] = module
+        spec.loader.exec_module(module)
+    finally:
+        if added_dir:
+            sys.path.remove(file_dir)
+
+    if symbol is None:
+        return module
+    try:
+        return getattr(module, symbol)
+    except AttributeError as exc:
+        raise ImportError(f"{symbol!r} not found in {file_path!r}") from exc
+
+
+def _import_kanta_object(path: str) -> Any:
+    """Import a Kanta object by module or filesystem path.
+
+    If ``path`` names an importable module, look up an object named
+    ``kanta`` in it; otherwise treat ``path`` as ``module.attr`` or
+    ``path/to/file.py[:kanta]`` referring directly to the object.
+    """
+    if _is_file_path(path):
+        if ":" in path:
+            return _import_from_file(path)
+        module = _import_from_file(path)
+        try:
+            return getattr(module, "kanta")
+        except AttributeError as exc:
+            raise ImportError(f"no 'kanta' object found in {path!r}") from exc
+    try:
+        spec = importlib.util.find_spec(path)
+    except ImportError:
+        spec = None
+    if spec is not None:
+        module = importlib.import_module(path)
+        try:
+            return getattr(module, "kanta")
+        except AttributeError as exc:
+            raise ImportError(f"no 'kanta' object found in module {path!r}") from exc
+    return _import_dotted(path)
+
+
+def _format_ts(dt) -> str:
+    """Return a local-looking timestamp without a timezone offset or microseconds."""
+    return dt.replace(tzinfo=None, microsecond=0).isoformat(sep=" ")
+
+
+def _package_version() -> str:
+    """Return the installed package version, or ``"unknown"`` from a source tree."""
+    try:
+        return importlib.metadata.version("kanta")
+    except importlib.metadata.PackageNotFoundError:
+        return "unknown"
+
+
+def _parse_args(argv: list[str] | None = None) -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        prog="kanta",
+        description=(
+            f"kanta {_package_version()} - read a kantadb file and print each"
+            " change record to the console."
+        ),
+    )
+    parser.add_argument(
+        "--version",
+        action="version",
+        version=f"%(prog)s {_package_version()}",
+    )
+    parser.add_argument(
+        "file",
+        help="Path to the kantadb file, or '-' to read from stdin.",
+    )
+    parser.add_argument(
+        "-d",
+        "--data",
+        metavar="MOD",
+        help=(
+            "Dotted path or filesystem path to the root data type."
+            " Examples: myapp.models.Data, myapp/models.py:Data."
+        ),
+    )
+    parser.add_argument(
+        "-m",
+        "--migrations",
+        metavar="MOD",
+        help=(
+            "Dotted path or filesystem path to the migrations module."
+            " Examples: myapp.migrations, myapp/migrations.py."
+        ),
+    )
+    parser.add_argument(
+        "-k",
+        "--kanta",
+        metavar="MOD",
+        help=(
+            "Module path or filesystem path to an existing Kanta object to use."
+            " Either a module containing an object named 'kanta' (e.g. myapp.db),"
+            " a dotted path to the object (e.g. myapp.db.kanta), or a file path"
+            " (e.g. myapp/db.py or myapp/db.py:kanta).  Its type, migrations, and"
+            " logfmt/logemit callbacks are used.  Cannot be combined with -d or -m."
+        ),
+    )
+    parser.add_argument(
+        "-o",
+        "--output",
+        help="Write the final replayed state as JSON to this file, or '-' for stdout.",
+    )
+    parser.add_argument(
+        "-q",
+        "--quiet",
+        action="store_true",
+        help="Suppress normal change/snapshot logs; only print warnings and errors.",
+    )
+    parser.add_argument(
+        "-r",
+        "--range",
+        help=(
+            "Python-style range to process.  Units: plain number = change index,"
+            " lN = line number, sN = snapshot, vN = version.  Negative snapshot"
+            " values count from the end (s-1 is the last snapshot).  Use ':' for"
+            " half-open ranges and '..' for inclusive end ranges.  Examples:"
+            " '2:5', '2..5', 'l10:l20', 's1:s3', 'v0:v2', 's-1:', ':-1', '-1'."
+        ),
+    )
+    parser.add_argument(
+        "-g",
+        "--grep",
+        action="append",
+        metavar="PATTERN",
+        help=(
+            "Print only change records matching PATTERN, structurally and"
+            " case-insensitively; matched regions get a yellow background,"
+            " and on a match the whole record is printed, not just the"
+            " matching line.  Repeatable: every pattern must match somewhere"
+            " in the same record, but different patterns may match different"
+            " lines of it.  A bare pattern matches the action or user as a"
+            " substring, a dotted path by element (each element in full,"
+            " unless it uses wildcards: 'users' matches 'users' anywhere but"
+            " not 'foousers', 'us*' does), or a value (strings by substring,"
+            " other values in full: 'true' matches a boolean, 'tru' does"
+            " not).  The 'path=value' form requires the path and the value"
+            " to match within the same change line; use '=value' or 'path='"
+            " to match values or paths only.  With -k, logfmt-prettified"
+            " values and users match alongside the raw ones.  Examples:"
+            " --grep alice, --grep 'users.*.email', --grep"
+            " 'users.alice.admin=true', --grep create_user --grep"
+            " '@example.com'."
+        ),
+    )
+    args = parser.parse_args(argv)
+    if args.kanta and (args.data or args.migrations):
+        parser.error("-k/--kanta cannot be used together with -d or -m")
+    return args
+
+
+def _print_change_log(
+    label: str,
+    record: ChangeRecord,
+    previous: dict[str, Any],
+    current: dict[str, Any],
+    kanta: Kanta[Any],
+    highlight: Any = None,
+) -> None:
+    """Log a single change record to stderr.
+
+    The record is dispatched as a :class:`LogEvent` through the Kanta
+    object's logemit handlers; the CLI's own rendering (with the ``l<N>``
+    label and timestamp) is the fallback when no handler claims the event.
+    ``highlight`` is an optional :class:`kanta.grep.GrepHighlighter` with
+    the record's matched regions, applied to the fallback rendering.
+    """
+    event = record_change_event(record, previous, current, kanta)
+
+    def render(ev) -> None:
+        ts = _format_ts(record.ts)
+        if highlight is not None:
+            header = format_action_header(
+                ev.action or "", ev.user, ev.extra, highlight=highlight
+            )
+            lines = format_diff(ev.diff, ev.previous, ev.logfmt, highlight=highlight)
+        else:
+            header, lines = ev.header, ev.diff_lines
+        if not lines:
+            _print(f"{label} {ts} {header}")
+        elif len(lines) == 1:
+            _print(f"{label} {ts} {header}{lines[0]}")
+        else:
+            _print(f"{label} {ts} {header}")
+            for line in lines:
+                _print(line)
+            _print()
+
+    emit_event(
+        event,
+        kanta._impl.callback_registry.logemit_handlers,
+        fallback=render,
+    )
+
+
+def _format_size(n: int) -> str:
+    """Return a human-readable byte size."""
+    if n < 1024:
+        return f"{n} B"
+    if n < 1024 * 1024:
+        return f"{n / 1024:.1f} kB"
+    return f"{n / (1024 * 1024):.1f} MB"
+
+
+def _find_venv_site_packages(start: Path) -> list[Path]:
+    """Return site-packages dirs of ``.venv`` directories from *start* to parents."""
+    py_dir = f"python{sys.version_info.major}.{sys.version_info.minor}"
+    found: list[Path] = []
+    for parent in [start, *start.parents]:
+        venv = parent / ".venv"
+        if not venv.is_dir():
+            continue
+        site_packages = venv / "lib" / py_dir / "site-packages"
+        if site_packages.is_dir():
+            found.append(site_packages)
+            continue
+        # Windows layout
+        win_site = venv / "Lib" / "site-packages"
+        if win_site.is_dir():
+            found.append(win_site)
+    return found
+
+
+@contextlib.contextmanager
+def _extra_import_paths():
+    """Temporarily add current dir and nearby venv site-packages to ``sys.path``.
+
+    The current directory is inserted first, then local ``.venv`` site-packages,
+    then any parent ``.venv`` site-packages.  Only paths that were not already
+    present are added, and only those added paths are removed on exit.
+    """
+    paths_to_add = [str(Path.cwd())]
+    paths_to_add.extend(str(p) for p in _find_venv_site_packages(Path.cwd()))
+    added: list[str] = []
+    for path in reversed(paths_to_add):
+        if path not in sys.path:
+            sys.path.insert(0, path)
+            added.append(path)
+    try:
+        yield
+    finally:
+        for path in added:
+            if path in sys.path:
+                sys.path.remove(path)
+
+
+def _print_snapshot_indicator(
+    label: str,
+    snap: Snapshot,
+    index: int,
+    serializer: Serializer,
+) -> None:
+    """Print a snapshot indicator line to stderr.
+
+    ``snapshot s<N>`` is rendered in bright white; the version, optional mtime
+    and data size are printed in normal and dark colors respectively.
+    """
+    ts = _format_ts(snap.ts)
+    line = Line().snapshot("snapshot").snapshot(f" s{index}")
+    line.target(f" v{snap.v}")
+    if snap.m is not None:
+        line.target(f" {_format_ts(snap.m)}")
+    size = len(serializer.encode(snap.state))
+    line.path_prefix(f" {_format_size(size)}")
+    _print(f"{label} {ts} {line}")
+
+
+async def _log_migration(
+    kanta: Kanta[Any],
+    filename: Path,
+    result,
+    previous_version: int,
+    quiet: bool,
+) -> None:
+    """Log an applied migration through the Kanta instance's callbacks.
+
+    Routes to the object's logmigr callbacks when registered (like
+    :meth:`KantaImpl._handle_migration_log`); otherwise emits a ``migrated``
+    event through its logemit handlers, falling back to a stderr line.
+    """
+    registry = kanta._impl.callback_registry
+    if registry.has("logmigr"):
+        try:
+            await registry.invoke(
+                "logmigr",
+                InjectionContext(kanta=kanta, report=result),
+            )
+        except Exception:
+            _logger.exception("logmigr callback failed")
+        return
+    if quiet:
+        return
+    descriptions = [f"{m.name} ({m.description})" for m in result.applied if m.changed]
+    emit_event(
+        LogEvent(
+            kind="migrated",
+            logger=migration_logger,
+            kanta=kanta,
+            filename=str(filename),
+            from_version=previous_version,
+            to_version=result.version,
+            migrations=descriptions,
+        ),
+        registry.logemit_handlers,
+        fallback=lambda ev: _print(ev.header),
+    )
+
+
+def _get_kanta(args: argparse.Namespace, filename: Path) -> tuple[Kanta[Any], bool]:
+    """Return the Kanta instance to work with, and whether the CLI owns it.
+
+    With ``-k`` the existing object is used as-is (and never closed by us);
+    otherwise an instance is constructed with an empty dict state.
+    """
+    if args.kanta:
+        try:
+            obj = _import_kanta_object(args.kanta)
+        except (ImportError, ValueError) as exc:
+            raise _CliError(f"Invalid --kanta value: {exc}") from exc
+        if not isinstance(obj, Kanta):
+            raise _CliError(
+                f"Invalid --kanta value: {args.kanta!r} is not a Kanta object"
+            )
+        return obj, False
+    try:
+        return Kanta(filename, {}, type=dict, migrations=args.migrations), True
+    except Exception as exc:
+        if args.migrations:
+            raise _CliError(f"Migration error: {exc}", EXIT_MIGRATION_ERROR) from exc
+        raise _CliError(f"Failed to initialize database: {exc}") from exc
+
+
+async def _run(args: argparse.Namespace) -> int:
+    cleanup_path: Path | None = None
+    if args.file == "-":
+        content = sys.stdin.buffer.read()
+        tmp = tempfile.NamedTemporaryFile(delete=False, suffix=".kantadb")
+        tmp.write(content)
+        tmp.close()
+        filename = Path(tmp.name)
+        cleanup_path = filename
+    else:
+        filename = Path(args.file)
+        if not filename.exists():
+            raise _CliError(f"File not found: {filename}")
+        content = filename.read_bytes()
+
+    data_type: type[Any] | None = None
+    if args.data:
+        with _extra_import_paths():
+            try:
+                data_type = _import_dotted(args.data)
+            except (ImportError, ValueError) as exc:
+                raise _CliError(f"Invalid --data value: {exc}") from exc
+
+    kanta: Kanta[Any] | None = None
+    kanta_owned = False
+    kanta_typed: Kanta[Any] | None = None
+    try:
+        with _extra_import_paths():
+            kanta, kanta_owned = _get_kanta(args, filename)
+        if data_type is None and args.kanta and kanta._impl.data_type is not dict:
+            data_type = kanta._impl.data_type
+
+        # Decode and validate the whole file into positioned events.
+        try:
+            events, change_count = scan_events(content, kanta)
+        except ReplayError as exc:
+            raise _CliError(str(exc), EXIT_PARSE_ERROR) from exc
+        except Exception as exc:
+            raise _CliError(
+                f"Failed to replay records from {filename}: {exc}",
+                EXIT_PARSE_ERROR,
+            ) from exc
+
+        snapshot_line_to_index = {
+            line: idx for idx, line in enumerate(_snapshot_lines(events))
+        }
+
+        # Resolve -r into a line range or a single snapshot selection.
+        try:
+            selection = (
+                select(args.range, events, change_count)
+                if args.range is not None
+                else Selection(0, end_of_file(events))
+            )
+        except RangeNotFoundError as exc:
+            raise _CliError(str(exc), EXIT_RANGE_ERROR) from exc
+        except ValueError as exc:
+            raise _CliError(f"Invalid --range value: {exc}") from exc
+
+        grep_patterns = [GrepPattern.parse(p) for p in args.grep or ()]
+
+        if selection.snapshot is not None:
+            snap_event = selection.snapshot
+            state = snap_event.snap.state
+            version = snap_event.snap.v
+            if not args.quiet:
+                _print_snapshot_indicator(
+                    record_label(snap_event.line_number, snap_event.record_index),
+                    snap_event.snap,
+                    snapshot_line_to_index[snap_event.line_number],
+                    kanta._impl.serializer,
+                )
+                _print()
+        else:
+            # Replay up to the range end, printing logs within the range.
+            state = {}
+            version = 0
+            printed = False
+            for event, previous, current in replay_events(events, selection.end_line):
+                state = current
+                version = event.version
+                if event.line_number < selection.start_line or args.quiet:
+                    continue
+                label = record_label(event.line_number, event.record_index)
+                if isinstance(event, SnapshotEvent):
+                    _print_snapshot_indicator(
+                        label,
+                        event.snap,
+                        snapshot_line_to_index[event.line_number],
+                        kanta._impl.serializer,
+                    )
+                else:
+                    assert previous is not None
+                    highlight = None
+                    if grep_patterns:
+                        # Build the same logfmt the rendering uses, so both
+                        # raw and prettified values are matched.
+                        logfmt = kanta._impl.callback_registry.build_logfmt(
+                            InjectionContext(
+                                kanta=kanta,
+                                previous_state=previous,
+                                current_state=current,
+                            )
+                        )
+                        highlight = evaluate(
+                            event.record, previous, grep_patterns, logfmt=logfmt
+                        )
+                        if highlight is None:
+                            continue
+                    _print_change_log(
+                        label, event.record, previous, current, kanta, highlight
+                    )
+                printed = True
+            if printed:
+                _print()
+
+        # Apply optional migrations to the range-end state.
+        if kanta._impl.migrations is not None:
+            try:
+                previous_version = version
+                result = kanta._impl.migrations.apply(state, version, kanta)
+                version = result.version
+            except Exception as exc:
+                raise _CliError(
+                    f"Migration error: {exc}", EXIT_MIGRATION_ERROR
+                ) from exc
+            if version != previous_version:
+                await _log_migration(
+                    kanta, filename, result, previous_version, args.quiet
+                )
+
+        output_state: dict[str, Any]
+        if data_type is not None:
+            try:
+                data = dict_to_struct(
+                    state, data_type, serializer=kanta._impl.serializer
+                )
+            except (
+                msgspec.ValidationError,
+                msgspec.DecodeError,
+                TypeError,
+                ValueError,
+            ) as exc:
+                raise _CliError(
+                    f"Validation error: {exc}", EXIT_VALIDATION_ERROR
+                ) from exc
+
+            if args.kanta:
+                # The file was already fully decoded and validated above with
+                # the object's own serializer, and its migrations were applied
+                # to the state; no need to re-open through a new instance.
+                _print(f"{data}")
+                output_state = struct_to_dict(data, serializer=kanta._impl.serializer)
+            else:
+                kanta_typed = Kanta(
+                    filename, data, type=data_type, migrations=args.migrations
+                )
+                try:
+                    await kanta_typed.open(create=False, readonly=True, log=False)
+                    _print(f"{data}")
+                except (msgspec.ValidationError, msgspec.DecodeError) as exc:
+                    raise _CliError(
+                        f"Validation error: {exc}", EXIT_VALIDATION_ERROR
+                    ) from exc
+                except DataIntegrityError as exc:
+                    raise _CliError(f"Parse error: {exc}", EXIT_PARSE_ERROR) from exc
+                except DatabaseError as exc:
+                    if not args.migrations or exc.cause_type == "ReplayError":
+                        raise _CliError(
+                            f"Parse error: {exc}", EXIT_PARSE_ERROR
+                        ) from exc
+                    raise _CliError(
+                        f"Migration error: {exc}", EXIT_MIGRATION_ERROR
+                    ) from exc
+                except Exception as exc:
+                    if args.migrations:
+                        raise _CliError(
+                            f"Migration error: {exc}", EXIT_MIGRATION_ERROR
+                        ) from exc
+                    raise _CliError(f"Failed to open {filename}: {exc}") from exc
+                output_state = kanta_typed._impl.statedict
+        else:
+            output_state = state
+
+        if args.output:
+            try:
+                out_bytes = msgspec.json.encode(output_state)
+                if args.output == "-":
+                    sys.stdout.buffer.write(out_bytes)
+                else:
+                    Path(args.output).write_bytes(out_bytes)
+            except Exception as exc:  # pragma: no cover
+                raise _CliError(f"Failed to write output: {exc}") from exc
+
+        return EXIT_SUCCESS
+    finally:
+        if kanta_typed is not None:
+            await kanta_typed.close()
+        if kanta is not None and kanta_owned:
+            await kanta.close()
+        if cleanup_path is not None:
+            cleanup_path.unlink(missing_ok=True)
+
+
+def main(argv: list[str] | None = None) -> int:
+    """Entry point for ``python -m kanta``."""
+    args = _parse_args(argv)
+    try:
+        return asyncio.run(_run(args))
+    except _CliError as exc:
+        _print(exc)
+        return exc.code
+
+
+if __name__ == "__main__":
+    sys.exit(main())

kanta/callbacks.py

@@ -1,8 +1,8 @@
 """Unified decorator-based callback registry for Kanta.
 
-Callbacks are registered once and invoked with arguments filled by their
-annotation types.  Unknown arguments are only permitted when they have a
-default value.
+Callbacks are registered once and invoked with arguments filled from their
+parameter names (state dicts: ``prev`` / ``state``) and annotation types.
+Unknown arguments are only permitted when they have a default value.
 
 Log formatters are a special case: they are called per value being rendered
 and receive the value plus an optional ``path`` string.  They return
@@ -23,10 +23,36 @@ from dataclasses import dataclass
 from typing import Annotated, Any, Union, get_args, get_origin
 
 from kanta.exceptions import DatabaseError
-from kanta.migrations import MigrationResult
+from kanta.migrations import MigrationReport
+
+DictPrev = DictPre = Annotated[dict, "prev"]
+DictState = DictPost = Annotated[dict, "state"]
+
+# State-dict injection keys, shared by parameter names and annotation tags:
+# a callback parameter named *or* tagged ``prev``/``state`` receives the
+# previous or current state dict respectively.  Matching by name does not
+# check the annotation; an explicit tag takes precedence over the name.
+_STATE_KINDS = {"prev": "previous_state", "state": "current_state"}
+
+
+def _state_key(text: Any) -> str | None:
+    """Return the state kind for a parameter name or annotation tag."""
+    return text if text in _STATE_KINDS else None
+
+
+def _state_tag(ann: Any) -> str | None:
+    """Return the state tag of an ``Annotated[dict, ...]`` annotation, if any."""
+    if get_origin(ann) is not Annotated:
+        return None
+    args = get_args(ann)
+    if not args or args[0] is not dict:
+        return None
+    for meta in args[1:]:
+        key = _state_key(meta)
+        if key is not None:
+            return key
+    return None
 
-DictPre = Annotated[dict, "pre"]
-DictPost = Annotated[dict, "post"]
 
 _logger = logging.getLogger(__name__)
 
@@ -35,16 +61,18 @@ class LogFmt:
     """Base class for stateful logfmt callbacks.
 
     Subclasses only need to override :meth:`resolve`.  The framework injects
-    ``previous_state`` and ``current_state`` through ``__init__``.
+    the previous and current state dicts through ``__init__`` and exposes them
+    as ``previous_state`` and ``state``.
     """
 
     def __init__(
         self,
-        previous: DictPre | None = None,
-        current: DictPost | None = None,
+        prev: dict | None = None,
+        state: dict | None = None,
     ) -> None:
-        self.previous_state = previous
-        self.current_state = current
+        self.previous_state = prev
+        self.state = state
+        self.current_state = state  # deprecated alias for ``state``
 
     def __call__(self, value: Any, path: str) -> str | None:
         return self.resolve(value, path)
@@ -67,7 +95,7 @@ class InjectionContext:
     error: DatabaseError | None = None
     previous_state: dict | None = None
     current_state: dict | None = None
-    migration_result: MigrationResult | None = None
+    report: MigrationReport | None = None
 
 
 @dataclass
@@ -108,6 +136,7 @@ class CallbackRegistry:
             "bootstrap": [],
             "fatal_error": [],
             "logmigr": [],
+            "validate": [],
         }
         self._logfmt_callbacks: list[_LogFmtFunctionSpec | _LogFmtClassSpec] = []
         self._logemit_callbacks: list[Callable[..., Any]] = []
@@ -146,6 +175,10 @@ class CallbackRegistry:
             raise TypeError(f"{kind} callbacks must be functions, not classes")
         if not callable(callback):
             raise TypeError(f"{kind} callback must be callable")
+        if kind == "validate" and inspect.iscoroutinefunction(callback):
+            raise TypeError(
+                "validate callbacks must not be async (transactions are synchronous)"
+            )
 
         params = self._validate_function(callback, kind)
         is_async = inspect.iscoroutinefunction(callback)
@@ -187,6 +220,16 @@ class CallbackRegistry:
                     break
         return results
 
+    def invoke_sync(self, kind: str, ctx: InjectionContext) -> None:
+        """Invoke all sync callbacks of *kind* in order; first exception raises.
+
+        Used for ``validate`` callbacks, which run inside synchronous
+        transactions and therefore must not be async.
+        """
+        for reg in self._callbacks[kind]:
+            kwargs = self._build_kwargs(reg.params, ctx)
+            reg.callback(**kwargs)
+
     def has(self, kind: str) -> bool:
         """Return True if any callback of *kind* is registered."""
         if kind == "logfmt":
@@ -261,6 +304,9 @@ class CallbackRegistry:
                 )
 
             if param.annotation is inspect.Parameter.empty:
+                if _state_key(name) is not None:
+                    params.append((name, dict))
+                    continue
                 if param.default is inspect.Parameter.empty:
                     raise TypeError(
                         f"{kind} callback {callback.__name__} has parameter "
@@ -269,6 +315,11 @@ class CallbackRegistry:
                 continue
 
             ann = self._resolve_raw_annotation(param.annotation, callback)
+            if _state_key(name) is not None:
+                # The name alone selects state injection; an explicit tag
+                # still overrides it.  The annotation is not checked.
+                params.append((name, ann))
+                continue
             if not self._is_allowed(kind, ann):
                 if param.default is inspect.Parameter.empty:
                     raise TypeError(
@@ -329,6 +380,9 @@ class CallbackRegistry:
                     f"*args or **kwargs"
                 )
             if param.annotation is inspect.Parameter.empty:
+                if _state_key(name) is not None:
+                    inject_params.append((name, dict))
+                    continue
                 if param.default is inspect.Parameter.empty:
                     raise TypeError(
                         f"logfmt callback {callback.__name__} has parameter "
@@ -337,6 +391,11 @@ class CallbackRegistry:
                 continue
 
             ann = self._resolve_raw_annotation(param.annotation, callback)
+            if _state_key(name) is not None:
+                # The name alone selects state injection; an explicit tag
+                # still overrides it.  The annotation is not checked.
+                inject_params.append((name, ann))
+                continue
             if name == "path" and self._unwrap_optional(ann) is str:
                 has_path = True
                 continue
@@ -392,6 +451,9 @@ class CallbackRegistry:
                     f"*args or **kwargs"
                 )
             if param.annotation is inspect.Parameter.empty:
+                if _state_key(name) is not None:
+                    inject_params.append((name, dict))
+                    continue
                 if param.default is inspect.Parameter.empty:
                     raise TypeError(
                         f"logfmt class {cls.__name__}.__init__ has parameter "
@@ -400,6 +462,11 @@ class CallbackRegistry:
                 continue
 
             ann = self._resolve_raw_annotation(param.annotation, cls.__init__)
+            if _state_key(name) is not None:
+                # The name alone selects state injection; an explicit tag
+                # still overrides it.  The annotation is not checked.
+                inject_params.append((name, ann))
+                continue
             if self._is_allowed("logfmt", ann):
                 inject_params.append((name, ann))
                 continue
@@ -464,7 +531,7 @@ class CallbackRegistry:
     ) -> dict[str, Any]:
         kwargs: dict[str, Any] = {}
         for name, ann in params:
-            value = self._resolve_annotation(ann, ctx)
+            value = self._resolve_annotation(name, ann, ctx)
             if value is _UNRESOLVED:
                 raise RuntimeError(f"no value available for annotation {ann!r}")
             kwargs[name] = value
@@ -472,52 +539,54 @@ class CallbackRegistry:
 
     def _is_allowed(self, kind: str, ann: Any) -> bool:
         bare = self._unwrap_optional(ann)
-        if self._matches_state_annotation(bare, "pre"):
-            return kind == "logfmt"
-        if self._matches_state_annotation(bare, "post"):
+        if _state_tag(bare) is not None:
             return kind == "logfmt"
         if bare is DatabaseError:
             return kind == "fatal_error"
-        if bare is MigrationResult:
+        if bare is MigrationReport:
             return kind == "logmigr"
         if self._data_type is not None and bare is self._data_type:
-            return kind == "bootstrap"
+            return kind in {"bootstrap", "validate"}
         if self._kanta_class is not None and bare is self._kanta_class:
             return kind in {
                 "bootstrap",
                 "fatal_error",
                 "logfmt",
                 "logmigr",
+                "validate",
             }
         return False
 
     def _allowed_message(self, kind: str) -> str:
         parts: list[str] = []
-        if kind == "bootstrap":
+        if kind in {"bootstrap", "validate"}:
             if self._data_type is not None:
                 parts.append(self._data_type.__name__)
-        if kind in {"bootstrap", "fatal_error", "logfmt", "logmigr"}:
+        if kind in {"bootstrap", "fatal_error", "logfmt", "logmigr", "validate"}:
             if self._kanta_class is not None:
                 parts.append(self._kanta_class.__name__)
         if kind == "fatal_error":
             parts.append("DatabaseError")
         if kind == "logmigr":
-            parts.append("MigrationResult")
+            parts.append("MigrationReport")
         if kind == "logfmt":
-            parts.append("Annotated[dict, 'pre']")
-            parts.append("Annotated[dict, 'post']")
+            parts.append("prev: dict")
+            parts.append("state: dict")
         return ", ".join(parts) if parts else "none"
 
-    def _resolve_annotation(self, ann: Any, ctx: InjectionContext) -> Any:
+    def _resolve_annotation(self, name: str, ann: Any, ctx: InjectionContext) -> Any:
         bare = self._unwrap_optional(ann)
-        if self._matches_state_annotation(bare, "pre"):
-            return ctx.previous_state
-        if self._matches_state_annotation(bare, "post"):
-            return ctx.current_state
+        # An explicit tag takes precedence over the parameter name.
+        tag = _state_tag(bare)
+        if tag is not None:
+            return getattr(ctx, _STATE_KINDS[tag])
+        key = _state_key(name)
+        if key is not None:
+            return getattr(ctx, _STATE_KINDS[key])
         if bare is DatabaseError:
             return ctx.error
-        if bare is MigrationResult:
-            return ctx.migration_result
+        if bare is MigrationReport:
+            return ctx.report
         if self._data_type is not None and bare is self._data_type:
             return ctx.data
         if self._kanta_class is not None and bare is self._kanta_class:
@@ -539,16 +608,6 @@ class CallbackRegistry:
                 ) from exc
         return raw_ann
 
-    @staticmethod
-    def _matches_state_annotation(ann: Any, marker: str) -> bool:
-        origin = get_origin(ann)
-        if origin is not Annotated:
-            return False
-        args = get_args(ann)
-        if not args:
-            return False
-        return args[0] is dict and marker in args[1:]
-
     @staticmethod
     def _unwrap_optional(ann: Any) -> Any:
         origin = get_origin(ann)

kanta/diff.py

@@ -1,63 +1,66 @@
-"""Diff computation and replay utilities."""
+"""Diff computation and replay utilities.
 
-import jsondiff
+Diff format: JSON-serializable dicts where ``$delete`` is the only command
+our producer emits; added keys and changed values (scalars, lists, type
+changes — lists always wholesale) are plain assignments. A dict value
+assigned over a non-dict needs no ``$replace``: the consumer can see from
+the old value whether to patch (old is a dict) or replace. User keys
+starting with ``$`` are escaped by prepending another ``$``
+(``$foo`` -> ``$$foo``); values are stored verbatim.
+
+The consumer additionally stays compatible with jsondiff's marshaled
+syntax, so it can replay diffs produced by jsondiff itself: ``$replace``,
+positional ``$insert``/``$delete`` and per-index nested diffs on lists,
+and jsondiff's escaping of ``$``-prefixed values.
+"""
 
 from kanta.structs import ChangeRecord
-from kanta.serialization.base import ReplayResult, replay
+from kanta.serialization.base import ReplayResult, apply_diff, replay
 from kanta.serialization.framing import LineFramer
 from kanta.serialization.json import JsonSerializer
 
+_UNCHANGED = object()
 
-def compute_diff(previous: dict, current: dict) -> dict | None:
-    """Compute a jsondiff patch between two dicts.
+
+def _escape_key(key: str) -> str:
+    """Escape a user key for use as a diff key (``$foo`` -> ``$$foo``)."""
+    if isinstance(key, str) and key.startswith("$"):
+        return "$" + key
+    return key
+
+
+def _diff(previous, current):
+    """Compute a raw diff, or _UNCHANGED if there is no difference."""
+    if isinstance(previous, dict) and isinstance(current, dict):
+        result = {}
+        deleted = [_escape_key(k) for k in previous if k not in current]
+        if deleted:
+            result["$delete"] = deleted
+        for key, new_value in current.items():
+            if key not in previous:
+                result[_escape_key(key)] = new_value
+            else:
+                sub = _diff(previous[key], new_value)
+                if sub is not _UNCHANGED:
+                    result[_escape_key(key)] = sub
+        return result if result else _UNCHANGED
+    if previous == current:
+        return _UNCHANGED
+    return current
+
+
+def diff(previous: dict, current: dict) -> dict | None:
+    """Compute a marshaled diff between two state dicts.
 
     Returns None if there is no difference.
     """
-    return jsondiff.diff(previous, current, marshal=True) or None
+    result = _diff(previous, current)
+    return result if result is not _UNCHANGED else None
 
 
-def _apply_diff(state: dict, diff: dict) -> dict:
-    """Apply a jsondiff patch manually, handling ``$replace`` and ``$delete``.
-
-    jsondiff.patch does not handle nested ``$replace`` commands when the
-    parent key is missing from the state.  This function recursively applies
-    diffs, treating ``$replace`` as full replacement and ``$delete`` as
-    key removal.
-    """
-    if not isinstance(diff, dict):
-        return diff
-
-    result = dict(state) if isinstance(state, dict) else state
-    if not isinstance(result, dict):
-        result = {}
-
-    for key, value in diff.items():
-        if key == "$replace":
-            return value
-        elif key == "$delete":
-            if isinstance(value, list):
-                for k in value:
-                    result.pop(k, None)
-            else:
-                result.pop(value, None)
-        elif isinstance(value, dict):
-            old = result.get(key, {})
-            if not isinstance(old, dict):
-                old = {}
-            result[key] = _apply_diff(old, value)
-        else:
-            result[key] = value
-
-    return result
-
-
-def patch_state(state: dict, diff: dict) -> dict:
-    """Apply a jsondiff patch to a state dict.
-
-    The diff was produced with ``marshal=True`` (string keys like
-    ``"$replace"`` and ``"$delete"``) and decoded from JSON.
-    """
-    return _apply_diff(state, diff)
+def patch(state: dict, diff: dict) -> dict:
+    """Apply a marshaled diff to a state dict."""
+    return apply_diff(state, diff)
 
 
 # Backward-compatible JSONL replay using the default serializer.

kanta/filelock.py

@@ -83,6 +83,10 @@ if sys.platform == "win32":
     ]
     _kernel32.CloseHandle.restype = wintypes.BOOL
     _kernel32.CloseHandle.argtypes = [wintypes.HANDLE]
+    _kernel32.SetEndOfFile.restype = wintypes.BOOL
+    _kernel32.SetEndOfFile.argtypes = [wintypes.HANDLE]
+    _kernel32.FlushFileBuffers.restype = wintypes.BOOL
+    _kernel32.FlushFileBuffers.argtypes = [wintypes.HANDLE]
 
     def _is_invalid_handle(handle) -> bool:
         return ctypes.c_void_p(handle).value == ctypes.c_void_p(-1).value
@@ -177,6 +181,22 @@ class LockedFile:
         os.lseek(self._fd, current, os.SEEK_SET)
         return end
 
+    def replace_content(self, data: bytes) -> None:
+        """Atomically-ish rewrite the file's content in place, lock retained.
+
+        Seeks to the start, truncates, writes *data* and fsyncs, all on the
+        already-locked descriptor. The path is never unlinked or renamed, so
+        no other process can observe a missing file or acquire its own lock.
+        Used by database rotation.
+        """
+        if self._fd is None:
+            raise RuntimeError("LockedFile.replace_content() called on a closed file")
+
+        if sys.platform == "win32":
+            self._replace_content_win32(data)
+        else:
+            self._replace_content_unix(data)
+
     def close(self) -> None:
         """Release the lock and close the file."""
         if self._fd is None:
@@ -227,6 +247,15 @@ class LockedFile:
         os.lseek(self._fd, 0, os.SEEK_END)
         os.write(self._fd, data)
 
+    def _replace_content_unix(self, data: bytes) -> None:
+        os.lseek(self._fd, 0, os.SEEK_SET)
+        os.ftruncate(self._fd, 0)
+        view = memoryview(data)
+        while view:
+            written = os.write(self._fd, view)
+            view = view[written:]
+        os.fdatasync(self._fd)
+
     # -- Windows -------------------------------------------------------------
 
     def _open_win32(self, path: Path, create: bool, readonly: bool) -> None:
@@ -288,3 +317,24 @@ class LockedFile:
         )
         if not ok:
             raise OSError(f"WriteFile failed: Windows error {ctypes.get_last_error()}")
+
+    def _replace_content_win32(self, data: bytes) -> None:
+        _kernel32.SetFilePointer(self._fd, 0, None, _FILE_BEGIN)
+        written = wintypes.DWORD()
+        ok = _kernel32.WriteFile(
+            self._fd,
+            data,
+            len(data),
+            ctypes.byref(written),
+            None,
+        )
+        if not ok:
+            raise OSError(f"WriteFile failed: Windows error {ctypes.get_last_error()}")
+        if not _kernel32.SetEndOfFile(self._fd):
+            raise OSError(
+                f"SetEndOfFile failed: Windows error {ctypes.get_last_error()}"
+            )
+        if not _kernel32.FlushFileBuffers(self._fd):
+            raise OSError(
+                f"FlushFileBuffers failed: Windows error {ctypes.get_last_error()}"
+            )

kanta/grep.py

@@ -0,0 +1,433 @@
+"""Structural grep matching and match highlighting for change records.
+
+Backs the ``--grep`` option of the ``python -m kanta`` CLI.  Patterns
+match against the structure of a transaction — its action, user, and the
+dotted paths and values of its diff — never against rendered output
+text.  Matching is case-insensitive.
+
+Path terms match element-wise: each dotted element of the term must match
+a whole path element (``users`` matches ``users`` anywhere in the path
+but not ``foousers``), unless the element uses shell wildcards
+(``us*``).  The term's elements match as a contiguous sequence, so
+``users.*.email`` matches the path ``users.alice.email``.  Only the
+matched elements are highlighted.
+
+Value terms match strings by substring and all other values (booleans,
+numbers, null) only in full: ``true`` matches a boolean but ``tru`` does
+not.  A term with wildcards matches the whole value text.  Matched
+substrings — or the whole scalar — are highlighted.  When a logfmt
+formatter (``-k``) prettifies a value or the user, both the raw and the
+prettified form are matched.
+
+A matched record carries a :class:`GrepHighlighter`, which the
+:kanta.logging formatters consult to wrap exactly the matched regions
+with a yellow background.
+"""
+
+from __future__ import annotations
+
+import dataclasses
+import fnmatch
+import json
+from collections.abc import Iterator
+from typing import Any
+
+from kanta.logging import _USER_PATH, _collect_changes, _get_nested
+from kanta.serialization.base import unmarshal
+from kanta.structs import ChangeRecord
+from kanta.tty import ANSI_RE, ESC, strip_ansi
+
+_GLOB_CHARS = frozenset("*?[")
+
+_MARK_BG = f"{ESC}48;5;220m"  # yellow background (xterm256 #ffd700) for matches
+_UNMARK_BG = f"{ESC}49m"  # back to the default background, foreground untouched
+
+
+def mark_spans(styled: str, spans: list[tuple[int, int]]) -> str:
+    """Wrap the given visible-text spans of a styled string with the mark color.
+
+    ``spans`` are ``(start, end)`` offsets into the visible text of
+    *styled*; ANSI sequences are not counted.  Overlapping and adjacent
+    spans are merged first, so overlapping matches from different patterns
+    produce one continuous highlight.  The set/clear codes are inserted at
+    the mapped positions in *styled*; only the background attribute is
+    touched, leaving foreground colors intact.
+    """
+    spans = _merge_spans(spans)
+    if not spans:
+        return styled
+    out: list[str] = []
+    plain_pos = 0
+    prev_end = 0
+    for match in ANSI_RE.finditer(styled):
+        out.append(_wrap_run(styled[prev_end : match.start()], plain_pos, spans))
+        plain_pos += match.start() - prev_end
+        out.append(match.group(0))
+        prev_end = match.end()
+    out.append(_wrap_run(styled[prev_end:], plain_pos, spans))
+    return "".join(out)
+
+
+def _merge_spans(spans: list[tuple[int, int]]) -> list[tuple[int, int]]:
+    """Return *spans* sorted, with overlapping and adjacent spans merged."""
+    merged: list[list[int]] = []
+    for start, end in sorted(spans):
+        if start >= end:
+            continue
+        if merged and start <= merged[-1][1]:
+            merged[-1][1] = max(end, merged[-1][1])
+        else:
+            merged.append([start, end])
+    return [(start, end) for start, end in merged]
+
+
+def _wrap_run(run: str, plain_start: int, spans: list[tuple[int, int]]) -> str:
+    """Wrap the intersections of *spans* with one escape-free text run."""
+    if not run:
+        return run
+    out: list[str] = []
+    pos = 0
+    for start, end in spans:
+        s = max(start - plain_start, 0)
+        e = min(end - plain_start, len(run))
+        if s >= e or e <= pos:
+            continue
+        out.append(run[pos:s])
+        out.append(f"{_MARK_BG}{run[s:e]}{_UNMARK_BG}")
+        pos = e
+    out.append(run[pos:])
+    return "".join(out)
+
+
+def _find_spans(text: str, needle: str) -> list[tuple[int, int]]:
+    """Return visible-text spans of every case-insensitive occurrence of *needle*."""
+    if not needle:
+        return []
+    haystack = strip_ansi(text).lower()
+    needle = needle.lower()
+    spans = []
+    pos = 0
+    while (found := haystack.find(needle, pos)) >= 0:
+        end = found + len(needle)
+        spans.append((found, end))
+        pos = end
+    return spans
+
+
+@dataclasses.dataclass(frozen=True)
+class GrepPattern:
+    """One parsed ``--grep`` pattern.
+
+    The bare form (``term`` set) matches the action, the user, or any
+    dotted path or value in the diff.  The ``path=value`` form
+    (``path_term`` and ``value_term`` set) requires both sides to match
+    within the same change line; either side may be left empty to match
+    values only (``=value``) or paths only (``path=``).
+    """
+
+    raw: str
+    term: str | None = None
+    path_term: str | None = None
+    value_term: str | None = None
+
+    @classmethod
+    def parse(cls, raw: str) -> GrepPattern:
+        """Parse a pattern, splitting the ``path=value`` form on the first ``=``."""
+        if "=" in raw:
+            path_term, value_term = raw.split("=", 1)
+            return cls(raw, path_term=path_term, value_term=value_term)
+        return cls(raw, term=raw)
+
+
+@dataclasses.dataclass(frozen=True)
+class _ValueMark:
+    """A value-side match.
+
+    ``needle`` is the text to locate in the displayed value ("" = a match
+    that marks nothing, e.g. from an empty term).  ``whole`` marks the
+    entire displayed value: used when the raw value matched but a logfmt
+    formatter displays something else, so no needle can be located.
+    """
+
+    needle: str = ""
+    whole: bool = False
+
+
+@dataclasses.dataclass
+class _Entry:
+    """One matchable ``(path, value)`` line of a record's flattened diff.
+
+    ``anchor`` is set for deleted content: the deleted path whose line is
+    displayed for this entry (the entry itself may sit below it).
+    """
+
+    path: list[str]
+    raw: Any
+    text: str
+    anchor: list[str] | None = None
+
+
+def _element_matches(pattern: str, element: str) -> bool:
+    """Match one path element: in full, or as a glob when it uses wildcards."""
+    pattern = pattern.casefold()
+    element = element.casefold()
+    if any(char in pattern for char in _GLOB_CHARS):
+        return fnmatch.fnmatchcase(element, pattern)
+    return element == pattern
+
+
+def _match_path(term: str, path: list[str]) -> frozenset[int] | None:
+    """Match a dotted term against *path* as a contiguous element sequence.
+
+    Returns the indices of the matched elements, or ``None``.  An empty
+    term matches anything and marks no elements.
+    """
+    if not term:
+        return frozenset()
+    patterns = term.split(".")
+    for start in range(len(path) - len(patterns) + 1):
+        if all(
+            _element_matches(pattern, path[start + i])
+            for i, pattern in enumerate(patterns)
+        ):
+            return frozenset(range(start, start + len(patterns)))
+    return None
+
+
+def _match_value(term: str, raw: Any, text: str) -> _ValueMark | None:
+    """Match a term against a value.
+
+    String values match by substring; containers do not match (their
+    leaves are matched individually); all other values match only in
+    full.  A term with wildcards matches the whole value text.
+    """
+    if not term:
+        return _ValueMark()
+    needle = term.casefold()
+    haystack = text.casefold()
+    if any(char in needle for char in _GLOB_CHARS):
+        return _ValueMark(text) if fnmatch.fnmatchcase(haystack, needle) else None
+    if isinstance(raw, str):
+        return _ValueMark(term) if needle in haystack else None
+    if isinstance(raw, (dict, list)):
+        return None
+    return _ValueMark(text) if needle == haystack else None
+
+
+def _dual_mark(term: str, raw: Any, text: str, pretty: str | None) -> _ValueMark | None:
+    """Match a term against both the raw and the prettified form of a value.
+
+    *pretty* is the logfmt-resolved display text, which is what the log
+    shows when set.  A match on the displayed form locates its needle
+    there; a match on the raw form alone marks the whole displayed value.
+    """
+    raw_mark = _match_value(term, raw, text)
+    if pretty is None:
+        return raw_mark
+    pretty_mark = _match_value(term, pretty, pretty)
+    if pretty_mark is not None:
+        return pretty_mark
+    if raw_mark is not None and (raw_mark.needle or raw_mark.whole):
+        return _ValueMark(whole=True)
+    return raw_mark
+
+
+def _value_text(value: Any) -> str:
+    """Render a value as matchable text, following the display conventions."""
+    if value is None:
+        return "null"
+    if isinstance(value, bool):
+        return "true" if value else "false"
+    if isinstance(value, (int, float)):
+        return str(value)
+    if isinstance(value, str):
+        return value
+    if isinstance(value, (dict, list)):
+        try:
+            return json.dumps(value, default=str)
+        except (TypeError, ValueError):
+            pass
+    return str(value)
+
+
+def _leaf_entries(
+    path: list[str], value: Any, anchor: list[str] | None
+) -> Iterator[_Entry]:
+    """Yield an entry for every value under *value*.
+
+    Added, replaced or deleted containers are a single change line in the
+    log but hold many values; descending into them lets patterns match
+    their content.  List elements are addressed by index (``users.0``).
+    """
+    items: Iterator[tuple[str, Any]]
+    if isinstance(value, dict):
+        items = ((str(key), item) for key, item in value.items())
+    elif isinstance(value, list):
+        items = ((str(index), item) for index, item in enumerate(value))
+    else:
+        return
+    for key, item in items:
+        item_path = [*path, key]
+        yield _Entry(item_path, item, _value_text(item), anchor)
+        yield from _leaf_entries(item_path, item, anchor)
+
+
+def _record_entries(record: ChangeRecord, previous: dict | None) -> list[_Entry]:
+    """Flatten a record's diff into matchable entries.
+
+    Uses the same traversal as the change-log rendering, so paths match
+    what the log shows; deleted paths carry their previous value.
+    """
+    changes: list[tuple[str, list[str], Any]] = []
+    _collect_changes(unmarshal(record.diff), [], changes, previous)
+    entries: list[_Entry] = []
+    for change_type, path, value in changes:
+        anchor = None
+        if change_type == "delete":
+            value = _get_nested(previous, path)
+            anchor = path
+        entries.append(_Entry(path, value, _value_text(value), anchor))
+        entries.extend(_leaf_entries(path, value, anchor))
+    return entries
+
+
+class GrepHighlighter:
+    """The matched regions of one record, wrapping rendered text on demand.
+
+    Implements the highlighter hook of the :mod:`kanta.logging`
+    formatters: ``path`` for path elements, ``value`` for values,
+    ``meta`` for header fields and ``delete`` for deletion markers.
+    All wrapping goes through :func:`mark_spans`, so overlapping matches
+    merge into one highlight.
+    """
+
+    def __init__(self) -> None:
+        self._lit_paths: set[str] = set()
+        self._lit_deletes: set[str] = set()
+        self._value_marks: dict[str, list[_ValueMark]] = {}
+        self._meta_marks: dict[str, list[_ValueMark]] = {}
+
+    def _light_elements(self, path: list[str], indices) -> None:
+        for i in indices:
+            self._lit_paths.add(".".join(path[: i + 1]))
+
+    def add(
+        self,
+        entry: _Entry,
+        elements: frozenset[int] | None,
+        vmark: _ValueMark | None,
+    ) -> None:
+        """Record one entry's match: lit element indices and/or a value mark."""
+        marked = vmark is not None and (vmark.needle or vmark.whole)
+        if entry.anchor is None:
+            if elements:
+                self._light_elements(entry.path, elements)
+            if marked:
+                key = ".".join(entry.path)
+                self._value_marks.setdefault(key, []).append(vmark)
+            return
+        # Deleted content: only the anchor path line is displayed.  Light
+        # the genuinely matched elements within it; a match on the removed
+        # value or below the anchor marks the deletion marker (✗) instead.
+        if elements:
+            shown = {i for i in elements if i < len(entry.anchor)}
+            if shown:
+                self._light_elements(entry.path, shown)
+            if len(shown) != len(elements):
+                self._lit_deletes.add(".".join(entry.anchor))
+        if marked:
+            self._lit_deletes.add(".".join(entry.anchor))
+
+    def add_meta(self, field: str, mark: _ValueMark) -> None:
+        """Record a header match on ``field`` (``"action"`` or ``"user"``)."""
+        if mark.needle or mark.whole:
+            self._meta_marks.setdefault(field, []).append(mark)
+
+    def path(self, text: str, path: str) -> str:
+        """Wrap a rendered path element when its element matched."""
+        if text and path in self._lit_paths:
+            return mark_spans(text, [(0, len(strip_ansi(text)))])
+        return text
+
+    def delete(self, text: str, path: str) -> str:
+        """Wrap the deletion marker when the removed content matched."""
+        if text and path in self._lit_deletes:
+            return mark_spans(text, [(0, len(strip_ansi(text)))])
+        return text
+
+    @staticmethod
+    def _apply_marks(text: str, marks: list[_ValueMark]) -> str:
+        if not text or not marks:
+            return text
+        spans: list[tuple[int, int]] = []
+        for mark in marks:
+            if mark.whole:
+                spans.append((0, len(strip_ansi(text))))
+            else:
+                spans.extend(_find_spans(text, mark.needle))
+        return mark_spans(text, spans)
+
+    def value(self, text: str, path: str) -> str:
+        """Wrap the matched regions of a rendered value."""
+        return self._apply_marks(text, self._value_marks.get(path, []))
+
+    def meta(self, text: str, field: str) -> str:
+        """Wrap the matched regions of a rendered header field."""
+        return self._apply_marks(text, self._meta_marks.get(field, []))
+
+
+def evaluate(
+    record: ChangeRecord,
+    previous: dict | None,
+    patterns: list[GrepPattern],
+    logfmt: Any = None,
+) -> GrepHighlighter | None:
+    """Match *patterns* against a record, returning its matched regions.
+
+    Returns ``None`` when any pattern matches nowhere in the transaction.
+    Otherwise every pattern contributed its matches — action, user, or
+    change lines — to the returned highlighter; different patterns may
+    match different lines of the same record.
+
+    ``logfmt`` is the optional composed logfmt callable; when given, both
+    the raw and the prettified form of each value (and of the user) are
+    matched.
+    """
+    entries = _record_entries(record, previous)
+    highlighter = GrepHighlighter()
+    for pattern in patterns:
+        matched = False
+        if pattern.term is not None:
+            mark = _match_value(pattern.term, record.a, record.a)
+            if mark is not None:
+                highlighter.add_meta("action", mark)
+                matched = True
+            if record.u:
+                pretty_user = (
+                    logfmt(record.u, _USER_PATH) if logfmt is not None else None
+                )
+                mark = _dual_mark(pattern.term, record.u, record.u, pretty_user)
+                if mark is not None:
+                    highlighter.add_meta("user", mark)
+                    matched = True
+        for entry in entries:
+            pretty = None
+            if logfmt is not None:
+                pretty = logfmt(entry.raw, ".".join(entry.path))
+            if pattern.term is not None:
+                elements = _match_path(pattern.term, entry.path)
+                vmark = _dual_mark(pattern.term, entry.raw, entry.text, pretty)
+                if elements is None and vmark is None:
+                    continue
+            else:
+                elements = _match_path(pattern.path_term or "", entry.path)
+                vmark = _dual_mark(
+                    pattern.value_term or "", entry.raw, entry.text, pretty
+                )
+                if elements is None or vmark is None:
+                    continue
+            matched = True
+            highlighter.add(entry, elements, vmark)
+        if not matched:
+            return None
+    return highlighter

kanta/kanta.py

@@ -2,7 +2,7 @@
 
 from __future__ import annotations
 import logging
-from datetime import datetime
+from datetime import datetime, timedelta
 from pathlib import Path
 from types import ModuleType, SimpleNamespace
 from typing import Any, Generic, TypeVar
@@ -53,6 +53,7 @@ class Kanta(Generic[T]):
         migrations: ModuleType | str | None = None,
         serializer: Serializer | None = None,
         flush_interval: float = 0.1,
+        retention: timedelta | int | None = None,
     ):
         """Initialize a Kanta persistence instance.
 
@@ -63,6 +64,13 @@ class Kanta(Generic[T]):
             migrations: Optional migrations module object or import path.
             flush_interval: Background flush interval in seconds.
             serializer: Optional serializer implementation.
+            retention: Optional history retention window, either a
+                :class:`~datetime.timedelta` or a plain number of days. When
+                set, opening the database rotates it: history older than
+                ``now - retention`` is moved to a ``{stem}@{timestamp}.kantadb``
+                sibling file and the main file is rewritten with a fresh
+                snapshot plus the retained records (see ``docs/rotation.md``).
+                ``None`` (default) disables rotation.
 
         Raises:
             ImportError: If ``migrations`` is a string path that cannot be imported.
@@ -78,6 +86,7 @@ class Kanta(Generic[T]):
             type=data_type,
             migrations=migrations,
             flush_interval=flush_interval,
+            retention=retention,
             kanta=self,
         )
 
@@ -240,6 +249,27 @@ class Kanta(Generic[T]):
             return _register
         return _register(fn)
 
+    def validate(self, fn):
+        """Register a data validation callback.
+
+        Used as ``@kanta.validate``. The callback receives the live data
+        object (and optionally the ``Kanta`` instance) and must raise an
+        exception when the data is inconsistent. Validators run after replay
+        during :meth:`open` (after msgspec decoding and migrations) and after
+        each transaction, before the change is committed to history. Multiple
+        validators run in registration order until the first failure.
+
+        Validators must be synchronous and must not modify the data — they
+        only fail. A failure inside a transaction rolls the transaction back;
+        a failure during open aborts the open.
+        """
+
+        def _register(callback):
+            self._impl.add_validate(callback)
+            return callback
+
+        return _register(fn)
+
     def fatal_error(self, fn=None):
         """Register fatal error handler callback.
 
@@ -281,7 +311,7 @@ class Kanta(Generic[T]):
         """Register a migration logging callback.
 
         Can be used as ``@kanta.logmigr``.
-        The callback receives a :class:`kanta.migrations.MigrationResult` and
+        The callback receives a :class:`kanta.migrations.MigrationReport` and
         may be sync or async. If registered, it replaces the default migration
         logger output; the application is responsible for emitting any log
         messages.

kanta/kantaimpl.py

@@ -6,7 +6,7 @@ import asyncio
 import copy
 import importlib
 import logging
-from datetime import UTC, datetime
+from datetime import UTC, datetime, timedelta
 from types import SimpleNamespace
 from typing import Any, Generic, TypeVar
 
@@ -19,8 +19,9 @@ from kanta.logging import (
     emit_event,
     migration_logger,
 )
-from kanta.migrations import MigrationResult, Migrations
+from kanta.migrations import MigrationReport, Migrations
 from kanta.persistence import PersistenceMixin
+from kanta.rotation import execute_rotation, plan_rotation
 from kanta.serialization import restore_data_in_place, struct_to_dict
 from kanta.serialization.base import replay
 
@@ -42,6 +43,10 @@ class KantaImpl(PersistenceMixin, Generic[T]):
         self.data: T = kwargs.pop("data")
         self._kanta = kwargs.pop("kanta", None)
         migrations = kwargs.pop("migrations", None)
+        retention = kwargs.pop("retention", None)
+        if isinstance(retention, int) and not isinstance(retention, bool):
+            retention = timedelta(days=retention)
+        self.retention: timedelta | None = retention
         self.ctx = SimpleNamespace()
         super().__init__(**kwargs)
         self.migrations: Migrations | None = None
@@ -91,25 +96,28 @@ class KantaImpl(PersistenceMixin, Generic[T]):
         """Register one migration logging callback."""
         self.callback_registry.register("logmigr", callback)
 
+    def add_validate(self, callback) -> None:
+        """Register one data validation callback."""
+        self.callback_registry.register("validate", callback)
+
     def add_logemit(self, callback) -> None:
         """Register one log emitter callback."""
         self.callback_registry.register("logemit", callback)
 
     async def _handle_migration_log(
         self,
-        migration_result: MigrationResult,
-        previous_version: int,
+        report: MigrationReport,
         log: bool | logging.Logger,
     ) -> None:
         """Route migration logging to callback or default logger."""
-        assert isinstance(migration_result, MigrationResult)
+        assert isinstance(report, MigrationReport)
 
         if self.callback_registry.has("logmigr"):
             await self.callback_registry.invoke(
                 "logmigr",
                 InjectionContext(
                     kanta=self._kanta,
-                    migration_result=migration_result,
+                    report=report,
                 ),
                 on_error=_log_callback_error,
             )
@@ -120,7 +128,7 @@ class KantaImpl(PersistenceMixin, Generic[T]):
 
         migration_log = log if isinstance(log, logging.Logger) else migration_logger
 
-        changed = [m for m in migration_result.migrations if m.changed]
+        changed = [m for m in report.applied if m.changed]
         if not changed:
             return
 
@@ -131,8 +139,8 @@ class KantaImpl(PersistenceMixin, Generic[T]):
                 logger=migration_log,
                 kanta=self._kanta,
                 filename=str(self.filename),
-                from_version=previous_version,
-                to_version=migration_result.version,
+                from_version=report.original,
+                to_version=report.version,
                 migrations=descriptions,
             ),
             self.callback_registry.logemit_handlers,
@@ -182,6 +190,9 @@ class KantaImpl(PersistenceMixin, Generic[T]):
         # From this point the file is open and must be closed via close().
         self.opened = True
 
+        if content and self.retention is not None and not readonly:
+            content = await self._maybe_rotate(content, log)
+
         if content:
             try:
                 rr = replay(
@@ -211,15 +222,15 @@ class KantaImpl(PersistenceMixin, Generic[T]):
                     cause_type=type(e).__name__,
                 ) from e
 
-            migration_result = None
+            migration_report = None
             state_before_migrations = None
             previous_version = rr.version
             if self.migrations is not None:
                 state_before_migrations = copy.deepcopy(rr.state)
-                migration_result = self.migrations.apply(
+                migration_report = self.migrations.apply(
                     rr.state, rr.version, self._kanta
                 )
-                rr.version = migration_result.version
+                rr.version = migration_report.version
 
             migrations_ran = rr.version != previous_version
 
@@ -240,6 +251,16 @@ class KantaImpl(PersistenceMixin, Generic[T]):
                 self.data_type,
                 serializer=self.serializer,
             )
+            if self.callback_registry.has("validate"):
+                try:
+                    self.callback_registry.invoke_sync(
+                        "validate",
+                        InjectionContext(data=self.data, kanta=self._kanta),
+                    )
+                except Exception:
+                    self.opened = False
+                    self.file.close()
+                    raise
             self.version = rr.version
             self.mtime = rr.m
             if log is not False and not migrations_ran:
@@ -267,10 +288,8 @@ class KantaImpl(PersistenceMixin, Generic[T]):
                 )
                 record = self.queue_change(action, normalized, mtime=False)
                 # The migration summary introduces the diff, so log it first.
-                if migrations_ran and migration_result is not None:
-                    await self._handle_migration_log(
-                        migration_result, previous_version, log
-                    )
+                if migrations_ran and migration_report is not None:
+                    await self._handle_migration_log(migration_report, log)
                 if (
                     record is not None
                     and log is not False
@@ -317,6 +336,12 @@ class KantaImpl(PersistenceMixin, Generic[T]):
                         InjectionContext(data=self.data, kanta=self._kanta),
                     )
 
+                if self.callback_registry.has("validate"):
+                    self.callback_registry.invoke_sync(
+                        "validate",
+                        InjectionContext(data=self.data, kanta=self._kanta),
+                    )
+
                 self.statedict = {}
                 current = struct_to_dict(self.data, serializer=self.serializer)
                 record = self.queue_change(
@@ -378,6 +403,30 @@ class KantaImpl(PersistenceMixin, Generic[T]):
         if not self.readonly:
             self.background_task = asyncio.create_task(self._background_loop())
 
+    async def _maybe_rotate(self, content: bytes, log: bool | logging.Logger) -> bytes:
+        """Rotate history older than the retention window (see docs/rotation.md).
+
+        Runs while the file is locked and quiescent, before replay. Returns
+        the (possibly replaced) content to replay. Rotation failures abort the
+        open with the original file intact.
+        """
+        cutoff = self.now() - self.retention
+        plan = await asyncio.to_thread(
+            plan_rotation,
+            content,
+            framer=self.framer,
+            serializer=self.serializer,
+            cutoff=cutoff,
+            now=self.now(),
+            min_diffs=self.snapshot.min_diffs,
+        )
+        if plan is None:
+            return content
+        await asyncio.to_thread(
+            execute_rotation, self.filename, self.file, plan, log=log
+        )
+        return plan.new_content
+
     async def close(self) -> None:
         """Stop the background task, flush pending changes, and release the file lock."""
         if not self.opened:

kanta/logging.py

@@ -5,6 +5,8 @@ through :func:`emit_event`, which runs any registered ``logemit`` callbacks
 and falls back to :func:`default_emit` for the built-in formatting.  Diff
 output is formatted in a human-readable path notation style with color
 coding; see :mod:`kanta.tty` for the color palette and line builder.
+ANSI codes are stripped at emit time when the standard error stream does
+not support color (``NO_COLOR``/``FORCE_COLOR``, tty and journald checks).
 """
 
 import logging
@@ -15,7 +17,8 @@ from typing import Any
 
 import msgspec
 
-from kanta.tty import Line, displaywidth
+from kanta.serialization.base import _apply, unmarshal
+from kanta.tty import Line, displaywidth, strip_ansi, use_color
 
 transaction_logger = logging.getLogger("kanta.transaction")
 bootstrap_logger = logging.getLogger("kanta.bootstrap")
@@ -124,19 +127,22 @@ class LogEvent(msgspec.Struct, kw_only=True):
 def emit_event(
     ev: LogEvent,
     handlers: Iterable[Callable[[LogEvent], Any]] = (),
+    *,
+    fallback: Callable[[LogEvent], None] | None = None,
 ) -> None:
     """Dispatch *ev* through registered logemit handlers.
 
     Each handler receives the event and may log it (or not) as it sees fit.
     A falsy return value stops the chain: the event is considered handled.
     A truthy return value passes the event — possibly modified — to the next
-    handler.  When all handlers pass, :func:`default_emit` renders the event
-    with the built-in formatting.
+    handler.  When all handlers pass, the *fallback* renders the event;
+    the default fallback is :func:`default_emit` with the built-in formatting.
 
     Logging must never break functionality: a crashing handler is reported
-    and the chain falls back to the built-in formatting, and a failure in
-    the built-in formatting itself is reported and swallowed.
+    and the chain falls back to the fallback rendering, and a failure in
+    the fallback itself is reported and swallowed.
     """
+    render = fallback if fallback is not None else default_emit
     try:
         for handler in handlers:
             try:
@@ -146,11 +152,16 @@ def emit_event(
                 break
             if not proceed:
                 return
-        default_emit(ev)
+        render(ev)
     except Exception:
         _logger.exception("failed to emit %s log event", ev.kind)
 
 
+def _maybe_strip(text: str) -> str:
+    """Strip ANSI codes from *text* when stderr has no color support."""
+    return text if use_color() else strip_ansi(text)
+
+
 def default_emit(ev: LogEvent) -> None:
     """Emit *ev* with Kanta's built-in formatting.
 
@@ -159,25 +170,28 @@ def default_emit(ev: LogEvent) -> None:
     logger so it can be silenced or routed separately from the headers.
     This is what runs when no logemit callback handles the event; custom
     callbacks may call it to delegate events they do not care about.
+
+    ANSI color codes are stripped after formatting when the standard error
+    stream does not support color (see :func:`kanta.tty.use_color`).
     """
     if ev.kind != "change":
-        ev.logger.log(ev.level, ev.header)
+        ev.logger.log(ev.level, _maybe_strip(ev.header))
         return
 
     diff_logger = logging.getLogger(f"{ev.logger.name}.diff")
     lines = ev.diff_lines if ev.show_diff and diff_logger.isEnabledFor(ev.level) else []
 
     if not lines:
-        ev.logger.log(ev.level, ev.header)
+        ev.logger.log(ev.level, _maybe_strip(ev.header))
         return
 
     if len(lines) == 1:
-        diff_logger.log(ev.level, f"{ev.header}{lines[0]}")
+        diff_logger.log(ev.level, _maybe_strip(f"{ev.header}{lines[0]}"))
         return
 
-    ev.logger.log(ev.level, ev.header)
+    ev.logger.log(ev.level, _maybe_strip(ev.header))
     for line in lines:
-        diff_logger.log(ev.level, line)
+        diff_logger.log(ev.level, _maybe_strip(line))
 
 
 def _join_path(path: str, key: str) -> str:
@@ -187,62 +201,91 @@ def _join_path(path: str, key: str) -> str:
     return f"{path}.{key}"
 
 
+def _dim_ellipsis() -> str:
+    """Return the truncation ellipsis in the palette's ellipsis color."""
+    return str(Line().ellipsis("…"))
+
+
 def _format_value(
     value: Any,
     path: str,
     *,
     max_len: int = 60,
     logfmt: Callable[[Any, str], str | None] | None = None,
+    highlight: Any = None,
 ) -> str:
-    """Format a value for display, truncating if needed."""
+    """Format a value for display, truncating if needed.
+
+    ``highlight`` is an optional hook with ``path(text, path)`` and
+    ``value(text, path)`` methods (see :class:`kanta.grep.GrepHighlighter`);
+    it wraps matched keys and scalar values, and recurses into containers.
+    """
     if logfmt is not None:
         resolved = logfmt(value, path)
         if resolved is not None:
+            if highlight is not None:
+                resolved = highlight.value(resolved, path)
             return resolved
 
+    def keyed(key: Any, key_path: str) -> str:
+        display = _format_value(key, key_path, max_len=30, logfmt=logfmt)
+        if highlight is not None:
+            display = highlight.path(display, key_path)
+        return display
+
     if value is None:
-        return "null"
-    if isinstance(value, bool):
-        return "true" if value else "false"
-    if isinstance(value, (int, float)):
-        return str(value)
-    if isinstance(value, str):
-        value = _UNSAFE_CHARS.sub("", value)
-        if len(value) > max_len:
-            return value[: max_len - 3] + "..."
-        return value
-    if isinstance(value, dict):
+        text = "null"
+    elif isinstance(value, bool):
+        text = "true" if value else "false"
+    elif isinstance(value, (int, float)):
+        text = str(value)
+    elif isinstance(value, str):
+        text = _UNSAFE_CHARS.sub("", value)
+        if len(text) > max_len:
+            text = text[: max_len - 1] + _dim_ellipsis()
+    elif isinstance(value, dict):
         if not value:
             return "{}"
         all_true = all(v is True for v in value.values())
         parts = []
         for k, v in value.items():
             key_path = _join_path(path, str(k))
-            key_display = _format_value(k, key_path, max_len=30, logfmt=logfmt)
+            key_display = keyed(k, key_path)
             if all_true:
                 parts.append(key_display)
             else:
-                val_display = _format_value(v, key_path, max_len=30, logfmt=logfmt)
+                val_display = _format_value(
+                    v, key_path, max_len=30, logfmt=logfmt, highlight=highlight
+                )
                 parts.append(f"{key_display}: {val_display}")
         return "{" + ", ".join(parts) + "}"
-    if isinstance(value, list):
+    elif isinstance(value, list):
         if not value:
             return "[]"
         parts = []
         for i, v in enumerate(value):
             item_path = _join_path(path, str(i))
-            parts.append(_format_value(v, item_path, max_len=30, logfmt=logfmt))
+            parts.append(
+                _format_value(
+                    v, item_path, max_len=30, logfmt=logfmt, highlight=highlight
+                )
+            )
         return "[" + ", ".join(parts) + "]"
-    text = str(value)
-    if len(text) > max_len:
-        text = text[: max_len - 3] + "..."
+    else:
+        text = str(value)
+        if len(text) > max_len:
+            text = text[: max_len - 1] + _dim_ellipsis()
+    if highlight is not None:
+        text = highlight.value(text, path)
     return text
 
 
 def _format_path_components(
-    path: list[str], logfmt: Callable[[Any, str], str | None] | None
+    path: list[str],
+    logfmt: Callable[[Any, str], str | None] | None,
+    highlight: Any = None,
 ) -> list[str]:
-    """Return path components after applying formatters."""
+    """Return path components after applying formatters and match highlights."""
     if not path:
         return []
     result = []
@@ -253,6 +296,8 @@ def _format_path_components(
             resolved = logfmt(component, prefix_path)
             if resolved is not None:
                 display = resolved
+        if highlight is not None:
+            display = highlight.path(display, prefix_path)
         result.append(display)
     return result
 
@@ -261,12 +306,13 @@ def _format_path(
     path: list[str],
     logfmt: Callable[[Any, str], str | None] | None,
     final_color: str = "path_final",
+    highlight: Any = None,
 ) -> str:
     """Format a path as dot notation with prefix in dark grey, final colored.
 
     *final_color* names a color in the :data:`kanta.tty.colors` palette.
     """
-    components = _format_path_components(path, logfmt)
+    components = _format_path_components(path, logfmt, highlight)
     if not components:
         return ""
     line = Line()
@@ -304,6 +350,13 @@ def _collect_changes(
         changes.append(("update" if existed else "add", path, diff))
         return
 
+    old_at_path = _get_nested(previous, path)
+    if isinstance(old_at_path, list):
+        # List edits ($insert/$delete/per-index) are shown as one whole-list
+        # update; the diff is already unmarshaled at this point.
+        changes.append(("update", path, _apply(old_at_path, diff)))
+        return
+
     for key, value in diff.items():
         if key == "$delete":
             if isinstance(value, list):
@@ -332,7 +385,14 @@ def _collect_changes(
                     ("update" if old_collection is not None else "add", path, value)
                 )
         elif isinstance(key, str) and key.startswith("$"):
-            changes.append(("add", path, {key: value}))
+            # Unknown $-command or (post-unmarshal) a user key starting with
+            # "$": treat as a normal key.
+            new_path = path + [str(key)]
+            existed = _get_nested(previous, new_path) is not None
+            if existed:
+                _collect_changes(value, new_path, changes, previous)
+            else:
+                changes.append(("add", new_path, value))
         else:
             new_path = path + [str(key)]
             existed = _get_nested(previous, new_path) is not None
@@ -347,29 +407,44 @@ def _format_change_lines(
     path: list[str],
     value: Any,
     logfmt: Callable[[Any, str], str | None] | None = None,
+    highlight: Any = None,
 ) -> list[str]:
     """Format a single change as one or more lines."""
     if change_type == "delete":
-        components = _format_path_components(path, logfmt)
+        components = _format_path_components(path, logfmt, highlight)
         line = Line()("  ")
         if len(components) > 1:
             line.path_prefix(".".join(components[:-1]) + ".")
-        line.delete(components[-1], " ✗")
+        marker = "✗"
+        if highlight is not None:
+            marker = highlight.delete(marker, ".".join(path))
+        line.delete(components[-1], " ", marker)
         return [str(line)]
 
     if change_type == "add":
-        path_str = _format_path(path, logfmt, final_color="add")
+        path_str = _format_path(path, logfmt, final_color="add", highlight=highlight)
         if isinstance(value, dict) and value:
             lines = [str(Line()("  ", path_str, " ").sep("="))]
-            formatted_items = []
             base_path = ".".join(path)
-            for k, v in value.items():
+            keys = []
+            for k in value:
                 key_path = _join_path(base_path, str(k))
                 key_display = _format_value(k, key_path, max_len=30, logfmt=logfmt)
-                v_str = _format_value(v, key_path, max_len=30, logfmt=logfmt)
-                formatted_items.append((key_display, v_str))
-            field_width = max(displaywidth(k) for k, _ in formatted_items)
+                if highlight is not None:
+                    key_display = highlight.path(key_display, key_path)
+                keys.append((k, key_display))
+            field_width = max(displaywidth(kd) for _, kd in keys)
             field_width = max(field_width, 12)
+            # Each item line is "    {key:{field_width}}: {value}"; budget the
+            # value so the whole line fits in 80 columns.
+            value_width = max(80 - 4 - field_width - 2, 20)
+            formatted_items = []
+            for (k, key_display), v in zip(keys, value.values()):
+                key_path = _join_path(base_path, str(k))
+                v_str = _format_value(
+                    v, key_path, max_len=value_width, logfmt=logfmt, highlight=highlight
+                )
+                formatted_items.append((key_display, v_str))
             return lines + [
                 str(
                     Line()("    ", k).sep(":")(
@@ -378,11 +453,13 @@ def _format_change_lines(
                 )
                 for k, v in formatted_items
             ]
-        value_str = _format_value(value, ".".join(path), logfmt=logfmt)
+        value_str = _format_value(
+            value, ".".join(path), logfmt=logfmt, highlight=highlight
+        )
         return [str(Line()("  ", path_str, " ").sep("=")(" ", value_str))]
 
-    value_str = _format_value(value, ".".join(path), logfmt=logfmt)
-    path_str = _format_path(path, logfmt=logfmt)
+    value_str = _format_value(value, ".".join(path), logfmt=logfmt, highlight=highlight)
+    path_str = _format_path(path, logfmt=logfmt, highlight=highlight)
     return [str(Line()("  ", path_str, " ").sep("=")(" ", value_str))]
 
 
@@ -390,6 +467,7 @@ def format_diff(
     diff: dict,
     previous: dict | None = None,
     logfmt: Callable[[Any, str], str | None] | None = None,
+    highlight: Any = None,
 ) -> list[str]:
     """Format a JSON diff as human-readable lines.
 
@@ -400,16 +478,18 @@ def format_diff(
             ``path`` is a dot-notation string; ``"$user"`` is used for the
             transaction actor.  If the callable returns ``None``, default
             formatting is used.
+        highlight: Optional match highlighter hook (see
+            :class:`kanta.grep.GrepHighlighter`) wrapping matched regions.
 
     Returns a list of formatted lines (without newlines).
     """
     changes: list[tuple[str, list[str], Any]] = []
-    _collect_changes(diff, [], changes, previous)
+    _collect_changes(unmarshal(diff), [], changes, previous)
     if not changes:
         return []
     lines = []
     for change_type, path, value in changes:
-        lines.extend(_format_change_lines(change_type, path, value, logfmt))
+        lines.extend(_format_change_lines(change_type, path, value, logfmt, highlight))
     return lines
 
 
@@ -417,12 +497,22 @@ def format_action_header(
     action: str,
     user: str | None = None,
     extra: Any = None,
+    highlight: Any = None,
 ) -> str:
-    """Format the default action header line."""
+    """Format the default action header line.
+
+    ``highlight`` is an optional hook with a ``meta(text, field)`` method
+    (see :class:`kanta.grep.GrepHighlighter`) wrapping matched regions of
+    the action and user fields.
+    """
+    if highlight is not None:
+        action = highlight.meta(action, "action")
     line = Line().action(action)
     if extra is not None and (extra := f"{extra}"):
         line(" ").target(extra)
     if user is not None and (user := f"{user}"):
+        if highlight is not None:
+            user = highlight.meta(user, "user")
         line(" by ").user(user)
     return str(line)
 

kanta/migrations.py

@@ -13,7 +13,7 @@ from dataclasses import dataclass
 from types import ModuleType
 from typing import Any
 
-from kanta.diff import compute_diff
+from kanta.diff import diff
 from kanta.exceptions import DatabaseError
 
 # Cache registries by imported module object so that many Kanta instances using
@@ -34,11 +34,20 @@ class MigrationInfo:
 
 
 @dataclass
-class MigrationResult:
-    """Result of applying migrations."""
+class MigrationReport:
+    """Report of applying migrations."""
 
     version: int
-    migrations: list[MigrationInfo]
+    original: int
+    applied: list[MigrationInfo]
+
+    @property
+    def migrations(self) -> list[MigrationInfo]:
+        """Deprecated alias for :attr:`applied`."""
+        return self.applied
+
+
+MigrationResult = MigrationReport  # deprecated alias for MigrationReport
 
 
 class Migrations:
@@ -57,13 +66,13 @@ class Migrations:
         def migrate_v2(d: dict) -> None:
             d.setdefault("version", 2)
 
-        result = migrations.apply(state, current_version=0, kanta=kanta)
-        new_version = result.version
+        report = migrations.apply(state, current_version=0, kanta=kanta)
+        new_version = report.version
 
     Or load from a module::
 
         migrations = Migrations.from_module("myapp.migrations")
-        result = migrations.apply(state, current_version=0, kanta=kanta)
+        report = migrations.apply(state, current_version=0, kanta=kanta)
     """
 
     def __init__(self) -> None:
@@ -137,7 +146,7 @@ class Migrations:
         data_dict: dict[str, Any],
         current_version: int,
         kanta: Any,
-    ) -> MigrationResult:
+    ) -> MigrationReport:
         """Apply pending migrations to *data_dict* in place.
 
         Missing intermediate migration steps are silently skipped.
@@ -146,8 +155,8 @@ class Migrations:
             DatabaseError: If the database version is newer than the highest
                 supported version or older than the minimum supported version.
 
-        Returns a :class:`MigrationResult` describing the new version and every
-        migration that ran.
+        Returns a :class:`MigrationReport` describing the original and new
+        versions and every migration that ran.
         """
         if current_version > self.dbver:
             raise DatabaseError(
@@ -161,6 +170,7 @@ class Migrations:
             )
 
         migrations: list[MigrationInfo] = []
+        original = current_version
         for version in sorted(self._migrations.keys()):
             if version <= current_version:
                 continue
@@ -169,7 +179,7 @@ class Migrations:
             self._call_migration(fn, data_dict, kanta)
             current_version = version
             changed = before != data_dict
-            diff = compute_diff(before, data_dict) if changed else None
+            delta = diff(before, data_dict) if changed else None
             desc = (fn.__doc__ or f"v{version}").split("\n")[0].rstrip(".")
             migrations.append(
                 MigrationInfo(
@@ -177,8 +187,10 @@ class Migrations:
                     description=desc,
                     version=version,
                     changed=changed,
-                    diff=diff,
+                    diff=delta,
                     before=before,
                 )
             )
-        return MigrationResult(version=current_version, migrations=migrations)
+        return MigrationReport(
+            version=current_version, original=original, applied=migrations
+        )

kanta/persistence.py

@@ -13,7 +13,7 @@ from pathlib import Path
 from typing import Any
 
 from kanta.callbacks import CallbackRegistry, InjectionContext
-from kanta.diff import compute_diff
+from kanta.diff import diff
 from kanta.exceptions import DatabaseError, DataIntegrityError
 from kanta.filelock import LockedFile
 from kanta.structs import ChangeRecord
@@ -158,11 +158,11 @@ class PersistenceMixin:
             The queued :class:`ChangeRecord`, or ``None`` if the diff was empty
             and *force* is ``False``.
         """
-        diff = compute_diff(self.statedict, current)
-        if not diff:
+        delta = diff(self.statedict, current)
+        if not delta:
             if not force:
                 return None
-            diff = {}
+            delta = {}
 
         # The clock is only read when a record is actually queued.
         now = self.now()
@@ -182,7 +182,7 @@ class PersistenceMixin:
             v=self.version,
             u=user,
             m=m,
-            diff=diff,
+            diff=delta,
         )
         self.pending_changes.append(record)
         self.statedict = copy.deepcopy(current)

kanta/replaylog.py

@@ -0,0 +1,378 @@
+"""Line-oriented replay and range selection for kantadb files.
+
+Support machinery for the ``python -m kanta`` CLI: decoding a file into
+positioned events, resolving ``-r`` range specifications to line numbers,
+replaying state over a line range, and building change log events.  Internal
+for now; not part of the public API.
+"""
+
+from __future__ import annotations
+
+import copy
+import dataclasses
+from collections.abc import Iterator
+from typing import TYPE_CHECKING, Any, Union
+
+import msgspec
+
+from kanta.callbacks import InjectionContext
+from kanta.diff import patch
+from kanta.exceptions import ReplayError
+from kanta.logging import _USER_PATH, LogEvent, transaction_logger
+from kanta.structs import ChangeRecord, Snapshot
+
+if TYPE_CHECKING:
+    from kanta import Kanta
+
+
+@dataclasses.dataclass
+class SnapshotEvent:
+    """A snapshot record positioned in the file."""
+
+    line_number: int
+    byte_pos: int
+    record_index: int
+    snap: Snapshot
+
+    @property
+    def version(self) -> int:
+        return self.snap.v
+
+
+@dataclasses.dataclass
+class ChangeEvent:
+    """A change record positioned in the file."""
+
+    line_number: int
+    byte_pos: int
+    record_index: int
+    record: ChangeRecord
+
+    @property
+    def version(self) -> int:
+        return self.record.v
+
+
+Event = Union[SnapshotEvent, ChangeEvent]
+
+
+class RangeNotFoundError(Exception):
+    """A single-item range specification that does not exist in the file.
+
+    The message is fully formatted for display, including the offending
+    input and how many items of that kind the file contains.
+    """
+
+
+@dataclasses.dataclass
+class Selection:
+    """A resolved range specification.
+
+    Either a ``[start_line, end_line)`` line range, or a single snapshot
+    (``snapshot`` set), used to show the snapshot state without replaying
+    further records.
+    """
+
+    start_line: int
+    end_line: int
+    snapshot: SnapshotEvent | None = None
+
+
+def record_label(line_number: int, record_index: int) -> str:
+    """Return a padded record label based on line number, falling back to record index."""
+    number = line_number if line_number else record_index
+    return f"l{number:<3}"
+
+
+def scan_events(content: bytes, kanta: Kanta[Any]) -> tuple[list[Event], int]:
+    """Decode all records, validating snapshot consistency.
+
+    Uses the Kanta instance's serializer and framer.  Returns the events in
+    file order and the number of change records.  Raises :class:`ReplayError`
+    with a located, display-ready message on decode failures or when a
+    snapshot does not match the replayed state.
+    """
+    impl = kanta._impl
+    state: dict[str, Any] = {}
+    events: list[Event] = []
+    change_count = 0
+    for is_snapshot, payload, line_number, byte_pos in impl.framer.iter_records(
+        content, 0
+    ):
+        record_index = len(events) + 1
+        label = record_label(line_number, record_index)
+        try:
+            if is_snapshot:
+                snap = impl.serializer.decode(payload, type=Snapshot)
+                if record_index > 1 and state != snap.state:
+                    raise ReplayError(
+                        f"Snapshot mismatch at {label}: replayed state"
+                        " does not equal the snapshot state.",
+                        line_number=line_number,
+                        byte_pos=byte_pos,
+                        record_type="snapshot",
+                    )
+                state = snap.state
+                events.append(SnapshotEvent(line_number, byte_pos, record_index, snap))
+            else:
+                record = impl.serializer.decode(payload, type=ChangeRecord)
+                state = patch(state, record.diff)
+                events.append(ChangeEvent(line_number, byte_pos, record_index, record))
+                change_count += 1
+        except msgspec.DecodeError as exc:
+            raise ReplayError(
+                f"Parse error at {label}: {exc}",
+                line_number=line_number,
+                byte_pos=byte_pos,
+            ) from exc
+    return events, change_count
+
+
+def replay_events(
+    events: list[Event], end_line: int
+) -> Iterator[tuple[Event, dict[str, Any] | None, dict[str, Any]]]:
+    """Replay events with line numbers below ``end_line``.
+
+    Yields ``(event, previous, state)`` per event: ``previous`` is the state
+    before a change (``None`` for snapshots) and ``state`` the state after
+    the event.
+    """
+    state: dict[str, Any] = {}
+    for event in events:
+        if event.line_number >= end_line:
+            break
+        if isinstance(event, SnapshotEvent):
+            state = event.snap.state
+            yield event, None, state
+        else:
+            previous = copy.deepcopy(state)
+            state = patch(state, event.record.diff)
+            yield event, previous, state
+
+
+def record_change_event(
+    record: ChangeRecord,
+    previous: dict[str, Any],
+    current: dict[str, Any],
+    kanta: Kanta[Any],
+) -> LogEvent:
+    """Build a change :class:`LogEvent` for a replayed record.
+
+    The Kanta instance's logfmt callbacks are used for value formatting and
+    for resolving the user/actor name; the event can then be dispatched with
+    :func:`kanta.logging.emit_event` and the instance's logemit handlers.
+    """
+    registry = kanta._impl.callback_registry
+    logfmt = registry.build_logfmt(
+        InjectionContext(
+            kanta=kanta,
+            previous_state=previous,
+            current_state=current,
+        )
+    )
+    user = record.u
+    if user is not None:
+        resolved = logfmt(user, _USER_PATH)
+        if resolved is not None:
+            user = resolved
+    return LogEvent(
+        kind="change",
+        logger=transaction_logger,
+        kanta=kanta,
+        action=record.a,
+        user=user,
+        diff=record.diff,
+        previous=previous,
+        logfmt=logfmt,
+    )
+
+
+def _plural(count: int, word: str) -> str:
+    """Return e.g. ``1 snapshot`` or ``2 snapshots``."""
+    return f"{count} {word}{'' if count == 1 else 's'}"
+
+
+def end_of_file(events: list[Event]) -> int:
+    """Return the sentinel line number just past the last line of the file."""
+    return events[-1].line_number + 1 if events else 0
+
+
+def _change_lines(events: list[Event]) -> list[int]:
+    """Return the line numbers of all change records, in file order."""
+    return [e.line_number for e in events if isinstance(e, ChangeEvent)]
+
+
+def _snapshot_lines(events: list[Event]) -> list[int]:
+    """Return the line numbers addressed by s0, s1, ...
+
+    If the file begins with a snapshot, s0 is that snapshot (l1) and s1 is
+    the next snapshot.  Otherwise the file begins with change records (empty
+    initial state): s0 is l0, the position before the start of the file, and
+    s1 is the first snapshot.
+    """
+    lines = [e.line_number for e in events if isinstance(e, SnapshotEvent)]
+    if events and isinstance(events[0], SnapshotEvent):
+        return lines
+    return [0, *lines]
+
+
+def _version_lines(events: list[Event]) -> dict[int, int]:
+    """Map each version to the line where it first appears; v0 is l0."""
+    lines: dict[int, int] = {0: 0}
+    for event in events:
+        lines.setdefault(event.version, event.line_number)
+    return lines
+
+
+def _event_at_line(events: list[Event], line: int) -> Event | None:
+    """Return the event whose file line number exactly matches ``line``."""
+    for event in events:
+        if event.line_number == line:
+            return event
+    return None
+
+
+def _parse_bound(bound_str: str) -> tuple[str, int | None]:
+    """Parse a range bound with optional unit prefix (l, s, v) or change index."""
+    if not bound_str:
+        return "change", None
+    unit_map = {"l": "line", "s": "snapshot", "v": "version"}
+    if bound_str[0] in unit_map:
+        unit = unit_map[bound_str[0]]
+        rest = bound_str[1:]
+        if not rest:
+            raise ValueError(f"empty value in {bound_str!r}")
+        return unit, int(rest)
+    return "change", int(bound_str)
+
+
+def _bound_to_line(
+    unit: str,
+    value: int | None,
+    events: list[Event],
+    total: int,
+    is_start: bool,
+) -> int:
+    """Convert a range bound to a line number.
+
+    Out-of-range values are truncated to l0 (before the first line) or to
+    the line just past the end of the file rather than erroring; a missing
+    bound means the corresponding file end.
+    """
+    eof = end_of_file(events)
+    if value is None:
+        return 0 if is_start else eof
+
+    if unit == "change":
+        lines = _change_lines(events)
+        if value < 0:
+            value = total + value
+        value = max(0, min(value, total))
+        return lines[value] if value < total else eof
+
+    if unit == "line":
+        if value < 0:
+            raise ValueError("line numbers do not support negative indexing")
+        return value
+
+    if unit == "snapshot":
+        lines = _snapshot_lines(events)
+        idx = len(lines) + value if value < 0 else value
+        if idx < 0:
+            return 0
+        return lines[idx] if idx < len(lines) else eof
+
+    if unit == "version":
+        if value < 0:
+            raise ValueError("version numbers do not support negative indexing")
+        return _version_lines(events).get(value, eof)
+
+    raise ValueError(f"unknown range unit: {unit}")
+
+
+def _resolve_range(range_str: str, events: list[Event], total: int) -> tuple[int, int]:
+    """Parse a range string into a [start_line, end_line) line range."""
+    sep = ".." if ".." in range_str else ":"
+    start_str, end_str = range_str.split(sep, 1)
+    start_unit, start_val = _parse_bound(start_str)
+    end_unit, end_val = _parse_bound(end_str)
+    start_line = _bound_to_line(start_unit, start_val, events, total, is_start=True)
+    end_line = _bound_to_line(end_unit, end_val, events, total, is_start=False)
+    # ``..`` makes the end bound inclusive.
+    if sep == ".." and end_val is not None:
+        end_line += 1
+    return min(start_line, end_line), end_line
+
+
+def _negative_check(unit: str, value: int) -> None:
+    if value < 0:
+        raise ValueError(f"{unit} numbers do not support negative indexing")
+
+
+def select(spec: str, events: list[Event], total: int) -> Selection:
+    """Resolve a range specification against the scanned events.
+
+    ``total`` is the number of change records.  Returns a :class:`Selection`:
+    a line range, or a single snapshot for snapshot selections (``sN``, or
+    ``lN`` pointing at a snapshot).  Ranges truncate out-of-bounds values;
+    a single index must exist and raises :class:`RangeNotFoundError`
+    otherwise.  Syntax errors raise :class:`ValueError`.
+    """
+    if ":" in spec or ".." in spec:
+        start_line, end_line = _resolve_range(spec, events, total)
+        return Selection(start_line, end_line)
+
+    # A single index must exist; out-of-bounds is an error.
+    unit, value = _parse_bound(spec)
+    if value is None:
+        raise ValueError("single bound must not be empty")
+
+    if unit == "snapshot":
+        lines = _snapshot_lines(events)
+        idx = len(lines) + value if value < 0 else value
+        n_snapshots = sum(isinstance(e, SnapshotEvent) for e in events)
+        count = _plural(n_snapshots, "snapshot")
+        if not 0 <= idx < len(lines):
+            raise RangeNotFoundError(f"Snapshot {spec!r} not found in file ({count})")
+        event = _event_at_line(events, lines[idx])
+        if event is None:
+            # s0 with an empty initial state (l0): not a real record, so it
+            # cannot be selected as a single item.
+            raise RangeNotFoundError(
+                f"Snapshot {spec!r} not found in file: the file starts"
+                f" with an empty initial state ({count})"
+            )
+        assert isinstance(event, SnapshotEvent)
+        return Selection(event.line_number, event.line_number + 1, event)
+
+    if unit == "line":
+        _negative_check(unit, value)
+        event = _event_at_line(events, value)
+        if event is None:
+            n_lines = events[-1].line_number if events else 0
+            raise RangeNotFoundError(
+                f"Line {spec!r} not found in file ({_plural(n_lines, 'line')})"
+            )
+        if isinstance(event, SnapshotEvent):
+            return Selection(value, value + 1, event)
+        return Selection(value, value + 1)
+
+    if unit == "version":
+        _negative_check(unit, value)
+        lines = _version_lines(events)
+        if value not in lines:
+            raise RangeNotFoundError(
+                f"Version {spec!r} not found in file ({_plural(len(lines), 'version')})"
+            )
+        start_line = lines[value]
+        later = [line for line in lines.values() if line > start_line]
+        return Selection(start_line, min(later) if later else end_of_file(events))
+
+    lines = _change_lines(events)
+    idx = total + value if value < 0 else value
+    if not 0 <= idx < total:
+        raise RangeNotFoundError(
+            f"Change index {spec!r} not found in file ({_plural(total, 'change')})"
+        )
+    end_line = lines[idx + 1] if idx + 1 < total else end_of_file(events)
+    return Selection(lines[idx], end_line)

kanta/rotation.py

@@ -0,0 +1,225 @@
+"""Database rotation: bound on-disk history to a retention window.
+
+See docs/rotation.md for the design. All planning happens on the in-memory
+bytes of the database file; the caller (KantaImpl.open) performs the actual
+copy-aside, in-place rewrite and rotated-file trimming under the file lock.
+"""
+
+from __future__ import annotations
+
+import copy
+import logging
+import shutil
+from dataclasses import dataclass
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+
+from kanta.exceptions import DatabaseError
+from kanta.structs import ChangeRecord, Snapshot
+from kanta.serialization.base import Serializer, apply_diff
+from kanta.serialization.framing import Framer
+
+_logger = logging.getLogger(__name__)
+
+
+@dataclass
+class RotationPlan:
+    """Everything needed to execute a rotation on disk."""
+
+    new_content: bytes
+    cutoff_end: int  # byte length of the dropped-history prefix
+    rotated_ts: datetime  # ts of the last dropped change record
+    retained_changes: int
+
+
+def rotated_path_for(path: Path, ts: datetime) -> Path:
+    """Sibling path for the rotated history: ``{stem}@{ISO-basic-ts}.kantadb``.
+
+    The timestamp uses ISO 8601 basic format at second precision (e.g.
+    ``20260902T143000Z``); the exact microsecond timestamp remains available
+    inside the file if ever needed. On collision an incrementing suffix is
+    inserted before the extension.
+    """
+    stamp = ts.strftime("%Y%m%dT%H%M%SZ")
+    candidate = path.with_name(f"{path.stem}@{stamp}.kantadb")
+    n = 1
+    while candidate.exists():
+        candidate = path.with_name(f"{path.stem}@{stamp}.{n}.kantadb")
+        n += 1
+    return candidate
+
+
+class _Entry:
+    """One parsed record frame with its byte range."""
+
+    __slots__ = ("is_snapshot", "record", "byte_pos", "end_pos")
+
+    def __init__(
+        self,
+        is_snapshot: bool,
+        record: ChangeRecord | Snapshot,
+        byte_pos: int,
+        end_pos: int,
+    ) -> None:
+        self.is_snapshot = is_snapshot
+        self.record = record
+        self.byte_pos = byte_pos
+        self.end_pos = end_pos
+
+
+def _scan(content: bytes, *, framer: Framer, serializer: Serializer) -> list[_Entry]:
+    """Decode every record in *content* with byte ranges."""
+    raw = list(framer.iter_records(content, 0))
+    entries: list[_Entry] = []
+    for i, (is_snapshot, payload, _line, byte_pos) in enumerate(raw):
+        end_pos = raw[i + 1][3] if i + 1 < len(raw) else len(content)
+        record = serializer.decode(
+            payload, type=Snapshot if is_snapshot else ChangeRecord
+        )
+        entries.append(_Entry(is_snapshot, record, byte_pos, end_pos))
+    return entries
+
+
+def plan_rotation(
+    content: bytes,
+    *,
+    framer: Framer,
+    serializer: Serializer,
+    cutoff: datetime,
+    now: datetime,
+    min_diffs: int,
+) -> RotationPlan | None:
+    """Plan a rotation of *content*, or return None when there is nothing to do.
+
+    Raises:
+        DatabaseError: If replay from the chosen base snapshot does not match
+            a snapshot found inside the file (corrupt history). Rotation must
+            be aborted and the original file left untouched.
+    """
+    if not content:
+        return None
+    entries = _scan(content, framer=framer, serializer=serializer)
+    changes = [e for e in entries if not e.is_snapshot]
+    if not changes:
+        return None  # snapshot-only file: already fully rotated
+
+    dropped = [e for e in changes if e.record.ts < cutoff]
+    if not dropped:
+        return None  # retention window covers all history
+
+    retained = [e for e in changes if e.record.ts >= cutoff]
+    rotated_ts = dropped[-1].record.ts
+    cutoff_end = retained[0].byte_pos if retained else len(content)
+
+    # Replay base: walk snapshots newest-first and take the first (newest)
+    # one predating the cutoff; fall back to start of file.
+    base: _Entry | None = None
+    for e in reversed([e for e in entries if e.is_snapshot]):
+        if e.record.ts <= cutoff:
+            base = e
+            break
+
+    state: dict[str, Any] = {}
+    version = 0
+    m: datetime | None = None
+    if base is not None:
+        snap = base.record
+        assert isinstance(snap, Snapshot)
+        state = dict(snap.state)
+        version = snap.v
+        m = snap.m
+
+    # The cutoff state starts from the replay base: when the base snapshot
+    # already predates the cutoff, it may itself be the cutoff state.
+    state_at_cutoff: dict[str, Any] | None = (
+        copy.deepcopy(state) if base is not None else None
+    )
+    version_at_cutoff = version
+    m_at_cutoff = m
+    final_version = version
+    final_m = m
+
+    for e in entries:
+        if base is not None and e.byte_pos <= base.byte_pos:
+            continue
+        if e.is_snapshot:
+            snap = e.record
+            assert isinstance(snap, Snapshot)
+            if snap.state != state:
+                raise DatabaseError(
+                    "rotation aborted: replayed state does not match snapshot "
+                    f"at byte {e.byte_pos}",
+                    action="rotate",
+                )
+            if snap.m is not None:
+                m = snap.m
+            continue
+        change = e.record
+        assert isinstance(change, ChangeRecord)
+        state = apply_diff(state, change.diff)
+        version = change.v
+        if change.m is not None:
+            m = change.m
+        if change.ts < cutoff:
+            state_at_cutoff = copy.deepcopy(state)
+            version_at_cutoff = version
+            m_at_cutoff = m
+        final_version = version
+        final_m = m
+
+    # There is at least one dropped change, so the cutoff state is known.
+    assert state_at_cutoff is not None
+
+    # Build the new content: leading cutoff snapshot, retained changes
+    # re-framed at fresh offsets, and a final snapshot when enough changes
+    # survived to warrant one (mirrors the regular snapshot policy).
+    out = bytearray()
+    leading = serializer.encode(
+        Snapshot(
+            ts=rotated_ts, v=version_at_cutoff, state=state_at_cutoff, m=m_at_cutoff
+        )
+    )
+    out += framer.frame_snapshot(leading, record_offset=0)
+    for e in retained:
+        payload = serializer.encode(e.record)
+        out += framer.frame_change(payload, record_offset=len(out))
+    if len(retained) >= min_diffs:
+        closing = serializer.encode(
+            Snapshot(ts=now, v=final_version, state=state, m=final_m)
+        )
+        out += framer.frame_snapshot(closing, record_offset=len(out))
+
+    return RotationPlan(
+        new_content=bytes(out),
+        cutoff_end=cutoff_end,
+        rotated_ts=rotated_ts,
+        retained_changes=len(retained),
+    )
+
+
+def execute_rotation(
+    path: Path, file, plan: RotationPlan, *, log: bool | logging.Logger = True
+) -> Path:
+    """Execute a planned rotation on disk. Caller must hold the lock on *file*.
+
+    1. Copy the original content aside to ``{stem}@{ts}.kantadb``.
+    2. Rewrite the locked file in place with the new content and fsync.
+    3. Trim the rotated copy to the dropped-history prefix.
+
+    Returns the rotated file path.
+    """
+    rotated = rotated_path_for(path, plan.rotated_ts)
+    shutil.copy2(path, rotated)
+    file.replace_content(plan.new_content)
+    with open(rotated, "r+b") as f:
+        f.truncate(plan.cutoff_end)
+    if log:
+        _logger.info(
+            "rotated %s: kept %d change record(s), history before %s moved to %s",
+            path,
+            plan.retained_changes,
+            plan.rotated_ts.isoformat(),
+            rotated,
+        )
+    return rotated

kanta/serialization/base.py

@@ -121,33 +121,100 @@ def replay(
 
 
 def _patch_state(state: dict, diff: dict) -> dict:
-    return _apply_diff(state, diff)
+    return apply_diff(state, diff)
 
 
-def _apply_diff(state: dict, diff: dict) -> dict:
+def _unescape(value: str) -> str:
+    """Reverse jsondiff's ``$$`` escaping; command strings pass through.
+
+    Only a ``$$`` prefix is stripped: jsondiff escapes ``$x`` to ``$$x``,
+    while single ``$`` strings occur verbatim in our own diffs (we do not
+    escape values) and must be left alone.
+    """
+    if value.startswith("$$"):
+        return value[1:]
+    return value
+
+
+def unmarshal(diff: Any) -> Any:
+    """Unescape a marshaled diff (keys, values and ``$delete`` entries).
+
+    Needed for jsondiff-produced diffs, which escape ``$``-prefixed values
+    as well as keys; our own producer escapes keys only, so unescaping
+    values is a no-op for them.
+    """
+    if isinstance(diff, dict):
+        return {
+            _unescape(k) if isinstance(k, str) else k: unmarshal(v)
+            for k, v in diff.items()
+        }
+    if isinstance(diff, list):
+        return [unmarshal(v) for v in diff]
+    if isinstance(diff, str):
+        return _unescape(diff)
+    return diff
+
+
+def apply_diff(state: Any, diff: Any) -> Any:
+    """Apply a diff.
+
+    Understands our own format (plain assignment + ``$delete``) and
+    jsondiff's marshaled syntax: ``$replace``, positional
+    ``$delete``/``$insert`` and per-index nested diffs on lists. A bare
+    dict over a non-dict old value is a wholesale replacement.
+    """
+    return _apply(state, unmarshal(diff))
+
+
+def _is_list_patch(diff: dict) -> bool:
+    """Whether a dict diff against a list state is a jsondiff list edit."""
+    for key in diff:
+        if key in ("$delete", "$insert"):
+            continue
+        try:
+            int(key)
+        except (ValueError, TypeError):
+            return False
+    return True
+
+
+def _apply(state: Any, diff: Any) -> Any:
     if not isinstance(diff, dict):
         return diff
+    if not diff:
+        return state
+    if "$replace" in diff:
+        return diff["$replace"]
 
-    result = dict(state) if isinstance(state, dict) else state
-    if not isinstance(result, dict):
-        result = {}
+    if isinstance(state, list):
+        if not _is_list_patch(diff):
+            # Our own producer replaces a list with a dict (or any other
+            # type) by plain assignment — no $replace wrapper.
+            return diff
+        result = list(state)
+        deletes = diff.get("$delete")
+        if deletes:
+            for pos in deletes:
+                result.pop(pos)
+        for pos, value in diff.get("$insert", []):
+            result.insert(pos, value)
+        for key, value in diff.items():
+            if key in ("$delete", "$insert"):
+                continue
+            pos = int(key)
+            result[pos] = _apply(result[pos], value)
+        return result
 
+    result = dict(state) if isinstance(state, dict) else {}
     for key, value in diff.items():
-        if key == "$replace":
-            return value
         if key == "$delete":
-            if isinstance(value, list):
-                for k in value:
-                    result.pop(k, None)
-            else:
-                result.pop(value, None)
+            keys = value if isinstance(value, list) else [value]
+            for k in keys:
+                result.pop(k, None)
+        elif key == "$insert":
             continue
-        if isinstance(value, dict):
-            old = result.get(key, {})
-            if not isinstance(old, dict):
-                old = {}
-            result[key] = _apply_diff(old, value)
-            continue
-        result[key] = value
-
+        elif key in result:
+            result[key] = _apply(result[key], value)
+        else:
+            result[key] = value
     return result

kanta/snapshot.py

@@ -35,6 +35,11 @@ class SnapshotState:
         """Force snapshot write on next check."""
         self._force_pending = True
 
+    @property
+    def min_diffs(self) -> int:
+        """Minimum accumulated changes before a snapshot may be written."""
+        return self._min_diffs
+
     def record_changes(self, count: int) -> None:
         self.changes += count
 

kanta/transaction.py

@@ -7,7 +7,7 @@ from contextlib import contextmanager
 from datetime import datetime
 from typing import Any
 
-from kanta.diff import compute_diff
+from kanta.diff import diff
 from kanta.exceptions import DataIntegrityError
 from kanta.callbacks import InjectionContext
 from kanta.logging import _USER_PATH, LogEvent, emit_event, transaction_logger
@@ -65,19 +65,19 @@ def transaction(
     if current_dict != impl.statedict:
         is_bootstrap = action in {"bootstrap"}
         if not (is_bootstrap and not impl.statedict):
-            diff = compute_diff(impl.statedict, current_dict)
-            if diff:
+            delta = diff(impl.statedict, current_dict)
+            if delta:
                 _logger.critical(
                     "Database state modified outside of transaction! "
                     "This indicates a bug where changes occurred without a transaction wrapper.\n"
                     "Changes detected: %s",
-                    diff,
+                    delta,
                 )
                 raise DataIntegrityError(
                     "Database state modified outside of transaction",
                     db_path=impl.db_path,
                     action=action,
-                    diff=diff,
+                    diff=delta,
                 )
 
     impl.in_transaction = True
@@ -86,8 +86,13 @@ def transaction(
     try:
         yield impl.data
         new_dict = struct_to_dict(impl.data, serializer=impl.serializer)
-        diff = compute_diff(impl.statedict, new_dict)
-        if diff:
+        delta = diff(impl.statedict, new_dict)
+        if delta:
+            if impl.callback_registry.has("validate"):
+                impl.callback_registry.invoke_sync(
+                    "validate",
+                    InjectionContext(data=impl.data, kanta=impl._kanta),
+                )
             previous = impl.statedict
             record = impl.queue_change(action, new_dict, user=user, mtime=mtime)
             if record is not None:

kanta/tty.py

@@ -10,8 +10,12 @@ color instead of emitting a separate one.
 
 from __future__ import annotations
 
+import io
+import os
 import re
+import sys
 import unicodedata
+from contextlib import suppress
 from typing import Any
 
 ESC = "\x1b["
@@ -25,6 +29,21 @@ def strip_ansi(text: str) -> str:
     return ANSI_RE.sub("", text)
 
 
+def use_color(stream: io.TextIOBase = sys.stderr) -> bool:
+    """Test if the stream supports color codes."""
+    if os.environ.get("NO_COLOR"):  # Non empty means no (no-color.org)
+        return False
+    if os.environ.get("FORCE_COLOR", "") not in {"", "0"}:  # force-color.org, node
+        return True
+    if hasattr(stream, "isatty") and stream.isatty():
+        return True
+    with suppress(KeyError, ValueError, OSError):  # Journald does color (-ocat)
+        dev, ino = map(int, os.environ["JOURNAL_STREAM"].split(":", 1))
+        st = os.fstat(stream.fileno())
+        return st.st_dev == dev and st.st_ino == ino
+    return False
+
+
 def displaywidth(text: str) -> int:
     """Return the terminal column width of *text*, ignoring ANSI sequences.
 
@@ -70,11 +89,13 @@ class Colors:
     action = "1;34"  # Bold blue for the action name
     user = "34"  # Blue for the user display
     target = "38;5;250"  # White for the extra/target display
+    snapshot = "97"  # Bright white for snapshot indicator text
     sep = "38;5;242"  # Dark grey for separators
     path_prefix = "38;5;242"  # Dark grey for the leading part of a dotted path
     path_final = "38;5;250"  # White for the final path element
     add = "32"  # Green for additions
     delete = "1;31"  # Bold red for deletions
+    ellipsis = "38;5;242"  # Dark grey for the truncation ellipsis
 
 
 colors = Colors()

pyproject.toml

@@ -17,20 +17,24 @@ readme = "README.md"
 requires-python = ">=3.11"
 dependencies = [
     "blake3>=1.0.8",
-    "jsondiff>=2.2.1",
     "msgspec>=0.20.0",
 ]
 
+[project.scripts]
+kanta = "kanta.__main__:main"
+
 [project.optional-dependencies]
 bin = [
     "blake3>=1.0.8",
 ]
 
 [project.urls]
+Homepage = "https://vasanko.com/coders/kanta"
 Repository = "https://git.zi.fi/LeoVasanko/kanta"
 
 [dependency-groups]
 dev = [
+    "jsondiff>=2.2.1",
     "pytest>=9.0.2",
     "pytest-asyncio>=1.3.0",
 ]

tests/test_callbacks.py

@@ -2,7 +2,7 @@ from typing import Any, Optional, Union
 
 import pytest
 
-from kanta import Kanta
+from kanta import DictPrev, DictState, Kanta
 from kanta.callbacks import DictPost, DictPre, LogFmt
 from kanta.exceptions import DatabaseError
 
@@ -166,6 +166,106 @@ async def test_logfmt_injects_states(tmp_path, format_config, caplog):
     assert "Alice" in caplog.text
 
 
+@pytest.mark.asyncio
+async def test_logfmt_injects_states_by_name(tmp_path, format_config, caplog):
+    import logging
+
+    caplog.set_level(logging.INFO, logger="kanta.transaction")
+    path = tmp_path / "test.db"
+    kanta = make_kanta(path, Data, format_config)
+
+    @kanta.logfmt
+    def resolve_users(value: str, prev, state: dict | None) -> str | None:
+        assert prev == {}
+        assert state is not None
+        return state.get("users", {}).get(value, {}).get("name")
+
+    await kanta.open()
+
+    with kanta.transaction(action="create_user") as data:
+        data.users["uuid-9"] = User(name="Carol")
+
+    await kanta.close()
+
+    assert "Carol" in caplog.text
+
+
+@pytest.mark.asyncio
+async def test_logfmt_state_name_ignores_annotation(tmp_path, format_config, caplog):
+    import logging
+
+    caplog.set_level(logging.INFO, logger="kanta.transaction")
+    path = tmp_path / "test.db"
+    kanta = make_kanta(path, Data, format_config)
+
+    # Matching by name does not check the annotation.
+    @kanta.logfmt
+    def resolve_users(value: str, state: int) -> str | None:
+        return state.get("users", {}).get(value, {}).get("name")
+
+    await kanta.open()
+
+    with kanta.transaction(action="create_user") as data:
+        data.users["uuid-10"] = User(name="Dave")
+
+    await kanta.close()
+
+    assert "Dave" in caplog.text
+
+
+@pytest.mark.asyncio
+async def test_logfmt_tag_takes_precedence_over_name(tmp_path, format_config, caplog):
+    import logging
+
+    caplog.set_level(logging.INFO, logger="kanta.transaction")
+    path = tmp_path / "test.db"
+    kanta = make_kanta(path, Data, format_config)
+
+    @kanta.logfmt
+    def check_prev(value: str, anything: DictPrev) -> str | None:
+        assert anything == {}
+        return None
+
+    @kanta.logfmt
+    def resolve_users(value: str, prev: DictState) -> str | None:
+        # The tag wins: prev receives the current state despite its name.
+        return prev.get("users", {}).get(value, {}).get("name")
+
+    await kanta.open()
+
+    with kanta.transaction(action="create_user") as data:
+        data.users["uuid-11"] = User(name="Erin")
+
+    await kanta.close()
+
+    assert "Erin" in caplog.text
+
+
+@pytest.mark.asyncio
+async def test_logfmt_class_state_attribute(tmp_path, format_config, caplog):
+    import logging
+
+    caplog.set_level(logging.INFO, logger="kanta.transaction")
+    path = tmp_path / "test.db"
+    kanta = make_kanta(path, Data, format_config)
+
+    @kanta.logfmt
+    class UserLogFmt(LogFmt):
+        def resolve(self, value: str, path: str) -> str | None:
+            if not isinstance(value, str):
+                return None
+            return self.state.get("users", {}).get(value, {}).get("name")
+
+    await kanta.open()
+
+    with kanta.transaction(action="create_user") as data:
+        data.users["uuid-12"] = User(name="Fred")
+
+    await kanta.close()
+
+    assert "Fred" in caplog.text
+
+
 @pytest.mark.asyncio
 async def test_logfmt_class_injection(tmp_path, format_config, caplog):
     import logging

tests/test_cli.py

@@ -0,0 +1,174 @@
+"""Tests for the ``python -m kanta`` CLI output formatting."""
+
+import sys
+from datetime import UTC, datetime
+
+import pytest
+
+from kanta.__main__ import (
+    _extra_import_paths,
+    _format_ts,
+    _import_dotted,
+    _import_kanta_object,
+    main,
+)
+from kanta.serialization import JsonSerializer
+from kanta.serialization.framing import LineFramer
+from kanta.structs import ChangeRecord, Snapshot
+
+
+def test_format_ts_strips_microseconds():
+    """Timestamps are rendered without microsecond precision."""
+    dt = datetime(2026, 8, 12, 10, 6, 52, 375398, tzinfo=UTC)
+    assert _format_ts(dt) == "2026-08-12 10:06:52"
+
+
+def test_extra_import_paths_are_temporary(tmp_path, monkeypatch):
+    """CWD and nearby venv site-packages are added only for the import block."""
+    parent_dir = tmp_path / "parent"
+    cwd = parent_dir / "child"
+    venv_site = (
+        cwd
+        / ".venv"
+        / "lib"
+        / f"python{sys.version_info.major}.{sys.version_info.minor}"
+        / "site-packages"
+    )
+    venv_site.mkdir(parents=True)
+    parent_venv_site = (
+        parent_dir
+        / ".venv"
+        / "lib"
+        / f"python{sys.version_info.major}.{sys.version_info.minor}"
+        / "site-packages"
+    )
+    parent_venv_site.mkdir(parents=True)
+
+    monkeypatch.chdir(cwd)
+    cwd_str = str(cwd)
+    venv = str(venv_site)
+    parent_venv = str(parent_venv_site)
+
+    before = sys.path.copy()
+    with _extra_import_paths():
+        during = sys.path.copy()
+        assert cwd_str in during
+        assert venv in during
+        assert parent_venv in during
+        assert during.index(cwd_str) < during.index(venv) < during.index(parent_venv)
+    assert sys.path == before
+
+
+def test_extra_import_paths_ignores_other_python_versions(tmp_path, monkeypatch):
+    """Only the site-packages for the running Python version is picked up."""
+    current_site = (
+        tmp_path
+        / ".venv"
+        / "lib"
+        / f"python{sys.version_info.major}.{sys.version_info.minor}"
+        / "site-packages"
+    )
+    other_site = tmp_path / ".venv" / "lib" / "python9.9" / "site-packages"
+    current_site.mkdir(parents=True)
+    other_site.mkdir(parents=True)
+
+    monkeypatch.chdir(tmp_path)
+    with _extra_import_paths():
+        assert str(current_site) in sys.path
+        assert str(other_site) not in sys.path
+
+
+def test_cli_snapshot_line_format(tmp_path, capsys, monkeypatch):
+    """Snapshot lines are timestamped and colored with metadata."""
+    monkeypatch.setenv("FORCE_COLOR", "1")
+    monkeypatch.delenv("NO_COLOR", raising=False)
+    path = tmp_path / "test.kantadb"
+    ts = datetime(2026, 8, 12, 10, 6, 52, 375398, tzinfo=UTC)
+    mtime = datetime(2026, 8, 12, 9, 0, 0, tzinfo=UTC)
+    serializer = JsonSerializer()
+    framer = LineFramer()
+
+    snapshot = Snapshot(ts=ts, v=1, m=mtime, state={"counter": 5})
+    change = ChangeRecord(ts=ts, a="inc", v=1, u="user1", diff={"counter": 6})
+    data = framer.frame_snapshot(
+        serializer.encode(snapshot), record_offset=0
+    ) + framer.frame_change(serializer.encode(change), record_offset=0)
+    path.write_bytes(data)
+
+    code = main([str(path)])
+    assert code == 0
+
+    err = capsys.readouterr().err
+    # No microsecond precision anywhere.
+    assert "10:06:52" in err
+    assert "10:06:52.375398" not in err
+
+    # Snapshot line: bright white snapshot/sN, white version/mtime, dark size.
+    assert "\x1b[97msnapshot s0" in err
+    assert "\x1b[38;5;250m v1 2026-08-12 09:00:00" in err
+    assert "\x1b[38;5;242m 13 B" in err
+
+
+def test_cli_strips_ansi_without_color_support(tmp_path, capsys, monkeypatch):
+    """Without a tty and with NO_COLOR set, output contains no ANSI codes."""
+    monkeypatch.setenv("NO_COLOR", "1")
+    monkeypatch.delenv("FORCE_COLOR", raising=False)
+    path = tmp_path / "test.kantadb"
+    ts = datetime(2026, 8, 12, 10, 6, 52, 375398, tzinfo=UTC)
+    serializer = JsonSerializer()
+    framer = LineFramer()
+
+    snapshot = Snapshot(ts=ts, v=1, m=None, state={"counter": 5})
+    change = ChangeRecord(ts=ts, a="inc", v=1, u="user1", diff={"counter": 6})
+    data = framer.frame_snapshot(
+        serializer.encode(snapshot), record_offset=0
+    ) + framer.frame_change(serializer.encode(change), record_offset=0)
+    path.write_bytes(data)
+
+    code = main([str(path)])
+    assert code == 0
+
+    err = capsys.readouterr().err
+    assert "\x1b[" not in err
+    assert "snapshot s0" in err
+
+
+def test_cli_version_on_help_and_version_flag(capsys):
+    """--help and --version print the installed package version."""
+    import importlib.metadata
+
+    version = importlib.metadata.version("kanta")
+
+    with pytest.raises(SystemExit) as help_exit:
+        main(["--help"])
+    assert help_exit.value.code == 0
+    assert f"kanta {version}" in capsys.readouterr().out
+
+    with pytest.raises(SystemExit) as version_exit:
+        main(["--version"])
+    assert version_exit.value.code == 0
+    assert capsys.readouterr().out.strip() == f"kanta {version}"
+
+
+def test_import_dotted_from_file_path(tmp_path):
+    """--data can be a filesystem path with an optional colon-separated symbol."""
+    module = tmp_path / "models.py"
+    module.write_text("class Data:\n    pass\n")
+    result = _import_dotted(f"{module}:Data")
+    assert result.__name__ == "Data"
+
+
+def test_import_kanta_object_from_file_path(tmp_path):
+    """--kanta can be a filesystem path; default symbol is ``kanta``."""
+    module = tmp_path / "database.py"
+    module.write_text("class Kanta:\n    pass\nkanta = Kanta()\n")
+    result = _import_kanta_object(str(module))
+    assert type(result).__name__ == "Kanta"
+
+
+def test_import_kanta_object_from_file_path_with_symbol(tmp_path):
+    """--kanta can be a filesystem path with an explicit colon-separated symbol."""
+    module = tmp_path / "database.py"
+    module.write_text("class CustomKanta:\n    pass\nmy_kanta = CustomKanta()\n")
+    result = _import_kanta_object(f"{module}:my_kanta")
+    assert type(result).__name__ == "CustomKanta"

tests/test_diff.py

@@ -1,16 +1,258 @@
-from kanta.diff import compute_diff
+"""Tests for our own diff producer/consumer and jsondiff compatibility.
+
+jsondiff is a dev dependency used only here, to verify that:
+
+- jsondiff.patch(..., marshal=True) can apply patches produced by
+  diff (our format is a subset of jsondiff's marshaled syntax);
+- apply_diff can apply patches produced by jsondiff.diff(..., marshal=True),
+  including positional $insert/$delete list edits and per-index nested diffs.
+"""
+
+import jsondiff
+import pytest
+
+from kanta.diff import diff, patch
+from kanta.logging import format_diff
+from kanta.serialization.base import apply_diff
+
+# --- Producer: diff ------------------------------------------------
 
 
 def test_no_diff():
-    assert compute_diff({"a": 1}, {"a": 1}) is None
+    assert diff({"a": 1}, {"a": 1}) is None
+    assert diff({}, {}) is None
 
 
 def test_simple_diff():
-    diff = compute_diff({"a": 1}, {"a": 2})
-    assert diff is not None
-    assert diff == {"a": 2}
+    delta = diff({"a": 1}, {"a": 2})
+    assert delta is not None
+    assert delta == {"a": 2}
 
 
 def test_nested_diff():
-    diff = compute_diff({"x": {"y": 1}}, {"x": {"y": 2}})
-    assert diff == {"x": {"y": 2}}
+    delta = diff({"x": {"y": 1}}, {"x": {"y": 2}})
+    assert delta == {"x": {"y": 2}}
+
+
+def test_key_added():
+    assert diff({"a": 1}, {"a": 1, "b": 2}) == {"b": 2}
+
+
+def test_key_removed():
+    assert diff({"a": 1, "b": 2}, {"a": 1}) == {"$delete": ["b"]}
+
+
+def test_last_key_removed_is_delete_not_replace():
+    # jsondiff's minimal-diff search emits {"$replace": {}} here; we emit
+    # what actually happened: the key was deleted.
+    assert diff({"a": 1}, {}) == {"$delete": ["a"]}
+    assert diff({"x": {"y": 1}}, {"x": {}}) == {"x": {"$delete": ["y"]}}
+
+
+def test_list_changes_are_full_assignment():
+    # No $insert/$delete positional edits: lists are replaced wholesale.
+    assert diff({"l": [1, 2]}, {"l": [1, 2, 3]}) == {"l": [1, 2, 3]}
+    assert diff({"l": [1, 2, 3]}, {"l": [1, 3]}) == {"l": [1, 3]}
+    assert diff({"l": [1]}, {"l": []}) == {"l": []}
+
+
+def test_list_with_unchanged_prefix_is_full_assignment():
+    delta = diff({"l": ["a", "b", "c"]}, {"l": ["a", "x", "b", "c"]})
+    assert delta == {"l": ["a", "x", "b", "c"]}
+
+
+def test_type_changes_are_full_assignment():
+    assert diff({"a": {"x": 1}}, {"a": [1]}) == {"a": [1]}
+    # A dict replacing a non-dict is a plain assignment too: the consumer
+    # sees from the old value whether to patch (dict) or replace.
+    assert diff({"a": [1]}, {"a": {"x": 1}}) == {"a": {"x": 1}}
+    assert diff({"a": 1}, {"a": None}) == {"a": None}
+
+
+def test_new_dict_value_assigned_wholesale():
+    assert diff({}, {"a": {"x": 1}}) == {"a": {"x": 1}}
+
+
+def test_dollar_keys_escaped():
+    assert diff({}, {"$weird": 1}) == {"$$weird": 1}
+    assert diff({"$weird": 1}, {"$weird": 2}) == {"$$weird": 2}
+    assert diff({"$weird": 1}, {}) == {"$delete": ["$$weird"]}
+
+
+def test_dollar_values_not_escaped():
+    # Only keys are escaped; values are stored verbatim, even "$delete".
+    assert diff({"s": 1}, {"s": "$y"}) == {"s": "$y"}
+    assert diff({"s": 1}, {"s": "$delete"}) == {"s": "$delete"}
+    assert diff({}, {"o": {"s": "$y", "l": ["$z"]}}) == {
+        "o": {"s": "$y", "l": ["$z"]}
+    }
+
+
+# --- Consumer: apply_diff / patch -------------------------------------
+
+
+def test_patch_delegates():
+    assert patch({"a": 1}, {"a": 2}) == {"a": 2}
+
+
+def test_apply_scalar_and_add():
+    assert apply_diff({"a": 1}, {"a": 2, "b": 3}) == {"a": 2, "b": 3}
+
+
+def test_apply_delete():
+    assert apply_diff({"a": 1, "b": 2}, {"$delete": ["b"]}) == {"a": 1}
+    assert apply_diff({"a": 1}, {"$delete": ["a"]}) == {}
+
+
+def test_apply_replace():
+    assert apply_diff({"a": 1, "b": 2}, {"$replace": {"c": 3}}) == {"c": 3}
+    assert apply_diff({"x": {"a": 1}}, {"x": {"$replace": [1]}}) == {"x": [1]}
+
+
+def test_apply_nested_delete():
+    diff = {"x": {"$delete": ["y"]}}
+    assert apply_diff({"x": {"y": 2, "z": 3}}, diff) == {"x": {"z": 3}}
+
+
+def test_apply_list_insert():
+    diff = {"l": {"$insert": [[1, "x"]]}}
+    assert apply_diff({"l": ["a", "b"]}, diff) == {"l": ["a", "x", "b"]}
+
+
+def test_apply_list_delete():
+    diff = {"l": {"$delete": [1]}}
+    assert apply_diff({"l": ["a", "b", "c"]}, diff) == {"l": ["a", "c"]}
+
+
+def test_apply_list_delete_multiple_positions():
+    # jsondiff emits positions in descending order for sequential pops.
+    diff = {"l": {"$delete": [4, 2, 0]}}
+    assert apply_diff({"l": [0, 1, 2, 3, 4]}, diff) == {"l": [1, 3]}
+
+
+def test_apply_list_insert_and_delete():
+    diff = {"l": {"$insert": [[0, 9], [2, 8], [4, 7]], "$delete": [2, 0]}}
+    assert apply_diff({"l": [0, 1, 2, 3]}, diff) == {"l": [9, 1, 8, 3, 7]}
+
+
+def test_apply_list_per_index_nested_diff():
+    diff = {"l": {"1": {"y": 3}}}
+    state = {"l": [{"x": 1}, {"y": 2}]}
+    assert apply_diff(state, diff) == {"l": [{"x": 1}, {"y": 3}]}
+
+
+def test_apply_escaped_keys_and_values():
+    assert apply_diff({}, {"$$weird": 1}) == {"$weird": 1}
+    assert apply_diff({"$weird": 1}, {"$delete": ["$$weird"]}) == {}
+    # jsondiff escapes $-values as "$$.."; those are unescaped on apply.
+    assert apply_diff({"s": 1}, {"s": "$$y"}) == {"s": "$y"}
+    assert apply_diff({"s": 1}, {"s": "$$delete"}) == {"s": "$delete"}
+    # Our own producer stores values verbatim; single-$ stays as-is.
+    assert apply_diff({"s": 1}, {"s": "$y"}) == {"s": "$y"}
+    assert apply_diff({"s": 1}, {"s": "$delete"}) == {"s": "$delete"}
+    assert apply_diff({}, {"o": {"s": "$$y", "l": ["$$z"]}}) == {
+        "o": {"s": "$y", "l": ["$z"]}
+    }
+
+
+def test_apply_empty_diff():
+    assert apply_diff({"a": 1}, {}) == {"a": 1}
+
+
+def test_apply_diff_on_missing_state():
+    assert apply_diff({}, {"a": {"b": 1}}) == {"a": {"b": 1}}
+
+
+def test_apply_bare_dict_replaces_non_dict():
+    # Our own producer emits no $replace; a dict over a non-dict old value
+    # is a wholesale replacement.
+    assert apply_diff({"a": [1, 2]}, {"a": {"x": 1}}) == {"a": {"x": 1}}
+    assert apply_diff({"a": 5}, {"a": {"x": 1}}) == {"a": {"x": 1}}
+    assert apply_diff({"a": None}, {"a": {"x": 1}}) == {"a": {"x": 1}}
+
+
+def test_apply_list_patch_still_works_on_lists():
+    # jsondiff-style per-index diff keeps list-op semantics on list state.
+    assert apply_diff({"l": [1, 2]}, {"l": {"1": 9}}) == {"l": [1, 9]}
+
+
+# --- jsondiff compatibility, both directions --------------------------------
+
+COMPAT_CASES = [
+    ("scalar change", {"a": 1}, {"a": 2}),
+    ("key add", {"a": 1}, {"a": 1, "b": 2}),
+    ("key remove", {"a": 1, "b": 2}, {"a": 1}),
+    ("last key removed", {"a": 1}, {}),
+    ("nested delete", {"a": {"x": 1, "y": 2}}, {"a": {"x": 1}}),
+    ("nested mixed", {"a": {"x": 1, "y": 2}}, {"a": {"x": 9, "z": 3}}),
+    ("list append", {"l": [1, 2]}, {"l": [1, 2, 3]}),
+    ("list insert mid", {"l": [1, 2, 3]}, {"l": [1, 9, 2, 3]}),
+    ("list remove mid", {"l": [1, 2, 3]}, {"l": [1, 3]}),
+    ("list remove many", {"l": [0, 1, 2, 3, 4]}, {"l": [1, 3]}),
+    ("list replace all", {"l": [1, 2]}, {"l": [3, 4]}),
+    ("list insert+delete", {"l": [0, 1, 2, 3]}, {"l": [9, 1, 8, 3, 7]}),
+    ("dict in list", {"l": [{"x": 1}, {"y": 2}]}, {"l": [{"x": 1}, {"y": 3}]}),
+    ("list to empty", {"l": [1]}, {"l": []}),
+    ("type change dict->list", {"a": {"x": 1}}, {"a": [1]}),
+    ("type change list->dict", {"a": [1]}, {"a": {"x": 1}}),
+    ("dollar key", {"$k": 1, "b": 1}, {"$k": 2}),
+    ("dollar value", {"s": "$x"}, {"s": "$y"}),
+    (
+        "deep nesting",
+        {"a": {"b": {"c": {"d": 1, "e": 2}}}},
+        {"a": {"b": {"c": {"d": 9}}}},
+    ),
+]
+
+
+# jsondiff.patch cannot apply our patches for "type change list->dict"
+# (we emit a bare dict where jsondiff needs $replace) and "dollar value"
+# (we do not escape "$"-prefixed values; jsondiff.patch would strip the
+# "$"), so those cases are excluded from this direction.
+JSONDIFF_APPLIES_CASES = [
+    c for c in COMPAT_CASES if c[0] not in {"type change list->dict", "dollar value"}
+]
+
+
+@pytest.mark.parametrize(
+    "name,old,new", JSONDIFF_APPLIES_CASES, ids=[c[0] for c in JSONDIFF_APPLIES_CASES]
+)
+def test_jsondiff_applies_our_patches(name, old, new):
+    delta = diff(old, new)
+    assert delta is not None
+    assert jsondiff.patch(old, delta, marshal=True) == new
+
+
+@pytest.mark.parametrize("name,old,new", COMPAT_CASES, ids=[c[0] for c in COMPAT_CASES])
+def test_we_apply_jsondiff_patches(name, old, new):
+    diff = jsondiff.diff(old, new, marshal=True)
+    assert apply_diff(old, diff) == new
+
+
+@pytest.mark.parametrize("name,old,new", COMPAT_CASES, ids=[c[0] for c in COMPAT_CASES])
+def test_our_own_round_trip(name, old, new):
+    delta = diff(old, new)
+    assert delta is not None
+    assert apply_diff(old, delta) == new
+
+
+def test_no_diff_means_equal_states():
+    for _name, old, new in COMPAT_CASES:
+        assert diff(old, new) is not None  # cases really differ
+    assert diff({"a": [1, {"b": "$x"}]}, {"a": [1, {"b": "$x"}]}) is None
+
+
+# --- Logging ----------------------------------------------------------------
+
+
+def test_format_diff_list_edit_shows_whole_list():
+    diff = jsondiff.diff({"l": [1, 2, 3]}, {"l": [1, 9, 3]}, marshal=True)
+    lines = format_diff(diff, previous={"l": [1, 2, 3]})
+    text = "\n".join(lines)
+    assert "$insert" not in text
+    assert "[1, 9, 3]" in text
+
+
+def test_format_diff_unescapes_dollar_keys():
+    lines = format_diff({"$$weird": 1}, previous={})
+    assert any("$weird" in line and "$$weird" not in line for line in lines)

tests/test_filelock.py

@@ -0,0 +1,55 @@
+"""Tests for LockedFile low-level behaviors."""
+
+from kanta.exceptions import FileLockError
+from kanta.filelock import LockedFile
+
+
+def test_replace_content_rewrites_in_place(tmp_path):
+    path = tmp_path / "data.kantadb"
+    path.write_bytes(b"original content here")
+
+    f = LockedFile()
+    f.open(path)
+    try:
+        f.replace_content(b"new")
+        assert f.size() == 3
+        f.write(b"!")
+    finally:
+        f.close()
+
+    assert path.read_bytes() == b"new!"
+
+
+def test_replace_content_keeps_lock(tmp_path):
+    path = tmp_path / "data.kantadb"
+    path.write_bytes(b"abc")
+
+    f = LockedFile()
+    f.open(path)
+    try:
+        f.replace_content(b"xyz")
+        other = LockedFile()
+        try:
+            other.open(path)
+            raise AssertionError("second open should fail while lock is held")
+        except FileLockError:
+            pass
+    finally:
+        f.close()
+
+
+def test_replace_content_grow_and_shrink(tmp_path):
+    path = tmp_path / "data.kantadb"
+    path.write_bytes(b"x" * 100)
+
+    f = LockedFile()
+    f.open(path)
+    try:
+        f.replace_content(b"")
+        assert f.size() == 0
+        f.replace_content(b"y" * 200)
+        assert f.size() == 200
+    finally:
+        f.close()
+
+    assert path.read_bytes() == b"y" * 200

tests/test_grep.py

@@ -0,0 +1,610 @@
+"""Tests for structural ``--grep`` matching and match highlighting."""
+
+from datetime import UTC, datetime
+
+from kanta.__main__ import main
+from kanta.grep import (
+    GrepPattern,
+    _Entry,
+    _find_spans,
+    _match_path,
+    _match_value,
+    evaluate,
+    mark_spans,
+)
+from kanta.logging import _USER_PATH
+from kanta.serialization import JsonSerializer
+from kanta.serialization.framing import LineFramer
+from kanta.structs import ChangeRecord, Snapshot
+from kanta.tty import strip_ansi
+
+TS = datetime(2026, 1, 1, tzinfo=UTC)
+
+MARK = "\x1b[48;5;220m"
+UNMARK = "\x1b[49m"
+
+
+def test_match_path_whole_elements_anywhere():
+    assert _match_path("users", ["users"]) == frozenset({0})
+    assert _match_path("users", ["data", "users", "alice"]) == frozenset({1})
+    # Partial element matches require explicit wildcards.
+    assert _match_path("users", ["foousers"]) is None
+    assert _match_path("us*rs", ["foousers"]) is None
+    assert _match_path("*users", ["foousers"]) == frozenset({0})
+
+
+def test_match_path_contiguous_element_sequence():
+    assert _match_path("alice.email", ["users", "alice", "email"]) == frozenset({1, 2})
+    # The sequence must be contiguous.
+    assert _match_path("users.email", ["users", "alice", "email"]) is None
+    assert _match_path("users.*.email", ["users", "alice", "email"]) == frozenset(
+        {0, 1, 2}
+    )
+    # Matching is case-insensitive and empty terms match without marking.
+    assert _match_path("USERS", ["users"]) == frozenset({0})
+    assert _match_path("", ["anything"]) == frozenset()
+
+
+def test_match_value_substring_for_strings_full_for_scalars():
+    assert _match_value("lice", "alice@example.com", "alice@example.com")
+    assert not _match_value("bob", "alice@example.com", "alice@example.com")
+    # Booleans, numbers and null match only in full.
+    assert _match_value("true", True, "true")
+    assert not _match_value("tru", True, "true")
+    assert _match_value("42", 42, "42")
+    assert not _match_value("4", 42, "42")
+    assert _match_value("null", None, "null")
+    # Wildcards match the whole value text of any type.
+    assert _match_value("tru*", True, "true")
+    assert _match_value("4*", 42, "42")
+    # Containers do not match; their leaves are matched individually.
+    assert not _match_value("email", {"email": "a@b.c"}, '{"email": "a@b.c"}')
+    # An empty term matches anything.
+    assert _match_value("", True, "true")
+
+
+def test_mark_spans_merges_overlaps_and_adjacents():
+    assert mark_spans("alice@example.com", [(0, 5), (3, 11)]) == (
+        f"{MARK}alice@examp{UNMARK}le.com"
+    )
+    assert mark_spans("aab", [(0, 1), (1, 3)]) == f"{MARK}aab{UNMARK}"
+    assert mark_spans("ab", [(1, 1), (2, 2)]) == "ab"
+
+
+def test_mark_spans_offsets_into_styled_text():
+    styled = "\x1b[32mal\x1b[0mlice"
+    # A span crossing an escape sequence wraps each escape-free run.
+    assert mark_spans(styled, [(0, 3)]) == (
+        f"\x1b[32m{MARK}al{UNMARK}\x1b[0m{MARK}l{UNMARK}ice"
+    )
+    # A span covering everything wraps each escape-free run separately.
+    assert mark_spans(styled, [(0, 6)]) == (
+        f"\x1b[32m{MARK}al{UNMARK}\x1b[0m{MARK}lice{UNMARK}"
+    )
+    assert mark_spans(styled, []) == styled
+
+
+def test_find_spans_case_insensitive_occurrences():
+    assert _find_spans("Alice likes ALICE", "alice") == [(0, 5), (12, 17)]
+    assert _find_spans("nope", "alice") == []
+    assert _find_spans("anything", "") == []
+
+
+def test_pattern_parse_forms():
+    bare = GrepPattern.parse("alice")
+    assert bare.term == "alice" and bare.path_term is None
+
+    pair = GrepPattern.parse("users.alice.age=30")
+    assert pair.term is None
+    assert pair.path_term == "users.alice.age"
+    assert pair.value_term == "30"
+
+    value_only = GrepPattern.parse("=alice@example.com")
+    assert value_only.path_term == "" and value_only.value_term == "alice@example.com"
+
+    path_only = GrepPattern.parse("users.alice=")
+    assert path_only.path_term == "users.alice" and path_only.value_term == ""
+
+    # Split on the first '=' only; the value may contain '='.
+    multi = GrepPattern.parse("key=a=b")
+    assert multi.path_term == "key" and multi.value_term == "a=b"
+
+
+def _patterns(*raws: str) -> list[GrepPattern]:
+    return [GrepPattern.parse(raw) for raw in raws]
+
+
+def test_evaluate_bare_term_against_path_value_action_user():
+    record = ChangeRecord(
+        ts=TS, a="create_user", u="admin", diff={"users": {"alice": {"age": 30}}}
+    )
+    assert evaluate(record, {}, _patterns("users.alice"))
+    assert evaluate(record, {}, _patterns("30"))
+    assert evaluate(record, {}, _patterns("CREATE_user"))
+    assert evaluate(record, {}, _patterns("admin"))
+    assert evaluate(record, {}, _patterns("age=30"))
+    assert not evaluate(record, {}, _patterns("bob"))
+    assert not evaluate(record, {}, _patterns("3"))  # not a full number match
+
+
+def test_evaluate_unescapes_dollar_keys():
+    record = ChangeRecord(ts=TS, a="set", diff={"$$config": 5})
+    assert evaluate(record, {}, _patterns("$config=5"))
+
+
+def test_evaluate_path_value_form_requires_same_line():
+    record = ChangeRecord(ts=TS, a="set", diff={"a": {"x": 1}, "b": {"y": 2}})
+    previous = {"a": {"x": 0}, "b": {"y": 0}}
+    assert evaluate(record, previous, _patterns("a.x=1"))
+    # 'a' matches one line's path, '2' another line's value: no match.
+    assert not evaluate(record, previous, _patterns("a=2"))
+
+
+def test_evaluate_all_patterns_same_record_any_line():
+    record = ChangeRecord(ts=TS, a="set", diff={"a": {"x": 1}, "b": {"y": 2}})
+    previous = {"a": {"x": 0}, "b": {"y": 0}}
+    assert evaluate(record, previous, _patterns("a.x", "=2"))
+    assert not evaluate(record, previous, _patterns("a.x", "=2", "missing"))
+
+
+def test_evaluate_matches_deleted_content_by_previous_value():
+    record = ChangeRecord(ts=TS, a="delete_user", diff={"users": {"$delete": "alice"}})
+    previous = {"users": {"alice": {"email": "alice@example.com"}}}
+    assert evaluate(record, previous, _patterns("alice@example.com"))
+    assert evaluate(record, previous, _patterns("users.alice.email"))
+    assert not evaluate(record, {}, _patterns("alice@example.com"))
+
+
+def test_highlighter_marks_exactly_the_matched_regions():
+    record = ChangeRecord(
+        ts=TS,
+        a="set",
+        diff={
+            "users": {"alice": {"email": "same@x.com"}, "bob": {"email": "same@x.com"}}
+        },
+    )
+    previous = {
+        "users": {"alice": {"email": "old@x.com"}, "bob": {"email": "same@x.com"}}
+    }
+    hl = evaluate(record, previous, _patterns("users.alice.email"))
+    assert hl is not None
+    # Path-side match: the matched elements are lit, the value is not.
+    assert hl.path("alice", "users.alice") == f"{MARK}alice{UNMARK}"
+    assert hl.value("same@x.com", "users.alice.email") == "same@x.com"
+    # Bob's identical value is a different path: untouched.
+    assert hl.path("bob", "users.bob") == "bob"
+    assert hl.value("same@x.com", "users.bob.email") == "same@x.com"
+
+    hl = evaluate(record, previous, _patterns("=same@x.com"))
+    assert hl is not None
+    # Both lines genuinely match the value: both are marked.
+    assert hl.value("same@x.com", "users.alice.email") == f"{MARK}same@x.com{UNMARK}"
+    assert hl.value("same@x.com", "users.bob.email") == f"{MARK}same@x.com{UNMARK}"
+    assert hl.path("alice", "users.alice") == "alice"
+
+
+def test_highlighter_merges_overlapping_needles_from_different_patterns():
+    record = ChangeRecord(ts=TS, a="set", diff={"email": "alice@example.com"})
+    hl = evaluate(record, {}, _patterns("alice", "lice@exam"))
+    assert hl is not None
+    assert hl.value("alice@example.com", "email") == (
+        f"{MARK}alice@exam{UNMARK}ple.com"
+    )
+
+
+def test_highlighter_marks_delete_marker_on_removed_content_match():
+    record = ChangeRecord(ts=TS, a="delete_user", diff={"users": {"$delete": "bob"}})
+    previous = {"users": {"bob": {"email": "bob@example.com"}}}
+    hl = evaluate(record, previous, _patterns("bob@example.com"))
+    assert hl is not None
+    # The matched content is gone: the deletion marker is lit, not the path.
+    assert hl.path("users", "users") == "users"
+    assert hl.path("bob", "users.bob") == "bob"
+    assert hl.delete("✗", "users.bob") == f"{MARK}✗{UNMARK}"
+    assert hl.delete("✗", "users.alice") == "✗"
+
+    # A path-side match lights the genuinely matched elements of the anchor,
+    # and the marker for the element below it.
+    hl = evaluate(record, previous, _patterns("users.bob.email"))
+    assert hl is not None
+    assert hl.path("users", "users") == f"{MARK}users{UNMARK}"
+    assert hl.path("bob", "users.bob") == f"{MARK}bob{UNMARK}"
+    assert hl.delete("✗", "users.bob") == f"{MARK}✗{UNMARK}"
+
+    # A path-side match of one displayed element lights only that element.
+    hl = evaluate(record, previous, _patterns("bob"))
+    assert hl is not None
+    assert hl.path("users", "users") == "users"
+    assert hl.path("bob", "users.bob") == f"{MARK}bob{UNMARK}"
+
+
+def test_highlighter_meta_marks_action_and_user():
+    record = ChangeRecord(ts=TS, a="create_user", u="admin", diff={"x": 1})
+    hl = evaluate(record, {}, _patterns("create", "ADM"))
+    assert hl is not None
+    assert hl.meta("create_user", "action") == f"{MARK}create{UNMARK}_user"
+    assert hl.meta("admin", "user") == f"{MARK}adm{UNMARK}in"
+    assert hl.meta("extra", "other") == "extra"
+
+
+def test_entry_dataclass_holds_anchor_for_deletes():
+    entry = _Entry(["a", "b"], 1, "1", anchor=["a"])
+    assert entry.anchor == ["a"]
+
+
+def _write_db(path, changes, state=None):
+    serializer = JsonSerializer()
+    framer = LineFramer()
+    snapshot = Snapshot(ts=TS, v=1, state=state or {})
+    data = framer.frame_snapshot(serializer.encode(snapshot), record_offset=0)
+    for change in changes:
+        data += framer.frame_change(serializer.encode(change), record_offset=0)
+    path.write_bytes(data)
+
+
+def _sample_changes():
+    return [
+        ChangeRecord(
+            ts=TS,
+            a="create_alice",
+            u="admin",
+            diff={
+                "users": {
+                    "alice": {"email": "alice@example.com", "admin": True},
+                }
+            },
+        ),
+        ChangeRecord(
+            ts=TS,
+            a="update_alice",
+            u="admin",
+            diff={"users": {"alice": {"age": 30}}},
+        ),
+        ChangeRecord(
+            ts=TS,
+            a="create_bob",
+            u="bob",
+            diff={"users": {"bob": {"email": "bob@example.com"}}},
+        ),
+    ]
+
+
+def _run_cli(tmp_path, capsys, monkeypatch, changes, *args, color=False):
+    if color:
+        monkeypatch.setenv("FORCE_COLOR", "1")
+        monkeypatch.delenv("NO_COLOR", raising=False)
+    else:
+        monkeypatch.setenv("NO_COLOR", "1")
+        monkeypatch.delenv("FORCE_COLOR", raising=False)
+    path = tmp_path / "test.kantadb"
+    _write_db(path, changes)
+    code = main([str(path), *args])
+    assert code == 0
+    return capsys.readouterr().err
+
+
+def test_cli_grep_filters_records(tmp_path, capsys, monkeypatch):
+    """Only matching change records are printed."""
+    err = _run_cli(tmp_path, capsys, monkeypatch, _sample_changes(), "--grep", "alice")
+    assert "create_alice" in err
+    assert "update_alice" in err
+    assert "create_bob" not in err
+    assert "bob@example.com" not in err
+
+
+def test_cli_grep_prints_entire_transaction(tmp_path, capsys, monkeypatch):
+    """A match prints the whole record, not just the matching line."""
+    err = _run_cli(
+        tmp_path, capsys, monkeypatch, _sample_changes(), "--grep", "users.alice.email"
+    )
+    assert "create_alice" in err
+    # Non-matching lines of the same record are printed too.
+    assert "admin" in err and "true" in err
+    # The update record does not contain the path.
+    assert "update_alice" not in err
+
+
+def test_cli_grep_repeated_patterns_must_all_match(tmp_path, capsys, monkeypatch):
+    """Repeated --grep options are ANDed within the same record."""
+    err = _run_cli(
+        tmp_path,
+        capsys,
+        monkeypatch,
+        _sample_changes(),
+        "--grep",
+        "alice",
+        "--grep",
+        "=alice@example.com",
+    )
+    assert "create_alice" in err
+    # update_alice matches 'alice' but has no email value.
+    assert "update_alice" not in err
+
+    err = _run_cli(
+        tmp_path,
+        capsys,
+        monkeypatch,
+        _sample_changes(),
+        "--grep",
+        "alice",
+        "--grep",
+        "bob",
+    )
+    assert "create_alice" not in err
+    assert "create_bob" not in err
+
+
+def test_cli_grep_no_match_exits_zero(tmp_path, capsys, monkeypatch):
+    """No matching records is not an error; snapshot lines still print."""
+    err = _run_cli(tmp_path, capsys, monkeypatch, _sample_changes(), "--grep", "nobody")
+    assert "snapshot s0" in err
+    assert "create_alice" not in err
+    assert "create_bob" not in err
+
+
+def test_cli_grep_path_value_forms(tmp_path, capsys, monkeypatch):
+    """The 'path=value', 'path=' and '=value' forms restrict the match side."""
+    err = _run_cli(
+        tmp_path, capsys, monkeypatch, _sample_changes(), "--grep", "users.*.admin=true"
+    )
+    assert "create_alice" in err
+    assert "create_bob" not in err
+
+    err = _run_cli(
+        tmp_path, capsys, monkeypatch, _sample_changes(), "--grep", "users.alice.age="
+    )
+    assert "update_alice" in err
+    assert "create_alice" not in err
+
+    err = _run_cli(
+        tmp_path, capsys, monkeypatch, _sample_changes(), "--grep", "=bob@example.com"
+    )
+    assert "create_bob" in err
+    assert "create_alice" not in err
+
+
+def test_cli_grep_path_elements_match_in_full(tmp_path, capsys, monkeypatch):
+    """'users' does not match a 'foousers' element; 'us*' does."""
+    changes = [ChangeRecord(ts=TS, a="trap", diff={"foousers": {"note": "x"}})]
+    err = _run_cli(tmp_path, capsys, monkeypatch, changes, "--grep", "users")
+    assert "trap" not in err
+
+    err = _run_cli(tmp_path, capsys, monkeypatch, changes, "--grep", "*users")
+    assert "trap" in err
+
+
+def _shared_email_changes():
+    return [
+        ChangeRecord(
+            ts=TS,
+            a="create_alice",
+            u="admin",
+            diff={"users": {"alice": {"email": "shared@example.com", "admin": True}}},
+        ),
+        ChangeRecord(
+            ts=TS,
+            a="create_bob",
+            u="admin",
+            diff={"users": {"bob": {"email": "shared@example.com"}}},
+        ),
+        ChangeRecord(
+            ts=TS,
+            a="delete_bob",
+            u="admin",
+            diff={"users": {"$delete": "bob"}},
+        ),
+    ]
+
+
+def _lines_with(err: str, text: str) -> list[str]:
+    return [line for line in err.splitlines() if text in strip_ansi(line)]
+
+
+def test_cli_highlight_marks_exactly_the_matched_regions(tmp_path, capsys, monkeypatch):
+    """A path-side match lights only the matched elements, not the value."""
+    err = _run_cli(
+        tmp_path,
+        capsys,
+        monkeypatch,
+        _shared_email_changes(),
+        "--grep",
+        "users.alice.email",
+        color=True,
+    )
+    (alice_line,) = _lines_with(err, "shared@example.com")
+    assert f"{MARK}alice{UNMARK}" in alice_line
+    assert f"{MARK}email{UNMARK}" in alice_line
+    # The value itself did not match, so it is not highlighted.
+    assert f"{MARK}shared@example.com{UNMARK}" not in alice_line
+    (header_line,) = _lines_with(err, "users =")
+    assert f"{MARK}users{UNMARK}" in header_line
+    assert "create_bob" not in err
+
+
+def test_cli_highlight_value_matches_on_all_matching_lines(
+    tmp_path, capsys, monkeypatch
+):
+    """A value-side match lights the value on every line that genuinely matched."""
+    err = _run_cli(
+        tmp_path,
+        capsys,
+        monkeypatch,
+        _shared_email_changes(),
+        "--grep",
+        "=shared@example.com",
+        color=True,
+    )
+    lines = _lines_with(err, "shared@example.com")
+    assert len(lines) == 2  # alice's and bob's records both matched
+    for line in lines:
+        assert f"{MARK}shared@example.com{UNMARK}" in line
+    # The deletion of bob matched by its previous (removed) value: the
+    # deletion marker is lit, not the deleted path.
+    (delete_line,) = _lines_with(err, "✗")
+    assert f"{MARK}✗{UNMARK}" in delete_line
+    assert f"{MARK}bob{UNMARK}" not in delete_line
+    assert f"{MARK}users{UNMARK}" not in delete_line
+
+
+def test_cli_highlight_merges_overlapping_matches(tmp_path, capsys, monkeypatch):
+    """Overlapping matches from different patterns form one continuous mark."""
+    err = _run_cli(
+        tmp_path,
+        capsys,
+        monkeypatch,
+        _shared_email_changes(),
+        "--grep",
+        "shared",
+        "--grep",
+        "red@exam",
+        color=True,
+    )
+    lines = _lines_with(err, "shared@example.com")
+    assert len(lines) == 2  # both records genuinely match both patterns
+    for line in lines:
+        assert f"{MARK}shared@exam{UNMARK}ple.com" in line
+
+
+def test_cli_highlight_full_scalar_and_header_fields(tmp_path, capsys, monkeypatch):
+    """Scalars are marked in full; matched action/user substrings are marked."""
+    err = _run_cli(
+        tmp_path,
+        capsys,
+        monkeypatch,
+        _shared_email_changes(),
+        "--grep",
+        "admin=true",
+        "--grep",
+        "create_al",
+        color=True,
+    )
+    (line,) = _lines_with(err, "shared@example.com")
+    assert f"{MARK}true{UNMARK}" in line
+    (header,) = _lines_with(err, "create_alice")
+    assert f"{MARK}create_al{UNMARK}ice" in header
+    # Only the record matching both patterns is printed.
+    assert "create_bob" not in err
+    assert "delete_bob" not in err
+
+
+def test_evaluate_matches_prettified_and_raw_value_forms():
+    """With logfmt, both the raw and the prettified form of a value match."""
+    record = ChangeRecord(ts=TS, a="set", diff={"when": 1767225600})
+
+    def logfmt(value, path):
+        return "2026-01-01" if value == 1767225600 else None
+
+    # The prettified form matches; the needle is located in the display text.
+    hl = evaluate(record, {}, _patterns("2026"), logfmt=logfmt)
+    assert hl is not None
+    assert hl.value("2026-01-01", "when") == f"{MARK}2026{UNMARK}-01-01"
+    # The raw form matches too; its needle is absent from the displayed
+    # text, so the whole displayed value is marked.
+    hl = evaluate(record, {}, _patterns("1767225600"), logfmt=logfmt)
+    assert hl is not None
+    assert hl.value("2026-01-01", "when") == f"{MARK}2026-01-01{UNMARK}"
+    # Without logfmt the raw value is matched and marked precisely.
+    hl = evaluate(record, {}, _patterns("1767225600"))
+    assert hl.value("1767225600", "when") == f"{MARK}1767225600{UNMARK}"
+    # Neither form matches.
+    assert evaluate(record, {}, _patterns("1999"), logfmt=logfmt) is None
+
+
+def test_evaluate_matches_prettified_user():
+    """The user field matches both the raw id and the logfmt-resolved name."""
+
+    def logfmt(value, path):
+        return "Alice Admin" if path == _USER_PATH else None
+
+    record = ChangeRecord(ts=TS, a="set", u="u123", diff={"x": 1})
+    hl = evaluate(record, {}, _patterns("alice"), logfmt=logfmt)
+    assert hl is not None
+    assert hl.meta("Alice Admin", "user") == f"{MARK}Alice{UNMARK} Admin"
+    # A raw-only user match marks the whole displayed name.
+    hl = evaluate(record, {}, _patterns("u123"), logfmt=logfmt)
+    assert hl is not None
+    assert hl.meta("Alice Admin", "user") == f"{MARK}Alice Admin{UNMARK}"
+    # The action is never prettified.
+    assert hl.meta("set", "action") == "set"
+
+
+def test_evaluate_without_logfmt_matches_raw_only():
+    record = ChangeRecord(ts=TS, a="set", u="u123", diff={"when": 1767225600})
+    assert evaluate(record, {}, _patterns("1767225600"))
+    assert evaluate(record, {}, _patterns("2026")) is None
+    assert evaluate(record, {}, _patterns("alice")) is None
+
+
+def test_cli_grep_matches_prettified_forms_with_kanta_object(
+    tmp_path, capsys, monkeypatch
+):
+    """End to end: a -k object's logfmt formatter doubles the match surface."""
+    db_path = tmp_path / "test.kantadb"
+    module = tmp_path / "dbmod.py"
+    module.write_text(
+        "from typing import Any\n"
+        "from kanta import Kanta\n"
+        f"kanta = Kanta({str(db_path)!r}, {{}}, type=dict)\n"
+        "@kanta.logfmt\n"
+        "def pretty(value: Any, path: str) -> str | None:\n"
+        "    if path == 'when':\n"
+        "        return 'Nov 3, 2025'\n"
+        "    if path == '$user':\n"
+        "        return 'Alice Admin'\n"
+        "    return None\n"
+    )
+    changes = [ChangeRecord(ts=TS, a="set", u="u123", diff={"when": 1767225600})]
+    kanta_args = ("-k", str(module))
+
+    # A term matching only the prettified value finds the record.
+    err = _run_cli(
+        tmp_path, capsys, monkeypatch, changes, *kanta_args, "--grep", "nov", color=True
+    )
+    (line,) = _lines_with(err, "Nov 3, 2025")
+    assert f"{MARK}Nov{UNMARK} 3, 2025" in line
+
+    # A term matching only the raw value marks the whole prettified display.
+    err = _run_cli(
+        tmp_path,
+        capsys,
+        monkeypatch,
+        changes,
+        *kanta_args,
+        "--grep",
+        "1767225600",
+        color=True,
+    )
+    (line,) = _lines_with(err, "Nov 3, 2025")
+    assert f"{MARK}Nov 3, 2025{UNMARK}" in line
+
+    # The prettified user matches, and the raw id marks the whole display.
+    err = _run_cli(
+        tmp_path,
+        capsys,
+        monkeypatch,
+        changes,
+        *kanta_args,
+        "--grep",
+        "alice",
+        color=True,
+    )
+    (header,) = _lines_with(err, "Alice Admin")
+    assert f"{MARK}Alice{UNMARK} Admin" in header
+    err = _run_cli(
+        tmp_path,
+        capsys,
+        monkeypatch,
+        changes,
+        *kanta_args,
+        "--grep",
+        "u123",
+        color=True,
+    )
+    (header,) = _lines_with(err, "Alice Admin")
+    assert f"{MARK}Alice Admin{UNMARK}" in header
+
+    # Without -k there is no prettified form to match.
+    err = _run_cli(tmp_path, capsys, monkeypatch, changes, "--grep", "nov")
+    assert not _lines_with(err, "u123")
+    err = _run_cli(tmp_path, capsys, monkeypatch, changes, "--grep", "1767225600")
+    assert _lines_with(err, "u123")

tests/test_kanta_integration.py

@@ -47,7 +47,7 @@ async def test_new_file_writes_bootstrap_record_without_handlers(
     records = read_changes(path, format_config)
     assert len(records) == 1
     assert records[0].a == "bootstrap"
-    assert records[0].diff == {"$replace": {"users": {}, "counter": 0}}
+    assert records[0].diff == {"users": {}, "counter": 0}
 
 
 @pytest.mark.asyncio
@@ -63,7 +63,8 @@ async def test_new_file_persists_initial_state_for_roundtrip(tmp_path, format_co
     assert len(records) == 1
     assert records[0].a == "bootstrap"
     assert records[0].diff == {
-        "$replace": {"users": {"alice": {"name": "Alice", "age": 0}}, "counter": 5}
+        "users": {"alice": {"name": "Alice", "age": 0}},
+        "counter": 5,
     }
 
     kanta2 = make_kanta(path, Data, format_config)
@@ -562,13 +563,12 @@ async def test_migration_with_changes_records_diff_and_snapshot(
 
     records = read_changes(path, format_config)
     migration_records = [r for r in records if r.a.startswith("migrate")]
-    assert len(migration_records) == 2
+    # The version migration and the msgspec normalization that follows it are
+    # grouped into a single migrate:vN record.
+    assert len(migration_records) == 1
     assert migration_records[0].a == "migrate:v1"
     assert migration_records[0].v == 1
-    assert migration_records[0].diff == {"counter": 2}
-    assert migration_records[1].a == "migrate:msgspec"
-    assert migration_records[1].v == 1
-    assert migration_records[1].diff == {"users": {}}
+    assert migration_records[0].diff == {"counter": 2, "users": {}}
 
     snap = read_last_snapshot(path, format_config)
     assert snap is not None
@@ -722,6 +722,41 @@ async def test_logmigr_callback_replaces_default_logging(
     assert not info_messages
 
 
+@pytest.mark.asyncio
+async def test_logmigr_callback_report(tmp_path, format_config, caplog):
+    import logging
+
+    from kanta import MigrationReport
+
+    path = tmp_path / "test.db"
+    seed_single_change(path, fixed_change("init", {"counter": 0}), format_config)
+
+    mod = type(sys)("test_migrations_report")
+
+    def migrate_v1(d):
+        """Bump counter."""
+        d["counter"] = 2
+
+    mod.__dict__["migrate_v1"] = migrate_v1
+
+    reports = []
+
+    kanta = make_kanta(path, Data, format_config, migrations=mod)
+
+    @kanta.logmigr
+    def collect(report: MigrationReport):
+        reports.append(report)
+
+    with caplog.at_level(logging.INFO, logger="kanta.migration"):
+        await kanta.open()
+        await kanta.close()
+
+    assert len(reports) == 1
+    assert reports[0].original == 0
+    assert reports[0].version == 1
+    assert [m.name for m in reports[0].applied] == ["migrate_v1"]
+
+
 @pytest.mark.asyncio
 async def test_transaction_log_false_suppresses_log(tmp_path, format_config, caplog):
     path = tmp_path / "test.db"

tests/test_logemit.py

@@ -127,6 +127,25 @@ def test_default_emit_created_and_migrated(capsys):
     assert "🛢️ x.kantadb migrated v0 -> v1: migrate_v1 (rename)" in err
 
 
+def test_default_emit_strips_ansi_without_color_support(capsys, monkeypatch):
+    """NO_COLOR output contains no ANSI codes; FORCE_COLOR keeps them."""
+    logging.getLogger("kanta").handlers.clear()
+    configure_logging()
+    monkeypatch.setenv("NO_COLOR", "1")
+    monkeypatch.delenv("FORCE_COLOR", raising=False)
+    emit_event(_change_event(diff={"counter": 1}))
+    err = capsys.readouterr().err
+    assert "\x1b[" not in err
+    assert "counter" in err
+
+    logging.getLogger("kanta").handlers.clear()
+    configure_logging()
+    monkeypatch.setenv("FORCE_COLOR", "1")
+    monkeypatch.delenv("NO_COLOR", raising=False)
+    emit_event(_change_event(diff={"counter": 1}))
+    assert "\x1b[" in capsys.readouterr().err
+
+
 @pytest.mark.asyncio
 async def test_logemit_receives_transaction_events(tmp_path, format_config):
     path = tmp_path / "test.db"
@@ -242,7 +261,11 @@ async def test_logmigr_failure_does_not_break_open(tmp_path, format_config):
 
 
 @pytest.mark.asyncio
-async def test_aborted_transaction_emits_event(tmp_path, format_config, caplog):
+async def test_aborted_transaction_emits_event(
+    tmp_path, format_config, caplog, monkeypatch
+):
+    monkeypatch.setenv("FORCE_COLOR", "1")
+    monkeypatch.delenv("NO_COLOR", raising=False)
     path = tmp_path / "test.db"
     kanta = make_kanta(path, Data, format_config)
     events = []

tests/test_logging.py

@@ -71,7 +71,9 @@ def test_log_change_no_diff(capsys):
     assert "test" in captured.err
 
 
-def test_log_change_appends_extra_string(capsys):
+def test_log_change_appends_extra_string(capsys, monkeypatch):
+    monkeypatch.setenv("FORCE_COLOR", "1")
+    monkeypatch.delenv("NO_COLOR", raising=False)
     kanta_logger = logging.getLogger("kanta")
     kanta_logger.handlers.clear()
     configure_logging()

tests/test_migrations.py

@@ -183,7 +183,7 @@ def test_apply_returns_change_information():
     assert result.migrations[0].name == "migrate_v1"
     assert result.migrations[0].description == "Set x"
     assert result.migrations[0].changed is True
-    assert result.migrations[0].diff == {"$replace": {"x": 1}}
+    assert result.migrations[0].diff == {"x": 1}
 
     assert result.migrations[1].name == "migrate_v2"
     assert result.migrations[1].description == "No-op"
@@ -206,3 +206,22 @@ def test_description_defaults_to_version_when_no_docstring():
 
     result = reg.apply({}, current_version=0, kanta=kanta)
     assert result.migrations[0].description == "v1"
+
+
+def test_report_fields():
+    from kanta import MigrationReport
+
+    reg = Migrations()
+    kanta = _DummyKanta()
+
+    @reg.register
+    def migrate_v1(d):
+        d["x"] = 1
+
+    report = reg.apply({"x": 0}, current_version=0, kanta=kanta)
+    assert isinstance(report, MigrationReport)
+    assert report.original == 0
+    assert report.version == 1
+    assert [m.name for m in report.applied] == ["migrate_v1"]
+    # Deprecated alias still works.
+    assert report.migrations is report.applied

tests/test_rotation.py

@@ -0,0 +1,214 @@
+"""Tests for retention-based database rotation (docs/rotation.md)."""
+
+from datetime import UTC, datetime, timedelta
+from pathlib import Path
+
+import pytest
+
+from kanta.structs import ChangeRecord, Snapshot
+from tests.support import Data, make_kanta, read_changes
+
+pytestmark = pytest.mark.asyncio
+
+DAY = timedelta(days=1)
+T0 = datetime(2026, 1, 1, tzinfo=UTC)
+
+
+def make_clock(cell: list[datetime]):
+    def clock() -> datetime:
+        return cell[0]
+
+    return clock
+
+
+async def write_history(path: Path, format_config, days: list[int]) -> None:
+    """Write one change per day offset (relative to T0) with a fake clock."""
+    cell = [T0 + (days[0] - 1) * DAY]  # bootstrap predates all history
+    kanta = make_kanta(path, Data, format_config)
+    kanta.clock(make_clock(cell))
+    await kanta.open(log=False)
+    for day in days:
+        cell[0] = T0 + day * DAY
+        with kanta.transaction(f"day{day}", log=False) as data:
+            data.counter += 1
+        await kanta.flush()
+    await kanta.close()
+
+
+def read_all(path: Path, format_config):
+    """All records (changes and snapshots) in file order."""
+    _, serializer_cls = format_config
+    serializer = serializer_cls()
+    framer = serializer.framer_cls()
+    out = []
+    for is_snapshot, payload, _, _ in framer.iter_records(path.read_bytes(), 0):
+        out.append(
+            serializer.decode(payload, type=Snapshot if is_snapshot else ChangeRecord)
+        )
+    return out
+
+
+def rotated_files(path: Path) -> list[Path]:
+    return sorted(path.parent.glob(f"{path.stem}@*.kantadb"))
+
+
+async def test_rotation_splits_history(tmp_path, format_config):
+    path = tmp_path / "data.kantadb"
+    await write_history(path, format_config, days=[-40, -20, -5])
+
+    cell = [T0]
+    kanta = make_kanta(path, Data, format_config, retention=30 * DAY)
+    kanta.clock(make_clock(cell))
+    await kanta.open(log=False)
+    assert kanta.data.counter == 3
+    await kanta.close()
+
+    rotated = rotated_files(path)
+    assert len(rotated) == 1
+
+    # Main file: leading snapshot (ts = last dropped record), the retained
+    # changes, and no final snapshot (too few retained changes).
+    records = read_all(path, format_config)
+    assert isinstance(records[0], Snapshot)
+    assert records[0].ts == T0 - 40 * DAY
+    assert records[0].state["counter"] == 1
+    changes = [r for r in records if isinstance(r, ChangeRecord)]
+    assert [c.a for c in changes] == ["day-20", "day-5"]
+
+    # Rotated file holds exactly the dropped history, ending at the last
+    # dropped record whose ts matches the filename.
+    stamp = (T0 - 40 * DAY).strftime("%Y%m%dT%H%M%SZ")
+    assert rotated[0].name == f"data@{stamp}.kantadb"
+    rrecords = read_all(rotated[0], format_config)
+    assert [r.a for r in rrecords] == ["bootstrap", "day-40"]
+
+
+async def test_rotation_reopens_cleanly_and_does_not_rerotate(tmp_path, format_config):
+    path = tmp_path / "data.kantadb"
+    await write_history(path, format_config, days=[-40, -5])
+
+    cell = [T0]
+    for expected_changes in (["day-5"], ["day-5"]):
+        kanta = make_kanta(path, Data, format_config, retention=30 * DAY)
+        kanta.clock(make_clock(cell))
+        async with kanta:
+            assert kanta.data.counter == 2
+        assert [c.a for c in read_changes(path, format_config)] == expected_changes
+
+    # Second open found a file whose history already fits the window.
+    assert len(rotated_files(path)) == 1
+
+
+async def test_rotation_noop_when_retention_covers_all(tmp_path, format_config):
+    path = tmp_path / "data.kantadb"
+    await write_history(path, format_config, days=[-5])
+    before = path.read_bytes()
+
+    cell = [T0]
+    kanta = make_kanta(path, Data, format_config, retention=30 * DAY)
+    kanta.clock(make_clock(cell))
+    async with kanta:
+        assert kanta.data.counter == 1
+
+    assert rotated_files(path) == []
+    assert path.read_bytes() == before
+
+
+async def test_rotation_aged_out_database_reduces_to_single_snapshot(
+    tmp_path, format_config
+):
+    path = tmp_path / "data.kantadb"
+    await write_history(path, format_config, days=[-40, -35])
+
+    cell = [T0]
+    kanta = make_kanta(path, Data, format_config, retention=30 * DAY)
+    kanta.clock(make_clock(cell))
+    async with kanta:
+        assert kanta.data.counter == 2
+
+    records = read_all(path, format_config)
+    assert len(records) == 1
+    assert isinstance(records[0], Snapshot)
+    assert records[0].state["counter"] == 2
+
+    # Opening again must not rotate the snapshot-only file.
+    before = path.read_bytes()
+    kanta = make_kanta(path, Data, format_config, retention=30 * DAY)
+    kanta.clock(make_clock(cell))
+    async with kanta:
+        assert kanta.data.counter == 2
+    assert path.read_bytes() == before
+    assert len(rotated_files(path)) == 1
+
+
+async def test_rotation_validates_against_internal_snapshots(tmp_path, format_config):
+    path = tmp_path / "data.kantadb"
+    cell = [T0 - 40 * DAY]
+    kanta = make_kanta(path, Data, format_config)
+    kanta.clock(make_clock(cell))
+    await kanta.open(log=False)
+    with kanta.transaction("old", log=False) as data:
+        data.counter = 1
+    await kanta.flush()
+    kanta.request_snapshot()
+    kanta._impl.maybe_snapshot()
+    cell[0] = T0 - 1 * DAY
+    with kanta.transaction("new", log=False) as data:
+        data.counter = 2
+    await kanta.flush()
+    await kanta.close()
+
+    cell[0] = T0
+    kanta = make_kanta(path, Data, format_config, retention=30 * DAY)
+    kanta.clock(make_clock(cell))
+    async with kanta:
+        assert kanta.data.counter == 2
+
+    records = read_all(path, format_config)
+    assert isinstance(records[0], Snapshot)
+    assert records[0].state["counter"] == 1
+    assert [r.a for r in records if isinstance(r, ChangeRecord)] == ["new"]
+
+
+@pytest.mark.parametrize("name", ["data", "data.db", "data.kantadb"])
+async def test_rotated_naming_normalizes_extension(tmp_path, format_config, name):
+    path = tmp_path / name
+    await write_history(path, format_config, days=[-40, -5])
+
+    cell = [T0]
+    kanta = make_kanta(path, Data, format_config, retention=30 * DAY)
+    kanta.clock(make_clock(cell))
+    async with kanta:
+        pass
+
+    stamp = (T0 - 40 * DAY).strftime("%Y%m%dT%H%M%SZ")
+    assert (tmp_path / f"data@{stamp}.kantadb").exists()
+
+
+async def test_retention_accepts_int_days(tmp_path, format_config):
+    path = tmp_path / "data.kantadb"
+    await write_history(path, format_config, days=[-40, -5])
+
+    cell = [T0]
+    kanta = make_kanta(path, Data, format_config, retention=30)
+    kanta.clock(make_clock(cell))
+    async with kanta:
+        assert kanta.data.counter == 2
+
+    assert len(rotated_files(path)) == 1
+    assert [c.a for c in read_changes(path, format_config)] == ["day-5"]
+
+
+async def test_rotation_disabled_by_default(tmp_path, format_config):
+    path = tmp_path / "data.kantadb"
+    await write_history(path, format_config, days=[-40, -5])
+    before = path.read_bytes()
+
+    cell = [T0]
+    kanta = make_kanta(path, Data, format_config)
+    kanta.clock(make_clock(cell))
+    async with kanta:
+        assert kanta.data.counter == 2
+
+    assert rotated_files(path) == []
+    assert path.read_bytes() == before

tests/test_validate.py

@@ -0,0 +1,116 @@
+"""Tests for the @kanta.validate integrity-validation callbacks."""
+
+import pytest
+
+from tests.support import Data, make_kanta
+
+pytestmark = pytest.mark.asyncio
+
+
+async def test_validate_passes_on_valid_data(tmp_path, format_config):
+    kanta = make_kanta(tmp_path / "d.kantadb", Data, format_config)
+    calls = []
+
+    @kanta.validate
+    def check(data: Data):
+        calls.append(data.counter)
+        assert data.counter >= 0
+
+    async with kanta:
+        with kanta.transaction("inc", log=False) as data:
+            data.counter = 1
+
+    assert calls  # ran during bootstrap/open and the transaction
+
+
+async def test_validate_failure_rolls_back_transaction(tmp_path, format_config):
+    kanta = make_kanta(tmp_path / "d.kantadb", Data, format_config)
+
+    @kanta.validate
+    def check(data: Data):
+        if data.counter < 0:
+            raise ValueError("counter must not go negative")
+
+    await kanta.open(log=False)
+    with pytest.raises(ValueError, match="negative"):
+        with kanta.transaction("dec", log=False) as data:
+            data.counter = -1
+    assert kanta.data.counter == 0  # rolled back
+    await kanta.close()
+
+    # The invalid change never reached the history.
+    kanta2 = make_kanta(tmp_path / "d.kantadb", Data, format_config)
+    async with kanta2:
+        assert kanta2.data.counter == 0
+
+
+async def test_validate_runs_on_open_after_replay(tmp_path, format_config):
+    kanta = make_kanta(tmp_path / "d.kantadb", Data, format_config)
+    async with kanta:
+        with kanta.transaction("set", log=False) as data:
+            data.counter = 5
+
+    kanta2 = make_kanta(tmp_path / "d.kantadb", Data, format_config)
+    seen = []
+
+    @kanta2.validate
+    def check(data: Data):
+        seen.append(data.counter)
+
+    async with kanta2:
+        pass
+    assert 5 in seen
+
+
+async def test_validate_failure_aborts_open(tmp_path, format_config):
+    kanta = make_kanta(tmp_path / "d.kantadb", Data, format_config)
+    async with kanta:
+        with kanta.transaction("set", log=False) as data:
+            data.counter = 5
+
+    kanta2 = make_kanta(tmp_path / "d.kantadb", Data, format_config)
+
+    @kanta2.validate
+    def check(data: Data):
+        raise ValueError("always inconsistent")
+
+    with pytest.raises(ValueError, match="inconsistent"):
+        await kanta2.open(log=False)
+
+    # The failed open released the file: a fresh instance can open it.
+    kanta3 = make_kanta(tmp_path / "d.kantadb", Data, format_config)
+    async with kanta3:
+        assert kanta3.data.counter == 5
+
+
+async def test_multiple_validators_stop_at_first_failure(tmp_path, format_config):
+    kanta = make_kanta(tmp_path / "d.kantadb", Data, format_config)
+    calls = []
+
+    @kanta.validate
+    def first(data: Data):
+        calls.append("first")
+        if data.counter > 1:
+            raise ValueError("too big")
+
+    @kanta.validate
+    def second(data: Data):
+        calls.append("second")
+
+    await kanta.open(log=False)
+    calls.clear()
+    with pytest.raises(ValueError, match="too big"):
+        with kanta.transaction("bump", log=False) as data:
+            data.counter = 2
+    assert calls == ["first"]
+    await kanta.close()
+
+
+async def test_validate_rejects_async_callback(tmp_path, format_config):
+    kanta = make_kanta(tmp_path / "d.kantadb", Data, format_config)
+
+    with pytest.raises(TypeError, match="must not be async"):
+
+        @kanta.validate
+        async def check(data: Data):
+            pass