Rendering every article per request is expensive. Bodies are now
RAM-cached keyed by the public base URL and cleared from
_invalidate_pages (the same hook that drops the page render cache on
any content/settings write). Responses carry a blake3 content-hash
ETag and answer 304, so polling feed readers revalidate cheaply;
matching page behavior, 304 revalidations are not recorded in
analytics.
New pagerite/feeds.py: /llms.txt (Markdown site map with excerpts for
LLM agents), /feed.json (JSON Feed 1.1) and /feed.xml (RSS 2.0 +
atom:link), all carrying every published article with full content and
absolutized URLs. Linked from every page <head> (feed alternates +
llms-txt) and from the sitemap, recorded in analytics like page GETs
and emoji-marked in the trails (🧠 llms.txt, 📡 feeds).
The robots and sitemap routes never called _record_get, so those
accesses were invisible. Recorded like page GETs, they surface as
crawler hits at display time (no activity message ever follows). In
the trail rendering, known non-article machinery paths (robots,
sitemap, future feed paths) get an emoji marker instead of a bare
slug.
The old 1.4-2.5 aspect window sent big near-square images (e.g.
1350x1200) to the compact layout despite ample resolution. The new
rule keeps square and portrait images compact (the compact box fits
them) while letting anything big and clearly wider than tall render
as a full-card cover.
Replace count-sorted, group-centered rows with a spring-like placement:
pills order by their weighted median target x and settle by clamped
coordinate descent, minimizing weighted horizontal connection distance
while keeping the minimum pill spacing.
The banner panel's card-image block gains a pasteboard button (an image
uploads; an image URL goes as the save message's image setting itself and
is fetched/stored server-side, cross-origin being CORS-blocked for the
browser) and a site-icon button saving the @favicon sentinel, which
resolves to Data.favicon at render time and follows favicon changes.
The resolved (inheriting) Node.image was checked before mining the
article, so a front-page banner image shadowed the articles' own
extracted figures everywhere: card covers, og:image and the editor's
card previews.
Render the seasonal typical estimate as a translucent muted fill (no
stroke, same 0.6 opacity as the data) instead of a grey line, on both day
and week views. Continue the fixed Monday-Monday week view with last
week's curve from the current bucket to the end of the week (secondary
accent, translucent fill), so the chart is useful early in the week and
last week is gradually replaced by the current one.
Gate the typical estimate on history spanning twice the view's full time
(third day / third week on) and the last-week tail on data reaching into
last week at all; the legend shows each entry only when present.
Replace the old "patches" format (path:lang composite keys, ordered
hunk lists, text-anchored matching) with Data.overrides:
path -> lang -> LangEdits, keyed throughout so a save's database diff
touches only the edited chunks. The old "patches" key is ignored on
decode, discarding legacy data without a migration.
- Whole-paragraph additions/deletions are structural: a drop flag on
the original chunk hash, and additions in their own dict anchored
from the neighboring chunks' before/after (first live referrer wins),
so they stay in place across retranslation and one-sided original
edits.
- Within-paragraph edits (up to a full paragraph rewrite or split) are
full-chunk replace patches applied by chunk hash alone: a
retranslation is overridden wholesale, so user edits survive AI
re-runs; editing the original changes the hash and orphans the patch.
The old search-matching staleness gate is gone.
- Saving a translation on a page without original chunks is rejected
(REST 400 / WS error); emptying the original afterwards renders the
translation empty, with the orphaned overrides inert.
- Added full article / with markdown mode for advanced LLMs, with local ollama and API support, new llm_translator script.
- Translate navigational titles as a whole (preserving site structure), and then retranslate as article headings with article context - potentially changing nav label
- Improved human overrides to translations, in particular when structure is affected
Reviewed-on: #2
Extra paragraphs exist only inside a patch hunk's replace text (anchored
on the preceding original block). Deleting one produced a (X, "") hunk,
and apply_patch removed the text but not a surrounding separator, leaving
an accumulating blank line in the served/editor Markdown. hybrid_markdown
now re-chunks the patched text (join_chunks(chunk_markdown(...))), which
drops the stray blank lines on the read path — repairing gaps left by
already-stored patches too.
Adjacent fix in make_patch: a hunk whose search text occurred more than
once in the page hit the FIRST occurrence at apply time, blanking or
rewriting the wrong instance when identical paragraph text appeared
earlier. Ambiguous hunks now grow block context (preceding block first)
until unique; unique hunks stay context-free so they keep surviving
translation refreshes.
load() closed the editor on every navigation, and the close raced the
navigation: the shell's close-time loadPlain re-render resolved last,
swapping the old page back in and replaceStating its URL. Navigation now
leaves the panel open and the existing pagerite:switch-editor retarget
fires while editing; the exception is /_a, where closeEditor({ navigating:
true }) closes the panel without the re-swap or title restore.
- Banner-corner auth link is now a button opening paskia-js's profile()
dialog (handles login too); auth re-probed when the dialog closes.
- Editor/analytics components call /_api via apiFetch/apiJson: an
expired session opens the login dialog and the request retries.
- pagerite.js: task-checkbox toggle uses apiJson (explicit edit attempt,
reverts on any failure incl. cancelled login); auth probes use
fetchJson (never a dialog); page-cache/navigation stay on plain fetch.
- Add the paskia npm dependency; document the convention.
Preview: the editor render handler now returns the complete article HTML
({cards} tags expanded, children's cards appended when the page has no
tag) and the frontend swaps article.innerHTML wholesale — the old
detach/keep/re-append hack for .cards (which duplicated stacks) is gone.
Semantics: one card per item, no subtree flattening (stacks removed).
Bare {cards} lists immediate children (front page: the other top-level
items); a page-less item is represented by its first leaf page, the
nav-link logic. {cards: path} renders that page alone, path/* its
direct children, path/** all published descendants.
The language-switch trail ping read <html lang> before the view
transition applied the swap, so it reported the previous language (and
read pings never update trail-item languages server-side): multi-
language visits recorded as single-language trails and the analytics
flag combining had nothing to show. The switch ping now passes the
picked tag explicitly.
Language negotiation no longer prefers the article's original language
anywhere in the Accept-Language list — nearly every browser lists
English as a fallback, so translations were almost never served. The
first servable header language now wins, the original counting only in
its natural position.
Area and bar fills at 0.6 opacity (were a near-invisible 0.15); the day
view's 'Last 24 hours' legend entry uses the same full-opacity accent
line as the other entries instead of a translucent bar swatch.
Port the seasonal.py demo algorithm to JS (analytics/seasonal.js): the
smoothed 5-minute history is folded onto a weekly grid with exponential
age decay (7-day half-life for the time-of-day pattern, 42 days for
per-weekday deviations, shrunk by effective weeks of data), capped at
180 days of history where the weights become negligible.
Week view: the fading past-week overlays are replaced by the full
Monday-first 'Typical week' estimate. Day view gains the same estimate
cut to the rolling 24h window as a smooth muted 'Typical <weekday>'
curve behind the bars. Longer ranges are unchanged.
Favicon flush to the badge's left edge, then the referer host and UTM
summary in one link, with a single newline-formatted tooltip (origin,
then each utm_* pair). Fixed --badge-bg/--badge-text/--badge-muted
palette (defined in pagerite.css, deliberately unthemed) so transparent
favicons stay legible in both light and dark themes; own-article trail
links remain plain, lifting referer info visually apart from them.
Replaces the outlined utm-tag pill.
GETs store the resolved page language, activity pings carry <html lang>
(including a new ping on in-place language switches, which also keeps the
switch's GET out of crawler classification), and trail items/crawler hits
surface it. The viewer shows flags discreetly: nothing on single-language
sites or primary-language-only rows, one leading flag for uniform visits,
transition flags on mixed trails, a flag row per crawler.
- UA lines click to copy the raw UA; abuse rows with rotating clients
copy every variation, one per line (uaRaws)
- "Copied!" popup is viewport-fixed at the click point; cell overflow
clipped the old absolute one
- language tags show their Intl.DisplayNames full name on hover
- a 🔗 link after the pretty UA opens the crawler info page uarite
provides
- the emoji-symbol dim-until-hover idiom (opacity 0.7 -> 1) is now one
global .icon-btn class in pagerite.css, shared by the edit pens,
auth links, editor icon buttons and the new UA link
The stored ua_pretty froze each record at the uarite version of its
record time (old records showed disguised Meta crawlers as
"Chrome/145 Windows"). Client now carries a display-time-only
"uarite" field holding the full uarite.UA dataclass (pretty/engine/
os/provider/kind/url), filled when the viewer payload is built, so old
data always follows the current uarite. uarite 0.2.0 fixes the Meta
misdetection itself; _compact_user_agent is gone, tracking.py calls
uaparse directly.
A closing ::: glued to a paragraph (no blank line before it) rode inside
the prose chunk and crossed to the translator as part of the text run;
when the model dropped it, validation passed and the splice lost the
fence — the rest of the page rendered inside the container (seen in the
Spanish translation). Container fence lines (::: openers and closers
alike) are now always their own prose-free chunk, never reaching the
translator. Affected pages re-chunk on next save and re-translate under
the new hashes, repairing themselves.
LangSelect's scoped CSS landed on the shared store chunk, whose stylesheet
the editor never loaded (only the public selector path injected it), so the
editor's language selector rendered unstyled on untranslated pages. Collect
editor stylesheets from the entry's imported chunks too (same traversal as
the langselect assets).
Also: hreflang alternates now skip languages disabled site-wide, and all
selectors (page editor, structure tab, public selector) order languages the
same way — primary first, then the lang tab's geographic grouping.
Replace the pre-classified store (visits/crawlers/abuse lists written at
collection time, plus abuse_ips and in-memory pending/session tables) with
a raw append-only log: one Get record per document GET (full path, true
HTTP status, referer origin, preload flag) and one Msg per /_ws activity
message. Visitor/crawler/abuse classification, visit grouping (30-minute
inactivity gap), status/referer/UTM attribution and all aggregates are
derived in Store.display(), so future rule changes never invalidate stored
data. The viewer payload keeps its exact shape.
Fixes structurally:
- Abuser 404s on slug-format paths showed up as "articles read": the
not-found branch recorded the request twice through separate status
plumbing. Each request is now recorded once with its true status.
- 404 trail links never rendered red: cache-served navigations issue no
GET and the only real GET (the idle preload) was discarded before status
recording. Preloads are now recorded with pre=True, never counted, and
used for status attribution.
- formatAbuseRows merged a path's 404 probes and 200 reads into one entry;
the collapse is now keyed by (path, status class).
Rule improvements enabled by the redesign:
- The plain-404 abuse threshold counts within a 1-hour sliding window, so
long-time readers accumulating misses never classify (scanners spray).
- Hidden (admin) clients never trigger abuse classification: editing means
visiting not-found pages.
- /.well-known/ probes (RFC 8615, e.g. Chrome devtools) are never abuse
evidence; //foo-style empty path segments are an instant telltale.
- Visits can no longer open on an external exit URL; favicon fetches skip
hidden clients' referers/exits.
Legacy analytics.json files are set aside as .bak-legacy on startup.
Past-week overlays can run far above the autoscaled y range, and the svg
is overflow: visible for the axis labels — wrap the data paths (bars,
skyline, area/line curves) in a clipped group so they cannot paint outside
the plot rect.
Visits with under 5 s of total reported reading time are JS-running bots:
display() converts them to crawler hits (one per trail page, with referer
and UTM query) and excludes them from every aggregate. Crawler referers
join the favicon fetch origins and render with their icon in the crawler
table. Abuse hits now record the real response status, so the abuse table
splits 404 probes from the articles the abuser actually read (200 GETs),
shown as trail links like the visitor/crawler tables.
All pending titles are now offered before any article chunks (stable
sort, menu order kept within each kind) — a page's name in the menu is
its most visible string.
Every request-driven kanta transaction now passes user= from the
Remote-User header the SSO/forward-auth proxy sets (the editor socket
reads it from the WebSocket headers; the translator worker keeps its
client key). Action labels are short identifiers naming the object, not
sentences: page / page:{lang} / page:title / page:{lang}:title /
page:language / page:slug / page:delete / structure:reorder / settings /
translate:reset for admin actions, translate:{lang}[ :title ] for worker
submissions (title results identified via the job kind, now tracked in
the connection state).
Weight-ratio mapping alone was routinely off by a word and could glue a
mark to its neighbor (losing the space between). Now each mark's source
words are aligned to the translation's words by form similarity
(sequence ratio + shared prefix, case-folded, capitalization bonus) with
cheap skip penalties, so inflection, dropped articles/prepositions and
reordering don't break the match; slices are cut exactly at word
boundaries. Alignments without an anchor pair fall back to the weight
ratio (still the CJK path).
Implement comprehensive content localization, admin panels for editing each language, AI translation interface with automatic updates when base language version is changed.
- SEO tags for all language URLs
- Uses accept-language by default, ?lang=en overrides temporarily
- User edits patched on top of translations
- RTL language supportReviewed-on: #1
word-break: keep-all does not suppress breaks at hard hyphens (an
explicit UAX #14 break opportunity), so `--arg` could split after the
dashes. Inline code is short — nowrap is the safe fix.
Space-separated brace attrs on a lone image's own line (e.g.
{style="max-width: 40em"}) are consumed onto the paragraph but leave
an empty text token in the inline children, which defeated the
lone-image check — the image stayed a bare inline <img> without the
figure wrapper (and without lightbox/zoom treatment). Strip empty text
tokens before the check.
Clicking a figure image opens a full-viewport overlay: the image as
large as fits (100vw, flex-shrunk to leave exactly the caption's
height) with the caption below. Click or any key closes it; wheel and
touch scrolling stop at the overlay (overscroll-behavior: contain).
Styling lives in the base theme: blurred dark backdrop (theme-tunable
via --lightbox-bg / --lightbox-text), soft shadow, reduced-motion-aware
open animation.
::: aside / .margin blocks no longer split the column segments: they
stay inside the .colseg at their anchor point, so the surrounding text
is measured and laid out as a single columned layout (split-off sides
previously lost .cols when too short on their own).
The zone placements (multicol side zone, sidebar track, wide gutter)
now position the boxes absolutely off the article's left border —
unaffected by any column layout inside — with the vertical spot coming
from the unset top (where the box occurs in the text). Narrow widths
keep the in-column float fallback. Trade-off: out-of-flow boxes no
longer stack via clear, so boxes anchored close together may overlap.
Inline code gets padding-inline, keep-all word-break, and a color nudged
30% toward --muted from the inherited color, so code inside
accent-colored text keeps its hue.
All color-mix calls now rely on the default interpolation space (oklab)
instead of declaring srgb/oklab explicitly.
migrate_v1 now also rebuilds the legacy flat pages store as the menu
tree (moved from the app lifespan, raw-dict level); migrate_v2 now also
backfills missing AVIF/WebP/JPEG derivatives on disk (moved from the
lifespan) and drops the obsolete version field. The version render
counter was cache-invalidation state, not database state: replaced by an
in-memory render generation that clears the page-body LRU and feeds page
ETags. The legacy Page struct and Data.pages/version fields are removed;
old databases lose the stale keys on re-serialization.
Uploaded images (SVGs rasterized, GIFs excepted) are stored as the
original (hash.orig.ext, internal only, never served) plus AVIF primary
and WebP/JPEG fallback derivatives re-encoded from it. Pages link the
bare hash; the server serves a format only when Accept lists it
explicitly (image/avif -> AVIF, image/webp -> WebP, else JPEG) with
vary: accept, while an explicit extension pins the format. Favicons go
through the same pipeline at 192px. migrate_v2 rewrites old
/_f/{hash}.avif article links, a startup backfill creates missing
derivatives, and twitter:image pins the .webp variant for X's scraper.
uploadImage and insertTable no longer inject at the cursor: on a
non-empty line (e.g. inside an existing image tag) the block goes on a
fresh blank-separated line after it, never into it.
PUT /_api/files now runs raster uploads through mediapreview.dispatch
(temp file for format routing: pyvips, ffmpeg for HEIC/HEIF/AVIF),
storing the untouched original as <hash>.orig<ext> and serving the
AVIF derivative <hash>.avif in links. SVG/GIF and failed conversions
fall back to plain <hash><ext> storage. FileStore.delete removes the
whole hash pair. Adds mediapreview[standard] dependency.
Cursor-driven editor→page scroll sync pinned the cursor's page position
at a fixed window height, so every cursor move dragged the page along.
Now the page scrolls only when the cursor's mapped position crosses a
viewport edge margin, and just enough to bring it back inside.
- Image insert always adds an empty "" caption with the cursor inside.
- Fenced blocks (``` code, ::: aside) share one toggle: clicked inside
one it is removed and the content selected; otherwise the selection
(expanded to whole lines) is wrapped, cursor left on the opener line.
- Code button: inline wrap toggles (selection preserved, backtick runs),
line-spanning selections make fenced blocks.
- Link button toggles: clicked inside [label](url) it unwraps.
- Block classes via two pickers (placement, AA size) with current-class
indication and a "normal" reset; placement replaces ::: container
names, fences get their own attribute line.
- New .small/.large/.huge text-size classes (0.7/1.5/3em); base body is
exactly 1rem so the scale is uniform.
- .left/.right floats generalized from figures to any block.
- Toolbar polish: non-emoji glyphs, scaled-up symbols, borderless
hover/active states, reordered (B/i and size last).
- Editor panel no longer closes on Escape.
- CodeMirror editors capture Tab/Shift-Tab for indent/dedent.
_block_attrs now applies a trailing {...} to the block when it ends the
last text line after whitespace (some text {.small}), not only on a line
of its own; a space is what keeps the braces off an image/link ending the
line. Glued-to-text braces stay literal.
Nitro's orange accent selection fill clashed with accent-colored text.
Introduce --selection-bg (base: accent 30% mix, as before) used by both
::selection and the editors' selection layer; nitro overrides it with a
neutral grey. CodeMirror's selection needed a baseTheme with its exact
&light/&dark selectors to win, and is now hidden when unfocused like a
normal input.
Live-filtering through slugify ate hyphens and spaces (trailing hyphens
are stripped) and jumped the cursor to the end on mid-text edits. Now
only spaces→hyphens and lowercasing happen oninput (length-preserving,
cursor stays put); the full slugify runs at commit before the server
call, which keeps its own validation.
The cube view-transition block moves out of pagerite.css into transition
designs (pagerite/themes/{name}/transition.css), handled like banner
designs: served from disk at /_themes/..., injected by the backend as
#pagerite-transition after the banner sheet, and picked site-wide in the
site settings (Data.transition, default cube; GET/PUT /_api/settings
carry transition + the designs found on disk). The site editor swaps the
sheet in place on change; the dev-mode order fixup knows the new id.
Designs: cube (unchanged), crossfade (all navigations), slide (sideways,
both pages moving together) and reveal (clip-path wipe over the
stationary old page) — the latter two mirrored on history-back and
crossfading within a section like cube.
Also compress the site settings form into a two-column grid: site name +
favicon on one row, theme + transition on the next.
popstate always passed back=true, so going forward after back still
rotated the cube backwards. Track an incrementing idx in history.state
and only mirror the transition when the target entry is actually behind;
replaceState calls now preserve the state object instead of wiping it.
LeoVasanko