Compare commits

...
10 Commits
27 changed files with 714 additions and 183 deletions
+3 -1
View File
@@ -29,6 +29,8 @@ Pagerite is a CMS. See `docs` for the full design and implementation details. Ke
- `frontend/src/` — Vue editor and public-page JS entries.
- `main.js` — Vue editor app entry.
- `analytics-main.js` — analytics page entry (mounts `AnalyticsView` at `/_a`).
- `langselect-main.js` + `LangSelector.vue` — public language selector, imported on demand by pagerite.js on pages with more than one hreflang alternate (the editors' `LangSelect` flag dropdown).
- `store.js` — the shared Pinia store (`useStore`, id `pagerite`) for cross-bundle UI state.
- `pagerite.js` — public page entry.
- `editorLang.js` + `LangSelect.vue` — the editor shell's shared language selection and its selector component (page + structure tabs; drives the page preview while the panel is open, via `swapdoc.setLangOverride`).
- `reconnect.js` — shared WebSocket pacing for all sockets (staggered connect slots, stuck-CONNECTING watchdog, exponential backoff): bursts and rapid retries trip the browser's WebSocket throttling.
@@ -64,6 +66,6 @@ Server run by CLI entry point `uv run pagerite` (no auto reloads, build needed).
## Conventions
- Keep dependencies minimal; add via `uv add` and mention it.
- The public URL space belongs to content (pretty slugs at root). Reserve only `/_` for the machinery (`/_api/`, `/_f/`, `/_assets/`), plus `/favicon.ico` from the build. Slugs are lowercase ASCII letters, digits, hyphens and underscores `[a-z0-9_-]` (the site editor filters input live via `slugify.js`, built on the `transliteration` npm package — unicode folds to ASCII, spaces become hyphens; an empty slug on a new page is derived from its title), may not begin with `_` or `.`, and such URLs are never looked up as content.
- The public URL space belongs to content (pretty slugs at root). Reserve only `/_` for the machinery (`/_api/`, `/_f/`, `/_assets/`), plus `/favicon.ico` (backend redirect to the configured site icon). Slugs are lowercase ASCII letters, digits, hyphens and underscores `[a-z0-9_-]` (the site editor filters input live via `slugify.js`, built on the `transliteration` npm package — unicode folds to ASCII, spaces become hyphens; an empty slug on a new page is derived from its title), may not begin with `_` or `.`, and such URLs are never looked up as content.
- No auth in core code; the SSO/reverse proxy gates all of `/_api` (forward-auth) and owns `/auth/` (login/logout, session validation). Pages render identically for everyone; pagerite.js adds the editing UI only after the auth server validates the session. The one keyed exception is `/_translate/{key}` (translator service; `Data.translate_keys`, see docs/localization.md).
- Update the relevant MarkDown files when architecture, tooling, or conventions change.
+2 -2
View File
@@ -73,10 +73,10 @@ Each `Client` record (shared by every event, keyed by hash):
A reverse-DNS lookup is attempted for each new client and the result, when
available, is stored as `host`; local/reserved/multicast addresses are
skipped. If a DB-IP MMDB file (`dbip-*.mmdb` or `dbip-*.mmdb.gz`) is present
in the repository root, it is loaded at startup and used to look up
in the working directory, it is loaded at startup and used to look up
`country`/`city`. These lookups run in background tasks after the event is
stored, so WebSocket message handling is never delayed. The decompressed
`dbip-*.mmdb` file is kept in the repository root and ignored by git. The
`dbip-*.mmdb` file is kept in the working directory and ignored by git. The
CLI flag `--dbip` (`uv run pagerite --dbip`) downloads the latest
`dbip-city-lite-YYYY-MM.mmdb.gz` from DB-IP at startup (in the app lifespan,
before the MMDB is opened), skipping the download when the local database is
+40 -11
View File
@@ -53,11 +53,13 @@ Region tags normalize to their base subtag (`fi-FI` → `fi`).
article's own language), `?lang=xx` when serving a translation — however
the language was arrived at (query or header).
- `<link rel="alternate" hreflang="…">` entries follow the canonical
directly (before the social meta tags) and are the same set on every
page — the site-wide configured languages (`translate_langs`, which the
translator works to fill in): `x-default` first, pointing at the plain
autodetecting URL, then every language explicitly with `?lang=`, the
page's own primary language included.
directly (before the social meta tags) and list the languages the page
is **actually available in**: `x-default` first, pointing at the plain
autodetecting URL, then every available language — the original again by
its plain URL, translations by `?lang=`. The public language selector
keys off these: pagerite.js mounts the editors' flag dropdown in the
top-right corner when the head advertises x-default plus more than one
language, loading its bundle (Vue + the flag SVG set) on demand.
- The override sticks for the session of clicks: a page requested with
`?lang=` replicates the query onto the navigation links it renders (nav,
sidebar, cards, brand — in-article links are content and stay as
@@ -66,6 +68,11 @@ Region tags normalize to their base subtag (`fi-FI` → `fi`).
`history.replaceState` (pretty, shareable URLs), remembers the language,
and adds it to every internal fetch that lacks one (preloads,
fetch-navigations, history traversals); history entries stay query-less.
- The public selector's pick is the same override, pure JS state
(`pagerite:set-session-lang`): the session language changes and the page
swaps in place — no `?lang=` in the address bar, no reload. The choice is
linked with the editor panel's language dropdown both ways; closing the
panel keeps the chosen language instead of reverting.
- A full page refresh or a shared link resets to automatic selection (header
only). This gives a clean one-time override without cookies.
@@ -88,6 +95,10 @@ Region tags normalize to their base subtag (`fi-FI` → `fi`).
### Rendering
- The translated Markdown goes through the same `markdown.render` pipeline.
- Section anchors (`#hash` ids on h1/h2 headings) stay in the original
language: render(anchors_from=...) pins the translated render's heading
ids to the original text's slugs, matched by heading position, so links
to sections don't break across languages.
- Navigation/sidebar titles come from the translation's title map, with
per-node fallback to the original title (a partially translated tree must
still render).
@@ -95,7 +106,9 @@ Region tags normalize to their base subtag (`fi-FI` → `fi`).
language like content pages, but over the **subtree's** combined
availability (`subtree_languages`) — they have no chunks of their own;
the heading, navigation and card text localize from the title map and
the target articles' translations.
the target articles' translations. Their hreflang alternates are
computed exactly like a content page's (a translated title counts as
availability, so the language selector is offered there too).
- Card descriptions and cover picks run on the target article's hybrid
Markdown where that page is available in the served language, with
per-card fallback to the original.
@@ -398,9 +411,14 @@ verbatim source substring — entity-decoded text, backslash escapes — is
skipped and stays in the original language), and the returned translations
are swapped in by offset. Markup corruption is therefore impossible by
construction; the failure modes that remain are a wrong segment count, an
empty segment, or markup injected INTO a segment (a `<br>` in a title
translation would splice live HTML) — each returned segment must parse as
pure prose, or the whole result is dropped and logged, and the (lang, key)
empty segment, markup injected INTO a segment (a `<br>` in a title
translation would splice live HTML), or a line that would start a new
block where the segment lands (a ``` or ::: fence line would eat the rest
of the block it splices into, closing fence included — segments are
inline prose, so `pure_prose` alone cannot see this) — each returned
segment must parse as
pure prose with no block-starting line or blank line, or the whole result
is dropped and logged, and the (lang, key)
pair is skipped for the rest of the server run (generation is
near-deterministic, so an immediate retry would re-fail; the fragment stays
pending and gets another chance on restart or `DELETE /_api/translations`).
@@ -451,14 +469,25 @@ stripped before the result goes back.
The same client-side enforcement covers markup bleed as a CLASS, not per
artifact: `<` is the prose/markup boundary on the wire and never appears in
a segment in either direction. Source pieces containing `<` are never
dispatched (they stay in the original language — segments.py), and the
a segment in either direction. A literal `<` in the source text (`<1MB` is
text, not markup — a tag needs a letter or `/!?`) crosses encoded as the
fullwidth `` and is decoded on return, before the result is validated and
spliced (segments.py) — the wire itself still never carries `<`, and the
reference client cuts the model's output at the first `<`
(scripts/translator.py) — echoed language tags, stray `<br>`s and any
future variant are one handled case. (The cut is post-decode, not a
generation stop string: Seed-X opens every generation with its `<s>`
framing token, which would trip a `<` stop immediately.)
Server-side, a second layer covers what the inline parser cannot: ASCII
punctuation that is plain prose on the wire but Markdown syntax in the
splice context — quotes (a translated `"` would close the quoted image
title it lands in), brackets (alt texts, re-inserted link texts), `|` in
table rows, `\` escapes. Rather than rejecting such results, `join` swaps
them for Unicode look-alikes before splicing (`_NEUTRAL` in
segments.py — curly quotes, fullwidth brackets; the renderer's
typographer curls straight quotes anyway).
Short fragments get more than a bare prompt: each segment may carry its
surround in `Job.contexts` — a title carries the article's opening prose
(its own block is just the title word), a segment carved out of a larger
+3
View File
@@ -37,3 +37,6 @@ __screenshots__/
# Playwright browser downloads (if ever installed locally)
.pw-browsers/
# npm project config (audit/fund off: the audit endpoint stalls installs)
!.npmrc
+2
View File
@@ -0,0 +1,2 @@
audit=false
fund=false
+1
View File
@@ -20,6 +20,7 @@
"codemirror": "^6.0.2",
"country-flag-icons": "^1.6.20",
"overlayscrollbars": "^2.16.0",
"pinia": "^4.0.3",
"transliteration": "^2.6.1",
"vue": "^3.5.26",
"vuedraggable": "^4.1.0"
+18 -9
View File
@@ -21,11 +21,11 @@ const currentPath = ref(props.pagePath)
const activeMode = ref(props.initialMode)
// The shared language selection (./editorLang, v-modeled by the tabs'
// LangSelects) also drives the page preview: while the shell is open it
// overrides the normal language preferences (?lang= / Accept-Language),
// so the page renders in the language being edited; closing restores.
// The primary selection pins by the CURRENT PAGE's own primary language
// (pages may differ — Node.language is inherited down the tree).
// LangSelects) is linked to the whole-page language: while the shell is
// open it drives the page preview (overrides ?lang= / Accept-Language),
// and closing keeps the pick as the session language. The primary
// selection pins by the CURRENT PAGE's own primary language (pages may
// differ — Node.language is inherited down the tree).
let pinned = false
function pinPreviewLang() {
pinned = true
@@ -34,6 +34,15 @@ function pinPreviewLang() {
setLangOverride(editorLang.value || pagePrimary.value || 'en')
loadPlain(currentPath.value)
}
// Opening the panel must not switch the page's language: adopt the
// session's chosen language (public selector / earlier pick) once, then
// pin. Runs only on (re)open — after that the selection is the user's.
function openShell() {
const session = window.__pageriteLang
if (!editorLang.value && session && session !== (pagePrimary.value || 'en'))
editorLang.value = session
pinPreviewLang()
}
function unpinPreviewLang() {
if (!pinned) return
pinned = false
@@ -89,13 +98,13 @@ function onSwitchEvent(ev) {
onMounted(() => {
document.body.dataset.editorMode = activeMode.value
addEventListener('pagerite:switch-editor', onSwitchEvent)
addEventListener('pagerite:editor-shown', pinPreviewLang)
addEventListener('pagerite:editor-shown', openShell)
addEventListener('pagerite:editor-hidden', unpinPreviewLang)
// The shell mounts visible (openEditor), so pin immediately. The site
// The shell mounts visible (openEditor), so open immediately. The site
// default primary language comes from the settings — it only fills the
// unknown; the page/structure tabs refine pagePrimary per page as they
// learn it (their knowledge is strictly better).
pinPreviewLang()
openShell()
fetch('/_api/settings').then((r) => r.json()).then((s) => {
if (!pagePrimary.value) pagePrimary.value = s.primary_lang || 'en'
}).catch(() => { /* keep the fallback */ })
@@ -103,7 +112,7 @@ onMounted(() => {
onUnmounted(() => {
removeEventListener('pagerite:switch-editor', onSwitchEvent)
removeEventListener('pagerite:editor-shown', pinPreviewLang)
removeEventListener('pagerite:editor-shown', openShell)
removeEventListener('pagerite:editor-hidden', unpinPreviewLang)
})
</script>
+30 -9
View File
@@ -3,7 +3,8 @@
// flag button opening a clean dropdown, v-modeled on the shared editorLang
// ('' = the primary language). The lang tab's flag grid is a different
// control (toggles, not a select) and stays as it is.
import { computed, ref } from 'vue'
import { computed, nextTick, ref } from 'vue'
import { usePopup } from './dropdown'
const props = defineProps({
modelValue: { type: String, default: '' },
@@ -13,8 +14,12 @@ const props = defineProps({
const emit = defineEmits(['update:modelValue'])
const open = ref(false)
const root = ref(null)
const toggleBtn = ref(null)
const pop = ref(null)
const popStyle = ref({})
// Closes on outside click / Escape (./dropdown), not on mouseleave.
usePopup(open, root)
const current = computed(
() => props.options.find((o) => o.tag === props.modelValue) ?? props.options[0],
)
@@ -26,6 +31,17 @@ function toggle() {
// onto the page area instead of being clipped by it.
const r = toggleBtn.value.getBoundingClientRect()
popStyle.value = { top: `${r.bottom + 2}px`, left: `${r.left}px` }
// A toggle mounted near the right window edge (the public page
// selector sits top-right) opens the popup flush against that edge.
nextTick(() => {
const p = pop.value?.getBoundingClientRect()
if (p && p.right > innerWidth - 4) {
popStyle.value = {
...popStyle.value,
left: `${Math.max(4, innerWidth - 4 - p.width)}px`,
}
}
})
}
}
@@ -36,7 +52,7 @@ function select(tag) {
</script>
<template>
<span v-if="options.length > 1" class="lang-select">
<span v-if="options.length > 1" ref="root" class="lang-select">
<button
ref="toggleBtn"
type="button"
@@ -47,7 +63,7 @@ function select(tag) {
: '')"
@click="toggle"
><span v-if="current?.flag" class="flag" v-html="current.flag" /></button>
<span v-if="open" class="lang-pop" :style="popStyle" @mouseleave="open = false">
<span v-if="open" ref="pop" class="lang-pop" :style="popStyle">
<button
v-for="o in options"
:key="o.code"
@@ -66,20 +82,23 @@ function select(tag) {
display: flex;
}
/* The closed state is just the small flag — no button chrome until hovered. */
/* The closed state is just the small flag — no button chrome at all, on
hover either (it sits among borderless emoji-icon buttons); like them it
rests dimmed and brightens on hover. */
.lang-current {
display: flex;
align-items: center;
padding: 2px;
background: none;
border: 1px solid transparent;
border: none;
border-radius: 4px;
cursor: pointer;
opacity: 0.7;
}
.lang-current:hover,
.lang-current.open {
border-color: var(--line);
opacity: 1;
}
/* The dropdown matches the page's existing popups (.picker-pop look).
@@ -127,11 +146,13 @@ function select(tag) {
color: var(--muted);
}
/* Flags render like in the analytics visitor cells. */
/* em-sized so the chip matches the surrounding text/icon size in each
context; the hairline border delineates white-flagged countries (not
button chrome). */
.flag {
display: inline-flex;
width: 18px;
height: 12px;
width: 1.5em;
height: 1em;
flex: 0 0 auto;
border-radius: 2px;
overflow: hidden;
+41
View File
@@ -0,0 +1,41 @@
<script setup>
// The public page's language selector: the editors' flag dropdown
// (LangSelect) as the first item of the banner's corner container, fed
// from the shared store (pagerite.js sets the page's hreflang alternates
// and served language per navigation). It binds the same store.lang the
// editor's dropdown binds, so both always show the same selection. A pick
// also dispatches pagerite:set-session-lang — pagerite.js swaps the page
// in place when the editor is closed (open, the editor reacts to the
// store and re-renders it).
import { computed } from 'vue'
import LangSelect from './LangSelect.vue'
import { flagFor, langName } from './langs'
import { useStore } from './store'
const store = useStore()
// The "(primary)" marker is admin-panel information; the public selector
// lists plain languages.
const options = computed(() =>
store.langAlternates.map((a) => ({
tag: a.tag,
code: a.tag,
name: langName(a.tag),
flag: flagFor(a.tag),
primary: false,
})),
)
const primaryTag = computed(() => store.langAlternates.find((a) => a.primary)?.tag ?? '')
// The explicit pick, else the served language (header-autodetected pages
// may have neither), else the primary.
const model = computed(() => store.lang || store.servedLang || primaryTag.value)
function go(tag) {
store.lang = tag === primaryTag.value ? '' : tag
dispatchEvent(new CustomEvent('pagerite:set-session-lang', { detail: { lang: tag } }))
}
</script>
<template>
<LangSelect :model-value="model" :options="options" @update:model-value="go" />
</template>
+33 -9
View File
@@ -26,6 +26,7 @@
// renders the version being edited, whichever language the page itself
// was loaded in.
import { computed, onActivated, onMounted, onUnmounted, ref, watch } from 'vue'
import { usePopup } from './dropdown'
import { EditorView, basicSetup } from 'codemirror'
import { Compartment, EditorState } from '@codemirror/state'
import { keymap } from '@codemirror/view'
@@ -621,8 +622,15 @@ const TABLE_MAX_ROWS = 6
// Class pickers: popup listing the block class toggles (placement ↔︎,
// text size AA), closed after applying. The block's current class of the
// group is marked; choosing "normal" (or the current class) removes it.
// All popups share the close behavior of ./dropdown (outside click /
// Escape; never mouseleave).
const classPicker = ref(null) // 'place' | 'size' | null
const activeClasses = ref(new Set())
const placeRoot = ref(null)
const sizeRoot = ref(null)
const tableRoot = ref(null)
usePopup(classPicker, computed(() => (classPicker.value === 'place' ? placeRoot : sizeRoot).value))
usePopup(tablePicker, tableRoot)
function openClassPicker(which) {
classPicker.value = classPicker.value === which ? null : which
@@ -1136,15 +1144,31 @@ onUnmounted(() => {
<div class="format-bar">
<button type="button" class="code-btn" title="code — inline wrap, or a fenced block for line-spanning selections; click again to unwrap" @click="insertCode"><code>&lt;/&gt;</code></button>
<button type="button" title="link (toggle: click inside a link to unwrap it)" @click="insertLink">🔗</button>
<button
type="button"
title="table"
:class="{ active: tablePicker }"
@click="tablePicker = !tablePicker"
></button>
<span class="picker" ref="tableRoot">
<button
type="button"
title="table"
:class="{ active: tablePicker }"
@click="tablePicker = !tablePicker"
></button>
<div v-if="tablePicker" class="table-picker" @mouseleave="tableSize = { cols: 0, rows: 0 }">
<div class="tp-grid" :style="{ gridTemplateColumns: `repeat(${TABLE_MAX_COLS}, 1fr)` }">
<button
v-for="n in TABLE_MAX_COLS * TABLE_MAX_ROWS"
:key="n"
type="button"
class="tp-cell"
:class="{ on: tableSize.cols >= (n - 1) % TABLE_MAX_COLS + 1 && tableSize.rows >= Math.floor((n - 1) / TABLE_MAX_COLS) + 1 }"
@mouseenter="tableSize = { cols: (n - 1) % TABLE_MAX_COLS + 1, rows: Math.floor((n - 1) / TABLE_MAX_COLS) + 1 }"
@click="insertTable(tableSize.cols, tableSize.rows)"
/>
</div>
<div class="tp-size">{{ tableSize.cols || '' }} × {{ tableSize.rows || '' }}</div>
</div>
</span>
<button type="button" title="insert image (upload) — pasting works too" @click="fileInput.click()">🖼</button>
<button type="button" title="aside box (::: aside) — wraps the selection or the cursor's line; clicked inside one, removes it" @click="insertAside"></button>
<span class="picker">
<span class="picker" ref="placeRoot">
<button
type="button"
title="block placement class"
@@ -1166,7 +1190,7 @@ onUnmounted(() => {
</span>
<button type="button" title="bold" @click="wrapInline('**')"><b>B</b></button>
<button type="button" title="italic" @click="wrapInline('*')"><i>i</i></button>
<span class="picker">
<span class="picker" ref="sizeRoot">
<button
type="button"
title="text size class"
@@ -1368,7 +1392,7 @@ onUnmounted(() => {
.table-picker {
position: absolute;
top: 100%;
left: 6.5rem;
left: 0;
z-index: 20;
padding: 0.5rem;
background: var(--bg);
+24
View File
@@ -0,0 +1,24 @@
// Shared popup open-state behavior: while `open` (a ref, truthy = open)
// is set, a pointerdown outside `root` (a template ref covering both the
// toggle button and the popup) or Escape resets it to null. One logic for
// every dropdown (LangSelect, the page editor's class/table pickers), so
// they can't drift apart.
import { onBeforeUnmount, watch } from 'vue'
export function usePopup(open, root) {
let off = null
const stop = watch(open, (v) => {
off?.()
off = null
if (!v) return
const down = (ev) => { if (!root.value?.contains(ev.target)) open.value = null }
const key = (ev) => { if (ev.key === 'Escape') open.value = null }
addEventListener('pointerdown', down, true)
addEventListener('keydown', key)
off = () => {
removeEventListener('pointerdown', down, true)
removeEventListener('keydown', key)
}
})
onBeforeUnmount(() => { off?.(); stop() })
}
+10 -5
View File
@@ -1,10 +1,15 @@
// The editor shell's shared language selection ('' = the primary language):
// one state, v-modeled by the LangSelect of every tab that has one (page,
// structure). While the panel is open it also drives the page preview —
// EditorShell applies it as the fetch-time language override (swapdoc).
import { ref } from 'vue'
// backed by the app-wide store (./store), so the editor tabs' LangSelects
// and the public corner selector bind the same value. Linked to the
// whole-page language: while the panel is open it drives the page preview
// (EditorShell applies it as the fetch-time language override, swapdoc).
import { computed, ref } from 'vue'
import { pinia, useStore } from './store'
export const editorLang = ref('')
export const editorLang = computed({
get: () => useStore(pinia).lang,
set: (v) => { useStore(pinia).lang = v },
})
// The CURRENT PAGE's primary language ('' = not yet learned): the shell's
// settings fetch fills it with the site default; the page/structure tabs
+50
View File
@@ -0,0 +1,50 @@
// Public language-selector entry: imported on demand by pagerite.js on
// pages advertising more than one language in their hreflang alternates.
// Vue, Pinia and the flag SVG set live in this chunk only — untranslated
// pages never pay for them. The selector's state lives in the shared
// store (./store), not the DOM: the corner container is rebuilt freely
// and ensureMounted re-mounts from the store.
import { createApp } from 'vue'
import LangSelector from './LangSelector.vue'
import { pinia, useStore } from './store'
let app = null
function store() {
return useStore(pinia)
}
// The current page's languages (called on every navigation).
export function setLanguages(alternates, current) {
Object.assign(store(), {
langAlternates: alternates,
servedLang: current,
langSelectorActive: true,
})
}
// The current page is single-language: the selector goes away.
export function hide() {
store().langSelectorActive = false
app?.unmount()
app = null
}
// Mount the selector as the container's first item; re-mount when its
// element went away with a container rebuild (a live app updates from the
// store reactively).
export function ensureMounted(host) {
if (!store().langSelectorActive || !host) {
app?.unmount()
app = null
return
}
if (app && host.contains(app._container)) return
app?.unmount()
const el = document.createElement('div')
el.id = 'lang-selector'
host.prepend(el)
app = createApp(LangSelector)
app.use(pinia)
app.mount(el)
}
+106 -29
View File
@@ -51,13 +51,18 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
url.searchParams.delete("lang");
history.replaceState(history.state, "", url);
}
// The session language. While the editor panel is open, its language
// selection overrides the normal preference (swapdoc.setLangOverride):
// internal fetches and prefetches follow it until the panel closes and
// the override clears (null restores the initial ?lang=, if any).
// The session language: the user's explicit pick (initial ?lang=, public
// selector, editor dropdown) is kept in chosenLang; while the editor is
// open its selection overrides it (swapdoc.setLangOverride), and closing
// falls back to chosenLang. JS state only — pretty URLs, no reloads.
// window.__pageriteLang is the pin for swapdoc.loadPlain's fetches.
let chosenLang = langParam;
let sessionLang = langParam;
window.__pageriteLang = sessionLang;
addEventListener("pagerite:session-lang", (ev) => {
sessionLang = ev.detail?.lang || langParam;
if (ev.detail?.lang) chosenLang = ev.detail.lang;
sessionLang = ev.detail?.lang || chosenLang;
window.__pageriteLang = sessionLang;
});
// An internal URL as fetched: carries the session's ?lang= unless the
// link already pins a language of its own. With no ?lang= on the initial
@@ -167,6 +172,22 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
return a;
}
// The banner top-right corner container: the language selector (first
// item) plus the admin pens and auth links. renderAuthUi rebuilds it from
// scratch; the selector's state lives in the shared store, not the DOM,
// so the langselect bundle re-mounts it into the fresh container.
function pensContainer() {
let pens = document.querySelector(".editor-pens");
if (!pens) {
const banner = document.getElementById("page-banner");
if (!banner) return null;
pens = document.createElement("div");
pens.className = "editor-pens";
banner.after(pens);
}
return pens;
}
function removePens() {
document.querySelectorAll(".editor-pens, #main article button.edit-link")
.forEach((el) => el.remove());
@@ -178,32 +199,33 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
// pens that may have been injected while the browser cache made us look
// authenticated.
removePens();
if (!authReady) return;
// Editing is open for admins and, as a dev/no-proxy fallback, when no
// Paskia SSO is detected at all.
const canEdit = isAdmin || !ssoAvailable;
// The analytics page is a read-only dashboard: editing pens and the side
// panel do not apply there. Login/logout links are still useful.
const onAnalytics = currentPath === "/_a";
const banner = document.getElementById("page-banner");
if (banner) {
const pens = document.createElement("div");
pens.className = "editor-pens";
if (canEdit && !onAnalytics) {
// Analytics viewer is now a normal page at /_a.
const a = document.createElement("a");
a.className = "edit-link analytics-link";
a.href = "/_a";
a.title = "analytics";
a.textContent = "📊";
pens.append(a);
pens.append(makePen("site"));
if (authReady) {
// Editing is open for admins and, as a dev/no-proxy fallback, when no
// Paskia SSO is detected at all.
const canEdit = isAdmin || !ssoAvailable;
// The analytics page is a read-only dashboard: editing pens and the side
// panel do not apply there. Login/logout links are still useful.
const onAnalytics = currentPath === "/_a";
if (document.getElementById("page-banner")) {
const pens = pensContainer();
if (canEdit && !onAnalytics) {
// Analytics viewer is now a normal page at /_a.
const a = document.createElement("a");
a.className = "edit-link analytics-link";
a.href = "/_a";
a.title = "analytics";
a.textContent = "📊";
pens.append(a);
pens.append(makePen("site"));
}
if (ssoAvailable) pens.append(makeAuthLink(isAdmin));
if (!pens.firstElementChild) pens.remove();
}
if (ssoAvailable) pens.append(makeAuthLink(isAdmin));
banner.after(pens);
if (canEdit && !onAnalytics) injectPagePen();
}
if (canEdit && !onAnalytics) injectPagePen();
// Re-mount the selector into the fresh container (no-op until the
// bundle has been loaded once).
langselectMod?.ensureMounted(document.querySelector(".editor-pens"));
}
async function setupAuth() {
@@ -407,6 +429,9 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
// copy pinned to a language (?lang=) caches under its own key, where
// navigation with the same session language finds it.
pageCache.set(rawKey(ev.detail.url), ev.detail.html);
// Editor-driven swaps don't go through load(): re-evaluate the
// language selector from the fresh copy too.
mountLangselect(new DOMParser().parseFromString(ev.detail.html, "text/html"));
});
// Editors mutate site-wide state (theme, structure, headings, banners),
@@ -742,6 +767,56 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
}
}
// --- Public language selector ------------------------------------------
// Pages translated into more than one language advertise it via hreflang
// alternates (x-default + one link per language). Those pages get the
// editors' flag dropdown as the first item of the corner container; its
// bundle (Vue + the flag SVG set) loads on demand. Re-evaluated from the
// fresh document on every swap (the head's own alternates stay stale).
let langselectMod = null;
async function mountLangselect(doc) {
const links = [...doc.head.querySelectorAll('link[rel="alternate"][hreflang]')];
const dflt = links.find((l) => l.hreflang === "x-default");
const langs = links.filter((l) => l.hreflang && l.hreflang !== "x-default");
if (!dflt || langs.length <= 1) return langselectMod?.hide();
try {
langselectMod ??= await import(/* @vite-ignore */ assets["pagerite:langselect-src"]);
for (const css of (assets["pagerite:langselect-css"] || "").split(",")) {
if (css && !document.querySelector(`link[href="${css}"]`)) {
const link = document.createElement("link");
link.rel = "stylesheet";
link.href = css;
link.dataset.pagerite = "langselect-css";
document.head.append(link);
}
}
langselectMod.setLanguages(
// The original's alternate is the plain URL — x-default's href —
// which also marks it as the primary option.
langs.map((l) => ({ tag: l.hreflang, href: l.href, primary: l.href === dflt.href })),
doc.documentElement.lang,
);
langselectMod.ensureMounted(pensContainer());
} catch (e) {
console.error("language selector mount failed:", e);
}
}
// The selector's pick (LangSelector dispatches this): make it the
// session language and swap the page in place. With the editor open the
// pick already landed in the shared store — the editor's watch re-renders
// the page itself, so there is nothing to do here.
addEventListener("pagerite:set-session-lang", async (ev) => {
const tag = ev.detail?.lang;
if (!tag || tag === sessionLang) return;
if (document.body.classList.contains("editing")) return;
chosenLang = sessionLang = tag;
window.__pageriteLang = tag;
const y = scrollY; // a language switch is not a navigation: keep scroll
await load(currentPath, false);
scrollTo(0, y);
});
// --- Fetch navigation ------------------------------------------------
async function load(url, push = true, back = false) {
// Navigating with the editor open closes it; unsaved edits are lost
@@ -843,6 +918,7 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
runScripts(document.getElementById("main"));
applyEffects();
mountAnalytics(doc);
mountLangselect(doc);
};
// Rotating cube page transition (styles injected as #pagerite-transition
// from the selected design's transition.css, e.g. themes/cube/);
@@ -1076,4 +1152,5 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
setupAuth();
applyEffects();
mountAnalytics(document);
mountLangselect(document);
})();
+26
View File
@@ -0,0 +1,26 @@
// The app's shared Pinia store — cross-bundle UI state lives here. Every
// entry chunk imports its own copy of this module, so the Pinia instance
// is parked on window (Vue itself is a shared chunk, so reactivity works
// across the copies). Pass `pinia` explicitly when calling useStore
// outside a component (module code, no active instance).
import { createPinia, defineStore } from 'pinia'
export const pinia = (window.__pageritePinia ??= createPinia())
export const useStore = defineStore('pagerite', {
state: () => ({
// The ONE language selection, v-modeled by both dropdowns (editor
// tabs, public corner selector): '' = no explicit pick (the page's
// primary / autodetect), else a concrete tag. A pick from either
// dropdown is visible to everyone immediately.
lang: '',
// The language the current page was actually served in (set by
// pagerite.js per navigation) — the selector's highlight fallback
// when there is no explicit pick.
servedLang: '',
// The public selector's page data: hreflang alternates
// ([{tag, href, primary}]) and whether to show at all.
langAlternates: [],
langSelectorActive: false,
}),
})
+12 -9
View File
@@ -12,12 +12,13 @@ export function dropPageCache() {
}
// The editor's language override (set by EditorShell): while the panel is
// open, its language selection wins over the normal preferences (?lang= /
// Accept-Language) — every in-place re-render asks for that language
// explicitly, and pagerite.js applies it to its own fetches and prefetches
// (pagerite:session-lang). The primary selection pins by its code:
// ?lang=<primary> selects the original explicitly (i18n.select_language).
let overrideLang = null // the ?lang= value in force, null = normal prefs
// open, its language selection wins over the normal preferences — every
// in-place re-render asks for that language explicitly, and pagerite.js
// applies it to its own fetches and prefetches (pagerite:session-lang).
// The primary selection pins by its code: ?lang=<primary> selects the
// original explicitly (i18n.select_language). Panel closed, the session's
// chosen language (window.__pageriteLang) takes over — the pick stays.
let overrideLang = null // the ?lang= value in force, null = the session's
export function setLangOverride(queryLang) {
overrideLang = queryLang || null
@@ -129,14 +130,16 @@ function swapRegions(doc) {
// Fetch /p, swap its regions into the live page and replaceState to it.
// Returns the final URL (after redirects), or null when the fetch did not
// yield a page. Category and missing URLs render a placeholder 404 page —
// fine to swap in (new pages are created by editing them). While the
// editor's language override is set the fetch pins that language.
// fine to swap in (new pages are created by editing them). The fetch pins
// the editor's language override, or — panel closed — the session's chosen
// language (window.__pageriteLang).
export async function loadPlain(p) {
let doc
let finalUrl = `/${p}`
let html
try {
const res = await fetch(overrideLang ? `${finalUrl}?lang=${overrideLang}` : finalUrl)
const pin = overrideLang || window.__pageriteLang
const res = await fetch(pin ? `${finalUrl}?lang=${pin}` : finalUrl)
const type = res.headers.get('content-type') || ''
if (!type.includes('text/html')) return null
if (res.redirected) finalUrl = res.url
+2 -1
View File
@@ -38,7 +38,7 @@ export default defineConfig({
chunkSizeWarningLimit: 1200,
// Mirror the URL space in the build output: hashed files land under
// frontend-build/_assets/ and the Frontend serves the build directory
// at the site root (frontend/public/favicon.ico -> /favicon.ico).
// at the site root.
manifest: true,
assetsDir: '_assets',
rollupOptions: {
@@ -50,6 +50,7 @@ export default defineConfig({
main: fileURLToPath(new URL('./src/main.js', import.meta.url)),
pagerite: fileURLToPath(new URL('./src/pagerite.js', import.meta.url)),
analytics: fileURLToPath(new URL('./src/analytics-main.js', import.meta.url)),
langselect: fileURLToPath(new URL('./src/langselect-main.js', import.meta.url)),
// Only the base CSS is built; theme/banner-design stylesheets live
// in pagerite/themes/{name}/ and are served by the backend as-is.
pagerite_base: fileURLToPath(new URL('./src/assets/pagerite.css', import.meta.url)),
+8
View File
@@ -503,6 +503,14 @@ async def editor_ws(ws: WebSocket) -> None:
# The title is injected as h1 when the markdown has
# none; the editor's title field edits live-preview.
title=msg.get("title") or (node.title if node else ""),
# Pin section anchors to the original language so the
# preview of a translation matches the served page
# (no-op when the previewed markdown is the original).
anchors_from=(
(node_markdown(data, node) or "", node.title)
if node
else None
),
)
await ws.send_json(
{
+2 -2
View File
@@ -48,7 +48,7 @@ logger = logging.getLogger(__name__)
# Vue build served at the site root, no SPA catch-all (assets only). The
# build mirrors the URL space: hashed, immutable files live under
# /_assets/ (assetsDir: '_/assets'), the favicon at /favicon.ico.
# /_assets/ (assetsDir: '_/assets').
frontend = Frontend(
Path(__file__).with_name("frontend-build"), spa=False, cached="/_assets/"
)
@@ -124,7 +124,7 @@ app.include_router(tracking.router)
app.include_router(files.router)
# Vue build asset routes are inserted at this position during load(): the
# build mirrors the URL space (/_assets/*, /favicon.ico at the root).
# build mirrors the URL space (/_assets/*).
frontend.route(app, "/")
# The content catch-all goes last: built assets win over content slugs,
+2 -2
View File
@@ -98,8 +98,8 @@ class Data(msgspec.Struct):
#: Trusted author content; not sanitized.
custom_css: str = ""
#: Favicon: content-addressed file name (served at "/_f/{name}"),
#: linked as <link rel="icon"> on every page. Empty = the build's
#: /favicon.ico.
#: linked as <link rel="icon"> on every page; /favicon.ico redirects
#: to it. Empty = no icon (and /favicon.ico 404s).
favicon: str = ""
#: API keys gating the translator service WebSocket (/_translate/{key};
#: the external forward-auth does not cover that route): key -> display
+19 -3
View File
@@ -6,7 +6,8 @@ when compression shrinks the body), served immutable at ``/_f/``. Raster
images and SVGs are recompressed into AVIF/WebP/JPEG derivatives
(``store_image`` and helpers); the untouched original is kept alongside as
``<hash>.orig<ext>`` (never served). Routes: upload/delete under
``/_api/files``, the favicon settings endpoints, the ``/_f/`` server with
``/_api/files``, the favicon settings endpoints, the /favicon.ico
redirect to the configured icon, the ``/_f/`` server with
Accept-negotiated formats, and the user assets (``/_themes/``, ``/_fonts/``).
"""
@@ -19,7 +20,7 @@ from pathlib import Path
import blake3
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import Response
from fastapi.responses import RedirectResponse, Response
from mediapreview import dispatch
from pagerite import views
@@ -252,6 +253,20 @@ async def delete_file(name: str) -> None:
file_store.delete(name)
@router.get("/favicon.ico", include_in_schema=False)
async def favicon_ico() -> Response:
"""The conventional /favicon.ico: redirect to the configured site icon.
Browsers request this path on their own (tabs, bookmarks, feeds and
other non-HTML contexts) regardless of the <link rel="icon"> pages
carry. Redirect to the icon's store URL, which negotiates the format
and caches immutably; 404 when no custom icon is configured.
"""
if not data.favicon:
raise HTTPException(404)
return RedirectResponse(f"/_f/{data.favicon}")
@router.put("/_api/settings/favicon")
async def put_favicon(request: Request) -> dict[str, str]:
"""Upload a favicon into the content-addressed store and activate it.
@@ -276,7 +291,8 @@ async def put_favicon(request: Request) -> dict[str, str]:
@router.delete("/_api/settings/favicon", status_code=204)
async def delete_favicon(request: Request) -> None:
"""Clear the custom favicon (back to the build's /favicon.ico).
"""Clear the custom favicon (/favicon.ico goes back to 404, pages drop
the <link rel="icon">).
The blob stays in the content-addressed store; only the reference goes.
"""
+55 -7
View File
@@ -402,7 +402,9 @@ def _heading_ids(state) -> None:
its self-link is ``href=""`` (back to the top of the page). An
author-set `{#id}` always wins; auto ids slugify the heading text
(python-slugify, mirroring the editor's slugify.js) and dedupe with
-2/-3 suffixes per render. Headings that already contain a link are
-2/-3 suffixes per render — unless env["anchor_ids"] presets them, as
render(anchors_from=...) does for translated pages so section URLs
stay in the original language. Headings that already contain a link are
``data-line`` records the heading's markdown source line (0-based, after
undoing the render(title=...) injection offset via ``env``) — the page
editor uses it for section pens and piecewise-linear scroll sync.
@@ -443,15 +445,25 @@ def _heading_ids(state) -> None:
if len(heads) < ANCHOR_MIN_HEADINGS:
return
seen: set[str] = set()
for i, token in heads:
preset = state.env.get("anchor_ids")
for k, (i, token) in enumerate(heads):
inline = tokens[i + 1]
hid = token.attrGet("id")
if not isinstance(hid, str) or not hid:
# Slug the visible text, not the raw markdown (`## [a](url)`).
text = "".join(
c.content for c in inline.children if c.type in ("text", "code_inline")
)
base = slugify(text) or "section"
if preset is not None and k < len(preset):
# Translated render: the original language's slug, matched
# by heading position (a translation never adds, removes or
# reorders headings; a patched one that does falls back to
# slugging its own text past the end of the list).
base = preset[k]
else:
# Slug the visible text, not the raw markdown (`## [a](url)`).
text = "".join(
c.content
for c in inline.children
if c.type in ("text", "code_inline")
)
base = slugify(text) or "section"
hid, n = base, 2
while hid in seen:
hid = f"{base}-{n}"
@@ -463,6 +475,36 @@ def _heading_ids(state) -> None:
wrap(i, token, f"#{hid}")
def anchor_ids(text: str, title: str | None = None) -> list[str]:
"""The section anchor ids of text, in heading order.
render(anchors_from=...) feeds these to _heading_ids via
env["anchor_ids"], pinning a translated render's anchors to the
original language's slugs. The selection mirrors _heading_ids exactly
(the same md instance assigns the ids during this parse, author-set
{#id} included as-is); the in-body title h1 is excluded.
"""
if title and not has_h1(text):
text = f"# {title}\n\n{text}"
tokens = md.parse(text, {"page_path": ""})
first_h1 = next(
(
i
for i, t in enumerate(tokens)
if t.type == "heading_open" and t.tag == "h1" and t.level == 0
),
None,
)
return [
t.attrGet("id")
for i, t in enumerate(tokens)
if t.type == "heading_open"
and t.tag in ("h1", "h2")
and t.level == 0
and i != first_h1
]
def make_md(*, verbatim: bool = False) -> MarkdownIt:
"""A fully configured parser. The module-level ``md`` (below) is the
render instance; ``verbatim=True`` builds the segmentation instance for
@@ -618,12 +660,16 @@ def render(
created: datetime | None = None,
modified: datetime | None = None,
title: str | None = None,
anchors_from: tuple[str, str] | None = None,
) -> Rendered:
"""Render Markdown text to the article body's HTML and layout flags.
``title`` injects a ``# {title}`` line at the top when the markdown has
no h1 of its own, so the implicit page title goes through the exact
same pipeline as an explicit one (first-h1 anchor treatment included).
``anchors_from`` is the (markdown, title) of the ORIGINAL language when
rendering a translation: section anchors are pinned to its slugs so
localized pages keep the original #hash URLs.
The top-level blocks are grouped into column segments: boundary blocks
(h1/h2 headings, .wide — see _is_boundary) are rendered bare, the runs
@@ -641,6 +687,8 @@ def render(
right after the article's h1.
"""
env = {"page_path": page_path, "line_offset": 0}
if anchors_from is not None:
env["anchor_ids"] = anchor_ids(*anchors_from)
if title and not has_h1(text):
text = f"# {title}\n\n{text}"
# The injected title shifts source lines by two; _heading_ids
+93 -20
View File
@@ -20,8 +20,13 @@ each segment's source span was located at dispatch (``split``), and
``join`` swaps in the translations. Markup therefore cannot break — it
never left the server. A returned segment must still be pure prose itself
(the model could inject markup INTO a segment); anything else — count
mismatch, empty segment, markup tokens — rejects the whole result and the
fragment stays pending.
mismatch, empty segment, markup tokens, a line that would start a new
block (a ``` or ::: fence would eat the rest of the block it lands in) —
rejects the whole result and the
fragment stays pending. Punctuation that is prose on the wire but syntax
in the splice context (quotes in a title attribute, brackets in an alt
text, "|" in a table row) is not worth a rejection either: it is swapped
for Unicode look-alikes (``_NEUTRAL``) before splicing.
A block of plain text, prose links and paired text formatting
(strong/em/s) crosses as ONE segment — link texts and formatted text
@@ -42,9 +47,11 @@ snippets that don't fit together. Blocks with any other inline markup
Locating is best effort: a run that is not a verbatim source substring
(entity-decoded text, backslash escapes) is skipped — it simply stays in
the original language. So is any piece containing "<": "<" is the
prose/markup boundary on the wire — translators cut their output there,
so such pieces could not survive the round trip.
the original language. A literal "<" in prose ("<1MB") is text, not
markup, but cannot cross as-is — "<" is the prose/markup boundary on the
wire, translators cut their output there — so it crosses encoded as the
fullwidth "" (``_encode``) and ``join`` decodes it back before
validating and splicing.
"""
import bisect
@@ -69,6 +76,38 @@ _ALERT = re.compile(r"^\[![A-Za-z]+\][ \t]*")
#: (inline attrs are consumed by the parser; a lone {dates} is not).
_BRACES = re.compile(r"\{[^{}\n]*\}")
def _encode(text: str) -> str:
"""Wire form of a segment or context: a literal "<" as fullwidth "".
A "<" in prose is text, not markup ("<1MB" — a tag needs a letter or
/!?), but "<" is the prose/markup boundary on the wire (translators
cut output at the first "<", scripts/translator.py), so it cannot
cross as-is. join decodes it back before the pure_prose check and
splicing — anything tag-like the model may have formed around it is
still rejected there.
"""
return text.replace("<", "")
#: ASCII punctuation that is plain prose to the inline parser (so
#: pure_prose cannot catch it) but Markdown SYNTAX in a splice context:
#: quotes close a quoted image/link title, brackets the [...] of alt and
#: re-inserted link texts, "|" splits a table row, and "\" escapes the
#: character after it (a trailing one eats a title's closing quote).
#: Neutralized to Unicode look-alikes (join), which Markdown treats as
#: plain text everywhere — the quotes are curled the way typographer=True
#: renders them anyway.
_NEUTRAL = str.maketrans(
{
'"': "",
"'": "",
"[": "",
"]": "",
"\\": "",
"|": "",
}
)
#: A link's tail after its text: "](dest)", "](dest \"title\")", "][ref]",
#: "[]" or a bare "]" (shortcut reference); the destination may nest one
#: level of parens. Best effort — a mis-scan fails the span-reconstruction
@@ -273,7 +312,7 @@ def _linked_block(
raw = "".join(text for text, _ in pieces)
lead = len(raw) - len(raw.lstrip())
wire = raw.strip()
if not _LETTER.search(wire) or "<" in wire or _BRACES.search(wire):
if not _LETTER.search(wire) or _BRACES.search(wire):
return None
# Locate each piece verbatim, in order; the source slices between the
# located pieces are then the link syntax, exact by construction.
@@ -338,7 +377,7 @@ def _linked_block(
rec.append(text_)
if source[span_start:span_end] != "".join(rec):
return None
return Span(span_start, span_end, _weight(wire), marks), wire
return Span(span_start, span_end, _weight(wire), marks), _encode(wire)
def split(text: str) -> tuple[list[Span], list[str], list[str]]:
@@ -367,10 +406,8 @@ def split(text: str) -> tuple[list[Span], list[str], list[str]]:
def emit(run: str, at: int, ctx: str) -> None:
"""Carve {...} spans out of the located run; emit the prose pieces,
stripped — padding whitespace stays in the template, off the wire.
Pieces containing "<" are never emitted: translators cut output at
the first "<" (the prose/markup boundary, scripts/translator.py),
so such a piece could not survive the round trip — it stays in the
original language instead."""
A literal "<" crosses encoded (``_encode``): it is text, not
markup, but the wire keeps "<" as the prose/markup boundary."""
pieces = []
pos = 0
for m in _BRACES.finditer(run):
@@ -380,10 +417,10 @@ def split(text: str) -> tuple[list[Span], list[str], list[str]]:
for p0, p1 in pieces:
raw = run[p0:p1]
piece = raw.strip()
if _LETTER.search(piece) and "<" not in piece:
if _LETTER.search(piece):
start = at + p0 + (len(raw) - len(raw.lstrip()))
spans.append(Span(start, start + len(piece), 0, []))
segments.append(piece)
segments.append(_encode(piece))
contexts.append(ctx)
tokens = _MD.parse(text)
@@ -409,7 +446,7 @@ def split(text: str) -> tuple[list[Span], list[str], list[str]]:
cursor = span.end
continue
runs = _runs(kids)
block = _block_text(kids).strip()
block = _encode(_block_text(kids).strip())
if alert and runs:
run = _ALERT.sub("", runs[0], count=1)
if _LETTER.search(run):
@@ -417,7 +454,7 @@ def split(text: str) -> tuple[list[Span], list[str], list[str]]:
else:
runs.pop(0)
for run in runs:
ctx = block if block and run.strip() != block else ""
ctx = block if block and _encode(run.strip()) != block else ""
pos = _locate(text, run, cursor)
if pos != -1:
emit(run, pos, ctx)
@@ -435,6 +472,22 @@ def split(text: str) -> tuple[list[Span], list[str], list[str]]:
return spans, segments, contexts
#: Block-level Markdown a translation must not introduce: a segment is
#: spliced INSIDE a block of the fragment, so a line starting a heading,
#: quote, list, code/container fence or a setext/thematic-break underline
#: would break the fragment's block structure — a ``` or ::: line eats the
#: rest of the fence it lands in, closing fence included. pure_prose only
#: parses inline and lets such lines through as softbreak prose, so join
#: rejects them here. Blank lines split the host block and are rejected
#: too (a faithful translation of a single block has none).
_BLOCK = re.compile(
r"^[ \t]*(?:#{1,6}(?:[ \t]|$)|>[ \t]?|(?:[-+*]|\d{1,9}[.)])[ \t]|`{3,}|~{3,}|:{3,}(?:[ \t]|$)"
r"|-(?:[ \t]*-){2,}[ \t]*$|=[ =]*$|_(?:[ \t]*_){2,}[ \t]*$)",
re.M,
)
_BLANK = re.compile(r"\n[ \t]*\n")
def pure_prose(text: str) -> bool:
"""True when the text parses as nothing but prose (text and softbreak
tokens) — the acceptance test for a translated segment: the model may
@@ -597,17 +650,37 @@ def _place_marks(translation: str, weight: int, marks: list[Mark]) -> str | None
def join(original: str, spans: list[Span], texts: list[str]) -> str | None:
"""Splice translated segments back into the original fragment; None on
any validation failure (count mismatch, empty or non-prose segment) —
the caller drops the result and the fragment stays pending. Segments
with marks (a block that crossed as one piece) get their links
re-inserted at weight-mapped positions after the prose check."""
any validation failure (count mismatch, empty, non-prose or
block-structure segment) — the caller drops the result and the fragment
stays pending. Segments with marks (a block that crossed as one piece)
get their links re-inserted at weight-mapped positions after the prose
check.
Markdown-significant ASCII punctuation that pure_prose cannot see
(plain text inline, syntax in the splice context — quoted titles, alt
and link texts, table rows) is neutralized to Unicode look-alikes
(``_NEUTRAL``) before splicing and mark placement (the swap is
char-for-char, so unit alignment is unaffected); lines that would
start a new block (a heading, a ``` or ::: fence — they would eat the
rest of the block/fence they land in) reject the result outright
(``_BLOCK``, ``_BLANK``)."""
if len(texts) != len(spans):
return None
out: list[str] = []
cursor = 0
for span, translation in zip(spans, texts):
if not translation.strip() or not pure_prose(translation):
# Decode the wire form ("" back to "<") first: pure_prose then
# validates exactly what gets spliced — a "" the model formed
# into anything tag-like is markup and rejects the result.
translation = translation.replace("", "<")
if (
not translation.strip()
or not pure_prose(translation)
or _BLOCK.search(translation)
or _BLANK.search(translation.strip())
):
return None
translation = translation.translate(_NEUTRAL)
if span.marks:
translation = _place_marks(translation, span.weight, span.marks)
if translation is None:
+1
View File
@@ -132,6 +132,7 @@ def _render_html(
data.theme,
data.favicon,
data.brand_html,
base_url,
transition=data.transition,
lang=lang,
translation=translation,
+9 -8
View File
@@ -44,8 +44,9 @@ _analytics_ws_clients: set[WebSocket] = set()
_analytics_broadcast_task: asyncio.Task | None = None
# Repository root from this file's location (pagerite/tracking.py -> ..).
_REPO_ROOT = Path(__file__).resolve().parent.parent
# DB-IP databases persist in the working directory (one download serves all
# sites run from it). Not the package directory: reinstalls/upgrades wipe it.
_DBIP_DIR = Path.cwd()
DBIP_URL = "https://download.db-ip.com/free/dbip-city-lite-{month}.mmdb.gz"
@@ -60,7 +61,7 @@ def _download_dbip() -> None:
existing = sorted(
p.stem.removeprefix("dbip-city-lite-").removesuffix(".mmdb")
for p in _REPO_ROOT.glob("dbip-city-lite-*.mmdb*")
for p in _DBIP_DIR.glob("dbip-city-lite-*.mmdb*")
)
if existing and existing[-1] >= months[0]:
logger.info("DB-IP database is current (%s), skipping download", existing[-1])
@@ -68,7 +69,7 @@ def _download_dbip() -> None:
for month in months:
url = DBIP_URL.format(month=month)
target = _REPO_ROOT / f"dbip-city-lite-{month}.mmdb.gz"
target = _DBIP_DIR / f"dbip-city-lite-{month}.mmdb.gz"
tmp = target.with_suffix(".mmdb.gz.tmp")
logger.info("Downloading %s", url)
try:
@@ -93,7 +94,7 @@ def _download_dbip() -> None:
continue
os.replace(tmp, target)
# Drop older databases so the app never picks up a stale one.
for old in _REPO_ROOT.glob("dbip-city-lite-*.mmdb*"):
for old in _DBIP_DIR.glob("dbip-city-lite-*.mmdb*"):
if old.name != target.name:
old.unlink()
logger.info("DB-IP database updated to %s", target.name)
@@ -102,13 +103,13 @@ def _download_dbip() -> None:
def _geoip_db_path() -> Path | None:
"""Find a DB-IP MMDB in the repo root, preferring an already-decompressed
"""Find a DB-IP MMDB in the working directory, preferring an already-decompressed
``.mmdb`` over the matching ``.mmdb.gz``. Returns None if none is present.
"""
mmdb = sorted(_REPO_ROOT.glob("dbip-*.mmdb"))
mmdb = sorted(_DBIP_DIR.glob("dbip-*.mmdb"))
if mmdb:
return mmdb[0]
gz = sorted(_REPO_ROOT.glob("dbip-*.mmdb.gz"))
gz = sorted(_DBIP_DIR.glob("dbip-*.mmdb.gz"))
if gz:
return gz[0]
return None
+42 -33
View File
@@ -101,7 +101,8 @@ ClientMsg = Hello | Result
def pending_items(data: Data, lang: str) -> list[TransItem]:
"""Fragments of the site still untranslated for ``lang``, deduped by key.
Every page node (published or not) contributes its title and each chunk
Every node (published or not, pages and pure category labels alike)
contributes its title; pages also contribute each chunk
that needs translation (``needs_translation``), is not editor-flagged
no-translate (``node.no_trans``) and has no ``trans`` entry for ``lang``
yet. Content-addressed text (shared paragraphs, repeated titles) appears
@@ -133,8 +134,10 @@ def pending_items(data: Data, lang: str) -> list[TransItem]:
path = f"{prefix}/{slug}" if prefix else slug
# An article whose primary language IS the target needs no
# translation into it — skip its title and chunks entirely.
# Category labels (chunks is None) contribute only their title:
# it is their nav-menu label.
node_lang = node.language or inherited
if node.chunks is not None and node_lang != lang:
if node_lang != lang:
if node.title:
emit(
chunk_key(node.title),
@@ -143,7 +146,7 @@ def pending_items(data: Data, lang: str) -> list[TransItem]:
"title",
context=opening(node),
)
for h in node.chunks:
for h in node.chunks or ():
text = data.chunks.get(h)
if (
text is not None
@@ -178,8 +181,8 @@ def store_results(data: Data, lang: str, items: list[TransResult]) -> list[str]:
for slug, node in sorted_nodes(nodes):
path = f"{prefix}/{slug}" if prefix else slug
node_lang = node.language or inherited
if node.chunks is not None and node_lang != lang:
keys = set(node.chunks)
if node_lang != lang:
keys = set(node.chunks or ())
if node.title:
keys.add(chunk_key(node.title))
if keys & stored:
@@ -277,34 +280,40 @@ class Dispatcher:
job = None
spans: list[Span] = []
original = ""
for lang in sorted(langs):
# Titles first: a page's name in the menu is its most
# visible string (stable: menu order kept within each kind).
for item in sorted(
pending_items(self.data, lang), key=lambda it: it.kind != "title"
):
if (lang, item.key) in inflight or (
lang,
item.key,
) in self.validation_failures:
continue
spans, texts, contexts = split(item.text)
if not texts:
continue # prose that could not be located for splicing
original = item.text
if item.kind == "title" and item.context:
# A title's surround is the article's opening prose
# (TransItem.context), not its own one-word block.
contexts = [item.context] * len(texts)
job = Job(
lang=lang,
key=item.key,
texts=texts,
path=item.path,
kind=item.kind,
contexts=contexts,
)
break
# Titles before articles — across languages too, so every menu
# is named before any article body is worked on (a page's name
# is its most visible string). pending_items emits in menu
# order, a page's title before its chunks; filtering by kind
# keeps that stable order within each kind.
pending = {lang: pending_items(self.data, lang) for lang in sorted(langs)}
for kind in ("title", "chunk"):
for lang in sorted(langs):
for item in pending[lang]:
if (
item.kind != kind
or (lang, item.key) in inflight
or (lang, item.key) in self.validation_failures
):
continue
spans, texts, contexts = split(item.text)
if not texts:
continue # prose that could not be located for splicing
original = item.text
if item.kind == "title" and item.context:
# A title's surround is the article's opening prose
# (TransItem.context), not its own one-word block.
contexts = [item.context] * len(texts)
job = Job(
lang=lang,
key=item.key,
texts=texts,
path=item.path,
kind=item.kind,
contexts=contexts,
)
break
if job is not None:
break
if job is not None:
break
if job is None:
+80 -23
View File
@@ -372,8 +372,8 @@ def _layout(
doc.meta(property=key, content=value)
else:
doc.meta(name=key, content=value)
# A custom favicon (from the site editor) is linked explicitly; without
# one, browsers fall back to the build's /favicon.ico by convention.
# A custom favicon (from the site editor) is linked explicitly;
# /favicon.ico redirects to the same store file for non-HTML contexts.
if favicon:
doc.link(rel="icon", href=f"/_f/{favicon}", id="pagerite-favicon")
# Asset URLs for the on-demand bundles (editor, analytics) for
@@ -385,10 +385,14 @@ def _layout(
# carries the on-demand URLs in one JSON script instead.
vite_url = os.environ.get("PAGERITE_VITE_URL")
editor_scripts, editor_css = _editor_assets()
langselect_scripts, langselect_css = _langselect_assets()
config = {
"pagerite:editor-src": editor_scripts[-1],
"pagerite:analytics-src": _analytics_assets()[0][0],
"pagerite:langselect-src": langselect_scripts[-1],
}
if langselect_css:
config["pagerite:langselect-css"] = ",".join(langselect_css)
if editor_css:
config["pagerite:editor-css"] = editor_css
if vite_url:
@@ -783,7 +787,11 @@ def page_content(
node = resolve(menu, path)[-1]
content = node_markdown(data, node) or ""
title = node.title
# The original text pins the section anchors: on a translated page the
# heading slugs (and thus #hash URLs) stay in the original language.
anchors_from = None
if translation:
anchors_from = (content, title)
if translation.markdown is not None:
content = translation.markdown
title = (
@@ -793,7 +801,9 @@ def page_content(
)
# The title is injected into the markdown (as # title when it has no
# h1 of its own), so title and content render as one article.
rendered = render(content, path, node.created, node.modified, title=title)
rendered = render(
content, path, node.created, node.modified, title=title, anchors_from=anchors_from
)
# Long articles get .multicol: the article column cap lifts (see the
# #content grid in pagerite.css) and the .cols segments lay out in at
# most two columns. The html is already segmented by render() — the
@@ -1011,6 +1021,39 @@ def _social_meta(
}
def _language_urls(
data: Data,
path: str,
node: Node,
lang: str,
original: str,
base_url: str,
) -> tuple[str, list[tuple[str, str]]]:
"""(canonical, hreflang alternates) for a page (docs/localization.md).
The canonical names the actually served language the plain URL for
the original (for SEO the non-query URL means the article's language),
?lang= for a translation regardless of how the language was arrived
at (query or header). The alternates list the languages the page is
actually available in (``node.langs``; a category label's title counts
as its content): x-default first (the plain, autodetecting URL), then
every available language the original again by its plain URL,
translations by ?lang=. The public language selector keys off these.
("", []) without a base_url.
"""
if not base_url:
return "", []
url = f"{base_url}/{path}"
canonical = url if lang == original else f"{url}?lang={lang}"
alternates = []
if data.translate_langs:
alternates = [("x-default", url)] + [
(tag, url if tag == original else f"{url}?lang={tag}")
for tag in sorted({original, *node.langs})
]
return canonical, alternates
def render_page(
menu: dict[str, Node],
data: Data,
@@ -1040,24 +1083,7 @@ def render_page(
title = _title(path.rpartition("/")[2], node, translation, path)
main = page_content(menu, data, path, translation, link_lang, lang)
social = _social_meta(node, path, title, str(main), brand, base_url)
# Canonical/hreflang URLs (docs/localization.md): the canonical names
# the actually served language — the plain URL for the original (for
# SEO the non-query URL means the article's language), ?lang= for a
# translation — regardless of how the language was arrived at (query
# or header). The alternates are site-wide, the same set on every
# page: the configured translate_langs (the translator works to fill
# them all in), x-default first (the plain, autodetecting URL), then
# every language explicitly, the page's own primary included.
canonical = ""
alternates = []
if base_url:
url = f"{base_url}/{path}"
canonical = url if lang == original else f"{url}?lang={lang}"
if data.translate_langs:
alternates = [("x-default", url)] + [
(tag, f"{url}?lang={tag}")
for tag in sorted({original, *data.translate_langs})
]
canonical, alternates = _language_urls(data, path, node, lang, original, base_url)
return str(
_layout(
*_page_assets(),
@@ -1090,6 +1116,7 @@ def render_category(
theme: str = "",
favicon: str = "",
brand_html: str = "",
base_url: str = "",
transition: str = "cube",
lang: str = i18n.ORIGINAL_LANGUAGE,
translation: Translation | None = None,
@@ -1105,12 +1132,16 @@ def render_category(
With a translation (titles only the category has no Markdown) the
heading, navigation and card text localize per target article
(docs/localization.md); ``link_lang`` replicates the ?lang= override
onto the navigation links as on content pages.
onto the navigation links as on content pages. The hreflang alternates
are computed as on content pages a translated title makes the
language available here too.
"""
node = resolve(menu, path)[-1]
original = i18n.primary_lang(menu, path)
if translation is None:
lang = i18n.primary_lang(menu, path)
lang = original
title = _title(path.rpartition("/")[2], node, translation, path)
_, alternates = _language_urls(data, path, node, lang, original, base_url)
doc = E.article
with doc:
doc.h1(title)
@@ -1127,6 +1158,7 @@ def render_category(
transition,
favicon,
lang=lang,
alternates=alternates,
)(
Title=f"{title} {brand}" if brand else title,
Brand=_brand_link(brand, brand_html, link_lang),
@@ -1222,6 +1254,31 @@ def _analytics_assets() -> tuple[list[str], list[str]]:
return _asset_cache["analytics"]
def _langselect_assets() -> tuple[list[str], list[str]]:
"""Script and stylesheet URLs for the on-demand public language selector."""
vite_url = os.environ.get("PAGERITE_VITE_URL")
if vite_url:
return [f"{vite_url}/src/langselect-main.js"], []
if "langselect" not in _asset_cache:
manifest = _manifest()
# import() loads no CSS automatically: collect the stylesheets of
# the entry and its imported chunks (LangSelect's ride on the
# shared langs chunk).
scripts, stylesheets, seen = [], [], set()
queue = ["src/langselect-main.js"]
for key in queue: # grows with imported chunks
if key in seen:
continue
seen.add(key)
entry = manifest[key]
if entry.get("isEntry"):
scripts.append(f"/{entry['file']}")
stylesheets += [f"/{css}" for css in entry.get("css", [])]
queue += entry.get("imports", [])
_asset_cache["langselect"] = scripts, stylesheets
return _asset_cache["langselect"]
def render_analytics(
menu: dict[str, Node],
brand: str = SITE_NAME,