Compare commits

..
16 Commits
Author SHA1 Message Date
LeoVasanko 7724290921 analytics viewer: referer + UTM as one neutral-grey badge
Favicon flush to the badge's left edge, then the referer host and UTM
summary in one link, with a single newline-formatted tooltip (origin,
then each utm_* pair). Fixed --badge-bg/--badge-text/--badge-muted
palette (defined in pagerite.css, deliberately unthemed) so transparent
favicons stay legible in both light and dark themes; own-article trail
links remain plain, lifting referer info visually apart from them.
Replaces the outlined utm-tag pill.
2026-09-15 05:20:57 +00:00
LeoVasanko 3b1804b093 analytics: record and show the rendered content language of page views
GETs store the resolved page language, activity pings carry <html lang>
(including a new ping on in-place language switches, which also keeps the
switch's GET out of crawler classification), and trail items/crawler hits
surface it. The viewer shows flags discreetly: nothing on single-language
sites or primary-language-only rows, one leading flag for uniform visits,
transition flags on mixed trails, a flag row per crawler.
2026-09-15 05:20:57 +00:00
LeoVasanko 4cd8dbfc73 Update fastapi-vue-setup 1.6.1 2026-09-15 05:20:57 +00:00
LeoVasanko dc4bdf6efc Avoid deprecation warning from pyvips; remove unnecessary logging config. 2026-09-15 05:20:57 +00:00
LeoVasanko a458bded09 Create the per-hostname data directory on first run 2026-09-15 05:20:57 +00:00
LeoVasanko 3a4745389e analytics viewer: copy-to-clipboard UAs, crawler info links, shared icon-btn
- UA lines click to copy the raw UA; abuse rows with rotating clients
  copy every variation, one per line (uaRaws)
- "Copied!" popup is viewport-fixed at the click point; cell overflow
  clipped the old absolute one
- language tags show their Intl.DisplayNames full name on hover
- a 🔗 link after the pretty UA opens the crawler info page uarite
  provides
- the emoji-symbol dim-until-hover idiom (opacity 0.7 -> 1) is now one
  global .icon-btn class in pagerite.css, shared by the edit pens,
  auth links, editor icon buttons and the new UA link
2026-09-09 00:55:44 +00:00
LeoVasanko 4eea3ae3af analytics: parse UAs at display time, never store ua_pretty
The stored ua_pretty froze each record at the uarite version of its
record time (old records showed disguised Meta crawlers as
"Chrome/145 Windows").  Client now carries a display-time-only
"uarite" field holding the full uarite.UA dataclass (pretty/engine/
os/provider/kind/url), filled when the viewer payload is built, so old
data always follows the current uarite.  uarite 0.2.0 fixes the Meta
misdetection itself; _compact_user_agent is gone, tracking.py calls
uaparse directly.
2026-09-09 00:11:41 +00:00
LeoVasanko 69583fb9fe Replace ua-parser with uarite for UA formatting and bot detection 2026-09-05 06:15:43 +00:00
LeoVasanko b57b7060ec Keep container fence lines out of prose chunks
A closing ::: glued to a paragraph (no blank line before it) rode inside
the prose chunk and crossed to the translator as part of the text run;
when the model dropped it, validation passed and the splice lost the
fence — the rest of the page rendered inside the container (seen in the
Spanish translation). Container fence lines (::: openers and closers
alike) are now always their own prose-free chunk, never reaching the
translator. Affected pages re-chunk on next save and re-translate under
the new hashes, repairing themselves.
2026-09-04 19:04:24 +00:00
LeoVasanko 3f27a0a292 Fix unstyled editor language selector, order all language menus logically
LangSelect's scoped CSS landed on the shared store chunk, whose stylesheet
the editor never loaded (only the public selector path injected it), so the
editor's language selector rendered unstyled on untranslated pages. Collect
editor stylesheets from the entry's imported chunks too (same traversal as
the langselect assets).

Also: hreflang alternates now skip languages disabled site-wide, and all
selectors (page editor, structure tab, public selector) order languages the
same way — primary first, then the lang tab's geographic grouping.
2026-09-04 18:52:22 +00:00
LeoVasanko b30d909a23 Don't extract GeoIP .mmdb.gz on filesystem, only in RAM. 2026-09-04 18:30:55 +00:00
LeoVasanko 13fecd2118 Hreflang alternates on category placeholder pages too 2026-09-04 18:12:48 +00:00
LeoVasanko 11a138e19f Translate category-label titles, not just page titles 2026-09-04 18:09:40 +00:00
LeoVasanko ebd5911a38 Store DBIP database in folder where the program is ran, not where it is installed. 2026-09-04 18:04:30 +00:00
LeoVasanko db57125953 Public language selector, linked with the editor's language selection. Shared popup menu operation with edit toolbar with consistent closing logic. 2026-09-04 17:52:51 +00:00
LeoVasanko 986e28c220 Serve /favicon.ico as a redirect to the configured site icon 2026-09-04 14:59:38 +00:00
42 changed files with 1224 additions and 486 deletions
+4 -2
View File
@@ -25,10 +25,12 @@ Pagerite is a CMS. See `docs` for the full design and implementation details. Ke
- `markdown.py` — markdown-it-py renderer.
- `views.py` — shared page layout and rendering; theme/user-font resolution across `THEME_DIRS` / `FONT_DIRS` (cwd, site, platform data roots, then built-in `pagerite/themes/`, see `docs/themes-and-assets.md`).
- `seed.py` — demo content, written only on first database creation.
- `analytics.py` — visit analytics collection (see `docs/analytics.md`).
- `analytics.py` — visit analytics collection (see `docs/analytics.md`). UA formatting/bot detection comes from the **uarite** package.
- `frontend/src/` — Vue editor and public-page JS entries.
- `main.js` — Vue editor app entry.
- `analytics-main.js` — analytics page entry (mounts `AnalyticsView` at `/_a`).
- `langselect-main.js` + `LangSelector.vue` — public language selector, imported on demand by pagerite.js on pages with more than one hreflang alternate (the editors' `LangSelect` flag dropdown).
- `store.js` — the shared Pinia store (`useStore`, id `pagerite`) for cross-bundle UI state.
- `pagerite.js` — public page entry.
- `editorLang.js` + `LangSelect.vue` — the editor shell's shared language selection and its selector component (page + structure tabs; drives the page preview while the panel is open, via `swapdoc.setLangOverride`).
- `reconnect.js` — shared WebSocket pacing for all sockets (staggered connect slots, stuck-CONNECTING watchdog, exponential backoff): bursts and rapid retries trip the browser's WebSocket throttling.
@@ -64,6 +66,6 @@ Server run by CLI entry point `uv run pagerite` (no auto reloads, build needed).
## Conventions
- Keep dependencies minimal; add via `uv add` and mention it.
- The public URL space belongs to content (pretty slugs at root). Reserve only `/_` for the machinery (`/_api/`, `/_f/`, `/_assets/`), plus `/favicon.ico` from the build. Slugs are lowercase ASCII letters, digits, hyphens and underscores `[a-z0-9_-]` (the site editor filters input live via `slugify.js`, built on the `transliteration` npm package — unicode folds to ASCII, spaces become hyphens; an empty slug on a new page is derived from its title), may not begin with `_` or `.`, and such URLs are never looked up as content.
- The public URL space belongs to content (pretty slugs at root). Reserve only `/_` for the machinery (`/_api/`, `/_f/`, `/_assets/`), plus `/favicon.ico` (backend redirect to the configured site icon). Slugs are lowercase ASCII letters, digits, hyphens and underscores `[a-z0-9_-]` (the site editor filters input live via `slugify.js`, built on the `transliteration` npm package — unicode folds to ASCII, spaces become hyphens; an empty slug on a new page is derived from its title), may not begin with `_` or `.`, and such URLs are never looked up as content.
- No auth in core code; the SSO/reverse proxy gates all of `/_api` (forward-auth) and owns `/auth/` (login/logout, session validation). Pages render identically for everyone; pagerite.js adds the editing UI only after the auth server validates the session. The one keyed exception is `/_translate/{key}` (translator service; `Data.translate_keys`, see docs/localization.md).
- Update the relevant MarkDown files when architecture, tooling, or conventions change.
+49 -13
View File
@@ -38,6 +38,9 @@ Each `Get` record (one per served document):
(`x-pagerite-preload` header): never counted as a view, crawler hit or
abuse — recorded only so a navigation later served from the in-memory page
cache (which issues no GET at all) can be attributed this GET's status,
- `lang` — rendered content language of the served document (the resolved
language of a localized page), `""` for non-localized responses (404
probes, reserved paths),
- `client` — 6-byte blake3 hash referencing `Analytics.clients`.
304 revalidation responses return before recording and are not logged.
@@ -51,7 +54,9 @@ Each `Msg` record (one per pagerite.js activity message over `/_ws`):
- `to` — navigation target (validated at record time: internal slug path or
external https URL; anything else is dropped — sanitation, not
classification),
- `read` — active seconds spent on `fr` since the previous report.
- `read` — active seconds spent on `fr` since the previous report,
- `lang` — rendered language reported by the client for the page the
activity happened on (the page's `<html lang>`; `""` from old clients).
Each `Client` record (shared by every event, keyed by hash):
@@ -63,20 +68,35 @@ Each `Client` record (shared by every event, keyed by hash):
when a database is available,
- `city` — city name from the DB-IP MMDB lookup, when available,
- `ua` — raw `User-Agent` string,
- `ua_pretty` — compact display form of the UA (browser/OS/device) when
parsable, otherwise the raw string,
- `hide` — true for admin clients (`hide` message field): everything this
client ever did is recorded but excluded from every statistic and from the
viewer payload. This is the one flag set at record time — it is a client
property, not a classification.
The viewer payload adds one display-time field to each client, never
persisted (stored records keep the default and old data always follows the
current uarite version):
- `uarite` — the `uarite.UA` dataclass from parsing the raw UA
(`pretty`/`engine`/`os`/`provider`/`kind`/`url`): the crawler name for
bots,
with a category suffix only where a provider runs crawlers of more than
one kind (`GPTBot (AI)` vs `OAI-SearchBot (search)`, `Googlebot (search)`
vs `Google-Extended (AI)`; single-kind providers stay plain: `Facebook`,
`WhatsApp`), `Browser/major OS` on the desktop, the device where that is
the relevant information (iPhone reports its iOS version, Android phones
their model instead of the OS), otherwise the raw string; `url` is the
crawler's info page when uarite knows one (rendered as a 🔗 link after the
pretty UA in the viewer), `kind` drives the bot classification.
A reverse-DNS lookup is attempted for each new client and the result, when
available, is stored as `host`; local/reserved/multicast addresses are
skipped. If a DB-IP MMDB file (`dbip-*.mmdb` or `dbip-*.mmdb.gz`) is present
in the repository root, it is loaded at startup and used to look up
in the working directory, it is loaded at startup and used to look up
`country`/`city`. These lookups run in background tasks after the event is
stored, so WebSocket message handling is never delayed. The decompressed
`dbip-*.mmdb` file is kept in the repository root and ignored by git. The
stored, so WebSocket message handling is never delayed. Only the downloaded
`.mmdb.gz` is kept on disk (in the working directory, ignored by git); it is
decompressed into RAM when opened. The
CLI flag `--dbip` (`uv run pagerite --dbip`) downloads the latest
`dbip-city-lite-YYYY-MM.mmdb.gz` from DB-IP at startup (in the app lifespan,
before the MMDB is opened), skipping the download when the local database is
@@ -89,7 +109,9 @@ The client (`pagerite.js`) keeps a WebSocket connection to `/_ws` for the
whole browsing session and sends activity messages over it — JSON text
frames matching the server's `Ping` msgspec struct with the fields `fr`
(source path), `to` (navigation target), `read` (active seconds on `fr`
since the last report) and `hide`; falsy fields are omitted. One channel
since the last report), `lang` (the rendered language of the page the
activity happened on — its `<html lang>`) and `hide`; falsy fields are
omitted. One channel
follows the session, so the activity of a visit stays tied together, and
while the user is active the accumulated reading time is flushed every few
seconds: the times are incremental, so a disconnection simply leaves the
@@ -158,17 +180,25 @@ for misses.
(`_SESSION_GAP`). A fresh page load with an already-open visit (second
tab) extends it, logging a `(direct)` transition. The visit's trail holds
first-seen targets in order; `read` updates accumulate active seconds on
the trail item matching `fr`. Each trail item's HTTP status comes from
the trail item matching `fr` (preferring the item whose language matches
the report, so seconds after a language switch land on the new-language
step). Each trail item's HTTP status comes from
the client's latest GET for that path — preloads included, which is what
allows 404 pages to render red in the viewer even when the navigation
itself was served from the page cache. The entry page's referer and
itself was served from the page cache. Each trail item also carries the
rendered language: the client's report, for the entry page falling back
to its GET's rendered language (old clients don't send one); a page
re-visited in a different language becomes a distinct trail step instead
of merging into the existing item. The entry page's referer and
`utm_*` tags come from the GET that loaded it (within 10 s before the
first message).
- **Crawler hits**: a document GET no activity message matched within
`_CRAWLER_TIMEOUT` (10 s) is a crawler hit — plain bots that only fetch
documents never register as visits. JS-running crawlers (Googlebot,
GoogleOther, Applebot, ...) do connect and send messages, but their UA
gives them away (`_is_bot_ua`): their messages are ignored at display
gives them away (`_is_bot_ua`, backed by `uarite.uaparse` — which
also knows the disguised ones: facebookexternalhit, Google-Extended,
WhatsApp, ...): their messages are ignored at display
time, so their GETs never match and land in the crawler list too. Real-
browser bots whose UA does not match are caught by engagement: a visit
whose total reported reading time is under 5 seconds (`_MIN_VISIT_READ`;
@@ -215,6 +245,10 @@ to tell misses from real pages at a glance.
The `Display` payload contains the derived `visits`, `crawlers` and `abuse`
rows (structs `Visit`/`Nav`/`TrailItem`, `CrawlerHit`, `AbuseHit` — display
DTOs only, never persisted), the visible `clients`, the fetched `favicons`,
the site language context (`multilingual` — translation languages are
configured, so the viewer can suppress language UI on single-language
sites — and `primary_lang` — the front page's primary language, so the
viewer can skip the primary-language default case),
and the aggregates below.
Each derived `Visit`:
@@ -226,8 +260,9 @@ Each derived `Visit`:
- `trail` — the entry page and everything seen afterwards, keyed by the
timestamp of first sight (insertion order = first-seen order). Each item
holds `to` (page path or external exit URL), the accumulated active
reading time in seconds (`read`) and the most recent HTTP status seen
for the target (`status`),
reading time in seconds (`read`), the most recent HTTP status seen
for the target (`status`) and the rendered language (`lang`; a page
seen in two languages within one visit gets one item per language),
- `navs` — every navigation (`fr`, `to`), keyed by its timestamp, repeats
included. The aggregates are computed from this log,
- `utm``utm_*` query parameters from the landing URL, as a dict.
@@ -240,7 +275,8 @@ Each derived `CrawlerHit`:
- `referer` — external https origin of the request, `""` for direct/none,
- `query` — raw query string of the request,
- `status` — HTTP status of the served response (200 for a real page, 404
for a category placeholder or missing page).
for a category placeholder or missing page),
- `lang` — rendered content language of the served document (from the GET).
Each derived `AbuseHit`:
+20 -7
View File
@@ -53,11 +53,13 @@ Region tags normalize to their base subtag (`fi-FI` → `fi`).
article's own language), `?lang=xx` when serving a translation — however
the language was arrived at (query or header).
- `<link rel="alternate" hreflang="…">` entries follow the canonical
directly (before the social meta tags) and are the same set on every
page — the site-wide configured languages (`translate_langs`, which the
translator works to fill in): `x-default` first, pointing at the plain
autodetecting URL, then every language explicitly with `?lang=`, the
page's own primary language included.
directly (before the social meta tags) and list the languages the page
is **actually available in**: `x-default` first, pointing at the plain
autodetecting URL, then every available language — the original again by
its plain URL, translations by `?lang=`. The public language selector
keys off these: pagerite.js mounts the editors' flag dropdown in the
top-right corner when the head advertises x-default plus more than one
language, loading its bundle (Vue + the flag SVG set) on demand.
- The override sticks for the session of clicks: a page requested with
`?lang=` replicates the query onto the navigation links it renders (nav,
sidebar, cards, brand — in-article links are content and stay as
@@ -66,6 +68,11 @@ Region tags normalize to their base subtag (`fi-FI` → `fi`).
`history.replaceState` (pretty, shareable URLs), remembers the language,
and adds it to every internal fetch that lacks one (preloads,
fetch-navigations, history traversals); history entries stay query-less.
- The public selector's pick is the same override, pure JS state
(`pagerite:set-session-lang`): the session language changes and the page
swaps in place — no `?lang=` in the address bar, no reload. The choice is
linked with the editor panel's language dropdown both ways; closing the
panel keeps the chosen language instead of reverting.
- A full page refresh or a shared link resets to automatic selection (header
only). This gives a clean one-time override without cookies.
@@ -99,7 +106,9 @@ Region tags normalize to their base subtag (`fi-FI` → `fi`).
language like content pages, but over the **subtree's** combined
availability (`subtree_languages`) — they have no chunks of their own;
the heading, navigation and card text localize from the title map and
the target articles' translations.
the target articles' translations. Their hreflang alternates are
computed exactly like a content page's (a translated title counts as
availability, so the language selector is offered there too).
- Card descriptions and cover picks run on the target article's hybrid
Markdown where that page is available in the served language, with
per-card fallback to the original.
@@ -137,7 +146,11 @@ served Markdown at render time.
`chunk_markdown(markdown)` splits the source into block-level chunks —
blank-line-separated blocks: headings, paragraphs, code fences (kept whole),
list blocks, tables, HTML blocks. A chunk's identity is its **source text**,
list blocks, tables, HTML blocks. Container fence lines (`::: name` openers
and `:::` closers) are always their own chunk, blank lines or not — folded
into a prose chunk the closer would cross to the translator as part of the
text, where the model can drop it (the rest of the page then renders inside
the container). A chunk's identity is its **source text**,
gettext-msgid style:
```python
+1
View File
@@ -20,6 +20,7 @@
"codemirror": "^6.0.2",
"country-flag-icons": "^1.6.20",
"overlayscrollbars": "^2.16.0",
"pinia": "^4.0.3",
"transliteration": "^2.6.1",
"vue": "^3.5.26",
"vuedraggable": "^4.1.0"
+40 -30
View File
@@ -23,6 +23,7 @@ import {
formatVisitRows,
} from './analytics/format.js'
import TrailLink from './TrailLink.vue'
import RefererBadge from './RefererBadge.vue'
import VisitorCell from './VisitorCell.vue'
import TransitionGraph from './TransitionGraph.vue'
import VisitorCharts from './VisitorCharts.vue'
@@ -160,9 +161,15 @@ watch(range, (r) => {
const clients = computed(() => data.value?.clients || {})
const favicons = computed(() => data.value?.favicons || {})
const visitRows = computed(() => formatVisitRows(visits.value, clients.value, pageTree.value, now.value))
// Site language context from the payload: drives the discreet rendered-
// language markers in the visit/crawler rows (multilingual sites only).
const site = computed(() => ({
multilingual: !!data.value?.multilingual,
primaryLang: data.value?.primary_lang || '',
}))
const visitRows = computed(() => formatVisitRows(visits.value, clients.value, pageTree.value, now.value, site.value))
const crawlers = computed(() => rangeData.value?.crawlers || [])
const crawlerRows = computed(() => formatCrawlerRows(crawlers.value, clients.value, pageTree.value, now.value))
const crawlerRows = computed(() => formatCrawlerRows(crawlers.value, clients.value, pageTree.value, now.value, site.value))
const abuseRows = computed(() => formatAbuseRows(rangeData.value?.abuse || [], clients.value, pageTree.value, now.value))
</script>
@@ -209,15 +216,16 @@ const abuseRows = computed(() => formatAbuseRows(rangeData.value?.abuse || [], c
<tbody>
<tr v-for="(v, i) in visitRows" :key="i">
<td class="trail">
<TrailLink v-if="v.refererStep" :step="v.refererStep" :favicons="favicons" @close="$emit('close')" />
<span v-if="v.utm && v.utm !== '—'" class="utm-tag small muted" :title="v.utmTitle">{{ v.utm }}</span>
<TrailLink v-for="(s, si) in v.trail" :key="si" :step="s" :favicons="favicons" @close="$emit('close')" />
<RefererBadge v-if="v.refererBadge" :badge="v.refererBadge" :favicons="favicons" />
<span v-if="v.rowFlag" class="flag" v-html="v.rowFlag" :title="v.rowFlagTitle"></span>
<TrailLink v-for="(s, si) in v.trail" :key="si" :step="s" :favicons="favicons" :flags="s.langFlags" @close="$emit('close')" />
</td>
<VisitorCell
:ip="v.ip"
:ip-display="v.ipDisplay"
:ua="v.ua"
:ua-raw="v.uaRaw"
:ua-url="v.uaUrl"
:country="v.country"
:city="v.city"
:lang="v.lang"
@@ -245,14 +253,16 @@ const abuseRows = computed(() => formatAbuseRows(rangeData.value?.abuse || [], c
<tbody>
<tr v-for="(c, i) in crawlerRows" :key="i">
<td class="trail">
<TrailLink v-if="c.refererStep" :step="c.refererStep" :favicons="favicons" @close="$emit('close')" />
<RefererBadge v-if="c.refererBadge" :badge="c.refererBadge" :favicons="favicons" />
<TrailLink v-for="(s, si) in c.pages" :key="si" :step="s" :count="s.count" @close="$emit('close')" />
<span v-for="(f, fi) in c.readFlags" :key="fi" class="flag" v-html="f.flag" :title="f.name"></span>
</td>
<VisitorCell
:ip="c.ip"
:ip-display="c.ipDisplay"
:ua="c.ua"
:ua-raw="c.uaRaw"
:ua-url="c.uaUrl"
:country="c.country"
:city="c.city"
:lang="c.lang"
@@ -299,6 +309,8 @@ const abuseRows = computed(() => formatAbuseRows(rangeData.value?.abuse || [], c
:ip-display="a.ipDisplay"
:ua="a.ua"
:ua-raw="a.uaRaw"
:ua-url="a.uaUrl"
:ua-raws="a.uaRaws"
:country="a.country"
:city="a.city"
:lang="a.lang"
@@ -468,16 +480,30 @@ const abuseRows = computed(() => formatAbuseRows(rangeData.value?.abuse || [], c
color: var(--error, #c00);
}
.visit-table .utm-tag {
display: inline-block;
/* The referer badge outgrows the 8rem trail-link cap (it carries the UTM
summary too); keep the inline-flex layout from the component. */
.visit-table .trail a.referer-badge {
display: inline-flex;
max-width: 100%;
padding: 0.05rem 0.4rem;
border: 1px solid var(--line);
border-radius: 0.25rem;
white-space: nowrap;
}
/* Same flag chip as the visitor cells (VisitorCell.vue); the flags here
mark the language the page was read in. */
.visit-table .flag {
display: inline-flex;
width: 18px;
height: 12px;
border-radius: 2px;
overflow: hidden;
text-overflow: ellipsis;
vertical-align: bottom;
border: 1px solid var(--line);
box-shadow: 0 0 0 1px rgba(0, 0, 0, 0.2) inset;
vertical-align: middle;
}
.visit-table .flag :deep(svg) {
width: 100%;
height: 100%;
display: block;
}
.visit-table .clickable-list {
@@ -506,22 +532,6 @@ const abuseRows = computed(() => formatAbuseRows(rangeData.value?.abuse || [], c
.visit-table .clickable-list,
.visit-table .last-seen {
cursor: pointer;
position: relative;
}
.visit-table :deep(.copy-popup) {
position: absolute;
bottom: calc(100% + 0.25rem);
left: 50%;
transform: translateX(-50%);
padding: 0.15rem 0.4rem;
background: var(--text, CanvasText);
color: var(--bg, Canvas);
border-radius: 0.25rem;
font-size: 0.75rem;
white-space: nowrap;
pointer-events: none;
z-index: 10;
}
.crawler-top-uas {
-8
View File
@@ -403,14 +403,6 @@ onUnmounted(() => {
margin-left: auto;
padding: 0 0.2rem;
font-size: 1rem;
background: none;
border: none;
cursor: pointer;
opacity: 0.7;
}
.block-head .icon-btn:hover {
opacity: 1;
}
/* The banner design selector stays compact; the upload button is pushed
+18 -9
View File
@@ -21,11 +21,11 @@ const currentPath = ref(props.pagePath)
const activeMode = ref(props.initialMode)
// The shared language selection (./editorLang, v-modeled by the tabs'
// LangSelects) also drives the page preview: while the shell is open it
// overrides the normal language preferences (?lang= / Accept-Language),
// so the page renders in the language being edited; closing restores.
// The primary selection pins by the CURRENT PAGE's own primary language
// (pages may differ — Node.language is inherited down the tree).
// LangSelects) is linked to the whole-page language: while the shell is
// open it drives the page preview (overrides ?lang= / Accept-Language),
// and closing keeps the pick as the session language. The primary
// selection pins by the CURRENT PAGE's own primary language (pages may
// differ — Node.language is inherited down the tree).
let pinned = false
function pinPreviewLang() {
pinned = true
@@ -34,6 +34,15 @@ function pinPreviewLang() {
setLangOverride(editorLang.value || pagePrimary.value || 'en')
loadPlain(currentPath.value)
}
// Opening the panel must not switch the page's language: adopt the
// session's chosen language (public selector / earlier pick) once, then
// pin. Runs only on (re)open — after that the selection is the user's.
function openShell() {
const session = window.__pageriteLang
if (!editorLang.value && session && session !== (pagePrimary.value || 'en'))
editorLang.value = session
pinPreviewLang()
}
function unpinPreviewLang() {
if (!pinned) return
pinned = false
@@ -89,13 +98,13 @@ function onSwitchEvent(ev) {
onMounted(() => {
document.body.dataset.editorMode = activeMode.value
addEventListener('pagerite:switch-editor', onSwitchEvent)
addEventListener('pagerite:editor-shown', pinPreviewLang)
addEventListener('pagerite:editor-shown', openShell)
addEventListener('pagerite:editor-hidden', unpinPreviewLang)
// The shell mounts visible (openEditor), so pin immediately. The site
// The shell mounts visible (openEditor), so open immediately. The site
// default primary language comes from the settings — it only fills the
// unknown; the page/structure tabs refine pagePrimary per page as they
// learn it (their knowledge is strictly better).
pinPreviewLang()
openShell()
fetch('/_api/settings').then((r) => r.json()).then((s) => {
if (!pagePrimary.value) pagePrimary.value = s.primary_lang || 'en'
}).catch(() => { /* keep the fallback */ })
@@ -103,7 +112,7 @@ onMounted(() => {
onUnmounted(() => {
removeEventListener('pagerite:switch-editor', onSwitchEvent)
removeEventListener('pagerite:editor-shown', pinPreviewLang)
removeEventListener('pagerite:editor-shown', openShell)
removeEventListener('pagerite:editor-hidden', unpinPreviewLang)
})
</script>
+30 -9
View File
@@ -3,7 +3,8 @@
// flag button opening a clean dropdown, v-modeled on the shared editorLang
// ('' = the primary language). The lang tab's flag grid is a different
// control (toggles, not a select) and stays as it is.
import { computed, ref } from 'vue'
import { computed, nextTick, ref } from 'vue'
import { usePopup } from './dropdown'
const props = defineProps({
modelValue: { type: String, default: '' },
@@ -13,8 +14,12 @@ const props = defineProps({
const emit = defineEmits(['update:modelValue'])
const open = ref(false)
const root = ref(null)
const toggleBtn = ref(null)
const pop = ref(null)
const popStyle = ref({})
// Closes on outside click / Escape (./dropdown), not on mouseleave.
usePopup(open, root)
const current = computed(
() => props.options.find((o) => o.tag === props.modelValue) ?? props.options[0],
)
@@ -26,6 +31,17 @@ function toggle() {
// onto the page area instead of being clipped by it.
const r = toggleBtn.value.getBoundingClientRect()
popStyle.value = { top: `${r.bottom + 2}px`, left: `${r.left}px` }
// A toggle mounted near the right window edge (the public page
// selector sits top-right) opens the popup flush against that edge.
nextTick(() => {
const p = pop.value?.getBoundingClientRect()
if (p && p.right > innerWidth - 4) {
popStyle.value = {
...popStyle.value,
left: `${Math.max(4, innerWidth - 4 - p.width)}px`,
}
}
})
}
}
@@ -36,7 +52,7 @@ function select(tag) {
</script>
<template>
<span v-if="options.length > 1" class="lang-select">
<span v-if="options.length > 1" ref="root" class="lang-select">
<button
ref="toggleBtn"
type="button"
@@ -47,7 +63,7 @@ function select(tag) {
: '')"
@click="toggle"
><span v-if="current?.flag" class="flag" v-html="current.flag" /></button>
<span v-if="open" class="lang-pop" :style="popStyle" @mouseleave="open = false">
<span v-if="open" ref="pop" class="lang-pop" :style="popStyle">
<button
v-for="o in options"
:key="o.code"
@@ -66,20 +82,23 @@ function select(tag) {
display: flex;
}
/* The closed state is just the small flag — no button chrome until hovered. */
/* The closed state is just the small flag — no button chrome at all, on
hover either (it sits among borderless emoji-icon buttons); like them it
rests dimmed and brightens on hover. */
.lang-current {
display: flex;
align-items: center;
padding: 2px;
background: none;
border: 1px solid transparent;
border: none;
border-radius: 4px;
cursor: pointer;
opacity: 0.7;
}
.lang-current:hover,
.lang-current.open {
border-color: var(--line);
opacity: 1;
}
/* The dropdown matches the page's existing popups (.picker-pop look).
@@ -127,11 +146,13 @@ function select(tag) {
color: var(--muted);
}
/* Flags render like in the analytics visitor cells. */
/* em-sized so the chip matches the surrounding text/icon size in each
context; the hairline border delineates white-flagged countries (not
button chrome). */
.flag {
display: inline-flex;
width: 18px;
height: 12px;
width: 1.5em;
height: 1em;
flex: 0 0 auto;
border-radius: 2px;
overflow: hidden;
+46
View File
@@ -0,0 +1,46 @@
<script setup>
// The public page's language selector: the editors' flag dropdown
// (LangSelect) as the first item of the banner's corner container, fed
// from the shared store (pagerite.js sets the page's hreflang alternates
// and served language per navigation). It binds the same store.lang the
// editor's dropdown binds, so both always show the same selection. A pick
// also dispatches pagerite:set-session-lang — pagerite.js swaps the page
// in place when the editor is closed (open, the editor reacts to the
// store and re-renders it).
import { computed } from 'vue'
import LangSelect from './LangSelect.vue'
import { flagFor, langName, langSort } from './langs'
import { useStore } from './store'
const store = useStore()
// The "(primary)" marker is admin-panel information; the public selector
// lists plain languages. Order: the primary language first, then the rest
// in the lang tab's geographic grouping (./langs langSort) — the head's
// hreflang order is just alphabetical.
const primaryTag = computed(() => store.langAlternates.find((a) => a.primary)?.tag ?? '')
const options = computed(() => {
const rest = langSort(
store.langAlternates.map((a) => a.tag).filter((t) => t !== primaryTag.value),
)
return [primaryTag.value, ...rest].filter(Boolean).map((tag) => ({
tag,
code: tag,
name: langName(tag),
flag: flagFor(tag),
primary: false,
}))
})
// The explicit pick, else the served language (header-autodetected pages
// may have neither), else the primary.
const model = computed(() => store.lang || store.servedLang || primaryTag.value)
function go(tag) {
store.lang = tag === primaryTag.value ? '' : tag
dispatchEvent(new CustomEvent('pagerite:set-session-lang', { detail: { lang: tag } }))
}
</script>
<template>
<LangSelect :model-value="model" :options="options" @update:model-value="go" />
</template>
+40 -14
View File
@@ -26,13 +26,14 @@
// renders the version being edited, whichever language the page itself
// was loaded in.
import { computed, onActivated, onMounted, onUnmounted, ref, watch } from 'vue'
import { usePopup } from './dropdown'
import { EditorView, basicSetup } from 'codemirror'
import { Compartment, EditorState } from '@codemirror/state'
import { keymap } from '@codemirror/view'
import { indentWithTab } from '@codemirror/commands'
import { markdown } from '@codemirror/lang-markdown'
import { cmHighlight, cmTheme } from './cmtheme'
import { flagFor, langName } from './langs'
import { flagFor, langName, langSort } from './langs'
import { editorLang, pagePrimary } from './editorLang'
import LangSelect from './LangSelect.vue'
import ConnNote from './ConnNote.vue'
@@ -120,11 +121,13 @@ function normPath(p) {
// localization settings tab).
// The picker's options: the primary language first, then the union of the
// page's translations and the site-wide configured targets, sorted.
// page's translations and the site-wide configured targets in the lang
// tab's geographic grouping (./langs langSort).
const langOptions = computed(() => {
const others = [...new Set([...siteLangs.value, ...pageLangs.value])]
.filter((l) => l && l !== primaryLang.value)
.sort()
const others = langSort(
[...new Set([...siteLangs.value, ...pageLangs.value])]
.filter((l) => l && l !== primaryLang.value),
)
return [primaryLang.value, ...others].map((code) => ({
tag: code === primaryLang.value ? '' : code,
code,
@@ -621,8 +624,15 @@ const TABLE_MAX_ROWS = 6
// Class pickers: popup listing the block class toggles (placement ↔︎,
// text size AA), closed after applying. The block's current class of the
// group is marked; choosing "normal" (or the current class) removes it.
// All popups share the close behavior of ./dropdown (outside click /
// Escape; never mouseleave).
const classPicker = ref(null) // 'place' | 'size' | null
const activeClasses = ref(new Set())
const placeRoot = ref(null)
const sizeRoot = ref(null)
const tableRoot = ref(null)
usePopup(classPicker, computed(() => (classPicker.value === 'place' ? placeRoot : sizeRoot).value))
usePopup(tablePicker, tableRoot)
function openClassPicker(which) {
classPicker.value = classPicker.value === which ? null : which
@@ -1136,15 +1146,31 @@ onUnmounted(() => {
<div class="format-bar">
<button type="button" class="code-btn" title="code — inline wrap, or a fenced block for line-spanning selections; click again to unwrap" @click="insertCode"><code>&lt;/&gt;</code></button>
<button type="button" title="link (toggle: click inside a link to unwrap it)" @click="insertLink">🔗</button>
<button
type="button"
title="table"
:class="{ active: tablePicker }"
@click="tablePicker = !tablePicker"
></button>
<span class="picker" ref="tableRoot">
<button
type="button"
title="table"
:class="{ active: tablePicker }"
@click="tablePicker = !tablePicker"
></button>
<div v-if="tablePicker" class="table-picker" @mouseleave="tableSize = { cols: 0, rows: 0 }">
<div class="tp-grid" :style="{ gridTemplateColumns: `repeat(${TABLE_MAX_COLS}, 1fr)` }">
<button
v-for="n in TABLE_MAX_COLS * TABLE_MAX_ROWS"
:key="n"
type="button"
class="tp-cell"
:class="{ on: tableSize.cols >= (n - 1) % TABLE_MAX_COLS + 1 && tableSize.rows >= Math.floor((n - 1) / TABLE_MAX_COLS) + 1 }"
@mouseenter="tableSize = { cols: (n - 1) % TABLE_MAX_COLS + 1, rows: Math.floor((n - 1) / TABLE_MAX_COLS) + 1 }"
@click="insertTable(tableSize.cols, tableSize.rows)"
/>
</div>
<div class="tp-size">{{ tableSize.cols || '' }} × {{ tableSize.rows || '' }}</div>
</div>
</span>
<button type="button" title="insert image (upload) — pasting works too" @click="fileInput.click()">🖼</button>
<button type="button" title="aside box (::: aside) — wraps the selection or the cursor's line; clicked inside one, removes it" @click="insertAside"></button>
<span class="picker">
<span class="picker" ref="placeRoot">
<button
type="button"
title="block placement class"
@@ -1166,7 +1192,7 @@ onUnmounted(() => {
</span>
<button type="button" title="bold" @click="wrapInline('**')"><b>B</b></button>
<button type="button" title="italic" @click="wrapInline('*')"><i>i</i></button>
<span class="picker">
<span class="picker" ref="sizeRoot">
<button
type="button"
title="text size class"
@@ -1368,7 +1394,7 @@ onUnmounted(() => {
.table-picker {
position: absolute;
top: 100%;
left: 6.5rem;
left: 0;
z-index: 20;
padding: 0.5rem;
background: var(--bg);
+68
View File
@@ -0,0 +1,68 @@
<script setup>
// Referer + UTM as one badge in the analytics visit/crawler tables: the
// referer's favicon flush on the left, its host as the link text, then the
// UTM summary smaller/muted inside the same badge. The whole badge links to
// the referer origin (external, new tab) and carries a single one-fact-
// per-line tooltip (badge.title: origin, then each utm pair) — no titles on
// the inner elements. Referers are external, so there is no close event.
import { computed } from 'vue'
const props = defineProps({
badge: { type: Object, required: true },
favicons: { type: Object, default: null },
})
const favicon = computed(() => (props.badge.origin ? props.favicons?.[props.badge.origin] : null))
</script>
<template>
<a class="referer-badge"
:href="badge.href || undefined"
:title="badge.title"
:target="badge.href ? '_blank' : undefined"
:rel="badge.href ? 'noopener' : undefined">
<img v-if="favicon" class="badge-favicon" :src="favicon" alt="" />
<span v-if="badge.label">{{ badge.label }}</span>
<small v-if="badge.utm" class="small">{{ badge.utm }}</small>
</a>
</template>
<style scoped>
/* Browser-chrome chip on a fixed neutral palette (--badge-* in
pagerite.css, deliberately unthemed): black-on-transparent and
white-on-transparent favicons both stay legible on it, and the text is
always dark regardless of the theme's link/text colors. Colors go on the
inner elements, so the theme's a / a:hover color rules (which target the
anchor) cannot cascade in. */
.referer-badge {
display: inline-flex;
align-items: center;
gap: 0.35em;
padding-right: 0.45em;
border-radius: 0.25rem;
background: var(--badge-bg);
color: var(--badge-text);
white-space: nowrap;
overflow: hidden;
}
/* Flush with the badge's top/left/bottom edges: a full-height square (the
badge has no padding on those sides), corners clipped by the badge's
overflow: hidden border-radius. */
.badge-favicon {
width: 1.5em;
height: 1.5em;
object-fit: cover;
flex: none;
}
.referer-badge > span,
.referer-badge > small {
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
}
.referer-badge > span { color: var(--badge-text); }
.referer-badge > small { color: var(--badge-muted); }
</style>
-8
View File
@@ -782,14 +782,6 @@ onUnmounted(() => {
margin-left: auto;
padding: 0 0.2rem;
font-size: 1rem;
background: none;
border: none;
cursor: pointer;
opacity: 0.7;
}
.block-head .icon-btn:hover {
opacity: 1;
}
.text-input {
+5 -4
View File
@@ -19,7 +19,7 @@ import { computed, inject, onActivated, onMounted, onUnmounted, provide, ref, wa
import StructureTree from './StructureTree.vue'
import LangSelect from './LangSelect.vue'
import { slugify } from './slugify'
import { flagFor, langName } from './langs'
import { flagFor, langName, langSort } from './langs'
import { editorLang, pagePrimary } from './editorLang'
import { dropPageCache, loadPlain } from './swapdoc'
@@ -40,9 +40,10 @@ const primaryLang = ref('en')
const siteLangs = ref([])
// The strip's options: the primary language first, then the configured
// translation targets (the lang tab manages that set).
// translation targets (the lang tab manages that set) in the lang tab's
// geographic grouping (./langs langSort).
const langOptions = computed(() =>
[primaryLang.value, ...siteLangs.value.filter((l) => l !== primaryLang.value)]
[primaryLang.value, ...langSort(siteLangs.value.filter((l) => l !== primaryLang.value))]
.map((code) => ({
tag: code === primaryLang.value ? '' : code,
code,
@@ -63,7 +64,7 @@ watch(lang, () => refreshPages())
// dropdown lists "inherit" first (naming what it resolves to), then every
// site language. Setting it on a section covers its whole subtree.
const rowLangChoices = computed(() =>
[primaryLang.value, ...siteLangs.value.filter((l) => l !== primaryLang.value)]
[primaryLang.value, ...langSort(siteLangs.value.filter((l) => l !== primaryLang.value))]
.map((code) => ({ tag: code, code, name: langName(code), flag: flagFor(code), primary: false })),
)
function rowLangOptions(el) {
+21
View File
@@ -6,6 +6,7 @@ const props = defineProps({
step: { type: Object, required: true },
count: { type: Number, default: 0 },
favicons: { type: Object, default: null },
flags: { type: Array, default: () => [] },
})
defineEmits(['close'])
@@ -38,6 +39,7 @@ const title = computed(() => {
<small v-if="count > 1" class="muted">{{ formatCount(count) }}×</small>
<img v-if="favicon" class="favicon" :src="favicon" alt="" />
<span>{{ step.slug }}</span>
<span v-for="(f, fi) in flags" :key="fi" class="flag" v-html="f"></span>
</a>
</template>
@@ -48,4 +50,23 @@ const title = computed(() => {
margin-right: 0.25em;
vertical-align: -0.1em;
}
/* Same flag chip as the visitor cells (VisitorCell.vue). */
.flag {
display: inline-flex;
width: 18px;
height: 12px;
margin-left: 0.25em;
border-radius: 2px;
overflow: hidden;
border: 1px solid var(--line);
box-shadow: 0 0 0 1px rgba(0, 0, 0, 0.2) inset;
vertical-align: middle;
}
.flag :deep(svg) {
width: 100%;
height: 100%;
display: block;
}
</style>
+23 -4
View File
@@ -4,15 +4,20 @@
// Clicking the IP copies the full address to the clipboard.
// ``variantCount`` overrides the UA line to warn when multiple client
// fingerprints share the same IP (e.g. a scanner rotating UAs).
// Clicking the UA line copies the raw UA(s) to the clipboard, one per line
// (``uaRaws`` carries every variation for multi-client IPs).
import { computed } from 'vue'
import * as flagSvgs from 'country-flag-icons/string/3x2'
import { copyIp, formatLang } from './analytics/format.js'
import { copyIp, copyList, formatLang } from './analytics/format.js'
import { langName } from './langs.js'
const props = defineProps({
ip: { type: String, default: '' },
ipDisplay: { type: String, default: '—' },
ua: { type: String, default: '' },
uaRaw: { type: String, default: '' },
uaRaws: { type: String, default: '' },
uaUrl: { type: String, default: '' },
country: { type: String, default: '' },
city: { type: String, default: '' },
lang: { type: String, default: '' },
@@ -26,6 +31,7 @@ const hasCity = computed(() => !!(props.city && props.city !== '—'))
const hasLocale = computed(() => hasCountry.value || hasCity.value)
const langValue = computed(() => props.langDisplay || formatLang(props.lang))
const showLang = computed(() => langValue.value && langValue.value !== '—')
const uaCopy = computed(() => props.uaRaws || props.uaRaw)
function flagSvg(code) {
return flagSvgs[code?.toUpperCase()] || ''
@@ -58,10 +64,16 @@ function countryName(code) {
</div>
<div class="visitor-row">
<div class="ua-line">
<small v-if="variantCount > 1" class="muted variant-hint">{{ variantCount }} client variations</small>
<small v-else class="muted" :title="uaRaw">{{ ua || '—' }}</small>
<small v-if="variantCount > 1" class="muted variant-hint clickable-ip"
:title="uaCopy"
@click="copyList(uaCopy, $event)">{{ variantCount }} client variations</small>
<small v-else class="muted clickable-ip" :title="uaRaw"
@click="copyList(uaCopy, $event)">{{ ua || '' }}</small><a v-if="uaUrl && variantCount <= 1"
class="ua-link icon-btn" :href="uaUrl"
target="_blank" rel="noopener noreferrer"
@click.stop>🔗</a>
</div>
<div v-if="showLang && variantCount <= 1" class="locale-lang"><small class="muted">{{ langValue }}</small></div>
<div v-if="showLang && variantCount <= 1" class="locale-lang"><small class="muted" :title="langName(lang)">{{ langValue }}</small></div>
</div>
</div>
</td>
@@ -121,6 +133,13 @@ function countryName(code) {
text-align: left;
}
.ua-link {
text-decoration: none;
font-size: 0.75em;
margin-left: 0.2em;
vertical-align: middle;
}
.locale-lang {
flex: 0 0 auto;
overflow: hidden;
+123 -41
View File
@@ -2,6 +2,7 @@
* Formatters and aggregators for summary sections: totals and the recent
* visit trail.
*/
import { flagFor, langName } from '../langs.js'
/**
* IPv4 unchanged, IPv6 returns the /64 network prefix in compact form.
@@ -25,32 +26,31 @@ export const hostIP = (ip) => {
}
}
function showCopiedFeedback(el) {
if (!el || typeof document === 'undefined') return
function showCopiedFeedback(el, event) {
if (typeof document === 'undefined') return
const popup = document.createElement('span')
popup.textContent = 'Copied!'
popup.className = 'copy-popup'
// Fixed to the viewport at the click point: table cells clip absolute
// popups with their overflow: hidden ellipsis styling.
const x = event?.clientX ?? 0
const y = event?.clientY ?? 0
popup.style.cssText =
'position:absolute;bottom:calc(100% + 0.25rem);left:50%;' +
'transform:translateX(-50%);padding:0.15rem 0.4rem;' +
`position:fixed;left:${x}px;top:${y}px;` +
'transform:translate(-50%, calc(-100% - 0.5rem));padding:0.15rem 0.4rem;' +
'background:var(--text, CanvasText);color:var(--bg, Canvas);' +
'border-radius:0.25rem;font-size:0.75rem;white-space:nowrap;' +
'pointer-events:none;z-index:10;'
el.classList.add('has-copy-popup')
el.appendChild(popup)
setTimeout(() => {
popup.remove()
el.classList.remove('has-copy-popup')
}, 1200)
'pointer-events:none;z-index:100;'
document.body.appendChild(popup)
setTimeout(() => popup.remove(), 1200)
}
/** Copy the full IP to the clipboard and show a brief "Copied!" popup. */
export async function copyIp(ip, event) {
if (!ip) return
const el = event?.currentTarget
try {
await navigator.clipboard.writeText(ip)
showCopiedFeedback(el)
showCopiedFeedback(event?.currentTarget, event)
} catch {
/* ignore */
}
@@ -59,10 +59,9 @@ export async function copyIp(ip, event) {
/** Copy arbitrary text to the clipboard and show a brief "Copied!" popup. */
export async function copyList(text, event) {
if (!text) return
const el = event?.currentTarget
try {
await navigator.clipboard.writeText(text)
showCopiedFeedback(el)
showCopiedFeedback(event?.currentTarget, event)
} catch {
/* ignore */
}
@@ -178,6 +177,32 @@ function stepOf(path, titles) {
return null
}
/**
* Badge data combining a visit's/crawler's external referer origin with the
* visit's UTM tags: the origin as the badge link/label (the favicon is
* looked up by origin in the component), the known UTM values as a short
* inline summary, and a one-fact-per-line tooltip — the full origin URL on
* the first line, then every ``utm_*=value`` pair. Null when there is no
* external referer and no UTM tag (a plain direct visit).
*/
function refererBadgeOf(referer, titles, utmTags = {}) {
const step = stepOf(referer, titles)
const external = step?.external ? step : null
const known = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content']
const utm = known.map((k) => utmTags[k]).filter(Boolean).join(' · ')
if (!external && !utm) return null
return {
href: external?.origin || '',
label: external?.slug || '',
origin: external?.origin || '',
utm,
title: [
...(external ? [external.origin] : []),
...Object.entries(utmTags).map(([k, value]) => `${k}=${value}`),
].join('\n'),
}
}
/**
* Human-readable relative timestamp. Adapted from cista-storage: uses
* ``Intl.RelativeTimeFormat`` for short intervals and a compact date for
@@ -342,7 +367,7 @@ export function countCrawlerUas(crawlers, clients) {
const counts = {}
for (const c of crawlers || []) {
const client = (clients || {})[c.client] || {}
const value = client.ua_pretty || client.ua || '(no UA)'
const value = client.uarite?.pretty || client.ua || '(no UA)'
counts[value] = (counts[value] || 0) + 1
}
return Object.entries(counts).sort((a, b) => b[1] - a[1])
@@ -370,12 +395,13 @@ export function mainDomain(host, limit = 24) {
/**
* Group raw crawler hits by client hash and format each group as a row showing
* every internal page that crawler visited. Rows are sorted by most recent hit
* first, with total hits as a tie-breaker. The group's ``refererStep`` is the
* latest external referer seen for the crawler — spiders often advertise
* their own site there — rendered with its favicon like visit referers.
* first, with total hits as a tie-breaker. The group's ``refererBadge`` is
* the latest external referer seen for the crawler — spiders often advertise
* their own site there — rendered as a badge with its favicon like visit
* referers.
* ``clients`` maps client hashes to client records.
*/
export function formatCrawlerRows(crawlers, clients, pageTree, now = Date.now()) {
export function formatCrawlerRows(crawlers, clients, pageTree, now = Date.now(), site = { multilingual: false, primaryLang: '' }) {
const titles = buildTitleMap(pageTree)
const groups = new Map()
for (const c of crawlers || []) {
@@ -386,10 +412,12 @@ export function formatCrawlerRows(crawlers, clients, pageTree, now = Date.now())
lastStart: 0,
referer: '',
pages: new Map(),
langs: new Set(),
}
const start = new Date(c.start).getTime()
if (start > g.lastStart) g.lastStart = start
if (c.referer) g.referer = c.referer
if (c.lang) g.langs.add(c.lang)
if (c.entry?.startsWith('/')) {
const existing = g.pages.get(c.entry) || { count: 0, status: c.status || 200 }
existing.count += 1
@@ -410,19 +438,28 @@ export function formatCrawlerRows(crawlers, clients, pageTree, now = Date.now())
const client = g.client || {}
const host = client.host || ''
const isHost = !!host
// Rendered languages read, shown only when they say something the
// primary language alone would not (multilingual sites only).
const langs = [...g.langs].sort()
const showLangs =
site.multilingual && (langs.length > 1 || (langs[0] && langs[0] !== site.primaryLang))
return {
lastSeen: formatWhen(g.lastStart, now),
lastSeenIso: formatWhenIso(g.lastStart),
lastSeenLocal: formatWhenLocal(g.lastStart),
refererStep: stepOf(g.referer, titles),
refererBadge: refererBadgeOf(g.referer, titles),
pages: [...g.pages.entries()]
.sort((a, b) => b[1].count - a[1].count)
.map(([path, info]) => ({ ...stepOf(path, titles), count: info.count, status: info.status })),
readFlags: showLangs
? langs.map((l) => ({ flag: flagFor(l), name: langName(l) })).filter((f) => f.flag)
: [],
ip: client.ip || '',
ipDisplay: isHost ? mainDomain(host) : hostIP(client.ip) || client.ip || '—',
isHost,
ua: client.ua_pretty || client.ua || '—',
ua: client.uarite?.pretty || client.ua || '—',
uaRaw: client.ua || '',
uaUrl: client.uarite?.url || '',
lang: client.lang || '—',
langDisplay: formatLang(client.lang),
country: client.country || '—',
@@ -505,6 +542,13 @@ export function formatAbuseRows(abuse, clients, pageTree, now = Date.now()) {
const client = (clients || {})[g.lastClient] || {}
const host = client.host || ''
const isHost = !!host
const uaRaws = [
...new Set(
[...g.clientHashes]
.map((h) => (clients || {})[h]?.ua)
.filter(Boolean),
),
].join('\n')
return {
lastSeen: formatWhen(g.lastStart, now),
lastSeenIso: formatWhenIso(g.lastStart),
@@ -527,8 +571,10 @@ export function formatAbuseRows(abuse, clients, pageTree, now = Date.now()) {
ip: client.ip || g.ip,
ipDisplay: isHost ? mainDomain(host) : hostIP(client.ip || g.ip) || client.ip || g.ip || '—',
isHost,
ua: client.ua_pretty || client.ua || '—',
ua: client.uarite?.pretty || client.ua || '—',
uaRaw: client.ua || '',
uaUrl: client.uarite?.url || '',
uaRaws,
lang: client.lang || '—',
langDisplay: formatLang(client.lang),
country: client.country || '—',
@@ -540,52 +586,88 @@ export function formatAbuseRows(abuse, clients, pageTree, now = Date.now()) {
/**
* Format raw visit records as rows for a technical table. Returns objects
* with display strings; missing values become "—". ``trail`` starts with the
* external referer (when present), then the entry page and any further internal
* pages or external exit origins. Only the 20 most recent visits are shown.
* ``clients`` maps client hashes to client records.
* with display strings; missing values become "—". The external referer
* (when present) and the UTM tags ride along as ``refererBadge``; ``trail``
* holds the entry page and any further internal pages or external exit
* origins; consecutive views of the same page (e.g. a
* language switch re-view) merge into one step that keeps the
* consecutive-distinct rendered languages, summed read time, and the latest
* status. On multilingual sites the rendered languages surface as flag
* icons: a visit read entirely in one non-primary language gets ``rowFlag``,
* and a visit spanning languages gets per-step ``langFlags`` markers where
* the language begins or changes. Only the 20 most recent visits are shown.
* ``clients`` maps client hashes to client records; ``site`` carries the
* payload's multilingual/primary-language context.
*/
export function formatVisitRows(visits, clients, pageTree, now = Date.now()) {
export function formatVisitRows(visits, clients, pageTree, now = Date.now(), site = { multilingual: false, primaryLang: '' }) {
const titles = buildTitleMap(pageTree)
return [...(visits || [])].reverse().slice(0, 20).map((v) => {
const client = (clients || {})[v.client] || {}
const trail = Object.values(v.trail || {})
const steps = Object.values(v.trail || {})
.map((item) => {
const step = stepOf(item.to, titles)
if (step) {
if (item.read) step.readSeconds = item.read
if (item.status) step.status = item.status
if (item.lang) step.lang = item.lang
}
return step
})
.filter(Boolean)
const utmKeys = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content']
const utmValues = utmKeys.map((k) => (v.utm || {})[k]).filter(Boolean)
const utm = utmValues.length ? utmValues.join(' · ') : ''
const utmTitle = Object.entries(v.utm || {})
.map(([k, value]) => `${k}=${value}`)
.join(', ')
const trail = []
for (const step of steps) {
const prev = trail[trail.length - 1]
if (prev && prev.path === step.path) {
if (step.lang && step.lang !== prev.langs[prev.langs.length - 1]) prev.langs.push(step.lang)
if (step.readSeconds) prev.readSeconds = (prev.readSeconds || 0) + step.readSeconds
if (step.status) prev.status = step.status
} else {
step.langs = step.lang ? [step.lang] : []
trail.push(step)
}
}
const distinctLangs = new Set(trail.flatMap((s) => s.langs))
const dash = (s) => (s || '—')
const host = client.host || ''
const isHost = !!host
return {
const row = {
lastSeen: formatWhen(v.start, now),
lastSeenIso: formatWhenIso(v.start),
lastSeenLocal: formatWhenLocal(v.start),
langDisplay: formatLang(client.lang),
trail,
refererStep: stepOf(v.referer, titles),
referer: dash(v.referer),
refererBadge: refererBadgeOf(v.referer, titles, v.utm),
ip: client.ip || '',
ipDisplay: isHost ? mainDomain(host) : hostIP(client.ip) || client.ip || '—',
isHost,
lang: dash(client.lang),
country: dash(client.country),
city: dash(client.city),
ua: client.ua_pretty || client.ua || '—',
ua: client.uarite?.pretty || client.ua || '—',
uaRaw: client.ua || '',
utm: utm || '',
utmTitle,
uaUrl: client.uarite?.url || '',
}
if (site.multilingual && distinctLangs.size) {
if (distinctLangs.size === 1) {
const [tag] = distinctLangs
const flag = flagFor(tag)
if (flag && tag !== site.primaryLang) {
row.rowFlag = flag
row.rowFlagTitle = langName(tag)
}
} else {
// Flag the steps where the rendered language begins or changes;
// lang-less steps keep the comparison chain going, they never flag.
let lastLang = null
for (const step of trail) {
if (!step.langs.length) continue
if (!lastLang || step.langs[step.langs.length - 1] !== lastLang) {
step.langFlags = step.langs.map(flagFor).filter(Boolean)
}
lastLang = step.langs[step.langs.length - 1]
}
}
}
return row
})
}
+23 -17
View File
@@ -26,6 +26,14 @@
/* Selection fill for page text and the CodeMirror editors; themes
override when the accent tint clashes with accent-colored text. */
--selection-bg: color-mix(var(--accent) 30%, transparent);
/* Referer-badge chip in the analytics viewer: a fixed neutral palette,
deliberately NOT themed — the chip sits behind transparent favicons, so
black-on-transparent and white-on-transparent glyphs must both stay
legible on every theme (dark greys kill black glyphs, pure white kills
white ones); its text likewise stays dark on any theme. */
--badge-bg: #c9d1d9;
--badge-text: #1f2328;
--badge-muted: #59636e;
/* Code highlighting palette, consumed by pygments.css: complete light and
dark sets (background included), resolved by light-dark() from the
used color-scheme. A theme picks a set simply by declaring
@@ -756,6 +764,20 @@ article {
position: relative;
}
/* Emoji/symbol icon buttons and links: dim until hovered. */
.icon-btn {
padding: 0;
font: inherit;
background: none;
border: none;
cursor: pointer;
opacity: 0.7;
}
.icon-btn:hover {
opacity: 1;
}
.edit-link {
position: absolute;
top: 0.2rem;
@@ -763,12 +785,6 @@ article {
left: -2.2rem;
z-index: 2;
/* stay above full-bleed .wide images */
font: inherit;
background: none;
border: none;
padding: 0;
cursor: pointer;
opacity: 0.7;
text-shadow: 0 0 0.1em black;
}
@@ -790,24 +806,14 @@ article h2 .edit-section {
opacity: 0.35;
}
.edit-link:hover {
opacity: 1;
}
/* Login/profile links injected by pagerite.js when Paskia SSO is in use.
They live inside the .editor-pens flex container in the banner's top-right
corner and inherit its reset; keep only their opacity/text-shadow tweaks. */
corner and inherit its reset; keep only their text-shadow tweak. */
.editor-pens a.login-link,
.editor-pens a.profile-link {
opacity: 0.7;
text-shadow: 0 0 0.1em black;
}
.editor-pens a.login-link:hover,
.editor-pens a.profile-link:hover {
opacity: 1;
}
article p,
article li,
article dd {
+24
View File
@@ -0,0 +1,24 @@
// Shared popup open-state behavior: while `open` (a ref, truthy = open)
// is set, a pointerdown outside `root` (a template ref covering both the
// toggle button and the popup) or Escape resets it to null. One logic for
// every dropdown (LangSelect, the page editor's class/table pickers), so
// they can't drift apart.
import { onBeforeUnmount, watch } from 'vue'
export function usePopup(open, root) {
let off = null
const stop = watch(open, (v) => {
off?.()
off = null
if (!v) return
const down = (ev) => { if (!root.value?.contains(ev.target)) open.value = null }
const key = (ev) => { if (ev.key === 'Escape') open.value = null }
addEventListener('pointerdown', down, true)
addEventListener('keydown', key)
off = () => {
removeEventListener('pointerdown', down, true)
removeEventListener('keydown', key)
}
})
onBeforeUnmount(() => { off?.(); stop() })
}
+10 -5
View File
@@ -1,10 +1,15 @@
// The editor shell's shared language selection ('' = the primary language):
// one state, v-modeled by the LangSelect of every tab that has one (page,
// structure). While the panel is open it also drives the page preview —
// EditorShell applies it as the fetch-time language override (swapdoc).
import { ref } from 'vue'
// backed by the app-wide store (./store), so the editor tabs' LangSelects
// and the public corner selector bind the same value. Linked to the
// whole-page language: while the panel is open it drives the page preview
// (EditorShell applies it as the fetch-time language override, swapdoc).
import { computed, ref } from 'vue'
import { pinia, useStore } from './store'
export const editorLang = ref('')
export const editorLang = computed({
get: () => useStore(pinia).lang,
set: (v) => { useStore(pinia).lang = v },
})
// The CURRENT PAGE's primary language ('' = not yet learned): the shell's
// settings fetch fills it with the site default; the page/structure tabs
+14
View File
@@ -30,6 +30,20 @@ export const LANG_GROUPS = [
const displayNames = new Intl.DisplayNames(['en'], { type: 'language' })
// Consistent menu ordering for language selectors: the geographic/cultural
// grouping above (similar languages sit together, and it does not vary with
// the display language the way alphabetical-by-name would). Tags outside
// the groups trail, ordered by tag. The primary language is not special
// here — callers put it first themselves.
const groupOrder = new Map(LANG_GROUPS.flat().map((c, i) => [c, i]))
export function langSort(codes) {
return [...codes].sort(
(a, b) =>
(groupOrder.get(a) ?? groupOrder.size) - (groupOrder.get(b) ?? groupOrder.size)
|| a.localeCompare(b),
)
}
// English display name for a language tag ("fi" -> "Finnish").
export function langName(tag) {
try {
+50
View File
@@ -0,0 +1,50 @@
// Public language-selector entry: imported on demand by pagerite.js on
// pages advertising more than one language in their hreflang alternates.
// Vue, Pinia and the flag SVG set live in this chunk only — untranslated
// pages never pay for them. The selector's state lives in the shared
// store (./store), not the DOM: the corner container is rebuilt freely
// and ensureMounted re-mounts from the store.
import { createApp } from 'vue'
import LangSelector from './LangSelector.vue'
import { pinia, useStore } from './store'
let app = null
function store() {
return useStore(pinia)
}
// The current page's languages (called on every navigation).
export function setLanguages(alternates, current) {
Object.assign(store(), {
langAlternates: alternates,
servedLang: current,
langSelectorActive: true,
})
}
// The current page is single-language: the selector goes away.
export function hide() {
store().langSelectorActive = false
app?.unmount()
app = null
}
// Mount the selector as the container's first item; re-mount when its
// element went away with a container rebuild (a live app updates from the
// store reactively).
export function ensureMounted(host) {
if (!store().langSelectorActive || !host) {
app?.unmount()
app = null
return
}
if (app && host.contains(app._container)) return
app?.unmount()
const el = document.createElement('div')
el.id = 'lang-selector'
host.prepend(el)
app = createApp(LangSelector)
app.use(pinia)
app.mount(el)
}
+116 -33
View File
@@ -51,13 +51,18 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
url.searchParams.delete("lang");
history.replaceState(history.state, "", url);
}
// The session language. While the editor panel is open, its language
// selection overrides the normal preference (swapdoc.setLangOverride):
// internal fetches and prefetches follow it until the panel closes and
// the override clears (null restores the initial ?lang=, if any).
// The session language: the user's explicit pick (initial ?lang=, public
// selector, editor dropdown) is kept in chosenLang; while the editor is
// open its selection overrides it (swapdoc.setLangOverride), and closing
// falls back to chosenLang. JS state only — pretty URLs, no reloads.
// window.__pageriteLang is the pin for swapdoc.loadPlain's fetches.
let chosenLang = langParam;
let sessionLang = langParam;
window.__pageriteLang = sessionLang;
addEventListener("pagerite:session-lang", (ev) => {
sessionLang = ev.detail?.lang || langParam;
if (ev.detail?.lang) chosenLang = ev.detail.lang;
sessionLang = ev.detail?.lang || chosenLang;
window.__pageriteLang = sessionLang;
});
// An internal URL as fetched: carries the session's ?lang= unless the
// link already pins a language of its own. With no ?lang= on the initial
@@ -127,16 +132,16 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
if (line != null) {
// Section pen on an anchored h2: opens the page editor at the
// section's markdown source line (data-line, from the backend).
btn.className = "edit-link edit-section";
btn.className = "edit-link edit-section icon-btn";
btn.title = "edit section";
btn.textContent = "🖊️";
btn.dataset.editorLine = line;
} else if (mode === "page") {
btn.className = "edit-link edit-page";
btn.className = "edit-link edit-page icon-btn";
btn.title = "edit page";
btn.textContent = "🖊️";
} else {
btn.className = "edit-link site-edit-link";
btn.className = "edit-link site-edit-link icon-btn";
btn.title = "site settings";
btn.textContent = "⚙️";
}
@@ -160,13 +165,29 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
function makeAuthLink(admin) {
const a = document.createElement("a");
a.className = admin ? "profile-link" : "login-link";
a.className = (admin ? "profile-link" : "login-link") + " icon-btn";
a.href = "/auth/";
a.title = admin ? "profile" : "log in";
a.textContent = admin ? "\u{1F510}" : "\u{1F511}";
return a;
}
// The banner top-right corner container: the language selector (first
// item) plus the admin pens and auth links. renderAuthUi rebuilds it from
// scratch; the selector's state lives in the shared store, not the DOM,
// so the langselect bundle re-mounts it into the fresh container.
function pensContainer() {
let pens = document.querySelector(".editor-pens");
if (!pens) {
const banner = document.getElementById("page-banner");
if (!banner) return null;
pens = document.createElement("div");
pens.className = "editor-pens";
banner.after(pens);
}
return pens;
}
function removePens() {
document.querySelectorAll(".editor-pens, #main article button.edit-link")
.forEach((el) => el.remove());
@@ -178,32 +199,33 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
// pens that may have been injected while the browser cache made us look
// authenticated.
removePens();
if (!authReady) return;
// Editing is open for admins and, as a dev/no-proxy fallback, when no
// Paskia SSO is detected at all.
const canEdit = isAdmin || !ssoAvailable;
// The analytics page is a read-only dashboard: editing pens and the side
// panel do not apply there. Login/logout links are still useful.
const onAnalytics = currentPath === "/_a";
const banner = document.getElementById("page-banner");
if (banner) {
const pens = document.createElement("div");
pens.className = "editor-pens";
if (canEdit && !onAnalytics) {
// Analytics viewer is now a normal page at /_a.
const a = document.createElement("a");
a.className = "edit-link analytics-link";
a.href = "/_a";
a.title = "analytics";
a.textContent = "📊";
pens.append(a);
pens.append(makePen("site"));
if (authReady) {
// Editing is open for admins and, as a dev/no-proxy fallback, when no
// Paskia SSO is detected at all.
const canEdit = isAdmin || !ssoAvailable;
// The analytics page is a read-only dashboard: editing pens and the side
// panel do not apply there. Login/logout links are still useful.
const onAnalytics = currentPath === "/_a";
if (document.getElementById("page-banner")) {
const pens = pensContainer();
if (canEdit && !onAnalytics) {
// Analytics viewer is now a normal page at /_a.
const a = document.createElement("a");
a.className = "edit-link analytics-link icon-btn";
a.href = "/_a";
a.title = "analytics";
a.textContent = "📊";
pens.append(a);
pens.append(makePen("site"));
}
if (ssoAvailable) pens.append(makeAuthLink(isAdmin));
if (!pens.firstElementChild) pens.remove();
}
if (ssoAvailable) pens.append(makeAuthLink(isAdmin));
banner.after(pens);
if (canEdit && !onAnalytics) injectPagePen();
}
if (canEdit && !onAnalytics) injectPagePen();
// Re-mount the selector into the fresh container (no-op until the
// bundle has been loaded once).
langselectMod?.ensureMounted(document.querySelector(".editor-pens"));
}
async function setupAuth() {
@@ -407,6 +429,9 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
// copy pinned to a language (?lang=) caches under its own key, where
// navigation with the same session language finds it.
pageCache.set(rawKey(ev.detail.url), ev.detail.html);
// Editor-driven swaps don't go through load(): re-evaluate the
// language selector from the fresh copy too.
mountLangselect(new DOMParser().parseFromString(ev.detail.html, "text/html"));
});
// Editors mutate site-wide state (theme, structure, headings, banners),
@@ -612,6 +637,8 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
if (to) msg.to = to;
const secs = Math.round(read / 1000);
if (secs > 0) msg.read = secs;
const lang = document.documentElement.lang;
if (lang) msg.lang = lang;
if (!msg.to && !msg.read) return;
report(msg);
}
@@ -742,6 +769,60 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
}
}
// --- Public language selector ------------------------------------------
// Pages translated into more than one language advertise it via hreflang
// alternates (x-default + one link per language). Those pages get the
// editors' flag dropdown as the first item of the corner container; its
// bundle (Vue + the flag SVG set) loads on demand. Re-evaluated from the
// fresh document on every swap (the head's own alternates stay stale).
let langselectMod = null;
async function mountLangselect(doc) {
const links = [...doc.head.querySelectorAll('link[rel="alternate"][hreflang]')];
const dflt = links.find((l) => l.hreflang === "x-default");
const langs = links.filter((l) => l.hreflang && l.hreflang !== "x-default");
if (!dflt || langs.length <= 1) return langselectMod?.hide();
try {
langselectMod ??= await import(/* @vite-ignore */ assets["pagerite:langselect-src"]);
for (const css of (assets["pagerite:langselect-css"] || "").split(",")) {
if (css && !document.querySelector(`link[href="${css}"]`)) {
const link = document.createElement("link");
link.rel = "stylesheet";
link.href = css;
link.dataset.pagerite = "langselect-css";
document.head.append(link);
}
}
langselectMod.setLanguages(
// The original's alternate is the plain URL — x-default's href —
// which also marks it as the primary option.
langs.map((l) => ({ tag: l.hreflang, href: l.href, primary: l.href === dflt.href })),
doc.documentElement.lang,
);
langselectMod.ensureMounted(pensContainer());
} catch (e) {
console.error("language selector mount failed:", e);
}
}
// The selector's pick (LangSelector dispatches this): make it the
// session language and swap the page in place. With the editor open the
// pick already landed in the shared store — the editor's watch re-renders
// the page itself, so there is nothing to do here.
addEventListener("pagerite:set-session-lang", async (ev) => {
const tag = ev.detail?.lang;
if (!tag || tag === sessionLang) return;
if (document.body.classList.contains("editing")) return;
chosenLang = sessionLang = tag;
window.__pageriteLang = tag;
const y = scrollY; // a language switch is not a navigation: keep scroll
await load(currentPath, false);
scrollTo(0, y);
// Log the switch as a trail event in the new language (load() updated
// <html lang>): the ping matches the switch's GET server-side, so it is
// not misclassified as a crawler hit.
ping({ to: currentPath });
});
// --- Fetch navigation ------------------------------------------------
async function load(url, push = true, back = false) {
// Navigating with the editor open closes it; unsaved edits are lost
@@ -843,6 +924,7 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
runScripts(document.getElementById("main"));
applyEffects();
mountAnalytics(doc);
mountLangselect(doc);
};
// Rotating cube page transition (styles injected as #pagerite-transition
// from the selected design's transition.css, e.g. themes/cube/);
@@ -1076,4 +1158,5 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
setupAuth();
applyEffects();
mountAnalytics(document);
mountLangselect(document);
})();
+26
View File
@@ -0,0 +1,26 @@
// The app's shared Pinia store — cross-bundle UI state lives here. Every
// entry chunk imports its own copy of this module, so the Pinia instance
// is parked on window (Vue itself is a shared chunk, so reactivity works
// across the copies). Pass `pinia` explicitly when calling useStore
// outside a component (module code, no active instance).
import { createPinia, defineStore } from 'pinia'
export const pinia = (window.__pageritePinia ??= createPinia())
export const useStore = defineStore('pagerite', {
state: () => ({
// The ONE language selection, v-modeled by both dropdowns (editor
// tabs, public corner selector): '' = no explicit pick (the page's
// primary / autodetect), else a concrete tag. A pick from either
// dropdown is visible to everyone immediately.
lang: '',
// The language the current page was actually served in (set by
// pagerite.js per navigation) — the selector's highlight fallback
// when there is no explicit pick.
servedLang: '',
// The public selector's page data: hreflang alternates
// ([{tag, href, primary}]) and whether to show at all.
langAlternates: [],
langSelectorActive: false,
}),
})
+12 -9
View File
@@ -12,12 +12,13 @@ export function dropPageCache() {
}
// The editor's language override (set by EditorShell): while the panel is
// open, its language selection wins over the normal preferences (?lang= /
// Accept-Language) — every in-place re-render asks for that language
// explicitly, and pagerite.js applies it to its own fetches and prefetches
// (pagerite:session-lang). The primary selection pins by its code:
// ?lang=<primary> selects the original explicitly (i18n.select_language).
let overrideLang = null // the ?lang= value in force, null = normal prefs
// open, its language selection wins over the normal preferences — every
// in-place re-render asks for that language explicitly, and pagerite.js
// applies it to its own fetches and prefetches (pagerite:session-lang).
// The primary selection pins by its code: ?lang=<primary> selects the
// original explicitly (i18n.select_language). Panel closed, the session's
// chosen language (window.__pageriteLang) takes over — the pick stays.
let overrideLang = null // the ?lang= value in force, null = the session's
export function setLangOverride(queryLang) {
overrideLang = queryLang || null
@@ -129,14 +130,16 @@ function swapRegions(doc) {
// Fetch /p, swap its regions into the live page and replaceState to it.
// Returns the final URL (after redirects), or null when the fetch did not
// yield a page. Category and missing URLs render a placeholder 404 page —
// fine to swap in (new pages are created by editing them). While the
// editor's language override is set the fetch pins that language.
// fine to swap in (new pages are created by editing them). The fetch pins
// the editor's language override, or — panel closed — the session's chosen
// language (window.__pageriteLang).
export async function loadPlain(p) {
let doc
let finalUrl = `/${p}`
let html
try {
const res = await fetch(overrideLang ? `${finalUrl}?lang=${overrideLang}` : finalUrl)
const pin = overrideLang || window.__pageriteLang
const res = await fetch(pin ? `${finalUrl}?lang=${pin}` : finalUrl)
const type = res.headers.get('content-type') || ''
if (!type.includes('text/html')) return null
if (res.redirected) finalUrl = res.url
+5 -5
View File
@@ -8,11 +8,11 @@
* - Disables Vite's screen clearing on startup
*
* Options:
* paths - Array of paths to proxy (default: ["/api"])
* paths - Array of paths to proxy (default: ['/api'])
*/
export default function fastapiVue({ paths = ["/api"] } = {}) {
const backendUrl = process.env.PAGERITE_BACKEND_URL || "http://localhost:8210"
export default function fastapiVue({ paths = ['/api'] } = {}) {
const backendUrl = process.env.PAGERITE_BACKEND_URL || 'http://localhost:8210'
// Build proxy configuration for each path
const proxy = {}
@@ -25,12 +25,12 @@ export default function fastapiVue({ paths = ["/api"] } = {}) {
}
return {
name: "vite-plugin-fastapi-pagerite",
name: 'vite-plugin-fastapi-pagerite',
config: () => ({
clearScreen: false,
server: { proxy },
build: {
outDir: "../pagerite/frontend-build",
outDir: '../pagerite/frontend-build',
emptyOutDir: true,
},
}),
+3 -2
View File
@@ -5,7 +5,7 @@ import { defineConfig } from 'vite'
import vue from '@vitejs/plugin-vue'
import vueDevTools from 'vite-plugin-vue-devtools'
const backendUrl = process.env.PAGERITE_BACKEND_URL || 'http://localhost:3200'
const backendUrl = process.env.PAGERITE_BACKEND_URL || 'http://localhost:8210'
// Proxy everything except Vite's own dev-time paths and the backend machinery
// to the FastAPI backend in dev. /_api, /_f, /_themes, /_fonts and /_a are
@@ -38,7 +38,7 @@ export default defineConfig({
chunkSizeWarningLimit: 1200,
// Mirror the URL space in the build output: hashed files land under
// frontend-build/_assets/ and the Frontend serves the build directory
// at the site root (frontend/public/favicon.ico -> /favicon.ico).
// at the site root.
manifest: true,
assetsDir: '_assets',
rollupOptions: {
@@ -50,6 +50,7 @@ export default defineConfig({
main: fileURLToPath(new URL('./src/main.js', import.meta.url)),
pagerite: fileURLToPath(new URL('./src/pagerite.js', import.meta.url)),
analytics: fileURLToPath(new URL('./src/analytics-main.js', import.meta.url)),
langselect: fileURLToPath(new URL('./src/langselect-main.js', import.meta.url)),
// Only the base CSS is built; theme/banner-design stylesheets live
// in pagerite/themes/{name}/ and are served by the backend as-is.
pagerite_base: fileURLToPath(new URL('./src/assets/pagerite.css', import.meta.url)),
+3 -3
View File
@@ -5,12 +5,12 @@ import os
from pathlib import Path
import msgspec
from fastapi_vue import server
from fastapi_vue import env, server
from pagerite.config import Config
DEFAULT_PORT = 8100
DEVMODE = os.getenv("PAGERITE_DEV") == "1"
os.environ["FASTAPI_VUE"] = "PAGERITE"
def main() -> None:
@@ -54,7 +54,7 @@ def main() -> None:
listen=args.listen,
default_port=DEFAULT_PORT,
server_header=False,
reload=Path(__file__).parent if DEVMODE else False,
reload=Path(__file__).parent if env.dev else False,
**run_args,
)
+140 -58
View File
@@ -2,8 +2,9 @@
Raw recording, display-time classification. Every document GET is appended
to ``Analytics.gets`` as a raw access-log line (path with query string, true
HTTP status, external referer origin, preload flag) and every pagerite.js
activity message from the /_ws WebSocket is appended to ``Analytics.msgs``
HTTP status, external referer origin, preload flag, rendered content
language) and every pagerite.js activity message from the /_ws WebSocket is
appended to ``Analytics.msgs``
(navigations ``fr`` -> ``to`` and active reading-time updates). Nothing is
classified when it is recorded: whether a client turns out to be a reader,
a crawler or a scanner is decided by ``Store.display()`` from the raw
@@ -60,32 +61,17 @@ from urllib.parse import parse_qs, urlencode, urlparse
import blake3
import msgspec
from ua_parser import parse
from uarite import UA, uaparse
def _compact_user_agent(ua: str) -> str:
"""Format a User-Agent string into a compact display form.
def _display_client(client: Client) -> Client:
"""Client copy with ``uarite`` filled in by the current uarite.
Returns the original UA when the parser cannot identify the browser/OS.
The parsed UA is a display-time field: stored records always carry the
default (None), so it never lands on disk, and old records always
follow current uarite rules.
"""
if not ua or not ua.strip() or ua == "-":
return ""
r = parse(ua)
browser = r.user_agent.family if r.user_agent else None
ver = r.user_agent.major if r.user_agent else ""
os_name = r.os.family if r.os else None
dev = r.device.family if r.device else None
if browser in (None, "Other") and os_name in (None, "Other"):
return ua
if browser and browser != "Other":
browser = browser.split()[0]
else:
browser = ""
os_name = os_name if os_name and os_name != "Other" else ""
if dev in (None, "Other") or dev == browser:
dev = ""
parts = [f"{browser}/{ver}" if browser else "", os_name, dev]
return " ".join(p for p in parts if p).strip()
return msgspec.structs.replace(client, uarite=uaparse(client.ua))
class Ping(msgspec.Struct, omit_defaults=True):
@@ -107,6 +93,9 @@ class Ping(msgspec.Struct, omit_defaults=True):
read: int = 0
#: Admin client: record but hide everything from the statistics.
hide: bool = False
#: Rendered language of the page the activity happened on (the page's
#: ``<html lang>``, sent by pagerite.js).
lang: str = ""
class Get(msgspec.Struct, omit_defaults=True):
@@ -130,6 +119,9 @@ class Get(msgspec.Struct, omit_defaults=True):
#: never counted as a view/crawler/abuse hit; recorded only so a later
#: cache-served navigation can be attributed this GET's status.
pre: bool = False
#: Rendered content language of the served document; "" for
#: non-localized responses (404 probes, reserved paths).
lang: str = ""
class Msg(msgspec.Struct, omit_defaults=True):
@@ -150,6 +142,9 @@ class Msg(msgspec.Struct, omit_defaults=True):
to: str = ""
#: Active reading time (seconds) spent on ``fr`` since the last report.
read: int = 0
#: Rendered language reported by the client for the page the activity
#: happened on.
lang: str = ""
class Client(msgspec.Struct, omit_defaults=True):
@@ -172,11 +167,14 @@ class Client(msgspec.Struct, omit_defaults=True):
city: str = ""
#: Raw User-Agent header.
ua: str = ""
#: Compact display form of ``ua`` (browser/OS/device) when parsable.
ua_pretty: str = ""
#: True for admin clients (hide=1 ping): everything this client ever did
#: is excluded from all statistics and from the viewer payload.
hide: bool = False
#: Display-time parsed UA (uarite.UA dataclass: pretty/engine/os/
#: provider/kind/url). Set only on the display-payload copies by
#: ``_display_client`` — stored records keep the default, so it is never
#: persisted and old data always follows the current uarite version.
uarite: UA | None = None
# --- Display DTOs -------------------------------------------------------
@@ -203,6 +201,8 @@ class TrailItem(msgspec.Struct, omit_defaults=True):
``read`` accumulates active reading time (seconds) across the whole
visit; ``status`` is the most recent HTTP status seen for the target.
A page seen in two rendered languages within one visit (a mid-article
language switch) gets one item per language.
"""
to: str
@@ -210,6 +210,10 @@ class TrailItem(msgspec.Struct, omit_defaults=True):
read: int = 0
#: Most recent HTTP status of the response (200 or 404).
status: int = 200
#: Rendered language of the target: the client's report, for the entry
#: page falling back to its GET's rendered language; "" when unknown
#: (old clients or data from before language recording).
lang: str = ""
class Visit(msgspec.Struct, omit_defaults=True):
@@ -218,8 +222,9 @@ class Visit(msgspec.Struct, omit_defaults=True):
``trail`` holds the entry page and everything seen afterwards, keyed by
the timestamp of first sight (insertion order = first-seen order);
re-visiting an already seen target updates its item instead of
appending. Client metadata is held in ``Analytics.clients`` keyed by
``client``.
appending — unless the client reports a different rendered language for
it, which appends a distinct item (a mid-article language switch).
Client metadata is held in ``Analytics.clients`` keyed by ``client``.
"""
start: datetime
@@ -253,6 +258,8 @@ class CrawlerHit(msgspec.Struct, omit_defaults=True):
query: str = ""
#: HTTP status of the served response (200 or 404 for content pages).
status: int = 200
#: Rendered content language of the served document (from the GET).
lang: str = ""
class AbuseHit(msgspec.Struct, omit_defaults=True):
@@ -331,6 +338,12 @@ class Display(msgspec.Struct, omit_defaults=True):
views: dict[str, dict[str, int]] = {}
#: New visits per 5-minute bucket: bucket ISO -> count (sparse).
site_visits: dict[str, int] = {}
#: Site context: true when translation languages are configured, so the
#: viewer can suppress language UI on single-language sites.
multilingual: bool = False
#: The site's primary language (the front page's), so the viewer can
#: skip the primary-language default case.
primary_lang: str = ""
def _bucket(now: datetime) -> str:
@@ -418,17 +431,22 @@ _MIN_VISIT_READ = 5
_FAVICON_RETRY = timedelta(days=7)
#: UAs of JS-running crawlers, which would register as visitors on their
#: activity messages. Anything calling itself a "bot" or "spider" matches;
#: known crawlers without those tokens (GoogleOther) are listed as extra
#: alternates. No source verification: a spoofed bot UA just lands in the
#: crawler list, and scanners that probe telltale paths are caught by the
#: abuse rules anyway.
_BOT_UA = re.compile(r"bot|spider|googleother", re.IGNORECASE)
#: activity messages. ``uarite`` knows the common crawlers
#: and link-preview fetchers (including disguised ones such as
#: facebookexternalhit and Google-Extended) plus any UA with a
#: bot/spider/crawler/scanner token. No source verification: a spoofed
#: bot UA just lands in the crawler list, and scanners that probe
#: telltale paths are caught by the abuse rules anyway.
def _is_bot_ua(ua: str) -> bool:
"""True when the UA claims a crawler identity (bot or spider)."""
return bool(_BOT_UA.search(ua))
"""True when the UA is not a regular browser.
Every real browser registers as ``kind == "browser"``; anything else
(recognized crawler/previewer, generic bot token, or an unclassified
HTTP client such as httpx) is not a visitor.
"""
return uaparse(ua).kind != "browser"
#: Plain-404 count per IP within ``_ABUSE_404_WINDOW`` that classifies it as
@@ -560,7 +578,6 @@ class Store:
self.data.clients[h] = Client(
ip=ip,
ua=ua,
ua_pretty=_compact_user_agent(ua),
lang=lang,
country=country,
)
@@ -602,22 +619,25 @@ class Store:
referer: str = "",
accept_language: str = "",
pre: bool = False,
lang: str = "",
) -> bytes | None:
"""Append one document GET to the raw log.
``path`` is the full request path, query string included; ``status``
the true HTTP status of the response; ``referer`` the raw Referer
header (reduced here to an external https origin, "" when internal
or absent); ``pre`` marks idle-time preloads from pagerite.js.
or absent); ``pre`` marks idle-time preloads from pagerite.js;
``lang`` the rendered content language of the served document (""
for non-localized responses such as 404 probes and reserved paths).
Returns the client hash when the client record was just created (so
the caller can schedule async enrichment), else None.
"""
lang, country = _parse_accept_language(accept_language)
client_hash = _client_hash(ip, ua, lang)
client_lang, country = _parse_accept_language(accept_language)
client_hash = _client_hash(ip, ua, client_lang)
new = client_hash not in self.data.clients
if new:
self._ensure_client(ip, ua, lang, country=country)
self._ensure_client(ip, ua, client_lang, country=country)
self.data.gets.append(
Get(
t=datetime.now(UTC),
@@ -626,6 +646,7 @@ class Store:
status=status,
ref=_origin(referer) or "",
pre=pre,
lang=lang,
)
)
self._save()
@@ -640,6 +661,7 @@ class Store:
accept_language: str = "",
hide: bool = False,
read: int = 0,
lang: str = "",
) -> bytes | None:
"""Append one client activity message (``Ping`` from pagerite.js) to
the raw log.
@@ -650,16 +672,18 @@ class Store:
stored raw and filtered at display time, so future rule changes lose
nothing. ``hide`` flags the client record as an admin; the message
itself is recorded normally and hidden at display time like
everything else the client ever did.
everything else the client ever did. ``lang`` is the rendered
language reported by the client for the page the activity happened
on.
Returns the client hash when the client record was just created (so
the caller can schedule async enrichment), else None.
"""
lang, country = _parse_accept_language(accept_language)
client_hash = _client_hash(ip, ua, lang)
client_lang, country = _parse_accept_language(accept_language)
client_hash = _client_hash(ip, ua, client_lang)
new = client_hash not in self.data.clients
if new:
self._ensure_client(ip, ua, lang, country=country)
self._ensure_client(ip, ua, client_lang, country=country)
if hide:
self.data.clients[client_hash].hide = True
fr = (_internal_path(fr) or "") if fr else ""
@@ -671,7 +695,14 @@ class Store:
target = _external_target(to) or ""
if target or read > 0:
self.data.msgs.append(
Msg(t=datetime.now(UTC), client=client_hash, fr=fr, to=target, read=read)
Msg(
t=datetime.now(UTC),
client=client_hash,
fr=fr,
to=target,
read=read,
lang=lang,
)
)
if target or read > 0 or hide:
self._save()
@@ -708,7 +739,13 @@ class Store:
self.data.favicons[origin] = Favicon(file=file, fetched=datetime.now(UTC))
self._save()
def display(self, in_menu: Callable[[str], bool] | None = None) -> Display:
def display(
self,
in_menu: Callable[[str], bool] | None = None,
*,
multilingual: bool = False,
primary_lang: str = "",
) -> Display:
"""Build the viewer payload from the raw events.
All classification happens here, so the stored data is independent
@@ -730,10 +767,17 @@ class Store:
- visits: the remaining messages, grouped per client with a new
visit after ``_SESSION_GAP`` of inactivity. Trail statuses come
from the client's GETs (preloads included — a cache-served
navigation's only GET is its preload); the entry referer and UTM
tags from the GET that loaded the entry page.
navigation's only GET is its preload); trail languages come from
the client's messages (the entry item falling back to its GET's
rendered language), and a page re-visited in a different rendered
language becomes a distinct trail step. The entry referer and
UTM tags come from the GET that loaded the entry page.
Hidden (admin) clients are excluded from every list and aggregate.
``multilingual`` and ``primary_lang`` are site context (translation
languages configured, the front page's primary language) copied
onto the payload so the viewer can suppress language UI on
single-language sites and skip the primary-language default case.
"""
in_menu = in_menu or (lambda path: False)
data = self.data
@@ -844,7 +888,11 @@ class Store:
g.path.split("?", 1)[1] if "?" in g.path else ""
)
visit.trail[m.t] = TrailItem(
to=m.to, status=status_at(h, m.to, m.t)
to=m.to,
status=status_at(h, m.to, m.t),
# The client's report wins; the entry GET fills
# in for old clients that don't send lang.
lang=m.lang or (g.lang if g is not None else ""),
)
visits.append(visit)
else:
@@ -852,18 +900,40 @@ class Store:
visit.navs[m.t] = Nav(fr=fr, to=m.to)
status = status_at(h, m.to, m.t)
# First-seen only: repeat pages and repeated exits
# update the existing trail item instead of appending.
# update the existing trail item instead of
# appending — but a repeat in a different rendered
# language (a mid-article language switch) becomes
# a distinct step.
for item in visit.trail.values():
if item.to == m.to:
item.status = status
if m.lang and item.lang and m.lang != item.lang:
visit.trail[m.t] = TrailItem(
to=m.to, status=status, lang=m.lang
)
else:
item.status = status
if not item.lang:
item.lang = m.lang
break
else:
visit.trail[m.t] = TrailItem(to=m.to, status=status)
visit.trail[m.t] = TrailItem(
to=m.to, status=status, lang=m.lang
)
if m.read > 0 and m.fr and visit is not None:
# A page appears in the trail once per language seen:
# land the seconds on the matching-language step when
# the client reports one, else on the first-seen item.
read_item: TrailItem | None = None
for item in visit.trail.values():
if item.to == m.fr:
item.read += m.read
if item.to != m.fr:
continue
if read_item is None:
read_item = item
if m.lang and item.lang == m.lang:
read_item = item
break
if read_item is not None:
read_item.read += m.read
last_t = m.t
# --- crawler hits: document GETs no message matched
@@ -897,6 +967,7 @@ class Store:
referer=g.ref,
query=query,
status=g.status,
lang=g.lang,
)
)
@@ -920,6 +991,7 @@ class Store:
referer=visit.referer if first else "",
query=query if first else "",
status=item.status,
lang=item.lang,
)
)
first = False
@@ -940,12 +1012,14 @@ class Store:
and not self._hidden(g.client)
and ip_of.get(g.client, "") in abuse_ips
],
clients={h: c for h, c in data.clients.items() if not c.hide},
clients={h: _display_client(c) for h, c in data.clients.items() if not c.hide},
favicons={
origin: f"/_f/{f.file}"
for origin, f in data.favicons.items()
if f.file
},
multilingual=multilingual,
primary_lang=primary_lang,
)
for visit in kept:
bucket = _bucket(visit.start)
@@ -967,6 +1041,14 @@ class Store:
nbuckets[nb] = nbuckets.get(nb, 0) + 1
return display
def display_json(self, in_menu: Callable[[str], bool] | None = None) -> str:
def display_json(
self,
in_menu: Callable[[str], bool] | None = None,
*,
multilingual: bool = False,
primary_lang: str = "",
) -> str:
"""The ``display()`` payload as a JSON string for the WebSocket."""
return msgspec.json.encode(self.display(in_menu)).decode()
return msgspec.json.encode(
self.display(in_menu, multilingual=multilingual, primary_lang=primary_lang)
).decode()
+4 -5
View File
@@ -36,11 +36,10 @@ from pathlib import Path
from fastapi import FastAPI, Request
from fastapi.responses import Response
from fastapi_vue import Frontend
from fastapi_vue import Frontend, env
from starlette.types import ASGIApp, Receive, Scope, Send
from pagerite import api, files, pages, tracking
from pagerite.__main__ import DEVMODE
from pagerite.files import file_store
from pagerite.state import analytics_store, config, kanta
@@ -48,7 +47,7 @@ logger = logging.getLogger(__name__)
# Vue build served at the site root, no SPA catch-all (assets only). The
# build mirrors the URL space: hashed, immutable files live under
# /_assets/ (assetsDir: '_/assets'), the favicon at /favicon.ico.
# /_assets/ (assetsDir: '_/assets').
frontend = Frontend(
Path(__file__).with_name("frontend-build"), spa=False, cached="/_assets/"
)
@@ -99,7 +98,7 @@ async def lifespan(_app: FastAPI) -> AsyncGenerator:
# is not meant to be browsable by the public anyway.
app = FastAPI(
title="Pagerite",
debug=DEVMODE,
debug=env.dev,
lifespan=lifespan,
docs_url=None,
redoc_url=None,
@@ -124,7 +123,7 @@ app.include_router(tracking.router)
app.include_router(files.router)
# Vue build asset routes are inserted at this position during load(): the
# build mirrors the URL space (/_assets/*, /favicon.ico at the root).
# build mirrors the URL space (/_assets/*).
frontend.route(app, "/")
# The content catch-all goes last: built assets win over content slugs,
+19 -3
View File
@@ -17,6 +17,13 @@ from pagerite.segments import has_prose
#: backticks or tildes (CommonMark).
_FENCE_OPEN = re.compile(r"^ {0,3}(`{3,}|~{3,})")
#: A container fence line (mdit-py-plugins container): the "::: aside"
#: opener and the ":::" closer alike. Always its own block, even with no
#: blank line around it: folded into a prose paragraph it would cross to
#: the translator as part of the text run, where the model can drop it —
#: the rest of the page then renders inside the container.
_CONTAINER = re.compile(r"^ {0,3}:{3,}(?:[ \t]|$)")
#: HTML block openers that may span blank lines (CommonMark types 1-5:
#: script/pre/style/textarea, comments, processing instructions,
#: declarations, CDATA) with their closing condition. Other HTML blocks
@@ -54,9 +61,11 @@ def chunk_markdown(markdown: str) -> list[str]:
Blocks are separated by blank lines; fenced code blocks and the
multi-line HTML blocks (comments, script/pre/style, CDATA...) are
kept atomic, even across blank lines, and end at their closing
condition. Chunks carry no surrounding blank lines and no trailing
newline; rejoining with ``join_chunks`` reproduces the source modulo
blank-line normalization.
condition. Container fence lines (:::, open and close alike) are
always their own block, blank lines or not (see _CONTAINER). Chunks
carry no surrounding blank lines and no trailing newline; rejoining
with ``join_chunks`` reproduces the source modulo blank-line
normalization.
"""
chunks: list[str] = []
buf: list[str] = []
@@ -91,6 +100,13 @@ def chunk_markdown(markdown: str) -> list[str]:
fence = m.group(1)
buf.append(line)
continue
if _CONTAINER.match(line):
# Container fence lines (open and close alike) are their own
# block — never part of a prose chunk (see _CONTAINER).
flush()
buf.append(line)
flush()
continue
if not buf:
for open_re, close_re in _HTML_ATOMIC:
if open_re.match(line):
+2 -2
View File
@@ -98,8 +98,8 @@ class Data(msgspec.Struct):
#: Trusted author content; not sanitized.
custom_css: str = ""
#: Favicon: content-addressed file name (served at "/_f/{name}"),
#: linked as <link rel="icon"> on every page. Empty = the build's
#: /favicon.ico.
#: linked as <link rel="icon"> on every page; /favicon.ico redirects
#: to it. Empty = no icon (and /favicon.ico 404s).
favicon: str = ""
#: API keys gating the translator service WebSocket (/_translate/{key};
#: the external forward-auth does not cover that route): key -> display
+21 -10
View File
@@ -6,7 +6,8 @@ when compression shrinks the body), served immutable at ``/_f/``. Raster
images and SVGs are recompressed into AVIF/WebP/JPEG derivatives
(``store_image`` and helpers); the untouched original is kept alongside as
``<hash>.orig<ext>`` (never served). Routes: upload/delete under
``/_api/files``, the favicon settings endpoints, the ``/_f/`` server with
``/_api/files``, the favicon settings endpoints, the /favicon.ico
redirect to the configured icon, the ``/_f/`` server with
Accept-negotiated formats, and the user assets (``/_themes/``, ``/_fonts/``).
"""
@@ -19,7 +20,7 @@ from pathlib import Path
import blake3
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import Response
from fastapi.responses import RedirectResponse, Response
from mediapreview import dispatch
from pagerite import views
@@ -38,10 +39,6 @@ from pagerite.state import (
logger = logging.getLogger(__name__)
# mediapreview logs pyvips noise ("VipsForeignSaveJpegTarget argument strip is
# deprecated", "threadpool completed with N workers") at INFO; keep warnings.
logging.getLogger("mediapreview").setLevel(logging.WARNING)
router = APIRouter()
@@ -158,14 +155,13 @@ def _svg_to_png(body: bytes, maxsize: int) -> bytes | None:
def _avif_to_format(avif: bytes, suffix: str, quality: int) -> bytes:
"""Re-encode the AVIF derivative into a fallback format (WebP/JPEG)
via pyvips. JPEG has no alpha, so it is flattened onto white;
``strip`` keeps metadata (EXIF) out of the fallbacks."""
via pyvips. JPEG has no alpha, so it is flattened onto white."""
import pyvips
img = pyvips.Image.new_from_buffer(avif, "")
if suffix == ".jpg" and img.hasalpha():
img = img.flatten(background=[255, 255, 255])
return img.write_to_buffer(suffix, Q=quality, strip=True)
return img.write_to_buffer(suffix, Q=quality, keep="none")
def _image_derivatives(
@@ -252,6 +248,20 @@ async def delete_file(name: str) -> None:
file_store.delete(name)
@router.get("/favicon.ico", include_in_schema=False)
async def favicon_ico() -> Response:
"""The conventional /favicon.ico: redirect to the configured site icon.
Browsers request this path on their own (tabs, bookmarks, feeds and
other non-HTML contexts) regardless of the <link rel="icon"> pages
carry. Redirect to the icon's store URL, which negotiates the format
and caches immutably; 404 when no custom icon is configured.
"""
if not data.favicon:
raise HTTPException(404)
return RedirectResponse(f"/_f/{data.favicon}")
@router.put("/_api/settings/favicon")
async def put_favicon(request: Request) -> dict[str, str]:
"""Upload a favicon into the content-addressed store and activate it.
@@ -276,7 +286,8 @@ async def put_favicon(request: Request) -> dict[str, str]:
@router.delete("/_api/settings/favicon", status_code=204)
async def delete_favicon(request: Request) -> None:
"""Clear the custom favicon (back to the build's /favicon.ico).
"""Clear the custom favicon (/favicon.ico goes back to 404, pages drop
the <link rel="icon">).
The blob stays in the content-addressed store; only the reference goes.
"""
+4 -3
View File
@@ -152,7 +152,8 @@ async def show_page(request: Request, path: str) -> Response:
if node is not None and node.published and node.chunks is not None:
# Language selection (docs/localization.md): ?lang= wins when a
# translation exists, else header logic. Analytics keep the raw
# Accept-Language header regardless of the selection.
# Accept-Language header regardless of the selection, and record
# the resolved language as the GET's rendered language.
query_lang = request.query_params.get("lang")
lang = i18n.select_language(
query_lang,
@@ -175,7 +176,7 @@ async def show_page(request: Request, path: str) -> Response:
if request.headers.get("if-none-match") == etag:
return Response(status_code=304)
if _is_trackable_path(path):
_record_get(request)
_record_get(request, lang=lang)
return _html_response(
request,
"page",
@@ -205,7 +206,7 @@ async def show_page(request: Request, path: str) -> Response:
)
link_lang = i18n.base_tag(query_lang or "")
if _is_trackable_path(path):
_record_get(request, status=404)
_record_get(request, status=404, lang=lang)
return _html_response(
request,
"category",
+7 -2
View File
@@ -22,11 +22,11 @@ from pathlib import Path
import blake3
from fastapi import HTTPException, Request
from fastapi.responses import Response
from fastapi_vue import env
from kanta import Kanta
from zstandard import ZstdCompressor
from pagerite import analytics, i18n, seed, translate, views
from pagerite.__main__ import DEVMODE
from pagerite.chunks import store_chunks
from pagerite.config import load
from pagerite.data import (
@@ -56,6 +56,10 @@ DB_PATH = os.getenv("PAGERITE_DB", str(SITE_DIR / "content.kantadb"))
# Visit analytics go to their own JSON file, not the kanta database.
ANALYTICS_PATH = Path(os.getenv("PAGERITE_ANALYTICS", str(SITE_DIR / "analytics.json")))
# The per-hostname data directory may not exist yet on first run; kanta
# creates the database file but not its parent directory.
Path(DB_PATH).parent.mkdir(parents=True, exist_ok=True)
ANALYTICS_PATH.parent.mkdir(parents=True, exist_ok=True)
analytics_store = analytics.Store(ANALYTICS_PATH)
# Content-addressed file store (uploads, seed assets, fetched favicons):
@@ -132,6 +136,7 @@ def _render_html(
data.theme,
data.favicon,
data.brand_html,
base_url,
transition=data.transition,
lang=lang,
translation=translation,
@@ -225,7 +230,7 @@ def _html_response(
# Absolute social/canonical URLs use the site's public origin; on
# localhost (varying ports) fall back to the request's own base URL.
base_url = SITE_URL or str(request.base_url).rstrip("/")
if DEVMODE:
if env.dev:
identity = _render_html(kind, path, base_url, lang, link_lang).encode()
body = _zstd.compress(identity) if zstd else identity
else:
+54 -33
View File
@@ -3,18 +3,19 @@
The visitor-activity WebSocket (``/_ws``, public) and the admin analytics
stream (``/_api/ws/analytics``) plus the ``/_a`` viewer page. Client IPs are
enriched in background tasks with reverse DNS (cached PTR lookups) and the
DB-IP city MMDB (``GeoIP``, decompressed and opened once at startup);
DB-IP city MMDB (``GeoIP``, decompressed into RAM and opened once at
startup);
external referrers get their favicon fetched and stored content-hashed.
Snapshot broadcasts to connected admin sockets are debounced.
"""
import asyncio
import gzip
import io
import ipaddress
import logging
import os
import re
import shutil
import socket
from datetime import date
from functools import lru_cache
@@ -25,8 +26,9 @@ import httpx
import msgspec
from fastapi import APIRouter, Request, WebSocket, WebSocketDisconnect
from fastapi.responses import Response
from uarite import uaparse
from pagerite import analytics
from pagerite import analytics, i18n
from pagerite.data import resolve
from pagerite.files import _hash_name, file_store
from pagerite.state import SITE_URL, _html_response, analytics_store, data
@@ -44,8 +46,9 @@ _analytics_ws_clients: set[WebSocket] = set()
_analytics_broadcast_task: asyncio.Task | None = None
# Repository root from this file's location (pagerite/tracking.py -> ..).
_REPO_ROOT = Path(__file__).resolve().parent.parent
# DB-IP databases persist in the working directory (one download serves all
# sites run from it). Not the package directory: reinstalls/upgrades wipe it.
_DBIP_DIR = Path.cwd()
DBIP_URL = "https://download.db-ip.com/free/dbip-city-lite-{month}.mmdb.gz"
@@ -60,7 +63,7 @@ def _download_dbip() -> None:
existing = sorted(
p.stem.removeprefix("dbip-city-lite-").removesuffix(".mmdb")
for p in _REPO_ROOT.glob("dbip-city-lite-*.mmdb*")
for p in _DBIP_DIR.glob("dbip-city-lite-*.mmdb*")
)
if existing and existing[-1] >= months[0]:
logger.info("DB-IP database is current (%s), skipping download", existing[-1])
@@ -68,7 +71,7 @@ def _download_dbip() -> None:
for month in months:
url = DBIP_URL.format(month=month)
target = _REPO_ROOT / f"dbip-city-lite-{month}.mmdb.gz"
target = _DBIP_DIR / f"dbip-city-lite-{month}.mmdb.gz"
tmp = target.with_suffix(".mmdb.gz.tmp")
logger.info("Downloading %s", url)
try:
@@ -93,7 +96,7 @@ def _download_dbip() -> None:
continue
os.replace(tmp, target)
# Drop older databases so the app never picks up a stale one.
for old in _REPO_ROOT.glob("dbip-city-lite-*.mmdb*"):
for old in _DBIP_DIR.glob("dbip-city-lite-*.mmdb*"):
if old.name != target.name:
old.unlink()
logger.info("DB-IP database updated to %s", target.name)
@@ -102,15 +105,19 @@ def _download_dbip() -> None:
def _geoip_db_path() -> Path | None:
"""Find a DB-IP MMDB in the repo root, preferring an already-decompressed
``.mmdb`` over the matching ``.mmdb.gz``. Returns None if none is present.
"""Find a DB-IP MMDB in the working directory: the ``.mmdb.gz`` download
is canonical (decompressed into RAM at open); a plain ``.mmdb`` left over
from older versions is still usable, and removed once the matching ``.gz``
is present so it does not linger on disk. Returns None if none is present.
"""
mmdb = sorted(_REPO_ROOT.glob("dbip-*.mmdb"))
gz = sorted(_DBIP_DIR.glob("dbip-*.mmdb.gz"))
if gz:
for stale in _DBIP_DIR.glob("dbip-*.mmdb"):
stale.unlink()
return gz[0]
mmdb = sorted(_DBIP_DIR.glob("dbip-*.mmdb"))
if mmdb:
return mmdb[0]
gz = sorted(_REPO_ROOT.glob("dbip-*.mmdb.gz"))
if gz:
return gz[0]
return None
@@ -123,28 +130,23 @@ class GeoIP:
def __init__(self) -> None:
self._reader: object | None = None
def _decompress(self, source: Path, target: Path) -> None:
if target.exists():
return
tmp = target.with_suffix(target.suffix + ".tmp")
with gzip.open(source, "rb") as src, open(tmp, "wb") as dst:
shutil.copyfileobj(src, dst)
os.replace(tmp, target)
def _load(self) -> None:
if self._reader is not None:
return
source = _geoip_db_path()
if source is None:
return
if source.suffix == ".gz":
target = source.with_suffix("")
self._decompress(source, target)
source = target
try:
import maxminddb
self._reader = maxminddb.open_database(str(source))
if source.suffix == ".gz":
# Only the .gz is kept on disk; the database is decompressed
# into RAM (MODE_FD makes the pure-Python Reader .read() the
# buffer — never mmap — and bypasses the C extension).
buf = io.BytesIO(gzip.decompress(source.read_bytes()))
self._reader = maxminddb.open_database(buf, maxminddb.MODE_FD)
else:
self._reader = maxminddb.open_database(str(source))
except Exception:
pass
@@ -332,7 +334,7 @@ async def _broadcast_analytics() -> None:
"""Send the current analytics snapshot to every connected WS client."""
if not _analytics_ws_clients:
return
payload = analytics_store.display_json(_in_menu)
payload = _display_json()
closed = set()
for ws in _analytics_ws_clients:
try:
@@ -368,12 +370,29 @@ def _in_menu(path: str) -> bool:
return resolve(data.menu, path.strip("/")) is not None
def _record_get(request: Request, *, status: int = 200) -> None:
def _display_json() -> str:
"""The current analytics snapshot as JSON for the admin stream.
Adds the site's language context: ``multilingual`` (translation
languages configured) lets the viewer suppress language UI on
single-language sites, ``primary_lang`` (the front page's) lets it skip
the primary-language default case.
"""
return analytics_store.display_json(
_in_menu,
multilingual=bool(data.translate_langs),
primary_lang=i18n.primary_lang(data.menu, ""),
)
def _record_get(request: Request, *, status: int = 200, lang: str = "") -> None:
"""Record the document GET as one raw access-log line in analytics.
Nothing is classified here the true HTTP status, the full request path
(query included), an external referer origin and the preload flag are
stored, and visitor/crawler/abuse classification happens at display time
(query included), an external referer origin, the preload flag and the
rendered content language (``lang``, "" for non-localized responses such
as 404 probes and reserved paths) are stored, and
visitor/crawler/abuse classification happens at display time
(see analytics.Store.display). Idle-time preloads from pagerite.js
(``x-pagerite-preload`` header) are recorded with ``pre=True``: never
counted, but a navigation later served from the in-memory page cache is
@@ -401,6 +420,7 @@ def _record_get(request: Request, *, status: int = 200) -> None:
referer=referer,
accept_language=request.headers.get("accept-language", ""),
pre=bool(request.headers.get("x-pagerite-preload")),
lang=lang,
)
if client_hash is not None:
_schedule_client_enrichment([client_hash])
@@ -442,7 +462,7 @@ async def activity_ws(ws: WebSocket) -> None:
# already printed there): compact UA plus the browser's language tag.
lang, _country = analytics._parse_accept_language(accept_language)
ws.scope.setdefault("state", {})["log_extra"] = " ".join(
part for part in (analytics._compact_user_agent(ua), lang) if part
part for part in (uaparse(ua).pretty, lang) if part
)
await ws.accept()
try:
@@ -460,6 +480,7 @@ async def activity_ws(ws: WebSocket) -> None:
accept_language,
hide=msg.hide,
read=msg.read,
lang=msg.lang,
)
if new_client is not None:
_schedule_client_enrichment([new_client])
@@ -476,7 +497,7 @@ async def analytics_websocket(ws: WebSocket) -> None:
endpoint. Powers the analytics viewer rendered at /_a.
"""
await ws.accept()
await ws.send_text(analytics_store.display_json(_in_menu))
await ws.send_text(_display_json())
_analytics_ws_clients.add(ws)
try:
while True:
+8 -5
View File
@@ -101,7 +101,8 @@ ClientMsg = Hello | Result
def pending_items(data: Data, lang: str) -> list[TransItem]:
"""Fragments of the site still untranslated for ``lang``, deduped by key.
Every page node (published or not) contributes its title and each chunk
Every node (published or not, pages and pure category labels alike)
contributes its title; pages also contribute each chunk
that needs translation (``needs_translation``), is not editor-flagged
no-translate (``node.no_trans``) and has no ``trans`` entry for ``lang``
yet. Content-addressed text (shared paragraphs, repeated titles) appears
@@ -133,8 +134,10 @@ def pending_items(data: Data, lang: str) -> list[TransItem]:
path = f"{prefix}/{slug}" if prefix else slug
# An article whose primary language IS the target needs no
# translation into it — skip its title and chunks entirely.
# Category labels (chunks is None) contribute only their title:
# it is their nav-menu label.
node_lang = node.language or inherited
if node.chunks is not None and node_lang != lang:
if node_lang != lang:
if node.title:
emit(
chunk_key(node.title),
@@ -143,7 +146,7 @@ def pending_items(data: Data, lang: str) -> list[TransItem]:
"title",
context=opening(node),
)
for h in node.chunks:
for h in node.chunks or ():
text = data.chunks.get(h)
if (
text is not None
@@ -178,8 +181,8 @@ def store_results(data: Data, lang: str, items: list[TransResult]) -> list[str]:
for slug, node in sorted_nodes(nodes):
path = f"{prefix}/{slug}" if prefix else slug
node_lang = node.language or inherited
if node.chunks is not None and node_lang != lang:
keys = set(node.chunks)
if node_lang != lang:
keys = set(node.chunks or ())
if node.title:
keys.add(chunk_key(node.title))
if keys & stored:
+100 -34
View File
@@ -21,6 +21,7 @@ import json
import os
import re
from fastapi_vue import env
from html5tagger import HTML, Document, E, Template
from platformdirs import site_data_dir, user_data_path
@@ -263,20 +264,31 @@ def _transition_css_url(transition: str) -> str | None:
def _editor_css_url(vite_url: str | None) -> str | None:
"""URL for the editor-specific stylesheet (Vue component styles).
"""URLs (comma-joined) for the editor-specific stylesheets (Vue
component styles).
This is linked by the public-page edit pen so the editor styles are
loaded before the editor JS dynamic-import resolves.
loaded before the editor JS dynamic-import resolves. Component styles
can land on shared chunks rather than the entry's own stylesheet —
LangSelect's ride on the shared store chunk, as it is also used by the
on-demand public language selector so collect the stylesheets of the
entry and its imported chunks (the same traversal _langselect_assets
does).
"""
if vite_url:
return None
manifest = _manifest()
entry = manifest["src/main.js"]
base = manifest.get(_BASE_CSS_KEY, {}).get("file")
for css in entry.get("css", []):
if css != base:
return f"/{css}"
return None
stylesheets, seen = [], set()
queue = ["src/main.js"]
for key in queue: # grows with imported chunks
if key in seen:
continue
seen.add(key)
entry = manifest[key]
stylesheets += [f"/{css}" for css in entry.get("css", []) if css != base]
queue += entry.get("imports", [])
return ",".join(stylesheets) or None
def _inline_asset(url: str) -> str:
@@ -326,7 +338,7 @@ def _layout(
) -> Template:
"""Page layout template with standard assets and ES-module scripts.
In dev (PAGERITE_VITE_URL set) assets are linked from the Vite dev
In dev (Vite dev-server URL set) assets are linked from the Vite dev
server and stylesheets use ``blocking="render"`` so the browser waits
for them before showing the page, avoiding a flash of unstyled content.
In production all page assets are inlined into the document: stylesheets
@@ -372,8 +384,8 @@ def _layout(
doc.meta(property=key, content=value)
else:
doc.meta(name=key, content=value)
# A custom favicon (from the site editor) is linked explicitly; without
# one, browsers fall back to the build's /favicon.ico by convention.
# A custom favicon (from the site editor) is linked explicitly;
# /favicon.ico redirects to the same store file for non-HTML contexts.
if favicon:
doc.link(rel="icon", href=f"/_f/{favicon}", id="pagerite-favicon")
# Asset URLs for the on-demand bundles (editor, analytics) for
@@ -383,12 +395,16 @@ def _layout(
# dev-server URLs as meta tags (Vite serves the modules and injects
# their CSS for hot reloads); production inlines all page assets and
# carries the on-demand URLs in one JSON script instead.
vite_url = os.environ.get("PAGERITE_VITE_URL")
vite_url = env.vite_url
editor_scripts, editor_css = _editor_assets()
langselect_scripts, langselect_css = _langselect_assets()
config = {
"pagerite:editor-src": editor_scripts[-1],
"pagerite:analytics-src": _analytics_assets()[0][0],
"pagerite:langselect-src": langselect_scripts[-1],
}
if langselect_css:
config["pagerite:langselect-css"] = ",".join(langselect_css)
if editor_css:
config["pagerite:editor-css"] = editor_css
if vite_url:
@@ -1017,6 +1033,42 @@ def _social_meta(
}
def _language_urls(
data: Data,
path: str,
node: Node,
lang: str,
original: str,
base_url: str,
) -> tuple[str, list[tuple[str, str]]]:
"""(canonical, hreflang alternates) for a page (docs/localization.md).
The canonical names the actually served language the plain URL for
the original (for SEO the non-query URL means the article's language),
?lang= for a translation regardless of how the language was arrived
at (query or header). The alternates list the languages the page is
actually available in (``node.langs``; a category label's title counts
as its content): x-default first (the plain, autodetecting URL), then
every available language the original again by its plain URL,
translations by ?lang=. The public language selector keys off these.
("", []) without a base_url.
"""
if not base_url:
return "", []
url = f"{base_url}/{path}"
canonical = url if lang == original else f"{url}?lang={lang}"
alternates = []
if data.translate_langs:
# Only languages the page actually has AND that are still enabled
# site-wide (a disabled target stops being advertised).
enabled = {original, *data.translate_langs}
alternates = [("x-default", url)] + [
(tag, url if tag == original else f"{url}?lang={tag}")
for tag in sorted({original, *node.langs} & enabled)
]
return canonical, alternates
def render_page(
menu: dict[str, Node],
data: Data,
@@ -1046,24 +1098,7 @@ def render_page(
title = _title(path.rpartition("/")[2], node, translation, path)
main = page_content(menu, data, path, translation, link_lang, lang)
social = _social_meta(node, path, title, str(main), brand, base_url)
# Canonical/hreflang URLs (docs/localization.md): the canonical names
# the actually served language — the plain URL for the original (for
# SEO the non-query URL means the article's language), ?lang= for a
# translation — regardless of how the language was arrived at (query
# or header). The alternates are site-wide, the same set on every
# page: the configured translate_langs (the translator works to fill
# them all in), x-default first (the plain, autodetecting URL), then
# every language explicitly, the page's own primary included.
canonical = ""
alternates = []
if base_url:
url = f"{base_url}/{path}"
canonical = url if lang == original else f"{url}?lang={lang}"
if data.translate_langs:
alternates = [("x-default", url)] + [
(tag, f"{url}?lang={tag}")
for tag in sorted({original, *data.translate_langs})
]
canonical, alternates = _language_urls(data, path, node, lang, original, base_url)
return str(
_layout(
*_page_assets(),
@@ -1096,6 +1131,7 @@ def render_category(
theme: str = "",
favicon: str = "",
brand_html: str = "",
base_url: str = "",
transition: str = "cube",
lang: str = i18n.ORIGINAL_LANGUAGE,
translation: Translation | None = None,
@@ -1111,12 +1147,16 @@ def render_category(
With a translation (titles only the category has no Markdown) the
heading, navigation and card text localize per target article
(docs/localization.md); ``link_lang`` replicates the ?lang= override
onto the navigation links as on content pages.
onto the navigation links as on content pages. The hreflang alternates
are computed as on content pages a translated title makes the
language available here too.
"""
node = resolve(menu, path)[-1]
original = i18n.primary_lang(menu, path)
if translation is None:
lang = i18n.primary_lang(menu, path)
lang = original
title = _title(path.rpartition("/")[2], node, translation, path)
_, alternates = _language_urls(data, path, node, lang, original, base_url)
doc = E.article
with doc:
doc.h1(title)
@@ -1133,6 +1173,7 @@ def render_category(
transition,
favicon,
lang=lang,
alternates=alternates,
)(
Title=f"{title} {brand}" if brand else title,
Brand=_brand_link(brand, brand_html, link_lang),
@@ -1186,7 +1227,7 @@ def _page_assets() -> tuple[list[str], list[str]]:
by the entry (e.g. overlayscrollbars.css) is extracted by Vite and must
be linked separately.
"""
vite_url = os.environ.get("PAGERITE_VITE_URL")
vite_url = env.vite_url
if vite_url:
return [f"{vite_url}/src/pagerite.js"], []
if "page" not in _asset_cache:
@@ -1204,7 +1245,7 @@ def _editor_assets() -> tuple[list[str], str | None]:
The shared CSS is already linked on the page, so the pen only needs the
editor-specific stylesheet.
"""
vite_url = os.environ.get("PAGERITE_VITE_URL")
vite_url = env.vite_url
if vite_url:
return [f"{vite_url}/@vite/client", f"{vite_url}/src/main.js"], None
if "editor" not in _asset_cache:
@@ -1216,7 +1257,7 @@ def _editor_assets() -> tuple[list[str], str | None]:
def _analytics_assets() -> tuple[list[str], list[str]]:
"""Script and stylesheet URLs for the analytics page entry."""
vite_url = os.environ.get("PAGERITE_VITE_URL")
vite_url = env.vite_url
if vite_url:
return [f"{vite_url}/src/analytics-main.js"], []
if "analytics" not in _asset_cache:
@@ -1228,6 +1269,31 @@ def _analytics_assets() -> tuple[list[str], list[str]]:
return _asset_cache["analytics"]
def _langselect_assets() -> tuple[list[str], list[str]]:
"""Script and stylesheet URLs for the on-demand public language selector."""
vite_url = env.vite_url
if vite_url:
return [f"{vite_url}/src/langselect-main.js"], []
if "langselect" not in _asset_cache:
manifest = _manifest()
# import() loads no CSS automatically: collect the stylesheets of
# the entry and its imported chunks (LangSelect's ride on the
# shared langs chunk).
scripts, stylesheets, seen = [], [], set()
queue = ["src/langselect-main.js"]
for key in queue: # grows with imported chunks
if key in seen:
continue
seen.add(key)
entry = manifest[key]
if entry.get("isEntry"):
scripts.append(f"/{entry['file']}")
stylesheets += [f"/{css}" for css in entry.get("css", [])]
queue += entry.get("imports", [])
_asset_cache["langselect"] = scripts, stylesheets
return _asset_cache["langselect"]
def render_analytics(
menu: dict[str, Node],
brand: str = SITE_NAME,
+2 -2
View File
@@ -17,7 +17,7 @@ readme = "README.md"
requires-python = ">=3.14"
dependencies = [
"blake3>=1.0.9",
"fastapi-vue~=1.4.2",
"fastapi-vue~=1.6.1",
"fastapi[standard]>=0.141.1",
"html5tagger>=2.0.0",
"httpx>=0.28.1",
@@ -29,7 +29,7 @@ dependencies = [
"platformdirs>=4.11.5",
"pygments>=2.20.0",
"python-slugify>=8.0.4",
"ua-parser>=1.0.2",
"uarite>=0.1.2",
"zstandard>=0.25.0",
]
+10 -5
View File
@@ -1,11 +1,12 @@
#!/usr/bin/env -S uv run
# auto-upgrade@fastapi-vue-setup - remove this if you modify this file
"""Run Vite development server for Vue app and FastAPI backend with auto-reload."""
import argparse
import asyncio
import os
import subprocess
import sys
from contextlib import suppress
from pathlib import Path
import tracerite
@@ -47,11 +48,11 @@ async def run_devserver(
os.environ["PAGERITE_DEV"] = "1"
async with ProcessGroup() as pg:
pg.create_task(check_ports_free(viteurl, backurl))
npm_i = await pg.spawn(*npm_install, cwd=front)
await check_ports_free(viteurl, backurl)
await pg.spawn(*pagerite, *(extra_args or []))
await pg.spawn(*pagerite, *(extra_args or []), vital=True)
await pg.wait(npm_i, ready(backurl, path=HEALTH))
await pg.spawn(*vite, cwd=front)
await pg.spawn(*vite, cwd=front, vital=True)
def main() -> None:
@@ -74,8 +75,12 @@ def main() -> None:
help=f"FastAPI (default: localhost:{DEFAULT_DEV_PORT})",
)
args, extra_args = parser.parse_known_args()
with suppress(KeyboardInterrupt):
try:
asyncio.run(run_devserver(args.listen, args.backend, extra_args))
except* KeyboardInterrupt:
pass # user stopped the devserver: normal exit
except* subprocess.SubprocessError, RuntimeError:
raise SystemExit(1) from None # logged in devutil already; exit 1
HELP_EPILOG = """
-1
View File
@@ -1,4 +1,3 @@
# ruff: noqa: INP001
"""Hatch build hook for building Vue frontend during package build."""
import sys
+2 -3
View File
@@ -1,4 +1,3 @@
# ruff: noqa: INP001
"""Utilities used at build time and in devserver script. No dependencies."""
import logging
@@ -33,7 +32,7 @@ def _check_node_version(node_path: str) -> None:
Raises RuntimeError if version is too old or cannot be determined.
"""
try:
result = subprocess.run( # noqa: S603
result = subprocess.run(
[node_path, "--version"],
capture_output=True,
text=True,
@@ -221,7 +220,7 @@ def build(folder: str = "frontend") -> None:
def run(cmd: list[str]) -> None:
display_cmd = [Path(cmd[0]).stem, *cmd[1:]]
logger.info("### %s", " ".join(display_cmd))
subprocess.run(cmd, check=True, cwd=folder) # noqa: S603
subprocess.run(cmd, check=True, cwd=folder)
try:
run(install_cmd)
+77 -97
View File
@@ -1,111 +1,90 @@
# ruff: noqa: INP001
"""Utilities meant for devserver script, used only in source repository with dev deps."""
from __future__ import annotations
import asyncio
import subprocess
import sys
from asyncio.subprocess import Process
from contextlib import suppress
from pathlib import Path
from typing import TYPE_CHECKING, Any, Self
from subprocess import CalledProcessError
from typing import TYPE_CHECKING, Any
from urllib.parse import urlsplit
from buildutil import find_dev_tool, find_install_tool, logger
from fastapi_vue.hostutil import parse_endpoint
from buildutil import find_dev_tool, find_install_tool, logger
if TYPE_CHECKING:
from collections.abc import Coroutine
from collections.abc import Awaitable
class ProcessGroup:
"""Manage async subprocesses with automatic cleanup, like TaskGroup for processes."""
class ProcessGroup(asyncio.TaskGroup):
"""TaskGroup with structured ownership of async subprocesses."""
def __init__(self) -> None:
"""Initialize empty process tracking."""
self._procs: list[asyncio.subprocess.Process] = []
self._cmds: dict[int, str] = {} # pid -> command name
def __init__(self, *, terminate_timeout: float = 10) -> None:
"""Set the grace period before terminate() escalates to kill()."""
super().__init__()
self._terminate_timeout = terminate_timeout
self._cmds: dict[Process, tuple[str, ...]] = {}
async def spawn(
self,
*cmd: str,
cwd: str | None = None,
) -> asyncio.subprocess.Process:
"""Spawn a subprocess and track it."""
cmd_name = Path(cmd[0]).stem
logger.info(">>> %s", " ".join([cmd_name, *cmd[1:]]))
proc = await asyncio.create_subprocess_exec(*cmd, cwd=cwd)
self._procs.append(proc)
self._cmds[proc.pid] = cmd_name
return proc
self, *cmd: str, cwd: str | None = None, vital: bool = False
) -> Process:
"""Spawn and own a subprocess. If a vital process exits, the group cancels."""
async def wait(
self,
*waitables: "asyncio.subprocess.Process | Coroutine[Any, Any, Any]",
) -> None:
"""Wait for processes/coroutines to complete, raise SystemExit on failure."""
async def run() -> None:
name = Path(cmd[0]).stem
logger.info(">>> %s", " ".join([name, *cmd[1:]]))
try:
proc = await asyncio.create_subprocess_exec(*cmd, cwd=cwd)
self._cmds[proc] = cmd
started.set_result(proc)
except Exception as e: # noqa: BLE001
started.set_exception(e)
return
async def wait_proc(proc: asyncio.subprocess.Process) -> None:
returncode = await proc.wait()
if returncode != 0:
cmd_name = self._cmds.get(proc.pid, "unknown")
raise subprocess.CalledProcessError(returncode, cmd_name)
tasks = [
wait_proc(w) if isinstance(w, asyncio.subprocess.Process) else w
for w in waitables
]
try:
await asyncio.gather(*tasks)
except subprocess.CalledProcessError as e:
logger.warning("%s failed with exit status %d", e.cmd, e.returncode)
raise SystemExit(1) from None
async def __aenter__(self) -> Self:
"""Enter the async context manager."""
return self
async def __aexit__(self, exc_type: type[BaseException] | None, *_: object) -> None:
"""Wait for one process to exit, terminate others, then wait for all."""
await self._cleanup(immediate=exc_type is not None)
async def _cleanup(self, *, immediate: bool = False) -> None:
running = [p for p in self._procs if p.returncode is None]
if not running:
return
if not immediate:
# Wait for any one process to exit
with suppress(asyncio.CancelledError):
await asyncio.wait(
[asyncio.create_task(p.wait()) for p in running],
return_when=asyncio.FIRST_COMPLETED,
)
# Terminate remaining processes
for p in self._procs:
if p.returncode is None:
try:
returncode = await proc.wait()
finally:
with suppress(ProcessLookupError):
p.terminate()
# Wait for all to finish (with overall timeout), shielded from cancellation
still_running = [p for p in self._procs if p.returncode is None]
if still_running:
with suppress(asyncio.CancelledError):
proc.terminate()
try:
await asyncio.shield(
asyncio.wait_for(
asyncio.gather(*[p.wait() for p in still_running]),
timeout=10,
),
)
await asyncio.wait_for(proc.wait(), self._terminate_timeout)
except TimeoutError:
for p in self._procs:
if p.returncode is None:
with suppress(ProcessLookupError):
p.kill()
await p.wait()
with suppress(ProcessLookupError):
proc.kill()
await proc.wait()
if vital:
logger.warning("Vital process %s exited", name)
raise CalledProcessError(returncode, cmd)
started = asyncio.get_running_loop().create_future()
self.create_task(run())
return await asyncio.shield(started)
async def wait(self, *waitables: Process | Awaitable) -> tuple[Any, ...]:
"""Wait concurrently and return results in argument order."""
async def task(w: Process | Awaitable) -> Any:
if not isinstance(w, Process):
return await w
if retcode := await w.wait():
cmd = self._cmds[w]
logger.warning(
"Process %s exited with status %d", Path(cmd[0]).stem, retcode
)
raise CalledProcessError(retcode, cmd)
return retcode
async with asyncio.TaskGroup() as group:
tasks = [group.create_task(task(w)) for w in waitables]
return tuple(task.result() for task in tasks)
async def http_get_server(url: str, timeout: float) -> str | None: # noqa: ASYNC109
async def http_get_server(url: str, timeout: float) -> str | None:
"""GET url with plain asyncio streams, return the response Server header.
Returns an empty string when the server responds without a Server header,
@@ -128,31 +107,32 @@ async def http_get_server(url: str, timeout: float) -> str | None: # noqa: ASYN
writer.close()
except OSError, EOFError, ValueError, TimeoutError:
return None
for line in data.decode("latin-1").split("\r\n"):
for line in data.decode(errors="replace").split("\r\n"):
if line.lower().startswith("server:"):
return line.split(":", 1)[1].strip()
return line[7:].strip()
return ""
async def check_ports_free(*urls: str) -> None:
"""Verify URLs are not responding (ports are free). Raise SystemExit if any respond."""
"""Verify URLs are not responding (ports are free).
async def check(url: str) -> None:
server = await http_get_server(url, timeout=0.1)
Meant to run as a task inside a TaskGroup. Logs the conflict and raises
RuntimeError (handled like a failed process) if any URL responds.
"""
servers = await asyncio.gather(*(http_get_server(url, timeout=0.1) for url in urls))
for url, server in zip(urls, servers, strict=True):
if server is not None:
logger.warning(
logger.error(
"Conflicting %s already running at %s", server or "server", url
)
raise SystemExit(1)
await asyncio.gather(*[check(url) for url in urls])
raise RuntimeError(url)
async def ready(url: str, path: str = "", max_attempts: int = 50) -> None:
"""Wait for the server to be ready by polling an endpoint.
Use empty path to disable the check and make this return immediately.
Raises SystemExit(1) if server doesn't start in time.
Logs, then raises RuntimeError if the server doesn't start in time.
"""
if not path:
return
@@ -162,8 +142,8 @@ async def ready(url: str, path: str = "", max_attempts: int = 50) -> None:
logger.info("✓ Backend ready!")
return
if attempt == max_attempts - 1:
logger.warning("Backend didn't start in time")
raise SystemExit(1)
logger.error("Backend at %s didn't start in time", url)
raise RuntimeError(url)
await asyncio.sleep(0.1)