diff --git a/docs/analytics.md b/docs/analytics.md index 474649b..021927b 100644 --- a/docs/analytics.md +++ b/docs/analytics.md @@ -38,6 +38,9 @@ Each `Get` record (one per served document): (`x-pagerite-preload` header): never counted as a view, crawler hit or abuse — recorded only so a navigation later served from the in-memory page cache (which issues no GET at all) can be attributed this GET's status, +- `lang` — rendered content language of the served document (the resolved + language of a localized page), `""` for non-localized responses (404 + probes, reserved paths), - `client` — 6-byte blake3 hash referencing `Analytics.clients`. 304 revalidation responses return before recording and are not logged. @@ -51,7 +54,9 @@ Each `Msg` record (one per pagerite.js activity message over `/_ws`): - `to` — navigation target (validated at record time: internal slug path or external https URL; anything else is dropped — sanitation, not classification), -- `read` — active seconds spent on `fr` since the previous report. +- `read` — active seconds spent on `fr` since the previous report, +- `lang` — rendered language reported by the client for the page the + activity happened on (the page's ``; `""` from old clients). Each `Client` record (shared by every event, keyed by hash): @@ -104,7 +109,9 @@ The client (`pagerite.js`) keeps a WebSocket connection to `/_ws` for the whole browsing session and sends activity messages over it — JSON text frames matching the server's `Ping` msgspec struct with the fields `fr` (source path), `to` (navigation target), `read` (active seconds on `fr` -since the last report) and `hide`; falsy fields are omitted. One channel +since the last report), `lang` (the rendered language of the page the +activity happened on — its ``) and `hide`; falsy fields are +omitted. One channel follows the session, so the activity of a visit stays tied together, and while the user is active the accumulated reading time is flushed every few seconds: the times are incremental, so a disconnection simply leaves the @@ -173,10 +180,16 @@ for misses. (`_SESSION_GAP`). A fresh page load with an already-open visit (second tab) extends it, logging a `(direct)` transition. The visit's trail holds first-seen targets in order; `read` updates accumulate active seconds on - the trail item matching `fr`. Each trail item's HTTP status comes from + the trail item matching `fr` (preferring the item whose language matches + the report, so seconds after a language switch land on the new-language + step). Each trail item's HTTP status comes from the client's latest GET for that path — preloads included, which is what allows 404 pages to render red in the viewer even when the navigation - itself was served from the page cache. The entry page's referer and + itself was served from the page cache. Each trail item also carries the + rendered language: the client's report, for the entry page falling back + to its GET's rendered language (old clients don't send one); a page + re-visited in a different language becomes a distinct trail step instead + of merging into the existing item. The entry page's referer and `utm_*` tags come from the GET that loaded it (within 10 s before the first message). - **Crawler hits**: a document GET no activity message matched within @@ -232,6 +245,10 @@ to tell misses from real pages at a glance. The `Display` payload contains the derived `visits`, `crawlers` and `abuse` rows (structs `Visit`/`Nav`/`TrailItem`, `CrawlerHit`, `AbuseHit` — display DTOs only, never persisted), the visible `clients`, the fetched `favicons`, +the site language context (`multilingual` — translation languages are +configured, so the viewer can suppress language UI on single-language +sites — and `primary_lang` — the front page's primary language, so the +viewer can skip the primary-language default case), and the aggregates below. Each derived `Visit`: @@ -243,8 +260,9 @@ Each derived `Visit`: - `trail` — the entry page and everything seen afterwards, keyed by the timestamp of first sight (insertion order = first-seen order). Each item holds `to` (page path or external exit URL), the accumulated active - reading time in seconds (`read`) and the most recent HTTP status seen - for the target (`status`), + reading time in seconds (`read`), the most recent HTTP status seen + for the target (`status`) and the rendered language (`lang`; a page + seen in two languages within one visit gets one item per language), - `navs` — every navigation (`fr`, `to`), keyed by its timestamp, repeats included. The aggregates are computed from this log, - `utm` — `utm_*` query parameters from the landing URL, as a dict. @@ -257,7 +275,8 @@ Each derived `CrawlerHit`: - `referer` — external https origin of the request, `""` for direct/none, - `query` — raw query string of the request, - `status` — HTTP status of the served response (200 for a real page, 404 - for a category placeholder or missing page). + for a category placeholder or missing page), +- `lang` — rendered content language of the served document (from the GET). Each derived `AbuseHit`: diff --git a/frontend/src/AnalyticsView.vue b/frontend/src/AnalyticsView.vue index f7a3e3a..77bfa92 100644 --- a/frontend/src/AnalyticsView.vue +++ b/frontend/src/AnalyticsView.vue @@ -160,9 +160,15 @@ watch(range, (r) => { const clients = computed(() => data.value?.clients || {}) const favicons = computed(() => data.value?.favicons || {}) -const visitRows = computed(() => formatVisitRows(visits.value, clients.value, pageTree.value, now.value)) +// Site language context from the payload: drives the discreet rendered- +// language markers in the visit/crawler rows (multilingual sites only). +const site = computed(() => ({ + multilingual: !!data.value?.multilingual, + primaryLang: data.value?.primary_lang || '', +})) +const visitRows = computed(() => formatVisitRows(visits.value, clients.value, pageTree.value, now.value, site.value)) const crawlers = computed(() => rangeData.value?.crawlers || []) -const crawlerRows = computed(() => formatCrawlerRows(crawlers.value, clients.value, pageTree.value, now.value)) +const crawlerRows = computed(() => formatCrawlerRows(crawlers.value, clients.value, pageTree.value, now.value, site.value)) const abuseRows = computed(() => formatAbuseRows(rangeData.value?.abuse || [], clients.value, pageTree.value, now.value)) @@ -211,7 +217,8 @@ const abuseRows = computed(() => formatAbuseRows(rangeData.value?.abuse || [], c {{ v.utm }} - + + formatAbuseRows(rangeData.value?.abuse || [], c + formatAbuseRows(rangeData.value?.abuse || [], c vertical-align: bottom; } +/* Same flag chip as the visitor cells (VisitorCell.vue); the flags here + mark the language the page was read in. */ +.visit-table .flag { + display: inline-flex; + width: 18px; + height: 12px; + border-radius: 2px; + overflow: hidden; + border: 1px solid var(--line); + box-shadow: 0 0 0 1px rgba(0, 0, 0, 0.2) inset; + vertical-align: middle; +} + +.visit-table .flag :deep(svg) { + width: 100%; + height: 100%; + display: block; +} + .visit-table .clickable-list { cursor: pointer; max-width: 22rem; diff --git a/frontend/src/TrailLink.vue b/frontend/src/TrailLink.vue index e59f0df..c892775 100644 --- a/frontend/src/TrailLink.vue +++ b/frontend/src/TrailLink.vue @@ -6,6 +6,7 @@ const props = defineProps({ step: { type: Object, required: true }, count: { type: Number, default: 0 }, favicons: { type: Object, default: null }, + flags: { type: Array, default: () => [] }, }) defineEmits(['close']) @@ -38,6 +39,7 @@ const title = computed(() => { {{ formatCount(count) }}× {{ step.slug }} + @@ -48,4 +50,23 @@ const title = computed(() => { margin-right: 0.25em; vertical-align: -0.1em; } + +/* Same flag chip as the visitor cells (VisitorCell.vue). */ +.flag { + display: inline-flex; + width: 18px; + height: 12px; + margin-left: 0.25em; + border-radius: 2px; + overflow: hidden; + border: 1px solid var(--line); + box-shadow: 0 0 0 1px rgba(0, 0, 0, 0.2) inset; + vertical-align: middle; +} + +.flag :deep(svg) { + width: 100%; + height: 100%; + display: block; +} diff --git a/frontend/src/analytics/format.js b/frontend/src/analytics/format.js index 97da743..2731e73 100644 --- a/frontend/src/analytics/format.js +++ b/frontend/src/analytics/format.js @@ -2,6 +2,7 @@ * Formatters and aggregators for summary sections: totals and the recent * visit trail. */ +import { flagFor, langName } from '../langs.js' /** * IPv4 unchanged, IPv6 returns the /64 network prefix in compact form. @@ -373,7 +374,7 @@ export function mainDomain(host, limit = 24) { * their own site there — rendered with its favicon like visit referers. * ``clients`` maps client hashes to client records. */ -export function formatCrawlerRows(crawlers, clients, pageTree, now = Date.now()) { +export function formatCrawlerRows(crawlers, clients, pageTree, now = Date.now(), site = { multilingual: false, primaryLang: '' }) { const titles = buildTitleMap(pageTree) const groups = new Map() for (const c of crawlers || []) { @@ -384,10 +385,12 @@ export function formatCrawlerRows(crawlers, clients, pageTree, now = Date.now()) lastStart: 0, referer: '', pages: new Map(), + langs: new Set(), } const start = new Date(c.start).getTime() if (start > g.lastStart) g.lastStart = start if (c.referer) g.referer = c.referer + if (c.lang) g.langs.add(c.lang) if (c.entry?.startsWith('/')) { const existing = g.pages.get(c.entry) || { count: 0, status: c.status || 200 } existing.count += 1 @@ -408,6 +411,11 @@ export function formatCrawlerRows(crawlers, clients, pageTree, now = Date.now()) const client = g.client || {} const host = client.host || '' const isHost = !!host + // Rendered languages read, shown only when they say something the + // primary language alone would not (multilingual sites only). + const langs = [...g.langs].sort() + const showLangs = + site.multilingual && (langs.length > 1 || (langs[0] && langs[0] !== site.primaryLang)) return { lastSeen: formatWhen(g.lastStart, now), lastSeenIso: formatWhenIso(g.lastStart), @@ -416,6 +424,9 @@ export function formatCrawlerRows(crawlers, clients, pageTree, now = Date.now()) pages: [...g.pages.entries()] .sort((a, b) => b[1].count - a[1].count) .map(([path, info]) => ({ ...stepOf(path, titles), count: info.count, status: info.status })), + readFlags: showLangs + ? langs.map((l) => ({ flag: flagFor(l), name: langName(l) })).filter((f) => f.flag) + : [], ip: client.ip || '', ipDisplay: isHost ? mainDomain(host) : hostIP(client.ip) || client.ip || '—', isHost, @@ -550,23 +561,44 @@ export function formatAbuseRows(abuse, clients, pageTree, now = Date.now()) { * Format raw visit records as rows for a technical table. Returns objects * with display strings; missing values become "—". ``trail`` starts with the * external referer (when present), then the entry page and any further internal - * pages or external exit origins. Only the 20 most recent visits are shown. - * ``clients`` maps client hashes to client records. + * pages or external exit origins; consecutive views of the same page (e.g. a + * language switch re-view) merge into one step that keeps the + * consecutive-distinct rendered languages, summed read time, and the latest + * status. On multilingual sites the rendered languages surface as flag + * icons: a visit read entirely in one non-primary language gets ``rowFlag``, + * and a visit spanning languages gets per-step ``langFlags`` markers where + * the language begins or changes. Only the 20 most recent visits are shown. + * ``clients`` maps client hashes to client records; ``site`` carries the + * payload's multilingual/primary-language context. */ -export function formatVisitRows(visits, clients, pageTree, now = Date.now()) { +export function formatVisitRows(visits, clients, pageTree, now = Date.now(), site = { multilingual: false, primaryLang: '' }) { const titles = buildTitleMap(pageTree) return [...(visits || [])].reverse().slice(0, 20).map((v) => { const client = (clients || {})[v.client] || {} - const trail = Object.values(v.trail || {}) + const steps = Object.values(v.trail || {}) .map((item) => { const step = stepOf(item.to, titles) if (step) { if (item.read) step.readSeconds = item.read if (item.status) step.status = item.status + if (item.lang) step.lang = item.lang } return step }) .filter(Boolean) + const trail = [] + for (const step of steps) { + const prev = trail[trail.length - 1] + if (prev && prev.path === step.path) { + if (step.lang && step.lang !== prev.langs[prev.langs.length - 1]) prev.langs.push(step.lang) + if (step.readSeconds) prev.readSeconds = (prev.readSeconds || 0) + step.readSeconds + if (step.status) prev.status = step.status + } else { + step.langs = step.lang ? [step.lang] : [] + trail.push(step) + } + } + const distinctLangs = new Set(trail.flatMap((s) => s.langs)) const utmKeys = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content'] const utmValues = utmKeys.map((k) => (v.utm || {})[k]).filter(Boolean) const utm = utmValues.length ? utmValues.join(' · ') : '' @@ -576,7 +608,7 @@ export function formatVisitRows(visits, clients, pageTree, now = Date.now()) { const dash = (s) => (s || '—') const host = client.host || '' const isHost = !!host - return { + const row = { lastSeen: formatWhen(v.start, now), lastSeenIso: formatWhenIso(v.start), lastSeenLocal: formatWhenLocal(v.start), @@ -596,5 +628,27 @@ export function formatVisitRows(visits, clients, pageTree, now = Date.now()) { utm: utm || '—', utmTitle, } + if (site.multilingual && distinctLangs.size) { + if (distinctLangs.size === 1) { + const [tag] = distinctLangs + const flag = flagFor(tag) + if (flag && tag !== site.primaryLang) { + row.rowFlag = flag + row.rowFlagTitle = langName(tag) + } + } else { + // Flag the steps where the rendered language begins or changes; + // lang-less steps keep the comparison chain going, they never flag. + let lastLang = null + for (const step of trail) { + if (!step.langs.length) continue + if (!lastLang || step.langs[step.langs.length - 1] !== lastLang) { + step.langFlags = step.langs.map(flagFor).filter(Boolean) + } + lastLang = step.langs[step.langs.length - 1] + } + } + } + return row }) } diff --git a/frontend/src/pagerite.js b/frontend/src/pagerite.js index 3fdc86c..d9179c5 100644 --- a/frontend/src/pagerite.js +++ b/frontend/src/pagerite.js @@ -637,6 +637,8 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect"; if (to) msg.to = to; const secs = Math.round(read / 1000); if (secs > 0) msg.read = secs; + const lang = document.documentElement.lang; + if (lang) msg.lang = lang; if (!msg.to && !msg.read) return; report(msg); } @@ -815,6 +817,10 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect"; const y = scrollY; // a language switch is not a navigation: keep scroll await load(currentPath, false); scrollTo(0, y); + // Log the switch as a trail event in the new language (load() updated + // ): the ping matches the switch's GET server-side, so it is + // not misclassified as a crawler hit. + ping({ to: currentPath }); }); // --- Fetch navigation ------------------------------------------------ diff --git a/pagerite/analytics.py b/pagerite/analytics.py index 75bc1fe..dbb0834 100644 --- a/pagerite/analytics.py +++ b/pagerite/analytics.py @@ -2,8 +2,9 @@ Raw recording, display-time classification. Every document GET is appended to ``Analytics.gets`` as a raw access-log line (path with query string, true -HTTP status, external referer origin, preload flag) and every pagerite.js -activity message from the /_ws WebSocket is appended to ``Analytics.msgs`` +HTTP status, external referer origin, preload flag, rendered content +language) and every pagerite.js activity message from the /_ws WebSocket is +appended to ``Analytics.msgs`` (navigations ``fr`` -> ``to`` and active reading-time updates). Nothing is classified when it is recorded: whether a client turns out to be a reader, a crawler or a scanner is decided by ``Store.display()`` from the raw @@ -92,6 +93,9 @@ class Ping(msgspec.Struct, omit_defaults=True): read: int = 0 #: Admin client: record but hide everything from the statistics. hide: bool = False + #: Rendered language of the page the activity happened on (the page's + #: ````, sent by pagerite.js). + lang: str = "" class Get(msgspec.Struct, omit_defaults=True): @@ -115,6 +119,9 @@ class Get(msgspec.Struct, omit_defaults=True): #: never counted as a view/crawler/abuse hit; recorded only so a later #: cache-served navigation can be attributed this GET's status. pre: bool = False + #: Rendered content language of the served document; "" for + #: non-localized responses (404 probes, reserved paths). + lang: str = "" class Msg(msgspec.Struct, omit_defaults=True): @@ -135,6 +142,9 @@ class Msg(msgspec.Struct, omit_defaults=True): to: str = "" #: Active reading time (seconds) spent on ``fr`` since the last report. read: int = 0 + #: Rendered language reported by the client for the page the activity + #: happened on. + lang: str = "" class Client(msgspec.Struct, omit_defaults=True): @@ -191,6 +201,8 @@ class TrailItem(msgspec.Struct, omit_defaults=True): ``read`` accumulates active reading time (seconds) across the whole visit; ``status`` is the most recent HTTP status seen for the target. + A page seen in two rendered languages within one visit (a mid-article + language switch) gets one item per language. """ to: str @@ -198,6 +210,10 @@ class TrailItem(msgspec.Struct, omit_defaults=True): read: int = 0 #: Most recent HTTP status of the response (200 or 404). status: int = 200 + #: Rendered language of the target: the client's report, for the entry + #: page falling back to its GET's rendered language; "" when unknown + #: (old clients or data from before language recording). + lang: str = "" class Visit(msgspec.Struct, omit_defaults=True): @@ -206,8 +222,9 @@ class Visit(msgspec.Struct, omit_defaults=True): ``trail`` holds the entry page and everything seen afterwards, keyed by the timestamp of first sight (insertion order = first-seen order); re-visiting an already seen target updates its item instead of - appending. Client metadata is held in ``Analytics.clients`` keyed by - ``client``. + appending — unless the client reports a different rendered language for + it, which appends a distinct item (a mid-article language switch). + Client metadata is held in ``Analytics.clients`` keyed by ``client``. """ start: datetime @@ -241,6 +258,8 @@ class CrawlerHit(msgspec.Struct, omit_defaults=True): query: str = "" #: HTTP status of the served response (200 or 404 for content pages). status: int = 200 + #: Rendered content language of the served document (from the GET). + lang: str = "" class AbuseHit(msgspec.Struct, omit_defaults=True): @@ -319,6 +338,12 @@ class Display(msgspec.Struct, omit_defaults=True): views: dict[str, dict[str, int]] = {} #: New visits per 5-minute bucket: bucket ISO -> count (sparse). site_visits: dict[str, int] = {} + #: Site context: true when translation languages are configured, so the + #: viewer can suppress language UI on single-language sites. + multilingual: bool = False + #: The site's primary language (the front page's), so the viewer can + #: skip the primary-language default case. + primary_lang: str = "" def _bucket(now: datetime) -> str: @@ -594,22 +619,25 @@ class Store: referer: str = "", accept_language: str = "", pre: bool = False, + lang: str = "", ) -> bytes | None: """Append one document GET to the raw log. ``path`` is the full request path, query string included; ``status`` the true HTTP status of the response; ``referer`` the raw Referer header (reduced here to an external https origin, "" when internal - or absent); ``pre`` marks idle-time preloads from pagerite.js. + or absent); ``pre`` marks idle-time preloads from pagerite.js; + ``lang`` the rendered content language of the served document ("" + for non-localized responses such as 404 probes and reserved paths). Returns the client hash when the client record was just created (so the caller can schedule async enrichment), else None. """ - lang, country = _parse_accept_language(accept_language) - client_hash = _client_hash(ip, ua, lang) + client_lang, country = _parse_accept_language(accept_language) + client_hash = _client_hash(ip, ua, client_lang) new = client_hash not in self.data.clients if new: - self._ensure_client(ip, ua, lang, country=country) + self._ensure_client(ip, ua, client_lang, country=country) self.data.gets.append( Get( t=datetime.now(UTC), @@ -618,6 +646,7 @@ class Store: status=status, ref=_origin(referer) or "", pre=pre, + lang=lang, ) ) self._save() @@ -632,6 +661,7 @@ class Store: accept_language: str = "", hide: bool = False, read: int = 0, + lang: str = "", ) -> bytes | None: """Append one client activity message (``Ping`` from pagerite.js) to the raw log. @@ -642,16 +672,18 @@ class Store: stored raw and filtered at display time, so future rule changes lose nothing. ``hide`` flags the client record as an admin; the message itself is recorded normally and hidden at display time like - everything else the client ever did. + everything else the client ever did. ``lang`` is the rendered + language reported by the client for the page the activity happened + on. Returns the client hash when the client record was just created (so the caller can schedule async enrichment), else None. """ - lang, country = _parse_accept_language(accept_language) - client_hash = _client_hash(ip, ua, lang) + client_lang, country = _parse_accept_language(accept_language) + client_hash = _client_hash(ip, ua, client_lang) new = client_hash not in self.data.clients if new: - self._ensure_client(ip, ua, lang, country=country) + self._ensure_client(ip, ua, client_lang, country=country) if hide: self.data.clients[client_hash].hide = True fr = (_internal_path(fr) or "") if fr else "" @@ -663,7 +695,14 @@ class Store: target = _external_target(to) or "" if target or read > 0: self.data.msgs.append( - Msg(t=datetime.now(UTC), client=client_hash, fr=fr, to=target, read=read) + Msg( + t=datetime.now(UTC), + client=client_hash, + fr=fr, + to=target, + read=read, + lang=lang, + ) ) if target or read > 0 or hide: self._save() @@ -700,7 +739,13 @@ class Store: self.data.favicons[origin] = Favicon(file=file, fetched=datetime.now(UTC)) self._save() - def display(self, in_menu: Callable[[str], bool] | None = None) -> Display: + def display( + self, + in_menu: Callable[[str], bool] | None = None, + *, + multilingual: bool = False, + primary_lang: str = "", + ) -> Display: """Build the viewer payload from the raw events. All classification happens here, so the stored data is independent @@ -722,10 +767,17 @@ class Store: - visits: the remaining messages, grouped per client with a new visit after ``_SESSION_GAP`` of inactivity. Trail statuses come from the client's GETs (preloads included — a cache-served - navigation's only GET is its preload); the entry referer and UTM - tags from the GET that loaded the entry page. + navigation's only GET is its preload); trail languages come from + the client's messages (the entry item falling back to its GET's + rendered language), and a page re-visited in a different rendered + language becomes a distinct trail step. The entry referer and + UTM tags come from the GET that loaded the entry page. Hidden (admin) clients are excluded from every list and aggregate. + ``multilingual`` and ``primary_lang`` are site context (translation + languages configured, the front page's primary language) copied + onto the payload so the viewer can suppress language UI on + single-language sites and skip the primary-language default case. """ in_menu = in_menu or (lambda path: False) data = self.data @@ -836,7 +888,11 @@ class Store: g.path.split("?", 1)[1] if "?" in g.path else "" ) visit.trail[m.t] = TrailItem( - to=m.to, status=status_at(h, m.to, m.t) + to=m.to, + status=status_at(h, m.to, m.t), + # The client's report wins; the entry GET fills + # in for old clients that don't send lang. + lang=m.lang or (g.lang if g is not None else ""), ) visits.append(visit) else: @@ -844,18 +900,40 @@ class Store: visit.navs[m.t] = Nav(fr=fr, to=m.to) status = status_at(h, m.to, m.t) # First-seen only: repeat pages and repeated exits - # update the existing trail item instead of appending. + # update the existing trail item instead of + # appending — but a repeat in a different rendered + # language (a mid-article language switch) becomes + # a distinct step. for item in visit.trail.values(): if item.to == m.to: - item.status = status + if m.lang and item.lang and m.lang != item.lang: + visit.trail[m.t] = TrailItem( + to=m.to, status=status, lang=m.lang + ) + else: + item.status = status + if not item.lang: + item.lang = m.lang break else: - visit.trail[m.t] = TrailItem(to=m.to, status=status) + visit.trail[m.t] = TrailItem( + to=m.to, status=status, lang=m.lang + ) if m.read > 0 and m.fr and visit is not None: + # A page appears in the trail once per language seen: + # land the seconds on the matching-language step when + # the client reports one, else on the first-seen item. + read_item: TrailItem | None = None for item in visit.trail.values(): - if item.to == m.fr: - item.read += m.read + if item.to != m.fr: + continue + if read_item is None: + read_item = item + if m.lang and item.lang == m.lang: + read_item = item break + if read_item is not None: + read_item.read += m.read last_t = m.t # --- crawler hits: document GETs no message matched @@ -889,6 +967,7 @@ class Store: referer=g.ref, query=query, status=g.status, + lang=g.lang, ) ) @@ -912,6 +991,7 @@ class Store: referer=visit.referer if first else "", query=query if first else "", status=item.status, + lang=item.lang, ) ) first = False @@ -938,6 +1018,8 @@ class Store: for origin, f in data.favicons.items() if f.file }, + multilingual=multilingual, + primary_lang=primary_lang, ) for visit in kept: bucket = _bucket(visit.start) @@ -959,6 +1041,14 @@ class Store: nbuckets[nb] = nbuckets.get(nb, 0) + 1 return display - def display_json(self, in_menu: Callable[[str], bool] | None = None) -> str: + def display_json( + self, + in_menu: Callable[[str], bool] | None = None, + *, + multilingual: bool = False, + primary_lang: str = "", + ) -> str: """The ``display()`` payload as a JSON string for the WebSocket.""" - return msgspec.json.encode(self.display(in_menu)).decode() + return msgspec.json.encode( + self.display(in_menu, multilingual=multilingual, primary_lang=primary_lang) + ).decode() diff --git a/pagerite/pages.py b/pagerite/pages.py index bd90ca7..dcd3898 100644 --- a/pagerite/pages.py +++ b/pagerite/pages.py @@ -152,7 +152,8 @@ async def show_page(request: Request, path: str) -> Response: if node is not None and node.published and node.chunks is not None: # Language selection (docs/localization.md): ?lang= wins when a # translation exists, else header logic. Analytics keep the raw - # Accept-Language header regardless of the selection. + # Accept-Language header regardless of the selection, and record + # the resolved language as the GET's rendered language. query_lang = request.query_params.get("lang") lang = i18n.select_language( query_lang, @@ -175,7 +176,7 @@ async def show_page(request: Request, path: str) -> Response: if request.headers.get("if-none-match") == etag: return Response(status_code=304) if _is_trackable_path(path): - _record_get(request) + _record_get(request, lang=lang) return _html_response( request, "page", @@ -205,7 +206,7 @@ async def show_page(request: Request, path: str) -> Response: ) link_lang = i18n.base_tag(query_lang or "") if _is_trackable_path(path): - _record_get(request, status=404) + _record_get(request, status=404, lang=lang) return _html_response( request, "category", diff --git a/pagerite/tracking.py b/pagerite/tracking.py index e76579a..0b652a8 100644 --- a/pagerite/tracking.py +++ b/pagerite/tracking.py @@ -28,7 +28,7 @@ from fastapi import APIRouter, Request, WebSocket, WebSocketDisconnect from fastapi.responses import Response from uarite import uaparse -from pagerite import analytics +from pagerite import analytics, i18n from pagerite.data import resolve from pagerite.files import _hash_name, file_store from pagerite.state import SITE_URL, _html_response, analytics_store, data @@ -334,7 +334,7 @@ async def _broadcast_analytics() -> None: """Send the current analytics snapshot to every connected WS client.""" if not _analytics_ws_clients: return - payload = analytics_store.display_json(_in_menu) + payload = _display_json() closed = set() for ws in _analytics_ws_clients: try: @@ -370,12 +370,29 @@ def _in_menu(path: str) -> bool: return resolve(data.menu, path.strip("/")) is not None -def _record_get(request: Request, *, status: int = 200) -> None: +def _display_json() -> str: + """The current analytics snapshot as JSON for the admin stream. + + Adds the site's language context: ``multilingual`` (translation + languages configured) lets the viewer suppress language UI on + single-language sites, ``primary_lang`` (the front page's) lets it skip + the primary-language default case. + """ + return analytics_store.display_json( + _in_menu, + multilingual=bool(data.translate_langs), + primary_lang=i18n.primary_lang(data.menu, ""), + ) + + +def _record_get(request: Request, *, status: int = 200, lang: str = "") -> None: """Record the document GET as one raw access-log line in analytics. Nothing is classified here — the true HTTP status, the full request path - (query included), an external referer origin and the preload flag are - stored, and visitor/crawler/abuse classification happens at display time + (query included), an external referer origin, the preload flag and the + rendered content language (``lang``, "" for non-localized responses such + as 404 probes and reserved paths) are stored, and + visitor/crawler/abuse classification happens at display time (see analytics.Store.display). Idle-time preloads from pagerite.js (``x-pagerite-preload`` header) are recorded with ``pre=True``: never counted, but a navigation later served from the in-memory page cache is @@ -403,6 +420,7 @@ def _record_get(request: Request, *, status: int = 200) -> None: referer=referer, accept_language=request.headers.get("accept-language", ""), pre=bool(request.headers.get("x-pagerite-preload")), + lang=lang, ) if client_hash is not None: _schedule_client_enrichment([client_hash]) @@ -462,6 +480,7 @@ async def activity_ws(ws: WebSocket) -> None: accept_language, hide=msg.hide, read=msg.read, + lang=msg.lang, ) if new_client is not None: _schedule_client_enrichment([new_client]) @@ -478,7 +497,7 @@ async def analytics_websocket(ws: WebSocket) -> None: endpoint. Powers the analytics viewer rendered at /_a. """ await ws.accept() - await ws.send_text(analytics_store.display_json(_in_menu)) + await ws.send_text(_display_json()) _analytics_ws_clients.add(ws) try: while True: