analytics viewer: copy-to-clipboard UAs, crawler info links, shared icon-btn

- UA lines click to copy the raw UA; abuse rows with rotating clients
  copy every variation, one per line (uaRaws)
- "Copied!" popup is viewport-fixed at the click point; cell overflow
  clipped the old absolute one
- language tags show their Intl.DisplayNames full name on hover
- a 🔗 link after the pretty UA opens the crawler info page uarite
  provides
- the emoji-symbol dim-until-hover idiom (opacity 0.7 -> 1) is now one
  global .icon-btn class in pagerite.css, shared by the edit pens,
  auth links, editor icon buttons and the new UA link

AGENTS.md

@@ -25,7 +25,7 @@ Pagerite is a CMS. See `docs` for the full design and implementation details. Ke
   - `markdown.py` — markdown-it-py renderer.
   - `views.py` — shared page layout and rendering; theme/user-font resolution across `THEME_DIRS` / `FONT_DIRS` (cwd, site, platform data roots, then built-in `pagerite/themes/`, see `docs/themes-and-assets.md`).
   - `seed.py` — demo content, written only on first database creation.
-  - `analytics.py` — visit analytics collection (see `docs/analytics.md`).
+  - `analytics.py` — visit analytics collection (see `docs/analytics.md`). UA formatting/bot detection comes from the **uarite** package.
 - `frontend/src/` — Vue editor and public-page JS entries.
   - `main.js` — Vue editor app entry.
   - `analytics-main.js` — analytics page entry (mounts `AnalyticsView` at `/_a`).

docs/analytics.md

@@ -63,13 +63,27 @@ Each `Client` record (shared by every event, keyed by hash):
   when a database is available,
 - `city` — city name from the DB-IP MMDB lookup, when available,
 - `ua` — raw `User-Agent` string,
-- `ua_pretty` — compact display form of the UA (browser/OS/device) when
-  parsable, otherwise the raw string,
 - `hide` — true for admin clients (`hide` message field): everything this
   client ever did is recorded but excluded from every statistic and from the
   viewer payload.  This is the one flag set at record time — it is a client
   property, not a classification.
 
+The viewer payload adds one display-time field to each client, never
+persisted (stored records keep the default and old data always follows the
+current uarite version):
+
+- `uarite` — the `uarite.UA` dataclass from parsing the raw UA
+  (`pretty`/`engine`/`os`/`provider`/`kind`/`url`): the crawler name for
+  bots,
+  with a category suffix only where a provider runs crawlers of more than
+  one kind (`GPTBot (AI)` vs `OAI-SearchBot (search)`, `Googlebot (search)`
+  vs `Google-Extended (AI)`; single-kind providers stay plain: `Facebook`,
+  `WhatsApp`), `Browser/major OS` on the desktop, the device where that is
+  the relevant information (iPhone reports its iOS version, Android phones
+  their model instead of the OS), otherwise the raw string; `url` is the
+  crawler's info page when uarite knows one (rendered as a 🔗 link after the
+  pretty UA in the viewer), `kind` drives the bot classification.
+
 A reverse-DNS lookup is attempted for each new client and the result, when
 available, is stored as `host`; local/reserved/multicast addresses are
 skipped.  If a DB-IP MMDB file (`dbip-*.mmdb` or `dbip-*.mmdb.gz`) is present
@@ -169,7 +183,9 @@ for misses.
   `_CRAWLER_TIMEOUT` (10 s) is a crawler hit — plain bots that only fetch
   documents never register as visits. JS-running crawlers (Googlebot,
   GoogleOther, Applebot, ...) do connect and send messages, but their UA
-  gives them away (`_is_bot_ua`): their messages are ignored at display
+  gives them away (`_is_bot_ua`, backed by `uarite.uaparse` — which
+  also knows the disguised ones: facebookexternalhit, Google-Extended,
+  WhatsApp, ...): their messages are ignored at display
   time, so their GETs never match and land in the crawler list too. Real-
   browser bots whose UA does not match are caught by engagement: a visit
   whose total reported reading time is under 5 seconds (`_MIN_VISIT_READ`;

frontend/src/AnalyticsView.vue

@@ -218,6 +218,7 @@ const abuseRows = computed(() => formatAbuseRows(rangeData.value?.abuse || [], c
                     :ip-display="v.ipDisplay"
                     :ua="v.ua"
                     :ua-raw="v.uaRaw"
+                    :ua-url="v.uaUrl"
                     :country="v.country"
                     :city="v.city"
                     :lang="v.lang"
@@ -253,6 +254,7 @@ const abuseRows = computed(() => formatAbuseRows(rangeData.value?.abuse || [], c
                     :ip-display="c.ipDisplay"
                     :ua="c.ua"
                     :ua-raw="c.uaRaw"
+                    :ua-url="c.uaUrl"
                     :country="c.country"
                     :city="c.city"
                     :lang="c.lang"
@@ -299,6 +301,8 @@ const abuseRows = computed(() => formatAbuseRows(rangeData.value?.abuse || [], c
                     :ip-display="a.ipDisplay"
                     :ua="a.ua"
                     :ua-raw="a.uaRaw"
+                    :ua-url="a.uaUrl"
+                    :ua-raws="a.uaRaws"
                     :country="a.country"
                     :city="a.city"
                     :lang="a.lang"
@@ -506,22 +510,6 @@ const abuseRows = computed(() => formatAbuseRows(rangeData.value?.abuse || [], c
 .visit-table .clickable-list,
 .visit-table .last-seen {
   cursor: pointer;
-  position: relative;
-}
-
-.visit-table :deep(.copy-popup) {
-  position: absolute;
-  bottom: calc(100% + 0.25rem);
-  left: 50%;
-  transform: translateX(-50%);
-  padding: 0.15rem 0.4rem;
-  background: var(--text, CanvasText);
-  color: var(--bg, Canvas);
-  border-radius: 0.25rem;
-  font-size: 0.75rem;
-  white-space: nowrap;
-  pointer-events: none;
-  z-index: 10;
 }
 
 .crawler-top-uas {

frontend/src/BannerEditor.vue

@@ -403,14 +403,6 @@ onUnmounted(() => {
   margin-left: auto;
   padding: 0 0.2rem;
   font-size: 1rem;
-  background: none;
-  border: none;
-  cursor: pointer;
-  opacity: 0.7;
-}
-
-.block-head .icon-btn:hover {
-  opacity: 1;
 }
 
 /* The banner design selector stays compact; the upload button is pushed

frontend/src/SiteEditor.vue

@@ -782,14 +782,6 @@ onUnmounted(() => {
   margin-left: auto;
   padding: 0 0.2rem;
   font-size: 1rem;
-  background: none;
-  border: none;
-  cursor: pointer;
-  opacity: 0.7;
-}
-
-.block-head .icon-btn:hover {
-  opacity: 1;
 }
 
 .text-input {

frontend/src/VisitorCell.vue

@@ -4,15 +4,20 @@
 // Clicking the IP copies the full address to the clipboard.
 // ``variantCount`` overrides the UA line to warn when multiple client
 // fingerprints share the same IP (e.g. a scanner rotating UAs).
+// Clicking the UA line copies the raw UA(s) to the clipboard, one per line
+// (``uaRaws`` carries every variation for multi-client IPs).
 import { computed } from 'vue'
 import * as flagSvgs from 'country-flag-icons/string/3x2'
-import { copyIp, formatLang } from './analytics/format.js'
+import { copyIp, copyList, formatLang } from './analytics/format.js'
+import { langName } from './langs.js'
 
 const props = defineProps({
   ip: { type: String, default: '' },
   ipDisplay: { type: String, default: '—' },
   ua: { type: String, default: '' },
   uaRaw: { type: String, default: '' },
+  uaRaws: { type: String, default: '' },
+  uaUrl: { type: String, default: '' },
   country: { type: String, default: '' },
   city: { type: String, default: '' },
   lang: { type: String, default: '' },
@@ -26,6 +31,7 @@ const hasCity = computed(() => !!(props.city && props.city !== '—'))
 const hasLocale = computed(() => hasCountry.value || hasCity.value)
 const langValue = computed(() => props.langDisplay || formatLang(props.lang))
 const showLang = computed(() => langValue.value && langValue.value !== '—')
+const uaCopy = computed(() => props.uaRaws || props.uaRaw)
 
 function flagSvg(code) {
   return flagSvgs[code?.toUpperCase()] || ''
@@ -58,10 +64,16 @@ function countryName(code) {
       </div>
       <div class="visitor-row">
         <div class="ua-line">
-          <small v-if="variantCount > 1" class="muted variant-hint">{{ variantCount }} client variations</small>
+          <small v-if="variantCount > 1" class="muted variant-hint clickable-ip"
-          <small v-else class="muted" :title="uaRaw">{{ ua || '—' }}</small>
+                 :title="uaCopy"
+                 @click="copyList(uaCopy, $event)">{{ variantCount }} client variations</small>
+          <small v-else class="muted clickable-ip" :title="uaRaw"
+                 @click="copyList(uaCopy, $event)">{{ ua || '—' }}</small><a v-if="uaUrl && variantCount <= 1"
+             class="ua-link icon-btn" :href="uaUrl"
+             target="_blank" rel="noopener noreferrer"
+             @click.stop>🔗</a>
         </div>
-        <div v-if="showLang && variantCount <= 1" class="locale-lang"><small class="muted">{{ langValue }}</small></div>
+        <div v-if="showLang && variantCount <= 1" class="locale-lang"><small class="muted" :title="langName(lang)">{{ langValue }}</small></div>
       </div>
     </div>
   </td>
@@ -121,6 +133,13 @@ function countryName(code) {
   text-align: left;
 }
 
+.ua-link {
+  text-decoration: none;
+  font-size: 0.75em;
+  margin-left: 0.2em;
+  vertical-align: middle;
+}
+
 .locale-lang {
   flex: 0 0 auto;
   overflow: hidden;

frontend/src/analytics/format.js

@@ -25,32 +25,31 @@ export const hostIP = (ip) => {
   }
 }
 
-function showCopiedFeedback(el) {
+function showCopiedFeedback(el, event) {
-  if (!el || typeof document === 'undefined') return
+  if (typeof document === 'undefined') return
   const popup = document.createElement('span')
   popup.textContent = 'Copied!'
   popup.className = 'copy-popup'
+  // Fixed to the viewport at the click point: table cells clip absolute
+  // popups with their overflow: hidden ellipsis styling.
+  const x = event?.clientX ?? 0
+  const y = event?.clientY ?? 0
   popup.style.cssText =
-    'position:absolute;bottom:calc(100% + 0.25rem);left:50%;' +
+    `position:fixed;left:${x}px;top:${y}px;` +
-    'transform:translateX(-50%);padding:0.15rem 0.4rem;' +
+    'transform:translate(-50%, calc(-100% - 0.5rem));padding:0.15rem 0.4rem;' +
     'background:var(--text, CanvasText);color:var(--bg, Canvas);' +
     'border-radius:0.25rem;font-size:0.75rem;white-space:nowrap;' +
-    'pointer-events:none;z-index:10;'
+    'pointer-events:none;z-index:100;'
-  el.classList.add('has-copy-popup')
+  document.body.appendChild(popup)
-  el.appendChild(popup)
+  setTimeout(() => popup.remove(), 1200)
-  setTimeout(() => {
-    popup.remove()
-    el.classList.remove('has-copy-popup')
-  }, 1200)
 }
 
 /** Copy the full IP to the clipboard and show a brief "Copied!" popup. */
 export async function copyIp(ip, event) {
   if (!ip) return
-  const el = event?.currentTarget
   try {
     await navigator.clipboard.writeText(ip)
-    showCopiedFeedback(el)
+    showCopiedFeedback(event?.currentTarget, event)
   } catch {
     /* ignore */
   }
@@ -59,10 +58,9 @@ export async function copyIp(ip, event) {
 /** Copy arbitrary text to the clipboard and show a brief "Copied!" popup. */
 export async function copyList(text, event) {
   if (!text) return
-  const el = event?.currentTarget
   try {
     await navigator.clipboard.writeText(text)
-    showCopiedFeedback(el)
+    showCopiedFeedback(event?.currentTarget, event)
   } catch {
     /* ignore */
   }
@@ -342,7 +340,7 @@ export function countCrawlerUas(crawlers, clients) {
   const counts = {}
   for (const c of crawlers || []) {
     const client = (clients || {})[c.client] || {}
-    const value = client.ua_pretty || client.ua || '(no UA)'
+    const value = client.uarite?.pretty || client.ua || '(no UA)'
     counts[value] = (counts[value] || 0) + 1
   }
   return Object.entries(counts).sort((a, b) => b[1] - a[1])
@@ -421,8 +419,9 @@ export function formatCrawlerRows(crawlers, clients, pageTree, now = Date.now())
         ip: client.ip || '',
         ipDisplay: isHost ? mainDomain(host) : hostIP(client.ip) || client.ip || '—',
         isHost,
-        ua: client.ua_pretty || client.ua || '—',
+        ua: client.uarite?.pretty || client.ua || '—',
         uaRaw: client.ua || '',
+        uaUrl: client.uarite?.url || '',
         lang: client.lang || '—',
         langDisplay: formatLang(client.lang),
         country: client.country || '—',
@@ -505,6 +504,13 @@ export function formatAbuseRows(abuse, clients, pageTree, now = Date.now()) {
       const client = (clients || {})[g.lastClient] || {}
       const host = client.host || ''
       const isHost = !!host
+      const uaRaws = [
+        ...new Set(
+          [...g.clientHashes]
+            .map((h) => (clients || {})[h]?.ua)
+            .filter(Boolean),
+        ),
+      ].join('\n')
       return {
         lastSeen: formatWhen(g.lastStart, now),
         lastSeenIso: formatWhenIso(g.lastStart),
@@ -527,8 +533,10 @@ export function formatAbuseRows(abuse, clients, pageTree, now = Date.now()) {
         ip: client.ip || g.ip,
         ipDisplay: isHost ? mainDomain(host) : hostIP(client.ip || g.ip) || client.ip || g.ip || '—',
         isHost,
-        ua: client.ua_pretty || client.ua || '—',
+        ua: client.uarite?.pretty || client.ua || '—',
         uaRaw: client.ua || '',
+        uaUrl: client.uarite?.url || '',
+        uaRaws,
         lang: client.lang || '—',
         langDisplay: formatLang(client.lang),
         country: client.country || '—',
@@ -582,8 +590,9 @@ export function formatVisitRows(visits, clients, pageTree, now = Date.now()) {
       lang: dash(client.lang),
       country: dash(client.country),
       city: dash(client.city),
-      ua: client.ua_pretty || client.ua || '—',
+      ua: client.uarite?.pretty || client.ua || '—',
       uaRaw: client.ua || '',
+      uaUrl: client.uarite?.url || '',
       utm: utm || '—',
       utmTitle,
     }

frontend/src/assets/pagerite.css

@@ -756,6 +756,20 @@ article {
   position: relative;
 }
 
+/* Emoji/symbol icon buttons and links: dim until hovered. */
+.icon-btn {
+  padding: 0;
+  font: inherit;
+  background: none;
+  border: none;
+  cursor: pointer;
+  opacity: 0.7;
+}
+
+.icon-btn:hover {
+  opacity: 1;
+}
+
 .edit-link {
   position: absolute;
   top: 0.2rem;
@@ -763,12 +777,6 @@ article {
   left: -2.2rem;
   z-index: 2;
   /* stay above full-bleed .wide images */
-  font: inherit;
-  background: none;
-  border: none;
-  padding: 0;
-  cursor: pointer;
-  opacity: 0.7;
   text-shadow: 0 0 0.1em black;
 }
 
@@ -790,24 +798,14 @@ article h2 .edit-section {
   opacity: 0.35;
 }
 
-.edit-link:hover {
-  opacity: 1;
-}
-
 /* Login/profile links injected by pagerite.js when Paskia SSO is in use.
    They live inside the .editor-pens flex container in the banner's top-right
-   corner and inherit its reset; keep only their opacity/text-shadow tweaks. */
+   corner and inherit its reset; keep only their text-shadow tweak. */
 .editor-pens a.login-link,
 .editor-pens a.profile-link {
-  opacity: 0.7;
   text-shadow: 0 0 0.1em black;
 }
 
-.editor-pens a.login-link:hover,
-.editor-pens a.profile-link:hover {
-  opacity: 1;
-}
-
 article p,
 article li,
 article dd {

frontend/src/pagerite.js

@@ -132,16 +132,16 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
     if (line != null) {
       // Section pen on an anchored h2: opens the page editor at the
       // section's markdown source line (data-line, from the backend).
-      btn.className = "edit-link edit-section";
+      btn.className = "edit-link edit-section icon-btn";
       btn.title = "edit section";
       btn.textContent = "🖊️";
       btn.dataset.editorLine = line;
     } else if (mode === "page") {
-      btn.className = "edit-link edit-page";
+      btn.className = "edit-link edit-page icon-btn";
       btn.title = "edit page";
       btn.textContent = "🖊️";
     } else {
-      btn.className = "edit-link site-edit-link";
+      btn.className = "edit-link site-edit-link icon-btn";
       btn.title = "site settings";
       btn.textContent = "⚙️";
     }
@@ -165,7 +165,7 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
 
   function makeAuthLink(admin) {
     const a = document.createElement("a");
-    a.className = admin ? "profile-link" : "login-link";
+    a.className = (admin ? "profile-link" : "login-link") + " icon-btn";
     a.href = "/auth/";
     a.title = admin ? "profile" : "log in";
     a.textContent = admin ? "\u{1F510}" : "\u{1F511}";
@@ -211,7 +211,7 @@ import { reconnectPolicy, socketSlot, watchConnecting } from "./reconnect";
         if (canEdit && !onAnalytics) {
           // Analytics viewer is now a normal page at /_a.
           const a = document.createElement("a");
-          a.className = "edit-link analytics-link";
+          a.className = "edit-link analytics-link icon-btn";
           a.href = "/_a";
           a.title = "analytics";
           a.textContent = "📊";

pagerite/analytics.py

@@ -60,32 +60,17 @@ from urllib.parse import parse_qs, urlencode, urlparse
 
 import blake3
 import msgspec
-from ua_parser import parse
+from uarite import UA, uaparse
 
 
-def _compact_user_agent(ua: str) -> str:
+def _display_client(client: Client) -> Client:
-    """Format a User-Agent string into a compact display form.
+    """Client copy with ``uarite`` filled in by the current uarite.
 
-    Returns the original UA when the parser cannot identify the browser/OS.
+    The parsed UA is a display-time field: stored records always carry the
+    default (None), so it never lands on disk, and old records always
+    follow current uarite rules.
     """
-    if not ua or not ua.strip() or ua == "-":
+    return msgspec.structs.replace(client, uarite=uaparse(client.ua))
-        return ""
-    r = parse(ua)
-    browser = r.user_agent.family if r.user_agent else None
-    ver = r.user_agent.major if r.user_agent else ""
-    os_name = r.os.family if r.os else None
-    dev = r.device.family if r.device else None
-    if browser in (None, "Other") and os_name in (None, "Other"):
-        return ua
-    if browser and browser != "Other":
-        browser = browser.split()[0]
-    else:
-        browser = ""
-    os_name = os_name if os_name and os_name != "Other" else ""
-    if dev in (None, "Other") or dev == browser:
-        dev = ""
-    parts = [f"{browser}/{ver}" if browser else "", os_name, dev]
-    return " ".join(p for p in parts if p).strip()
 
 
 class Ping(msgspec.Struct, omit_defaults=True):
@@ -172,11 +157,14 @@ class Client(msgspec.Struct, omit_defaults=True):
     city: str = ""
     #: Raw User-Agent header.
     ua: str = ""
-    #: Compact display form of ``ua`` (browser/OS/device) when parsable.
-    ua_pretty: str = ""
     #: True for admin clients (hide=1 ping): everything this client ever did
     #: is excluded from all statistics and from the viewer payload.
     hide: bool = False
+    #: Display-time parsed UA (uarite.UA dataclass: pretty/engine/os/
+    #: provider/kind/url).  Set only on the display-payload copies by
+    #: ``_display_client`` — stored records keep the default, so it is never
+    #: persisted and old data always follows the current uarite version.
+    uarite: UA | None = None
 
 
 # --- Display DTOs -------------------------------------------------------
@@ -418,17 +406,22 @@ _MIN_VISIT_READ = 5
 _FAVICON_RETRY = timedelta(days=7)
 
 #: UAs of JS-running crawlers, which would register as visitors on their
-#: activity messages.  Anything calling itself a "bot" or "spider" matches;
+#: activity messages.  ``uarite`` knows the common crawlers
-#: known crawlers without those tokens (GoogleOther) are listed as extra
+#: and link-preview fetchers (including disguised ones such as
-#: alternates.  No source verification: a spoofed bot UA just lands in the
+#: facebookexternalhit and Google-Extended) plus any UA with a
-#: crawler list, and scanners that probe telltale paths are caught by the
+#: bot/spider/crawler/scanner token.  No source verification: a spoofed
-#: abuse rules anyway.
+#: bot UA just lands in the crawler list, and scanners that probe
-_BOT_UA = re.compile(r"bot|spider|googleother", re.IGNORECASE)
+#: telltale paths are caught by the abuse rules anyway.
 
 
 def _is_bot_ua(ua: str) -> bool:
-    """True when the UA claims a crawler identity (bot or spider)."""
+    """True when the UA is not a regular browser.
-    return bool(_BOT_UA.search(ua))
+
+    Every real browser registers as ``kind == "browser"``; anything else
+    (recognized crawler/previewer, generic bot token, or an unclassified
+    HTTP client such as httpx) is not a visitor.
+    """
+    return uaparse(ua).kind != "browser"
 
 
 #: Plain-404 count per IP within ``_ABUSE_404_WINDOW`` that classifies it as
@@ -560,7 +553,6 @@ class Store:
             self.data.clients[h] = Client(
                 ip=ip,
                 ua=ua,
-                ua_pretty=_compact_user_agent(ua),
                 lang=lang,
                 country=country,
             )
@@ -940,7 +932,7 @@ class Store:
                 and not self._hidden(g.client)
                 and ip_of.get(g.client, "") in abuse_ips
             ],
-            clients={h: c for h, c in data.clients.items() if not c.hide},
+            clients={h: _display_client(c) for h, c in data.clients.items() if not c.hide},
             favicons={
                 origin: f"/_f/{f.file}"
                 for origin, f in data.favicons.items()

pagerite/tracking.py

@@ -26,6 +26,7 @@ import httpx
 import msgspec
 from fastapi import APIRouter, Request, WebSocket, WebSocketDisconnect
 from fastapi.responses import Response
+from uarite import uaparse
 
 from pagerite import analytics
 from pagerite.data import resolve
@@ -443,7 +444,7 @@ async def activity_ws(ws: WebSocket) -> None:
     # already printed there): compact UA plus the browser's language tag.
     lang, _country = analytics._parse_accept_language(accept_language)
     ws.scope.setdefault("state", {})["log_extra"] = " ".join(
-        part for part in (analytics._compact_user_agent(ua), lang) if part
+        part for part in (uaparse(ua).pretty, lang) if part
     )
     await ws.accept()
     try:

pyproject.toml

@@ -29,7 +29,7 @@ dependencies = [
     "platformdirs>=4.11.5",
     "pygments>=2.20.0",
     "python-slugify>=8.0.4",
-    "ua-parser>=1.0.2",
+    "uarite>=0.1.2",
     "zstandard>=0.25.0",
 ]