27 Commits
Author SHA1 Message Date
LeoVasanko 9779857dcd Update documentation for the v0.5.0 release 2026-09-10 21:30:24 +00:00
LeoVasanko 9ec4f877eb Resume tracking: per-episode positions, player-agnostic fallback, series continue point
- Fix merged playback-state dropping season/episode, which broke series
  resume restore entirely.
- Track watch progress per episode (episodes map keyed S..E.., with done
  markers on completion) while keeping one continue point per series as
  the last watched episode, used for spoiler protection and season
  selection. MPC-BE tracker resumes from each episode's own position.
- Assumed-playback fallback for any player and server-only mode: a launch
  starts a session and the guessed position (base + elapsed, capped at
  runtime) is written when frontend activity resumes; the real MPC-BE
  tracker overrides guesses via timestamps. Frontend reports input
  activity (throttled) via POST /api/activity.
- Ignore watches under 5 minutes in both trackers.
- Episode tiles show a small quadrant-circle watch indicator; near end
  counts as fully watched, no data shows nothing.
- Fix episode audio ownership: single audioOwnerKey shared by mouse hover
  and keyboard/gamepad focus, no longer clobbered by playback sync;
  idle fade can be re-armed by continued activity.
2026-09-10 20:56:03 +00:00
LeoVasanko 6a6a012efe Series detail: curtain poster browser for seasons
Replace the 3D ring jukebox with a flat curtain browser: selected season
faces the screen as top of the left stack, side cards rotate about their
outer edge and recede into the screen, each side evenly spaced with pitch
adapting to its card count so cards never leave the stage.

- Mount episode videos only after the season switch settles, then
  preload=auto + autoplay (avoids mid-animation media setup jank)
- Start videos immediately (no stagger) once an episode is pointed at;
  keep staggered starts while navigating the season selector
- Keyboard/gamepad entry into the season row lands on the selected
  season via data-nav-entry-col instead of the x-closest card
- Black episode tiles; hidden (ahead-of-cursor) tiles get a sheen and
  vignette veil that cross-fades on reveal/conceal
- Nudge season info panel 8px left for balanced gaps
2026-09-10 16:47:33 +00:00
LeoVasanko 2c28ab1f25 Series detail: one season at a time with 3D jukebox season selector
Render only the selected season's episodes; switch via a perspective-arc
poster carousel with floating season info, deferred video loading with
still-image fade-in, and spoiler fade for episodes ahead of the cursor.
2026-09-10 03:34:52 +00:00
LeoVasanko e6eadb2ecd Working dev reload on Windows; fast quiet scanner shutdown
uvicorn's reloader is unusable on Windows for this app: it restarts the
server child via CTRL_C_EVENT, which is never delivered to a plain spawn
child (no console process group of its own), so the child kept running
old code and the reloader blocked in process.join() after the first
reload — the scanner included.  And when a child does restart, uvicorn
hands it parent-bound sockets that ProactorEventLoop cannot register
with IOCP (WinError 87 on accept), while the selector loop would break
asyncio subprocesses (ffmpeg/ffprobe showreel generation).

In dev mode on Windows, mediahive now runs a small supervisor instead:
it watches the package with watchfiles and respawns a fresh child
process (which binds its own sockets) on every change, and terminates
the child on Ctrl-C.  POSIX keeps uvicorn's native reload.

Shutdown: RootScanner.stop() no longer does a final best-effort state
save — those to_thread writes ran on the asyncio default executor, whose
non-daemon threads blocked interpreter exit, so log lines appeared after
the shell prompt returned.  All scanner state is already persisted after
each completed scan and when the showreel queue drains.  Shutdown-time
log lines (scan cancelled, showreel worker stopping) move to debug.
2026-09-09 15:15:27 +00:00
LeoVasanko ff6b195973 Fix rescan loop: short-circuited registration dropped files from state
The directory worker registered video files with
'node["changed"] = node["changed"] or await _register_candidate(...)',
so once one file in a directory was found changed, the rest were never
registered: they fell out of known_paths, were pruned from the persisted
seen-mtimes at finalize, and were rediscovered as new on every scan.
That rotating ~10-item subset was reprocessed (metadata, TMDb lookups,
logs) every 30 seconds forever.

Also fix the self-sustaining reprocess loop for items that produce no
index entries (no playable file, no parseable episodes, non-media):
they never appear in the index, so the DB-aware mtime gating alone could
never skip them.  They are now recorded in a persisted empty_mtimes map
(scan-state.json) and skipped while their mtime is unchanged.

Episode/playable/blu-ray finder caches are now cleared at the start of
each scan: an empty result cached before a download finished (or during
a transient network-mount error) previously stuck for the process
lifetime and reported "no episodes found" for series that have episodes.

Logging: a no-change scan now emits a single line with item count and
duration; per-item "up to date" chatter moved to debug; change scans
report the new/changed count plus per-item reason lines (stored vs
current mtime, index/empty-marker membership), finalize prune/commit
traces, and scan-state load/save confirmations.

Add AGENTS.md noting the project's Python 3.14 baseline (PEP 758
unparenthesized except) so tooling does not flag it as a syntax error.
2026-09-09 14:40:32 +00:00
LeoVasanko 36e2fdd5ff Preserve browse scroll positions exactly when returning from details
Skip ensure-visible and synced-row animations on focus restore, and
snapshot/restore panel scrollTop plus synced-row scrollLeft around
detail navigation so the browse page comes back pixel-identical.
2026-09-09 04:51:10 +00:00
LeoVasanko 8f462b9e1d Track backoff state only for large empty subtrees
dir_state now records only subtrees with >= _MIN_TRACKED_DIRS (8)
directories that yielded no items of interest; hot or tiny trees fall
back to default every-pass scheduling with no bookkeeping. Per-directory
classification already works at any depth, so e.g. torrents/Music is
pruned at its own root while torrents/Movies stays hot; backed-off
subtrees are still re-examined when their (exponentially growing, 1h
cap) timer expires.
2026-09-09 04:27:15 +00:00
LeoVasanko 838db5b55c Make rescans robust: DB-aware mtime gating, wipe-proof discovery
- RootScanner takes indexed_paths (store torrent keys); a candidate is
  skipped only when its mtime is unchanged AND it is present in the index,
  so an emptied/partial index self-heals on the next pass (non-indexable
  OTHER candidates still skip on mtime alone)
- discovery: a directory that cannot be listed (transient SMB failure)
  now carries its previously known paths into the Sync set instead of
  dropping them, and an unreadable media root fails the whole scan —
  previously both produced a partial/empty known_paths that made
  sync_torrent_paths delete items that still exist on disk
- worker: broad per-directory exception guard (carry known paths, stay
  hot, keep other workers running); node completion hardened against
  double-decrement; backoff state not updated from cancelled walks
2026-09-09 04:21:17 +00:00
LeoVasanko e5300eaac0 Faster GUI exit, graceful second-instance startup, pretty crash window
- scanner.stop: await cancelled tasks concurrently under one 2s budget
  and persist scan/reel/probe state concurrently (was sequential)
- supervisor.shutdown: stop root contexts concurrently (per-root scanner
  waits and network-mount snapshot flushes no longer add up)
- winmain: retry log-file rotation while a previous instance still holds
  mediahive.log (Windows denies rename of an open file — the old instance
  dying slowly made new instances crash with a PermissionError); after 2s
  show a 'waiting for previous instance' window, retry up to 15s, then
  fall back to a per-pid log file
- winmain: gui_main() wraps winmain() and renders unhandled exceptions as
  a TraceRite HTML page in a pywebview window instead of PyInstaller's
  plain-text fatal-error dialog; __main__ GUI mode uses it too
2026-09-09 03:46:27 +00:00
LeoVasanko d1f1b9ecb8 Parallel adaptive discovery walk; fix PyInstaller tracerite data files
- scanner: replace serial DFS with 16-worker priority work-queue, one
  scandir round trip per directory (DirEntry mtimes), per-subtree
  exponential backoff (persisted in scan-state.json v2), leaf candidates
  gated on max(dir, newest video file) mtime so growing downloads and
  lazy NTFS dir mtimes are handled
- spec: collect_data_files('tracerite') so style.css/script.js ship
- guibuild: tolerate non-UTF-8 console when printing status glyphs
- ruff format pass on indexer.py/index_store.py (hook requirement)
2026-09-09 03:37:31 +00:00
LeoVasanko 26af7c633b Remove rtorrent scripting from the repository. 2026-09-09 02:14:11 +00:00
LeoVasanko c072f15cb5 Make helper scripts directly executable with uv run shebang
guibuild.py and release.py use the project environment; rtorrent-manager.py
is standalone and declares its own dependency via uv script metadata.
2026-09-09 02:08:49 +00:00
LeoVasanko 23030cd1c4 Remove wheel build hook, following upstream fastapi-vue convention
The wheel hook was a local addition (76cd022) that rebuilt the frontend
on every wheel build. Only the sdist build should invoke node; wheel
builds (including wheel-from-sdist) package the prebuilt frontend-build.
Restores scripts/fastapi-vue/buildhook.py to its pristine upstream state.
2026-09-09 02:02:25 +00:00
LeoVasanko 2a39e1f0ea Fix scanner rescan bugs and wasted work
Bugs fixed:
- Partial rescans no longer replace whole index entries: Upsert events now
  carry the scanned torrent paths and IndexStore merges partial rebuilds,
  so touching one season no longer drops the others from listings.
- Deleted torrents are now detected: discovery reports the full candidate
  set via a new Sync event after each completed scan, and the store prunes
  file entries/episodes/seasons/items whose torrents vanished.
- Seen mtimes are committed only after a scan completes successfully, so
  cancelled/failed scans retry their items.
- Showreel worker no longer rebroadcasts stale whole items; it sends narrow
  MovieShowreel/EpisodeReel events that update only reel fields.
- Permanently failing reel generations (e.g. DoVi/libplacebo on GPU-less
  machines) and short-video re-queueing are no longer retried every scan:
  reel outcomes persist in reel-state.json with exponential backoff.

Optimizations:
- Persist scan state (scan-state.json) and ffmpeg probe results
  (probe-cache.json, keyed by mtime+size, failures included) under
  .mediahive/, written only when changed. A warm restart over an unchanged
  library drops from ~57 s + 467 queued reel tasks to ~0.2 s with an empty
  queue (measured on a 163-item root).
- Bounded parallelism (semaphores + gather) for cast-profile downloads,
  TMDb title lookups, and season-detail fetches.
- Incremental TMDb-id index bookkeeping instead of rebuilding per upsert;
  removed dead trigger_scan.

See docs/scanning-review.md for the full findings/fixes/measurements report.
2026-09-04 00:00:53 +00:00
LeoVasanko d3addadf14 Work around Starlette StreamingResponse bug with not aclosing the async generator by implementing a custom StreamingFileResponse class. 2026-09-03 13:20:18 +00:00
LeoVasanko 0a4d54c1b7 Update GUI logging to fastapi-vue too. 2026-09-03 12:30:53 +00:00
LeoVasanko c2776d2e2d fastapi-vue-setup 1.4.1 upgrade, replaces our own access logging and more. 2026-09-03 12:19:50 +00:00
LeoVasanko e5bc736ffa Fix episode preview playback and add staggered stopping
- Observe the video element itself instead of el.closest('.episode-tile'):
  ref callbacks can fire before ancestors are attached, so no tile was
  ever observed and the visibility allowlist blocked all playback.
- Use a blocklist for off-screen culling (play until reported off-screen)
  so observer lag/failure degrades to the old always-on behavior.
- Default to the first season playing on page view again.
- Keep preload="auto" so tiles show a first-frame preview picture;
  playback gating already prevents GPU use from inactive videos.
- Stop videos staggered (same 500ms step as startup) on idle stop,
  season switch and scroll culling; same for movie page showreels.
2026-09-03 03:19:35 +00:00
LeoVasanko 258fc79753 Reduce preview video GPU load: idle stop, browse-gated seasons, viewport culling
- Add useIdlePreviewPlayback composable: stops preview videos after 30s
  without input and restarts them staggered on activity; also stops
  while the tab is hidden.
- SeriesFullView: no episode videos play until a season is browsed
  (keyboard/gamepad focus or mouse hover); only near-viewport tiles of
  the active season play (IntersectionObserver); inactive seasons no
  longer preload video data; sync is incremental so playing tiles are
  not rewound on scroll.
- MediaDetail: header showreel videos use cancellable stagger timers,
  pause when scrolled out of view, and honor the idle stop.
- Images: lazy/async decoding on series page posters; content-visibility:
  auto on media cards to skip rendering off-screen row items.
2026-09-03 02:23:22 +00:00
LeoVasanko 7a60ef5384 Implement Ctrl-R/F5 reload handler with intent to allow reloads in GUI build. 2026-09-02 19:42:17 +00:00
LeoVasanko 22454f2d29 Update README 2026-06-03 00:41:52 +00:00
LeoVasanko 589c789d4d Fix pywebview qt dep on non-Windows platforms. 2026-06-03 00:33:37 +00:00
LeoVasanko 5f2454d8e8 Simpler and cross-platform added_on timestamp determination. 2026-06-03 00:01:06 +00:00
LeoVasanko 2fa15132fb Make default script run with GUI if possible. Add --gui to force GUI. 2026-06-02 23:11:08 +00:00
LeoVasanko 76cd0224de Attempt build fix to allow running from git. 2026-06-02 22:45:09 +00:00
LeoVasanko b0d13a67a0 Fix config saving by not trying to save None default values. 2026-06-02 22:35:44 +00:00
42 changed files with 4332 additions and 2025 deletions
+20
View File
@@ -0,0 +1,20 @@
# MediaHive agent notes
## Python 3.14 syntax: unparenthesized `except` is valid
This project targets Python **>= 3.14** (see `requires-python` in
`pyproject.toml`). Per [PEP 758](https://peps.python.org/pep-0758/) (Final,
Python 3.14), multiple exception types may be caught **without parentheses**:
```python
except OSError, ValueError: # valid Python 3.14+, equivalent to except (OSError, ValueError):
```
Parentheses are still required when an `as` clause is used:
`except (OSError, ValueError) as e:`.
Do not "fix" these into tuple form, and do not flag them as Python 2 remnant
syntax errors — that rule is obsolete training data. Any syntax validation,
compilation check, or linting of this codebase must run under Python 3.14+
(e.g. `python -m py_compile` with a 3.14 interpreter); older interpreters
will report false SyntaxErrors on this and other 3.14-only constructs.
+9 -4
View File
@@ -6,15 +6,20 @@ Netflix style browsing of your local media archive. Supports keyboard, mouse and
**[Windows and Mac portable ZIP downloads](https://git.zi.fi/LeoVasanko/mediahive/releases)**
## Getting Started
- Windows and macOS: Download the portable ZIP from the releases page, extract it anywhere, and run `MediaHive`.
- Linux and other platforms: Install [UV](https://docs.astral.sh/uv/getting-started/installation/) and run directly with `uvx --from mediahive[gui] mediahive`.
## What It Does
- Scans your chosen media folder for all movies and series that can be found
- Produces preview video clips and downloads metadata
- Search on cast and character names, not just titles
- Search on names and other metadata, not just titles
- Remembers per-episode playback positions and offers series continue points
- Hand off playback to your preferred system player
- Implement gamepad controls for MPC-BE on Windows (where needed)
Extract the ZIP in some place and run MediaHive.exe to start the app. Currently we have no installer, but you can pin to start/taskbar for easier access. On the first startup the app asks for your media folder, that can later be changed by clicking in-app folder icon.
Windows and macOS builds are currently portable-only (no installer). On first startup, the app asks for your media folder, which you can later change using the in-app folder icon.
Note that `.mediahive` folder is created in your media folder to hold all the metadata and preview clips, avoiding the lengthy processing that you will see on initial startup.
@@ -25,7 +30,7 @@ MediaHive is designed to work with a mouse, keyboard, or gamepad.
| Input | Controls |
| --- | --- |
| Mouse | Click posters, rows, search, play, and folder actions directly. |
| Keyboard | Arrow keys move focus, `Enter` activates the focused item, `Escape` goes back, and `/` jumps to search. |
| Keyboard | Arrow keys move focus, `Enter` activates the focused item, `Escape` goes back, and `/` or `Ctrl`/`Cmd`+`F` jumps to search. |
| Gamepad | D-pad or left stick moves focus, `A` selects or plays, and `B` goes back. `RB`/`LB` browses adjacent items, and the Search bar has an OSD keyboard. Player controls during playback. |
## Recommended Players
+27 -11
View File
@@ -10,23 +10,39 @@ All media paths are scoped to a **root**, identified by a friendly `root_id`
| --- | --- | --- |
| `GET` | `/api/health` | Lightweight health check. |
| `GET` | `/api/config` | Returns the current root configuration. |
| `PUT` | `/api/config/roots` | Atomically replace the full root set. |
| `POST` | `/api/play/{root_id}` | Opens a media file with the system player. |
| `POST` | `/api/open-folder/{root_id}` | Opens a folder in the system file explorer. |
| `GET` | `/api/meta/{root_id}/{meta_key}` | Returns allowed metadata from `<root>/.mediahive`. |
| `GET` | `/api/meta/playback-state` | Returns merged resume positions across all roots. |
| `GET` | `/api/player/status` | Returns whether remote player control is currently available. |
| `GET` | `/api/mpcbe/status` | Reports whether MPC-BE's local web interface is reachable. |
| `PUT` | `/api/config/roots` | Atomically replace the full root set. Returns `{ "status": "ok", "accepted": [{path, root_id}], "failed": [...] }`. |
| `POST` | `/api/play/{root_id}` | Opens a media file with a media player. Also starts an assumed-playback session (see notes). |
| `GET` | `/api/players` | Lists detected media players. Returns `{ "players": [{id, name, family, path}] }`, including synthetic `default` and `custom` entries. |
| `POST` | `/api/activity` | Reports user input activity; finalizes any assumed-playback session. Returns `{ "status": "ok", "finalized": bool }`. |
| `POST` | `/api/open-folder/{root_id}` | Opens a folder in the system file explorer; given a file path, selects the file instead. |
| `GET` | `/api/meta/{root_id}/{meta_key}` | Returns allowed metadata from `<root>/.mediahive`, as `{ "key": meta_key, "data": ... }`. |
| `GET` | `/api/meta/playback-state` | Returns merged resume positions across all roots, as `{ "key": "playback-state", "data": ... }`. Series entries carry one continue point per series (`season`/`episode` = last watched) plus a per-episode watch map (`episodes`: `"S<season>E<episode>"``{pos, ts, done}`); completing an episode marks it done and advances the point to the next episode. |
| `POST` | `/api/meta/playback-state` | Updates one resume entry (`root_id`, `file_path`, `pos`; null `pos` clears a movie or advances a series' continue point). Returns `{ "status": "ok", "slug", "pos" }` plus `season`/`episode` when the continue point advances. |
| `GET` | `/api/player/status` | Returns whether remote player control is currently available. Accepts an optional `?port=` override (default 13579). |
| `GET` | `/api/mpcbe/status` | Reports whether MPC-BE's local web interface is reachable, as `{ "reachable": true|false }`. Accepts an optional `?port=` override; always `false` on non-Windows. |
| `GET` | `/api/media/{root_id}/{file_path:path}` | Serves files from the specified root. |
| `GET` | `/api/assets/{root_id}/{asset_type}/{asset_path:path}` | Serves typed assets from `<root>/.mediahive`. |
| `WS` | `/api/ws` | Streams roots, index updates, and task progress for all roots. |
| `WS` | `/api/ws` | Streams roots, index updates, and task progress for all roots (see WebSocket notes). |
## Notes
- `PUT /api/config/roots` accepts `{ "roots": { "name": "/absolute/path", ... } }`, validates paths, and atomically swaps the active set.
- `POST /api/play/{root_id}` and `POST /api/open-folder/{root_id}` expect JSON request bodies with `file_path` / `folder_path` relative to the root.
- `GET /api/media/{root_id}/{file_path:path}` is constrained to the specified root; path traversal outside the root is rejected.
- `POST /api/play/{root_id}` and `POST /api/open-folder/{root_id}` expect JSON request bodies with `file_path` / `folder_path` relative to the root. The play body additionally accepts `player_id` (a value from `GET /api/players`; unknown ids yield 400) and `player_custom_cmd` (command template used when `player_id` is `custom`).
- `GET /api/media/{root_id}/{file_path:path}` is constrained to the specified root; path traversal outside the root is rejected. Single-range requests are supported (`206` with `Content-Range`, `416` on invalid ranges), responses carry a weak `ETag` (`If-None-Match` yields `304`) and `Cache-Control: public, max-age=604800, immutable`.
- `GET /api/assets/{root_id}/{asset_type}/{asset_path:path}` is constrained to `<root>/.mediahive/{asset_type}` where `asset_type` is one of `movies`, `series`, `people`.
- `GET /api/meta/{root_id}/{meta_key}` supports metadata keys currently limited to `playback-state` and `scanignore`.
- `GET /api/player/status` returns `{ "remote": true|false }`.
- `GET /api/mpcbe/status` returns `false` on non-Windows platforms.
- Roots may also be provided at startup via the `MEDIAHIVE_ROOTS` environment variable (JSON dict of name → path), which overrides the persisted configuration.
- Assumed playback: after `POST /api/play/{root_id}` the launched item is assumed to be playing while the frontend reports no input activity. On the next `POST /api/activity` the guessed position (`resume base + elapsed`, capped at the TMDb runtime) is written once; watches under 5 minutes are discarded (a peek is not progress). A resume entry written by another tracker (e.g. the GUI's MPC-BE tracker) during the session overrides the guess. The MPC-BE tracker likewise ignores sessions shorter than 5 minutes.
## WebSocket
`GET /api/ws` sends tagged msgspec JSON messages as binary frames (message shapes are defined in `mediahive/models/protocol.py`):
- `roots` — full root list and per-root status: `{roots: [{root_id, path, status, error, snapshot_loaded, movies, series}]}`.
- `init` — full index payload `{roots: {root_id: {movies, series, people}}}`, re-sent when the root set changes or a snapshot finishes loading.
- `upsert` — single item inserted or updated: `{root_id, kind ("movie"|"series"), id, item, people?}`.
- `remove` — single item removed: `{root_id, kind, id}`.
- `task` — background task progress: `{root_id, data}`.
Clients must send (any) text frame to keep the receive loop alive.
+8 -6
View File
@@ -1,6 +1,6 @@
# Development
This document covers the developer-facing ways to run MediaHive locally. The main [README.md](../README.md) is aimed at Windows end users.
This document covers the developer-facing ways to run MediaHive locally. The main [README.md](../README.md) covers end-user startup across platforms (portable ZIPs on Windows/macOS, `uvx --from mediahive[gui] mediahive` on Linux/other).
## Requirements
@@ -40,13 +40,14 @@ uv run --extra gui python -m mediahive.winmain /path/to/media/folder
This launches the same pywebview-based desktop flow used by the Windows build.
## Migrate Existing Index Snapshots
## Building And Releasing
```bash
uv run python scripts/indexmigr.py /path/to/media/root --write
```
The helper scripts are directly executable via their `uv run` shebang (on Windows, run them with `uv run scripts/<name>.py`):
This applies versioned snapshot migrations to `.mediahive/index.json` outside the main application. Use it before starting a newer build against an older index.
- `./scripts/guibuild.py` builds the PyInstaller desktop app and a versioned portable ZIP under `build/`.
- `./scripts/release.py` publishes a release to the Gitea releases page.
Python packaging builds the frontend automatically through the hatch build hook `scripts/fastapi-vue/buildhook.py` (see `pyproject.toml`), so wheels and sdists always ship a fresh `mediahive/frontend-build`.
## Notes
@@ -54,3 +55,4 @@ This applies versioned snapshot migrations to `.mediahive/index.json` outside th
- The desktop app remembers the chosen folder between launches.
- HTTP and WebSocket endpoints are documented in [API.md](API.md).
- MPC-BE integration details (Windows only) live in [mpc-be.md](mpc-be.md).
- Scanner/indexer design notes and the v0.5.0 rescan fixes are reviewed in [scanning-review.md](scanning-review.md).
+3 -1
View File
@@ -611,7 +611,9 @@ Current native command usage is centered on:
- `889` for play/pause
- `816` for exit
- `-1&position=HH:MM:SS` for exact 4-second seeking
- `-1&position=HH:MM:SS` to seek to the stored resume position when playback starts
The GUI also polls `/variables.html` for the live position and duration and posts resume positions back to the MediaHive backend (`/api/meta/playback-state`), which is how per-episode resume positions and series continue points are tracked. Sessions shorter than 5 minutes are ignored.
## Guidance
+352
View File
@@ -0,0 +1,352 @@
# Scanner review — findings, fixes, and measured results
Date: 2026-09-03 (review); fixes implemented same day.
Scope: `mediahive/hivescan/*`, `mediahive/index_store.py`, `mediahive/root_registry.py`,
`mediahive/models/events.py`
Method: code review plus instrumented runs of the real `RootScanner` (monkeypatched
timers around every ffmpeg invocation, HTTP request, and filesystem primitive).
Structure of this document:
- **Section 1** records the findings as measured against the *pre-fix* code
(line references are from that revision).
- **Section 2** describes the fixes that were implemented for each finding.
- **Section 3** gives before/after measurements.
- **Appendix A** (blob storage options) is kept for reference only; it was
explicitly decided **not** to change the on-disk storage format for now.
Test environment (details omitted intentionally): the media library lives on a
network-mounted filesystem (SMB/CIFS). Two roots were measured:
- **Subset root**: 163 torrents (121 movies, 40 series entries), stale/empty index.
- **Full library root**: existing index with 1127 movies, 159 series,
1608 episodes, ~2983 video files, ~35k people records.
Environment characteristic that dominates several measurements:
`stat`/`scandir`/`exists` on the network mount are ~0.1 ms (attribute caching
works), but **every small file write costs ~0.2 s** (synchronous write-through).
The scanner writes thousands of small files into `.mediahive/` on that mount.
---
## 1. Findings (pre-fix)
### F1 — Partial rescan replaces whole entries (the "disappearing seasons" bug)
**This is a correctness bug, not a performance issue.**
Chain of events:
1. Discovery mtime-gates per torrent directory/file. Touching one season
directory of a series yields a `downloads` list containing *only that
season*.
2. `_process_series` rebuilds the `Series` object from only the items it was
given.
3. `IndexStore.upsert_series` **replaced the entire entry** and broadcast the
partial series to all connected clients.
Measured end-to-end on a 5-season series (5 separate season torrents,
110 episodes):
```
full scan -> index entry seasons [1,2,3,4,5] (110 episodes)
touch season 4 -> rescan 0.2 s, emits ONE upsert: seasons [4] (22 episodes)
-> index entry is now seasons [4] — seasons 1-3,5 gone
```
The missing seasons returned only when a scan happened to include all seasons
again — in practice the next **process restart**, because the seen-mtimes map
(F2) was memory-only and forced a full rediscovery at startup.
Same bug class, other variants:
- **Movies**: a touched version directory dropped the other versions of the
same movie from the listing.
- **TMDb dedupe collapse**: when a partial entry arrived under a *different*
item id for an already-known TMDb id, the old **complete** entry was
explicitly deleted.
- **Deletions were never detected**: discovery only ever added to
`_seen_mtimes`; nothing emitted removals. A torrent deleted from disk stayed
in the index forever.
### F2 — No persistent scan state: every restart was a full reprocess
All scanner state was process memory: `_seen_mtimes`, the ffmpeg probe cache,
and the episode/playable-file/bluray-probe caches.
Consequences measured:
- **Steady-state rescan within one process: 0.2 s** (subset root, nothing
changed) — mtime gating worked fine while the process lived.
- **Warm-restart scan (fresh process caches, all disk caches warm): 56.5 s**
for the same 163 items, of which **52.9 s (94 %) was re-running ffmpeg probes
on all 469 video files** (21 s `ffmpeg -i` + 32 s `showinfo` passes on HDR
files). TMDb was 100 % disk-cache hits and cost 0.8 s total.
- Scaled to the full library: every application restart re-probed **~2983
files ≈ 68 minutes** of sequential ffmpeg, during which the whole index was
re-derived and re-upserted item by item (see F4).
### F3 — Preview (showreel) generation: restart storms and infinite retries
- **Every scan that included an item enqueued all of its reel tasks**, whether
or not the reels existed. Existence was only checked later by the serial
worker. A full scan of the subset root enqueued **467** tasks; the full
library would enqueue ~2700.
- **Nothing was persisted about the queue.** A restart before the queue drained
started everything over.
- **Failures were never recorded.** In the drain test, 3 of 18 movies failed
deterministically (DoVi profile 7 titles require `libplacebo` tonemapping,
which fails on GPU-less machines; one file has a matroska demux error). The
same files were retried on every subsequent drain — ~1.5 s of probing plus
crop detection plus an error task broadcast to every client, **forever**.
- The reel-existence check required **all five** reels; short videos
legitimately produce fewer, so they were treated as "missing" and re-queued
on every scan, generating nothing new each time.
- Measured generation pace with software AV1 encoding: ~1628 s per movie
(5 clips), ~45 s per episode clip.
- The reel worker rebroadcast **whole items** built from stale scan data,
clobbering newer store state.
### F4 — Degraded operation while a full scan is in progress
- Items were re-upserted one by one as processed; combined with F1, any
partial rescan interleaved with normal use made listings lose data until the
next restart.
- The showreel worker broadcast an **error task for every permanent failure on
every scan** (F3), producing user-visible noise.
- The index snapshot was rewritten every 5 s while dirty; with 1286+ items
that is a ~7.5 MB serialize + write per flush, continuously, for the
duration of a scan.
- `upsert_*` rebuilt the TMDb-id lookup maps on **every** upsert — O(n²) per
scan. Measured negligible; fixed anyway as part of the merge work.
### F5 — Cold-scan cost was serialized small-file I/O, not TMDb
Cold scan of the subset root: **975 s for 163 items**.
| Time | Share | Where |
|---|---|---|
| 763 s | 78 % | `download_cast_profile`: 3494 cast images, strictly serialized; ≈0.22 s each ≈ 0.19 s network-mount write + 0.03 s HTTP |
| ~100 s | 10 % | ffmpeg probes (469 files, incl. 83 HDR `showinfo` passes) |
| 73 s | 7 % | TMDb API layer: 217 uncached requests (27 s HTTP) **plus ~41 s writing per-request cache JSON files** to the network mount |
| ~75 s | 8 % | covers / backdrops / season posters (same small-write cost) |
| 0.3 s | — | filesystem discovery walk |
Notes:
- The TMDb disk cache itself is fine: once warm it serves 282 requests in
0.8 s. Cache *reads* need no optimization.
- The full cast of every title was downloaded sequentially, one tiny file per
person (the full library has ~35k people records). Re-runs are cheap
(exists-check), so this was a cold-scan-only cost — but it made the first
scan of a new root take ~8× longer than everything else combined.
### F6 — Measured as noise (not worth effort)
- The 30-second rescan loop's tree walk: 3.56 s per pass over the full
library (~1900 directories). Continuous but light.
- `get_directory_size` per torrent: 0.1 s total in the scan.
- TMDb disk-cache reads: sub-second per scan.
- `trigger_scan` was dead code — nothing called it.
- `_seen_mtimes` was updated *before* processing; a cancelled/failed scan
permanently lost that update until the next restart.
- The in-process probe cache was keyed by path only; a replaced file kept
stale probe data until restart.
---
## 2. Implemented fixes
All proposals P1P5 from the review were implemented, keeping the existing
on-disk format unchanged (no blob storage — see Appendix A).
### F1 → merge-semantics upserts + deletion sync
- The `Upsert` event now carries `scanned: list[str]` — the media-root-relative
torrent paths whose content was (re)scanned to build the item
(`mediahive/models/events.py`; `_process_movies`/`_process_series` yield it).
- `IndexStore.upsert_movie/upsert_series` (`mediahive/index_store.py`) merge a
partial rebuild into the existing entry instead of replacing it:
file entries belonging to scanned torrents are replaced, everything else is
preserved, episodes/seasons emptied by the merge are dropped, same-episode
multi-release files are unioned, and non-None scalar fields from the fresh
scan win. The store broadcasts only when the merged result actually changed.
- The TMDb dedupe collapse now folds duplicates through the same merge (the
complete entry's scalars win), so a partial candidate can no longer delete a
complete entry; the reverse TMDb-id map is fixed up incrementally instead of
rebuilding both maps per upsert (also F4/P5).
- **Deletion sync**: discovery collects the full set of candidate torrent
paths; after a fully completed scan the scanner emits a new `Sync` event and
`IndexStore.sync_torrent_paths` drops file entries whose torrent path is
gone, cascading to empty episodes/seasons/items with proper removals.
### F2 → persisted scan state and probe cache
Three small JSON files under `.mediahive/` per root, loaded at scanner start
and written atomically (tmp + rename) **only when changed**:
- `scan-state.json`: relpath → mtime. A restart over an unchanged library now
discovers "0 new items" and finishes in walk time. Mtimes are committed
**after** the scan completes successfully (fixes the pre-commit nit from F6):
a cancelled/failed scan retries its items.
- `probe-cache.json`: path → {mtime, size, probe fields} for every probed
file, **failures included**. Keying by mtime+size makes it self-invalidating
when a file is replaced (also fixes the stale-probe nit from F6). Non-plain
paths (bluray:/concat: URIs) fail `stat` and stay memory-cached only.
- `reel-state.json`: see F3 below.
### F3 → reel-state persistence, backoff, and queue gating
- `reel-state.json` records, per media folder (movies) or per episode
(`folder#SxxEyy`), the video's mtime+size, status (`done`/`failed`),
attempt count, and last-attempt timestamp.
- `_reel_needed` gates both scan-time queueing and the worker:
`done` entries are skipped; `failed` entries back off exponentially
(6 h → 12 h → … capped at 1 week); entries with no record fall back to a
cheap on-disk existence check, and existing reels are silently recorded as
`done` so future scans take the cheap path. This ends both the infinite
retries of unreadable files and the re-queueing of short videos.
- The worker no longer rebroadcasts whole (stale) items: it sends narrow
`MovieShowreel` / `EpisodeReel` events, and the store updates only the reel
fields of the current entry (`set_movie_showreel` / `set_episode_reel`).
- Reel state is persisted at scan finalize, on scanner `stop()`, and as soon
as the reel queue drains (a crash between scans no longer loses records).
### F5 → bounded parallelism for downloads and TMDb fetches
- Cast-profile downloads: `asyncio.gather` with a semaphore of 8.
- TMDb title lookups (movies and series) are prefetched in parallel
(semaphore of 4) before the grouping loops, which then read the per-call
caches.
- Season-detail fetches are prefetched in parallel (semaphore of 4) before the
season loop.
- Per-item cover/backdrop/poster logic is unchanged (exists-check-fast when
warm).
### Housekeeping (P5)
- Dead `trigger_scan` removed (`is_scanning` kept).
- TMDb-id index bookkeeping is incremental (see F1 above).
- `_rebuild_tmdb_indexes` remains only for snapshot load and dedupe.
---
## 3. Measured results
Subset root (163 items, 121 movies / 40 series entries, 469 video files),
same network mount:
| Scenario | Before | After |
|---|---|---|
| Touch one season of a 5-season series | other 4 seasons vanish until next full scan | 0.3 s rescan, one partial upsert (`scanned=[S04]`), store keeps all 5 seasons |
| Steady rescan, nothing changed (same process) | 0.2 s | 0.2 s, 0 upserts, no writes |
| Warm restart, unchanged library (fresh process) | 56.5 s (94 % ffmpeg re-probes) + 467 reel tasks queued | **0.2 s, 0 upserts, reel queue 0** |
| First scan with warm TMDb cache but no probe cache | 56.5 s | 51.6 s once — writes `probe-cache.json` (469 records, 207 KB); subsequent runs skip all probing |
| Permanently unreadable files (3 DoVi/libplacebo movies) | retried on every scan and every startup, error broadcast each time | recorded as failed once, skipped within backoff |
| Cold scan, nothing cached | 975 s | not re-measured end-to-end; the dominant terms are now 8-way parallel (cast images: 3494 downloads measured at 3.7 s when warm) |
| Deleted torrent | stayed in index forever | removed on the next completed scan via `Sync` |
Unit-level checks (synthetic `IndexStore` + scanner state, no filesystem
library involved): partial upsert preserves untouched seasons and replaces
rescanned torrent files; multi-release episode union; sync removal cascades;
single-episode reel updates; probe-cache save/load roundtrip; reel backoff
math; reel-state persistence roundtrip across scanner instances.
---
## Appendix A — Blob storage options: scan-time vs runtime concerns
**Status: reference only — not implemented.** The current on-disk format
(one file per artifact) was deliberately kept. This appendix stays as
documentation of the options if write amplification or file counts ever
become an operational issue.
The two concerns have opposite constraints, so they should be decided
separately:
- **Scan-time data** is written and read only by the scanner. Nothing in the
server serves it. Storage format is therefore a pure implementation detail
and can be changed freely.
- **Runtime data** is delivered to the frontend as plain files via
`/api/assets/{root}/{movies|series|people}/{path}` (covers, posters,
backdrops, person photos) and `/api/media/{root}/{path}` (media files), with
etag/range streaming. Anything that replaces files here must keep an HTTP
serving story working.
### A.1 Artifact inventory (measured)
| Artifact | Class | Avg size | Count (full library) | Total | Written |
|---|---|---|---|---|---|
| TMDb response cache JSON | scan-time | 25 KB (median 9 KB) | ~8,500 | ~215 MB | on every uncached API request |
| Probe results / scan state / reel state | scan-time | ~0.5 KB/record | ~3,000 records | ~1.5 MB | per processed file |
| Person photos | runtime | 11.4 KB | ~22,500 | ~262 MB | once per person |
| `cover.jpg` / season posters | runtime | ~77 KB | ~2,100 | ~164 MB | once per title/season |
| `backdrop.jpg` | runtime | ~147 KB | ~1,460 | ~220 MB | once per title |
| Reel clips (WebM/AV1) | runtime | ~450 KB | ~8,700 | ~4.0 GB | once per title/episode |
| `index.json` snapshot | both | 7.5 MB | 1 | 7.5 MB | debounced, only when dirty |
Reference point for the write-amplification math: on the network mount one
small-file write costs ~0.2 s, while a single 7.5 MB sequential write costs
the same ~0.2 s (~40 MB/s). So ~31,000 tiny files ≈ 1.7 hours of serialized
write time, versus ~6 s for the same bytes as one bulk dump.
### A.2 Scan-time blob store (TMDb cache, probe cache, scan state)
Nothing here is served, so the only requirement is fast lookup + cheap
persistence. Two workable shapes:
- **RAM map + debounced atomic dump.** Plain dicts keyed by request hash /
file path, dumped as one binary file (length-prefixed msgspec or JSON blob,
optionally zstd-compressed) with tmp-write + rename, on the same
dirty-flag + debounce discipline `index.json` already uses. Effects: the
~8,500 individual cache writes collapse into a handful of bulk flushes;
warm lookups become dict hits with zero filesystem calls. TMDb JSON
compresses ~10× (215 MB → ~2025 MB), so a full dump is a sub-second write.
Caveat: holding all responses parsed in RAM costs ~200 MB for the full
library; storing raw response *bytes* and parsing lazily, or capping to
entries referenced by known index items, keeps this modest.
- **SQLite (stdlib, WAL mode).** One database file, incremental commits, crash
safety without full dumps, and kernel page cache instead of explicit RAM
management. Better fit if the cache is allowed to grow unbounded, at the
price of slightly more code.
Either way, keep the existing cache *semantics* unchanged: cache HTTP-level
failures, never cache network errors. With persisted scan state in place the
TMDb cache becomes write-rarely (new items only), which further lowers the
value of elaborate engineering here — the simple dump is likely enough.
### A.3 Runtime-served artifacts
- **Reels, covers, backdrops, season posters: keep as files.** They are few
per title, tens-to-hundreds of KB, written exactly once, and benefit from
the existing etag/range file serving. No write-amplification problem.
- **Person photos** (~22.5k files × 11.4 KB) are the one runtime class where
tiny files hurt at scan time. Three options, in increasing invasiveness:
1. **Keep files, fix only the scan-time behavior** — bounded-parallel
downloads, optionally capped to top-N billed cast. Zero changes to
serving; the cold-scan cost drops ~8× but the file count stays.
*(This is the option currently implemented.)*
2. **Blob db + serve from the db.** Person photos move into the same store
as above; the assets handler gains one branch for the `people` asset
type that streams bytes from the db instead of the filesystem.
Eliminates all 22.5k tiny files. If full RAM residency (262 MB) is
undesirable, use SQLite and let the page cache handle it.
3. **Hybrid lazy materialization.** The db is authoritative at scan time
(no tiny writes during scans); the assets handler writes the photo to the
conventional path on first request and serves it as a file thereafter.
Serving logic and URLs stay unchanged; disk usage appears only for
people actually viewed.
Note that `index.json` itself is already the right shape: one atomic 7.5 MB
file, rewritten only when dirty. The goal for everything else is simply to
reach the same shape per concern.
### A.4 Recommendation (if revisited)
| Concern | Recommended treatment |
|---|---|
| TMDb response cache | RAM map + debounced single-file dump (SQLite if growth matters) |
| Probe cache, seen-mtimes, reel-failure records | same dump mechanism, separate small files *(currently: three small JSON files, written only when dirty)* |
| Person photos | option 1 now (parallel + lazy fetching); option 2 or 3 if file count becomes an operational issue |
| Covers, posters, backdrops, reels | unchanged — plain files |
| `index.json` | unchanged |
+74 -8
View File
@@ -177,6 +177,8 @@
:all-movies="mediaIndex?.movies ?? []"
:focus-episode="focusEpisode"
:has-resume-position="hasResumePosition"
:get-resume-point="getResumePoint"
:get-resume-episodes="getResumeEpisodes"
:get-root-name="getRootName"
@close="closeDetail"
@play="handlePlay"
@@ -201,6 +203,7 @@ import type {
MediaItem,
EpisodeWithSeries,
TaskInfo,
SeriesResumePoint,
} from "./types"
import {
playMedia,
@@ -208,9 +211,12 @@ import {
isMpcBeReachable,
fetchResumePositions,
getPlayerStatus,
type ResumePositionEntry,
type EpisodeWatchEntry,
} from "./api"
import { useSettings } from "./composables/useSettings"
import { useKeyboardNavigation, setActiveNavigationScope } from "./composables/useKeyboardNavigation"
import type { SyncedRowScrollSnapshot } from "./composables/useKeyboardNavigation"
import { useMediaWebSocket } from "./composables/useMediaWebSocket"
import Header from "./components/Header.vue"
import CollageHero from "./components/CollageHero.vue"
@@ -219,7 +225,13 @@ import MediaDetail from "./components/MediaDetail.vue"
import type { SearchResultItem, SearchResponseMessage } from "./search-worker"
// Initialize keyboard navigation
const { getFocusState, restoreFocusState, focusElement } = useKeyboardNavigation()
const {
getFocusState,
restoreFocusState,
focusElement,
snapshotSyncedRowScroll,
restoreSyncedRowScroll,
} = useKeyboardNavigation()
const router = useRouter()
const route = useRoute()
@@ -490,7 +502,7 @@ const settings = useSettings()
const searchResults = ref<MediaItem[]>([])
const isSearching = ref(false)
const mpcBeConnected = ref(false)
const resumePositions = ref<Record<string, number>>({})
const resumePositions = ref<Record<string, ResumePositionEntry>>({})
const searchQuery = ref(getRouteSearchQuery())
const searchReturnPath = ref<string | null>(null)
const browsePanelRef = ref<HTMLElement | null>(null)
@@ -533,6 +545,10 @@ async function refreshResumePositions() {
resumePositions.value = await fetchResumePositions()
}
function refreshResumePositionsAsEvent() {
void refreshResumePositions()
}
async function refreshPlayerStatus() {
if (!isMpcFamilySelected()) {
mpcBeConnected.value = false
@@ -548,7 +564,25 @@ async function refreshPlayerStatus() {
function hasResumePosition(mediaId: string | null) {
if (!mediaId) return false
return Number(resumePositions.value[mediaId] || 0) > 0
return (resumePositions.value[mediaId]?.pos || 0) > 0
}
function getResumePoint(mediaId: string | null): SeriesResumePoint | null {
if (!mediaId) return null
const entry = resumePositions.value[mediaId]
if (!entry || entry.season === null || entry.episode === null) return null
return {
seasonNumber: entry.season,
episodeNumber: entry.episode,
positionSeconds: entry.pos,
}
}
function getResumeEpisodes(
mediaId: string | null,
): Record<string, EpisodeWatchEntry> | null {
if (!mediaId) return null
return resumePositions.value[mediaId]?.episodes ?? null
}
function startMpcBePolling() {
@@ -674,6 +708,23 @@ const searchCategories = ref<{ name: string; items: MediaItem[] }[]>([])
const focusStateMap = new Map<string, { row: number; col: number }>()
// Track the last viewed item ID to restore focus to the right card
const lastViewedItemId = ref<string | null>(null)
let browseScrollSnapshot: { panelTop: number; rows: SyncedRowScrollSnapshot } | null = null
function captureBrowseScrollSnapshot() {
browseScrollSnapshot = {
panelTop: browsePanelRef.value?.scrollTop ?? 0,
rows: snapshotSyncedRowScroll(),
}
}
function restoreBrowseScrollSnapshot() {
if (!browseScrollSnapshot) return
if (browsePanelRef.value) {
browsePanelRef.value.scrollTop = browseScrollSnapshot.panelTop
}
restoreSyncedRowScroll(browseScrollSnapshot.rows)
browseScrollSnapshot = null
}
// Save current focus state for a page
function saveFocusForPage(page: string) {
@@ -693,22 +744,24 @@ function restoreFocusForPage(page: string) {
if (lastViewedItemId.value) {
// Use nextTick + timeout to ensure DOM is updated after navigation
setTimeout(() => {
restoreBrowseScrollSnapshot()
const itemId = lastViewedItemId.value
// Find the element with matching item id
const element = document.querySelector(`[data-item-id="${itemId}"]`) as HTMLElement | null
if (element) {
focusElement(element)
focusElement(element, { preserveScroll: true })
lastViewedItemId.value = null
return
}
// Fallback to saved focus state
const state = focusStateMap.get(page)
restoreFocusState(state || null)
restoreFocusState(state || null, { preserveScroll: true })
lastViewedItemId.value = null
}, 100)
} else {
restoreBrowseScrollSnapshot()
const state = focusStateMap.get(page)
restoreFocusState(state || null)
restoreFocusState(state || null, { preserveScroll: true })
}
}
@@ -910,6 +963,7 @@ onMounted(() => {
document.addEventListener("keydown", handleDetailAdjacentKey)
window.addEventListener("mediahive:gamepad-action", onGamepadAction as EventListener)
window.addEventListener("mediahive:gamepad-button", onRawGamepadButton as EventListener)
window.addEventListener("mediahive:resume-updated", refreshResumePositionsAsEvent)
})
onUnmounted(() => {
@@ -917,6 +971,7 @@ onUnmounted(() => {
document.removeEventListener("keydown", handleDetailAdjacentKey)
window.removeEventListener("mediahive:gamepad-action", onGamepadAction as EventListener)
window.removeEventListener("mediahive:gamepad-button", onRawGamepadButton as EventListener)
window.removeEventListener("mediahive:resume-updated", refreshResumePositionsAsEvent)
stopMpcBePolling()
})
@@ -1025,6 +1080,12 @@ function showDetail(item: MediaItem) {
const currentPage = route.path === "/series" ? "series" : "movies"
saveFocusForPage(currentPage)
// Capture the exact browse scroll positions to restore on return,
// but only when leaving the browse page (not for detail-to-detail hops)
if (!isDetailOpen.value) {
captureBrowseScrollSnapshot()
}
// Check if there are matched episodes to focus on
if (item.type === "series" && item.searchMatchInfo?.matchedEpisodes?.length) {
const firstMatch = item.searchMatchInfo.matchedEpisodes[0]
@@ -1090,10 +1151,15 @@ function focusDetailEntryTarget(item: MediaItem): boolean {
'[data-nav-release-item="true"][data-nav-row="2"][data-nav-col="0"][data-nav-focusable="true"]',
) as HTMLElement | null
} else if (item.type === "series") {
// Initial episode tile (first season, first episode) maps to row 2 / col 0.
// Row 2 is the season selector strip; land on the selected season poster.
target = detailPanel.querySelector(
'.episode-tile[data-nav-row="2"][data-nav-col="0"][data-nav-focusable="true"]',
'.season-poster-card.season-poster-card--selected[data-nav-focusable="true"]',
) as HTMLElement | null
if (!target) {
target = detailPanel.querySelector(
'.episode-tile[data-nav-focusable="true"]',
) as HTMLElement | null
}
}
if (target) {
+63 -5
View File
@@ -148,10 +148,25 @@ function splitAssetTypePath(assetPath: string): { assetType: string; relativePat
return { assetType: assetType.toLowerCase(), relativePath: rest.join("/") }
}
/** Watch progress for one episode of a series. */
export interface EpisodeWatchEntry {
pos: number
done: boolean
}
/** One stored continue point. season/episode are set for series, null for movies. */
export interface ResumePositionEntry {
pos: number
season: number | null
episode: number | null
/** Per-episode watch progress for series, keyed "S<season>E<episode>". */
episodes?: Record<string, EpisodeWatchEntry>
}
/**
* Fetch merged resume positions from all roots.
*/
export async function fetchResumePositions(): Promise<Record<string, number>> {
export async function fetchResumePositions(): Promise<Record<string, ResumePositionEntry>> {
try {
const response = await fetch("/api/meta/playback-state")
if (!response.ok) return {}
@@ -160,12 +175,32 @@ export async function fetchResumePositions(): Promise<Record<string, number>> {
if (!positions || typeof positions !== "object") {
return {}
}
const normalized: Record<string, number> = {}
const normalized: Record<string, ResumePositionEntry> = {}
for (const [slug, value] of Object.entries(positions as Record<string, unknown>)) {
if (!value || typeof value !== "object") continue
const pos = (value as { pos?: unknown }).pos
if (typeof pos === "number" && Number.isFinite(pos) && pos > 0) {
normalized[slug] = pos
const entry = value as { pos?: unknown; season?: unknown; episode?: unknown }
if (typeof entry.pos !== "number" || !Number.isFinite(entry.pos) || entry.pos < 0) {
continue
}
normalized[slug] = {
pos: entry.pos,
season: typeof entry.season === "number" ? entry.season : null,
episode: typeof entry.episode === "number" ? entry.episode : null,
}
const rawEpisodes = (entry as { episodes?: unknown }).episodes
if (rawEpisodes && typeof rawEpisodes === "object") {
const watches: Record<string, EpisodeWatchEntry> = {}
for (const [key, watch] of Object.entries(
rawEpisodes as Record<string, unknown>,
)) {
if (!watch || typeof watch !== "object") continue
const w = watch as { pos?: unknown; done?: unknown }
if (typeof w.pos !== "number" || !Number.isFinite(w.pos)) continue
watches[key] = { pos: w.pos, done: w.done === true }
}
if (Object.keys(watches).length > 0) {
normalized[slug].episodes = watches
}
}
}
return normalized
@@ -174,6 +209,29 @@ export async function fetchResumePositions(): Promise<Record<string, number>> {
}
}
/**
* Report that the user is actively interacting with the UI.
*
* Ends any server-side assumed-playback session (launched item is assumed
* watched while the UI sees no input). Throttled; fire-and-forget.
*/
let lastActivityReportAt = 0
export function reportUserActivity(): void {
const now = Date.now()
if (now - lastActivityReportAt < 5000) return
lastActivityReportAt = now
void fetch("/api/activity", { method: "POST" })
.then(async (response) => {
if (!response.ok) return
const data = await response.json().catch(() => null)
if (data?.finalized) {
// An assumed-playback position was just written; let views refetch.
window.dispatchEvent(new Event("mediahive:resume-updated"))
}
})
.catch(() => {})
}
/**
* Replace the full root set atomically
*/
+1
View File
@@ -18,6 +18,7 @@
:src="posterImageUrl"
:alt="item.title || 'Unknown'"
loading="lazy"
decoding="async"
@error="imageError = true"
/>
<div v-else class="media-card-placeholder">
+101 -28
View File
@@ -5,7 +5,8 @@
:series="item.data as Series"
:all-movies="allMovies"
:focus-episode="focusEpisode"
:has-resume-position="hasResumePosition"
:resume-point="getResumePoint(item.id)"
:resume-episodes="getResumeEpisodes(item.id)"
:get-root-name="getRootName"
@close="$emit('close')"
@play="handlePlay"
@@ -17,7 +18,7 @@
<div v-else class="movie-page">
<div class="movie-page-content">
<!-- Diagonal collage header -->
<div class="collage-header">
<div ref="collageHeaderRef" class="collage-header">
<!-- Background collage of showreel videos -->
<div class="collage-grid">
<div
@@ -233,7 +234,8 @@
<script setup lang="ts">
import { computed, ref, watch, onMounted, onUnmounted, nextTick } from "vue"
import type { CastMember, MediaItem, Movie, MovieUi, Series, Torrent } from "../types"
import type { CastMember, MediaItem, Movie, MovieUi, Series, SeriesResumePoint, Torrent } from "../types"
import type { EpisodeWatchEntry } from "../api"
import {
getCoverUrl,
getVideoPreviewUrl,
@@ -253,12 +255,15 @@ import {
FOCUSABLE_ATTR,
setModalOpen,
} from "../composables/useKeyboardNavigation"
import { useIdlePreviewPlayback } from "../composables/useIdlePreviewPlayback"
const props = defineProps<{
item: MediaItem
allMovies: MovieUi[]
focusEpisode?: { seasonNumber: number; episodeNumber: number } | null
hasResumePosition: (mediaId: string | null) => boolean
getResumePoint: (mediaId: string | null) => SeriesResumePoint | null
getResumeEpisodes: (mediaId: string | null) => Record<string, EpisodeWatchEntry> | null
getRootName: (rootId: string | null | undefined) => string | null
}>()
const emit = defineEmits<{
@@ -278,6 +283,22 @@ const safariAutoplay = isSafariBrowser()
const COLLAGE_START_OFFSETS_SECONDS = [0, 8, 6, 4, 2]
const DESKTOP_NAV_SHORTCUT_MIN_WIDTH = 900
const collageHeaderRef = ref<HTMLElement | null>(null)
let collageHeaderVisible = true
let collageHeaderObserver: IntersectionObserver | null = null
const staggerTimers = new Set<ReturnType<typeof setTimeout>>()
const COLLAGE_STOP_STEP_MS = 500
let staggerToken = 0
const { stopped: previewPlaybackStopped } = useIdlePreviewPlayback({
onStop: stopPreviews,
onRestart: () => startStaggeredPlayback(),
})
function previewsSuppressed(): boolean {
return previewPlaybackStopped.value || !collageHeaderVisible || document.hidden
}
let disposeOutOfBoundsHandler: (() => void) | null = null
let lastReleaseShortcutRow: number | null = null
@@ -362,15 +383,35 @@ function isVideoReady(index: number): boolean {
return videoStates.value[index] === "ready"
}
function cancelStaggeredPlayback() {
staggerToken += 1
for (const timer of staggerTimers) {
clearTimeout(timer)
}
staggerTimers.clear()
}
function scheduleStaggeredStart(token: number, start: () => void, delayMs: number) {
const timer = setTimeout(() => {
staggerTimers.delete(timer)
if (token !== staggerToken || previewsSuppressed()) return
start()
}, delayMs)
staggerTimers.add(timer)
}
// Start staggered video playback
function startStaggeredPlayback() {
cancelStaggeredPlayback()
const token = staggerToken
const videos = videoRefs.value.filter((v) => v !== null) as HTMLVideoElement[]
if (videos.length === 0) return
if (videos.length === 0 || previewsSuppressed()) return
if (safariAutoplay) {
videos.forEach((video, index) => {
const offset = COLLAGE_START_OFFSETS_SECONDS[index] ?? 0
const startVideo = () => {
if (token !== staggerToken || previewsSuppressed()) return
video.currentTime = offset
video.play().catch(() => {})
}
@@ -390,11 +431,46 @@ function startStaggeredPlayback() {
// Set up staggered start for remaining videos
for (let i = 1; i < videos.length; i++) {
setTimeout(() => {
const video = videos[i]
if (!video) return
video.play().catch(() => {})
}, i * 2000)
scheduleStaggeredStart(
token,
() => {
const video = videos[i]
if (!video) return
video.play().catch(() => {})
},
i * 2000,
)
}
}
function scheduleStaggeredStop(token: number, stop: () => void, delayMs: number) {
const timer = setTimeout(() => {
staggerTimers.delete(timer)
if (token !== staggerToken) return
stop()
}, delayMs)
staggerTimers.add(timer)
}
// Stop all collage previews with a stagger, and cancel pending staggered starts
function stopPreviews() {
cancelStaggeredPlayback()
const token = staggerToken
clearHoverAudioIdleTimer()
hoveredVideoIndex = null
for (const interval of volumeFadeIntervals.values()) {
clearInterval(interval)
}
volumeFadeIntervals.clear()
let stopIndex = 0
for (const video of videoRefs.value) {
if (video && !video.paused && !video.ended) {
scheduleStaggeredStop(token, () => video.pause(), stopIndex * COLLAGE_STOP_STEP_MS)
stopIndex += 1
} else {
video?.pause()
}
}
}
@@ -497,6 +573,19 @@ onMounted(() => {
startStaggeredPlayback()
}, 100)
// Pause the collage videos while the header is scrolled out of view
if (collageHeaderRef.value) {
collageHeaderObserver = new IntersectionObserver((entries) => {
collageHeaderVisible = entries[0]?.isIntersecting ?? true
if (collageHeaderVisible) {
startStaggeredPlayback()
} else {
stopPreviews()
}
})
collageHeaderObserver.observe(collageHeaderRef.value)
}
registerMovieOutOfBoundsShortcut()
})
@@ -798,14 +887,6 @@ const ratingClass = computed(() => {
return "rating-low"
})
const seasons = computed(() => {
if (props.item.type !== "series") return []
const series = props.item.data as Series
return series.seasons || []
})
const selectedSeasonIndex = ref<number>(0)
const versionActionMenu = ref<{
visible: boolean
x: number
@@ -892,17 +973,6 @@ function handleMovieMenuKeydown(event: KeyboardEvent) {
}
}
// Select first season by default
watch(
seasons,
(s) => {
if (s.length > 0 && selectedSeasonIndex.value >= s.length) {
selectedSeasonIndex.value = 0
}
},
{ immediate: true },
)
function handlePlay(filePath: string | null) {
if (filePath) {
emit("play", filePath)
@@ -990,6 +1060,9 @@ onUnmounted(() => {
window.removeEventListener("mousemove", handleHoverAudioMouseMove)
window.removeEventListener("mediahive:gamepad-action", handleGamepadAction as EventListener)
clearHoverAudioIdleTimer()
cancelStaggeredPlayback()
collageHeaderObserver?.disconnect()
collageHeaderObserver = null
disposeOutOfBoundsHandler?.()
disposeOutOfBoundsHandler = null
lastReleaseShortcutRow = null
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,74 @@
import { onUnmounted, ref, type Ref } from "vue"
export const PREVIEW_IDLE_MS = 30_000
const ACTIVITY_EVENTS = ["mousemove", "mousedown", "wheel", "keydown", "touchstart"] as const
const GAMEPAD_ACTIVITY_EVENT = "mediahive:gamepad-action"
interface IdlePreviewPlaybackOptions {
idleMs?: number
onStop: () => void
onRestart: () => void
}
// Stops preview videos after a period without user input and restarts them
// (via the component's staggered startup) when activity resumes. Also stops
// previews while the tab is hidden. Activity listeners run in the capture
// phase so the stopped flag clears before hover/focus handlers react.
export function useIdlePreviewPlayback(options: IdlePreviewPlaybackOptions): {
stopped: Ref<boolean>
} {
const idleMs = options.idleMs ?? PREVIEW_IDLE_MS
const stopped = ref(false)
let idleTimer: ReturnType<typeof setTimeout> | null = null
function clearIdleTimer() {
if (idleTimer !== null) {
clearTimeout(idleTimer)
idleTimer = null
}
}
function stop() {
clearIdleTimer()
if (stopped.value) return
stopped.value = true
options.onStop()
}
function handleActivity() {
if (document.hidden) return
if (stopped.value) {
stopped.value = false
options.onRestart()
}
clearIdleTimer()
idleTimer = setTimeout(stop, idleMs)
}
function handleVisibilityChange() {
if (document.hidden) {
stop()
} else {
handleActivity()
}
}
for (const eventName of ACTIVITY_EVENTS) {
window.addEventListener(eventName, handleActivity, { passive: true, capture: true })
}
window.addEventListener(GAMEPAD_ACTIVITY_EVENT, handleActivity, { passive: true, capture: true })
document.addEventListener("visibilitychange", handleVisibilityChange)
idleTimer = setTimeout(stop, idleMs)
onUnmounted(() => {
for (const eventName of ACTIVITY_EVENTS) {
window.removeEventListener(eventName, handleActivity, { capture: true })
}
window.removeEventListener(GAMEPAD_ACTIVITY_EVENT, handleActivity, { capture: true })
document.removeEventListener("visibilitychange", handleVisibilityChange)
clearIdleTimer()
})
return { stopped }
}
@@ -1,5 +1,8 @@
import { reportUserActivity } from "../api"
type InputModality = "mouse" | "keyboard" | "gamepad"
const MOUSE_IDLE_MS = 1400
const MOUSE_INTENT_DISTANCE_PX = 28
const MOUSE_INTENT_WINDOW_MS = 700
@@ -90,6 +93,7 @@ function registerMouseIntentTravel(event: MouseEvent): boolean {
}
function handleMouseMove(event: MouseEvent) {
reportUserActivity()
showPointerFromMotion()
if (modality === "mouse") {
@@ -112,6 +116,7 @@ function handleMouseOver(event: MouseEvent) {
}
function handleMouseIntentAction(event: MouseEvent | WheelEvent) {
reportUserActivity()
pointerVisible = true
if (isMouseIntentTarget(event.target)) {
activateMouseInput()
@@ -124,6 +129,7 @@ function handleMouseIntentAction(event: MouseEvent | WheelEvent) {
function handleKeyboardActivity(event: KeyboardEvent) {
if (event.metaKey || event.ctrlKey || event.altKey) return
reportUserActivity()
activateNonMouseInput("keyboard")
}
@@ -702,7 +702,7 @@ function findNextElement(
}
}
function focusElement(element: HTMLElement | null) {
function focusElement(element: HTMLElement | null, options?: { preserveScroll?: boolean }) {
if (!element) return
if (!isElementInActiveScope(element)) return
@@ -714,8 +714,10 @@ function focusElement(element: HTMLElement | null) {
element.classList.add("nav-focused")
element.focus({ preventScroll: true })
ensureElementVisibleVertically(element)
syncRowsToElement(element)
if (!options?.preserveScroll) {
ensureElementVisibleVertically(element)
syncRowsToElement(element)
}
focusedElement.value = element
}
@@ -727,13 +729,16 @@ function getFocusState(): { row: number; col: number } | null {
return { row, col }
}
function restoreFocusState(state: { row: number; col: number } | null) {
function restoreFocusState(
state: { row: number; col: number } | null,
options?: { preserveScroll?: boolean },
) {
if (!state) return
const target = findElementAt(state.row, state.col)
if (target) {
setTimeout(() => {
focusElement(target.element)
focusElement(target.element, options)
}, 50)
}
}
@@ -893,6 +898,32 @@ export function installKeyboardNavigation() {
})
}
export interface SyncedRowScrollSnapshot {
rows: [HTMLElement, number][]
offset: number
targetOffset: number
}
function snapshotSyncedRowScroll(): SyncedRowScrollSnapshot {
return {
rows: getSyncedRows().map((row) => [row, row.scrollLeft]),
offset: syncedRowsCurrentOffset,
targetOffset: syncedRowsTargetOffset,
}
}
function restoreSyncedRowScroll(snapshot: SyncedRowScrollSnapshot) {
stopSyncedRowAnimation()
for (const [row, scrollLeft] of snapshot.rows) {
if (row.isConnected) {
row.scrollLeft = scrollLeft
}
}
syncedRowsCurrentOffset = snapshot.offset
syncedRowsTargetOffset = snapshot.targetOffset
lastSyncedRowsAnimationAt = null
}
export function useKeyboardNavigation() {
return {
focusedElement,
@@ -901,6 +932,8 @@ export function useKeyboardNavigation() {
focusAt,
getFocusState,
restoreFocusState,
snapshotSyncedRowScroll,
restoreSyncedRowScroll,
}
}
+17
View File
@@ -6,10 +6,27 @@ import { installKeyboardNavigation } from "./composables/useKeyboardNavigation"
import { installGamepadNavigation } from "./composables/useGamepadNavigation"
import { installInputModalityTracking } from "./composables/useInputModality"
function installReloadShortcut() {
document.addEventListener(
"keydown",
(event) => {
if (
event.key === "F5" ||
((event.ctrlKey || event.metaKey) && event.key.toLowerCase() === "r")
) {
event.preventDefault()
window.location.reload()
}
},
{ capture: true },
)
}
// Install global keyboard navigation handlers immediately
installInputModalityTracking()
installKeyboardNavigation()
installGamepadNavigation()
installReloadShortcut()
// Unregister any legacy service workers — MediaHive no longer uses a PWA/SW.
if ("serviceWorker" in navigator) {
+4
View File
@@ -420,6 +420,10 @@ html:not(.mouse-active) .btn-secondary.nav-focused {
outline: none;
text-decoration: none;
color: inherit;
/* Skip rendering work for cards scrolled out of view (long rows). Width is
fixed; the intrinsic height is only a pre-first-render estimate. */
content-visibility: auto;
contain-intrinsic-height: auto 330px;
}
html.mouse-active .media-card:hover,
+7
View File
@@ -109,6 +109,13 @@ export interface Series {
seasons: Season[]
}
/** A series' single continue point (last watched position). */
export interface SeriesResumePoint {
seasonNumber: number
episodeNumber: number
positionSeconds: number
}
export interface MovieUi extends Movie {
id: string
root_id: string | null
+3 -1
View File
@@ -5,13 +5,14 @@
* Configures Vite for FastAPI backend integration:
* - Proxies /api/* requests to the FastAPI backend
* - Builds to the Python module's frontend-build directory
* - Disables Vite's screen clearing on startup
*
* Options:
* paths - Array of paths to proxy (default: ["/api"])
*/
export default function fastapiVue({ paths = ["/api"] } = {}) {
const backendUrl = process.env.MEDIAHIVE_BACKEND_URL || "http://localhost:8420"
const backendUrl = process.env.MEDIAHIVE_BACKEND_URL || "http://localhost:8421"
// Build proxy configuration for each path
const proxy = {}
@@ -26,6 +27,7 @@ export default function fastapiVue({ paths = ["/api"] } = {}) {
return {
name: "vite-plugin-fastapi-mediahive",
config: () => ({
clearScreen: false,
server: { proxy },
build: {
outDir: "../mediahive/frontend-build",
+79 -2
View File
@@ -33,6 +33,54 @@ def _derive_name(path: str) -> str:
return p.name or p.anchor.strip("/\\").lower() or "media"
def _dev_reload_supervisor() -> None:
"""Windows dev-mode reloader: restart the server process on changes.
uvicorn's own reload cannot work here: it restarts the child with
CTRL_C_EVENT, which is never delivered to a plain spawn child (no own
console process group), so the reloader blocks in join() after the
first reload and the old server — scanner included — keeps running.
And even when the child does restart, uvicorn passes it sockets bound
by the parent; ProactorEventLoop cannot register inherited sockets
with IOCP (WinError 87 on accept), while the selector loop would lose
asyncio subprocess support (ffmpeg/ffprobe showreel generation).
So: watch the package directory ourselves and respawn a fresh child
process that binds its own sockets. The child runs with
MEDIAHIVE_DEV_CHILD=1 and reload disabled. Scanner state is persisted
after every scan, so a non-graceful child exit on reload loses nothing.
"""
import subprocess
import watchfiles
watch_dir = Path(__file__).parent
argv = [sys.executable, "-m", "mediahive", *sys.argv[1:]]
child_env = dict(os.environ, MEDIAHIVE_DEV_CHILD="1")
print(f"Dev reloader: watching {watch_dir}", file=sys.stderr)
proc = subprocess.Popen(argv, env=child_env)
try:
for changes in watchfiles.watch(watch_dir):
changed = sorted({str(Path(p).name) for _, p in changes})
print(
f"Dev reloader: change in {', '.join(changed[:5])} — restarting",
file=sys.stderr,
)
proc.terminate()
proc.wait()
proc = subprocess.Popen(argv, env=child_env)
except KeyboardInterrupt:
pass
finally:
proc.terminate()
try:
proc.wait(timeout=5)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait()
def main() -> None:
_configure_windows_event_loop_policy()
@@ -54,9 +102,30 @@ def main() -> None:
action="append",
help=(f"Endpoint (default: localhost:{DEFAULT_PORT})."),
)
parser.add_argument(
"--gui",
action="store_true",
help="Run with GUI (fails if GUI dependencies are not installed)",
)
args = parser.parse_args()
# --listen implies server-only mode; use --gui to force GUI even with --listen.
use_gui = args.gui or not args.listen
if use_gui:
try:
from mediahive.winmain import gui_main
except ImportError as exc:
if args.gui:
raise RuntimeError(
"GUI dependencies are not installed. "
"Install with: uv pip install mediahive[gui]"
) from exc
else:
gui_main()
return
if args.media_folders:
roots: dict[str, str] = {}
for path in args.media_folders:
@@ -73,13 +142,21 @@ def main() -> None:
roots[name] = p.as_posix()
os.environ["MEDIAHIVE_ROOTS"] = json.dumps(roots)
dev = {"reload": True, "reload_dirs": ["mediahive"]}
if (
DEVMODE
and sys.platform == "win32"
and os.environ.get("MEDIAHIVE_DEV_CHILD") != "1"
):
_dev_reload_supervisor()
return
server.run(
"mediahive.server:app",
listen=args.listen,
default_port=DEFAULT_PORT,
server_header=False,
loop="none" if sys.platform == "win32" else "auto",
**(dev if DEVMODE and sys.platform != "win32" else {}),
reload=Path(__file__).parent if DEVMODE and sys.platform != "win32" else False,
)
-253
View File
@@ -1,253 +0,0 @@
"""Custom access logging middleware for FastAPI/Uvicorn."""
import logging
import sys
import time
from ipaddress import IPv6Address
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import Response
logger = logging.getLogger("mediahive.access")
_RESET = "\033[0m"
_STATUS_INFO = "\033[32m" # 1xx (green)
_STATUS_OK = "\033[1;92m" # 2xx (bright green)
_STATUS_REDIRECT = "\033[32m" # 3xx (green)
_STATUS_CLIENT_ERR = "\033[0;31m" # 4xx (red)
_STATUS_SERVER_ERR = "\033[1;91m" # 5xx (bold bright red)
_METHOD_READ = "\033[0;34m" # GET, HEAD, OPTIONS (blue)
_METHOD_WRITE = "\033[1;94m" # POST, PUT, DELETE, PATCH (bold bright blue)
_HOST = "\033[38;5;242m" # hostname (dark grey)
_PATH = "\033[38;5;250m" # path (white)
_TIMING = "\033[38;5;242m" # timing/devmode (dark grey)
_WS_OPEN = "\033[38;5;226m" # WebSocket connect (brightest yellow from 6x6x6 cube)
_WS_CLOSE = "\033[38;5;142m" # WebSocket disconnect (significantly dimmer yellow)
_WS_STATUS = "\033[38;5;242m" # WebSocket close status (dark grey)
def format_ipv6_network(ip: str) -> str:
"""Format IPv6 address to show only network part (first 64 bits).
Special addresses are returned as-is for clarity:
- ::1 (loopback)
- :: (unspecified)
- ::ffff:x.x.x.x (IPv4-mapped, returns just the IPv4 part)
- fe80:: (link-local, returned as-is since interface-specific)
"""
try:
# Strip brackets that some proxies add around IPv6
ip = ip.strip("[]")
# Strip zone ID (e.g., fe80::1%eth0)
if "%" in ip:
ip = ip.split("%")[0]
addr = IPv6Address(ip)
# Special cases - return as-is or with minimal processing
if addr.is_loopback: # ::1
return "::1"
if addr.is_unspecified: # ::
return "::"
if addr.ipv4_mapped: # ::ffff:x.x.x.x
return str(addr.ipv4_mapped)
if addr.is_link_local: # fe80::/10 - interface-specific, keep full
return str(addr)
# Regular addresses: truncate to /64 network prefix
network_int = int(addr) >> 64
# Format as IPv6 with trailing ::
# Split into 4 groups of 16 bits
groups = []
for _ in range(4):
groups.insert(0, format(network_int & 0xFFFF, "x"))
network_int >>= 16
# Compress consecutive zero groups
result = ":".join(groups) + "::"
# Simplify leading zeros in groups and compress, then strip trailing ::
return str(IPv6Address(result + "0")).removesuffix("::")
except Exception:
return ip
def format_client_ip(ip: str) -> str:
"""Format client IP, compressing IPv6 to network part only."""
if not ip or ip == "-":
return "-"
# Strip brackets for detection (some proxies add them)
stripped = ip.strip("[]")
if ":" in stripped:
return format_ipv6_network(ip)
return ip
def status_color(status: int) -> str:
"""Return color code based on HTTP status."""
if status < 200:
return _STATUS_INFO
if status < 300:
return _STATUS_OK
if status < 400:
return _STATUS_REDIRECT
if status < 500:
return _STATUS_CLIENT_ERR
return _STATUS_SERVER_ERR
def method_color(method: str) -> str:
"""Return color code based on HTTP method."""
if method in ("GET", "HEAD", "OPTIONS"):
return _METHOD_READ
return _METHOD_WRITE
def format_access_log(
client: str,
status: int,
method: str,
host: str,
path: str,
duration_ms: float,
extra: str = "",
) -> str:
"""Format access log line with colors and aligned fields."""
# Format components with fixed widths for alignment
ip = format_client_ip(client).ljust(19) # IPv6 network max 19 chars
timing = f"{duration_ms:.0f}ms"
method_padded = method.ljust(7) # Longest method is OPTIONS (7)
status_str = f"{status_color(status)}{status}{_RESET}"
timing_str = f"{_TIMING}{timing}{_RESET}"
method_str = f"{method_color(method)}{method_padded}{_RESET}"
host_str = f"{_HOST}{host}{_RESET}"
path_str = f"{_PATH}{path}{_RESET}"
# Format: "IP STATUS METHOD host path [extra] TIMING"
extra_str = f" {_TIMING}{extra}{_RESET}" if extra else ""
return (
f"{ip} {status_str} {method_str} {host_str}{path_str}{extra_str} {timing_str}"
)
# WebSocket connection counter (mod 100)
_ws_counter = 0
def _next_ws_id() -> int:
"""Get next WebSocket connection ID (0-99)."""
global _ws_counter
ws_id = _ws_counter
_ws_counter = (_ws_counter + 1) % 100
return ws_id
def log_ws_open(ws) -> int:
"""Log WebSocket connection open. Returns connection ID for use in close."""
ws_id = _next_ws_id()
client = ws.client.host if ws.client else "-"
host = ws.headers.get("host", "-")
path = ws.url.path
origin = ws.headers.get("origin")
ip = format_client_ip(client).ljust(19)
# ID right-aligned like status codes (3 chars), emoji formatted like method
id_str = f"{_WS_OPEN}{str(ws_id).rjust(3)}{_RESET}"
# Emoji (2 display width) + 6 spaces = 8 display chars, but within color for alignment
emoji_str = f"{_METHOD_READ}🔌 {_RESET}"
# Determine if origin should be shown (omit when same as host)
# Origin header includes scheme (e.g., "https://example.com"), compare host part
origin_host = origin.split("://", 1)[-1] if origin else None
show_origin = origin_host and origin_host != host
host_str = f"{_HOST}{host}{_RESET}"
path_str = f"{_PATH}{path}{_RESET}"
origin_str = f" {_RESET}from {_HOST}{origin_host}{_RESET}" if show_origin else ""
logger.info(f"{ip} {id_str} {emoji_str}{host_str}{path_str}{origin_str}")
return ws_id
# WebSocket close codes to human-readable status
WS_CLOSE_CODES = {
1000: "ok",
1001: "going away",
1002: "protocol error",
1003: "unsupported",
1005: "no status",
1006: "abnormal",
1007: "invalid data",
1008: "policy violation",
1009: "too large",
1010: "extension required",
1011: "server error",
1012: "restarting",
1013: "try again",
1014: "bad gateway",
1015: "tls error",
}
def log_ws_close(ws_id: int, close_code: int | None, duration: float) -> None:
"""Log WebSocket connection close with duration and status."""
# ID right-aligned like status codes (3 chars), "closed" formatted like method
id_str = f"{_WS_CLOSE}{str(ws_id).rjust(3)}{_RESET}"
# Pad within the dim color to keep full width in color (8 display chars)
closed_str = f"{_TIMING}closed {_RESET}"
timing = f"{duration * 1000:.0f}ms"
# Convert close code to status text
if close_code is None:
code = "----"
status = "unknown"
else:
code = str(close_code)
status = WS_CLOSE_CODES.get(close_code, f"code {close_code}")
# Status code and text in normal color, not dim
status_str = f"{code} {status}"
timing_str = f"{_TIMING}{timing}{_RESET}"
logger.info(f"{' ' * 19} {id_str} {closed_str}{status_str} {timing_str}")
class AccessLogMiddleware(BaseHTTPMiddleware):
"""Middleware that logs HTTP requests with custom format."""
async def dispatch(self, request: Request, call_next) -> Response:
start = time.perf_counter()
response = await call_next(request)
duration_ms = (time.perf_counter() - start) * 1000
client = request.client.host if request.client else "-"
host = request.headers.get("host", "-")
method = request.method
path = request.url.path
if request.url.query:
path = f"{path}?{request.url.query}"
status = response.status_code
extra = getattr(request.state, "log_extra", "")
line = format_access_log(
client, status, method, host, path, duration_ms, extra=extra
)
logger.info(line)
return response
def configure_access_logging():
"""Configure the access logger to output to stderr."""
handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(logging.Formatter("%(message)s"))
logger.addHandler(handler)
logger.setLevel(logging.INFO)
logger.propagate = False
# Suppress uvicorn access logs to avoid duplicate request lines.
logging.getLogger("uvicorn.access").setLevel(logging.WARNING)
# Suppress uvicorn websocket "connection open/closed" messages.
logging.getLogger("uvicorn.error").setLevel(logging.WARNING)
# Suppress watchfiles "X changes detected" INFO messages (keep WARNING for reload notification)
logging.getLogger("watchfiles.main").setLevel(logging.WARNING)
+1 -1
View File
@@ -14,7 +14,7 @@ import msgspec
import msgspec.toml
class Config(msgspec.Struct):
class Config(msgspec.Struct, omit_defaults=True):
media_folder: str | None = None
roots: dict[str, str] | None = None
+103 -17
View File
@@ -148,20 +148,30 @@ async def _cache_people_profiles(
if not info or not info.cast:
return info, people
semaphore = asyncio.Semaphore(8)
async def fetch_profile(cast_credit, person):
async with semaphore:
return cast_credit.id, await download_cast_profile(
person.profile_path,
media_folder,
person.name,
cast_credit.id,
)
tasks = []
for cast_credit in info.cast:
if cast_credit.id is None:
continue
person = people.get(cast_credit.id)
if person is None or not person.profile_path:
continue
downloaded_path = await download_cast_profile(
person.profile_path,
media_folder,
person.name,
cast_credit.id,
)
tasks.append(fetch_profile(cast_credit, person))
for cast_id, downloaded_path in await asyncio.gather(*tasks):
if downloaded_path:
people[cast_credit.id] = Person(
person = people[cast_id]
people[cast_id] = Person(
name=person.name,
profile_path=Path(downloaded_path).name,
gender=person.gender,
@@ -378,6 +388,25 @@ async def _build_seasons_data(
seasons_map[season_num] = {}
seasons_map[season_num][episode_num] = files
# Prefetch all missing season details in parallel; the loop below then
# reads them straight from season_cache.
if tmdb_id:
missing = [
season_num
for season_num in seasons_map
if (tmdb_id, season_num) not in season_cache
]
if missing:
semaphore = asyncio.Semaphore(4)
async def prefetch(num: int) -> None:
async with semaphore:
season_cache[tmdb_id, num] = await fetch_season_details(
tmdb_id, num
)
await asyncio.gather(*(prefetch(num) for num in missing))
seasons_data = []
for season_num in sorted(seasons_map.keys()):
episodes_in_season = seasons_map[season_num]
@@ -442,11 +471,15 @@ async def _process_movies(
generate_showreels: bool,
media_root: str | None = None,
root_id: str | None = None,
) -> AsyncIterator[tuple[str, Movie, tuple[str, Path, str] | None, dict[int, Person]]]:
) -> AsyncIterator[
tuple[str, Movie, tuple[str, Path, str] | None, dict[int, Person], list[str]]
]:
"""Async generator that processes all movies.
Yields:
Tuples of ``(Movie, showreel_task_or_None)`` as each movie is processed.
Tuples of ``(movie_id, Movie, showreel_task_or_None, people, scanned)``
as each movie is processed. ``scanned`` lists the media-root-relative
torrent paths whose content was (re)scanned to build the movie.
"""
_ = root_id
@@ -499,6 +532,21 @@ async def _process_movies(
len(categories[ContentType.MOVIE]),
) if movie_groups else None
# Prefetch TMDb lookups for all unique titles in parallel; the grouping
# loop below then reads them straight from movie_tmdb_cache.
if movie_groups:
semaphore = asyncio.Semaphore(4)
first_by_key = {
f"{items[0].title.lower()}:{items[0].year}": items[0]
for items in movie_groups.values()
}
async def prefetch(item: ParsedContent) -> None:
async with semaphore:
await get_movie_tmdb(item.title, item.year)
await asyncio.gather(*(prefetch(item) for item in first_by_key.values()))
for idx, (_movie_key, items) in enumerate(movie_groups.items(), 1):
first_item = items[0]
logger.debug(
@@ -640,7 +688,10 @@ async def _process_movies(
showreel_source_sets=showreel_source_sets or None,
files=files,
)
yield item_id, movie, showreel_task, people
scanned = [
make_relative_path(item.path.as_posix(), media_root) for item in items
]
yield item_id, movie, showreel_task, people, scanned
# Process movies without TMDb info
for group_data in no_tmdb_movie_groups.values():
@@ -712,7 +763,10 @@ async def _process_movies(
showreel_source_sets=showreel_source_sets or None,
files=files,
)
yield item_id, movie, showreel_task, {}
scanned = [
make_relative_path(item.path.as_posix(), media_root) for item in items
]
yield item_id, movie, showreel_task, {}, scanned
async def _process_series(
@@ -723,12 +777,16 @@ async def _process_series(
media_root: str | None = None,
root_id: str | None = None,
) -> AsyncIterator[
tuple[str, Series, list[tuple[str, Path, int, int, str]], dict[int, Person]]
tuple[
str, Series, list[tuple[str, Path, int, int, str]], dict[int, Person], list[str]
]
]:
"""Async generator that processes all series.
Yields:
Tuples of ``(Series, episode_reel_tasks)`` as each series is processed.
Tuples of ``(series_id, Series, episode_reel_tasks, people, scanned)``
as each series is processed. ``scanned`` lists the media-root-relative
torrent paths whose content was (re)scanned to build the series.
"""
_ = root_id
@@ -783,6 +841,20 @@ async def _process_series(
len(categories[ContentType.SERIES]),
)
# Prefetch TMDb lookups for all unique titles in parallel; the grouping
# loop below then reads them straight from series_tmdb_cache.
if series_groups:
semaphore = asyncio.Semaphore(4)
first_by_key = {
items[0].title.lower(): items[0] for items in series_groups.values()
}
async def prefetch(item: ParsedContent) -> None:
async with semaphore:
await get_series_tmdb(item.title)
await asyncio.gather(*(prefetch(item) for item in first_by_key.values()))
for idx, (_series_key, items) in enumerate(series_groups.items(), 1):
first_item = items[0]
logger.debug(" [%d/%d] %s", idx, len(series_groups), first_item.title)
@@ -879,7 +951,11 @@ async def _process_series(
)
if not seasons_data:
logger.info(" Skipping %s - no episodes found", display_title)
logger.info(
" Skipping %s - no episodes in the %d scanned torrent(s)",
display_title,
len(items),
)
continue
different_titles = sorted(
@@ -898,7 +974,10 @@ async def _process_series(
backdrop_path=make_relative_path(backdrop_path, media_root),
seasons=seasons_data,
)
yield series_id, series, ep_reel_tasks, people
scanned = [
make_relative_path(item.path.as_posix(), media_root) for item in items
]
yield series_id, series, ep_reel_tasks, people, scanned
# Process series without TMDb info
for group_data in no_tmdb_groups.values():
@@ -928,7 +1007,11 @@ async def _process_series(
)
if not seasons_data:
logger.info(" Skipping %s - no episodes found", title)
logger.info(
" Skipping %s - no episodes in the %d scanned torrent(s)",
title,
len(items),
)
continue
item_timestamps = [await get_added_timestamp(item.path) for item in items]
@@ -941,4 +1024,7 @@ async def _process_series(
cover_path=make_relative_path(cover_path, media_root),
seasons=seasons_data,
)
yield series_id, series, ep_reel_tasks, {}
scanned = [
make_relative_path(item.path.as_posix(), media_root) for item in items
]
yield series_id, series, ep_reel_tasks, {}, scanned
File diff suppressed because it is too large Load Diff
+12 -1
View File
@@ -28,12 +28,23 @@ VIDEO_EXTENSIONS = {
".m2ts",
}
# Caches for expensive operations
# Caches for expensive operations. These are per-scan only: the scanner
# clears them at the start of every scan. Caching across scans is wrong —
# an empty result recorded before a download finished (or during a transient
# network-mount error) would stick for the process lifetime and report
# "no episodes found" for series that do have episodes.
_episode_files_cache: dict[str, dict[tuple[int, int], list[tuple[str, int]]]] = {}
_playable_file_cache: dict[str, str | None] = {}
_bluray_probe_file_cache: dict[str, str | None] = {}
def clear_scan_caches() -> None:
"""Drop all per-scan filesystem caches; called at the start of each scan."""
_episode_files_cache.clear()
_playable_file_cache.clear()
_bluray_probe_file_cache.clear()
def _scandir_split(
directory: Path,
stop_event: threading.Event,
+94 -1
View File
@@ -7,13 +7,14 @@ and HDR passthrough.
import asyncio
import contextlib
import json
import logging
import re
import shlex
import subprocess
import sys
from collections import Counter
from dataclasses import dataclass
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Any
@@ -425,6 +426,85 @@ class MediaProbeInfo:
_media_probe_cache: dict[str, MediaProbeInfo] = {}
# ---------------------------------------------------------------------------
# Persistent probe records
#
# Probe results are keyed by (path, mtime, size) and persisted to
# ``probe-cache.json`` under the root's .mediahive folder so that process
# restarts do not re-run ffmpeg on unchanged files. The in-RAM structures
# are process-global (keyed by absolute path, so sharing across roots is
# safe); each root loads/saves its own file, merging into the same dict.
# ---------------------------------------------------------------------------
_probe_records: dict[str, dict] = {}
_probe_records_path: Path | None = None
_probe_records_dirty = False
def load_probe_records(path: Path) -> None:
"""Load persisted probe records from ``path`` (missing file is fine)."""
global _probe_records_path
_probe_records_path = path
try:
data = json.loads(path.read_text(encoding="utf-8"))
except FileNotFoundError:
return
except OSError, ValueError:
logger.exception("Failed to load probe cache from %s", path)
return
records = data.get("records")
if isinstance(records, dict):
_probe_records.update(records)
logger.info("Loaded probe cache: %d records from %s", len(records), path)
def probe_records_dirty() -> bool:
return _probe_records_dirty
def save_probe_records() -> None:
"""Persist probe records if any were added since the last save."""
global _probe_records_dirty
if not _probe_records_dirty or _probe_records_path is None:
return
_probe_records_dirty = False
try:
payload = json.dumps({"version": 1, "records": _probe_records})
tmp = _probe_records_path.with_suffix(".tmp")
tmp.parent.mkdir(parents=True, exist_ok=True)
tmp.write_text(payload, encoding="utf-8")
tmp.replace(_probe_records_path)
except OSError, TypeError, ValueError:
logger.exception("Failed to save probe cache to %s", _probe_records_path)
def _record_probe(video_path: str, stat_info, info: MediaProbeInfo) -> None:
global _probe_records_dirty
if stat_info is None:
return # Non-plain paths (bluray:/concat: URIs) are not persisted
_probe_records[video_path] = {
"mtime": int(stat_info.st_mtime),
"size": stat_info.st_size,
"info": asdict(info),
}
_probe_records_dirty = True
def _lookup_probe_record(video_path: str, stat_info) -> MediaProbeInfo | None:
rec = _probe_records.get(video_path)
if rec is None or stat_info is None:
return None
if rec.get("mtime") != int(stat_info.st_mtime):
return None
if rec.get("size") != stat_info.st_size:
return None
try:
return MediaProbeInfo(**rec["info"])
except TypeError, KeyError:
return None
_duration_re = re.compile(r"Duration:\s*(\d+):(\d+):(\d+(?:\.\d+)?)")
_dimension_re = re.compile(r"(\d{2,5})x(\d{2,5})")
_dovi_profile_re = re.compile(
@@ -449,11 +529,23 @@ async def probe_media_info(video_path: str) -> MediaProbeInfo:
if cached is not None:
return cached
# Stat once: used both to validate persisted records and to key new ones.
# Non-plain paths (bluray:/concat: URIs) fail stat and stay memory-cached.
stat_info = None
with contextlib.suppress(OSError, ValueError):
stat_info = await AsyncPath(video_path).stat()
recorded = _lookup_probe_record(video_path, stat_info)
if recorded is not None:
_media_probe_cache[video_path] = recorded
return recorded
info = MediaProbeInfo()
cmd = ["ffmpeg", "-hide_banner", "-i", video_path]
ffmpeg_result = await _run_ffmpeg(cmd, timeout_seconds=30, allow_nonzero_exit=True)
if ffmpeg_result is None:
_media_probe_cache[video_path] = info
_record_probe(video_path, stat_info, info)
return info
stdout, stderr = ffmpeg_result
@@ -552,6 +644,7 @@ async def probe_media_info(video_path: str) -> MediaProbeInfo:
info.subtitle_languages = subtitle_languages or None
_media_probe_cache[video_path] = info
_record_probe(video_path, stat_info, info)
return info
+5 -18
View File
@@ -2,7 +2,6 @@
import os
import re
import time
import unicodedata
from pathlib import Path
@@ -11,9 +10,6 @@ from aiopathlib import AsyncPath
# Default output folder name (created at common root of scanned paths)
DEFAULT_OUTPUT_FOLDER = ".mediahive"
# Threshold for considering atime "too close" to current time (1 hour)
_ATIME_FRESHNESS_THRESHOLD = 3600
# Resolution priority for quality sorting (higher = better)
RESOLUTION_PRIORITY = {
"8K": 5,
@@ -108,10 +104,9 @@ def normalize_resolution_label(value: str | None) -> str | None:
async def get_added_timestamp(path: Path) -> int | None:
"""Get the timestamp when a torrent was added to the collection.
Heuristic:
- For directories: use ctime (most accurate for torrent folder creation)
- For files: use atime unless it's too close to current time (suggesting
the filesystem updates atime on reads), otherwise use max(mtime, ctime)
Best-effort rule:
- On Windows: use ctime (creation-time semantics)
- On other OSes: use mtime (ctime is metadata-change time on Unix)
Returns:
Unix timestamp as int, or None if path doesn't exist
@@ -122,17 +117,9 @@ async def get_added_timestamp(path: Path) -> int | None:
stat_info = await ap.stat()
except OSError, PermissionError:
return None
if await ap.is_dir():
if os.name == "nt":
return int(stat_info.st_ctime)
now = time.time()
atime = stat_info.st_atime
if now - atime < _ATIME_FRESHNESS_THRESHOLD:
return int(max(stat_info.st_mtime, stat_info.st_ctime))
return int(atime)
return int(stat_info.st_mtime)
def get_directory_size(path: Path) -> int:
+291 -18
View File
@@ -18,10 +18,13 @@ from aiopathlib import AsyncPath
from fastapi import WebSocket
from mediahive.models.data import (
Episode,
IndexSnapshot,
Movie,
Season,
Series,
TaskInfo,
Torrent,
)
from mediahive.models.events import Remove, Task, Upsert
from mediahive.models.tmdb import Person
@@ -147,6 +150,120 @@ class IndexStore:
return None
return item.info.tmdb_id
def torrent_paths(self) -> set[str]:
"""All media-root-relative torrent paths currently in the index.
The scanner uses this to reprocess items that are missing from the
index even though their mtime is unchanged (e.g. after the snapshot
was wiped or an upsert never landed).
"""
paths: set[str] = set()
for movie in self.movies.values():
paths.update(movie.files)
for show in self.series.values():
for season in show.seasons:
for episode in season.episodes:
paths.update(episode.files)
return paths
@staticmethod
def _newest_from_files(files: dict[str, Torrent]) -> int | None:
timestamps = [t.added_at for t in files.values() if t.added_at]
return max(timestamps) if timestamps else None
# ------------------------------------------------------------------
# Merge helpers (partial rescan support)
# ------------------------------------------------------------------
def _merge_movie(self, existing: Movie, new: Movie, scanned: set[str]) -> Movie:
"""Merge a partially rebuilt movie into the existing entry.
File entries belonging to torrents in ``scanned`` are replaced by the
new data; everything else is preserved.
"""
files = {k: v for k, v in existing.files.items() if k not in scanned}
files.update(new.files)
return Movie(
title=new.title or existing.title,
info=new.info or existing.info,
year=new.year if new.year is not None else existing.year,
newest=(
self._newest_from_files(files)
or max(filter(None, [existing.newest, new.newest]), default=None)
),
cover_path=new.cover_path or existing.cover_path,
backdrop_path=new.backdrop_path or existing.backdrop_path,
showreel_images=new.showreel_images or existing.showreel_images,
showreel_source_sets=new.showreel_source_sets
or existing.showreel_source_sets,
files=files,
)
def _merge_series(self, existing: Series, new: Series, scanned: set[str]) -> Series:
"""Merge a partially rebuilt series into the existing entry.
File entries belonging to torrents in ``scanned`` are replaced by the
new data; seasons/episodes/files from torrents that were not rescanned
are preserved. Episodes and seasons left without files are dropped.
"""
seasons: dict[int, Season] = {}
for season in existing.seasons:
episodes: dict[int, Episode] = {}
for ep in season.episodes:
files = {k: v for k, v in ep.files.items() if k not in scanned}
if files:
episodes[ep.episode_number] = msgspec.structs.replace(
ep, files=files
)
if episodes:
seasons[season.season_number] = msgspec.structs.replace(
season,
episodes=list(episodes.values()),
episode_count=len(episodes),
)
for season in new.seasons:
current = seasons.get(season.season_number)
if current is None:
seasons[season.season_number] = season
continue
episodes = {ep.episode_number: ep for ep in current.episodes}
for ep in season.episodes:
old = episodes.get(ep.episode_number)
if old is None:
episodes[ep.episode_number] = ep
continue
# Same episode from an unscanned torrent too: union the files,
# prefer fresh metadata/reel info from the new scan.
files = dict(old.files)
files.update(ep.files)
episodes[ep.episode_number] = msgspec.structs.replace(
ep,
files=files,
reel_image=ep.reel_image or old.reel_image,
reel_sources=ep.reel_sources or old.reel_sources,
)
ordered = [episodes[k] for k in sorted(episodes)]
seasons[season.season_number] = msgspec.structs.replace(
season,
episodes=ordered,
episode_count=len(ordered),
poster_path=season.poster_path or current.poster_path,
)
alt_titles = sorted(
set(existing.alternative_titles or []) | set(new.alternative_titles or [])
)
return Series(
title=new.title or existing.title,
info=new.info or existing.info,
alternative_titles=alt_titles or None,
newest=max(filter(None, [existing.newest, new.newest]), default=None),
cover_path=new.cover_path or existing.cover_path,
backdrop_path=new.backdrop_path or existing.backdrop_path,
seasons=[seasons[k] for k in sorted(seasons)],
)
def _rebuild_tmdb_indexes(self) -> None:
"""Rebuild TMDb id lookup maps from the current in-memory items."""
self._movie_tmdb_ids.clear()
@@ -189,22 +306,34 @@ class IndexStore:
self._rebuild_tmdb_indexes()
def _collapse_movie_tmdb_duplicates(self, tmdb_id: int, keep_id: str) -> None:
"""Remove other movie entries that share a TMDb id."""
"""Fold other entries that share a TMDb id into the kept one."""
for item_id, movie in list(self.movies.items()):
if item_id == keep_id:
continue
if self._get_tmdb_id(movie) == tmdb_id:
self.movies.pop(item_id, None)
self._rebuild_tmdb_indexes()
if self._get_tmdb_id(movie) != tmdb_id:
continue
kept = self.movies.get(keep_id)
if kept is not None:
# Preserve any file versions the duplicate alone carried.
self.movies[keep_id] = self._merge_movie(movie, kept, set())
self.movies.pop(item_id, None)
if self._movie_tmdb_ids.get(tmdb_id) == item_id:
self._movie_tmdb_ids[tmdb_id] = keep_id
def _collapse_series_tmdb_duplicates(self, tmdb_id: int, keep_id: str) -> None:
"""Remove other series entries that share a TMDb id."""
"""Fold other entries that share a TMDb id into the kept one."""
for item_id, series in list(self.series.items()):
if item_id == keep_id:
continue
if self._get_tmdb_id(series) == tmdb_id:
self.series.pop(item_id, None)
self._rebuild_tmdb_indexes()
if self._get_tmdb_id(series) != tmdb_id:
continue
kept = self.series.get(keep_id)
if kept is not None:
# Preserve any seasons/episodes the duplicate alone carried.
self.series[keep_id] = self._merge_series(series, kept, set())
self.series.pop(item_id, None)
if self._series_tmdb_ids.get(tmdb_id) == item_id:
self._series_tmdb_ids[tmdb_id] = keep_id
async def _write_snapshot(self) -> None:
"""Write current index to disk (called from debounce task)."""
@@ -300,8 +429,14 @@ class IndexStore:
item_id: str,
item: Movie,
people: dict[int, Person] | None = None,
scanned: list[str] | None = None,
) -> bool:
"""Insert or update a movie. Returns True if it was a real change."""
"""Insert or update a movie. Returns True if it was a real change.
When ``scanned`` is given, the item is a partial rebuild covering only
those torrent paths; it is merged into the existing entry instead of
replacing it.
"""
tmdb_id = self._get_tmdb_id(item)
existing_id = self._movie_tmdb_ids.get(tmdb_id) if tmdb_id is not None else None
if existing_id is not None and existing_id != item_id:
@@ -311,6 +446,10 @@ class IndexStore:
if tmdb_id is not None:
self._movie_tmdb_ids[tmdb_id] = item_id
self._collapse_movie_tmdb_duplicates(tmdb_id, item_id)
existing = self.movies.get(item_id)
if existing is not None and scanned is not None:
item = self._merge_movie(existing, item, set(scanned))
if existing is not None:
if msgspec.json.encode(existing) == msgspec.json.encode(item):
@@ -334,8 +473,14 @@ class IndexStore:
item_id: str,
item: Series,
people: dict[int, Person] | None = None,
scanned: list[str] | None = None,
) -> bool:
"""Insert or update a series. Returns True if it was a real change."""
"""Insert or update a series. Returns True if it was a real change.
When ``scanned`` is given, the item is a partial rebuild covering only
those torrent paths; it is merged into the existing entry instead of
replacing it.
"""
tmdb_id = self._get_tmdb_id(item)
existing_id = (
self._series_tmdb_ids.get(tmdb_id) if tmdb_id is not None else None
@@ -347,6 +492,10 @@ class IndexStore:
if tmdb_id is not None:
self._series_tmdb_ids[tmdb_id] = item_id
self._collapse_series_tmdb_duplicates(tmdb_id, item_id)
existing = self.series.get(item_id)
if existing is not None and scanned is not None:
item = self._merge_series(existing, item, set(scanned))
if existing is not None:
if msgspec.json.encode(existing) == msgspec.json.encode(item):
@@ -367,22 +516,146 @@ class IndexStore:
def remove_movie(self, item_id: str) -> None:
"""Remove a movie from the index and broadcast."""
self.movies.pop(item_id, None)
for tmdb_id, mapped_id in list(self._movie_tmdb_ids.items()):
if mapped_id == item_id:
self._movie_tmdb_ids.pop(tmdb_id, None)
movie = self.movies.pop(item_id, None)
if movie is None:
return
tmdb_id = self._get_tmdb_id(movie)
if tmdb_id is not None and self._movie_tmdb_ids.get(tmdb_id) == item_id:
self._movie_tmdb_ids.pop(tmdb_id, None)
self._schedule_snapshot()
self._broadcast(Remove(kind="movie", id=item_id))
def remove_series(self, item_id: str) -> None:
"""Remove a series from the index and broadcast."""
self.series.pop(item_id, None)
for tmdb_id, mapped_id in list(self._series_tmdb_ids.items()):
if mapped_id == item_id:
self._series_tmdb_ids.pop(tmdb_id, None)
series = self.series.pop(item_id, None)
if series is None:
return
tmdb_id = self._get_tmdb_id(series)
if tmdb_id is not None and self._series_tmdb_ids.get(tmdb_id) == item_id:
self._series_tmdb_ids.pop(tmdb_id, None)
self._schedule_snapshot()
self._broadcast(Remove(kind="series", id=item_id))
# ------------------------------------------------------------------
# Scanner-driven maintenance
# ------------------------------------------------------------------
def sync_torrent_paths(self, paths: set[str]) -> None:
"""Drop file entries whose torrent path no longer exists on disk.
``paths`` is the complete set of media-root-relative torrent paths the
scanner found during a fully completed discovery pass. Episodes and
seasons left without files are dropped; items left without any files
are removed entirely.
"""
for item_id, movie in list(self.movies.items()):
kept = {k: v for k, v in movie.files.items() if k in paths}
if len(kept) == len(movie.files):
continue
if not kept:
self.remove_movie(item_id)
continue
updated = msgspec.structs.replace(
movie, files=kept, newest=self._newest_from_files(kept)
)
self.movies[item_id] = updated
self._schedule_snapshot()
self._broadcast(Upsert(kind="movie", id=item_id, item=updated))
for item_id, series in list(self.series.items()):
removed_any = False
new_seasons: list[Season] = []
for season in series.seasons:
new_episodes: list[Episode] = []
for ep in season.episodes:
files = {k: v for k, v in ep.files.items() if k in paths}
if len(files) < len(ep.files):
removed_any = True
if files:
new_episodes.append(msgspec.structs.replace(ep, files=files))
else:
removed_any = True
if not new_episodes:
removed_any = True
continue
if len(new_episodes) < len(season.episodes):
season = msgspec.structs.replace(
season,
episodes=new_episodes,
episode_count=len(new_episodes),
)
new_seasons.append(season)
if not removed_any:
continue
if not new_seasons:
self.remove_series(item_id)
continue
updated_series = msgspec.structs.replace(series, seasons=new_seasons)
self.series[item_id] = updated_series
self._schedule_snapshot()
self._broadcast(Upsert(kind="series", id=item_id, item=updated_series))
def set_movie_showreel(
self,
item_id: str,
showreel_images: list[str] | None,
showreel_source_sets: list[list[str]] | None,
) -> None:
"""Update only the showreel fields of a movie (reel worker callback)."""
movie = self.movies.get(item_id)
if movie is None:
return
updated = msgspec.structs.replace(
movie,
showreel_images=showreel_images,
showreel_source_sets=showreel_source_sets,
)
if msgspec.json.encode(updated) == msgspec.json.encode(movie):
return
self.movies[item_id] = updated
self._schedule_snapshot()
self._broadcast(Upsert(kind="movie", id=item_id, item=updated))
def set_episode_reel(
self,
item_id: str,
season_num: int,
episode_num: int,
reel_image: str | None,
reel_sources: list[str] | None,
) -> None:
"""Update only the reel fields of one episode (reel worker callback)."""
series = self.series.get(item_id)
if series is None:
return
for season in series.seasons:
if season.season_number != season_num:
continue
for ep in season.episodes:
if ep.episode_number != episode_num:
continue
if ep.reel_image == reel_image and ep.reel_sources == reel_sources:
return
new_episodes = [
msgspec.structs.replace(
e, reel_image=reel_image, reel_sources=reel_sources
)
if e is ep
else e
for e in season.episodes
]
new_seasons = [
msgspec.structs.replace(s, episodes=new_episodes)
if s is season
else s
for s in series.seasons
]
updated = msgspec.structs.replace(series, seasons=new_seasons)
self.series[item_id] = updated
self._schedule_snapshot()
self._broadcast(Upsert(kind="series", id=item_id, item=updated))
return
# ------------------------------------------------------------------
# WebSocket management
# ------------------------------------------------------------------
+39 -2
View File
@@ -14,12 +14,20 @@ from .tmdb import Person
class Upsert(msgspec.Struct, tag="upsert"):
"""Single item inserted or updated."""
"""Single item inserted or updated.
``scanned`` lists the media-root-relative torrent paths whose content was
(re)scanned to build this item. When present, the store merges the item
into the existing entry instead of replacing it wholesale: only data
belonging to the scanned torrents is replaced. ``None`` means full
replacement (legacy behaviour).
"""
kind: str # "movie" or "series"
id: str
item: Movie | Series
people: dict[int, Person] | None = None
scanned: list[str] | None = None
class Remove(msgspec.Struct, tag="remove"):
@@ -29,6 +37,35 @@ class Remove(msgspec.Struct, tag="remove"):
id: str
class Sync(msgspec.Struct, tag="sync"):
"""Full set of media-root-relative torrent paths currently on disk.
Sent by the scanner after a successfully completed discovery pass so the
store can drop entries whose files no longer exist. Internal only
never forwarded to WebSocket clients.
"""
paths: list[str]
class MovieShowreel(msgspec.Struct, tag="movie-showreel"):
"""Reel worker result for a movie (internal, scanner → store)."""
id: str
showreel_images: list[str] | None = None
showreel_source_sets: list[list[str]] | None = None
class EpisodeReel(msgspec.Struct, tag="episode-reel"):
"""Reel worker result for one episode (internal, scanner → store)."""
id: str
season: int
episode: int
reel_image: str | None = None
reel_sources: list[str] | None = None
class Task(msgspec.Struct, tag="task"):
"""Task progress broadcast."""
@@ -36,4 +73,4 @@ class Task(msgspec.Struct, tag="task"):
# Union of scan events (scanner → server) and WS broadcast messages
ScanEvent = Upsert | Task
ScanEvent = Upsert | Sync | MovieShowreel | EpisodeReel | Task
+34 -5
View File
@@ -11,7 +11,14 @@ import msgspec
from mediahive.config import load_config, save_config
from mediahive.index_store import IndexStore
from mediahive.models.events import ScanEvent, Task, Upsert
from mediahive.models.events import (
EpisodeReel,
MovieShowreel,
ScanEvent,
Sync,
Task,
Upsert,
)
logger = logging.getLogger("mediahive.root_registry")
@@ -158,9 +165,27 @@ class RootContext:
event = await self._events.get()
if isinstance(event, Upsert):
if event.kind == "movie":
self.store.upsert_movie(event.id, event.item, event.people)
self.store.upsert_movie(
event.id, event.item, event.people, event.scanned
)
else:
self.store.upsert_series(event.id, event.item, event.people)
self.store.upsert_series(
event.id, event.item, event.people, event.scanned
)
elif isinstance(event, Sync):
self.store.sync_torrent_paths(set(event.paths))
elif isinstance(event, MovieShowreel):
self.store.set_movie_showreel(
event.id, event.showreel_images, event.showreel_source_sets
)
elif isinstance(event, EpisodeReel):
self.store.set_episode_reel(
event.id,
event.season,
event.episode,
event.reel_image,
event.reel_sources,
)
elif isinstance(event, Task):
self.store.broadcast_task(event.data)
except asyncio.CancelledError:
@@ -328,6 +353,10 @@ class Supervisor:
async def shutdown(self) -> None:
async with self._lock:
for ctx in list(self._contexts.values()):
await ctx.stop()
# Stop roots concurrently — each may wait on task cancellation and
# network-mount snapshot flushes, and those delays must not add up.
await asyncio.gather(
*(ctx.stop() for ctx in list(self._contexts.values())),
return_exceptions=True,
)
self._contexts.clear()
+456 -80
View File
@@ -34,12 +34,6 @@ from fastapi.responses import FileResponse, Response, StreamingResponse
from fastapi_vue import Frontend
from mediahive.__main__ import DEVMODE
from mediahive.access_logging import (
AccessLogMiddleware,
configure_access_logging,
log_ws_close,
log_ws_open,
)
from mediahive.config import load_config
from mediahive.hivescan.images import close_image_client
from mediahive.hivescan.scanner import RootScanner
@@ -63,8 +57,6 @@ from mediahive.root_registry import Supervisor
logger = logging.getLogger("mediahive.server")
configure_access_logging()
MPC_BE_DEFAULT_PORT = 13579
# Suppress console windows when spawning subprocesses on Windows
@@ -89,16 +81,62 @@ if sys.platform == "win32":
@dataclass
class _PlaybackEntry:
"""Single resume position entry with timestamp."""
class _EpisodeWatch:
"""Per-episode watch progress within a series entry."""
pos: int
ts: datetime
done: bool = False
def to_dict(self) -> dict:
return {"pos": self.pos, "ts": self.ts, "done": self.done}
@staticmethod
def from_dict(data: dict) -> _EpisodeWatch | None:
if not isinstance(data, dict):
return None
pos = data.get("pos")
ts = data.get("ts")
if not isinstance(pos, int) or pos < 0:
return None
if isinstance(ts, str):
try:
ts = datetime.fromisoformat(ts)
except ValueError, TypeError:
return None
elif not isinstance(ts, datetime):
return None
return _EpisodeWatch(pos=pos, ts=ts, done=bool(data.get("done")))
def _episode_watch_key(season: int, episode: int) -> str:
return f"S{season}E{episode}"
@dataclass
class _PlaybackEntry:
"""Resume entry for one media slug.
season/episode form the series' single continue point (last watched
episode), None for movies. episodes holds per-episode watch progress
for series, keyed "S<season>E<episode>".
"""
pos: int
ts: datetime
season: int | None = None
episode: int | None = None
episodes: dict[str, _EpisodeWatch] = field(default_factory=dict)
def to_dict(self) -> dict:
return {
"pos": self.pos,
"ts": self.ts,
"season": self.season,
"episode": self.episode,
"episodes": {
key: watch.to_dict() for key, watch in sorted(self.episodes.items())
},
}
@staticmethod
@@ -118,7 +156,22 @@ class _PlaybackEntry:
pass
else:
return None
return _PlaybackEntry(pos=pos, ts=ts)
season = data.get("season")
episode = data.get("episode")
episodes: dict[str, _EpisodeWatch] = {}
raw_episodes = data.get("episodes")
if isinstance(raw_episodes, dict):
for key, watch_data in raw_episodes.items():
watch = _EpisodeWatch.from_dict(watch_data)
if watch is not None:
episodes[str(key)] = watch
return _PlaybackEntry(
pos=pos,
ts=ts,
season=season if isinstance(season, int) else None,
episode=episode if isinstance(episode, int) else None,
episodes=episodes,
)
@dataclass
@@ -131,7 +184,8 @@ class _PlaybackRootSnapshot:
class PlaybackStateCache:
"""Background cache for merged playback-state across all active roots.
Stores resume positions by movie slug with timestamps. When merging
Stores resume positions by media slug (movie id, or series id with a
season/episode continue point) with timestamps. When merging
across roots, picks the most recent entry for each slug.
"""
@@ -164,7 +218,7 @@ class PlaybackStateCache:
"""Return merged resume entries keyed by slug (most recent wins)."""
with self._lock:
return {
slug: _PlaybackEntry(e.pos, e.ts)
slug: _PlaybackEntry(e.pos, e.ts, e.season, e.episode, dict(e.episodes))
for slug, e in self._merged_entries.items()
}
@@ -174,15 +228,50 @@ class PlaybackStateCache:
root_path: Path,
slug: str,
pos: int | None,
season: int | None = None,
episode: int | None = None,
*,
done_episode: tuple[int, int] | None = None,
) -> None:
"""Read-modify-write one root file and refresh the in-memory cache immediately."""
"""Read-modify-write one root file and refresh the in-memory cache immediately.
For series entries, updates both the series continue point (last
watched episode) and the per-episode watch map. done_episode marks a
completed episode as fully watched without touching the continue
point position semantics (used together with advancing the point).
"""
file_path = root_path / ".mediahive" / "playback-state.json"
entries = self._read_resume_entries(file_path)
if pos is None:
if pos is None and done_episode is None:
entries.pop(slug, None)
else:
entries[slug] = _PlaybackEntry(pos=pos, ts=datetime.now())
now = datetime.now()
entry = entries.get(slug)
if entry is None:
entry = _PlaybackEntry(pos=pos or 0, ts=now)
entries[slug] = entry
if pos is not None:
entry.pos = pos
entry.ts = now
entry.season = season
entry.episode = episode
if season is not None and episode is not None and pos > 0:
entry.episodes[_episode_watch_key(season, episode)] = _EpisodeWatch(
pos=pos, ts=now
)
elif done_episode is not None:
# Final episode completed: no continue point remains, but the
# per-episode watch history is kept for indicators.
entry.pos = 0
entry.ts = now
entry.season = None
entry.episode = None
if done_episode is not None:
done_season, done_ep = done_episode
entry.episodes[_episode_watch_key(done_season, done_ep)] = (
_EpisodeWatch(pos=0, ts=now, done=True)
)
self._write_resume_entries(file_path, entries)
@@ -211,7 +300,6 @@ class PlaybackStateCache:
previous = self._roots
next_roots: dict[str, _PlaybackRootSnapshot] = {}
merged: dict[str, _PlaybackEntry] = {}
for root_id, ctx in contexts.items():
file_path = ctx.root_path / ".mediahive" / "playback-state.json"
@@ -226,15 +314,9 @@ class PlaybackStateCache:
next_roots[root_id] = snapshot
# Merge: for each slug, keep the entry with the most recent timestamp
for slug, entry in snapshot.entries.items():
existing = merged.get(slug)
if existing is None or entry.ts > existing.ts:
merged[slug] = entry
with self._lock:
self._roots = next_roots
self._merged_entries = merged
self._merged_entries = self._build_merged_entries(next_roots)
@staticmethod
def _signature(path: Path) -> tuple[bool, int, int]:
@@ -288,12 +370,33 @@ class PlaybackStateCache:
def _build_merged_entries(
roots: dict[str, _PlaybackRootSnapshot],
) -> dict[str, _PlaybackEntry]:
"""Merge resume entries across roots.
Newest continue point per slug, and newest watch state per episode
key within each slug.
"""
merged: dict[str, _PlaybackEntry] = {}
for snapshot in roots.values():
for slug, entry in snapshot.entries.items():
existing = merged.get(slug)
if existing is None or entry.ts > existing.ts:
merged[slug] = entry
if existing is None:
merged[slug] = _PlaybackEntry(
entry.pos,
entry.ts,
entry.season,
entry.episode,
dict(entry.episodes),
)
continue
if entry.ts > existing.ts:
existing.pos = entry.pos
existing.ts = entry.ts
existing.season = entry.season
existing.episode = entry.episode
for key, watch in entry.episodes.items():
current = existing.episodes.get(key)
if current is None or watch.ts > current.ts:
existing.episodes[key] = watch
return merged
@@ -378,21 +481,195 @@ def _expand_torrent_playable_path(file_key: str, playable_file: str | None) -> s
return f"{file_key}/{playable_file}"
def _resolve_movie_slug_for_file_path(ctx, file_path: str) -> str | None:
def _resolve_media_ref_for_file_path(
ctx, file_path: str
) -> tuple[str, int | None, int | None] | None:
"""Resolve a playable file path to (slug, season_number, episode_number).
Movies return (movie_id, None, None); series episode files return
(series_id, season_number, episode_number).
"""
target = _normalize_media_path_value(file_path)
for movie_id, movie in ctx.store.movies.items():
for file_key, torrent in movie.files.items():
normalized_key = _normalize_media_path_value(file_key)
if normalized_key == target:
return movie_id
return movie_id, None, None
playable_path = _expand_torrent_playable_path(
file_key, torrent.playable_file
)
if _normalize_media_path_value(playable_path) == target:
return movie_id
return movie_id, None, None
for series_id, show in ctx.store.series.items():
for season in show.seasons:
for episode in season.episodes:
for file_key, torrent in episode.files.items():
normalized_key = _normalize_media_path_value(file_key)
if normalized_key == target:
return series_id, season.season_number, episode.episode_number
playable_path = _expand_torrent_playable_path(
file_key, torrent.playable_file
)
if _normalize_media_path_value(playable_path) == target:
return series_id, season.season_number, episode.episode_number
return None
def _next_episode_ref(
show, season_number: int, episode_number: int
) -> tuple[int, int] | None:
"""Return the (season_number, episode_number) following the given episode."""
for season_index, season in enumerate(show.seasons):
if season.season_number != season_number:
continue
for episode_index, episode in enumerate(season.episodes):
if episode.episode_number != episode_number:
continue
if episode_index + 1 < len(season.episodes):
return season.season_number, season.episodes[
episode_index + 1
].episode_number
if season_index + 1 < len(show.seasons):
next_season = show.seasons[season_index + 1]
if next_season.episodes:
return next_season.season_number, next_season.episodes[
0
].episode_number
return None
return None
# --- Assumed playback tracking (player-agnostic fallback) ---
#
# Launching an external player returns immediately and no player API is
# guaranteed, so for arbitrary players we cannot observe real progress.
# Instead: when the user launches an item and the frontend then sees no
# input activity, the item is assumed to be playing. The resume position is
# written once, when frontend activity resumes (i.e. the user came back).
# The MPC-BE tracker in the GUI overrides this: if a newer entry for the
# slug was written while the session ran, the guess is discarded.
ASSUMED_PLAYBACK_MIN_WATCH_S = 300
@dataclass
class _AssumedPlaybackSession:
root_id: str
slug: str
season: int | None
episode: int | None
base_pos_s: int
started_mono: float
started_wall: datetime
duration_s: int | None
_assumed_playback: _AssumedPlaybackSession | None = None
def _media_duration_seconds(
ctx, slug: str, season_number: int | None, episode_number: int | None
) -> int | None:
"""Best-known runtime in seconds (TMDb, minutes) for a media ref."""
minutes: int | None = None
if season_number is None:
movie = ctx.store.movies.get(slug)
if movie is not None and movie.info is not None:
minutes = movie.info.runtime
else:
show = ctx.store.series.get(slug)
if show is not None:
for season in show.seasons:
if season.season_number != season_number:
continue
for episode in season.episodes:
if episode.episode_number == episode_number:
minutes = episode.runtime
break
break
return minutes * 60 if minutes else None
def _start_assumed_playback(ctx, root_id: str, file_path: str) -> None:
"""Begin a guessed-watch session for a freshly launched file."""
global _assumed_playback
# Time between two launches counts as watching the previous item.
_finalize_assumed_playback()
ref = _resolve_media_ref_for_file_path(ctx, file_path)
if ref is None:
return
slug, season_number, episode_number = ref
entry = playback_state_cache.get_merged_entries().get(slug)
base_pos_s = 0
if entry is not None:
if season_number is None or (entry.season, entry.episode) == (
season_number,
episode_number,
):
base_pos_s = entry.pos
_assumed_playback = _AssumedPlaybackSession(
root_id=root_id,
slug=slug,
season=season_number,
episode=episode_number,
base_pos_s=base_pos_s,
started_mono=time.monotonic(),
started_wall=datetime.now(),
duration_s=_media_duration_seconds(ctx, slug, season_number, episode_number),
)
def _finalize_assumed_playback() -> bool:
"""Close the guessed-watch session, writing the assumed position.
Returns True when a resume position was actually written.
"""
global _assumed_playback
session = _assumed_playback
_assumed_playback = None
if session is None:
return False
elapsed_s = int(time.monotonic() - session.started_mono)
if elapsed_s < ASSUMED_PLAYBACK_MIN_WATCH_S:
return False
pos_s = session.base_pos_s + elapsed_s
if session.duration_s:
pos_s = min(pos_s, session.duration_s)
if pos_s <= 0:
return False
# A newer entry written while this session ran (e.g. the GUI's real
# MPC-BE tracker finalizing on player close) overrides the guess.
current = playback_state_cache.get_merged_entries().get(session.slug)
if current is not None and current.ts > session.started_wall:
return False
ctx = supervisor.get(session.root_id)
if ctx is None:
return False
playback_state_cache.update_resume_position(
session.root_id,
ctx.root_path,
session.slug,
pos_s,
session.season,
session.episode,
)
logger.info(
"Assumed playback: %s S%sE%s +%ds -> pos %ds",
session.slug,
session.season,
session.episode,
elapsed_s,
pos_s,
)
return True
def _load_root_metadata(root_path: Path, meta_key: str):
"""Load allowed per-root metadata values from .mediahive."""
key = meta_key.strip().lower().strip("/")
@@ -660,7 +937,12 @@ async def _attach_scanners() -> None:
for ctx in supervisor.all_contexts().values():
if ctx.scanner is None and ctx.status == "ready":
try:
scanner = RootScanner(ctx.root_id, ctx.root_path, ctx.send_event)
scanner = RootScanner(
ctx.root_id,
ctx.root_path,
ctx.send_event,
ctx.store.torrent_paths,
)
await scanner.start()
ctx.scanner = scanner
except Exception:
@@ -748,6 +1030,7 @@ async def lifespan(_app: FastAPI):
try:
yield
finally:
_finalize_assumed_playback()
playback_state_cache.stop()
await event_loop_lag_monitor.stop()
@@ -769,9 +1052,6 @@ async def lifespan(_app: FastAPI):
app = FastAPI(title="MediaHive Server", lifespan=lifespan, debug=DEVMODE)
# Custom access logging (uvicorn access logs are suppressed in access_logging)
app.add_middleware(AccessLogMiddleware)
# Allow CORS for development
app.add_middleware(
CORSMiddleware,
@@ -829,9 +1109,6 @@ async def ws_endpoint(ws: WebSocket) -> None:
attached_contexts = supervisor.all_contexts()
outbound: asyncio.Queue[bytes] = asyncio.Queue()
start = time.perf_counter()
ws_id = log_ws_open(ws)
close_code: int | None = None
prev_root_ids: set[str] = set()
prev_meta: dict[str, tuple[str, str, str | None, bool]] = {}
@@ -918,9 +1195,7 @@ async def ws_endpoint(ws: WebSocket) -> None:
try:
while True:
await ws.receive_text()
except WebSocketDisconnect as exc:
close_code = exc.code
except OSError, RuntimeError:
except WebSocketDisconnect, OSError, RuntimeError:
pass
finally:
sender_task.cancel()
@@ -935,8 +1210,6 @@ async def ws_endpoint(ws: WebSocket) -> None:
if ctx is not None:
ctx.store.remove_listener(listener)
log_ws_close(ws_id, close_code, time.perf_counter() - start)
# --- Media actions ---
@@ -999,6 +1272,10 @@ async def play_media(root_id: str, request: Request, response: Response):
launch_ms = (time.perf_counter() - launch_t0) * 1000.0
total_ms = (time.perf_counter() - req_start) * 1000.0
# Player-agnostic fallback: assume the launched item is being watched
# until frontend activity resumes (real MPC-BE tracking overrides).
_start_assumed_playback(ctx, root_id, req.file_path)
loop_lag_ms, loop_lag_max_ms = event_loop_lag_monitor.snapshot()
if trace_id:
@@ -1110,6 +1387,16 @@ async def root_metadata(root_id: str, meta_key: str):
return {"key": meta_key, "data": _load_root_metadata(ctx.root_path, meta_key)}
@app.post("/api/activity")
async def report_activity():
"""Report user input activity in the frontend.
Ends any assumed-playback session: activity means the user is back at
the UI, so the launched item's guessed watch time is written out.
"""
return {"status": "ok", "finalized": _finalize_assumed_playback()}
@app.get("/api/meta/playback-state")
async def merged_playback_state():
"""Return merged playback-state resume positions from in-memory cache.
@@ -1124,18 +1411,66 @@ async def merged_playback_state():
@app.post("/api/meta/playback-state")
async def write_playback_state(request: Request):
"""Update one playback-state entry via backend-managed read-modify-write."""
"""Update one playback-state entry via backend-managed read-modify-write.
A series episode played to completion (pos null) is marked fully watched
in the per-episode watch map and advances the series' single continue
point to the next episode (pos 0); finishing the final episode clears
the continue point but keeps the watch history.
"""
req = msgspec.json.decode(await request.body(), type=PlaybackStateUpdateRequest)
ctx = _get_context(req.root_id)
slug = _resolve_movie_slug_for_file_path(ctx, req.file_path)
if slug is None:
ref = _resolve_media_ref_for_file_path(ctx, req.file_path)
if ref is None:
raise HTTPException(
status_code=404,
detail=f"Movie not found for file path: {req.file_path}",
detail=f"Media not found for file path: {req.file_path}",
)
pos = None if req.pos is None or req.pos <= 0 else int(req.pos)
playback_state_cache.update_resume_position(req.root_id, ctx.root_path, slug, pos)
slug, season_number, episode_number = ref
if req.pos is None or req.pos <= 0:
if season_number is None:
playback_state_cache.update_resume_position(
req.root_id, ctx.root_path, slug, None
)
return {"status": "ok", "slug": slug, "pos": None}
show = ctx.store.series.get(slug)
next_ref = (
_next_episode_ref(show, season_number, episode_number)
if show is not None
else None
)
done = (season_number, episode_number)
if next_ref is None:
playback_state_cache.update_resume_position(
req.root_id, ctx.root_path, slug, None, done_episode=done
)
return {"status": "ok", "slug": slug, "pos": None}
next_season, next_episode = next_ref
playback_state_cache.update_resume_position(
req.root_id,
ctx.root_path,
slug,
0,
next_season,
next_episode,
done_episode=done,
)
return {
"status": "ok",
"slug": slug,
"pos": 0,
"season": next_season,
"episode": next_episode,
}
pos = int(req.pos)
playback_state_cache.update_resume_position(
req.root_id, ctx.root_path, slug, pos, season_number, episode_number
)
return {"status": "ok", "slug": slug, "pos": pos}
@@ -1186,6 +1521,74 @@ def _resolve_root_scoped_path(base: Path, raw_path: str) -> Path:
return candidate
class StreamingFileResponse(StreamingResponse):
"""Stream a file from disk with optional single-range support.
Unlike plain ``StreamingResponse`` with a generator, the file handle is
held in ``stream_response`` scope across the send loop (the same pattern
as Starlette's ``FileResponse``), so it is always closed promptly and in
flow including on client disconnect, where a generator's cleanup would
be deferred to GC and its exceptions lost.
"""
chunk_size = 64 * 1024
def __init__(
self,
path: Path,
*,
file_size: int,
etag: str,
cache_control: str,
media_type: str,
range_header: str | None = None,
) -> None:
if range_header:
self._start, self._end = _parse_range_header(range_header, file_size)
status_code = 206
else:
self._start, self._end = 0, file_size - 1
status_code = 200
headers = {
"Cache-Control": cache_control,
"ETag": etag,
"Accept-Ranges": "bytes",
"Content-Length": str(self._end - self._start + 1),
}
if status_code == 206:
headers["Content-Range"] = f"bytes {self._start}-{self._end}/{file_size}"
super().__init__( # body_iterator is unused; stream_response is overridden
content=(),
status_code=status_code,
headers=headers,
media_type=media_type,
)
self.path = path
async def stream_response(self, send) -> None:
await send({
"type": "http.response.start",
"status": self.status_code,
"headers": self.raw_headers,
})
async with aiofiles.open(self.path, "rb") as f:
await f.seek(self._start)
remaining = self._end - self._start + 1
while remaining > 0:
chunk = await f.read(min(self.chunk_size, remaining))
if not chunk:
break
remaining -= len(chunk)
await send({
"type": "http.response.body",
"body": chunk,
"more_body": True,
})
await send({"type": "http.response.body", "body": b"", "more_body": False})
def _serve_file_response(full_path: Path, file_path: str, request: Request):
"""Serve a file with range + cache support."""
if not full_path.exists():
@@ -1219,40 +1622,13 @@ def _serve_file_response(full_path: Path, file_path: str, request: Request):
headers={"Cache-Control": cache_control, "ETag": etag},
)
async def stream_file(start: int, end: int):
async with aiofiles.open(full_path, "rb") as f:
await f.seek(start)
remaining = end - start + 1
while remaining > 0:
chunk = await f.read(min(64 * 1024, remaining))
if not chunk:
break
remaining -= len(chunk)
yield chunk
headers = {
"Cache-Control": cache_control,
"ETag": etag,
"Accept-Ranges": "bytes",
}
if range_header:
start, end = _parse_range_header(range_header, file_size)
headers["Content-Range"] = f"bytes {start}-{end}/{file_size}"
headers["Content-Length"] = str(end - start + 1)
return StreamingResponse(
stream_file(start, end),
status_code=206,
media_type=content_type,
headers=headers,
)
headers["Content-Length"] = str(file_size)
return StreamingResponse(
stream_file(0, file_size - 1),
return StreamingFileResponse(
full_path,
file_size=file_size,
etag=etag,
cache_control=cache_control,
media_type=content_type,
headers=headers,
range_header=range_header,
)
+312 -53
View File
@@ -9,6 +9,7 @@ import asyncio
import contextlib
import ctypes
import html
import importlib.metadata
import json
import logging
import os
@@ -53,6 +54,9 @@ MPC_BE_STATE_RUNNING = 2
MPC_BE_SEEK_BEGIN_COMMAND = 1085
MPC_BE_RESUME_APPLY_THRESHOLD_MS = 15000
MPC_BE_RESUME_CLEAR_MARGIN_MS = 15000
# Watching (or presumably watching) less than this leaves no position data:
# brief peeks and seeks back to re-view a scene are not true progress.
MPC_BE_RESUME_MIN_WATCH_MS = 5 * 60 * 1000
MPC_BE_PLAYBACK_STATE_FLUSH_SECONDS = 1.0
VOLUME_MIN = 0.0
VOLUME_MAX = 1.5
@@ -139,7 +143,20 @@ def _expand_playable_file(file_key: str, playable_file: str | None) -> str:
return f"{file_key}/{playable_file}"
def _fetch_resume_positions(backend_url: str) -> dict[str, int]:
def _fetch_resume_positions(
backend_url: str,
) -> tuple[
dict[str, tuple[int, int | None, int | None]],
dict[tuple[str, int, int], int],
]:
"""Fetch resume state from the backend.
Returns (continue_points, episode_positions): continue_points map a slug
to (pos_ms, season_number, episode_number) season/episode set for the
series' single continue point, None for movies. episode_positions map
(slug, season, episode) to pos_ms for partially watched episodes;
fully watched episodes are absent.
"""
req = urllib.request.Request(
url=f"{backend_url}/api/meta/playback-state",
method="GET",
@@ -148,21 +165,45 @@ def _fetch_resume_positions(backend_url: str) -> dict[str, int]:
with urllib.request.urlopen(req, timeout=2) as resp:
raw = json.loads(resp.read().decode("utf-8"))
except OSError, TimeoutError, urllib.error.URLError, json.JSONDecodeError:
return {}
return {}, {}
data = raw.get("data") if isinstance(raw, dict) else None
positions = data.get("resume_positions") if isinstance(data, dict) else None
if not isinstance(positions, dict):
return {}
return {}, {}
cleaned: dict[str, int] = {}
cleaned: dict[str, tuple[int, int | None, int | None]] = {}
episode_positions: dict[tuple[str, int, int], int] = {}
for slug, value in positions.items():
if not isinstance(slug, str) or not isinstance(value, dict):
continue
pos = value.get("pos")
season = value.get("season")
episode = value.get("episode")
if isinstance(pos, int) and pos > 0:
cleaned[slug] = pos * 1000
return cleaned
cleaned[slug] = (
pos * 1000,
season if isinstance(season, int) else None,
episode if isinstance(episode, int) else None,
)
elif pos == 0 and isinstance(season, int) and isinstance(episode, int):
# Series episode boundary marker (previous episode completed).
cleaned[slug] = (0, season, episode)
episodes = value.get("episodes")
if not isinstance(episodes, dict):
continue
for key, watch in episodes.items():
match = re.fullmatch(r"S(\d+)E(\d+)", str(key))
if not match or not isinstance(watch, dict):
continue
ep_pos = watch.get("pos")
if watch.get("done") or not isinstance(ep_pos, int) or ep_pos <= 0:
continue
episode_positions[slug, int(match.group(1)), int(match.group(2))] = (
ep_pos * 1000
)
return cleaned, episode_positions
def _post_resume_position(
@@ -190,51 +231,105 @@ def _post_resume_position(
return False
def _load_movie_slug_map(index_path: Path) -> dict[str, str]:
def _media_file_key(
mapping: dict[str, str], file_key: str, torrent: object, media_key: str
) -> None:
"""Map both the raw file key and its expanded playable path to a media key."""
mapping[_normalize_media_path(file_key)] = media_key
playable_file = torrent.get("playable_file") if isinstance(torrent, dict) else None
expanded = _expand_playable_file(
file_key, playable_file if isinstance(playable_file, str) else None
)
mapping[_normalize_media_path(expanded)] = media_key
def _split_media_key(media_key: str) -> tuple[str, int | None, int | None]:
"""Split a media key into (slug, season_number, episode_number)."""
slug, separator, ep_ref = media_key.partition("#")
if not separator:
return slug, None, None
match = re.fullmatch(r"S(\d+)E(\d+)", ep_ref)
if not match:
return slug, None, None
return slug, int(match.group(1)), int(match.group(2))
def _load_media_key_map(index_path: Path) -> dict[str, str]:
"""Map normalized playable file paths to media keys.
Movies map to their movie id; series episodes map to
"<series_id>#S<season>E<episode>" so episode switches are detected while
the backend keeps a single continue point per series.
"""
try:
raw = json.loads(index_path.read_text(encoding="utf-8"))
except OSError, TypeError, json.JSONDecodeError:
return {}
movies = raw.get("movies") if isinstance(raw, dict) else None
if not isinstance(movies, dict):
if not isinstance(raw, dict):
return {}
mapping: dict[str, str] = {}
for movie_id, movie in movies.items():
if not isinstance(movie_id, str) or not isinstance(movie, dict):
continue
files = movie.get("files")
if not isinstance(files, dict):
continue
for file_key, torrent in files.items():
if not isinstance(file_key, str):
movies = raw.get("movies")
if isinstance(movies, dict):
for movie_id, movie in movies.items():
if not isinstance(movie_id, str) or not isinstance(movie, dict):
continue
normalized_key = _normalize_media_path(file_key)
mapping[normalized_key] = movie_id
playable_file = (
torrent.get("playable_file") if isinstance(torrent, dict) else None
)
expanded = _expand_playable_file(
file_key, playable_file if isinstance(playable_file, str) else None
)
mapping[_normalize_media_path(expanded)] = movie_id
files = movie.get("files")
if not isinstance(files, dict):
continue
for file_key, torrent in files.items():
if not isinstance(file_key, str):
continue
_media_file_key(mapping, file_key, torrent, movie_id)
series = raw.get("series")
if isinstance(series, dict):
for series_id, show in series.items():
if not isinstance(series_id, str) or not isinstance(show, dict):
continue
seasons = show.get("seasons")
if not isinstance(seasons, list):
continue
for season in seasons:
if not isinstance(season, dict):
continue
season_number = season.get("season_number")
episodes = season.get("episodes")
if not isinstance(season_number, int) or not isinstance(episodes, list):
continue
for episode in episodes:
if not isinstance(episode, dict):
continue
episode_number = episode.get("episode_number")
files = episode.get("files")
if not isinstance(episode_number, int) or not isinstance(
files, dict
):
continue
media_key = f"{series_id}#S{season_number}E{episode_number}"
for file_key, torrent in files.items():
if not isinstance(file_key, str):
continue
_media_file_key(mapping, file_key, torrent, media_key)
return mapping
def _media_key_for_filepath(
filepath: str, roots: dict[str, Path]
) -> tuple[str | None, str, str] | None:
"""Resolve a filepath to a (movie_slug, root_id, relative_key) tuple."""
"""Resolve a filepath to a (media_key, root_id, relative_key) tuple."""
for root_id, root in roots.items():
try:
relative = Path(filepath).resolve().relative_to(root.resolve())
relative_key = relative.as_posix()
index_path = root / ".mediahive" / "index.json"
movie_slug = _load_movie_slug_map(index_path).get(
media_key = _load_media_key_map(index_path).get(
_normalize_media_path(relative_key)
)
return movie_slug, root_id, relative_key
return media_key, root_id, relative_key
except OSError, RuntimeError, ValueError:
continue
return None
@@ -347,7 +442,7 @@ def _start_gamepad_remote(
status_miss_count = 0
playback_state = _default_playback_state()
resume_positions = _fetch_resume_positions(backend_url)
resume_positions, episode_positions = _fetch_resume_positions(backend_url)
tracked_media_key: str | None = None
tracked_root_id: str | None = None
tracked_relative_path = ""
@@ -399,18 +494,34 @@ def _start_gamepad_remote(
position_ms = player_position_ms or 0
duration_ms = player_duration_ms or 0
tracked_slug, tracked_season, tracked_episode = _split_media_key(
tracked_media_key
)
if _should_clear_resume(position_ms, duration_ms):
resume_positions.pop(tracked_media_key, None)
resume_positions.pop(tracked_slug, None)
if tracked_season is not None and tracked_episode is not None:
episode_positions.pop(
(tracked_slug, tracked_season, tracked_episode), None
)
if tracked_root_id and tracked_relative_path:
_post_resume_position(
backend_url, tracked_root_id, tracked_relative_path, None
)
elif position_ms <= MPC_BE_RESUME_CLEAR_MARGIN_MS:
# Ignore brief starts; keep the previous saved resume position.
elif position_ms < MPC_BE_RESUME_MIN_WATCH_MS:
# Peeks and brief seeks are not true progress; keep the previous
# saved resume position.
pass
else:
position_seconds = max(0, position_ms // 1000)
resume_positions[tracked_media_key] = position_ms
resume_positions[tracked_slug] = (
position_ms,
tracked_season,
tracked_episode,
)
if tracked_season is not None and tracked_episode is not None:
episode_positions[tracked_slug, tracked_season, tracked_episode] = (
position_ms
)
if tracked_root_id and tracked_relative_path:
_post_resume_position(
backend_url,
@@ -453,8 +564,36 @@ def _start_gamepad_remote(
if resume_applied_for_key == tracked_media_key:
return
saved_position = resume_positions.get(tracked_media_key)
if not isinstance(saved_position, int):
tracked_slug, tracked_season, tracked_episode = _split_media_key(
tracked_media_key
)
if tracked_season is not None and tracked_episode is not None:
# Series: the episode's own saved position wins; fall back to the
# series continue point when it points at this very episode.
saved_position = episode_positions.get((
tracked_slug,
tracked_season,
tracked_episode,
))
if saved_position is None:
saved = resume_positions.get(tracked_slug)
if saved is None or (saved[1], saved[2]) != (
tracked_season,
tracked_episode,
):
# The series continue point belongs to a different episode.
resume_applied_for_key = tracked_media_key
return
saved_position = saved[0]
else:
saved = resume_positions.get(tracked_slug)
if saved is None:
resume_applied_for_key = tracked_media_key
return
saved_position = saved[0]
if saved_position <= 0:
# Episode boundary marker (previous episode completed): start at 0.
resume_applied_for_key = tracked_media_key
return
if player_position_ms is None or player_duration_ms is None:
@@ -463,7 +602,11 @@ def _start_gamepad_remote(
resume_applied_for_key = tracked_media_key
return
if _should_clear_resume(saved_position, player_duration_ms):
resume_positions.pop(tracked_media_key, None)
resume_positions.pop(tracked_slug, None)
if tracked_season is not None and tracked_episode is not None:
episode_positions.pop(
(tracked_slug, tracked_season, tracked_episode), None
)
resume_applied_for_key = tracked_media_key
return
if len(pending_requests) >= MPC_BE_MAX_INFLIGHT_REQUESTS:
@@ -687,6 +830,46 @@ def _start_gamepad_remote(
return thread
def _rotate_and_open_log(log_path: Path):
"""Rotate mediahive.log to .log.1 and open a fresh log file.
Raises OSError when a previous MediaHive instance still holds the file
open (Windows forbids renaming a file that is open without delete
sharing) callers treat that as "previous instance not dead yet".
"""
prev = log_path.with_suffix(".log.1")
if log_path.exists():
if prev.exists():
prev.unlink()
log_path.rename(prev)
fd = os.open(log_path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC)
return os.fdopen(fd, "w", encoding="utf-8", buffering=1) # line-buffered
def _wait_for_previous_instance(log_path: Path, timeout: float = 15.0):
"""Show a waiting notice while a previous MediaHive instance exits.
Returns an open log file handle, or None on timeout.
"""
result: list = []
window = webview.create_window(
"MediaHive", html=_WAIT_HTML, width=520, height=280, resizable=False
)
def poll() -> None:
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
try:
result.append(_rotate_and_open_log(log_path))
break
except OSError:
time.sleep(0.5)
window.destroy()
webview.start(func=poll, icon=_icon_path(), **_webview_start_kwargs())
return result[0] if result else None
def _setup_logging() -> Path:
"""Redirect stdout/stderr and configure logging to a file in %APPDATA%/mediahive/.
@@ -701,19 +884,30 @@ def _setup_logging() -> Path:
log_dir.mkdir(parents=True, exist_ok=True)
log_path = log_dir / "mediahive.log"
# Rotate: keep previous run as .log.1
prev = log_path.with_suffix(".log.1")
if log_path.exists():
if prev.exists():
prev.unlink()
log_path.rename(prev)
try:
log_file = _rotate_and_open_log(log_path)
except OSError:
# A previous instance still holds the log file. It is usually on its
# way out — give it a couple of seconds silently first.
log_file = None
deadline = time.monotonic() + 2.0
while log_file is None and time.monotonic() < deadline:
time.sleep(0.25)
with contextlib.suppress(OSError):
log_file = _rotate_and_open_log(log_path)
if log_file is None:
log_file = _wait_for_previous_instance(log_path)
if log_file is None:
# Never fail startup over logging: fall back to a per-process file.
log_path = log_dir / f"mediahive-{os.getpid()}.log"
with contextlib.suppress(OSError):
fd = os.open(log_path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC)
log_file = os.fdopen(fd, "w", encoding="utf-8", buffering=1)
fd = os.open(log_path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC)
log_file = os.fdopen(fd, "w", encoding="utf-8", buffering=1) # line-buffered
# Redirect raw stdout/stderr so print() and tracebacks go to the file
sys.stdout = log_file
sys.stderr = log_file
if log_file is not None:
# Redirect raw stdout/stderr so print() and tracebacks go to the file
sys.stdout = log_file
sys.stderr = log_file
# force=True removes handlers added by uvicorn/fastapi during import so that
# basicConfig actually takes effect (without it, it's a silent no-op)
@@ -742,6 +936,54 @@ _SETUP_HTML = """<!DOCTYPE html>
<div><h1>MediaHive</h1><p>Choose a folder that contains your media</p></div>
</body></html>"""
# Shown when a previous instance is still shutting down.
_WAIT_HTML = """<!DOCTYPE html>
<html><head><meta charset="utf-8"><style>
* { box-sizing: border-box; margin: 0; padding: 0; }
body { background: #141414; color: #fff;
font-family: 'Segoe UI', system-ui, sans-serif;
display: flex; align-items: center; justify-content: center;
height: 100vh; text-align: center; }
h1 { font-size: 2rem; color: #e50914; margin-bottom: .5rem; }
p { color: #aaa; }
</style></head><body>
<div><h1>MediaHive</h1>
<p>Waiting for the previous MediaHive instance to finish exiting</p></div>
</body></html>"""
def _show_fatal_error(exc: BaseException) -> None:
"""Show an unhandled exception as a TraceRite HTML page in a webview.
Frozen --windowed builds otherwise surface crashes only as PyInstaller's
plain-text error dialog (or nothing at all).
"""
try:
from tracerite.html import html_traceback
fragment = str(html_traceback(exc))
except Exception: # noqa: BLE001 - error reporting must never raise
return
page = (
"<!DOCTYPE html><html><head><meta charset='utf-8'>"
"<title>MediaHive — Error</title></head>"
f"<body style='margin:1.5rem'>{fragment}</body></html>"
)
try:
webview.create_window("MediaHive — Error", html=page, width=1100, height=750)
webview.start(icon=_icon_path(), **_webview_start_kwargs())
except Exception:
logger.exception("Could not display the error window")
def gui_main() -> None:
"""Run the GUI, rendering fatal exceptions as a TraceRite HTML window."""
try:
winmain()
except Exception as exc:
logger.exception("Fatal error")
_show_fatal_error(exc)
class JsApi:
"""Python methods exposed to the frontend via window.pywebview.api."""
@@ -893,7 +1135,7 @@ def winmain() -> None:
nargs="?",
help="Path to the media folder (default: saved config or initial setup dialog)",
)
args = parser.parse_args()
args, _unknown = parser.parse_known_args()
_prepend_meipass_to_path()
@@ -936,14 +1178,31 @@ def winmain() -> None:
backend_url = f"http://{BACKEND_HOST}:{backend_port}"
os.environ["MEDIAHIVE_BACKEND_URL"] = backend_url
# Run the FastAPI backend on a background thread
# Startup banner, same as fastapi-vue's server.run() prints in CLI mode.
# Goes to stderr, which frozen builds redirect to the log file.
from fastapi_vue.startupbox import print_box
try:
version = importlib.metadata.version("mediahive")
except importlib.metadata.PackageNotFoundError:
version = "dev"
print_box(f"MediaHive {version}\n{backend_url}")
# Run the FastAPI backend on a background thread. fastapi-vue's patched
# log config wires up its access-log middleware, emoji level prefixes and
# tracerite tracebacks (colors are auto-disabled when stderr is not a tty,
# e.g. redirected to the log file in frozen builds).
from fastapi_vue.logging import patch_log_config
config = uvicorn.Config(
"mediahive.server:app",
host=BACKEND_HOST,
port=backend_port,
loop="asyncio",
log_level="warning",
server_header=False,
timeout_graceful_shutdown=0,
access_log=False, # fastapi-vue's middleware replaces uvicorn's
log_config=patch_log_config(uvicorn.config.LOGGING_CONFIG),
)
server = uvicorn.Server(config)
backend_thread = threading.Thread(
@@ -1038,4 +1297,4 @@ def winmain() -> None:
if __name__ == "__main__":
winmain()
gui_main()
+8 -7
View File
@@ -8,7 +8,7 @@ dependencies = [
"aiofiles>=25.1.0",
"aiopathlib>=0.6.0",
"bencodepy>=0.9.5",
"fastapi-vue>=0.5.2",
"fastapi-vue>=1.4.1",
"fastapi[standard]>=0.128.0",
"httpx[http2]>=0.28.1",
"msgspec>=0.19",
@@ -36,7 +36,11 @@ artifacts = ["mediahive/frontend-build"]
only-packages = true
[tool.hatch.build.targets.sdist.hooks.custom]
path = "scripts/fastapi-vue/build-frontend.py"
path = "scripts/fastapi-vue/buildhook.py"
[tool.hatch.build.targets.sdist.force-include]
"scripts/fastapi-vue/buildhook.py" = "scripts/fastapi-vue/buildhook.py"
"scripts/fastapi-vue/buildutil.py" = "scripts/fastapi-vue/buildutil.py"
[tool.uv]
package = true
@@ -46,8 +50,8 @@ parse-torrent-title = { git = "https://github.com/platelminto/parse-torrent-titl
[project.optional-dependencies]
gui = [
"pywebview>=6.2.1; platform_system != 'Darwin'",
"pywebview[qt5]>=6.2.1; platform_system == 'Darwin'",
"pywebview[qt]>=6.2.1; platform_system != 'Windows'",
"pywebview>=6.2.1; platform_system == 'Windows'",
"qtpy>=2.4.1; platform_system == 'Darwin'",
"PyQt5>=5.15.11; platform_system == 'Darwin'",
"pythonnet>=3.1.0rc0; platform_system == 'Windows' and python_version >= '3.14'",
@@ -116,6 +120,3 @@ ignore = [
# Allow unused local variables in ctypes COM boilerplate
"F841",
]
[tool.ruff.lint.per-file-ignores]
"mediahive/access_logging.py" = ["BLE001", "G004"]
-162
View File
@@ -1,162 +0,0 @@
#!/usr/bin/env python3
"""RTorrent Client - Communicate with rtorrent via XMLRPC over SCGI socket."""
import socket
import xmlrpc.client
from pathlib import Path
class SCGITransport(xmlrpc.client.Transport):
"""SCGI transport for communicating with rtorrent via Unix socket."""
def __init__(self, socket_path: str) -> None:
super().__init__()
self.socket_path = socket_path
def single_request(self, _host, _handler, request_body, _verbose=False):
# Create SCGI request
headers = f"CONTENT_LENGTH\x00{len(request_body)}\x00SCGI\x001\x00"
request = f"{len(headers)}:{headers},{request_body.decode('utf-8')}"
# Connect to socket
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.connect(self.socket_path)
sock.send(request.encode("utf-8"))
# Read response
response = b""
while True:
data = sock.recv(4096)
if not data:
break
response += data
sock.close()
# Parse response - skip HTTP headers
if b"\r\n\r\n" in response:
response = response.split(b"\r\n\r\n", 1)[1]
return self.parse_response(response)
def parse_response(self, response_body):
p, u = xmlrpc.client.getparser()
p.feed(response_body)
p.close()
return u.close()
class RTorrentClient:
"""Client for communicating with rtorrent via XMLRPC over SCGI socket."""
def __init__(
self, socket_path: str = "/home/user/rtorrent/.session/rpc.socket"
) -> None:
self.socket_path = socket_path
transport = SCGITransport(socket_path)
self.proxy = xmlrpc.client.ServerProxy(
"http://localhost/RPC2", transport=transport
)
def get_loaded_hashes(self) -> set[str]:
"""Get set of info hashes for all currently loaded torrents."""
try:
downloads = self.proxy.download_list("")
return {h.upper() for h in downloads}
except (OSError, xmlrpc.client.Error) as e:
print(f"Error getting loaded torrents: {e}")
return set()
def load_torrent(self, torrent_path: Path, download_dir: Path) -> bool:
"""Load a torrent file and set its download directory.
Uses load.start_verbose to load and immediately start/hash-check.
Args:
torrent_path: Path to the .torrent file
download_dir: Directory where the data already exists
Returns:
True if successful, False otherwise
"""
try:
# load.start_verbose with d.directory.set to specify download location
# This will hash-check existing files instead of re-downloading
self.proxy.load.start_verbose(
"", str(torrent_path), f'd.directory.set="{download_dir}"'
)
return True
except (OSError, xmlrpc.client.Error) as e:
print(f"Error loading torrent {torrent_path}: {e}")
return False
def get_torrent_info(self, info_hash: str) -> dict | None:
"""Get info about a loaded torrent."""
try:
name = self.proxy.d.name(info_hash)
message = self.proxy.d.message(info_hash)
tied_file = self.proxy.d.tied_to_file(info_hash)
directory = self.proxy.d.directory(info_hash)
base_path = self.proxy.d.base_path(info_hash) # Actual data path
is_multi_file = self.proxy.d.is_multi_file(info_hash)
return {
"hash": info_hash,
"name": name,
"message": message,
"tied_file": tied_file,
"directory": directory,
"base_path": base_path, # Full path to data (file or folder)
"is_multi_file": is_multi_file,
}
except (OSError, xmlrpc.client.Error) as e:
print(f"Error getting torrent info for {info_hash}: {e}")
return None
def get_unregistered_torrents(self) -> list[dict]:
"""Find all torrents with 'unregistered' or 'not registered' tracker errors.
Returns:
List of torrent info dicts for torrents with registration errors
"""
unregistered = []
try:
hashes = self.proxy.download_list("")
for info_hash in hashes:
try:
message = self.proxy.d.message(info_hash)
if message and (
"unregistered" in message.lower()
or "not registered" in message.lower()
):
info = self.get_torrent_info(info_hash)
if info:
unregistered.append(info)
except OSError, xmlrpc.client.Error:
continue
except (OSError, xmlrpc.client.Error) as e:
print(f"Error scanning for unregistered torrents: {e}")
return unregistered
def remove_torrent(self, info_hash: str, delete_files: bool = False) -> bool:
"""Remove a torrent from rtorrent.
Args:
info_hash: The info hash of the torrent to remove
delete_files: If True, also delete downloaded files (default: False)
Returns:
True if successful, False otherwise
"""
try:
if delete_files:
# This would delete the data - NOT what we want
self.proxy.d.erase(info_hash)
else:
# Just remove from rtorrent, keep files
self.proxy.d.erase(info_hash)
return True
except (OSError, xmlrpc.client.Error) as e:
print(f"Error removing torrent {info_hash}: {e}")
return False
+4
View File
@@ -11,6 +11,7 @@ import sys
import mediahive.winmain
import mediahive.server
from pathlib import Path
from PyInstaller.utils.hooks import collect_data_files
block_cipher = None
@@ -32,6 +33,9 @@ _datas = [
# Bundled Vue frontend served by the FastAPI backend
(str(_frontend_build), "mediahive/frontend-build"),
]
# tracerite (indirect dep) loads style.css / script.js at runtime; PyInstaller
# has no hook for it, so collect its package data explicitly
_datas += collect_data_files("tracerite")
if _icon_win.exists():
_datas.append((str(_icon_win), "mediahive/assets"))
if _icon_mac.exists():
Regular → Executable
+15 -7
View File
@@ -9,9 +9,11 @@ import sys
from contextlib import suppress
from pathlib import Path
import tracerite
# Import util.py from scripts/fastapi-vue (not a package, so we adjust sys.path)
sys.path.insert(0, str(Path(__file__).with_name("fastapi-vue")))
from devutil import ( # type: ignore[import-not-found]
from devutil import (
ProcessGroup,
check_ports_free,
logger,
@@ -22,11 +24,15 @@ from devutil import ( # type: ignore[import-not-found]
DEFAULT_VITE_PORT = 8420
DEFAULT_DEV_PORT = 8421
HEALTH = "/api/health?from=devserver.py"
async def run_devserver(
listen: str, backend: str, extra_args: list[str] | None = None
listen: str,
backend: str,
extra_args: list[str] | None = None,
) -> None:
"""Start Vite and FastAPI dev servers with hot reload."""
reporoot = Path(__file__).parent.parent
front = reporoot / "frontend"
if not (front / "package.json").exists():
@@ -36,7 +42,7 @@ async def run_devserver(
viteurl, npm_install, vite = setup_vite(listen, DEFAULT_VITE_PORT)
backurl, mediahive = setup_cli("mediahive", backend, DEFAULT_DEV_PORT)
# Tell the everyone by environment (vite proxy and backend devmode use these)
# Tell everyone via environment (vite proxy and backend devmode use these)
os.environ["MEDIAHIVE_VITE_URL"] = viteurl
os.environ["MEDIAHIVE_BACKEND_URL"] = backurl
os.environ["MEDIAHIVE_DEV"] = "1"
@@ -45,11 +51,13 @@ async def run_devserver(
npm_i = await pg.spawn(*npm_install, cwd=front)
await check_ports_free(viteurl, backurl)
await pg.spawn(*mediahive, *(extra_args or []))
await pg.wait(npm_i, ready(backurl, path="/api/health?from=devserver.py"))
await pg.wait(npm_i, ready(backurl, path=HEALTH))
await pg.spawn(*vite, cwd=front)
def main() -> None:
"""Parse CLI arguments and run the devserver."""
tracerite.load()
parser = argparse.ArgumentParser(
description="Run Vite and FastAPI development servers",
formatter_class=argparse.RawDescriptionHelpFormatter,
@@ -58,12 +66,12 @@ def main() -> None:
parser.add_argument(
"-l",
"--listen",
metavar="host:port",
metavar="addr",
help=f"Vite (default: localhost:{DEFAULT_VITE_PORT})",
)
parser.add_argument(
"--backend",
metavar="host:port",
metavar="addr",
help=f"FastAPI (default: localhost:{DEFAULT_DEV_PORT})",
)
args, extra_args = parser.parse_known_args()
@@ -72,7 +80,7 @@ def main() -> None:
HELP_EPILOG = """
scripts/devserver.py [args to mediahive]
Other options are forwarded to mediahive [args]
JS_RUNTIME environment variable can be used to select the JS runtime:
npm, deno, bun, or full path to the runtime executable (node maps to npm).
-17
View File
@@ -1,17 +0,0 @@
"""Hatch build hook for building Vue frontend during package build."""
import sys
from pathlib import Path
from hatchling.builders.hooks.plugin.interface import ( # type: ignore[import-not-found]
BuildHookInterface,
)
sys.path.insert(0, str(Path(__file__).parent))
from buildutil import build
class CustomBuildHook(BuildHookInterface):
def initialize(self, version, build_data) -> None:
super().initialize(version, build_data)
build("frontend")
+18
View File
@@ -0,0 +1,18 @@
"""Hatch build hook for building Vue frontend during package build."""
import sys
from pathlib import Path
from hatchling.builders.hooks.plugin.interface import BuildHookInterface
sys.path.insert(0, str(Path(__file__).parent))
from buildutil import build
class CustomBuildHook(BuildHookInterface): # type: ignore[misc]
"""Hatch build hook that builds Vue frontend during package build."""
def initialize(self, version: str, build_data: dict) -> None: # type: ignore[override]
"""Build frontend before package is built."""
super().initialize(version, build_data)
build("frontend")
+95 -58
View File
@@ -7,6 +7,8 @@ import shutil
import subprocess
from pathlib import Path
MIN_NODE_VERSION = 20
class _PrefixFormatter(logging.Formatter):
"""Formatter that adds prefix based on log level."""
@@ -31,81 +33,118 @@ def _check_node_version(node_path: str) -> None:
"""
try:
result = subprocess.run(
[node_path, "--version"], capture_output=True, text=True, check=True
[node_path, "--version"],
capture_output=True,
text=True,
check=True,
)
version_str = result.stdout.strip()
# Parse version like "v20.10.0" or "v18.17.1"
match = re.match(r"v(\d+)", version_str)
if match:
major_version = int(match.group(1))
if major_version >= 20:
if major_version >= MIN_NODE_VERSION:
return
raise RuntimeError(
f"Node.js {version_str} found, but v20+ required (install with nvm)"
)
msg = f"Node.js {version_str} found, but v{MIN_NODE_VERSION}+ required"
raise RuntimeError(msg)
except subprocess.CalledProcessError, FileNotFoundError, ValueError:
pass
raise RuntimeError("Could not determine Node.js version")
msg = "Could not determine Node.js version"
raise RuntimeError(msg)
def _validate_npm_runtime(tool: str) -> bool:
"""Validate npm runtime by checking Node.js version. Returns True if valid."""
node_path = shutil.which("node", path=str(Path(tool).parent))
if node_path is None:
return False
try:
_check_node_version(node_path)
except RuntimeError:
return False
return True
def _find_runtime_from_env(options: list[str]) -> tuple[str, str] | None:
"""Find runtime specified by JS_RUNTIME environment variable."""
js_runtime_env = os.environ.get("JS_RUNTIME")
if not js_runtime_env:
return None
js_runtime = js_runtime_env
js_path = Path(js_runtime)
runtime_name = js_path.name
# Map node to npm
if runtime_name == "node":
runtime_name = "npm"
js_runtime = str(js_path.parent / "npm") if js_path.parent.name else "npm"
for option in options:
if option != runtime_name and not runtime_name.startswith(option):
continue
tool = shutil.which(js_runtime)
if tool is None:
msg = f"JS_RUNTIME={js_runtime_env}: {option} not found"
raise RuntimeError(msg)
if option == "npm":
node_path = shutil.which("node", path=str(Path(tool).parent))
if node_path is None:
msg = f"JS_RUNTIME={js_runtime_env}: node not found"
raise RuntimeError(msg)
_check_node_version(node_path)
return tool, option
msg = f"JS_RUNTIME={js_runtime_env} not recognized"
raise RuntimeError(msg)
def _auto_detect_runtime(options: list[str]) -> tuple[str, str]:
"""Auto-detect JavaScript runtime from available options."""
node_version_error: RuntimeError | None = None
for option in options:
tool = shutil.which(option)
if not tool:
continue
if option == "npm" and not _validate_npm_runtime(tool):
try:
node_path = shutil.which("node", path=str(Path(tool).parent))
if node_path:
_check_node_version(node_path)
except RuntimeError as e:
node_version_error = e
continue
return tool, option
if node_version_error:
raise node_version_error
msg = "Node.js (v20+), Deno or Bun is required but none was found"
raise RuntimeError(msg)
def find_js_runtime() -> tuple[str, str]:
"""Find a JavaScript runtime from JS_RUNTIME env or auto-detect.
Returns (tool_path, tool_name) where tool_name is "deno", "npm", or "bun".
Raises JSRuntimeError if no suitable runtime is found.
Raises RuntimeError if no suitable runtime is found.
"""
options = ["npm", "deno", "bun"]
node_version_error: RuntimeError | None = None
# Check for JS_RUNTIME environment variable
if js_runtime_env := os.environ.get("JS_RUNTIME"):
js_runtime = js_runtime_env
js_path = Path(js_runtime)
runtime_name = js_path.name
# Map node to npm
if runtime_name == "node":
runtime_name = "npm"
js_runtime = str(js_path.parent / "npm") if js_path.parent.name else "npm"
for option in options:
if option == runtime_name or runtime_name.startswith(option):
tool = shutil.which(js_runtime)
if tool is None:
raise RuntimeError(
f"JS_RUNTIME={js_runtime_env}: {option} not found"
)
# Check Node.js version if using npm
if option == "npm":
node_path = shutil.which("node", path=str(Path(tool).parent))
if node_path is None:
raise RuntimeError(
f"JS_RUNTIME={js_runtime_env}: node not found"
)
_check_node_version(node_path) # Raises on failure
return tool, option
raise RuntimeError(f"JS_RUNTIME={js_runtime_env} not recognized")
if result := _find_runtime_from_env(options):
return result
# Auto-detect
for option in options:
if tool := shutil.which(option):
# Check Node.js version if using npm
if option == "npm":
node_path = shutil.which("node", path=str(Path(tool).parent))
if node_path is None:
continue
try:
_check_node_version(node_path)
except RuntimeError as e:
node_version_error = e
continue # Try next runtime
return tool, option
# No runtime found - provide helpful error
if node_version_error:
raise node_version_error
raise RuntimeError("Node.js (v20+), Deno or Bun is required but none was found")
return _auto_detect_runtime(options)
def find_build_tool():
def find_build_tool() -> tuple[list[str], list[str]]:
"""Find JavaScript runtime and construct install/build commands.
Returns (install_cmd, build_cmd) tuples of command lists.
@@ -143,9 +182,7 @@ def find_dev_tool() -> list[str]:
if name == "bun":
logger.warning(
"Bun has a bug in WS proxying "
"(https://github.com/oven-sh/bun/issues/9882). "
"Consider using npm instead."
"Bun has a WS proxy bug (github.com/oven-sh/bun/issues/9882). Consider npm.",
)
return [tool, *dev_args[name]]
@@ -178,9 +215,9 @@ def build(folder: str = "frontend") -> None:
install_cmd, build_cmd = find_build_tool()
except RuntimeError as e:
logger.warning(e)
raise SystemExit(1)
raise SystemExit(1) from None
def run(cmd) -> None:
def run(cmd: list[str]) -> None:
display_cmd = [Path(cmd[0]).stem, *cmd[1:]]
logger.info("### %s", " ".join(display_cmd))
subprocess.run(cmd, check=True, cwd=folder)
@@ -190,4 +227,4 @@ def build(folder: str = "frontend") -> None:
logger.info("")
run(build_cmd)
except subprocess.CalledProcessError:
raise SystemExit(1)
raise SystemExit(1) from None
+76 -49
View File
@@ -1,33 +1,32 @@
"""Utilities for the devserver script in the source repository.
Used only with development dependencies.
"""
"""Utilities meant for devserver script, used only in source repository with dev deps."""
import asyncio
import subprocess
import sys
from collections.abc import Coroutine
from contextlib import suppress
from pathlib import Path
from typing import Any, Self
from typing import TYPE_CHECKING, Any, Self
from urllib.parse import urlsplit
import httpx
from buildutil import find_dev_tool, find_install_tool, logger
from fastapi_vue.hostutil import parse_endpoint
if TYPE_CHECKING:
from collections.abc import Coroutine
class ProcessGroup:
"""Manage async subprocesses with automatic cleanup.
Acts like TaskGroup for processes.
"""
"""Manage async subprocesses with automatic cleanup, like TaskGroup for processes."""
def __init__(self) -> None:
"""Initialize empty process tracking."""
self._procs: list[asyncio.subprocess.Process] = []
self._cmds: dict[int, str] = {} # pid -> command name
async def spawn(
self, *cmd: str, cwd: str | None = None
self,
*cmd: str,
cwd: str | None = None,
) -> asyncio.subprocess.Process:
"""Spawn a subprocess and track it."""
cmd_name = Path(cmd[0]).stem
@@ -38,7 +37,8 @@ class ProcessGroup:
return proc
async def wait(
self, *waitables: asyncio.subprocess.Process | Coroutine[Any, Any, Any]
self,
*waitables: asyncio.subprocess.Process | Coroutine[Any, Any, Any],
) -> None:
"""Wait for processes/coroutines to complete, raise SystemExit on failure."""
@@ -59,18 +59,14 @@ class ProcessGroup:
raise SystemExit(1) from None
async def __aenter__(self) -> Self:
"""Return this process group context manager."""
"""Enter the async context manager."""
return self
async def __aexit__(
self,
exc_type: type[BaseException] | None,
*_: object,
) -> None:
async def __aexit__(self, exc_type: type[BaseException] | None, *_: object) -> None:
"""Wait for one process to exit, terminate others, then wait for all."""
await self._cleanup(immediate=exc_type is not None)
async def _cleanup(self, immediate: bool = False) -> None:
async def _cleanup(self, *, immediate: bool = False) -> None:
running = [p for p in self._procs if p.returncode is None]
if not running:
return
@@ -98,7 +94,7 @@ class ProcessGroup:
asyncio.wait_for(
asyncio.gather(*[p.wait() for p in still_running]),
timeout=10,
)
),
)
except TimeoutError:
for p in self._procs:
@@ -108,46 +104,71 @@ class ProcessGroup:
await p.wait()
async def check_ports_free(*urls: str) -> None:
"""Verify URLs are not responding (ports are free).
async def http_get_server(url: str, timeout: float) -> str | None: # noqa: ASYNC109
"""GET url with plain asyncio streams, return the response Server header.
Raise SystemExit if any endpoint responds.
Returns an empty string when the server responds without a Server header,
and None when the server is unreachable or doesn't answer in time.
"""
parts = urlsplit(url)
host = parts.hostname or "localhost"
port = parts.port or (443 if parts.scheme == "https" else 80)
path = parts.path or "/"
if parts.query:
path += f"?{parts.query}"
try:
async with asyncio.timeout(timeout):
reader, writer = await asyncio.open_connection(host, port)
try:
writer.write(f"GET {path} HTTP/1.0\r\nHost: {host}\r\n\r\n".encode())
await writer.drain()
data = await reader.readuntil(b"\r\n\r\n")
finally:
writer.close()
except OSError, EOFError, ValueError, TimeoutError:
return None
for line in data.decode("latin-1").split("\r\n"):
if line.lower().startswith("server:"):
return line.split(":", 1)[1].strip()
return ""
async def check(client: httpx.AsyncClient, url: str) -> None:
with suppress(httpx.RequestError):
res = await client.get(url, timeout=0.1)
server = res.headers.get("server", "server")
logger.warning("Conflicting %s already running at %s", server, url)
async def check_ports_free(*urls: str) -> None:
"""Verify URLs are not responding (ports are free). Raise SystemExit if any respond."""
async def check(url: str) -> None:
server = await http_get_server(url, timeout=0.1)
if server is not None:
logger.warning(
"Conflicting %s already running at %s", server or "server", url
)
raise SystemExit(1)
async with httpx.AsyncClient() as client:
await asyncio.gather(*[check(client, url) for url in urls])
await asyncio.gather(*[check(url) for url in urls])
async def ready(url: str, path: str = "") -> None:
async def ready(url: str, path: str = "", max_attempts: int = 50) -> None:
"""Wait for the server to be ready by polling an endpoint.
Use empty path to disable the check and make this return immediately.
Raises SystemExit(1) if server doesn't start in time.
"""
max_attempts = 50
full_url = f"{url}{path}"
if not path:
return
async with httpx.AsyncClient() as client:
for attempt in range(max_attempts):
try:
await client.get(full_url, timeout=1.0)
logger.info("✓ Backend ready!")
return
except httpx.RequestError:
if attempt == max_attempts - 1:
logger.warning("Backend didn't start in time")
raise SystemExit(1)
await asyncio.sleep(0.1)
for attempt in range(max_attempts):
if await http_get_server(f"{url}{path}", timeout=1.0) is not None:
logger.info("✓ Backend ready!")
return
if attempt == max_attempts - 1:
logger.warning("Backend didn't start in time")
raise SystemExit(1)
await asyncio.sleep(0.1)
def setup_vite(
endpoint: str, default_port: int = 5173
endpoint: str,
default_port: int = 5173,
) -> tuple[str, list[str], list[str]]:
"""Parse frontend endpoint and build commands.
@@ -173,7 +194,9 @@ def setup_vite(
def setup_fastapi(
endpoint: str, module: str, default_port: int = 8000
endpoint: str,
module: str,
default_port: int = 8000,
) -> tuple[str, list[str]]:
"""Parse backend endpoint and build uvicorn command.
@@ -205,7 +228,9 @@ def setup_fastapi(
def setup_cli(
cli: str, endpoint: str, default_port: int = 8000
cli: str,
endpoint: str,
default_port: int = 8000,
) -> tuple[str, list[str]]:
"""Parse backend endpoint and build CLI command.
@@ -221,5 +246,7 @@ def setup_cli(
host = endpoints[0]["host"]
port = endpoints[0]["port"]
cmd = [cli, f"--listen={host}:{port}"]
# Run the package as a module with the current interpreter, instead of
# relying on a PATH-installed CLI entry point.
cmd = [sys.executable, "-m", cli, f"--listen={host}:{port}"]
return f"http://{host}:{port}", cmd
Regular → Executable
+4
View File
@@ -1,3 +1,4 @@
#!/usr/bin/env -S uv run
"""Build the desktop GUI application and package it as a version-numbered ZIP.
Usage:
@@ -238,6 +239,9 @@ def create_zip(version: str) -> Path:
def main() -> None:
# Windows consoles default to cp1252, which can't encode ✓/✗
sys.stdout.reconfigure(errors="replace")
sys.stderr.reconfigure(errors="replace")
try:
version = read_version()
print(f"MediaHive version: {version}")
Regular → Executable
+1
View File
@@ -1,3 +1,4 @@
#!/usr/bin/env -S uv run
"""Publish a MediaHive release to Gitea.
Usage:
-467
View File
@@ -1,467 +0,0 @@
#!/usr/bin/env python3
"""Torrent Scanner - Scans for .torrent files and analyzes their trackers."""
import argparse
import hashlib
import shutil
from collections.abc import Iterator
from dataclasses import dataclass
from pathlib import Path
import bencodepy
from rtorrent_client import RTorrentClient
def _expand_path_pattern(pattern: str) -> list[Path]:
"""Expand a user-provided path or glob pattern with pathlib."""
expanded = Path(pattern).expanduser()
pattern_text = str(expanded)
has_glob = any(ch in pattern_text for ch in "*?[")
if not has_glob:
return [expanded] if expanded.exists() else []
normalized = pattern_text.replace("\\", "/")
if expanded.is_absolute():
root = Path(expanded.anchor)
remainder = normalized[len(expanded.anchor) :].lstrip("/")
return list(root.glob(remainder)) if remainder else []
return list(Path().glob(normalized))
@dataclass
class TorrentInfo:
"""Information extracted from a torrent file."""
path: Path
name: str
trackers: list[str]
size: int | None = None
files: list[str] | None = None
info_hash: str | None = None
is_multi_file: bool = False
def has_tracker(self, domain: str) -> bool:
"""Check if any tracker URL contains the given domain."""
return any(domain.lower() in tracker.lower() for tracker in self.trackers)
def get_download_directory(self) -> Path:
"""Get the download directory (parent of .torrents folder).
Assumes .torrent files are in <download_dir>/.torrents/
so the actual downloads are one level up.
"""
return self.path.parent.parent
def get_expected_data_path(self) -> Path:
"""Get the expected path where downloaded data should exist.
For multi-file torrents: download_dir/torrent_name/ (directory)
For single-file torrents: download_dir/torrent_name (file)
"""
return self.get_download_directory() / self.name
def verify_download_exists(self) -> tuple[bool, str]:
"""Verify that the downloaded data exists on disk.
Returns:
Tuple of (exists: bool, message: str)
"""
expected_path = self.get_expected_data_path()
if self.is_multi_file:
# Multi-file torrent: expect a directory
if not expected_path.exists():
return False, f"Directory not found: {expected_path}"
if not expected_path.is_dir():
return False, f"Expected directory but found file: {expected_path}"
# Optionally check if at least some files exist
existing_files = list(expected_path.rglob("*"))
file_count = sum(1 for f in existing_files if f.is_file())
if file_count == 0:
return False, f"Directory exists but is empty: {expected_path}"
return True, f"Directory exists with {file_count} files"
# Single-file torrent: expect a file
if not expected_path.exists():
return False, f"File not found: {expected_path}"
if expected_path.is_dir():
return False, f"Expected file but found directory: {expected_path}"
return True, f"File exists: {expected_path}"
def parse_torrent(filepath: Path) -> TorrentInfo | None:
"""Parse a .torrent file and extract relevant information.
Args:
filepath: Path to the .torrent file
Returns:
TorrentInfo object or None if parsing fails
"""
try:
data = bencodepy.decode(Path(filepath).read_bytes())
except (OSError, ValueError, TypeError) as e:
print(f"Error parsing {filepath}: {e}")
return None
# Extract trackers
trackers = []
# Main announce URL
if b"announce" in data:
announce = data[b"announce"]
if isinstance(announce, bytes):
trackers.append(announce.decode("utf-8", errors="replace"))
# Announce list (multiple trackers)
if b"announce-list" in data:
for tier in data[b"announce-list"]:
for tracker in tier:
if isinstance(tracker, bytes):
url = tracker.decode("utf-8", errors="replace")
if url not in trackers:
trackers.append(url)
# Extract name
info = data.get(b"info", {})
name = info.get(b"name", b"Unknown").decode("utf-8", errors="replace")
# Calculate info hash
info_hash = hashlib.sha1(bencodepy.encode(info)).hexdigest().upper()
# Extract size and files
size = None
files = None
is_multi_file = False
if b"length" in info:
# Single file torrent
size = info[b"length"]
files = [name]
is_multi_file = False
elif b"files" in info:
# Multi-file torrent
files = []
size = 0
is_multi_file = True
for file_info in info[b"files"]:
file_path = "/".join(
p.decode("utf-8", errors="replace") for p in file_info.get(b"path", [])
)
files.append(file_path)
size += file_info.get(b"length", 0)
return TorrentInfo(
path=filepath,
name=name,
trackers=trackers,
size=size,
files=files,
info_hash=info_hash,
is_multi_file=is_multi_file,
)
def scan_torrent_directories(paths: list[str]) -> Iterator[Path]:
"""Scan directories for .torrent files.
Args:
paths: List of directory paths or glob patterns to scan
Yields:
Path objects for each .torrent file found
"""
for pattern in paths:
for torrent_dir in _expand_path_pattern(pattern):
if torrent_dir.is_dir():
yield from torrent_dir.glob("*.torrent")
def find_torrents_with_tracker(
tracker_domain: str, paths: list[str]
) -> list[TorrentInfo]:
"""Find all torrents that have a specific tracker domain.
Args:
tracker_domain: Domain to search for in tracker URLs (e.g., "hdbits.org")
paths: List of directory paths or glob patterns to scan
Returns:
List of TorrentInfo objects for matching torrents
"""
matching_torrents = []
for torrent_path in scan_torrent_directories(paths):
info = parse_torrent(torrent_path)
if info and info.has_tracker(tracker_domain):
matching_torrents.append(info)
return matching_torrents
def format_size(size_bytes: int | None) -> str:
"""Format bytes as human-readable size."""
if size_bytes is None:
return "Unknown"
for unit in ["B", "KB", "MB", "GB", "TB"]:
if size_bytes < 1024:
return f"{size_bytes:.2f} {unit}"
size_bytes /= 1024
return f"{size_bytes:.2f} PB"
def main() -> None:
"""Run the torrent scanner command-line workflow."""
parser = argparse.ArgumentParser(
description="Scan and manage torrent files",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
%(prog)s /path/to/torrents*/.torrents/
%(prog)s /mnt/disk1/torrents/.torrents/ /mnt/disk2/torrents/.torrents/
%(prog)s /torrents*/.torrents/ --tracker hdbits.org
%(prog)s /torrents*/.torrents/ --dry
""",
)
parser.add_argument(
"paths",
nargs="+",
help="Directories or glob patterns containing .torrent files",
)
parser.add_argument(
"--dry",
action="store_true",
help="Dry run - show what would be done without making changes",
)
parser.add_argument(
"--tracker",
default="hdbits.org",
help="Tracker domain to filter by (default: hdbits.org)",
)
args = parser.parse_args()
dry_run = args.dry
tracker_domain = args.tracker
# Expand glob patterns
expanded_paths = []
for pattern in args.paths:
matches = [str(path) for path in _expand_path_pattern(pattern)]
if matches:
expanded_paths.extend(matches)
else:
expanded_paths.append(pattern)
if dry_run:
print("=" * 60)
print("DRY RUN MODE - No changes will be made")
print("=" * 60)
print("Scanning for torrents...")
print(f"Search paths: {expanded_paths}")
print("-" * 60)
# Parse all torrents
all_torrents: list[TorrentInfo] = []
for torrent_path in scan_torrent_directories(expanded_paths):
info = parse_torrent(torrent_path)
if info:
all_torrents.append(info)
# Separate by tracker
with_hdbits = [t for t in all_torrents if t.has_tracker(tracker_domain)]
without_hdbits = [t for t in all_torrents if not t.has_tracker(tracker_domain)]
# Print stats
print(f"\n{'=' * 60}")
print("SUMMARY")
print(f"{'=' * 60}")
print(f"Total torrents scanned: {len(all_torrents)}")
print(f"With {tracker_domain}: {len(with_hdbits)}")
print(f"Without {tracker_domain}: {len(without_hdbits)}")
# Add hdbits torrents to rtorrent
if with_hdbits:
print(f"\n{'=' * 60}")
print("VERIFYING DOWNLOADS & ADDING TO RTORRENT")
print(f"{'=' * 60}")
# First, verify which torrents have their data
verified = []
missing_data = []
for torrent in with_hdbits:
exists, message = torrent.verify_download_exists()
if exists:
verified.append(torrent)
else:
missing_data.append((torrent, message))
print("\nVerification results:")
print(f" Downloads found: {len(verified)}")
print(f" Downloads missing: {len(missing_data)}")
# Report missing downloads
if missing_data:
print(f"\n{'=' * 60}")
print("TORRENTS WITH MISSING DATA (will not add)")
print(f"{'=' * 60}")
for torrent, message in missing_data:
print(f"\n Name: {torrent.name}")
print(f" Torrent: {torrent.path}")
print(f" Reason: {message}")
# Now add verified torrents to rtorrent
if verified:
print(f"\n{'=' * 60}")
print(f"ADDING {len(verified)} VERIFIED TORRENTS TO RTORRENT")
print(f"{'=' * 60}")
client = RTorrentClient()
loaded_hashes = client.get_loaded_hashes()
print(f"Currently loaded in rtorrent: {len(loaded_hashes)} torrents")
added = 0
skipped = 0
failed = 0
for torrent in verified:
if torrent.info_hash and torrent.info_hash in loaded_hashes:
print(f"Skipping (already loaded): {torrent.name}")
skipped += 1
else:
download_dir = torrent.get_download_directory()
if dry_run:
print(f"Would add: {torrent.name}")
print(f" Download dir: {download_dir}")
added += 1
else:
print(f"Adding: {torrent.name}")
print(f" Download dir: {download_dir}")
if client.load_torrent(torrent.path, download_dir):
added += 1
else:
failed += 1
if dry_run:
print(f"\nDry run: {added} would be added, {skipped} already loaded")
else:
print(
f"\nRtorrent results: {added} added, "
f"{skipped} skipped, {failed} failed"
)
# Clean up unregistered torrents from rtorrent
print(f"\n{'=' * 60}")
print("CHECKING FOR UNREGISTERED TORRENTS")
print(f"{'=' * 60}")
client = RTorrentClient()
unregistered = client.get_unregistered_torrents()
if unregistered:
print(f"Found {len(unregistered)} unregistered torrent(s):\n")
removed_from_rtorrent = 0
removed_torrent_files = 0
removed_downloads = 0
for torrent_info in unregistered:
# Determine the download path (base_path is the actual file/folder)
download_path = (
Path(torrent_info["base_path"]) if torrent_info["base_path"] else None
)
if dry_run:
status = "[DRY]"
if download_path:
print(f" {status} {download_path}")
else:
print(f" {status} {torrent_info['name']} (no data path)")
# Remove from rtorrent (keeps downloaded files)
elif client.remove_torrent(torrent_info["hash"]):
removed_from_rtorrent += 1
# Delete the .torrent file if it exists
tied_file = torrent_info["tied_file"]
if tied_file:
torrent_file = Path(tied_file)
if torrent_file.exists():
try:
torrent_file.unlink()
removed_torrent_files += 1
except OSError:
pass
# Delete the downloaded files
if download_path and download_path.exists():
try:
if download_path.is_dir():
shutil.rmtree(download_path)
else:
download_path.unlink()
removed_downloads += 1
print(f" [DEL] {download_path}")
except OSError as e:
print(f" [ERR] {download_path}: {e}")
else:
print(f" [DEL] {torrent_info['name']} (no data)")
else:
print(f" [ERR] {torrent_info['name']}: failed to remove from rtorrent")
print()
if dry_run:
print(
f"Dry run: {len(unregistered)} would be removed "
"(rtorrent + .torrent + downloads)"
)
else:
print(
f"Cleanup: {removed_from_rtorrent} from rtorrent, "
f"{removed_torrent_files} .torrents, {removed_downloads} downloads"
)
else:
print("No unregistered torrents found.")
# List torrents without hdbits.org
if without_hdbits:
print(f"\n{'=' * 60}")
print(f"TORRENTS WITHOUT {tracker_domain.upper()}")
print(f"{'=' * 60}")
for torrent in without_hdbits:
print(f"\nName: {torrent.name}")
print(f"Path: {torrent.path}")
print(f"Size: {format_size(torrent.size)}")
if torrent.trackers:
print("Trackers:")
for tracker in torrent.trackers:
print(f" - {tracker}")
else:
print("Trackers: (none)")
# Remove the non-hdbits torrent files
print(f"\n{'=' * 60}")
if dry_run:
print(f"WOULD REMOVE {len(without_hdbits)} TORRENT FILE(S)")
print(f"{'=' * 60}")
for torrent in without_hdbits:
print(f"Would remove: {torrent.path}")
else:
print(f"REMOVING {len(without_hdbits)} TORRENT FILE(S)")
print(f"{'=' * 60}")
for torrent in without_hdbits:
try:
torrent.path.unlink()
print(f"Removed: {torrent.path}")
except OSError as e:
print(f"Failed to remove {torrent.path}: {e}")
if __name__ == "__main__":
main()