Commit Graph
132 Commits
Author SHA1 Message Date
LeoVasanko b13f08eab2 Avoid InvalidStateError on conversion tasks trying to set their result after preview cancelled. 2026-05-02 16:04:21 +00:00
LeoVasanko 5d566d6deb Restore CISTA_HOME internal environment passing of config dir removed in commit 31e0197, that was not being passed across processes. 2026-05-02 05:47:24 +00:00
LeoVasanko bd7291e9ef Lint: unused arguments 2026-05-02 05:26:47 +00:00
LeoVasanko 2fa52229cc Exception nazi and other suppression removals. Added tests on auth flows that were simplified to linter requirements. 2026-05-02 05:22:56 +00:00
LeoVasanko 2dea459d8f Update fastapi-vue-setup 1.3.1 to avoid ruff errors. Fixed devserver script on platforms where Sanic needs AppServer. 2026-05-02 05:03:43 +00:00
LeoVasanko 922069c603 Imports to top of file (ruff rule now). Refactor a couple of import cycles by implementing clear hierarchy of modules. 2026-05-02 04:46:34 +00:00
LeoVasanko 5c7c7343ad Fix logging regressions from unrelated changes and ruff changing emoji into i. Simplified custom logger. 2026-05-02 04:16:38 +00:00
LeoVasanko 593d16d8c5 Use tracerite for human-readable tracebacks. 2026-05-02 03:50:20 +00:00
LeoVasanko 20d8d317fa Ruff linting. 2026-05-02 03:31:51 +00:00
LeoVasanko 1bec73f4cd onlyoffice previews much faster and more robust, added --oosetup helper, improved error/log handling 2026-05-02 03:10:17 +00:00
LeoVasanko 4dd1d4c7e6 Fix AVIF preview encoding for yuvj video frames 2026-05-02 00:37:53 +00:00
LeoVasanko 84ef91a360 preview: move OnlyOffice unavailable from warning log to access log extra 2026-05-01 23:20:01 +00:00
LeoVasanko 8b4e622aef preview: remove ffprobe fallback and unused json import
_get_image_dimensions now relies solely on pyvips header reading.
If pyvips cannot read the header the function returns None and the
caller encodes at full resolution rather than falling back to ffprobe.
Remove unused stdlib json import.
2026-05-01 22:51:58 +00:00
LeoVasanko 13f32c57ab preview: ffmpeg CLI fallback for HEIC/HDR, worker pool fixes, logging improvements
- Use ffmpeg CLI directly for HEIC/HEIF previews (bypassing pyvips which
  cannot encode >8-bit AVIF on current system). ffmpeg handles tile
  assembly, EXIF rotation, HDR metadata and ICC profiles automatically.
- pyvips remains primary for other image formats; ffmpeg is fallback.
- Simplify _get_image_dimensions() to use pyvips header read for all
  formats including HEIC, with ffprobe as fallback.
- Move all imports to top of preview.py; remove lazy OnlyOffice import.
- Worker pool: add readiness handshake (\x01 byte), bounded idle wait
  with PREVIEW_TIMEOUT, eager module import before signalling.
- Fix dispatch data check bug (was 'if data is not None' -> 'if data').
- Add logger.exception for unhandled preview/dispatch errors.
- EmojiFormatter now includes tracebacks for exception logs.
- Return 503 on preview timeouts instead of 504.
2026-05-01 22:48:30 +00:00
LeoVasanko 25a2a5f20c Separate OnlyOffice async handling, add office_previews WS flag, framed worker protocol
- Move OnlyOffice conversion out of worker subprocesses into async event loop
  using httpx.AsyncClient with shared client and clean shutdown hook
- Add OOConversionManager with in-flight deduplication (asyncio.Future) and
  configurable concurrency limit (OO_MAX_CONCURRENT=2)
- Add 10s total timeout for office previews via asyncio.wait_for
- Return 503 when OnlyOffice is unavailable, 504 on timeout
- Remove office handling from worker dispatch(); workers now only do
  images, video, and PDFs
- Replace PNG tempfile bridge with framed binary input protocol on worker
  stdin: (json_size)(data_size)(json)(raw_data)
- Add process_image_buffer() for in-memory AVIF conversion via pyvips
- Add office_previews to WS server message, cached every 30s from OO
  availability check
- Frontend gates only office document previews on office_previews flag;
  images, video, PDFs remain unconditional
- Change default OnlyOffice port from 8080 to 8988
2026-05-01 03:42:03 +00:00
LeoVasanko 041090cce9 preview: increase timeout to 10s, add priority queue scheduling (images > video > pdf > office) 2026-04-27 04:51:41 +00:00
LeoVasanko 575df1214b session: rename cookie to 'cista', add __Host- prefix on HTTPS 2026-04-27 03:56:07 +00:00
LeoVasanko 1927c24053 Revert an accidental change of functionality that was breaking file listings, originally done for linting purposes in commit 3767fb0. 2026-04-26 07:47:27 +00:00
LeoVasanko 6242c76be8 Add share token support with virtual filesystem and selection toolbar button 2026-04-26 07:46:14 +00:00
LeoVasanko 8c93a4f2b5 Add OnlyOffice-based preview for office documents
Replace Aspose.Words with OnlyOffice Document Server for generating
bitmap previews of office documents (Word, Excel, PowerPoint, etc.).

Backend:
- Add cista/onlyoffice.py conversion client
- Convert office docs directly to PNG via OnlyOffice, then AVIF via pyvips
- Make office previews optional based on OnlyOffice availability
- Remove Aspose.Words dependency and all related code
- Add spreadsheet and presentation format support

Frontend:
- Mark office files as previewable in Document.ts
- Add office extensions to MediaPreview.vue preview list
- Fix pre-existing @ts-ignore in HeaderMain.vue

Tests:
- Fix test_lrucache.py parameter name (open -> opener)

Also run ruff format across the codebase to satisfy linter checks.
2026-04-26 06:59:01 +00:00
LeoVasanko 942b54d795 lint: apply manual ruff cleanup (non-preview files) 2026-04-26 06:16:42 +00:00
LeoVasanko 18ee0f3f56 sso: use /auth/api/check for token permission checks 2026-04-26 05:46:09 +00:00
LeoVasanko f8b2c9494a WebDAV sync support, access tokens, REST control endpoints (#10)
Implement complete WebDAV file serving compatible with various clients from Windows File Explorer to more specialized sync tools. The old control WebSocket has been updated to part-DAV, part REST API instead. Implemented user:pass BASIC auth. Added UI and backend for creating tokens that avoid the need to use actual username and password for requests from CLI or DAV.
2026-04-26 04:22:52 +00:00
LeoVasanko 9ac571463a Uploads by PUT range requests rather than WS, remove dead code WS handlers (#9)
Increases block size to 16 MiB and adjusts progress display to work smoothly with that. Also removes download WS that was already unused. Provides faster upload speed than over WS.
2026-04-25 01:55:52 +00:00
LeoVasanko d52b24372a Remove CISTA_PIL and Pillow preview mode 2026-04-25 02:06:40 +00:00
LeoVasanko 742b3006c9 Adjust emoji log spacing 2026-04-25 01:59:14 +00:00
LeoVasanko 01a979c00a Improve preview worker error propagation and logging 2026-04-25 01:51:49 +00:00
LeoVasanko f1e16b7abe Preview: add pdf+pyvips encode path and simplify timings 2026-04-25 00:20:19 +00:00
LeoVasanko 0ebff0ec17 Preview: simplify pyvips timing to single processing ms + req total. 2026-04-25 00:06:35 +00:00
LeoVasanko d5ff7757c8 Preview: persistent worker pool, fast AVIF encode, access-log timing
- Add preview_worker.py: long-lived subprocess; JSONL request / framed binary
  response protocol (BLAKE3 checksum + LE uint32 sizes + msgspec JSON + payload)
- Add _PreviewWorkerPool: asyncio queue-based pool; kill+replace workers on timeout,
  checksum error, or protocol error; PREVIEW_TIMEOUT=3s, PREVIEW_WORKERS=cpu-scaled
- Use pyvips effort=0,strip=True for fast AVIF thumbnails; CISTA_PIL=1 env forces Pillow
- Report load/process/save ms in PreviewResponse; inject into access log via req.ctx._log_extra
- Wire start/shutdown_preview_workers into Sanic lifecycle hooks (app.py)
- Call configure_main_logging() after Sanic() to install emoji formatter
- Silence worker stderr (DEVNULL)
2026-04-24 23:53:23 +00:00
LeoVasanko cd604eb10a Logging: emoji formatter and access-log extra field 2026-04-24 23:52:34 +00:00
LeoVasanko abcf5d9940 Use client IP (potentially proxied) in logs, handle [ipv6]. 2026-04-24 20:13:11 +00:00
LeoVasanko 0190bda853 Avoid useless check. 2026-04-24 00:04:58 +00:00
LeoVasanko da4bba95be Handle missing preview files with silent 404 2026-04-23 19:02:54 +00:00
LeoVasanko e07ab220cb Add cleaner access logging for HTTP requests and WebSockets 2026-04-23 18:06:22 +00:00
LeoVasanko ded7ce65bc Faster image previews by pyvips. 2026-04-23 17:19:24 +00:00
LeoVasanko 0b269aef7f Set avif=1 flag for SVT-AV1 when encoding a single image, for increased efficiency and less memory usage. 2026-04-23 16:57:27 +00:00
LeoVasanko 497de296f2 Consistent forwarding of host/origin/ua over HTTP and WS to Paskia. 2026-02-10 23:56:42 +00:00
LeoVasanko 134b216f4c Cleanup 2026-02-10 23:52:07 +00:00
LeoVasanko c51552ea29 Cleaner WebSocket/Paskia proxying. 2026-02-10 23:42:13 +00:00
LeoVasanko 00645fc8ff Attempt to get allocated size on Windows which needs WINAPI. Fallback to file size if not possible. 2026-02-05 20:31:46 +00:00
LeoVasanko 8480a73839 Fix --listen docopt parsing (only -l was working). 2026-02-05 20:01:26 +00:00
LeoVasanko 302ed684e7 Remove invalid Sanic kwarg when binding to all interfaces (not needed anyway). 2026-02-05 19:56:54 +00:00
LeoVasanko af35e0480a Better proxy header processing, pass original user-agent without httpx overriding it. 2026-02-05 17:53:52 +00:00
LeoVasanko f578a50007 Pass the new allocated field through search filtering. Fixes issue where 0 allocated was reported in search results. 2026-02-04 23:55:05 +00:00
LeoVasanko f40d9c1abd Cache previews in server RAM for much faster access (they are about 70kB each). 2026-02-04 20:56:16 +00:00
LeoVasanko 3d8845cf99 Brighter low free space colors. 2026-02-04 20:40:59 +00:00
LeoVasanko 87e1443e7d Auth restricted endpoint is picky about the trailing slash, use it consistently. 2026-02-04 20:35:29 +00:00
LeoVasanko f45c57e901 Auth mode indication on startupbox 2026-02-04 20:24:21 +00:00
LeoVasanko 41686d1dd1 Correctly handle customized server name in admin settings dialog. 2026-02-04 20:15:56 +00:00