Vasanko/cista-storage
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
240 Commits 1 Branch 58 Tags
31e0197fd9867f1ee87786e0dae1896d7f3ae428
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Leo Vasanko 31e0197fd9 onlyoffice previews much faster and more robust, added --oosetup helper, improved error/log handling 2026-05-02 03:10:17 +00:00
Leo Vasanko 0ec3b3d93a Fix AVIF preview encoding for yuvj video frames 2026-05-02 00:37:53 +00:00
Leo Vasanko e0807c0500 preview: move OnlyOffice unavailable from warning log to access log extra 2026-05-01 23:21:35 +00:00
Leo Vasanko dff95c414b frontend: trim document formats, simplify type getters, use FILE_TYPES consistently 
- Trim document extension list to common office formats only
  (doc/docx/xls/xlsx/ppt/pptx/odt/ods/odp/rtf), removing txt/md/csv/html/xml/etc.
- Simplify Doc getters: merge guard clauses into single returns
  (img, previewable, previewurl, ext).
- Simplify MediaPreview.preview() into single boolean expression.
- Use global FILE_TYPES instead of inline extension lists.
 2026-05-01 23:17:40 +00:00
Leo Vasanko b6f4172514 preview: remove ffprobe fallback and unused json import 
_get_image_dimensions now relies solely on pyvips header reading.
If pyvips cannot read the header the function returns None and the
caller encodes at full resolution rather than falling back to ffprobe.
Remove unused stdlib json import.
 2026-05-01 22:51:58 +00:00
Leo Vasanko 6de9b443ae preview: ffmpeg CLI fallback for HEIC/HDR, worker pool fixes, logging improvements 
- Use ffmpeg CLI directly for HEIC/HEIF previews (bypassing pyvips which
  cannot encode >8-bit AVIF on current system). ffmpeg handles tile
  assembly, EXIF rotation, HDR metadata and ICC profiles automatically.
- pyvips remains primary for other image formats; ffmpeg is fallback.
- Simplify _get_image_dimensions() to use pyvips header read for all
  formats including HEIC, with ffprobe as fallback.
- Move all imports to top of preview.py; remove lazy OnlyOffice import.
- Worker pool: add readiness handshake (\x01 byte), bounded idle wait
  with PREVIEW_TIMEOUT, eager module import before signalling.
- Fix dispatch data check bug (was 'if data is not None' -> 'if data').
- Add logger.exception for unhandled preview/dispatch errors.
- EmojiFormatter now includes tracebacks for exception logs.
- Return 503 on preview timeouts instead of 504.
 2026-05-01 22:48:30 +00:00
Leo Vasanko 49cb4a8255 Separate OnlyOffice async handling, add office_previews WS flag, framed worker protocol 
- Move OnlyOffice conversion out of worker subprocesses into async event loop
  using httpx.AsyncClient with shared client and clean shutdown hook
- Add OOConversionManager with in-flight deduplication (asyncio.Future) and
  configurable concurrency limit (OO_MAX_CONCURRENT=2)
- Add 10s total timeout for office previews via asyncio.wait_for
- Return 503 when OnlyOffice is unavailable, 504 on timeout
- Remove office handling from worker dispatch(); workers now only do
  images, video, and PDFs
- Replace PNG tempfile bridge with framed binary input protocol on worker
  stdin: (json_size)(data_size)(json)(raw_data)
- Add process_image_buffer() for in-memory AVIF conversion via pyvips
- Add office_previews to WS server message, cached every 30s from OO
  availability check
- Frontend gates only office document previews on office_previews flag;
  images, video, PDFs remain unconditional
- Change default OnlyOffice port from 8080 to 8988
 2026-05-01 03:42:03 +00:00
Leo Vasanko cb954db537 preview: increase timeout to 10s, add priority queue scheduling (images > video > pdf > office) 2026-04-27 04:51:41 +00:00
Leo Vasanko 4a66d2749c Add sort-order keycap hints next to search bar 
Show 'Order [1] [2] [3]' keycaps to the right of the search bar
when the viewport is at least 800px wide. These visual hints
match the existing '/' search keycap style and correspond to
the existing keyboard shortcuts for sorting:
  1 = name (alphabetical)
  2 = modified (newest first)
  3 = size (largest first)

Hints are hidden on narrow viewports and when text input fields are focused.
 2026-04-27 04:10:13 +00:00
Leo Vasanko bcbcef219e session: rename cookie to 'cista', add __Host- prefix on HTTPS 2026-04-27 04:00:14 +00:00
Leo Vasanko d6d5eb85d6 Revert an accidental change of functionality that was breaking file listings, originally done for linting purposes in commit 3767fb0. 2026-04-27 04:00:14 +00:00
Leo Vasanko 1e75763f77 Add share token support with virtual filesystem and selection toolbar button 2026-04-26 07:46:14 +00:00
Leo Vasanko e3282d1456 Add benchmark script for OnlyOffice output formats 
Benchmarks three preview pipelines across all sample documents:
- BMP → AVIF (via pyvips)
- PNG → AVIF (via pyvips) — selected for production
- PNG only (no compression)

Results confirm PNG → AVIF as the optimal path:
- ~30 ms AVIF encode overhead
- 2.1× size reduction vs raw PNG
- Slightly faster than BMP → AVIF
 2026-04-26 07:28:52 +00:00
Leo Vasanko fb0ddd20e0 Add OnlyOffice-based preview for office documents 
Replace Aspose.Words with OnlyOffice Document Server for generating
bitmap previews of office documents (Word, Excel, PowerPoint, etc.).

Backend:
- Add cista/onlyoffice.py conversion client
- Convert office docs directly to PNG via OnlyOffice, then AVIF via pyvips
- Make office previews optional based on OnlyOffice availability
- Remove Aspose.Words dependency and all related code
- Add spreadsheet and presentation format support

Frontend:
- Mark office files as previewable in Document.ts
- Add office extensions to MediaPreview.vue preview list
- Fix pre-existing @ts-ignore in HeaderMain.vue

Tests:
- Fix test_lrucache.py parameter name (open -> opener)

Also run ruff format across the codebase to satisfy linter checks.
 2026-04-26 06:59:01 +00:00
Leo Vasanko c49e66323f frontend: add biome checks and pre-commit integration (excluding preview files) 2026-04-26 06:43:06 +00:00
Leo Vasanko 3767fb0cec lint: apply manual ruff cleanup (non-preview files) 2026-04-26 06:16:42 +00:00
Leo Vasanko 9321961a11 sso: use /auth/api/check for token permission checks 2026-04-26 05:46:09 +00:00
Leo Vasanko a88a22571a Remove CISTA_PIL and Pillow preview mode 2026-04-25 02:06:40 +00:00
Leo Vasanko d58748439a Adjust emoji log spacing 2026-04-25 02:05:20 +00:00
Leo Vasanko c51701c252 Improve preview worker error propagation and logging 2026-04-25 01:51:49 +00:00
Leo Vasanko 88a032acfe Preview: add pdf+pyvips encode path and simplify timings 2026-04-25 00:20:19 +00:00
Leo Vasanko bbb73be07e Preview: simplify pyvips timing to single processing ms + req total. 2026-04-25 00:06:35 +00:00
Leo Vasanko 48112dec26 Frontend: don't show img preview for video/audio/doc files 
- MediaPreview.vue: only render <img> for non-video, non-audio items
  (video/audio already have their own player containers)
- Document.ts: limit previewurl to previewable files (was checking only .complete)
 2026-04-24 23:54:02 +00:00
Leo Vasanko fc2f357470 Preview: persistent worker pool, fast AVIF encode, access-log timing 
- Add preview_worker.py: long-lived subprocess; JSONL request / framed binary
  response protocol (BLAKE3 checksum + LE uint32 sizes + msgspec JSON + payload)
- Add _PreviewWorkerPool: asyncio queue-based pool; kill+replace workers on timeout,
  checksum error, or protocol error; PREVIEW_TIMEOUT=3s, PREVIEW_WORKERS=cpu-scaled
- Use pyvips effort=0,strip=True for fast AVIF thumbnails; CISTA_PIL=1 env forces Pillow
- Report load/process/save ms in PreviewResponse; inject into access log via req.ctx._log_extra
- Wire start/shutdown_preview_workers into Sanic lifecycle hooks (app.py)
- Call configure_main_logging() after Sanic() to install emoji formatter
- Silence worker stderr (DEVNULL)
 2026-04-24 23:53:23 +00:00
Leo Vasanko e579dbc61f Logging: emoji formatter and access-log extra field 2026-04-24 23:52:34 +00:00
Leo Vasanko 3e337bf1f2 Use client IP (potentially proxied) in logs, handle [ipv6]. 2026-04-24 20:13:11 +00:00
Leo Vasanko 522dd49f3f Avoid useless check. 2026-04-24 00:04:58 +00:00
Leo Vasanko 53b3835ba2 Upload end of file first for large files (for preview to function). 2026-04-24 00:02:17 +00:00
Leo Vasanko bb25d8db74 Avoid showing previews while file upload is in progress or the file is incomplete. This avoids 500 errors on server trying to generate the preview. 2026-04-23 19:34:16 +00:00
Leo Vasanko 7b3252c588 Handle missing preview files with silent 404 2026-04-23 19:02:54 +00:00
Leo Vasanko 4626f1cdff Add cleaner access logging for HTTP requests and WebSockets 2026-04-23 18:06:22 +00:00
Leo Vasanko a2e2cccebe Don't show broken media previews during uploads, instead show a spinner indicating completion. 2026-04-23 17:40:03 +00:00
Leo Vasanko 1795506090 Faster image previews by pyvips. 2026-04-23 17:19:24 +00:00
Leo Vasanko 314422c96c Allow duplicate selected entries of the same name from different paths. 2026-04-23 17:01:03 +00:00
Leo Vasanko e490cbe0fe Set avif=1 flag for SVT-AV1 when encoding a single image, for increased efficiency and less memory usage. 2026-04-23 16:57:27 +00:00
Leo Vasanko 2c74779102 Consistent forwarding of host/origin/ua over HTTP and WS to Paskia. 2026-02-10 23:56:42 +00:00
Leo Vasanko b65404beba Cleanup 2026-02-10 23:52:07 +00:00
Leo Vasanko 4aef6c73d2 pytest deps, remove unused test group. 2026-02-10 23:46:44 +00:00
Leo Vasanko 67d1fa65fb Cleaner WebSocket/Paskia proxying. 2026-02-10 23:42:13 +00:00
Leo Vasanko 6b009dea40 Attempt to get allocated size on Windows which needs WINAPI. Fallback to file size if not possible. 2026-02-05 20:31:46 +00:00
Leo Vasanko ad224dd97d Fix vite argument passing on deno (fastapi-vue-setup upstream). 2026-02-05 20:13:03 +00:00
Leo Vasanko 7b19739875 Fix --listen docopt parsing (only -l was working). 2026-02-05 20:01:26 +00:00
Leo Vasanko 362a9f47c9 Remove invalid Sanic kwarg when binding to all interfaces (not needed anyway). 2026-02-05 19:56:54 +00:00
Leo Vasanko 7b3a39a58e Better proxy header processing, pass original user-agent without httpx overriding it. 2026-02-05 17:53:52 +00:00
Leo Vasanko 0b08acdcc3 Intl support for search hotkey (where US keyboard has it). 2026-02-05 00:57:42 +00:00
Leo Vasanko 298f389dad Keyboard navigation fixes (still not perfect but better). 2026-02-05 00:54:15 +00:00
Leo Vasanko a163798178 Fix search bar losing focus under specific conditions, mainly that when the search takes more than 50ms, causing results to be emptied, with a file priorly focused, was causing breadcrumbs to get focused. (reactivity is hard) 2026-02-05 00:07:45 +00:00
Leo Vasanko 204a308396 Pass the new allocated field through search filtering. Fixes issue where 0 allocated was reported in search results. 2026-02-04 23:55:05 +00:00
Leo Vasanko 82062d1d7a Cache previews in server RAM for much faster access (they are about 70kB each). 2026-02-04 20:56:16 +00:00
Leo Vasanko ad37a48df9 Brighter low free space colors. 2026-02-04 20:40:59 +00:00
Leo Vasanko 84d80518a6 Auth restricted endpoint is picky about the trailing slash, use it consistently. 2026-02-04 20:35:29 +00:00
Leo Vasanko e520d9361c Auth mode indication on startupbox 2026-02-04 20:24:21 +00:00
Leo Vasanko 5b4be95ace Correctly handle customized server name in admin settings dialog. 2026-02-04 20:15:56 +00:00
Leo Vasanko 69c26ff85b Space usage widget show full name if possible, allow longer names. 2026-02-04 20:12:49 +00:00
Leo Vasanko 84fcf25cc5 Rename HeaderSelected to SelectionToolbar. 2026-02-04 20:01:05 +00:00
Leo Vasanko 09ef869d7e Improved selection toolbar UX. 2026-02-04 19:57:31 +00:00
Leo Vasanko cb9652567b README 2026-02-04 19:23:48 +00:00
Leo Vasanko 4ea1982312 Make devserver script take --listen argument for vite, while forwarding other args correctly to backend 2026-02-04 19:23:34 +00:00
Leo Vasanko 1f757ef335 Consistent header and footer sizing. 2026-02-04 19:19:41 +00:00
Leo Vasanko b3df715e25 Fix free/used disk space handling. Implement file disk usage tracking. Display indicators on FileExplorer for incomplete files (sparse allocation, upload in progress). 2026-02-04 19:15:09 +00:00
Leo Vasanko 0656a027f4 Implement server name config to override the default of using share folder name. 2026-02-04 18:00:21 +00:00
Leo Vasanko aab1c59022 Punch a hole in the pie to make a donut. 2026-02-04 17:27:56 +00:00
Leo Vasanko 395d6c3cce Unused imports 2026-02-04 17:09:14 +00:00
Leo Vasanko c698f74319 Don't allow header scale larger with the window above 1000px, while the rest of the app keeps scaling by root font size. 2026-02-04 17:08:32 +00:00
Leo Vasanko 56aacb8f87 Revisited diskspace widget animations and styling. 2026-02-04 17:01:22 +00:00
Leo Vasanko 78890347cc More consistent and properly scaling header layout and positioning. Search keyboard tooltip hidden for mobile users. 2026-02-04 16:13:03 +00:00
Leo Vasanko 598c63278a New much prettier disk space widget. 2026-02-04 15:14:06 +00:00
Leo Vasanko 0b8462d0dc Refactor to make full file list completely non-reactive because pinia persistence was causing long delays especially while searching when there were a lot of files. Implement better ghosts that do not alter the file list. 2026-02-04 02:03:41 +00:00
Leo Vasanko d7aae07af0 Prebuild lookup structures to reduce UI lag on very large file lists. 2026-02-04 00:18:28 +00:00
Leo Vasanko e532e8dcfa Make devserver script pass extra args to cista CLI. 2026-02-04 00:01:02 +00:00
Leo Vasanko f479115bd6 Require new fastapi-vue for unix socket compatibility. 2026-02-01 07:10:21 +00:00
Leo Vasanko 510a908841 Disk space pie chart. 2026-02-01 06:59:15 +00:00
Leo Vasanko 15d227fda2 Implement a larger set of file menu options. 2026-02-01 06:34:50 +00:00
Leo Vasanko 3a86fe26a7 Remove refresh wwwroot support and static files serving, only using vite to serve frontend in dev mode (avoid accidentally using stale builds). 2026-02-01 06:14:35 +00:00
Leo Vasanko 5c530652fa Change default port to 8989 like Droppy. Using 8999 for backend in dev mode. 2026-02-01 06:09:56 +00:00
Leo Vasanko 1e207b38b7 Use fastapi-vue parse_endpoint instead of our own logic. 2026-02-01 06:00:30 +00:00
Leo Vasanko e8517c75e2 Better UI feedback on search order changes. 2026-02-01 05:48:15 +00:00
Leo Vasanko b1fae56c86 Fix a search caching issue. 2026-02-01 05:38:48 +00:00
Leo Vasanko 098febec99 New header/footer layout and styles. Use cursorTooltip in more places, make it prettier. Fix tooltip still sometimes activating with touch (now only with mouse). 2026-02-01 05:22:48 +00:00
Leo Vasanko 6264f9624f Remove download to folder, just download zip or single file. The folder download is still accessible by Alt+Click on the download button (in the few browsers that support it). 2026-02-01 03:47:14 +00:00
Leo Vasanko e0937ed233 Automatically navigate into newly uploaded folders. 2026-02-01 03:27:13 +00:00
Leo Vasanko 51bf029d2a Implement ghost items reflecting pending changes from frontend (e.g. files being uploaded) until the server acknowledges them. This gives immediate UI feedback and is clearer in cases. 2026-02-01 03:23:43 +00:00
Leo Vasanko abe967de1b Remove dead code. 2026-02-01 02:49:50 +00:00
Leo Vasanko 80453e98ab Unified watcher that *may* receive events from inotify and other sources. Added change notify messages from control and upload WebSockets. Cleanup debug printouts. 2026-02-01 02:44:52 +00:00
Leo Vasanko add9d7ac82 New inotify watcher with efficient algorithms. 2026-01-31 23:46:43 +00:00
Leo Vasanko 7e4b56d9ae Performance profile file change watching. 2026-01-31 22:53:17 +00:00
Leo Vasanko 5ead0f4deb Debouncing to avoid UI flicker while typing in search. 2026-01-31 22:45:27 +00:00
Leo Vasanko 9828597e6b Search optimizations and fixes. Prevent re-renders on hash change (redux). Clear previous search results at the start of a new search. 2026-01-31 22:37:47 +00:00
Leo Vasanko d1440f539b Optimizing the search faster; don't trigger full re render by URL changes. 2026-01-31 22:17:05 +00:00
Leo Vasanko 79875d3190 Simplified startup box unix socket formatting. 2026-01-31 22:09:57 +00:00
Leo Vasanko a79159aad5 Upgrade to docopt-ng because docopt is no longer maintained. 2026-01-31 22:06:03 +00:00
Leo Vasanko 5590a756ca Better error logging on preview generation errors. 2026-01-31 22:00:26 +00:00
Leo Vasanko bb0c4075d0 Basic details in startup box. 2026-01-31 21:55:08 +00:00
Leo Vasanko 40fb7bf398 Implement background worker to make search response faster and avoid hanging the UI when there are a lot of files. Implement better toast messages for transitional events that automatically disappear. 2026-01-31 21:02:16 +00:00
Leo Vasanko 1e74245721 Smarter ZIP download naming. 2026-01-31 20:59:43 +00:00
Leo Vasanko 9879eea7c4 More robust zip download processing; don't let aborted requests hang threads, use a larger threadpool separate of preview generation. 2026-01-31 19:43:23 +00:00
Leo Vasanko b61da257cb Update README for Paskia SSO 2026-01-31 05:15:52 +00:00
Leo Vasanko 61e59c9b45 Fixes for public mode authentication flows. 2026-01-31 04:48:44 +00:00
Leo Vasanko 9c173ff0bf Finalize Paskia integration and built-in authentication. 2026-01-31 00:48:02 +00:00
Leo Vasanko 1dab7f0fd1 Bundle icons into a single file. 2026-01-30 19:12:27 +00:00