9 Commits
Author SHA1 Message Date
LeoVasanko 720c4f06e5 logging: add opt-in quiet_vips_logging() helper
pyvips redirects all GLib messages ("VIPS: threadpool completed ..."
per operation) onto the 'pyvips' logger at INFO. The helper caps that
logger at WARNING while keeping 'pyvips.voperation' (deprecated-argument
notices) at INFO. Library import remains free of logging side effects;
the CLI and worker entry points call the helper instead of their old
bare setLevel, which also hid the deprecation notices.
2026-09-18 03:42:36 +00:00
Leo Vasanko c15e964e6d backends: route office formats through dispatch(); diagnostic unknown-extension errors
Synchronous dispatch() now converts office documents via OnlyOffice
(asyncio.run around OOConversionManager.convert), refusing to run inside
a live event loop and pointing async callers at
pool.generate_office_preview(). The office import stays lazy so the base
install works without the 'office' extra.

Unknown file types now name the offending extension (or its absence)
instead of a generic 'preview unsupported'.
2026-09-15 02:11:35 +00:00
LeoVasanko 7633ee0d84 onlyoffice: bind temp file server to bridge IP, silence handler tracebacks, self-limit lifetime
The temporary HTTP server that OnlyOffice downloads the source document
from was bound to 0.0.0.0, so internet scanners could (and did) connect,
and socketserver dumped a full traceback to stderr for every dropped
connection. It also stayed up for the whole conversion attempt, leaving
the port exposed when conversions hang.

- Bind only to the callback host (oonet gateway by default) so the port
  is unreachable from the internet.
- Override handle_error to log at debug level instead of printing
  tracebacks.
- Watchdog shuts the server down ~2s after the file is fetched, or at
  max_lifetime (request_timeout + 30s), and the socket is closed with
  server_close() in the normal path.
2026-09-10 20:22:41 +00:00
LeoVasanko d4dfc57994 onlyoffice: hardcode oonet gateway as callback host, drop docker detection
The cista service account has no docker CLI access, so detecting the
gateway via docker network inspect silently fell back to docker0/legacy
behavior. With the pinned subnet the gateway is always 172.30.0.1;
ONLYOFFICE_CALLBACK_HOST remains as an env override.
2026-08-13 07:38:52 +00:00
LeoVasanko 65e2fddf1a onlyoffice: drop legacy localhost:8988 URL fallback
The container always lives on the isolated oonet network at the fixed
IP; falling back to a published localhost port silently masked broken
setups with confusing 'not reachable at localhost:8988' diagnostics.
2026-08-13 07:26:57 +00:00
LeoVasanko 4103b8928c onlyoffice: fixed container IP instead of published port; fetch-aware timeout
Docker silently discards published ports on internal networks, so the
isolated oonet setup left the container unreachable at localhost:8988.
Reach it at its fixed IP (172.30.0.2) on the bridge instead; the host
is the gateway, so this needs no port publishing at all. The oosetup
CLI drops its now-meaningless <port> argument.

Also: record whether OnlyOffice fetched the input file from the
temporary callback server and report it in the timeout error message
('input file never fetched' = network/callback failure, vs a stalled
conversion). Convert POST timeout is 7s (conversion runs inside the
request), result PNG download stays at 2s.
2026-08-13 06:51:31 +00:00
LeoVasanko 3c1894f337 onlyoffice: isolated docker network, pinned server sources, strict deadlines
- Dockerfile: pin ONLYOFFICE server sources to a known-good commit
  (build-arg ONLYOFFICE_SERVER_REF) instead of tracking master; master
  gained a 'memory runtime' branch that forks no converter workers on
  community edition, silently breaking all conversions. Also patch that
  branch out as defense in depth.
- setup_docker: run the container on an internal-only network (oonet,
  172.30.0.0/24) with no outbound internet; only host callback traffic
  is possible.
- _get_callback_host: resolve the oonet gateway via docker network
  inspect instead of assuming docker0; fall back to docker0.
- OOConversionManager: cancel the background conversion task when its
  last waiter is cancelled (strict preview deadline), releasing the
  semaphore slot and aborting the hung HTTP request; shield the shared
  future so one waiter's cancellation does not affect others.
- Split timeouts: convert POST 7s (conversion happens inside the
  request with async:false), result PNG download 2s.
2026-08-13 06:40:01 +00:00
LeoVasanko 7ac373179b Shorter short error messages. 2026-08-13 05:19:50 +00:00
LeoVasanko a99996bd9c Quiet expected preview failures; rename pyvips to vips
Worker logs expected PreviewError failures as warnings; tracebacks are
reserved for unanticipated internal errors. pdf read errors report
backend "pdf" (vips not yet reached); the stage field is dropped as the
backend name now encodes the failing pipeline step. Backend label
"pyvips" renamed to "vips" everywhere.
2026-08-13 04:46:08 +00:00
13 changed files with 353 additions and 116 deletions
+9 -15
View File
@@ -2,16 +2,16 @@
Usage: Usage:
mediapreview <path> [-o OUTPUT] [-q QUALITY] [--maxsize N] [--maxzoom Z] mediapreview <path> [-o OUTPUT] [-q QUALITY] [--maxsize N] [--maxzoom Z]
mediapreview oosetup [<name>] [<port>] mediapreview oosetup [<name>]
mediapreview (-h | --help) mediapreview (-h | --help)
Generate an AVIF preview for a media file (one-shot, in-process), or set up Generate an AVIF preview for a media file (one-shot, in-process), or set up
the bundled OnlyOffice container. the bundled OnlyOffice container (isolated network, reachable from the host
at its fixed container IP).
Arguments: Arguments:
<path> media file to preview <path> media file to preview
<name> container name [default: onlyoffice-mediapreview] <name> container name [default: onlyoffice-mediapreview]
<port> container host port [default: 8988]
Options: Options:
-o OUTPUT output .avif file (default: write AVIF bytes to stdout) -o OUTPUT output .avif file (default: write AVIF bytes to stdout)
@@ -32,18 +32,17 @@ from docopt import docopt
from mediapreview.backends import dispatch from mediapreview.backends import dispatch
from mediapreview.exceptions import PreviewError from mediapreview.exceptions import PreviewError
from mediapreview.util.logformat import EmojiFormatter from mediapreview.util.logformat import EmojiFormatter, quiet_vips_logging
def _configure_logging() -> None: def _configure_logging() -> None:
handler = logging.StreamHandler(sys.stderr) handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(EmojiFormatter()) handler.setFormatter(EmojiFormatter())
logging.basicConfig(level=logging.INFO, handlers=[handler]) logging.basicConfig(level=logging.INFO, handlers=[handler])
# pyvips is chatty at INFO ("threadpool completed ..." per operation). quiet_vips_logging()
logging.getLogger("pyvips").setLevel(logging.WARNING)
def _oosetup(name: str, port: int) -> None: def _oosetup(name: str) -> None:
try: try:
# Lazy import: keeps the base CLI free of office-extra concerns. # Lazy import: keeps the base CLI free of office-extra concerns.
from mediapreview.office import setup_docker # noqa: PLC0415 from mediapreview.office import setup_docker # noqa: PLC0415
@@ -52,7 +51,7 @@ def _oosetup(name: str, port: int) -> None:
sys.exit(1) sys.exit(1)
try: try:
# Logs go to stderr; stdout carries only the secret line below. # Logs go to stderr; stdout carries only the secret line below.
secret = setup_docker(name=name, port=port) secret = setup_docker(name=name)
except Exception as e: except Exception as e:
sys.stderr.write(f"error: OnlyOffice setup failed: {e}\n") sys.stderr.write(f"error: OnlyOffice setup failed: {e}\n")
sys.exit(1) sys.exit(1)
@@ -102,13 +101,8 @@ def main() -> None:
# by the <path> pattern if it came second. Dispatch it before parsing; # by the <path> pattern if it came second. Dispatch it before parsing;
# the main help above still documents both modes. # the main help above still documents both modes.
if sys.argv[1:2] == ["oosetup"]: if sys.argv[1:2] == ["oosetup"]:
args = docopt( args = docopt("Usage:\n mediapreview oosetup [<name>]", argv=sys.argv[1:])
"Usage:\n mediapreview oosetup [<name>] [<port>]", argv=sys.argv[1:] _oosetup(args["<name>"] or "onlyoffice-mediapreview")
)
_oosetup(
args["<name>"] or "onlyoffice-mediapreview",
int(args["<port>"] or 8988),
)
return return
_preview(docopt(__doc__)) _preview(docopt(__doc__))
+45 -6
View File
@@ -5,6 +5,7 @@ plus quality/size parameters and returning `(avif_bytes, PreviewResponse)`.
`dispatch` picks the right backend for a path. `dispatch` picks the right backend for a path.
""" """
import asyncio
import logging import logging
import mimetypes import mimetypes
@@ -16,7 +17,7 @@ from mediapreview.backends.image import (
from mediapreview.backends.pdf import process_pdf from mediapreview.backends.pdf import process_pdf
from mediapreview.backends.video import process_video from mediapreview.backends.video import process_video
from mediapreview.exceptions import PreviewError, backend_error from mediapreview.exceptions import PreviewError, backend_error
from mediapreview.formats import DOC_PREVIEW_SUFFIXES from mediapreview.formats import DOC_PREVIEW_SUFFIXES, OFFICE_PREVIEW_SUFFIXES
__all__ = [ __all__ = [
"dispatch", "dispatch",
@@ -34,7 +35,7 @@ def dispatch(path, quality, maxsize, maxzoom, data=None):
backend = "unknown" backend = "unknown"
try: try:
if data: if data:
backend = "pyvips" backend = "vips"
return process_image_buffer( return process_image_buffer(
data, quality=quality, maxsize=maxsize, maxzoom=maxzoom data, quality=quality, maxsize=maxsize, maxzoom=maxzoom
) )
@@ -42,16 +43,52 @@ def dispatch(path, quality, maxsize, maxzoom, data=None):
if suffix in DOC_PREVIEW_SUFFIXES: if suffix in DOC_PREVIEW_SUFFIXES:
backend = "pdf" backend = "pdf"
return process_pdf(path, quality=quality, maxsize=maxsize, maxzoom=maxzoom) return process_pdf(path, quality=quality, maxsize=maxsize, maxzoom=maxzoom)
if suffix in OFFICE_PREVIEW_SUFFIXES:
backend = "onlyoffice"
try:
from mediapreview.office import ( # noqa: PLC0415
close_oo_client,
get_oo_manager,
)
except ImportError as e:
raise ImportError(
"Office document previews require the 'office' extra:"
" pip install mediapreview[office]"
) from e
try:
asyncio.get_running_loop()
except RuntimeError:
pass # no event loop, asyncio.run() is safe
else:
raise RuntimeError(
"Office preview via dispatch() cannot be called inside a running"
" event loop; use mediapreview.pool.generate_office_preview() instead"
)
async def _convert_office() -> bytes:
manager = get_oo_manager()
try:
return await manager.convert(path)
finally:
await close_oo_client()
png_bytes = asyncio.run(_convert_office())
result, resp = process_image_buffer(
png_bytes, quality=quality, maxsize=maxsize, maxzoom=maxzoom
)
if resp is not None:
resp.backend = "onlyoffice+" + (resp.backend or "vips")
return result, resp
mime_type, _ = mimetypes.guess_type(path.name) mime_type, _ = mimetypes.guess_type(path.name)
if mime_type and mime_type.startswith("video/"): if mime_type and mime_type.startswith("video/"):
backend = "video" backend = "video"
return process_video(path, quality=quality, maxsize=maxsize) return process_video(path, quality=quality, maxsize=maxsize)
if mime_type and mime_type.startswith("image/"): if mime_type and mime_type.startswith("image/"):
backend = "pyvips" backend = "vips"
return process_image(path, quality=quality, maxsize=maxsize) return process_image(path, quality=quality, maxsize=maxsize)
except PreviewError: except PreviewError:
# Already structured (e.g. a stage of a combined pipeline like # Already structured (e.g. a failing stage of a combined pipeline
# pdf+pyvips) — keep the original backend/stage identity. # like pdf+vips) — keep the original backend identity.
raise raise
except ValueError as e: except ValueError as e:
raise backend_error(backend, str(e)) from e raise backend_error(backend, str(e)) from e
@@ -62,4 +99,6 @@ def dispatch(path, quality, maxsize, maxzoom, data=None):
except Exception as e: except Exception as e:
logger.exception("Preview dispatch failed for %s", path) logger.exception("Preview dispatch failed for %s", path)
raise backend_error(backend, str(e)) from e raise backend_error(backend, str(e)) from e
raise backend_error(backend, "preview unsupported") if not suffix:
raise backend_error(backend, "unknown file type: no file extension")
raise backend_error(backend, f"unknown file extension: {suffix!r}")
+2 -2
View File
@@ -148,7 +148,7 @@ def process_image_pyvips(path, *, maxsize, quality):
) )
except pyvips.error.Error as e: except pyvips.error.Error as e:
raise ValueError(f"cannot decode image: {e}") from e raise ValueError(f"cannot decode image: {e}") from e
backend = "pyvips" backend = "vips"
t_end = perf_counter() t_end = perf_counter()
return ret, PreviewResponse( return ret, PreviewResponse(
@@ -181,7 +181,7 @@ def process_image_buffer(data: bytes, *, quality, maxsize, maxzoom):
return ret, PreviewResponse( return ret, PreviewResponse(
ok=True, ok=True,
mime="image/avif", mime="image/avif",
backend="pyvips", backend="vips",
timings=[round((t_end - t_start) * 1000, 1)], timings=[round((t_end - t_start) * 1000, 1)],
width=orig_w, width=orig_w,
height=orig_h, height=orig_h,
+5 -4
View File
@@ -1,4 +1,4 @@
"""PDF/XPS/EPUB preview conversion via PyMuPDF + pyvips.""" """PDF/XPS/EPUB preview conversion via PyMuPDF + vips."""
from time import perf_counter from time import perf_counter
@@ -13,7 +13,7 @@ try:
except ImportError: # pragma: no cover - optional pdf extra except ImportError: # pragma: no cover - optional pdf extra
pymupdf = None pymupdf = None
BACKEND = "pdf+pyvips" BACKEND = "pdf+vips"
def process_pdf(path, *, maxsize, maxzoom, quality, page_number=0): def process_pdf(path, *, maxsize, maxzoom, quality, page_number=0):
@@ -31,7 +31,8 @@ def process_pdf(path, *, maxsize, maxzoom, quality, page_number=0):
pix = page.get_pixmap(matrix=mat) pix = page.get_pixmap(matrix=mat)
samples, width, height, n = pix.samples_mv, pix.width, pix.height, pix.n samples, width, height, n = pix.samples_mv, pix.width, pix.height, pix.n
except Exception as e: except Exception as e:
raise backend_error(BACKEND, str(e), stage="pdf") from e # vips was never reached — this is a plain pdf error.
raise backend_error("pdf", str(e)) from e
t_load_end = perf_counter() t_load_end = perf_counter()
t_save_start = perf_counter() t_save_start = perf_counter()
@@ -39,7 +40,7 @@ def process_pdf(path, *, maxsize, maxzoom, quality, page_number=0):
img = pyvips.Image.new_from_memory(samples, width, height, n, "uchar") img = pyvips.Image.new_from_memory(samples, width, height, n, "uchar")
ret = img.write_to_buffer(".avif", Q=quality, effort=AVIF_FAST_EFFORT, keep="none") ret = img.write_to_buffer(".avif", Q=quality, effort=AVIF_FAST_EFFORT, keep="none")
except Exception as e: except Exception as e:
raise backend_error(BACKEND, str(e), stage="pyvips") from e raise backend_error(BACKEND, str(e)) from e
t_save_end = perf_counter() t_save_end = perf_counter()
return ret, PreviewResponse( return ret, PreviewResponse(
+17 -4
View File
@@ -36,10 +36,17 @@ RUN apt-get update -qq && \
ca-certificates && \ ca-certificates && \
rm -rf /var/lib/apt/lists/* rm -rf /var/lib/apt/lists/*
# Clone the open-source server components (shallow, ~15 MB). # Pin the open-source server components to a known-good commit (~15 MB).
# The master branch is used because the Linux/web tags are not published # The master branch is a moving target (the Linux/web tags are not published
# in the server repo; the license.js file has been stable for years. # in the server repo): a 2026 convertermaster change there detects community
RUN git clone --depth 1 https://github.com/ONLYOFFICE/server.git /opt/oo-server # edition as a "memory runtime" and forks NO converter workers, silently
# breaking all conversions. Pinned to the commit proven in production.
ARG ONLYOFFICE_SERVER_REF=4e56b8d056640557ffcd8c860a65535ab6cbd95b
RUN git init /opt/oo-server && \
cd /opt/oo-server && \
git remote add origin https://github.com/ONLYOFFICE/server.git && \
git fetch --depth 1 origin "$ONLYOFFICE_SERVER_REF" && \
git checkout FETCH_HEAD
# Patch license.js so the converter worker count is read from an env var # Patch license.js so the converter worker count is read from an env var
# instead of being hardcoded to 1. # instead of being hardcoded to 1.
@@ -47,6 +54,12 @@ RUN sed -i \
's/count: 1,/count: parseInt(process.env.WORKERS, 10) || 8,/' \ 's/count: 1,/count: parseInt(process.env.WORKERS, 10) || 8,/' \
/opt/oo-server/Common/sources/license.js /opt/oo-server/Common/sources/license.js
# Defense in depth: never take the "memory runtime" branch that forks no
# converter workers, even if the pinned ref is bumped carelessly.
RUN sed -i \
's/runtimeProfile\.isMemoryRuntime()/false \/* patched: always fork converter workers *\//g' \
/opt/oo-server/FileConverter/sources/convertermaster.js
# Install npm dependencies for the modules the FileConverter touches. # Install npm dependencies for the modules the FileConverter touches.
# DocService deps are also needed because converter.js pulls in baseConnector. # DocService deps are also needed because converter.js pulls in baseConnector.
RUN cd /opt/oo-server/Common && npm ci --no-audit --no-fund RUN cd /opt/oo-server/Common && npm ci --no-audit --no-fund
+27 -19
View File
@@ -17,9 +17,10 @@ The hierarchy is intentionally small:
- ``OnlyOfficeError`` covers all OnlyOffice failures; optional fields - ``OnlyOfficeError`` covers all OnlyOffice failures; optional fields
(``code``, ``status``, ``url``, ``snippet``) describe the specific failure. (``code``, ``status``, ``url``, ``snippet``) describe the specific failure.
- ``PreviewBackendError`` covers backend conversion failures (ffmpeg, pyvips, - ``PreviewBackendError`` covers backend conversion failures (ffmpeg, vips,
pdf, etc.); ``stage`` identifies the failing step of a combined pipeline pdf, etc.). Combined pipelines report the failing step in ``backend``
(e.g. "pdf" vs "pyvips" in the "pdf+pyvips" backend). (e.g. "pdf" if pdf reading failed before vips was reached, "pdf+vips"
for a vips write failure).
- ``PreviewTimeoutError`` covers timeouts for any backend. - ``PreviewTimeoutError`` covers timeouts for any backend.
- ``PreviewCancelledError`` covers cancellations (e.g. pool shutdown). - ``PreviewCancelledError`` covers cancellations (e.g. pool shutdown).
@@ -80,17 +81,6 @@ class OnlyOfficeError(PreviewError):
class PreviewBackendError(PreviewError): class PreviewBackendError(PreviewError):
"""Backend conversion failure (image/video/pdf/etc).""" """Backend conversion failure (image/video/pdf/etc)."""
def __init__(
self,
message: str = "preview failed",
short: str = "error",
*,
stage: str | None = None,
backend: str | None = None,
):
super().__init__(message, short, backend=backend)
self.stage = stage
class PreviewTimeoutError(PreviewError): class PreviewTimeoutError(PreviewError):
"""Preview conversion exceeded its timeout for a given backend.""" """Preview conversion exceeded its timeout for a given backend."""
@@ -102,9 +92,11 @@ class PreviewTimeoutError(PreviewError):
*, *,
timeout_seconds: float = 0.0, timeout_seconds: float = 0.0,
backend: str | None = None, backend: str | None = None,
fetched: bool | None = None,
): ):
super().__init__(message, short, backend=backend) super().__init__(message, short, backend=backend)
self.timeout_seconds = timeout_seconds self.timeout_seconds = timeout_seconds
self.fetched = fetched
class PreviewCancelledError(PreviewError): class PreviewCancelledError(PreviewError):
@@ -163,22 +155,38 @@ def onlyoffice_no_fileurl_error(snippet: str | None = None) -> OnlyOfficeError:
return OnlyOfficeError(log, "no-fileurl error", snippet=snippet) return OnlyOfficeError(log, "no-fileurl error", snippet=snippet)
def backend_error(backend: str, message: str, *, stage: str | None = None) -> PreviewBackendError: def backend_error(backend: str, message: str) -> PreviewBackendError:
short = message.splitlines()[0][:60] short = message.splitlines()[0]
# Many backend messages look like "source: summary: detail ...".
# Drop the source prefix and any trailing detail so the short label
# is usable in UIs with limited space.
if ": " in short:
short = short.split(": ", 1)[1]
if ": " in short:
short = short.split(": ", 1)[0]
short = short[:60]
return PreviewBackendError( return PreviewBackendError(
f"[{backend}] preview failed: {message}", f"[{backend}] preview failed: {message}",
short, short,
backend=backend, backend=backend,
stage=stage,
) )
def preview_timeout_error(backend: str, timeout_seconds: float) -> PreviewTimeoutError: def preview_timeout_error(
backend: str, timeout_seconds: float, fetched: bool | None = None
) -> PreviewTimeoutError:
log = f"{backend.capitalize()} preview timed out after {timeout_seconds}s"
if fetched is not None:
# OnlyOffice: whether it ever downloaded the input file from our
# callback server distinguishes network/callback failures from a
# stalled conversion.
log += " (input file fetched)" if fetched else " (input file never fetched)"
return PreviewTimeoutError( return PreviewTimeoutError(
f"{backend.capitalize()} preview timed out after {timeout_seconds}s", log,
"timeout", "timeout",
backend=backend, backend=backend,
timeout_seconds=timeout_seconds, timeout_seconds=timeout_seconds,
fetched=fetched,
) )
+1 -1
View File
@@ -90,5 +90,5 @@ def expected_backend(path: Path) -> str:
if mime_type and mime_type.startswith("video/"): if mime_type and mime_type.startswith("video/"):
return "video" return "video"
if mime_type and mime_type.startswith("image/"): if mime_type and mime_type.startswith("image/"):
return "pyvips" return "vips"
return "preview" return "preview"
+152 -49
View File
@@ -47,6 +47,16 @@ except ImportError: # pragma: no cover - optional office extra
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# Isolated docker network for the OnlyOffice container: internal-only (no
# outbound internet), the container can only reach the host on this bridge.
# Docker discards published ports on internal networks, so the container is
# reached at its fixed IP instead of a published localhost port. The host is
# always the first address of the pinned subnet (the bridge gateway).
OO_NETWORK = "oonet"
OO_SUBNET = "172.30.0.0/24"
OO_GATEWAY = "172.30.0.1"
OO_CONTAINER_IP = "172.30.0.2"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Configuration helpers # Configuration helpers
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -57,9 +67,11 @@ _httpx_client_loop: asyncio.AbstractEventLoop | None = None
def _get_onlyoffice_url() -> str: def _get_onlyoffice_url() -> str:
# The container runs on the isolated oonet network at a fixed IP; the
# host is the bridge gateway and reaches it directly, no published port.
return os.environ.get( return os.environ.get(
"ONLYOFFICE_URL", "ONLYOFFICE_URL",
os.environ.get("ONLYOFFICE_CISTA_URL", "http://localhost:8988"), os.environ.get("ONLYOFFICE_CISTA_URL", f"http://{OO_CONTAINER_IP}"),
) )
@@ -69,26 +81,14 @@ def _get_jwt_secret() -> str:
@lru_cache(maxsize=1) @lru_cache(maxsize=1)
def _get_callback_host() -> str: def _get_callback_host() -> str:
"""Return the host IP that OnlyOffice (usually in Docker) can use to reach us.""" """Return the host IP that OnlyOffice (in Docker) can use to reach us.
The host is always the gateway of the pinned oonet subnet; no detection
is needed (the cista service account may not have docker CLI access).
"""
if host := os.environ.get("ONLYOFFICE_CALLBACK_HOST"): if host := os.environ.get("ONLYOFFICE_CALLBACK_HOST"):
return host return host
# Try to auto-detect docker bridge IP return OO_GATEWAY
try:
result = subprocess.run(
["/sbin/ip", "-4", "addr", "show", "docker0"],
capture_output=True,
text=True,
timeout=2,
check=False,
)
for line in result.stdout.splitlines():
if "inet " in line:
parts = line.strip().split()
addr_part = parts[1] # e.g. 172.17.0.1/16
return addr_part.split("/")[0]
except Exception:
logger.debug("Failed to auto-detect docker bridge IP")
return "127.0.0.1"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -156,12 +156,14 @@ def log_reachable_info() -> None:
logger.warning("OnlyOffice probe failed%s", suffix) logger.warning("OnlyOffice probe failed%s", suffix)
def setup_docker(name: str = "onlyoffice-mediapreview", port: int = 8988) -> str: def setup_docker(name: str = "onlyoffice-mediapreview") -> str:
"""Build and run the patched OnlyOffice Docker image. """Build and run the patched OnlyOffice Docker image.
Uses ONLYOFFICE_JWT_SECRET if set, otherwise generates a random secret. The container runs on an isolated internal network (OO_NETWORK) with no
Returns the secret used, so the caller is responsible for persisting it outbound internet and no published ports; the host reaches it at
(the CLI prints it as `ONLYOFFICE_JWT_SECRET=<token>`). OO_CONTAINER_IP. Uses ONLYOFFICE_JWT_SECRET if set, otherwise generates a
random secret. Returns the secret used, so the caller is responsible for
persisting it (the CLI prints it as `ONLYOFFICE_JWT_SECRET=<token>`).
The Docker build context ships inside the package at `mediapreview/docker`. The Docker build context ships inside the package at `mediapreview/docker`.
""" """
if secret := _get_jwt_secret(): if secret := _get_jwt_secret():
@@ -182,13 +184,33 @@ def setup_docker(name: str = "onlyoffice-mediapreview", port: int = 8988) -> str
if result.returncode != 0: if result.returncode != 0:
raise RuntimeError("Failed to build OnlyOffice image") raise RuntimeError("Failed to build OnlyOffice image")
# Isolated network: internal-only, so the container has no outbound
# internet access and can only reach the host on this bridge (needed
# for the preview file callback). Already-exists is fine.
net_cmd = [
"docker",
"network",
"create",
"--internal",
"--subnet",
OO_SUBNET,
OO_NETWORK,
]
result = subprocess.run(net_cmd, capture_output=True, check=False) # noqa: S603
if result.returncode != 0 and b"already exists" not in result.stderr:
raise RuntimeError(
f"Failed to create docker network {OO_NETWORK}: {result.stderr.decode(errors='replace').strip()}"
)
logger.info("Starting OnlyOffice container") logger.info("Starting OnlyOffice container")
run_cmd = [ run_cmd = [
"docker", "docker",
"run", "run",
"-d", "-d",
"-p", "--network",
f"{port}:80", OO_NETWORK,
"--ip",
OO_CONTAINER_IP,
"-e", "-e",
f"JWT_SECRET={secret}", f"JWT_SECRET={secret}",
"-e", "-e",
@@ -203,7 +225,10 @@ def setup_docker(name: str = "onlyoffice-mediapreview", port: int = 8988) -> str
result = subprocess.run(run_cmd, check=False, shell=False) # noqa: S603 result = subprocess.run(run_cmd, check=False, shell=False) # noqa: S603
if result.returncode != 0: if result.returncode != 0:
raise RuntimeError("Failed to start OnlyOffice container") raise RuntimeError("Failed to start OnlyOffice container")
logger.info("OnlyOffice is running on http://localhost:%d", port) # Docker discards published ports on internal networks, so the container
# is reached at its fixed IP; no localhost port is exposed.
logger.info("OnlyOffice is running on http://%s", OO_CONTAINER_IP)
logger.info("Callback host for file downloads: %s", OO_GATEWAY)
return secret return secret
@@ -258,29 +283,65 @@ async def is_available_cached() -> bool:
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
class _TempServer(socketserver.TCPServer):
"""TCPServer that logs handler errors instead of dumping tracebacks to stderr."""
daemon_threads = True
oo_fetched: bool
def handle_error(self, request, client_address) -> None: # noqa: ARG002
# Dropped connections (client disconnects mid-request, port scanners)
# are routine noise; socketserver's default prints a full traceback.
logger.debug("Temp file server: error from %s", client_address)
class _QuietHandler(SimpleHTTPRequestHandler): class _QuietHandler(SimpleHTTPRequestHandler):
server: _TempServer
def log_message(self, fmt, *args) -> None: def log_message(self, fmt, *args) -> None:
pass # Any request logged here means a client (OnlyOffice) connected to
# fetch the file; record it for timeout diagnostics.
self.server.oo_fetched = True
def _get_free_port() -> int: def _get_free_port(host: str) -> int:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(("0.0.0.0", 0)) # noqa: S104 s.bind((host, 0))
return s.getsockname()[1] return s.getsockname()[1]
def _serve_file_temporarily(file_path: Path): def _serve_file_temporarily(file_path: Path, max_lifetime: float = 60.0):
"""Start a temporary HTTP server for *file_path* and return (url, server).""" """Start a temporary HTTP server for *file_path* and return (url, server).
The server binds only to the callback host address (the docker bridge
gateway by default), not 0.0.0.0, so it is unreachable from the internet.
It shuts itself down shortly after the file has been fetched, or when
*max_lifetime* elapses, so a hung OnlyOffice request cannot leave the
port open indefinitely.
"""
directory = str(file_path.parent) directory = str(file_path.parent)
filename = file_path.name filename = file_path.name
port = _get_free_port() host = _get_callback_host()
port = _get_free_port(host)
handler = partial(_QuietHandler, directory=directory) handler = partial(_QuietHandler, directory=directory)
httpd = socketserver.TCPServer(("0.0.0.0", port), handler) # noqa: S104 httpd = _TempServer((host, port), handler)
httpd.oo_fetched = False
thread = threading.Thread(target=httpd.serve_forever, daemon=True) thread = threading.Thread(target=httpd.serve_forever, daemon=True)
thread.start() thread.start()
host = _get_callback_host() def _watchdog() -> None:
deadline = perf_counter() + max_lifetime
while perf_counter() < deadline and not httpd.oo_fetched:
threading.Event().wait(0.1)
if httpd.oo_fetched:
# Brief grace so the in-flight response finishes transferring.
threading.Event().wait(2.0)
httpd.shutdown()
httpd.server_close()
threading.Thread(target=_watchdog, daemon=True).start()
url = f"http://{host}:{port}/{quote(filename)}" url = f"http://{host}:{port}/{quote(filename)}"
return url, httpd return url, httpd
@@ -297,9 +358,14 @@ def _build_jwt_token(payload: dict) -> str | None:
return jwt.encode(payload, secret, algorithm="HS256") return jwt.encode(payload, secret, algorithm="HS256")
async def convert_to_png_async(file_path: Path, request_timeout: float = 5.0) -> bytes: async def convert_to_png_async(
file_path: Path, request_timeout: float = 7.0, download_timeout: float = 2.0
) -> bytes:
"""Convert *file_path* to PNG using OnlyOffice Document Server (async). """Convert *file_path* to PNG using OnlyOffice Document Server (async).
With ``async: false`` the conversion itself runs inside the POST request,
so *request_timeout* must cover full conversion time. *download_timeout*
covers fetching the resulting one-page PNG, which is pure transfer.
Returns the PNG bytes. Raises RuntimeError on failure. Returns the PNG bytes. Raises RuntimeError on failure.
""" """
if httpx is None or jwt is None: if httpx is None or jwt is None:
@@ -310,8 +376,12 @@ async def convert_to_png_async(file_path: Path, request_timeout: float = 5.0) ->
convert_url = f"{oo_url}/ConvertService.ashx" convert_url = f"{oo_url}/ConvertService.ashx"
client = get_httpx_client() client = get_httpx_client()
# Start temporary HTTP server so OnlyOffice can fetch the file # Start temporary HTTP server so OnlyOffice can fetch the file. The
doc_url, httpd = await asyncio.to_thread(_serve_file_temporarily, file_path) # watchdog lifetime covers the full conversion plus slack so a hung
# conversion cannot leave the port open forever.
doc_url, httpd = await asyncio.to_thread(
_serve_file_temporarily, file_path, request_timeout + 30.0
)
try: try:
suffix = file_path.suffix.lstrip(".").lower() suffix = file_path.suffix.lstrip(".").lower()
payload = { payload = {
@@ -340,7 +410,9 @@ async def convert_to_png_async(file_path: Path, request_timeout: float = 5.0) ->
) )
response.raise_for_status() response.raise_for_status()
except httpx.TimeoutException as e: except httpx.TimeoutException as e:
raise preview_timeout_error("onlyoffice", request_timeout) from e raise preview_timeout_error(
"onlyoffice", request_timeout, fetched=httpd.oo_fetched
) from e
except httpx.HTTPStatusError as e: except httpx.HTTPStatusError as e:
raise onlyoffice_http_error(e.response.status_code) from e raise onlyoffice_http_error(e.response.status_code) from e
except httpx.RequestError as e: except httpx.RequestError as e:
@@ -367,10 +439,10 @@ async def convert_to_png_async(file_path: Path, request_timeout: float = 5.0) ->
# Download converted PNG # Download converted PNG
try: try:
png_response = await client.get(file_url, timeout=request_timeout) png_response = await client.get(file_url, timeout=download_timeout)
png_response.raise_for_status() png_response.raise_for_status()
except httpx.TimeoutException as e: except httpx.TimeoutException as e:
raise preview_timeout_error("onlyoffice", request_timeout) from e raise preview_timeout_error("onlyoffice", download_timeout) from e
except httpx.HTTPStatusError as e: except httpx.HTTPStatusError as e:
raise onlyoffice_http_error(e.response.status_code) from e raise onlyoffice_http_error(e.response.status_code) from e
except httpx.RequestError as e: except httpx.RequestError as e:
@@ -380,6 +452,7 @@ async def convert_to_png_async(file_path: Path, request_timeout: float = 5.0) ->
return png_response.content return png_response.content
finally: finally:
await asyncio.to_thread(httpd.shutdown) await asyncio.to_thread(httpd.shutdown)
await asyncio.to_thread(httpd.server_close)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -391,12 +464,23 @@ async def convert_to_png_async(file_path: Path, request_timeout: float = 5.0) ->
OO_MAX_CONCURRENT = max(2, min(8, cpu_count())) OO_MAX_CONCURRENT = max(2, min(8, cpu_count()))
class _InFlight:
"""A deduplicated conversion: shared future, its task, and waiter count."""
__slots__ = ("future", "task", "waiters")
def __init__(self, future: asyncio.Future[bytes], task: asyncio.Task[None]):
self.future = future
self.task = task
self.waiters = 0
class OOConversionManager: class OOConversionManager:
"""Manages async OnlyOffice conversions with deduplication and concurrency limits.""" """Manages async OnlyOffice conversions with deduplication and concurrency limits."""
def __init__(self, max_concurrent: int = OO_MAX_CONCURRENT): def __init__(self, max_concurrent: int = OO_MAX_CONCURRENT):
self._semaphore = asyncio.Semaphore(max_concurrent) self._semaphore = asyncio.Semaphore(max_concurrent)
self._in_flight: dict[str, asyncio.Future[bytes]] = {} self._in_flight: dict[str, _InFlight] = {}
self._tasks: set[asyncio.Task[None]] = set() self._tasks: set[asyncio.Task[None]] = set()
self._lock = asyncio.Lock() self._lock = asyncio.Lock()
@@ -408,31 +492,50 @@ class OOConversionManager:
key = f"{filepath}:{stat.st_mtime_ns}" key = f"{filepath}:{stat.st_mtime_ns}"
async with self._lock: async with self._lock:
if key in self._in_flight: entry = self._in_flight.get(key)
future = self._in_flight[key] if entry is None:
else:
future = asyncio.get_running_loop().create_future() future = asyncio.get_running_loop().create_future()
self._in_flight[key] = future
task = asyncio.create_task(self._do_convert(filepath, key, future)) task = asyncio.create_task(self._do_convert(filepath, key, future))
self._tasks.add(task) self._tasks.add(task)
task.add_done_callback(self._tasks.discard) task.add_done_callback(self._tasks.discard)
entry = _InFlight(future, task)
self._in_flight[key] = entry
entry.waiters += 1
return await future try:
# shield: one waiter's cancellation must not cancel the future
# shared with other waiters.
return await asyncio.shield(entry.future)
except asyncio.CancelledError:
# The caller hit the (strict) preview deadline or disconnected.
# When no other waiter remains, cancel the background task so it
# releases its semaphore slot and aborts the HTTP request instead
# of running orphaned and piling load onto OnlyOffice.
async with self._lock:
entry.waiters -= 1
orphan = entry.waiters == 0
if orphan:
entry.task.cancel()
raise
async def _do_convert( async def _do_convert(
self, filepath: Path, key: str, future: asyncio.Future[bytes] self, filepath: Path, key: str, future: asyncio.Future[bytes]
) -> None: ) -> None:
try: try:
async with self._semaphore: async with self._semaphore:
png_bytes = await convert_to_png_async(filepath, request_timeout=5.0) png_bytes = await convert_to_png_async(filepath)
except asyncio.CancelledError:
# All waiters gave up; cancel the future so nothing hangs on it.
if not future.done():
future.cancel()
raise
except Exception as e: except Exception as e:
if not future.done(): if not future.done():
future.set_exception(e) future.set_exception(e)
async with self._lock:
self._in_flight.pop(key, None)
else: else:
if not future.done(): if not future.done():
future.set_result(png_bytes) future.set_result(png_bytes)
finally:
async with self._lock: async with self._lock:
self._in_flight.pop(key, None) self._in_flight.pop(key, None)
+1 -1
View File
@@ -477,7 +477,7 @@ async def generate_office_preview(
img, resp = await run_preview(filepath, quality, maxsize, maxzoom, data=png_bytes) img, resp = await run_preview(filepath, quality, maxsize, maxzoom, data=png_bytes)
if resp is not None: if resp is not None:
resp.backend = "onlyoffice+" + (resp.backend or "pyvips") resp.backend = "onlyoffice+" + (resp.backend or "vips")
if resp.timings: if resp.timings:
resp.timings = [round((t_oo_end - t_oo_start) * 1000, 1), *resp.timings] resp.timings = [round((t_oo_end - t_oo_start) * 1000, 1), *resp.timings]
return img, resp return img, resp
+15
View File
@@ -31,3 +31,18 @@ class EmojiFormatter(logging.Formatter):
def format(self, record: logging.LogRecord) -> str: def format(self, record: logging.LogRecord) -> str:
return format_level_prefix(record.levelno) + record.getMessage() return format_level_prefix(record.levelno) + record.getMessage()
def quiet_vips_logging() -> None:
"""Silence libvips per-operation chatter without hiding deprecations.
pyvips redirects every GLib message ("VIPS: threadpool completed ...")
onto the ``pyvips`` logger at INFO; cap that logger at WARNING. pyvips's
own diagnostics (e.g. deprecated-argument notices) are logged on the
``pyvips.voperation`` child logger and stay at the default INFO.
Opt-in: applications that want quiet vips output call this once during
their own logging setup. mediapreview never calls it on import.
"""
logging.getLogger("pyvips").setLevel(logging.WARNING)
logging.getLogger("pyvips.voperation").setLevel(logging.INFO)
+10 -4
View File
@@ -38,8 +38,9 @@ except ImportError: # pragma: no cover - optional worker extra
sys.exit(1) sys.exit(1)
from mediapreview.backends import dispatch from mediapreview.backends import dispatch
from mediapreview.exceptions import PreviewError
from mediapreview.protocol import PreviewRequest, PreviewResponse from mediapreview.protocol import PreviewRequest, PreviewResponse
from mediapreview.util.logformat import format_level_prefix from mediapreview.util.logformat import format_level_prefix, quiet_vips_logging
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -140,7 +141,13 @@ def _run_loop() -> None:
) )
_write_response(resp, result or b"") _write_response(resp, result or b"")
except Exception as e: except Exception as e:
logger.exception("Preview worker error for %s", req.path) # PreviewError is an expected failure (broken input, missing
# extra, backend error) — a warning suffices. Tracebacks are
# reserved for internal errors we did not anticipate.
if isinstance(e, PreviewError):
logger.warning("Preview failed for %s: %s", req.path, e)
else:
logger.exception("Preview worker error for %s", req.path)
captured = stderr_capture.getvalue().strip() captured = stderr_capture.getvalue().strip()
_write_response( _write_response(
PreviewResponse( PreviewResponse(
@@ -164,8 +171,7 @@ def main() -> None:
handler = logging.StreamHandler(sys.stderr) handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(_WorkerLogFormatter()) handler.setFormatter(_WorkerLogFormatter())
logging.basicConfig(level=logging.INFO, handlers=[handler]) logging.basicConfig(level=logging.INFO, handlers=[handler])
# pyvips is chatty at INFO ("threadpool completed ..." per operation). quiet_vips_logging()
logging.getLogger("pyvips").setLevel(logging.WARNING)
# NOTE: standalone package no longer depends on cista config loading. # NOTE: standalone package no longer depends on cista config loading.
# Consumers can load their own configuration before starting workers. # Consumers can load their own configuration before starting workers.
if len(sys.argv) > 1: if len(sys.argv) > 1:
+17 -5
View File
@@ -64,15 +64,23 @@ def test_onlyoffice_no_fileurl_error():
assert err.short == "no-fileurl error" assert err.short == "no-fileurl error"
def test_backend_error_stage(): def test_backend_error_pipeline_backend():
"""Combined pipelines tag the failing stage.""" """Combined pipelines report the failing step in the backend name."""
err = backend_error("pdf+pyvips", "cannot read document", stage="pdf") err = backend_error("pdf", "cannot read document")
assert err.backend == "pdf+pyvips" assert err.backend == "pdf"
assert err.stage == "pdf"
assert err.short == "cannot read document" assert err.short == "cannot read document"
assert isinstance(err, PreviewBackendError) assert isinstance(err, PreviewBackendError)
def test_backend_error_short_message_strips_source_and_detail():
"""Backend messages like "source: summary: detail" become just the summary."""
err = backend_error(
"vips",
"pyvips: cannot decode image: unable to load from file b'/mnt/c/Users...",
)
assert err.short == "cannot decode image"
def test_error_pickle_round_trip(): def test_error_pickle_round_trip():
"""Exceptions survive pickling (the worker pool wire) intact.""" """Exceptions survive pickling (the worker pool wire) intact."""
err = onlyoffice_error_from_code("-8") err = onlyoffice_error_from_code("-8")
@@ -90,7 +98,11 @@ async def test_generate_office_preview_raises_structured_error(monkeypatch):
async def fake_convert(_filepath: Path, request_timeout: float = 5.0) -> bytes: async def fake_convert(_filepath: Path, request_timeout: float = 5.0) -> bytes:
raise onlyoffice_error_from_code("-8") raise onlyoffice_error_from_code("-8")
async def fake_available() -> bool:
return True
monkeypatch.setattr(office, "convert_to_png_async", fake_convert) monkeypatch.setattr(office, "convert_to_png_async", fake_convert)
monkeypatch.setattr(office, "is_available_cached", fake_available)
with pytest.raises(OnlyOfficeError) as exc_info: with pytest.raises(OnlyOfficeError) as exc_info:
await generate_office_preview( await generate_office_preview(
+52 -6
View File
@@ -28,6 +28,7 @@ from mediapreview.backends.image import (
) )
from mediapreview.backends.pdf import process_pdf from mediapreview.backends.pdf import process_pdf
from mediapreview.backends.video import process_video from mediapreview.backends.video import process_video
from mediapreview.exceptions import PreviewBackendError
from mediapreview.office import is_available_async from mediapreview.office import is_available_async
from mediapreview.pool import generate_office_preview from mediapreview.pool import generate_office_preview
@@ -56,7 +57,7 @@ def _assert_ok(data, resp, backend: str | None = None) -> None:
def test_process_image_exif_orientations(path: Path) -> None: def test_process_image_exif_orientations(path: Path) -> None:
"""Every EXIF orientation fixture must produce a valid preview.""" """Every EXIF orientation fixture must produce a valid preview."""
data, resp = process_image(path, maxsize=512, quality=60) data, resp = process_image(path, maxsize=512, quality=60)
_assert_ok(data, resp, backend="pyvips") _assert_ok(data, resp, backend="vips")
assert resp.width in (1200, 1800) assert resp.width in (1200, 1800)
assert resp.height in (1200, 1800) assert resp.height in (1200, 1800)
@@ -74,7 +75,7 @@ def test_process_image_pyvips() -> None:
"""The pyvips-only image backend works on a plain JPEG.""" """The pyvips-only image backend works on a plain JPEG."""
path = FILES / "Landscape_1.jpg" path = FILES / "Landscape_1.jpg"
data, resp = process_image_pyvips(path, maxsize=512, quality=60) data, resp = process_image_pyvips(path, maxsize=512, quality=60)
_assert_ok(data, resp, backend="pyvips") _assert_ok(data, resp, backend="vips")
def test_process_image_buffer() -> None: def test_process_image_buffer() -> None:
@@ -83,7 +84,7 @@ def test_process_image_buffer() -> None:
data, resp = process_image_buffer( data, resp = process_image_buffer(
path.read_bytes(), maxsize=512, quality=60, maxzoom=2.0 path.read_bytes(), maxsize=512, quality=60, maxzoom=2.0
) )
_assert_ok(data, resp, backend="pyvips") _assert_ok(data, resp, backend="vips")
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -127,7 +128,7 @@ def test_process_pdf() -> None:
maxzoom=2.0, maxzoom=2.0,
quality=60, quality=60,
) )
_assert_ok(data, resp, backend="pdf+pyvips") _assert_ok(data, resp, backend="pdf+vips")
assert resp.width == 595 assert resp.width == 595
assert resp.height == 842 assert resp.height == 842
@@ -138,9 +139,9 @@ def test_process_pdf() -> None:
DISPATCH_FIXTURES = [ DISPATCH_FIXTURES = [
("Landscape_1.jpg", "pyvips", 1800, 1200), ("Landscape_1.jpg", "vips", 1800, 1200),
("sample-1mb.mp4", "video", 854, 480), ("sample-1mb.mp4", "video", 854, 480),
("sample.pdf", "pdf+pyvips", 595, 842), ("sample.pdf", "pdf+vips", 595, 842),
] ]
@@ -159,6 +160,51 @@ def test_dispatch(
assert resp.height == expected_height assert resp.height == expected_height
def test_dispatch_office(monkeypatch) -> None:
"""dispatch() converts office documents via OnlyOffice when called directly."""
fake_png = (FILES / "Landscape_1.jpg").read_bytes()
class _FakeManager:
async def convert(self, filepath: Path) -> bytes:
assert filepath == FILES / "file-sample_100kB.docx"
return fake_png
async def _noop() -> None:
return None
monkeypatch.setattr("mediapreview.office.get_oo_manager", _FakeManager)
monkeypatch.setattr("mediapreview.office.close_oo_client", _noop)
data, resp = dispatch(
FILES / "file-sample_100kB.docx",
quality=60,
maxsize=512,
maxzoom=2.0,
)
_assert_ok(data, resp)
assert resp.backend == "onlyoffice+vips"
def test_dispatch_unknown_extension(tmp_path: Path) -> None:
"""Unsupported extensions produce a diagnostic naming the extension."""
path = tmp_path / "unknown-file.xyz"
path.write_text("not a previewable file")
with pytest.raises(PreviewBackendError) as exc_info:
dispatch(path, quality=60, maxsize=512, maxzoom=2.0)
assert "unknown file extension: '.xyz'" in str(exc_info.value)
assert exc_info.value.backend == "unknown"
def test_dispatch_no_extension(tmp_path: Path) -> None:
"""Files without an extension produce a diagnostic saying so."""
path = tmp_path / "unknown-file-no-ext"
path.write_text("not a previewable file")
with pytest.raises(PreviewBackendError) as exc_info:
dispatch(path, quality=60, maxsize=512, maxzoom=2.0)
assert "unknown file type: no file extension" in str(exc_info.value)
assert exc_info.value.backend == "unknown"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Office previews via OnlyOffice # Office previews via OnlyOffice
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------