sanic/tests/test_middleware.py

324 lines
7.9 KiB
Python
Raw Normal View History

import logging
from asyncio import CancelledError
Streaming Server (#1876) * Streaming request by async for. * Make all requests streaming and preload body for non-streaming handlers. * Cleanup of code and avoid mixing streaming responses. * Async http protocol loop. * Change of test: don't require early bad request error but only after CRLF-CRLF. * Add back streaming requests. * Rewritten request body parser. * Misc. cleanup, down to 4 failing tests. * All tests OK. * Entirely remove request body queue. * Let black f*ckup the layout * Better testing error messages on protocol errors. * Remove StreamBuffer tests because the type is about to be removed. * Remove tests using the deprecated get_headers function that can no longer be supported. Chunked mode is now autodetected, so do not put content-length header if chunked mode is preferred. * Major refactoring of HTTP protocol handling (new module http.py added), all requests made streaming. A few compatibility issues and a lot of cleanup to be done remain, 16 tests failing. * Terminate check_timeouts once connection_task finishes. * Code cleanup, 14 tests failing. * Much cleanup, 12 failing... * Even more cleanup and error checking, 8 failing tests. * Remove keep-alive header from responses. First of all, it should say timeout=<value> which wasn't the case with existing implementation, and secondly none of the other web servers I tried include this header. * Everything but CustomServer OK. * Linter * Disable custom protocol test * Remove unnecessary variables, optimise performance. * A test was missing that body_init/body_push/body_finish are never called. Rewritten using receive_body and case switching to make it fail if bypassed. * Minor fixes. * Remove unused code. * Py 3.8 check for deprecated loop argument. * Fix a middleware cancellation handling test with py38. * Linter 'n fixes * Typing * Stricter handling of request header size * More specific error messages on Payload Too Large. * Init http.response = None * Messages further tuned. * Always try to consume request body, plus minor cleanup. * Add a missing check in case of close_if_idle on a dead connection. * Avoid error messages on PayloadTooLarge. * Add test for new API. * json takes str, not bytes * Default to no maximum request size for streaming handlers. * Fix chunked mode crash. * Header values should be strictly ASCII but both UTF-8 and Latin-1 exist. Use UTF-8B to cope with all. * Refactoring and cleanup. * Unify response header processing of ASGI and asyncio modes. * Avoid special handling of StreamingHTTPResponse. * 35 % speedup in HTTP/1.1 response formatting (not so much overall effect). * Duplicate set-cookie headers were being produced. * Cleanup processed_headers some more. * Linting * Import ordering * Response middleware ran by async request.respond(). * Need to check if transport is closing to avoid getting stuck in sending loops after peer has disconnected. * Middleware and error handling refactoring. * Linter * Fix tracking of HTTP stage when writing to transport fails. * Add clarifying comment * Add a check for request body functions and a test for NotImplementedError. * Linter and typing * These must be tuples + hack mypy warnings away. * New streaming test and minor fixes. * Constant receive buffer size. * 256 KiB send and receive buffers. * Revert "256 KiB send and receive buffers." This reverts commit abc1e3edb21a5e6925fa4c856657559608a8d65b. * app.handle_exception already sends the response. * Improved handling of errors during request. * An odd hack to avoid an httpx limitation that causes test failures. * Limit request header size to 8 KiB at most. * Remove unnecessary use of format string. * Cleanup tests * Remove artifact * Fix type checking * Mark test for skipping * Cleanup some edge cases * Add ignore_body flag to safe methods * Add unit tests for timeout logic * Add unit tests for timeout logic * Fix Mock usage in timeout test * Change logging test to only logger in handler * Windows py3.8 logging issue with current testing client * Add test_header_size_exceeded * Resolve merge conflicts * Add request middleware to hard exception handling * Add request middleware to hard exception handling * Request middleware on exception handlers * Linting * Cleanup deprecations Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com> Co-authored-by: Adam Hopkins <admhpkns@gmail.com>
2021-01-10 22:45:36 +00:00
from itertools import count
2021-04-05 16:01:48 +01:00
from sanic.exceptions import NotFound
from sanic.request import Request
from sanic.response import HTTPResponse, json, text
# ------------------------------------------------------------ #
# GET
# ------------------------------------------------------------ #
2018-12-30 11:18:06 +00:00
2018-08-26 15:43:14 +01:00
def test_middleware_request(app):
results = []
@app.middleware
2018-10-22 21:25:38 +01:00
async def handler1(request):
results.append(request)
2018-12-30 11:18:06 +00:00
@app.route("/")
2018-10-22 21:25:38 +01:00
async def handler2(request):
2018-12-30 11:18:06 +00:00
return text("OK")
2018-12-30 11:18:06 +00:00
request, response = app.test_client.get("/")
2018-12-30 11:18:06 +00:00
assert response.text == "OK"
assert type(results[0]) is Request
2021-02-15 19:50:20 +00:00
def test_middleware_request_as_convenience(app):
results = []
@app.on_request
async def handler1(request):
results.append(request)
@app.on_request()
2021-02-15 19:50:20 +00:00
async def handler2(request):
results.append(request)
@app.route("/")
async def handler3(request):
2021-02-15 19:50:20 +00:00
return text("OK")
request, response = app.test_client.get("/")
assert response.text == "OK"
assert type(results[0]) is Request
assert type(results[1]) is Request
2021-02-15 19:50:20 +00:00
2018-08-26 15:43:14 +01:00
def test_middleware_response(app):
results = []
2018-12-30 11:18:06 +00:00
@app.middleware("request")
2018-10-22 21:25:38 +01:00
async def process_request(request):
results.append(request)
2018-12-30 11:18:06 +00:00
@app.middleware("response")
async def process_response(request, response):
results.append(request)
results.append(response)
2018-12-30 11:18:06 +00:00
@app.route("/")
async def handler(request):
2018-12-30 11:18:06 +00:00
return text("OK")
2018-12-30 11:18:06 +00:00
request, response = app.test_client.get("/")
2018-12-30 11:18:06 +00:00
assert response.text == "OK"
assert type(results[0]) is Request
2021-02-15 19:50:20 +00:00
assert type(results[1]) is Request
assert isinstance(results[2], HTTPResponse)
def test_middleware_response_as_convenience(app):
results = []
@app.on_request
async def process_request(request):
results.append(request)
@app.on_response
async def process_response_1(request, response):
results.append(request)
results.append(response)
@app.on_response()
async def process_response_2(request, response):
2021-02-15 19:50:20 +00:00
results.append(request)
results.append(response)
@app.route("/")
async def handler(request):
return text("OK")
request, response = app.test_client.get("/")
assert response.text == "OK"
assert type(results[0]) is Request
assert type(results[1]) is Request
assert isinstance(results[2], HTTPResponse)
assert type(results[3]) is Request
assert isinstance(results[4], HTTPResponse)
2021-02-15 19:50:20 +00:00
def test_middleware_response_as_convenience_called(app):
results = []
@app.on_request()
async def process_request(request):
results.append(request)
@app.on_response()
async def process_response(request, response):
results.append(request)
results.append(response)
@app.route("/")
async def handler(request):
return text("OK")
request, response = app.test_client.get("/")
assert response.text == "OK"
assert type(results[0]) is Request
assert type(results[1]) is Request
assert isinstance(results[2], HTTPResponse)
2018-08-26 15:43:14 +01:00
def test_middleware_response_exception(app):
Streaming Server (#1876) * Streaming request by async for. * Make all requests streaming and preload body for non-streaming handlers. * Cleanup of code and avoid mixing streaming responses. * Async http protocol loop. * Change of test: don't require early bad request error but only after CRLF-CRLF. * Add back streaming requests. * Rewritten request body parser. * Misc. cleanup, down to 4 failing tests. * All tests OK. * Entirely remove request body queue. * Let black f*ckup the layout * Better testing error messages on protocol errors. * Remove StreamBuffer tests because the type is about to be removed. * Remove tests using the deprecated get_headers function that can no longer be supported. Chunked mode is now autodetected, so do not put content-length header if chunked mode is preferred. * Major refactoring of HTTP protocol handling (new module http.py added), all requests made streaming. A few compatibility issues and a lot of cleanup to be done remain, 16 tests failing. * Terminate check_timeouts once connection_task finishes. * Code cleanup, 14 tests failing. * Much cleanup, 12 failing... * Even more cleanup and error checking, 8 failing tests. * Remove keep-alive header from responses. First of all, it should say timeout=<value> which wasn't the case with existing implementation, and secondly none of the other web servers I tried include this header. * Everything but CustomServer OK. * Linter * Disable custom protocol test * Remove unnecessary variables, optimise performance. * A test was missing that body_init/body_push/body_finish are never called. Rewritten using receive_body and case switching to make it fail if bypassed. * Minor fixes. * Remove unused code. * Py 3.8 check for deprecated loop argument. * Fix a middleware cancellation handling test with py38. * Linter 'n fixes * Typing * Stricter handling of request header size * More specific error messages on Payload Too Large. * Init http.response = None * Messages further tuned. * Always try to consume request body, plus minor cleanup. * Add a missing check in case of close_if_idle on a dead connection. * Avoid error messages on PayloadTooLarge. * Add test for new API. * json takes str, not bytes * Default to no maximum request size for streaming handlers. * Fix chunked mode crash. * Header values should be strictly ASCII but both UTF-8 and Latin-1 exist. Use UTF-8B to cope with all. * Refactoring and cleanup. * Unify response header processing of ASGI and asyncio modes. * Avoid special handling of StreamingHTTPResponse. * 35 % speedup in HTTP/1.1 response formatting (not so much overall effect). * Duplicate set-cookie headers were being produced. * Cleanup processed_headers some more. * Linting * Import ordering * Response middleware ran by async request.respond(). * Need to check if transport is closing to avoid getting stuck in sending loops after peer has disconnected. * Middleware and error handling refactoring. * Linter * Fix tracking of HTTP stage when writing to transport fails. * Add clarifying comment * Add a check for request body functions and a test for NotImplementedError. * Linter and typing * These must be tuples + hack mypy warnings away. * New streaming test and minor fixes. * Constant receive buffer size. * 256 KiB send and receive buffers. * Revert "256 KiB send and receive buffers." This reverts commit abc1e3edb21a5e6925fa4c856657559608a8d65b. * app.handle_exception already sends the response. * Improved handling of errors during request. * An odd hack to avoid an httpx limitation that causes test failures. * Limit request header size to 8 KiB at most. * Remove unnecessary use of format string. * Cleanup tests * Remove artifact * Fix type checking * Mark test for skipping * Cleanup some edge cases * Add ignore_body flag to safe methods * Add unit tests for timeout logic * Add unit tests for timeout logic * Fix Mock usage in timeout test * Change logging test to only logger in handler * Windows py3.8 logging issue with current testing client * Add test_header_size_exceeded * Resolve merge conflicts * Add request middleware to hard exception handling * Add request middleware to hard exception handling * Request middleware on exception handlers * Linting * Cleanup deprecations Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com> Co-authored-by: Adam Hopkins <admhpkns@gmail.com>
2021-01-10 22:45:36 +00:00
result = {"status_code": "middleware not run"}
2018-12-30 11:18:06 +00:00
@app.middleware("response")
2017-11-15 15:46:39 +00:00
async def process_response(request, response):
2018-12-30 11:18:06 +00:00
result["status_code"] = response.status
return response
@app.exception(NotFound)
async def error_handler(request, exception):
2018-12-30 11:18:06 +00:00
return text("OK", exception.status_code)
2018-12-30 11:18:06 +00:00
@app.route("/")
async def handler(request):
2018-12-30 11:18:06 +00:00
return text("FAIL")
2018-12-30 11:18:06 +00:00
request, response = app.test_client.get("/page_not_found")
assert response.text == "OK"
assert result["status_code"] == 404
2018-10-22 21:25:38 +01:00
def test_middleware_response_raise_cancelled_error(app, caplog):
app.config.RESPONSE_TIMEOUT = 1
2018-12-30 11:18:06 +00:00
@app.middleware("response")
async def process_response(request, response):
2018-12-30 11:18:06 +00:00
raise CancelledError("CancelledError at response middleware")
2018-12-30 11:18:06 +00:00
@app.get("/")
def handler(request):
2018-12-30 11:18:06 +00:00
return text("OK")
with caplog.at_level(logging.ERROR):
2018-12-30 11:18:06 +00:00
reqrequest, response = app.test_client.get("/")
2022-09-19 14:04:09 +01:00
assert response.status == 500
assert (
"sanic.error",
logging.ERROR,
"Exception occurred while handling uri: 'http://127.0.0.1:42101/'",
) not in caplog.record_tuples
def test_middleware_response_raise_exception(app, caplog):
2018-12-30 11:18:06 +00:00
@app.middleware("response")
async def process_response(request, response):
2018-12-30 11:18:06 +00:00
raise Exception("Exception at response middleware")
2021-02-08 10:18:29 +00:00
app.route("/")(lambda x: x)
with caplog.at_level(logging.ERROR):
reqrequest, response = app.test_client.get("/fail")
assert response.status == 404
# 404 errors are not logged
assert (
"sanic.error",
logging.ERROR,
2018-12-30 11:18:06 +00:00
"Exception occurred while handling uri: 'http://127.0.0.1:42101/'",
) not in caplog.record_tuples
# Middleware exception ignored but logged
assert (
2018-12-30 11:18:06 +00:00
"sanic.error",
logging.ERROR,
2018-12-30 11:18:06 +00:00
"Exception occurred in one of response middleware handlers",
) in caplog.record_tuples
2018-08-26 15:43:14 +01:00
def test_middleware_override_request(app):
@app.middleware
async def halt_request(request):
2018-12-30 11:18:06 +00:00
return text("OK")
2018-12-30 11:18:06 +00:00
@app.route("/")
async def handler(request):
2018-12-30 11:18:06 +00:00
return text("FAIL")
2021-02-08 10:18:29 +00:00
_, response = app.test_client.get("/", gather_request=False)
assert response.status == 200
2018-12-30 11:18:06 +00:00
assert response.text == "OK"
2018-08-26 15:43:14 +01:00
def test_middleware_override_response(app):
2018-12-30 11:18:06 +00:00
@app.middleware("response")
async def process_response(request, response):
2018-12-30 11:18:06 +00:00
return text("OK")
2018-12-30 11:18:06 +00:00
@app.route("/")
async def handler(request):
2018-12-30 11:18:06 +00:00
return text("FAIL")
2018-12-30 11:18:06 +00:00
request, response = app.test_client.get("/")
assert response.status == 200
2018-12-30 11:18:06 +00:00
assert response.text == "OK"
2018-08-26 15:43:14 +01:00
def test_middleware_order(app):
order = []
2018-12-30 11:18:06 +00:00
@app.middleware("request")
async def request1(request):
order.append(1)
2018-12-30 11:18:06 +00:00
@app.middleware("request")
async def request2(request):
order.append(2)
2018-12-30 11:18:06 +00:00
@app.middleware("request")
async def request3(request):
order.append(3)
2018-12-30 11:18:06 +00:00
@app.middleware("response")
async def response1(request, response):
order.append(6)
2018-12-30 11:18:06 +00:00
@app.middleware("response")
async def response2(request, response):
order.append(5)
2018-12-30 11:18:06 +00:00
@app.middleware("response")
async def response3(request, response):
order.append(4)
2018-12-30 11:18:06 +00:00
@app.route("/")
async def handler(request):
2018-12-30 11:18:06 +00:00
return text("OK")
2018-12-30 11:18:06 +00:00
request, response = app.test_client.get("/")
assert response.status == 200
2018-10-22 21:25:38 +01:00
assert order == [1, 2, 3, 4, 5, 6]
Streaming Server (#1876) * Streaming request by async for. * Make all requests streaming and preload body for non-streaming handlers. * Cleanup of code and avoid mixing streaming responses. * Async http protocol loop. * Change of test: don't require early bad request error but only after CRLF-CRLF. * Add back streaming requests. * Rewritten request body parser. * Misc. cleanup, down to 4 failing tests. * All tests OK. * Entirely remove request body queue. * Let black f*ckup the layout * Better testing error messages on protocol errors. * Remove StreamBuffer tests because the type is about to be removed. * Remove tests using the deprecated get_headers function that can no longer be supported. Chunked mode is now autodetected, so do not put content-length header if chunked mode is preferred. * Major refactoring of HTTP protocol handling (new module http.py added), all requests made streaming. A few compatibility issues and a lot of cleanup to be done remain, 16 tests failing. * Terminate check_timeouts once connection_task finishes. * Code cleanup, 14 tests failing. * Much cleanup, 12 failing... * Even more cleanup and error checking, 8 failing tests. * Remove keep-alive header from responses. First of all, it should say timeout=<value> which wasn't the case with existing implementation, and secondly none of the other web servers I tried include this header. * Everything but CustomServer OK. * Linter * Disable custom protocol test * Remove unnecessary variables, optimise performance. * A test was missing that body_init/body_push/body_finish are never called. Rewritten using receive_body and case switching to make it fail if bypassed. * Minor fixes. * Remove unused code. * Py 3.8 check for deprecated loop argument. * Fix a middleware cancellation handling test with py38. * Linter 'n fixes * Typing * Stricter handling of request header size * More specific error messages on Payload Too Large. * Init http.response = None * Messages further tuned. * Always try to consume request body, plus minor cleanup. * Add a missing check in case of close_if_idle on a dead connection. * Avoid error messages on PayloadTooLarge. * Add test for new API. * json takes str, not bytes * Default to no maximum request size for streaming handlers. * Fix chunked mode crash. * Header values should be strictly ASCII but both UTF-8 and Latin-1 exist. Use UTF-8B to cope with all. * Refactoring and cleanup. * Unify response header processing of ASGI and asyncio modes. * Avoid special handling of StreamingHTTPResponse. * 35 % speedup in HTTP/1.1 response formatting (not so much overall effect). * Duplicate set-cookie headers were being produced. * Cleanup processed_headers some more. * Linting * Import ordering * Response middleware ran by async request.respond(). * Need to check if transport is closing to avoid getting stuck in sending loops after peer has disconnected. * Middleware and error handling refactoring. * Linter * Fix tracking of HTTP stage when writing to transport fails. * Add clarifying comment * Add a check for request body functions and a test for NotImplementedError. * Linter and typing * These must be tuples + hack mypy warnings away. * New streaming test and minor fixes. * Constant receive buffer size. * 256 KiB send and receive buffers. * Revert "256 KiB send and receive buffers." This reverts commit abc1e3edb21a5e6925fa4c856657559608a8d65b. * app.handle_exception already sends the response. * Improved handling of errors during request. * An odd hack to avoid an httpx limitation that causes test failures. * Limit request header size to 8 KiB at most. * Remove unnecessary use of format string. * Cleanup tests * Remove artifact * Fix type checking * Mark test for skipping * Cleanup some edge cases * Add ignore_body flag to safe methods * Add unit tests for timeout logic * Add unit tests for timeout logic * Fix Mock usage in timeout test * Change logging test to only logger in handler * Windows py3.8 logging issue with current testing client * Add test_header_size_exceeded * Resolve merge conflicts * Add request middleware to hard exception handling * Add request middleware to hard exception handling * Request middleware on exception handlers * Linting * Cleanup deprecations Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com> Co-authored-by: Adam Hopkins <admhpkns@gmail.com>
2021-01-10 22:45:36 +00:00
def test_request_middleware_executes_once(app):
i = count()
@app.middleware("request")
async def inc(request):
nonlocal i
next(i)
@app.route("/")
async def handler(request):
await request.app._run_request_middleware(request)
return text("OK")
request, response = app.test_client.get("/")
assert next(i) == 1
request, response = app.test_client.get("/")
assert next(i) == 3
def test_middleware_added_response(app):
@app.on_response
def display(_, response):
response["foo"] = "bar"
return json(response)
@app.get("/")
async def handler(request):
return {}
_, response = app.test_client.get("/")
assert response.json["foo"] == "bar"
def test_middleware_return_response(app):
response_middleware_run_count = 0
request_middleware_run_count = 0
@app.on_response
def response(_, response):
nonlocal response_middleware_run_count
response_middleware_run_count += 1
@app.on_request
def request(_):
nonlocal request_middleware_run_count
request_middleware_run_count += 1
@app.get("/")
async def handler(request):
resp1 = await request.respond()
return resp1
_, response = app.test_client.get("/")
assert response_middleware_run_count == 1
assert request_middleware_run_count == 1