* Allow Pathlib Path objects to be passed to the app.static file endpoint register helper.
* fixed import sort
* Raise error if static file path is not an accepted object type
Added more tests to improve coverage on the new type checks.
* Streaming request by async for.
* Make all requests streaming and preload body for non-streaming handlers.
* Cleanup of code and avoid mixing streaming responses.
* Async http protocol loop.
* Change of test: don't require early bad request error but only after CRLF-CRLF.
* Add back streaming requests.
* Rewritten request body parser.
* Misc. cleanup, down to 4 failing tests.
* All tests OK.
* Entirely remove request body queue.
* Let black f*ckup the layout
* Better testing error messages on protocol errors.
* Remove StreamBuffer tests because the type is about to be removed.
* Remove tests using the deprecated get_headers function that can no longer be supported. Chunked mode is now autodetected, so do not put content-length header if chunked mode is preferred.
* Major refactoring of HTTP protocol handling (new module http.py added), all requests made streaming. A few compatibility issues and a lot of cleanup to be done remain, 16 tests failing.
* Terminate check_timeouts once connection_task finishes.
* Code cleanup, 14 tests failing.
* Much cleanup, 12 failing...
* Even more cleanup and error checking, 8 failing tests.
* Remove keep-alive header from responses. First of all, it should say timeout=<value> which wasn't the case with existing implementation, and secondly none of the other web servers I tried include this header.
* Everything but CustomServer OK.
* Linter
* Disable custom protocol test
* Remove unnecessary variables, optimise performance.
* A test was missing that body_init/body_push/body_finish are never called. Rewritten using receive_body and case switching to make it fail if bypassed.
* Minor fixes.
* Remove unused code.
* Py 3.8 check for deprecated loop argument.
* Fix a middleware cancellation handling test with py38.
* Linter 'n fixes
* Typing
* Stricter handling of request header size
* More specific error messages on Payload Too Large.
* Init http.response = None
* Messages further tuned.
* Always try to consume request body, plus minor cleanup.
* Add a missing check in case of close_if_idle on a dead connection.
* Avoid error messages on PayloadTooLarge.
* Add test for new API.
* json takes str, not bytes
* Default to no maximum request size for streaming handlers.
* Fix chunked mode crash.
* Header values should be strictly ASCII but both UTF-8 and Latin-1 exist. Use UTF-8B to
cope with all.
* Refactoring and cleanup.
* Unify response header processing of ASGI and asyncio modes.
* Avoid special handling of StreamingHTTPResponse.
* 35 % speedup in HTTP/1.1 response formatting (not so much overall effect).
* Duplicate set-cookie headers were being produced.
* Cleanup processed_headers some more.
* Linting
* Import ordering
* Response middleware ran by async request.respond().
* Need to check if transport is closing to avoid getting stuck in sending loops after peer has disconnected.
* Middleware and error handling refactoring.
* Linter
* Fix tracking of HTTP stage when writing to transport fails.
* Add clarifying comment
* Add a check for request body functions and a test for NotImplementedError.
* Linter and typing
* These must be tuples + hack mypy warnings away.
* New streaming test and minor fixes.
* Constant receive buffer size.
* 256 KiB send and receive buffers.
* Revert "256 KiB send and receive buffers."
This reverts commit abc1e3edb2.
* app.handle_exception already sends the response.
* Improved handling of errors during request.
* An odd hack to avoid an httpx limitation that causes test failures.
* Limit request header size to 8 KiB at most.
* Remove unnecessary use of format string.
* Cleanup tests
* Remove artifact
* Fix type checking
* Mark test for skipping
* Cleanup some edge cases
* Add ignore_body flag to safe methods
* Add unit tests for timeout logic
* Add unit tests for timeout logic
* Fix Mock usage in timeout test
* Change logging test to only logger in handler
* Windows py3.8 logging issue with current testing client
* Add test_header_size_exceeded
* Resolve merge conflicts
* Add request middleware to hard exception handling
* Add request middleware to hard exception handling
* Request middleware on exception handlers
* Linting
* Cleanup deprecations
Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com>
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>
In ASGI-mode, don't do sanic-side response chunk encoding, leave that to the ASGI-response-transport
Don't set content-length when using chunked-encoding in ASGI mode, this is incompatible with ASGI Chunked Transport-Encoding.
* Ignore writing headers when in ASGI mode for streaming responses
* Move asgi set on streaming until after response type check
* Adds multidict==5.0.0 to pass tests
* Bump version to 20.9.1
This allows blueprint registration to add the bp's static routes to its list of own routes. So now blueprint middlewares will apply to a blueprint's static file routes.
Fixes#1953
* feat: fixes exception due to unread bytes in stream
* feat: additonal unit tests to cover changes
* fix: automated changes by `make fix-import`
* fix: additonal changes by `make fix-import`
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>
* Added fix to include subprotocols from scope
* Added unit test to validate fix
* Changes by black
* Made changes to WebsocketConnection protocol
* Linter changes
* Added unit tests
* Fixing bugs in linting due to isort import checks
* Reverting compat import changes
* Fixing linter errors in compat.py
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>
* Version
* Version 20.6.1
* Fix imports and isort to remove from Makefile deprecated options
* duplicate the mypy ignore hint across both lines
after splitting the `from trio import ...` statement onto two lines, need to duplicate the mypy ignore hint across both lines to keep mypy from complaining
Co-authored-by: Ashley Sommer <ashleysommer@gmail.com>
* Socket binding implemented properly for IPv6 and UNIX sockets.
- app.run("::1") for IPv6
- app.run("unix:/tmp/server.sock") for UNIX sockets
- app.run("localhost") retains old functionality (randomly either IPv4 or IPv6)
Do note that IPv6 and UNIX sockets are not fully supported by other Sanic facilities.
In particular, request.server_name and request.server_port are currently unreliable.
* Fix Windows compatibility by not referring to socket.AF_UNIX unless needed.
* Compatibility fix.
* Fix test of existing unix socket.
* Cleaner unix socket removal.
* Remove unix socket on exit also with workers=1.
* More pedantic UNIX socket implementation.
* Refactor app to take unix= argument instead of unix:-prefixed host. Goin' fast @ unix-socket fixed.
* Linter
* Proxy properties cleanup. Slight changes of semantics. SERVER_NAME now overrides everything.
* Have server fill in connection info instead of request asking the socket.
- Would be a good idea to remove request.transport entirely but I didn't dare to touch it yet.
* Linter 💣🌟✊💀
* Fix typing issues. request.server_name returns empty string if host header is missing.
* Fix tests
* Tests were failing, fix connection info.
* Linter nazi says you need that empty line.
* Rename a to addr, leave client empty for unix sockets.
* Add --unix support when sanic is run as module.
* Remove remove_route, deprecated in 19.6.
* Improved unix socket binding.
* More robust creating and unlinking of sockets. Show proper and not temporary name in conn_info.
* Add comprehensive tests for unix socket mode.
* Hide some imports inside functions to avoid Windows failure.
* Mention unix socket mode in deployment docs.
* Fix merge commit.
* Make test_unix_connection_multiple_workers pickleable for spawn mode multiprocessing.
Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com>
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>
Moves the websocket_handler subfunction out to a class-level method, which can be more easily pickled by the built-in python Pickler.
Also includes a similar fix for the add_task deferred task scheduler subfunction.
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>
* Bug fix for host parameter issue with lists
As explained in #1772 there is an issue when using a list as an argument for the host parameter in the Blueprint.route() decorator. I've traced the issue back to this line, and the if conditional should ensure that the name attribute isn't accessed when route is None.
* Unit tests for blueprint.route host paramter set to list.
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>
* fix-#1856: adjust websockets version to setup.py and make nightly (py39) tests pass
* fix-#1856: set min websockets version to 8.1
* fix-#1856: suppress timeout for CI to pass
* fix-#1856: timeout -> close_timeout due to deprecation warning
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>
Co-authored-by: 7 <yunxu1992@gmail.com>
* Fix watchdog reload worker repeatedly if there are multiple changed files
* Simplify autoreloader, don't need multiprocessing.Process. Now works on OSX py38.
* Allow autoreloader with multiple workers and run it earlier.
* This works OK on Windows too.
* I don't see how cwd could be different here.
* app.run and app.create_server argument fixup.
* Add test for auto_reload (coverage not working unfortunately).
* Reloader cleanup, don't use external kill commands and exit normally.
* Strip newlines on test output (Windows-compat).
* Report failures in test_auto_reload to avoid timeouts.
* Use different test server ports to avoid binding problems on Windows.
* Fix previous commit
* Listen on same port after reload.
* Show Goin' Fast banner on reloads.
* More robust testing, also -m sanic.
* Add a timeout to terminate process
* Try a workaround for tmpdir deletion on Windows.
* Join process also on error (context manager doesn't).
* Cleaner autoreloader termination on Windows.
* Remove unused code.
* Rename test.
* Longer timeout on test exit.
Co-authored-by: Hùng X. Lê <lexhung@gmail.com>
Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com>
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>
Moves the subfunction _handler out to a module-level function, and parameterizes it with functools.partial().
Fixes the case when picking a sanic app which has a registered static route handler. This is usually encountered when attempting to use multiprocessing or auto_reload on OSX or Windows.
Fixes#1774
* Remove remove_route, deprecated in 19.6.
* No need for py35 compat anymore.
* Rewrite asyncio.coroutines with async/await.
* Remove deprecated request.raw_args.
* response.text() takes str only: avoid deprecation warning in all but one test.
* Remove unused import.
* Revert unnecessary deprecation warning.
* Remove apparently unnecessary py38 compat.
* Avoid asyncio.Task.all_tasks deprecation warning.
* Avoid warning on a test that tests deprecated response.text(int).
* Add pytest-asyncio to tox deps.
* Run the coroutine returned by AsyncioServer.close.
Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com>
* Fix Ctrl+C on Windows.
* Disable testing of a function N/A on Windows.
* Add test for coverage, avoid crash on missing _stopping.
* Initialise StreamingHTTPResponse.protocol = None
* Improved comments.
* Reduce amount of data in test_request_stream to avoid failures on Windows.
* The Windows test doesn't work on Windows :(
* Use port numbers more likely to be free than 8000.
* Disable the other signal tests on Windows as well.
* Windows doesn't properly support SO_REUSEADDR, so that's disabled in Python, and thus rebinding fails. For successful testing, reuse port instead.
* app.run argument handling: added server kwargs (alike create_server), added warning on extra kwargs, made auto_reload explicit argument. Another go at Windows tests
* Revert "app.run argument handling: added server kwargs (alike create_server), added warning on extra kwargs, made auto_reload explicit argument. Another go at Windows tests"
This reverts commit dc5d682448.
* Use random test server port on most tests. Should avoid port/addr reuse issues.
* Another test to random port instead of 8000.
* Fix deprecation warnings about missing name on Sanic() in tests.
* Linter and typing
* Increase test coverage
* Rewrite test for ctrlc_windows_workaround
* py36 compat
* py36 compat
* py36 compat
* Don't rely on loop internals but add a stopping flag to app.
* App may be restarted.
* py36 compat
* Linter
* Add a constant for OS checking.
Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com>
* Compatibility and deprecations for Sanic 20.3 in preparation of the streaming branch.
* Add test for new API.
* isort tests
* More coverage
* json takes str, not bytes
Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com>
* HTTP1 header formatting moved to headers.format_headers and rewritten.
- New implementation is one line of code and twice faster than the old one.
- Whole header block encoded to UTF-8 in one pass.
- No longer supports custom encode method on header values.
- Cookie objects now have __str__ in addition to encode, to work with this.
* Linter
* format_http1_response
* Replace encode_body with faster implementation based on f-string.
Benchmarks:
def encode_body(data):
try:
# Try to encode it regularly
return data.encode()
except AttributeError:
# Convert it to a str if you can't
return str(data).encode()
def encode_body2(data):
return f"{data}".encode()
def encode_body3(data):
return str(data).encode()
data_str, data_int = "foo", 123
%timeit encode_body(data_int)
928 ns ± 2.96 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
%timeit encode_body2(data_int)
280 ns ± 2.09 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
%timeit encode_body3(data_int)
387 ns ± 1.7 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
%timeit encode_body(data_str)
202 ns ± 1.9 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
%timeit encode_body2(data_str)
197 ns ± 0.507 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
%timeit encode_body3(data_str)
313 ns ± 1.28 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
* Wtf linter
* Content-type fixes.
* Body encoding sanitation, first pass.
- body/data type autodetection fixed.
- do not repr(body).encode() bytes-ish values.
- support __html__ and _repr_html_ in sanic.response.html().
* <any type>-to-str response autoconversion limited to sanic.response.text() only.
* Workaround MyPy issue.
* Add an empty line to make isort happy.
* Add html test for __html__ and _repr_html_.
* Remove StreamingHTTPResponse.get_headers helper function.
* Add back HTTPResponse Keep-Alive removed by earlier merge or something.
* Revert "Remove StreamingHTTPResponse.get_headers helper function."
Tests depend on this otherwise useless function.
This reverts commit 9651e6ae01.
* Add deprecation warnings; instead of assert for wrong HTTP version, and for non-string response.text.
* Add back missing import.
* Avoid duplicate response header tweaking code.
* Linter errors
* Quick fixes to make Sanic usable on hypercorn -k trio myweb.app
* Quick'n dirty compatibility and autodetection of hypercorn trio mode.
* mypy ignore for aiofiles/trio.
* lint
* Default error handler now only logs traceback on 500 errors and all responses are HTML formatted.
* Tests passing.
* Ability to flag any exception object with self.quiet = True following @ashleysommer suggestion.
* Refactor HTML formatting into errorpages.py. String escapes for debug tracebacks.
* Remove extra includes
* Auto-set quiet flag also when decorator is used.
* Cleanup, make error pages (probably) HTML5-compliant and similar for debug and non-debug modes.
* Fix lookup of non-existant status codes
* No logging of 503 errors after all.
* lint
* Allow route decorators to stack up without causing a function signature inspection crash
Fix#1742
* Apply fix to @websocket routes docorator too
Add test for double-stacked websocket decorator
remove introduction of new variable in route wrapper, extend routes in-place.
Add explanation of why a handler will be a tuple in the case of a double-stacked route decorator
* HTTP1 header formatting moved to headers.format_headers and rewritten.
- New implementation is one line of code and twice faster than the old one.
- Whole header block encoded to UTF-8 in one pass.
- No longer supports custom encode method on header values.
- Cookie objects now have __str__ in addition to encode, to work with this.
* Add an import missed in merge.
* Begin swap of requests-async for httpx
* Finalize httpx adoption and resolve tests
Resolve linting and formatting
* Remove documentation references to requests-async in favor of httpx
* GIT-37: fix blueprint middleware application
1. If you register a middleware via `@blueprint.middleware` then it will apply only to the routes defined by the blueprint.
2. If you register a middleware via `@blueprint_group.middleware` then it will apply to all blueprint based routes that are part of the group.
3. If you define a middleware via `@app.middleware` then it will be applied on all available routes
Fixes#37
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* GIT-37: add changelog
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* pass request_buffer_queue_size argument to HttpProtocol
* fix to use simultaneously only one task to put body to stream buffer
* add a test code for REQUEST_BUFFER_QUEUE_SIZE
* 🐛 fix SERVER_NAME enforcement in url_for
fixes#1707
* 💡 add additional documentation for using request.args
fixes#1704
* ✅ add additional test to check url_for without SERVER_NAME
* 📝 add changelog for fixes
* Fixes ability to trigger "after_server_start", "before_server_stop", "after_server_stop" server events when using app.create_server to start your own asyncio_server
See example file run_async_advanced for a full example
* Fix a missing method on AsyncServer that some tests need
Add a tiny bit more documentation in-code
Change name of AsyncServerCoro to AsyncioServer
* Added support for HTTP Forwarded header and combined parsing of other proxy headers.
- Accessible via request.forwarded that tries parse_forwarded and then parse_xforwarded
- parse_forwarded uses the Forwarded header, if config.FORWARDED_SECRET is provided and a matching header field is found
- parse_xforwarded uses X-Real-IP and X-Forwarded-* much alike the existing implementation
- This commit does not change existing request properties that still use the old code and won't make use of Forwarded headers.
* Use req.forwarded in req properties server_name, server_port, scheme and remote_addr.
X-Scheme handling moved to parse_xforwarded.
* Cleanup and fix req.server_port; no longer reports socket port if any forwards headers are used.
* Update docstrings to incidate that forwarded header is used first.
* Remove testing function.
* Fix tests and linting.
- One test removed due to change of semantics - no socket port will be used if any forwarded headers are in effect.
- Other tests augmented with X-Forwarded-For, to allow the header being tested take effect (shouldn't affect old implementation).
* Try to workaround buggy tools complaining about incorrect ordering of imports.
* Cleanup forwarded processing, add comments. secret is now also returned.
* Added tests, fixed quoted string handling, cleanup.
* Further tests for full coverage.
* Try'n make linter happy.
* Add support for multiple Forwarded headers. Unify parse_forwarded parameters with parse_xforwarded.
* Implement multiple headers support for X-Forwarded-For.
- Previously only the first header was used, so this BUGFIX may affect functionality.
* Bugfix for request.server_name: strip port and other parts.
- request.server_name docs claim that it returns the hostname only (no port).
- config.SERVER_NAME may be full URL, so strip scheme, port and path
- HTTP Host and consequently forwarded Host may include port number, so
strip that also for forwarded hosts (previously only done for HTTP Host).
- Possible performance benefit of limiting to one split.
* Fallback to app.url_for and let it handle SERVER_NAME if defined (until a proper solution is implemented).
* Revise previous commit. Only fallback for full URL SERVER_NAMEs; allows host to be defined and proxied information still being used.
* Heil lintnazi.
* Modify testcase not to use underscores in URLs. Use hyphens which the spec allows for.
* Forwarded and Host header parsing improved.
- request.forwarded lowercases hosts, separates host:port into their own fields and lowercases addresses
- forwarded.parse_host helper function added and used for parsing all host-style headers (IPv6 cannot be simply split(":")).
- more tests fixed not to use underscores in hosts as those are no longer accepted and lead to the field being rejected
* Fixed typo in docstring.
* Added IPv6 address tests for Host header.
* Fix regex.
* Further tests and stricter forwarded handling.
* Fix merge commit
* Linter
* Linter
* Linter
* Add to avoid re-using the variable. Make a few raw strings non-raw.
* Remove unnecessary or
* Updated docs (work in progress).
* Enable REAL_IP_HEADER parsing irregardless of PROXIES_COUNT setting.
- Also cleanup and added comments
* New defaults for PROXIES_COUNT and REAL_IP_HEADER, updated tests.
* Remove support for PROXIES_COUNT=-1.
* Linter errors.
- This is getting ridiculous: cannot fit an URL on one line, linter requires
splitting the string literal!
* Add support for by=_proxySecret, updated docs, updated tests.
* Forwarded headers' semantics tuning.
- Forwarded host is now preserved in original format
- request.host now returns a forwarded host if available, else the Host header
- Forwarded options are preserved in original order, and later keys override earlier ones
- Forwarded path is automatically URL-unquoted
- Forwarded 'by' and 'for' are omitted if their value is unknown
- Tests modified accordingly
- Cleanup and improved documentation
* Add ASGI test.
* Linter
* Linter #2
* PEP 594 has cgi module scheduled for deprecation in Python 3.8. Reimplement
cgi.parse_header in Sanic. The new implementation is much faster than either
cgi.parse_header or equivalent werkzeug.parse_options_header, and unlike the
two, handles also quoted values with semicolons or \" in them.
* Fix string escape.
* Useless linter complaints.
* More linter issues
* Add return type hint.
* Do not support quoted-pair escapes.
- Improved documentation and renamed the function more aptly as it only seems
to apply to content-type and content-disposition headers.
* Unquote filenames also in normal mode.
* Add tests for headers. Adapted from CPython parse_header tests with changes on the final test.
* Linter
* Revert "Unquote filenames also in normal mode."
This reverts commit bf0d502bcd.
* Improved parse_content_header and added tests with Firefox and Chrome.
- Unescaping of quotes moved to parse_content_header because it affects all fields,
not just filenames.
- It is impossible to handle all cases correctly but the current heuristics should
suffice well for typical cases and beyond.
- Added comparisons with cgi.parse_header and werkzeug.parse_options_header.
* Updated comments as well.
If no X-Forwarded-Port nor Host headers are present, Sanic uses "sockname"
to determine the port. This expected (host, port) tuple to be returned but
for IPv6 a 4-tuple is returned instead. Changed code so that port is picked
up in either case. Handling of "peername" was already correct in this regard.
_get_address and server_port both still return incorrect data or crash for
other socket types (e.g unix). Socket type should checked before any queries.
Bandit is a python package for staticly scanning code for security issues.
* Added to tox.ini
* Added to setup.py
* Added to .travis.yml
As part of CI/CD pipeline
Update all tests to be compatible with requests-async
Cleanup testing client changes with black and isort
Remove Python 3.5 and other meta doc cleanup
rename pyproject and fix pep517 error
Add black config to tox.ini
Cleanup tests and remove aiohttp
tox.ini change for easier development commands
Remove aiohttp from changelog and requirements
Cleanup imports and Makefile
* assign app before handle_request so that request.app could be used in case of connection timeout
* gitignore pip-wheel-metadata/
* remove default app for request class and fix lint issue
* Drop dependency on distutils
While distutils is part of stdlib, it feels odd to use distutils in main application code.
I personally use a (lean)[https://hub.docker.com/r/haizaar/python-minimal/tags] Python distribution for running my applications that does not include distutils.
* Flake8 fixes
* "black" fixes
* strtobool should actually return bool
We stop getting:
ValueError: could not convert string to float: '12.34.56'
when passing 12.34.56 as a number route parameter argument.
By accepting ".12" and "12.", this is a non-breaking change. All valid
floats described by [0-9\.]+ are still accepted, just invalid ones are
now rejected.
Couldn't delete the release on github so we go with the next best thing
which is to just bump the patch version
Signed-off-by: Eli Uriegas <seemethere101@gmail.com>
* add Request.not_grouped_args, deprecation warning Request.raw_args
* add 1 more test for coverage
* custom parser for Request.args and Request.query_args, some additional tests
* add docs for custom queryset parsing
* fix import sorting
* docstrings for get_query_args and get_args methods
* lost import
