* HTTP1 header formatting moved to headers.format_headers and rewritten.
- New implementation is one line of code and twice faster than the old one.
- Whole header block encoded to UTF-8 in one pass.
- No longer supports custom encode method on header values.
- Cookie objects now have __str__ in addition to encode, to work with this.
* Linter
* format_http1_response
* Replace encode_body with faster implementation based on f-string.
Benchmarks:
def encode_body(data):
try:
# Try to encode it regularly
return data.encode()
except AttributeError:
# Convert it to a str if you can't
return str(data).encode()
def encode_body2(data):
return f"{data}".encode()
def encode_body3(data):
return str(data).encode()
data_str, data_int = "foo", 123
%timeit encode_body(data_int)
928 ns ± 2.96 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
%timeit encode_body2(data_int)
280 ns ± 2.09 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
%timeit encode_body3(data_int)
387 ns ± 1.7 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
%timeit encode_body(data_str)
202 ns ± 1.9 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
%timeit encode_body2(data_str)
197 ns ± 0.507 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
%timeit encode_body3(data_str)
313 ns ± 1.28 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
* Wtf linter
* Content-type fixes.
* Body encoding sanitation, first pass.
- body/data type autodetection fixed.
- do not repr(body).encode() bytes-ish values.
- support __html__ and _repr_html_ in sanic.response.html().
* <any type>-to-str response autoconversion limited to sanic.response.text() only.
* Workaround MyPy issue.
* Add an empty line to make isort happy.
* Add html test for __html__ and _repr_html_.
* Remove StreamingHTTPResponse.get_headers helper function.
* Add back HTTPResponse Keep-Alive removed by earlier merge or something.
* Revert "Remove StreamingHTTPResponse.get_headers helper function."
Tests depend on this otherwise useless function.
This reverts commit 9651e6ae01.
* Add deprecation warnings; instead of assert for wrong HTTP version, and for non-string response.text.
* Add back missing import.
* Avoid duplicate response header tweaking code.
* Linter errors
* Quick fixes to make Sanic usable on hypercorn -k trio myweb.app
* Quick'n dirty compatibility and autodetection of hypercorn trio mode.
* mypy ignore for aiofiles/trio.
* lint
* Default error handler now only logs traceback on 500 errors and all responses are HTML formatted.
* Tests passing.
* Ability to flag any exception object with self.quiet = True following @ashleysommer suggestion.
* Refactor HTML formatting into errorpages.py. String escapes for debug tracebacks.
* Remove extra includes
* Auto-set quiet flag also when decorator is used.
* Cleanup, make error pages (probably) HTML5-compliant and similar for debug and non-debug modes.
* Fix lookup of non-existant status codes
* No logging of 503 errors after all.
* lint
* Allow route decorators to stack up without causing a function signature inspection crash
Fix#1742
* Apply fix to @websocket routes docorator too
Add test for double-stacked websocket decorator
remove introduction of new variable in route wrapper, extend routes in-place.
Add explanation of why a handler will be a tuple in the case of a double-stacked route decorator
* HTTP1 header formatting moved to headers.format_headers and rewritten.
- New implementation is one line of code and twice faster than the old one.
- Whole header block encoded to UTF-8 in one pass.
- No longer supports custom encode method on header values.
- Cookie objects now have __str__ in addition to encode, to work with this.
* Add an import missed in merge.
* Begin swap of requests-async for httpx
* Finalize httpx adoption and resolve tests
Resolve linting and formatting
* Remove documentation references to requests-async in favor of httpx
* GIT-37: fix blueprint middleware application
1. If you register a middleware via `@blueprint.middleware` then it will apply only to the routes defined by the blueprint.
2. If you register a middleware via `@blueprint_group.middleware` then it will apply to all blueprint based routes that are part of the group.
3. If you define a middleware via `@app.middleware` then it will be applied on all available routes
Fixes#37
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* GIT-37: add changelog
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* pass request_buffer_queue_size argument to HttpProtocol
* fix to use simultaneously only one task to put body to stream buffer
* add a test code for REQUEST_BUFFER_QUEUE_SIZE
* 🐛 fix SERVER_NAME enforcement in url_for
fixes#1707
* 💡 add additional documentation for using request.args
fixes#1704
* ✅ add additional test to check url_for without SERVER_NAME
* 📝 add changelog for fixes
* Fixes ability to trigger "after_server_start", "before_server_stop", "after_server_stop" server events when using app.create_server to start your own asyncio_server
See example file run_async_advanced for a full example
* Fix a missing method on AsyncServer that some tests need
Add a tiny bit more documentation in-code
Change name of AsyncServerCoro to AsyncioServer
* Added support for HTTP Forwarded header and combined parsing of other proxy headers.
- Accessible via request.forwarded that tries parse_forwarded and then parse_xforwarded
- parse_forwarded uses the Forwarded header, if config.FORWARDED_SECRET is provided and a matching header field is found
- parse_xforwarded uses X-Real-IP and X-Forwarded-* much alike the existing implementation
- This commit does not change existing request properties that still use the old code and won't make use of Forwarded headers.
* Use req.forwarded in req properties server_name, server_port, scheme and remote_addr.
X-Scheme handling moved to parse_xforwarded.
* Cleanup and fix req.server_port; no longer reports socket port if any forwards headers are used.
* Update docstrings to incidate that forwarded header is used first.
* Remove testing function.
* Fix tests and linting.
- One test removed due to change of semantics - no socket port will be used if any forwarded headers are in effect.
- Other tests augmented with X-Forwarded-For, to allow the header being tested take effect (shouldn't affect old implementation).
* Try to workaround buggy tools complaining about incorrect ordering of imports.
* Cleanup forwarded processing, add comments. secret is now also returned.
* Added tests, fixed quoted string handling, cleanup.
* Further tests for full coverage.
* Try'n make linter happy.
* Add support for multiple Forwarded headers. Unify parse_forwarded parameters with parse_xforwarded.
* Implement multiple headers support for X-Forwarded-For.
- Previously only the first header was used, so this BUGFIX may affect functionality.
* Bugfix for request.server_name: strip port and other parts.
- request.server_name docs claim that it returns the hostname only (no port).
- config.SERVER_NAME may be full URL, so strip scheme, port and path
- HTTP Host and consequently forwarded Host may include port number, so
strip that also for forwarded hosts (previously only done for HTTP Host).
- Possible performance benefit of limiting to one split.
* Fallback to app.url_for and let it handle SERVER_NAME if defined (until a proper solution is implemented).
* Revise previous commit. Only fallback for full URL SERVER_NAMEs; allows host to be defined and proxied information still being used.
* Heil lintnazi.
* Modify testcase not to use underscores in URLs. Use hyphens which the spec allows for.
* Forwarded and Host header parsing improved.
- request.forwarded lowercases hosts, separates host:port into their own fields and lowercases addresses
- forwarded.parse_host helper function added and used for parsing all host-style headers (IPv6 cannot be simply split(":")).
- more tests fixed not to use underscores in hosts as those are no longer accepted and lead to the field being rejected
* Fixed typo in docstring.
* Added IPv6 address tests for Host header.
* Fix regex.
* Further tests and stricter forwarded handling.
* Fix merge commit
* Linter
* Linter
* Linter
* Add to avoid re-using the variable. Make a few raw strings non-raw.
* Remove unnecessary or
* Updated docs (work in progress).
* Enable REAL_IP_HEADER parsing irregardless of PROXIES_COUNT setting.
- Also cleanup and added comments
* New defaults for PROXIES_COUNT and REAL_IP_HEADER, updated tests.
* Remove support for PROXIES_COUNT=-1.
* Linter errors.
- This is getting ridiculous: cannot fit an URL on one line, linter requires
splitting the string literal!
* Add support for by=_proxySecret, updated docs, updated tests.
* Forwarded headers' semantics tuning.
- Forwarded host is now preserved in original format
- request.host now returns a forwarded host if available, else the Host header
- Forwarded options are preserved in original order, and later keys override earlier ones
- Forwarded path is automatically URL-unquoted
- Forwarded 'by' and 'for' are omitted if their value is unknown
- Tests modified accordingly
- Cleanup and improved documentation
* Add ASGI test.
* Linter
* Linter #2
* PEP 594 has cgi module scheduled for deprecation in Python 3.8. Reimplement
cgi.parse_header in Sanic. The new implementation is much faster than either
cgi.parse_header or equivalent werkzeug.parse_options_header, and unlike the
two, handles also quoted values with semicolons or \" in them.
* Fix string escape.
* Useless linter complaints.
* More linter issues
* Add return type hint.
* Do not support quoted-pair escapes.
- Improved documentation and renamed the function more aptly as it only seems
to apply to content-type and content-disposition headers.
* Unquote filenames also in normal mode.
* Add tests for headers. Adapted from CPython parse_header tests with changes on the final test.
* Linter
* Revert "Unquote filenames also in normal mode."
This reverts commit bf0d502bcd.
* Improved parse_content_header and added tests with Firefox and Chrome.
- Unescaping of quotes moved to parse_content_header because it affects all fields,
not just filenames.
- It is impossible to handle all cases correctly but the current heuristics should
suffice well for typical cases and beyond.
- Added comparisons with cgi.parse_header and werkzeug.parse_options_header.
* Updated comments as well.
If no X-Forwarded-Port nor Host headers are present, Sanic uses "sockname"
to determine the port. This expected (host, port) tuple to be returned but
for IPv6 a 4-tuple is returned instead. Changed code so that port is picked
up in either case. Handling of "peername" was already correct in this regard.
_get_address and server_port both still return incorrect data or crash for
other socket types (e.g unix). Socket type should checked before any queries.
Bandit is a python package for staticly scanning code for security issues.
* Added to tox.ini
* Added to setup.py
* Added to .travis.yml
As part of CI/CD pipeline
Update all tests to be compatible with requests-async
Cleanup testing client changes with black and isort
Remove Python 3.5 and other meta doc cleanup
rename pyproject and fix pep517 error
Add black config to tox.ini
Cleanup tests and remove aiohttp
tox.ini change for easier development commands
Remove aiohttp from changelog and requirements
Cleanup imports and Makefile
* assign app before handle_request so that request.app could be used in case of connection timeout
* gitignore pip-wheel-metadata/
* remove default app for request class and fix lint issue
* Drop dependency on distutils
While distutils is part of stdlib, it feels odd to use distutils in main application code.
I personally use a (lean)[https://hub.docker.com/r/haizaar/python-minimal/tags] Python distribution for running my applications that does not include distutils.
* Flake8 fixes
* "black" fixes
* strtobool should actually return bool
We stop getting:
ValueError: could not convert string to float: '12.34.56'
when passing 12.34.56 as a number route parameter argument.
By accepting ".12" and "12.", this is a non-breaking change. All valid
floats described by [0-9\.]+ are still accepted, just invalid ones are
now rejected.
Couldn't delete the release on github so we go with the next best thing
which is to just bump the patch version
Signed-off-by: Eli Uriegas <seemethere101@gmail.com>
* add Request.not_grouped_args, deprecation warning Request.raw_args
* add 1 more test for coverage
* custom parser for Request.args and Request.query_args, some additional tests
* add docs for custom queryset parsing
* fix import sorting
* docstrings for get_query_args and get_args methods
* lost import
* enable blueprint group middleware support
This commit will enable the users to implement a middleware at the
blueprint group level whereby enforcing the middleware automatically to
each of the available Blueprints that are part of the group.
This will eanble a simple way in which a certain set of common features
and criteria can be enforced on a Blueprint group. i.e. authentication
and authorization
This commit will address the feature request raised as part of Issue #1386
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* enable indexing of BlueprintGroup object
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* rename blueprint group file to fix spelling error
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* add documentation and additional unit tests
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* cleanup and optimize headers in unit test file
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* fix Bluprint Group iteratable method
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* add additional unit test to check StopIteration condition
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* cleanup iter protocol implemenation for blueprint group and add slots
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* fix blueprint group middleware invocation identification
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* feat: enable list behavior on blueprint group object and use append instead of properly to add blueprint to group
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
* suport filename length is 0
* 1. suport filename length is zero for multipart/form-data.
2. Now recognizes non-ASCII filenames in RFC 2231, "filename*" format
3. Add some test cases in tests/test_requests.py::test_request_multipart_files.
* reformat sanic/request.py
- adding 2 new parameters to Sanic.create_server:
* return_asyncio_server=False - defines whether there's
a need to return an asyncio.Server or run it right away
* asyncio_server_kwargs=None - for python 3.7 uvloop doesn't
support all necessary features like "start_serving",
so, in order to make sanic work well with asyncio from 3.7
there's a need to introduce generic way for passing
kwargs for "loop.create_server"
Closes: #1469
* Add tests for remove_route()
* Add tests for sanic/router.py
* Add tests for sanic/cookies.py
* Disable reset logging in test_logging.py
* Add tests for sanic/request.py
* Add tests for ContentRangeHandler
* Add tests for exception at response middleware
* Fix cached_handlers for ErrorHandler.lookup()
* Add test for websocket request timeout
* Add tests for getting cookies of StreamResponse, Remove some unused variables in tests/test_cookies.py
* Add tests for nested error handle
* created methods to append and finish body content on request.py so the underlying body instance can have certain flexibility; modified server.py to reflect these changes
* - made some adjustments (including the Request.body_init method) as requested by @ahopkins;
- created a new test with a custom Request class implementation of the flexibility provided by the new methods;
The current implementation of `sanic` attempts to make use of `ujson` if
it's available in the system and if not, it will default to the inbuilt
`json` module provided by python.
The current implementation of `ujson` does not provide a mechanism to
provide a custom `seperators` parameter as part of the `dumps` method
invocation and the default behavior of the module is to strip all the
spaces around seperators such as `:` and `,`. This leads to an
inconsistency in the response length when the response is generated
using the `ujson` and in built `json` module provided by python.
To maintain the consistency, this commit overrides the default behavior
of the `dumps` method provided by the `json` module to add a `seperators`
argument that will strip the white spaces around these character like
the default behavior of `ujson`
This addresses the issue referenced in #1398
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
This commit fixes the issue in the `Range` header handling that was done
while serving the file contents.
As per the HTTP response standards, a status code of 206 will be used in
case if the Range is returning a partial value and default of 200 in
other cases.
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
Change the string passed in the "name" section of the namedtuples in Blueprint to match the name of the Blueprint module attribute name.
This allows blueprints to be pickled and unpickled, without errors, which is a requirment of running Sanic in multiprocessing mode in Windows.
Added a test for pickling and unpickling blueprints
Added a test for pickling and unpickling sanic itself
Added a test for enabling multiprocessing on an app with a blueprint (only useful to catch this bug if the tests are run on Windows).
* This commit adds handlers for the asyncio/uvloop protocol callbacks for pause_writing and resume_writing.
These are needed for the correct functioning of built-in tcp flow-control provided by uvloop and asyncio.
This is somewhat of a breaking change, because the `write` function in user streaming callbacks now must be `await`ed.
This is necessary because it is possible now that the http protocol may be paused, and any calls to write may need to wait on an async event to be called to become unpaused.
Updated examples and tests to reflect this change.
This change does not apply to websocket connections. A change to websocket connections may be required to match this change.
* Fix a couple of PEP8 errors caused by previous rebase.
* update docs
add await syntax to response.write in response-streaming docs.
* remove commented out code from a test file
* Fix two problems with the auto_reloader in Linux.
1) Change 'posix' to 'linux' in sys.plaform check, because 'posix' is an invalid value and 'linux' is the correct value to use here.
2) In kill_process_children, don't just kill the 2nd level procs, also kill the 1st level procs.
Also in kill_process_children, catch and ignore errors in the case that the child proc is already killed.
* Fix flake8 formatting on PR
The Canceled exception handler purposely sets `response` to `None` to drop references to the handler coroutine, in an attempt to preemptively release resources.
This commit also fixes a possible reference-before-assignment of the `response` variable in the `handle_request` function.
Finally, another byproduct of this change is that ResponseMiddleware will no longer run if the `response` is `None`.
* Add content_type flag to Sanic.static
Fixes#1266
* Fix flake8 error in travis
Add line to document `content_type` arg
* Fix content_type for file streams
Update tests
herp derp
* Remove content_type as an arg to HTTPResponse
`response.HTTPResponse` will default to `headers['Content-Type']` instead of `content_type`
https://github.com/channelcat/sanic/pull/1267#discussion_r204190913
* URL Quote the URL before redirecting
* Use safe url instead of unsafe one
* Fix query params
* fix build
* Whitelist all reserved characters from rfc3986
* Add tests for redirect url sanitizing
* Remove check for resulting URL on header injection test
The thing the tests are testing for can be implemented in other
ways that don't redirect to 100% the same address, but they'll all have
to match the remaining parts of the test to succeed.
* add unit tests, which should fail
* fix CIDict
* moving CIDict to avoid circular imports
* fix unit tests
* use multidict for headers
* fix cookie
* add version constraint for multidict
* omit test coverage for __main__.py
* make flake8 happy
* consolidate check in for loop
* travisci retry build
* introduced basic entity and hopbyhop header identification
* removed entity headers
* coding style fixes
* remove unneeded header check
* moved from bytes to unicode in headers
* changed list to tuple in empty response statuses
causes a route to be added twice. One without the slash, the other with the
Setting strict_slashes to false when a route does not end with slashes
slash. This is ok if the Router._add method runs linearly, but problematic
when it runs recursively. Unfortunately recursion is triggered when
the host param to the Router._add function is a list of hosts.
A field name in the Content-Disposition header is required by the multipart/form-data spec. If one field/part does not have it, it will be omitted from the request. When this happens, we log it to DEBUG.
Sanic automatically assumes that a form field is a file if it has a content-type header, even though the header is text/plain or application/json. This is a fix for it, I took into account the RFC7578 specification regarding the defaults.
When passing a null value as parameter (ex.: 0, None or False), Sanic said "Error: Required parameter `param` was not passed to url_for"
Example:
```
@app.route("/<idx>")
def route(rq, idx):
pass
```
```
url_for("route", idx=0)
```
No longer raises: `Error: Required parameter `idx` was not passed to url_for`
Change auto reloader enviroment varible name to SANIC_SERVER_RUNNING
Fix some typo mistakes, flake uncompatibilities and such problems.
Raise NotImplementedError for operating systems except posix systems for auto reloading.
Before: Even after raising ResponseTimeout, server still processes
remaining tasks until it is done
After: Before raising ResponseTimeout, server stops working task.
