Commit Graph

485 Commits

Author SHA1 Message Date
L. Kärkkäinen
1aac4f546b Merge branch 'master' into streaming 2020-03-26 08:59:33 +02:00
L. Kärkkäinen
120f0262f7
Fix Ctrl+C and tests on Windows. (#1808)
* Fix Ctrl+C on Windows.

* Disable testing of a function N/A on Windows.

* Add test for coverage, avoid crash on missing _stopping.

* Initialise StreamingHTTPResponse.protocol = None

* Improved comments.

* Reduce amount of data in test_request_stream to avoid failures on Windows.

* The Windows test doesn't work on Windows :(

* Use port numbers more likely to be free than 8000.

* Disable the other signal tests on Windows as well.

* Windows doesn't properly support SO_REUSEADDR, so that's disabled in Python, and thus rebinding fails. For successful testing, reuse port instead.

* app.run argument handling: added server kwargs (alike create_server), added warning on extra kwargs, made auto_reload explicit argument. Another go at Windows tests

* Revert "app.run argument handling: added server kwargs (alike create_server), added warning on extra kwargs, made auto_reload explicit argument. Another go at Windows tests"

This reverts commit dc5d682448.

* Use random test server port on most tests. Should avoid port/addr reuse issues.

* Another test to random port instead of 8000.

* Fix deprecation warnings about missing name on Sanic() in tests.

* Linter and typing

* Increase test coverage

* Rewrite test for ctrlc_windows_workaround

* py36 compat

* py36 compat

* py36 compat

* Don't rely on loop internals but add a stopping flag to app.

* App may be restarted.

* py36 compat

* Linter

* Add a constant for OS checking.

Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com>
2020-03-25 21:42:46 -07:00
L. Kärkkäinen
5832764aab Merge branch 'master' into streaming 2020-03-25 14:58:45 +02:00
L. Kärkkäinen
42af6e1af6 Add a check for request body functions and a test for NotImplementedError. 2020-03-25 14:35:09 +02:00
L. Kärkkäinen
4db075ffc1
Streaming migration for 20.3 release (#1800)
* Compatibility and deprecations for Sanic 20.3 in preparation of the streaming branch.

* Add test for new API.

* isort tests

* More coverage

* json takes str, not bytes

Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com>
2020-03-24 10:11:09 -07:00
L. Kärkkäinen
9dc2ec966c Linting 2020-03-09 15:44:06 +02:00
L. Kärkkäinen
4c34508695 Merge branch 'master' into streaming 2020-03-06 16:30:58 +02:00
L. Kärkkäinen
319388d78b
Remove the old request context API deprecated in 19.9. Use request.ctx instead. (#1801)
Co-authored-by: L. Kärkkäinen <tronic@users.noreply.github.com>
2020-03-05 21:40:46 -08:00
L. Kärkkäinen
eb66621544 json takes str, not bytes 2020-03-04 13:21:49 +02:00
L. Kärkkäinen
dc6b4925df Add test for new API. 2020-03-04 12:25:46 +02:00
L. Kärkkäinen
e73f26b0a3 More specific error messages on Payload Too Large. 2020-03-02 15:32:12 +02:00
L. Kärkkäinen
9baa2419cd Merge branch 'master' into streaming 2020-03-02 14:05:15 +02:00
Subham Roy
ce71514d71
bump httpx dependency version to 0.11.1 (#1794) 2020-03-01 11:42:11 -08:00
L. Kärkkäinen
5a96996003 A test was missing that body_init/body_push/body_finish are never called. Rewritten using receive_body and case switching to make it fail if bypassed. 2020-03-01 16:34:20 +02:00
L. Kärkkäinen
fc16594138 Disable custom protocol test 2020-03-01 14:53:04 +02:00
L. Kärkkäinen
f8298939c0 Merge branch 'master' into streaming 2020-03-01 13:13:39 +02:00
L. Kärkkäinen
2840e4cfc8 Everything but CustomServer OK. 2020-03-01 13:10:53 +02:00
L. Kärkkäinen
85be5768c8 Remove keep-alive header from responses. First of all, it should say timeout=<value> which wasn't the case with existing implementation, and secondly none of the other web servers I tried include this header. 2020-03-01 12:15:16 +02:00
L. Kärkkäinen
7f41c5fa6b Even more cleanup and error checking, 8 failing tests. 2020-02-29 18:59:19 +02:00
L. Kärkkäinen
85c67a0014 Remove tests using the deprecated get_headers function that can no longer be supported. Chunked mode is now autodetected, so do not put content-length header if chunked mode is preferred. 2020-02-28 18:40:44 +02:00
L. Kärkkäinen
d5971377c5 Remove StreamBuffer tests because the type is about to be removed. 2020-02-28 17:38:09 +02:00
L. Kärkkäinen
29c6f3c49f Entirely remove request body queue. 2020-02-26 19:23:55 +02:00
Mykhailo Yusko
16961fab9d
Use f-strings instead of str.format() (#1793) 2020-02-25 14:01:13 -06:00
L. Kärkkäinen
3d05e1ec07 Change of test: don't require early bad request error but only after CRLF-CRLF. 2020-02-24 17:27:04 +02:00
L. Kärkkäinen
861e87347a
Fix #1788 incorrect url_for for routes with hosts, added tests. (#1789)
* Fix #1788 incorrect url_for for routes with hosts, added tests.

* Linter

* Remove debug print
2020-02-21 09:10:22 -08:00
L. Kärkkäinen
fe64a2764d Make all requests streaming and preload body for non-streaming handlers. 2020-02-21 13:28:50 +02:00
L. Kärkkäinen
bffdb3b5c2 More robust response datatype handling (#1674)
* HTTP1 header formatting moved to headers.format_headers and rewritten.

- New implementation is one line of code and twice faster than the old one.
- Whole header block encoded to UTF-8 in one pass.
- No longer supports custom encode method on header values.
- Cookie objects now have __str__ in addition to encode, to work with this.

* Linter

* format_http1_response

* Replace encode_body with faster implementation based on f-string.

Benchmarks:

def encode_body(data):
    try:
        # Try to encode it regularly
        return data.encode()
    except AttributeError:
        # Convert it to a str if you can't
        return str(data).encode()

def encode_body2(data):
    return f"{data}".encode()

def encode_body3(data):
    return str(data).encode()

data_str, data_int = "foo", 123

%timeit encode_body(data_int)
928 ns ± 2.96 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

%timeit encode_body2(data_int)
280 ns ± 2.09 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

%timeit encode_body3(data_int)
387 ns ± 1.7 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

%timeit encode_body(data_str)
202 ns ± 1.9 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

%timeit encode_body2(data_str)
197 ns ± 0.507 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

%timeit encode_body3(data_str)
313 ns ± 1.28 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

* Wtf linter

* Content-type fixes.

* Body encoding sanitation, first pass.
- body/data type autodetection fixed.
- do not repr(body).encode() bytes-ish values.
- support __html__ and _repr_html_ in sanic.response.html().

* <any type>-to-str response autoconversion limited to sanic.response.text() only.

* Workaround MyPy issue.

* Add an empty line to make isort happy.

* Add html test for __html__ and _repr_html_.

* Remove StreamingHTTPResponse.get_headers helper function.

* Add back HTTPResponse Keep-Alive removed by earlier merge or something.

* Revert "Remove StreamingHTTPResponse.get_headers helper function."

Tests depend on this otherwise useless function.

This reverts commit 9651e6ae01.

* Add deprecation warnings; instead of assert for wrong HTTP version, and for non-string response.text.

* Add back missing import.

* Avoid duplicate response header tweaking code.

* Linter errors
2020-01-20 10:34:32 -06:00
Ashley Sommer
801595e24a Add server.start_serving and server.serve_forever to AsyncioServer proxy object, to match asyncio-python3.7 example doc, fixes #1754 (#1762) 2020-01-20 09:00:48 -06:00
L. Kärkkäinen
ba9b432993 No tracebacks on normal errors and prettier error pages (#1768)
* Default error handler now only logs traceback on 500 errors and all responses are HTML formatted.

* Tests passing.

* Ability to flag any exception object with self.quiet = True following @ashleysommer suggestion.

* Refactor HTML formatting into errorpages.py. String escapes for debug tracebacks.

* Remove extra includes

* Auto-set quiet flag also when decorator is used.

* Cleanup, make error pages (probably) HTML5-compliant and similar for debug and non-debug modes.

* Fix lookup of non-existant status codes

* No logging of 503 errors after all.

* lint
2020-01-20 08:58:14 -06:00
Ashley Sommer
b565072ed9 Allow route decorators to stack up again (#1764)
* Allow route decorators to stack up without causing a function signature inspection crash
Fix #1742

* Apply fix to @websocket routes docorator too
Add test for double-stacked websocket decorator
remove introduction of new variable in route wrapper, extend routes in-place.
Add explanation of why a handler will be a tuple in the case of a double-stacked route decorator
2020-01-10 21:50:16 -08:00
Liran Nuna
fccbc1adc4 Allow empty body without Content-Type; Introduce response.empty() (#1736) 2019-12-23 14:16:53 -06:00
Adam Hopkins
3f6a978328 Swap out requests-async for httpx (#1728)
* Begin swap of requests-async for httpx

* Finalize httpx adoption and resolve tests

Resolve linting and formatting

* Remove documentation references to requests-async in favor of httpx
2019-12-20 19:23:52 -08:00
Harsha Narayana
a6077a1790 GIT-37: fix blueprint middleware application (#1690)
* GIT-37: fix blueprint middleware application

1. If you register a middleware via `@blueprint.middleware` then it will apply only to the routes defined by the blueprint.
2. If you register a middleware via `@blueprint_group.middleware` then it will apply to all blueprint based routes that are part of the group.
3. If you define a middleware via `@app.middleware` then it will be applied on all available routes

Fixes #37

Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>

* GIT-37: add changelog

Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
2019-12-20 10:01:04 -06:00
Junyeong Jeong
ecbe5c839f pass request_buffer_queue_size argument to HttpProtocol (#1717)
* pass request_buffer_queue_size argument to HttpProtocol

* fix to use simultaneously only one task to put body to stream buffer

* add a test code for REQUEST_BUFFER_QUEUE_SIZE
2019-11-21 09:33:50 -06:00
Harsha Narayana
e81a8ce073 fix SERVER_NAME enforcement in url_for and request.args documentation (#1708)
* 🐛 fix SERVER_NAME enforcement in url_for

fixes #1707

* 💡 add additional documentation for using request.args

fixes #1704

*  add additional test to check url_for without SERVER_NAME

* 📝 add changelog for fixes
2019-11-01 10:32:49 -07:00
Harsha Narayana
e506c89304 deprecate None value support for app name (#1705)
*  deprecate None value support for app name

* 🚨 cleanup linter issues across the codebase
2019-10-23 09:12:20 -07:00
L. Kärkkäinen
c54a8b10bb Implement dict-like API on request objects for custom data. (#1666)
* Implement dict-like API on request objects for custom data.
* Updated docs about custom_context.
2019-09-26 14:11:31 -07:00
Ashley Sommer
927c0e082e Fix tests for multiprocessing pickle app and pickle blueprint (#1680)
The old tests were not quite checking for the right thing.
Fixing the test does not change Sanic code, expose any bugs, or fix any bugs.
2019-09-18 10:22:24 -07:00
Ashley Sommer
7674e917e4 Fixes "after_server_start" when using return_asyncio_server. (#1676)
* Fixes ability to trigger "after_server_start", "before_server_stop", "after_server_stop" server events when using app.create_server to start your own asyncio_server
See example file run_async_advanced for a full example

* Fix a missing method on AsyncServer that some tests need
Add a tiny bit more documentation in-code
Change name of AsyncServerCoro to AsyncioServer
2019-09-16 10:59:16 -07:00
L. Kärkkäinen
1e4b1c4d1a Forwarded headers and otherwise improved proxy handling (#1638)
* Added support for HTTP Forwarded header and combined parsing of other proxy headers.

- Accessible via request.forwarded that tries parse_forwarded and then parse_xforwarded
- parse_forwarded uses the Forwarded header, if config.FORWARDED_SECRET is provided and a matching header field is found
- parse_xforwarded uses X-Real-IP and X-Forwarded-* much alike the existing implementation
- This commit does not change existing request properties that still use the old code and won't make use of Forwarded headers.

* Use req.forwarded in req properties server_name, server_port, scheme and remote_addr.

X-Scheme handling moved to parse_xforwarded.

* Cleanup and fix req.server_port; no longer reports socket port if any forwards headers are used.

* Update docstrings to incidate that forwarded header is used first.

* Remove testing function.

* Fix tests and linting.

- One test removed due to change of semantics - no socket port will be used if any forwarded headers are in effect.
- Other tests augmented with X-Forwarded-For, to allow the header being tested take effect (shouldn't affect old implementation).

* Try to workaround buggy tools complaining about incorrect ordering of imports.

* Cleanup forwarded processing, add comments. secret is now also returned.

* Added tests, fixed quoted string handling, cleanup.

* Further tests for full coverage.

* Try'n make linter happy.

* Add support for multiple Forwarded headers. Unify parse_forwarded parameters with parse_xforwarded.

* Implement multiple headers support for X-Forwarded-For.

- Previously only the first header was used, so this BUGFIX may affect functionality.

* Bugfix for request.server_name: strip port and other parts.

- request.server_name docs claim that it returns the hostname only (no port).
- config.SERVER_NAME may be full URL, so strip scheme, port and path
- HTTP Host and consequently forwarded Host may include port number, so
  strip that also for forwarded hosts (previously only done for HTTP Host).
- Possible performance benefit of limiting to one split.

* Fallback to app.url_for and let it handle SERVER_NAME if defined (until a proper solution is implemented).

* Revise previous commit. Only fallback for full URL SERVER_NAMEs; allows host to be defined and proxied information still being used.

* Heil lintnazi.

* Modify testcase not to use underscores in URLs. Use hyphens which the spec allows for.

* Forwarded and Host header parsing improved.

- request.forwarded lowercases hosts, separates host:port into their own fields and lowercases addresses
- forwarded.parse_host helper function added and used for parsing all host-style headers (IPv6 cannot be simply split(":")).
- more tests fixed not to use underscores in hosts as those are no longer accepted and lead to the field being rejected

* Fixed typo in docstring.

* Added IPv6 address tests for Host header.

* Fix regex.

* Further tests and stricter forwarded handling.

* Fix merge commit

* Linter

* Linter

* Linter

* Add  to avoid re-using the  variable. Make a few raw strings non-raw.

* Remove unnecessary or

* Updated docs (work in progress).

* Enable REAL_IP_HEADER parsing irregardless of PROXIES_COUNT setting.

- Also cleanup and added comments

* New defaults for PROXIES_COUNT and REAL_IP_HEADER, updated tests.

* Remove support for PROXIES_COUNT=-1.

* Linter errors.

- This is getting ridiculous: cannot fit an URL on one line, linter requires
  splitting the string literal!

* Add support for by=_proxySecret, updated docs, updated tests.

* Forwarded headers' semantics tuning.

- Forwarded host is now preserved in original format
- request.host now returns a forwarded host if available, else the Host header
- Forwarded options are preserved in original order, and later keys override earlier ones
- Forwarded path is automatically URL-unquoted
- Forwarded 'by' and 'for' are omitted if their value is unknown
- Tests modified accordingly
- Cleanup and improved documentation

* Add ASGI test.

* Linter

* Linter #2
2019-09-02 08:50:56 -05:00
L. Kärkkäinen
2011f3a0b2 PEP 594 has cgi module scheduled for deprecation in Python 3.8 (#1649)
* PEP 594 has cgi module scheduled for deprecation in Python 3.8. Reimplement
cgi.parse_header in Sanic. The new implementation is much faster than either
cgi.parse_header or equivalent werkzeug.parse_options_header, and unlike the
two, handles also quoted values with semicolons or \" in them.

* Fix string escape.

* Useless linter complaints.

* More linter issues

* Add return type hint.

* Do not support quoted-pair escapes.

- Improved documentation and renamed the function more aptly as it only seems
  to apply to content-type and content-disposition headers.

* Unquote filenames also in normal mode.

* Add tests for headers. Adapted from CPython parse_header tests with changes on the final test.

* Linter

* Revert "Unquote filenames also in normal mode."

This reverts commit bf0d502bcd.

* Improved parse_content_header and added tests with Firefox and Chrome.

- Unescaping of quotes moved to parse_content_header because it affects all fields,
  not just filenames.
- It is impossible to handle all cases correctly but the current heuristics should
  suffice well for typical cases and beyond.
- Added comparisons with cgi.parse_header and werkzeug.parse_options_header.

* Updated comments as well.
2019-08-27 08:30:23 -05:00
Adam Hopkins
aa270d3ac2 Add content-type headers in response in ASGI mode 2019-08-11 11:29:08 +03:00
Harsha Narayana
97f288a534
fix: GIT-1623: handle cookie duplication and serialization issue
Signed-off-by: Harsha Narayana <harsha2k4@gmail.com>
2019-07-08 13:03:33 +05:30
7
1eaa2e3a5f
Merge pull request #1614 from huge-success/asgi-custom-request
Add custom request support to ASGI mode; fix a couple tests
2019-07-06 11:47:58 -07:00
BananaWanted
72b445621b Respect X-Forward-* headers and generate correct URLs in url_for (#1465)
* handle forwarded address in request

* added test cases

* Fix lint errors

* Fix uncovered code branch

* Update docstrings

* Update documents

* fix import order
2019-07-04 07:13:43 -05:00
Adam Hopkins
a2666a2b8a Add custom request support to ASGI mode; fix a couple tests
Undo change to request stream test
2019-06-24 22:59:23 +03:00
7
891f99d71d
Merge pull request #1475 from tomchristie/asgi-refactor-attempt
ASGI refactoring attempt
2019-06-20 16:33:44 -07:00
Adam Hopkins
b1c23fdbaa Increase testing coverage for ASGI
Beautify

Specify websockets version
2019-06-19 00:38:58 +03:00
7
8fbbe94fe1
Merge pull request #1436 from jotagesales/config_from_object_string
Config from object string
2019-06-16 16:58:43 -07:00
Adam Hopkins
ab706dda7d Resolve linting issues with imports 2019-06-11 11:21:37 +03:00