Merge branch 'master' into asgi-refactor-attempt

.github/stale.yml

@@ -0,0 +1,19 @@
+# Number of days of inactivity before an issue becomes stale
+daysUntilStale: 90
+# Number of days of inactivity before a stale issue is closed
+daysUntilClose: 30
+# Issues with these labels will never be considered stale
+exemptLabels:
+  - bug
+  - urgent
+  - necessary
+  - help wanted
+# Label to use when marking an issue as stale
+staleLabel: stale
+# Comment to post when marking an issue as stale. Set to `false` to disable
+markComment: >
+  This issue has been automatically marked as stale because it has not had
+  recent activity. It will be closed if no further activity occurs. If this 
+  is incorrect, please respond with an update. Thank you for your contributions.
+# Comment to post when closing a stale issue. Set to `false` to disable
+closeComment: false

README.rst

@@ -18,7 +18,7 @@ Sanic | Build fast. Run fast.
     * - Support
       - | |Forums| |Join the chat at https://gitter.im/sanic-python/Lobby| |Awesome|
     * - Stats
-      - | |Downloads|
+      - | |Downloads| |Conda downloads|
 
 .. |Forums| image:: https://img.shields.io/badge/forums-community-ff0068.svg
    :target: https://community.sanicframework.org/
@@ -50,6 +50,9 @@ Sanic | Build fast. Run fast.
 .. |Downloads| image:: https://pepy.tech/badge/sanic/month
     :alt: Downloads
     :target: https://pepy.tech/project/sanic
+.. |Conda downloads| image:: https://img.shields.io/conda/dn/conda-forge/sanic.svg
+    :alt: Downloads
+    :target: https://anaconda.org/conda-forge/sanic
 
 .. end-badges
 
@@ -57,7 +60,7 @@ Sanic is a **Python 3.6+** web server and web framework that's written to go fas
 
 `Source code on GitHub <https://github.com/huge-success/sanic/>`_ | `Help and discussion board <https://community.sanicframework.org/>`_. 
 
-The project is maintained by the community, for the community **Contributions are welcome!**
+The project is maintained by the community, for the community. **Contributions are welcome!**
 
 The goal of the project is to provide a simple way to get up and running a highly performant HTTP server that is easy to build, to expand, and ultimately to scale.
 
@@ -75,6 +78,12 @@ Installation
        $ pip3 install --no-binary :all: sanic
 
 
+.. note::
+
+  If you are running on a clean install of Fedora 28 or above, please make sure you have the ``redhat-rpm-config`` package installed in case if you want to
+  use ``sanic`` with ``ujson`` dependency.
+
+
 Hello World Example
 -------------------
 

SECURITY.md

@@ -0,0 +1,25 @@
+# Security Policy
+
+## Supported Versions
+
+Sanic releases long term support release once a year in December. LTS releases receive bug and security updates for **24 months**. Interim releases throughout the year occur every three months, and are supported until the subsequent interim release.
+
+| Version | LTS                | Supported          |
+| ------- | ------------------ | ------------------ |
+| 19.6.0  |                    | :white_check_mark: |
+| 19.3.1  |                    | :heavy_check_mark: |
+| 18.12.0 | :heavy_check_mark: | :heavy_check_mark: |
+| 0.8.3   |                    | :x:                |
+| 0.7.0   |                    | :x:                |
+| 0.6.0   |                    | :x:                |
+| 0.5.4   |                    | :x:                |
+| 0.4.1   |                    | :x:                |
+| 0.3.1   |                    | :x:                |
+| 0.2.0   |                    | :x:                |
+| 0.1.9   |                    | :x:                |
+
+## Reporting a Vulnerability
+
+If you discover a security vulnerability, we ask that you **do not** create an issue on GitHub. Instead, please [send a message to the core-devs](https://community.sanicframework.org/g/core-devs) on the community forums. Once logged in, you can send a message to the core-devs by clicking the message button.
+
+This will help to not publicize the issue until the team can address it and resolve it.

docs/sanic/getting_started.md

@@ -1,14 +1,16 @@
 # Getting Started
 
 Make sure you have both [pip](https://pip.pypa.io/en/stable/installing/) and at
-least version 3.5 of Python before starting. Sanic uses the new `async`/`await`
+least version 3.6 of Python before starting. Sanic uses the new `async`/`await`
 syntax, so earlier versions of python won't work.
 
 ## 1. Install Sanic
 
-  ```
+>   If you are running on a clean install of Fedora 28 or above, please make sure you have the ``redhat-rpm-config`` package installed in case if you want to use ``sanic`` with ``ujson`` dependency.
-  pip3 install sanic
+
-  ```
+```bash
+pip3 install sanic
+```
 
 To install sanic without `uvloop` or `ujson` using bash, you can provide either or both of these environmental variables
 using any truthy string like `'y', 'yes', 't', 'true', 'on', '1'` and setting the `SANIC_NO_X` (`X` = `UVLOOP`/`UJSON`) 
@@ -18,6 +20,13 @@ to true will stop that features installation.
 SANIC_NO_UVLOOP=true SANIC_NO_UJSON=true pip3 install sanic
 ```
 
+You can also install Sanic from [`conda-forge`](https://anaconda.org/conda-forge/sanic) 
+
+```bash
+conda config --add channels conda-forge
+conda install sanic
+```
+
 ##  2. Create a file called `main.py`
 
   ```python

docs/sanic/index.rst

@@ -1,9 +1,9 @@
 Sanic
 =================================
 
-Sanic is a Flask-like Python 3.5+ web server that's written to go fast.  It's based on the work done by the amazing folks at magicstack, and was inspired by `this article <https://magic.io/blog/uvloop-blazing-fast-python-networking/>`_.
+Sanic is a Python 3.6+ web server and web framework that's written to go fast. It allows the usage of the async/await syntax added in Python 3.5, which makes your code non-blocking and speedy.
 
-On top of being Flask-like, Sanic supports async request handlers.  This means you can use the new shiny async/await syntax from Python 3.5, making your code non-blocking and speedy.
+The goal of the project is to provide a simple way to get up and running a highly performant HTTP server that is easy to build, to expand, and ultimately to scale.
 
 Sanic is developed `on GitHub <https://github.com/channelcat/sanic/>`_. Contributions are welcome!
 
@@ -23,3 +23,7 @@ Sanic aspires to be simple
 
     if __name__ == "__main__":
         app.run(host="0.0.0.0", port=8000)
+
+.. note::
+
+    Sanic does not support Python 3.5 from version 19.6 and forward. However, version 18.12LTS is supported thru December 2020. Official Python support for version 3.5 is set to expire in September 2020.

environment.yml

@@ -13,7 +13,7 @@ dependencies:
   - sphinx==1.8.3
   - sphinx_rtd_theme==0.4.2
   - recommonmark==0.5.0
-  - requests-async==0.4.0
+  - requests-async==0.5.0
   - sphinxcontrib-asyncio>=0.2.0
   - docutils==0.14
   - pygments==2.3.1

sanic/__init__.py

@@ -2,6 +2,6 @@ from sanic.app import Sanic
 from sanic.blueprints import Blueprint
 
 
-__version__ = "19.03.1"
+__version__ = "19.6.0"
 
 __all__ = ["Sanic", "Blueprint"]

sanic/app.py

@@ -443,8 +443,12 @@ class Sanic:
     ):
         """Decorate a function to be registered as a websocket route
         :param uri: path of the URL
+        :param host: Host IP or FQDN details
+        :param strict_slashes: If the API endpoint needs to terminate
+                with a "/" or not
         :param subprotocols: optional list of str with supported subprotocols
-        :param host:
+        :param name: A unique name assigned to the URL so that it can
+                be used with :func:`url_for`
         :return: decorated function
         """
         self.enable_websocket()

setup.py

@@ -89,8 +89,8 @@ tests_require = [
     "multidict>=4.0,<5.0",
     "gunicorn",
     "pytest-cov",
-    "httpcore==0.1.1",
+    "httpcore==0.3.0",
-    "requests-async==0.4.0",
+    "requests-async==0.5.0",
     "beautifulsoup4",
     uvloop,
     ujson,

tests/test_keep_alive_timeout.py

@@ -16,41 +16,28 @@ from sanic.response import text
 from sanic.testing import HOST, PORT, SanicTestClient
 
 
-# import traceback
-
-
 CONFIG_FOR_TESTS = {"KEEP_ALIVE_TIMEOUT": 2, "KEEP_ALIVE": True}
 
 old_conn = None
 
 
 class ReusableSanicConnectionPool(httpcore.ConnectionPool):
-    async def acquire_connection(self, url, ssl, timeout):
+    async def acquire_connection(self, origin):
         global old_conn
-        if timeout.connect_timeout and not timeout.pool_timeout:
+        connection = self.active_connections.pop_by_origin(origin, http2_only=True)
-            timeout.pool_timeout = timeout.connect_timeout
+        if connection is None:
-        key = (url.scheme, url.hostname, url.port, ssl, timeout)
+            connection = self.keepalive_connections.pop_by_origin(origin)
-        try:
-            connection = self._keepalive_connections[key].pop()
-            if not self._keepalive_connections[key]:
-                del self._keepalive_connections[key]
-            self.num_keepalive_connections -= 1
-            self.num_active_connections += 1
 
-        except (KeyError, IndexError):
+        if connection is None:
-            ssl_context = await self.get_ssl_context(url, ssl)
+            await self.max_connections.acquire()
-            try:
+            connection = httpcore.HTTPConnection(
-                await asyncio.wait_for(
+                origin,
-                    self._max_connections.acquire(), timeout.pool_timeout
+                ssl=self.ssl,
+                timeout=self.timeout,
+                backend=self.backend,
+                release_func=self.release_connection,
             )
-            except asyncio.TimeoutError:
+        self.active_connections.add(connection)
-                raise PoolTimeout()
-            release = functools.partial(self.release_connection, key=key)
-            connection = httpcore.connections.Connection(
-                timeout=timeout, on_release=release
-            )
-            self.num_active_connections += 1
-            await connection.open(url.hostname, url.port, ssl=ssl_context)
 
         if old_conn is not None:
             if old_conn != connection:
@@ -66,62 +53,6 @@ class ReusableSanicAdapter(requests.adapters.HTTPAdapter):
     def __init__(self):
         self.pool = ReusableSanicConnectionPool()
 
-    async def send(
-        self,
-        request,
-        stream=False,
-        timeout=None,
-        verify=True,
-        cert=None,
-        proxies=None,
-    ):
-
-        method = request.method
-        url = request.url
-        headers = [
-            (_encode(k), _encode(v)) for k, v in request.headers.items()
-        ]
-
-        if not request.body:
-            body = b""
-        elif isinstance(request.body, str):
-            body = _encode(request.body)
-        else:
-            body = request.body
-
-        if isinstance(timeout, tuple):
-            timeout_kwargs = {
-                "connect_timeout": timeout[0],
-                "read_timeout": timeout[1],
-            }
-        else:
-            timeout_kwargs = {
-                "connect_timeout": timeout,
-                "read_timeout": timeout,
-            }
-
-        ssl = httpcore.SSLConfig(cert=cert, verify=verify)
-        timeout = httpcore.TimeoutConfig(**timeout_kwargs)
-
-        try:
-            response = await self.pool.request(
-                method,
-                url,
-                headers=headers,
-                body=body,
-                stream=stream,
-                ssl=ssl,
-                timeout=timeout,
-            )
-        except (httpcore.BadResponse, socket.error) as err:
-            raise ConnectionError(err, request=request)
-        except httpcore.ConnectTimeout as err:
-            raise requests.exceptions.ConnectTimeout(err, request=request)
-        except httpcore.ReadTimeout as err:
-            raise requests.exceptions.ReadTimeout(err, request=request)
-
-        return self.build_response(request, response)
-
 
 class ResusableSanicSession(requests.Session):
     def __init__(self, *args, **kwargs) -> None:
@@ -149,13 +80,14 @@ class ReuseableSanicTestClient(SanicTestClient):
         uri="/",
         gather_request=True,
         debug=False,
-        server_kwargs={"return_asyncio_server": True},
+        server_kwargs=None,
         *request_args,
         **request_kwargs,
     ):
         loop = self._loop
         results = [None, None]
         exceptions = []
+        server_kwargs = server_kwargs or {"return_asyncio_server": True}
         if gather_request:
 
             def _collect_request(request):
@@ -183,7 +115,6 @@ class ReuseableSanicTestClient(SanicTestClient):
                 )
                 results[-1] = response
             except Exception as e2:
-                # traceback.print_tb(e2.__traceback__)
                 exceptions.append(e2)
 
         if self._server is not None:
@@ -201,7 +132,6 @@ class ReuseableSanicTestClient(SanicTestClient):
                 loop._stopping = False
                 _server = loop.run_until_complete(_server_co)
             except Exception as e1:
-                # traceback.print_tb(e1.__traceback__)
                 raise e1
             self._server = _server
         server.trigger_events(self.app.listeners["after_server_start"], loop)
@@ -253,14 +183,10 @@ class ReuseableSanicTestClient(SanicTestClient):
         request_keepalive = kwargs.pop(
             "request_keepalive", CONFIG_FOR_TESTS["KEEP_ALIVE_TIMEOUT"]
         )
-        if self._session:
+        if not self._session:
-            _session = self._session
+            self._session = ResusableSanicSession()
-        else:
-            _session = ResusableSanicSession()
-            self._session = _session
-        async with _session as session:
         try:
-                response = await getattr(session, method.lower())(
+            response = await getattr(self._session, method.lower())(
                 url,
                 verify=False,
                 timeout=request_keepalive,
@@ -315,6 +241,7 @@ def test_keep_alive_timeout_reuse():
     """If the server keep-alive timeout and client keep-alive timeout are
      both longer than the delay, the client _and_ server will successfully
      reuse the existing connection."""
+    try:
         loop = asyncio.new_event_loop()
         asyncio.set_event_loop(loop)
         client = ReuseableSanicTestClient(keep_alive_timeout_app_reuse, loop)
@@ -326,12 +253,14 @@ def test_keep_alive_timeout_reuse():
         request, response = client.get("/1")
         assert response.status == 200
         assert response.text == "OK"
+    finally:
         client.kill_server()
 
 
 def test_keep_alive_client_timeout():
     """If the server keep-alive timeout is longer than the client
     keep-alive timeout, client will try to create a new connection here."""
+    try:
         loop = asyncio.new_event_loop()
         asyncio.set_event_loop(loop)
         client = ReuseableSanicTestClient(keep_alive_app_client_timeout, loop)
@@ -350,6 +279,7 @@ def test_keep_alive_client_timeout():
         assert exception is not None
         assert isinstance(exception, ValueError)
         assert "got a new connection" in exception.args[0]
+    finally:
         client.kill_server()
 
 
@@ -358,6 +288,7 @@ def test_keep_alive_server_timeout():
     keep-alive timeout, the client will either a 'Connection reset' error
     _or_ a new connection. Depending on how the event-loop handles the
     broken server connection."""
+    try:
         loop = asyncio.new_event_loop()
         asyncio.set_event_loop(loop)
         client = ReuseableSanicTestClient(keep_alive_app_server_timeout, loop)
@@ -379,4 +310,5 @@ def test_keep_alive_server_timeout():
             "Connection reset" in exception.args[0]
             or "got a new connection" in exception.args[0]
         )
+    finally:
         client.kill_server()

tests/test_request_stream.py

@@ -71,15 +71,13 @@ def test_request_stream_app(app):
     @app.post("/post/<id>", stream=True)
     async def post(request, id):
         assert isinstance(request.stream, StreamBuffer)
-
+        result = ""
-        async def streaming(response):
         while True:
             body = await request.stream.read()
             if body is None:
                 break
-                await response.write(body.decode("utf-8"))
+            result += body.decode("utf-8")
-
+        return text(result)
-        return stream(streaming)
 
     @app.put("/_put")
     async def _put(request):
@@ -89,15 +87,13 @@ def test_request_stream_app(app):
     @app.put("/put", stream=True)
     async def put(request):
         assert isinstance(request.stream, StreamBuffer)
-
+        result = ""
-        async def streaming(response):
         while True:
             body = await request.stream.read()
             if body is None:
                 break
-                await response.write(body.decode("utf-8"))
+            result += body.decode("utf-8")
-
+        return text(result)
-        return stream(streaming)
 
     @app.patch("/_patch")
     async def _patch(request):
@@ -107,15 +103,14 @@ def test_request_stream_app(app):
     @app.patch("/patch", stream=True)
     async def patch(request):
         assert isinstance(request.stream, StreamBuffer)
-
+        result = ""
-        async def streaming(response):
         while True:
             body = await request.stream.read()
             if body is None:
                 break
-                await response.write(body.decode("utf-8"))
+            result += body.decode("utf-8")
+        return text(result)
 
-        return stream(streaming)
 
     assert app.is_request_stream is True
 
@@ -166,15 +161,13 @@ def test_request_stream_handle_exception(app):
     @app.post("/post/<id>", stream=True)
     async def post(request, id):
         assert isinstance(request.stream, StreamBuffer)
-
+        result = ""
-        async def streaming(response):
         while True:
             body = await request.stream.read()
             if body is None:
                 break
-                await response.write(body.decode("utf-8"))
+            result += body.decode("utf-8")
-
+        return text(result)
-        return stream(streaming)
 
     # 404
     request, response = app.test_client.post("/in_valid_post", data=data)
@@ -222,15 +215,13 @@ def test_request_stream_blueprint(app):
     @bp.post("/post/<id>", stream=True)
     async def post(request, id):
         assert isinstance(request.stream, StreamBuffer)
-
+        result = ""
-        async def streaming(response):
         while True:
             body = await request.stream.read()
             if body is None:
                 break
-                await response.write(body.decode("utf-8"))
+            result += body.decode("utf-8")
-
+        return text(result)
-        return stream(streaming)
 
     @bp.put("/_put")
     async def _put(request):
@@ -240,15 +231,13 @@ def test_request_stream_blueprint(app):
     @bp.put("/put", stream=True)
     async def put(request):
         assert isinstance(request.stream, StreamBuffer)
-
+        result = ""
-        async def streaming(response):
         while True:
             body = await request.stream.read()
             if body is None:
                 break
-                await response.write(body.decode("utf-8"))
+            result += body.decode("utf-8")
-
+        return text(result)
-        return stream(streaming)
 
     @bp.patch("/_patch")
     async def _patch(request):
@@ -258,27 +247,23 @@ def test_request_stream_blueprint(app):
     @bp.patch("/patch", stream=True)
     async def patch(request):
         assert isinstance(request.stream, StreamBuffer)
-
+        result = ""
-        async def streaming(response):
         while True:
             body = await request.stream.read()
             if body is None:
                 break
-                await response.write(body.decode("utf-8"))
+            result += body.decode("utf-8")
-
+        return text(result)
-        return stream(streaming)
 
     async def post_add_route(request):
         assert isinstance(request.stream, StreamBuffer)
-
+        result = ""
-        async def streaming(response):
         while True:
             body = await request.stream.read()
             if body is None:
                 break
-                await response.write(body.decode("utf-8"))
+            result += body.decode("utf-8")
-
+        return text(result)
-        return stream(streaming)
 
     bp.add_route(
         post_add_route, "/post/add_route", methods=["POST"], stream=True
@@ -388,15 +373,13 @@ def test_request_stream(app):
     @app.post("/stream", stream=True)
     async def handler(request):
         assert isinstance(request.stream, StreamBuffer)
-
+        result = ""
-        async def streaming(response):
         while True:
             body = await request.stream.read()
             if body is None:
                 break
-                await response.write(body.decode("utf-8"))
+            result += body.decode("utf-8")
-
+        return text(result)
-        return stream(streaming)
 
     @app.get("/get")
     async def get(request):

tests/test_request_timeout.py

@@ -13,37 +13,26 @@ class DelayableSanicConnectionPool(httpcore.ConnectionPool):
         self._request_delay = request_delay
         super().__init__(*args, **kwargs)
 
-    async def request(
+    async def send(
         self,
-        method,
+        request,
-        url,
-        headers=(),
-        body=b"",
         stream=False,
         ssl=None,
         timeout=None,
     ):
-        if ssl is None:
+        connection = await self.acquire_connection(request.url.origin)
-            ssl = self.ssl_config
+        if connection.h11_connection is None and connection.h2_connection is None:
-        if timeout is None:
+            await connection.connect(ssl=ssl, timeout=timeout)
-            timeout = self.timeout
-
-        parsed_url = httpcore.URL(url)
-        request = httpcore.Request(
-            method, parsed_url, headers=headers, body=body
-        )
-        connection = await self.acquire_connection(
-            parsed_url, ssl=ssl, timeout=timeout
-        )
         if self._request_delay:
-            print(f"\t>> Sleeping ({self._request_delay})")
             await asyncio.sleep(self._request_delay)
-        response = await connection.send(request)
-        if not stream:
         try:
-                await response.read()
+            response = await connection.send(
-            finally:
+                request, stream=stream, ssl=ssl, timeout=timeout
-                await response.close()
+            )
+        except BaseException as exc:
+            self.active_connections.remove(connection)
+            self.max_connections.release()
+            raise exc
         return response
 
 

tox.ini

@@ -12,8 +12,8 @@ deps =
     pytest-cov
     pytest-sanic
     pytest-sugar
-    httpcore==0.1.1
+    httpcore==0.3.0
-    requests-async==0.4.0
+    requests-async==0.5.0
     chardet<=2.3.0
     beautifulsoup4
     gunicorn