Version 20.9 (#1940)

* New aproach for uploading sanic app config.

* Update config.rst

Co-authored-by: tigerthelion <bjt.thompson@gmail.com>
Co-authored-by: Adam Hopkins <admhpkns@gmail.com>

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,5 @@
+blank_issues_enabled: true
+contact_links:
+  - name: Questions and Help
+    url: https://community.sanicframework.org/c/questions-and-help
+    about: Do you need help with Sanic? Ask your questions here.

.github/ISSUE_TEMPLATE/help-wanted.md

@@ -1,13 +0,0 @@
----
-name: Help wanted
-about: Do you need help? Try community.sanicframework.org
-
----
-
-*DELETE ALL BEFORE POSTING*
-*Post your HELP WANTED questions on [the community forum](https://community.sanicframework.org/)*.
-
-Checkout the community forum before posting any question here.
-We prefer if you put these kinds of questions here:
-
-https://community.sanicframework.org/c/questions-and-help

CHANGELOG.rst

@@ -1,3 +1,73 @@
+Version 20.9.0
+===============
+
+
+Features
+********
+
+  *
+    `#1887 <https://github.com/huge-success/sanic/pull/1887>`_
+    Pass subprotocols in websockets (both sanic server and ASGI)
+
+  *
+    `#1894 <https://github.com/huge-success/sanic/pull/1894>`_
+    Automatically set ``test_mode`` flag on app instance
+  
+  *
+    `#1903 <https://github.com/huge-success/sanic/pull/1903>`_ 
+    Add new unified method for updating app values
+  
+  *
+    `#1906 <https://github.com/huge-success/sanic/pull/1906>`_,
+    `#1909 <https://github.com/huge-success/sanic/pull/1909>`_
+    Adds WEBSOCKET_PING_TIMEOUT and WEBSOCKET_PING_INTERVAL configuration values
+  
+  *
+    `#1935 <https://github.com/huge-success/sanic/pull/1935>`_
+    httpx version dependency updated, it is slated for removal as a dependency in v20.12
+
+  *
+    `#1937 <https://github.com/huge-success/sanic/pull/1937>`_ 
+    Added auto, text, and json fallback error handlers (in v21.3, the default will change form html to auto)
+
+Bugfixes
+********
+  
+  *
+    `#1897 <https://github.com/huge-success/sanic/pull/1897>`_
+    Resolves exception from unread bytes in stream
+
+Deprecations and Removals
+*************************
+
+  *
+    `#1903 <https://github.com/huge-success/sanic/pull/1903>`_
+    config.from_envar, config.from_pyfile, and config.from_object are deprecated and set to be removed in v21.3
+
+Developer infrastructure
+************************
+
+  *
+    `#1890 <https://github.com/huge-success/sanic/pull/1890>`_,
+    `#1891 <https://github.com/huge-success/sanic/pull/1891>`_
+    Update isort calls to be compatible with new API
+
+  *
+    `#1893 <https://github.com/huge-success/sanic/pull/1893>`_
+    Remove version section from setup.cfg
+
+  *
+    `#1924 <https://github.com/huge-success/sanic/pull/1924>`_
+    Adding --strict-markers for pytest
+
+Improved Documentation
+**********************
+
+  *
+    `#1922 <https://github.com/huge-success/sanic/pull/1922>`_
+    Add explicit ASGI compliance to the README
+
+
 Version 20.6.3
 ===============
 
@@ -299,7 +369,7 @@ Features
 
   * 
     `#1562 <https://github.com/huge-success/sanic/pull/1562>`_
-    Remove ``aiohttp`` dependencey and create new ``SanicTestClient`` based upon
+    Remove ``aiohttp`` dependency and create new ``SanicTestClient`` based upon
     `requests-async <https://github.com/encode/requests-async>`_
 
   * 

Makefile

@@ -71,7 +71,7 @@ black:
 	black --config ./.black.toml sanic tests
 
 fix-import: black
-	isort -rc sanic tests
+	isort sanic tests
 
 
 docs-clean:

README.rst

@@ -58,6 +58,8 @@ Sanic | Build fast. Run fast.
 
 Sanic is a **Python 3.6+** web server and web framework that's written to go fast. It allows the usage of the ``async/await`` syntax added in Python 3.5, which makes your code non-blocking and speedy.
 
+Sanic is also ASGI compliant, so you can deploy it with an `alternative ASGI webserver <https://sanic.readthedocs.io/en/latest/sanic/deploying.html#running-via-asgi>`_.
+
 `Source code on GitHub <https://github.com/huge-success/sanic/>`_ | `Help and discussion board <https://community.sanicframework.org/>`_. 
 
 The project is maintained by the community, for the community. **Contributions are welcome!**
@@ -104,7 +106,7 @@ Hello World Example
     if __name__ == '__main__':
         app.run(host='0.0.0.0', port=8000)
         
-Sanic can now be easily run using ``python3 hello.py``.
+Sanic can now be easily run using ``sanic hello.app``.
 
 .. code::
 

SECURITY.md

@@ -5,10 +5,15 @@
 Sanic releases long term support release once a year in December. LTS releases receive bug and security updates for **24 months**. Interim releases throughout the year occur every three months, and are supported until the subsequent interim release.
 
 | Version | LTS           | Supported          |
-| ------- | ------------------ | ------------------ |
-| 19.6.0  |                    | :white_check_mark: |
-| 19.3.1  |                    | :heavy_check_mark: |
-| 18.12.0 | :heavy_check_mark: | :heavy_check_mark: |
+| ------- | ------------- | ------------------ |
+| 20.9    |               | :heavy_check_mark: |
+| 20.6    |               | :x:                |
+| 20.3    |               | :x:                |
+| 19.12   | until 2021-12 | :white_check_mark: |
+| 19.9    |               | :x:                |
+| 19.6    |               | :x:                |
+| 19.3    |               | :x:                |
+| 18.12   | until 2020-12 | :white_check_mark: |
 | 0.8.3   |               | :x:                |
 | 0.7.0   |               | :x:                |
 | 0.6.0   |               | :x:                |
@@ -18,6 +23,9 @@ Sanic releases long term support release once a year in December. LTS releases r
 | 0.2.0   |               | :x:                |
 | 0.1.9   |               | :x:                |
 
+:white_check_mark: = security/bug fixes
+:heavy_check_mark: = full support
+
 ## Reporting a Vulnerability
 
 If you discover a security vulnerability, we ask that you **do not** create an issue on GitHub. Instead, please [send a message to the core-devs](https://community.sanicframework.org/g/core-devs) on the community forums. Once logged in, you can send a message to the core-devs by clicking the message button.

changelogs/1892.removal.rst

@@ -0,0 +1 @@
+Remove [version] section.

changelogs/1904.feature.rst

@@ -0,0 +1,3 @@
+Adds WEBSOCKET_PING_TIMEOUT and WEBSOCKET_PING_INTERVAL configuration values
+
+Allows setting the ping_interval and ping_timeout arguments when initializing `WebSocketCommonProtocol`.

docs/sanic/config.rst

@@ -12,9 +12,9 @@ Sanic holds the configuration in the `config` attribute of the application objec
 
     app = Sanic('myapp')
     app.config.DB_NAME = 'appdb'
-    app.config.DB_USER = 'appuser'
+    app.config['DB_USER'] = 'appuser'
 
-Since the config object actually is a dictionary, you can use its `update` method in order to set several values at once:
+Since the config object has a type that inherits from dictionary, you can use its ``update`` method in order to set several values at once:
 
 .. code-block:: python
 
@@ -47,9 +47,90 @@ Then the above variable would be `MYAPP_REQUEST_TIMEOUT`. If you want to disable
 
     app = Sanic(__name__, load_env=False)   
 
+From file, dict, or any object (having __dict__ attribute).
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can store app configurations in: (1) a Python file, (2) a dictionary, or (3) in some other type of custom object.
+
+In order to load configuration from ove of those, you can use ``app.upload_config()``.
+
+**1) From file**
+
+
+Let's say you have ``my_config.py`` file that looks like this:
+
+.. code-block:: python
+
+    # my_config.py
+    A = 1
+    B = 2
+
+Loading config from this file is as easy as:
+
+.. code-block:: python
+
+    app.update_config("/path/to/my_config.py")
+ 
+You can also use environment variables in the path name here.
+
+Let's say you have an environment variable like this:
+
+.. code-block:: shell
+
+    $ export my_path="/path/to"
+    
+Then you can use it like this:
+
+.. code-block:: python
+
+    app.update_config("${my_path}/my_config.py")
+
+.. note::
+
+    Just remember that you have to provide environment variables in the format ${environment_variable} and that $environment_variable is not expanded (is treated as "plain" text).
+
+**2) From dict**
+
+You can also set your app config by providing a ``dict``:
+
+.. code-block:: python
+
+    d = {"A": 1, "B": 2}
+    
+    app.update_config(d)
+ 
+**3) From _any_ object**
+
+App config can be taken from an object. Internally, it uses ``__dict__`` to retrieve keys and values.
+
+For example, pass the class:
+
+.. code-block:: python
+
+    class C:
+        A = 1
+        B = 2
+        
+    app.update_config(C)
+
+or, it can be instantiated:
+
+.. code-block:: python
+
+    c = C()
+    
+    app.update_config(c)
+    
+- From an object (having __dict__ attribute)
+
+
 From an Object
 ~~~~~~~~~~~~~~
 
+.. note::
+
+     Deprecated, will be removed in version 21.3.
+
 If there are a lot of configuration values and they have sensible defaults it might be helpful to put them into a module:
 
 .. code-block:: python
@@ -71,6 +152,10 @@ You could use a class or any other object as well.
 From a File
 ~~~~~~~~~~~
 
+.. note::
+
+     Deprecated, will be removed in version 21.3.
+
 Usually you will want to load configuration from a file that is not part of the distributed application. You can load configuration from a file using `from_pyfile(/path/to/config_file)`. However, that requires the program to know the path to the config file. So instead you can specify the location of the config file in an environment variable and tell Sanic to use that to find the config file:
 
 .. code-block:: python
@@ -98,7 +183,7 @@ The config files are regular Python files which are executed in order to load th
 Builtin Configuration Values
 ----------------------------
 
-Out of the box there are just a few predefined values which can be overwritten when creating the application.
+Out of the box there are just a few predefined values which can be overwritten when creating the application. Note that websocket configuration values will have no impact if running in ASGI mode.
 
 +---------------------------+-------------------+-----------------------------------------------------------------------------+
 | Variable                  | Default           | Description                                                                 |
@@ -123,6 +208,10 @@ Out of the box there are just a few predefined values which can be overwritten w
 +---------------------------+-------------------+-----------------------------------------------------------------------------+
 | WEBSOCKET_WRITE_LIMIT     | 2^16              | High-water limit of the buffer for outgoing bytes                           |
 +---------------------------+-------------------+-----------------------------------------------------------------------------+
+| WEBSOCKET_PING_INTERVAL   | 20                | A Ping frame is sent every ping_interval seconds.                           |
++---------------------------+-------------------+-----------------------------------------------------------------------------+
+| WEBSOCKET_PING_TIMEOUT    | 20                | Connection is closed when Pong is not received after ping_timeout seconds   |
++---------------------------+-------------------+-----------------------------------------------------------------------------+
 | GRACEFUL_SHUTDOWN_TIMEOUT | 15.0              | How long to wait to force close non-idle connection (sec)                   |
 +---------------------------+-------------------+-----------------------------------------------------------------------------+
 | ACCESS_LOG                | True              | Disable or enable access log                                                |

docs/sanic/testing.rst

@@ -58,6 +58,36 @@ More information about
 the available arguments to `httpx` can be found
 [in the documentation for `httpx <https://www.encode.io/httpx/>`_.
 
+Additionally, Sanic has an asynchronous testing client. The difference is that the async client will not stand up an
+instance of your application, but will instead reach inside it using ASGI. All listeners and middleware are still
+executed.
+
+.. code-block:: python
+    @pytest.mark.asyncio
+    async def test_index_returns_200():
+        request, response = await app.asgi_client.put('/')
+        assert response.status == 200
+.. note::
+
+    Whenever one of the test clients run, you can test your app instance to determine if it is in testing mode:
+    `app.test_mode`.
+
+Additionally, Sanic has an asynchronous testing client. The difference is that the async client will not stand up an
+instance of your application, but will instead reach inside it using ASGI. All listeners and middleware are still
+executed.
+
+.. code-block:: python
+
+    @pytest.mark.asyncio
+    async def test_index_returns_200():
+        request, response = await app.asgi_client.put('/')
+        assert response.status == 200
+
+.. note::
+
+    Whenever one of the test clients run, you can test your app instance to determine if it is in testing mode:
+    `app.test_mode`.
+
 
 Using a random port
 -------------------

docs/sanic/websocket.rst

@@ -51,5 +51,9 @@ You could setup your own WebSocket configuration through ``app.config``, like
     app.config.WEBSOCKET_MAX_QUEUE = 32
     app.config.WEBSOCKET_READ_LIMIT = 2 ** 16
     app.config.WEBSOCKET_WRITE_LIMIT = 2 ** 16
+    app.config.WEBSOCKET_PING_INTERVAL = 20
+    app.config.WEBSOCKET_PING_TIMEOUT = 20
+
+These settings will have no impact if running in ASGI mode.
 
 Find more in ``Configuration`` section.

sanic/__version__.py

@@ -1 +1 @@
-__version__ = "20.6.3"
+__version__ = "20.9.0"

sanic/app.py

@@ -90,6 +90,7 @@ class Sanic:
         self.named_response_middleware = {}
         # Register alternative method names
         self.go_fast = self.run
+        self.test_mode = False
 
     @property
     def loop(self):
@@ -1451,3 +1452,13 @@ class Sanic:
         self.asgi = True
         asgi_app = await ASGIApp.create(self, scope, receive, send)
         await asgi_app()
+
+    # -------------------------------------------------------------------- #
+    # Configuration
+    # -------------------------------------------------------------------- #
+    def update_config(self, config: Union[bytes, str, dict, Any]):
+        """Update app.config.
+
+        Please refer to config.py::Config.update_config for documentation."""
+
+        self.config.update_config(config)

sanic/asgi.py

@@ -98,7 +98,9 @@ class MockTransport:
     def create_websocket_connection(
         self, send: ASGISend, receive: ASGIReceive
     ) -> WebSocketConnection:
-        self._websocket_connection = WebSocketConnection(send, receive)
+        self._websocket_connection = WebSocketConnection(
+            send, receive, self.scope.get("subprotocols", [])
+        )
         return self._websocket_connection
 
     def add_task(self) -> None:

sanic/compat.py

@@ -14,7 +14,8 @@ class Header(CIMultiDict):
 use_trio = argv[0].endswith("hypercorn") and "trio" in argv
 
 if use_trio:
-    from trio import open_file as open_async, Path  # type: ignore
+    from trio import Path  # type: ignore
+    from trio import open_file as open_async  # type: ignore
 
     def stat_async(path):
         return Path(path).stat()

sanic/config.py

@@ -1,8 +1,15 @@
-import os
-import types
+from os import environ
+from typing import Any, Union
 
-from sanic.exceptions import PyFileError
-from sanic.helpers import import_string
+# NOTE(tomaszdrozdz): remove in version: 21.3
+# We replace from_envvar(), from_object(), from_pyfile() config object methods
+# with one simpler update_config() method.
+# We also replace "loading module from file code" in from_pyfile()
+# in a favour of load_module_from_file_location().
+# Please see pull request: 1903
+# and issue: 1895
+from .deprecated import from_envvar, from_object, from_pyfile  # noqa
+from .utils import load_module_from_file_location, str_to_bool
 
 
 SANIC_PREFIX = "SANIC_"
@@ -24,12 +31,15 @@ DEFAULT_CONFIG = {
     "WEBSOCKET_MAX_QUEUE": 32,
     "WEBSOCKET_READ_LIMIT": 2 ** 16,
     "WEBSOCKET_WRITE_LIMIT": 2 ** 16,
+    "WEBSOCKET_PING_TIMEOUT": 20,
+    "WEBSOCKET_PING_INTERVAL": 20,
     "GRACEFUL_SHUTDOWN_TIMEOUT": 15.0,  # 15 sec
     "ACCESS_LOG": True,
     "FORWARDED_SECRET": None,
     "REAL_IP_HEADER": None,
     "PROXIES_COUNT": None,
     "FORWARDED_FOR_HEADER": "X-Forwarded-For",
+    "FALLBACK_ERROR_FORMAT": "html",
 }
 
 
@@ -56,76 +66,23 @@ class Config(dict):
     def __setattr__(self, attr, value):
         self[attr] = value
 
-    def from_envvar(self, variable_name):
-        """Load a configuration from an environment variable pointing to
-        a configuration file.
-
-        :param variable_name: name of the environment variable
-        :return: bool. ``True`` if able to load config, ``False`` otherwise.
-        """
-        config_file = os.environ.get(variable_name)
-        if not config_file:
-            raise RuntimeError(
-                "The environment variable %r is not set and "
-                "thus configuration could not be loaded." % variable_name
-            )
-        return self.from_pyfile(config_file)
-
-    def from_pyfile(self, filename):
-        """Update the values in the config from a Python file.
-        Only the uppercase variables in that module are stored in the config.
-
-        :param filename: an absolute path to the config file
-        """
-        module = types.ModuleType("config")
-        module.__file__ = filename
-        try:
-            with open(filename) as config_file:
-                exec(  # nosec
-                    compile(config_file.read(), filename, "exec"),
-                    module.__dict__,
-                )
-        except IOError as e:
-            e.strerror = "Unable to load configuration file (%s)" % e.strerror
-            raise
-        except Exception as e:
-            raise PyFileError(filename) from e
-
-        self.from_object(module)
-        return True
-
-    def from_object(self, obj):
-        """Update the values from the given object.
-        Objects are usually either modules or classes.
-
-        Just the uppercase variables in that object are stored in the config.
-        Example usage::
-
-            from yourapplication import default_config
-            app.config.from_object(default_config)
-
-            or also:
-            app.config.from_object('myproject.config.MyConfigClass')
-
-        You should not use this function to load the actual configuration but
-        rather configuration defaults. The actual config should be loaded
-        with :meth:`from_pyfile` and ideally from a location not within the
-        package because the package might be installed system wide.
-
-        :param obj: an object holding the configuration
-        """
-        if isinstance(obj, str):
-            obj = import_string(obj)
-        for key in dir(obj):
-            if key.isupper():
-                self[key] = getattr(obj, key)
+    # NOTE(tomaszdrozdz): remove in version: 21.3
+    # We replace from_envvar(), from_object(), from_pyfile() config object
+    # methods with one simpler update_config() method.
+    # We also replace "loading module from file code" in from_pyfile()
+    # in a favour of load_module_from_file_location().
+    # Please see pull request: 1903
+    # and issue: 1895
+    from_envvar = from_envvar
+    from_pyfile = from_pyfile
+    from_object = from_object
 
     def load_environment_vars(self, prefix=SANIC_PREFIX):
         """
         Looks for prefixed environment variables and applies
         them to the configuration if present.
         """
-        for k, v in os.environ.items():
+        for k, v in environ.items():
             if k.startswith(prefix):
                 _, config_key = k.split(prefix, 1)
                 try:
@@ -135,23 +92,47 @@ class Config(dict):
                         self[config_key] = float(v)
                     except ValueError:
                         try:
-                            self[config_key] = strtobool(v)
+                            self[config_key] = str_to_bool(v)
                         except ValueError:
                             self[config_key] = v
 
+    def update_config(self, config: Union[bytes, str, dict, Any]):
+        """Update app.config.
 
-def strtobool(val):
-    """
-    This function was borrowed from distutils.utils. While distutils
-    is part of stdlib, it feels odd to use distutils in main application code.
+        Note:: only upper case settings are considered.
 
-    The function was modified to walk its talk and actually return bool
-    and not int.
-    """
-    val = val.lower()
-    if val in ("y", "yes", "t", "true", "on", "1"):
-        return True
-    elif val in ("n", "no", "f", "false", "off", "0"):
-        return False
-    else:
-        raise ValueError("invalid truth value %r" % (val,))
+        You can upload app config by providing path to py file
+        holding settings.
+
+            # /some/py/file
+            A = 1
+            B = 2
+
+        config.update_config("${some}/py/file")
+
+        Yes you can put environment variable here, but they must be provided
+        in format: ${some_env_var}, and mark that $some_env_var is treated
+        as plain string.
+
+        You can upload app config by providing dict holding settings.
+
+            d = {"A": 1, "B": 2}
+            config.update_config(d)
+
+        You can upload app config by providing any object holding settings,
+        but in such case config.__dict__ will be used as dict holding settings.
+
+            class C:
+                A = 1
+                B = 2
+            config.update_config(C)"""
+
+        if isinstance(config, (bytes, str)):
+            config = load_module_from_file_location(location=config)
+
+        if not isinstance(config, dict):
+            config = config.__dict__
+
+        config = dict(filter(lambda i: i[0].isupper(), config.items()))
+
+        self.update(config)

sanic/deprecated.py

@@ -0,0 +1,106 @@
+# NOTE(tomaszdrozdz): remove in version: 21.3
+# We replace from_envvar(), from_object(), from_pyfile() config object methods
+# with one simpler update_config() method.
+# We also replace "loading module from file code" in from_pyfile()
+# in a favour of load_module_from_file_location().
+# Please see pull request: 1903
+# and issue: 1895
+import types
+
+from os import environ
+from typing import Any
+from warnings import warn
+
+from sanic.exceptions import PyFileError
+from sanic.helpers import import_string
+
+
+def from_envvar(self, variable_name: str) -> bool:
+    """Load a configuration from an environment variable pointing to
+    a configuration file.
+
+    :param variable_name: name of the environment variable
+    :return: bool. ``True`` if able to load config, ``False`` otherwise.
+    """
+
+    warn(
+        "Using `from_envvar` method is deprecated and will be removed in "
+        "v21.3, use `app.update_config` method instead.",
+        DeprecationWarning,
+        stacklevel=2,
+    )
+
+    config_file = environ.get(variable_name)
+    if not config_file:
+        raise RuntimeError(
+            f"The environment variable {variable_name} is not set and "
+            f"thus configuration could not be loaded."
+        )
+    return self.from_pyfile(config_file)
+
+
+def from_pyfile(self, filename: str) -> bool:
+    """Update the values in the config from a Python file.
+    Only the uppercase variables in that module are stored in the config.
+
+    :param filename: an absolute path to the config file
+    """
+
+    warn(
+        "Using `from_pyfile` method is deprecated and will be removed in "
+        "v21.3, use `app.update_config` method instead.",
+        DeprecationWarning,
+        stacklevel=2,
+    )
+
+    module = types.ModuleType("config")
+    module.__file__ = filename
+    try:
+        with open(filename) as config_file:
+            exec(  # nosec
+                compile(config_file.read(), filename, "exec"),
+                module.__dict__,
+            )
+    except IOError as e:
+        e.strerror = "Unable to load configuration file (e.strerror)"
+        raise
+    except Exception as e:
+        raise PyFileError(filename) from e
+
+    self.from_object(module)
+    return True
+
+
+def from_object(self, obj: Any) -> None:
+    """Update the values from the given object.
+    Objects are usually either modules or classes.
+
+    Just the uppercase variables in that object are stored in the config.
+    Example usage::
+
+        from yourapplication import default_config
+        app.config.from_object(default_config)
+
+        or also:
+        app.config.from_object('myproject.config.MyConfigClass')
+
+    You should not use this function to load the actual configuration but
+    rather configuration defaults. The actual config should be loaded
+    with :meth:`from_pyfile` and ideally from a location not within the
+    package because the package might be installed system wide.
+
+    :param obj: an object holding the configuration
+    """
+
+    warn(
+        "Using `from_object` method is deprecated and will be removed in "
+        "v21.3, use `app.update_config` method instead.",
+        DeprecationWarning,
+        stacklevel=2,
+    )
+
+    if isinstance(obj, str):
+        obj = import_string(obj)
+    for key in dir(obj):
+        if key.isupper():
+            self[key] = getattr(obj, key)

sanic/errorpages.py

@@ -1,88 +1,75 @@
 import sys
+import typing as t
 
+from functools import partial
 from traceback import extract_tb
 
-from sanic.exceptions import SanicException
+from sanic.exceptions import InvalidUsage, SanicException
 from sanic.helpers import STATUS_CODES
-from sanic.response import html
+from sanic.request import Request
+from sanic.response import HTTPResponse, html, json, text
 
 
-# Here, There Be Dragons (custom HTML formatting to follow)
+try:
+    from ujson import dumps
+
+    dumps = partial(dumps, escape_forward_slashes=False)
+except ImportError:  # noqa
+    from json import dumps  # type: ignore
 
 
-def escape(text):
-    """Minimal HTML escaping, not for attribute values (unlike html.escape)."""
-    return f"{text}".replace("&", "&amp;").replace("<", "&lt;")
-
-
-def exception_response(request, exception, debug):
-    status = 500
-    text = (
-        "The server encountered an internal error "
-        "and cannot complete your request."
+FALLBACK_TEXT = (
+    "The server encountered an internal error and "
+    "cannot complete your request."
 )
+FALLBACK_STATUS = 500
 
-    headers = {}
-    if isinstance(exception, SanicException):
-        text = f"{exception}"
-        status = getattr(exception, "status_code", status)
-        headers = getattr(exception, "headers", headers)
-    elif debug:
-        text = f"{exception}"
 
-    status_text = STATUS_CODES.get(status, b"Error Occurred").decode()
-    title = escape(f"{status} — {status_text}")
-    text = escape(text)
+class BaseRenderer:
+    def __init__(self, request, exception, debug):
+        self.request = request
+        self.exception = exception
+        self.debug = debug
 
-    if debug and not getattr(exception, "quiet", False):
-        return html(
-            f"<!DOCTYPE html><meta charset=UTF-8><title>{title}</title>"
-            f"<style>{TRACEBACK_STYLE}</style>\n"
-            f"<h1>⚠️ {title}</h1><p>{text}\n"
-            f"{_render_traceback_html(request, exception)}",
-            status=status,
+    @property
+    def headers(self):
+        if isinstance(self.exception, SanicException):
+            return getattr(self.exception, "headers", {})
+        return {}
+
+    @property
+    def status(self):
+        if isinstance(self.exception, SanicException):
+            return getattr(self.exception, "status_code", FALLBACK_STATUS)
+        return FALLBACK_STATUS
+
+    @property
+    def text(self):
+        if self.debug or isinstance(self.exception, SanicException):
+            return str(self.exception)
+        return FALLBACK_TEXT
+
+    @property
+    def title(self):
+        status_text = STATUS_CODES.get(self.status, b"Error Occurred").decode()
+        return f"{self.status} — {status_text}"
+
+    def render(self):
+        output = (
+            self.full
+            if self.debug and not getattr(self.exception, "quiet", False)
+            else self.minimal
         )
+        return output()
 
-    # Keeping it minimal with trailing newline for pretty curl/console output
-    return html(
-        f"<!DOCTYPE html><meta charset=UTF-8><title>{title}</title>"
-        "<style>html { font-family: sans-serif }</style>\n"
-        f"<h1>⚠️ {title}</h1><p>{text}\n",
-        status=status,
-        headers=headers,
-    )
-
-
-def _render_exception(exception):
-    frames = extract_tb(exception.__traceback__)
-    frame_html = "".join(TRACEBACK_LINE_HTML.format(frame) for frame in frames)
-    return TRACEBACK_WRAPPER_HTML.format(
-        exc_name=escape(exception.__class__.__name__),
-        exc_value=escape(exception),
-        frame_html=frame_html,
-    )
-
-
-def _render_traceback_html(request, exception):
-    exc_type, exc_value, tb = sys.exc_info()
-    exceptions = []
-    while exc_value:
-        exceptions.append(_render_exception(exc_value))
-        exc_value = exc_value.__cause__
-
-    traceback_html = TRACEBACK_BORDER.join(reversed(exceptions))
-    appname = escape(request.app.name)
-    name = escape(exception.__class__.__name__)
-    value = escape(exception)
-    path = escape(request.path)
-    return (
-        f"<h2>Traceback of {appname} (most recent call last):</h2>"
-        f"{traceback_html}"
-        "<div class=summary><p>"
-        f"<b>{name}: {value}</b> while handling path <code>{path}</code>"
-    )
+    def minimal(self):  # noqa
+        raise NotImplementedError
+
+    def full(self):  # noqa
+        raise NotImplementedError
 
 
+class HTMLRenderer(BaseRenderer):
     TRACEBACK_STYLE = """
         html { font-family: sans-serif }
         h2 { color: #888; }
@@ -95,18 +82,15 @@ TRACEBACK_STYLE = """
         .tb-header { background: #eee; padding: 0.3rem; font-weight: bold }
         .frame-descriptor { background: #e2eafb; font-size: 14px }
     """
-
     TRACEBACK_WRAPPER_HTML = (
         "<div class=tb-header>{exc_name}: {exc_value}</div>"
         "<div class=tb-wrapper>{frame_html}</div>"
     )
-
     TRACEBACK_BORDER = (
         "<div class=frame-border>"
         "The above exception was the direct cause of the following exception:"
         "</div>"
     )
-
     TRACEBACK_LINE_HTML = (
         "<div class=frame-line>"
         "<p class=frame-descriptor>"
@@ -115,3 +99,232 @@ TRACEBACK_LINE_HTML = (
         "<p class=frame-code><code>{0.line}</code>"
         "</div>"
     )
+    OUTPUT_HTML = (
+        "<!DOCTYPE html><html lang=en>"
+        "<meta charset=UTF-8><title>{title}</title>\n"
+        "<style>{style}</style>\n"
+        "<h1>{title}</h1><p>{text}\n"
+        "{body}"
+    )
+
+    def full(self):
+        return html(
+            self.OUTPUT_HTML.format(
+                title=self.title,
+                text=self.text,
+                style=self.TRACEBACK_STYLE,
+                body=self._generate_body(),
+            ),
+            status=self.status,
+        )
+
+    def minimal(self):
+        return html(
+            self.OUTPUT_HTML.format(
+                title=self.title,
+                text=self.text,
+                style=self.TRACEBACK_STYLE,
+                body="",
+            ),
+            status=self.status,
+            headers=self.headers,
+        )
+
+    @property
+    def text(self):
+        return escape(super().text)
+
+    @property
+    def title(self):
+        return escape(f"⚠️ {super().title}")
+
+    def _generate_body(self):
+        _, exc_value, __ = sys.exc_info()
+        exceptions = []
+        while exc_value:
+            exceptions.append(self._format_exc(exc_value))
+            exc_value = exc_value.__cause__
+
+        traceback_html = self.TRACEBACK_BORDER.join(reversed(exceptions))
+        appname = escape(self.request.app.name)
+        name = escape(self.exception.__class__.__name__)
+        value = escape(self.exception)
+        path = escape(self.request.path)
+        lines = [
+            f"<h2>Traceback of {appname} (most recent call last):</h2>",
+            f"{traceback_html}",
+            "<div class=summary><p>",
+            f"<b>{name}: {value}</b> while handling path <code>{path}</code>",
+            "</div>",
+        ]
+        return "\n".join(lines)
+
+    def _format_exc(self, exc):
+        frames = extract_tb(exc.__traceback__)
+        frame_html = "".join(
+            self.TRACEBACK_LINE_HTML.format(frame) for frame in frames
+        )
+        return self.TRACEBACK_WRAPPER_HTML.format(
+            exc_name=escape(exc.__class__.__name__),
+            exc_value=escape(exc),
+            frame_html=frame_html,
+        )
+
+
+class TextRenderer(BaseRenderer):
+    OUTPUT_TEXT = "{title}\n{bar}\n{text}\n\n{body}"
+    SPACER = "  "
+
+    def full(self):
+        return text(
+            self.OUTPUT_TEXT.format(
+                title=self.title,
+                text=self.text,
+                bar=("=" * len(self.title)),
+                body=self._generate_body(),
+            ),
+            status=self.status,
+        )
+
+    def minimal(self):
+        return text(
+            self.OUTPUT_TEXT.format(
+                title=self.title,
+                text=self.text,
+                bar=("=" * len(self.title)),
+                body="",
+            ),
+            status=self.status,
+            headers=self.headers,
+        )
+
+    @property
+    def title(self):
+        return f"⚠️ {super().title}"
+
+    def _generate_body(self):
+        _, exc_value, __ = sys.exc_info()
+        exceptions = []
+
+        # traceback_html = self.TRACEBACK_BORDER.join(reversed(exceptions))
+        lines = [
+            f"{self.exception.__class__.__name__}: {self.exception} while "
+            f"handling path {self.request.path}",
+            f"Traceback of {self.request.app.name} (most recent call last):\n",
+        ]
+
+        while exc_value:
+            exceptions.append(self._format_exc(exc_value))
+            exc_value = exc_value.__cause__
+
+        return "\n".join(lines + exceptions[::-1])
+
+    def _format_exc(self, exc):
+        frames = "\n\n".join(
+            [
+                f"{self.SPACER * 2}File {frame.filename}, "
+                f"line {frame.lineno}, in "
+                f"{frame.name}\n{self.SPACER * 2}{frame.line}"
+                for frame in extract_tb(exc.__traceback__)
+            ]
+        )
+        return f"{self.SPACER}{exc.__class__.__name__}: {exc}\n{frames}"
+
+
+class JSONRenderer(BaseRenderer):
+    def full(self):
+        output = self._generate_output(full=True)
+        return json(output, status=self.status, dumps=dumps)
+
+    def minimal(self):
+        output = self._generate_output(full=False)
+        return json(output, status=self.status, dumps=dumps)
+
+    def _generate_output(self, *, full):
+        output = {
+            "description": self.title,
+            "status": self.status,
+            "message": self.text,
+        }
+
+        if full:
+            _, exc_value, __ = sys.exc_info()
+            exceptions = []
+
+            while exc_value:
+                exceptions.append(
+                    {
+                        "type": exc_value.__class__.__name__,
+                        "exception": str(exc_value),
+                        "frames": [
+                            {
+                                "file": frame.filename,
+                                "line": frame.lineno,
+                                "name": frame.name,
+                                "src": frame.line,
+                            }
+                            for frame in extract_tb(exc_value.__traceback__)
+                        ],
+                    }
+                )
+                exc_value = exc_value.__cause__
+
+            output["path"] = self.request.path
+            output["args"] = self.request.args
+            output["exceptions"] = exceptions[::-1]
+
+        return output
+
+    @property
+    def title(self):
+        return STATUS_CODES.get(self.status, b"Error Occurred").decode()
+
+
+def escape(text):
+    """Minimal HTML escaping, not for attribute values (unlike html.escape)."""
+    return f"{text}".replace("&", "&amp;").replace("<", "&lt;")
+
+
+RENDERERS_BY_CONFIG = {
+    "html": HTMLRenderer,
+    "json": JSONRenderer,
+    "text": TextRenderer,
+}
+
+RENDERERS_BY_CONTENT_TYPE = {
+    "multipart/form-data": HTMLRenderer,
+    "application/json": JSONRenderer,
+    "text/plain": TextRenderer,
+}
+
+
+def exception_response(
+    request: Request,
+    exception: Exception,
+    debug: bool,
+    renderer: t.Type[t.Optional[BaseRenderer]] = None,
+) -> HTTPResponse:
+    """Render a response for the default FALLBACK exception handler"""
+
+    if not renderer:
+        renderer = HTMLRenderer
+
+        if request:
+            if request.app.config.FALLBACK_ERROR_FORMAT == "auto":
+                try:
+                    renderer = JSONRenderer if request.json else HTMLRenderer
+                except InvalidUsage:
+                    renderer = HTMLRenderer
+
+                content_type, *_ = request.headers.get(
+                    "content-type", ""
+                ).split(";")
+                renderer = RENDERERS_BY_CONTENT_TYPE.get(
+                    content_type, renderer
+                )
+            else:
+                render_format = request.app.config.FALLBACK_ERROR_FORMAT
+                renderer = RENDERERS_BY_CONFIG.get(render_format, renderer)
+
+    renderer = t.cast(t.Type[BaseRenderer], renderer)
+    return renderer(request, exception, debug).render()

sanic/exceptions.py

@@ -169,6 +169,10 @@ class Unauthorized(SanicException):
             }
 
 
+class LoadFileException(SanicException):
+    pass
+
+
 def abort(status_code, message=None):
     """
     Raise an exception based on SanicException. Returns the HTTP response

sanic/response.py

@@ -14,10 +14,10 @@ from sanic.helpers import has_message_body, remove_entity_headers
 try:
     from ujson import dumps as json_dumps
 except ImportError:
-    from json import dumps
-
     # This is done in order to ensure that the JSON response is
     # kept consistent across both ujson and inbuilt json usage.
+    from json import dumps
+
     json_dumps = partial(dumps, separators=(",", ":"))
 
 
@@ -249,7 +249,10 @@ def raw(
     :param content_type: the content type (string) of the response.
     """
     return HTTPResponse(
-        body=body, status=status, headers=headers, content_type=content_type,
+        body=body,
+        status=status,
+        headers=headers,
+        content_type=content_type,
     )
 
 

sanic/server.py

@@ -14,11 +14,13 @@ from ipaddress import ip_address
 from signal import SIG_IGN, SIGINT, SIGTERM, Signals
 from signal import signal as signal_func
 from time import time
+from typing import Dict, Type, Union
 
 from httptools import HttpRequestParser  # type: ignore
 from httptools.parser.errors import HttpParserError  # type: ignore
 
 from sanic.compat import Header, ctrlc_workaround_for_windows
+from sanic.config import Config
 from sanic.exceptions import (
     HeaderExpectationFailed,
     InvalidUsage,
@@ -416,6 +418,7 @@ class HttpProtocol(asyncio.Protocol):
     async def stream_append(self):
         while self._body_chunks:
             body = self._body_chunks.popleft()
+            if self.request:
                 if self.request.stream.is_full():
                     self.transport.pause_reading()
                     await self.request.stream.put(body)
@@ -844,6 +847,7 @@ def serve(
     app.asgi = False
 
     connections = connections if connections is not None else set()
+    protocol_kwargs = _build_protocol_kwargs(protocol, app.config)
     server = partial(
         protocol,
         loop=loop,
@@ -852,6 +856,7 @@ def serve(
         app=app,
         state=state,
         unix=unix,
+        **protocol_kwargs,
     )
     asyncio_server_kwargs = (
         asyncio_server_kwargs if asyncio_server_kwargs else {}
@@ -948,6 +953,21 @@ def serve(
         remove_unix_socket(unix)
 
 
+def _build_protocol_kwargs(
+    protocol: Type[HttpProtocol], config: Config
+) -> Dict[str, Union[int, float]]:
+    if hasattr(protocol, "websocket_handshake"):
+        return {
+            "websocket_max_size": config.WEBSOCKET_MAX_SIZE,
+            "websocket_max_queue": config.WEBSOCKET_MAX_QUEUE,
+            "websocket_read_limit": config.WEBSOCKET_READ_LIMIT,
+            "websocket_write_limit": config.WEBSOCKET_WRITE_LIMIT,
+            "websocket_ping_timeout": config.WEBSOCKET_PING_TIMEOUT,
+            "websocket_ping_interval": config.WEBSOCKET_PING_INTERVAL,
+        }
+    return {}
+
+
 def bind_socket(host: str, port: int, *, backlog=100) -> socket.socket:
     """Create TCP server socket.
     :param host: IPv4, IPv6 or hostname may be specified

sanic/testing.py

@@ -11,6 +11,8 @@ from sanic.response import text
 
 
 ASGI_HOST = "mockserver"
+ASGI_PORT = 1234
+ASGI_BASE_URL = f"http://{ASGI_HOST}:{ASGI_PORT}"
 HOST = "127.0.0.1"
 PORT = None
 
@@ -22,6 +24,14 @@ class SanicTestClient:
         self.port = port
         self.host = host
 
+        @app.listener("after_server_start")
+        def _start_test_mode(sanic, *args, **kwargs):
+            sanic.test_mode = True
+
+        @app.listener("before_server_end")
+        def _end_test_mode(sanic, *args, **kwargs):
+            sanic.test_mode = False
+
     def get_new_session(self):
         return httpx.AsyncClient(verify=False)
 
@@ -95,7 +105,9 @@ class SanicTestClient:
 
         if self.port:
             server_kwargs = dict(
-                host=host or self.host, port=self.port, **server_kwargs,
+                host=host or self.host,
+                port=self.port,
+                **server_kwargs,
             )
             host, port = host or self.host, self.port
         else:
@@ -185,30 +197,33 @@ async def app_call_with_return(self, scope, receive, send):
     return await asgi_app()
 
 
-class SanicASGIDispatch(httpx.ASGIDispatch):
-    pass
-
-
 class SanicASGITestClient(httpx.AsyncClient):
     def __init__(
         self,
         app,
-        base_url: str = f"http://{ASGI_HOST}",
+        base_url: str = ASGI_BASE_URL,
         suppress_exceptions: bool = False,
     ) -> None:
         app.__class__.__call__ = app_call_with_return
         app.asgi = True
 
         self.app = app
-
-        dispatch = SanicASGIDispatch(app=app, client=(ASGI_HOST, PORT or 0))
-        super().__init__(dispatch=dispatch, base_url=base_url)
+        transport = httpx.ASGITransport(app=app, client=(ASGI_HOST, ASGI_PORT))
+        super().__init__(transport=transport, base_url=base_url)
 
         self.last_request = None
 
         def _collect_request(request):
             self.last_request = request
 
+        @app.listener("after_server_start")
+        def _start_test_mode(sanic, *args, **kwargs):
+            sanic.test_mode = True
+
+        @app.listener("before_server_end")
+        def _end_test_mode(sanic, *args, **kwargs):
+            sanic.test_mode = False
+
         app.request_middleware.appendleft(_collect_request)
 
     async def request(self, method, url, gather_request=True, *args, **kwargs):

sanic/utils.py

@@ -0,0 +1,99 @@
+from importlib.util import module_from_spec, spec_from_file_location
+from os import environ as os_environ
+from re import findall as re_findall
+from typing import Union
+
+from .exceptions import LoadFileException
+
+
+def str_to_bool(val: str) -> bool:
+    """Takes string and tries to turn it into bool as human would do.
+
+    If val is in case insensitive (
+        "y", "yes", "yep", "yup", "t",
+        "true", "on", "enable", "enabled", "1"
+    ) returns True.
+    If val is in case insensitive (
+        "n", "no", "f", "false", "off", "disable", "disabled", "0"
+    ) returns False.
+    Else Raise ValueError."""
+
+    val = val.lower()
+    if val in {
+        "y",
+        "yes",
+        "yep",
+        "yup",
+        "t",
+        "true",
+        "on",
+        "enable",
+        "enabled",
+        "1",
+    }:
+        return True
+    elif val in {"n", "no", "f", "false", "off", "disable", "disabled", "0"}:
+        return False
+    else:
+        raise ValueError(f"Invalid truth value {val}")
+
+
+def load_module_from_file_location(
+    location: Union[bytes, str], encoding: str = "utf8", *args, **kwargs
+):
+    """Returns loaded module provided as a file path.
+
+    :param args:
+        Coresponds to importlib.util.spec_from_file_location location
+        parameters,but with this differences:
+        - It has to be of a string or bytes type.
+        - You can also use here environment variables
+          in format ${some_env_var}.
+          Mark that $some_env_var will not be resolved as environment variable.
+    :encoding:
+        If location parameter is of a bytes type, then use this encoding
+        to decode it into string.
+    :param args:
+        Coresponds to the rest of importlib.util.spec_from_file_location
+        parameters.
+    :param kwargs:
+        Coresponds to the rest of importlib.util.spec_from_file_location
+        parameters.
+
+    For example You can:
+
+        some_module = load_module_from_file_location(
+            "some_module_name",
+            "/some/path/${some_env_var}"
+        )
+    """
+
+    # 1) Parse location.
+    if isinstance(location, bytes):
+        location = location.decode(encoding)
+
+    # A) Check if location contains any environment variables
+    #    in format ${some_env_var}.
+    env_vars_in_location = set(re_findall(r"\${(.+?)}", location))
+
+    # B) Check these variables exists in environment.
+    not_defined_env_vars = env_vars_in_location.difference(os_environ.keys())
+    if not_defined_env_vars:
+        raise LoadFileException(
+            "The following environment variables are not set: "
+            f"{', '.join(not_defined_env_vars)}"
+        )
+
+    # C) Substitute them in location.
+    for env_var in env_vars_in_location:
+        location = location.replace("${" + env_var + "}", os_environ[env_var])
+
+    # 2) Load and return module.
+    name = location.split("/")[-1].split(".")[
+        0
+    ]  # get just the file name without path and .py extension
+    _mod_spec = spec_from_file_location(name, location, *args, **kwargs)
+    module = module_from_spec(_mod_spec)
+    _mod_spec.loader.exec_module(module)  # type: ignore
+
+    return module

sanic/websocket.py

@@ -3,6 +3,7 @@ from typing import (
     Awaitable,
     Callable,
     Dict,
+    List,
     MutableMapping,
     Optional,
     Union,
@@ -34,6 +35,8 @@ class WebSocketProtocol(HttpProtocol):
         websocket_max_queue=None,
         websocket_read_limit=2 ** 16,
         websocket_write_limit=2 ** 16,
+        websocket_ping_interval=20,
+        websocket_ping_timeout=20,
         **kwargs
     ):
         super().__init__(*args, **kwargs)
@@ -44,6 +47,8 @@ class WebSocketProtocol(HttpProtocol):
         self.websocket_max_queue = websocket_max_queue
         self.websocket_read_limit = websocket_read_limit
         self.websocket_write_limit = websocket_write_limit
+        self.websocket_ping_interval = websocket_ping_interval
+        self.websocket_ping_timeout = websocket_ping_timeout
 
     # timeouts make no sense for websocket routes
     def request_timeout_callback(self):
@@ -118,6 +123,8 @@ class WebSocketProtocol(HttpProtocol):
             max_queue=self.websocket_max_queue,
             read_limit=self.websocket_read_limit,
             write_limit=self.websocket_write_limit,
+            ping_interval=self.websocket_ping_interval,
+            ping_timeout=self.websocket_ping_timeout,
         )
         # Following two lines are required for websockets 8.x
         self.websocket.is_client = False
@@ -137,9 +144,11 @@ class WebSocketConnection:
         self,
         send: Callable[[ASIMessage], Awaitable[None]],
         receive: Callable[[], Awaitable[ASIMessage]],
+        subprotocols: Optional[List[str]] = None,
     ) -> None:
         self._send = send
         self._receive = receive
+        self.subprotocols = subprotocols or []
 
     async def send(self, data: Union[str, bytes], *args, **kwargs) -> None:
         message: Dict[str, Union[str, bytes]] = {"type": "websocket.send"}
@@ -164,7 +173,14 @@ class WebSocketConnection:
     receive = recv
 
     async def accept(self) -> None:
-        await self._send({"type": "websocket.accept", "subprotocol": ""})
+        await self._send(
+            {
+                "type": "websocket.accept",
+                "subprotocol": ",".join(
+                    [subprotocol for subprotocol in self.subprotocols]
+                ),
+            }
+        )
 
     async def close(self) -> None:
         pass

sanic/worker.py

@@ -5,7 +5,7 @@ import signal
 import sys
 import traceback
 
-import gunicorn.workers.base as base  # type: ignore
+from gunicorn.workers import base as base  # type: ignore
 
 from sanic.server import HttpProtocol, Signal, serve, trigger_events
 from sanic.websocket import WebSocketProtocol

setup.cfg

@@ -11,11 +11,3 @@ line_length = 79
 lines_after_imports = 2
 lines_between_types = 1
 multi_line_output = 3
-not_skip = __init__.py
-
-[version]
-current_version = 19.12.0
-files = sanic/__version__.py
-current_version_pattern = __version__ = "{current_version}"
-new_version_pattern = __version__ = "{new_version}"
-

setup.py

@@ -81,7 +81,7 @@ requirements = [
     "aiofiles>=0.3.0",
     "websockets>=8.1,<9.0",
     "multidict>=4.0,<5.0",
-    "httpx==0.11.1",
+    "httpx==0.15.4",
 ]
 
 tests_require = [

tests/performance/wheezy/simple_server.py

@@ -1,7 +1,6 @@
 # Run with: gunicorn --workers=1 --worker-class=meinheld.gmeinheld.MeinheldWorker simple_server:main
 """ Minimal helloworld application.
 """
-
 import ujson
 
 from wheezy.http import HTTPResponse, WSGIApplication
@@ -39,6 +38,7 @@ main = WSGIApplication(
 
 if __name__ == "__main__":
     import sys
+
     from wsgiref.simple_server import make_server
 
     try:

tests/static/app_test_config.py

@@ -0,0 +1 @@
+TEST_SETTING_VALUE = 1

tests/test_app.py

@@ -3,6 +3,7 @@ import logging
 import sys
 
 from inspect import isawaitable
+from unittest.mock import patch
 
 import pytest
 
@@ -125,7 +126,7 @@ def test_app_handle_request_handler_is_none(app, monkeypatch):
     def handler(request):
         return text("test")
 
-    request, response = app.test_client.get("/test")
+    _, response = app.test_client.get("/test")
 
     assert (
         "'None' was returned while requesting a handler from the router"
@@ -148,6 +149,43 @@ def test_app_enable_websocket(app, websocket_enabled, enable):
     assert app.websocket_enabled == True
 
 
+@patch("sanic.app.WebSocketProtocol")
+def test_app_websocket_parameters(websocket_protocol_mock, app):
+    app.config.WEBSOCKET_MAX_SIZE = 44
+    app.config.WEBSOCKET_MAX_QUEUE = 45
+    app.config.WEBSOCKET_READ_LIMIT = 46
+    app.config.WEBSOCKET_WRITE_LIMIT = 47
+    app.config.WEBSOCKET_PING_TIMEOUT = 48
+    app.config.WEBSOCKET_PING_INTERVAL = 50
+
+    @app.websocket("/ws")
+    async def handler(request, ws):
+        await ws.send("test")
+
+    try:
+        # This will fail because WebSocketProtocol is mocked and only the call kwargs matter
+        app.test_client.get("/ws")
+    except:
+        pass
+
+    websocket_protocol_call_args = websocket_protocol_mock.call_args
+    ws_kwargs = websocket_protocol_call_args[1]
+    assert ws_kwargs["websocket_max_size"] == app.config.WEBSOCKET_MAX_SIZE
+    assert ws_kwargs["websocket_max_queue"] == app.config.WEBSOCKET_MAX_QUEUE
+    assert ws_kwargs["websocket_read_limit"] == app.config.WEBSOCKET_READ_LIMIT
+    assert (
+        ws_kwargs["websocket_write_limit"] == app.config.WEBSOCKET_WRITE_LIMIT
+    )
+    assert (
+        ws_kwargs["websocket_ping_timeout"]
+        == app.config.WEBSOCKET_PING_TIMEOUT
+    )
+    assert (
+        ws_kwargs["websocket_ping_interval"]
+        == app.config.WEBSOCKET_PING_INTERVAL
+    )
+
+
 def test_handle_request_with_nested_exception(app, monkeypatch):
 
     err_msg = "Mock Exception"
@@ -222,3 +260,28 @@ def test_handle_request_with_nested_sanic_exception(app, monkeypatch, caplog):
 def test_app_name_required():
     with pytest.deprecated_call():
         Sanic()
+
+
+def test_app_has_test_mode_sync():
+    app = Sanic("test")
+
+    @app.get("/")
+    def handler(request):
+        assert request.app.test_mode
+        return text("test")
+
+    _, response = app.test_client.get("/")
+    assert response.status == 200
+
+
+# @pytest.mark.asyncio
+# async def test_app_has_test_mode_async():
+#     app = Sanic("test")
+
+#     @app.get("/")
+#     async def handler(request):
+#         assert request.app.test_mode
+#         return text("test")
+
+#     _, response = await app.asgi_client.get("/")
+#     assert response.status == 200

tests/test_asgi.py

@@ -208,6 +208,53 @@ async def test_websocket_receive(send, receive, message_stack):
     assert text == msg["text"]
 
 
+@pytest.mark.asyncio
+async def test_websocket_accept_with_no_subprotocols(
+    send, receive, message_stack
+):
+    ws = WebSocketConnection(send, receive)
+    await ws.accept()
+
+    assert len(message_stack) == 1
+
+    message = message_stack.popleft()
+    assert message["type"] == "websocket.accept"
+    assert message["subprotocol"] == ""
+    assert "bytes" not in message
+
+
+@pytest.mark.asyncio
+async def test_websocket_accept_with_subprotocol(send, receive, message_stack):
+    subprotocols = ["graphql-ws"]
+
+    ws = WebSocketConnection(send, receive, subprotocols)
+    await ws.accept()
+
+    assert len(message_stack) == 1
+
+    message = message_stack.popleft()
+    assert message["type"] == "websocket.accept"
+    assert message["subprotocol"] == "graphql-ws"
+    assert "bytes" not in message
+
+
+@pytest.mark.asyncio
+async def test_websocket_accept_with_multiple_subprotocols(
+    send, receive, message_stack
+):
+    subprotocols = ["graphql-ws", "hello", "world"]
+
+    ws = WebSocketConnection(send, receive, subprotocols)
+    await ws.accept()
+
+    assert len(message_stack) == 1
+
+    message = message_stack.popleft()
+    assert message["type"] == "websocket.accept"
+    assert message["subprotocol"] == "graphql-ws,hello,world"
+    assert "bytes" not in message
+
+
 def test_improper_websocket_connection(transport, send, receive):
     with pytest.raises(InvalidUsage):
         transport.get_websocket_connection()

tests/test_errorpages.py

@@ -0,0 +1,86 @@
+import pytest
+
+from sanic import Sanic
+from sanic.errorpages import exception_response
+from sanic.exceptions import NotFound
+from sanic.request import Request
+from sanic.response import HTTPResponse
+
+
+@pytest.fixture
+def app():
+    app = Sanic("error_page_testing")
+
+    @app.route("/error", methods=["GET", "POST"])
+    def err(request):
+        raise Exception("something went wrong")
+
+    return app
+
+
+@pytest.fixture
+def fake_request(app):
+    return Request(b"/foobar", {}, "1.1", "GET", None, app)
+
+
+@pytest.mark.parametrize(
+    "fallback,content_type, exception, status",
+    (
+        (None, "text/html; charset=utf-8", Exception, 500),
+        ("html", "text/html; charset=utf-8", Exception, 500),
+        ("auto", "text/html; charset=utf-8", Exception, 500),
+        ("text", "text/plain; charset=utf-8", Exception, 500),
+        ("json", "application/json", Exception, 500),
+        (None, "text/html; charset=utf-8", NotFound, 404),
+        ("html", "text/html; charset=utf-8", NotFound, 404),
+        ("auto", "text/html; charset=utf-8", NotFound, 404),
+        ("text", "text/plain; charset=utf-8", NotFound, 404),
+        ("json", "application/json", NotFound, 404),
+    ),
+)
+def test_should_return_html_valid_setting(
+    fake_request, fallback, content_type, exception, status
+):
+    if fallback:
+        fake_request.app.config.FALLBACK_ERROR_FORMAT = fallback
+
+    try:
+        raise exception("bad stuff")
+    except Exception as e:
+        response = exception_response(fake_request, e, True)
+
+    assert isinstance(response, HTTPResponse)
+    assert response.status == status
+    assert response.content_type == content_type
+
+
+def test_auto_fallback_with_data(app):
+    app.config.FALLBACK_ERROR_FORMAT = "auto"
+
+    _, response = app.test_client.get("/error")
+    assert response.status == 500
+    assert response.content_type == "text/html; charset=utf-8"
+
+    _, response = app.test_client.post("/error", json={"foo": "bar"})
+    assert response.status == 500
+    assert response.content_type == "application/json"
+
+    _, response = app.test_client.post("/error", data={"foo": "bar"})
+    assert response.status == 500
+    assert response.content_type == "text/html; charset=utf-8"
+
+
+def test_auto_fallback_with_content_type(app):
+    app.config.FALLBACK_ERROR_FORMAT = "auto"
+
+    _, response = app.test_client.get(
+        "/error", headers={"content-type": "application/json"}
+    )
+    assert response.status == 500
+    assert response.content_type == "application/json"
+
+    _, response = app.test_client.get(
+        "/error", headers={"content-type": "text/plain"}
+    )
+    assert response.status == 500
+    assert response.content_type == "text/plain; charset=utf-8"

tests/test_keep_alive_timeout.py

@@ -3,6 +3,7 @@ import asyncio
 from asyncio import sleep as aio_sleep
 from json import JSONDecodeError
 
+import httpcore
 import httpx
 
 from sanic import Sanic, server
@@ -12,67 +13,26 @@ from sanic.testing import HOST, SanicTestClient
 
 CONFIG_FOR_TESTS = {"KEEP_ALIVE_TIMEOUT": 2, "KEEP_ALIVE": True}
 
-old_conn = None
 PORT = 42101  # test_keep_alive_timeout_reuse doesn't work with random port
 
+from httpcore._async.base import ConnectionState
+from httpcore._async.connection import AsyncHTTPConnection
+from httpcore._types import Origin
 
-class ReusableSanicConnectionPool(
-    httpx.dispatch.connection_pool.ConnectionPool
-):
-    @property
-    def cert(self):
-        return self.ssl.cert
 
-    @property
-    def verify(self):
-        return self.ssl.verify
+class ReusableSanicConnectionPool(httpcore.AsyncConnectionPool):
+    last_reused_connection = None
 
-    @property
-    def trust_env(self):
-        return self.ssl.trust_env
-
-    @property
-    def http2(self):
-        return self.ssl.http2
-
-    async def acquire_connection(self, origin, timeout):
-        global old_conn
-        connection = self.pop_connection(origin)
-
-        if connection is None:
-            pool_timeout = None if timeout is None else timeout.pool_timeout
-
-            await self.max_connections.acquire(timeout=pool_timeout)
-            ssl_config = httpx.config.SSLConfig(
-                cert=self.cert,
-                verify=self.verify,
-                trust_env=self.trust_env,
-                http2=self.http2,
-            )
-            connection = httpx.dispatch.connection.HTTPConnection(
-                origin,
-                ssl=ssl_config,
-                backend=self.backend,
-                release_func=self.release_connection,
-                uds=self.uds,
-            )
-
-        self.active_connections.add(connection)
-
-        if old_conn is not None:
-            if old_conn != connection:
-                raise RuntimeError(
-                    "We got a new connection, wanted the same one!"
-                )
-        old_conn = connection
-
-        return connection
+    async def _get_connection_from_pool(self, *args, **kwargs):
+        conn = await super()._get_connection_from_pool(*args, **kwargs)
+        self.__class__.last_reused_connection = conn
+        return conn
 
 
 class ResusableSanicSession(httpx.AsyncClient):
     def __init__(self, *args, **kwargs) -> None:
-        dispatch = ReusableSanicConnectionPool()
-        super().__init__(dispatch=dispatch, *args, **kwargs)
+        transport = ReusableSanicConnectionPool()
+        super().__init__(transport=transport, *args, **kwargs)
 
 
 class ReuseableSanicTestClient(SanicTestClient):
@@ -258,6 +218,7 @@ def test_keep_alive_timeout_reuse():
         request, response = client.get("/1")
         assert response.status == 200
         assert response.text == "OK"
+        assert ReusableSanicConnectionPool.last_reused_connection
     finally:
         client.kill_server()
 
@@ -270,7 +231,6 @@ def test_keep_alive_client_timeout():
         asyncio.set_event_loop(loop)
         client = ReuseableSanicTestClient(keep_alive_app_client_timeout, loop)
         headers = {"Connection": "keep-alive"}
-        try:
         request, response = client.get(
             "/1", headers=headers, request_keepalive=1
         )
@@ -279,11 +239,7 @@ def test_keep_alive_client_timeout():
         loop.run_until_complete(aio_sleep(2))
         exception = None
         request, response = client.get("/1", request_keepalive=1)
-        except ValueError as e:
-            exception = e
-        assert exception is not None
-        assert isinstance(exception, ValueError)
-        assert "got a new connection" in exception.args[0]
+        assert ReusableSanicConnectionPool.last_reused_connection is None
     finally:
         client.kill_server()
 
@@ -298,7 +254,6 @@ def test_keep_alive_server_timeout():
         asyncio.set_event_loop(loop)
         client = ReuseableSanicTestClient(keep_alive_app_server_timeout, loop)
         headers = {"Connection": "keep-alive"}
-        try:
         request, response = client.get(
             "/1", headers=headers, request_keepalive=60
         )
@@ -307,13 +262,6 @@ def test_keep_alive_server_timeout():
         loop.run_until_complete(aio_sleep(3))
         exception = None
         request, response = client.get("/1", request_keepalive=60)
-        except ValueError as e:
-            exception = e
-        assert exception is not None
-        assert isinstance(exception, ValueError)
-        assert (
-            "Connection reset" in exception.args[0]
-            or "got a new connection" in exception.args[0]
-        )
+        assert ReusableSanicConnectionPool.last_reused_connection is None
     finally:
         client.kill_server()

tests/test_load_module_from_file_location.py

@@ -0,0 +1,35 @@
+from pathlib import Path
+from types import ModuleType
+
+import pytest
+
+from sanic.exceptions import LoadFileException
+from sanic.utils import load_module_from_file_location
+
+
+@pytest.fixture
+def loaded_module_from_file_location():
+    return load_module_from_file_location(
+        str(Path(__file__).parent / "static/app_test_config.py")
+    )
+
+
+@pytest.mark.dependency(name="test_load_module_from_file_location")
+def test_load_module_from_file_location(loaded_module_from_file_location):
+    assert isinstance(loaded_module_from_file_location, ModuleType)
+
+
+@pytest.mark.dependency(depends=["test_load_module_from_file_location"])
+def test_loaded_module_from_file_location_name(
+    loaded_module_from_file_location,
+):
+    assert loaded_module_from_file_location.__name__ == "app_test_config"
+
+
+def test_load_module_from_file_location_with_non_existing_env_variable():
+    with pytest.raises(
+        LoadFileException,
+        match="The following environment variables are not set: MuuMilk",
+    ):
+
+        load_module_from_file_location("${MuuMilk}")

tests/test_request_data.py

@@ -33,6 +33,23 @@ def test_custom_context(app):
             }
         )
 
+    @app.middleware("response")
+    def modify(request, response):
+        # Using response-middleware to access request ctx
+        try:
+            user = request.ctx.user
+        except AttributeError as e:
+            user = str(e)
+        try:
+            invalid = request.ctx.missing
+        except AttributeError as e:
+            invalid = str(e)
+
+        j = loads(response.body)
+        j["response_mw_valid"] = user
+        j["response_mw_invalid"] = invalid
+        return json(j)
+
     request, response = app.test_client.get("/")
     assert response.json == {
         "user": "sanic",
@@ -41,6 +58,8 @@ def test_custom_context(app):
         "has_session": True,
         "has_missing": False,
         "invalid": "'types.SimpleNamespace' object has no attribute 'missing'",
+        "response_mw_valid": "sanic",
+        "response_mw_invalid": "'types.SimpleNamespace' object has no attribute 'missing'",
     }
 
 

tests/test_request_stream.py

@@ -1,9 +1,12 @@
+import asyncio
+
 import pytest
 
 from sanic.blueprints import Blueprint
 from sanic.exceptions import HeaderExpectationFailed
 from sanic.request import StreamBuffer
 from sanic.response import json, stream, text
+from sanic.server import HttpProtocol
 from sanic.views import CompositionView, HTTPMethodView
 from sanic.views import stream as stream_decorator
 
@@ -337,6 +340,22 @@ def test_request_stream_handle_exception(app):
     assert "Method GET not allowed for URL /post/random_id" in response.text
 
 
+@pytest.mark.asyncio
+async def test_request_stream_unread(app):
+    """ensure no error is raised when leaving unread bytes in byte-buffer"""
+
+    err = None
+    protocol = HttpProtocol(loop=asyncio.get_event_loop(), app=app)
+    try:
+        protocol.request = None
+        protocol._body_chunks.append("this is a test")
+        await protocol.stream_append()
+    except AttributeError as e:
+        err = e
+
+    assert err is None and not protocol._body_chunks
+
+
 def test_request_stream_blueprint(app):
     """for self.is_request_stream = True"""
     bp = Blueprint("test_blueprint_request_stream_blueprint")

tests/test_request_timeout.py

@@ -1,64 +1,54 @@
 import asyncio
 
+from typing import cast
+
+import httpcore
 import httpx
 
+from httpcore._async.base import (
+    AsyncByteStream,
+    AsyncHTTPTransport,
+    ConnectionState,
+    NewConnectionRequired,
+)
+from httpcore._async.connection import AsyncHTTPConnection
+from httpcore._async.connection_pool import ResponseByteStream
+from httpcore._exceptions import LocalProtocolError, UnsupportedProtocol
+from httpcore._types import TimeoutDict
+from httpcore._utils import url_to_origin
+
 from sanic import Sanic
 from sanic.response import text
 from sanic.testing import SanicTestClient
 
 
-class DelayableHTTPConnection(httpx.dispatch.connection.HTTPConnection):
-    def __init__(self, *args, **kwargs):
-        self._request_delay = None
-        if "request_delay" in kwargs:
-            self._request_delay = kwargs.pop("request_delay")
-        super().__init__(*args, **kwargs)
-
-    async def send(self, request, timeout=None):
-
-        if self.connection is None:
-            self.connection = await self.connect(timeout=timeout)
+class DelayableHTTPConnection(httpcore._async.connection.AsyncHTTPConnection):
+    async def arequest(self, *args, **kwargs):
+        await asyncio.sleep(2)
+        return await super().arequest(*args, **kwargs)
 
+    async def _open_socket(self, *args, **kwargs):
+        retval = await super()._open_socket(*args, **kwargs)
         if self._request_delay:
             await asyncio.sleep(self._request_delay)
-
-        response = await self.connection.send(request, timeout=timeout)
-
-        return response
+        return retval
 
 
-class DelayableSanicConnectionPool(
-    httpx.dispatch.connection_pool.ConnectionPool
-):
+class DelayableSanicConnectionPool(httpcore.AsyncConnectionPool):
     def __init__(self, request_delay=None, *args, **kwargs):
         self._request_delay = request_delay
         super().__init__(*args, **kwargs)
 
-    async def acquire_connection(self, origin, timeout=None):
-        connection = self.pop_connection(origin)
-
-        if connection is None:
-            pool_timeout = None if timeout is None else timeout.pool_timeout
-
-            await self.max_connections.acquire(timeout=pool_timeout)
-            connection = DelayableHTTPConnection(
-                origin,
-                ssl=self.ssl,
-                backend=self.backend,
-                release_func=self.release_connection,
-                uds=self.uds,
-                request_delay=self._request_delay,
-            )
-
-        self.active_connections.add(connection)
-
-        return connection
+    async def _add_to_pool(self, connection, timeout):
+        connection.__class__ = DelayableHTTPConnection
+        connection._request_delay = self._request_delay
+        await super()._add_to_pool(connection, timeout)
 
 
 class DelayableSanicSession(httpx.AsyncClient):
     def __init__(self, request_delay=None, *args, **kwargs) -> None:
-        dispatch = DelayableSanicConnectionPool(request_delay=request_delay)
-        super().__init__(dispatch=dispatch, *args, **kwargs)
+        transport = DelayableSanicConnectionPool(request_delay=request_delay)
+        super().__init__(transport=transport, *args, **kwargs)
 
 
 class DelayableSanicTestClient(SanicTestClient):

tests/test_requests.py

@@ -12,7 +12,14 @@ from sanic import Blueprint, Sanic
 from sanic.exceptions import ServerError
 from sanic.request import DEFAULT_HTTP_CONTENT_TYPE, Request, RequestParameters
 from sanic.response import html, json, text
-from sanic.testing import ASGI_HOST, HOST, PORT, SanicTestClient
+from sanic.testing import (
+    ASGI_BASE_URL,
+    ASGI_HOST,
+    ASGI_PORT,
+    HOST,
+    PORT,
+    SanicTestClient,
+)
 
 
 # ------------------------------------------------------------ #
@@ -59,7 +66,10 @@ async def test_ip_asgi(app):
 
     request, response = await app.asgi_client.get("/")
 
-    assert response.text == "http://mockserver/"
+    if response.text.endswith("/") and not ASGI_BASE_URL.endswith("/"):
+        response.text[:-1] == ASGI_BASE_URL
+    else:
+        assert response.text == ASGI_BASE_URL
 
 
 def test_text(app):
@@ -573,7 +583,7 @@ async def test_standard_forwarded_asgi(app):
     assert response.json() == {"for": "127.0.0.2", "proto": "ws"}
     assert request.remote_addr == "127.0.0.2"
     assert request.scheme == "ws"
-    assert request.server_port == 80
+    assert request.server_port == ASGI_PORT
 
     app.config.FORWARDED_SECRET = "mySecret"
     request, response = await app.asgi_client.get("/", headers=headers)
@@ -1044,9 +1054,9 @@ def test_url_attributes_no_ssl(app, path, query, expected_url):
 @pytest.mark.parametrize(
     "path,query,expected_url",
     [
-        ("/foo", "", "http://{}/foo"),
-        ("/bar/baz", "", "http://{}/bar/baz"),
-        ("/moo/boo", "arg1=val1", "http://{}/moo/boo?arg1=val1"),
+        ("/foo", "", "{}/foo"),
+        ("/bar/baz", "", "{}/bar/baz"),
+        ("/moo/boo", "arg1=val1", "{}/moo/boo?arg1=val1"),
     ],
 )
 @pytest.mark.asyncio
@@ -1057,7 +1067,7 @@ async def test_url_attributes_no_ssl_asgi(app, path, query, expected_url):
     app.add_route(handler, path)
 
     request, response = await app.asgi_client.get(path + f"?{query}")
-    assert request.url == expected_url.format(ASGI_HOST)
+    assert request.url == expected_url.format(ASGI_BASE_URL)
 
     parsed = urlparse(request.url)
 
@@ -1955,10 +1965,7 @@ def test_server_name_and_url_for(app):
     app.config.SERVER_NAME = "my-server"  # This means default port
     assert app.url_for("handler", _external=True) == "http://my-server/foo"
     request, response = app.test_client.get("/foo")
-    assert (
-        request.url_for("handler")
-        == f"http://my-server/foo"
-    )
+    assert request.url_for("handler") == f"http://my-server/foo"
 
     app.config.SERVER_NAME = "https://my-server/path"
     request, response = app.test_client.get("/foo")

tests/test_unix_socket.py

@@ -4,6 +4,7 @@ import os
 import subprocess
 import sys
 
+import httpcore
 import httpx
 import pytest
 
@@ -139,8 +140,9 @@ def test_unix_connection():
 
     @app.listener("after_server_start")
     async def client(app, loop):
+        transport = httpcore.AsyncConnectionPool(uds=SOCKPATH)
         try:
-            async with httpx.AsyncClient(uds=SOCKPATH) as client:
+            async with httpx.AsyncClient(transport=transport) as client:
                 r = await client.get("http://myhost.invalid/")
                 assert r.status_code == 200
                 assert r.text == os.path.abspath(SOCKPATH)
@@ -179,8 +181,9 @@ async def test_zero_downtime():
     from time import monotonic as current_time
 
     async def client():
+        transport = httpcore.AsyncConnectionPool(uds=SOCKPATH)
         for _ in range(40):
-            async with httpx.AsyncClient(uds=SOCKPATH) as client:
+            async with httpx.AsyncClient(transport=transport) as client:
                 r = await client.get("http://localhost/sleep/0.1")
                 assert r.status_code == 200
                 assert r.text == f"Slept 0.1 seconds.\n"

tests/test_update_config.py

@@ -0,0 +1,36 @@
+from pathlib import Path
+
+import pytest
+
+
+_test_setting_as_dict = {"TEST_SETTING_VALUE": 1}
+_test_setting_as_class = type("C", (), {"TEST_SETTING_VALUE": 1})
+_test_setting_as_module = str(
+    Path(__file__).parent / "static/app_test_config.py"
+)
+
+
+@pytest.mark.parametrize(
+    "conf_object",
+    [
+        _test_setting_as_dict,
+        _test_setting_as_class,
+        pytest.param(
+            _test_setting_as_module,
+            marks=pytest.mark.dependency(
+                depends=["test_load_module_from_file_location"],
+                scope="session",
+            ),
+        ),
+    ],
+    ids=["from_dict", "from_class", "from_file"],
+)
+def test_update(app, conf_object):
+    app.update_config(conf_object)
+    assert app.config["TEST_SETTING_VALUE"] == 1
+
+
+def test_update_from_lowercase_key(app):
+    d = {"test_setting_value": 1}
+    app.update_config(d)
+    assert "test_setting_value" not in app.config

tox.ini

@@ -12,12 +12,13 @@ deps =
     pytest-cov
     pytest-sanic
     pytest-sugar
+    pytest-benchmark
+    pytest-dependency
     httpcore==0.3.0
-    httpx==0.11.1
+    httpx==0.15.4
     chardet<=2.3.0
     beautifulsoup4
     gunicorn
-    pytest-benchmark
     uvicorn
     websockets>=8.1,<9.0
 commands =
@@ -30,13 +31,13 @@ commands =
 deps =
     flake8
     black
-    isort
+    isort>=5.0.0
     bandit
 
 commands =
     flake8 sanic
     black --config ./.black.toml --check --verbose sanic/
-    isort --check-only --recursive sanic
+    isort --check-only sanic
 
 [testenv:type-checking]
 deps =
@@ -55,6 +56,9 @@ commands =
 [pytest]
 filterwarnings =
     ignore:.*async with lock.* instead:DeprecationWarning
+addopts = --strict-markers
+markers =
+    asyncio
 
 [testenv:security]
 deps =