Testing decorated functions in Python web frameworks

The meaty parts of Web frameworks, the parts that determine what happens when you hit a certain URL, often have a similar structure. Here's an example from Bottle, a Python framework:

hats.py

@app.route('/hats')
def get_hat_names():
    hat_names = db.connection.execute("SELECT name FROM hats").fetchall()
    return hat_template(hat_names = hat_names)

Very similar patterns exist in Django and Flask. (Also in non-Python frameworks, but for concreteness I'll use only Python examples.)

Now: how do you test get_hat_names?

The answer starts with the observation that things are not, in practice, as simple as the example above. Let's make the example slightly more realistic:

hats.py

# You have some imports

# Somewhere you have a Django Router, a Flask object, or a Bottle command.

@app.route('/hats')
def show_hats(style: str = Form(...)): # In Django and other frameworks, there would be a `request` parameter here
    style_enum_value = Style[style].value # Style is an enum you're importing
    hat_names = db.connection.execute("SELECT name FROM hats WHERE style = {style_enum_value}").fetchall()
    sale_hat_names = db.connection.execute("SELECT hat_name FROM sales WHERE status = 'active'").fetchall()
    if not hat_names:
        return no_hats_template()
    sale_items = [hat_name for hat_name in hat_names if hat_name in sale_hat_names]
    return hat_template(hat_names = hat_names, sale_items = sale_items)

There are at least four problems that arise in testing a function like this:

1. The function is more complicated than you'd want it to be for testing.
2. The function uses (closes over, if you prefer) items from the surrounding context These items often include routers, environment variables, and database connections.
3. You tend to need mocks: e.g., of database connections and of Django Request objects.
4. The function is decorated, and there's a lot of behavior happening in the wrapped function (but not the first-order show_hats function) that will interfere with your attempts to interact with it in tests.

(Those problems are interrelated, but also multifaceted. You might say those are actually two problems or actually six. Never mind.)

Here's how I test these functions without creating an opaque wad of mocks and set-up functions.

Step one: put persistence-related information behind an interface

Create an interface that abstracts away the details of a particular database configuration:

hat_repository.py

class HatRepository:
    @abstractmethod
    def get_hat_names(self, style_enum_value):
        raise NotImplementedError

    @abstractmethod
    def get_sale_hat_names(self, style_enum_value):
        raise NotImplementedError

class SQLHatRepository(HatRepository):
    def __init__(self, db_path):
        # Do your database setup
        pass

    def get_hat_names(self, style_enum_value):
        return db.connection.execute("SELECT name FROM hats WHERE style = {style_enum_value}").fetchall()

    def get_sale_hat_names(self, style_enum_value):
        return db.connection.execute("SELECT hat_name FROM sales WHERE status = 'active' AND style = {style_enum_value}").fetchall()

This will help us a lot here. It will help us a lot everywhere, in fact: it's a powerful abstraction. (Architecture Patterns with Python is the best treatment I've read of the so-called "repository pattern.")

Then rewrite your decorated framework functions to depend not on a raw database connection but on an instance of the repository:

hats.py

# You have some imports

# Somewhere you have a Django Router, a Flask object, or a Bottle command

repo = SQLHatRepository("db_path.db")

@app.route('/hats')
def show_hats(style: str = Form(...)): # In Django and other frameworks, there would be a `request` parameter here
    style_enum_value = Style[style].value
    hat_names = repo.get_hat_names(style_enum_value)
    sale_hat_names = repo.get_sale_hat_names(style_enum_value)
    if not hat_names:
        return no_hats_template()
    sale_items = [hat_name for hat_name in hat_names if hat_name in sale_hat_names]
    return hat_template(hat_names = hat_names, sale_items = sale_items)

Step two: put all substantial logic in a helper function

In our example, this would work like this:

hat_helpers.py

def _show_hats(repo, style_enum_value):
    hat_names = repo.get_hat_names(style_enum_value)
    sale_hat_names = repo.get_sale_hat_names(style_enum_value)
    sale_items = [hat_name for hat_name in hat_names if hat_name in sale_hat_names]
    return (hat_names, sale_items)

hats.py

# You have some imports

# Somewhere you have a Django Router, a Flask object, or a Bottle command

from hat_helpers import _show_hats

repo = SQLHatRepository("db_path.db")

@app.route('/hats')
def show_hats(style: str = Form(...)): # In Django and other frameworks, there would be a `request` parameter here
    style_enum_value = Style[style].value
    hat_names, sale_items = _show_hats(repo, style_enum_value)
    if not hat_names:
        return no_hats_template()
    return hat_template(hat_names = hat_names, sale_items = sale_items)

The division of work above is common:

1. The main decorated function parses inputs, which are often require transformation from what is passed in to the function. Most commonly, this is a matter of parsing strings or extracting fields from a request.
2. The main decorated function feeds values into templates.
3. The helper function does everything else.

Step three: test the helper function

Now you have a helper function, _show_hats, that is perfectly testable:

1. You can create a test-friendly subclass of HatRepository.
2. No mocks are required.
3. If the logic is complicated, you can (and should!) split it out and test those smaller functions separately.
4. You can import the helper without invoking side effects from your Web framework's router initialization, database connection, or anything else.

This looks like an anti-pattern, but it isn't

You might resist this solution: aren't these just "pass-through functions" of the sort John Osterhout warns us about? To have a function doing little more than passing arguments through is a well-known anti-pattern. What I'm recommending here, however, only look like pass-through functions. There's a lot of action happening in the decorator. To avoid pulling as much as possible out of these functions is to be deceived by syntax.