This is part two in a three-part series:

1. Making React and Django play well together
2. Making React and Django play well together - the “hybrid app” model
3. Making React and Django play well together - the “single page app” model

Making React and Django play well together - the “hybrid app” model

In my last post I discussed the trade-offs involved in choosing an architecture for integrating React with Django.

I described an alternative between two models:

• The “single page app” model: a standalone JavaScript frontend makes API requests to a backend running on another domain.
• The “hybrid app” model: the same backend serves HTML pages embedding JavaScript components and API requests.

I promised that I would describe how to implement each model.

Today I’m starting with the “hybrid app” model.

Let’s bootstrap a todolist app!¹

Disclaimer: I’m starting with default project templates and making minimal changes in order to focus on the integration between frontend and backend.

As a consequence, I’m ignoring many best practices for Django and React projects. Keep in mind that I’m describing only one piece of the puzzle and that many variants are possible!

Why build a “hybrid app”?

Here are the main reasons:

• Your app needs good SEO (e-commerce, classifieds, portfolios, etc.)
• You prefer treating your app as a single unit because that’s more convenient for building new features, running integration tests, and deploying new versions.
• You have a team of full-stack developers — perhaps a team of one — and you’d rather avoid the overhead of managing separate backend and frontend projects.
• You’re comfortable with building full-stack apps with Django and you want to try a modern JavaScript framework while leveraging your experience.

While the “hybrid app” model feels a bit old-school, it’s good for enhancing the frontend of backend-heavy apps — whether new or pre-existing — without disruption.

Initialization

Since the frontend and the backend are deployed together, it makes sense to maintain them in the same code repository.

Let’s initialize Django and React applications in backend and frontend directories at the root of the repository.²

Django

Start a shell, go to the root of the code repository and bootstrap the backend:

mkdir backend
cd backend
poetry init
poetry shell
poetry add django
django-admin startproject todolist .

For convenience, edit the Django settings as follows. This will make it easier to switch between development and production mode.

# backend/todolist/settings.py

# insert these lines after the definition of BASE_DIR
BACKEND_DIR = BASE_DIR  # rename variable for clarity
FRONTEND_DIR = BASE_DIR.parent / 'frontend'

# modify the definition of DEBUG and ALLOWED_HOSTS
import os
DEBUG = os.environ.get('DJANGO_ENV') == 'development'
ALLOWED_HOSTS = ['localhost']

Start the development server:

# in the backend directory, after executing poetry shell
DJANGO_ENV=development ./manage.py migrate
DJANGO_ENV=development ./manage.py runserver

Open http://localhost:8000/ in a browser to confirm that everything is working.

React

Start another shell, go to the root of the code repository and bootstrap the frontend:

npx create-react-app frontend
cd frontend

Start the development server:

# in the frontend directory
yarn start

http://localhost:3000/ opens automatically in a browser.

Starting point

Django and React development servers are now running on http://localhost:8000/ and http://localhost:3000/ but they don’t know anything about each other.

The source tree contains:

.
├── backend
│   ├── db.sqlite3
│   ├── manage.py
│   ├── poetry.lock
│   ├── pyproject.toml
│   └── todolist
│       ├── __init__.py
│       ├── asgi.py
│       ├── settings.py
│       ├── urls.py
│       └── wsgi.py
└── frontend
    ├── README.md
    ├── package.json
    ├── public
    │   ├── favicon.ico
    │   ├── index.html
    │   ├── logo192.png
    │   ├── logo512.png
    │   ├── manifest.json
    │   └── robots.txt
    ├── src
    │   ├── App.css
    │   ├── App.js
    │   ├── App.test.js
    │   ├── index.css
    │   ├── index.js
    │   ├── logo.svg
    │   ├── reportWebVitals.js
    │   └── setupTests.js
    └── yarn.lock

You can confirm this by running tree -I '__pycache__|node_modules'.

Running yarn build in the frontend directory adds:

.
└── frontend
    ├── build
    │   ├── asset-manifest.json
    │   ├── favicon.ico
    │   ├── index.html
    │   ├── logo192.png
    │   ├── logo512.png
    │   ├── manifest.json
    │   ├── robots.txt
    │   └── static
    │       ├── css
    │       │   ├── main.8c8b27cf.chunk.css
    │       │   └── main.8c8b27cf.chunk.css.map
    │       ├── js
    │       │   ├── 2.fa9ea962.chunk.js
    │       │   ├── 2.fa9ea962.chunk.js.LICENSE.txt
    │       │   ├── 2.fa9ea962.chunk.js.map
    │       │   ├── 3.2bb8242a.chunk.js
    │       │   ├── 3.2bb8242a.chunk.js.map
    │       │   ├── main.9fa1c34e.chunk.js
    │       │   ├── main.9fa1c34e.chunk.js.map
    │       │   ├── runtime-main.15e620d4.js
    │       │   └── runtime-main.15e620d4.js.map
    │       └── media
    │           └── logo.6ce24c58.svg

Production setup

Let’s start with the configuration for running the app in production.

Many developers would think about their development setup first.

However, production has more constraints than development, notably security and performance. As a consequence, I prefer starting with the production setup and then building a development setup that provides adequate dev / prod parity.

Serving HTML

In the “hybrid app” model, Django is in charge of serving HTML pages.

Let’s add a view that renders the index.html generated by create-react-app with the Django template engine.³

A catchall URL pattern routes any unrecognized URL with that view. Then the frontend URL router takes over and renders the appropriate page on the client side.

Configure the template engine:

# backend/todolist/settings.py

TEMPLATES = [
    {
        'DIRS': [FRONTEND_DIR / 'build'],
        ...,
    },
]

Create the view in a new module:

# backend/todolist/views.py

from django.views.generic import TemplateView

catchall = TemplateView.as_view(template_name='index.html')

Add a catchall URL pattern to the URLconf:

# backend/todolist/urls.py

from django.contrib import admin
from django.urls import path, re_path

from . import views

urlpatterns = [
    path('admin/', admin.site.urls),
    re_path(r'', views.catchall),
]

Build the frontend:

# in the frontend directory
yarn build

Install waitress, a production WSGI server, and start it locally:⁴

# in the backend directory, after executing poetry shell
poetry add waitress
waitress-serve todolist.wsgi:application

At this point, you can access the HTML page generated by create-react-app and served by Django at http://localhost:8080/. The page title displayed in the browser tab says “React App”, which confirms that we’re getting the intended HTML page.

However, the page is blank because loading the CSS and JS failed. Let’s fix that!

Serving static files

The “hybrid app” model allows us to take advantage of the staticfiles contrib app and its ecosystem. I find it convenient to serve static files with WhiteNoise and cache them with a CDN for performance.

The changes described below:

• Configure django.contrib.staticfiles and WhiteNoise;
• Declare that frontend/build/static/ contains static files;
• Serve them at /static/ where create-react-app expects them;
• Serve the few other files in the frontend/build/ directory at the server root.⁵

# backend/todolist/settings.py

INSTALLED_APPS = [
    ...,
    # before django.contrib.staticfiles
    'whitenoise.runserver_nostatic',
    ...
]

MIDDLEWARE = [
    ...,
    # just after django.middleware.security.SecurityMiddleware
    'whitenoise.middleware.WhiteNoiseMiddleware',
    ...,
]

STATICFILES_DIRS = [FRONTEND_DIR / 'build' / 'static']

STATICFILES_STORAGE = (
    'whitenoise.storage.CompressedManifestStaticFilesStorage')

STATIC_ROOT = BACKEND_DIR / 'static'

STATIC_URL = '/static/'  # already declared in the default settings

WHITENOISE_ROOT = FRONTEND_DIR / 'build' / 'root'

This setup requires moving the files to serve at the server root in a root subdirectory. That’s everything in frontend/build except index.html and static.

# in the frontend directory
mkdir build/root
for file in $(ls build | grep -E -v '^(index\.html|static|root)$'); do
    mv "build/$file" build/root;
done

Stop the WSGI server, install WhiteNoise, collect static files, and restart the server:⁶

# in the backend directory, after executing poetry shell
poetry add whitenoise
./manage.py collectstatic
waitress-serve todolist.wsgi:application

Now, if you refresh the page at http://localhost:8080/, the create-react-app welcome page loads. Success!

Also, the browser tab displays the React icon, which means that serving files like favicon.ico from the server root works.

Deploying

This setup is compatible with any method for deploying Django to production, provided the server that builds the application prior to deploying it is able to build the frontend and the backend. This boils down to installing Node.js and yarn.

A complete build script for the application looks like:

cd frontend

# 1. Build the frontend
yarn build

# 2. Move files at the build root inside a root subdirectory
mkdir -p build/root
for file in $(ls build | grep -E -v '^(index\.html|static|root)$'); do
    mv "build/$file" build/root;
done

cd ..

cd backend

poetry shell

# 3. Build the backend
./manage.py collectstatic --no-input

cd ..

Development setup

Now that we’re happy with the production setup, let’s build the development setup.

The challenge is to optimize dev / prod parity while preserving all features of the development servers.

Remove all build artifacts to make sure we don’t accidentally rely on them:

# in the backend directory
rm -r static

# in the frontend directory
rm -r build

If the Django and React development servers aren’t running anymore, start them again with:

# in the backend directory, after executing poetry shell
DJANGO_ENV=development ./manage.py runserver

# in the frontend directory
yarn start

Serving HTML

In production, we’re loading frontend/build/index.html as a template.

In development, it’s available at http://localhost:3000/.

It would be possible to build a Django template loader that loads templates from an URL. That would be optimal in terms of dev / prod parity but I don’t think it’s worth the complexity.

Instead I’m just going to write an alternative catchall view that proxies requests for index.html to the React development server.

Proxying is more appropriate than redirecting: redirecting could change the behavior of the browser and introduce significant differences between dev and prod.

Change the views module to:

# backend/todolist/views.py

import urllib.request

from django.conf import settings
from django.http import HttpResponse
from django.template import engines
from django.views.generic import TemplateView

def catchall_dev(request, upstream='http://localhost:3000'):
    upstream_url = upstream + request.path
    with urllib.request.urlopen(upstream_url) as response:
        response_text = response.read().decode()
        content = engines['django'].from_string(response_text).render()
        return HttpResponse(content)

catchall_prod = TemplateView.as_view(template_name='index.html')

catchall = catchall_dev if settings.DEBUG else catchall_prod

I’m keeping the original implementation in production because it benefits from Django’s caching of compiled templates.

Refresh http://localhost:8000/: while the page is blank, the page title says “React App”, which is good.

However, there’s a stack trace and a HTTP 500 error in the logs of runserver: "GET /static/js/vendors~main.chunk.js HTTP/1.1" 500. Oops, our catchall view is also receiving requests for static files and it crashes when it attempts to compile them as Django templates!

That was our next step anyway, so let’s improve catchall_dev to proxy requests for CSS and JS assets.

Serving static files

Here’s a version that runs only HTML responses through Django’s template engine.

# backend/todolist/views.py

import urllib.request

from django.conf import settings
from django.http import HttpResponse
from django.template import engines
from django.views.generic import TemplateView

def catchall_dev(request, upstream='http://localhost:3000'):
    upstream_url = upstream + request.path
    with urllib.request.urlopen(upstream_url) as response:
        content_type = response.headers.get('Content-Type')

        if content_type == 'text/html; charset=UTF-8':
            response_text = response.read().decode()
            content = engines['django'].from_string(response_text).render()
        else:
            content = response.read()

        return HttpResponse(
            content,
            content_type=content_type,
            status=response.status,
            reason=response.reason,
        )

catchall_prod = TemplateView.as_view(template_name='index.html')

catchall = catchall_dev if settings.DEBUG else catchall_prod

Refresh http://localhost:8000/: the application loads. Hurray!

Fixing autoreload

Modify the App.js file in the frontend. If you used create-react-app before, you expect the application to reload automatically, but it doesn’t.

Indeed, the browser’s console shows this error:

WebSocket connection to 'ws://localhost:8000/sockjs-node' failed: ...
The development server has disconnected.
Refresh the page if necessary.

This happens because our catchall view is unable to proxy WebSocket connections.

The question of dev / prod parity no longer matters here because there’s no autoreload in production. We can tell our application where webpack’s endpoint for hot module reloading listens with the WDS_SOCKET_PORT environment variable. Create a .env file in the frontend directory containing this line:

WDS_SOCKET_PORT=3000

Restart the frontend development server with yarn start. Refresh your browser at http://localhost:8000/ — be careful, running yarn start opens http://localhost:300/ instead — and modify App.js again. Now the browser reflects your changes immediately.

Optimized proxying

Development builds of the frontend can grow very large because they aren’t optimized for file size like production builds. If this makes the Django development server slow, you can avoid buffering entire files in memory with Django’s streaming response API. This can make a noticeable difference.

# backend/todolist/views.py

import urllib.request

from django.conf import settings
from django.http import HttpResponse, StreamingHttpResponse
from django.template import engines
from django.views.generic import TemplateView

def iter_response(response, chunk_size=65536):
    try:
        while True:
            data = response.read(chunk_size)
            if not data:
                break
            yield data
    finally:
        response.close()

def catchall_dev(request, upstream='http://localhost:3000'):
    upstream_url = upstream + request.path
    response = urllib.request.urlopen(upstream_url)
    content_type = response.getheader('Content-Type')

    if content_type == 'text/html; charset=UTF-8':
        response_text = response.read().decode()
        response.close()
        return HttpResponse(
            engines['django'].from_string(response_text).render(),
            content_type=content_type,
            status=response.status,
            reason=response.reason,
        )
    else:
        return StreamingHttpResponse(
            iter_response(response),
            content_type=content_type,
            status=response.status,
            reason=response.reason,
        )

catchall_prod = TemplateView.as_view(template_name='index.html')

catchall = catchall_dev if settings.DEBUG else catchall_prod

I didn’t explain all the design choices, so you may be wondering…

Why this design?

The setup I described takes advantage of features available in create-react-app and django.contrib.staticfiles to optimize page load performance and dev / prod parity with very little code and without introducing any additional dependencies.

Caching

Optimizing how browsers cache static assets is a performance-critical requirement.

The most reliable solution consists in:

• Inserting a hash of the contents of each file in the name of the file;
• Telling the browser that it can cache static files forever.

If the contents of a file changes, then its name changes and the browser loads the new version.

In practice this is more complicated than hashing and renaming files. For example, when CSS references an image, inserting a hash in the name of the image changes the contents of the CSS, which changes its hash. Implementing this behavior involves parsing static assets and understanding dependencies.

Django is able to parse and to modify CSS to handle dependencies. This is implemented in in ManifestStaticFilesStorage, which WhiteNoise’s CompressedManifestStaticFilesStorage builds upon.

Code splitting

Delivering the application gradually with code splitting is another performance-critical requirement.

When code splitting is enabled, a JS loader downloads chunks that define modules and imports them as needed. This is a relatively new requirement: code splitting wasn’t mainstream until 2016.

Django is unable to parse JS and understand dependencies between JS files. For this reason, the bundler needs to be responsible for inserting hashes in file names.⁷ Generally speaking, since the bundler is responsible for creating an optimized build of frontend assets, it makes sense to let it take care of inserting hashes in file names.

Putting it all together

Regardless of which system performs the hashing, a mapping from original file names to hashed file names is needed in order to substitute the hashed file name automatically whenever the developer references a static asset by its original file name. For example, main.js must be replaced with main.a3b22bcc.js.

The crux of the issue is to transmit this mapping from the bundler, which creates it when it builds the frontend, to the backend, which needs it to reference static files in HTML pages.

In our setup, the mapping is already applied in frontend/build/index.html when Django loads it as a HTML template, so the backend doesn’t need to do anything.

An alternative solution involves dumping the mapping in a JSON file with a webpack plugin, then loading it and applying it with a Django storage engine. webpack-bundle-tracker and django-webpack-loader implement this.

One more thing!

If you’ve been following closely, you noticed that our setup hashes frontend assets twice: webpack hashes them during yarn build and Django hashes them again during ./manage.py collectstatic. This produces files such as backend/static/js/main.a3b22bcc.5c290d7ff561.js.

These double-hashed files are never referenced.

However, generating them makes ./manage.py collectstatic a bit slower than it could be, especially on large apps. It’s possible to optimize this by subclassing CompressedManifestStaticFilesStorage, if you accept relying on private APIs.

Since I know you’re going to ask, here’s an implementation. It’s a fun hack and it works for me. Don’t shout at me it if breaks :-)

# todolist/storage.py

import collections
import re

from django.contrib.staticfiles.storage import  (
    ManifestStaticFilesStorage)
from whitenoise.storage import (
    CompressedStaticFilesMixin, HelpfulExceptionMixin)


class SkipHashedFilesMixin:

    _already_hashed_pattern = re.compile(r'\.[0-9a-f]{8}\.')

    def is_already_hashed(self, path):
        """
        Determine if a file is already hashed by webpack.

        The current implementation is quite lax. Adapt as needed.

        """
        return self._already_hashed_pattern.search(path)

    def post_process(self, paths, dry_run=False, **options):
        """
        Skip files already hashed by webpack.

        """
        if dry_run:
            return

        unhashed_paths = collections.OrderedDict()
        for path, path_info in paths.items():
            if self.is_already_hashed(path):
                yield path, None, False
            else:
                unhashed_paths[path] = path_info

        yield from super().post_process(
            unhashed_paths, dry_run=dry_run, **options)

    def stored_name(self, name):
        if self.is_already_hashed(name):
            return name
        else:
            return super().stored_name(name)


class StaticFilesStorage(
        HelpfulExceptionMixin, CompressedStaticFilesMixin,
        SkipHashedFilesMixin, ManifestStaticFilesStorage):
    """
    Similar to whitenoise.storage.CompressedManifestStaticFilesStorage.

    Add a mixin to avoid hashing files that are already hashed by webpack.

    """

After you create this file, enable the storage engine with:

STATICFILES_STORAGE = 'todolist.storage.StaticFilesStorage'