Managing multiple geographies and time zones in Django
“Oh, I’ll just write a small Django app…”
For the last couple years, I’ve been building and maintaining a Django application to manage mobile push notification campaigns for myCANAL.
Through the Django admin, campaign managers define their messages and schedule campaigns targeted at user segments. A handful of cron jobs take care of importing segments from our data lake and sending campaigns via mobile push notification APIs.
At this point, Python glues together a few components. Django makes the web interface a breeze. SSO works seamlessly with django-auth-adfs.
All is well.
“So, we’d like to expand to new geographies…”
When I started building this app, I was only targeting French users of myCANAL. However, CANAL+ serves customers in 36 countries! We’re busy making myCANAL fully available in every geography1, including mobile push notifications.
Each geography has its own contents, events, and marketing team. As a consequence, it needs its own segments and campaigns. Furthermore, we’d like to isolate geographies from one another, so that campaign managers can only schedule campaigns in their own geography.
I could deploy one instance of the application per geography but that didn’t seem like a good choice. I didn’t want to deploy a dozen copies of an application that gets very little traffic. It has just one user and her backup. They use it no more than a few minutes every day.
Generally speaking, as we’re building a global video platform, we avoid local instantiation as much as possible.
This is a simple case of multitenancy. In multitenant B2B applications, company accounts are isolated from one another and users accounts are attached to company accounts.
Here, instead of company accounts, we have geographies, but it’s the same concept. The techniques I’m going to describe generalize well to more complex cases.
Mixins to the rescue
First, I need:
- a model to represent each geography,
- a data migration to create the geography that all instances are, implicitly, attached to until now.
# models.py
class Geography(models.Model):
"""
Area in which a subsidiary operates.
"""
name = models.CharField(
verbose_name="name",
max_length=100,
unique=True,
)
class Meta:
verbose_name = "geography"
verbose_name_plural = "geographies"
def __str__(self):
return self.name
# migrations/xxxx_create_default_geography.py
from django.db import migrations
DEFAULT_GEOGRAPHY_NAME = "France"
def create_default_geography(apps, schema_editor):
Geography = apps.get_model("...", "Geography")
Geography.objects.create(
id=1,
name=DEFAULT_GEOGRAPHY_NAME,
)
def delete_default_geography(apps, schema_editor):
Geography = apps.get_model("...", "Geography")
Geography.objects.get(id=1).delete()
class Migration(migrations.Migration):
dependencies = [
("...", "..."),
]
operations = [
migrations.RunPython(
create_default_geography,
delete_default_geography,
)
]
The actual model has a few other fields that store technical identifiers of geographies for interoperating with other systems.
Then, I create a model mixin that defines a foreign key to the Geography model:
# models.py
class Geographical(models.Model):
"""
Mixin to associate objects to a geography.
"""
geography = models.ForeignKey(
to=Geography,
on_delete=models.PROTECT,
related_name="+",
verbose_name="geography",
)
class Meta:
abstract = True
I disable reverse accessors with related_name="+" because I dislike how Django calls the default reverse accessors <something>_set and because I don’t need them.
Now attaching a geography to all models in the application is simply a matter of inheriting the mixin and creating migrations.
# models.py
from django.contrib.auth.models import AbstractUser
class User(AbstractUser, Geographical):
...
class Segment(Geographical):
...
class Campaign(Geographical):
...
# etc.
Perhaps you noticed that I enforced id=1 in the data migration that creates the default geography. This makes it safe to set a default value of 1 for the geography field in migrations:
$ django-admin makemigrations
You are trying to add a non-nullable field 'geography' to device without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
2) Quit, and let me add a default in models.py
Select an option: 1
Please enter the default value now, as valid Python
The datetime and django.utils.timezone modules are available, so you can do e.g. timezone.now
Type 'exit' to exit this prompt
>>> 1
At this point, I hit a roadbump: new users can’t log in via SSO anymore. Indeed, when a new user authenticates, their account is created automatically; this fails because the geography field doesn’t have a value.
For lack of a better solution, I’m going to set an arbitrary default value — at least that beats crashing.
# models.py
DEFAULT_GEOGRAPHY_NAME = "France"
class User(AbstractUser, Geographical):
def save(self, *args, **kwargs):
if self.geography_id is None:
self.geography = Geography.objects.get(
name=DEFAULT_GEOGRAPHY_NAME,
)
super().save(*args, **kwargs)
Admin magic
Writing a model admin for the Geography model is straightforward:
# admin.py
@admin.register(models.Geography)
class Geography(admin.ModelAdmin):
fields = ["name"]
list_display = ["name"]
ordering = ["name"]
search_fields = ["name"]
Now comes the more interesting part. I want the concept of geographies to be completely hidden from users — except superusers, if only because they must be able to attach user accounts to the correct geography; I’ll get to that later.
To achieve this, a model admin mixin restricts all querysets to objects in the user’s geography and creates new objects in the user’s geography:
# admin.py
class Geographical(admin.ModelAdmin):
"""
Mixin for objects associated to a geography.
"""
def get_queryset(self, request):
"""
Only show objects in the user's geography, except for superusers.
"""
qs = super().get_queryset(request)
if not request.user.is_superuser:
qs = qs.filter(geography=request.user.geography)
return qs
def save_model(self, request, obj, form, change):
"""
Create objects in the user's geography, except for superusers.
"""
if not change and not request.user.is_superuser:
assert obj.geography_id is None
obj.geography = request.user.geography
return super().save_model(request, obj, form, change)
I add the mixin to the model admins, like I did for models:
# admin.py
from django.contrib.auth.admin import UserAdmin
@admin.register(models.User)
class User(UserAdmin, Geographical):
...
@admin.register(models.Segment)
class Segment(Geographical):
...
@admin.register(models.Campaign)
class Campaign(Geographical):
...
# etc.
And… that’s it!
The two method overrides are quite easy to understand. They produce exactly the behavior I want. Think about everything the override of get_queryset gives me for free! This is where the power of the Django admin really shows.
Now, for superusers, I want the admin to show the geography field. This requires adjusting model admin configurations dynamically when the logged-in user is a superuser. I override a few other methods in the mixin:
# admin.py
class Geographical(admin.ModelAdmin):
...
def get_autocomplete_fields(self, request):
"""
Display the geography field, only for superusers.
"""
autocomplete_fields = super().get_autocomplete_fields(request)
if request.user.is_superuser:
autocomplete_fields = ["geography"] + list(autocomplete_fields)
return autocomplete_fields
def get_fieldsets(self, request, obj=None):
"""
Display the geography field, only for superusers.
"""
fieldsets = super().get_fieldsets(request, obj)
if request.user.is_superuser:
# Insert "geography" as the first field of the first fieldset.
fieldsets = [
(
fieldsets[0][0],
{
**fieldsets[0][1],
"fields": ["geography"] + list(fieldsets[0][1]["fields"]),
},
)
] + list(fieldsets[1:])
return fieldsets
def get_list_display(self, request):
"""
Display the geography field, only for superusers.
"""
list_display = super().get_list_display(request)
if request.user.is_superuser:
list_display = list(list_display) + ["geography"]
return list_display
def get_list_filter(self, request):
"""
Allow filtering by the geography field, only for superusers.
"""
list_filter = super().get_list_filter(request)
if request.user.is_superuser:
list_filter = list(list_filter) + ["geography"]
return list_filter
Notice how all these methods carefully avoid mutating values returning by the super() calls. Instead they build a new value with appropriate changes.
Indeed, the default implementation of get_list_display() returns the list_display class attribute directly. Mutating it would affect all future requests. The same pitfall exists in the other get_*() methods.
One more thing - time zones
I earned my commit bit to Django in 2011 by adding support for datetimes with time zone. I needed this to avoid datetime arithmetic problems around DST changes.
This feature also supports multiple time zones in a Django site, so that each user sees dates and times in their time zone. Back then I documented how to achieve this with a middleware but I never had the opportunity to do it in a real project — until now!
Luckily, our geographies don’t span multiple time zones. This means I can attach a time zone to each geography and activate it for users in this geography.
I add a timezone attribute to geographies and create a migration:
# models.py
class Geography(models.Model):
...
timezone = models.CharField(
verbose_name="time zone",
max_length=30,
)
I modify the model admin accordingly:
# admin.py
import pytz
class GeographyForm(forms.ModelForm):
timezone = forms.ChoiceField(
choices=[
(timezone, timezone)
for timezone in pytz.common_timezones
],
)
@admin.register(models.Geography)
class Geography(admin.ModelAdmin):
fields = ["name", "timezone"]
form = GeographyForm
list_display = ["name", "timezone"]
ordering = ["name"]
search_fields = ["name"]
I’m restricting choices to common time zones from the time zone database in the admin rather than setting choices in the model field to avoid serializing the large value of pytz.common_timezones in the migration. Not only would this make the migration file unreadable, but it would also generate meaningless migrations if the list of common time zones changes. This hack has no downsides here.
Finally I create a middleware to activate the time zone of the geography of the current user. The middleware has to support unauthenticated requests, at least for the login page.
# middleware.py
import pytz
from django.utils import timezone
def set_timezone(get_response):
def middleware(request):
if request.user.is_authenticated:
timezone.activate(
pytz.timezone(request.user.geography.timezone)
)
else:
timezone.deactivate()
return get_response(request)
return middleware
I add it to the MIDDLEWARE setting and that’s it!
I had a great experience writing that code. In a few hours, I added support for multiple geographies and multiple time zones. The end result is quite clean and maintainable.
This is only possible thanks to the robust basis and customization options provided by Django. That’s why I’m still enjoying writing Django code after a decade :-)
So I proudly enter the meeting where I’m going to explain how to send mobile push notification campaigns in our new geographies.
We make introductions:
Hello, I’m the community manager for geographies A, B and C.
Oops.
Stay tuned for the next post!
-
We say “geography” rather than “country” because areas where CANAL+ services are available don’t always match frontiers. ↩