A minimal REST API Django setup as a microservices

November 12, 2019 - Jan Pieter Bruins Slot

In this post I will set out on how to set up a django project that can be used as a REST API microservice. To see the end result, you can investigate the code here. An important disclaimer: the code presented here is to be used in a development environment, review security practices for the framework(s) when you want to use it in production or expose it to the internet.

1. Project structure

We will using the following stack:

Let’s first start with the project structure and create some files and folders. Create the following files and folders:

$ tree -L 1 --dirsfirst
.
├── config/
├── docker-compose.yml
├── Dockerfile
└── README.md

First update our Dockerfile:

$ cat Dockerfile

FROM python:3.8

COPY . /srv/minimal-django
RUN pip3 install -r /srv/minimal-django/config/requirements.txt

CMD [ "gunicorn", "--config", \
    "/srv/minimal-django/config/gunicorn.py", \
    "minimal_django.wsgi" \
]

We will use a Python base image, and of course you can choose your own base image as you like. In the Dockerfile we’ll define that we need to copy the code, and install the Python package requirements. Lastly, we use Gunicorn as the WSGI application server.

Now, for convenience sake we’re using docker-compose to orchestrate our docker containers. So we will add the following contents to our docker-compose.yml file:

$ cat docker-compose.yml

version: "3"

volumes:
  postgres:
    driver: local

services:
    postgres:
      image: postgres:12.0
      environment:
        - POSTGRES_NAME=postgres
        - POSTGRES_USER=postgres
        - POSTGRES_PASSWORD=mysecretpassword
        - PGDATA=pgdata
      volumes:
        - postgres:/var/lib/postgresql/data/pgdata

    minimal_django:
      build:
        context: .
      ports:
        - "4001:4001"
      volumes:
        - .:/srv/minimal_django
      depends_on:
        - postgres

We will be using PostgreSQL as our datastore in this example, and we will calling our Django application minimal_django.

Next, we need to add the configuration files in the folder config/, and we will add the following files:

$ ls config/
gunicorn.py  requirements.txt

But first, let’s add the python packages we will using in the Django application, and we will be adding those in the requirements.txt file.

$ cat config/requirements.txt

django==2.2.7
gunicorn==19.9.0
psycopg2-binary==2.8.4
djangorestframework==3.10.3

As you can see we will be using Django REST Framework for the REST API. Next up we will create the Django project.

2. Create the Django project

Now, that we have the basic structure set up. We need to create an actual Django project, and we will be using the Docker container for this. This way we don’t necessarily install Django on our host system.

First, build the minimal_django container.

$ docker-compose build minimal_django

Second, we will start an interactive shell in the minimal_django container.

$ docker-compose run minimal_django sh

Third, we will create the Django project by executing the following command in the attached shell:

# cd /srv/minimal-django
# django-admin startproject minimal_django

We’ve created the project ‘inside’ the container, and because we’ve attached a volume to the location of the files. So now we need to fix the file permissions on our host system, because they have been created as the root account. So, exit from the shell in the docker container, and change the owner of the folder that we’ve just created.

$ chown -R $USER:$USER minimal_django/

3. Update settings.py

Because this will be a minimal Django implementation we will updating the settings.py file, and remove the non-essential elements.

1. Update ALLOWED_HOSTS

   ALLOWED_HOSTS = ["*"]
   

2. Update INSTALLED_APPS

   INSTALLED_APPS = [
       'rest_framework',
       'api',
   ]
   

3. Remove MIDDLEWARE, and its contents

4. Remove TEMPLATES, and its contents

5. Update the DATABASES dict

   DATABASES = {
       'default': {
           'ENGINE': 'django.db.backends.postgresql_psycopg2',
           'NAME': "postgres",
           'USER': "postgres",
           'PASSWORD': "mysecretpassword",
           'HOST': "postgres",
           'PORT': "5432",
       }
   }
   

6. Remove AUTH_PASSWORD_VALIDATORS, and its contents

7. Remove STATIC_URL, and its contents

8. Remove LANGUAGE_CODE, USE_I18N, USE_L10N, and its contents

9. Add the following Django REST Framework configuration

   REST_FRAMEWORK = {
       "DEFAULT_RENDERER_CLASSES": [
           'rest_framework.renderers.JSONRenderer',
       ],
       "DEFAULT_AUTHENTICATION_CLASSES": [],
       "DEFAULT_PERMISSION_CLASSES": [],
       "UNAUTHENTICATED_USER": None,
       'PAGE_SIZE': 10,
       'DEFAULT_PAGINATION_CLASS': \
           'rest_framework.pagination.LimitOffsetPagination',
       'DEFAULT_FILTER_BACKENDS': [
           'rest_framework.filters.OrderingFilter'
       ],
   }
   

10. Adding logging configuration, this is optional and you can edit/update it to your preferences.

    LOGGING = {
        'version': 1,
        'disable_existing_loggers': True,
        'formatters': {
            'verbose': {
                'format':
                '%(pathname)s:%(lineno)d (%(funcName)s) '
                '%(levelname)s %(message)s',
            },
            'simple': {
                'format': '%(levelname)s %(message)s'
            },
        },
        'handlers': {
            'null': {
                'level': 'DEBUG', 'class': 'logging.NullHandler',
            },
            'stderr': {
                'level': 'DEBUG',
                'formatter': 'simple',
                'class': 'logging.StreamHandler',
            },
        },
        'loggers': {
            'django': {
                'handlers': ['stderr'],
                'propagate': True,
                'level': 'WARNING',
            },
        },
    }
    

NOTE: Again, this is not a production ready configuration!

4. Configuring the Django api app

Now we are ready to create the Django app. We will call it api, and within it we will create a simple REST API. First, create a folder in which it will be contained.

$ mkdir minimal_django/api/

Inside this folder create the following files:

$ ls minimal_django/api/
__init__.py  models.py  serializers.py  urls.py  views.py

We will start with the file models.py, and we will create a very simple model.

# minimal_django/api/models.py
from django.db import models


class Pet(models.Model):
    name = models.CharField(max_length=64)

Next, update the views.py file where we will add the Django Rest Framework ViewSet.

# minimal_django/api/views.py
from rest_framework import viewsets

from api.models import Pet
from api.serializers import PetSerializer


class PetViewSet(viewsets.ModelViewSet):
    queryset = Pet.objects.all()
    serializer_class = PetSerializer

Create and update the serializers.py file that will be used in our application.

# minimal_django/api/serializers.py
from rest_framework import serializers

from api.models import Pet


class PetSerializer(serializers.ModelSerializer):
    class Meta:
        model = Pet
        fields = ('name', )

Update the urls.py file, and we will use the Django Rest Framework default routers.

# minimal_django/api/urls.py
from django.conf.urls import include, url
from rest_framework.routers import DefaultRouter

from api import views

router = DefaultRouter()
router.register(r'pets', views.PetViewSet)

urlpatterns = [url(r'^', include(router.urls))]

Also update the urls.py file, of the Django project itself and remove the paths to the admin environment.

# minimal_django/urls.py
from django.conf.urls import url, include

urlpatterns = [
    url(r'^', include('api.urls')),
]

5. Configuring Gunicorn

We will be using Gunicorn as the WSGI application server. Create and update the gunicorn.py file with the following content. Refer to the configuration documentation for more configuration options.

# config/gunicorn.py

# Documentation at: http://docs.gunicorn.org/en/latest/index.html
django_project_name = "minimal_django"

# Chdir to specified directory before apps loading
chdir = "/srv/minimal-django/minimal_django"

# The socket to bind to
bind = ":4001"

# The class of worker processes for handling requests
worker_class = "sync"

# Is a number of OS processes for handling requests
workers = 4

# Is a maximum count of active greenlets grouped in a pool that will be
# allowed in each process
worker_connections = 1000

# The maximum number of requests a worker will process before restarting
max_requests = 5000

# Workers silent for more than this many seconds are killed and restarted
timeout = 120

# Set environment variable
raw_env = \
    ["DJANGO_SETTINGS_MODULE={}.settings".format(django_project_name)]

# The access log file to write to, "-" means to stderr
accesslog = "-"

# The error log file to write to, "-" means to stderr
errorlog = "-"

6. Start it up

Before we can run the application, we need to make initial migrations and propagate the model definition into our database schema. We will be using our Docker container to create those migrations, and again as a result we need to change the folder permissions in the end.

# First start the postgresql container
$ docker-compose up -d postgres

# Create the migrations
$ docker-compose run minimal_django \
    /srv/minimal-django/minimal_django/manage.py \
    makemigrations api

# Actually migrate the schema
$ docker-compose run minimal_django \
    /srv/minimal-django/minimal_django/manage.py \
    migrate

# Reset the folder permissions of the created migrations folder
$ chown -R $USER:$USER minimal_django

With that done we’re able to run our application, since we already started our postgresql container we will able to execute the following command.

$ docker-compose up minimal_django

Now, the application should be running and you’ll be able to access the API with curl or any other method.

$ curl http://localhost:4001/pets/