Speed it up! Recommendations to make your Django app faster

Let’s admit it, Django rules. It is simple, trustworthy, fast and it is written in Python. What else do you need?

However, we tend to abuse its easiness and we quickly forget that an application grows; and with it the amount of code processed as well as the amount of data saved on the database. This affects directly the application’s response time. That is why our application is fast in the beginning and then one day you check it and it is very slow for Django standards.

But it is not Django’s fault, it is our fault. We tend to think “Django ORM is so cool, let’s use it everywhere all the time with no considerations” or ” Django views rules, I may calculate everything for my view and send the information on the context and the app is still fast” or simply “Do not worry, I will improve that code on the next release” and we never commit with that refactor.

For these reasons, and considering that we are the main reason why the software fails, let’s check some considerations to make our Django app faster.

Reduce the amount of queries

One thing that we usually forget is that each query to the database is expensive; especially if you are using filters and if your database has a lot of information. For this reason, they are the main slowness factor on a Django app. The problem is that we use them carelessly because they are usually just a couple of lines and because we are not dealing with the database directly but through Django’s ORM; even if we wrote Python, it is running SQL, and SQL may be very slow.

That is why it is important to debug and keep track of your queries. One awesome way to do it, is to use Django Debug Toolbar library. It has a SQL panel that not only show you the amount of queries made to the database and their execution time; but gives you a detailed report of what does each query do, how many times a query is repeated, what are the most expensive queries, etc. If you are using Pydanny’s Cookie Cutter, this library is already installed for local instances.

On the other hand, If your app is already on production, you may use a monitoring system such as The Elastic Stack. These kind of tools may give you more detailed information, based on your site’s visits such as the slowest view in the app, its most visited view, its average loading time, etc. This may allow you to optimize the views that really matter first and prioritize the work to ensure that your visitors have a neat experience on your site.
Using these tools, I was able to reduce the queries in a very heavy view from 1500 to 31 in less than an hour working on it without losing information; reducing the load time from several seconds to less than one second. It was amazing!

Here are some tricks to reduce the queries in your app:

• Use select_related and prefetch_related when working with model relationships (one to one, many to many, foreign keys, etc).
• Check your filters, sometimes it’s faster to select all items and filter them with python directly.
• Assign your query sets into variables and keep using those variables instead of calling the query every time you need it.
• If you just need to check if the queryset has items or the number of items on the query, use exists or count function for that. It is much faster that checking the length of the query result.
• If you need the latest or the earliest item on the query set, use these methods directly instead of using order_by and then calling the last result of the query.
• It is obvious, but read Django’s querySets documentation. It contains a lot of good advices and recommendations to improve your queries’ times.

Go asynchronous wherever you may with Celery

There are certain processes that do not require to be executed synchronously that may take time and slow our app. For example, sending an email to the user after he/she performs an action, calculating values for a report or updating some information inside the database. As well, there are some processes that may take too long to finish that will affect our app directly.

For all these kind of actions, you may use Celery. According to its documentation:

Celery is a task queue with focus on real-time processing, while also supporting task scheduling.

http://docs.celeryproject.org/en/latest/index.html

With celery, you may create tasks. A task is simply a method that may be executed asynchronously. What celery does is to create several workers (4 by default) that will listen to the task query. Every time a task is added to the task query, the first available worker will get it and execute it. If several tasks are added at the same time, each worker will get one until they are all busy. Once one of them finishes, it will get the next task in the query until it is clean. The workers run simultaneously, so several tasks may be running at once. As well, being an asynchronous process, there is no control on the order the tasks finish. The first sent to the task query may be the last one to end its execution.

The workers run in a separate service from your app, so your app does not get affected by their running time. Even if a task raises an error, your app will not be affected by it (being a good tool to isolate error prone processes). For this reason, you may execute as tasks all those processes that you do not care where are going to be run.

As well, for very slow processes like data analysis, you may use celery along with a REST API or websockets, consuming these with a JS framework such as Vue. It is better for a user to see a loading message on the screen that nothing at all. As well, considering that celery is asynchronous, you may start displaying the results that are already calculated on the page without waiting for all the processes to finish.

Almost all our Django applications use Celery, you should give it a try!

Do not repeat yourself

One of the first rules of coding is the DRY principle, Don’t repeat yourself. You should always avoid coding functionalities or objects that already exist; especially when you are working with a framework such as Django. I bring this up because usually the code we repeat is slower than the one that comes with the library core. As well, using the tools given by the library allows you to check for performance consideration inside the documentation, better practices, etc.

I know this recommendation is kind of obvious, but it is impressive how many times I have found handmade code that was already implemented on the Django core. Check Django’s documentation or browse the web before coding; you may be pleasantly surprised.

Cache the predictable data

If you have a method with a result that will not change in a lot of time you may consider caching it. Django provides a low level cache that allows you to store specific data in a cache layer without caching the whole page.

Here is a small example:

from django.core.cache import cache
cache.set('my_key', 'hello, world!', 30)
cache.get('my_key')
cache.clear()

With this cache, you may calculate a heavy value once in a specific period of time reducing the response time on all the other requests. It is very useful if you do not need to return up to date information, if you are using an external API whose result updates once a day or once a week, etc. You may even mix it with celery and update the cache once the task finishes.

Please consider that if you are going to save a lot of information on the cache, you may consider creating a model to hold the information on the database instead. Using timestamp and celery processes may be a better way if you need to use the data for other analysis and processes. It depends on your requirements.

Keep your code clean

Keeping your code clean and organized is not only a good way to ease the development process but may impact directly on your app timing. Here are some considerations that I have found that are directly related to the app performance:

• Check for repeated code. If a section of code appears twice or more times on your code, consider creating a function or a method with it.
• In general, a method or a function should only be called once per data per process. If you need to use its result, store it on a local variable and use this variable instead.
• Avoid unnecessary processes on the __init__ method of your objects. If there is something heavy, like an external request, keep it on a different method so you may control when to call it.
• Check for unused variables and method calls. You may be spending time running a method whose result is not being used. Several code editors may help you to check this.
• Once again, read Django’s documentation and find the Django way to do what you need to do; it is usually cleaner and faster than other approaches.

Keep calm and enjoy Django!

I know that you may be very worried after reading this blog post. But take it easy! go and check you app and start improving its performance. One of the best things of python and Django is that they are very versatile so refactoring your code to improve your app will be much easier than you thought. So, happy coding!