Many of us have experienced slow websites and laggy apps at least once while programming. These issues are inherent to the design of the systems we use each day: some operations take a (relatively) long time to complete. Transferring bytes over thousands of kilometres between you and the server, reading tiny magnetic fields on the spinning disk, and other such activities may take a moment. However, while your system is waiting for these resources it is essential to remain usable and responsive, and not waste precious CPU cycles. The concepts of concurrency and asynchronous programming were introduced to address these concerns.
Asynchronous programming is a technique that enables your program to start a potentially long-running task and still be able to be responsive to other events while that task runs, rather than having to wait until that task has finished – Mozilla
Synchronous programming, on the other hand, is when a program does all tasks strictly one-after-the-other, and will wait for extended operations to remain idle.
Why is it so important to let the system take care of other tasks while it waits for some long IO operation? To give you some perspective, a typical 3Ghz processor can perform 30,000,000 operations in 10ms, the average time it takes for a hard drive to read some data. Imagine how wasteful our systems would be if they were not able to multitask!
In this article I will teach you how to use the power of asynchronous programming in Python, using its asyncio library. We will go over some general concepts, implementation details, usage examples, and advanced capabilities of asyncio. After reading my manual, you will be able to use asynchronous programming in Python with confidence and use its capabilities in your projects.
How async works on low level
Asynchronous I/O is a form of input/output processing that permits other processing to continue before the transmission has finished – Wikipedia
To enable asynchronous programming in high-level languages such as Python, the underlying infrastructure (OS) must support it. As of today, all systems support some way of asynchronous operations. There are many different implementations varying by the ease of use and efficiency, and I will briefly talk about some of them.
Multiprocessing
A simple approach to asynchronous IO would be to take all the blocking operations and place them in another process. Then, the main process would communicate with the IO process using sockets.
The advantage of this method is relative simplicity for OS developers – once you implement multitasking, the system will support asynchronous programming. On the downside, this may result in wasted resources and complicated programming for the software itself.
Polling
This is a very common implementation of asynchronous IO. In this case, the OS is responsible for managing the IO operations. The program will instruct the system to perform the IO operation specified, and the system will do so in the background. The program will periodically ask (poll) the system on the status of the operation. Once finished, the poll will return the data.
The advantage of this method is it is easy to use in the programs, and it is pretty intuitive. However, this can result in wasted resources, as the program would spend CPU cycles asking the system about the status of the operations (potentially millions of times!).
Interrupts/Callbacks
This is the most performant and elegant approach to asynchronous IO. The program will ask the system to perform an IO operation, and then continue working. Once the IO operation is done, the system will notify the program by a signal, interrupt, or callback. This will cause the program to stop whatever it was doing and deal with the incoming data.
One important implication of this approach is that the program can be interrupted at an unpleasant time – for example, in the middle of a blocking operation or a long computation. Several methods are used to mitigate this, but they are out of the scope of this article.
How async works in Python
Now that you have an understanding of how asynchronous operations are handled by the system, we can move on to the Python implementation. In Python, the library responsible for asynchronous programming is asyncio
.
First introduced in Python 3.4, asyncio lacked many features that were important to developers; mainly the lack of async/await
syntax was missed. Asyncio evolved rapidly since then and by version 3.7, Python can boast first-class support for contemporary asynchronous programming support.
But how does asyncio actually work? Let’s try to analyze it ourselves. For asynchronous IO to happen, Python must be doing more than one thing at once – at the very least, it must be able to wait for some blocking operation and execute the rest of the code at the same time. Since there is only one Python, and many things it needs to do at the same time, it must be switching between them quickly to make an impression of concurrency.
To make things simpler, let’s introduce an example. In chess, there are such things as simultaneous exhibitions or simultaneous displays. During such games, a highly-skilled player (such as a grandmaster) plays many games simultaneously with a number of other players. For example, a skilled player (X) may play 5 games at the same time with 5 other players (A, B, C, D, and E). X will go from table to table, taking 10 seconds to make a move before moving on to the next table. Other players will have 40 seconds to make their move before X comes around to them. Supposing the average game is 50 moves, it would take about 40 minutes to play all 5 games. However, if these games were played sequentially (one after the other), it would take 3.5 hours!
Now, imagine the grandmaster X as the Python interpreter. It has many instructions in the program and many long operations that it waits for. Python will go from one operation to the other and see if it is finished and if anything must be done. It will go round and round until the program is finished.
This is a rough description of Python async internals and the workings of something called the event loop. In the next section, we will go over some of the main concepts and definitions of the asyncio
library in Python.
Main concepts of asyncio in Python
To properly master the asyncio
library, you will need to understand a number of concepts and definitions that are used in it.
Generator
Generators in Python are special kinds of functions that can suspend their execution and resume it later. They are often used as iterator functions; the most common one is range
. Generators use the yield
keyword to return a value without completely exiting the function – it can be continued to get more values. Below is a very simple example of a generator that returns numbers from 1 to 5:
def get_numbers():
yield 1
yield 2
yield 3
yield 4
yield 5
for i in get_numbers():
print(i)
>> 1
>> 2
>> 3
>> 4
>> 5
In Python 3.3, a new keyword was introduced to generators: yield from
. It allowed users to combine generators together by indicating that the next value should be yielded from another generator. Consider this example, which does the same thing as the first but with extra steps:
def get_all_numbers():
yield 1
yield from get_some_numbers()
yield 5
def get_some_numbers():
yield 2
yield 3
yield 4
for i in get_all_numbers():
print(i)
>> 1
>> 2
>> 3
>> 4
>> 5
It may be unclear at first as to why this is important or even remotely useful, but this has been a key step in implementing the async/await
syntax in Python which I will talk about next.
Coroutine
A coroutine is a function that can be paused (suspended) and resumed at a later time. Unlike regular functions (or subroutines), which can only be run from start to finish, coroutines can be started, stopped, and resumed many times.
Coroutines are a more generalized form of subroutines. Subroutines are entered at one point and exited at another point. Coroutines can be entered, exited, and resumed at many different points. They can be implemented with the
async def
statement. – Python docs
So far it sounds like coroutines are just generators. Indeed, they are very similar, but there are a few key differences. Coroutines are defined using the async def
keyword, and can use the await
syntax inside. await
keyword is very similar to yield from
– it turns the control over to another function and tells the caller that it now has to wait for it to complete.
Coroutines are basic building blocks of asynchronous workflows in Python and we will get a chance to work with them a lot.
Task
If coroutines are special kinds of functions, tasks are entities that run such functions. A task accepts a coroutine and provides an interface to run and oversee the coroutine – check its completion status, get the resulting data, or see the error message. The closest relative to the Task
object are Promises from JavaScript or Futures from Rust.
Tasks are used to run coroutines in event loops. If a coroutine awaits on a Future, the Task suspends the execution of the coroutine and waits for the completion of the Future. When the Future is done, the execution of the wrapped coroutine resumes – Python docs
Event loop
I gave a short introduction to the event loop in the chess example, but now we know enough concepts to more precisely define it. The event loop is the engine behind asynchronous applications, and it is responsible to running Tasks.
The event loop holds in memory all Tasks that are currently running in the program. It will walk them one-by-one, and see if a Task is waiting on something or ready to be executed. It will execute the pending code, and the Task will either end up waiting for something else or exit. The event loop will run endlessly until the whole program terminates.
Python is not a pioneer in using an event loop model for asynchronous tasks. It has been used in JavaScript for quite some time with great success.
I will mention that regular application developers will rarely need access to the event loop itself. This is a low-level API and is intended for library and framework developers. However, I will explore how to use it in the last parts of this article.
Creating and running coroutines
At last, we have all the necessary knowledge of asyncio
internals, and we are ready for some hands-on practical learning! In this section, I will explore how to create and run some basic subroutines using asyncio
in Python.
To create functions in Python, we write def functionname(arguments...)
. Coroutines are special kinds of functions, and thus have special kind of definitions – async def
:
async def my_coroutine():
...
In the function body, we can call other subroutines and await
for them:
import asyncio
async def my_coroutine():
await asyncio.sleep(1)
asyncio.sleep()
is also a coroutine – this is why we can await
it. await
means that we give control to another coroutine and must wait until it is finished. In this simple example, we will wait 1 second.
Finally, after creating a subroutine, we will need a way to run it. With asyncio
, it is very simple to run coroutines. To do so, we will use asyncio.run()
function:
import asyncio
async def my_coroutine():
print("before sleeping")
await asyncio.sleep(1)
print("after sleeping")
asyncio.run(my_coroutine())
>> before sleeping
>> after sleeping
asyncio.run()
accepts a coroutine and runs it in the main event loop. Note that the call to run
is synchronous and blocking: it starts an event loop, which will only be running coroutines. For example, consider this code:
import asyncio
async def my_coroutine():
print("before sleeping")
await asyncio.sleep(1)
print("after sleeping")
asyncio.run(my_coroutine())
print("after event loop")
>> before sleeping
>> after sleeping
>> after event loop
After learning how to create and run simple coroutines, let’s explore some real-world examples of using asyncio.
Running blocking operations asynchronously
Creating coroutines just to asyncio.sleep
does not sound very useful. After all, I promised you to teach how to run long blocking operations asynchronously. I will show you how to do it using a very straightforward example – making HTTP requests.
I mentioned before that all you need to do to wait for a routine is to add await
in front of it. So can it be that making HTTP requests is the same? Let’s try. We will call an API called CatFacts, which is supposed to give us a random cat fact:
import asyncio
import requests
ENDPOINT = "https://catfact.ninja/fact"
async def call_api():
result = await requests.get(ENDPOINT)
print(result.json())
asyncio.run(call_api())
We expect this code to call the endpoint specified and print the results. Try running it! (remember to install the requests
package if you do not have it). To much surprise, we get an error:
Traceback ...
...
File "api-request-bad-example.py", line 8, in call_api
result = await requests.get(ENDPOINT)
TypeError: object Response can't be used in 'await' expression
What does this error mean? It says that we cannot await
a Response
object. This is true – you can only await
coroutines – that is, functions defined with async def
. But requests.get
is not a coroutine! It is written as a plain old blocking function, not adapted for asyncio
. While there are other HTTP request libraries that support asyncio
natively, for the purposes of this section we will work with requests
.
So how can we make blocking functions compatible with asyncio
? We can do this by getting access to the event loop and using a function called run_in_executor
. This function will take a runnable and run it in a separate thread, controlled by the main event loop. In such a case, your code will run in a separate thread, allowing your program to run without blocks, and at the same time allow such code to interface with asyncio – so we can await
it. Below is the API call example rewritten using run_in_executor
:
import asyncio
import requests
ENDPOINT = "https://catfact.ninja/fact"
async def call_api():
loop = asyncio.get_event_loop()
result = await loop.run_in_executor(None, lambda: requests.get(ENDPOINT))
print(result.json())
asyncio.run(call_api())
Let’s examine this example closely. On the first line of the call_api
function, we use asyncio.get_event_loop
to get access to the event loop which is responsible for running our code. Then we use the loop.run_in_executor
function to execute our blocking code in a separate executor (another name for thread). The first argument to this function is the executor to use – just set it to None
for now to use the default one. The second argument is the function to execute. I used an inline function that runs and returns requests.get(ENDPOINT)
. Lastly, we await
the result of the execution and print it. Here is the cat fact I got:
{'fact': 'In Holland’s embassy in Moscow, Russia, the staff noticed that the two Siamese cats kept meowing and clawing at the walls of the building. Their owners finally investigated, thinking they would find mice. Instead, they discovered microphones hidden by Russian spies. The cats heard the microphones when they turned on.', 'length': 318}
Before we move on, let’s refactor this code a little bit to make our next examples simpler. I defined this function to make API calls to the cat fact service:
import asyncio
import requests
ENDPOINT = "https://catfact.ninja/fact"
async def get_cat_fact():
loop = asyncio.get_event_loop()
result = await loop.run_in_executor(None, lambda: requests.get(ENDPOINT))
return result.json().get('fact')
I put this in the file aiocat.py
and you can use it in other examples like:
import asyncio
from aiocat import get_cat_fact
async def main():
print(await get_cat_fact())
asyncio.run(main())
Now we have set up the infrastructure for writing many examples in this tutorial, and we will move on to the next topic.
Run multiple coroutines together
So far, we have worked with asyncio
without using the benefits of asynchronicity: our coroutines ran one after another. Now we will really put our system’s resources to use by running multiple coroutines at the same time. We will start with an easy example – fetching 5 cat facts from the cat fact service. Before we begin, let’s measure how much time it will take us to run these requests one at a time:
import asyncio
import time
from aiocat import get_cat_fact
async def main():
for i in range(5):
print(f'Cat fact #{i + 1}')
print(await get_cat_fact())
start = time.perf_counter()
asyncio.run(main())
print(f'Completed in {time.perf_counter() - start} seconds')
This code would output something like this:
Cat fact #1
...
Cat fact #2
...
Cat fact #3
...
Cat fact #4
...
Cat fact #5
...
Completed in 0.8712859000006574 seconds
We see that it takes roughly 1 second to run 5 API calls one after another. Now let’s rewrite it using asyncio.gather
, a function from asyncio to run multiple coroutines in parallel:
import asyncio
import time
from aiocat import get_cat_fact
async def main():
facts = await asyncio.gather(
get_cat_fact(),
get_cat_fact(),
get_cat_fact(),
get_cat_fact(),
get_cat_fact()
)
print(*facts, sep='\\n')
start = time.perf_counter()
asyncio.run(main())
print(f'Completed in {time.perf_counter() - start}s')
Take a look at this piece of code. We are using function asyncio.gather
and passing our 5 coroutines to it. Then, we await the result and print it.
gather
will take all the awaitables we pass to it, and await them all at once. They will start at the same time, and the function will return when all of them finish. If you worked with Promises in JS, this is similar to Promise.all
function.
You can try it out by running this code – your runtime should decrease drastically. For me, this code runs in 0.23 seconds – almost 5 times faster than the sequential version!
Let’s write a more challenging example. Suppose you want to ask the user how many cat facts they would like to read, and then fetch them. You can try writing it yourself, or read my take on it:
import asyncio
from aiocat import get_cat_fact
async def main():
n = int(input('How many cat facts would you like? '))
requests = [get_cat_fact() for i in range(n)]
facts = await asyncio.gather(*requests)
print(*facts, sep='\\n\\n')
asyncio.run(main())
You can see that we prompt the user for the number of facts. Then we use the list comprehension notation to get a list of awaitables of cat facts. Then we pass this list to asyncio.gather
– note that we need to unpack it with *
operator – gather
does not accept lists. Lastly, we print the facts, separated by 2 new lines.
Tasks in asyncio
By now you should be comfortable creating and running simple coroutines. To further extend our abilities and tooling, we will move on and learn about another powerful feature of asyncio
– Tasks.
A Task
is a class within asyncio
that gives us powerful ways to schedule, control, and observe coroutines. A task is simply a wrapper around a coroutine that allows us to inspect it and perform certain actions (cancel, for example).
To create a Task, you can use the function asyncio.create_task
. You will need to pass a coroutine instance to it. Consider this example:
import asyncio
from aiocat import get_cat_fact
async def main():
get_fact_task = asyncio.create_task(get_cat_fact())
result = await get_fact_task
print(result)
asyncio.run(main())
You can see that we create a get_fact_task
by using asyncio.create_task
and passing get_cat_fact()
coroutine to it. Then we await
the result of this task, which is simply the result of the underlying coroutine. After it finishes, we print the result.
So far it does not seem like much – we achieve the same things without using tasks, and adding in some complexity. Let’s dive into some additional features within tasks.
Naming tasks
You can give unique names to tasks to identify them and use them for logging purposes. To do this, you can call set_name
function on the task object, and use get_name
to get the name of the task. For example,
import asyncio
from aiocat import get_cat_fact
async def main():
get_fact_task = asyncio.create_task(get_cat_fact())
get_fact_task.set_name('Task to get a random fact about cats')
result = await get_fact_task
print(f'Finished running task: {get_fact_task.get_name()}')
print(result)
asyncio.run(main())
This code would produce:
Finished running task: Task to get a random fact about cats
A cat only has the ability to move their jaw up and down, not side
to side like a human can.
Check the status of the task
After you have created the task, it is scheduled to run in the event loop. You can check if it is done at any point in time using the function .done(). You can also cancel the task using .cancel() and check if it was cancelled using .cancelled()
:
import asyncio
from aiocat import get_cat_fact
async def main():
task1 = asyncio.create_task(get_cat_fact())
task2 = asyncio.create_task(get_cat_fact())
print(f'Task 1: done: {task1.done()}, cancelled: {task1.cancelled()}')
print(f'Task 2: done: {task2.done()}, cancelled: {task2.cancelled()}')
task1.cancel() # cancel first task
await task2 # wait until second task is done
print(f'Task 1: done: {task1.done()}, cancelled: {task1.cancelled()}')
print(f'Task 2: done: {task2.done()}, cancelled: {task2.cancelled()}')
asyncio.run(main())
This is the output expected:
Task 1: done: False, cancelled: False
Task 2: done: False, cancelled: False
Task 1: done: True, cancelled: True
Task 2: done: True, cancelled: False
Scheduling tasks
The way tasks are executed is a little different than how coroutines are executed directly. If you write results = await get_cat_fact()
, the coroutine is created and execution is suspended until we get a result. When we write asyncio.create_task(get_cat_fact())
, the routine is wrapped up into a task, but it does not execute right away. Instead, it is scheduled to run in the event loop. Basically, asyncio
tells you that this task will run at some point, but does not specify when.
Practically, to get to run this task, the event loop must have some time to spare. If you will be waiting for something (some other coroutine) it may execute pending tasks in the background. You can also await the task directly (like in the example above) and this will be the cue for asyncio
to run it immediately.
To illustrate how tasks are scheduled and run, let’s write one more example. Firstly, we will modify our get_cat_fact()
function. As is, API requests happen too fast and with no errors, so I will add some random delays and errors to illustrate the features of tasks:
async def real_get_cat_fact():
await asyncio.sleep(random.random())
if (random.random() > .3):
return get_cat_fact()
else:
raise Exception('API Error!')
You can see that we will sleep up to 3 seconds and with probability 30% will throw an error. Now let’s move on to our next example. In this one, we will create a couple of tasks, and try sleeping until all of them are complete while watching their progress:
import asyncio
from aiocat import real_get_cat_fact
def check_task_status(task: asyncio.Task):
# check if task is done and print results
if task.done():
if task.exception() is not None:
print(f'Task #{task.get_name()} failed')
return True
print(f'Task #{task.get_name()} complete, result - {task.result()}')
return True
return False
# else task is in progress
async def main():
# create array of tasks with numbered names
tasks = [asyncio.create_task(real_get_cat_fact(), name=str(i+1)) for i in range(5)]
while tasks:
# check status of tasks and remove those that are finished
tasks = [task for task in tasks if not check_task_status(task)]
await asyncio.sleep(0.1)
asyncio.run(main())
Firstly, take a look at the check_task_status
function. It is using the .done()
and .result()
functions that we have seen, and a new one – .exception()
, which returns an exception if your task throws one. The check_task_status
function checks if a task is finished executing, print its results and return true. If it is still in progress, it returns false.
In the main function, we first create a list of tasks, numbered from 1 to 5. Then, we go in a loop, that runs until we run out of tasks in the tasks array. We check the status of each task and remove the finished ones. Then we sleep for 100ms.
If you run this, you will get an output similar (but different) to this:
Task #2 failed
Task #1 complete, result - The leopard is the most widespread of all big cats.
Task #3 complete, result - Blue-eyed, pure white cats are frequently deaf.
Task #4 complete, result - When your cats rubs up against you, she is actually marking you as 'hers' with her scent. If your cat pushes his face against your head, it is a sign of acceptance and affection.
Task #5 complete, result - The Ancient Egyptian word for cat was mau, which means "to see".
Remember when I said that the event loop will execute our background tasks in its free time? In this example, the execution happens while we await asyncio.sleep
, or when we instruct the interpreter to wait 100 ms. This is another way of using tasks – scheduling background operations to run when our code is not doing anything else.
Task callbacks
The last feature of tasks that we will look at is callback functions. You can set up functions that will be called automatically when a task finishes. Let’s rewrite the above example using such functions. To add a callback to the task, you can use the function add_done_callback
. You pass in the callback and it will be called with the task object once the task is done, failed, or cancelled.
import asyncio
from aiocat import real_get_cat_fact
def check_task_status(task: asyncio.Task):
if task.done():
if task.exception() is not None:
print(f'Task #{task.get_name()} failed')
return
print(f'Task #{task.get_name()} complete, result - {task.result()}')
async def main():
tasks = []
for i in range(5):
task = asyncio.create_task(real_get_cat_fact())
task.set_name(str(i + 1))
task.add_done_callback(check_task_status)
tasks.append(task)
await asyncio.gather(*tasks, return_exceptions=True)
asyncio.run(main())
In this example, we create a list of tasks using a for loop and use the check_task_status function as a callback. It will print the result of the task or an error message. Lastly, use asyncio.gather
to wait until all tasks are complete.
Iterators in asyncio
Now it is time to move on to our next topic – asynchronous iterators. Iterators in Python are special classes that are used to iterate over a sequence of items. It may be finite, like an iterator going over a list of objects, or infinite, like producing random numbers.
The difference between a regular iterator and the asynchronous one is that you need to await the result of the asynchronous iterator. For example, when traversing an iterator, you would typically use the for
operator. However, for the asynchronous iterator, you should use async for
. Likewise, instead of using the next()
function to get the next value in the iterator, you should use await anext()
.
To implement a custom asynchronous iterator, you need to define a class that implements __aiter()__
and __anext()__
functions. __aiter()__
should return an instance of the iterator, and __anext()__
should return an awaitable result of the iterator.
We will look into these functions by writing a simple example. Let’s write an iterator CatFactIterator
that will produce 5 cat facts asynchronously:
import asyncio
from aiocat import get_cat_fact
class CatFactIterator:
# how many facts were produced
count: int
def __init__(self):
# initialize
self.count = 0
def __aiter__(self):
return self
def __anext__(self):
if self.count == 5:
# stop after 5 facts
raise StopAsyncIteration("Ran out of cat facts!")
self.count += 1
return get_cat_fact()
async def main():
# create instance of iterator
facts = CatFactIterator()
# iterate asynchronously over iterator
async for fact in facts:
print(fact)
asyncio.run(main())
Look at the CatFactIterator
class. It is supposed to return 5 cat facts. We keep track of how many were already produced in the count
attribute. The aiter
function just returns the instance of the class and anext
returns the next cat fact. To indicate that the iterator is complete, we raise the StopAsyncIteration
exception.
Context managers in asyncio
Another area in Python that benefits from asyncio is context management. Context managers are used to allocate and release resources that are used in your program. One popular context manager is the open
function:
with open('file', 'r') as f:
# file opens
f.write('something')
# file closes automatically
Context managers are typically used when you are working with resources that you have to allocate (open, connect) and release (close, disconnect) when you are done. With the arrival of asyncio, we can now perform these operations asynchronously.
To create an asynchronous context manager, you need to define a class which implements __aenter__
and __aexit__
functions. aenter
is called when you enter a context manager and aexit
is called when you exit the context manager. To use it, you need to write async with
instead of regular with
. You will see some examples of using asynchronous context managers in the next section.
Making HTTP requests in asyncio the right way
So far, we have explored all the main topics of asyncio, and that should be enough to start working on most of the everyday tasks of a Python developer. As a bonus, I would like to introduce you to the library aiohttp
, an asynchronous HTTP client.
If you recall our earlier example, when we used requests
to call an API, it was kind of hacky. We had to use loop.run_in_executor
, because requests
does not support asynchronous programming natively.
Instead of doing that, I highly recommend using a native asynchronous library such as aiohttp
when you need to make HTTP calls. It is faster, safer, and easier to use.
To install aiohttp
, run the following command:
pip install aiohttp
Now, let’s make a call to the cat fact API we have been using so far. The code below does exactly that:
import asyncio
import aiohttp
async def main():
async with aiohttp.ClientSession() as session:
async with session.get('<https://catfact.ninja/fact>') as response:
data = await response.json()
print(data.get('fact'))
asyncio.run(main())
We will take a closer look at this code. In the main
function, we first use the ClientSession async context manager to create a session. Then we use another context manager session.get to perform an HTTP GET request. Then we await its json, and lastly, print the cat fact.
This is a much better way of getting cat facts (and other data) than our earlier implementation. I encourage you to read more about aiohttp
on its official site.
Conclusion
In this article, I went over the main fundamentals of using asynchronous programming with asyncio in Python. By now, you should be comfortable using async libraries and calls in your code, as well as writing your own async functions. Have you used asyncio in your projects? Let me know about your experiences in the comments!