Services in Systemd: In-Depth Tutorial

systemd is the glue that holds Linux systems together. systemd is a collection of building blocks, which handle services, processes, logging, network connectivity and even authentication. In this article, I will show you how to work with services in systemd

What exactly is systemd?

systemd is a suite of computer programs aimed to manage and interconnect different parts of system together. It is designed as a replacement for SystemV and LSB-style startup scrips, which were prevalent since 1980s. systemd consists of these components:

  • systemd, which is the system and service manager
  • systemctl, a command line tool to interact with systemd
  • journald, a unified logging framework
  • logind, a daemon that handles user logins and seats
  • resolved, timesyncd, and networkd, which are responsible for network connection, domain name resolution and synchronizing time with internet resources

This is a very high-level overview of systemd‘s architecture, but will suit us for now.

Managing existing services

A service is essentially a process, running in the background and managed/provisioned by systemd. Stuff you would like to run as services includes various servers (HTTP, SSH, FTP), synchronization utilities (Syncthing, rsync), virtualisation hypervisors (Docker, K8s), and many more. Let’s start by listing all available services on your system by running this command:

$ systemctl

You will be presented with a table that looks like this:

By default, this command will list devices, mount points, namespaces, and services, which we are interested it. Scroll down to see the service section (you will notice everything having a .service extension):

Here is the structure of this table:

  • Name of the service. (Matches the name of the .service config file)
  • Current status of the service. loaded means systemd knows about it, active means systemd ran it successfully
  • Current status of the process. It indicates if the process is running or exited. Note that an active service can be exited (for example, 1 action has to happen at boot, and the process than returns. The service is still considered active.
  • Description

If you want to see some more details of a specific service, use this command:

$ systemctl status SERVICE_NAME

For example, running systemctl status dbus on my system will produce the following:

This command will tell you more info than the previous some, for example the uptime, the path to .service file, the process name and its PID. While this is helpful, status is usually used to troubleshoot a service. For example, I have a problem with a service syncthing-discosrv. Running the same command on it will tell me this:

You can notice the logs, conveniently printed at the end. This tells me precisely what when wrong when systemd tried to start this service and I can now fix that easily.

Lastly, the most interesting commands. To start/stop services, use this:

$ systemctl start/stop SERVICE_NAME

Similarly, you can enable or disable the services. Enabled means it will run on boot:

$ systemctl enable/disable SERVICE_NAME

Now, to the interesting part.

Creating your own services

systemd won’t be very useful if it did not let you create your own services. Thankfully, this is quite easy to do. Let’s start with the most basic example: running a bash script on boot. Suppose you have a script, located at /usr/bin/runme.sh that does some sort of cleanup. Now, to run it on boot, you need to create a unit file. Create it in /etc/systemd/system/runme.service:

[Unit]
Description=Cleaning service

[Service]
Type=simple
ExecStart=/bin/bash /usr/bin/runme.sh

[Install]
WantedBy=multi-user.target

This should be pretty straightforward. The Description sets the description (as seen earlier in the article). Type sets the type of our service, which can also be forking, if service spawns multiple processes. ExecStart is the actual command used to run the service. Lastly, the WantedBy specifies when this service should run. multi-user means regulal system startup and what you want most of the time. Note that if you start in single-user mode (recovery), this service would not run (which may or may not be what you want). Now, to enable this service:

$ systemctl enable runme
$ systemctl status runme
// .... ENABLED .....

If you want, you can also run it once right now:

$ systemctl start runme

Now, let’s consider a slightly more sophisticated example. Suppose you have a Python script (located at /etc/scripts/server.py) that must be run at boot, but after network is up. Also, you want it to restart on crash, but not too often. If this sounds like your case, use something like this for your service file:

[Unit]
Description=My awesome service
After=network.target

[Service]
type=Simple
User=username
Restart=on-failure
RestartSec=1
StartLimitBurst=5
StartLimitIntervalSec=10
StartLimitAction=reboot
ExecStart=/bin/python /etc/scripts/server.py

[Install]
WantedBy=multi-user.target

Quite a few new options here. The After directive specifies this service’s dependency on network target, which initializes network connectivity. Later, we specify the User to run the script as username. Now, there are 5(!) options to control the resilience of our service. Restart specifies when the service is to be restarted. RestartSec will not let systemd restart more than once a second. StartLimitBurst and StartLimitIntervalSec say that service can be restarted at most 5 times in a 10 second interval. Lastly, StartLimitAction asks to reboot the system if the service still fails after all those restats. Obviously, this is not advised if there are many vital services on your machine. Once set up, you can enable and start this service just like the rest of them.

Closing notes

Thank you for reading, I hope you liked this article. Please let me know your opinion about systemd, I hear there is quite a lot of controversy there.

Resources

Get new content delivered to your mailbox:

2 comments

  1. This tutorial is very helpful for beginners, thank you.

  2. wolfgangw | october 9, 2022

    good article, thanks!

    some typos:
    “the process than returns.“ -> “the process then returns.”
    “what when wrong“ -> “what went wrong”

leave a comment