TL;DR
First step after editing any unit file: systemctl daemon-reload, or systemd keeps using the old config. systemctl restart = stop+start; systemctl reload sends SIGHUP without interrupting the process (service must support it). In scripts use systemctl is-active svc (exit code 0=active, 3=inactive, 4=unknown) — never parse status output text. Top pitfalls: ① ExecStart does not support pipes/redirection/$VAR — wrap shell syntax as /bin/sh -c '...'; ② % is an escape character in units — write literal %%; ③ journald's default size limit is 10% of disk or 4G (whichever is smaller), it rotates automatically, but journalctl --vacuum-size=200M frees space immediately.
Service Lifecycle Command Table
| Action | Command | Notes / exit codes |
|---|---|---|
| Start | systemctl start svc | No-op if already running; returns 0 |
| Stop | systemctl stop svc | SIGTERM first, SIGKILL after timeout (default 90s) |
| Restart | systemctl restart svc | = stop + start; use after config changes |
| Reload config | systemctl reload svc | Sends SIGHUP, no downtime; service must implement it |
| Enable on boot | systemctl enable --now svc | enable + start in one step |
| Disable on boot | systemctl disable svc | Removes symlink only; does not stop a running process |
| Mask | systemctl mask svc | Fully disabled; even start is refused (incl. dependency pulls) |
| Show status | systemctl status svc | Shows active/inactive/failed + recent logs |
| Script check | systemctl is-active svc | Exit: 0=active, 3=inactive, 4=unknown, non-zero=failed |
| Failed check | systemctl is-failed svc | 0=failed; handy for monitoring alerts |
Common Query Commands
| Need | Command | Notes | |
|---|---|---|---|
| List running services | systemctl list-units --type=service --state=running | Count with `\ | wc -l` |
| List enabled services | systemctl list-unit-files --state=enabled | Shows enabled/disabled/static | |
| Show dependency tree | systemctl list-dependencies svc | Includes .target nodes | |
| Show process tree | systemctl status svc (CGroup section) | Main PID + children | |
| Show full unit | systemctl cat svc | Merged result incl. overrides | |
| Override part of config | systemctl edit svc | Creates drop-in: /etc/systemd/system/<svc>.d/override.conf | |
| Rewrite whole unit | systemctl edit --full svc | Copies full unit to /etc then edits | |
| Reload systemd itself | systemctl daemon-reload | Required after editing/installing units | |
| Show default target | systemctl get-default | Usually multi-user.target or graphical.target | |
| Switch target | systemctl isolate multi-user.target | Old-style runlevel 3 |
Core Unit Directives (copy-paste for .service files)
| Section/Key | Purpose | Example |
|---|---|---|
[Unit] Description= | Human-readable name shown by systemctl status | Description=My Web API |
[Unit] After= | Ordering only (no dependency) | After=network-online.target |
[Unit] Wants= / Requires= | Weak/strong dependency; Requires failure fails this unit too | Wants=mysql.service |
[Service] Type= | simple (default; ExecStart stays in foreground) / forking (daemonizes; needs PIDFile=) / notify (process sends READY=1 via sd_notify) | Type=simple |
[Service] ExecStart= | Start command; no pipes/redirection/$VAR; use /bin/sh -c '...' for shell; escape % as %% | ExecStart=/usr/bin/python3 /opt/app/main.py |
[Service] Environment= / EnvironmentFile= | Env vars; file uses one KEY=VALUE per line; a missing file fails startup | EnvironmentFile=/etc/app.env |
[Service] User= / Group= | Drop privileges — always set for security | User=www-data |
[Service] Restart= | Restart policy: no (default) / on-failure / always / on-abnormal; pair with RestartSec=3 | Restart=on-failure |
[Service] WorkingDirectory= | Working dir; relative ExecStart paths resolve from here | WorkingDirectory=/opt/app |
[Service] ExecStartPre= / ExecStartPost= | Pre/post-start hooks (multiple allowed, run in order) | ExecStartPre=/bin/mkdir -p /run/app |
[Install] WantedBy= | Target to attach to when enabled | WantedBy=multi-user.target |
systemd vs SysVinit Comparison
| Scenario | SysVinit (legacy) | systemd (modern) |
|---|---|---|
| Start service | service nginx start | systemctl start nginx |
| Stop service | service nginx stop | systemctl stop nginx |
| Enable on boot | chkconfig nginx on (or /etc/rc.d scripts) | systemctl enable nginx |
| Check status | service nginx status | systemctl status nginx |
| Runlevel | runlevel / telinit 3 | systemctl get-default / isolate multi-user.target |
| Logs | App-written /var/log files + syslog | journalctl -u nginx centralized |
| Startup script | /etc/init.d/nginx (shell script) | /etc/systemd/system/nginx.service (declarative) |
Copy-Paste Full Unit (Python service)
# /etc/systemd/system/myapp.service
[Unit]
Description=My Python Web API
After=network-online.target mysql.service
Wants=mysql.service
[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/opt/myapp
EnvironmentFile=/etc/myapp.env
ExecStart=/usr/bin/python3 -m uvicorn main:app --host 127.0.0.1 --port 8000
Restart=on-failure
RestartSec=3
[Install]
WantedBy=multi-user.target
# Deploy a service (with permissions + reload)
sudo cp myapp.service /etc/systemd/system/myapp.service
sudo chmod 644 /etc/systemd/system/myapp.service
sudo systemctl daemon-reload
sudo systemctl enable --now myapp
# Verify
systemctl is-active myapp # prints active, exit code 0
systemctl status myapp --no-pager | head -n 15
journalctl -u myapp -n 50 --no-pager # last 50 log lines
# Check service state in a script (use exit codes, not status text)
import subprocess
r = subprocess.run(["systemctl", "is-active", "myapp"], capture_output=True, text=True)
print(r.returncode, r.stdout.strip()) # 0 active / 3 inactive / non-zero failed
# Type=notify with sdnotify (systemd considers the service up only after READY=1)
try:
import sdnotify
n = sdnotify.SystemdNotifier()
n.notify("READY=1")
except ImportError:
pass # still works when running outside systemd (local dev)
High-Frequency Pitfall Troubleshooting
| Symptom | Cause → Fix | ||
|---|---|---|---|
| Config change ignored after restart | Missing systemctl daemon-reload → reload, then restart | ||
| Service fails instantly with few log lines | Usually a missing EnvironmentFile= → verify path and permissions (file must be readable, 644 even for root) | ||
Failed to execute: No such file or directory | Wrong ExecStart path or script without exec bit → use absolute path, chmod +x, run the command manually first | ||
| `\ | or >` in config has no effect | ExecStart has no shell syntax → use `/bin/sh -c 'cmd \ | grep x'` |
% characters eaten in logs | % is an escape char in units → write literal %%, e.g. ExecStart=/bin/echo 100%% | ||
| Legacy daemonizing script "fails" on start/stop | Missing Type=forking + PIDFile= → declare Type=forking and PIDFile=/run/app.pid | ||
systemctl stop hangs for a long time | Process ignores SIGTERM; waits out the default 90s timeout before SIGKILL → graceful shutdown in the app, or shorten with TimeoutStopSec=10 | ||
| Service keeps restarting (Restart=always + crash on boot) | Infinite restart loop → use Restart=on-failure + StartLimitIntervalSec, check n-restarts in systemctl status | ||
| journalctl fills the disk | Default limit is 10% of disk or 4G (smaller) → journalctl --vacuum-size=200M immediately; long-term: set SystemMaxUse= in /etc/systemd/journald.conf |
FAQ
Why don't my systemd service config changes take effect?
Almost always because you skipped systemctl daemon-reload. systemd loads units into memory at startup; after editing a file you must run daemon-reload, then systemctl restart svc for the new config to apply.
Is "active (exited)" in systemctl status normal?
Yes. exited means the main process ran and finished as expected (one-shot tasks, Type=oneshot init scripts) — it is as healthy as active (running). Only failed is abnormal.
How do I make a service start on boot?
sudo systemctl enable svc (or enable --now to also start it immediately). The unit needs an [Install] section with WantedBy=multi-user.target or enable will error. Use disable to remove autostart, mask to fully block it.
journalctl logs are eating the disk — what do I do?
Run journalctl --vacuum-size=200M to free space immediately. To cap it long-term, set SystemMaxUse=200M in /etc/systemd/journald.conf and systemctl restart systemd-journald. The default limit is 10% of the disk or 4G, whichever is smaller.
My service keeps failing to start — fastest way to debug?
systemctl status svc for the error and recent logs → journalctl -u svc -n 50 --no-pager for the full picture → run the ExecStart command manually to prove the command itself works (mind environment variables and working-directory differences).
Sources
- systemd official docs (freedesktop.org/software/systemd), verified 2026-08-11
- man pages: systemctl / systemd.service / journalctl
- Arch Linux Wiki: systemd (wiki.archlinux.org/title/Systemd), verified 2026-08-11