← @LucasWerkmeistr Twitter archive

Lucas Werkmeister

@LucasWerkmeistr

systemctl list-units --state failed: list all failed units. (Add --type service to exclude other kinds of units.) #systemdTOTD

4/4/2017, 11:47:10 PM

Favs: 3

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

(Since list-units is the default command, you can abbreviate this to systemctl --state failed if you really want to. A bit too terse IMHO.)

4/4/2017, 11:47:51 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemctl start --no-block UNIT: start a unit and don’t wait for startup to complete. Mostly useful for long-running oneshots. #systemdTOTD

4/4/2017, 11:49:41 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

(Oneshot services are ones that run a command and then exit. Mostly useful for timers: man-db, updatedb, logrotate…)

4/4/2017, 11:50:44 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

Most systemd commands support globbing (wildcards) for units. E. g. see status of all tor instances: `systemctl status tor@*` #systemdTOTD

4/6/2017, 11:42:38 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

Or to see all log messages of a socket-activated Accept=yes service: journalctl -u git@* #systemdTOTD

4/6/2017, 11:43:57 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

Doesn’t just work for templates, of course. Check status of systemd’s own services: `systemctl status systemd-*.service` #systemdTOTD

4/7/2017, 12:25:18 AM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

Forget about failed instances of a service: systemctl reset-failed git@* #systemdTOTD

4/9/2017, 12:10:13 AM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

Send a reminder notification in two hours: systemd-run --user --on-active 2h notify-send "empty the dryer" #systemdTOTD

4/13/2017, 11:00:14 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

(there are a million ways to do this, of course, this is just one)

4/13/2017, 11:00:25 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemd-socket-proxyd(8) allows you to socket activate services that don’t support inheriting sockets from their environment. #systemdTOTD

4/13/2017, 11:05:14 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

proxyd inherits socket from systemd and forwards everything to a specified address, where the other service listens by itself. See manpage.

4/13/2017, 11:06:21 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

That in itself isn’t terribly exciting, but it gets way better:

4/13/2017, 11:07:15 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemd-socket-proxyd(8) allows you to network isolate a svc that should only listen on one connection, no other communication. #systemdTOTD

4/13/2017, 11:09:10 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

See the manpage for an example. Service+proxyd have PrivateNetwork=yes, proxyd JoinsNamespaceOf= service.

4/13/2017, 11:10:13 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemd opens socket in real network namespace. proxyd inherits it and forwards it to a port in private namespace shared with service.

4/13/2017, 11:11:16 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

(If the other service can listen on a Unix domain socket instead of an Internet address, you don’t even need JoinsNamespaceOf=.)

4/13/2017, 11:12:31 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

ANNOUNCEMENT: systemdTOTD is no longer a daily series, but I’ll continue to occasionally post tips under the hashtag as I discover them.

4/17/2017, 8:02:28 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

If I tweet multiple tips in a day, imagine I’m still playing catch-up ;) but I don’t have enough ideas for one tip every day anymore.

4/17/2017, 8:03:21 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

But for a quick-fire round, here’s the last few ideas I still had saved up:

4/17/2017, 8:03:54 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemd-mount /dev/WHAT [/WHERE]: create a transient mount unit, just like systemd-run creates a transient service (+ opt. timer) unit.

4/17/2017, 8:10:51 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

So why use systemd-mount over mount? A few possible reasons:

4/17/2017, 8:11:07 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

- The mount unit is a systemd unit like any other unit: systemd can manage dependencies (parent mounts), apply common settings, etc.

4/17/2017, 8:12:05 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

- If you omit the WHERE path, systemd chooses a suitable folder (based on the device label) and creates it. Pure laziness, in other words :)

4/17/2017, 8:13:07 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

- With --discover (on by default if WHERE is omitted), systemd also reads some more metadata, e. g. for the unit description.

4/17/2017, 8:14:03 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

- But most importantly: if device is removable, systemd automatically creates an automount with short idle timeout instead of normal mount.

4/17/2017, 8:14:46 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

This means that the device is always unmounted when it’s not used for a while, making it safe to remove without ejecting it first.

4/17/2017, 8:15:49 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

(Automounts are also a systemd unit type, so if you think this sounds cool, you can also do this for other mounts you have :) )

4/17/2017, 8:19:59 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemd-mount --list: list mountable devices with path, model, type, label, UUID, and other info in one nice table. #systemdTOTD

4/17/2017, 8:21:41 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

I’m definitely going to use this instead of blkid in the future when I need UUIDs for /etc/fstab – so much easier to use and remember :D

4/17/2017, 8:22:15 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

SystemCallFilter=: add a blacklist or whitelist of syscalls to a service. See systemd.exec(5) for details. #systemdTOTD

4/17/2017, 8:24:40 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

Blocked system calls can either kill the service immediately (uncatchable SIGSYS) or return a specified error number.

4/17/2017, 8:26:23 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

And since specifying individual syscalls is tedious, systemd also defines some sets you can use, e. g.:

SystemCallFilter=~@​mount @​raw-io

4/17/2017, 8:27:51 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

The tilde indicates that the filter is a blacklist, not a whitelist, and the at means that the name is a system call set.

4/17/2017, 8:29:17 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

Most services systemd ships use a blacklist of sets (see for example systemd-{journald,logind,importd}).

4/17/2017, 8:30:30 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

Specifying a whitelist of individual syscalls would be more restrictive, but more likely to break if systemd’s dependencies change behavior.

4/17/2017, 8:31:30 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

journalctl --since 06:30 --until 07:00 # what happened during today’s unattended upgrade? #systemdTOTD

6/23/2017, 8:40:17 PM

Favs: 4

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemd-analyze set-log-level debug && systemd-analyze set-log-target console # change log behavior of systemd daemon itself #systemdTOTD

9/7/2017, 11:08:34 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemd-analyze syscall-filter @​basic-io @​file-system # print syscalls in a syscall group, useful to debug a SystemCallFilter #systemdTOTD

9/8/2017, 12:01:26 AM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemctl --failed # shorter alias for systemctl [list-units] --state=failed (list-units is the default command) #systemdTOTD

10/22/2017, 11:30:11 AM

Favs: 2

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemctl set-property user.slice MemorySwapMax=0 # keep user sessions in RAM (requires cgroups v2) #systemdTOTD

11/4/2017, 2:50:50 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

echo 1 | sudo tee /sys/fs/cgroup/memory/user.slice/memory.swappiness # similar effect in cgroups v1, in conjunction with…

11/4/2017, 2:50:59 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

echo 100 | sudo tee /sys/fs/cgroup/memory/system.slice/memory.swappiness # …raise swappiness of all system services, or alternatively…

11/4/2017, 2:51:09 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

echo 100 | sudo tee /sys/fs/cgroup/memory/system.slice/rdf2hdt.service/memory.swappiness # …raise swappiness of a single service.

11/4/2017, 2:51:18 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

alias iotop='sudo systemd-run -qt -p DynamicUser=yes -p AmbientCapabilities=CAP_NET_ADMIN iotop' # restrict iotop capabilities #systemdTOTD

11/5/2017, 12:26:25 AM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

.link files: configure physical network devices. Processed by udev.
.netdev files: configure virtual network devices. Created by systemd-networkd.
.network files: configure networks applying to devices. Processed by systemd-networkd. Apply to links, refer to netdevs. #systemdTOTD

11/11/2017, 10:49:57 AM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

.link: set MTU, enable Wake-on-LAN
.netdev: create bridge
.network: enable DHCP, connect to bridge

11/11/2017, 10:50:10 AM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

# list units by IP traffic #systemdTOTD
systemctl --no-legend |
while read -r unit _; do
bytes=$(systemctl show -p IPEgressBytes --value -- "$unit")
[[ $bytes == 18446744073709551615 ]] && bytes=-1
((bytes > 0)) && printf '% 15d %s\n' "$bytes" "$unit"
done | sort -rn

7/27/2018, 11:46:22 AM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

Requires DefaultIPAccounting=yes in systemd-system.conf(5) or IPAccounting=yes on individual units.

7/27/2018, 11:47:33 AM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

(The screenshot shows a system with systemd v237; on v238+, you will also get aggregated statistics for slice units.)

7/27/2018, 11:47:43 AM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

TIL: systemd reads unit files not just from /usr/lib/systemd (and /run, /etc, and some others), but also from /usr/local/lib/systemd, compatible with the standard prefix in a GNU-style build. #systemdTOTD

7/27/2018, 2:15:44 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemd debugging tip: temporarily increase the log level. #systemdTOTD
level=$(systemd-analyze log-level)
sudo systemd-analyze log-level debug
sudo systemctl start whatever.service
sudo systemd-analyze log-level "$level"
journalctl -b -e

8/22/2018, 12:03:53 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

if you want, you can get a bit fancier with the journalctl arguments, e. g.
journalctl -b _SYSTEMD_UNIT=whatever.service + _SYSTEMD_UNIT=init.scope UNIT=whatever.service
or even
journalctl -b CODE_FILE=../systemd-stable/src/basic/mount-util.c
(exact path will depend on distro)

8/22/2018, 12:07:01 PM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemd debugging tip: you can simply strace PID1. #systemdTOTD

sudo strace -p1 -f -yy # terminal 1
sudo systemctl start whatever.service # terminal 2

8/24/2018, 11:45:33 AM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemd is single-threaded and usually spends most of its idle time in a blocking epoll_wait() syscall, so there’s typically not much unrelated noise in the strace output.

8/24/2018, 11:46:26 AM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

if you want to automate this without using two terminals:

sudo sh -c 'strace -p1 -f -yy -o/tmp/trace & systemd-run …; kill %1'

(this time using systemd-run instead of systemctl start – both might be useful depending on situation)

8/24/2018, 11:48:52 AM

Favs: 0

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

systemd-inhibit --what=sleep:idle:handle-lid-switch --who="$USER" --why=Music --mode=block vlc *.mp3 # use your laptop as a portable music player without it going to sleep when you shut the lid #systemdTOTD

10/18/2018, 12:38:18 PM

Favs: 1

Retweets: 0

Lucas Werkmeister

@LucasWerkmeistr

can also be useful to charge another device via USB while you’re not using the laptop (use `sleep infinity` for the command in that case)

10/18/2018, 12:38:51 PM

Favs: 1

Retweets: 0