Signal handling in long-lived daemons
A daemon that holds state in memory and exits on SIGTERM will eventually lose that state. The window is small until it is not — a write buffer flush, an fsync that takes longer than the systemd timeout, and the process is gone with a partially written file.
The pattern that survives: the signal handler sets a flag and nothing else. The
main loop notices the flag, stops accepting new work, drains in-flight requests
with a deadline, flushes, then exits. sd_notify tells the supervisor
when the drain is done so that ordering is explicit rather than guessed.
Record the exit path in the log at least once. The number of services that complete their drain path successfully under real load is much lower than the number that have one in the source tree.
← All notes