Backend / Linux & bash / 10_signals_traps.md

Signals and Traps

Updated 6 interview angles 7 min read source
On this page14
  1. The signals you need
  2. SIGKILL vs SIGTERM — the most asked
  3. SIGSTOP vs SIGTSTP
  4. sending signals
  5. What signals get sent automatically
  6. Trap — handling signals in bash
  7. The cleanup pattern
  8. Graceful shutdown in Python
  9. Docker / Kubernetes signal flow
  10. SIGPIPE — the silent killer
  11. Process groups and pgid
  12. Common pitfalls
  13. Common interview confusions
  14. Interview angle

Signals and Traps

Signals are how the kernel and processes tell each other “something happened.” For Python developers the relevant ones are: SIGTERM (please shut down), SIGKILL (force shutdown), SIGINT (Ctrl-C), SIGHUP (terminal closed / often “reload”), SIGUSR1/USR2 (app-defined). Knowing how to send and catch them is essential for graceful shutdown.

The signals you need

Name Number Meaning Catchable? Default action
SIGHUP 1 terminal hangup; convention: reload config yes terminate
SIGINT 2 interrupt (Ctrl-C) yes terminate
SIGQUIT 3 quit + dump core (Ctrl-) yes terminate + core
SIGKILL 9 force kill NO terminate
SIGTERM 15 polite termination yes terminate
SIGSTOP 19 pause NO stop
SIGTSTP 20 suspend (Ctrl-Z) yes stop
SIGCONT 18 continue (resume after stop) yes resume
SIGUSR1 10 user-defined #1 yes terminate
SIGUSR2 12 user-defined #2 yes terminate
SIGCHLD 17 child process state changed yes ignored
SIGPIPE 13 wrote to a closed pipe yes terminate

Numbers vary by architecture; names are portable. Use names (kill -TERM) not numbers.

SIGKILL vs SIGTERM — the most asked

SIGTERM: “please shut down.” The process can:

  • Catch the signal.
  • Run cleanup (close connections, flush buffers, persist state).
  • Exit when ready.

SIGKILL: “die now.” The kernel terminates the process immediately. The process cannot:

  • Catch it.
  • Clean up.
  • Do anything.

kill <pid> sends SIGTERM. kill -9 <pid> sends SIGKILL. Use TERM first, give the process a few seconds, then KILL only if it’s hung.

SIGSTOP vs SIGTSTP

Both pause a process. Difference:

  • SIGSTOP (signal 19): uncatchable. The kernel stops the process immediately. Resume with SIGCONT.
  • SIGTSTP (signal 20, sent by Ctrl-Z): catchable. Apps that want graceful suspend can handle it.

Useful: kill -STOP <pid> to freeze a runaway process so you can investigate, then kill -CONT <pid> to resume (or kill -KILL to terminate).

sending signals

bash
kill 1234                       # SIGTERM (default)
kill -TERM 1234                 # explicit
kill -15 1234                   # by number
kill -9 1234                    # SIGKILL
kill -HUP 1234                  # SIGHUP (often = reload)
kill -USR1 1234                 # send USR1
kill -l                         # list all signal names
killall gunicorn                # by name
pkill -HUP nginx                # by pattern

To kill all processes of a user (don’t run on yourself):

bash
pkill -u alice

What signals get sent automatically

Action Signal sent
Ctrl-C in terminal SIGINT to foreground process group
Ctrl-Z in terminal SIGTSTP to foreground process group
Ctrl-\ in terminal SIGQUIT to foreground process group
Closing terminal SIGHUP to processes attached to that TTY
Process writes to a closed pipe SIGPIPE to the writer
Child exits SIGCHLD to parent
OOM killer fires SIGKILL to chosen victim
systemctl stop SIGTERM, then SIGKILL after timeout

Trap — handling signals in bash

bash
#!/usr/bin/env bash
set -euo pipefail

cleanup() {
    echo "cleaning up..."
    rm -rf "$work_dir"
}

trap cleanup EXIT                       # always run on exit
trap 'echo "interrupted"; exit 130' INT  # Ctrl-C
trap 'echo "terminated"; cleanup; exit 143' TERM
Trap pseudo-signal When
EXIT script exits, any reason
ERR any command fails (with set -e)
DEBUG before every command (debugging)
RETURN when a function returns

trap - INT resets the INT handler to default.

The cleanup pattern

bash
#!/usr/bin/env bash
set -euo pipefail

work_dir=$(mktemp -d)
trap 'rm -rf "$work_dir"' EXIT

# ... do work in $work_dir ...
# Even on Ctrl-C, error, or normal exit, work_dir is cleaned up

This is the canonical “make a scratch directory, guarantee cleanup” pattern. Combine with mktemp for safety.

Graceful shutdown in Python

A real Python service should catch SIGTERM:

python
import signal
import sys

def graceful_shutdown(signum, frame):
    print("shutdown requested, flushing...")
    flush_buffers()
    close_connections()
    sys.exit(0)

signal.signal(signal.SIGTERM, graceful_shutdown)
signal.signal(signal.SIGINT, graceful_shutdown)

Most frameworks (Django, Flask, FastAPI via uvicorn/gunicorn) handle this for you. Check your worker class’s docs:

  • Gunicorn (sync): SIGTERM = graceful shutdown (finish in-flight requests, then exit). SIGINT = same. SIGQUIT = quick shutdown. SIGKILL = die now.
  • Uvicorn: similar; SIGTERM gives --timeout-graceful-shutdown to drain.
  • Celery: SIGTERM = warm shutdown; SIGQUIT = cold.

Docker / Kubernetes signal flow

When docker stop or kubectl delete pod:

  1. SIGTERM sent to PID 1 in the container.
  2. Wait for --timeout (default 10s in Docker, terminationGracePeriodSeconds in k8s, default 30s).
  3. SIGKILL if still running.

Two common bugs:

PID 1 problem

dockerfile
CMD ./start.sh

If start.sh is a shell script that execs the real binary, the shell gets the signal but doesn’t forward it. Two fixes:

  1. exec in the script: exec gunicorn ... (replaces shell with the binary as PID 1).
  2. Init system: tini or dumb-init as PID 1 forwards signals correctly.
dockerfile
ENTRYPOINT ["tini", "--"]
CMD ["python", "app.py"]

App ignores SIGTERM

If the app catches SIGTERM and does nothing (or doesn’t catch it), the 10/30-second graceful period is wasted, then SIGKILL kills it mid-request. Symptoms: dropped connections during deploys.

Always implement graceful shutdown in production services.

SIGPIPE — the silent killer

When you cmd1 | cmd2 and cmd2 exits early, cmd1 writing to the closed pipe gets SIGPIPE. Default action: terminate cmd1.

This is usually fine (yes | head -5 works because of SIGPIPE), but in scripts can produce surprising failures:

bash
generate_huge_output | head -5
# generate_huge_output gets SIGPIPE after head closes — exits 141 (128+13)

With set -e -o pipefail, this can fail your script. Workarounds:

bash
generate_huge_output | head -5 || true
# or
{ generate_huge_output; } | head -5

Process groups and pgid

Signals can be sent to a process group (all processes started together):

bash
# negative PID = send to process group 1234
kill -- -1234

Used by shells when you Ctrl-C — the signal goes to the foreground process group, including children.

Common pitfalls

  • Using kill -9 first — skips cleanup. Try TERM, wait 5–10s, then KILL.
  • Trapping EXIT but using exit inside the trap — fine; the trap doesn’t re-run on exit triggered by itself.
  • Catching SIGKILL or SIGSTOP — impossible. The kernel handles them directly.
  • Container app not handling SIGTERMdocker stop always feels like 10 seconds, then nukes you. Implement graceful shutdown.
  • Shell script as PID 1 without exec — signals get swallowed. Use exec or tini.

Common interview confusions

  • “SIGTERM and SIGKILL are the same.” — TERM is polite; the process can clean up. KILL is forced; the process can’t react.
  • “You can catch SIGKILL with the right code.” — no. Uncatchable, by kernel design.
  • kill deletes a process.” — sends a signal. Default signal is SIGTERM. The process exits if it doesn’t have a handler.
  • “Ctrl-C kills the program.” — sends SIGINT. Default is to terminate, but the program can catch and ignore (some interactive programs do).

Interview angle 6

  • “What’s the difference between SIGTERM and SIGKILL?” — TERM is catchable (process can clean up); KILL is uncatchable, immediate. Always TERM first; KILL only if it doesn’t respond.
  • “Why can’t you catch SIGKILL?” — kernel handles it directly; no signal handler runs. Same with SIGSTOP. By design — there has to be a guaranteed way to terminate any process.
  • “How does docker stop work?” — SIGTERM to PID 1; wait 10 seconds (default); SIGKILL. App needs to handle SIGTERM gracefully.
  • “Why do you need tini in a Docker image?” — PID 1 has special signal-handling rules. A typical Python interpreter or shell script as PID 1 may not properly reap zombies or forward signals. tini is a tiny init that does both.
  • “What’s the trap pattern for cleaning up a temp directory?”work=$(mktemp -d); trap 'rm -rf "$work"' EXIT. Runs cleanup on any exit (success, error, signal).
  • “What’s SIGHUP traditionally used for?” — terminal hangup (close). By convention many daemons treat it as “reload config” (nginx, postfix). Not enforced — the daemon has to catch and interpret it.