Handle graceful shutdown and signals¶

The controller drives a bounded, ordered shutdown: it cancels the context every service shares, runs each WithStop in reverse registration order, and abandons a stuck stop, or a supervisor whose WithStart never returns, at a deadline so the shutdown sequence itself can never hang. This guide covers the timeout, learning whether a stop released everything, signal handling, bounding your wait with WaitContext, and how a service distinguishes a controlled stop from an upstream cancellation.

The shutdown sequence¶

Whether triggered by a signal, a direct Stop(), or parent-context cancellation, shutdown always runs the same sequence:

1. Transition the controller to Stopping, from Running or from UnableToStart. Readiness is already false in both cases from this point on, so traffic stops being routed here before any service is stopped.
2. Detach OS-signal handling.
3. Cancel the controller context with the cause ErrShutdown, unblocking every WithStart that waits on ctx.Done().
4. Run each WithStop in reverse registration order, bounded by the shutdown timeout.
5. Await supervisor exit with the remaining shutdown budget, logging a WARN naming any service whose WithStart failed to return (it is abandoned).
6. Transition to Stopped and release Wait().

Set the shutdown timeout¶

The timeout bounds the whole stop phase. It defaults to DefaultShutdownTimeout (5s); override it with WithShutdownTimeout:

c := controls.NewController(ctx,
    controls.WithShutdownTimeout(30*time.Second),
)

Each WithStop receives a context carrying this deadline. A well-behaved stop respects it. http.Server.Shutdown(ctx), for example, drains in-flight requests until the deadline, then returns.

Zero is not "use the default". WithShutdownTimeout(0), or any negative duration, creates a budget that has already expired, so every stop callback is abandoned the instant it is launched and may never run at all. Omit the option if you want the 5s default.

A StopFunc can run more than once, so make it idempotent. Shutdown calls it once; a health-threshold restart calls it again before each restart of that service. An error-triggered restart does not call it; see Configure restart policy.

The deadline context is fresh, not the cancelled controller context. The ctx passed to WithStop is derived from context.Background() with the shutdown timeout, not from the already-cancelled controller context. That is deliberate: a context that was dead on arrival would make http.Server.Shutdown fail instantly instead of draining.

Abandoning a stop that ignores its context¶

Each WithStop runs in its own goroutine and is awaited against the deadline. If a stop function ignores its context and runs long, the controller abandons it when the deadline elapses and moves on to the next service:

controls.WithStop(func(ctx context.Context) {
    // BAD: ignores ctx, blocks for 10s regardless of the shutdown timeout.
    time.Sleep(10 * time.Second)
})

With a 5s timeout, the controller waits 5s, then abandons this goroutine (it is left to finish on its own) and continues shutting down the remaining services. This guarantees Wait() returns within roughly the shutdown timeout even when a stop misbehaves, but the abandoned work is not cleanly completed. Always honour the context:

controls.WithStop(func(ctx context.Context) {
    select {
    case <-workDone:
    case <-ctx.Done(): // give up at the deadline
    }
})

Know whether a stop released everything¶

A WithStop callback returns nothing, so after an abandoned stop the controller knows only that the deadline passed. WithStopErr registers a stop that reports how it ended, and the controller records the answer on ServiceInfo.StopErr:

c.Register("api",
    controls.WithStart(serve),
    controls.WithStopErr(func(ctx context.Context) error {
        // nil means every resource is released; anything else means the
        // service may still be holding something.
        return srv.Shutdown(ctx)
    }),
)

// later, after c.Wait()
info, _ := c.GetServiceInfo("api")
if info.StopErr != nil {
    log.Printf("api did not release cleanly: %v", info.StopErr)
}

The result is recorded, not acted upon: no restart is refused and no policy is altered on the strength of it. WithStop is unchanged and equivalent to a WithStopErr that always returns nil, so nothing that already works needs to move. Set one or the other; with both, only the error-reporting one runs.

A panic inside either kind of stop is contained and lands on StopErr too, on both paths that call it: shutdown and a health-threshold restart. Before this existed, a stop that ignored its context was abandoned at the deadline and nothing recorded that it had happened.

Bound your wait against a stuck start¶

The bare Wait() is unbounded: it returns only once every supervisor goroutine has unwound, which requires every WithStart to return after cancellation. If a start wraps third-party code that may ignore its context, use WaitContext so your process cannot hang on a controller that already reports Stopped:

c.Start()

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

if err := c.WaitContext(ctx); err != nil {
    // The wait was abandoned: some StartFunc never returned. The shutdown
    // sequence itself completed; the stuck service was named in a WARN log.
}

On the abandon path the stuck supervisor goroutine is deliberately leaked, the same trade as an abandoned stop. See D10 in Concurrency and shutdown correctness.

Decide an exit code from the outcome¶

Done() closes when the shutdown sequence has finished, within the shutdown timeout of whatever triggered it, and needs no deadline of your own. Then Outcome() says what triggered it and whether every step finished:

c.Start()
<-c.Done()

o, _ := c.Outcome()

switch {
case !o.Complete():
    for _, u := range o.Unfinished {
        log.Printf("%s: %s %v", u.Service, u.Reason, u.Err)
    }

    os.Exit(1)
case o.Cause == controls.CauseSignal:
    os.Exit(128 + int(o.Signal.(syscall.Signal)))
case o.Cause == controls.CauseGaveUp:
    log.Printf("%s gave up: %v", o.Service, o.Err)
    os.Exit(1)
}

CauseGaveUp and CauseCompleted come only from the opt-ins in Stop when a service gives up; Outcome.Service names the service. Decide on Cause before reading Err, which under CauseGaveUp can be context.Canceled.

Unfinished lists each step in the order shutdown reached it: a stop that was abandoned at the deadline (StopAbandoned), one whose turn came after the budget had gone (StopUnbudgeted), one that returned an error or panicked (StopFailed), and a supervisor that had not exited (SupervisorAbandoned). A hung service usually appears twice, once for its stop and once for its supervisor. A stop that returns because the deadline fired may be reported either as StopFailed or as StopAbandoned; Complete() is false either way.

Outcome() returns ok == false until the controller's own shutdown sequence has written it, so read it after Done().

Signal handling¶

The controller installs no signal handler by default. Signal disposition is process-global state, so it belongs to whichever layer is outermost: usually your main, or the CLI framework wrapping it. A library that registers its own handler silently becomes a second owner of that global, and signal.Notify is additive: every registered channel receives a copy, so two handlers means two shutdowns racing.

So the controller observes rather than intercepts. Cancel the context you passed to NewController and the services shut down gracefully; see Stopping from a parent context.

Opting in from a standalone main¶

When the controller genuinely is the outermost layer, a daemon with no CLI framework above it, ask for signals explicitly:

c := controls.NewController(ctx, controls.WithSignals())

The first SIGINT/SIGTERM initiates a graceful Stop. A second signal exits the signal handler immediately, so a caller can escalate (for example to os.Exit) if a shutdown is wedged.

Tests need no option¶

Because signals are off by default, a test just constructs a controller and drives shutdown explicitly:

c := controls.NewController(ctx)
c.Start()
// ... exercise the services ...
c.Stop()
c.Wait()

Stopping from a parent context¶

Cancelling the context you passed to NewController triggers the same graceful sequence as Stop(). So does its deadline expiring:

ctx, cancel := context.WithCancel(context.Background())
c := controls.NewController(ctx)
c.Start()

cancel()  // -> ordered, bounded shutdown, exactly as Stop() would
c.Wait()

The controller does not merely inherit that cancellation. It reacts to it, running the full shutdown sequence. That is what makes ErrShutdown reliable below, and it means an expired parent deadline gives your services an orderly teardown bounded by WithShutdownTimeout, rather than a context that is already dead when WithStop receives it.

Recognise a controlled stop¶

When the controller shuts a service down it cancels the context with a specific cause, controls.ErrShutdown, so a service can recognise an orderly teardown and return an expected end-of-run rather than an error:

controls.WithStart(func(ctx context.Context) error {
    <-ctx.Done()

    if errors.Is(context.Cause(ctx), controls.ErrShutdown) {
        // Orderly shutdown: an expected end-of-run, not a failure.
        return ctx.Err()
    }

    return context.Cause(ctx)
})

ErrShutdown is the cause of every stop the controller drives: a direct Stop(), a parent cancellation, an expired parent deadline, or a signal when you opted into WithSignals. One rule, no exceptions.

Related¶

• Concurrency and shutdown correctness: what the shutdown bound covers (and D10, the stuck-start abandon policy), and why no goroutine leaks or busy-spins.
• Architecture: the state machine and control goroutines behind the sequence above.
• Controller reference: Stop, Wait, WaitContext and every option, including what each does when called in the wrong state.