Skip to content

Controller reference

Controller is the supervisor: it owns the registered services, the control goroutines, the lifecycle state and the shutdown sequence. This page lists its constructor, its options and every method, with the behaviour of each when it is called at the wrong time.

NewController: what it sets up

func NewController(ctx context.Context, opts ...ControllerOpt) *Controller

NewController allocates the controller and applies the options. It starts no goroutine, installs no OS signal handler and touches no process-global state, so constructing a controller you never start is harmless.

The ctx you pass is watched, not inherited. The controller derives its own context with context.WithCancelCause(context.WithoutCancel(ctx)) and watches ctx.Done() separately: completing your context, by cancel or by deadline, triggers the ordinary shutdown sequence, so every service still observes ErrShutdown as its context cause. Values on your context are preserved and visible to services. See Cause determinism.

State immediately after NewController, before any option is applied:

Field Value
Lifecycle state NeverStarted
Logger slog.New(slog.DiscardHandler), so nothing is logged
Shutdown timeout DefaultShutdownTimeout (5s)
Message channel unbuffered chan Message
Error channel unbuffered chan error
Signal channel nil: no OS signal handling
Valid-error predicate none: every non-nil error from a StartFunc is a failure
Registered services / health checks none

Controller options

Each option is a ControllerOpt passed to NewController. There is no way to apply one after construction that is safe once the controller is running; see Setters.

Option Signature Default when omitted What it does, and what a bad value does
WithLogger WithLogger(l *slog.Logger) discard handler Stores l.With("component", "controller"). Passing nil panics with a nil pointer dereference inside NewController. Omit the option instead if you want no logging.
WithShutdownTimeout WithShutdownTimeout(d time.Duration) DefaultShutdownTimeout (5s) Bounds the whole stop phase, not each callback. A zero or negative d is not "use the default": it produces an already-expired budget, so stop callbacks are abandoned the instant they are launched and may never run at all.
WithSignals WithSignals() no signal handling Creates a signal channel with a one-slot buffer. signal.Notify for SIGINT/SIGTERM is registered in Start, not at construction, and detached again at shutdown. Do not use it under a CLI framework that already turns signals into cancellation.
WithOnEvent WithOnEvent(fn func(ServiceEvent)) none Delivers every terminal service failure, and the latest of each run of retries, to fn on a goroutine of its own, in order. A retry waiting behind a slow fn is replaced by a newer one. fn may call Stop. A panic in it is recovered and logged. See Events.
WithStopOnGiveUp WithStopOnGiveUp() nothing a service does stops the controller Stops it with CauseGaveUp when a registered service fails for good: an exhausted policy, a failed start with no policy, or a death after a clean start, which it watches for in a service with a Status and no policy. See Stop when a service gives up.
WithDeathWatch WithDeathWatch(interval time.Duration, threshold int) 10s, 3 The death watch's poll interval and the consecutive failures that end a service. A zero or negative value keeps its default. Does nothing without WithStopOnGiveUp.
WithCompletion WithCompletion(p CompletionPolicy) CompleteOnFirst Which WithRunToCompletion service's completion ends the controller: the first, or under CompleteOnAll the last outstanding. An unknown value acts as CompleteOnFirst, with a warning at Start.
WithOnPanic WithOnPanic(fn func(PanicReport)) none Delivers every panic the controller recovers to fn on a goroutine of its own, from Start until shutdown's cutoff. It is beside the failure the panic causes, never instead of it. See Panics.
WithProbePanicsAsFailures WithProbePanicsAsFailures() a polled Status panic breaches at once Counts a panic in a WithStatus probe polled by a restart policy as one failure towards HealthFailureThreshold. Nothing else changes.
WithValidError WithValidError(fn func(error) bool) none A StartFunc error for which fn returns true is classified as a normal end of run: not restarted, not counted, not forwarded on the error channel. Applies to every registered service; there is no per-service variant. (A Supervisor's Child has its own ValidError field.)

Lifecycle methods

Method Blocks? Effect Called at the wrong time
Start() no NeverStarted → Running, then launches one supervisor goroutine per registered service, the async health-check goroutines, the event forwarders it needs, and the control goroutines (message processor, parent watch, and the signal handler if WithSignals was passed). Any state other than NeverStarted: logs WARN "Start called, but controller has already started; ignoring" and returns. A controller is single-use; after Stopped it cannot be started again.
Stop() no Running → Stopping, or UnableToStart → Stopping, recording CauseStop and the shutdown deadline, then sends Stop on the message channel. Returns before the shutdown sequence has finished. Safe to call from a WithOnEvent callback. Any other state: logs WARN "Stop called, but not in expected state, unable to continue" and returns. Before any Start that means it is a no-op and the controller stays startable. Safe to call concurrently or repeatedly.
Wait() yes, unbounded Blocks until the shutdown sequence has completed, every async health check has exited, and every service has either started cleanly or had its supervisor goroutine exit. The sequence includes delivering queued events, so a slow consumer delays it by up to the shutdown budget. Before Start, the wait group is empty and it returns immediately. It never returns while a service whose Start failed has a retry that ignores cancellation; use Done or WaitContext for that case.
WaitContext(ctx) yes, bounded Returns nil when what Wait waits for is done, or ctx.Err() when ctx completes first. On the abandoned path the stuck supervisor goroutines and one internal helper goroutine are deliberately leaked.
Done() n/a Returns a channel closed when the shutdown sequence has finished, whatever triggered it, within the shutdown timeout of the trigger. The same channel every call. Before Start it stays open: nothing has stopped.
Outcome() no Returns (Outcome, true) once the controller's own shutdown sequence has written it, which happens as it reaches Stopped. A copy each call. (Outcome{}, false) before that, and after a SetState(Stopped) from outside, which runs no shutdown.

Wait and WaitContext are methods on the concrete *Controller only; they are on no interface, including Controllable.

Registration methods

Method Returns Rules
Register(id string, opts ...ServiceOption) nothing Must be called before Start. Names are not checked for uniqueness. Called after Start: logs WARN "Register called after Start; service will not be supervised", and the service is added to the collection but never started, stopped or supervised, while still appearing in every health report. See Services.
RegisterHealthCheck(check HealthCheck) error error Must be called before Start. Returns cannot register health check after start in any other state, and duplicate health check name: "<name>" when a check of that name is already registered. Uniqueness is checked only against other health checks, never against service names.

Health and introspection methods

Method Returns Notes
Status() HealthReport Every service (via its WithStatus probe) and every health check regardless of CheckType. Not a gate: an async check that has not run yet counts as OK.
Liveness() HealthReport Every service (WithLiveness, falling back to WithStatus) plus checks of type CheckTypeLiveness or CheckTypeBoth.
Readiness() HealthReport Every service (WithReadiness, falling back to WithStatus) plus checks of type CheckTypeReadiness or CheckTypeBoth. Ready only while Running: in every other state OverallHealthy is false regardless of what the probes say. Also fails closed on an async check with no result yet, reported ERROR.
GetServiceInfo(name string) (ServiceInfo, bool) false when no service of that name was registered. When two services share a name, only the most recently registered one's info is stored.
GetCheckResult(name string) (CheckResult, bool) false when the name is unknown or the check has not yet produced a result, which for a synchronous check means until a report including it has been built.
GetState() State One of NeverStarted, Running, UnableToStart, Stopping, Stopped, or Unknown for a Controller built without NewController.
IsRunning(), IsStopping(), IsStopped() bool Equality tests against Running, Stopping, Stopped. There is no predicate for NeverStarted, UnableToStart or Unknown; compare GetState() directly.
GetContext() context.Context The controller's own context, the one services receive. It is cancelled with cause ErrShutdown during shutdown.
GetLogger() *slog.Logger The configured logger, or the discard logger.
WaitGroup() *sync.WaitGroup The wait group Wait blocks on. On no interface.

All three reports are safe to call concurrently while the controller runs, and stay responsive during shutdown, because the stop sequence does not hold the services mutex.

Channels

Method Channel Default
Messages() chan Message unbuffered
Errors() chan error unbuffered
Signals() chan os.Signal nil unless WithSignals was passed; buffered with one slot when it was

Stop is the only Message the controller defines and the only one the message processor acts on. Sending controls.Stop directly on Messages() drives the same shutdown sequence as calling Stop().

The controller never reads the error channel. It logs each failure at ERROR where it happens, with service_name and kind, and delivers the error to the channel through a forwarder that is the channel's only sender, so a slow reader never blocks a service.

  • Calling Errors() subscribes. Failures before your first call are not delivered. Call it before Start, or use WithOnEvent, if you must see a service fail at boot.
  • Read the controller's own channel until it closes, which it does before Done closes. A reader that stops early holds Done, and Wait, back by up to the shutdown budget whenever errors are queued for it.
  • A channel you install with SetErrorsChannel before Start is subscribed from Start. It is yours, so the controller never closes it, and sends nothing on it after Done. Read it until Done, then take whatever is still buffered.
  • Never send on Errors(). Nothing receives, so the send blocks until the channel closes, and then panics.

The channel carries the same failures as WithOnEvent, as bare errors.

Service events

type ServiceEvent struct {
    Service  string
    Kind     EventKind
    Err      error
    Restarts int
}
Kind When Err
EventRetrying A run failed and the restart policy will run it again, including after a health-threshold breach. The run's error, or health check failed: … for a breach.
EventFailed Terminal. The service started cleanly at least once, then failed and exhausted its policy, or had none and died under the death watch. Satisfies errors.Is(err, ErrRestartsExhausted) and the last error beside it: after health breaches, max restarts exceeded: health check failed: ….
EventUnableToStart Terminal. The service never started cleanly and will not be tried again, including a service with no restart policy failing once. As above, or the StartFunc's error for a service with no policy.

Restarts is the consecutive-restart count, which resets after a healthy run; for a terminal event it is the count the service gave up at. A failure noticed after shutdown has cancelled the services' context is not reported. Events queued when shutdown begins are delivered within the remaining budget, and the number left undelivered at the deadline is logged.

A WithOnEvent callback can run after the controller reaches Stopped, and one still running at the deadline can outlive Done, because a callback cannot be interrupted. A callback that calls Wait or blocks on Done holds up its own drain until the budget expires.

A failure caused by a panic has an Err satisfying errors.Is(err, ErrPanicked), on the event, the error channel and ServiceInfo.Error alike, and its log line carries a stack attribute naming the line that panicked.

Panics

Every panic the controller recovers, in a start, stop or probe, or in a standalone health check, is an error wrapping ErrPanicked, with the panic site's stack as its errors.OriginStack. WithOnPanic also hands each one to a callback, for an error tracker:

type PanicReport struct {
    Service string    // the service or health check that panicked
    Site    PanicSite // SiteStart, SiteStop, SiteProbe or SiteHealthCheck
    Value   any       // the recovered value
    Err     error     // wraps ErrPanicked
    Repeats int       // panics merged into this report while it waited
}
  • It runs from Start until shutdown's cutoff. A panic before Start, or in a probe called during the drain or after Done, is still recovered and still reported unhealthy, but is not delivered.
  • Repeats merge. While a report waits behind a busy callback, a newer panic from the same service, or health check, at the same site replaces its Value and Err and adds one to Repeats. A tracker sees the latest panic at each site with a count of how many it stands for.
  • It is drained like the event callback, within the remaining budget after the error channel and WithOnEvent, and what is left at the deadline is logged as undelivered with consumer=WithOnPanic.
  • A callback may call Stop. A panic in it is recovered and logged, never reported.
  • Nothing orders a report against the event or log line for the same failure.

The shutdown outcome

type Outcome struct {
    Cause      StopCause
    Err        error     // context.Cause of the parent for CauseParent; the failure for CauseGaveUp
    Signal     os.Signal // for CauseSignal
    Service    string    // for CauseGaveUp and CauseCompleted
    Unfinished []Unfinished
}

type Unfinished struct {
    Service string
    Reason  UnfinishedReason
    Err     error // the stop's error, for StopFailed
}
StopCause Trigger
CauseStop Stop(), or Stop sent on Messages()
CauseSignal the first signal on a channel WithSignals owns
CauseParent the parent context's cancellation or deadline
CauseGaveUp a service that failed for good, under WithStopOnGiveUp, or a WithRunToCompletion service. Service names it, and Err is its terminal event's error, which can be context.Canceled, so decide on Cause first.
CauseCompleted a WithRunToCompletion service's work finished, the last one outstanding under CompleteOnAll. Service names it, and Err is nil.

Exactly one cause is recorded however many triggers race, and the shutdown deadline is recorded with it, so the budget runs from the trigger. A service giving up after another trigger changes nothing, silently, though its event is still published.

UnfinishedReason Meaning
StopAbandoned The stop had budget left when called and had not returned when it expired. StopErr holds ErrStopAbandoned, and a later return does not overwrite it.
StopUnbudgeted The budget had already expired when this stop's turn came. It is still called, and not awaited. StopErr holds ErrStopAbandoned.
StopFailed The stop returned an error, or panicked, inside the budget.
SupervisorAbandoned The service's supervisor goroutine had not exited by the deadline: a Start that ignores cancellation, or a Status that blocks under the death watch.

Complete() is true when Unfinished is empty.

Setters, and when they are safe to call

SetLogger, SetShutdownTimeout, SetErrorsChannel, SetMessageChannel, SetSignalsChannel, SetWaitGroup and SetState mutate fields that the control goroutines read after Start. They carry no internal synchronisation.

Call them only during construction, before Start, which is what the WithX options do internally. Calling one on a running controller races the running goroutines and is a programming error, not a supported reconfiguration path.

SetState is a different case, and the earlier version of this page had it wrong. It is not there for fakes: StateAccessor is meant to be consumed and implemented outside this module, and a consumer driving a controller it owns needs to say what state it is in. What is true is that it bypasses the compare-and-set transitions, so calling it on a controller you did not construct races the control goroutines, and since readiness is gated on the state it can take a healthy process out of rotation. See wiki spec 0003 D7.

SetSignalsChannel detaches any previous signal.Notify registration before storing the new channel, so swapping the channel cannot leave an orphaned registration receiving signals nobody reads.

Package-level values

Symbol Value Meaning
ErrShutdown errors.NewSentinel("controls.shutdown", "controller shutdown") The cause attached to the controller context for every stop the controller drives. Test for it with errors.Is(context.Cause(ctx), controls.ErrShutdown).
ErrRestartsExhausted errors.NewSentinel("controls.restarts_exhausted", "max restarts exceeded") Inside the error a service leaves on ServiceInfo.Error, Errors() and its terminal ServiceEvent, and a child on Failure.Err, when it has used up its restart policy. errors.Is matches it and the last error beside it.
ErrPanicked errors.NewSentinel("controls.panicked", "panicked") Inside every error converted from a recovered panic, a child's included. The messages read as before: probe panicked: <value>.
ErrStopAbandoned errors.NewSentinel("controls.stop_abandoned", "stop not awaited within the shutdown budget") On ServiceInfo.StopErr for a stop abandoned at the deadline, or not awaited because the budget had gone.
DefaultShutdownTimeout 5 * time.Second Applied when WithShutdownTimeout is not passed.
DefaultRestartResetInterval 30 * time.Second Applied when RestartPolicy.RestartResetInterval is zero.
Stop Message("stop") The only control message.
CauseStop, CauseSignal, CauseParent, CauseGaveUp, CauseCompleted StopCause values What started a shutdown.
CompleteOnFirst, CompleteOnAll CompletionPolicy values Which completion ends the controller.
StopAbandoned, StopUnbudgeted, StopFailed, SupervisorAbandoned UnfinishedReason values Why a shutdown step did not finish.
EventRetrying, EventFailed, EventUnableToStart EventKind values The kinds of ServiceEvent.
SiteStart, SiteStop, SiteProbe, SiteHealthCheck PanicSite values Where a recovered panic was called from.
NeverStarted, Running, UnableToStart, Stopping, Stopped State values The lifecycle states, in order.
Unknown State value Not part of that sequence: the state could not be determined.

The Supervisor sentinels and DefaultFailureBufferSize are on the Supervisor page. Generational's four sentinels are on the Generational page.