Services and restart policy reference¶
A service is a name plus a set of callbacks, registered with Register and
described with ServiceOptions. This page lists the options, the contract each
callback must honour, and every RestartPolicy field with its default.
Register¶
Register has no error return. Three consequences are worth knowing before you
call it:
- Names are not checked for uniqueness. Registering two services under the
same name is accepted. Both are started and stopped, both appear in every
health report as separate entries with the same
name, andGetServiceInfo(name)returns only the most recently registered one's info. - Registration after
Startis not supervised. The service is appended and logged asWARN "Register called after Start; service will not be supervised", but itsStartFuncis never called, itsStopFuncnever runs, and no restart policy applies. It still appears inStatus(),Liveness()andReadiness(), as"OK"if it has no probe. Treat that warning as a bug in your wiring. - Both lifecycle callbacks are optional. A service with no
WithStartgets a no-op that returnsnil; one with noWithStopgets a no-op stop. Neither can be nil at run time, so neither can panic on a nil call.
Service options¶
| Option | Type it takes | Omitted | Notes |
|---|---|---|---|
WithStart |
func(context.Context) error |
no-op returning nil |
Called once per run by the supervisor. The context is the controller's, cancelled at shutdown with cause ErrShutdown. |
WithStop |
func(context.Context) |
no-op | Called during shutdown in reverse registration order, and before each health-triggered restart. The context carries the shutdown deadline. |
WithStopErr |
func(context.Context) error |
none | A stop that reports whether every resource was released. The result lands on ServiceInfo.StopErr and changes nothing else. If both WithStop and WithStopErr are set, only WithStopErr runs. |
WithStatus |
func() error |
none | General health signal: feeds Status(), is the fallback for Liveness()/Readiness(), and is the probe the restart supervisor polls. |
WithLiveness |
func() error |
falls back to WithStatus |
Feeds Liveness() only. |
WithReadiness |
func() error |
falls back to WithStatus |
Feeds Readiness() only. |
WithRestartPolicy |
RestartPolicy (by value) |
no restarts at all | The policy is copied at registration; later mutation of your struct has no effect. |
WithRunToCompletion |
— | a nil return is a clean background start |
Declares the service a unit of work: a nil return with the controller live stops it with CauseCompleted, and any error return, context.Canceled and a valid error included, is a failure that stops it with CauseGaveUp once the service fails for good. A completed service is not health-monitored. See Stop when a service gives up. |
WithRestartResetInterval |
time.Duration |
DefaultRestartResetInterval (30s) |
Sets RestartPolicy.RestartResetInterval, creating a zero-valued policy first if the service has none, so this option alone enables restarts, with every other field at its default, including unlimited MaxRestarts. Order relative to WithRestartPolicy does not matter. |
What each callback must honour¶
StartFunc: func(ctx context.Context) error¶
| Return | Classified as | Restarted? | Reported as a failure? |
|---|---|---|---|
nil |
clean start: the service is serving in the background | never | no |
any error while ctx is already cancelled, or context.Canceled |
cancelled | never | no |
an error matching the WithValidError predicate |
cancelled | never | no |
| any other non-nil error | failure | yes, if a RestartPolicy is attached |
yes |
It must return after its context is cancelled. A StartFunc that ignores
cancellation pins its supervisor goroutine; the shutdown sequence abandons it at
the deadline and names it in a WARN, but a bare Wait() then blocks forever.
Use WaitContext when a service wraps third-party code that may not honour
cancellation.
A panic inside a StartFunc is that service's failure. It is recovered,
wrapped as controls: service "<name>" panicked: <value> satisfying
errors.Is(err, ErrPanicked), and handled like any other error: retried under a
restart policy, UnableToStart without one. It is never a valid error, whatever
WithValidError says. A goroutine the StartFunc launches is not covered;
recover inside it.
A restart calls it again with the same closure. Anything the closure
captured is shared between runs, so a listener, a server or a supervisor built
at wiring time is the previous run's by the second call. Build single-use things
inside the run, or wrap them in a
Generational.
StopFunc: func(ctx context.Context)¶
- The context is derived from
context.Background()with the shutdown timeout, not from the already-cancelled controller context, sohttp.Server.Shutdownand friends can still drain in-flight work. - It may be called more than once, and must be idempotent. Shutdown calls it once; a health-threshold restart calls it again before every restart of that service.
- It is not called between error-triggered restarts. When a
StartFuncreturns a genuine error and the policy restarts the service,StopFuncdoes not run in between; only a health-threshold restart stops the service first. - A panic inside a
StopFuncis recovered, recorded onServiceInfo.StopErr, and the shutdown sequence continues with the next service. - Ignoring the context does not hang shutdown: the callback is abandoned at the deadline and left to finish on its own. The work it was doing is not completed.
StopErrFunc: func(ctx context.Context) error¶
Everything above, plus a return value. nil means every resource the run
acquired has been released; non-nil means the budget expired or release failed,
and the service may still be holding something. The controller stores the
answer on ServiceInfo.StopErr and does nothing else with it: no restart is
refused and no policy is altered on its strength. Before it existed, a stop that
ignored its context was abandoned at the deadline and nothing recorded that it
had happened.
StatusFunc and ProbeFunc: func() error¶
- Return
nilfor healthy, any error for unhealthy. The error text is copied into the report entry. - Probes run inline while a report is built, so a slow probe delays the report and every caller waiting on it. Apply your own timeout inside a probe that does I/O.
- A panic in a probe called from
Status(),Liveness()orReadiness()is recovered and reported asprobe panicked: <value>. - A panic in a
WithStatusprobe polled by the restart supervisor breaches at once, whateverHealthFailureThresholdis: the service is stopped and restarted under its policy, withhealth check failed: probe panicked: ….WithProbePanicsAsFailuresmakes it count as one failure instead. - It must report the current run. A status cell the closure captured and never cleared between restarts keeps reporting the previous run's failure, which trips the health threshold again.
RestartPolicy fields¶
type RestartPolicy struct {
MaxRestarts int
InitialBackoff time.Duration
MaxBackoff time.Duration
HealthFailureThreshold int
HealthCheckInterval time.Duration
RestartResetInterval time.Duration
}
| Field | Zero value means | Default applied | What it controls |
|---|---|---|---|
MaxRestarts |
unlimited restarts | — | The cap on consecutive failures. MaxRestarts: 2 allows two restarts, so the service runs at most three times before the supervisor gives up. |
InitialBackoff |
use the default | 1s | The wait before the first restart. |
MaxBackoff |
use the default | 30s | The ceiling the wait doubles towards. The multiplier is fixed at 2.0 and there is no jitter. |
HealthFailureThreshold |
health monitoring off | — | Consecutive WithStatus failures that trigger a restart. Requires a WithStatus probe: with no probe, health monitoring stays off however high you set this. |
HealthCheckInterval |
use the default | 10s | How often the supervisor polls WithStatus while health monitoring is on. |
RestartResetInterval |
use the default | 30s (DefaultRestartResetInterval) |
How long a run must last before the consecutive-failure counter and the backoff reset. |
Attaching a policy is what enables restarts at all. Without one, a StartFunc
error is recorded in ServiceInfo, reported as an EventUnableToStart, and the
service is left stopped.
The same type governs a Supervisor's children, read through the same helpers,
with HealthFailureThreshold and HealthCheckInterval inert there because a
child has no Status probe. See the Supervisor reference.
What the supervisor does when restarts run out¶
When the consecutive-failure counter reaches MaxRestarts, the supervisor stops
supervising that service and records an error that errors.Is matches against
ErrRestartsExhausted:
- from an error-driven restart loop,
max restarts exceeded: <last start error>, anderrors.Isagainst the last error still matches too, since the error unwraps to both; - from a health-driven restart loop,
max restarts exceeded: health check failed: <last health error>, matching the health error the same way.
That error is stored on ServiceInfo.Error and carried by the terminal
ServiceEvent, EventFailed or EventUnableToStart, and on the error channel.
The controller keeps running, unless you opt in. A service exhausting its
restarts, or failing with no policy at all, does not shut the process down by
default. WithStopOnGiveUp makes it stop the controller with CauseGaveUp; see
Stop when a service gives up.
What it does change is readiness, but only if the service never started. A
service that has never started cleanly and has exhausted its policy will never
start, so the controller moves to UnableToStart and stops reporting ready. With
no policy at all that is the first genuine error, since there are no retries. A
service that did start cleanly and failed later is an ordinary failure however
many restarts it then used up, and so is one that fails once and recovers on a
restart: both conditions are required. Nothing is stopped either way. See
the health model.
ServiceInfo¶
type ServiceInfo struct {
Name string
RestartCount int
LastStarted time.Time
LastStopped time.Time
Error error
StopErr error
}
GetServiceInfo returns a copy, so reading it never blocks the supervisor.
| Field | Meaning |
|---|---|
Name |
The id passed to Register. |
RestartCount |
Consecutive failures so far, not lifetime restarts. Reset to zero when a run lasts at least RestartResetInterval. |
LastStarted |
When the most recent run began. |
LastStopped |
When the most recent run returned. Set on every return, including a clean start that is still serving in the background, so it is not evidence the service has stopped. |
Error |
The most recent classified error, or once the supervisor gave up the max restarts exceeded error, which errors.Is matches against ErrRestartsExhausted and against the last error. nil after a clean run. |
StopErr |
How the last stop ended: nil when every resource was released, non-nil when it was not (including a recovered panic), and always nil for a service registered with WithStop, which cannot report either way. |
Ordering guarantees¶
- Startup is concurrent. One supervisor goroutine is launched per service in
registration order; the controller does not wait for a
StartFuncto return or a probe to pass before launching the next. There is no dependency ordering or readiness gate between services. Gate inside your ownStartFuncif you need one. - Shutdown is sequential and reversed. Stop callbacks run one at a time in
reverse registration order: register
databasethenhttp-api, andhttp-apiis stopped first.
Related¶
- Configure restart policy: the task-oriented recipe.
- The restart supervisor: why a clean start is never restarted.
- Defaults and timings: the same defaults alongside every other one.