Prometheus

Prometheus is a monitoring system that pulls its data: it requests metrics from each monitored server at a regular interval, a process called scraping. To take part, Stalwart serves its metrics in the Prometheus text format. Prometheus itself can read them, and so can compatible collectors such as VictoriaMetrics or the OpenTelemetry Collector’s Prometheus receiver.

Endpoint

When the exporter is enabled, the metrics are served at /metrics/prometheus, on every HTTP listener of the server. Metric names are converted to Prometheus conventions by replacing dots and hyphens with underscores, so delivery.attempt-time becomes delivery_attempt_time.

In a cluster, each node serves only its own metrics. Prometheus must therefore be configured to collect from every node.

Configuration

The exporter is configured in the prometheus field of the Metrics object (found in the WebUI under Settings › Telemetry › Metrics › General, Settings › Telemetry › Metrics › OpenTelemetry, Settings › Telemetry › Metrics › Prometheus). Its @type is Disabled or Enabled, and the endpoint is served only with Enabled.

The main decision is whether to require authentication. Without credentials, anyone who can reach an HTTP listener can read the metrics, which include message counts, queue sizes and error rates. Set credentials when the HTTP listeners are reachable from untrusted networks.

The Enabled variant has two optional fields for this, authUsername and authSecret. When both are set, the endpoint requires HTTP Basic authentication. The two fields must be set together: when only one is set, the endpoint is disabled, and the configuration reports one of these messages:

Prometheus authUsername is set but authSecret is not, Prometheus endpoint disabled
Prometheus authSecret is set but authUsername is not, Prometheus endpoint disabled

A request with wrong or missing credentials is refused and logged as an auth.failed event. That event is logged at debug, so a tracer at the usual info level does not show it.

Example

The following value enables the endpoint with Basic authentication, reading the password from a file:

{
  "prometheus": {
    "@type": "Enabled",
    "authUsername": "prometheus",
    "authSecret": {"@type": "File", "filePath": "/run/secrets/prometheus-password"}
  }
}

The matching Prometheus job collects from two nodes, with the same credentials:

scrape_configs:
  - job_name: stalwart
    scheme: https
    metrics_path: /metrics/prometheus
    basic_auth:
      username: prometheus
      password_file: /etc/prometheus/stalwart-password
    static_configs:
      - targets: ["mail1.example.com:443", "mail2.example.com:443"]