Since we are already using Prometheus, it makes sense to deploy Prometheus' companion Alertmanager. It will receive alerts, filter and forward them to the end-points we'll define. Slack will be the first.
Alertmanager Docker image expect us to define a configuration file that defines routes, receivers, and a few other things. One possible configuration can be as follows.
route:
receiver: "slack"
repeat_interval: 1h
receivers:
- name: "slack"
slack_configs:
- send_resolved: true
text: "Something horrible happened! Run for your lives!"
api_url: "https://hooks.slack.com/services\/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"
The configuration defines the route with slack as the receiver of the alerts. In the receivers ...
