Alerting

FusionReactor Cloud Alerting allows you to create checks based on the values of metric data gathered by a FusionReactor agent. The alerting system is comprised of several components:

  • Checks are the basis of the conditions that are processed by the alerting engine
  • Alerts are generated if the conditions of a check are met
  • Subscriptions can be added to a check and are used to notify different channels when an alert is generated

Checks

Checks can take two forms:

  1. Threshold checks are used to alert when a metric value crosses a defined threshold
  2. Status checks are used to alert when the status online / offline of a server, or group of servers changes

Checks are created using the following interface

!Screenshot

When all of the required fields for a check are completed, a preview graph showing the current data and the check period is generated.

The alerting engine runs every 60 seconds - if data is available for the check period selected the check will be processed immediately.

Once a condition has been created, a subscription can be added to the check.

!Screenshot

See the Subscriptions section below for information on creating a subscription.

The final screen shows a preview of the check that has been created and any additional options that can be added at this stage.

!Screenshot

Alerts

Viewing alerts

!Screenshot

To view your alerts:

  1. Go to Alerting
  2. View the "Alerts" tab

Here you will see all the alerts for your account that are recorded. There are various options to filter the alert log available in the filter bar.

When a check has been processed, it can fall into one of the following states:

  • PENDING
  • DISABLED
  • DELETED
  • NO_DATA
  • OK
  • WARNING
  • ERROR

When a check has fallen into or out of OK, WARNING or ERROR state you will receive an alert notifying you of this.

The alert will be sent to the subscription you have enabled on the check.

Subscriptions

The following alerting subscriptions are currently supported:

Configuring subscriptions

!Screenshot

Alerting configurations can be found under settings > configuration > alert services. To configure a service click the Configuration button next to that service.

Email

No configuration is required.

Flowdock

More information: https://www.flowdock.com/

  • From your main Flowdock screen (after logging in), click your username on the navigation bar and select 'Account'
  • Select 'API tokens' on the left navigation
  • Scroll down until you see 'Flow API tokens' heading
  • Select the inbox you wish to use and take the API Token provided
Hipchat (V1 API)

The owner of the desired chat room must configure this service

More information: http://hipchat.com/

  • After you login to HipChat on the web, click 'Rooms' just under 'Welcome {username}'
  • Select the room you wish the alert messages to go to
  • Select 'Tokens' on the left navigation
  • Enter a label under 'Create new token' ensuring you have the 'Send Notification' scope selected
  • Ensure you also have the 'View Group' and 'View Room' scopes selected to make setting up your sevices easier
Hipchat (V2 API)

The owner of the desired chat room must configure this service

More information: http://hipchat.com/

  • After you login to HipChat on the web, click 'Rooms' just under 'Welcome {username}'
  • Select the room you wish the alert messages to go to
  • Select 'Tokens' on the left navigation
  • Enter a label under 'Create new token' ensuring you have the 'Send Notification' scope selected
  • Ensure you also have the 'View Group' and 'View Room' scopes selected to make setting up your sevices easier
Http (Webhook)
  • Enter Webhook URL in the individual sevices after creating an alert check.
OpsGenie

More information: https://www.opsgenie.com/

  • After you login to OpsGenie, click 'Integrations' on the left navigation bar
  • Select the 'Configure Integrations' tab
  • Select Default API (Rest API HTTPS over JSON)
  • Use the API Key listed on that page.
PagerDuty

More information: https://www.pagerduty.com/

  • After logging into the PagerDuty service, you can select 'Configuration' on the top navigation bar
  • Here you can crete a service to represent your FRCloud alerts
  • View a service and select the 'Settings' tab
  • Provide us with the integration key
Pushover

More information: https://pushover.net/

  • After you login to Pushover, 'Register an Application'
  • Enter the details required
  • After redirection, use 'API Token/Key'
Slack

More information: https://slack.com/

VictorOps

More information: https://portal.victorops.com

  • Sign into VictorOps and click 'Settings' on the top navigation bar
  • Select the 'Integrations' tab
  • Select 'REST Endpoint' on the list to the right
  • Select 'Enable Integration'
  • Provide the Post URL

Resetting subscriptions

!Screenshot

Above: Confirmation dialog for resetting HipChat service

You can reset the configuration for a service by clicking the Reset button next to that service.

When a service configuration is reset the following will happen:

  • You will no longer be able to select and use that service when Adding sevices to a check
  • Each sevice that is linked to that service will receive an Alert informing them that the service has been reset.
  • Each sevice that is linked to that service will no longer receive Alerts triggered by Custom checks and Automatic checks.