Why performance budgets?
Performance budgets are a vital tool in your web performance toolbox. They take the hassle out of monitoring your page performance by notifying you whenever your metrics cross a certain threshold.
Why performance budget alerts?
We alert you whenever one or more of your performance budgets is crossed. We send alerts via email and to webhook endpoints like Slack and HipChat.
It's important to remember this only happens when the budget is crossed, either from under budget to over budget, or over budget to under budget. If your test is over budget 10 times or hours in a row, we only send a notification on the first crossover. We don't send 10 notifications. This is done to reduce spam.
How to create a performance budget
Performance budgets are attached to a Favorites chart, and can be set on any of the metrics that are plotted on the chart. To learn how to create a Favorites chart, read Create custom dashboards and charts.
To add a performance budget to a Favorites chart, click the cog icon to edit the chart.
Then click on the BUDGET tab.
Performance budgets have three parts:
- The metric that you want to set a budget on. You can choose any of the metrics that are currently plotted on the chart. If you’d like to choose another metric, go to the FILTERS tab and add that metric first.
- The budget thresholds. Budget Threshold is an absolute threshold, which means the budget is crossed when any point on the chart goes over or under this value. Budget Rate of Change is a relative threshold, which means the budget is crossed when any point on the chart changes by more than the specified percentage in relation to the previous point. See the How thresholds work section below for more information.
- The budget alert settings. Budgets are most useful when they alert you to performance issues, but you can choose not to enable alerts if you prefer. You can also choose to receive alerts only if the budget is crossed a certain number of times. This is useful for reducing the number of false alarms for metrics that fluctuate around the budget threshold. See the Tuning performance budgets section below for more information.
Once you have set the budget metric and threshold, click SAVE. A red threshold line should appear on the chart.
How thresholds work
There are two types of performance budget thresholds: absolute, and relative (or rate of change).
Absolute thresholds are fixed values represented by a horizontal red line on a chart. A metric will cross over an absolute threshold when its value is over the threshold value. A metric will cross under an absolute threshold when its value is under or equal to the threshold value.
Relative thresholds (rate of change)
Relative thresholds (sometimes called rate of change) are dynamic values specified as a percentage. A metric crosses the relative threshold whenever its value changes by more than the threshold percentage.
Relative thresholds are not represented in SpeedCurve dashboards, but they are represented in alerts as a horizontal green bar showing the acceptable rate of change for a metric. For example, in the alert below the relative threshold is 15%. The previous value in the chart is 2.52, and so the acceptable rate of change is any value between 2.14 (15% below 2.52) and 2.90 (15% above 2.52). The latest value in the chart is 4.17, which is 44% higher than the upper value of the acceptable rate of change.
Relative thresholds with a larger bucket size
The alert "bucket size" is determined by the Only alert when the budget is crossed setting: <N> times for synthetic or <N> hours in-a-row for LUX. When a budget has a bucket size larger than 1, relative thresholds are calculated by comparing the latest N values to the previous N values.
In this alert below, the bucket size is 3, and the relative threshold is 20%. The threshold boundaries are therefore calculated as 20% above and below the average of the previous 3 values. An alert is sent only if the latest 3 values are outside of the threshold boundaries.
How to configure alerts
You can configure where your performance budget alerts are sent by visiting your account settings, and looking for the Notifications section.
How to tune your performance budgets (e.g. to reduce the number of emails)
SpeedCurve sends alert emails whenever the state of a budget changes, i.e. whenever the metric crosses the threshold in either direction. This can result in a large number of alerts being sent when a metric fluctuates around the threshold, for example in this chart below:
The good news is that you can still set up meaningful performance budgets around unstable metrics by modifying the alert bucket size. This number controls how many times in a row a metric must cross the budget thresholds before any alerts are sent. For the chart above, a value of 3 would be sufficient to avoid being sent too many alerts while still enabling the team to be notified when there are considerable changes to the metric.
You can think of this setting as controlling how many data points or hours are required to go over the threshold before an alert is sent. In the example below, the alert was only sent after the final point went over the threshold.