- Why performance budgets?
- Why performance budget alerts?
- How to create a performance budget
- How thresholds work
- Suggested performance budgets
- How to configure alerts
- How to tune your performance budgets
Why performance budgets?
Performance budgets are a vital tool in your web performance toolbox. They take the hassle out of monitoring your page performance by notifying you whenever your Synthetic and/or LUX metrics cross a certain threshold.
Why performance budget alerts?
SpeedCurve alerts you whenever one or more of your performance budgets is crossed. We send alerts via email and to webhook endpoints like Slack and HipChat.
As well as sending alerts when your budgets go over their threshold, we also send you alerts when they go back under the threshold. We do this so that it's easier for you to find out whether a change you made has fixed a performance issue.
We don't want to send you too many alerts, so we only send them when the budget status changes, i.e. when it is under and goes over, or when it is over and goes back under. If your test is over budget 10 times or hours in a row, we only send a notification on the first crossover.
How to create a performance budget
Performance budgets are attached to a Favorites chart, and can be set on any of the Synthetic and LUX (RUM) metrics that are plotted on the chart. (To learn how to create a Favorites chart, read Create custom dashboards and charts.)
To add a performance budget to a Favorites chart, click the EDIT button in the chart header (or click ADD CHART at the top of the page to create a new chart).
Then expand the Set a performance budget section.
Performance budgets have three parts:
- The metric that you want to set a budget on. You can choose any of the metrics that are currently plotted on the chart. If you’d like to choose another metric, expand the Define the data source section and add that metric first. (To get you started, SpeedCurve automatically gives you a 'Performance Budgets' dashboard in your Favorites, with some default metrics and budgets. More on this in 'Suggested performance budgets' further down this page.)
- The budget thresholds. All budgets must have an absolute threshold, which means the budget is crossed when any point on the chart goes over or under this value. Rate of change threshold is a relative threshold, which means the budget is crossed when the metric changes by a certain amount, relative to the previous values. See the How thresholds work section below for more information.
- The budget alert settings. Budgets are most useful when they alert you to performance issues, but you can choose not to enable alerts if you prefer. You can also choose to receive alerts only if the budget is crossed a certain number of times. This is useful for reducing the number of false alarms for metrics that fluctuate around the budget threshold. See the Tuning performance budgets section below for more information.
As you tune your performance budget, you should notice a red line appear on the chart preview at the top of the editor.
Once you have finished configuring the budget, click SAVE.
How thresholds work
There are two types of performance budget thresholds: absolute, and relative (or rate of change). These thresholds are totally independent of each other, and essentially work as two separate budgets.
Absolute thresholds let you know when your metrics exceed a certain value. A metric will cross over an absolute threshold when its value is over the threshold value. A metric will cross under an absolute threshold when its value is under or equal to the threshold value. Absolute thresholds are represented by a horizontal red line on a chart.
Relative thresholds (rate of change)
Relative thresholds (sometimes called rate of change) let you know when your metrics change by a certain amount. A metric goes over or under the relative threshold whenever its value changes by more than the threshold percentage.
Relative thresholds are not represented in SpeedCurve charts, but they are represented in alerts as a horizontal green bar showing the acceptable rate of change for a metric. For example, in the alert below the relative threshold is 15%. The previous value in the chart is 2.52, and so the acceptable rate of change is any value between 2.14 (15% below 2.52) and 2.90 (15% above 2.52). The latest value in the chart is 4.17, which is 44% higher than the upper value of the acceptable rate of change.
Relative thresholds with a larger bucket size
The alert "bucket size" is determined by the Only alert when the budget is crossed setting: <N> times for synthetic or <N> hours in-a-row for LUX. When a budget has a bucket size larger than 1, relative thresholds are calculated by comparing the latest N values to the previous N values.
In this alert below, the bucket size is 3, and the relative threshold is 20%. The threshold boundaries are therefore calculated as 20% above and below the average of the previous 3 values. An alert is sent only if the latest 3 values are outside of the threshold boundaries.
Suggested performance budgets
When you sign up for a SpeedCurve plan, we automatically give you a 'Performance Budgets' dashboard in your Favorites. This dashboard includes some default metrics and budgets (follow the links to learn why we think they're good metrics to consider):
- Start render (Synthetic and LUX): 2s
- Speed Index (Synthetic): 4s
- CPU scripting fully loaded (Synthetic): 2s
- Last Painted Hero (Synthetic): 5s
IMPORTANT: These defaults are intended to be a jumping-off point for your own analysis. You may find that some of these metrics and numbers are relevant for your site, while others may be less relevant. And you'll probably want to add some metrics that aren't there by default.
For example, if you're tracking custom metrics, you probably want to create performance budgets for them.
And if you're using LUX, our real user monitoring solution, we'd encourage you to consider tracking these metrics:
We strongly encourage you to look at your own data in your other SpeedCurve dashboards in order to create the most meaningful performance budgets for your site. We also encourage you to monitor your competitors' sites so you can create budgets that will help you stay faster than the competition.
How to configure alerts
By default, budget alerts are sent to the email addresses and webhooks are configured in your account settings. You can find them in the Alerts section of your settings.
You can also configure separate alert settings for individual Favorites dashboards. To do this, click on the cog icon at the top-right of your dashboard and click Edit Dashboard. You can configure the alerts in the Budget Alert Settings section.
Note: When using custom alert settings for a dashboard, the account-level alert settings will be ignored.
How to tune your performance budgets (e.g. to reduce the number of emails)
SpeedCurve sends alert emails whenever the state of a budget changes, i.e. whenever the metric crosses the threshold in either direction. This can result in a large number of alerts being sent when a metric fluctuates around the threshold, for example in this chart below:
The good news is that you can still set up meaningful performance budgets around unstable metrics by modifying the alert bucket size. This number controls how many times in a row a metric must cross the budget thresholds before any alerts are sent. For the chart above, a value of 3 would be sufficient to avoid being sent too many alerts while still enabling the team to be notified when there are considerable changes to the metric.
You can think of this setting as controlling how many data points or hours are required to go over the threshold before an alert is sent. In the example below, the alert was only sent after the final point went over the threshold.