At SpeedCurve, we think Speed Index is a pretty good synthetic metric for measuring user-perceived performance, but it doesn't work for every page. Here's how it's measured, and why it doesn't work on some pages.
How Speed Index is measured
In the history of performance metrics, Speed Index was a huge evolutionary leap from previously popular metrics like start render and load time. It represented the first time any tool attempted to tackle the issue of measuring what users actually see.
Speed Index is the average time at which visible parts of the page are displayed. It's measured in WebPageTest (which SpeedCurve Synthetic is built on top of) and it identifies when the visible parts of a page are displayed.
You can find the official Speed Index definition here, but my favourite explainer is from this PerfNow talk by Google engineer Paul Irish. (His explanation of how Speed Index is calculated starts at around the 7:45 mark.)
Speed Index is based on measuring when visual content (e.g., images and video) renders within the viewport. Speed Index evaluates the percentage of visual completion of a page at every point in time. It does this by looking at a filmstrip view of your page rendering, analyzing it on a frame-by-frame basis, and looking for changes in the colour palette.
Then it calculates visual completion as a percentage over time and calculates the total area above the percentage curve. This is expressed in milliseconds:
At SpeedCurve, we convert those milliseconds to seconds (e.g., a Speed Index score of 1219 becomes 1.22 seconds) because it makes it easier to display alongside the other time-based metrics in your charts.
Why Speed Index isn't great for some pages
For many pages Speed Index can be a good indicator of visual completeness, but because of how it's calculated, it doesn't work on all pages. It's not a specific point-in-time, but instead is an aggregate stat, so it's a bit apples-to-oranges when compared to time-based metrics.
We like Speed Index (which is why we've included it in SpeedCurve), but because it's dependent on how the page is built, it's not a one-size-fits-all metric. For example, I recently spoke with a customer who was getting confusing Speed Index results for mobile. It turned out that the reason why their Speed Index was fast – despite the fact that their viewport was almost empty – was because most of their "above the fold" content was not image-based.