April 2017 Urban Broadband Report
The ISP Webpage Surfing gap
In this months report we have to ask why Vodafone is much slower than Orcon for webpage surfing! Why are there such large gaps between ISP's surfing performance on the same plans?
Special Chart 1b shows Webpage average speed by time of day for Fibre 100Mb/s and above. Click on the interactive chart to see the speeds at different time periods.The Peak speed gap between Orcon (43Mb/s) is three times that of Vodafone (18Mb/s).
What are the underlying issues creating such a huge difference in the same product for the same market? This difference is worst for Fibre, but nearly as extreme in the slower speed technologies of ADSL and VDSL. The key to good surfing conditions within NZ that an ISP can influence may include sufficient capacity on all route links, DNS response times, peering at the internet exchanges, and the number of devices over a route.
Vodafone, Trustpower and BigPipe speeds during peak hours, are about half the speed of other ISPs like Orcon, 2Degrees and MyRepublic when surfing NZ webpages (Kiwibank, NZTA, PriceMe, Radio NZ, Real Estate, Scoop and The Warehouse).Special Chart 1 (Chart 1b has the same data but for the full Time of Day comparison)
We will add Slingshot again next month to Fibre, VDSL and ADSL after a successful drive to get more Slingshot volunteers.
Note: Charts use the name "Wireless" to indicate "Fixed Wireless", a service being delivered over mobile 4G networks by Spark, Skinny and Vodafone. We do not have enough Vodafone panelists to report yet.
- Technology comparison
- Fibre & Cable
- Copper (ADSL and VDSL)
- Fibre & Cable
- Copper (ADSL and VDSL)
Summary of Performance Measures
Table 1: Summary of Performance Measures
Note: Webpage average speed is the website size divided by the time to download. Webpages are often sent from the owners site in many small files, which means the speed is not as fast as that achieved during a single file download (Peak Speed).
A simple analogy to explain Peak vs Average speed, assumes you have a very fast car like a Ferrari
Speed Min/Max compares slowest hour to the fastest hour (as fast as your line will allow).
The 9pm column is the actual average speed at 9pm for each product. The average is taken over all ISPs because for copper, actual speed is subject to the distance between our test points and the exchange equipment. The range of offered speeds for Fibre is large so we have provided an average speed for all fibre connections, which matches the methodology used for copper.
Responsive website browsing is valued by most Internet users, and conversely, slow-loading sites can be extremely frustrating.
TrueNet tests Internet browsing by downloading a selection of Live Webpages from NZ, Australia, and the USA; measuring the size of each webpage, and the time to fully download all files on the page. From this we calculate the speed. These pages are changed from time-to-time so the actual average webpage speeds cannot be compared between months. This month there are 7 NZ pages, 3 Australian, and 5 USA pages. DO NOT COMPARE SPEEDS BETWEEN COUNTRIES because average webpage speed is dependent on the actual webpages downloaded, not the ISPs. The list of test webpages is in the Glossary
We download webpages from each connection, and compare the average speed achieved for each download. This test replicates daily activity for many people, and we then group the webpages into the regions NZ, Australia and USA, so that readers can compare ISPs based on their own preferences.
Fibre webpage average speeds show no material difference based on Fibre speeds sold, so we show them inclusive of 100, 200Mb/s, and GigE services.
NZ Webpage Surfing
Chart 1a - Shows a comparison of average speed between FIbre, Cable, Copper, and Fixed Mobile when surfing a set of NZ Webpages.
- ADSL and VDSL have consistent speed throughout the day.
- Fixed Wireless continues to experience a broad reduction in speed during the day, with speed dropping below ADSL.
- Cable and Fibre also reduce in speed during the day, and into the evening. Vodafone fibre is the main contributor to the fibre time of day (ToD) variation.
Fibre and Cable webpage surfing show similar results to March, with Orcon still well ahead, MyRepublic shows continued improvement and is now on par with 2Degrees. Vodafone fibre and cable once again showing not only ToD issues, but also with speed in general.Chart 1b: Fibre & Cable - Webpage Average Speed of Selected NZ Webpages by Time of Day
Copper webpage Surfing speed is limited by the speed of copper, VDSL has an average peak speed of 30Mb/s and ADSL about 10Mb/s, but there is a wide range of speeds about these averages.
VDSL webpage surfing (Solid lines) 2Degrees remain ahead of the other ISPs, both Voyager and Orcon show improvement. Vodafone continue showing the broad roll-off during the day.
ADSL webpage surfing (dotted lines) - Show a slight increase in Spark performance.
Chart 1c: Copper - Webpage Average Speed of Selected NZ Webpages by Time of Day
Chart 1d - Spark and Skinny's webpage surfing speeds continue about the same this month, with both showing the steady slow down that occurs during the day and on into the evening.
Chart 1d: Fixed Wireless - Webpage Average Speed of Selected NZ Webpages by Time of Day
US Webpage Surfing
|We download popular US webpages from each connection, and compare the average time taken to download all webpages. This test replicates the daily activity of many people accessing webpages from the USA, and includes the impact of good network design at an ISP, where NZ caching improves international website performance.|
Chart 2a - Spark is much improved on March results averaging 10Mb/s faster this month, ahead of the other ISPs. Overall there was an increase in average speed in April, with the exception of Trustpower fibre, which has half of Sparks speed at 16 Mb/s and who with Vodafone Cable remain well below the other products in this test.
Chart 2a: Fibre & Cable - Webpage Average Speed of Selected US Webpages by Time of Day
Chart 2b - VDSL US webpage surfing: The four major ISPs are very closely grouped in these April results, Orcon with better results and Vodafone and Spark down slightly from last month. No signficant time of day performance issues to report. The average speed of Voyager VDSL improved slightly to sit just above the best ADSL ISP.
ADSL US webpage surfing: There is an improved performance from Spark, improving by about 1Mb/s.
Chart 2b: Copper - Webpage Average Speed of Selected US Webpages by Time of Day
Chart 2c - Spark results continue similar to Skinny in April, the same as seen in the NZ webpage results, with very similar speeds.
Chart 2c: Fixed Wireless - Webpage Average Speed of Selected US Webpages by Time of Day
Australian Webpage Surfing
We download popular Australian webpages from each connection, and compare the average time taken to download all webpages. This test replicates daily activity for many people, but to provide the ability for readers to compare between ISPs based on their own preferences, we group the webpages into regions, NZ, Australia and USA.
Remember: DO NOT COMPARE SPEEDS BETWEEN COUNTRIES because average webpage speed is dependent on the size of webpages downloaded, which are different for each country.
Chart 3a - Vodafone Cable improved speed to 40Mb/s from March, while Trustpower's average fell to a similar level. Voyager had an unusual slowdown between 4am and 11am, and it will be interesting to see if this perists, or is one off for April, this slowdown was across the board for all Voyager panelists.
Chart 3a: Fibre & Cable - Average Speed of Selected Australian Webpages by Time of Day
Chart 3b - VDSL Australian webpage surfing has a similar order of ISPs and performance to previous months. Voyager results display the 4am and 11am slowdown as seen in the Fibre results in chart 3a above, with speeds less than the best performing ADSL
ADSL Australian webpage surfing - Spark improved to be on par with 2Degrees in April.
Chart 3b: Copper - Webpage Average Speed of Selected Australian Webpages by Time of Day
As in previous months results, each chart for Fixed Wireless in April shows the drop off in performance throughout the day for both ISPs.Chart 3c: Fixed Wireless - Webpage Average Speed of Selected Australian Webpages by Time of Day
For TrueNet's peak speed tests each panelist's probe regularly downloads a 2MB and/or 5MB file from Auckland, Wellington, Dallas and Sydney. We identify the fastest quartile/decile as the Peak Speed. The faster the connection, the larger the file we need to download to ensure that the maximum speed is reached during our test for most ISPs. 100Mb/s connections easily reach full speed before 2MB of data is downloaded from NZ or Australia. Slower connections can test accurately with much smaller files.
International tests take the result from each test run from Dallas or Sydney.
Comparing performance by time-of-day is important as it shows the service degradation when everyone is using the Internet during the evening hours of 8pm to 10pm. A poor result typically shows the line drop below 90%, which usually occurs in the busy period between 7pm and 10pm, i.e. if this is true, the average user for that ISP is getting less than 90% of their line capability.
NZ peak speed can be influenced by how well ISPs peer at the internet exchanges, where they connect to TrueNet's server provider for our file test. Without effective peering a file can travel from a TrueNet test probe located in Wellington, through to Auckland, on to Sydney then back to the Wellington exchange, creating a problem with slowdown in service delivery during busy hours.
TrueNet however, recommends you compare ISPs based on webpage surfing speeds rather than peak speed. To make the comparison fairer, hosted webpages are popular sites that all ISPs have equal access to.
TrueNet uses the best quartile of a pair of Auckland and Wellington tests to calculate the median NZ results by hour, over the month for each monitored connection. We take the average of all median results with each ISP for each hour.
Chart 4 shows a comparison of Time of Day performance of Fibre, Cable, VDSL, ADSL and Fixed Wireless. Fibre, VDSL and ADSL were above 95% of best hour Peak Speed, which is a great result. At 9pm Cable dropped to 93% of the best Peak hour speed, an improvement on March results . Fixed Wireless shows a steady decline during the "waking" hours to a poor result of 72% of fastest .
Chart 4: Fibre, Cable and Copper (DSL) Peak Speed by Time of Day from NZ
Fibre & Cable Comparison - Peak Speeds by Time of Day
Chart 5a - There were further improvements in April, with most Fibre ISPs staying above 95% of best peak hour speed. The main exceptions were MyRepublic and Vodafone Fibre, staying mostly above 90% of best peak hour speed (90% is an excellent 4 star result - see star ratings in the Glossary here), with 95% at 5 star.
Note: With Fibre speeds, the ability of the backhaul network to "ramp-up" to full speed can be compromised, we use a 2MB file which for some ISPs is more than adequate, but for others this is not enough to reach the maximum speed possible. We prefer the 2MB file because that provides a good test for any ramp-up issues.Chart 5a: 100Mb/s and greater Fibre & Cable File Peak Speed by Time of Day from NZ
DSL Performance by Time of Day (ToD) Indexed to Peak Speeds
|TrueNet uses an index for copper (DSL) reporting because ADSL and VDSL are sold using the phrase "as fast as your line will allow", so we find the maximum speed, and compare each result with that maximum. Hence every line reaches 100% at some hour.|
High Speed Copper (VDSL) Peak Speed by Time of Day from NZ
Chart 5b - 2Degrees had exceptional (5 star) results in April, consistently above 99% of peak speed (see star ratings in the Glossary here). Spark and Voyager also remained above 95% of best hour Peak Speed at all hours of the day.
Vodafone had a broad roll-off in speed as reported last month, but the evening busy hour result of 91% is still a good result.Chart 5b: VDSL File Peak Speed by Time of Day from NZ
Low Speed Copper (ADSL) File Peak Speed by Time of Day from NZ
Chart 5c - ALL ISP's show excellent performance this month, consistently above 95% at all times of day, with Spark achieving above 99%. We are seeking more ADSL volunteers for each of the ISPs in the chart.Chart 5c: ADSL File Peak Speed by Time of Day from NZ
Fixed Wireless Peak Speed by Time of Day from NZ
Chart 5d - There is little difference between both ISP's, with users experiencing an all day decline to a 8-9pm low point, which is now a well established pattern.Chart 5d: Fixed Wireless File Peak Speed by Time of Day from NZ
To compare international performance, TrueNet downloads files every hour to measure Peak Speeds from Dallas and Sydney. The Peak international Speed reported is compared to the maximum NZ peak speed, ie this is the potential peak speed for each connection.
We ensure the download file is not held in New Zealand (cached), so that the test truly measures international performance.
The connection between the ISP and the Sydney peering exchange is critical to many services based in Australia, and the rest of the world. Many services are "cached" in Sydney on servers that are supplied by third parties to improve download speeds in remote places. Two examples are Apple or Microsoft software updates, which are cached by Akamai in NZ. We test by downloading our file from a server connected to the Sydney exchange. This file is random and has checks on it to ensure it is not cached in NZ, which means it is an honest measure from Sydney.
High Speed Fibre & Cable - Australian Peak Speed by Time of Day
Chart 6a - In this chart, results are compared as a percentage of NZ speeds so care needs to be taken interpreting the results: Trustpower stands out because their actual speeds are slower than others (e.g. 100, 200Mb/s and GigE) with a lower average NZ speed than the larger ISPs (110Mb/s compared to about 140 for others). Time of Day variation is negligable for most ISPs, which is a great result.Chart 6a: Australian File Peak Speed for High Speed Connections (2MB file size only) by Time of Day
ADSL & VDSL (Copper Connections) - Australian Peak Speeds by Time of Day
|Copper connections (ADSL & VDSL) have a speed that is dependent on the distance between the home modem, and the exchange equipment which means that ISPs have little influence on the peak speed of each connection. To overcome this limitation, the Australia, and USA speeds are referenced to the average NZ download speed of the respective ISPs.|
VDSL (High Speed Copper) Australian Peak Speeds by Time of Day
Chart 6b, shows two very different ISP groups - 2Degrees and Spark with faster results, and then Orcon, Vodafone and Voyager together with similar but slower speeds. In March 2Degrees and Spark had virtually identical results, but Spark has improved in the April results.Chart 6b: VDSL Peak Speed from Sydney, Australia (2MB file size only) by Time of Day
ADSL (Low Speed Copper) Australian Peak Speeds by Time of Day
ADSL continues to have great results in the Australian peak speed test. The time of day performance versus maximum NZ speed would be good for tests within NZ, and it is more impressive to get these results over the trans-Tasman link. Spark and Vodafone maintained speeds equivalent within 95% maximum NZ speed. Orcon improved from March, to be very similar to the other ISPs in these results. ADSL shows the best results of the test technologies in the Sydney peak speed test.Chart 6c: ADSL peak speed performance from Sydney, Australia (2MB file size only) by Time of Day
Fixed Wireless Australian Peak Speeds by Time of Day
Chart 6d - Spark and Skinny again have nearly identical results, with both experiencing a gradual slowdown during daylight hours. Note this measure shows the speed as a percentage of the best NZ speed recorded regularly, ie it is most likely to be the percentage of the median Spark or Skinny Fixed Wireless NZ speed at 2-4am. The shape of the Time of Day in Chart 6d remains constant between months, though the overall % is a few points lower than in March.Chart 6d: Fixed Wireless FIle Download performance from Sydney, Australia (2MB file size only) by Time of Day
|Upload speed is important to users sending large amounts of data through the Internet, or uploading files to the Cloud. TrueNet's upload test sends a 1MB file to our Wellington server, and records the results as the median of the 10 deciles measured for each download.|
By Technology / Service Upload Speed
The Upload Speed performance measured for each technology is one of the few measures that remains stable over long periods of time.
The 50Mb/s Fibre upload service averages slightly under the advertised speed, although VERY quick, but other Fibre upload speeds are equal to, or greater than the advertised speed.
April Cable results have been separated into 20Mb/s and Full Speed Upload Services. Cable Full Speed upload averaged around 34Mb/s, and the 20Mb/s service was at advertised speed. Note that a 1MB test file was used, hence may not have sufficient time to ramp-up depending on latency.Chart 7: Upload Speed by Technology
Upload Speed - by ISP
Chart 8 - Fibre & Cable 20Mb/s upload speeds have small variations between ISPs, and we note that Vodafone Cable delivers almost precisely the advertised 20Mb/s, where the fibre results are all somewhat above 20Mb/s, including the Vodafone Fibre. With the inclusion of Full Speed cable (FibreX Max), users do gain better upload speeds than 20Mb/s service, but averaging less that 50Mb/s fibre upload services.Chart 8: Fibre & Cable Upload Speed by ISP
TrueNet's video test chooses specific videos (Two videos, a HD one and an identical content 4K video) The video will be cached (stored locally) by ISPs in NZ for all viewers, simply because we are downloading the video seven times a minute from more than 400 different NZ locations.
Each test downloads 10 seconds of content at a time, to fill any buffer to last 40 seconds of viewing. So on startup, the test will immediately download (at the comnnection's full speed) 40 seconds worth of content, after a time gap (to allow for playback) it will then download a second 40 seconds of content, simulating keeping the 40 second playback buffer full. This appears to be an international standard for consumer video testing.
TrueNet records buffer events by counting "Stops", where the simulated buffer would become empty. We also record Resume and Finished events. Chart 1 shows a count of the number of "Stops" (sometimes more than one stop in a single download attempt).
We also measure the start delay, ie the time taken for a video to start downloading so it can play. The typical period for this is so small, usually less than 1 second, that we have ceased publishing this statistic.
Buffering Events during Youtube watching
There was an increase of buffering events for ADSL and Fixed Wireless in April, but it is generally absent for other technologies. Our experience is that Youtube buffering problems are often unique to a few panelists rather than spread over many technologies, so if you have an issue with buffering, try these tips, check your Wifi, your Connection (eg the line to the street) or ask for help from your ISP. Note, only Fixed Wireless appeared to experience increased buffering during the evening peak period, almost doubling the average rate of buffering events.Chart 9a: Buffering Events by Day
Chart 9b: Buffering Events for the Month