March 2017 Urban Broadband Report
Vodafone's Homegrown Congestion
Vodafone shows a strong decline during the day and into the evening, in stark contrast to all other ISP results on any fixed line technology. The trend in Special Chart 3, has been evident for some time, indicating long standing congestion on the Vodafone Network -particularly with regards to NZ when connecting to major NZ websites.
Special Chart 1 - shows all four technologies webpage average speed by time of day, decline is experienced across the board on all of Vodafones technologies.
Special Chart 1: Vodafone Speed by Technology
Special Chart 2 - is specific to location, the most notable feature being the huge 9pm dip for the selection of NZ ISPs, Banks, and NZ webpages.
Special Chart 2: Vodafone Speed by Location
Special Chart 3: NZ Webpage download speeds for VDSL by ISP, minimum speed during the day divided by the maximum speed, plotted over the last 12 months
Note: Charts use the name "Wireless" to indicate "Fixed Wireless", a new service being delivered over mobile 4G networks.
- Technology comparison
- Fibre & Cable
- Copper (ADSL and VDSL)
- Fibre & Cable
- Copper (ADSL and VDSL)
Summary of Performance MeasuresTable 1: Summary of Performance Measures
*Note: Webpage average speed is the website size divided by the time to download. Webpages are often sent from the owners site in many small files, which means the speed is not as fast as that of a single file download.
Speed Min/Max compares slowest hour to the fastest hour (as fast as your line will allow).
The 9pm column is the actual average speed at 9pm for each product. The average is taken over all ISPs because for copper, actual speed is subject to the distance between our test points and the exchange equipment. The range of offered speeds for Fibre is large so we have provided an average speed for all fibre connections, which matches the methodology used for copper.
Responsive website browsing is valued by most Internet users, and conversely, slow-loading sites can be extremely frustrating.
TrueNet tests Internet browsing by downloading a selection of Live Webpages from NZ, Australia, and the USA; measuring the size of each webpage, and the time to fully download all files on the page. From this we calculate the speed. These pages are changed from time-to-time so the actual average webpage speeds cannot be compared between months. This month there are 7 NZ pages, 3 Australian, and 5 USA pages. DO NOT COMPARE SPEEDS BETWEEN COUNTRIES because average webpage speed is dependent on the actual webpages downloaded, not the ISPs. The list of test webpages is in the Glossary
We download webpages from each connection, and compare the average speed achieved for each download. This test replicates daily activity for many people, and we then group the webpages into the regions NZ, Australia and USA, so that readers can compare ISPs based on their own preferences.
Fibre webpage average speeds show no material difference based on Fibre speeds sold, so we show them inclusive of 100, 200Mb/s, and GigE services.
Chart 1a - shows a comparison of the average speed of the main access technologies when downloading a set of NZ Webpages.
ADSL and VDSL remain consistent throughout the day.
Fixed Wireless is showing a broad slowdown throughout the day, speed is comparable (slightly slower) to ADSLChart 1a: Technology Comparison - Webpage Average Speed of Selected NZ Webpages by ToD
Chart 1b - Orcon fibre shows great results this month, and along with 2 Degrees are averaging better download speeds than the other major ISPs (who were all about equal last month). My Republic is showing improvement on last month's results remaining consistent throughout the day, but once again both Vodafone's fibre and cable network show a broad slow down in performance for a large portion of the day.Chart 1b: Fibre & Cable - Webpage Average Speed of Selected NZ Webpages by ToD
ADSL - 2 Degrees (in chart 1c) has improved on February's results, ahead of the other ISP's with a consistently reliable performance.
VDSL - Spark was well in the lead for VDSL in February, but this month they have dropped below 2 Degrees. Orcon shows a broad drop off in VDSL performance this month to just above Vodafone who, once again for ToD results, sit below the other ISPs for copper services.
Chart 1d - Spark Fixed Wireless results are on par with Skinny this month, and again both with the steady slow down that occurs during daylight hours and into the evening.Chart 1d: Fixed Wireless - Webpage Average Speed of Selected NZ Webpages by ToD
|We download popular US webpages from each connection, and compare the average time taken to download all webpages. This test replicates the daily activity of many people accessing webpages from the USA, and includes the impact of good network design at an ISP, where NZ caching improves international website performance.|
Chart 2a - Vodafone and Orcon clearly have the best average speeds for US Webpages, however Voyager and Spark both improved alot. My Republic also shows great improvement although still to a level well below other ISPs. Trustpower (who we started measuring again this month) trails behind the others along with Vodafone Cable.Chart 2a: Fibre & Cable - Webpage Average Speed of Selected US Webpages by ToD
VDSL: Vodafone (chart 2b) again has good performance, leading in both VDSL and fibre this month, but is the only ISP showing a dip around the busy 8pm time slot. 2Degrees is much improved on last month, but Voyager VDSL, although slightly improved, is again comparable to ADSL speed this month.
ADSL: Not much change from February
Chart 2b: Copper - Webpage Average Speed of Selected US Webpages by ToD
Chart 2c - Fixed Wireless performance in downloading US webpages - Spark and Skinny have a steady reduction in speed during the day, and into the evening, the same pattern as shown in the NZ webpages (Chart 1d)Chart 2c: Fixed Wireless - Webpage Average Speed of Selected US Webpages by ToD
We download popular Australian webpages from each connection, and compare the average time taken to download all webpages. This test replicates daily activity for many people, but to provide the ability for readers to compare between ISPs based on their own preferences, we group the webpages into regions, NZ, Australia and USA.
Remember: DO NOT COMPARE SPEEDS BETWEEN COUNTRIES because average webpage speed is dependent on the size of webpages downloaded, which are different for each country.
Chart 3a - results are very comparable to last month, My Republic and Orcon are the only ISP's to show noticeable improvement, with Spark and Orcon now the best.Chart 3a: Fibre & Cable - Average Speed of Selected Australian Webpages by ToD
Chart 3b - ADSL had virtually no change on last month, and all ISPs remain grouped together.
2Degrees, however improved over February, almost matching Spark. Orcon managed to drop significantly to match Vodafone.
Chart 3b: Copper - Webpage Average Speed of Selected Australian Webpages by ToD
Chart 3c - Each chart for Fixed Wireless is showing the same steady decline during the day.Chart 3c: Fixed Wireless - Webpage Average Speed of Selected Australian Webpages by ToD
Speed (File Peak Speed)
For TrueNet's speed tests each panelist's probe regularly downloads a 2MB and/or 5MB file from Auckland, Wellington, Dallas and Sydney. The faster the connection, the larger the file we need to download to ensure that the maximum speed is reached during our test. 100Mb/s connections easily reach full speed before 2MB of data is downloaded from NZ or Australia. Slower connections can test accurately with much smaller files.
International tests take the result from each test run from Dallas or Sydney.
NZ file peak speed can be influenced by how well ISPs peer at the internet exchanges, with TrueNet's server provider where our file test originates. Without effective peering a file can travel from a TrueNet test probe located in Wellington, through to Auckland, on to Sydney then back to the Wellington exchange, creating a problem with slowdown in service delivery during busy hours.
TrueNet recomends you compare ISPs based on webpage download speeds rather than peak speed. To make the comparison fairer, hosted webpages are popular sites that all ISPs have equal access to.
Comparing performance by time-of-day is important as it shows the service degradation when everyone is using the Internet during the evening hours of 8pm to 10pm. TrueNet uses the best quartile of a pair of Auckland and Wellington tests to calculate the median NZ results by hour, over the month for each monitored connection. We take the average of all median results with each ISP for each hour. Note: This month we have included an additional Auckland test location as testing to the main test point inadvertently ceased during the Month. Therefore, results are chosen from the best of 3 test points - Wellington and 2 x Auckland locations.
A poor result typically shows the line drop below 90%, which usually occurs in the busy period between 7pm and 10pm, i.e. if this is true, the average user for that ISP is getting less than 90% of their line capability.
Chart 4, shows a comparison of Time of Day performance of Fibre, Cable, VDSL, ADSL and Fixed Wireless. In March ADSL and VDSL remained well above the target 95% of best hour Peak Speed. Fibre is consistent this month staying above 95% of best-hour peak speed. Cable, however is quite erratic dropping down to 90% of best hour peak speed at midday, and this is evident on many of the 20 probes we measure. Fixed Wireless shows a strong, steady decline during the "waking" hours to a poor result of 70.1% of best hour peak speed.Chart 4: Fibre, Cable and Copper (DSL) File Peak Speed by ToD from NZ
Fibre & Cable Comparison - File Peak Speeds by ToD
Chart 5a - We see slightly better results than in recent months. Voyager shows a big improvement on February results, with MyRepublic showing more consistancy this month. 90% is an excellent result, we equate that to a 4 star result (see star ratings in the Glossary here), with 95% at 5 star, meaning TrueNet has no issue with these speeds.
Note: With Fibre speeds, the ability of the backhaul path to ramp-up to full speed can be compromised, we use a 2MB file which for some ISPs is more than adequate, but for others this is not enough to reach the maximum speed possible. We prefer the 2MB file because that demonstrates rampup issues, with poor results typically due to this.
Chart 5a: 100Mb/s and greater Fibre & Cable File Peak Speed by ToD from NZ
DSL Performance by Time of Day (ToD) Indexed to File Peak Speeds
|TrueNet uses an index for copper (DSL) reporting because ADSL and VDSL are sold using the phrase "as fast as your line will allow", so we find the maximum speed, and compare each result with that maximum.|
High Speed Copper (VDSL) File Peak Speed by ToD from NZ
Chart 5b - 2 Degrees and Spark show exceptional (5 star) results in March, consistently above 99% of peak speed (see star ratings in the Glossary here).
Voyager is showing a sharp dip at 6pm - the only ISP to dip under 90% (4 star) this month. We are seeking more Voyager volunteers, which may improve the sharp variations, but is unlikely to change the trend.Chart 5b: VDSL File Peak Speed by ToD from NZ
Low Speed Copper (ADSL) File Peak Speed by ToD from NZ
Chart 5c - ALL ISP's show great performance this month consistently above 95% with the only exception being Orcon, which is partly impacted by the sample size reducing, we are also seeking more Orcon ADSL volunteers.Chart 5c: ADSL File Peak Speed by ToD from NZ
Fixed Wireless File Peak Speed by ToD from NZ
Chart 5d - Skinny has the better performance this month, but both ISP's experience the relentless decline to the 8-9pm low point.Chart 5d: Fixed Wireless File Peak Speed by ToD from NZ
To compare international performance, TrueNet downloads files every hour to measure Peak Speeds from Dallas and Sydney.
The File Peak Speed reported is compared to the maximum NZ peak speed, ie this is the potential peak speed for each connection.
We ensure the download file is not held in New Zealand (cached), so that the test truly measures international performance. Tests are based on a 2MB file size.
High Speed Fibre & Cable - Australian File Peak Speed by ToD
The connection between the ISP and the Sydney peering exchange is critical to many services based in Australia, and the rest of the world. Many services are "cached" in Sydney on servers that are supplied by thrid parties to improve download speeds in remote places. Two examples are Apple or Microsoft software updates, which are cached by Akamai in NZ. We test by downloading our file from a server connected to the Sydney exchange. This file is random and has checks on it to ensure it is not cached in NZ, which means it is an honest measure from Sydney.
Chart 6a - In this chart, results are compared as a percentage of NZ speeds. While it would also be valid to compare as a percentage of best speed Sydney speed, we believe it is relevant to know what percentage of perceived "NZ Speed" is attained when downloading data from Australia. Care needs to be taken interpreting the results: Trustpower stands out because their mix of offered speeds (e.g. 30, 100, 200Mb/s and GigE) has a lower average NZ speed than the larger ISPs. The primary focus is inteneded to be on Time of Day variation, which is negligable for most ISPs.Chart 6a: Australian File Peak Speed for High Speed Connections (2MB file size only) by ToD
ADSL & VDSL (Copper Connections) - Australian File Download Speeds by ToD
|Copper connections (ADSL & VDSL) have a speed that is dependent on the distance between the home modem, and the exchange equipment which means that ISPs have little influence on the peak speed of each connection. To overcome this limitation, the Australia, and USA speeds are referenced to the average NZ download speed of the respective ISPs.|
VDSL (High Speed Copper) Australian File Download Speeds by ToD
Chart 6b, shows two very different ISP groups - 2Degrees and Spark are virtually identical, delivering around 94%-95% of maximum NZ speed. Orcon, Vodafone and Voyager are grouped around 60% - 65% of maximum NZ speed.Chart 6b: VDSL File Peak Speed from Sydney, Australia (2MB file size only) by ToD
ADSL (Low Speed Copper) Australian File Download Speeds by ToD
Chart 6c, ADSL providers had excellent results again in March, with 2Degrees reaching speeds equivalent to maximum NZ speed. Vodafone maintained the improvement from February, also achieving very close to 100% in this test. ADSL shows the best results of the test technologies in the Sydney file download test.Chart 6c: ADSL File Download performance from Sydney, Australia (2MB file size only) by ToD
Fixed Wireless Australian File Download Speeds by ToD
Chart 6d, shows a very similar performance from Spark and Skinny with both experiencing a gradual slowdown during daylight hours. Note this measure shows the speed as a percentage of the best NZ speed recorded regularly, ie it is most likely to be the percentage of the median Spark or Skinny Fixed Wireless NZ speed at 2-4am.Chart 6d: Fixed Wireless FIle Download performance from Sydney, Australia (2MB file size only) by ToD
|Upload speed is important to users sending large amounts of data through the Internet, or uploading files to the Cloud. TrueNet's upload test sends a 1MB file to our Wellington server, and records the results as the median of the 10 deciles measured for each download.|
By Technology / Service Upload Speed
Chart 7 - the Upload Speed performance measured for each technology is one of the few measures that remains stable over long periods of time.
The 50Mb/s Fibre upload service averages under the advertised speed, but other Fibre upload speeds are equal to, or greater than the advertised speed.
Cable 100Mb/s upload is included in these results, averaging around 31Mb/s. Note that a 1MB test file was used, hence may not have sufficient time to ramp-up depending on latency. However, the average upload of the 50Mb/s Fibre outperforms this group of Cable users.Chart 7: Upload Speed by Technology
Upload Speed - by ISP
Chart 8 - Fibre & Cable upload service speeds show small differences between the ISPs in the average speed of 20Mb/s Upload services, with each providing better-than-advertised speed. The 10Mb/s Upload services also achieve, or exceed the advertised speed. Orcon, with the largest group of 50Mb/s upload panelists in our tests, delivers around 50Mb/s. Given that the average of all ISPs is consistently around 44-45Mb/s, services from the other ISPs average well under advertised speed.
Note: The average of the Median Decile Speed is the calculated average Median speed taken from the total number of test results inside a 10 point speed range.
Chart 8: Fibre & Cable Upload Speed by ISP
TrueNet's video test chooses specific videos (This month two videos, a HD one and an identical content 4K video) The video will be cached (stored locally) by ISPs in NZ for all viewers, simply because we are downloading the video seven times a minute from 400+ different NZ addresses.
Each test downloads 10 seconds of content at a time, to fill any buffer to last 40 seconds. So on startup, the test will immediately download (at full speed) 40 seconds of content, after a time gap (to allow for playback) it will then download a second 40 seconds of content, simulating keeping the 40 second playback buffer full. This appears to be an international standard for consumer video testing.
TrueNet records buffer events by counting "Stops", where the simulated buffer would become empty. We also record Resume and Finished events. Chart 1 shows a count of the number of "Stops" (sometimes more than one in a single download attempt).
We also measure the start delay, ie the time taken for a video to start downloading so it can play. The typical period for this is so small, usually less than 1 second, that we are unlikely to continue publishing this statistic.
Buffering Events during Youtube watching
Minor buffering occurs on average for ADSL, but it is generally absent for other technologies. Our experience is that Youtube buffering problems are often unique to a single panelist rather than spread over many technologies, so if you have an issue with buffering, try these tips, check your Wifi, your Connection (eg the line to the street) or ask for help from your ISP. Note, also that only Wireless appeared to experience increased buffering during the evening peak period.Chart 9a: Buffering Events by Day Chart 9b: Buffering Events for the Month
Glossary available here
Chorus continue to fund our fixed wireless probes, these are purchased by our panelists then refunded by TrueNet for exclusive testing.