October 2016 Urban Broadband Report
NZ ISPs lead a major Australian ISP
Our Australian based TPG panelists lines on ADSL in Melbourne and Sydney are worse at downloading Australian websites than any NZ ISP we measure.
We are focussing this month a lot more on website download performance than speed. Speed is now sufficiently reliable in NZ that the total user experience is now the critical measure, and we are investigating more measurements to demonstrate the differences between ISPs.
The ADSL website performance range continues, with Slingshot panelists taking just over half the time of Vodafone panelists to download the NZ websites.
Please consider Volunteering as a panelist, we need more panelists - FAQ at truenet.nz/faq
- Technology comparison
- Fibre & Cable
- Copper (ADSL and VDSL)
- Fibre & Cable
- Copper (ADSL and VDSL)
Summary of Performance Measures
The major point we noticed this month is the gap opening up between NZ ISPs and our sample Australian ISP TPG. This is despite the impact of the distance across the Tasman.
To make it more difficult for Australia, our ADSL and VDSL average speeds (over 200 panelists with more than 150,000 tests) improved to 11Mb/s for ADSL, while VDSL improved to 35Mb/s average. With the increasing number and speed of Fibre and VDSL connections the NZ average speed is approaching or exceeding the Australian "target" of 25Mb/s.
Small increases in average VDSL and ADSL 9pm Speed. The USA Live Webpage results returned to typical figures after unusual results in September.Table 1: Summary of Performance Measures
*Note: Cable and Fibre speed comparisons show minimum hour (slowest) speed to Advertised Service Speed; ADSL & VDSL compare minimum hour (slowest) speed to the hour with maximum (as fast as your line will allow) speed. The 9pm column is the actual average speed at 9pm for each product, for ADSL and VDSL that average is taken over all ISPs because actual speed is subject to the distance between our test points and the exchange equipment - we do not compare ISPs directly on speed because distance sets DSL speeds, not the ISPs.
Website download times are a direct measure of many internet features, speed, latency and DNS lookup times. This month NZ ISPs are continuing to show much better performance than the samples we have in Australia (we have a small panelist group in Sydney and Melbourne on TPG and are trying to get a few more on Telstra) TPG is a lot worse than all NZ ISPs for downloading NZ websites, we may expect that due to downloading across the Tasman, BUT TPG are worse at downloading Australian and USA websites than any NZ ISPs we measure.
Responsive website browsing is valued by most Internet users, and conversely, slow-loading sites can be extremely frustrating.
TrueNet tests Internet browsing by downloading a selection of Live Webpages from NZ, Australia, and the USA, measuring the time to fully download all files on the page. These pages are changed from time-to-time. This month there are 7 NZ pages, 3 Australian, and 4 USA pages.
The charts in this section have three points for each ISP entry showing the Min. Hour (minimum time to download a webpage); the Max. Hour (maximum time to download a webpage); and the Median of all hours. The best ISP has a low median, and a short arrow showing the spread between peak and off-peak performance.
We download websites from each connection, and compare the average time taken to download them. This test replicates daily activity for many people, and we group the websites into regions, NZ, Australia and USA, so that readers can compare ISPs based on their own preferences.
Max = Busy hour,
Median = the most likely time to download a page outside the busy hour,
Min = The best time all day, ie likely very early in the morning.
The general trend across technologies and speeds is similar to last month. Vodafone ADSL median result was much closer to the other ISPs this month, while the Australian provider TPG had longer web page download times than in September. As a reference, note Spark 200Mb/s Fibre has a slighter faster webpage download time than Spark 100Mb/s fibre, then compare results with the USA and Australia Webpage Charts to follow.Chart 1: NZ Average Webpage Download Time - in seconds
|We download popular US websites from each connection, and compare the average time taken to download all websites. This test replicates daily activity for many people, but to provide the ability for readers to compare between ISPs based on their own preferences, we group the websites into regions, NZ, Australia and USA.|
September results showed longer than normal webpage download times to the sample US websites due to one website having issues, however that situation was resolved by the owner, and is now showing more typical webpage downloading times. Test results show that faster technologies do not necessarily have any impact if a website has delivery issues.
Chart 2: Fibre 30Mb/s and 100Mb/s services yield similar results, with Spark 200Mb/s appearing to take longer than the 100Mb/s service, as reported in August. Vodafone 100Mb/s Cable is conspicuous due to longer webpage download times than Fibre100Mb/s services - as well as more spread between max and min times. Vodafone is currently updating its technology to allow higher speeds, however this has not translated into improvements in website download times.Chart 2: USA Average Webpage Download Time - in seconds
Not all websites respond equally. This chart of Alternate US websites responds with similar website download times irrespective of technology. The results are dominated by Netflix and Vice websites. There is either limited connectivity to these sites, or the speed accessed when browsing may be intentionally limited, we suspect the latter.Chart 2A: Alternate websites - average download time (Netflix, Outbrain and Vice)
We have seen similar effects in the past with Youtube, where there is a specific max speed any probe achieved (approx 30Mb/s), and, this speed has been reduced over time.
|We download popular Australian websites from each connection, and compare the average time taken to download all websites. This test replicates daily activity for many people, but to provide the ability for readers to compare between ISPs based on their own preferences, we group the websites into regions, NZ, Australia and USA.|
TrueNet tests show normal results for Australian Webpage tests this month. There is very little sign of contention (min - max spread), with only small spreads in TPG, 2Degrees and Slingshot ADSL. TPG is a major Australian ISP, but their performance downloading Australian websites (from Australia), demonstrates how far behind NZ they are, with all NZ ISPs having a far better performance for downloading the same Australian websites. Vodafone ADSL website performance is worse than all other ISPs.
Note that for the Australian webpages, the Spark 200Mb/s service is almost identical to the 100Mb/s service.Chart 3: Australian Average Webpage Download Time - in seconds
Speed (File Download Performance)
NZ speeds are all good and hardly worth commenting on, so we simply describe the charts.
For TrueNet's speed tests each panelist's probe regularly downloads a 1MB or 2MB file from Auckland, Wellington, Dallas and Sydney. The faster the connection, the larger the file we need to download to ensure that the maximum speed is reached during our test. 100Mb/s connections easily reach full speed before 2MB of data is downloaded. Slower connections can test accurately with much smaller files. Our tests on Satellite, Wireless and ADSL can tolerate a file size of just 300k, although these days with higher caps we can use a 1MB file.
International tests take the result from each test run from Dallas or Sydney.
Update 12 October: Many tests were updated mid September to 2MB or 5MB, so new reports will be based on 2MB tests for faster connections to allow for ramp-up (the time taken to reach the fastest speed possible on a line, which varies by ISP and technology).
New Zealand File Download Speed
Comparing performance by time-of-day is important as it shows the service degradation when everyone is using the Internet during the evening hours of 8pm to 10pm. TrueNet uses the best of a pair of Auckland and Wellington download tests to calculate the median NZ results by hour over the month for each monitored connection. We take the average of all median results with each ISP for each hour.
A poor result typically shows the line drop below 90%, which usually occurs in the busy period between 7pm and 10pm, i.e. if this is true, the average user for that ISP is getting less than 90% of their line capability.
TrueNet examines all probes in each Technology or service speed. Cable & Fibre 100Mb/s average well above advertised rates, even if speed trends downward across the afternoon and evening. In contrast, 30Mb/s and 200Mb/s services are just under their respective advertised speeds.
Copper Services - VDSL and ADSL - mirror each other well, having a slight evening dip, but still excellent performance by ToD during the month.Chart 4: Fibre, Cable and Copper (DSL) File Download Speed
Fibre & Cable Comparison - File Download Speeds
Chart 5 shows good results, with all of the ISPs above advertised rate at all times of the day.Chart 5: 100Mb/s Fibre & Cable Download Speed from NZ
DSL Performance by Time of Day (ToD) Indexed speeds
|We use an index for copper reporting because ADSL and VDSL are sold using the phrase "as fast as your line will allow", so we find the maximum speed, and compare each result with that maximum.|
High Speed Copper (VDSL) File Download Speed
This is the second straight month with improvements in the evening busy period. All ISPs are achieving better than 90%, although a couple may be starting to show congestion. The serious congestion on Vodafone lines in August and September is no longer evident.Chart 6: VDSL File Download Speed by Time of Day
Low Speed Copper (ADSL) File Download Speed
Chart 7 shows very solid Time of Day performance for ADSL services.Chart 7: ADSL File Download Speed by Time of Day
TrueNet tests File Download Speeds from our overseas servers to measure performance from our international neighbours.
We ensure the download file is not held in New Zealand (cached), so that the test truly measures international performance. Test file sizes were increased to 2MB for this test during September.
High Speed Fibre & Cable Time of Day
The comparison between September and October is very different, all Fibre results now look great, sitting between 80% and 90% of their advertised speed of 100Mb/s.
Vodafone Cable Australian results are slow throughout the day, down on September where they reached over 60% during off peak hours. However, October results (over 40%) are better in the evening compared with September where they were just 36%.Chart 8: Australian download speeds for High Speed Connections (2MB file size only)
ADSL & VDSL (Copper Connections) - File Download Speeds
|Copper connections (ADSL & VDSL) have a speed that is dependent on the distance between the home modem, and the exchange equipment which means that ISPs do not have any influence on the peak speed of each connection. To overcome this limitation, the Australia, and USA speeds are referenced to the average NZ download speed of the respective ISPs.|
October shows a big improvement in relative speeds (indexed to NZ speed tests) for Spark (improving from 53% up to 67%), 2Degrees (56% to 80%), and Vodafone (41% to 65%).
Voyager performance changed the other way, dropping from 67% to 52%.Chart 9: VDSL ToD performance from Sydney, Australia (1MB & 2MB Files used)
There were no Time of Day issues in ADSL results for the Sydney download test.
Both 2Degrees (71% to 83%) and Spark (67% to 77%) improved this month, while Slingshot's relative speed dropped a little to equal the Vodafone results.
TPG continues to have better speed from our NZ test points than from the Sydney server. Unfortunately for the users, this does not translate to better speeds for NZ webpage downloads, as shown in the Webpage Download section above.Chart 10: ADSL ToD performance from Sydney, Australia (1MB & 2MB files used)
|Upload speed is important to users sending large amounts of data through the Internet, or uploading files to the Cloud. TrueNet's upload test sends a 1MB file to our Wellington server, and records the results using a similar method to the download tests, but measured from Wellington only.|
By Technology / Service Speed
October test results show subtle increases in upload speeds for 10Mb/s Fibre, and VDSL services.Chart 11: Upload Performance by Technology
Upload Speed - by ISP
Trustpower, and Vodafone 20Mb/s upload services improved to be above the advertised speed, and closer to that of the other ISPs in the chart.Chart 12: Fibre & Cable Upload Speed by ISP