August 2016 Urban Broadband Report
The Need for Speed
Throughout August all 4 major ISPs with Fibre 100 service options (Vodafone, Spark, 2Degrees and Orcon), delivered more than 100Mb/s at all times. In addition, MyRepublic improved significantly from previous months to 89% at 9pm.
TrueNet focused on the webpage download capability for ADSL this month, because our tests showed a wide spread of results with Slingshot being the standout performer, and Flip not far behind.
Five ISPs are achieving better than 95% of maximum possible speed for ADSL, and VDSL. (ADSL - Flip, 2Degrees, Spark, Slingshot, and Orcon), (VDSL - Voyager, Spark, 2Degrees, Orcon, and Slingshot).
- Technology comparison
- Fibre & Cable
- Copper (ADSL and VDSL)
- Fibre & Cable
- Copper (ADSL and VDSL)
TrueNet is seeking many more volunteers to be panelists, especially ADSL or VDSL (ordinary copper telephone line internet access). Go here to volunteer
Summary of Performance Measures
All 4 major ISPs (Spark, Orcon, 2Degrees and Vodafone) plus Other exceeded 100% of advertised speed in August, that means for Fibre 100 all of these ISPs were delivering more than 100Mb/s at all times.
In 2014, Initial reporting of 100Mb/s Fibre by TrueNet showed that while the delivery speed of the wholesale product was promised at 100Mb/s, the retail product sold by ISPs had to be delivered at a slower speed due to internet overheads, typically we saw 94Mb/s as the maximum average speed from launch in January 2014 to May 2016. In response, Chorus published a new product called Chorus Accelerate. "Chorus Accelerate Fibre products will also be slightly over-specified for their speeds, meaning that Retail Service Providers can be confident that if they pay for 100Mbps as a wholesale product, the end user will see 100Mbps."
That's history now with all major ISPs currently delivering to our panelists an average of at better than 100Mb/s during the peak traffic hour.
TrueNet maintains the following chart every month in our "ISP Performance" trends dropdown menu above where that change is evident (plus Vodafone 100Mb/s Cable)100Mb/s evening busy hour speed trend.
MyRepublic 100Mb/s fibre has continued its improvements in Time of Day performance, now achieving 89% of advertised speed at the busiest time of day. However, the target has since moved with the Major ISPs now reaching 100% for Fibre.
It is evident in our ADSL results from TPG, that Australia still has a long way to go before they can match the Webpage download times of any of the ISPs that TrueNet reports on. Note that most NZ ISPs are quicker at downloading Australian websites than TPG , so there is no technical reason for the difference.Table 1: Summary of Performance Measures
Cable and Fibre speed comparisons show minimum hour (slowest) speed to Advertised Service Speed; ADSL & VDSL compare minimum hour (slowest) speed to the hour with maximum (as fast as your line will allow) speed. The 9pm column is the actual average speed at 9pm for each product, for ADSL and VDSL that average is taken over all ISPs because actual speed is subject to the distance between our test points and the exchange equipment - ISPs do not influence this.
Webpage Download Time
August is showing consistantly good results across the board, with very little variation by time of day for all technologies, apart from ADSL.
Responsive website browsing is valued by most Internet users, and conversely, slow-loading sites can be extremely frustrating.
TrueNet tests Internet browsing by downloading a selection of Live Webpages from NZ, Australia, and the USA, measuring the time to fully download all files on the page. These pages are changed from time-to-time. This month there are 7 NZ pages, 3 Australian, and 9 USA pages (recently adjusted to reflect survey opinions).
The charts in this section have three points for each ISP entry showing the Min. Hour (minimum time to download a webpage); the Max. Hour (maximum time to download a webpage); and the Median of all hours. The best ISP has a low median, and a short arrow showing the spread between peak and off-peak performance.
We download websites from each connection, and compare the average time taken to download them. This test replicates daily activity for many people, and we group the websites into regions, NZ, Australia and USA, so that readers can compare ISPs based on their own preferences.
Max = Busy hour,
Median = the most likely time to download a page outside the busy hour,
Min = The best time all day, ie likely very early in the morning.
Chart 1: TPG stands out this month taking more than twice the average time to download our sample of websites in the busy hour (Max). However the Vodafone spread on ADSL is very large, that means downloads in the evening busy hour take twice as long as Flip and Slingshot.Chart 1: NZ Average Webpage Download Time - in seconds
The ADSL results above are effectively a summary, with Max = the worst time to download and Min being the best. Below is a chart to show in detail how that varies through the day, much like our speed by time of day, but measured in time taken to download the webpages, congestion shows as an increase in time.Chart 2: Time of Day to download NZ websites for ADSL
|We download popular US websites from each connection, and compare the average time taken to download all websites. This test replicates daily activity for many people, but to provide the ability for readers to compare between ISPs based on their own preferences, we group the websites into regions, NZ, Australia and USA.|
Chart 3: Smaller ISPs continue to be sluggish for USA webpage downloads, in ADSL, VDSL and Fibre. Strangely, Spark 200Mb/s appears to be taking longer than Spark 100Mb/s for downloads from our popular USA websites. The best result stands out and is from Trustpower 50Mb/s fibre service.Chart 3: USA Average Webpage Download Time - in seconds
|We download popular Australian websites from each connection, and compare the average time taken to download all websites. This test replicates daily activity for many people, but to provide the ability for readers to compare between ISPs based on their own preferences, we group the websites into regions, NZ, Australia and USA.|
Chart 4: Vodafone ADSL is again taking longer in the evening than other named ISPs. The smaller ISPs in "Other" continue to take the longest for both ADSL and VDSL, but not Fibre this time.
TPG cannot compete with NZ ISPs, again coming in almost last for downloading Australian websites from Australia, compared with NZ ISPs which download Australian websites from NZ.Chart 4: Australian Average Webpage Download Time - in seconds
Speed (File Download Performance)
In the charts below, summaries of Time of Day results are comprised of very large numbers of test observations. For each hour (eg 1am) TrueNet collects a total of approximately 25,000 test run observations during the month.
For TrueNet's speed tests each panelist's probe regularly downloads a 1MB or 2MB file from Auckland, Wellington, Dallas and Sydney. The faster the connection, the larger the file we need to download to ensure that the maximum speed is reached during our test. 100Mb/s connections easily reach full speed before 2MB of data is downloaded. Slower connections can test accurately with much smaller files. Our tests on Satellite, Wireless and ADSL can tolerate a file size of just 300k, although these days with higher caps we can use a 1MB file.
International tests take the result from each test run from Dallas or Sydney.
Update 12 October: Many tests were updated mid September to 2MB or 5MB, so new reports will be based on 2MB tests for faster connections to allow for rampup.
Comparing performance by time-of-day is important as it shows the service degradation when everyone is using the Internet during the evening hours of 8pm to 10pm. TrueNet uses the best of a pair of Auckland and Wellington download tests to calculate the median NZ results by hour over the month for each monitored connection. We take the average of all median results with each ISP for each hour.
A poor result typically shows the line drop below 90%, which usually occurs in the busy period between 7pm and 10pm, i.e. if this is true, the average user for that ISP is getting less than 90% of their line capability.
100Mb/s Fibre and 50Mb/s Cable stay above advertised speed all the time.
200Mb/s Fibre delivered an average 180Mb/s or more across all hours of the day, however this shows in Chart 5 below as about 90% all day.
VDSL, ADSL and 30Mb/s Fibre all reach very close to their advertised speeds at all times as expected for DSL, it is impossible for DSL to exceed maximum speed.Chart 5: Fibre, Cable, and Copper (DSL) File Download Speed
Fibre & Cable Comparison - File Download Speeds
Chart 6: The 100Mb/s services have clustered together in August (a good thing). Vodafone Cable, and in particular My Republic both showed very much improved evening performance compared to the past few months. The results by Time of Day (ToD) for Voyager and MyRepublic vary more than others because the sample size is smaller.
Chart 6: 100Mbs Fibre & Cable Performance from NZ
Chart 7: Lower speed Fibre & Cable are very stable by ToD in August. Vodafone Cable 50Mb/s, and 2Degrees 30Mb were above advertised speed, while Trustpower 50Mb/s, and 30Mb/s "Other ISPs" averaged a bit under their advertised speed.
Chart 7: Other Fibre & Cable Performance from NZ
DSL Performance by Time of Day (ToD) Indexed speeds
|We use an index for copper reporting because ADSL and VDSL are sold using the phrase "as fast as your line will allow", so we find the maximum speed, and compare each result with it.|
High Speed Copper (VDSL) File Download Speed
The VDSL download performance by ToD was tightly grouped again this month, with the exception of Vodafone in the evening.Chart 8: VDSL File Download Speed by Time of Day
Sample size for Slingshot and Voyager is smaller per datapoint compared to the other ISPs, which can result in more variation
Low Speed Copper (ADSL) File Download Speed
Chart 9: There is consistent Time of Day performance for ISP's ADSL services. As above, Vodafone is down a small amount in the evening busy period.
We need more ADSL volunteers! Go here to assistChart 9: ADSL File Download Speed by Time of Day
Sample size for TPG is smaller per datapoint based (on tests to Sydney rather than Auckland or Wellington), resulting in greater variation.
TrueNet tests File Download Speeds from our overseas servers to measure performance from our international neighbours.
We ensure the download file is not held in New Zealand (cached), so that the test truly measures international performance.
High Speed Fibre and Cable Time of Day
We measure the speed of a file downloaded from our Sydney and Dallas Servers. Our tests have been based on 1MB files, assuming correctly that most connections are able to reach the maximum speed their line will allow within the time taken to download a 1MB file.
However, we have identified that Ramp-up, the time taken to reach the fastest speed possible on a line, varies by ISP and technology. Ramp-up only affects small file downloads where the maximum speed cannot be reached, this is common for websites, and mostly explains why lines with speeds greater than about 20Mb/s (See Charts 1, 3 and 4 above for Fibre, Cable and the best VDSL) do not improve website downloads.
Fibre lines on 100Mb/s or more take almost exactly 1MB to reach 100Mb/s on the best ISP lines, we measure the speed of each quarter of a file as it is downloaded, so our tests show only the last quarter measured as the best possible speed. Therefore, our current test results are published using the fastest speed reached in a 1MB file download.Chart 10: Australian download speeds for High Speed Connections (Speed Limited to that reached within a 1MB file)
Note Chart 10 shows the maximum speed reached during a 1MB file download, the speed may go on to higher levels with more data ( i.e. using a bigger file size).
Our opinion is that downloading large files from overseas is rare, most large files are stored as copies in NZ, so the main files are going to be website files with pictures in the range 200kB to 1MB. We plan to do further research into file sizes in websites. Meanwhile, we have decided to increase our international file size to 2MB.
This section reviews our research to develop a fair and reliable test.
What is the speed of download? Is it the speed a file eventually gets to in a 100MB download for example, or is it a speed that is reached in a specified time? This is a difficult question that is often decided as being the fastest file download speed achieved when downloading a massive file, which is often not in common use. We normally use a 1MB file, and that has been sufficient until faster connections became more prevalent.
Downloading a file starts slowly, and as more of the file is downloaded it increases speed until the maximum speed is detected by the equipment at each end, this process is called Ramp-up. Ramp-up tends to take longer the further the distance the file travels, however Australia is only about twice the distance from Auckland as Dunedin, an important consideration for those ISPs with services based from Auckland only.
We were approached by Spark, concerned that our measurement of Australian downloads was using a file that is too small (1MB) for Ramp-up. We checked our results, and found that slower service speeds of 50Mb/s and below are fine. However, Spark lines definitely do not reach full speed by the time our current test finishes for service speeds of 100Mb/s or more. Almost all copper lines do not exceed 50Mb/s so are reporting accurately.
In order to research what file is best to use we compared the Ramp-up on three different sizes -1, 2 and 5MB files. We found results depend on the ISP, some ISPs reach the peak speed during smaller file size downloads and some do not. Below are the comparison of four key ISPs download averages over a large sample of files with the three different file sizes tested. As a result of our testing we are choosing to use the 2MB file for future downloads because that will ensure the best testing on the limited data caps that our volunteers are prepared to provide.Chart 11, Our previous standard 1MB file Ramp-up
It is evident from Chart 11 that during the last quarter of the 1MB file download, the best quartile speed is short of 100Mb/s, and it may still be increasing in speed. Vodafone are able to Rampup quicker than other ISPs, and achieve 80Mb/s average over that last quarter, while Spark is trailing behind at 50Mb/s. The last quarter is the average speed of the whole of the last quarter, so if the file does reach 100Mb/s before finishing it may not be measured if some of that final quarter was slower.
Chart 12: We doubled the file size, to see if speeds reach the capacity of the line and found that they do for Vodafone, but maybe not for the other three ISPs, with Spark only just reaching 50% of line speed.
For the 2MB file, the chart once again shows Vodafone's faster Rampup with 100Mb/s panelists averaging almost 90Mb/s from Sydney in the last quartile. Although the other ISPs are still increasing in speed they fail to reach the 100Mb/s before the file has completed downloading.
Chart 12, A new 2MB test file Ramp-up
Chart 13: We tried a 5MB file, ie five times our standard file size. While Ramp-up is still occuring on 2Degrees and Spark, it clearly finishes on Orcon and Vodafone, well before the first 2MB is downloaded.
With this much larger 5MB file, we can see that Vodafone download average reaches close to 100Mb/s, and maybe 2Degress may get there. Orcon appears to have reached a cap of a bit over 80Mb/s. 2Degrees & Spark may still be increasing in speed. Note 2MB is 40% of a 5MB file.Chart 13, A new 5MB test file Ramp-up
ADSL & VDSL (Copper Connections) - File Download Speeds
|Copper connections (ADSL & VDSL) have a speed that is dependent on the distance between the home modem, and the exchange equipment which means that ISPs do not have any influence on the peak speed of each connection. To overcome this limitation, the Australia, and USA speeds are referenced to the average NZ download speed of the respective ISPs. The speed reported is from the 8pm-10pm evening period.|
Having found that slower speeds are not affected by the smaller file, our results for VDSL using a 1MB file size are included as normal, however we have changed the file for future reporting to a 2MB file.
Chart 14: Slingshot has been re-added to the chart and shows the most pronounced evening congestion, although with a smaller sample size, variation is pronounced. Orcon and Slingshot are Vocus-group brands both showing ToD variation; also refer to ADSL below.Chart 14: VDSL ToD performance from Sydney Australia using a 1MB file
Chart 15: Flip, Orcon, Slingshot (Vocus group) each have a dip in evening speeds. Just the slightest dip in the Spark and Vodafone services is evident.
TPG shows results of downloads from NZ, indexed to the best download from Australia for TPG. ie a reflection of the NZ ISP results.Chart 15: ADSL ToD performance from Sydney Australia 1MB file
|Upload speed is important to users sending large amounts of data through the Internet, or loading files to the Cloud. TrueNet's upload test sends a 1MB file to our Wellington server, and records the results using a similar method to the download tests, but measured from Wellington only.|
By Technology / Service Speed
Upload speeds are generally very consistent in our testing and after years of testing we expect to see the upload speed advertised being measured almost every time.
Chart 16: The Upload speed of the main Technologies and service speeds remains consistent with previous months. Fibre tends to average over the advertised speed, with the exception of 50Mb/s Uploads. 10Mb/s Cable and VDSL typically average under 10Mb/s.Chart 16: Upload Performance by Technology
Upload Speed - by ISP
The average Fibre & Cable upload speed by ISP and Service Speed is shown in Chart 17.
All ISP 20Mb/s services averaged above advertised, except Trustpower, which was just under 20Mb/s.Chart 17: Fibre & Cable Upload Speed by ISP