July 2017 Urban Broadband Report

Try TrueNet Comparison Tools

TrueNet now has two Interactive tools (Table 1 & 2 below) to assist in the choice of the best ISP for you.  

Table 1 helps you to compare technologies by sorting based on your preference, eg Surfing Speeds which show how fast each technology can download our sample of webpages; or Peak Speeds showing the file download speed of each technology at 9pm. 

Table 2 shows a more detailed analysis which helps you to sort based on technology, NZ website speed, ISP, or price within each technology.  

Try them out.

As with previous month's the speed differences for NZ websites continue to show a wide spread.  All ISPs are able to deliver similar speeds for each technology, the surfing charts simply show how fast each service is when downloading a range of popular websites in NZ and overseas.

Index

Webpage Average Speed

  • NZ
  • USA
  • AUS

NZ File Peak Speed by Time of Day (ToD)

  • Technology comparison 
  • Fibre & Cable
  • Copper (ADSL and VDSL)
  • Fixed Wireless

International File Peak Speed

  • Fibre & Cable
  • Copper (ADSL and VDSL)
  • Fixed Wireless

Upload Median Decile Speed

Video test results

Glossary

Summary of Performance Measures

 

Notable features of our table comparing technologies: 

  • Fixed Wireless has a faster peak speed than ADSL, but is about the same for surfing speed.  This is expected due to the higher levels of Latency on Fixed Wireless - see https://truenet.nz/story/2017/03/first-report-fixed-wireless-technology, Chart 7
  • Peak speeds at 9pm are very close to advertised peak speeds
  • Fibre and Copper (ADSL & VDSL) are all achieving better than 95% minimum/maximum peak speeds
Table 1: Technology Performance Summary

Notable features of our table comparing ISPs:

  • NZ Surfing Speed fibre results are spread from 31.7Mb/s (Voyager) to the best at 41.2Mb/s (Orcon, despite being the lowest priced in our table)
  • International Surfing Speed fibre results are a completely different ranking but also a big spread, with Orcon top at 35.5Mb/s, and Trustpower last at 24Mb/s.
  • If you select DSL, then Voyager VDSL service is actually slower for NZ surfing speeds than some ADSL speeds.  Similarly with International surfing speeds, but this time Slingshot VDSL is slower than some ADSL speeds.
  • Price is not related to quality, the best ISP in any technology is not always the most expensive for unlimited naked service.

TrueNet's second table explores performance with unlimited, naked long-term price by ISP for each of the technologies.  

To explore this table, first type into the search bar to filter the table -(eg AD will get only ADSL or Fi will compare Fixed Wireless and Fibre, sp will get all Spark services and so on.) Then click on any header in the blue bar to sort each column.- (eg prices will show ascending or descending price).  HINT: Right click on the table and select "Open frame in new tab" then select 25 lines to see the full table (Edit - This does not work in Chrome).

Table 2: Summary of Performance Measures 

Note: Webpage average speed is the webpage size divided by the time to download.  Webpages are often sent from the owners site in many small files, which means the speed is not as fast as that achieved during a single file download (Peak Speed).  

A simple analogy to explain Peak vs Average speed, assumes you have a very fast car like a Ferrari

Peak speed is the speed reached during a time trial on a track - say 100's of km/hr
Average Speed (webpage surfing speed) is the average speed you would achieve in Featherston street, Wellington, after navigating lights, say 30km/hr

Peak Speed Ratio compares the slowest hour to the fastest hour (as fast as your line will allow) in the File download Peak Speed test.  

The Price column is the price of Unlimited, Naked services for the given ISP and technology. A full list of ISP Prices can also be seen here.

 

Webpage Surfing (Webpage Average Speed)

Methodology

Responsive website surfing is valued by most Internet users, and conversely, slow-loading sites can be extremely frustrating.

TrueNet tests Internet surfing by downloading a selection of Live Webpages from NZ, Australia, China, UK and USA; measuring the size of each webpage, and the time to fully download all files on the page.  From this we calculate the speed.  These pages are changed from time-to-time so the actual average webpage speeds cannot be compared between months.  This month there are 15 NZ pages, 3 Australian, 1 UK, 2 China and 10 USA pages.  DO NOT COMPARE SPEEDS BETWEEN COUNTRIES because average webpage speed is dependent on the actual webpages downloaded, not the ISPs.  The list of test webpages is in the Glossary.

We download webpages from each connection, and compare the average speed achieved for each download.  This test replicates daily activity for many people, and we then group the webpages into NZ and International, so that readers can compare ISPs based on their own preferences.

Fibre webpage average speeds show no material difference based on Fibre speeds sold, so we show them inclusive of 100Mb/s, 200Mb/s, and Full Speed (sometimes called GigE) services.

NZ Webpage Surfing

Technology Comparison - Webpage Average Speed of Selected NZ Webpages by Time of Day.

Please click on the different time periods in the interactive chart (Peak, Business, Off-Peak) to see the performance changes during each period.

200Mb/s Cable sold by Vodafone in Wellington and Christchurch as FibreX is clearly slower than Fibre, even during off-peak hours. 

The Average Surfing Speed difference between 100 and 200Mb/s fibre is minimal.  The surprise is that Fixed Wireless (sold by Spark, Skinny and Vodafone) is about the same surfing speed as ADSL during the peak evening periods - but check the differences between ISPs in our comparison later in this report.

Chart 1a: Technology Comparison - Webpage Average Speed of Selected NZ Webpages by Time of Day

Fibre and Cable (FibreX)

Orcon and MyRepublic Average Speed were again best in July, with most of the other Fibre ISPs in a tight grouping (Chart 1b).

Bigpipe Fibre and Vodafone Cable both had average speed 15% below the main ISP grouping.  Vodafone Fibre at about 35Mb/s is faster than Vodafone Cable (FibreX) at about 26Mb/s during peak hours.

Chart 1b: Fibre & Cable - Webpage Average Speed of Selected NZ Webpages by Time of Day

Copper webpage surfing speed is limited by the speed of internet over copper (a standard telephone line connection). VDSL has an average peak speed of 40Mb/s and ADSL about 10Mb/s, but there is a wide range of speeds about these averages.  We do not report the actual average speed of each ISP on these technologies, because that is dependent on the location of our panelists, not the ISPs network.  The speed at any time of day, relative to the maximum speed possible, is however within the control of the ISP on copper.

 

Copper (VDSL and ADSL)

VDSL (Solid Lines) - NZ Webpage surfing speeds show a large spread between ISPs, 2Degrees and Spark achieving the best speeds with no variation by time of day.  The worst VDSL supplier, Slingshot at about 12Mb/s contrasts with the best 2Degrees at about 19Mb/s.

ADSL (dotted lines) - Average speeds for ADSL remain very consistant at all times of day.  Orcon ADSL is the slowest at about 7Mb/s while other ISPs are about 10Mb/s 

Chart 1c: Copper - Webpage Average Speed of Selected NZ Webpages by Time of Day

Fixed Wireless: Vodafone had the best speed in July.  As previously reported, Fixed Wireless speeds are at a peak in the early morning, then tend to roll off during the day to a low during the evening peak period.  The NZ Webpage results in Chart 1d show that Skinny / Spark performance is relatively constant during working hours, while Vodafone Wireless speed rolls-off progressively during the day. This difference is seen in Internet surfing results, but is not seen in Peak Speed test results Chart 4d.

Chart 1d: Fixed Wireless - Webpage Average Speed of Selected NZ Webpages by Time of Day

 

International Webpage Surfing

We download popular Webpages from Australia, China, US, and the UK, and compare the average time taken to download all webpages.  This test replicates the daily activity of many people accessing webpages from various countries, and includes the impact of network design at an ISP, where NZ caching, international links and other enhancements improve international website performance.

Fast Broadband: Spark, MyRepublic, and Orcon are well above other ISPs for the selected Webpages. Tight grouping of remaining ISPs is apparent, except for TrustPower and Vodafone Cable which sit last at about 24Mb/s. Vodafone Cable has a dip in the evening busy hour. Note - these results cannot be compared directly with June results especially because extra Webpages and Countries have been included.

Chart 2a: Fibre & Cable - Webpage Average Speed of Selected International Webpages by Time of Day

Copper: VDSL speeds vary from almost 16Mb/s, down to ADSL-like speeds, however the relative order of ISPs has not changed from June.

ADSL  Spark had the best speed again, as was the case for VDSL. There is no evidence at all of Time of Day issues in these Copper results.   TrueNet assumes this is a new "normal" due to the number of connections upgrading to fibre and leaving the copper network unloaded.

Chart 2b: Copper - Webpage Average Speed of Selected International Webpages by Time of Day

Fixed Wireless - Vodafone maintains a small, consistent speed gap over Skinny & Spark (which have virtually identical results, despite their price difference).

Chart 2c: Fixed Wireless - Webpage Average Speed of Selected International Webpages by Time of Day

 

Peak Speed

For TrueNet's peak speed tests each panelist's probe regularly downloads a 2MB and/or 5MB file from Auckland, Wellington, Dallas and Sydney.  We identify the fastest quartile/decile as the Peak Speed.  The faster the connection, the larger the file we need to download to ensure that the maximum speed can be reached during our test for ISPs. 100Mb/s connections easily reach full speed before 2MB of data is downloaded from NZ or Australia.  Slower connections can test accurately with much smaller files. 

Comparing performance by time-of-day is important as it shows the service degradation when everyone is using the Internet during the evening hours of 8pm to 10pm.  A poor result typically shows the line drop below 90%, which usually occurs in the busy period between 7pm and 10pm, i.e. if this is true, the average user for that ISP is getting less than 90% of their line capability.  

We set the "Zero" line at 70% because panelists with less than 70% report difficulty using the internet.

New Zealand Peak Speed

NZ peak speed can be influenced by how well ISPs peer at the internet exchanges, where they connect to TrueNet's server provider for our file test. Without effective peering a file can travel from a TrueNet test probe located in Wellington, through to Auckland, on to Sydney then back to the Wellington exchange, creating a problem with slowdown in service delivery during busy hours. 

TrueNet however, recommends you also compare ISPs based on webpage surfing speeds rather than peak speed alone.  To make the surfing comparison fairer,  hosted webpages are popular sites that all ISPs have equal access to.

 

NOTE

TrueNet uses the best quartile of a pair of Auckland and Wellington tests to calculate the median NZ results by hour, over the month for each monitored connection.  We take the average of all median results with each ISP for each hour.  

Technology Comparison

Chart 3 compares Peak Speed of technologies in Off-peak, Business, and Peak hour periods. Cable 200Mb and Fixed Wireless services had the most noticeable change in speed between Peak and Off-peak periods.  Note that this comparison is deceptive, your line speed is being measured here but that does not necessarily correspond with your experience of the internet, Webpage Surfing is far more likely to correspond to your experience

Chart 3: Fibre, Cable and Copper (DSL) Peak Speed by Time of Day  from NZ

 

Fast Broadband: - Peak Speeds by Time of Day

Fast Broadband connections with an advertised speed of 100Mb/s are shown in chart 4a. Slingshot and MyRepublic had the top results, however all ISPs in the chart except TrustPower had peak speeds above the advertised speed of 100Mb/s. There are no indications of Time of Day congestion.

Chart 4a: 100Mb/s and greater Fibre & Cable File Peak Speed by Time of Day from NZ

DSL Performance by Time of Day (ToD) Indexed to Peak Speeds

TrueNet uses an index for copper (DSL) reporting because ADSL and VDSL are sold using the phrase "as fast as your line will allow", so we find the maximum speed, and compare each result with that maximum.  Hence every line reaches 100% at some hour.  

The Copper network is becoming "deloaded" as more users upgrade to Fibre.  This generally means that performance is excellent.  Where once we hoped that ISPs could achieve 90%, we now see most doing better than 95%.

High Speed Copper (VDSL) Peak Speed by Time of Day from NZ

VDSL Time of Day performance was again excellent in July, with all ISPs above 95% of best hour speed during the month.

Chart 4b: VDSL File Peak Speed by Time of Day from NZ

Low Speed Copper (ADSL) File Peak Speed by Time of Day  from NZ

ADSL has for several months shown excellent Time of Day performance. Most ISP services were at or above 98% of best hour speed.

Chart 4c: ADSL File Peak Speed by Time of Day from NZ

Fixed Wireless Peak Speed by Time of Day from NZ

Fixed Wireless has larger variation in Peak Speed over the day than other technologies, though results have slowly been improving. Skinny had better performance in our tests in July, with Spark and Vodafone about the same. However, the peak hour result improved compared to June, and Skinny's result was up from 76% at 8pm (June) to 84% (July). 

Chart 4d: Fixed Wireless File Peak Speed by Time of Day  from NZ

International Peak Speed

To compare international performance, TrueNet downloads files every hour to measure Peak Speeds from Sydney.  

We ensure the download file is not held in New Zealand (cached), so that the test truly measures international performance.   

The connection between the ISP and the Sydney peering exchange is critical to many services based in Australia, and the rest of the world.  Many services are "cached" in Sydney on servers that are supplied by third parties to improve download speeds in remote places.  Two examples are Apple or Microsoft software updates, which are cached by Akamai in NZ.  

We test by downloading our file from a server connected to the Sydney exchange. This file is random and has checks on it to ensure it is not cached in NZ, which means it is an honest measure from Sydney.

Fast Broadband - Australian Peak Speed by Time of Day

Peak Speed from Australia for 100Mb/s Fibre connections are shown in Chart 5a. There are a group of 6 ISPs that are roughly comparable between 85 and 95Mb/s, however Voyager and MyRepublic speeds are below 80Mb/s.

Chart 5a: Australian File Peak Speed for High Speed Connections (2MB file size only) by Time of Day

Copper Connections - Australian Peak Speeds by Time of Day

Copper connections (ADSL & VDSL) have a speed that is dependent on the distance between the home modem, and the exchange equipment which means that ISPs have little influence on the peak speed of each connection.

VDSL (High Speed Copper) Australian Peak Speeds by Time of Day

Australian Peak speed needs to be compared to the best NZ Peak speed by ISP due to the influence of distance from the exchange, which has a direct affect on Peak Speed.  We show the Australian Peak speed as a percentage of the NZ Peak speed (Chart 5b).  

2Degrees and Spark achieved close to NZ Peak speed during July (90-95%), and there is a slight dip in 2Degrees results in the evening busy period. Orcon, Slingshot, Vodafone and Voyager results remain clustered around 60% of best hour NZ Peak Speed.

Chart 5b:  VDSL Peak Speed from Sydney, Australia (2MB file size only) by Time of Day

ADSL (Low Speed Copper) Australian Peak Speeds by Time of Day

ADSL - Autralian Peak speeds are very close to NZ Peak speed for most ISPs. However, July results show Slingshot well below the other ADSL services.

Chart 5c: ADSL peak speed performance from Sydney, Australia (2MB file size only) by Time of Day

Fixed Wireless: Australian Peak Speeds by Time of Day

Fixed Wireless Peak Speed from Australia are also compared to the NZ Peak speed because the actual speed of each connection is limited by, and different for each Fixed Wireless user, usually due to the distance from the nearest tower, but also by the location of the modem within the house. Peak Speed from Australia for Skinny, Spark , and Vodafone as a percentage ot their NZ speeds is roughly equal, with results also similar to those in June.

Chart 5d: Fixed Wireless FIle Download performance from Sydney, Australia (2MB file size only) by Time of Day 

Upload Median Decile Speed - Performance

Upload speed is important to users sending large amounts of data through the Internet, or uploading files to the Cloud.  TrueNet's upload test sends a 1MB file to our Wellington server, and records the results as the median of the 10 deciles measured for each download.

By Technology / Service Upload Speed

Upload Speed Performance of monitored Technologies has been stable over long periods of time. Results have shown increases in VDSL - up a small amount again in July.

The 50Mb/s Fibre upload service averages slightly under the advertised speed at 46Mb/s, but other Fibre upload speeds are equal to, or greater than the advertised speed.

Cable results have been separated into 20Mb/s and Full Speed Upload Services. Cable Full Speed upload averaged 33Mb/s, and the 20Mb/s service was at advertised speed. 

Chart 6: Upload Speed by Technology

Upload Speed - by ISP

Fast Broadband upload speeds are compared in Chart 7.   20Mb/s service results vary between 21 and 24 Mb/s. Orcon 50Mb/s was right around advertised speed.

Chart 7: Fibre & Cable Upload Speed by ISP

Fixed Wireless Upload performance by Time of Day is shown in Chart 8. Performance of each Fixed Wireless ISP is similar, with Spark marginally better by Time of Day performance.

Chart 8: Fixed Wireless Upload Speed - Time of Day

Video Test Results

TrueNet's video test chooses specific videos (Two videos, a HD one and an identical content 4K video)  The video will be cached (stored locally) by ISPs in NZ for all viewers, simply because we are downloading the video seven times a minute from more than 400 different NZ locations.

Each test downloads 10 seconds of content at a time, to fill any buffer to last 40 seconds of viewing.   So on startup, the test will immediately download (at the connection's full speed) 40 seconds worth of content, after a time gap (to allow for playback) it will then download a second 40 seconds of content, simulating keeping the 40 second playback buffer full.  This appears to be an international standard for consumer video testing.

TrueNet records buffer events by counting "Stops", where the simulated buffer would become empty.   We also record Resume and Finished events.   Chart 1 shows a count of the number of "Stops" (sometimes more than one stop in a single download attempt).

We also measure the start delay, ie the time taken for a video to start downloading so it can play.  The typical period for this is so small, usually less than 1 second, that we have ceased publishing this statistic.

Buffering Events during Youtube watching

In July, Fixed Wireless was the only technology with much evidence of buffering. There were very low levels of buffering in other technologies on specific days, but otherwise none.  The background buffering rate for Fixed Wireless increases from Off-peak to Business Hours, to Peak Hours (17%).

For technologies other than Fixed Wireless, our experience is that Youtube buffering problems are often unique to a few panelists rather than spread throughout the technologies, so if you have an issue with buffering, try these tips - check your Wifi, your Connection (eg the line to the street) or ask for help from your ISP.

Chart 9a: Buffering Events by Day

Buffering during Peak hours or after 6pm each night, is most pronounced in Fixed Wireless, hitting 17%

Chart 9b: Buffering Events by Time of Day

Note (edit 7 Sept 2017): All Fixed Wireless data collection is supported by contract with Chorus.  All other technologies are supported by contract with the Commerce Commission.  Analysis of the data and the production of reports are the work of TrueNet.

Comments

Athlonite's picture

where are the gigabit connection results

john's picture

We call it Full Speed or FS. We cannot see any difference in performance compared to 200mbps so group them together for webpage speeds. We do not have the gear to accurately measure peak speed so do not report it.

Subscription

Subscribe to our mailing list

* indicates required



Email Format

View previous campaigns.