September 2017 Urban Broadband Report

Focus on Fixed Wireless (AKA Mobile Data)

There is little change this month from last month, however the speed performance from Spark & Skinny on Fixed Wireless improved recently to almost match their Vodafone competitor.  We compare May 2017 (our first month of Fixed Wireless comparisons) with this month, and the change is obvious.  This change occured a couple of months ago, but is now a confirmed trend, see here

The Performance of Spark & Skinny used to be almost identical, but we are now seeing changes.  Spark lines are now better at downloading Websites, but Skinny lines on average have a faster Peak Speed matching Vodafone, ie the speed reached when downloading our 2MB files.

Index

Summary of Performance Measures

Webpage Average Speed

  • NZ
  • International

NZ File Peak Speed by Time of Day (ToD)

  • Technology comparison 
  • Fibre & Cable
  • Copper (ADSL and VDSL)
  • Fixed Wireless

International File Peak Speed

  • Fibre & Cable
  • Copper (ADSL and VDSL)
  • Fixed Wireless

Upload Median Decile Speed

Video test results

Glossary

Summary of Performance Measures

Our interactive charts enable comparisons according to your preferences. The default sort of both Table 1: ISPs, and Table 2: Technologies, prioritises Surfing on NZ websites, this includes Trademe, ISPs, and Banks amongst almost 30 sites.

TrueNet's Table 1 compares broadband performance with unlimited, naked, long-term price by ISP, for each of the technologies.  

To explore Table 1, first type into the search bar to filter the table  (eg AD will get only ADSL or Fi will compare Fixed Wireless and Fibre, sp will get all Spark services and so on.) You can check technology performance with price at this point.

Click on any header in the blue bar to sort by each column, (e.g. prices will show ascending or descending price).  

HINT for researchers: Right click on the table to select "Open frame in new tab" then select 25 lines to see the full table (Note - this does not work in Chrome).

Table 1: ISPs

Notable features of Table 2 when comparing technologies: 

Fixed Wireless has a faster peak speed than ADSL, but is about the same for surfing speed.  Higher levels of Latency on Fixed Wireless are expected to be an issue for Fixed Wireless - see our page showing Latency by technology here
Peak speeds at 9pm are very close to advertised peak speeds, but the raw data speed, or peak speed, has little impact on the overall speed experienced when surfing.
Fibre 100 and Copper (ADSL & VDSL) are all achieving better than 95% minimum/maximum peak speeds

Table 2: Technology

Note: Webpage average speed is the webpage size divided by the time to download.  Webpages are often sent from the owners site in many small files, which means the speed is not as fast as that achieved during a single file download (Peak Speed).  

A simple analogy to explain Peak vs Average speed, assumes you have a very fast car like a Ferrari.

Peak speed is the speed reached during a time trial on a track - say 100's of km/hr
Average Speed (webpage surfing speed) is the average speed you would achieve in Featherston street, Wellington, after navigating lights, say 30km/hr

Peak Speed Ratio compares the slowest hour to the fastest hour (as fast as your line will allow) in the File download Peak Speed test.  

The Price column is the price of Unlimited, Naked services for the given ISP and technology. A full list of ISP Prices can also be seen here.

Webpage Surfing (Webpage Average Speed)

Methodology

Responsive website surfing is valued by most Internet users, and conversely, slow-loading sites can be extremely frustrating.

TrueNet tests Internet surfing by downloading a selection of Live Webpages from NZ, Australia, China, UK and USA; measuring the size of each webpage, and the time to fully download all files on the page.  From this we calculate the speed.  These pages are changed from time-to-time so the actual average webpage speeds cannot be compared between months.  This month there are 15 NZ pages, 3 Australian, 1 UK, 2 China and 10 USA pages.  DO NOT COMPARE SPEEDS BETWEEN COUNTRIES because average webpage speed is dependent on the actual webpages downloaded, not the ISPs.  The list of test webpages is in the Glossary.

We download webpages from each connection, and compare the average speed achieved for each download.  This test replicates daily activity for many people, and we then group the webpages into NZ and International, so that readers can compare ISPs based on their own preferences.

Fibre webpage average speeds show no material difference based on Fibre speeds sold, so we show them inclusive of 100Mb/s, 200Mb/s, and Full Speed (sometimes called GigE) services.

NZ Webpage Surfing

Technology Comparison - Webpage Average Speed of Selected NZ Webpages by Time of Day.

Please click on the different time periods in the interactive chart (Peak, Business, Off-Peak) to see the performance changes during each period.

 

 

There is minimal difference in average surfing speed between Fibre 100Mb/s & 200 Mb/s services.  200Mb/s Cable (sold as FibreX) was again slower than Fibre including during off-peak hours.

Fixed Wireless and ADSL Surfing speeds were the same at almost half that of VDSL.

Chart 1a: Technology Comparison - Webpage Average Speed of Selected NZ Webpages by Time of Day

Fibre and Cable (FibreX)

Orcon Average speed is barely above that of Slingshot and My Republic this month, with Vodafone climbing the ranks averaging higher than Trustpower and the tightly grouped bunch of Spark, 2Degrees and Voyager

Bigpipe fibre and Vodafone Cable remain last this month.

Chart 1b: Fibre & Cable - Webpage Average Speed of Selected NZ Webpages by Time of Day

Copper webpage surfing speed is limited by the speed of internet over copper (a standard telephone line connection). VDSL has an average peak speed of 40Mb/s and ADSL about 10Mb/s, but there is a wide range of speeds about these averages.  We do not report the actual average speed of each ISP on these technologies, because that is dependent on the location of our panelists, not the ISPs network.  The speed at any time of day, relative to the maximum speed possible, is however within the control of the ISP on copper.

Copper (VDSL and ADSL)

VDSL (Solid Lines) - There is a change in order this month with Vodafone slightly ahead in NZ webpage average speeds. Spark, 2Degrees and Orcon are almost the same.  Slingshot VDSL is well back with the same speeds as the best performing ADSL (Spark), there was a small improvement in speed from Voyager VDSL.

ADSL (dotted lines) - Spark continues to have the best speed for NZ webpages again this month, and Orcon was again consistently half the speed of Spark. 

Chart 1c: Copper - Webpage Average Speed of Selected NZ Webpages by Time of Day

Fixed Wireless

Vodafone continues to lead in average surfing speed ahead of Spark and Skinny, however change is on the way, take a look at the same chart we produced in May 2017, duplicated as a second chart below.  All ISPs still show a steady drop in speed through to peak time.

Chart 1d: Fixed Wireless - Webpage Average Speed of Selected NZ Webpages by Time of Day

Compare with May 2017, Spark and Skinny are catching up.

International Webpage Surfing

Popular Webpages from Australia, China, US, and the UK are downloaded, and we compare the average speed to download all webpages.  This test replicates the daily activity of many people accessing webpages from various countries, and includes the impact of network design at an ISP, where NZ caching, international links and other enhancements improve international website performance.

Fibre & Cable

There are two distinct clusters of ISPs this month. Spark (fastest results), My Republic, Vodafone and Slingshot all improving on their previous results. In the second group, Bigpipe's average speed has also improved sitting slightly behind Trustpower, 2Degrees and Orcon, and closer to the second grouping of ISPs than to Vodafone cable as in previous months.

Chart 2a: Fibre & Cable - Webpage Average Speed of Selected International Webpages by Time of Day

Copper (VDSL and ADSL)

VDSL (Solid Lines) - Vodafone has moved up the ranks of VDSL this month, with all time of day results remaining stable. Slingshot, and in particular Voyager, show speeds on par with ADSL.

ADSL (Dashed Lines) - The relative order of ISPs are unchanged from August, Spark had the fastest Average Speed by a significant amount, while Orcon is again bringing up the rear, averaging half the speed of Spark.

Chart 2b: Copper - Webpage Average Speed of Selected International Webpages by Time of Day

Fixed Wireless

Spark for the first time since we started testing, has achieved faster speeds on Fixed Wireless than Vodafone. There is still quite a sharp time of day dip, but this is shown across all ISPs.  There is a change comparing Spark & Skinny, which normally show almost identical results, it is possible that Skinny has a different international connection to Spark.

Chart 2c: Fixed Wireless - Webpage Average Speed of Selected International Webpages by Time of Day

 

This compares with May 2017, Australia websites were shown separately then, but had a similar shape and were faster.

Peak Speed

For TrueNet's peak speed tests each panelist's probe regularly downloads a 2MB and/or 5MB file from Auckland, Wellington, Dallas and Sydney.  We identify the fastest quartile/decile as the Peak Speed.  The faster the connection, the larger the file we need to download to ensure that the maximum speed can be reached during our test for ISPs. 100Mb/s connections easily reach full speed before 2MB of data is downloaded from NZ or Australia.  Slower connections can test accurately with much smaller files. 

Comparing performance by time-of-day is important as it shows the service degradation when everyone is using the Internet during the evening hours of 8pm to 10pm.  A poor result typically shows the line drop below 90%, which usually occurs in the busy period between 7pm and 10pm, i.e. if this is true, the average user for that ISP is getting less than 90% of their line capability.  

We set the "Zero" line at 70% because panelists with less than 70% report difficulty using the internet.

New Zealand Peak Speed

NZ peak speed can be influenced by how well ISPs peer at the internet exchanges, where they connect to TrueNet's server provider for our file test. Without effective peering a file can travel from a TrueNet test probe located in Wellington, through to Auckland, on to Sydney then back to the Wellington exchange, creating a problem with slowdown in service delivery during busy hours. 

TrueNet however, recommends you also compare ISPs based on webpage surfing speeds rather than peak speed alone.  To make the surfing comparison fairer,  hosted webpages are popular sites that all ISPs have equal access to.

 

NOTE:

TrueNet uses the best quartile of a pair of Auckland and Wellington tests to calculate the median NZ results by hour, over the month for each monitored connection.  We take the average of all median results with each ISP for each hour.

Technology Comparison

Chart 3 compares Peak Speed of technologies in Off-peak, Business, and Peak hours between 6pm to midnight. Cable and Fixed Wireless results have the biggest swings between Peak Hours and Off-Peak periods. Fibre and DSL remain stable across time of day results.

Chart 3: Fibre, Cable and Copper (DSL) Peak Speed by Time of Day from NZ

Fast Broadband: - Peak Speed by Time of Day

Fast Broadband connections with an advertised speed of 100Mb/s are shown in Chart 4a.  Orcon, Slingshot, Voyager are top of the chart this month. However the Voyager result shows a distinct roll off during the evening peak.

Chart 4a: 100Mb/s Fibre & Cable File Peak Speed by Time of Day from NZ

DSL Performance by Time of Day Indexed to Peak Speeds

TrueNet uses an index for copper (DSL) reporting because ADSL and VDSL are sold using the phrase "as fast as your line will allow", so we find the maximum speed, and compare each result with that maximum.  Hence every line reaches 100% at some hour.  

The Copper network is becoming "deloaded" as more users upgrade to Fibre.  This generally means that performance is excellent.  Where once we hoped that ISPs could achieve 90%, we now see most doing better than 95%.

High Speed Copper (VDSL) Peak Speed by Time of Day from NZ

Solid results again this month. There was a bit more speed variation by time of day, but all ISPs are staying within the 95% of peak speed (the short dip below 95% for Voyager is likely to be due to the smaller sample size).

Chart 4b: VDSL File Peak Speed by Time of Day from NZ

Low Speed Copper (ADSL) File Peak Speed by Time of Day from NZ

As seen with VDSL results, there is a bit more random variation by time of day, but again maintaining excellent overall performance.  Sample sizes are again reducing due to panelists migrating to Fibre so we are seeking more ADSL and VDSL volunteers from the ISPs in Chart 4c, and Slingshot.

Chart 4c: ADSL File Peak Speed by Time of Day from NZ

Fixed Wireless Peak Speed by Time of Day

 Spark time of day fluctuations are more extreme than previous months, whereas Vodafone's results are very similar to Skinny.  This chart compares performance by time of day,  so the much slower speed visible in the afternoon/evening is likely to be severe for some on slower than average speeds.

Chart 4d: Fixed Wireless File Peak Speed by Time of Day from NZ

International Peak Speed

To compare international performance, TrueNet downloads files every hour to measure Peak Speeds from Sydney.  

We ensure the download file is not held in New Zealand (cached), so that the test truly measures international performance.   

The connection between the ISP and the Sydney peering exchange is critical to many services based in Australia, and the rest of the world.  Many services are "cached" in Sydney on servers that are supplied by third parties to improve download speeds in remote places.  Two examples are Apple, or Microsoft software updates, which are cached by Akamai in NZ.  

We test by downloading our file from a server connected to the Sydney exchange. This file is random and has checks on it to ensure it is not cached in NZ, which means it is an honest measure from Sydney.

Fast Broadband - Australian Peak Speed by Time of Day

ISP results are grouped around 90Mb/s from Australia, or about 90% of advertised speed, with Voyager an exception at around 70Mb/s. Trustpower & Slingshot had among the best results, achieving similar speeds downloading from Australia compared to the NZ advertised speed.

Chart 5a: Australian File Peak Speed for High Speed Connections (2MB file size only) by Time of Day

Copper Connections - Australian Peak Speeds by Time of Day

Copper connections (ADSL & VDSL) have a speed that is dependent on the distance between the home modem, and the exchange equipment which means that ISPs have little influence on the peak speed of each connection.

 

Australian Peak speed needs to be compared to the best NZ Peak speed by ISP due to the influence of distance from the exchange, which has a direct affect on Peak Speed.  We show the Australian Peak speed as a percentage of the NZ Peak speed (Chart 5b). 

VDSL (High Speed Copper) Australian Peak Speeds by Time of Day

Australian download performance is split into 2 distinct tiers: 2Degrees and Spark at 90% - 95% of peak NZ speed; and the other named ISPs averaging 50% - 60% of peak NZ speed. There is no obvious reason for this difference shown between ISPs for VDSL, this difference is not seen on Fibre or ADSL results.

Chart 5b:  VDSL Peak Speed from Sydney, Australia (2MB file size only) by Time of Day

ADSL (Low Speed Copper) Australian Peak Speeds by Time of Day

The ISPs are tightly grouped between the low-90's and 100% of best NZ speed, which means ADSL users typically get similar speeds downloading either Australian or NZ files.

Chart 5c: ADSL peak speed performance from Sydney, Australia (2MB file size only) by Time of Day

Fixed Wireless: Australian Peak Speeds by Time of Day

Fixed Wireless Peak Speeds from Australia are also compared to the NZ Peak speed, because the actual speed of each connection is different for each Fixed Wireless user. Peak Speed from Australia as a percentage of NZ Peak Speed for Skinny and Vodafone remained roughly equal in September, with Spark's results below those of the other two ISPs throughout the day and into the evening period.  This difference mimics the NZ Peak Speed results.

Chart 5d: Fixed Wireless FIle Download performance from Sydney, Australia (2MB file size only) by Time of Day 

Upload Speed - Performance

Upload speed is important to users sending large amounts of data through the Internet, or uploading files to the Cloud.  TrueNet's upload test sends a 1MB file to our Wellington server, and records the results as the median of the 10 deciles measured for each download.

By Technology / Service Upload Speed

Upload Speed Performance of monitored Technologies has been stable over a long period of time, but there has been incremental change. In September, VDSL increased a small amount to 15Mb/s average upload speed . Full Speed Cable improved to 36 Mb/s, better than recent months.

The 50Mb/s Fibre upload service averages slightly under the advertised speed at 47Mb/s, but other Fibre upload speeds are equal to, or greater than the advertised speed. 

Cable results have been separated into 20Mb/s, and Full Speed Upload Services. Cable Full Speed upload averaged 36Mb/s, and the 20Mb/s service was at advertised speed. 

While Fixed Wireless is often comparable with ADSL, in Upload speeds there is no comparison, with Fixed Wireless running almost 10 times the speed of ADSL.  This makes a big difference in some situations, eg the quality of the video while chatting with friends, or the speed of uploads for dropbox, file sharing and posting photos.

Chart 6: Upload Speed by Technology

Upload Speed - by ISP

Fibre and Cable results are given by ISP (where available) in Chart 7. The 20Mb/s services continue to meet or exceed advertised speed. The order of ISPs is unchanged from August results. 

Chart 7: Fibre & Cable Upload Speed by ISP

The Upload performance of Fixed Wireless is shown in Chart 8, by Time of Day. There is a general trend for the upload speed to decrease during the day, increasing again around 5pm, near the end of the business day. There is no distinct difference between the ISPs, and the dip is less than download speed dips, especially for Spark.

Chart 8: Fixed Wireless Upload Speed by Time of Day

Video Test Results

TrueNet's video test chooses specific videos (Two videos, a HD one and an identical content 4K video)  The video will be cached (stored locally) by ISPs in NZ for all viewers, simply because we are downloading the video seven times a minute from more than 400 different NZ locations.

Each test downloads 10 seconds of content at a time, to fill any buffer to last 40 seconds of viewing.   So on startup, the test will immediately download (at the connection's full speed) 40 seconds worth of content, after a time gap (to allow for playback) it will then download a second 40 seconds of content, simulating keeping the 40 second playback buffer full.  This appears to be an international standard for consumer video testing.

TrueNet records buffer events by counting "Stops", where the simulated buffer would become empty.   We also record Resume and Finished events.   Chart 1 shows a count of the number of "Stops" (sometimes more than one stop in a single download attempt).

We also measure the start delay, ie the time taken for a video to start downloading so it can play.  The typical period for this is so small, usually less than 1 second, that we have ceased publishing this statistic.

Buffering Events during Youtube watching

Buffering on fixed internet lines in NZ is usually very rare on any of our 480 test sites.  If you have buffering on your connection, check your wifi.  We do not test over wifi and that is often the culprit when it comes to buffering.

The rate of buffering events for Fixed Wireless were very low, improving further from August. Other technologies showed extremely low levels of buffering (under 1%) with buffering on specific days, but otherwise none.  The background buffering rate for Fixed Wireless increases from Off-peak (2%), to Peak Hours (3%).

For technologies other than Fixed Wireless, our experience is that Youtube buffering problems are often unique to a few panelists rather than spread throughout the technologies.

Chart 9a: Buffering Events by Day

Buffering for different times of day is shown in Chart 9b. Click on the different periods (Peak, Business, Off-Peak) in the chart to see changes in performance.  This means that even during peak hours, the worst technology, Fixed Wireless, does not buffer 97% of the time.

Chart 9b: Buffering Events by Time of Day

Note: All Fixed Wireless data collection is supported by a contract with Chorus.  All other technologies are supported by a contract with the Commerce Commission.  Analysis of the data and the production of reports are the work of TrueNet under the Commerce Commission contract.

 

Subscription

Subscribe to our mailing list

* indicates required



Email Format

View previous campaigns.