January 2017 Urban Broadband Report

Orcon Fibre Consistent - Vodafone Congested?

Orcon consistently tops the Fibre webpage download speeds in January from NZ, Australia and the USA.

In the copper technologies of ADSL and VDSL,  the most consistent ISP is Spark, although 2Degrees is almost as good but their speeds from Australia are a bit slower. Orcon, while consistently good in fibre are in the middle range for copper technologies.

Vodafone Fibre, Cable and VDSL results show a broad roll-off in performance during waking hours for live webpage download speeds from NZ.  This indicates potential congestion from NZ and Australia because average webpage speeds drop during the middle of the day with the most severe drop during the evening busy hours of 8-9pm.  Conversely, Vodafone is consistently best at downloading US websites.

TrueNet's first Video report is published this month. TrueNet is reporting the download performance of two video files from YouTube.  We test for start delay and buffering events, and have found no sign of any video download issues for ISPs we measure in NZ.

Panelists keep upgrading to Fibre - no surprise with a very high growth rate of fibre services, but that means we are running short of ADSL and VDSL panelists.  Please volunteer here if you are interested in helping keep the ISPs honest.  We provide you with a great access point wifi as well as private personal charts showing the performance of your network.

Index

Webpage Average Speed

  • NZ
  • USA
  • AUS

NZ File Peak Speed by Time of Day (ToD)

  • Technology comparison
  • Fibre & Cable
  • Copper (ADSL and VDSL)

International File Peak Speed

  • Fibre & Cable
  • Copper (ADSL and VDSL)

Upload Median Decile Speed

Initial Video test results
Glossary

Summary of Performance Measures

In January 2017, there are signs of congestion returning (Time of Day performance differences) in some services. ToD variation evident in both Peak File Speed and Webpage Average Speed results.

Webpage Average Speed is showing a wide range of speeds between different ISPs for each technology.   As expected, the differences are more muted for distant webpages, i.e. USA average speeds are more similar between ISPs than for NZ webpage average speed.  In the high speed Fibre & Cable (service speed greater than 50Mb/s), speeds for NZ Webpages were from 39.4Mb/s down to 19 Mb/s. Vodafone's average webpage speeds slow down (compared to max) throughout the day.

Peak File Speed: MyRepublic and Vodafone Time of Day variations are below 90% of maximum speed around 9pm.  MyRepublic had a fairly sharp reduction in speed in the evening, but otherwise were performing well at other times of day.

Table 1: Summary of Performance Measures

*Note: Webpage average speed is the website size divided by the time to download.  Webpages are often sent from the owners site in many small files, which means the speed is not as fast as a single file will download.

Speed Min/Max compares slowest hour to the fastest hour (as fast as your line will allow).  

The 9pm column is the actual average speed at 9pm for each product.  The average is taken over all ISPs because for copper, actual speed is subject to the distance between our test points and the exchange equipment.  The range of offered speeds for Fibre is large so we have provided an average speed for all fibre connections, which matches the DSL speeds listed.

The average speed of VDSL at 41Mb/s is extraordinary given that last January the average speed for VDSL was just 30Mb/s and the previous January 2015 just 23Mb/s.   Similarly, the average speed for Fibre is well above the 100Mb/s typical of last year.

Webpage Average Speed

In the January results we continue with the new measure of average speed achieved when downloading a specific set of webpages. This method shows speed differences between technologies, and can also show differences between ISPs in a given technology. 

Methodology

Responsive website browsing is valued by most Internet users, and conversely, slow-loading sites can be extremely frustrating.

TrueNet tests Internet browsing by downloading a selection of Live Webpages from NZ, Australia, and the USA, measuring the size of each webpage, and the time to fully download all files on the page.  From this we calculate the speed.  These pages are changed from time-to-time so the actual average webpage speeds cannot be compared between months.  This month there are 7 NZ pages, 3 Australian, and 5 USA pages.  DO NOT COMPARE SPEEDS BETWEEN COUNTRIES because average webpage speed is dependent on the actual webpages downloaded, not the ISPs.

We download webpages from each connection, and compare the average speed achieved for each download.  This test replicates daily activity for many people, and we group the webpages into the regions NZ, Australia and USA, so that readers can compare ISPs based on their own preferences.

Fibre webpage average speeds show no material difference based on Fibre speeds sold, so we show them inclusive of 100, 200Mb/s, and GigE services.

NZ Webpages

Chart 1a below compares the average speed of the main access technologies when downloading a set of NZ Webpages. Note the large differences between ADSL; VDSL & Cable; and Fibre.  

Fibre and Cable both show some slow down during the evening, whereas ADSL and VDSL hold steady throughout the day. 

Chart 1a: Technology Comparison - Webpage Average Speed of Selected NZ Pages by ToD

Chart 1b: The Fibre & Cable results indicate distinct speed groupings of different ISPs.   Orcon and 2Degrees generally averaged better webpage download speeds, while Vodafone Cable averaged significantly slower speeds to the NZ Webpages.

Further, MyRepublic and Vodafone performance varied strongly depending on the time of day. MyRepublic appears to have a capacity issue at "peak" times. Vodafone Fibre & Cable, on the other hand, has a broad roll-off in performance during waking hours compared to speeds around the peak of 2am to 4am.

The obvious grouping of ISPs into pairs in this chart (pages from 7 independent sites) is difficult to explain.  Orcon & 2Degrees at about 40Mb/s, Slingshot (Orcon stablemate) and Voyager at 35Mb/s while Spark, Vodafone & MyRepublic are at a bit over 30Mb/s, with Vodafone having a ToD variation to drop to below 25Mb/s a lot of the day.

Chart 1b: Fibre & Cable - Webpage Average Speed of Selected NZ Pages by ToD

The Vodafone ToD pattern from Fibre and cable is also seen in Vodafone VDSL and ADSL results, but being from a lower speed appears less impacted. VDSL average speed in chart 1c is roughly twice that of ADSL, but is well below the other major ISPs for both technologies.

Chart 1c: Copper - Webpage Average Speed of Selected NZ Pages by ToD

US Webpages

We download popular US webpages from each connection, and compare the average time taken to download all webpages.   This test replicates daily activity for many people from the USA, including the impact of good design at an ISP where NZ caching improves international website performance.

Chart 2a: Average speeds attained for US Webpages varied from 14Mb/s to 26Mb/s. Vodafone, Orcon, Voyager and Spark Fibre had the best speeds.  Both MyRepublic and Spark experience a dip around 8pm, although MyRepublic speed is generally much slower at 16Mb/s.  In contrast, Vodafone Cable was relatively constant by Time of Day, although only half the speed of their fibre services.

Chart 2a: Fibre & Cable - Webpage Average Speed of Selected US Pages by ToD

Chart 2b: Most VDSL results are grouped together, with the exception of Voyager.   Note that Spark dips in the evening period for both ADSL and VDSL results, similar to the dip seen in the fibre results above.

Chart 2b: Copper - Webpage Average Speed of Selected US Pages by ToD

Australian Webpages

We download popular Australian webpages from each connection, and compare the average time taken to download all webpages.  This test replicates daily activity for many people, but to provide the ability for readers to compare between ISPs based on their own preferences, we group the webpages into regions, NZ, Australia and USA.

Remember: DO NOT COMPARE SPEEDS BETWEEN COUNTRIES because average webpage speed is dependent on the actual webpages downloaded, not the ISPs

Chart 3a: As with the Webpage charts above, the Australian webpage results for Fibre & Cable span a wide range of results - from 70 Mb/s to below 40Mb/s.

Spark and Orcon results are nearly identical at the top, this contrasts with Spark Fibre results seen in  Charts 2a (USA) and 1b (NZ).  

MyRepublic had a relatively sharp dip in the evening, which mirrors the NZ Webpage results as well as the NZ Speed tests in the next section.  Vodafone Cable had the slowest speeds, and the same Time of Day profile as the NZ Webpages, though more muted.

Chart 3a: Fibre & Cable - Webpage Average Speed of Selected Australian Pages by ToD

Chart 3b: Spark VDSL and ADSL continue to have the best speeds, well clear of the other VDSL providers but only just at the top of ADSL, equal to 2Degrees

Note, VDSL is the solid lines and ADSL is the lines with dots.

Chart 3b: Copper - Webpage Average Speed of Selected Australian Pages by ToD

Notice Orcon is consistent with their Fibre products, nearly always the best, but with copper (ADSL and VDSL) Orcon is slower than the other major ISPs in webpage speeds.

Speed (File Peak Speed)

For TrueNet's speed tests each panelist's probe regularly downloads a 2MB and/or 5MB file from Auckland, Wellington, Dallas and Sydney.  The faster the connection, the larger the file we need to download to ensure that the maximum speed is reached during our test. 100Mb/s connections easily reach full speed before 2MB of data is downloaded from NZ or Australia.  Slower connections can test accurately with much smaller files. 

International tests take the result from each test run from Dallas or Sydney.  

New Zealand File Peak Speed

Comparing performance by time-of-day is important as it shows the service degradation when everyone is using the Internet during the evening hours of 8pm to 10pm.  TrueNet uses the best quartile of a pair of Auckland and Wellington tests to calculate the median NZ results by hour over the month for each monitored connection.  We take the average of all median results with each ISP for each hour.  

A poor result typically shows the line drop below 90%, which usually occurs in the busy period between 7pm and 10pm, i.e. if this is true, the average user for that ISP is getting less than 90% of their line capability.

Technology Comparison

Time of Day performance of Fibre, Cable, VDSL, and ADSL is compared in Chart 4. Overall performance was good in December - all but Cable remained in the 95% - 100% range, and cable remained above 90%.

Chart 4: Fibre, Cable and Copper (DSL) File Peak Speed by ToD from NZ

 

Fibre & Cable Comparison - File Peak Speeds by ToD

Chart 5: Performance of Fibre and Cable is varying more in recent times. With the high speed services of 200Mb/s and "GigE", there is more variation in speed test results than we have seen before.  January results generally stayed within the 90%-100% of peak speed, where December results were generally above 95% of peak speed.  Voyager had a broad reduction in peak speed over the course of the day, while MyRepublic had a sharp dip in the evening busy period, but had excellent performance at other times.

Chart 5: 100Mb/s and greater Fibre & Cable File Peak Speed by ToD from NZ

DSL Performance by Time of Day (ToD) Indexed to File Peak Speeds

TrueNet use an index for copper (DSL) reporting because ADSL and VDSL are sold using the phrase "as fast as your line will allow", so we find the maximum speed, and compare each result with that maximum.

High Speed Copper (VDSL) File Peak Speed by ToD from NZ

Chart 6: 2Degrees, Orcon, and Spark had great results in January (remember the average actual speed is 41Mb/s, almost half that of Fibre actual speeds).   Vodafone and Voyager variations by ToD dropped compared to last month. Vodafone results mirror the broad reduction in speed throughout the day that was seen in Cable Webpage results.  The Voyager result compares with their result in Fibre peak speeds.

Chart 6: VDSL File Peak Speed by ToD from NZ

Low Speed Copper (ADSL) File Peak Speed by ToD from NZ

Chart 7: ADSL had excellent Time of Day results in January for all ISPs apart from Vodafone which repeats the variation in ToD that we have seen in almost all Vodafone measurements in January.   However the reduction is just below 95%, which is good by past standards.

Chart 7: ADSL File Peak Speed by ToD from NZ

International File Peak Speed

TrueNet Downloads Files every hour to measure Peak Speeds from Dallas and Sydney to compare international performance.

The File Peak Speed reported is compared to the maximum NZ peak speed, ie this is the potential peak speed for each connection.

We ensure the download file is not held in New Zealand (cached), so that the test truly measures international performance.   Tests are based on a 2MB file size.

High Speed Fibre & Cable- Australian File Peak Speed by ToD

Chart 8 shows Peak speeds in downloading a test file from Sydney, indexed to peak NZ speed for each ISP Fibre or Cable service.  The overall range of results is comparable to last month, and we note almost a factor of two between the lowest and highest relative speeds. Vodafone Fibre again is the best on the chart, while Vodafone cable had the slowest results. Slingshot, Orcon and MyRepublic each slowed down a little in the evening business period.

Chart 8: Australian File Peak Speed for High Speed Connections (2MB file size only) by ToD

ADSL & VDSL (Copper Connections) - Australian File Download Speeds by ToD

Copper connections (ADSL & VDSL) have a speed that is dependent on the distance between the home modem, and the exchange equipment which means that ISPs do not have any influence on the peak speed of each connection.  To overcome this limitation, the Australia, and USA speeds are referenced to the average NZ download speed of the respective ISPs.

VDSL (High Speed Copper) Australian File Download Speeds by ToD

Chart 9: VDSL results in the Sydney speed test showed a 2-tiered outcome, with Spark and 2Degrees achieving a higher percentage (~90%), while Orcon, Vodafone, and Voyager results were below 70% of NZ peak speed.

Chart 9:  VDSL File Peak Speed from Sydney, Australia (2MB file size only) by ToD

ADSL (Low Speed Copper) Australian File Download Speeds by ToD

Chart 10: ADSL results for our Sydney speed test continued to improve in December, with the majority of results between 90% - 100% of peak speed within NZ. This would be a fairly good result for NZ test points, and is a great result for an international link. Slingshot performance on this test remained well below the other listed ISPs.

Chart 10: ADSL FIle Download performance from Sydney, Australia (2MB file size only) - ToD

Upload Median Decile Speed - Performance

Upload speed is important to users sending large amounts of data through the Internet, or uploading files to the Cloud.   TrueNet's upload test sends a 1MB file to our Wellington server, and records the results as the median of the 10 deciles measured for each download. 

By Technology / Service Upload Speed

Chart 11: The Upload Speed performance measured for each technology is one of the few measures that remains stable over long periods of time. The 50Mb/s Fibre upload service averages under the advertised speed, but other Fibre and Cable upload speeds are equal to or greater than the advertised speed.

Chart 11: Upload Speed by Technology

Upload Median Decile Speed - by ISP

Chart 12: Fibre & Cable upload service speeds show small differences between the ISPs in the average speed of 20Mb/s Upload services, with each providing better-than-advertised speed. The 10Mb/s Upload services also achieve or better the advertised speed.

Chart 12: Fibre & Cable Upload Speed by ISP

 

Video Test Results

We have uploaded 2 video files (HD and 4K) to YouTube to test for start delay and buffering events.  Our files are likely to have been cached almost immediately, being downloaded up to 800 times a day, this is expected.  However with this common scenario, neither start delays nor buffering of the Video are of any concern, across all ISPs on technologies ADSL, VDSL, Cable and Fibre.   We are researching how we could use video's that are not cached, but since most videos are likely to be cached we think this report demonstrates that NZ video performance is very good in the main with very minor issues.

Methodology

TrueNet's video test chooses specific videos ( This month two videos, an HD one and an identical content 4K video)  The video will be cached (stored locally) by ISPs in NZ for all viewers, simply because we are downloading the video seven times a minute from 400+ different NZ addresses.

Each test downloads 10 seconds of content at a time, to fill any buffer to last 40 seconds.   So on startup, the test will immediately download (at full speed) 40 seconds of content, after a time gap (to allow for playback) it will then download a second 40 seconds of content, simulating keeping the 40 second playback buffer full.  This appears to be an international standard for consumer video testing.

TreuNet records buffer events by counting "Stops", where the simulated buffer would become empty.   We also record Resume and Finished events.   Chart 1 shows a count of the number of "Stops" (sometimes more than one in a single download attempt).

We also measure the start delay, ie the time taken for a video to start downloading so it can play.  The typical period for this is so small, usually less than 1 second, that we are unlikely to continue publishing this statistic.

One single connection has a severe number of buffering events, with about 30 times as many buffering events as the total of all other connections, and until we have found a reason we have excluded this connection as it may be faulty.

Buffering is where a video stops for some time (usually with a circling icon) before the download catches up with viewing.  We mimic that by downloading in short sections with a delay a bit less than the time taken to view the downloaded section.  

The start delay is the time taken for a video to start downloading after being requested.  

There is no sign in our testing of any issues in NZ with delivery of cached videos.   If you have an issue with video buffering try bypassing your Wifi, which is often the cause of video buffering. 

Chart 13: Buffering Events during Video Tests

Notice that the worst day is the 30th January, where the total buffering events was only 22 out of over 600 videos, some of which were in the same video.  ie very rare.

Chart 14: Start Delay of Video Tests

Start delays are almost all less than 1 second, ie very quick and not noticable.

The Australian equivalent to the Commerce Commission (ACCC) has completed a report based on a small trial in Melbourne.  TrueNet results compared with ACCC results show a quicker response in NZ, but in both cases the delay is minimal.

ACCC published Start Delay, September 2015  (<1000ms = <1 second)

Subscription

Subscribe to our mailing list

* indicates required



Email Format

View previous campaigns.