Online Gaming Latency Tests
Which ISP is Best for Online Gaming
The Latency Effect on Gaming
Latency - or ping time - is often discussed on game site forums. High latency is the gamers enemy, where "lag" results in a noticeable delay between the action of the player and the reaction of the server in a video game. Lag may be caused by high latency, and in a fast paced game can ruin the experience for players.
With revenue for the online game market forecast by DFC Intelligence to reach US$100 billiion by 2018 from a gaming audience now estimated at 1.4 billion consumers, and ISP performance for gaming websites is an important component in the mix of measurements that TrueNet incudes in their research.
In this report TrueNet investigates the latency of popular games sites, testing five popular game servers, all located in Sydney:
- Battlefield 4, IP Address = 184.108.40.206
- DOTA2, 220.127.116.11
- League of Legends2, 18.104.22.168
- World of Warcraft, 22.214.171.124
- Starcraft, 126.96.36.199
TrueNet measured round trip latency to each of these sites, and calculated the Median for each site for each ISP. These are then looked at by Technology, and split by North and South Islands, because the greater distance from the server for probes in the South Island will increase the latency by a small amount. Faster technologies do have lower latency as seen in our Quarterly Reports. We have included a discovery set of charts to provide visual aids, and to help understanding of our research findings - feedback below would be appreciated.
The following two charts show the Latency results for Fibre & Cable (Chart 1, higher speed broadband), and Copper (Chart 2, ADSL & VDSL). As these games are hosted on servers in Australia, we have used as a benchmark the latency we regularly report for Sydney and NZ. These are used as reference points in most charts.
To see in better detail, double click on any chart to expand.Chart 1: Game Site Latency for Cable and Fibre, by North Island and South Island
There is strong trend in both graphs in terms of relative game latency - the order displayed is typically highest to lowest, Battefield 4 is highest for most ISP/Technology combinations, and World of Warcraft is usually lowest in each case. Latency for Starcraft and World of Warcraft was consistently the lowest amongst the services shown in Charts 1 & 2. We later split the sites into two groups, "Fast games" and "Slow games" - the names chosen to fit the legend.
From Chart 1, there is a consistent difference of about 10ms between the North and South Islands, although the North Island Cable results also match the South Island results.Chart 2: Game Site Latency for ADSL and VDSL, by North Island and South Island
In Chart 2 the same applies, except for Spark and Vodafone ADSL North Island Latency is similar to the Latency of all ISPs in the South Island. 2Degrees typically had better latency than the other ISPs in the North Island for both ADSL and VDSL.
Also note that several of the ISPs had Starcraft and WoW ping times below the typical latency we measure to our peered Sydney server.
Spread of Results (Bell Curve)
Another aspect of interest is the variability of the ping results (Latency), or "Bell curve", ie the spread over a wide range of results. An average can vary significantly from the results of individual tests, especially where you have a long tail of results that have very high latency not reflected in the average. So the greater the range of results the less reliable the average will be.
Chart 3: To experience low latency, the user would want results towards the left of the chart, and ideally all similar. This chart compares two groups of sites, where the response times are compared with our test sites in NZ and Sydney. The "Fast game" sites include League of Legends, Starcraft and World of Warcraft. The "Slow game" sites are Battlefield 4 and DOTA 2.Chart 3: Frequency of Latency Results
As expected, since all games are Sydney based our NZ server sites are much quicker. The fast game sites are almost identical to the tests from our server in Sydney at 40-50ms. However the slow sites are significantly slower, running between 50-70ms
Bell Curve by ISP
Remember, that results with a low average latency will have a much bigger spike showing up in the left side of each chart.
ADSL Test Distribution
The Bell curve by ISP of the Slow Game sites, is very similar between ISPs, but we have noticed some differences when looking at the "Fast game" sites: League of Legends, Starcraft and World of Warcraft.
Chart 4 covers only the "Fast game" sites: Sample sizes are all similar apart from Flip where we have a sample about 50% of the others. The dotted lines are to remind you that the last point is the sum of the outliers, or where the response time is greater than 100ms and may be up to 500ms.
ISPs have differing results including longer "tails" of higher latency for some ISPs and some very small variances for others.Chart 4: Fast Gaming sites by ISP with ADSL
The best ISP will depend on your choice of probability of consistency in performance. eg in Chart 4, Orcon has a peak well to the left, but a large (~25%) and long tail skewed to the right. Flip has every test between 40-60ms. 2Degrees has a wider spread, but it is centred on 40-50 almost like Slingshot but more to the left. Vodafone and Spark however have a very long tail, extending above 100ms for more than 10% of tests.
Fibre Test Distribution
To add more detail, Fibre is not quite so spread so to demonstrate we have also produced a chart for Fibre - all speeds, and as many ISPs we have with 6 or more panelists. (the dashed lines show where we have fewer panelists)Chart 5: Fast Gaming Sites by ISP with Fibre
There is very little difference between ISPs in chart 5, please take a little care with interpreting MyRepublic (984 tests), BigPipe (675 tests) and Voyager (885 tests), while all others have many more than 1000 tests. There are 26,000 tests represented in this data.
We welcome constructive comments so feel free to respond below, as we are interested in your opinion.
Rural vs Urban
Chart 6: For copper based services (ADSL, VDSL) we also checked for any Rural - Urban difference. From our results, there is not a large difference between Rural DSL users and Urban users. The South Island does show a difference in ADSL results (We do not have enough Rural VDSL in the South Island to allow a comparison). The poor Rural ADSL South result may be because of much greater distances in the South Island. No matter the cause, the large latency of Rural South Island ADSL users would give an unpleasant gaming experience.Chart 6: Rural & Urban ADSL, VDSL Game Latency Comparison
We require further volunteers to assist with testing, especially in the Rural sector where we need another 50.