Measuring Mobile Broadband in the UK

September-December 2010

Executive Summary

Background

1.1 Ofcom's primary duty under the Communications Act 2003 (the "Act") is to further the interests of UK citizens and consumers in carrying out our functions . In addition to securing the availability of a wide range of electronic communications services including broadband services, encouraging investment and innovation in relevant markets and the availability and use of high-speed data services, we must have regard to the interests of consumers in respect of price, quality of service and value for money. Our duties include the requirement to carry out research into consumers' experiences of the way services are provided and to publish and take account of the results of such research.

1.2 Ofcom has an ongoing programme of research into the performance of fixed-line broadband. We believe that similar research into mobile broadband is useful as take-up of mobile broadband increases and an increasing number of households use mobile broadband as their only internet connection. Our research finds that there were around 4.8 million active mobile broadband subscribers using datacards or dongles at the end of 2010 compared to 2.6 million at the end of 2008 ; 7% of UK households have it as their only means of internet access, compared to just 3% in Q1 2009.

About this research

1.3 Ofcom partnered with broadband test and measurement specialists Epitiro to measure the network performance delivered by the UK's five mobile networks (3', O2, Orange, T-Mobile, Vodafone), and to understand the consumer experience of users accessing data services via USB modems or datacards.

1.4 Measurements were made on key performance indicators which affect the consumer experience of mobile broadband applications such as web browsing, downloading files, on-line gaming and streaming video. Measurements included upload and download speeds, web page download times, latency, DNS resolution times, packet loss and jitter.

1.5 Over 4.2 million tests were run from September to December 2010 using three data collection methodologies.

  • Static test probes installed in 97 locations across the UK, each with connections to all five operators and testing hourly. The objective here was to research the performance of 3G/HSPA networks, so locations were chosen where all networks had good 3G coverage and any data collected from 2G connections was discarded.
  • Drive testing in dense urban city, urban sprawl, provincial town, rural and semi-rural county locations (performing tests while stationary). The objective here was to explore how performance varied both within and between areas with different demographic characteristics.
  • A consumer panel of over 1,000 mobile broadband USB modem and datacard users with an installed test application that tested up to 4 times per day. The objective here was to research the performance actually delivered to mobile broadband users.

Network capability results (static probe testing)

1.6 From our deployment of static probes we analysed only the faster 3G and HSPA data (2G connections were excluded) to understand network capability. From the locations tested we determined the following:

  • The average download speed was 2.1Mbit/s. This means, for example, that a 5MB music track would download in about 20 seconds and a 250MB video file (for example a standard definition 30min TV programme) would download in about 17minutes.
  • The average time for each test probe to download web pages from popular UK web sites was 2.2 seconds. Over half of all pages (59%) were downloaded in less than 2 seconds, but 12% of requests took longer than 4 seconds.
  • Latency, a measure of the time it takes a single packet of data to travel from a user's PC to a third-party server and back again, and therefore a measure of responsiveness, was an average of 117 milliseconds. Nearly two thirds of tests took more than 100 milliseconds, indicating that mobile broadband connections may not be optimal for some online games.

1.7 Average mobile broadband speeds and other key performance indicators varied significantly by time of day, with services on average around a quarter slower in peak evening periods than in the off-peak early hours of the morning. This indicates that there is contention for services in mobile broadband networks, although this study does not attempt to establish where the contention lies.

1.8 There were some differences in average performance between operators which are outlined below. Key findings for each operator are expressed as a 95% confidence interval around the mean; we do this in order to acknowledge the limited statistical accuracy of the research - a 95% confidence interval means that if we repeated the research again with a different sample assembled in the same way there would be a 95% probability that the mean value would be in the range shown. From this analysis, we conclude:

  • O2, Vodafone and 3 delivered on average faster download speeds over a 24-hour period than Orange and T-Mobile; O2 was faster than all the other operators in the peak evening period of 8-10pm.
  • O2 had on average lower latency than 3, Orange and Vodafone over a 24-hour period.
  • Web page download tests (downloading the full HTML - but not the images - from some of the UK's most popular web sites) found that on average O2 delivered pages faster than the other four operators.

Variation by regions (drive testing)

1.9 Our drive test research analysed performance from all available connections (2G, 3G and HSPA bearers) and showed a significant variance in service quality across rural, urban, semi-urban and inner city areas.

  • Urban areas outperformed rural areas in mobile broadband performance, in the areas we tested. More than 50% of connections in the rural/semi-rural area surveyed (Herefordshire and Shropshire) had speeds of less than 500kbit/s, compared to 25% of tests in the city we surveyed (Birmingham).This is likely to be largely driven by network availability of 2G and 3G/HSPA: over 95% of tests in Birmingham connected to a 3G/HSPA bearer, compared to under 60% of tests in Herefordshire/Shropshire.
  • Despite the potential for good mobile broadband speeds in the urban city locations covered, the measured performance was highly variable across the city, with no guarantee of good performance offered by a city centre location.

Consumer panel results

1.10 Our consumer panel survey included data from all connections (2G, 3G and HSPA bearers) to examine the service levels actually experienced by consumers.

  • The average download speed experienced by the consumer panel based on all connections was 1.5Mbit/s. Seven percent of consumers received average speeds of less than 0.5Mbit/s and 6% received average speeds of more than 3Mbit/s.
  • Panelists were able to download basic web pages from popular websites in an average of 8.5 seconds, with fewer than 10% of pages downloading in less than 2 seconds.
  • Average latency was 192 milliseconds and around a third of tests delivered latency of higher than 200 milliseconds, making the connection not suitable for internet services which require a high degree of responsiveness, such as online gaming or VoIP telephony.

Mobile broadband services generally perform below fixed services

1.11 For some consumers, mobile broadband, given its costs and added flexibility is already proving to be an adequate substitute for fixed broadband. This is despite the fact that average speed, latency and web page download times for mobile broadband perform at levels lower than those typically delivered by fixed broadband services. The average mobile broadband speed of 1.5Mbit/s based on the consumer panel results compares with the average fixed broadband speed of 6.2Mbit/s.

1.12 It should be noted, however, that the actual speeds available through fixed and mobile broadband depend very much on a combination of location and network quality. Consumers should therefore check the speed available on their individual fixed line as well as the level of mobile broadband coverage before making any choice between the two.

1.13 Network latency on mobile broadband services is almost double the average delivered on some fixed line services. This means that webpage browsing is typically considerably slower using a mobile broadband connection rather than a fixed broadband connection. The lower latencies required by activities such as online games may also make mobile broadband unsuitable for some types of internet use.

Conclusions and next steps

1.14 Geographic location is likely to be the largest single determinant of mobile broadband performance. There are very significant differences in performance by the type of network connection (2G, 3G or HSPA), and our drive testing indicates that the availability of 2G, 3G or HSPA networks, and the performance delivered, vary significantly even within small geographic areas. Before purchasing mobile broadband, consumers are advised to check the network availability in the locations where they expect to use the service using operators' own postcode-based coverage checkers.

1.15 In areas where there is good 3G/HSPA coverage, our research finds that there are some differences between operators in the performance delivered. Consumers should be aware that service quality will differ based on their location and choice of provider.

1.16 Mobile broadband networks and services, and broadband services in general, are evolving rapidly, and it is essential that consumers have information which enables them to make informed choices about the services available to them. Ofcom will continue to work with industry with the objective of ensuring that up-to-date information about the following is available:

  • robust and reliable comparative information on network coverage by postcode, including where 3G/HSPA services are available;
  • the performance of mobile broadband services delivered via dongles/ datacards, including its relative performance compared to fixed-line broadband service; and
  • the performance of mobile broadband services delivered to smartphones and other devices.