Test and Measurement

Fibre optic and high speed wireless broadband infrastructure deployments are accelerating across the globe as the demand for faster broadband access increases. In today’s ‘open world’, end customers have the ability to assess the performance of their broadband service using generic, easily accessible tools. These tools provide an indication of connection performance and help the end user judge the value of their broadband service.

The Business Challenge

The provision of test and measurement assessments of broadband implementations are required by funding organisations, regulatory bodies and companies responsible for building and operating the networks. It is important to provide definitive test and measurement information for broadband networks, to understand whether the tests available to end users are representative of a more specialised and controlled test suite and methodology.

In 2012, Gravelroad were asked to review and assess a cellular carrier’s deployment that was being funded to provide a broadband service using 3G technology. This assessment was focused on whether the capability that was being built met the contracted performance parameters.

The Gravelroad Solution

In developing the test and measurement methodology, Gravelroad was able to provide understanding of the overall performance of the broadband service being provided and to answer essential questions:

• Was available internet bandwidth dependant on distance from a cell site?
• What was the nature of service availability near to (or beyond) the edge of the specified coverage areas?

Gravelroad completed a test & measurement service in the field at a set of locations through a sample service area comprising a number of cell sites. The wireless broadband service was delivered to Small Office / Home Office (SoHo) and residential customers in rural areas.

Measurements were completed by deploying a mobile test rig on a 4 wheel drive vehicle. This test rig simulated a SoHo installation with an additional server and laptops to perform a suite of tests, control the test process and to run web based measurements using speedtest.net.

In order that the devices at both ends of the link were under the control of the mobile test program software, Gravelroad installed a server at a peering exchange in the carrier’s network.

The automated test program provided a conventional test suite that included:

• Ping Round Trip Time (RTT)
• Download Speed, and
• Upload Speed

Other data gathered was:

• Radio Signal Strength and position data
• Round Trip Time (RTT) performance whilst the downloading test is underway
• Tests to find the links responsible for significant Round Trip Time degradation
• Impact of changes to MTU (Maximum Transmission Unit) and TCP/IP Window size during tests
• Comparison between UDP and TCP performance
• Performance as the radio link dropped out and failed
• Packet loss performance during an extended (simulated) VoIP/Skype call, and
• VoIP call performance during a download test.

The test system incorporated the capability to repeat the test suite with additional antenna attenuation. This provided a greater understanding of the sensitivity of the service due to potential issues such as installation errors, extended antenna cable runs, antenna pointing errors, failure of cable waterproofing etc.

The regime included tests to confirm that the system itself did not impact the test results.

A variant of the test methodology was produced to run unattended. This allowed an automated test suite to be run (at hourly intervals) of performance mapping against the time of day.

The Outcome

As a result of Gravelroad’s Test and Measurement process, a clear understanding of the service being provided was derived, where its strengths lay and weaknesses were likely to be located.

The project revealed some technical issues that the broadband provider was able to correct, to improve end user performance.

Client understanding has developed in relation to:

• the relationship between on-going performance and the simple web tests that customers are able to perform for themselves
• performance that can be expected near to edge of service area maps, and critically
• performance of the constructed broadband with regard to the contracted service performance levels.

This independent review of network performance has helped both the client and broadband service provider to understand the end user performance experience, the variability of this performance, and the value of the service level agreement between the parties.