Home | What is CVM? | Partners | Case Study | How we work | CVM Asia Pacific

Case Study

VODAFONE CALLING QUALITY
Presentation to The Market Research Society of New Zealand July 2000

by Melody McGinley - Vodafone NZ Ltd
and Rodger Gallagher, Deborah Hill & Owen Mayall - CVM Partners

Executive Summary

This study specifically covers the quality of mobile phone calls and the way calling quality was improved.

Vodafone New Zealand through the adoption of the Customer Value Added (CVA) market research approach have identified the product and service features that are most important to mobile phone customers. This approach has been successful in supporting Vodafone's key marketing strategy goal of increasing market share through acquisition of new customers and reducing the churn of existing customers, while also providing quality products and services.

Prior to 1998 Vodafone's predecessor BellSouth saw 'coverage' and the 'dropped call' rate as the main component of calling quality, with all optimisation initiatives being driven solely by internal network metrics. This investigation shows that market research techniques can be used to disaggregate mobile phone calling quality, measure the quality of component parts, and be used to improve the quality of calls. Significant improvements, measured with CVA surveys, were made to the subcomponents of calling quality at minimal business cost.

Successive waves of CVA research, with questionnaires developed from extensive qualitative research, and using sample sizes designed to deliver robust regional information, had established that making and receiving calls was important both to Vodafone customers and customers of competitors. Vodafone customers formerly with Telecom rated Vodafone performance on making and receiving calls lower than those who had not previously been with a competitor network. Vodafone's efforts to improve customer perceptions of the company's performance, therefore, required a detailed look at the issues customers have in mind when rating this issue and where improvements needed to be made.

Both qualitative and quantitative market research techniques were used in the project with survey measures also correlated to new customer focussed internal service metrics. This information was then used to drive an improvement project that saw significant improvements in the clarity and overall calling quality of mobile phone calls.

The test of any market research project must be whether the business objectives were achieved. In Vodafone's case its quarterly Net-adds/ Win rate market share was 19% in 1997 before this project commenced. By June 1998 the Net-adds market share had been increased to 30%, and by June 1999 it was up to 42%, lifting to 60% by June 2000. Service improvements achieved through the work undertaken on the Calling Quality project covered in this paper as well as a number of other CVA initiatives have contributed to lifting the overall value delivered to Vodafone's customers. This superior value has delivered the gains in market share for Vodafone.

Introduction

This case study demonstrates the synergy obtained when the full range of market research approaches: secondary data, qualitative and quantitative are coupled with a company's internal metrics and used by a project team to focus improvement to deliver better value to customers. This study specifically covers the quality of mobile phone calls and the way calling quality was improved.

Background

Vodafone New Zealand, through the adoption of the Customer Value Added (CVA) market research approach, have identified the product and service features that are most important to mobile phone customers. This approach has been successful in supporting Vodafone's key marketing strategy goal of increasing market share through acquisition of new customers and reducing the churn of existing customers while also providing quality products and services.

The CVA programme placed the customers views firmly in front of Vodafone, identifying an issue that historically had been specified and delivered mainly based on technology-based standards.

Prior to 1998 Vodafone's predecessor BellSouth saw 'coverage' and the 'dropped call' rate as the main component of calling quality, with all optimisation initiatives being driven solely by internal network metrics. This investigation shows that market research techniques can be used to disaggregate mobile phone calling quality, measure the quality of component parts, and be used to improve the quality of calls. Significant improvements, measured with CVA surveys, were made to the subcomponents of calling quality at minimal business cost.

Investigation into Call Quality

Under the CVA umbrella, a project was commissioned in the last quarter of 1998 to further investigate the process of making and receiving calls. Successive waves of CVA research, with questionnaires developed from extensive qualitative research, and using sample sizes designed to deliver robust regional information, had established that making and receiving calls was important both to Vodafone customers and customers of competitors. Vodafone customers formerly with Telecom rated Vodafone performance on making and receiving calls lower than those who had not previously been with a competitor network. Vodafone's efforts to improve customer perceptions of the company's performance therefore required a detailed look at the issues customers have in mind when rating this issue, and where Vodafone performance was perceived to fall short of that of Telecom.

To provide this knowledge, five focus groups were conducted - three with switchers to Vodafone from Telecom, one with Vodafone customers who had not been with Telecom, and one with Telecom customers not previously with Vodafone. The groups addressed two key issues:

1. What mobile phone users had in mind when rating "making and receiving calls".
2. In what way Vodafone performance was perceived to be inferior to Telecom's performance on those issues.

Data were analysed to develop a structured model of the making and receiving calls process. The process of making and receiving calls is disaggregated as follows:

In essence, what customers and non customers mean when they talk about calling quality and coverage and care most about are:

• the ability to make and receive a call when and where they want - they are relatively forgiving in rural areas, but absolutely expect to be able to initiate, receive and maintain connection of a call within a known coverage area e.g. within towns or cities, and on major routes e.g. State Highway 27.

• that calls once established, will be free of distortion e.g. robotic voice characteristics and will not break up e.g. interrupted speech.

• that network services will be available where the supplier has promised e.g. within publicised coverage areas, customers absolutely expect no disruption to service. For some, but not all customers, the benchmark is land-line performance

Those Vodafone customers who have not previously been with Telecom, compare their experiences when making or receiving calls with some prior level of expectation - to the degree that when performance falls short of that expectation, they downgrade their rating. Expectations may be minimum acceptable, most likely from a named supplier, normative or may reflect some "ideal" standard.

Vodafone customers who have experienced Telecom service, use that experience as the benchmark. On critical issues, usually bundled in their minds as "coverage", namely geographic coverage, holes within known coverage areas, signal strength and call disconnection, Vodafone was perceived to perform less well than Telecom. More specifically:

• In urban areas, such as Auckland, it was found that customers expected to be able to initiate and complete (or receive and complete) a call with uninterrupted speech, without losing the signal or dropping the call while maintaining intelligible speech.

• Dead patches or pockets of no coverage are not expected to occur in areas that are "covered". In other words, if there is coverage in Auckland, customers expected to be able to make or receive a call in their home in Howick.

• While expectations are not as high about the ability to make and receive calls outside towns and cities, customers expect there to be a reasonable "radius" around populated areas and major routes.

The recommendations from the qualitative research were:

1. That the items included in the making and receiving calls section of the CVA questionnaire be reworded to reduce ambiguity of terms currently used e.g. coverage.

2. That the focus group findings be shared with both Vodafone's engineering function and the marketing function to ensure there is no potential for misalignment between customer perspectives and Vodafone understanding of what customers mean when they use these words to rate Vodafone performance.

3. That efforts continue to develop and align internal metrics with customer needs as now defined.

4. That these measures be available to both engineering and marketing functions, and to maintain alignment between service delivery capabilities and communication strategy.

Network Service Consultancy - Internal Metrics

The engineering group of Vodafone had been making excellent progress (last quarter 1998) in reducing the mean level of dropped calls reported from the network management system. Drive testing results from the same period found a slightly higher level of dropped calls from the competitors network compared to the Vodafone network, yet the CVA results didn't reflect this. (Drive testing consists of a specially equipped car driving roads within the mobile phone coverage area and carrying out a series of tests that simulate the calls made by a customer. The testing covers both Vodafone and competitor network performance and provides comparative data.)

Rodger Gallagher of Customer Value Management was commissioned to investigate and work towards establishing linkages between CVA, drive test and network management data.

Preliminary analysis indicated that an alignment between the CVA data and the drive test results could be established where a service standard of 5% or greater dropped calls was used. With these early findings the next step was to test the relationship between the drive test results and information collected from the network management system.

A correlation was found between the drive test results (mean of dropped calls greater than 5%) for each region and the cell site mean dropped call rate from the network management system. Clearly from an engineering view point the overall network performance level for dropped calls was excellent, however as long as there were individual cell sites with performance below the mean, individual customers could receive a lower than satisfactory service when operating in these areas.

Working in parallel with the network consultancy the "Calling" section of the CVA questionnaire was reworded to reflect the findings from the qualitative work - a Call Clarity CVA measure was established.

Further analysis was undertaken to form links between the new CVA data, the drive test results and the network management data. The outcome from this work was the introduction of a service standard (bit error rate) across all Vodafone cell-sites. As the bit error rate increases, the incidence of robotic-like speech increases. By establishing technical standards based on customer needs revealed through market research and econometric analysis, Vodafone has been able to design and manage its network to deliver improved clarity for the mobile phone calls made by its customers.

The above graph depicts the situation before and after the forming of the Optimisation project team where the network management system internal metrics were given a customer focus. As can be seen, CVA results track the internal metric service standard data. (Service standard data is shown monthly from October 1998 on. The CVA Call Clarity scores are represented by the "dots" in the months of October, February, June and October.)

After the introduction of the service standard Vodafone achieved a significant improvement in the Call Clarity score. The table below shows that Vodafone has achieved parity with competitors in Call Clarity.

(CVA results from Business market segment)
(Relative scores shown are VF mean score/Competitor mean score *100)

Comparative drive testing comparing the call clarity of Vodafone compared to its competitor confirms the same improving trend. As Vodafone's quality of calls continues to improve and customer perceptions catch up with actual performance, the survey results continue to show a positive trend for Vodafone.

Market Share Gains

The test of any market research project must be whether the business objectives were achieved. In Vodafone's case its quarterly Net-adds/ Win rate market share was 19% in 1997 before this project commenced. By June 1998 the Net-adds market share had been increased to 30%, and by June 1999 it was up to 42%, lifting to 60% by June 2000. Service improvements achieved through the work undertaken on the Calling Quality project covered in this paper as well as a number of other CVA initiatives have contributed to lifting the overall value delivered to Vodafone's customers. This superior value has delivered the gains in market share for Vodafone.

The full case study on how Vodafone NZ used the CVA approach
to get ahead of Telecom NZ is covered in
Mastering Customer Value by Ray Kordupleski.
Click here for more information

The Vodafone Story Continues...

"Vodafone NZ edges ahead of Telecom"
Click here to read the full NZ Herald article.

© CVM 2002 : T&C | Home | What is CVM? | Partners | Case Study | How we work | CVM Asia Pacific |