XT not yet reliable: independent report

Friday 7th May 2010

Telecom's troubled XT network is not yet reliable because "as yet unidentified software issues" remain unresolved, says a May 3 report to the company from independent UK-based investigators, Analysys Mason.

The report paints a picture of XT as a victim of its own success, piling on so many new customers after its high profile launch last year that the "network and its supporting operations were not ready to effectively handle the large amounts of traffic the network experienced through the successful acquistiion of XT customers and the migration of Telecom's CDMA customers to the XT network."

There were five key areas identified as responsible for the failures, which Analysys Mason said was compounded by "a number of parallel key factors that contributed to the reliability and performance issues experienced".

The chief executive of Telecom, New Zealand's largest telco, Paul Reynolds, said the review was "both chastening and heartening".

"Clearly some serious errors were made but the report shows that XT is fundamentally sound, and that Telecom and our partner, Alcatel-Lucent, are on the right track."

The report does not seek to ascribed blame for the outages, although it notes that "Telecom and Alcatel-Lucent have recognised that capacity planning and network dimensioning is an area for improvement."

Among actions listed by Telecom to prevent recurrences are "new senior management structure in place" and "change in culture to allow escalation of concerns." Telecom's chief transformation officer, Frank Mount, resigned as the string of XT outages grew earlier this year, and the chief executive of Alcatel-Lucent in New Zealand, Steve Low, resigned.

The five key areas of failure the report identifies are:

· Unreadiness to manage immediate traffic growth

"The approach to operating the network was not appropriate for a new, quickly changing network that included leading-edge technology. Operational systems/processes were not mature enough to predict and pre-empt the issues." Analysys Mason recommends that a longer term - at least 12 month - view should be taken to managing traffic growth;

· Software issues contributed to network instability

"As yet unidentified software issues, in particular those related to radio network controllers, contributed to parts of the network becoming unstable under certain circumstances." Such problems are not uncommon in new software deployments, and are manageable, the report says. "However, with other parallel contributing factors, the network and operational support were not ready to mitigate the risks." Because not all contributing factors had been built in, recovery plans were inadequate;

· The initial configuration of the XT network did not match the intention to deliver the same coverage as the CDMA network it replaced

"A number of target network design improvements were not achieved when building the network due to physical limitations", and an error in setting one parameter exacerbated the problem. "Due to the lack of operational system and process maturity, in particular the lack fo effective network performance measuring systems, it was difficult to address coverage issues quickly."

· Some aspects of the network architecture are "overly complex, meaning that any faults are difficult find and rectify"

Telecom says it has been reviewing the network architecture "with a view to removing complexity", has upgraded four RNC's, will be instal another eight by June next year, and has added fibre backhaul to a further 23 sites.

· Immature operational management systems and procedss failures contributed to the impact of the network issues

The start-up XT network was treated as if it were a mature, well-established network. Among issues identified were "unclear areas of responsibility, the change management process not being able to manage the high level of change being introduced in this new and rapidly evolving network, and inadequate support systems to be able to monitor, identify, prioritise and manage faults."

The report noted that XT's handling of 111 emergency calls was "aligned to international standards."

Businesswire.co.nz

Comments from our readers

No comments yet