Telecommunications - A Case Study

A large CLEC's search for the next-generation Enterprise Management System that would effectively manage its converged service offerings and internal processes while offering its customers a unique window into each customer's Service Level Management information led it to IBM's Netcool suite of products. With these goals in mind, this CLEC then selected Sentini Technologies to help rollout the Netcool product suite (along with other standard EMS toolsets) at its state-of-the-art 24/7 Operations Support Center (OSC).

Bringing It All Together
The CLEC had a multitude of needs and needed a way of tying everything together so that its Operational Services' Support team (a) had a single place from which to manage its converged service offerings, (b) could provide multi-layer data visibility to different groups and departments, (c) gauge the effects of seemingly disparate events on its business groups and service offerings, (d) and last but not the least, provide its customers a window into how it was managing the customers' Service Levels from its end.

Managing Converged Service Offerings
Since being founded in mid-nineties, the CLEC has grown rapidly and today its footprint spans much of the midatlantic and southeastern regions of the United States. Its offerings in these markets include local & long distance calling plans and enhanced Data & Internet services which it offers to its customers in convenient bundled forms.

Having used Lucent's NOC-1 platform for managing the CLEC's mission-critical Lucent-5ESS switches over the years, the CLEC was interested in a unified management system that would monitor the CLEC's Voice, Data and Internet services with equal finesse.

Providing Multi-Layer Data Visibility To Different Internal Groups
Large CLECs with separate groups that handle the provisioning of new customers and the support and maintenance of existing ones rarely work tightly together. As a result, there is often a delay in newly provisioned customers being monitored by the support and maintenance group immediately.

Additionally, the billing department is not completely aware of a customer's service level status and how service level breaches affect the customer's monthly charges. Finally, the sales department has to be aware of the real-time and historical service level status of the company's customers to help with continuing and potential future business relationships with them.

This CLEC was no different and was looking at a creative approach to pushing the same data to different groups simultaneously while ensuring the presentation format was optimized for each group so that each group then had everything it needed to look at every single customer from all operational aspects.

Estimating The Impact On Business Units And Services
Also important to the CLEC was to gain understanding of how each piece of data collected for each customer and for each internal resource affected the bottom line. How was a business unit being affected by the outage of a key resource? How was downtime on a critical circuit affecting its Service Level agreements with its customers and thereby, in turn, penalizing the CLEC in dollar terms?

Also, often, data collected on various resources in a company exists in multiple, disconnected repositories and effective data mining remains forever a pipedream.

This CLEC however was ambitious in wanting to correlate collected fault and alarm data for its device, system, machine and application (both internal and customer facing) resources with its massive data repositories in such a way that management had a clear picture of the effect of every single piece of data collected on different business units, in Quality of Service (QoS) terms, and where applicable, in monetary terms.

Giving Customers What They Want - A Window Into The Customer's Own Data
The CLEC wanted to offer a radical new service to its customers by granting the customer total visibility into the customer's own data, from the CLEC's side.

Though aggregating different data entities into a single "pane of glass" for each customer was no easy task, it was essential to the CLEC to help it differentiate itself to its customers in a crowded marketplace.

How The Telco Benefited From Sentini Technologies's Ems Expertise

• "Customer First" vision - Sentini Technologies always puts customer needs first and this one was no different. The result? A satisfied customer whose needs were met by Sentini Technologies's team going the extra mile for them.
   
• Extensive collective experience (20+ years) drawn upon. - Our vast experience in EMS design resulted in a fast and efficient build-out of the customer's OSC (Operations' Support Center) that not only met every one of its requirements, but was also completed ahead of schedule.
   
• Customer involvement at every turn. - By involving the customer's engineers and managers at every step of the EMS rollout, Sentini Technologies has ensured that the CLEC was continually "in the know". This level of involvement on the part of the CLEC ensures that the CLEC can comfortably use and administer the EMS from the day the Sentini Technologies team is offsite (knowing though, that Sentini Technologies's expertise and support are never more than a phone call or an email away).
   

Proactive Voice Service Management
The CLEC's voice environment, consisting of its core Lucent-5ESS switches, SCPs, STPs, an Alcatel (DSC) Dex/Megahub switch, Alcatel DACS and Tellabs DACS, was highly suited for Netcool Voice Probes and TSMs.

Following Sentini Technologies's EMS deployment, the customer is now able to use these probes and TSMs to capture alarms from these switches and immediately channel them to Netcool's high-performance memory-resident database where they are normalized and de-duplicated thereby presenting the OSC users with a clear picture of the voice network's status.

Some highlights of Sentini Technologies enabled Voice Service Management:

• Historical Raw Data Browse Channels. - The OSC operators rely heavily on having the ability to view raw Lucent switch data. Using Java-based browse channels, the Sentini Technologies team ensured that the CLEC had continued access to these all-important tools through Netcool.
   
• Java-based Connectivity Tools. - Other "must-have" OSC user requirements involved connecting to STPs, DEXs and connecting to the Lucent 5ESS' MCC and TLWS ports. Once again, using Java-based connectivity tools, the Sentini Technologies team not only empowered users to gain visibility into these switches but also enabled them to gain true platform-independence using these Java-based tools.
   
• Customizing Netcool Data Presentation Mechanisms. - Extensive customization made to Netcool's Unix, NT and Web clients ensure that data from the CLEC's disparate data sources was presented across multiple platforms as richly and clearly as possible. Noteworthy here are the maps that the Sentini Technologies team built to help the OSC visually identify hot spots with a mere glance at the maps that are constantly displayed on the OSC monitors.
   

Managing Internet And Data Services
Naviscore is an application that is made extensive use of to monitor Lucent CBX switches and since out-of-the-box Netcool probes have the ability to manage SNMP-enabled devices (such as Routers, Switches and other network devices) and other Network Management Systems and applications, Netcool's data probes are helping the CLEC manage not only its Lucent CBX switches, but also other Network Management Systems like HP OpenView.

Included, also, in Netcool's managed scope are the CLEC's SNMP capable PBX switches, along with the SNMP capable Juniper routers.

On the Internet front, the CLEC's Internet Services' are constantly monitored with Netcool's Internet Service Monitors that manage the Internet Service layer from the customers' perspective by simulating end-user experience via synthetic transactions.

Providing Multi-Layer Data Visibility
With the high emphasis placed on multi-layer data visibility across different departments within the CLEC, Sentini Technologies, with its custom-developed connectivity and integration tools ensured that starting with provisioning a new customer to monitoring the Service Levels of every single customer for exceptions, data collected by Netcool was used in the most optimal way.

Part of the provisioning was automated using custom Netcool tools and this has increased the efficiency of the provisioning process; The Support and Maintenance group is given instant visibility into a newly provisioned customer thanks to the automation introduced in the Provisioning step, so that there is no lag between provisioning and service/circuit monitoring; The Billing department is notified of any interruptions in any customer's service so that customers are accurately charged for services provided; Sales personnel have constant access into real-time customer service levels.

Measuring Business Unit Impact
Using Netcool Impact and Sentini Technologies's custom developed tools, the impact of each piece of fault data is carefully studied and presented to the OSC users.

Sentini Technologies's custom-developed tools further correlate all collected data so that the CLEC's managers have a real-time view into the "health" of any business unit (or service) at any instant and can also observe the trending of the health of various business units and services. Furthermore, by aggregating and intelligently correlating the health of various business services and units, Sentini Technologies has been able to translate into real-world terms the impact of various events affecting the CLEC's infrastructure at any given time. Business decisions are also facilitated since business unit health information is archived and available to managers in the form of useful graphs and tables (Graphs that show, for example, how dips in a particular business service's health reflect a certain occurrence on a key resource thereby enabling the offending resource issue to be adequately addressed).

Putting The Customer In Charge Of The Customer's Data
To provide each customer with as much visibility into the customer's own data as possible, Sentini Technologies created a Customer Information Portal (CIP) that lets customers login into a customized web site that displays different information modules pertaining to the customer's service: existing fault and event data (pulled from IBM Netcool), existing trouble tickets (pulled from the Helpdesk application that is integrated with the IBM Netcool system to facilitate automatic trouble ticket creation, etc.), trouble tickets closed during the previous 30 days, service/circuit utilization (using Concord's eHealth suite), current and historical Service Level status and Service Level exception history (using Concord's eHealth suite and Netcool Internet Service Monitors).

As a corollary to providing customers with this information, the CLEC's customers now feel more in control of their information, have an increased level of trust in the CLEC's commitment to its Service Level agreements and there are fewer billing disputes over their Service Level agreements.

Fully Redundant, Central Management Platform
Using a combination of clients supported on a variety of platforms, the CLEC's OSC users and non-OSC users alike have a clear, consolidated view into Netcool's central database at any given time.

Since the entire Enterprise Management System is built around open, off-the-shelf and lightweight Netcool components, the CLEC can now quite seamlessly integrate the management of most future devices and applications into the existing Netcool-based Central Management Platform.

The fully redundant system design implemented by Sentini Technologies allows users and managers to transparently fail-over to the backup system in case of a failure on the part of the primary system, continue to reliably manage their network(s) and quickly diagnose issues before they develop into problems affecting customers.

Cost-Effective, Unified Service Level Management
Managing the CLEC's converged service offerings ultimately boils down to effectively maintaining and maximizing the CLEC's high service level standards for each of its services while minimizing the associated costs.

Given that operational efficiencies are intimately connected to service delivery and service assurance, Sentini Technologies's approach to the EMS rollout has consistently been to ensure that the CLEC can focus, to the maximum extent possible, on problem resolution once it is detected and identified by Netcool and not on detection per se. Furthermore, using complex correlation at the device data acquisition level (the probe level), Netcool filters out acquired data that does not truly represent problems in the customer's network.

Thus, by integrating standalone devices, systems and applications under the open and highly customizable Netcool suite of products, enabling multi-layer data visibility and putting the CLEC's customers in control of their own information, IBM and Sentini Technologies have not only enabled the CLEC to proactively and centrally manage its services, but also left the door open for future devices, systems and applications to be easily integrated with the existing Enterprise Management System.