Predictive service operations: How to improve IT operations management to respond decisively to the current economic crisis
Table of contents The need for transformation: IT operations today . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Workflow in the reactive operations team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Transform IT with HP predictive service operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Health monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Incident and problem management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Change management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Automated release management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Bringing it all together—unified data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Making the move from reactive to proactive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Reactive stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Control stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Manage stage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Proactive stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Predictive stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Predictive service operations solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 HP business service management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 HP business service automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 HP IT service management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Is it time for your operations organization to move to predictive service operations? . . . . . . . . . . . . . . . 11
As businesses struggle to cope with today’s economic crisis, IT operations teams are faced with challenges as daunting as any they have ever confronted. Businesses will inevitably cut IT funding— but they will still expect operations to deliver innovative business applications, pursue new technology initiatives, and upgrade existing business systems. This paper explains how IT operations organizations can meet these challenges by replacing current reactive processes with a predictive, preventative, productive approach that unifies health monitoring, incident and problem management, change management, and release management to drive efficiencies. HP customer spotlight • Shortened mean time to repair (MTTR) by 22 percent, reduced the number of outages per quarter by 50 percent, and increased end-user productivity by 60 percent (financial services provider) • Automatically detected 95 percent of network events, reducing international IT costs by 30 percent in the process (communications provider) • Increased the number of changes moving into production by 40 percent and at the same time cut the emergency change rate by 50 percent (global manufacturer) • Improved provisioning by 92 percent, patching by 99 percent, and application deployment by 99 percent (global data service firm) • Reduced time required to deploy a business service by 320 man hours (telecommunications provider)
IT operations teams are no strangers to hard times and tight budgets. When the technology bubble burst in 2001 and the market plunged, businesses slashed IT budgets, and IT responded with outsourcing, consolidation, and workforce reduction. Ultimately, these measures helped businesses survive the downturn and set the stage for the IT innovation that would drive the recovery. Now, business is faced with another down cycle, and IT organizations can expect to see budget cuts. But IT can’t allow belt-tightening to slow down the deployment of mission-critical business applications. Cutting operational spend across the board is also not the answer. In the past, when budgets were tight, IT dramatically squeezed maintenance activities wherever they could. Today, with IT so deeply embedded in every critical business process, this approach puts the business at grave risk. For IT operations, the key lies in a new approach that makes it possible to operate within strict budget constraints without putting business outcomes at risk— an approach called “predictive service operations.” Predictive service operations transforms IT operations from a reactive cycle to a predictive process. It enables teams to work together across all operations areas to prioritize and isolate events before they impact the business, analyze the impact of changes to prevent them from disrupting services, automate change deployments to help ensure consistent auditable processes are followed, and manage IT-enabled business services.
The need for transformation: IT operations today As it exists today, the typical IT operations organization isn’t well suited to cope with today’s challenges. Most operations teams spend a majority of their time and budget on maintaining the environment rather than on innovation or on expanding service offerings. This is in part a result of cost cutting in previous economic down cycles. Many businesses responded by cutting spending on innovation and reducing maintenance activities where they could. This put operations teams in reactive mode; with limited resources, they had to concentrate on coping with a rising volume of service requests, events, and changes, and they couldn’t get ahead of demand. The result was service disruptions, unplanned downtime, and cost overruns.
Workflow in the reactive operations team One side effect of reactive operations is that functions are performed in silos. IT operations breaks down into four primary functions: health monitoring, incident and problem management, change management, and release management. These functions align with key processes within the ITIL Service Operations and Service Transition phases and with similar best-practice frameworks, and they represent the dayto-day activities that all operations organizations must manage. Most of these organizations have invested in management tools and process training, but the investments have been made in silos, with each
3
Figure 1. The transition points between functional areas are zones of potential problems for IT operations
Unknown state
s e s a e l e r l a u n a M
Release management
Health monitoring
Change management
Incident and problem management
I n c i d e n t s t o r m s
Ad hoc change management
functional team attempting to be a standalone center of excellence. As a result, the benefits are contained in the silos. Automation doesn’t extend across the workflow, and data isn’t shared. And because of that, the transitions from one functional area to the next are potential trouble points.
25 percent or more of their application or service downtime was the result of unsuccessful changes and configurations. And because manual releases and deployments result in an unknown state of the IT environment, operations teams can’t accurately gauge the business impacts, which can include:
Determining the cause, impact, and priority of events uncovered during health monitoring is a challenge for many organizations. According to the 2007 IDC survey “Business Service Management: Survey Shows Rising Customer Adoption and Increasing Maturity,” over 38 percent of respondents spend 30 percent or more of the incident resolution process on problem isolation.
• IT cost overruns
This causes poorly diagnosed and unqualified events to be sent to the service desk for incident and problem management. With so many events to manage, coordinating response teams and avoiding duplicate efforts is extremely difficult. More than 55 percent of companies surveyed by IDC report that four or more staff members become involved before the cause of a business-critical problem is identified. In addition, the large number of incidents and requests coming in to the service desk often result in an ad hoc approach to change management. The IDC survey indicated that 37 percent of IT organizations handle more than 500 changes per week, and that the number is expected to increase by at least 10 percent in the coming year. Ad hoc change management leads to poor planning, poor risk assessment, and a lack of stakeholder involvement in the approval process.
• Lost end-user productivity • Negative revenue and brand impact • Penalties for missed ser vice level agreements • Inability to report on or comply with industry and government requirements
Transform IT with HP predictive service operations To meet today’s extreme challenges, IT operations organizations need to evolve from their reactive state by adopting a predictive service operations approach. They need rigorous application of standardized best practices so that they can predict business outcomes. They need wide-spread adoption of infrastructure automation to reduce costs. And they need to make and prioritize decisions with business services in mind to reduce risk and improve availability. All this is accomplished through a predictive approach to managing IT services.
Industry best practices such as ITIL provide a fundamental framework for success, but more immediate engagement and automation are still needed. To become truly effective in managing When changes are deployed, they are often deployed today’s complex IT environments, organizations must transform the way they manage the core functions of manually or with only partial automation, and they health monitoring, incident and problem management, create the risk of unplanned downtime. In fact, 34 percent of the respondents to the IDC survey said that change management, and release management.
4
Figure 2. Predictive service operations is a holistic approach that benefits every function and functional area.
Increase staff productivity and service reliability
Predict problems before they impact the business
Desired state e s a e l e r
d e t a m o t u a , d e m r o f n I
Automated release management
Health monitoring Unified data
Change management
Incident and problem management
F e w e ,r q u a l i f i e d i n c i d e n t s
Rational, informed changes
Predict impact of change before and during release
HP customer spotlight Using HP health monitoring solutions, a publishing firm reduced MTTR by 50 percent, the average number of faults per application by 25 percent, and the number of escalated faults by 50 percent.
Provide the right data to the right users
HP predictive service operations is an approach for helping IT operations organizations move from reactive to predictive service delivery. It treats IT operations both holistically and by function, with the goal of leveraging the strength of the whole organization, across functions, to predictably deliver the quality and cost of services expected by the business. The approach focuses on the core operations functions to help IT organizations: • Proactively identify, isolate, and prioritize events before they impact the business • Predict the impact of changes on infrastructure, services, and staff during planning and execution • Increase the productivity of the IT staff through automation of processes and tasks and through information sharing Adoption of this predictive service operations approach provides guidance to IT organizations in creating centers of excellence within the core functions, and at the same time contributes to the maturation of all operations functions. This holistic approach drives improvements in costs, quality, and effectiveness that are transparent to the business.
Understand problem source for rapid resolution
events before they impact business services, end-users, and customers. It lets them monitor entire ser vices— from the business process level down through the network, servers, and storage—improving correlation and providing needed insight. And it raises awareness to changing business conditions before IT alarms typically would have been raised. Second, it allows IT teams to manage entire distributed business services, across both physical and virtual environments, and across all elements and applications. Third, it provides a complete view of all issues, alarms, events, and impacts relating to a service. When an event storm occurs, the root cause is quickly identified and prioritized so that end-users rarely notice a problem with key business services. This complete oversight means that fewer issues are sent to the service desk and that those incidents that are forwarded are pre-qualified and prioritized—allowing the response teams to focus on quickly restoring services and avoiding lengthy diagnosis.
Incident and problem management With pre-qualified and prioritized incidents arriving at the service desk, the response team can focus all its Health monitoring efforts on problem resolution. Pre-qualified incidents The primary goal of health monitoring is to identify also help drive improved process automation, ensuring and isolate problems before they impact the business, that incidents are routed to the right teams and ideally before an actual outage occurs. HP solutions prioritized appropriately. This level of automation help accomplish this goal in three ways. helps avoid cross-functional finger-pointing and helps each team focus on maximizing the availability and First, it provides truly proactive management capabilities from issue detection through remediation. performance of its portion of the business services. It enables operations teams to isolate and address
5
HP customer spotlight Using HP incident and problem management solutions, one financial services firm broke even on its investment in eight months and achieved an ROI of 456 percent, and another found that its response teams could handle two to four times as many users per month.
Best-practice automation for incident and problem management helps drive productivity across IT operations. The HP solution provides industry-standard ITIL process models and workflows that drive rapid time to value. Process automation and standardization not only drive efficiency improvements but also improve the quality of data for continual service improvement and process reporting. In addition to automating processes, HP solutions help operations automate routine tasks—driving further efficiencies and improving control of core processes. By automating the routine, human error can be factored out, and results become more predictable and measurable. Having higher-quality response-team metrics makes assigning staff resources to services more predictable. Based on historical information, HP solutions provide the capability to perform what-if scenarios based on service value, assigned staff, and desired service levels. Using this information, IT staff deployments can be optimized to meet the priorities agreed to by the business and IT, and staff resources can be shifted from maintenance to business-critical initiatives.
With better control over inbound incidents, the IT operations staff can be more responsive to business requests. Additionally, the urgency of having to deal with an overwhelming number of issues is reduced, and IT can make more informed and rational recommendations for change based on data-driven criteria.
Change management The predictive service operations approach to change management helps ensure that processes are consistently followed, that decisions about risk and timing are data-driven, and that all stakeholders have been provided with the information they need to make solid decisions without having to spend an inordinate amount of time in change advisory board (CAB) meetings. In short, the HP solution includes credible, reliable change control that follows defined processes while ensuring that industry and government regulatory compliance requirements are satisfied. As it does for incident and problem management, HP predictive service operations provides best practices for implementing effective change management that make sure the right processes are followed, the right teams are included in planning, and the right stakeholders are involved in approvals. These processes provide a coordinated approach for managing services from request through delivery. One of the major challenges IT organizations have with change management is getting appropriate buy-in or business participation in the CAB. Business stakeholders often avoid CAB meetings because they are IT focused, complex, and lengthy. Without business involvement, IT is forced to make de cisions without the proper visibility into critical business timelines. This can make planned outages more disruptive than they should be.
6
HP customer spotlight Using HP change management solutions, a health insurance company reduced CAB meeting time by 75 percent while increasing CAB effectiveness.
HP customer spotlight Using HP automated release solutions, a financial service firm reduced the time required to add a new database cluster from eight days to 1.5 hours and improved its server-toadmin ratio from 50:1 to 120:1.
HP predictive service operations helps break down that roll-backs can be implemented if needed and the barriers to a successfully functioning CAB by support compliance reporting requirements. eliminating the need for many face-to-face meetings. Automation has been identified as one of the most Instead, it provides the right people with the right effective ways to address the increasing pace of information at the time they need to make appropriate change. In a 2008 IDC survey entitled “Datacenter decisions about upcoming changes. This “virtual Automation: Accelerating Market Maturity Through CAB” approach eliminates long meetings and calls Investment in IT,” 86 percent of respondents agreed and allows CAB participants to rely on shared data or strongly agreed that automation is important to accurately classify the risk and impact of proposed changes. Additionally, a forward calendar of changes for reliable and cost-effective change and release management. makes it possible to easily assess business-significant blackout periods, key resource schedules, and other HP continues to provide new levels of automation, time-based factors. including complete network lifecycle management for monitoring, advanced diagnosis, and automated With the HP predictive approach, the impact of resolution of performance degradations and faults. proposed and scheduled changes can be easily Automation allows IT teams to be more productive identified while highlighting any unintended by carrying out routine tasks while avoiding manual consequences to other important services. This errors. Automation also makes sure that processes are combination of real-time visibility into change planning followed and that when changes are made, the results and process automation helps ensure that regulatory will be the desired state for business services and their requirements are met and that when needed, audit supporting infrastructure. reports can be easily generated. When the time comes to finally deploy a change, the release process Bringing it all together—unified data can be executed reliably and with confidence. Moving IT organizations from reactive to predictive service operations requires a sharing of knowledge Automated release management across functions, and coherent data to drive effective By automating the release of both common and process automation. IT must have a unified view of complex tasks, operations teams can achieve new service information to help ensure that each function levels of efficiency and productivity. HP solutions is responding to demand with the same set of core for data center and client automation help IT priorities. Sharing information breaks down these operations teams manage distributed environments silos and makes sure that there is a common visibility with consistency and centralized control. The result into changes, incident records, asset histories, service is consistency in delivery through documented, reliability, service levels, and the cost of the services repeatable processes that are tunable for continual improvement. Documented processes also help ensure provided.
7
Figure 3. The HP maturity model provides a roadmap for moving from reactive to predictive.
1
2
4
3
5 PROACTIVE
PREDICTIVE
MANAGE REACTIVE
Health
Consolidate domain management, network, OS, servers, and storage
Capture/discover domain-specific data Asset tracking
HP customer spotlight Using HP automated discovery, a pharmaceutical firm reduced the required level of effort for change management and application mapping for its Siebel consolidation by 90 percent.
End-user experience End-to-end transaction
Configuration management
Automated release
Unified data
CONTROL
Consolidate service desk for incident and problem management
Incident and problem
Change
Problem isolation and diagnostics
Service-level management: health
Business and transaction management
Knowledge management
Service-level management: Response time
Continual service improvement
Problem isolation and diagnostics Change process automation
Run-book automation CAB decision support
Continual service improvement
Service catalog
Implement on-time decision support
Automated release: network
Automated release: server and storage
Implement on-time decision support
Coordinated workflow automation
Automated release: client automation
Contact and software compliance
Consolidate domain management, network, OS, servers, and storage
HP predictive service operations provides a complete view across IT, making information available when and where it is needed and in a context that is appropriate for each user. The goal of this shared information is to keep teams and tools synchronized so that all IT functions can be more productive. In contrast to manual spreadsheet tracking, automated discovery keeps the information current to enable the key processes in health monitoring, incident and problem management, change management, and automated release. Without real-time updates on infrastructure and service topology, it is impossible to reliably manage the impact of changes and the related risks. Similarly, without an automatically discovered service mapping of applications to infrastructure, proactive problem isolation and prioritization cannot be achieved. With HP automated discovery, many applications are discoverable without customization, and others, once mapped, are maintained automatically. HP automated discovery shares the same DNA as the integrated configuration management database (CMDB) defined in ITIL, which does not assume that all data resides in a single repository, but instead recognizes the reality that multiple data repositories form a configuration management system as set forth in ITIL.
Service model in CMDB
Making the move from reactive to proactive For many organizations this move will take time and effort. But the question is not when to start, but where. IT organizations must assess their current capabilities across operations and plan for maturing their organization in a way that best meets the ongoing needs of the business in both good and bad economic times. HP predictive service operations provides such an approach. The move to predictive service operations can be accomplished in any number of ways. Some organizations will choose to create centers of excellence around particular IT operations functions, while others will adopt a methodical, unified approach of improving the entire operations group from one stage to the next. However, the end result—the truly predictive operations environment—does require a holistic approach; none of the individual functions can be left behind. To help customers plan for their transition to predictive service operations, HP provides a maturity model that identifies five common stages: reactive, control, manage, proactive, and predictive.
8
Reactive stage Reactive organizations are those that are still working with siloed IT functions, pockets of automation, and disparate processes. The first step to move from “reactive” to ‘”control” is to get a consolidated view of the infrastructure, its health, configuration, and the related assets. Common problems at this stage may include decentralized IT services; a focus on quick fixes and a lack of information; the need to respond to repetitive end-user requests and react to the same problems over and over; and limited ability to locate and track assets and their usage.
Proactive stage The proactive stage is characterized by a focus on managing service levels jointly defined by the business and IT. To do this effectively, IT must understand the priorities and costs of the services provided and be able to communicate its understanding of these priorities to the business teams in a meaningful way. In this stage, service expectations have been defined across the IT organization so that each function understands its contribution to the business. Finally, additional automation is employed to further drive process consistency and efficiency.
Control stage The control stage often involves consolidation of processes and tools to drive efficiencies. This stage includes monitoring the health of business ser vices from the perspective of the end user; consolidating the service desk to maximize the benefit of incident and problem management best practices across the enterprise; automating releases for network elements and coordinating workflow automation; cross-domain discovery; and software license and asset contract management.
Predictive stage The predictive stage is characterized by the IT operations organization’s ability to predictably manage the IT environment, meeting the service availability, reliability, and cost needs of the business. At this stage the correct balance between strategic innovation and maintenance activities has been fully achieved. IT is able to proactively identify and respond to changing business conditions and to act as a trusted advisor for the business. Continual service improvement across IT services is attained in this stage because the IT organization has all the proper Manage stage metrics required to evaluate the trade-offs related In the manage stage, IT organizations have control of to service delivery. And lastly, implementation-time the fundamentals and are able to actively manage the decision support provides real-time visibility into the IT environment to rapidly respond to service requests, environment and services, allowing change managers disruptions, and required changes. This stage includes and practitioners to predict the impact of schedule effective problem isolation and diagnosis; end-user and scope changes, making deployments consistent self service capabilities through the service catalog; and cost-effective while providing stable updates to knowledge sharing through the use of integrated the existing environment. knowledge management; change process automation; automated release for servers, storage and clients; As organizations move up through the maturity model and dependency mapping and consolidated topology. from reactive to predictive, they gain both IT and business benefits. IT benefits include increased service At this stage, IT operations teams are able to deliver availability and performance, reduced change-related standardized services reliably and within the cost risk, and improved staff productivity. Business benefits guidelines of the business.
9
include improved end-user productivity, lower costs of service, higher-quality, more stable service delivery, and improved responsiveness and innovation from IT. This maturity model, derived from HP customer experience, provides a guideline, but is not a lock-step formula. Different organizations will find it practical to implement predictive capabilities in whatever order best matches their unique business requirements.
Predictive service operations solutions The HP predictive service operations is supporte d by HP solutions that work together to help operations executives deliver business value and reduce risk across the operations workflow. HP business service management HP business service management solutions help keep services up and running, and through automation they provide a consistent and efficient way to handle incidents and to monitor and report service-level achievement. They let IT operations see business services from the top down, the way business users see them. They also provide the bottom-up infrastructure view—the performance and availability of networks, systems, storage, and software—needed by specialists. Then HP software combines these two perspectives so operations staff can understand the business impact of infrastructure or application problems. Restoration efforts can then be focused where there is the most business need. When service impairment is detected—often before users experience problems—infrastructure-wide views allow operations staff to quickly identify
the source of the problem. Automated procedures enable generalists to perform basic troubleshooting and restoration steps. When handoffs are required, they go to the right specialist team with the right information the first time. Specialist tools—based on common information and providing a common view— permit specialists to come up to speed quickly and to zero in on problems. HP business service automation HP business service automation solutions provide a set of integrated tools to automate change and configuration management across the environment— user desktops, network devices, servers, and storage. High levels of automated change management are the only way to deal efficiently and reliably with the numbers and variety of devices prevalent in today’s IT infrastructures. HP business service automation allows IT to establish standard configurations, deploy them quickly, verify their implementation, and continually monitor their state. Inventory information is continually collected, so IT knows what is in the environment, can identify exceptions to policies, and can quickly assess the impact of planned changes. Since operations processes span team silos, HP business services automation uses workflow technology to orchestrate not only changes, but service management processes like problem and incident management as well as routine maintenance tasks across the data center, the network operations center, and application support teams. ITIL processes can be implemented, managed, and documented.
10
HP IT service management HP solutions for IT service management enable IT teams to consolidate service desks, implement self-service catalogs, continuously improve services, manage project portfolios and finances, consolidate event management, and control change, configuration, and release management to deliver value to the business at low cost and risk. They enable operations organizations to proactively monitor service levels and report performance. They allow IT teams to maintain an automated IT service desk to speed resolution when users experience problems or require new services. A ser vice catalog allows users to request the IT assets they need and initiate the acquisition process. Self-serve facilities let them access information and find solutions to common problems. And automation speeds things up for business users and increases staff efficiency, driving down cost and increasing consistency and repeatability.
Is it time for your operations organization to move to predictive service operations? As your business prepares to weather the current economic downcycle, your IT operations organization can be a key contributor to survival and recovery. But if your team still operates in reactive mode, then its ability to contribute is limited. The more progressive your operations capabilities are in terms of the maturity model, the greater your team’s ability help the business contain costs without putting business goals at risk. HP predictive service operations can give your IT operations managers the tools, the ser vices, and the approach they need to move from reactive to predictive by pragmatically improving key functions that drive service quality, reduce the risk of change, and support regulatory requirements, all the while maintaining costs at levels agreed on with the business. To learn more about how HP software and ser vices can help you move from reactive to predictive service operations, contact your HP representative today or visit us online at www.hp.com/go/btosoftware.
11
This is an HP Indigo print.
Technology for better business outcomes To learn more, visit: www.hp.com/go/btosoftware. ©2008 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Oracle is a registered U.S. trademark of Oracle Corporation, Redwood City, California. 4AA0-xxxxENW, November 2008
