The World Bank
The World Bank Rolls Out a New Culture of Collaboration Along with a Single, Unified View of its Global Infrastructure
67% reduction in performance problems per application
Dev teams see the same data as Ops teams
Increased accuracy in performance reporting and alerting
Despite a substantial investment in more than 50 monitoring tools, there was little insight into what was being monitored or why an application or service was experiencing a problem. As a result, application issues tended to set off a cascade of finger-pointing.
Michael Makar, the senior IT manager in charge of enterprise monitoring and performance testing, remembered what it was like, “Executive management would get contacted by vice presidents or a country office director complaining that they couldn’t use a mobile approval application, or they were locked out of an HR application. Applications and services were down all the time. There was a lack of visibility across the different teams, and there was no management reporting. The outage calls could take days and it could take weeks for a root cause analysis to identify an issue with an application.” Without an effective way of communicating about performance issues or resolving them, a culture of blame took root.
“AppDynamics collects so many more metrics than what we saw with other vendors.”Michael MakarSenior IT Manager @The World Bank
About The World Bank
The largest and most famous development bank in the world, the World Bank, comprises five international organizations: The International Bank for Reconstruction and Development, the International Finance Corporation, the International Development Association, the International Centre for Settlement of Investment Disputes, the Multilateral Investment Guarantee Agency. Until recently, the applications used by the different entities were all monitored separately.
In 2015, the bank made a commitment to consolidate its operations with the goal of achieving a single, unified view into everything everywhere, whether the code was running in a data center or on AWS or Azure.
"It was critical for us to modernize and bring in the right tools to make sure that we were proactively monitoring the right applications and the right services, and providing the bank’s executive management with performance-based monitoring from all across the world,” Makar said.
After reviewing possible application performance monitoring solutions, the bank selected AppDynamics. “The AppDynamics value proposition for us was around handling the complexity of our applications. We make heavy use of Java and .NET, and we needed to build a view of those applications across the different APIs, services, and microservices that our teams use today,” Makar said. Visibility into user sessions and the ability to drill down into back-end transactions for troubleshooting was critical. With stakeholders requesting more and more native mobile applications, support for Xamarin as a development platform and views of the user experience on a mobile device were important as well.
“I can’t remember the last time we had a war room scenario. The outages that occur are solved so quickly, we don’t need to go into big war room scenarios anymore.”Michael MakarSenior IT Manager @The World Bank
Makar and his team deployed AppDynamics to 400 production applications. “We realized a lot of value from AppDynamics right away,” Makar said. Once the Development Teams were able to see the same data as the Operations Team, the finger-pointing stopped. They no longer had to bother web administrators for data they now had at their fingertips. They could see locate any problem they were having with an application and solve it. “The teams stopped blaming each other after we deployed AppDynamics,” he said.
A self-service model quickly emerged, with teams adding AppDynamics to new applications as they were delivered. “We changed the culture,” Makar said. Application owners, business owners, and ops team leaders were all given the tools they needed to access the information through AppDynamics. If an application went down, they couldn’t blame the Monitoring Team for not telling them. They were responsible for configuring their own metrics and alerts.
Goodbye to war rooms and alert blizzards
The results? Far fewer outages and no more calls in the middle of the night. “I can’t remember the last time we had a war room scenario,” Makar said. “The outages that occur are solved so quickly, we don’t need to go into big war room scenarios anymore.” On a per application basis, the average number of problem tickets has declined 67% since AppDynamics was implemented.
By configuring AppDynamics to send all alerts to PagerDuty, alerts are put through an on-call process and teams are only alerted when there is an issue they need to respond to. This is a big improvement over the blizzard of indiscriminate email alerts and text messages that used to be part of a DevOps engineer’s daily life.
Using data to solve business problems
Makar finds he couldn’t stop talking about the data that is now available. “AppDynamics collects so many more metrics than what we saw with other vendors,” he said. Not only is the data indispensable for troubleshooting, but it also is valuable for everything from capacity planning to monitoring the performance of the bank’s infrastructure over time. “Our CIO came up with multiple different KPIs that he wants around performance, availability, mean time to resolution, and mean time between service failures,” he said. “We’ll be automating these reports on a quarterly basis.”
Makar is particularly excited about the opportunity to use the data to solve business problems. Project and loan-related information that is collected along with user activity can be mined for insight. “As our investment officers go through projects, we’re seeing project IDs and the stage of workflow that they’re in,” Makar said. “We’re able to put together reports for management now that give them a better sense of how the business is performing in real time and where the issues are cropping up. And I believe we’ve only scratched the surface of what we can do with the data analytics capability today.”
A year and a half into the AppDynamics implementation, Makar marvels at the difference the application monitoring solution has made. “AppDynamics has changed the culture so much, I don’t think we could live without it,” he said.
“AppDynamics has changed the culture so much, I don’t think we could live without it.”
Michael Makar, Senior IT Manager, The World Bank