Most Complex Project : Infosys Technologies : Pro-active Performance Risk Analysis and Remediation Governance Model

Infosys runs 150+ web based custom applications to support its internal

functioning. This includes a heady mix of J2EE, .NET, ASP and SQL 2005-based

applications. As can be imagined, managing the performance of so many

applications is a major challenge. This project has been specifically deployed

for the application development teams working within the Information Systems

division to ensure smooth functioning of these applications.

The case for a pro-active approach



With each passing year the complexity of Infosys business has been steadily

increasing with the addition of new subsidiaries, special services, and new

acquisitions. This puts additional pressure on the Information Systems division

and there is constant challenge for scaling up its applications while committing

on the required quality of services to end users. This means they need to

eventually ensure flawless quality during production.

The organization decided to catch hold of the horse before it bolted. They

adopted the proactive monitoring and detection of performance defects early

during the evolution of a Web application. For instance, they would look at long

running queries within applications that would cause slow page responses. They

would also check for relativity amongst procedure calls to ensure an optimally

written code was used and there was no unnecessary lag because of this issue.

Such an approach also set into motion a governance framework for performance

risk analysis and remediation for all enterprise Web applications, and cause

isolation of problematic applications from a shared environment. This system

provides a relevant information for troubleshooting and debugging of various

applications.

What was deployed



The solution enables automatic collection, analysis and  correlation of

various performance metrics from the production environments and acts as a

gateway to take proactive measures, and to build a good closed-loop governance

mechanism. The solution automatically detects the performance defects using ADDS

engine, which raises a performance defect in an Enterprise Project Management

System, sending early warning of the performance incidents and an escalation

mail to delivery managers and the practice unit head if the defects were not

closed within the defined threshold time.

Various application performance monitoring tools have been strategically

deployed at various points in the project architecture. The Usage Analyzer tool

creates usage reports by application/page by geography, and concurrency  reports

for accurate performance  modeling. The Performance Indicator measures and

categorizes application's performance (categorizing them into safe, warning and

threat zones) quantitatively using performance risk indicator (PRI) model.

Challenges faced



With the solution notifying both Web and database related performance

issues, there was resistance from  stakeholders in accepting application

performance defects that were identified in production, and raising the same as

project defects in the development cycles. SAO practices and industry best

standards were followed for intergrating the ADDS engine with Enterprise Project

Management System. The PRI shows the degree of performance risk present in an

application. This is a new approach consisting of integrating individual

performance measures of the pages, database objects in a way that computes

overall risk of an application. With each source of performance information

being voluminous (approximately 35GB), the testing methods adopted  to verify

and validate the results of the solution were challenging once the  processing

was done. Cross- verifications  were done manually as well as by pushing the

data to local databases and thereby constructing SQL queries that erify and

validate the results of the  processed data.

How the organization benefits



Pro-active Performance Risk Analysis and Remediation Governance Model has

been successfully implemented and adopted in the digital business engine of

Infosys. All Web applications are governed by this  framework. It analyzes the

performance of 150+ web applications, 20 IIS Web servers, 4 Weblogic servers and

8 database servers which contain 15000+ Asp & ASPX pages, 3000+ Java pages and

19000+  database objects. A great way of measuring the efficacy of this project

is by analyzing the percentage of users for an application at any given point in

time.

Typically, the variation in this percentage should not be too high. Here, it

has been reduced to a variation of just 1% from 10-35% earlier. And this while

96-97% of users are accessing them concurrently.