Performance Evaluation of I.T Project Management in Developing Countries

Published Online March 2013 in MECS

Failure rate of IT Project is very high as compared to other fields/Industries [2]. Extensive work has been done in developed countries regarding IT Project Management failure. However, such studies are rarely done in the developing countries like Pakistan.

There are many common reasons of IT Project failures in both developed and developing countries. But some factors are dedicated to developing counties like ignoring the international standards, professional environment, incompetent project management and inexperienced human resource. So it is very important to address failure causes in developing countries which have huge potential of IT Projects.

Another issue is that in developing countries, there are some private organizations whose success rate is higher, as they follow International Standards. However, most of the small private sector IT companies and public sector IT organization are not performing well. This study focuses on performance evaluation of IT Project Management in both public and private sector organizations and some common reasons behind the failure of the projects in developing counties.

The main content of the paper is organized in a following way. The First portion describes the literature review related to IT Projects Management in Developing Countries. Already worked done for Managing the IT Projects is discussed. Second Portion describes the flaws and previous research done for the IT Projects Management Issues Specially in developing Countries. Factors effecting the management issues are highlighted. Analysis on the findings gathered from the second portion is done in the third portion of the paper. Forth portion describes the proposed method for management along with the results. Limitations are discussed that are observed during the research after implementing the proposed method. Fifth portion contains the conclusion remarks for all research done and reference for the research material studied. In the end biography of authors is discussed.

Different stakeholders, consultants and project managers have presented their personal view about the eventual causes of failure of IT projects. But their opinion is sturdily biased by their field of expertise. Even very experienced project managers are faced with unmet circumstances.[1]

The following surveys - some of which have also been presented in the statistics over IT failure rate - are an effort to bring some objectivity in determining the causes of IT project failure. This extends the number of the situations considered, but this also reduces the depth of insight or the expertise of the analysis that one gains by going through a full project lifecycle. The most famous surveys are presented as:

• •    The Bull Survey (1998) [14]

• •    The Robbins-Gioia Survey (2001) [15]

• •    The Conference Board Survey (2001) [16]

• •    The KPMG Canada Survey (1997) [17]

• •    The Chaos Report (1995) [18]

• •    The OASIG Study (1995) [19]

According to the Bull Survey of 1998:

The major Factors of IT project failure identified by the professionals and project managers were:

Fig. 1: IT project failure factors

The major success Factors acknowledged were:

Fig. 2: Success factors

These important studies in the field of project management were done in the developed countries. Very little research is done in the developing countries on such issues. Although there are many common reasons behind the project failure in all countries but there are some reasons that are specific to the developing countries in general and Pakistan in particular. We will explore those reasons later in the paper.

• II.    Related Work

We have used a case study approach suitable for our research and to justify the arguments in a logical fashion [9] [10] [11]. We chose 6 different organizations, in which 3 are public sector and 3 are private sector, based in Islamabad and Lahore, for our study. Six project managers and three client managers agreed to participate in this study. All these participants were actively involved in the project management process in their respective organizations. The research also involved discussion with some of end users - who were the recipients of the service provided by the respective organization. Questionnaires relying on best practices in five categories of issues were prepared based on the discussion with the project managers and end users. Follow-up formal interviews were afterwards arranged with the same managers and end-users in order to confirm the results and clarify the ambiguities [11]. The questions were clearly focused on issues related to IT project management.

All record of interviews was maintained in a logbook and later transliterate into the computer. The interviews were analyzed with the organization’s profile which helped us to confirm and authenticate the observed findings. At the end, data analysis was done by Qualitative techniques for analysis like Brainstorming, Interpretation and Partial Situation Analysis which were then divided and categorized into appropriate sections.

• III.    Division of Issues in Five Categories

We have observed wide range of causes of failures. In order to separate various types of issues, we have divided them in five categories as under.

• 1)    Project Management:

Issues related to project management are included in this category. The PMBOK [12] and our discussion with the project managers have been selected as reference for compiling questionnaire. Some questions have also been taken from [13].

• 2)    Risk Management Issue

Risk management is a set of principles and practices to identify analyze and handle risk factors in software projects. Risk management can significantly improve software project performance, current approaches tend to be narrow in focus, understate management in favor of technique, and not always work well in practice [14].

• 3)    Requirement Engineering And Designing Issues

These are basically the issues related to best practices in software engineering. We have used SWEBOK [15] and informal interviews with the development team as reference for compiling questionnaire for this category.

• 4)    Administrative Issues

Some other issues like governance, culture, politics, environment etc. were included in this category. Project managers’ input was vital in preparing the questionnaire of this category.

• 5)    Budgetary Issues

The issues related to budget, finance, funding etc were discussed in this category.

• IV.    Organization and Project Profile

  The public and private sector organizations in the study varied significantly in size, ranging from very small (40 employees) to very large (5000 employees). Participants in the study were mostly project managers, as shown.

Fig. 3: Management Affect in percentage

Most of the products in the study were web applications and transaction-based systems development, as shown in graph.

Fig. 4: web application and transaction based system development

In the study more than half of the projects were less than 12 months long in duration on completion. Some were large and ongoing, in which case interviews focused on the completed phases of the projects.

Fig. 5: Time completion Ration of Projects

• V.    Findings

• 5.1    Project Management:

• 5.2    Risk Management

The scores are awarded as under. For yes/no questions, yes=10 and no=0. For other questions scores are given from 0 to 10. Exceptions are mentioned with the concerned questions. Scores are mentioned for each project in the respective column with their average in the right most columns.

Table 1: Project management

Questions	Score	Range
Was there a full-time project manager for the project?	Yes	Yes No
To which extent was the project professionally managed?	8	1-10
Had the project manager a clear vision of the project?	6.5	1-10
Was the project manager replaced during the project?	No	Yes No
To which extent was the project planning effective?	6	4-10
How much the methodology used was suitable?	4	1-10
How well was progress controlled against the plan?	5	1-10
Was project manager aware of the project’s status most of the time?	7.5	1-10
Were the issues handled soundly during the project?	6	4-10
Did the project have efficient change control mechanism?	4.5	2-10
Was effective formal post-project review occurred?	4	1-10

Ratings on project management (Table 1) tended to be lower, and displayed greater variance, particularly between public and private sector organizations.

Interestingly, small private sector organizations responses tend to be closer to public sector organizations. We are mentioning the average score here.

Average scores on risk management practices are depicted in Table 2. The responses show that most of the organizations in the study were not following specific/formal risk management methodology, and were quite ambiguous about the prioritization of risks. These responses are showing that a majority of projects did come across unexpected threats and they were considered to be strategically important, indicating the importance of having strong risk management. No organization reported practicing quantitative risk assessment. Organizations that assessed risk used qualitative scales.

Table 2: Risk Management

Questions	Score	Range
Was specific risk management methodology applied?	3	1-10
How accurately were risks recognized as the projects start?	4.5	1-10
How well were risks handled during the project?	3.5	1-10
How well was risk prioritization done when identified?	4	1-10
Were mitigation/contingency plans well defined?	4	1-10
Was responsibility accurately assigned for managing risks?	6	1-10
Did unexpected problems occur during the project?	5	1-10

• 5.3    Administrative Issues

• 5.4    Budgetary Issues

5.5 Requirement Engineering and Designing Issues

Table3 depicts responses relating to project governance. Most of the projects reported to a steering committee but the level of involvement of the top management varied a lot. Top management of the public sector organizations was not that much involved as compared to the private sector organizations (both small and large). Similarly executive support was more evident in the private sector organizations.

Table 3: Project governance

Questions	Score	Range
Was the project team reporting to a steering committee?	Yes	Yes No
How much steering committee effective was effective?	5	3-10
Yes Was there a project sponsor?	Yes
How involved was the project sponsor?	3	1-10
To which extent was the top management committed?	6.5	4-10
How involved was top management?	5.5	3-10
Was this commitment beneficial for the project?	4.5	1-10
Did the project team believe they had top management support?	5	4-10
Who was the most senior official directly Involved in the project? Director CEO Business unit manager	Yes

Conventionally budgets are allocated for one year in public sector IT organizations. But projects consent, bidding, vendor selection and contract sign off takes about 8 months. Therefore it is not possible that IT project could complete in one fiscal year. On the other hand, responses gathered from private sector organizations revealed that mostly their budget is allocated properly and within time. But both sectors have witnessed over budgeting issues.

Issues related to requirement phase, designing and development are discussed in this section. Feasibility, requirements elicitation, design, development and technology related questions are asked. Table 5 gives average score obtained in our findings.

Table 4: Budgetary Issues

Questions	Score (1-10)
What percentage was project overbudgeted? (if over-budgeted)	25
Did the top management sanctioned budget for the project?	8
Was proper budget allocated?	7
Was budget made available at required time?	7

Table 5: Requirement Engineering and designing Issues

Questions	Score (1-10)
Did the project management team completed feasibility study for the project?	4
How accurately was requirement phase and analysis done?	5.5
Was there involvement of all stakeholders for the requirement gathering?	7
What was the performance level of users who participated in requirement engineering process?	5
Did the project manager officially sign off requirement specification document?	2.5
How frequently requirements altered after requirements engineering process?	7.5
How much efficient was requirements change management process?	6
Was specific “performance evaluation and quality analysis” technique used for design?	6
Were specific performance measures used to assess the quality of software design?	5
To what extent were client managers satisfied on quality of software design?	6
Did development phase involve developing skills in new technology/tools?	6.5
What percentage did the acquiring skills in new technology responsible for delaying the project?	18
How accurate was unit and integration testing performed?	5.5
How well was SQA process done by the contractor?	6.5
What percentage of total functionality completed by the third party?	55

• VI.    Observations and Analysis

• 6.1    Ownership of Project

• 6.2    Unrealistic Expectations

• 6.3    Stakeholder Politics

• 6.4    Team Building Issues

• 6.5    Communication and Issues

• 6.6    Conflicts Among Top Management

Different findings related to five categories of issues were discussed. As observed earlier that mostly the reasons of IT projects failure is common in developed countries and developing countries including Pakistan. This is not claimed on basis of any evidence (as we could not find any study related to this), but this can be said by observing many similarities in developing countries, like political instability, economic situation, law and order situation, poor education and computer skills, dearth of certified software companies, reluctance to automate business functions, etc. However we have explored some issues which are more relevant to developing countries in general and Pakistan in particular. These issues are given below:

In our informal discussions with the project managers, mostly it was revealed that it is very important to enforce the work/task by the project manager. It rarely happens that the team members themselves organize and perform the task in time. PM must interfere and remind his team time and again to enforce the tasks. Lack of ownership is very common issue in our environment, where team members prefer to take minimum responsibility.

Due to lack of IT literacy, customer’s expectations are very high in our environment. They expect that a product provide all the solutions and their wish list to be implemented completely. Naturally every product has some limitations but customers do not accept limitations of the product/solution. These issues are faced in requirements engineering phase. Our survey related to requirements engineering issues has revealed that often requirement specification document is signed off without assessing the unrealistic expectations. Due to this, Customer/user expectations are not managed properly.

Departmental Politics and leg pulling adversely affects overall performance of the project. Sometimes it is Inter department and other times it may be intra department. If Inter department conflicts arise during the project, PM is responsible to resolve such issues. Study reveals that occasionally, Intra department conflicts also arise in organizations where some departments interference in others issues. During our study we found that often some departments do not want any project to be done by some other department. Such instances are less observed in corporate sector because of commitment of the top management. Executive managers are responsible to monitor overall performance along with business managers.

In private sector, HR skills enhancement and impartial evaluation are considered very important aspects of organizational growth. It leads to high success rate of projects. In our study, we have found that evaluation can be annual, biannual, or quarterly (in some cases).It is observed that in public sector, evaluation is not strictly based on merit; instead it is based on favoritism. In small private sector organizations, staff is often not appreciated nor rewarded for working long hours.

In most of the organizations that we have included in our research, Communication issues, among the team and with the client, are considered the big hurdle in project success. Misunderstanding often arises in the requirements phase which leads to undesirable results. Particularly in offshore development, such misunderstandings arise. We have also observed that in mature organizations proper tools, processes, mechanism, and formal communication protocol exists that reduces the chances of communication issues.

We have also observed some conflicts among the top management. They were found mostly in private sector organizations, sometimes between managers of different departments, who are dependent on each other, and sometimes between function manager and project manager. It happens that function manager does not feel the need to ask the PM for some issues or report properly. It yields drastic effects on project performance. On the other hand, in public sector organizations, most of the times; it is not possible to bypass the senior manager.

• 6.7    Budgetary Issues

• 6.8    Neglect of Project Risk Management

• 6.9    Resistance to Change

• 6.10    Maturity Level of It Companies

We have observed in our study that issues related to budget contributed in delay of the project by about 10 months. In public sector organizations, there is traditional bureaucratic style of working where red-tapeism and delay in issuance of funds are very common phenomenon. On the other hand, responses gathered from private sector organizations have shown that mostly their budget is allocated within time. But both sectors have witnessed over budgeting issues which ultimately leads to failure of project.

Project risk management is almost fully neglected in both public and private sector organizations in Pakistan. Studies in developed countries have revealed the importance of risk management for project success. In fact by adopting proper risk management methodologies, major causes of IT project failure like exceeded budget, missed deadlines, poor communication, etc. can be overcome.

IT companies have often faced difficulties in requirements gathering due to non cooperation of the client’s staff. They are resistant to change due to supposed threat of transparency and not having computer skills. Sometimes they do not cooperate, give wrong information and resist deployment of the systems. They consider IT an extra work and try to avoid it.

It was revealed in a study that about 60% of the project managers in Pakistan have no formal project management training. PMP certified project managers are just 35% while the remaining manages projects using their own experience, knowledge and personal skills.

IT companies are not yet mature to manage the issues discussed in this paper. Out of about 1500 IT companies registered with Pakistan Software Export Board, only two companies CMMI level 5, three level 3 and sixteen are level 2 certified [16].

• VII.    Research Implications

In Pakistan, as a developing country, this research field has not been explored much as compared to developed counties. Working on this research field may give a start for further and more spotlighted research work. The major objective of this paper was to perform a comparative study of IT project management approaches followed in public and private sector organizations in Pakistan. Findings of this paper elaborate certain concerns which add more towards failure rather than success of projects in developing countries. There is huge potential of IT projects in Pakistan but both the public and private sector organizations require focusing more on these concerning issues. Lack of this type of research work in Pakistan indicates that public and private sector organizations have not realized causes of their failure. This research highlights that professional approach, top management support, excellent governance, taking ownership of projects, good team member selection, effective communication, risk management and relying on software engineering process and standards are major fundamentals which can lead to successful IT projects.

• VIII.    Limitations

There are some limitations of our study so findings should be interpreted cautiously. First, it is based on study of limited projects. Second, organizations were based on just two cities of Pakistan. For better analysis, the study could have included some of the projects from other cities. Third, responses to questionnaire were depending on the understandings of study participants from six o rganizations which cannot be verified. Finally, all these projects were based in Pakistan. Inclusion of some of the outsourced software developed projects in this study could have further provided our analysis more rational and logical results.

• IX.    Conclusion

The purpose of this research was to evaluate the performance of IT project management in developing countries. Software industry of Pakistan is well below its potential but it is growing rapidly. There is great room to enhance the performance of IT project management. Lessons should be learnt from mature private sector organizations as they have successfully delivered in the same environment. Public sector and small IT companies should follow them to explore their potential. Issues identified in this paper must be dealt effectively in order to increase the success rate of IT projects in developing countries.