Comparison of Requirement Prioritization Techniques to Find Best Prioritization Technique

Published Online November 2015 in MECS

In software construction, often comes a circumstances where decision among numerous choices are made. Firesmith [1] notice that software system made of hundreds or thousands of software requirements. Due to limited resources in terms of time, budget and other resources which are considered during development of software and to get customer satisfaction we need to prioritize software requirements. As all the requirements can’t be programmed in a single increment, also we don’t know which requirements are of higher priority regarding customer satisfaction and which are not. In software development, if their involve only one stakeholder it make it easy to recognize   high priority and less priority requirement but when the number of stakeholder increases, in the development of software the case becomes for complex to recognize high and less priority requirements. Different stakeholders have their opinions to implement requirements, one requirement will be of higher priority to one stakeholder and the same requirement will be of lesser priority to other stakeholder. Requirement prioritization helps different stakeholders to agree upon requirements. . For decision making, requirements prioritization are considered one of the most important processes to construct software project [2]. Software projects have low success rates yet these days. The Chaos story 2009 (as written in New Dawn Technologies) [13] shows that, deficiency of user interest, confined resources, astonishing results, vibrant nature of requirements and requirements particularization, results the software project to decline. Due to Requirement Prioritization user interest are increased with the project as it permits the shareholder to stipulate those requirements which are of their own interest. Requirement prioritization also helps to eliminate the disparity amongst the different shareholders. Karlsson and Ryan are of the view that requirements prioritization suggest stakeholders to assign resources to requirements on the basis of their priorities. [14]. Karlsson, Wohlin and Regnell say that requirements prioritization support about the knowledge of the problems in the requirements such as misinterpretation of requirements or any vague requirement so that it does not causes problem later on [5]. Hatton says that requirement prioritization is very.

important nowadays in software development for the expansion of projects in order to minimize the fiasco of the projects [10]

Take example of FBI Virtual Case File (VCF) which comes in category of very large software system, its budget was 170 Million dollar [3]. Cleland-Huang and Mobasher did investigation of the VCF project and think that project failure was somehow occurs due to miss management of requirements and not performing requirements prioritization [4].Software project or product development becomes very fast. Allot of competitor available in market. So within the allocated time and budget software product development must be completed. For this reason and easy to use, easy to understand, generate efficient and reliable results and fault tolerance requirement prioritization techniques must be used. Requirements prioritization techniques must support prioritization of interdependent requirements. The AHP is analytical tools for Requirements prioritization. The AHP is applied to break down large unstructured decision problems into controllable and measurable modules. Despite the increasing AHP has started to be engaged in Requirement prioritization in software engineering fields. Though, there is still a lack of papers presenting the use of AHP in typical Requirement Prioritization.

This article provides an examination of different requirement prioritization methods:  analytic network process (ANP), analytic  hierarchy process (AHP), hierarchy AHP, spanning tree matrix, bubble sort, binary search tree and priority groups. To elaborate these methods, each requirement prioritization method was critically reviewed and studied in order to know how to prioritized requirements. We then categorized the requirement prioritization techniques from a user’s perspective according to a number of criteria such as ease of use, required completion time, reliability of results. Regardless of its problems of scaling-up, we found the Analytic Hierarchy Process to be the most reliable and promising technique for Software requirements prioritization.

• II.    Prioritizing Methods

Some necessary methods of software requirements prioritization have been describe below in this section, which shows us that how basic requirements can be prioritize. To know shortcoming of requirements prioritization methods, requirements are prioritized with each method.

• A.    Numerical Assignment

Numerical Assignment prioritization method is mention by larger number of studies such as [5],[6],[7],[8],[9],[10]. Numerical Assignment is fundamental technique for requirement prioritization in which several groups of requirements prioritization are made and then requirements are assigned to one of these groups on the bases of their priority. Different groups made in numerical assignment may be differing but few of them are same.

Such as low, medium and high are common groups. While prioritizing requirements, when requirements are assign to any of the group in numerical assignment, then priority of each requirement inside that group will be same.

• B.    MoScoW (Museum of Soviet Calculators on the Web)

MoScoW is a type of numerical assignment technique which is described in [11],[12]. Four priority groups which are MUST have, SHOULD have, COULD have and WONT have, are made in MoScoW. To prioritize requirements, each requirement will be place in one of the group based on their priority.

• •    Must have means that requirements in this group must be implemented in the software before it goes to release.

• •    Should have means that if requirements from this group are implemented then it will be good for the product/software.

• •    Could have means that if requirement from this group exist then it will be good for the project/software.

• •    Won’t have means that requirements present in this groups can’t be implemented in current iteration as these requirements are of low priority.

• C.    Simple Ranking

Berander and Andrews [7] and Hatton [10] suggested simple ranking requirement prioritization technique. In simple ranking requirements are ranged from 1 to n where n is any integer value. Higher priority requirement are ranked by 1 and lower priority requirement are ranked by n.

• D.    Bubble Sort

Bubble sort is a technique which is used for sorting of elements. Hopcroft, Aho and Ullman [12], Karlsson [5] prioritize software requirements first by using Bubble sort technique. To prioritize requirements using bubble sort, you have to take two requirements and compare these requirements with each other, if the requirement are not in sequence then swap the requirements and then take another requirement and compare with it. Continue this process for each requirement until you have priority list of requirements from higher to lower requirements.

• E.    Binary Search Tree

Binary search tree method was suggested by Hopcroft, Aho and Ullman [12] which is another technique used for sorting. In binary search tree all buds have mostly two children’s. First time the binary search tree was presented for requirement prioritization by Karlsson [5]. Each bud in binary search tree shows a requirement. The low priority requirement are those requirements which are arranged on the left side of bud and high priority requirements are those requirements which are placed on the right side of the bud in binary search tree. In we do requirement prioritization such as initially we take a requirement and accommodate that requirement as a base bud. After that we take another random requirement and analyze it to the base bud. If that requirement is negligibly essential than the base bud we then measure it to the left child bud of the root bud and if the requirement is of larger importance than the base bud then we analyze it to the right side child of root buds . The base bud if do not have any child buds then place that requirement as a new child of the root bud. If the requirement having larger preferences then root bud place that provision as a child of the root bud and if it’s of less preference then root bud place that requirement to the left side bud as a new child of the root bud . We are repeating this process till whole of the requirements are placed in the binary search tree.

• F.    Hundred Dollar Method

The Cumulative voting method which is also called as Hundred Dollar prioritization method mentioned in [5], [11], [6], is also an easy requirement prioritization method. In the hundred dollars technique 100 points (hours, money etc) are disposed to the stake holders to divide the 100 points amongst the requirements. Then every stake holder disposes the points among the requirements on the basis of its composition. For example if there are 100 points distributed to the every stake holder and there are 5 requirements to prioritize , so that every stake holder has an option to dispose 20 points to every requirement , or may issue all of the 100 points to a single requirement considering it the most significant requirement.

• G.    Analytic Hierarchy Process (ANP)

Table 1. Scale for Pairwise Comparison in AHP [16]

Intensity	D efinition	Explanation
1	of equal value	Two requirements are of equal value
2	Slightly more value	Experience slightly favors one requirement over another
5	Essential or strong value	Experience strongly favors one requirement over another
7	Very   strong value	A requirement is strongly favored and its dominance is demonstrated in practice
9	Extreme value	The evidence favoring one over another is of the highest possible order of affirmation
2,4,6,8	Intermediate values between adjacent judgments	When compromise is needed

Analytic hierarchy process is a famous requirement prioritization method. Analytic hierarchy process has been presented by Saaty [16]. In AHP, initially whole requirements are recognized and then criteria under which these requirements will be preferred. In AHP we pair wise analyzing between the probable pairs of the hierarchy. Now users can recognize the possible relationship between the hierarchies. We then pair wise analyze them and users can select its preferences from the scale which ranges from 1 to 9. The scale is given in Table 1. Now AHP alters the user’s attention to numeric numbers and numeric numbers are given to every element in the hierarchy. During the use of AHP there is an option that redundancy may occur while preeminence of requirements, so consistency ratio must be known after using the analytic hierarchical process to view the prioritization completed are legal. AHP is used to analyze software requirements and among them which one is of top requirement and to which degree. If there are n requirements to be analyzed by AHP then the number of pair wise analysis required are n (n-1)/2.

• H.    Hierarchy AHP

The most complex level software requirements are located on the apex of hierarchy and the most accurate level requirements are located at the base of hierarchy. Karlsson et al presented hierarchy AHP to software requirement prioritization; prefer those requirements which are presented at the same planes. [5] In hierarchy AHP, whole of the exclusive pairs of requirements are located at one plane. Now in this way not only whole of the requirements are pair wise analyzed to one another. Only those requirements are analyzed which are on the same plane.

• I.    Minimal Spanning Tree

Minimal spanning tree in other method of prioritization of software requirements which is by Karlsson et al for referencing software requirements [5] In Minimal spanning tree prioritization technique has idea that if the resolution making is made certainly consistent, then the excess could be overwhelmed. Like if the requirement 1 is superior to requirement 2 and requirement 2 known to be superior than requirement 3 , then requirement 1 be of superior variety than requirement 3 but AHP admits the customer to analyze this kind of pair wise analysis also , which is by this time completed and thence raises the overabundance .

• J.    Planning Game

Extreme Programming (XP) is consist of 12 basic guidelines, planning game is one of them[15], which is used for the prioritization of software requirements in a XP programming project. XP is a combination of two different requirements prioritization techniques. It combines the numerical assignment prioritization technique and ranking prioritization technique to prioritize requirements. In planning game (also called PG) requirements are first elicited from the users, and written these requirements on story board. Then these requirements are prioritized by stakeholder into three different piles: (1) those requirement without which the software will not work/perform, (2) those requirements that are fewer important but give noteworthy business worth, and (3) those requirements that would be good to have [15].At the same time engineers calculates the time needed to developed each requirement and hence distribute the requirements on the bases of risk into three groups (1) those that they can approximate accurately, (2) those that they can approximate logically fine, and (3) those that they cannot approximate at all.

• III.    Evaluation of Requirements Prioritization Techniques

In this section we study the different studies about the evaluation of requirements prioritization techniques. Five papers have been critically reviewed in order to know which requirement prioritization technique is best to prioritize software requirements. Karlsson et al found AHP to be the most capable technique amongst all the prioritization techniques. It provides the most efficient results which are on ratio scale. It is fault- tolerant and provides a consistency check. [5]. Hatton [10] suggested that AHP is the hardest to use amongst all the prioritization methods, also taking long time to compete the prioritization process and having less confidence form users but according to Karlsson et al [5] suggested that AHP is most suitable method for requirement prioritizations amongst six requirement prioritization techniques. Five requirements prioritization techniques are evaluated which are AHP, Binary Search Tree, Planning Game (PG), 100 point and PG with AHP. AHP with PG are considered to be the best requirements prioritization technique [17]. The author concluded that AHP is more accurate, faster and helpful then the numerical assignment [18].

• A.    The Evaluation of AHP, Hierarchy AHP, Spanning Tree, Bubble Sort, Priority Groups an Binary Search Tree Requirement Prioritization Methods

In this paper Karlsson et al carry out an experiment to evaluate six requirements prioritization techniques which are spanning tree, bubble sort, binary search tree, priority groups, AHP and hierarchy AHP [5]. The author took 13 quality requirements for the experiment. In the experiment two types of measurements are evaluated. One is objective measure and the other is subjective measure. The objective measures for the experiment are, required number of decisions, total time consumption, and time consumption per decision. The subjective measures for the experiment are: ease of use, reliability of results, and fault tolerance. While performing the experiment the authors have identified some risks which are describe as Requirements are interdependent. Usually there exist interdependencies between the requirements and criteria under which we are going to prioritize the requirements. None of the prioritization technique in this experiment have the ability to consider the interdependency of the requirements and criteria. Infect still no requirements prioritization technique has covered this problem of calculating the priority of dependent requirements and criteria.

Few persons involved in the experiment. Another risk or problem with the experiment was that only three persons were involve in the experiment. The result of this article is shown in the following tables:

Table 2. Objective Measurement for Experiment [5]

Evaluation Criteria	AHP	Hierarchy AHP	Spanning Tree	Bubble Sort	Binary Search	Priority Group
Consistency Index(Yes/No)	Yes	Yes	No	No	No	No
Scale of Measurement	ratio	Ratio	Ratio	Ordinal	Ordinal	Ordinal

Table 3. Objective Measurement for Experiment [5]

Evaluation Criteria	AHP	Hierarchy AHP	Spanning Tree	Bubble Sort	Binary Search	Priority Group
Required number of decisions	78	26	12	78	29,33, 38	34,35, 36
Total time consumpti on (ordinal scale from 1-6)	6	2	1	3	5	4
Total time consumpti on per decision (ordinal scale form 1-6)	2	4	5	1	6	3

Table 4. Subjective Measurement for Experiment [5]

Evaluation Criteria	AHP	Hierarchy AHP	Spanning Tree	Bubble Sort	Binary Search	Priority Group
Ease of Use	3	4	2	1	5	6
Reliability of Results	1	3	6	2	4	5
Fault tolerance	1	3	6	2	4	5

As from the Table 2, 3 and 4 it can be seen that amongst all the prioritization techniques AHP is the most capable technique amongst all the prioritization techniques. It provides the most efficient results which are on ratio scale. It is fault- tolerant and provides a consistency check. Other good techniques, such as bubble sort has lack of such important features. Using AHP the difference between the priorities of the requirements is very clear while the other prioritization techniques only provide the correct order. The consistency test is very essential as person judgments are far from perfect. On the other side AHP may be challenging for the bigger problems. It has problem scaling up as the same is the case with the bubble sort. So for this purpose tool support will be needed to overcome this problem.

• B.    The Evaluation of Simple Ranking, MoScoW, ANP and Hundred Dollar Requirement Prioritization Techniques

Hatton [10] conducted a case study to evaluate simple ranking, MoScoW, hundred dollar and AHP requirements prioritization techniques. Criteria used for the evaluation of these requirements prioritization techniques are: ease of use, time to compete the overall prioritization process and the stakeholder confidence. 12 requirements associated with cell phone features are taken for the evaluation of the requirements prioritization techniques. These requirements are provided to the users. A broad collection of human from different ages, sex, level of education and occupations were shortlisted as contestants. 31 studies were fulfilled and the results were used for data analysis. An ordinal scale from 1 to 10 was used, to measure the difficulty and user confidence of the prioritization method. The research results are shown in the table 5, 6.

Table 5. Time taken (minutes), median confidence and median difficulty (1-10) by different requirement prioritizations techniques measures for experiment [10]

	Minimum Time	Maximum Time	Mean Time	Slandered Deviation	Median Confidence	Median Difficulty
MoScoW	1	5	1.78	1.083	8	2
Simple Ranking	1	4	1.5	0.73	8	3
100 Dollar	1	8	3.6	2.42	7	4
AHP	7	22	14.03	4.4	2	9

Table 6. Characteristics of prioritization methods [10]

Evaluation Criteria	Simple Ranking	MoScoW	100 dollar	AHP
Ratio Scale Information			Yes	Yes
High Confidence from User	Yes	Yes	Yes
Consistent	Yes	Yes	Yes	Yes
Low difficulty	Yes	Yes	Yes
Low effort	Yes	Yes	Yes
Able to handle large number of alternatives		Yes

According to this article the author has shown AHP as the worst method amongst the four prioritization methods, but karlsson [5] has shown AHP as the best method among the six prioritization techniques. AHP is difficult to scale and too much pairwise comparisons are needed and costs much time to complete the prioritization process, but it provides best results, as karlsson et al [5] show either of the prioritization techniques provide the facility of fault tolerance and consistency check. Only AHP provides it and also the result of AHP is on ratio scale, It shows exactly the difference between the requirements due to which the ambiguity and conflict amongst the stakeholders can be resolved. As tools are available for AHP, if we use tools for the prioritization of requirements then we can obtain good results with less time consumed over the prioritization process.

• C.    The Evaluation of An Experimental Comparison of five Prioritization Methods

In this thesis, the author has evaluated and compares the five prioritization techniques which are: AHP, Binary Search Tree, Planning Game (PG), 100 point and PG with AHP [17]. The author has compared the five prioritization techniques on the basis of Time, Scalability, comparison, ease and consistency/accuracy.

For the comparison of the five prioritization techniques, the author has developed a web base interface, where online survey was carried about the five prioritization techniques and to gather the requirements for the experiment from the web portal.

They gathered requirements from the user in the following way

• •    Introduction

• •    Methods

• •    End Questions

In this way the survey was carried out about the prioritization techniques and requirements were gathered for the experiment. After that some mathematical and probability functions were performed on the mathematical experiments.

The following statistical result were found from the experiment

Easy

When the question was asked that which prioritization method is easy one to perform the prioritization process, the answer was found that PG is the most easy method to perform the prioritization process followed by BST and 100 point methods. AHP is considered to be the most difficult method amongst the five prioritization techniques.

Certainty/Accuracy

Another question was that, how certain they were about the final result after the prioritization process had been done. There was mainly no difference between the methods; all the methods had almost similar results. For the question that how accurate the prioritization method compare to each other, then according to this author 100 point and BST are the most accurate methods followed by PG, PGAHP and AHP.

Time

Each participant was asked the question to record the time, that how much time it took to complete the experiment. Then among the five prioritization techniques AHP took most of the time to compete the prioritization process followed by the BST. 100 point and PG are fastest to complete the prioritization process; PGAHP is in the middle of the methods.

Scalable

When the question was asked, that which prioritization method is the most scalable among the five, then BST and 100 point was considered to be the most scalable and AHP was considered to be worst to scale with requirements. PG and PGAHP were considered in the middle.

Best

The last question was asked that which method of prioritization is the best method amongst the five prioritization methods. BST and PGAHP were considered to be the best methods while AHP was considered the worst method among the five prioritization methods. PG and 100point lie in between of the above too.

As it can be seen from the results that AHP with PG are consider to best method for requirements prioritization technique.

• D.    The evaluation and study of prioritization techniques using student as a subject

In this article the authors [18] have done the evaluation of two requirements prioritization techniques which are: AHP and numerical assignment technique. For this purpose the authors have used student as a subject in the experiment. The requirements that have to be prioritized are taken from two different projects. One is from the real estate agency and the other is from the library management system. Two groups of students are taken. One group is from the software engineering students of subject requirement engineering and the other group is from the MS or PHD research students. The objective of this article is to compare the AHP and numerical assignment

Requirements prioritization method on the basis of ease of use and accuracy, that among these two methods which one is the best requirements prioritization method.

While conducting the experiment first in the class room, 8 students took part. 9 requirements for requirements prioritization were taken from the estate agency system. Each student took 25 minutes to prioritize the requirements on both of the requirements prioritization methods.

The result shows that AHP method provides more reliable, accurate and more helpful results then the numerical assignment methods.

Now prioritizing the requirements with the research students, for this purpose they take the nine (9) requirements from the library management system (LMS) and seven (7) research students took part in the experiment. Each student started prioritizing the requirements with both the methods: numerical assignment and AHP.

It is concluded that pairwise comparison (AHP) is more precise, quicker and useful then the numerical assignment. One of the advantages of AHP is that here we have to calculate, that how much one requirement is more or less than the other requirement while in the numerical assignment you only calculate which requirements is of greater priority and which is not. Due to which the participants are not forced to focus each requirement which results in less accuracy. While in AHP each requirement which we are going to prioritize must be studied in detail in order to know the importance of it, so as we take interest in the requirement, As a result we get the requirements clear and fully understandable due to which the ambiguity is removed.

So it is concluded that AHP is more useful, precise and quicker than the numerical assignment method. This is also proved by Karlsson [5].

• E.    The evaluation of various elicitation techniques and requirement prioritization techniques

In this article the authors talk about the different techniques used for the requirements elicitation and requirements prioritization.[19] In this article the authors have summarized the different prioritization techniques(Pair wise comparison approach, Analytic hierarchical process (AHP),100 Point, Requirements triage method and Binary search tree (BST)) with advantages and disadvantages.

The authors of the article have also mentioned that it is very important to identify the dependencies between the requirements before, and then prioritize the requirements. As when two requirements are dependent on each other and we place them in separate increments, then it will be problem. As there is no such requirement prioritization technique who calculates the dependency among the requirements and the criteria under which requirements are prioritized.

Among the all requirements prioritization techniques AHP is very reliable since large quantity of redundancy in the pair wise comparison makes the process literally tactless to judgmental errors. Another benefit of AHP is that, the result of the AHP is on ratio scale, which allows for useful evaluation of requirements.

• IV.    Conclusion