QoS Metrics for Cloud Computing Services Evaluation

Published Online November 2014 in MECS

Cloud computing has got to appear like a benchmark to achieve demand resources (e.g., Platform, software, infrastructure, and so on). For users almost like diverse energy (e.g., Gas, electrical power and water). The 3 primary solutions are given by the Cloud computer's product in keeping with the demands of IT clients [1]. At first, Platform as a Service (PaaS) offers a platform for producing various applications on top of it, such as the Google App Engine (GAE) [2]. Next, software as a Service (SaaS) presents access to complete programs as a service, similar to customer Relationship Management (CRM) [3]. Finally, Infrastructure as a Service (IaaS) delivers a workplace to releasing, running and handling VMs and also space for storing. Actually, IaaS gives modern scalability (scale down and scale up) of calculating sources and on demand storage devices. Cloud as a man of the main novice calculating paradigms [4], is now increasingly popular in companies.

Correspondingly, lots of and a lot of Cloud services provided by growing various companies are obtainable in the industry [5]. Because of this kind of commercial benefits supplied by Cloud systems, several institutions have begun constructing programs on the Cloud system and also creating their organization's flexible by exploitation elastic and agile Cloud solutions. However, transferring products and/or information into the Cloud isn't really easy. Varied problems can be found to control the entire capability that Cloud computing claims. The Cloud delivers its solutions to the purchasers via online services solely. Therefore, services play an important role in these systems. Clouds also take over open standards, scalable scheme and service-oriented architecture (SOA) and also present genuine solutions on-demand in a flexible or finance manner. The products are supported on Cloud providers, thus their stability and then efficiency is intimately connected with the Cloud services' situation and  standard.  Because  of the heterogeneous and loosely coupled environment of Cloud, exact QoS isn't recognized to service consumers until runtime [6, 7]. On the other hand, with the increase of general Cloud offerings, for buyers it is becoming more and more troublesome to choose that vendors can fulfill their QoS needs. Every Cloud provider offers very similar products at distinct (completely different) costs and capability degrees with various groups of choices. Whereas one seller may be inexpensive for memorial solutions, they will be costly for calculation. Moreover, the services on the server side are perceived as a black box to system customers. Therefore, quality evaluation of services before distributing is vital in a Cloud environment. The services ought to be examined by QoS metrics to form specified; the services are satisfying the consumer’s expectancy. Once assessing Cloud services, a collection of appropriate measurement metrics or criteria should be chosen. In fact, based on the produced research within the analysis of ancient computing devices, the choice of metrics plays a vital role in analyzing implementations [8]. However, compared to the massive quantity of the study attempts into the conditions for the Cloud [9, 10, 11], to the easiest of our information, there's not one methodical article concerning metrics for evaluating Cloud services even so. As a result, it's difficult to denote the fury of Cloud computing and a total scope of criteria for assessing completely different business Cloud solutions. The aim of this paper is to determine QoS metrics that could be applied by the service manufacturers to evaluate if the services fulfill the consumer’s priority. The remainder of this paper is organized as follows:

Section 2 provides a survey on the cloud characteristics and models. Section 3 provides a brief description of cloud service metrics and describes our research method. A discussion of the results is provided in Section 4 and presents the conclusion of this paper.

• II.    Cloud Characteristics

Today, most people, each IT organization are discussing about the Clouds. Tho' there is no precise definition of Cloud systems, you'll be able to know about it in several ways [12]. Cloud computing could be a model for sanction present, simple, on demand access to a joint pool of computing services (e.g., servers, networks, programs, storage, and so on) that may be immediately provisioned and also discharged with lowest administration effort. The us. Government might be a main customer of electronic services and, thus, one in every of the principal consumers of Cloud systems. The United States National Institute of Standards and Technology (NIST) has a collection of operating definitions that distinguish Cloud computing into service models and deployment models. These models and their connection to the vital features of Cloud systems are shown in Fig. 1 [13]. Cloud is TCP/IP primarily based development of computer technologies like large memory, fast microprocessor, reliable system design and highspeed network. While not the standard interconnection protocols and mature of set data centre concepts, Cloud system wouldn't reality too. The varied definitions and interpretations of “Clouds” and / or “Cloud computing” can be found. With particular respect to the multiple usage scopes, the term is used to. We'll try to deliver a representative (as opposition complete) set of definitions as a referral towards future usage within the Cloud computing connected research area. Fig. 1 demonstrates the most aspects forming a Cloud system [14].

Fig. 1. Perspective on the key aspects of a cloud computing environment

As “Clouds” don't focus on a specific technology, nevertheless, to an all-purpose provisioning paradigm with increased abilities, it's essential to complicate on these attributes. The European Commission “Expert group Report” [14] creates a summary of most common Cloud computer properties classified in three sets:

-Nonfunctional features – “represent attributes or characteristics of a product, rather than particular technological requirements."

-Economic concerns – “one of the crucial leads to introduce Cloud techniques in a commercial environment within the initial instance."

-Technological aspects – originate from the contrary of non-functional and economic features and “typically imply a specific realization."

This part specifies the tangible abilities related to Clouds which are considered important (needed in almost any Cloud computing environment) and also related

(preferably supported, however, might be limited to particular use cases). We could thereby identify nonfunctional, financial and technical characteristics addressed, severally to be handled by Cloud technologies. This is often no full and in detail group of characteristics, even so, displays the primarily generally referred ones seen in the articles. In Table 1 is proposed the characteristics to apply within the Cloud computing environment [15].

Table 1. Cloud Computing Characteristics

Aspect	Characteristic	Description	Example
Nonfunctional	Elasticity	An important primary characteristic of Cloud devices and circumscribes the competence of the under-lying infrastructure to adjust to modify , possibly nonfunctional requirements , for instance quantity and size of data covered by an program , variety of parallel consumers and so on .	Amazon EC2
	Reliability	Refers to the ability to ensure a continuous process of the program without loss.	VMware
	Quality of Service	A significant feature that's vital in many use cases where ever particular requirements must be satisfied by the outsourced providers and / or services .	Amazon S3
	Agility and adaptability	It bears on-time response to adjustments in the amount of demands and also scale of resources , even so moreover adjust to modifications within the environmental situations that e .g . require varying types of resources , unique quality or even distinct routes , or others .	FlexNet
	Availability	It can be found in the capability to present redundancy for providers and information therefore errors could be masked clear.	MS Azure
Economic	Cost reduction	Most significant worries to make-up a Cloud program which could adjust to the changing buyer behavior and also decrease costs for infrastructure maintenance and order.	Google Apps Engine
	Pay per use	The ability to build up price based on the real usage of resources.	PayPal
	Improved time to market	Vital especially for small to medium enterprises that require to vend their services immediately and also easily with lower waiting times due to getting and establishing the infrastructure , particularly in a scope compatible and competitive with superior companies .	Animoto
	Return of investment (ROI)	Is crucial for any traders and cannot often be assured – in reality . A number of Cloud systems presently lose out this feature .	-
	Turning CAPEX (capital expenditure) into OPEX (operational expenditure)	Is an implicit , and even far argued feature of Cloud devices , like the real cost benefit ( CF . ROI ) is possibly not obvious .	Efficient Servers
	Going Green	Is related not just to decrease extra fees of power consumption , it even to minimize the carbon footprint .	-
Technological	Virtualization	A key technical attribute of Clouds which hides the engineering complication from the consumer and also allows improved flexibleness .	Virtual Box
	Multi-tenancy	The place of code and / or even data is primarily not known as well as the equal resource may be allocated to several users ( maybe at the same time ) .	MS SQL
	Security, privacy and compliance	Undoubtedly vital in most systems coping with potentially critical information and code .	Almost all
	Data Management	A crucial feature especially for storage Clouds , in which information are flexibly dispersed across various sites .	WebSphere
	APIs and / or Programming Enhancements	Are equipment presented to use the Cloud capabilities .	Hadoop
	Metering	Any type of device and programs consumption is important to be able to provide elastic charges , pricing and also billing .	-
	Tools	Essential to assistance production , adaptation and also using of Cloud programs .	-

• III.  Metrics For Cloud Services Evaluation

Clearly the selection of acceptable metrics depends on the service features to be evaluated [16].

Therefore, we naturally organized the known evaluation metrics in keeping with their corresponding Cloud service options. According to the options to be evaluated, the collected metrics is associated with three aspects of Cloud services, namely Performance, Economics, and Security [17]. What we are really describing are three broad classes of capabilities that reside on top of physical cloud infrastructure, as depicted in Fig. 2.

Fig. 2. The SPI model: software, platform, and infrastructure as a service

• A.    Performance Metrics

There are several distinct options provided by Cloud vendors dealing with the IT needs of multiple companies. Each decision has got very different efficiency regarding performance, service latency and precision. Institutions ought to recognize however their programs can do on the numerous Clouds and also whether or not those deployments satisfy their goals. Performance means diverse things in many contexts. Generally, it's relevant to response time (the time could it require to process a demand) , throughput (how much a number of requires over-all might be done per unit of the time) , or even timeliness (capability to meet deadlines, i.e. , To process requesting in a settled and appropriate time period) . Many aspects of performance characteristics identified from the aforesaid major researches could be summarized as shown in Table 2 [17, 18, 19].

Table 2. Performance Features of Cloud Services

Features	Description	Metrics
Communication	Indicated to the data transfer between internal service instances (or completely different Cloud services), or between external consumer and also the Cloud.	Packet Loss Frequency
		Connection Error Rate
		MPI Transfer bit/Byte Speed
		MPI Transfer Delay
Computation	Denotes the computing data or task (job) processing in the Cloud systems.	CPU Load (%)
		Benchmark OP (FLOP) Rate
		Instance Efficiency (% CPU peak)
Memory	Designed for rapid use of temporarily store information that may be obtained from slow-accessed hard disk drive.	Mean Hit Time (s)
		Memory bit/Byte Speed (MB/s, GB/s)
		Random Memory Update Rate
		Response Time (ms)
Time	Project completion within time while maintaining quality is very important for the success of projects.	Computation time
		Communication time

• B.    Economics Metrics

It is to compare to check totally different costs of services as they provide variety of options and so have got several dimensions. However, an equivalent supplier offers completely various Virtual Machines that can meet users’ requirements. Economics has been typically considered a driving factor in the adoption of Cloud

Computing. In line with the discussion regarding Cloud Computing from the view of Berkeley [20], the economics facet of a commercial Cloud service contains two properties: price and elasticity. Thus, we collected and organized relevant metrics for these two properties respectively, as shown in Table 3 [17, 21, 22].

Table 3. Economic Features of Cloud Services

Features	Description	Metrics
Cost	Important and direct indicator to point out however economical once applying Cloud Computing. In theory, the price might cover a large range of things if moving computing to the Cloud.	Total Cost ($)
		FLOP Cost (cent/FLOP, $/GFLOP)
		Supported Users on a Fixed Budget
		Component Resource Cost ($)
		Price/Performance Ratio
		Cost over a Fixed Time ($/year)
Elasticity	Describes the aptitude of each adding and removing Cloud resources quickly in a very fine-grain manner. In different words, an elastic Cloud service issue each growth and reduction of workload, and significantly emphasizes the speed of response to modified workload.	Boot Time (second)
		Suspend Time (second)
		Delete Time (second)
		Provision (or Deployment) Time (second)
		Total Acquisition Time (second)

• C.    Security Metrics

Information security and privacy are crucial problems for almost each one company. Sharing data underneath a different organization’s administration is usually a significant concern which requires strict safety plans utilized by Cloud companies. For instance, economic institutions generally need compliance with the laws regarding data integrity and also security. The protection of commercial Cloud services has got many dimensions and challenges individuals must be worried about [20, 23] . Even so, not many safety feedbacks were mirrored in the famous major reports. Security and Safety are multi-dimensional naturally and also contains numerous characteristics similar to saving trust and safety, data integrity and availability. Table 4 shows the relevant metrics [17, 24, 25] .

Table 4. Security Features of Cloud Services

Features	Description	Metrics
Data Security	Represents a vast group of plans , systems , and also equipment utilized to guard to save , programs , as well as the related infrastructure of Cloud systems .	Is SSL Applicable
		Communication Latency over SSL
		Audit ability
Authentication	The process of determinative whether or not somebody or something is, in fact, who or what it's declared to be. Authentication usually involves verifying the validity of a minimum of one type of identification.	Meaning
		Sensitivity
		Effectiveness
		Confidentiality

• D.    General Metrics

In our work, a Cloud service is known as a process resource or a network path or a middleware service or a Cloud application. Within the Cloud, the services usually generally distributed across totally different Cloud suppliers. We tend to think about that these features are important for service suppliers. They cowl needed characteristics of Cloud solutions and supply the simplest way to assess the QoS for the Cloud services quantitative manner. As mentioned in the previous section, we tend to target QoS metrics of Cloud services that may be monitored and measured. Table 5 presents the classification of our chosen general metrics [14, 15, 17, 25, 26, 27].

Table 5. General Features of Cloud Services

Features	Description	Metrics
Availability	In classic device users entry services from web browser via the net, however right here the accessibility to that services is not assured .	Flexibility
		Accuracy
		Response time
Scalability	Represents the potentiality of enhancing the calculating power of the service provider's program as well as the product ability to process a number of clients' demands at a certain time period.	Average of assigned resources among the requested resources
Reliability	Is the power of a service to remain functional with time without malfunction . The capability of the service to keep on operating with a certain degree of efficiency timely .	Service Constancy
		Accuracy of Service
		Fault Tolerance
		Maturity
		Recoverability
Efficiency	The resources employed for services when providing the wanted operation and also therefore the amount of efficiency , underneath settled situation , that means it evaluate however nice the web services employs the sources .	Utilization of Resource
		Ratio of waiting time
		Time behaviour
Reusability	Known as the level to which a application component or even other work system may be used in a number of program or application .	Readability
		Coverage of variability
		Publicity
Compos ability	Related to the interoperability characteristics.	Service Modularity
		Service interoperability
Adaptability	The level of efficiency in adjusting the solutions for the utilization of every service based software .	Completeness o/ Variant Set
		Coverage of Variability
Usability	The quantity to which a service could be used by particular consumers to gain certain aims with usefulness , effectiveness and also approval in a certain background of usage .	Operability
		Attractiveness
		Learn ability
Modifiability	The capability to make modifications to a product rapidly and cost-effectively, in case service interfaces have to be need to , the customization might generate difficulties as a result of when service interfaces are released and used by programs.	MTTC (Mean Time To Change)
Sustainability	For defined in terms of the environmental effect of the Cloud service employed . It might be calculated as the usual carbon footprint or even energy capable of the Cloud services .	DPPE (Data Centre Performance per Energy) parameter
		PUE (Power Usage Efficiency)

• IV.    Conclusions And Future Work

Cloud is now a crucial paradigm for outsourcing diverse computer needs of institutions. Presently, there are lots of Cloud vendors that supply totally various Cloud solutions with different cost and also functionality characteristics. With the increasing variety of Cloud products, while it opens the opportunity to control the almost unlimited calculating sources of the Cloud, it's besides be hard for Cloud consumers look for the ideal Cloud providers who could fulfill their QoS requirements relating to variables similar to privacy and performance. To select suitable between several Cloud companies, clients ought to have the method to recognize and also evaluate crucial performance standards, which are necessary to their programs. The choice of metrics has been known as being essential within the evaluation of computer systems. In fact, the metrics choice is that the requirement of the many different evaluations steps including benchmark selection. Within the context of Cloud Computing; however, we've not found any systematic discussion regarding the evaluation metrics. Therefore, we tend to be planned an investigation into the metrics appropriate for Cloud service's evaluation. Because of the lack of consensus on the standard definition of Cloud, it's tough to denote the complete scope of metrics ahead for evaluating totally different Cloud services. As mentioned within the motivation of constructing this metrics catalogue, you can easily in turn apply the developed catalogue to help the long-term work of assessment of commercial Cloud companies. Consequently, this metrics catalogue is used to facilitate.