Evaluation of the quality of innovative software products on the basis measurement values of functional indices

However, the use of information systems and ISP in itself is not yet a guarantee of the financial well-being of industrial enterprises in the industry. The successful development of enterprises largely depends on the quality of the development of the information systems and ISP used. It is in determining the quality of these products that the biggest problem lies. To solve it, it is necessary to determine suitable methods and technologies, the composition of functional indicators, formulate clear requirements for them, select metrics and measurement scales to obtain qualitative and quantitative values of indicators.

Research purpose. The purpose of this study is to form an approach to measuring the qualitative and quantitative values of functional indicators of information systems and ISPs for assessing their quality in multicriteria expert systems.

Material and methods. It is obvious that in the process of carrying out production activities by enterprises, the new information constantly appears and there is a loss of relevance of existing information. This circumstance should be taken into account when evaluating the quality of information systems and ISP. That is, we should talk about the preestablished qualitative and quantitative values of functional indicators and their compliance or non-compliance with the calculated values obtained as a result of measurements at a specific time.

There are many definitions of ISP quality based on the aggregation of various functional indicators. In the future, they are used to assess the consistency of the calculated values with the established requirements [4, 5]. For example, the International Organization for Standardization (ISO) defines quality as the completeness of the properties and characteristics of a product, process or service that ensure the ability to meet stated or implied needs [6]. At the same time, each property can correspond to one or more ISP functional indicators, which are measurable attributes of a qualitative assessment of this property. These indicators can be evaluated both by individual and complex quality indicators. A single quality indicator, as a rule, refers only to one of the properties of the ISP. While a complex indicator of the quality of an ISP always refers to several of its properties. Accordingly, a quantitative assessment of these characteristics will allow us to determine the degree of presence of a specific property in the ISP, as well as the level of its quality [7, 8, 9].

Therefore, when conducting this study, we will be interested in choosing metrics and scales, as well as methods suitable for measuring quantitative values of the quality of specific functional indicators and attributes of various properties of an ISP and their subsequent evaluation. It should be noted that the quality assessment of an ISP is influenced not only by functional indicators determined by the composition of the requirements for its functional purpose, but also by indicators determined by the composition of the requirements for the information content of specific areas of knowledge [10]. A set of these characteristics is described in the ISP quality model included in the ISO/IEC 25010:2011 Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models [11]. Its analogue in Russia was GOST R ISO/IEC 25010-2015. Information technologies. System and software engineering. Requirements and quality assessment of systems and software (SQuaRE). Quality models of systems and software products [12].

Nowadays, there is a fairly representative range of methods and tools for measuring the values of specific functional indicators and attributes of their properties, followed by an assessment of the quality of both the ISP as a whole and its individual components (modules). Among them, we will highlight the methods of the applied statistical analysis, expert assessments, as well as more complex methods of intellectual analysis, such as neural networks, methods of machine data analysis and a number of others.

The use of different models, methods and tools for assessing the quality of information systems and ISP has its own characteristics. Thus, the use of mathematical methods is aimed at improving the objectivity of quality assessment. However, as practice shows, the use of expert assessment methods allows to increase the efficiency of assessing the quality of information systems and ISP. They provide the construction of the most adequate and complete models of the processes of functioning of information systems and ISP, the quality of which is subject to measurement. Expert methods include a sufficiently developed set of techniques and tools based on the use of the qualifications, experience, and creative potential of one or more specialists in a particular field of knowledge [13, 14, 15].

Results and discussion

Given the diversity of user interests, assessing the quality of information systems and ISP seems to be quite a difficult task. As a rule, it is impossible to use only one universal quality measure to solve it. Most often, a set of certain quality indicators and their variable values are formed, covering the entire set of requirements for the quality of functioning of modern information systems and ISP. Currently, various models for assessing the quality of their most important component - software-have been widely used to solve this problem [16, 17].

To assess the quality of information systems and ISP, a set of indicators corresponding to their functional characteristics is most often used. It is combined into a multi-criteria expert system, provided that for each indicator, each of the characteristics included in this system can be measured using different measuring scales and can be expressed in certain units of measurement that allow establishing certain relationships between them.

Variable values of quality indicators can act as independent attributes of information systems and ISP. They are used to measure individual indicators of the quality of information systems and ISP based on a specific metric scale – an accepted measurement system at the level of individual indicators or at the level of their variable values. The combination of several variable values allows you to determine the quality of a separate functional indicator. Initially, it is formed taking into account the requirements for the functioning of information systems and ISP.

In modern conditions, the hierarchical model of the relationship between the quality indicators of information systems and ISP is the most common one. First, a system of indicators is formed, which can include such indicators as: efficiency, integrity of information content, correctness of relationships between individual modules, ease of maintenance, measurement capability, flexibility, adaptation to user requests, mobility, and the possibility of interaction. Next, variable values of its quality are determined for each indicator. They may include general or initial utility, ease of operation, etc. Each functional indicator of the quality of information systems and ISP is assigned its own group of variable values. Below is a number of possible variable values for some functional indicators:

• -    operability – the ability to function on different information platforms, the efficiency of memory use, communication, the amount of information input, the speed of information input and output, the accuracy of calculations;

• -    integrity of information contentregulation and control of access to information, depending on the established user permissions;

• -    relationships between individual modules – the correctness of information transmission, completeness and consistency of information flows between modules;

• -    reliability – resistance to user errors, consistency and availability of information content by knowledge areas;

• -    ease of maintenance – simplicity, brevity, informative interfaces, information interconnection of modules;

• -    the possibility of measurement – the availability of methods and tools, the simplicity of measurements, the informativeness of the calculated values;

• -    flexibility – the ability to distribute on different media, the ability to quickly format, the ability to function with a limited number of modules;

• -    adaptability – information content of interfaces, information interconnection of modules, independence of technical and software;

• -    mobility – information content, modularity, independence of technical and software;

• -    the possibility of interaction is modularity, unification of communication and data transmission procedures [18].

With metrics and measurement scales, it is possible to obtain an assessment of the quality of information systems and ISP through the use of variable values of their functional indicators.

With the help of measuring scales, the range of values of the measured functional indicators is usually set, as well as the specified accuracy and units of measurement are set. Different measuring scales are used to measure qualitative and quantitative indicators. Qualitative indicators are usually measured using nominal or rank measurement scales, and quantitative indicators are measured using interval scales or ratio scales.

The nominal scale of measurement of functional indicators of the quality of information systems and ISP is convenient to use when all indicators are divided into certain classes on one basis, and quality indicators from different classes have different values. But when using the nominal measurement scale, there are a number of disadvantages. They consist in the limited possibilities of statistical processing of variable values of functional quality indicators. So, when using nominal scales, the operations of determining the arithmetic mean or median do not make sense. For variable values of functional indicators of the quality of information systems and ISP, using nominal scales, it is possible to calculate the mode, analyze frequencies and various groupings. These actions are based on the division of the general data set into categories of variable values of functional quality indicators.

The rank scale of measurements of functional quality indicators of information systems and ISP is used in the case when the variable values of quality indicators are divided into classes, and the classes themselves are ordered. An important characteristic of the rank scale is the empirical significance of the variable values of the functional quality indicators measured in it. For variable values of functional quality indicators of the rank scale, you can determine the values of the median and mode, as well as perform any monotonic transformations, procedures for testing the hypothesis of equality of the median to a given value, testing the hypothesis of equality of two medians, using rank correlation methods with the calculation of the Kendall and Spearman coefficients. As a disadvantage of using rank scales, it should be noted that it is unacceptable to perform any arithmetic operations.

For the interval scale of measurements, not only the absolute values of functional quality indicators, but also the difference (interval) between them have empirical significance. Numerical values of functional quality indicators when using interval scales are determined with accuracy up to linear transformations. In the case of using relationship scales, the numerical values of functional quality indicators are determined up to the similarity transformation. At the same time, the relations between the values of functional quality indicators already have empirical significance here. Practice shows that the difference between the variable values of functional quality indicators related to the interval scale or the scale of relations is usually insignificant. The variable values of functional quality indicators determined by the interval scale usually differ little from their similar values determined by the ratio scale.

The choice of measurement scales is an important stage in assessing the quality of information systems and ISP at the level of variable values of functional quality indicators. The possibility of applying a number of operations that are applicable only to specific measurement scales depends on the correct choice of the measurement scale. However, it should be remembered that for some meas- urement scales, it does not make sense to perform a number of operations.

Basing on the possibility of measuring the variable values of functional indicators and their accuracy, all the attributes of information systems and ISP quality indicators can be divided into 3 categories that have their own characteristics.

• 1.    Interval or relative scales should be used to determine quantitative indicators. They allow you to objectively measure and compare the calculated values obtained with the initial requirements for quantitative indicators in numerical terms.

• 2.    To determine categorical indicators, nominal scales of measurement of individual categories and their properties should be used.

• 3.    To determine the quality indicators, subjective assessments should be used with the involvement of experts. These estimates contain separate features-categories, for the description of which ordinal or nominal scales can be used.

Currently, there are many metrics and scales for evaluating the quality indicators of information systems and ISP. All of them can be classified on different grounds. The classification of metrics and scales according to the directions of qualitative assessment is proposed below:

• 1)    analysis of the reliability of information systems and ISP, which allows you to investigate situations of possible failures and predict the probability of their occurrence;

• 2)    analysis of language tools, their level and expediency of use in a specific information system or ISP;

• 3)    analysis of the performance of information systems and ISP by identifying implementation errors, ways to improve their efficiency;

• 4)    analysis of information and topological complexity of information systems and ISP.

• 5)    analysis of the complexity of perception of information systems and ISP, cognitive ergonomics, the results of which are valuable at the stages of development, design, as well as making changes and maintaining information systems and ISP.

• 6)    cost analysis of the developer, necessary for the feasibility study of the project for the development of information systems and ISP,

as well as for forecasting the timing of the entire project and its individual stages.

The quantitative analysis of the qualitative characteristics of the functional indicators of information systems and ISP can be evaluated on the basis of a set of metrics classified by the type of information received or by the type of information used:

• -    by the type of the obtained information on information systems and ISP metrics are classified as:

• а)    determining the deviation from the given norm of indicators;

• b)    indicators’ forecasting values;

• c)    determining the fact of the correspondence of information systems and ISP with formed requirements;

• -    by the type of the used information on information systems and ISP metrics are classified as:

• а)    metrics, based on the lexical analysis of information systems and ISP (Halstead, Jilb, Chepin metrics etc.);

• b)    metrics based on the assessment of information systems and ISP management flows (McCabe metric);

• c)    metrics that evaluate the intermodule and intramodule connections of information

systems and ISP;

• d)    metrics that analyze data flows of information systems and ISP.

The main idea of all these 4 groups of metrics is to determine the correctness and complexity of the information systems and ISP being developed, as well as to predict the parameters and characteristics of future versions of information systems and ISP. By combining the metrics of groups b), c), d), it is possi- ble to obtain the so-called metrics of functional connectivity and coupling of modules. At the design stage, the calculation of these metrics allows us to determine the reliability of information systems and ISP by evaluating the minimum coupling and maximum strength of the modules being developed.

Opinions

The conducted research allowed us to identify the features of measuring the quantitative values of the functional indicators of information systems and ISP for assessing their quality in multi-criteria expert systems. An approach to assessing the quality of information systems and ISP is proposed from the standpoint of pre-determined quantitative values of their functional indicators at a specific time and their compliance or noncompliance with the calculated values obtained as a result of measurements and subse- quent evaluations. It is established that it is advisable to use various methods, models, metrics and measurement scales to measure the quality of information systems and ISP, with the help of which it is possible to determine the calculated values of variable characteristics of their quality indicators.

The main conclusion from the results of the study is that there is no universal multicriteria expert system for assessing the quality of each specific information system or ISP. To do this, it is necessary to develop an individual multi-criteria expert system of quality indicators. At the same time, the integral assessment of the quality of a specific information system or ISP will be determined based on the calculated values of the variable values of their functional indicators.