The factor analysis and the search for an objective meaning of factors as a function of meanings of (names) features

Our approach to the theory of the factor analysis is closely connected with the theory of committee decisions, so I have decided to pay attention to the origins of this theory.

A factor – a latent source of dynamics of interdependent features of objects and phenomena.

The factor analysis is a method of multidimensional mathematical statistics. It is used in order to search for statistically associated features and for their assessment, in order to emphasize key hidden factors, functions of which are presented by features.

The founder of a factor analysis – English explorer, sir Francis Galton (1822–1911), geographer, anthropologist, psychologist, founder of differential psychology and psychometrics, statistician. In the 1850s he developed root ideas of a factor analysis implementing them into psychological problems of individual differences. The goal is to create a mathematical model of individual differences.

Then in 1901 English mathematician, statistician and biologist, Karl Pearson (1857–1936), founded mathematical statistics and biometrics; and suggested an idea of principal axis method.

English psychologist and mathematician Charles Edward Spearman (1863–1945) studied a two-factor model of human intelligence and distinguished a general factor.

The traditional procedure of finding the factors is performed through equations of the dependency of features from factors [1–3]. I have suggested a complex method – search for dependencies of factors from features.

The traditional approach includes factors taxonomy [4]. Along with this one relies on the statistics of learning material. In my approach we deal with algebraic procedures without mathematical statistics.

Mathematical models and methods of solutions committee dealing with pattern recognition problems are considered, including discriminant analysis, taxonomy and assessment of information capacity of signs subsystems [5]. Among the committee structures there is the committee of the majority which is the main one. This is one of the models of the experts’ council. The main objective is to find a decision rule of pattern recognition.

The objective consists in the following. We need to find a committee of discriminant functions for the case of sets A and B. The discriminant function f , if it exists, satisfies the system of inequalities (*):

F (a) > 0 for all а from the set А, f (b) < 0 for all b from the set В, f can be found in the functional class F.

However, this system can often be inconsistent, and then, instead of one function we create the committee of C functions.

This is such a finite sequence

С = [f1, …, fq], that more than half of the functions from the set C satisfy each inequality of the system (*). Some of the functions of the set can be repeated.

The connection of these methods with the factor analysis is studied in this article which allows finding deep connections in the observations table, as well as with artificial neural networks. In contrast to traditional methods based on mathematical statistics (which require large amounts of observations and require search of dependencies of the features from the factors), we suggest an algebraic approach based on the committees method.

Our work is supported by RAS academician V.I. Berdyshev and by RAS academician Yu.I. Zhuravlev who is in charge of the entire area of algebraic models and methods of recognition and their mathematical and practical justification in the Russian Federation.

However, it is necessary to start with the initiative of the outstanding mathematicians S.B. Stechkin and I.I. Eremin who set the task to prove necessary and sufficient conditions for the existence of the committee system of linear inequalities for me in 1965.

The results of fundamental studies by RAS academician I.I. Eremin in the field of theory and methods of solutions and optimal correction of inconsistent systems of equations and inequalities, and conflicting objectives of an efficient (in particular optimal) choice determined the direction of further development of the theory and methods of operations research and pattern recognition.

One approach to this correction is associated with the collective search for generalized solutions to infeasible systems of constraints and this approach relies on different voting logic (democracy), the simplest of which is associated with decision making by majority vote [6].

The original sources of the committee theory can be found in some American works on artificial neural networks – in Nils Nilsson’s, Ablow’s and Keillor’s algorithms [7]. However, they believed that the neural network is an engineering discipline, and therefore, they did not set the task of rigorous mathematical justification of the relevant algorithms for themselves.

There is a variety of conceptions on the basis of which decision rules for diagnosis and classification are made [8, 9].

The method of collective decisions has a wide range of applications in the area of pattern recognition and classification of objects and situations, where the learning algorithms known as committee machines, associative machines and boosting machines. Despite the apparent proximity of these approaches, for some reasons they have been developed independently for a long time [10, 11].

М.Yu. Khachay in his doctorate [12] noticed that it is possible to combine the committee theory with the theory of empirical risk.

The researches of Institute of Mechanics and Mathematics URAN are currently under development (Mazurov, Tyagunov, Kazantsev, Krivonogov, Sachkov, Beletskiy, Gaynanov, Matveev, Khachay) and their aim is to identify deep connections between these approaches that will contribute to further development of these approaches [13, 14]. So, for example, an original and profound book by D.N. Gayvanov was published which is based on combinatorial geometry and graph theory [15].

Methods of the synthesis of neural networks on the basis of the committee method have been developing.

On the basis of the problem of the minimal affine separating committee (the simplest piecewise linear classifier based on majority vote) a game theoretic approach to the development and justification of the approximate, particular polynomial learning algorithms for the recognition and classification of objects and situations is studied.

The problem of development of affine separating committee is a discrete generalization of the problem of a separating hyperplane in Euclidean space for the case of the partial sets, convex hulls of which intersect. If the partial sets intersect then the statement of this problem in this case is naturally immersed in a finite-dimensional space of a suitable dimension [16, 17].

One of the committee methods uses the analysis of finite and infinite systems of inequalities – linear and non-linear – they may be joint or disjoint. The committee method is connected with this approach [18].

However, on purpose the committee method is not limited to the separation of two finite sets of the same function. It possesses other features: there is no need to formulate separability hypotheses, and another feature is about the existence of the compactness axioms. But the only weakest necessary condition is supposed to be fulfilled: for training sets of different classes not to intersect. It is important that according to this minimal condition there is always a committee consisting of affine functions. G.S. Rubinstein noted the connection of the committee theory with the task of systems of different representatives of sets [14].

Note that the committee C is actually a set of factors.

Another approach involves the minimization of the empirical risk (V.N. Vapnik [19]). V.N. Vapnik has developed the theory of statistical learning problems. He has generalized Glivenko’ theory [20] and he has developed the theory of uniform convergence of frequencies of events to their probabilities and he has introduced a diversity measure of function classes.

Y.I. Zhuravlev‘s approach [21, 22] – evaluation method is connected with mathematical principles of classification, this evaluation method covers many algorithms of recognition including heuristics. In particular, he develops the algebra of algorithms, including heuristic ones. And in this algebra he finds the optimal decision rule.

The great variety of books and articles is devoted to the factor analysis, but there is still some kind of mystery in this topic. There is even a completely informal part of the algorithm. This is the process when a factor obtains its meaning which combines the signs with their names. Since we deal with the names of the signs and factors, we use the methods of mathematical linguistics [23, 24].

The naming algorithm obtains a fairly complete formalization.

Here the comment by J.F. Lyotard becomes appropriate: in Sensus communities we come across with the way of thinking that is not purely philosophical or mathematical [25].

Now to the algorithm of calculation of a factor name according to the names of signs that are included in the corresponding taxon. We apply the factor analysis to the observation table of object\sign.

The first stage is creating taxon columns of signs according to their correspondence to objects. In the matrix object\sign: lines are sign values when objects are presented on the line, columns are signs when they appear in objects.

The method consists of the following: it is necessary to divide a finite set P into taxons in the space Rm. And let the form of the taxon be formulated.

The second stage for each taxon (which corresponds with the taxon object) is to write the word from the names of the signs. This word will be the name of the factor.

The third stage is the compression of a bigger word in order to convert it into the name of the factor.

Now we put it down using symbols. The observation matrix A is presented in two ways – through lines and through columns:

А = [С 1 …C m ]* = [P 1 …P n ].

Here С* j – lines, Р i – columns, * – sign of transposition, а (c* j ) – name of the object, а (P i ) – name of the sign.

Let us take one taxon Т from the columns set:

Т = {Р i : i ∈ I}.

The method of its finding consists of the following. Р – finite set in the space Rm. And taxon form is presented:

Т = { x : f ( x ) < 0} ∩ P.

Here f is taken from a possible set of functions F.

Taxon Т corresponds with the factor with the name [ a (P i ): i ∈ I]. This “big” word consists of “small” words a (i). This is the name of the factor. This word can be compressed, if needed.

It is possible to suggest another approach to the definition of the meaning of the factors. Namely the meaning of the sign or the factor is the set of their contexts. The meaning of the factor a is the set of its values or meanings V( a ) . So, the meaning of the sign b is the set of its meanings or interpretations V( b ) . If the factor is a combination of signs a then its meaning is in the intersection of the corresponding sets V( a ) . This set is done using an electronic dictionary of synonyms.

We have already noted that the factor analysis began with the analysis of psychological researches. Let us give some information about the publications. Apparently the first work in this area belongs to F. Galton on tests theory. In 1901 К. Pearson published an article “On lines and planes of closest fit to sys- tem of points in space”, where the idea of the principal axes was discussed. Then in 1904 Ch. Spearman published an article “General intelligence, objectively determined and measured” in “American journal of psychology”. G. Rorschach introduced psychological tests in 1921.

The factor analysis originated in sociology in 1940. From that moment the process of wide application of this data processing tool has begun.

The research is supported by Russian Science Foundation grant no. 14-11-00109.