Cognitive fuzzy control system: examples of intelligent control and therapy of the autism applications
Автор: Reshetnikov Andrey, Ulyanov Sergey, Mamaeva Alla
Журнал: Сетевое научное издание «Системный анализ в науке и образовании» @journal-sanse
Статья в выпуске: 4, 2017 года.
Бесплатный доступ
The article show the possibility of neurointerface application based on cognitive helmet with different types of controls for driving of the vehicle. An assessment of possibilities of application of intelligent computational intelligence and toolkits to improve the reliability of the fuzzy control system operation.
Cognitive system, intelligent system, adaptation, neurointerface
Короткий адрес: https://sciup.org/14122659
IDR: 14122659
Текст научной статьи Cognitive fuzzy control system: examples of intelligent control and therapy of the autism applications
КОГНИТИВНАЯ НЕЧЕТКАЯ СИСТЕМА УПРАВЛЕНИЯ: ПРИМЕРЫ ИНТЕЛЛЕКТУАЛЬНОГО УПРАВЛЕНИЯ И ПРИЛОЖЕНИЯ К ТЕРАПИИ
АУТИСТИЗМА
Решетников Андрей Геннадьевич1, Ульянов Сергей Викторович2, Мамаева Алла Александровна3
1 Кандидат технических наук, доцент;
ГБОУ ВО МО «Университет «Дубна»,
Институт системного анализа и управления;
141980, Московская обл., г. Дубна, ул. Университетская, 19;
2 Доктор физико-математических наук, профессор;
ГБОУ ВО МО «Университет «Дубна»,
Институт системного анализа и управления;
141980, Московская обл., г. Дубна, ул. Университетская, 19;
3 Аспирант;
ГБОУ ВО МО «Университет «Дубна»,
Институт системного анализа и управления;
141980, Московская обл., г. Дубна, ул. Университетская, 19;
Introduction: the problem of cognitive and intelligent control
So far the theory and design processes of intelligent control systems (ICS) as knowledge-based control systems (in the form of relevant knowledge base (KB)) was carried out by an experts themselves through computational intelligence as soft computing using genetic algorithms or fuzzy neural networks.
The role of the human operator control circuit was not considered in an explicit form or it was described by transferring simplified functions. The very inclusion of human operator control circuit is often seen as a source of emergency or increasing risk information from the decision. It has so far proved [1] that in a circuit of control of big multiple loop connected management systems to 75% of information often there is an excess quantity which isn't used or hinders decision-making. Therefore, one of the central problems of developing cognitive control system (CCS) was finding a constructive solution of design KB tasks in a given issue oriented applications.
However, cognitive abilities of operator (such as intuition, instinct and emotions) decision in difficult situations is both informational resource that enables to improve the effectiveness of the development and application of KB. Experimental studies of the brain and behavior of the human operator confirmed the hypothesis about the relationship of electrical excitation EEG of individual sections of the brain (neurons or groups of neurons) with determination and prediction of human behavior.
Thus, there is an opportunity to apply cognitive processes of the brain the human operator as friendly "brain-computer interface" with a view to enhancing the effectiveness of predictive control to guarantee the achievement of management objectives in the face of uncertainty, contingencies and the increasing risk of information uncertainty.
The aim of this work is to show the possibilities of experimentally effective application of cognitive interface ("brain-computer-actuating device") on the example of driving (mobile robot), reveal the modern management technologies and show the role and necessity of application of intelligent computing in the work of the "brain-computer interface" to improve the reliability and robustness of the control system.
In particular, the paper considers the possibility to control the movement of the object (forward, backward, left, right, bypass obstacles) through cognitive helmet using building block commands recognition and different types of control systems, including those based on soft computing.
For removal of brain, activity used cognitive helmet Emotiv EPOC+ (fig. 1), in conjunction with the supplied software (EPOC Control Panel).
The sensors

The software includes a block record and recognition of mental commands generated by the operator. EPOC has 14 electrodes, which are passive sensors that allow registering electromagnetic signals and transmit them via Bluetooth to a computer for processing. The electrodes are mounted on the surface of the skin (not submersible, non-invasive interface) and require wetting special liquid for better contact of the so-called "wet" interface.
Structurally the Emotiv EPOC+ consists of 14 channels AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4 (plus CMS/DRL and P/3P4).
Sampling method sampling is serial. Sample rate 128 samples per second (inner 2048 Hz). Resolution is of 14 bits. Bandwidth of 0.2-45 Hz, digital notch filters for 50 Hz and 60 Hz. There is an additional filtering of digital filter.
The software supplied allows you to retrieve, identify and record the EEG signal with helmet, mental commands and assign certain actions to them (for example, to send a signal to a Bluetooth device for autonomous robot movement forward, backward, left and right).
Mental training signal is resting and recording conditions, when the subject is the image of any action, concentrating in certain brain activity part of the brain. The entry of these signals takes 8 seconds.
Figure 2 shows an example of the EEG signal, as well as the arrangement of electrodes of cognitive helmet.

Figure 2. On the top the signal is EEG electrode from EROS. The below arrangement of electrodes on the head

In fig. 3 electrodes are presented in accordance with the functional areas of the brain [2-4].
Operator training takes place as follows: the operator is put on a helmet and through special software recorded signal its "neutral status" (resting State), then it is a signal of mental commands. To that end, the operator monitors and control object generates a mental command to its movement in the right direction, focusing and stimulating certain parts of the brain (the frontal, occipital or temporal lobes). From the point of view of the control system, the operator must be able to once again generate the recorded signals that are interpreted by the control system to control the movement of the device, for each mental commands recorded individual signal. In other words, the recording signal to move forward the operator initiates the frontal brain share and for team back-occipital, etc. (fig 3.).

Figure 3. Functional areas of the brain
One of the major technology component of cognitive neurointerface is gaming simulators. It is important to note that in the cases training, as an object of control model of the system acts on the screen, this in turn allows you to not only train your brain to generate mental commands, but also customize the object control system control adjusting it to the operator, in order to enhance the effectiveness of the system of "brain-computer". This kind of feature is due to the fact that how a person learns to work with the helmet, and the program itself should be adjusted to the particular person by configuring a control system.
Recognition quality of such control system is usually associated with the level of training of the operator to generate different mental commands. Bundled software supplied specialized games-trainers, that using in the process of learning and training. Usually, for this use of computer games, where the operator must hold level an activity that will be associated with a system command that is used to manage the object. While playing, the operator develops his skill in working with a helmet that gives you the ability to control in the future and the real technical devices (manipulators, wheelchairs, etc. devices).
Figure 4 presents software games simulators-left program Tetris in which the operator is invited to training commands left and right, as well as switching between forms, the program on the right - Spirit Mountain, where the operator training the brain for generate the comands forward, backward, left, right, up, down, for controlling the character in the virtual world [5].

Depending on the number of mental commands that you must generate, workout divided into levels of complexity. Workouts can be active-with external stimuli, such as light of a certain frequency, video and audio material, pictures, etc., and passive, if in the process of generating mental commands the operator presents it without external influence. Duration and classes of workouts affect recognition quality and quantity of mental commands. If the first time a person can learn to generate signals in the brain to one command. For good recognition of two or more commands, need some training.
Accordingly, the control and the achievement of control objectives will influence how the Psychophysiological features of the human condition (including his position in space) and its level of training. Normally (by the manufacturer) for a good operation of the system using four commands, it is necessary to conduct regular workout 2-3 weeks, and after practice, the operator was experiencing fatigue and accordingly need time to recuperate.
Figure 5 shows the EEG, shot in a neutral State and in the State where the operator is far ahead.

Figure 5. EEG without mental commands (neutral State) and EEG with mental command (forward)
Usually, for the evaluation and recognition of signals used statistical techniques, including calculation of dispersion and math expectation. Figure 7 contains a table that highlights the most distinct signal associated with the traffic ahead and neutral State.
FCS |
FC6 |
О1 |
02 |
P7 |
PS |
T7 |
T8 |
Neutral |
126,66 |
3459,63 |
2108,80 |
3791,51 |
2940,07 |
3757,97 |
3768,52 |
2523,59 |
|
103,47 |
4201,47 |
1644,94 |
3894,85 |
3072,52 |
3129,58 |
3455,78 |
1613,89 |
Forward |
Figure 6. Signal Dispersion during the neutral status and with the mental command.
The table highlights the most distinct signal associated with the operator with the traffic ahead and neutral State. Dispersion of signals generated by the operator, it is quite a simple and effective method of comparing signals mental commands. Further, various methods are well showed in [2]. Figure 6 shows that on electrodes FC6, T8 and r8 the difference signal dispersion is most essential. This gives grounds to classify control signal in software.
In fig. 7 shows the location of these electrodes.

Figure 7. Electrodes with the greatest difference in dispersion
It is important to note than in the case of training, the object control system is also tuned to improve the efficiency and robust of the brain-computer system. Thus, the control object is the system is the control model. This allows you to train your brain to generate mental commands and tuning software of the control system. This kind of feature is because how a person learns to work with the helmet and the program itself should be tuned to the particular person by adaptation a control system.
Structure of cognitive regulators
One of the objectives of this paper is to compare different regulators for operating a device using cognitive Slam, evaluate the possibility of using methods and means of intelligent control and soft computing optimizer (SCO)[8] of KB to improve reliability and efficiency of the on-board control system.
-
- Proportional controller .
The structure of this control system is based on the principle of the relay and making the control action proportional to the detected signal. From the power of recognition, the value of the output signals are received in the range of -1 to + 1. Number of outputs equals the number of recognition recorded commands. For our task it was team forward, backward, left, right. Movement device begins when output signal of recognition becomes more (or less) of a certain specified threshold, the direction of movement is determined by the sign of the output values and speed of movement (control) is proportional to the output signal size from recognition. Thus, for the command "movement forward" and "backward movement" was chosen range from 1 to -1 respectively.
Block diagram of the system is presented in fig. 8.

Figure 8. Block diagram of proportional regulator circuit cognitive control
Recognition signal unit is supplied together with the used equipment and integrates as a library in the development Wednesday.
-
- Proportional-integral (PI) controller with fuzzy output block.
In the shown structure of output and signal recognition unit come on fuzzy output unit with an integrated knowledge base. The knowledge base has two inputs for proportional and integral component of the signal recognition unit, knowledge base thus implements the PI regulator input values are the corresponding values of the signal, and output value of control action fed engines of vehicle. Structural scheme shown in fig. 9.

Figure 9. Block diagram of the program with a block of fuzzy inference and knowledge base
For more effective operation of the control system, you can use a more complex controller, for example, proportional – integral - derivative (PID) controller, where depending on the received command with the block of recognition, is implemented by maintaining an appropriate speed and rotation angle to achieve certain values of control. Control effect in this block is consulate to the follow formula:
u (t) = P +1 + D = K e(t) + K Je(t)dT + Kd — p i 0 d dt
-
- PID controller with different coefficients of amplification (fig.10).
supervision over
block signal recognition
Figure 10. Block diagram of the program with PID regulator
PID has three components, which depending on the incoming signal with the block recognizer in sum produce control signal. PI and PID regulators respectively set limits on the terms integral and differential component.
This work does not considered task of tuning gains of ratios of PID regulator, but shows that the choice of reinforcement ratios significantly affects system performance. Next, in the conclusions of the work shows the possibility of using modern types of calculations to add functions of learning and adaptation in the structure of the control system on the software level.
Designing cognitive control regulator for autonomous robot
For the experiment was been select the object of control - mobile robot in the form of three-wheel vehicle with Bluetooth-control (fig. 11).
Microcontroller Arduino
Motors
BlueTooth receiver
Power source
Figure 11. Control Unit
The control device as a control processor used the Arduino Uno. Together with the engine driver- Polo-lu Dual MC33926. 2 (micromotor) Motor- DC 9V Motor Bluetooth module- HC-05. Power supply serves 3 3.7V Li-On battery.
The first and easiest implementation regulator for vehicle is a proportional controller. Such a regulator sends a proportional value of motors cars depending on which team has the greatest affinity to recorded in advance mental command (fig. 12 ).
level of activation of team a 0.7 --——--—--- -
_ entrance signal in
° в0 the regulator g C E level of activation of team
0.4
0,3 forward backwards
0.2
0.1 mental team
Figure 12. Activating the commands in the proportional controller

Figure 13. Control impact produced p-controller while moving back and forth
For example, activation command associated in pairs of movement forward and backward was made according to the difference of activation levels for these commands.
Next, let us look at the process of design PI regulator using SCO. To do this, in the first phase the expert generates a training signal, driving based on proportional regulator machine. During system operation, recording the signals received from the block recognizer. Coming from this signal by adding formed integral component (fig. 14). Then the expert put the respective control impact based on previous experience with the system.
To approximate the teaching signal (fig. 15) applied the developed SCO with selected the model of fuzzy inference (Sugeno type models). As teaching signal used the signal from the block signal recognition EPOC, as well as the integral value of the signal.

Figure 14. The training signal.

Figure 15. Neural network fuzzy inference
At the next stage of design for fuzzy knowledge base withdrawal is carried out in automatic mode formation full knowledge base and further optimization of right-hand sides (fig.16).
program

operator
regulations




Figure 16. Block diagram the formation rules in the knowledge base
In other words, at the entrance to the neural net receives commands from the software module signal recognition (forward, backward, left and right) the output value is the commands then receives vehicle.
The knowledge base is applied in conjunction with PI controller. Using soft computing need to build more "soft" structure to control. For this purpose created linguistic variables (LV) for each of the commands recorded in the system, was formed a complete knowledge base (fig. 17). The right side of the regulator contains appropriate values for control action using PI controller. Thus, the activation level of rules in base corresponds to the activation level of the control action

Activation level in system of recognition
Figure 17. An example of the linguistic variable for the team forward
Figure 18 shows the result of cognitive motion control of mobile robot in maze based on the PI-regulator.

Figure 18. When controlling the trajectory of mobile robot based on the PI-regulator
The following verified regulator to control machines was PID controller with constant coefficients. The coefficients of the regulators were PID 1 [1 0.1 1] and for PID 2 [3 0.1 10].
Results: modeling and experiment
In fig. 19 and fig. 20 shown the commands of control systems to manage control object. The first chart, green introduced the target signal, which corresponds to a movement back and forth, and the rest of the colors allocated to the activation levels (Epoc) and PI controller with knowledge base.

Figure 19. Controlling actions produced by standard signal and PI regulator. Forward and backward
As can be seen from the graphs in Fig. 19, when the task motion vehicle is back, and concentration occurs thinking process on that team, recognition block is not always correctly identifies and control the machine. Vehicle work in spurts or even goes to the other side (false positives), PI control compensates this, and additional add-in as an integral component in the knowledge base, allows a smooth sequence of commands and reduce errors in reaching the goal. Moreover, the system becomes adaptive and learnable, because The basis of the base is the software tool SCO.
On fig. 20, move to right corresponds to 1, and the movement to the left corresponds to -1.

Figure 20. Control actions produced by relay and PI adjuster when moving left and right.
Additionally there was decry the problem of the motion using control system with PID regulator
(fig. 21).

Figure 21. Detour obstacles control system with PID regulator
Figure 22 presents the results of an experiment using PID controllers with different coefficients of gain control action. The odds were set in manual mode. Differential component in PID controller associated with the speed of the operator activates the mental command.

Figure 22. Control actions produced PID regulators when driving forward
As you can see, the choice of gain factors influence naturally on the computation the action of controller and the operation of the system as a whole. However, establishing the optimal values of coefficients for each point in time is relevant and very interesting task. When incorrect (false) installation values the same way there has been an incorrect actuation, control object moves in spurts.
To compare the results obtained in the experiments used value is the mean deviation from the desired result. As can be seen from table 1 and fig. 23, using a more complex controller, the deviation has reduced. However, the wrong setting of the gain increases the deviation of the system from the intended target.

Figure 23. Cumulative score deviation module
Table 1. Compare mean deviation of different controllers
/ |
P |
PI |
PID1 |
PID2 |
Mean deviation |
0,846 |
0,853 |
0,860 |
0,505 |
Analysis of results of experiments showed that quality control is greatly improved when more complex control schemes.
Therapy of the autism using the intelligent cognitive system
It is believe that effective therapy for autism is extremely expensive. It is not because it is complicated, but because the small number of the specialists who own behavioral techniques. There is a situation when the majority of families do not have access to the necessary treatment [10].
A cognitive-intellectual system for diagnosing, adapting and educating autistic children was developed, which adapts to the concrete child. It can make autism therapy more accessible.
CISDAEAC is designed to extract, process and formulate a learning program based on cognitive processes, in particular EEG signals, adaptation of autistic children to society and training in basic household skills.
One of the tasks of this work is the processing of the EEG signal, based on the recognition of emotions, and forming an encephalographic portrait of the child on the next step.
Before working with the system, a detailed assessment of the current level of social interaction of the child, revealing the difference between the difficulties in acquiring.
Next, creating a minimum training package to determine the starting point, consisting of basic logical tasks.
To receive feedback, using the Emotiv EPOC + cognitive helmet, which allows recording the brain activity signal and transferring it to the data processing module. Next, by the EEG signal, evaluating the child's reaction to the interaction with environment, and monitoring parameters for solving the tasks.
Next, forming the strategy of learning, using the technology of soft computing. When forming a neural network of fuzzy output in the developed SCO. The signal from the EPOC signal recognition unit, the decision time, the correctness of the solution and the task identification number are using as the input data.
After this, the system sets the appropriate coefficients for adjusting the training program.
In connection with the specifics of processing large amounts of data in real time, for the processing of the EEG signal, a spectral-correlation method of signal analysis was chosen.
Figure 24 shows the spectral power graph of the EEG signal taken from the AF3 channel, obtained by the fast Fourier transform with a sampling frequency of 512 Hz. The graph is symmetric relative to the N/2 harmonics corresponding to half the sampling frequency (Nyquist frequency).

Figure 24. Spectral power graph of the EEG
Since the range of significant frequencies is from 1 to 50 Hz, the use of a fast Fourier transform with time-thinning (sampling each 5th value) optimizes the speed of signal processing. In accordance with the Nyquist -Shannon theorem at this sampling frequency, the upper frequency band transmitted without distortion.

Figure 25. Spectral power graph of fast Fourier transform with time thinning
Total values of the spectral power in the frequency ranges of delta, theta, alpha, gamma and beta rhythms serves as characteristic features of the signal. Thus, each channel of the EEG is associated with five features. Figure 25 shows the distribution of the values of the total power density of the EEG signal from the AF3 sensor in accordance with the frequency range.
The experiment
Emotional experience, as a type of cognitive activity, forms characteristic spatial-frequency EEG patterns. Emotions, regardless of their sign, can generate in both cerebral hemispheres, but a number of studies focuses attention on the involvement of the anterior brain in the generation of emotions, and also reveal frontal asymmetries for emotions of different signs.
In the course of the study, the EEG signal was taken from the sensors of the anterior sections of the brain symmetrically from the right and left hemispheres (see Figure 26)[9].


Figure 26. Position of the electrodes
Using the fast Fourier transform with time thinning, the spectral power density curves of the signals from the sensors AF3, AF4, F7, F3, F4, F8 were plotted. FC5 and FC6 in the range of significant frequencies from 1 to 50 Hz for the right and left hemispheres of the brain with positive and negative emotions (see Figure 27).

Figure 27. Spectral power graph of the fast Fourier transform of the anterior brain for emotions of different signs
Frontal asymmetries were traced for emotions of different signs. At a sufficiently high level of the initial tonic activity of the leading (speech, most often left) hemisphere of the GM, a positive background prevails, while at a low level, the negative hemisphere is negative.
Based on the estimation of the general tonic activity of the leading hemisphere, as well as the values of the total spectral power density of the allocated frequency ranges, the emotional background of the child is evaluated. Parameters that reflect the child's response to tasks will allow the system to adapt to the specifics of thinking, take into account personal characteristics and provide adaptability to various situations, taking into account psychophysiological conditions and deviations through soft calculations.
Development prospects
Currently, for the training of cognitive abilities of the operator are widely used simulators-games. One of the main problems of the application of the cognitive system control technologies "brain-computer", is the problem with the physiological features of the human condition-operator. Exhaustion, excitement, distracting noises, etc. affect the physical condition that naturally affects recognition quality teams and quality of control target device. At the moment do not possess specialized simulators software module for adaptation and learning control system (the program itself) to the characteristics of the operator. Experience with such weights indicates that virtually no mechanisms of adaptation and learning for practices imposed by the entry signal and its repetition which is interpreted as the generation of commands.
In turn, proved in a wide class of areas Soft Computing (genetic algorithms, fuzzy logic and fuzzy neural networks), can design intelligent cognitive control system with improve robust. On fig. 28 shown robust cognitive control system.

supervision over

brain activity

Figure 28. Robust cognitive control system
In such a structure, the control of the central element is a fuzzy system with integrated knowledge base to it, knowledge base contains type if-then rules. In the right parts of the Regulation contain relevant factors in the formulation of control action. In the left parts are indicators of recognized and unrecognized commands, for example, the values of the variances from sensors of the helmet.
For the design of this kind, you must use the appropriate knowledge base. Generalized technology design can be represented as a diagram presented in fig. 29.

Figure 29. The structure of the design technology of cognitive control systems
In such a structure control system with model is a central element, that rendering action that needs to be done. For the control of vehicle it can be a game where you must control the machine.
The first step is getting the teaching signal, for example, on the screen of the operator demonstrates the movement of cars on a certain track (line, circle). The signal of movement vehicle is recorded at the same time with EEG sensors. This signal will be base material for approximation of the fuzzy neural network and establishment of the left parts of the rules.
The right-hand sides of the rules matched in the second stage of the design process, when the operator needs to operate a vehicle. While his attempt to repeat previous (recorded) signal is not required since the right parts are not formed. They are formed in the process of the movement of cars on the highway, where traffic speed is determined on the basis of the PID controller.
The conclusions
The results showed that quality of control greatly improved with intelligent cognitive control scheme based on SCO. This work demonstrates the need to develop a unified information technology of design of control system for neurointerface.
It is worth noting that the further development of cognitive technologies control inseparable with brain training methods. Now, operators using specialized filters, smoothing signals of the EEG and remove interferences and noises caused by psycho-physiological condition and external factors, but for global better the use of intelligent cognitive technologies, the software level of control system should be have executive mechanisms for learning and adapting the control system.
Список литературы Cognitive fuzzy control system: examples of intelligent control and therapy of the autism applications
- Petrov B. N., Ulanov G. M., Ulyanov S.V., e.t., Information and semantic problems in the management of processes and organization. - М.: Nauka. - 1977.
- Sotnikov P. I. Review methods of EEG signal processing in the brain-computer interface // Inzhenernyy vestnik, Elektronnyy zhurnal. - 2014. - № 10.
- Kane M. J. Working Memory, Attention Control, and the N-Back Task: A Question of Construct Validity, University of North Carolina at Greensboro. - 2014.
- Anatomy of the Brain, Mayfield Certified Health [Info materials are written and developed by the Mayfield Clinic]. Mayfield Brain & Spine, 2016. Available at: http://www.mayfieldclinic.com/PE-AnatBrain.htm.
- Anguera J. A. et all. Video game training enhances cognitive control in older adults // Nature. - 2013. - Vol. 501. - Pp. 97-101.
- Barker A.T., Jalinous R, Freeston I.L. Non-invasive magnetic stimulation of human motor cortex // Lancet. - 1985. - Vol. 11. - Pp. 1106-1107.
- Brandt S.A., Stark L.W. Spontaneous eye movements during visual imagery reflect the content of the visual scene // J. Cogn. Neurosci. - 1997. - Vol. 9. - Pp. 27-38.
- Litvintseva L. V., Ulyanov S. V., Ulyanov S. S. Design of robust knowledge bases of fuzzy controllers for intelligent control of substantially nonlinear dynamic systems: II. A soft computing optimizer and robustness of intelligent control systems // J. of Computer and Systems Sciences Intern., 2006. - Vol. 45. - № 5. - Pp. 744-771.
- EDN: LJSYWP
- Sergey V. Ulyanov, Andrey G. Reshetnikov, Alla A. Mamaeva, Fuzzy Cognitive Control System of Autonomous Vehicle: Brain Neurointerface and Soft Computing Modes, 9th International Conference on Theory and Application of Soft Computing, Computing with Words and Perception, ICSCCW 2017, 24-25 August 2017, Budapest, Hungary. - Pp. 53-66.
- Ognjen (Oggi) Rudovic, Jaeryoung Lee, Miles Dai1, Björn Schuller and Rosalind W. Picard, Personalized Machine Learning for Robot Perception of Affect and Engagement in Autism Therapy arXiv:1802.01186v1 [cs.RO] 4 Feb 2018.