Methods of speech and text databases development for QA-systems

The paper is devoted to the problems of question-answer systems development (QA-systems). The subject of the study is discussion of approaches to the automatic filling of the database of the QA-system based on the analysis of the unstructured text sources currently available in the public domain of the Internet. The analysis reveals that the following ways of implementing QA-systems are distinguished: based on inference for ontologies, rules and syntax, using artificial neural networks. The methods for automatically search of question-answer pairs based on the structure of sentences and on the basis of associative-ontological analysis has been developed and tested in the research. The method based on the analysis of the structure of sentences is effective for texts such as lists of frequently asked questions (FAQ), as well as literature texts containing dialogs, direct speech, based on preliminary processing of the text, expressed in the form of a heuristic rule. The method based on associative-ontological analysis is focused to the class of reference and dictionary texts and is based on the assumption that in the descriptive text there is a sentence (or a group of sentences) containing the main idea of the text. In this case, the title of the text can be considered a question, and this sentence (or a group of sentences) is the answer. We need to make the selection of meaning-generating sentences due to the semantic reduction of the text automation. For this purpose, algorithms of self-referencing are applied based on the associative-ontological approach to the processing of texts in natural language. For the experimental verification of the possibility of creating an open QA-system based on the automatic collection of question-answer pairs from the Internet, a prototype of a collection module for the database of the QA-system has been developed.