Application of AI chatbots in doctoral students’ English language training

М.Г. Кузьмин1, ,

К.М. Коротков2, ,

Nowadays neural networks are penetrating into many spheres of our life, AI tools are starting to be used even in the educational university process to create personalized learning with the help of assessment methods coordination and teacher-student interactions. AI technologies help to adapt teaching strategies and educational content to the students’ individual needs. According to S. Sharma, a personalized learning improves students’ motivation and promotes an effective learning process [18]. The “National AI Deve- lopment Strategy until 2030” aims at the active AI tools use in the scientific-technological spheres [24, 26]. During the process of integrating AI tools into various processes of functioning of public agencies, we need to concentrate on the information society development. It is reflected in the RF President’s Decree dated October 1, 2017 [25].

However, for teaching a foreign language to doctoral students AI methods have not been developed [1, 5]. Thus a complex of AI chatbots should necessarily be developed.

In reality not all educators and doctoral students are aware of the importance of AI tools application in educational university process. Among the reasons: little knowledge, devoting much time and endeavor to implement AI tools, also no desire to use them in the educational process for doctoral students [6, 11].

The research aim is to analyze the complex of AI chatbots usage at English language classes for doctoral students of South Ural State University (SUSU). The tasks are as following: to analyze the concept of “AI chatbots”; to find out doctoral students’ awareness of AI tools in the educational university process (we applied a questionnaire and interview among 30 doctoral students) to satisfy the doctoral students’ needs; to analyze the regularity of AI tools usage a questionnaire, then an interview (26 educators) are applied; to analyze the educators’ obstacles in AI tools application the educators’ needs (lecturers of English) were analyzed; a complex of AI chatbots was implemented in the process of teaching English to doctoral students (AI chatbots: ChatGPT, GigaChat and Kandinsky).

Literature Review

Nowadays the main priorities in the Russian educational system development are becoming intelligent technologies with the AI use not excluding traditional educational technologies. Researchers analyze the problem relevance and the AI development perspectives [13], and study the practice of introducing AI into the higher education system [17, 21]. In the scientific literature you can find a huge number of definitions of AI. However, a single universal way to understand the essence of artificial intelligence has not yet emerged. Among the reasons the lack of agreement in the understanding of the concept of intelligence, the unsuccessful translation of the term “artificial intelligence”, the new technologies appearance that change the content, also different spheres of AI application can be seen [7, 9, 10, 20].

In the study we rely on the following definition of AI. The National Standard of RF (the Russian Federation) “AI Technologies in Education” proves that AI represents some technological results imitating human cognitive functions such as self-learning without a ready-made algorithm, and the results can be compared to the ones of human intellectual actions [8].

Accordingly, AI tool is considered to be a general term including AI chatbots, neural network sites and programmes [12].

Different researchers such as I.I. Sagdullina, V.I. Toktarova, etc. [22] study personalized learning, evaluation of students’ work, etc., considering these directions to be the key areas for using AI tools in the higher education.

The AI technologies application has proved to be much wider [17, 21, 22]. ChatGPT is the most popular AI tool for writing and editing texts. Educators believe that the use of AI tools increases the risk of students not completing work independently [2–4]. But most academics are willing to inform undergraduates and graduate students about the limitations and capabilities of AI and thus prepare them for the AI tools use in the new world [6, 7, 9, 14].

When considering AI tools, it is necessary to expand on the concept of “neural network”. R. Callan believes that a neural network is a collection of elements connected to each other to ensure interaction and form a self-learning system. Neuron elements are capable of calculating the output signal from a set of input signals [5]. I.A. Filipova considers an artificial neural network as a mathematical model operating on the nervous system, capable of performing specific tasks, including those of a creative nature [7]. Neural networks are adaptive to various types of data and tasks due to their structure; they are capable of processing complex nonlinear information.

In our study, a neural network is understood as a modern AI technology that is capable of creating suitable content for educational purposes based on a given request.

There are many works by foreign and Russian researchers on the topic of neural network technologies in education. They are devoted to the AI role in education, troubles and perspectives for their use, their advantages in adaptive learning, data-based decision making, etc. [1, 9, 12, 15]. Some researchers present AI technology functions in education: performing assessment tasks, teaching, administration, etc. [14, 23]. Russian researchers pay close attention to the implementation of neural networks in the educational process, describing possible areas of their use and multitasking [2, 16]. There are interdisciplinary, specialized and auxiliary neural networks [6]. Foreign researchers emphasize the advantages of using neural networks in education: increasing the efficiency of individual learning, creating a game-based learning experience and focusing on personalizing learning [3].

In general, neural networks for creating edu- cational content [6] (texts, images and videos, etc.) are service sites, also chatbots that are represented as computer programs based on AI. We highlight AI chatbots in the work as working with doctoral students AI chatbots are the most convenient for an active form of learning with the help of AI.

In our understanding AI chatbot is a computer program that interactively simulates human speech (spoken or written) and allows you to communicate with digital devices as if they were real people. In our research we need a complex of AI chatbots as a complex consists of many different and connected parts and contributes to better learning than just one tool. Thus we apply AI chatbots in our study: ChatGPT, GigaChat and Kandinsky for doctoral students’ language learning.

Materials and Methods

In order to achieve the main goal of the study and solve the above set tasks we used the method of literature analysis of state-of-the-art methodological questions and most debated problems in teaching English on doctoral level. To check the most recent state of students’ and educators’ knowledge about applying AI tools, a number of empirical methods were applied, such as questionnaires, interviews. Cronbach’s alpha coefficient was calculated to identify the consistency of the sample in the research.

The theoretical method enabled us to analyze the concept of “AI chatbots”. Questionnaires and interviews made it possible to reveal doctoral students’ awareness of AI tools in the educational process of the higher school (among 30 doctoral students). The gaps found in the study will shed light upon doctoral students’ needs in their efficient use of the tools to further make the educational process more enjoyable and time-saving, to improve the doctoral students’ interest to the English subject.

Participants. Two groups of 56 participants took part in the research – 26 ESL lecturers and 30 first- and second-year doctoral students of South Ural State University from humanitarian and engineering fields of science. The group of ESL lecturers is characterized by the following: 60% senior lecturers and 40% Associate Professors, 96% of them have teaching experience of more than 10 years. The participants from the doctoral students group are aged between 22 and 38, but mostly (53% of the respondents) are 25– 26 years old, 80% male and 20% female doctoral students.

Instruments. The questionnaire was conducted online using Yandex Forms for doctoral students about using AI tools in learning English and ESL lecturers about using AI tools in teaching English. Both lecturers and doctoral students were also interviewed personally to confirm the correctness of their understanding the questions. The questionnaire for doctoral students contains 2 parts and 10 questions: the first part is focused on personal background of the respondents and the second part reveals the doctoral students’ knowledge about artificial intelligence tools, frequency and efficiency - how they are used for learning English, what information and support is needed about the possibilities of using AI tools and about personal needs in learning English that can be met by the AI tools.

ESL lecturers were also questioned by means of an online Yandex Forms tool to find out if the lecturers know artificial intelligence tools, what chatbots they use in class, what their attitude is and what difficulties lecturers experience in working with AI tools. All the questions were closed to further process the results and calculate the reliability coefficient – the value of Cron-bach's alpha.

Research procedure. To analyze the use of a complex of AI chatbots in the educational process we used the example of English language classes for doctoral students of South Ural State University.

Firstly, two questionnaires were created both for ESL lecturers and doctoral students and then sent to the respondents via email as links in Yandex Forms. Secondly, the answers were transferred to the Excel tables for processing and identifying the trends in them. Finally, Cronbach’s alpha was calculated using Python programming language and Pandas (Python data analysis library).

Results and discussion

Before presenting the main results of the research which was focused on studying the responses of two groups of participants, we tested the answers for the main five questions of each questionnaire from the point of the consistency of the sample. The data in Table 1 below showing the reliability coefficient of 26 responses from educators for 5 questions was found to be 0,89. Table 2 demonstrates the value for the doctoral students’ questionnaire containing also five questions about AI tools – 0,99.

The results of the questionnaire of doctoral students show that the most well-known AI tool among them is ChatGPT (50,94%), while all the other AI chatbots is below 20% of familiarity each (Fig. 1).

The same ChatGPT takes a leading place as an AI tool used for learning English by doctoral students (39,53%). On the second place among the suggested options is “None” (23,26%) which means that a considerable amount (second place in the row of options for the question) of the approached respondents do not use AI tools for learning English (Fig. 2).

Table 1 Consistency of the sample for educators’ questionnaire

Chronbach’s Alpha	№ of questions	№ of respondents
0,89	5	26

Table 2

Consistency of the sample for doctoral students’ questionnaire

Chronbach’s Alpha	№ of questions	№ of respondents
0,99	5	30

Further on, we set two more questions: about thei r awareness of how AI t ools help i n learning English and the tasks that they solve in learning English by AI tools. The first place in a wareness and using AI tools takes “translation”. The second place is equally di v ided by “generating text s ” and “correcting writt e n mistakes ” (Fig. 3). Oth e r options chosen by do c toral stude n ts cannot be c onsidered to be valid as more point s are collect e d for practical use of AI tools than for awareness about their application (Fig. 4).

The second group of the respondents is presented by ESL lecturers who were firstly asked about their attitude to AI tools – positive, negative or neutral. It appeared that neither of them are negative about AI tools, and can be divided into two equal groups with positive and neutral attitude. More than half o f them with positive attitude are associate profe s sors. The n e xt question “Do you use AI tools in teaching?” was also ans w ered positively by 60 % of all the respondents, 60% among them are also associate profess o rs. On the other hand, senior lecturers are

Fig. 1. Doctoral students’ awareness about AI tools

Fig. 2. AI tools applied by doctoral students for learning English

among the majority of those who do not use AI tools and have a neutral rather than positive attitude to such tools. In the next two questions a list of different names of AI tools was presented and the respondents had to choose those they know and those they use in their work. The most well- known AI tools – chatbots are ChatGPT (27,7%), Kandinsky (21,3%) and GigaChat (16%) as most of the respondents among ESL chose them. All the other AI tools take less than 12% out of the eight tools suggested in the questionnaire and are the least-known (Fig. 5).

Fig. 3. Doctoral students’ awareness about how AI tools help in learning English

Fig. 4. Doctoral students’ purposes in using AI tools for learning English

However, there is only one tool that takes a leading place in using which is ChatGPT, it takes nearly the third part (28,07%) of all of the seven suggested AI tools for ESL lecturers in a questionnaire in terms of applying them in teaching English. All the other six tools (Talk AI, Deeple Translator, Kandinsky, Twee, Fliki, GigaChat) take less than 20% in the whole range of the AI tools used by educators. The least used tools are Fliki and GigaChat (Fig. 6).

The ESL lecturers were also asked about the difficulties that they experience in using AI tools. It turned out that there are two main of them suggested in the questionnaire – “Have a limited number of credits on free account” (32,5%) and “Make mistakes” (32,5 %). All the other types of difficulties are less experienced by ESL lecturers (Fig. 7).

Based on the research results, it becomes clear that both ESL lectur e rs and doctoral students are acquainted mostl y with one AI tool – ChatGPT and use it in a limited variation for teaching and learning English. But there a re many other AI tools (chatbots) for different purposes that can make the process of teaching and learning English enjoyable and the result tangible, suc h as GigaChat and Kandi n sky.

The received results suggest that it is necessary to increase ESL lect u rers’ awareness and skills in applying AI tools as well as to t rain doctoral students’ skills in applying the tools. So, we sug g est a complex of AI tools that are AI chatbots: ChatGPT, GigaChat and Kandin s ky to be implemented in the process of teaching English to doctoral students, as they are the m ost comfortable and motivating one s .

Fig. 7. Difficulties that ESL lecturers experience in using AI tools

Table 3

Tasks and prompts for the chatbot

Unit №	Topic	Task	Chatbot prompts for Editing Writing
1	Introduction to editing	Read samples A and B and decide which is easier to read, then compare your idea with the ideas generated by two AI chatbots (Talk AI and Giga Chat). Act like an editor! Study the Introduction of the research paper and say what recommendations you can make to improve it (clarity and style; grammar issues; punctuation errors; awkward phrasing; inconsistent language)	Review the samples A and B for clarity and style; grammar issues; punctuation errors; awkward phrasing; inconsistent language. Suggest improvements to enhance overall readability
2	Structural editing	Trim the text using the eight strategies (mass-delete whatever doesn’t belong; delete long lead-ins; pare down unnecessarily wordy phrases; delete redundant words; delete filler expressions and words; delete needless adverbs; favour short, plain words and use jargon selectively; simplify verbs), then compare your idea with the ideas generated by two AI chatbots (Talk AI and Giga Chat)	Evaluate the text for long leadins, wordy phrases, redundant words, filler expressions, needless adverbs. Recommend changes to enhance it
3	Paragraph level revision	Read two paragraphs from Introduction to the paper. Identify where the paragraphs need to be revised to improve their effectiveness, then compare your idea with the ideas generated by two AI chatbots (Talk AI and Giga Chat)	Examine the two paragraphs for their effective structure. Provide recommendations to optimize them

According to the research results, doctoral students use AI tools mostly for translating texts, whereas, as the interview showed, they do not use them for generating academic articles. It is quite clear because AI tools are not able to scan and implement researcher’s background, specific vocabulary, approaches, concepts and references. Though, they are helpful in teaching English editing practice which is proved by a set of exercises implemented in a special course on research paper editing during the fourth semester of doctoral studies.

The most well-known AI tool ChatGPT is well-known for ESL lecturers and doctoral students. Along with it, we implemented ChatGPT analogues that work on Russian platforms, such as Giga Chat. The chatbot is suggested to be used for training doctoral students’ competence in editing own research papers and research papers of their peers.

ChatGPT and GigaChat chatbots. ChatGPT is a chatbot with artificial intelligence from the company OpenAI, co-founded by Elon Musk ChatGPT, developed by OpenAI, has gained broad recognition as a powerful AI chatbot. This technology’s prowess stems from its extensive training on a substantial corpus of text data, enabling it to converse on a wide spectrum of subjects. However, while ChatGPT excels in text generation, it has one significant limitation – it lacks the ability to produce images. This short- coming confines the chatbot to text-based interactions and responses, which might limit its utility in contexts requiring multimodal communication.

GigaChat, on the other hand, is a worthy competitor to ChatGPT. The purpose of GigaChat is to address the drawbacks of its rival. This tool is trained in Russian and is equipped with multimodal features. In Table 3 the examples of tasks and prompts for the chatbot are presented in the context of training editing skills.

Kandinsky . Kandinsky 2.0 is another AI tool designed to transform texts into images and allow users to inpaint existing images. It can perform text-to-image generation and is the perfect tool for those looking to create unique visual content. The tool can be used for generating images for texts and tasks by ESL lecturers to increase doctoral students’ attention to theoretical material and train their vocabulary when writing prompts.

Doctoral students are suggested to generate pictures for the edited texts in different genres. The main thing here is to write the name of the object and describe it in details. The examples for some vocabulary and topics on editing are given in Table 4.

In teaching practice, we share the approach of using AI chatbots in combination with traditional ones [20]. It means that the first stage is always about students’ studying the basic material presented by educators or found by students themselves.

Table 4

Vocabulary and topics on editing

Unit №	Topic	Vocabulary	Kandinsky Prompts for generating pictures
1	Introduction to editing	Developmental editor light rewriting	Developmental editor sitting at the table, making light rewriting, highly concentrated and busy with work
2	Structural editing	Draft gaps in information spelling errors	Draft of a research paper with gaps in information and spelling errors on the table of an editor
3	Paragraph level revision	Paragraph’s focus close-reading unrelated ideas	Editor aimed at close reading of a research paper paragraph which has unrelated ideas in it

Conclusion

AI tool is a general term including AI chatbots, neural network sites and programmes [12]. AI chatbots are neural networks represented as computer programs based on AI. In our research we highlight AI chatbots as working with doctoral students they are the most convenient for an active form of learning with the help of AI.

The research results revealed doctoral students’ and educators' knowledge and skills about AI tools concerning learning a foreign language. In the study we see that the majority of doctoral students know about some AI chatbots, but never used them in editing texts. It means that educators do not use AI chatbots extensively because of the fact that their applying is tedious and they have a lack of knowledge how to make prompts. To master and adapt AI activities to the doctoral educa- tional process for some lecturers is considered to be demanding acquiring much time and endeavor.

However, doctoral students and educators have a positive relation to master and implement a complex of AI chatbots. For doctoral students it is natural to learn and communicate using chatbots, the Internet, mobile phones [4, 19], because they matured enveloped by digital gadgets. AI chatbots provide services to doctoral students’ all requirements. SUSU also upgrades traditional teaching approaches and modernizes the academic environment. The complex of AI chatbots presents an attractive learning environment with effective training and also active interaction in a new AI academic setting [10, 11]. Moreover, this complex of AI chatbots is oriented on doctoral students in the purposeful environment and is important in modern learning conditions.