Early Formalization of AI-tools Usage in Software Engineering in Europe: Study of 2023

Since its inception in the XX century, the IT domain has been subject to significant changes driven by the influence of the external environment. Technological pressures have compelled IT companies to adapt their organizational and production paradigms to meet the escalating expectations of users, employers, partners, and clients. The effective management of these changes has emerged as a formidable competitive advantage, optimizing the speed and efficacy of tactical and strategic IT company operations [1].

In the years 2022-2023, AI-augmented Software Engineering and the widespread adoption of fully remote working are shaping a new reality for software development companies. The associated competitive advantages of fully remote working have prompted a shift in organizational paradigms, transitioning from the traditional "5 days in office" approach to more flexible "hybrid" or fully remote modes. Recent studies like [2] have demonstrated that Fully Remote Mode (FRM) is now established as a robust practice, adopted by leading European software companies, and is no longer solely driven by pandemic-related concerns.

The Pan-European study has been conducted to solve mentioned above problem and to define the implications and potential consequences of incorporating AI-tools in SD projects in IT companies. The goal, method and details of the study are described in next section.

2.    The Goal, Method and Details of the Study

The Pan-European study has been conducted in June-July of 2023 and covers the experience of 35 teams in IT companies, system integrators and banks with strong in-house software development: from Russia and Kazakhstan to UK and Spain. The teams are representing different kinds of software developments business:

• •    Independent software vendors, including in-house product development like Miro, Google, Finastra, Finshape, Sber, VTB, Playrix, OZON;

• •    Custom development and outsourcing software services like Atos IT solutions, SOFTEC, First Line Software and Epam;

• •    System Integrators like Thoughtworks and Auxo;

• •    Other IT institutions like Capgemini Engineering, ZEISS Digital and Ericsson.

The goal of this research is to gather the opinions and experience of the senior software engineers and managers into solid vision of AI-tools usage in 2023, that might help in the future management of the practical implementation of AI in IT business and the production processes. The study is devoted to the issue of early formalization of the use of AI-tools in the software engineering. The hypothesis of the author in the study: “Wide media coverage of LLMs since the second half of 2022 has seriously popularized the use of AI in software engineering. Currently, more and more European engineers are using various AI-tools in solving their daily tasks in the software projects. IT organizations have already begun a centralized transformation of the production processes associated with the formalization of the use of AI-tools”. The study has been conducted via Google.Forms instrument with questionnaire form and video interviews. All structured results have been sent in July, 2023 to experts to give them an opportunity to comment it before study results are finalized.

The expert’s panel might be described in following Table 1. There are three main characteristics – level of professional experience (in years), region in Europe and type of IT business. Such kind of characteristics are expressing the variability of software development experience in expert’s panel.

Table 1. Experts in the research

№	The characteristic	Representation in the expert’s study
1	Level of professional experience in software engineering	Less than 5 years	5-10 years	More than 10 years
		2 %	9%	89%
2	Region of described the experience in usage of AI-tools	North and Western Europe (UK, Spain, Sweden, France, Germany, Swiss, etc)	Central and South Europe (Poland, Czechia, Hungary, Serbia Bulgaria, Cyprus, etc)	Eastern Europe and CIS (Ukraine, Russia, Armenia, Turkey Kazakhstan, Georgia, etc)
		29%	20%	51%
3	Types of IT-business	Independent software vendors, including inhouse product development	Custom development and outsourcing software services	System Integrators	Other type of IT business
		46%	29%	11%	14%

As it’s clear from the Table 1 the Pan-European study covers the main geographical regions and let the senior engineers and managers take part in construction of the new vision of AI-tools using in software development. Main results of the study are described in following section of the article.

The study's results were acquired through a systematic two-phase process. In the initial phase, the raw opinions from every expert have been obtained via tool Google.Forms and automatically consolidated in a working document. The initial results have been proceeding in the official report for the participants and sent them via electronic channels. This laid the groundwork for subsequent interactions with experts in the second phase. During the second phase, targeted discussions were held with domain experts. These discussions aimed to delve deeper into the collected data and refine the final outcomes. To ensure data accuracy, Google Forms was employed for data processing. Approximately 54% of experts engaged in follow-up discussions, which allowed them to contribute to the final results. The conclusive results of the study are presented in the following section, drawing from the collaboration between empirical insights and expert perspectives. This marks the culmination of the research process, emphasizing rigorous methodology and academic integrity.

3.    Main Results of the Study

This study is structured into three integral sections, each contributing valuable insights into the utilization of AI-tools in software production companies. The first section is devoted to the examination of the prevailing practices of AI-tools current usage within these companies. These tools, often made through large language models (LLMs), rely on inbuilt artificial neural networks. These LLMs undergo training using self-supervised and semi-supervised learning techniques and typically comprise tens of millions to billions of weights [4]. Their training process is facilitated through specialized AI accelerator hardware, enabling parallel processing and utilizing vast amounts of software code from various sources, the majority of which are accessible openly. As of 2023, the software engineering domain comprises an extensive array of approximately 50 distinctive tools based on LLMs, with notable mentions being ChatGPT, Copilot, and cnvrg.io.

The second section of this study delves into the common challenges that arise during the implementation of LLMs in real-world software development (SD) practices, including the formalization of their usage at the official corporate level. The third and final section is dedicated to presenting the forecasted trajectory of AI-tools in the software engineering domain. This section offers a comprehensive forecast regarding the technological future of AI-tool development in software engineering. This forecast encompasses aspects such as the integration of AI elements into diverse information systems within SD projects and the potential long-term impact on the skill sets of software developers in the near future. Furthermore, this section features invaluable insights from experts, encapsulating their private opinions concerning various dimensions of AI implementation in software engineering, including aspects related to copyright, legal matters, and regulatory concerns.

With this clarification, the study's structure is now fully represented, encompassing all three key sections, each contributing substantively to the understanding of AI-tool deployment and its potential impact in the software development landscape. The collective benefits derived from the use of AI tools have led to an enhanced crossfunctionality among engineers, empowering them to tackle entire classes of straightforward tasks autonomously. In this context, the advent of AI in software engineering is poised to impact various levels of the IT industry's organizational and production paradigms, namely:

• •    Corporate level: With the introduction of AI, core business processes are poised to undergo transformation.

• •    Project level: Roles and areas of responsibility within projects are subject to change.

• •    Personal level: Software developers are compelled to update their skill sets to effectively collaborate with AI-based tools.

All those levels would be analyzed in current research, but firstly should be learned the current processes of LLMs usage in software production processes. The study shows that significant share (20%) of teams and organizations already has started the implementation of AI-tools in real software production and it’s planned to do in near future for around 43% of teams. Personally around 23% of experts are using AI tools in their regular job with the high frequency and it has a strong impact on their personal tasks in SD projects. And 20% of experts estimated the impact of the AI-tools on their professional work as average and valuable for some particular tasks in the regular work.

Expert panel estimated the main advantages of AI \ LLMs usage in software development in the real practice:

• •    Automation of routine operations and time saving –63% of experts;

• •    Speed up of operations in the team \ organization – 43% of experts;

• •    Software product excellence, incl. software quality, UX and documentation – 31% of experts;

By the middle of 2023 the AI-tools (like Watson or ChatGPT) are not in high demand in SD projects for business and system analysis. More than 70% of experts aren’t using it at all in this type of tasks and experts confirmed that AI-tools are in demand for only tasks of business modeling and data analysis (14%). Much more successful the practice of

AI-tools usage in software design and construction. There are most popular types of tasks for LLMs:

• •    Coding (incl. unit-tests, stored procedures, etc) – 31% of experts;

• •    Code reviews (incl. code optimization and refactoring) – 23% of experts;

• •    Fast software prototyping – 14% of experts;

According to this research the work with code is the most popular propose for LLMs usage. In area of software quality assurance, the demand is lower and also aimed on the working with the code. There are most popular types of tasks for LLMs in software quality management:

• •    searching of the errors and vulnerabilities in the code – 11% of experts;

• •    writing the auto-tests – around 9% of experts;

• •    non-functional testing (integration, load, stress, etc) - around 9% of experts.

Around 77% of experts don’t use AI-tools in software quality management.

Also LLMs are useful in some other SD project activities:

• •    Software documentation (the manuals, video instructions, etc) – around 29% of experts;

• •    Software product support (the issue tickets, help for users, etc) – 11% of experts.

For around 9% of teams there is a demand of AI-tools in the project management.

Study (in its second section) confirmed that the implementation of LLMs is still a potential competition opportunity for IT companies: not a lot of them in Europe had started the centralized LLMs implementation in production processes. Only 15% experts noticed, that their companies are executing the corporate plan to implement LLMs in software development. And around 20% of teams \ IT organizations are in the process of the discussion about its centralized implementation and let the process go on the level of the single teams \ individuals.

For sure, the implementation of LLMs has their specific features and risks. Expert’s panel figured the main barriers in the implementation of any AI-instruments in the software development in their teams and organizations in 2023:

• •    Lack of the resources (money, time, knowledge, HR-capital) – 43% of experts;

• •    Organizational resistance of engineers and managers – 34% of experts;

Around 37% of involved in the study SD teams are in the active process of AI-tools Implementation in the software engineering practices. In 12% of teams there is an official plan or a corporate policy with the official recommendations how to use AI-instruments in SD projects, and in 31% of teams its usage is continuing now in test mode. Together with that around 20% of teams has no need to implement it in 2023. Moreover, in 9% of teams there is a strong negative attitude to AI-instruments in the software development (including its official prohibition).

Expert’s panel estimated (from the personal point of view) the advisability of the centralized corporate investments in the improvement of AI \ LLMs usage in 2023:

• •    Might be useful, but it needs more preparation (R&D project, economical model, etc) – 43% of experts;

• •    It's the best time to do it (ahead of the competition) – 40% of experts;

• •    Too early, better to start from individual and team’s usage before centralized investments – 17% of experts.

Trend of the usage of the LLMs in software development is actual. The study confirmed the initial author’s hypothesis: more and more European engineers are using various AI-tools in solving their tasks in software projects, and innovative IT-companies have already begun a centralized transformation of production processes associated with the formalization of the use of AI tools. Current AI-instruments usage is focused on the working with the software code (in different kind of ways) and with SD project\product documentation. There is a big potential in LLMs usage in the software engineering and the lack of the centralized efforts on the corporate level might lead to the missing of the competition advantage in the software production.

Implementation of the LLMs in the real corporate software production practices is going in normal way according to Everett Rogers categorization from well-known theory of “Diffusion of Innovations” [5]:

• •    there are formed categories with the “innovators” and “early adopters” and it’s clear: the formalization of AI-tools usage is started (the teams are executing the corporate plan to implement LLMs in software development and started its centralized management via corporate policy or recommendations);

• •    there is process of formation of “an early majority” category – those teams are interested in following learning of AI-tools via different ways (R&D projects, individual\team experiment, etc).

Last section of the research is dedicated to the forecast about the near tech future. Despite of accepting the fact of that AI impact is a significant at the moment [6], current study’s topic is in the early stage of formalization and all assumptions are preliminary. There are the several time horizons for the development of AI-tools and its implementation in IT domain:

• •    short term – 3-7 years;

• •    middle-term – next 10 years.

Expert’s panel estimated the potential power of AI (incl. LLMs) in acceptance testing, validation and release software production. In next short term (3-7 years) AI-tools would:

• •    be an ordinary tool for human, who accept and validate new releases of software – 48% of experts;

• •    play the central role, unified the human efforts and QA tools (tests, scripts) – 26% of experts.

Experts estimated the potential power of AI (incl. LLMs) in software design and coding in short term:

• •    be an ordinary tool for human, who do coding or design – 51% of experts;

• •    expert, who would help, teach and check the humans in limited number of cases – 21% of experts;

Senior IT engineers and managers totally denied the idea of human replacing by AI-tools in all areas of software engineering at mid-term perspective or even never.

In middle term the transformation of software development paradigms (incl. tools and processes) by AI would change the everyday needed skills of developers. About 46% of experts predicted the strong need of special knowledge in IT-profession about managing with AI and about 31% of experts are sure in the continuing transformation of the whole IT software engineer job in the SD projects.

Also, expert panel estimated the value and the role of LLMs in learning and in the excellence of the software development skills in 2023:

• •    The impact of usage LLMs in professional learning is very high – 34% of experts;

• •    It’s just one more useful tool on the board – 54% of experts.

Expert’s panel confirmed that in next 10 years the usage of AI-tools would significantly change the production processes in software development industry: around 63% of experts noticed that process of the global shift in the industry is started. Also around 31% of experts are still waiting for positive moves in this direction for the most innovative IT companies. This part of teams \ IT organizations is evaluating the impact of LLMs on software engineering as modest, but they are expecting that leaders of the IT-market would use AI-tools and develop it. Seems like, it’s the first sign of appearing of “late majority” according to Everett Rogers categorization from well-known theory of “Diffusion of Innovations”.

Experts confirmed that speed of AI-tools development is very high and they’re expected in next 3-7 years:

• •    It’s appearing in all advanced IDE (code redactor, release building, documentation, etc) – 94% of experts;

• •    It’s appearing in the software project management systems (like Trello, MS Project, Jira, etc) – 73% of experts;

• •    It’s appearing in product management tools (UX\UI, feature analysis, user tests, etc) – 60% of experts;

• •    It’s appearing in DevOps tools (from CI\CD to user support software) – 57% of experts.

By the way, Apple integrated the set of AI-tools in their X-code IDE (iOS software development) in the July of 2023 [7]. And other market players are on the same way at the moment [8, 9].

In mid-term experts define the expectation of the fast state or industry regulation of the AI-usage in IT domain in different parts of Europe. There are some interesting assumptions and comments about the threads and risks in the processes of the AI-tools usage and its implementation in SD projects:

• •    Information security issues in AI usage are critical. Current speed of AI implementation is sacrificing some needs of that kind;

• •    In Western Europe legal perspectives (absence of the regulations and legal background) of AI-tools usage might be the main barrier in its implementation. Some basic ideas might be found in [10];

• •    Setting of AI-tools for the complex projects is a big activity and has a lot of risks. Data sources would be more and more specific for different AI-tools in different SD projects;

• •    AI will lead to a cyclopean growth in the mutual integration of various systems in complex IT landscapes on next level of scale, including old and domestic systems in different tech stacks.

4.    Conclusions

Experts also shared the doubts about ethical aspect of AI-tools usage: data source for global LLMs learning couldn’t be forever in “the common ownership”. There are a lot of issues with author’s rights, copyrights, patents and corporate “now-how” competition advantages that can’t be shared world-wide in old way before expansion of AI-tools [11]. Experts are expecting the moves in legal formalization of AI-tools usage in IT-domain.

The author envisages a forthcoming discourse to deliberate upon the principal findings elicited by this study. Additionally, the author is poised to embark upon a subsequent phase of investigation that centers around the formal integration of AI tools within the milieu of software engineering, slated for the timeline encompassing the years 2024 to 2026. It is conjectured that the maturation of corporate AI tools targeting software engineering between 2023 and 2024 will act as an accelerant for the formalization process. This hypothesis is underpinned by the ongoing endeavors of notable entities, including Google, Microsoft, AIOpen, IBM, and approximately 50 other stakeholders within the marketplace. The palpable strides and concentrated endeavors undertaken by these influential industry participants lend credence to the sustainability of the aforementioned supposition.

This comprehensive study offers valuable insights into the current state and future trajectory of AI-tools, specifically large language models (LLMs), in software development processes within European IT companies. The research highlights the growing significance of AI-tools and their potential to revolutionize the software engineering landscape. The findings demonstrate that AI-tools, particularly LLMs, have already become instrumental in various aspects of software development, streamlining processes and enhancing overall productivity.

One of the key observations is the prominent role of AI-tools in software design and coding activities. The ability of LLMs to generate code, perform code reviews, and assist in fast software prototyping showcases their potential to accelerate the software development life cycle. However, the study also reveals the limited demand for AI-tools in certain areas, such as business and system analysis, which calls for further exploration and targeted development.

Furthermore, the study emphasizes the transformative impact of AI-tools on software engineers' skill sets. As AI-tools gain prominence, developers are expected to adapt and acquire new competencies to collaborate effectively with these technologies. The need for expertise in managing AI-tools and their integration into existing development practices emerges as a pivotal factor for successful software developers.

While the benefits of AI-tools in software development are evident [12], the research also sheds light on the challenges faced by IT companies during their integration. Resource constraints, including financial, temporal, and human, pose a significant hurdle to widespread implementation. Organizational resistance to change and the need for cultural transformation further complicate the smooth adoption of AI-tools. Addressing these challenges will require strategic planning, strong leadership, and clear communication to ensure successful implementation.

Additionally, the study emphasizes the ethical considerations surrounding AI-tool usage. The ownership of data used for training LLMs raises questions about privacy and intellectual property rights. As AI-tools become more pervasive, it is essential to establish appropriate legal frameworks and ethical guidelines to protect both individuals and companies from potential misuse of data and ensure responsible AI-tool usage. Some basic ideas about legal frameworks [13] and ethical guidelines [14] are demonstrating the strong need of designing new patterns in the domain’s regulation: from labor aspects and guaranties to paying the commissions to authors of the data source for training the AI. This issue is also in the very early stage of formalization and might be started only by the competent and powerful governmental bodies.

Like any other production innovations in the organization of the labor, this trend cannot be universal and lightning fast, however, according to the study, it is he who will determine the competitiveness of IT companies and transform the usual role of an engineer in a software project over the next ten years. Diffusion of the innovations is a well-learned topic of researches in XX c and in most cases all high-tech innovations are diffusing in domain in accordance with Everett Rodgers theory about the five categories of innovation’s adopters: innovators, early adopters, early majority, late majority, and laggards. An interesting observation is that the introduction of AI tools into the actual practice of software production follows the classification of Everett Rogers from the well-known theory of “diffusion of innovations”:

• •    categories with "innovators" and "early adopters" according to E. Rogers were formed and the formalization of the use of AI tools began (project teams carry out a corporate plan for introducing AI into software development, create and use centralized corporate policies and/or recommendations);

• •    the process of forming the category of "early majority" according to E. Rogers is underway - teams interested in further studying AI tools in various ways (R&D, individual / team experiment, etc.).

If the trend of AI-tools usage in business and production processes in high tech companies would have the character of "S curves" in the time by the analogy with other technology innovations like Internet [15] than we may expect the strong demand on AI tools and shift in organization of software production in mid-term perspectives. It’s planned to repeat the similar study in European region within 2026. Meanwhile, the study of 2023 confirmed that AI tools have already become part of the usual processes in software projects: from working with documentation to writing and improving program code. Further growth in the automation of software development processes is associated with the development of such tools and their integration into the production systems of software projects (from IDE and CI \ CD to project and product management).

In light of the research findings, it is evident that the software engineering community is at a crucial juncture. The successful integration of AI-tools into software development processes will require collaboration, knowledge sharing, and innovative approaches. Companies that embrace AI-tools as a means to enhance productivity and deliver high-quality software products are likely to gain a competitive edge in the fast-paced IT landscape.

This research significantly contributes to the field by shedding light on the current state of AI-tool adoption in software production, the challenges that come with it, and the potential impact on the industry's future. The findings provide a roadmap for both practitioners and policymakers to navigate the integration of AI-tools effectively and ethically. As the industry moves forward, the scientific justification provided by this study becomes a cornerstone for advancing AI utilization within the software engineering landscape.

The study also points to the need for continuous research and development to harness the full potential of AI-tools in software engineering. Addressing the current limitations and exploring new application domains will be vital for unleashing the true transformative power of AI-tools in the future. The findings pave the way for a future where AI-tools become an integral part of the software engineering process, driving innovation, efficiency, and excellence in software development.