The application of artificial intelligence tools in the legal profession

Artificial intelligence (hereinafter: AI) is a key component of the Fifth Industrial Revolution, which should facilitate the performance of numerous jobs. However, various dilemmas and controversies are also associated with AI, especially regarding the concern that many jobs will disappear. As the legal profession is based on knowledge, the question of its future is often considered in literature.

Among modern AI tools, ChatGPT stands out in terms of popularity. Namely, ChatGPT is a chatbot tool of generative AI, a language simulator composed of a series of algorithms that enables conversation with end users. Developed by OpenAI and released in 2020, it contains a huge number of text prompts, which enables it to provide human-like text. ChatGPT is used for various purposes, such as translation, summarizing, and answering questions (Perlman, 2022).

The first part of this paper offers a general overview of the possibilities for the application of AI tools, primarily ChatGPT, in different legal professions. The second part analyzes a segment of the basic legal and ethical regulations in this area followed by an overview of earlier studies that dealt with surveying lawyers regarding the application of AI tools. Furthermore, the research conducted for the purposes of this paper among legal professionals in Serbia regarding their opinion on the use of AI and ChatGPT in their work is presented. The results of the questionnaire are analyzed as well. Finally, the main conclusions are presented.

Characteristics of the Use of AI in Legal Professions

As AI technology advances, so do new opportunities for its application. A subfield of AI applications in law are technological solutions that help (or even substitute) lawyers in performing certain legal tasks. This field is very diverse and includes automated contracts, online dispute resolution, legal analytics, etc., and is expected to have the most disruptive impact on the legal profession (Bues and Matthaei, 2017).

The popular and widely used ChatGPT is the subject of various research and analyses, both regarding its positive sides and advantages and its negative aspects. The main advantage of ChatGPT in the legal profession is, first of all, the possibility of processing a large amount of data in the shortest possible time. For the purposes of t his paper, the authors consulted ChatGPT regarding its role in the legal

© 2024 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license .

profession. Although the answer contained some grammatical and terminological errors, it was interesting to note that it was both concise and precise. So, paraphrasing this smart assistant, the following stands out:

• •    Legal advice and consultation, researching legal issues, and providing detailed analysis of legal problems or cases;

• •    Preparation of legal documents, such as contracts, requests, and the like;

• •    Assistance in representing clients before the court;

• •    Assistance in cases involving international transactions, agreements, and disputes;

• •    Legal technology, such as contract automation tools, electronic document storage, etc.

Research conducted by Iu and Wong (2023) showed that ChatGPT has advanced drafting capabilities for various legal documents. Also, it could understand simple facts and articulate the legal basis of a claim. It even showed excellent skills in analyzing more complex cases. However, its data sources were limited, especially in relation to recent legal cases. In this context, the authors conclude that ChatGPT at this stage of development presents a good additional tool to lawyers but cannot be seen as their replacement.

Within the judiciary, AI can find its application in limiting the discretion of judicial authorities, by finding algorithmic solutions for analyzing legal problems and making decisions (Mijatović, 2024).

Apart from the already mentioned numerous possibilities of facilitating the performance of legal work, another possibility for the implementation of AI in the legal profession is the creation of a new legal profession – a legal prompt engineer, who would be in charge of entering current data on the basis of which the tools would generate new data for end users.

On the other hand, generative AI tools like ChatGPT have their limitations and downsides. In the legal context, Mijatović (2024) states that AI cannot provide complete legal certainty, given its inherent limitations. As mentioned earlier, it can be useful in data analysis, outcome prediction, and the like, but it does not have the capacity for empathy like a human, i.e. it cannot manage more complex situations and their contextualization. The author concludes that it is unlikely that AI will replace judges or lawyers in making legal decisions in the foreseeable future. On a similar note, the authors Avramović and Jovanov (2023) conducted an analysis which indicates that AI contributes to greater impartiality in the judicial decision-making process, but only as an auxiliary tool. Establishing legal certainty is too big a task for AI, but it can have its application in increasing the predictability of actions. Moreover, the research of Guleria (2024) showed that in its current form, ChatGPT has a limitation in the field of forensic science and law, and should be used with caution, regardless of its many positive aspects.

Therefore, the application of AI in legal science is difficult, due to the numerous variables that affect its processes. Nevertheless, it can facilitate and speed up legal decision-making, and in that context, AI represents a tool for increasing efficiency and effectiveness (Radosavljevic et al., 2023).

Legal and Ethical Regulation of AI

Given that AI is still in its infancy, it is understandable that lawmakers around the world are just beginning to deliberate on the right way to regulate it. All the more so, it is not clear in which direction the development of this technology will go. At the international level, a significant step forward was made in the European Union (EU) with the adoption of the Artificial Intelligence Act, the world’s first comprehensive AI law. It aims to protect human rights, democracy, the rule of law, and environmental sustainability from high-risk AI, while at the same time encouraging innovation and ensuring Europe’s leading role in this field (Paunović, 2024).

To this date, Serbia has not yet adopted any laws on AI, but harmonization with EU regulations is expected. Among the strategic documents, there is the Artificial Intelligence Development Strategy for the period 2020-2025, which determines the goals and measures for the development of AI, the implementation of which should result in economic growth, the improvement of public services, the improvement of scientific staff and the development of skills for the jobs of the future. Also, the implementation of the measures of the Strategy should ensure that AI in the Republic of Serbia is developed and applied in a safe manner and in accordance with internationally recognized ethical principles in order to use the potential of this technology to improve the quality of life of each individual and society as a whole, as well as to achieve the Sustainable Development Goals (“Official Gazette of the RS”, no. 96/2019).

The Government of Serbia has also adopted the Ethical Guidelines for the Development, Implementation, and Use of Reliable and Responsible AI, which recommends that state administration bodies and holders of public authority apply Ethical Guidelines when developing, implementing, and using systems that can be classified as AI systems or procurement thereof. It is recommended that all people

(including natural and legal persons) who develop, implement, and use systems that can be classified as AI systems, or acquire them, should apply the Ethical Guidelines (“Official Gazette of the RS”, No. 23/2023).

The use of AI in practice has already led to certain sanctions due to its careless application. An interesting example is one from the United States of America, where the use of AI is already relatively widespread. Namely, the US Federal Court fined lawyers who used ChatGPT to write lawsuits. The biggest problem was that ChatGPT used quotes from cases that did not exist, and the lawyers “didn’t bother to check the accuracy of the information they were using” (Binder, 2023).

Therefore, legal and ethical regulations in the field of AI are still deficient, and rudimentary norms are only slowly emerging, primarily in the form of guidelines and ethical rules.

Overview of previous research

To the best of the authors’ knowledge, research on the impact of AI on the legal profession is very rare in Serbia, so several interesting foreign studies will be presented. In a study conducted by the Thomson Reuters Institute (2023), there were over 400 respondents, lawyers in medium and large law firms in the United States of America, the United Kingdom, and Canada. The study showed that only 3% of respondents used AI. Over 30% of respondents indicated that their firm was still considering whether to incorporate the use of generative AI for legal work. About 15% of respondents said they were warned not to use unauthorized artificial intelligence at work, while 6% said companies banned such use. The largest number of respondents stated that they do not fully trust generative AI tools, especially the ChatGPT tool when it comes to confidential customer data. Also, the results showed that the awareness of AI risks is growing, as 62% percent of respondents expressed concern over the use of Chat GPT for business purposes. The respondents expressed most concerns regarding the accuracy and security of the technology, especially regarding the privacy of clients, loss of work, etc. The authors concluded that it is becoming certain that ChatGPT will be as common in the legal profession as online research and electronic contracts.

In a study conducted by The Law360 Pulse AI 2023 and 2024 which included 384 respondents, mostly senior male lawyers, the results showed a growing interest in AI and its implementation in the legal sector, with as many as 41% of firms having an AI provider. They mostly use it for legal research and document creation, but only 16% of firms encourage AI use, while 14% offer training. The respondents also expressed concerns over human substitution, ethical implications, and the accuracy of generated content. The study also showed that companies are skeptical about complex legal tasks, where human supervision is needed.

A study conducted among over 4.000 participants (lawyers, law students, and consumers) from the USA in March 2023 showed that lawyers have a high awareness of generative AI tools like ChatGPT (86%) (LexisNexis, 2023). The use of AI tools for various legal tasks was confirmed by 36% of lawyers, but they also had concerns regarding ethical consequences and data protection. When asked about the impact generative AI will have on legal practice, a large majority of lawyers (89%) and law students (92%) stated it will have at least some impact, and 39% of lawyers and 46% of law students said that impact will be “significant” or “transformative.” Thus, the legal community expressed cautious optimism, recognizing the possibilities of AI in the legal profession, but emphasizing the need for ethical handling of these smart tools and human oversight.

Materials and Methods

For the purposes of this paper, research was conducted through a survey created using the Google Forms tool. A total of 143 respondents from various legal professions participated in the research. The online questionnaire consisted of 23 questions with open and closed-answer modalities. The questions were grouped into two parts: the socio-demographic characteristics of the research participants and the attitude of the research participants towards the use of AI within the legal profession. The online questionnaire was distributed via e-mail addresses and Viber. Participation in the research was anonymous and voluntary and without any financial compensation. The study was conducted in accordance with the Helsinki Declaration during the month of June 2024.

The collected data were analyzed using adequate statistical methods and presented graphically. A statistically significant association between two categorical variables was tested using the Chi-square test of independence. The significance level in all tests was 5.

Results

The majority of respondents were female (65%). The age range was from 23 to 65 and the average age of the respondents was 36 years. In terms of education, the same percentage of respondents had completed bachelor’s and master’s studies (46.2% each) and the smallest percentage had a PhD (7.7%). The largest number of respondents lived in the province of Vojvodina (55.9%), while the rest lived in the region of Belgrade (21%), Šumadija and Western Serbia (13.3%), Southern and Eastern Serbia (7.7%), with the smallest percentage of participants being from Kosovo and Metohija (2.1%). The largest percentage of respondents practiced law in court as attorenys/barristers (28%), followed by respondents who were employed in state, provincial, and local authorities or the public service (21.7%), or who worked in the judiciary and performed other jobs within the judiciary (20.3%). The next highest percentage of respondents were legal experts employed in the commercial sector (18.2%) and had an academic career (9.8%), and the lowest percentage were respondents employed in education (5.6%).

The segment of the questionnaire entitled “Questions about artificial intelligence” contained several questions related to the use of AI tools and the respondents’ perception of their usefulness and expediency in the legal profession. Thus, the first question concerned familiarity with the term AI. The vast majority of respondents indicated that they were familiar with this term, partially (52.4%), or fully (44.8%). Only a few respondents stated that they were not familiar with this term or that they had heard of it, but could not explain what it meant (0.7% each). In terms of familiarity with the term “ChatGPT“, the largest percentage of respondents were fully familiar with this term (41.3%), followed by those who were partially familiar (34.3%), while a smaller percentage had not heard of this tool at all, or had heard of it (11.9%), but could not explain what it was specifically (4.2%).

When asked if they ever use some of the AI tools in their work, more than half of the respondents stated that they never use them (60.1%), 16.1% of respondents said they rarely do so, while 14% used them occasionally and 7.7% of respondents used them regularly. Similarly to the previous answer, when it comes to ChatGPT, as many as 65% have never used this tool at work, while 14% rarely use it, 12.6% use it occasionally, and only 6.3% use it regularly.

Figure 1. Do you ever use ChatGPT in your work?

When assessing the usefulness of AI at work, a total of 57.4% of the respondents stated that it can be partially or completely useful, 18.9% believe that it could be, but they were not certain in what way, 11.9% believed that it is not useful, and the smallest percentage stated that it can even be harmful (4.9%). Regarding the usefulness of ChatGPT in legal work, our respondents stated in a relatively high percentage that this tool can be useful (49.7%), followed by respondents who thought it could be useful, but they were not sure in what way (18.9%), while 11.2 % of respondents believed that it is not useful or that it can even be harmful (6.3%).

Of the respondents who considered AI useful in the work of legal professionals, most of them stated that the reason for this is faster access to data (73.4%). Furthermore, they stated that it can help organize documents (46.9%), facilitate the preparation of submissions and other legal documents (32.8%), allow more time for other activities (28.1%), or facilitate communication with clients (18%). In the form of a free answer, the respondents could write the reasons for the usefulness of AI, and one answer can be high-lighted: “It can direct the user towards finding a solution to the problem more easily, by selecting the appropriate topic”.

Figure 2. What legal task is AI most useful for?

As the future of many knowledge workers is a current issue, the respondents were asked whether they thought that AI tools, such as ChatGPT, could replace the legal work they do in the future. The majority of respondents (58%) answered that they could not, about a fifth of respondents said they could partially, and 9% said they could completely. 14.7% of respondents did not know, that is, they were not sure.

Regarding the ethical justification of the use of AI in law, the respondents’ answers were divided. T largest percentage of respondents (40.6%) answered that it was partially justified, and 10.9% that it was completely justified. A quarter of respondents thought that it was not justified (25.2%), while 23.8% did not know, that is, they were not sure about this issue.

When it comes to legal regulation in AI, the largest percentage of respondents answered that they were not familiar with it (37.8%), 37.1% were partially familiar with it, and only 6.3% of respondents stated that they were fully familiar with it. More than half of the respondents stated that Serbia needs a law on artificial intelligence (64.3%), and only 4.9% said that it is not needed. A certain percentage of respondents were not sure whether it was needed or not (10.5%).

When asked how the use of AI tools like ChatGPT should be regulated in the legal profession, the largest percentage of respondents stated that the use of this tool should be limited, but not prohibited (59.4%). Some respondents believed that the use of this tool is unacceptable in the judiciary (7%) and in legal practices (for attorneys/barristers) (2.1%). About 7% of the respondents stated that the use of this tool should be prohibited in any legal work, and 8.4% of the respondents stated that the use of this tool is unacceptable for certain legal tasks, while it is fine for others. Also, respondents expressed the need to protect data in order to prevent misuse and plagiarism.

16%      7%  2%

• ■    The use of Chat GPT in any legal profession should be prohibited.

• ■    The use of ChatGPT in the judiciary is unacceptable.

• ■    The use of ChatGPT in the legal practice (for attorneys/barristers) is unacceptable.

I The use of ChatGPT is unacceptable for certian legal tasks, but is acceptable for others.

• ■    The use of ChatGPT should be limited, but not prohibited.

• ■    I do not know - I do not have an opinion

Figure 3. How should the use of ChatGPT in the legal profession be regulated?

When asked if they were ready to pay for some additional performance of the AI tool, the majority of respondents answered that they were not ready (36.4%), and a smaller percentage said that they were partially ready for it (24.5%), or that they were fully ready (9.8%).

When evaluating the legal tasks for which AI would be most useful, most respondents decided that it was the search for legal regulations (76.2%), followed by help with translation (48.3%), the search for basic legal terms (42%), drafting submissions, contracts, decisions and other legal acts (47.6%), assistance to students and interns, and communication with clients. In a free answer, respondents stated that they used AI when creating procedures, for planning, and for explaining unclear concepts.

Finally, the respondents chose the statements about artificial intelligence with which they agreed most. Thus, the largest percentage of them chose the statement “AI can be a useful tool if it is used carefully and in accordance with legal and ethical standards” (49%). The next most frequently chosen statement was that AI will never be able to replace humans and that attorneys/barristers and other legal professionals will always be needed (28.7%). The claim that AI is the future of the legal profession and that it will speed up and facilitate business was the most acceptable one for 11.2% of respondents. A certain number of respondents believed that they did not have enough knowledge about AI to form an opinion (6.3%). About 4% of respondents chose the statement that AI is a danger to the legal profession. The fewest respondents chose the statement that AI will completely replace legal professionals (0.7%).

In the form of a free answer, the respondents could express some additional opinions on this topic. We will highlight the answer that “AI will never be able to understand emotions, non-verbal reactions, and feelings”, as well as the one that “AI does not have the possibility of improvisation, argumentation and finding creative solutions” which is needed in the legal profession. Also, one respondent expressed fear that AI “will distance legal professionals from reading books and communicating with colleagues about professional concepts and cases”.

Statistically significant differences were found in only one crossing of variables. Thus, there are statistically significant differences in familiarity with legal regulations in the field of AI depending on the type of legal work respondents did (Chi-square=44.621, df=21, p=0.002), whereby academics stated that they were the most familiar, while legal professionals working in education were the least familiar with it.

Discussion

Based on the presented results, summary conclusions can be drawn. The largest percentage of our respondents were familiar with AI but to a lesser extent with ChatGPT. However, the largest percentage of respondents never used AI tools in their professional work. This result is not surprising, given that ChatGPT is a relatively new phenomenon. These results are in line with foreign research, namely the LexisNexis (2023) study conducted in the USA, but our respondents stated that they use AI tools to a greater extent than the respondents from some other foreign studies (around 30% of our respondents said that they did use AI tools, although rarely, compared to just 3% in the Thomson Reuters study).

The largest percentage of respondents positively evaluated AI in terms of its usefulness in legal professions, but with awareness of the need for its regulation, and use in accordance with legal and ethical rules. Thus, the optimal way to regulate ChatGPT, according to the majority of respondents, would be to limit, but not prohibit the use of this tool.

Respondents mostly had an affirmative attitude towards the survival of the legal profession and its necessity in the future, but the research results also showed that they understand and accept the useful aspects of AI. The most useful aspect of the AI tool is seen in the faster access to the necessary data, and even the performance of more complex functions, such as the drafting of submissions and other legal documents, which a large number of respondents rated as a positive side of the AI. The negative sides are the possibility of alienation from colleagues, excessive reliance on technology, and the issue of protecting sensitive data.

It should be emphasized that these results, especially the low usage of smart tools, do not come as a surprise and that they are in line with the data that the participation of generative AI in the Serbian economy is almost non-existent, so it is no wonder that they are poorly applied in the legal profession as well. Moreover, although popular large language models, such as ChatGPT, have improved their understanding of the Serbian language, they are still not well adapted to it, and even less to a specific legal language (Uspenski, 2023). Although there are some domestic tools that represent a step forward in this context (Bogdanović et al., 2024), the use of this or a similar tool cannot be more widespread and fruitful until its base is adapted to domestic sources and language.

Conclusions

Using intelligent tools like ChatGPT in the legal profession can be interesting and bring some relief, but smart tools should be handled with care. Firstly, it is necessary to keep in mind the legal and ethical codes. Secondly, ChatGPT itself has its limitations, and it should be used with caution, always checking the data it generates, as it may produce incorrect or inappropriate responses.

In this paper, the goal was to gain a general insight into the problems of applying AI tools in legal practice, and therefore a survey was conducted among various legal experts working on different types of legal work. The main findings are that legal professionals in Serbia are well-informed about AI and ChatGPT, but that these tools are rarely used in their work. Nevertheless, our respondents recognized the possibilities of AI tools when performing various legal tasks. Also, the results showed that most respondents had a moderate attitude towards AI: they believed that the use of computer tools is useful, but that it should be limited by legal and ethical regulations. A significant number of respondents were also ready to pay for additional AI performance, which indicates a relative openness to new technologies within the legal profession. The research also showed that there is a large space for improvement of smart tools, so as to make them more useful and reliable. Therefore, the results of this research can be of importance to decision-makers, creators of new technologies, as well as to legal professionals themselves.

The limitations of this research concern, first of all, the sample size and structure. Thus, the opportunity for future research is very broad and can go in two directions. The first one concerns the narrowing of the sample to a specific legal profession. The second one concerns expanding the survey questions to additional aspects of AI use, which would lead to new insights.

In conclusion, intelligent tools should be used with caution, with the best results provided by a combination of technology and expertise. AI can facilitate and expedite the work, however, there should always be corrective action performed by a legal professional. The goal of AI should be assistance in performing tasks, to the satisfaction of both legal professionals and their clients.

Acknowledgments

We would like to thank all the research participants, especially the Association of Legal Practitioners of Serbia (Udruženje pravnika Srbije) and the Kec Group (Kec grupa), who helped distribute the survey to their members.

Conflict of interests

The authors declare no conflict of interest.