Development of distributed applications for data collection and analysis on the basis of a microservice architecture

The article discusses the use of the microservice architecture in the field of computational science. Microservice architecture is mainly used for distributed applications in industrial systems and is not quite common in the field of scientific computing and data analysis. However, its use is now becoming increasingly important. Decomposition of scientific applications related to computing and data processing to the level of microservices has potential benefits, known in the field of industrial systems: increased flexibility, reduced support complexity, simplification of application scaling, etc. Decomposition of a system to microservices is challenging and there are several techniques that enable extraction of the services from a monolithic system. There is no uniform approach to system decomposition in industrial systems; for each system you need to choose a methodology, based on the application domain features, the connectivity of subsystems and other system properties. The paper provides an overview of the main approaches to the implementation of scientific computing systems based on the microservice architecture, given the differences of scientific systems and industrial systems, and highlights the aspects that enable easier application of the microservice architecture in scientific applications in comparison with industrial systems. Based on the example of the system for data collection and analysis of the Sergey Korolev computing cluster load, we demonstrate the principles of determining the service boundaries during the initial decomposition of a monolithic system. The migration of an existing application to the microservice architecture is a unique process for each system and cannot be performed in a single iteration. As an example of such migration, we demonstrate the transformation of the Templet Web cloud service from a monolithic version of the system with 3-tier architecture to the microservice version of the system with a dedicated Service Discovery / API Gateway service. The tightly integrated parts of the system have not been broken down into microservices in order to not complicate the system design. The authors provide practical guidance solutions for problems arising from the operation of the system based on microservices. In conclusion, the major differences between the microservice version of the system and the monolithic are provided, as well as the benefits that enable the microservice architecture to address the larger range of tasks comparedtomonolithicsystems.