Application of the computer vision system reconstructing the three-dimensional form of space technology objects

Trends in the development of the spacecraft (SC) industry present increasingly complex tasks for mechatronic systems. Due to the complexity of the assembly processes of large-sized transformable structures and solar panels, the current methods of opening structures are becoming less reliable and relevant. The development of orbital stations in addition leads to a whole range of new tasks. Using mechatronic manipulators is one of the promising approaches. However, the functionality of their application is severely limited by the algorithms for moving the executive body and manipulating objects, predetermined during development. In this regard, there is a need to develop new approaches to finding and determining the shape of the target object, for further calculation of the algorithm for its interaction with the mechatronic manipulator gripper. The article presents a method for reconstructing the three-dimensional shape of objects, based on using a machine vision system. A stereo camera is used to obtain the scene's spatial characteristics. Based on the data obtained, the scene is divided into several independent geometric surfaces, followed by staged processing by neural network algorithms. The required parameters of the target objects are extracted at each stage of the algorithm. YolactEdge is used as a neural network architecture, which performs semantic segmentation and classification of objects. A separate task can be the correlation of the spatial characteristics of the target objects and the replacement three-dimensional model. To ensure this correlation, the neural network architecture was supplemented with a keypoint detection branch which provides a prediction of the positions of the reference points of objects that uniquely determine the spatial characteristics of the target object. As a result, the system obtained is able to provide the construction of a three-dimensional map of the turning radius area in real time. In addition, based on the received telemetric information, it is possible to calculate the trajectory of the manipulator's executive body and its interaction with objects.