Record linkage in data integration problem under big data conditions

The problem of identifying records refer to the same entity arises appears during the integration data from multiple sources. The application of probabilistic record linkage is one of the key to solve described problem. In this article defined and tried that application of locality-sensitive hashing and vector space model on the blocking stage allow to reach the efficient implementation of described above decision. The implementation is tested in Apache Spark on two registers of companies GLEIF and Companies House.