Optimal Autoencoder Structure Selection Using Bayesian Optimization

The paper considers the problem of the structure selection for autoencoder model. An autoencoder is a model differentiable by parameters, represented as a composition of two functions: an encoder representing the input object as a latent vector representation, and a decoder transforming the latent vector representation into the original feature space. The structure of the autoencoder model is represented as a set of hyperparameters, for the selection of which it is proposed to apply Bayesian optimization methods. A two-stage modification of the Bayesian optimization method is proposed: at each search iteration, a set of points with the best estimate of the model quality is selected, and then the best one is selected from them taking into account the dynamics of training. The theoretical justification of the algorithm is given, and experiments on CIFAR and Fashion-MNIST samples confirm the effectiveness of the proposed approach.