Interference-aware Deep Quantum Neural Network for NOMA Channel Estimation via Adaptive Energy Valley Optimization

To address the growing multi-user interference in dense wireless networks, we propose an interference-aware Deep Quantum Neural Network (DQNN) for channel estimation in the Non-orthogonal multiple access (NOMA) systems. The proposed method incorporates a hybrid classical-quantum architecture. A Transformer-encoder processes the pilot signals to extract spatiotemporal features. A parameterized quantum circuit maps the processed features into a high-dimensional Hilbert space. The enhancement hinges on an Adaptive Energy Valley Optimization (AEVO) algorithm, which modifies the optimization trajectory using interference-aware preconditioners derived from the interference covariance structure. With the aid of these preconditioners, the DQNN can steer through the NOMA's non-convex terrain characterized by interference to enhance estimation performance. Moreover, interference-aware preconditioning is achieved through a lightweight neural network which adapts to time-varying interference. The successive interference cancellation decoder uses the estimated channel matrix to recover symbols. By further analysing the results, it is noticed that the quantum-enhanced machine learning delivers better results than the classical ones. The proposed framework enhances the state-of-the-art in NOMA channel estimation, while also providing a general framework for interference-aware optimization in quantum machine learning. At 10 dB SNR, the AEVO-DQNN method with a 16x16 antenna array obtained a minimum NMSE of 0.012288 and a minimum BER of 0.013023. Further, the proposed method outperforms the competing methods in terms of NMSE/BER mean with 95% confidence intervals, interference rejection ratio analysis, sensitivity to estimation error and estimated interference covariance, and paired t-test analysis.