KL-triggered Continual Adaptation for Nonstationary Resource Allocation: An Off-policy Actor–critic Approach with Nash Social Welfare

This paper proposes a drift-aware off-policy deterministic actor–critic framework for constrained continuous resource allocation in non-stationary environments. Feasible allocations are ensured by a simplex-parameterized policy using softmax normalization with budget scaling, avoiding projection or Lagrangian tuning. The reward integrates Nash social welfare via mean log-utility, efficiency, fairness, and constraint-violation penalties with adaptive weights. To improve sample efficiency, we adopt prioritized experience replay based on TD error and state novelty. Non-stationarity is detected by KL divergence between recent and historical state-visitation distributions; detected drift triggers buffer refresh and incremental fine-tuning, while Elastic Weight Consolidation mitigates catastrophic forgetting. Experiments across six application-motivated domains (food, medical, housing, education services, employment support, and elderly care) demonstrate improved utilization and welfare with reduced inequality and low decision latency compared with optimization, heuristic, and DRL baselines. Results are reported over multiple runs with mean ± standard deviation and corrected significance tests.