Pointwise AttnRes

DeRes: Decoupling Residual Stability and Adaptivity for Scalable CTR Prediction

arXiv:2606.07980v1 Announce Type: new Abstract: Transformer-based CTR models face a growing bottleneck at the residual connection: under Pre-Norm, early user-interest signals are diluted layer by layer; the identity skip cannot forget stale interests; and each layer sees only its immediate predecessor, losing long-range cross-layer dependencies. Recent attention-based residual variants (AttnRes) address parts of this in language models, but drop the protective identity skip and have not been...