LRD
No mentions found
This entity hasn't been tracked yet, or Iris is still building its knowledge base.
Related Articles from SNS
Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View
Announce Type: new Abstract: Modern Transformer architectures frequently employ normalization mechanisms such as RMSNorm and Query-Key Normalization, making parts of the model approximately scale-invariant with respect to weight magnitudes. In this regime, standard Frobenius-norm weight decay acts purely along the radial direction of the weight space and cannot directly simplify the function represented by the normalized layer. We study grokking in small algorithmic tasks through this lens...
Black hole feeding bursts may explain JWST's Little Red Dots in early universe
June 8, 2026 report Black hole feeding bursts may explain JWST's Little Red Dots in early universe Shreejaya Karantha Author Sadie Harley Scientific Editor Robert Egan Associate Editor A new theoretical study may have cracked one of the most puzzling discoveries of the James Webb Space Telescope (JWST): Little Red Dots, spotted across the early universe. The paper, posted to the arXiv preprint server on May 29, argues that these objects could be black holes caught in rare, violent bursts of...