BiKD

Balancing Knowledge Distillation for Imbalance Learning with Bilevel Optimization

arXiv:2605.17839v3 Announce Type: replace Abstract: Knowledge distillation transfers knowledge from a high capacity teacher to a compact student using a mixture of hard and soft losses. On imbalanced data, a fixed weighting between hard and soft losses becomes brittle the learning process. Recent studies try to reweight these components in long-tailed settings.