ZO
No mentions found
This entity hasn't been tracked yet, or Iris is still building its knowledge base.
Related Articles from SNS
Dominant-Layer ZO: A Single Layer Dominates Zeroth-Order Fine-Tuning of LLMs
arXiv:2606.05516v1 Announce Type: new Abstract: Zeroth-order (ZO) optimization enables memory-efficient fine-tuning of large language models (LLMs) using only forward passes, but it remains unclear how useful adaptation is distributed across layers. In this work, we reveal a surprising phenomenon: ZO fine-tuning is sharply dominated by a single decoding layer. Across multiple LLM families and downstream tasks, fine-tuning this dominant layer alone consistently matches or even exceeds...
Revisiting Zeroth-Order Hessian Approximation: A Single-Step Policy Optimization Lens
Announce Type: new Abstract: Accurate Zeroth-Order (ZO) Hessian estimation is a cornerstone of derivative-free methods, essential for tasks such as bilevel optimization, Bayesian inference, and uncertainty quantification. However, obtaining a complete suite of low-variance estimators for the Hessian and its inverse in high-dimensional settings remains a significant challenge. To address this, we propose a unified framework that reinterprets ZO Hessian approximation through the lens of...
Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs
Announce Type: replace Abstract: Zeroth-order optimizers have recently emerged as an attractive approach for fine-tuning large language models (LLMs), as they avoid backpropagation and can substantially reduce memory overhead relative to standard first-order training. However, existing zeroth-order methods rely on hand-crafted, static sampling strategies that are not adaptable to model-specific structures. To address this, we propose ZO-Finetuner, a learning-based zeroth-order optimizer for...
GRZO: Group-Relative Zeroth-Order Optimization for Large Language Model Fine-Tuning
Announce Type: new Abstract: Zeroth-order (ZO) optimization is a memory-efficient alternative to backpropagation for fine-tuning large language models, but its deployment is limited by the high variance of gradient estimation. We propose GRZO, a Group-Relative Zeroth-Order optimizer that draws one pseudo-independent perturbation per mini-batch example and aggregates the per-example losses through group-relative normalization, raising the effective gradient-direction count from one to the...
Protecting K-Nearest Neighbor Queries from Location Inference Attacks
Announce Type: new Abstract: The k-nearest neighbor query (kNNQ) is a core component of modern location-based services (LBS) and has been widely adopted in popular features such as ``people nearby''. However, its potential privacy risks have long been overlooked. In this work, we present the first two attacks against kNNQ, namely the geometric intersection location inference attack (GI-LIA) and the zero-order optimization location inference attack (ZO-LIA), revealing the inherent location...
ZOAF: Towards Efficient Zeroth-Order Optimization for Analog/RF Circuit Design
Announce Type: new Abstract: Circuit optimization is an indispensable step in analog/RF IC design. Classical fast gradient-based optimization methods are typically infeasible due to lack of access to simulator source code and the technical barriers to implementing adjoint methods. Therefore, surrogate-based black-box optimization is widely used in practice; however, it can be costly to build and sensitive to hyperparameters, whereas population heuristics often suffer from slow convergence...
Day after truck driver’s murder, Manipur highway found dug up
Imphal/Churachandpur: A section of NH-202, the Imphal-Ukhrul lifeline, was found dug up on Saturday near Shangkai Kuki village, allegedly using a JCB excavator. Sources said the road was deliberately excavated by Kuki villagers to disrupt vehicular movement along the highway. Meanwhile, Manipur Police said an attack on Friday at Roudei (TM Kasom) village was allegedly carried out by “armed Kuki militants” positioned along the route from the Patleijang hill area.
Emerge Career (YC S22) Is Hiring a Founding Growth Marketer
About the Role We are looking for our first AI-forward growth marketer who wants to outperform entire marketing teams. The engine is already running. You'll inherit real traction and real data, not a blank slate.
Zeroth-Order Non-Log-Concave Sampling with Variance Reduction and Applications to Inverse Problems
arXiv:2605.30573v1 Announce Type: new Abstract: Sampling from high-dimensional, non-log-concave distributions with unnormalized densities remains a fundamental challenge in machine learning, particularly in black-box settings where gradient information is inaccessible or computationally prohibitive. While Langevin dynamics provides a principled framework for sampling when gradients are accessible, its extension to the black-box settings suffers from high variance and lacks non-asymptotic...