OpenResearchAlpha

arXiv · 19 Nov 2026

A new algorithmic approach for finding flat minima: critically damped Polyak stepsizes

Francesco Caporali, Jason M. Altschuler, Boris Hanin

0
Citations
no database records one yet
0
Citations in 2026
so far
–
Leading author's h-index
no author has a profile yet
0
References
works it cites, from open citation data

Citations by database

Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.

  • OpenCitations0

Citations per year

The current year is still counting.

  • 2026 so far0

Abstract

We present a new adaptive stepsize rule that biases gradient descent toward flatter minima in both theory and practice, and establish the first uniform-in-time guarantees for both sharpness and loss reduction for a broad class of locally quadratic objectives. Our critically damped Polyak (CDP) stepsizes multiply the classical Polyak rule by $4$ and a mild damping factor. The factor $4$ is critical in that it produces equioscillations around the manifold of minimizers $\mathcal{M}$ to leading order. Damping provably drives convergence to $\mathcal{M}$. CDP thus operates at the edge of stability by adapting its stepsizes to local curvature. Once the loss is small, its projected iterates follow gradient descent on $\log{λ_1(\nabla^2 \mathcal{L})}$ along $\mathcal{M}$ to leading order, paralleling the flatness bias of normalized gradient descent (Arora et al., 2022). Unlike normalized gradient descent, CDP step lengths shrink with the distance to $\mathcal{M}$, enabling convergence to an interpolating solution that locally minimizes sharpness at finite learning rate. We establish our theory for a broad class of non-negative losses with locally quadratic Morse-Bott structure near $\mathcal{M}$, capturing regular interpolation regimes in smooth overparameterized neural networks. Empirically, CDP stepsizes produce a flattening effect similar to that of Sharpness-Aware Minimization (SAM, Foret et al., 2021), reducing sharpness even in settings beyond our theory, including nonlinear teacher-student networks and fine-tuning of pre-trained LLMs.

Machine learningArtificial intelligenceOptimization and control

What the data shows

  • No database records a citation yet, as is usual for a paper this new.

Graph

This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.

Citing paper shares an authorAuthor without a profileSize shows citations, or h-index for authors

Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.

Start with

Authors (3)

Each author's standing in QAI's scholar index: the works their ORCID iD is attached to, and for anyone whose ORCID record names a lab OpenResearch follows, their arXiv preprints matched by name and a shared co-author. Names are as printed on the paper.

ProfileMost-cited paper

Cited by 0 papers

Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.

0 shown
No citing papers for this filter.

References (0)

What it cites, from open citation data, with Crossref's record of each.

No references in the open citation data yet.
A new algorithmic approach for finding flat minima: critically damped Polyak stepsizes - QAI OpenResearch