File Download
There are no files associated with this item.
Supplementary
-
Citations:
- Appears in Collections:
Conference Paper: Toward Better PAC-Bayes Bounds for Uniformly Stable Algorithms
| Title | Toward Better PAC-Bayes Bounds for Uniformly Stable Algorithms |
|---|---|
| Authors | |
| Issue Date | 10-Dec-2023 |
| Abstract | We give sharper bounds for uniformly stable randomized algorithms in a PAC-Bayesian framework, which improve the existing results by up to a factor of √nn (ignoring a log factor), where nn is the sample size. The key idea is to bound the moment generating function of the generalization gap using concentration of weakly dependent random variables due to Bousquet et al (2020). We introduce an assumption of sub-exponential stability parameter, which allows a general treatment that we instantiate in two applications: stochastic gradient descent and randomized coordinate descent. Our results eliminate the requirement of strong convexity from previous results, and hold for non-smooth convex problems. |
| Persistent Identifier | http://hdl.handle.net/10722/347750 |
| DC Field | Value | Language |
|---|---|---|
| dc.contributor.author | Zhou, Sijia | - |
| dc.contributor.author | Lei, Yunwen | - |
| dc.contributor.author | Kaban, Ata | - |
| dc.date.accessioned | 2024-09-28T00:30:21Z | - |
| dc.date.available | 2024-09-28T00:30:21Z | - |
| dc.date.issued | 2023-12-10 | - |
| dc.identifier.uri | http://hdl.handle.net/10722/347750 | - |
| dc.description.abstract | <p>We give sharper bounds for uniformly stable randomized algorithms in a PAC-Bayesian framework, which improve the existing results by up to a factor of √nn (ignoring a log factor), where nn is the sample size. The key idea is to bound the moment generating function of the generalization gap using concentration of weakly dependent random variables due to Bousquet et al (2020). We introduce an assumption of sub-exponential stability parameter, which allows a general treatment that we instantiate in two applications: stochastic gradient descent and randomized coordinate descent. Our results eliminate the requirement of strong convexity from previous results, and hold for non-smooth convex problems.<br></p> | - |
| dc.language | eng | - |
| dc.relation.ispartof | Advances in Neural Information Processing Systems (10/12/2023-16/12/2023, New Orleans) | - |
| dc.title | Toward Better PAC-Bayes Bounds for Uniformly Stable Algorithms | - |
| dc.type | Conference_Paper | - |
| dc.identifier.spage | 29602 | - |
| dc.identifier.epage | 29614 | - |