AccSGD {sjSDM}

AccSGD

Description

accelerated stochastic gradient, see Kidambi et al., 2018 for details

Usage

AccSGD(kappa = 1000, xi = 10, small_const = 0.7, weight_decay = 0)

Arguments

kappa

long step

xi

advantage parameter

small_const

small constant

weight_decay

l2 penalty on weights

Value

Anonymous function that returns optimizer when called.

References

Kidambi, R., Netrapalli, P., Jain, P., & Kakade, S. (2018, February). On the insufficiency of existing momentum schemes for stochastic optimization. In 2018 Information Theory and Applications Workshop (ITA) (pp. 1-9). IEEE.