ScheduledOptim{T<:AbstractSchedule, O, F}
ScheduledOptim(schedule::AbstractSchedule, opt, update_func)
ScheduledOptim(schedule::AbstractSchedule, opt; update_func = (o, s) -> (o.eta = s))
(::Type{<:AbstractSchedule})(opt; update_func = (o, s) -> (o.eta = s), kwargs...)

Wrap a schedule and opt together into a ScheduledOptim. The schedule is iterated on every call to Flux.apply!. The ScheduledOptim can be used anywhere a Flux optimizer is used.

The keyword argument constructor sets update_func(opt, s) to schedule the learning rate of opt to s on every iteration. You can update any field of opt by passing your own update_func.

Instead of constructing schedule and opt separately, you can use the ::Type{<:AbstractSchedule} constructor (e.g. Exp below):

# create a Flux.Momentum optimizer
# where the learning rate is adjusted with
# Exp(λ = 0.1, γ = 0.8) decay schedule
opt = Exp(Momentum(); λ = 0.1, γ = 0.8)

Arguments

• schedule::AbstractSchedule: the schedule to use
• opt: a Flux optimizer
• update_func: a mutating function of with inputs (optim, param) that updates optim based on the current param value