Decay schedule API reference

Exp{T}(start, decay)
Exp(; start, decay)

A exponential decay schedule at rate decay. The output conforms to

start * decay^{t - 1}

Arguments:

• start: the base value
• decay: the decay rate

Inv{T, S<:Integer}(start, decay, degree)
Inv(; start, decay, degree)

A decay schedule that inversely decays with rate decay. The output conforms to

start / (1 + (t - 1) * decay)^degree

Arguments

• start: the base value
• decay: the decay rate
• degree::Integer: the degree of decay

Poly{T, S<:Integer}(start, degree, max_iter)
Poly(; start, degree, max_iter)

A polynomial schedule decays with degree degree. The output conforms to

start / (1 - (t - 1) / max_iter)^degree

Arguments

• start: the base value
• degree::Integer: the degree of the polynomial
• max_iter::Integer: the total number of iterations

Step{T, S<:Integer}(start, decay, step_sizes)
Step(; start, decay, step_sizes)

A step schedule decays exponentially by decay every step in step_sizes. The output conforms to

start * decay^{i - 1}

where sum(step_sizes[1:(i - 1)]) < t <= sum(step_sizes[1:i])

Arguments

• start: the starting value
• decay: the decay rate
• step_sizes::Union{<:Integer, <:Vector}: the step sizes