Getting started

All schedules types in ParameterSchedulers.jl behave as indexable iterators. For example, we can index the simple exponential decay schedule (Exp) below:

s = Exp(λ = 0.1, γ = 0.8)
println("s[1]: $(s[1])")
println("s[5]: $(s[5])")

s[1]: 0.1
s[5]: 0.04096000000000001

Info

The indices are unitless. So, if you index a schedule every epoch, then the s[i] is parameter value at epoch i.

We can also use the schedule in an iterable context like a for-loop:

for (i, param) in enumerate(s)
    (i > 10) && break
    println("s[$i]: $param")
end

s[1]: 0.1
s[2]: 0.08000000000000002
s[3]: 0.06400000000000002
s[4]: 0.051200000000000016
s[5]: 0.04096000000000001
s[6]: 0.03276800000000001
s[7]: 0.026214400000000013
s[8]: 0.020971520000000007
s[9]: 0.016777216000000008
s[10]: 0.013421772800000007

Warning

Many schedules such as Exp are infinite iterators, so iterating over them will result in an infinite loop. You can use Base.IteratorSize to check if a schedule has infinite length.

Notice that the value of s[1] and s[5] is unchanged even though we accessed the schedule once via getindex and again in the for-loop. This is because all schedules in ParameterSchedulers.jl are immutable. If you want a stateful schedule, then you can use ScheduleIterator:

stateful_s = ScheduleIterator(s)
println("s: $(ParameterSchedulers.next!(stateful_s))")
println("s: $(ParameterSchedulers.next!(stateful_s))")
println(stateful_s)

s: 0.1
s: 0.08000000000000002
ScheduleIterator{Exp{Float64},Int64}(Exp{Float64}(0.1, 0.8), 3)

We used ParameterSchedulers.next! to advance the stateful iterator. Notice that stateful_s stores a reference to s and the current iteration state (which is 3 since we advanced the iterator twice). Also note that ScheduleIterator cannot be indexed (or iterated with Base.iterate):

try stateful_s[1]
catch e
    println(e)
end

MethodError(getindex, (ScheduleIterator{Exp{Float64},Int64}(Exp{Float64}(0.1, 0.8), 3), 1), 0x0000000000007056)

Tutorials

Interfaces

Getting started