Complex schedule API reference

ComposedSchedule([(s, ps) -> T(ps...),] schedule::T, parameters)

A schedule whose fields are given by parameters.(t) at iteration t.

At each step t, this gets a new set of parameters with parameters.(t), then creates a new schedule given the first (optional) argument. The new schedule(t) is the returned value.

Constant{T}
Constant(value)

A constant schedule that is always value.

Interpolator{T, S, F}
Interpolator(schedule, rate, ceil_fn = x -> ceil(Int, x))

A schedule whose output is schedule(t / rate) (i.e. it interpolates schedule(t)).

This can be useful when your code iterates over real numbers at a fixed rate (e.g. in a fixed time step differential solver), but you want to use a schedule that iterates discretely over integers.

It could also be used to specify schedule in units of epochs, while iterating it in units of mini-batches.

Specify ceil_fn to apply a ceiling (or flooring) function to t / rate.

Loop{T, S<:Integer}
Loop(f, period)

Create a schedule that loops f every period iterations. f must be callabe (a function or schedule).

Arguments

• f: the schedule to loop
• period::Integer: how often to loop

Sequence{T, S}
Sequence(schedules, step_sizes)
Sequence(schedule1 => step1, schedule2 => step2, ...)

A sequence of schedules. The output of this schedule is the concatenation of schedules where each schedule is evaluated for each step size in step_sizes.

Note that schedules can also be a vector of numbers (not just schedules).

Arguments

• schedules: a vector of schedules or numbers
• step_sizes: a vector of iteration lengths for each schedule

Shifted(schedule, offset)

A schedule who's starting iteration is shifted to offset. (i.e. calling an Shifted with t = 1 is equivalent to calling schedule with t = offset)

Shortened{T}
Shortened(schedule, nsteps)

A schedule that mimics schedule but throws a BoundsError if accessed beyond nsteps.

OneCycle(nsteps, maxval;
         startval = maxval / 25,
         endval = maxval / 1f5,
         percent_start = 0.25)

Creates a one-cycle cosine schedule over nsteps steps warming up from startval up to maxval for ceil(percent_start * nsteps), then back to endval (see Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates).