neon.optimizers.optimizer.Schedule

class neon.optimizers.optimizer.Schedule(step_config=None, change=1.0)[source]

Bases: neon.NervanaObject

Learning rate schedule.

By default implements a constant learning rate:

# Constant learning rate of 0.01 across training epochs
optimizer = GradientDescentMomentum(0.01, 0.9, schedule = Schedule())

Otherwise, the schedule multiplies the learning rate by change at every element in step_config. For example,

schedule = Schedule(step_config=[2, 6], change=0.5)
optimizer = GradientDescentMomentum(1.0, 0.9, schedule = Schedule())

will yield a learning rate schedule of:

Epoch LR
0 1.0
1 1.0
2 0.5
3 0.5
4 0.5
5 0.5
6 0.25
7 0.25
8 0.25
9 0.25
__init__(step_config=None, change=1.0)[source]

Class constructor.

Parameters:
  • step_config (list, optional) – Configure the step times (list of epoch indices). Defaults to None (constant).
  • change (int, optional) – The learning rate is multiplied by change ** steps, where steps is the number of steps in the step schedule that have passed.

Methods

__init__([step_config, change]) Class constructor.
gen_class(pdict)
get_description([skip]) Returns a dict that contains all necessary information needed to serialize this object.
get_learning_rate(learning_rate, epoch) Returns the current learning rate given the epoch and initial learning rate.
recursive_gen(pdict, key) helper method to check whether the definition
be = None
classnm

Returns the class name.

gen_class(pdict)
get_description(skip=[], **kwargs)

Returns a dict that contains all necessary information needed to serialize this object.

Parameters:skip (list) – Objects to omit from the dictionary.
Returns:Dictionary format for object information.
Return type:(dict)
get_learning_rate(learning_rate, epoch)[source]

Returns the current learning rate given the epoch and initial learning rate.

Parameters:
  • learning_rate (float) – Initial learning rate
  • epoch (int) – Current epoch, used to calculate the adjusted learning rate
Returns:

The adjusted learning rate

Return type:

(float)

modulenm

Returns the full module path.

recursive_gen(pdict, key)

helper method to check whether the definition dictionary is defining a NervanaObject child, if so it will instantiate that object and replace the dictionary element with an instance of that object