

PREV CLASS NEXT CLASS  FRAMES NO FRAMES  
SUMMARY: NESTED  FIELD  CONSTR  METHOD  DETAIL: FIELD  CONSTR  METHOD 
java.lang.Object com.aliasi.stats.AnnealingSchedule
public abstract class AnnealingSchedule
An AnnealingSchedule
instance implements a method to
return the learning rate for a specified epoch. It also has a
method to receive callback reports on the resulting error rate for
an epoch so that implementations may adapt the learning rate to
received error.
This class provides an abstract base class implementation along with static factory methods to create the two most popular annealing schedules, constant, exponential decay and inverse scaling.
The contant learning rate schedule always returns the same learning rate, which is fixed at construction time.
The exponential decay annealing schedule sets the learning rate as follows based on a specfieid exponential base:
Under this schedule, the learning rate undergoes exponential decay starting at the initial learning rate and decaying exponentially at a rate determined by the base of the exponent.learningRate(epoch) = initialLearningRate * java.lang.Math.pow(base,epoch)
The exponential learning rate can find solutions quickly in fairly wellbehaved spaces, but may stop short of the minimum error solution due to too much decay in later epochs.
The inverse learning rate scaling sets the learning rate as:
The inverse scaling annealing schedule lowers the rate more quickly than the exponential rates initially and then more slowly for later epochs.learningRate(epoch) = initialLearningRate / (1 + epoch/annealingRate)
This is a popular learning rate because it is guaranteed to converge in the limit. It can be slower to converge once it gets near a solution than exponential decay.
receivedError()
method is a
boolean flag indicating whether to accept the updates to the
underlying vectors or not. This allows samplingbased annealing
schedules to be implemented that evaluate several learning
rates and accept just the one with the most error reduction.
The inverse scale metric is popular because it is theoretically guaranteed to converge in the limit.
An annealing rate will converge in the limit within arbitrary precision of a solution if the learning rate satisfies:
and:Σ_{epoch} learningRate(epoch) = ∞
Σ_{epoch} learningRate(epoch)^{2} < ∞
Constructor Summary  

AnnealingSchedule()
Donothing constructor to be used by concrete implementations. 
Method Summary  

static AnnealingSchedule 
constant(double learningRate)
Return the annealing schedule for the specified constant learning rate. 
static AnnealingSchedule 
exponential(double initialLearningRate,
double base)
Return the exponential annealing schedule with the specified initial learning rate and exponent. 
static AnnealingSchedule 
inverse(double initialLearningRate,
double annealingRate)
Return the inverse annealing schedule with the specified initial learning rate and annealing rate. 
abstract double 
learningRate(int epoch)
Return the learning rate for the specified epoch. 
boolean 
receivedError(int epoch,
double rate,
double error)
Receive a report from an optimizer about the effect of the specified learning rate in the specified epoch and return true if the update producing the error should
be accepted or rejected. 
Methods inherited from class java.lang.Object 

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait 
Constructor Detail 

public AnnealingSchedule()
Method Detail 

public abstract double learningRate(int epoch)
epoch
 Epoch being evaluated.
public boolean receivedError(int epoch, double rate, double error)
true
if the update producing the error should
be accepted or rejected.
This abstract class's implementation of this method is to do nothing. Concrete subclasses which adapt learning rates based on empirical error reports from the optimizer must override this method.
epoch
 Training epoch.rate
 Training rate.error
 Training error.public static AnnealingSchedule inverse(double initialLearningRate, double annealingRate)
initialLearningRate
 Initial learning rate for epoch zero.annealingRate
 Rate at which initial learning rate anneals.
IllegalArgumentException
 If the initial learning rate or
the annealing rates are not finite and positive.public static AnnealingSchedule exponential(double initialLearningRate, double base)
initialLearningRate
 Initial learning rate for epoch 0.base
 Base of the exponential decay.
IllegalArgumentException
 If the initial learning rate is
not finite and positive, or if the exponent is not between 0.0 (exclusive)
and 1.0 (inclusive).public static AnnealingSchedule constant(double learningRate)
learningRate
 The constant rate returned by this
annealing schedule.
IllegalArgumentException
 If the learning rate is not
finite and positive.


PREV CLASS NEXT CLASS  FRAMES NO FRAMES  
SUMMARY: NESTED  FIELD  CONSTR  METHOD  DETAIL: FIELD  CONSTR  METHOD 