com.aliasi.corpus
Class XValidatingObjectCorpus<E>

java.lang.Object
  com.aliasi.corpus.Corpus<ObjectHandler<E>>
      com.aliasi.corpus.XValidatingObjectCorpus<E>

Type Parameters:

E - the type of objects handled.

All Implemented Interfaces:

Handler, ObjectHandler<E>

public class XValidatingObjectCorpus<E>
extends Corpus<ObjectHandler<E>>
implements ObjectHandler<E>

An XValidatingObjectCorpus holds a list of items which it uses to provide training and testing items using cross-validation.

Handler Implementation

The method handle(Object) is used to add items to the corpus. The items will be stored in the order in which they are received (though they may be permuted later).

When used as a handler, this class simply collects the items and stores them in a list. This allows an instance of this class to be used like any other object handler.

Cross Validation

Cross-validation divides a corpus up into roughly equal sized parts, called folds, assigning one of the parts as the test section and the other parts as training sections. A typical number of folds is 10, with 90% of the data being used for training and 10% for testing. The number of folds is set in the constructor.

Initially, the fold will be set to 0, but the fold may be reset later using setFold(int). Iterating between 0 and the number of folds minus 1 will work through all folds. The method size() returns the size of the corpus and fold() is the current fold.

For cases where numFolds() is greater than zero, the start and end of a fold are defined by:

 start(fold) = (int) (size() * fold() / (double) numFolds())
 
 end(fold) = start(fold+1)

If numFolds() is 0, the start and end for the fold are 0, so that visiting the training part of the corpus visits the entire corpus.

Permuting the Corpus

The randomization method permuteCorpus(Random) randomizes the list of items. This can be useful for removing local dependencies.

Use Without Cross Validation

No matter how the folds are set, using Corpus.visitCorpus(Handler) will run the specified handler over all of the data collected in this corpus.

If the number of folds is set to 1, then Corpus.visitTest(Handler) visits the entire corpus.

If the number of folds is set to 0, Corpus.visitTrain(Handler) visits the entire corpus.

Thead Safety

This class must be used with external read/write synchronization. The write operations include the constructor, set-fold, set number of folds, permute corpus, and handle methods. The read operations include the visit num instances and fold reporting methods.

Since:

LingPipe3.9

Version:

3.9

Author:

Bob Carpenter

Constructor Summary

XValidatingObjectCorpus(int numFolds)
Construct a cross-validating corpus with the specified number of folds.

Method Summary

int	fold() Returns the current fold.
void	handle(E e) Add the specified item to the end of the corpus.
int	numFolds() Return the number of folds for this cross-validating corpus.
void	permuteCorpus(Random random) Randomly permutes the corpus using the specified randomizer.
void	setFold(int fold) Set the current fold to the specified value.
void	setNumFolds(int numFolds) Sets the number of folds to the specified value.
int	size() Return the number of items in this corpus.
void	visitCorpus(ObjectHandler<E> handler) Visit the entire corpus, sending all extracted events to the specified handler.
void	visitCorpus(ObjectHandler<E> trainHandler, ObjectHandler<E> testHandler) Visit the entire corpus, first sending training events to the specified training handler and then sending testing events to the test handler.
void	visitTest(ObjectHandler<E> handler) Send all of the test items to the specified handler.
void	visitTrain(ObjectHandler<E> handler) Send all of the training items to the specified handler.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

XValidatingObjectCorpus

public XValidatingObjectCorpus(int numFolds)

Construct a cross-validating corpus with the specified number of folds. The initial fold is set to 0.

See the class documentation above for information on how the number of folds is used.

Parameters:

numFolds - Number of folds in the corpus.

Throws:

IllegalArgumentException - If the number of folds is negative.

Method Detail

numFolds

public int numFolds()

Return the number of folds for this cross-validating corpus.

Returns:

Current number of folds.

setNumFolds

public void setNumFolds(int numFolds)

Sets the number of folds to the specified value.

See the class documentation above for information on how the number of folds is used.

Parameters:

numFolds - Number of folds.

Throws:

IllegalArgumentException - If the number of folds is negative.

fold

public int fold()

Returns the current fold.

Returns:

The current fold.

permuteCorpus

public void permuteCorpus(Random random)

Randomly permutes the corpus using the specified randomizer.

Parameters:

random - Randomizer to use for permutation.

setFold

public void setFold(int fold)

Set the current fold to the specified value.

Warning: If the number of folds is set to zero, this method will throw an exception.

Throws:

IllegalArgumentException - If the fold is not greater than or equal to 0 and less than the number of folds.

size

public int size()

Return the number of items in this corpus.

Returns:

Number of items.

handle

public void handle(E e)

Add the specified item to the end of the corpus.

Specified by:

handle in interface ObjectHandler<E>

Parameters:

e - Item to add to corpus.

visitTrain

public void visitTrain(ObjectHandler<E> handler)

Send all of the training items to the specified handler. See the class documentation above for a specification of which items are visited based on the value of the number of folds and the current fold.

Overrides:

visitTrain in class Corpus<ObjectHandler<E>>

Parameters:

handler - Handler receiving training items.

visitTest

public void visitTest(ObjectHandler<E> handler)

Send all of the test items to the specified handler. See the class documentation above for a specification of which items are visited based on the value of the number of folds and the current fold.

Overrides:

visitTest in class Corpus<ObjectHandler<E>>

Parameters:

handler - Handler receiving training items.

visitCorpus

public void visitCorpus(ObjectHandler<E> handler)

Description copied from class: Corpus

Visit the entire corpus, sending all extracted events to the specified handler.

This is just a convenience method that is defined by:

 visitCorpus(handler,handler);
 

Overrides:

visitCorpus in class Corpus<ObjectHandler<E>>

Parameters:

handler - Handler for events extracted from the corpus.

visitCorpus

public void visitCorpus(ObjectHandler<E> trainHandler,
                        ObjectHandler<E> testHandler)

Description copied from class: Corpus

Visit the entire corpus, first sending training events to the specified training handler and then sending testing events to the test handler.

This is just a convenience method that is defined by:

 visitTrain(trainHandler);
 visitTest(testHandler);
 

Overrides:

visitCorpus in class Corpus<ObjectHandler<E>>

Parameters:

trainHandler - Handler for training events from the corpus.

testHandler - Handler for testing events from the corpus.