com.aliasi.classify
Class ConditionalClassifierEvaluator<E>

java.lang.Object
  extended by com.aliasi.classify.BaseClassifierEvaluator<E>
      extended by com.aliasi.classify.RankedClassifierEvaluator<E>
          extended by com.aliasi.classify.ScoredClassifierEvaluator<E>
              extended by com.aliasi.classify.ConditionalClassifierEvaluator<E>
Type Parameters:
E - The type of objects being classified by the evaluated classifier.
All Implemented Interfaces:
Handler, ObjectHandler<Classified<E>>
Direct Known Subclasses:
JointClassifierEvaluator

public class ConditionalClassifierEvaluator<E>
extends ScoredClassifierEvaluator<E>

A ConditionalClassifierEvaluator provides an evaluation harness for conditional probability-based n-best classifiers. It extends the scored classifier evaluator with conditional probability specific evaluation metrics.

Thread Safety

This class must be read-write synchronized externally for use in multiple threads.

Since:
LingPipe3.9.1
Version:
3.9.1
Author:
Bob Carpenter

Constructor Summary
ConditionalClassifierEvaluator(ConditionalClassifier<E> classifier, String[] categories, boolean storeInputs)
          Construct a scored classifier evaluator with the specified classifier, categories and flag indicating whether or not to store inputs.
 
Method Summary
 double averageConditionalProbability(String refCategory, String responseCategory)
          Returns the average conditional probability of the specified response category for test cases with the specified reference category.
 double averageConditionalProbabilityReference()
          Returns the average over all test cases of the conditional probability of the response that matches the reference category.
 ConditionalClassifier<E> classifier()
          Return the classifier being evaluated.
 ScoredPrecisionRecallEvaluation conditionalOneVersusAll(String refCategory)
          Returns a scored precision-recall evaluation of the classifcation of the specified reference category versus all other categories using the conditional probability scores.
 void handle(Classified<E> classified)
          Add the specified classified object to this evaluator.
 void setClassifier(ConditionalClassifier<E> classifier)
          Set the classifier being evaluated to the specified value.
 
Methods inherited from class com.aliasi.classify.ScoredClassifierEvaluator
averageScore, averageScoreReference, scoredOneVersusAll, setClassifier
 
Methods inherited from class com.aliasi.classify.RankedClassifierEvaluator
averageRank, averageRankReference, meanReciprocalRank, rankCount, setClassifier
 
Methods inherited from class com.aliasi.classify.BaseClassifierEvaluator
addClassification, categories, confusionMatrix, falseNegatives, falsePositives, numCases, numCategories, oneVersusAll, setClassifier, toString, trueNegatives, truePositives
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Constructor Detail

ConditionalClassifierEvaluator

public ConditionalClassifierEvaluator(ConditionalClassifier<E> classifier,
                                      String[] categories,
                                      boolean storeInputs)
Construct a scored classifier evaluator with the specified classifier, categories and flag indicating whether or not to store inputs.

Parameters:
classifier - Classifier to evaluate.
categories - Complete list of categories.
storeInputs - Set to true to store input objects.
Method Detail

setClassifier

public void setClassifier(ConditionalClassifier<E> classifier)
Set the classifier being evaluated to the specified value. This method is useful to evaluate multiple classifiers with the same evaluator, for instance for use in cross-validation.

Parameters:
classifier - New classifier for this evaluation.
Throws:
IllegalArgumentException - If called from an evaluator with a runtime type other than ConditionalClassifierEvaluator.

classifier

public ConditionalClassifier<E> classifier()
Return the classifier being evaluated.

Overrides:
classifier in class ScoredClassifierEvaluator<E>
Returns:
The classifier for this evaluator.

handle

public void handle(Classified<E> classified)
Description copied from class: BaseClassifierEvaluator
Add the specified classified object to this evaluator.

Specified by:
handle in interface ObjectHandler<Classified<E>>
Overrides:
handle in class ScoredClassifierEvaluator<E>
Parameters:
classified - Classified object to add to evaluation.

averageConditionalProbability

public double averageConditionalProbability(String refCategory,
                                            String responseCategory)
Returns the average conditional probability of the specified response category for test cases with the specified reference category. If there are no cases matching the reference category, the result is Double.NaN. If the conditional classifiers' results are properly normalized, the sum of the averages over all categories will be 1.0.

Better classifiers return high values when the reference and response categories are the same and lower values when they are different. The log value would be extremely volatile given the extremely low and high conditional estimates of the language model classifiers.

Parameters:
refCategory - Reference category.
responseCategory - Response category.
Returns:
Average conditional probability of response category in cases for specified reference category.
Throws:
IllegalArgumentException - If the either category is unknown.

averageConditionalProbabilityReference

public double averageConditionalProbabilityReference()
Returns the average over all test cases of the conditional probability of the response that matches the reference category. Better classifiers return higher values for this average.

As a normalized value, the average conditional probability always has a sensible interpretation across training instances.

Returns:
The average conditional probability of the reference category in the response.

conditionalOneVersusAll

public ScoredPrecisionRecallEvaluation conditionalOneVersusAll(String refCategory)
Returns a scored precision-recall evaluation of the classifcation of the specified reference category versus all other categories using the conditional probability scores. This method may only be called for evaluations that have scores.

Parameters:
refCategory - Reference category.
Returns:
The conditional one-versus-all precision-recall evaluatuion.
Throws:
IllegalArgumentException - If the specified category is unknown.