edu.umass.cs.mallet.base.fst
Class MultiSegmentationEvaluator

java.lang.Object
  extended byedu.umass.cs.mallet.base.fst.TransducerEvaluator
      extended byedu.umass.cs.mallet.base.fst.MultiSegmentationEvaluator

public class MultiSegmentationEvaluator
extends TransducerEvaluator


Field Summary
 
Fields inherited from class edu.umass.cs.mallet.base.fst.TransducerEvaluator
alwaysEvaluateWhenFinished, checkpointFilePrefix, checkpointIterationsToSkip, checkpointTransducer, numIterationsToSkip, numIterationsToWait, printModelAtEnd, viterbiOutput, viterbiOutputEncoding, viterbiOutputFilePrefix, viterbiOutputIterationsToSkip, viterbiOutputIterationsToWait
 
Constructor Summary
MultiSegmentationEvaluator(java.lang.Object[] segmentStartTags, java.lang.Object[] segmentContinueTags)
           
MultiSegmentationEvaluator(java.lang.Object[] segmentStartTags, java.lang.Object[] segmentContinueTags, boolean showViterbi)
           
 
Method Summary
 void batchTest(InstanceList data, java.util.ArrayList predictedSequences, java.lang.String description, java.io.PrintStream viterbiOutputStream)
          Tests segmentation using an ArrayList of predicted Sequences instead of a Transducer.
 boolean evaluate(Transducer model, boolean finishedTraining, int iteration, boolean converged, double cost, InstanceList training, InstanceList validation, InstanceList testing)
          Evaluates a Tranducers on a given training, validation, and testing set.
 int numIncorrectSegments(Sequence trueOutput, Sequence predOutput)
          returns the number of incorrect segments in predOutput
 void test(Transducer model, InstanceList data, java.lang.String description, java.io.PrintStream viterbiOutputStream)
           
 
Methods inherited from class edu.umass.cs.mallet.base.fst.TransducerEvaluator
getCheckpointFilePrefix, getCheckpointIterationsToSkip, getCheckpointTransducer, getNumIterationsToSkip, getNumIterationsToWait, getViterbiOutput, getViterbiOutputEncoding, getViterbiOutputFilePrefix, getViterbiOutputIterationsToSkip, getViterbiOutputIterationsToWait, isAlwaysEvaluateWhenFinished, isPrintModelAtEnd, setAlwaysEvaluateWhenFinished, setCheckpointFilePrefix, setCheckpointIterationsToSkip, setCheckpointTransducer, setNumIterationsToSkip, setNumIterationsToWait, setPrintModelAtEnd, setViterbiOutput, setViterbiOutputEncoding, setViterbiOutputFilePrefix, setViterbiOutputIterationsToSkip, setViterbiOutputIterationsToWait, shouldDoEvaluate
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

MultiSegmentationEvaluator

public MultiSegmentationEvaluator(java.lang.Object[] segmentStartTags,
                                  java.lang.Object[] segmentContinueTags,
                                  boolean showViterbi)

MultiSegmentationEvaluator

public MultiSegmentationEvaluator(java.lang.Object[] segmentStartTags,
                                  java.lang.Object[] segmentContinueTags)
Method Detail

evaluate

public boolean evaluate(Transducer model,
                        boolean finishedTraining,
                        int iteration,
                        boolean converged,
                        double cost,
                        InstanceList training,
                        InstanceList validation,
                        InstanceList testing)
Description copied from class: TransducerEvaluator
Evaluates a Tranducers on a given training, validation, and testing set. .

The default implementation calls the evaluator's test on the training, validation, and testing sets.

Overrides:
evaluate in class TransducerEvaluator
Parameters:
model - The transducer to evaluate.
finishedTraining - Whether training has finished. Evaluators can be set to always evaluate when training completes.
iteration - The current iteration
converged - Whether training has converged. Most evaluators ignore this.
cost - The current cost during training. Some evaluators may include this in the output.
training - The training set. If null, the argument is ignored.
validation - The validation set. If null, the argument is ignored.
testing - The testing set. If null, the argument is ignored.
Returns:
Whether training should continue.

test

public void test(Transducer model,
                 InstanceList data,
                 java.lang.String description,
                 java.io.PrintStream viterbiOutputStream)
Specified by:
test in class TransducerEvaluator

numIncorrectSegments

public int numIncorrectSegments(Sequence trueOutput,
                                Sequence predOutput)
returns the number of incorrect segments in predOutput

Parameters:
trueOutput - truth
predOutput - predicted
Returns:
number of incorrect segments

batchTest

public void batchTest(InstanceList data,
                      java.util.ArrayList predictedSequences,
                      java.lang.String description,
                      java.io.PrintStream viterbiOutputStream)
Tests segmentation using an ArrayList of predicted Sequences instead of a Transducer. If predictedSequence is null, don't include in stats (useful for error analysis).

Parameters:
data - list of instances to be segmented
predictedSequences - predictions
description - description of trial
viterbiOutputStream - where to print the Viterbi paths