com.aliasi.tokenizer
Class LengthStopFilterTokenizer

java.lang.Object
  extended by com.aliasi.tokenizer.Tokenizer
      extended by com.aliasi.tokenizer.FilterTokenizer
          extended by com.aliasi.tokenizer.StopFilterTokenizer
              extended by com.aliasi.tokenizer.LengthStopFilterTokenizer
All Implemented Interfaces:
Iterable<String>

Deprecated. Use TokenLengthTokenizerFactory or ModifyTokenTokenizerFactory.modify(Tokenizer) instead.

@Deprecated
public class LengthStopFilterTokenizer
extends StopFilterTokenizer

A StopFilterTokenizer removes tokens that exceed a specified length.

Since:
LingPipe3.6
Version:
3.8
Author:
Bob Carpenter

Field Summary
 
Fields inherited from class com.aliasi.tokenizer.FilterTokenizer
mTokenizer
 
Constructor Summary
LengthStopFilterTokenizer(Tokenizer tokenizer, int maxTokenLength)
          Deprecated. Use ModifyTokenTokenizerFactory.modify(Tokenizer) instead.
 
Method Summary
 int maxTokenLength()
          Deprecated. Returns the maximum token length for this filter.
 boolean stop(String token)
          Deprecated. Returns true if the specified token exceeds the maximum length.
 String toString()
          Deprecated. Returns a string representation of this tokenizer including the class and maximum token length.
 
Methods inherited from class com.aliasi.tokenizer.StopFilterTokenizer
nextToken
 
Methods inherited from class com.aliasi.tokenizer.FilterTokenizer
baseTokenizer, lastTokenEndPosition, lastTokenStartPosition, nextWhitespace, setTokenizer
 
Methods inherited from class com.aliasi.tokenizer.Tokenizer
iterator, tokenize, tokenize
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Constructor Detail

LengthStopFilterTokenizer

@Deprecated
public LengthStopFilterTokenizer(Tokenizer tokenizer,
                                            int maxTokenLength)
Deprecated. Use ModifyTokenTokenizerFactory.modify(Tokenizer) instead.

Construct a length filtering tokenizer with the specified maximum length.

Parameters:
maxTokenLength - Maximum token length that is accepted by the filter.
Method Detail

maxTokenLength

public int maxTokenLength()
Deprecated. 
Returns the maximum token length for this filter.

Returns:
The maximum token length for this filter.

stop

public boolean stop(String token)
Deprecated. 
Returns true if the specified token exceeds the maximum length.

Specified by:
stop in class StopFilterTokenizer
Parameters:
token - Token to filter.
Returns:
true if the specified token exceeds the maximum length.

toString

public String toString()
Deprecated. 
Returns a string representation of this tokenizer including the class and maximum token length.

Overrides:
toString in class FilterTokenizer
Returns:
A string representation of this tokenizer.