com.g11ntoolkit.token
Class Token

java.lang.Object
  |
  +--com.g11ntoolkit.token.Token

public class Token

extends java.lang.Object

A token is a unique id for an extracted string.

The token is put in place of the string so we know where the string came from. It is used later to put the string back where it belongs. A token is made up of a prefix and a number. This generally makes each string in a set of files unique. This is good because sets of strings may be combined to make translation easier and more efficient.

The prefix usually is some string which will help in identify the source of the string (it could be the file name which contains the string). This can sometimes provide context for translation. It can also make finding the string easier if it is needed after the extract is done.

The number is just a sequence number. The number can be reset when the prefix is reset. It must be unique within a prefix so that each string is uniquely identified.

A token may be decorated with some unique string in front of it and behind it. A decorated token can make it easier to find the tokens in a tokenized file. The default decoration for tokens is stored in the vars.properties file. You can change it by using the appropriate methods of this class, but, it is best to change it in the properties file. Some higher level tools will rely on the token decoration to identify tokens in the tokenized file. If you change it using the methods on this class then it may make the tokenized file difficult to use. It is a good idea to decide on the decoration, set it in the properties file, and leave it alone.

Version:

2005/06/20

Author:

Bill Rich, Wilandra Consulting LLC
Copyright © 2001-2005, Wilandra Consulting LLC. All rights reserved.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

See License Agreement.

Field Summary

private java.lang.String	decoration The decoration is some unique string which will be placed in front of and behind the token.
private static java.util.logging.Logger	log The log used for all messages from this class.
private static java.util.ResourceBundle	mrb Messages Resource Bundle.
private int	tokenNumber The token number is initialized when the class is instantiated and incremented each time a token is requested.
private java.lang.String	tokenPrefix The token prefix is initialized when the class is instantiated or by the set method.
private int	tokenSequenceNumber A token number to use for the sequence.
private static java.util.ResourceBundle	vrb Variables Resource Bundle.
private static java.util.ResourceBundle	xrb Constants, messages, and variables used by the tools and classes for XML processing.

Constructor Summary

Token()
Establishes an instance with no initial token prefix information.

Token(java.lang.String possibleToken)
Establishes an instance based on the specified string.

Token(java.lang.String prefix, int start)
Establishes an instance with a specific token prefix and a specific starting number.

Method Summary

int	compareTo(java.lang.Object anotherToken) Shows how the specified Object compares to this Token.
int	compareTo(Token anotherToken) Shows how the specified Token compares to this Token.
void	debugDump() Dumps the contents of the object to the console.
boolean	equals(java.lang.Object anotherToken) Tests if the specified Object is equal to this Token.
boolean	equals(Token anotherToken) Tests if the specified Token is equal to this Token.
java.lang.String	getDecoratedToken() Return the current token in the sequence with some decoration in front of and behind it.
java.lang.String	getDecoration() Returns the decoration used on the tokens.
int	getNumber() Returns the current token number.
java.lang.String	getPrefix() Return the token prefix value.
java.lang.String	getToken() Return the current token in the sequence.
int	hashCode() Returns a hash code for this token.
java.lang.String	nextDecoratedToken() Return the next token in the sequence with some decoration in front of and behind it.
java.lang.String	nextToken() Return the next token in the sequence.
void	setDecoration(java.lang.String deco) Sets the decoration to some specified string.
void	setParts(java.lang.String wt) Sets up the two individual parts of the token.
void	setPrefix(java.lang.String prefix) Sets the prefix to some specified string.
void	setStartNumber(int start) Sets the token starting number.
void	setToken(java.lang.String possibleToken) Sets the current token values.
void	setToken(java.lang.String possibleToken, java.lang.String deco) Sets the current token values.
java.lang.String	stripDeco(java.lang.String wt) Removes the token decorations from a string and returns what is left.
boolean	validate(java.lang.String wt) Validates that the specified string is a token.
void	writeXML(java.io.Writer out) Writes the contents of the object to the specified writer.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

private static java.util.logging.Logger log

The log used for all messages from this class.

vrb

private static java.util.ResourceBundle vrb

Variables Resource Bundle.

mrb

private static java.util.ResourceBundle mrb

Messages Resource Bundle.

xrb

private static java.util.ResourceBundle xrb

Constants, messages, and variables used by the tools and classes for XML processing.

tokenPrefix

private java.lang.String tokenPrefix

The token prefix is initialized when the class is instantiated or by the set method.

It is used to give the token some context and to make sure the token ids are unique.

tokenNumber

private int tokenNumber

The token number is initialized when the class is instantiated and incremented each time a token is requested.

decoration

private java.lang.String decoration

The decoration is some unique string which will be placed in front of and behind the token.

This will make the token easier to identify in a file. The default decoration is stored in the vars.properties file. The decoration may be set at any time.

tokenSequenceNumber

private int tokenSequenceNumber

A token number to use for the sequence.

Constructor Detail

Token

public Token()

Establishes an instance with no initial token prefix information.

You must use the set method to set the token prefix before you ask for the first token or all you will get is a number preceded by a dot. This is valid if that is what you want. The token number is started at 0.

Token

public Token(java.lang.String possibleToken)
      throws MalformedToken

Establishes an instance based on the specified string.

The string is assumed to be a well formed token with no decoration or with the default (@@) decoration in front of it and behind it.

Parameters:

possibleToken - a String that contains the proposed token value

Throws:

MalformedToken - if the proposed string is not a valid token

Token

public Token(java.lang.String prefix,
             int start)

Establishes an instance with a specific token prefix and a specific starting number.

Parameters:

prefix - a String containing the desired prefix

start - int containing the desired token number

Method Detail

setPrefix

public void setPrefix(java.lang.String prefix)

Sets the prefix to some specified string.

Can be used to override the prefix set in the constructor.

Parameters:

prefix - a String containing the desired prefix

See Also:

getPrefix()

setStartNumber

public void setStartNumber(int start)

Sets the token starting number.

Can be used to change the next number to be used at any time. The number set as the starting token number will be the number used by the nextToken method on the next call to it.

Parameters:

start - an int containing the desired token number

getDecoration

public java.lang.String getDecoration()

Returns the decoration used on the tokens.

Returns:

a String containing the token decoration

See Also:

setDecoration(java.lang.String)

setDecoration

public void setDecoration(java.lang.String deco)

Sets the decoration to some specified string.

Can be used to override what is set in the constructor.

Warning: use this method with caution. If you change the token decoration from what is shown in the properties file, FileDetok may not be able to detokenize the file.

Parameters:

deco - a String containing the decoration string to use

See Also:

getDecoration()

getPrefix

public java.lang.String getPrefix()

Return the token prefix value.

Returns:

a String containing the prefix value

See Also:

setPrefix(java.lang.String)

nextToken

public java.lang.String nextToken()

Return the next token in the sequence.

It increments the current token number then returns the value.

Returns:

a String containing the next token

getToken

public java.lang.String getToken()

Return the current token in the sequence.

This does not increment the token number.

Returns:

a String containing the next token

See Also:

setToken(java.lang.String)

setToken

public void setToken(java.lang.String possibleToken)
              throws MalformedToken

Sets the current token values.

The specified possible token is assumed to be a well formed token with no decoration or with the default (found in the vars.properties file) decoration in front of it and behind it.

Parameters:

possibleToken - a String that contains the proposed token value

Throws:

MalformedToken - if the proposed string is not a valid token

See Also:

getToken(), nextToken()

setToken

public void setToken(java.lang.String possibleToken,
                     java.lang.String deco)
              throws MalformedToken

Sets the current token values.

The specified possible token is assumed to be a well formed token with the specified decoration in front of it and behind it.

Warning: use this method with caution. If you change the token decoration from what is shown in the properties file, FileDetok may not be able to detokenize the file.

Parameters:

possibleToken - a String that contains the proposed token value

deco - a String containing the decoration string to use

Throws:

MalformedToken - if the proposed string is not a valid token

See Also:

getToken(), nextToken(), getDecoration()

nextDecoratedToken

public java.lang.String nextDecoratedToken()

Return the next token in the sequence with some decoration in front of and behind it.

The decoration is to allow you to identify a token in a file more easily. The decoration is the one specified in the global decoration variable in front of an behind the token. It increments the current token number then returns the value.

Returns:

a String containing the next decorated token

See Also:

setToken(java.lang.String)

getDecoratedToken

public java.lang.String getDecoratedToken()

Return the current token in the sequence with some decoration in front of and behind it.

The decoration is to allow you to identify a token in a file more easily. The decoration is the one specified in the global decoration variable in front of an behind the token. This does not increment the token number.

Returns:

a String containing the next decorated token

See Also:

setToken(java.lang.String)

validate

public boolean validate(java.lang.String wt)
                 throws MalformedToken

Validates that the specified string is a token.

The validation is looking for a well formed token which may or may not have the currently set decoration in front of it or behind it. Any decoration found will be removed before the validation is continued. The validation is to see if the string has two parts, a prefix and a token number separated by a dot. If so, it is valid and we return a true; if not, we return a false.

Parameters:

wt - a String containing the proposed well formed token

Returns:

a boolean indicating if the specified string contains a well formed token (true) or not (false)

Throws:

MalformedToken - if the proposed string is not a valid token

See Also:

stripDeco(java.lang.String), setParts(java.lang.String)

compareTo

public int compareTo(Token anotherToken)

Shows how the specified Token compares to this Token.

Parameters:

anotherToken - a Token object

Returns:

an int set to -1 if this Token is less than the specified Token, set to 0 if this Token is equal to the specified Token, or set to 1 if this Token is greater than the specified Token

compareTo

public int compareTo(java.lang.Object anotherToken)

Shows how the specified Object compares to this Token.

Parameters:

anotherToken - an Object specifying the object to compare this instance to

Returns:

an int set to -1 if this Token is less than the specified Token, set to 0 if this Token is equal to the specified Token, set to 1 if this Token is greater than the specified Token, or -2 if the objects can not be compared

equals

public boolean equals(Token anotherToken)

Tests if the specified Token is equal to this Token.

Equality means that the token prefixes and token numbers are equal. Decorations are ignored.

Parameters:

anotherToken - a Token object specifying the object to compare this instance to

Returns:

a boolean set to true if this Token and the specified Token are equal or false if they are not equal.

equals

public boolean equals(java.lang.Object anotherToken)

Tests if the specified Object is equal to this Token.

Equality means that the token prefixes and token numbers are equal. Decorations are ignored.

Overrides:

equals in class java.lang.Object

Parameters:

anotherToken - an Object specifying the object to compare this instance to

Returns:

boolean set to true if this Token and the specified Object are equal or false if they are not equal.

See Also:

equals(com.g11ntoolkit.token.Token)

getNumber

public int getNumber()

Returns the current token number.

Returns:

an int containing the current token number

hashCode

public int hashCode()

Returns a hash code for this token.

The hash code for a token is defined as the sum of the hash code for the token prefix and the token number.

Overrides:

hashCode in class java.lang.Object

Returns:

an int containing the hash code for this Token.

stripDeco

public java.lang.String stripDeco(java.lang.String wt)

Removes the token decorations from a string and returns what is left.

The decoration must be both at the beginning of and at the end of the possible token before it is assumed to be decoration.

Parameters:

wt - a String containing the token

Returns:

a String containing the token with the decoration removed from the beginning and end of string

setParts

public void setParts(java.lang.String wt)
              throws MalformedToken

Sets up the two individual parts of the token.

Sets the global variables tokenPrefix and tokenNumber. Also checks to see that the number is in fact all digits.

Parameters:

wt - a String containing the proposed token

Throws:

MalformedToken - if the proposed string is not a valid token

writeXML

public void writeXML(java.io.Writer out)
              throws java.io.IOException

Writes the contents of the object to the specified writer.

Parameters:

out - a Writer to use for the output

Throws:

java.io.IOException - for any errors detected by the specified Writer

debugDump

public void debugDump()

Dumps the contents of the object to the console.