com.g11ntoolkit.tokfile
Class TokFile

java.lang.Object
  |
  +--com.g11ntoolkit.tokfile.TokFile

public class TokFile

extends java.lang.Object

Create and maintain a TokFile with its associated buffer and hashtable.

The TokFile is the file that contains the original input file where the strings have been replaced by a token id. It is an intermediate file used by the L10N Leveraging tools.

The input file can be in either a plain text form or in an XML form. Which form is determined by which load method is called.

Version:

2005/06/20

Author:

Bill Rich, Wilandra Consulting LLC
Copyright © 2001-2005, Wilandra Consulting LLC. All rights reserved.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

See License Agreement.

Field Summary

private java.util.Hashtable	context Contains a table of context information.
private int	currentRecordStart Beginning position of the current record in the buffer.
private java.lang.String[]	dqStyle Contains the string double-quote style.
private java.lang.String[]	fileType Contains the file type we are given.
private java.util.ArrayList	lineStart Contains the starting positions of the lines from the source file in the buffer.
private static java.util.logging.Logger	log The log used for all messages from this class.
private boolean	moreRecords Indicates that there are more records available in the buffer.
private static java.util.ResourceBundle	mrb Message Resource Bundle.
private java.util.Hashtable	revContext Contains a table of the Tokens associated with their keys.
private int	sourceStart Contains the starting position in the file image.
private java.lang.StringBuffer	theBuffer Collection all the data from the specified input file.
private java.lang.String	theImage Contains the image of the source file.
private static java.util.ResourceBundle	vrb Variables Resource Bundle.
private static java.util.ResourceBundle	xrb Constants, messages, and variables used by the tools and classes for XML processing.

Constructor Summary

TokFile()
Creates an empty buffer and an empty hashtable.

TokFile(java.io.InputStreamReader isReader)
Creates a buffer that contains the records read from the specified input stream reader.

Method Summary

void	append(char aChar) Adds the specified character to the end of the buffer.
void	append(java.lang.String aString) Adds the specified string to the end of the buffer.
void	closeFileImageToBuffer() Place the last part of the image file into the buffer.
private static java.lang.String	convertToFileURL(java.lang.String filename) Returns the filename converted to a file URL.
void	copyOf(TokFile tf) Makes a copy of the specified TokFile object contents in this object.
void	debugDump() Dumps the contents of the object to the console for debugging.
void	debugDump(java.lang.String what) Dumps the contents of the object to the console for debugging.
int	findOffset(int row, int col) Returns the offset into the image area based on the row and col numbers provided.
java.lang.StringBuffer	getBuffer() Returns the buffer.
TokContext	getContext(Token tok) Returns the context for a specified token in the file.
java.util.Hashtable	getContextTable() Returns the context hashtable.
java.lang.String	getDQStyle() Returns the double-quote style for this file.
java.lang.String	getFirstRecord() Returns the first record in the buffer.
java.lang.String	getNextRecord() Returns the next record in the buffer.
java.lang.String	getRecord(int offset) Returns the record at the specified offset in the buffer.
java.lang.String	getRecord(java.lang.String aString) Returns the record for the specified string.
java.lang.String	getRecord(Token aToken) Returns the record for the specified token.
Token	getRevContext(TokContext ctx) Returns the token for a specified context in the file.
java.util.Hashtable	getRevContextTable() Returns the reverse context hashtable.
java.lang.String	getType() Returns the type of file we are dealing with.
boolean	hasMoreRecords() Returns true if there are more records in the buffer.
int	length() Returns the length of this buffer.
void	load(java.io.InputStreamReader isReader) Loads the buffer with the records read from the specified input stream reader.
void	load(java.io.InputStreamReader isReader, boolean xmlfile, java.lang.String xmlFileName) Loads the buffer with the records read from the specified input stream reader.
void	loadFileImage(java.lang.String inFileName) Load the source file image.
java.lang.String[]	parse(java.lang.String rec) Parses the specified string into a context string and a token string.
void	replaceString(java.lang.String aString, Token aToken) Replaces the specified string with a token.
void	replaceStringWithToken(int strStartRow, int strStartCol, int strEndRow, int strEndCol, Token token) Replaces the string in the file image with a token in the buffer.
void	replaceToken(Token aToken, java.lang.String aString) Replaces the specified token with a string.
void	setBuffer(java.lang.StringBuffer aBuffer) Set the buffer contents.
private void	setContext() Sets the context for all the tokens in the file based on the context table found in the file.
void	setContext(TokContext ctx, Token tok) Sets the context for the specified token in the file.
void	setContextTable(java.util.Hashtable ch) Set the context hashtable.
private void	setDQStyle() Determines the double-quote style in the file and remembers it.
void	setDQStyle(java.lang.String dqs) Sets the double-quote style for this file.
private void	setRevContext(Token tok, TokContext ctx) Sets the token for each context in the file.
void	setRevContextTable(java.util.Hashtable rch) Set the reverse context hashtable.
private void	setType() Determines the type of file we are dealing with and remembers it.
void	setType(java.lang.String ft) Determines the type of file we are dealing with and remembers it.
void	write(java.io.OutputStreamWriter osWriter) Writes the buffer out to the specified output stream writer and closes it.
void	writeXML(java.io.OutputStreamWriter osWriter) Writes the buffer out to the specified output stream writer in XML format.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

private static java.util.logging.Logger log

The log used for all messages from this class.

mrb

private static java.util.ResourceBundle mrb

Message Resource Bundle.

vrb

private static java.util.ResourceBundle vrb

Variables Resource Bundle.

xrb

private static java.util.ResourceBundle xrb

Constants, messages, and variables used by the tools and classes for XML processing.

theBuffer

private java.lang.StringBuffer theBuffer

Collection all the data from the specified input file.

On load, this is the raw data from the file except that all encoded characters have been decoded to their native codes. The buffer can be updated with various methods to replace the strings with tokens.

currentRecordStart

private int currentRecordStart

Beginning position of the current record in the buffer.

moreRecords

private boolean moreRecords

Indicates that there are more records available in the buffer.

fileType

private java.lang.String[] fileType

Contains the file type we are given.

This is important because we need to know what type of file it is in order to properly parse the file to set up the context hashtables.

dqStyle

private java.lang.String[] dqStyle

Contains the string double-quote style.

This indicates whether the string was wrapped in a pair of double-quotes or not.

context

private java.util.Hashtable context

Contains a table of context information.

This is a table of the keys in the file associated with their tokens.

revContext

private java.util.Hashtable revContext

Contains a table of the Tokens associated with their keys.

This is a table of the tokens in the file associated with their keys (i.e., a reverse context).

theImage

private java.lang.String theImage

Contains the image of the source file.

sourceStart

private int sourceStart

Contains the starting position in the file image.

lineStart

private java.util.ArrayList lineStart

Contains the starting positions of the lines from the source file in the buffer.

Constructor Detail

TokFile

public TokFile()

Creates an empty buffer and an empty hashtable.

This may be handy if the buffer is going to be used to create a new TokFile.

TokFile

public TokFile(java.io.InputStreamReader isReader)
        throws TokFileError

Creates a buffer that contains the records read from the specified input stream reader.

Parameters:

isReader - an InputStreamReader mapped to the input file

Throws:

TokFileError - for errors in processing the input file

Method Detail

load

public void load(java.io.InputStreamReader isReader)
          throws TokFileError

Loads the buffer with the records read from the specified input stream reader.

Passes the work on to the load(InputStreamReader, boolean, String) method indicating that the file is in a text form.

Parameters:

isReader - an InputStreamReader mapped to the input file

Throws:

TokFileError - for errors in processing the input file

See Also:

load(InputStreamReader, boolean, String)

load

public void load(java.io.InputStreamReader isReader,
                 boolean xmlfile,
                 java.lang.String xmlFileName)
          throws TokFileError

Loads the buffer with the records read from the specified input stream reader.

The file may be either a text file or an XML file.

The TokFile buffer will contain an exact copy of the data in the file.

Parameters:

isReader - an InputStreamReader mapped to the input file or null if xmlfile is true

xmlfile - a boolean specifying whether the file is in XML form (true) or in text form (false)

xmlFileName - a String specifying the name of the XML file or null if xmlfile is false

Throws:

TokFileError - for errors in processing the input file

See Also:

load(InputStreamReader)

setType

public void setType(java.lang.String ft)

Determines the type of file we are dealing with and remembers it.

Parameters:

ft - a String containing the file type

See Also:

getType()

setType

private void setType()
              throws TokFileError

Determines the type of file we are dealing with and remembers it.

We take care of setting it for you. This is not something we want to have set from outside the control of this class.

Throws:

TokFileError - for errors in processing the input file

See Also:

getType()

getType

public java.lang.String getType()

Returns the type of file we are dealing with.

Returns:

a String containing the file type description

See Also:

setType(java.lang.String)

setDQStyle

private void setDQStyle()
                 throws TokFileError

Determines the double-quote style in the file and remembers it.

Throws:

TokFileError - for errors in processing the input file

See Also:

getDQStyle()

setDQStyle

public void setDQStyle(java.lang.String dqs)

Sets the double-quote style for this file.

We take care of setting it for you. This is not something we want to have set from outside the control of this class.

Parameters:

dqs - a String containing the double-quote style for the file

See Also:

getDQStyle()

getDQStyle

public java.lang.String getDQStyle()

Returns the double-quote style for this file.

Returns:

a String containing the double-quote style for the file

See Also:

setDQStyle()

setContext

public void setContext(TokContext ctx,
                       Token tok)

Sets the context for the specified token in the file.

setContext() will also set the reverse context since it knows it already.

Parameters:

ctx - a TokContext containing the context for the specified token

tok - a Token containing the token for the specified context

See Also:

getContext(com.g11ntoolkit.token.Token)

getContext

public TokContext getContext(Token tok)

Returns the context for a specified token in the file.

Parameters:

tok - a Token containing the token for which the context is wanted

Returns:

a TokContext containing the context for the specified token

See Also:

setContext(com.g11ntoolkit.levblock.TokContext, com.g11ntoolkit.token.Token)

setContext

private void setContext()
                 throws TokFileError

Sets the context for all the tokens in the file based on the context table found in the file.

We take care of setting it for you. This is not something we want to have set from outside the control of this class.

Throws:

TokFileError - for errors in processing the input file

See Also:

getContext(com.g11ntoolkit.token.Token)

parse

public java.lang.String[] parse(java.lang.String rec)

Parses the specified string into a context string and a token string.

Parameters:

rec - a String containing the record

Returns:

a String array containing one string for the context and one for the token contained in the specified record

setRevContext

private void setRevContext(Token tok,
                           TokContext ctx)

Sets the token for each context in the file.

We take care of setting it for you. This is not something we want to have set from outside the control of this class.

Parameters:

tok - a Token containing the token

ctx - a TokContext containing the context

See Also:

getRevContext(com.g11ntoolkit.levblock.TokContext)

getRevContext

public Token getRevContext(TokContext ctx)

Returns the token for a specified context in the file.

Parameters:

ctx - a TokContext contains the context

Returns:

a Token containing the token for the specified context

See Also:

setRevContext(com.g11ntoolkit.token.Token, com.g11ntoolkit.levblock.TokContext)

write

public void write(java.io.OutputStreamWriter osWriter)
           throws TokFileError

Writes the buffer out to the specified output stream writer and closes it.

Parameters:

osWriter - an OutputStreamWriter mapped to the output file

Throws:

TokFileError - for errors in processing the input file

writeXML

public void writeXML(java.io.OutputStreamWriter osWriter)
              throws TokFileError,
                     gnu.regexp.REException,
                     java.io.IOException

Writes the buffer out to the specified output stream writer in XML format.

Parameters:

osWriter - an OutputStreamWriter mapped to the output file

Throws:

TokFileError - for errors in processing the buffer

gnu.regexp.REException - for errors in setting up the regular expression

java.io.IOException - for errors in writing the file

getFirstRecord

public java.lang.String getFirstRecord()
                                throws RecordNotFound

Returns the first record in the buffer.

This resets the current record position to 0 then gets the next record.

Returns:

a String containing the first record in the buffer

Throws:

RecordNotFound - if no record is available

getNextRecord

public java.lang.String getNextRecord()
                               throws RecordNotFound

Returns the next record in the buffer.

Returns:

a String containing the next record in the buffer

Throws:

RecordNotFound - if no record is available

getRecord

public java.lang.String getRecord(Token aToken)
                           throws TokenNotFound

Returns the record for the specified token.

Parameters:

aToken - a Token containing the token for the desired record

Returns:

a String containing the record for the specified token

Throws:

TokenNotFound - if the token is not found

getRecord

public java.lang.String getRecord(java.lang.String aString)
                           throws StringNotFound

Returns the record for the specified string.

Parameters:

aString - a String containing the string

Returns:

a String containing the record for the specified string

Throws:

StringNotFound - if the string is not found

getRecord

public java.lang.String getRecord(int offset)
                           throws RecordNotFound

Returns the record at the specified offset in the buffer.

This updates the current record position to point to the position after the returned record. When the end of the buffer is reached a flag is set to indicate that no more records are available in the buffer.

Parameters:

offset - an int specifying the offset for the record

Returns:

a String containing the desired record

Throws:

RecordNotFound - if the record is not found

replaceString

public void replaceString(java.lang.String aString,
                          Token aToken)
                   throws TokFileError

Replaces the specified string with a token.

Parameters:

aString - a String contains the string to replace

aToken - a Token contains the token to be put in place of the specified string

Throws:

TokFileError - for errors in processing the input file

replaceToken

public void replaceToken(Token aToken,
                         java.lang.String aString)
                  throws TokFileError

Replaces the specified token with a string.

Parameters:

aToken - a Token containing the token to be replaced

aString - a String containing the string to replace the token

Throws:

TokFileError - for errors in processing the input file

append

public void append(java.lang.String aString)

Adds the specified string to the end of the buffer.

This will extend the buffer so there will be more records available in the buffer and the flag will be set to show this. Don't forget to add appropriate line breaks if you want them. I don't do it for you.

Parameters:

aString - a String containing the string

append

public void append(char aChar)

Adds the specified character to the end of the buffer.

This will extend the buffer so there will be more records available in the buffer and the flag will be set to show this.

Parameters:

aChar - a char containing the character

length

public int length()

Returns the length of this buffer.

Returns:

an int containing the current length of the buffer

hasMoreRecords

public boolean hasMoreRecords()

Returns true if there are more records in the buffer.

Returns:

a boolean to indicate that there are more records in the buffer (true) or not (false)

setBuffer

public void setBuffer(java.lang.StringBuffer aBuffer)

Set the buffer contents.

Parameters:

aBuffer - a StringBuffer specifying the buffer contents

See Also:

getBuffer()

getBuffer

public java.lang.StringBuffer getBuffer()

Returns the buffer.

Returns:

a StringBuffer representing the buffer

See Also:

setBuffer(java.lang.StringBuffer)

setContextTable

public void setContextTable(java.util.Hashtable ch)

Set the context hashtable.

Parameters:

ch - a Hashtable specifying the context hashtable contents

See Also:

getContextTable()

getContextTable

public java.util.Hashtable getContextTable()

Returns the context hashtable.

Returns:

a Hashtable representing the context hashtable

See Also:

setContextTable(java.util.Hashtable)

setRevContextTable

public void setRevContextTable(java.util.Hashtable rch)

Set the reverse context hashtable.

Parameters:

rch - a Hashtable specifying the reverse context hashtable contents

See Also:

getRevContextTable()

getRevContextTable

public java.util.Hashtable getRevContextTable()

Returns the reverse context hashtable.

Returns:

a Hashtable representing the reverse context hashtable

See Also:

setRevContextTable(java.util.Hashtable)

copyOf

public void copyOf(TokFile tf)

Makes a copy of the specified TokFile object contents in this object.

Parameters:

tf - a TokFile object specifying the TokFile object to copy

debugDump

public void debugDump()

Dumps the contents of the object to the console for debugging.

This version of the method assumes that you want all of the contents.

debugDump

public void debugDump(java.lang.String what)

Dumps the contents of the object to the console for debugging.

You can control what gets dumped by specifying a parameter. The parameter can be ALL, BUFFER, or CONTEXT. The parameter must be specified in all uppercase characters.

Parameters:

what - a String containing the desired part of the object to display on the console

loadFileImage

public void loadFileImage(java.lang.String inFileName)
                   throws java.io.IOException

Load the source file image.

This is used in replacing the strings in the file with their tokens. This is only done for files parsed by the JavaCC parser.

Parameters:

inFileName - a String specifying the name of the source file to load

Throws:

java.io.IOException - when an IO error occurs

replaceStringWithToken

public void replaceStringWithToken(int strStartRow,
                                   int strStartCol,
                                   int strEndRow,
                                   int strEndCol,
                                   Token token)

Replaces the string in the file image with a token in the buffer.

This is used when the file is being parsed by the JavaCC parser.

Parameters:

strStartRow - an int specifying the starting line number of the string in the file image that is being replaced

strStartCol - an int specifying the starting column number in the starting line of the string in the file image that is being replaced

strEndRow - an int specifying the ending line number of the string in the file image that is being replaced

strEndCol - an int specifying the ending column number in the ending line of the string in the file image that is being replaced

token - a Token specifying the token that is replacing the string

closeFileImageToBuffer

public void closeFileImageToBuffer()

Place the last part of the image file into the buffer.

This is used when the file is being parsed by the JavaCC parser.

findOffset

public int findOffset(int row,
                      int col)

Returns the offset into the image area based on the row and col numbers provided.

This is used when the file is being parsed by the JavaCC parser.

Parameters:

row - an int specifying the line number

col - an int specifying the column number in the line number

Returns:

int containing the offset into the file image area

convertToFileURL

private static java.lang.String convertToFileURL(java.lang.String filename)

Returns the filename converted to a file URL.

Parameters:

filename - a String containing the file name to be converted

Returns:

String containing the converted file name