Edinburgh Speech Tools  2.4-release
 All Classes Functions Variables Typedefs Enumerations Enumerator Friends Pages
EST_Token Class Reference

#include <include/EST_Token.h>

Public Member Functions

 EST_Token (const EST_String p)
void init ()
void set_row (int r)
void set_col (int c)
void set_filepos (int c)
 Set file position in original {EST_TokenStream}.
EST_String lstring ()
 Return lower case version of token name.
EST_String ustring ()
 Return upper case version of token name.
int row (void) const
 Line number in original {EST_TokenStream}.
int col (void) const
 Line position in original {EST_TokenStream}.
int filepos (void) const
 file position in original {EST_TokenStream}.
const EST_String pos_description () const
 A string describing current position, suitable for error messages.
EST_Tokenoperator= (const EST_Token &a)
EST_Tokenoperator= (const EST_String &a)
int operator== (const EST_String &a)
int operator!= (const EST_String &a)
int operator== (const char *a)
int operator!= (const char *a)
Basic access to fields
void set_token (const EST_String &p)
 set token from a string
void set_token (const char *p)
void set_whitespace (const EST_String &p)
 set whitespace of token.
void set_whitespace (const char *p)
void set_punctuation (const EST_String &p)
 set (post) punctuation of token.
void set_punctuation (const char *p)
void set_prepunctuation (const EST_String &p)
 set prepunction
void set_prepunctuation (const char *p)
const EST_Stringwhitespace ()
const EST_Stringpunctuation ()
const EST_Stringprepunctuation ()
Access token as a string
const EST_Stringstring () const
const EST_StringS () const
 Access token as a string.
const EST_StringString () const
 Access token as a string.
 operator EST_String () const
 For automatic coercion to {EST_String}.
Access token as a int
int Int (bool &valid) const
int Int () const
int I (bool &valid) const
int I () const
 operator int () const
Access token as a long
long Long (bool &valid) const
long Long () const
long L (bool &valid) const
long L () const
 operator long () const
Access token as a float
float Float (bool &valid) const
float Float () const
float F (bool &valid) const
float F () const
 operator float () const
Access token as a double
double Double (bool &valid) const
double Double () const
double D (bool &valid) const
double D () const
 operator double () const
void set_quoted (int q)
 Note that this token was quoted (or not)
int quoted () const
 TRUE is token was quoted.


ostream & operator<< (ostream &s, const EST_Token &p)

Detailed Description

This class is similar to {EST_String} but also maintains the original punctuation and whitespace found around the token.

{EST_Token}'s primary use is with {EST_TokenStream} class which allows easy tokenizing of ascii files.

A token consists of four parts, any of which may be empty: a name, the actual token, preceding whitespace, preceding punctuation, the name and succeeding punctuation.

Alan W Black (awb@c.nosp@m.str..nosp@m.ed.ac.nosp@m..uk): April 1996

Definition at line 73 of file EST_Token.h.

The documentation for this class was generated from the following files: