Tokens.py File Reference

This file defines several classes that are used to retrieve tokens from a stream. More...

Classes
class	Library.Parse.Scan.Tokens.AToken
	A generic token. More...
class	Library.Parse.Scan.Tokens.AWordToken
	A token that represents a single word or identifier. More...
class	Library.Parse.Scan.Tokens.ANumberToken
	A token that represents a numeric value. More...
class	Library.Parse.Scan.Tokens.AStringToken
	A token that represents a string literal. More...
class	Library.Parse.Scan.Tokens.ASpecialToken
	A token that represents a special character which is not necessarily a keyword or operator (such a list delimiter, brackets, etc.). More...
class	Library.Parse.Scan.Tokens.ASpaceToken
	A token that represents whitespace (not a token from space). More...
class	Library.Parse.Scan.Tokens.ALineEndingToken
	A token which represents an end of line character (or sequence of characters, for Windows-derived streams). More...
class	Library.Parse.Scan.Tokens.AStreamEndingToken
	A token that represents the end of the stream. More...
class	Library.Parse.Scan.Tokens.AnErrorToken
	An erroneous token read from the source. More...

Detailed Description

This file defines several classes that are used to retrieve tokens from a stream.