Arena
1.0
A content management system
|
A token that represents a special character which is not necessarily a keyword or operator (such a list delimiter, brackets, etc.). More...
Public Member Functions | |
def | __init__ |
Constructor. | |
def | readFrom |
Read the token from the specified stream. More... | |
Public Member Functions inherited from Library.Parse.Scan.Tokens.AToken | |
def | __init__ |
Constructor. More... | |
def | init |
Initializer. | |
def | rewind |
Return the token to the input stream. More... | |
def | toString |
Return a string representation of the token. More... | |
def | readFrom |
Read the token from the specified stream. More... | |
def | writeTo |
Write the token to the specified stream. More... | |
def | code |
Retrieve the token code. | |
def | literal |
Retrieve the token literal. | |
def | Language |
Retrieve the language used to parse the token. More... | |
Public Member Functions inherited from Library.Base.Classwork.AnObject | |
def | init |
Initialize a new object instance. More... | |
def | __init__ |
Construct and initialize a new instance of AnObject. More... | |
def | __del__ |
Destroy the object instance. More... | |
def | displayName |
Construct a "pretty" display name for the class instance, based on the name of its class. More... | |
def | implements |
Determine whether the class instance is related to the specified class. More... | |
def | toString |
Return a string representation of the class, suitable for output to a console or text-based device. More... | |
def | __str__ |
Return a string representation of the class. More... | |
def | classname |
Return the name of the class from which the instance was created. More... | |
def | name |
Return the name of the class. More... | |
Public Member Functions inherited from Library.String.Utilities.IsPrintable | |
def | toString |
Return a string representation of the class conveniently. More... | |
def | __str__ |
Return a string representation of the class. More... | |
def | printTo |
Print a string representation of the class to the specified stream. More... | |
Public Member Functions inherited from Library.IO.Streams.IsStreamable | |
def | writeTo |
Write class instance data to the specified stream. More... | |
def | readFrom |
Read class instance data from the specified stream. More... | |
Additional Inherited Members | |
Static Public Member Functions inherited from Library.Parse.Scan.Tokens.AToken | |
def | fromSource |
Construct a new token based on the next character read from the specified stream. More... | |
A token that represents a special character which is not necessarily a keyword or operator (such a list delimiter, brackets, etc.).
As with AWordToken, this class accumulates characters until it encounters one that is not allowed by the language to be a symbol. It then checks the accumulated string against a list of registered keywords and operators. If no match is found, the class returns the rightmost character from its string to the stream and checks again. The process continues until a match is found or the token literal is only a single character in length.
def Library.Parse.Scan.Tokens.ASpecialToken.readFrom | ( | Self, | |
Source | |||
) |
Read the token from the specified stream.
This routine is used to tokenize text in the specified stream.