causerie

Introduction

Units

Class Hierarchy

Classes, Interfaces, Objects and Records

Types

Variables

Constants

Functions and Procedures

Identifiers

Classes hierarchy graph

Class ATokenFromSource

DescriptionHierarchyFieldsMethodsProperties

Unit

parsing

Declaration

type ATokenFromSource = class(AToken)

Description

This class represents a token that is parsed from a source stream. Its methods are defined to retrieve the literal text of the token and match it with an opcode defined by a parsed language. It has methods that enable a parser to check syntax and can also write itself to an intermediate code stream for further processing by an interpeter.

Hierarchy

Overview

Fields

Protected MyLanguage: AParsedLanguage;
Protected myText: string;

Methods

Public constructor using(const ThisLanguage: AParsedLanguage); virtual;
Public class function readUsing(const ThisLanguage: AParsedLanguage; const Source: ATextInputStream): AToken; virtual;
Public function init: boolean; override;
Public procedure returnTo(const Source: AStream); override;
Public function isPartOf(const rule: TSortKey): boolean; overload; virtual;
Public function isPartOfAny(const rules: array of TSortKey):TSortKey; overload; virtual;
Public function shallowCopyFrom(const Other: AnObject): boolean; override;
Public function readFrom(const Source: AStream): TStreamIOSize; override;
Public function toString: AnsiString; override;
Public function Language: AParsedLanguage;
Public function text: string; virtual;

Description

Fields

Protected MyLanguage: AParsedLanguage;

Refers to the language definition used to parse the token from the source.

Protected myText: string;

Stores the literal text of the token, as parsed from the source

Methods

Public constructor using(const ThisLanguage: AParsedLanguage); virtual;

Construct a new instance of ATokenFromSource that will use the specified language to tokenize the source.

Public class function readUsing(const ThisLanguage: AParsedLanguage; const Source: ATextInputStream): AToken; virtual;

Construct and return an instance of ATokenFromSource or its descendants.

This method is a factory method designed to be called by a scanner on an instance of ATextInputStream. It reads a character from Source and, depending on how that character is defined within ThisLanguage, constructs the appropriate instance of ATokenFromSource or its descendants which will return that token to the caller.

Public function init: boolean; override;

Initializer

Public procedure returnTo(const Source: AStream); override;

Return the token to its source. The token will be read again the next time the source is queried for a token.

This method overrides the behavior inherited from AToken. It calls AStream.rewindBy on Source, passing the size of its literal token string as the number of bytes by which the stream position should be rewound.

This method is defined to provide a way for a scanner to "peek" at the next token in the source stream without actually having to process it.

Public function isPartOf(const rule: TSortKey): boolean; overload; virtual;

Determine whether or not the token is allowed by the specified rule.

This method looks up the rule specified by rule, then passes that reference to AToken.isPartOf and returns the result. If rule specifies a rule that cannot be found, this method returns False.

Public function isPartOfAny(const rules: array of TSortKey):TSortKey; overload; virtual;

Determine whether or not the token is allowed by one of the specified rules.

This method checks the token opcode against each rule specified in rules. The ID of the first rule that contains the token is returned. If no rule contains the token, this method returns -1.

This method is defined to allow a parser to determine whether a given token may appear at the current source location; in other words, it allows a parser to check syntax.

Public function shallowCopyFrom(const Other: AnObject): boolean; override;

Construct a shallow copy of the other object.

This method extends the behavior inherited from AToken.shallowCopyFrom: it calls that method, then checks to see whether Other is an instance of ATokenFromSource. If so, it copies the value of ATokenFromSource.text from Other to Self, overwriting the value in Self.

Note that this method does NOT copy any sibling or child nodes and so cannot be used to create a full copy of any descendant of ANode. The copy will NOT be automatically placed in the list to which Other belongs, if any, but the caller is free to do so.

Public function readFrom(const Source: AStream): TStreamIOSize; override;

Read the token from the given source stream.

This method overrides the behavior inherited from AToken. It is written with the assumption that Source represents an instance of ATextInputStream. However, in the base implementation of ATokenFromSource, it does nothing at all; it is for descendants of ATokenFromSource to retrieve and parse the token text.

Returns

The total number of bytes retrieved from Source. In the base implementation provided by ATokenFromSource, this value will always be zero (0).

Public function toString: AnsiString; override;

Construct a string representation of the token, suitable for output to a text-based device, such as a console.

This method overrides the behavior inherited from AToken. It simply returns a quoted form of Self.text, as obtained by a call to Charstring.quote. It is assumed that this method will be called by a parser that needs to output a warning or error message concerning a given token.

Public function Language: AParsedLanguage;

Retrieve a reference to the language definition used to parse the token.

The caller may make use of this reference to directly query the language definition being used, but it should NOT attempt to free the reference.

Public function text: string; virtual;

Retrieve the literal text of the token, as read from the source stream.

Generated by PasDoc 0.13.0 on 2015-01-10 17:13:18