Class ASpaceToken

Indicates whether successive instances of a given whitespace character should be consolidated into one token or not

Initializer

Construct a shallow copy of the other object.

This method extends the behavior inherited from ATokenFromSource.shallowCopyFrom: it calls that method, then checks to see whether Other is an instance of ASpaceToken. If so, it copies the value of ASpaceToken.consolidation from Other to Self, overwriting the value in Self.

Note that this method does NOT copy any sibling or child nodes and so cannot be used to create a full copy of any descendant of ANode. The copy will NOT be automatically placed in the list to which Other belongs, if any, but the caller is free to do so.

Read the token from the given source stream.

This method reads characters from Source until one is encountered that is not considered whitespace.

If ASpaceToken.consolidation is True, then this method reads consecutive instances of the first whitespace character encountered until a character is encountered that is neither whitespace nor the same as the first whitespace character read. Otherwise, this method reads only a single whitespace character from the source.

This method is written with the assumption that Source represents an instance of ATextInputStream. To read non-symbolic tokens from an intermediate code stream, an instance of AToken should be used instead.

Returns

The total number of bytes read from Source.

Determine whether or not consecutive instances of a given whitespace character will be consolidated into one token or not.

If this function returns True, then occurrences of more than one consecutive instance of the first whitespace character encountered will be consolidated into a single whitespace token. In other words, if the source contains four space characters in a row, as is customary with languages like Python, then those four characters will be consolidated into one token. Otherwise, they will be read as four successive tokens.

For instances of ASpaceToken, this method will return False by default. For instances of ALineEndingToken, this method will return True by default. To change this behavior, call ASpaceToken.setConsolidation before calling ASpaceToken.readFrom or ALineEndingToken.readFrom.

Set whether or not consecutive instances of a given whitespace character are consolidated into one token.

If flag is True, then occurrences of more than one consecutive isntance of the first whitespace character encountered will be consolidated into a single token; in other words, if the source contains four space characters in a row, as is customary with languages like Python, then those four space characters will be collected into one token. Otherwise, they will be read as four distinct tokens.

Returns

The previous value of ASpaceToken.consolidation.