Package org.openoffice.da.comp.w2lcommon.tex.tokenizer

Class Token

  • All Implemented Interfaces:
    java.lang.Cloneable
    public class Token
extends java.lang.Object
implements java.lang.Cloneable
    This class represent a token in TeX

    • Constructor Summary

      Constructors 
      Constructor Description
      Token()
      Construct a new Token, initialized as a TokenTYPE.ENDINPUT-token

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      protected void addChar​(char c)
      Append a character to the character content of this token
      protected void clearChars()
      Delete the character content of this token
      java.lang.Object clone()  
      char getChar()
      Get the first character in this token
      java.lang.String getString()
      Get the character content of this token as a string
      TokenType getType()
      Get the TokenType of this token
      boolean is​(char c, TokenType type)
      Test wether this token is a character token of the given type (that is, a single character with a token type that is neither COMMAND_SEQUENCE nor ENDINPUT)
      boolean isCS​(java.lang.String sName)
      Test wether this token is a COMMAND_SEQUENCE token with a given name
      protected void set​(char c, TokenType type)
      Set this token as a character token with a specific TokenType
      protected void setChar​(char c)
      Set the character content of this token to a single character (the type of the token is not changed)
      protected void setType​(TokenType type)
      Set the type of this token to a specific TokenType (the character content is not changed)
      java.lang.String toString()  

      • Methods inherited from class java.lang.Object

        equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Constructor Detail

      • Token

        public Token()
        Construct a new Token, initialized as a TokenTYPE.ENDINPUT-token

    • Method Detail

      • setType

        protected void setType​(TokenType type)
        Set the type of this token to a specific TokenType (the character content is not changed)
        Parameters:
        type - the new TokenType

      • setChar

        protected void setChar​(char c)
        Set the character content of this token to a single character (the type of the token is not changed)
        Parameters:
        c - the character

      • set

        protected void set​(char c,
                   TokenType type)
        Set this token as a character token with a specific TokenType
        Parameters:
        c - the character
        type - the TokenType to use

      • clearChars

        protected void clearChars()
        Delete the character content of this token

      • addChar

        protected void addChar​(char c)
        Append a character to the character content of this token
        Parameters:
        c - the character to be appended

      • is

        public boolean is​(char c,
                  TokenType type)
        Test wether this token is a character token of the given type (that is, a single character with a token type that is neither COMMAND_SEQUENCE nor ENDINPUT)
        Parameters:
        c - the character to test
        type - the TokenType to test
        Returns:
        true if the test was successful

      • isCS

        public boolean isCS​(java.lang.String sName)
        Test wether this token is a COMMAND_SEQUENCE token with a given name
        Parameters:
        sName - the name of the command sequence
        Returns:
        true if the test was successful

      • getType

        public TokenType getType()
        Get the TokenType of this token
        Returns:
        the type

      • getChar

        public char getChar()
        Get the first character in this token
        Returns:
        the character or U+FFFF is no characters exist

      • getString

        public java.lang.String getString()
        Get the character content of this token as a string
        Returns:
        the character content

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object

      • clone

        public java.lang.Object clone()
        Overrides:
        clone in class java.lang.Object