All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Groups Pages
pp::Tokenizer Class Reference

#include <Tokenizer.h>

Inherits pp::Lexer.


struct  Context

Public Member Functions

 Tokenizer (Diagnostics *diagnostics)
 ~Tokenizer ()
bool init (size_t count, const char *const string[], const int length[])
virtual void lex (Token *token)
void setFileNumber (int file)
void setLineNumber (int line)
- Public Member Functions inherited from pp::Lexer
virtual ~Lexer ()

Static Public Attributes

static const std::size_t kMaxTokenLength = 256

Detailed Description

Definition at line 19 of file Tokenizer.h.

Constructor & Destructor Documentation

pp::Tokenizer::Tokenizer ( Diagnostics diagnostics)

Definition at line 2297 of file Tokenizer.cpp.

References pp::Tokenizer::Context::diagnostics.

pp::Tokenizer::~Tokenizer ( )

Definition at line 2302 of file Tokenizer.cpp.

Member Function Documentation

bool pp::Tokenizer::init ( size_t  count,
const char *const  string[],
const int  length[] 

Definition at line 2307 of file Tokenizer.cpp.

References pp::Tokenizer::Context::input.

void pp::Tokenizer::setFileNumber ( int  file)

Definition at line 2315 of file Tokenizer.cpp.

References ppset_column().

void pp::Tokenizer::setLineNumber ( int  line)

Definition at line 2322 of file Tokenizer.cpp.

References ppset_lineno().

Member Data Documentation

const size_t pp::Tokenizer::kMaxTokenLength = 256

Definition at line 35 of file Tokenizer.h.

Referenced by lex().

The documentation for this class was generated from the following files: