CSC 468
Tokenizer

Files:

readme.txt   : this file
tokenizer.txt: tokenizer spec (list of recognized tokens)

MXBtokenizer.java : main tokenizer class
TokenList.java    : list of tokens
Token.java        : token

tokenizer.java    : test driver.