Commit Graph

4 Commits

Author SHA1 Message Date
greg
169e662049 Collapse Separator tokens
only ever gonna be one in a row
2017-09-19 22:10:52 -07:00
greg
bfa16fd6fb Added Keyword lexical class 2017-09-19 22:10:52 -07:00
greg
1059a88ee6 Separate parsing into module 2017-09-19 22:10:52 -07:00
greg
429ace73bd Move tokenizing into separate module 2017-09-19 22:10:52 -07:00