This is another one for the list of occam oddities: there are situations where
both a hex literal and a preprocessor directive are legal, so the two can't be
ambiguous -- no #FACE or #ADA in the future!
The occam parser is now a GenParser Token OccState, rather than a GenParser
Char OccState, and a lot of now-redundant code has been removed. The parser is
also somewhat faster, which wasn't intended but is nice anyway.
I've also modified the Rain parser to not rely on the old preprocessing code;
it wasn't appropriate for Rain's syntax anyway, so I assume Neil will be
replacing it eventually.
This is because the structure pass needs to be able to tell where new lines
start, and if the line number changes in the middle of a token then it'll get
confused.
This looks more complicated than it is because it meant adding state to the
lexer -- it's now quite a neat little three-state machine.
This also renames DecimalLiteral to IntLiteral to match the rest of the code.
The parser now gets a stream of tokens, rather than needing to worry about
loading files itself.
This also reworks the lexer's idea of what constitutes a Token -- it's now a
pair (Meta, TokenType), so it's always easy to pull out/rewrite the metadata --
and adds proper support for lexing preprocessor directives, rather than just
treating them as reserved words.
The aim is to separate lexing from the existing parser, which should make
indentation processing a good deal less painful.
The lexer is compiled using Alex, which generates Haskell code from the .x
file; it's not particularly elegant but the input syntax is nice and it does
seem to work well.
It turns out there's a Prelude function that does what readSource does already
-- so just use that. (I was in the process of moving readSource into Utils when
I found it, since I want to use it in other places.)
The compiler itself is under the GPLv2+; the support code that gets built into
user programs is under the LGPLv2+. This matches the existing practice for the
KRoC project.
(As with Occade, I've used the new GPLv3-style license header in the source
files, though, since that avoids having to update the FSF's postal address.)
