[project @ 2003-09-08 11:52:24 by simonmar]

[project @ 2003-09-08 11:52:24 by simonmar]
Replace the handwritten lexer with one generated by Alex.

YOU NOW NEED ALEX (v 2.0 or later) TO COMPILE GHC FROM CVS.

Highlights:

  - Faster than the previous lexer (about 10% of total parse time,
    depending on the token mix).

  - More correct than the previous lexer: a couple of minor wibbles
    in the syntax were fixed.

  - Completely accurate source spans for each token are now collected.
    This information isn't used yet, but it will be used to give much
    more accurate error messages in the future.

  - SrcLoc now contains a column field as well as a line number,
    although this is currently ignored when printing out SrcLocs.

  - StringBuffer is now based on a ByteArray# rather than a Ptr, which
    means that StringBuffers are now garbage collected.  Previously
    StringBuffers were hardly ever released, so a GHCi session would
    leak space as more source files were loaded in.

  - Code size reduction: Lexer.x is about the same size as the old
    Lex.lhs, but StringBuffer.lhs is significantly shorter and
    simpler.  Sadly I wasn't able to get rid of parser/Ctypes.hs
    (yet).