[project @ 2002-06-05 14:08:23 by simonpj]

[project @ 2002-06-05 14:08:23 by simonpj]
------------------------------------------------
	Fix the (new) lexer, and make the derived read
	and show code work according to the new H98 report
	------------------------------------------------

The new lexer, based on Koen's cunning parser (Text.ParserCombinators.ReadP)
wasn't quite right.  It's all very cool now.

In particular:

* The H98 "lex" function should return the exact string parsed, and it
now does, aided by the new combinator ReadP.gather.

* As a result the Text.Read.Lex Lexeme type is much simpler than before
    data Lexeme
      = Char   Char	-- Quotes removed,
      | String String	-- 	escapes interpreted
      | Punc   String 	-- Punctuation, eg "(", "::"
      | Ident  String	-- Haskell identifiers, e.g. foo, baz
      | Symbol String	-- Haskell symbols, e.g. >>, %
      | Int Integer
      | Rat Rational
      | EOF
     deriving (Eq,Show)

* Multi-character punctuation, like "::" was getting lexed as a Symbol,
but it should be a Punc.

* Parsing numbers wasn't quite right.  "1..n" got it confused because it
got committed to a decimal point and then found a second '.'.

* The new H98 spec for Show is there, which ignores associativity.