Failure in combinator parser

I'm running on 2.9.1

Hey there,
I'm trying to emit readable parser errors, and am following this here.
On the page 637 on this PDF, there's an example on how to emit failures, by using the failure method on the Parsers class. I wrote it down and simplified it a tiny bit to be easy to work with. It's here.
My problem is that this snippet does not produce what the PDF claims. The failure comes from the "false" parser and not the failure(".."). Why is that, and what should it look like?
I'm a bit lost here...
Regards,
Andrés

On Sun, Jan 15, 2012 at 11:47, Andres Taylor
wrote:
> Hey there,
>
> I'm trying to emit readable parser errors, and am following this here.
>
> On the page 637 on this PDF, there's an example on how to emit failures, by
> using the failure method on the Parsers class. I wrote it down and
> simplified it a tiny bit to be easy to work with. It's here.
>
> My problem is that this snippet does not produce what the PDF claims. The
> failure comes from the "false" parser and not the failure(".."). Why is
> that, and what should it look like?
>
> I'm a bit lost here...

The technical reason is that the "false" parser skips whitespace, and
the failure parser doesn't. This means that "false" managed to parse
more of the input than failure, so it's likely in a better position to
produce an error.

Because of this unfortunate interaction between whitspace-skipping in
Regex and JavaTokenParsers and the way failure error messages are
selected, I opened this ticket with an alternative way of providing
error messages: https://issues.scala-lang.org/browse/SI-5192.

On Sun, Jan 15, 2012 at 7:44 AM, Daniel Sobral wrote:
> On Sun, Jan 15, 2012 at 11:47, Andres Taylor
> wrote:
>> Hey there,
>>
>> I'm trying to emit readable parser errors, and am following this here.
>>
>> On the page 637 on this PDF, there's an example on how to emit failures, by
>> using the failure method on the Parsers class. I wrote it down and
>> simplified it a tiny bit to be easy to work with. It's here.
>>
>> My problem is that this snippet does not produce what the PDF claims. The
>> failure comes from the "false" parser and not the failure(".."). Why is
>> that, and what should it look like?
>>
>> I'm a bit lost here...
>
> The technical reason is that the "false" parser skips whitespace, and
> the failure parser doesn't. This means that "false" managed to parse
> more of the input than failure, so it's likely in a better position to
> produce an error.
>
> Because of this unfortunate interaction between whitspace-skipping in
> Regex and JavaTokenParsers and the way failure error messages are
> selected, I opened this ticket with an alternative way of providing
> error messages: https://issues.scala-lang.org/browse/SI-5192.
>
> --

A remedy is to use StandardTokenParsers, like this:

import scala.util.parsing.combinator.syntactical._
import scala.util.parsing.input._

object ParserTest extends StandardTokenParsers {
lexical.reserved += ("null", "true", "false")
lexical.delimiters += ("{", "}", "[", "]", ",", ":")

def main(args: Array[String]) {
val json = """{ "address book": John }"""

val r = parse(json)
println(r)
}

def obj: Parser[Map[String, Any]] =
"{" ~> repsep(member, ",") <~ "}" ^^ (Map() ++ _)

def arr: Parser[List[Any]] =
"[" ~> repsep(value, ",") <~ "]"

def member: Parser[(String, Any)] =
stringLit ~ ":" ~ value ^^ {
case name ~ ":" ~ value => (name, value)
}

def value: Parser[Any] = obj | arr | stringLit | numericLit ^^
(_.toDouble) | "null" ^^ (x => null) | "true" ^^ (x => true) | "false"
^^ (x => false) | failure("illegal start of value")

def parse(txt: String) = phrase(obj)(new lexical.Scanner(txt))
}

On 16/01/2012, at 2:44 AM, Daniel Sobral wrote:

> On Sun, Jan 15, 2012 at 11:47, Andres Taylor
> wrote:
>> Hey there,
>>
>> I'm trying to emit readable parser errors, and am following this here.
>>
>> On the page 637 on this PDF, there's an example on how to emit failures, by
>> using the failure method on the Parsers class. I wrote it down and
>> simplified it a tiny bit to be easy to work with. It's here.
>>
>> My problem is that this snippet does not produce what the PDF claims. The
>> failure comes from the "false" parser and not the failure(".."). Why is
>> that, and what should it look like?
>>
>> I'm a bit lost here...
>
> The technical reason is that the "false" parser skips whitespace, and
> the failure parser doesn't. This means that "false" managed to parse
> more of the input than failure, so it's likely in a better position to
> produce an error.
>
> Because of this unfortunate interaction between whitspace-skipping in
> Regex and JavaTokenParsers and the way failure error messages are
> selected, I opened this ticket with an alternative way of providing
> error messages: https://issues.scala-lang.org/browse/SI-5192.

That's a nice enhancement.

Another way to avoid the whitespace-skipping problem that I've used on occasion is to define

override def err (msg : String) : Parser[Nothing] =
"" ~> super.err (msg)

override def failure (msg : String) : Parser[Nothing] =
"" ~> super.failure (msg)

so that the errors and failures are reported after the whitespace has been skipped.

Tony