2.9.0 Regression

On 5/18/11 6:43 AM, Chris Marshall wrote:
> In 2.8

Undefined behavior.

> In 2.9

Same behavior.

When you find yourself calling a destructive, invalidating method on an
Iterator, and then think to yourself "I believe I'll call another one!"
you have embraced the undefined.

Exactly who says that the Iterator.toStream method has undefined behaviour? Certainly not ScalaDoc:
"a stream containing all elements of this traversable or iterator"
Or is it undefined because it doesn't state exactly how it will mutate the underlying iterator?

> Date: Wed, 18 May 2011 08:32:00 -0700
> From: paulp@improving.org
> To: scala-internals@googlegroups.com
> CC: oxbow_lakes@hotmail.com
> Subject: Re: [scala-internals] 2.9.0 Regression
>
> On 5/18/11 6:43 AM, Chris Marshall wrote:
> > In 2.8
>
> Undefined behavior.
>
> > In 2.9
>
> Same behavior.
>
> When you find yourself calling a destructive, invalidating method on an
> Iterator, and then think to yourself "I believe I'll call another one!"
> you have embraced the undefined.

On 5/18/11 9:16 AM, Chris Marshall wrote:
> Exactly who says that the Iterator.toStream method has undefined
> behaviour? Certainly not ScalaDoc:

That's good, neither did I.

Is it really not apparent at this late date that repeatedly calling
destructive methods on single-usage mutable structures does not have a
defined outcome. Here:

scala> val it = (1 to 3).iterator
it: Iterator[Int] = non-empty iterator

scala> it.toList
res0: List[Int] = List(1, 2, 3)

// is it a regression if this one changes?
scala> it.toList
res1: List[Int] = List()

Whatever behavior it is you think you can rely upon on that second call,
you are incorrect. There are two[1] methods you can call on an iterator
and have a good idea about the state of the iterator afterward. The
rest involve opening the box and killing the cat[2].

[1] There are more than two, but let's err on the side of caution.
[2] Don't come here expecting sensible metaphors.

Understood - but when one tries something, finds that it works (basically I wanted an Iterator.nextOption and found that this was possible via Iterator.toStream.headOption) and doesn't see anything in the documentation to *explicitly* state that it is unsafe, then it is not entirely unreasonable to assume that the behaviour is valid. Especially when one considers the (admittedly historic) paucity of documentation (which at one time, might have led us to assume that all behaviour was undefined) and also the fact that said documentation explicitly described Stream (or did when I wrote this code) as being a "lazy list where elements are only evaluated when needed". I guess the Stream might "need" the elements more often than I do.
On a related note: Can we have Iterator.nextOption as a method, to avoid these misunderstandings in future?
Chris

> Date: Wed, 18 May 2011 09:44:01 -0700
> From: paulp@improving.org
> To: scala-internals@googlegroups.com
> CC: oxbow_lakes@hotmail.com
> Subject: Re: [scala-internals] 2.9.0 Regression
>
> On 5/18/11 9:16 AM, Chris Marshall wrote:
> > Exactly who says that the Iterator.toStream method has undefined
> > behaviour? Certainly not ScalaDoc:
>
> That's good, neither did I.
>
> Is it really not apparent at this late date that repeatedly calling
> destructive methods on single-usage mutable structures does not have a
> defined outcome. Here:
>
> scala> val it = (1 to 3).iterator
> it: Iterator[Int] = non-empty iterator
>
> scala> it.toList
> res0: List[Int] = List(1, 2, 3)
>
> // is it a regression if this one changes?
> scala> it.toList
> res1: List[Int] = List()
>
> Whatever behavior it is you think you can rely upon on that second call,
> you are incorrect. There are two[1] methods you can call on an iterator
> and have a good idea about the state of the iterator afterward. The
> rest involve opening the box and killing the cat[2].
>
> [1] There are more than two, but let's err on the side of caution.
> [2] Don't come here expecting sensible metaphors.

On 5/18/11 10:31 AM, Chris Marshall wrote:
> Understood - but when one tries something, finds that it works
> (basically I wanted an Iterator.nextOption and found that this was
> possible via Iterator.toStream.headOption) and doesn't see anything in
> the documentation to *explicitly* state that it is unsafe, then it is
> not entirely unreasonable to assume that the behaviour is valid.

It is, in fact, entirely unreasonable. Any part of you which feels
otherwise will only lead you to sorrow.

So would scala methods benefit from scaladoc flags 'mutates me' and 'doesn't mutate me' so that we can instantly identify things that may or may not cause excitement?
Matthew

On 18 May 2011 18:35, Paul Phillips <paulp@improving.org> wrote:

On 5/18/11 10:31 AM, Chris Marshall wrote:
> Understood - but when one tries something, finds that it works
> (basically I wanted an Iterator.nextOption and found that this was
> possible via Iterator.toStream.headOption) and doesn't see anything in
> the documentation to *explicitly* state that it is unsafe, then it is
> not entirely unreasonable to assume that the behaviour is valid.

It is, in fact, entirely unreasonable.  Any part of you which feels
otherwise will only lead you to sorrow.

--
Matthew Pocockmailto: turingatemyhamster@gmail.comgchat: turingatemyhamster@gmail.com msn: matthew_pocock@yahoo.co.ukirc.freenode.net: drdozer(0191) 2566550

What is slightly odd about this is that the following methods have not changed:
Iterator.toStreamTreeMap.iteratorRedBlack#Tree.toStream
To be honest, I'm a bit at a loss to understand why the behaviour has changed anyway, given that it all flows from these.
I will take your advice to heart - sorrow leads to rancour, rancour leads to state and state leads to buffering. Which none of us want.


> Date: Wed, 18 May 2011 10:35:52 -0700
> From: paulp@improving.org
> To: scala-internals@googlegroups.com
> CC: oxbow_lakes@hotmail.com
> Subject: Re: [scala-internals] 2.9.0 Regression
>
> On 5/18/11 10:31 AM, Chris Marshall wrote:
> > Understood - but when one tries something, finds that it works
> > (basically I wanted an Iterator.nextOption and found that this was
> > possible via Iterator.toStream.headOption) and doesn't see anything in
> > the documentation to *explicitly* state that it is unsafe, then it is
> > not entirely unreasonable to assume that the behaviour is valid.
>
> It is, in fact, entirely unreasonable. Any part of you which feels
> otherwise will only lead you to sorrow.

Possibly...
But more useful still would be a full effect system, so you can specify where you only expect pure functions an immutable behaviour - and the compiler then throws an error if you attempt to do something side-effecting where it's forbidden.

On 18 May 2011 18:56, Matthew Pocock <turingatemyhamster@gmail.com> wrote:

So would scala methods benefit from scaladoc flags 'mutates me' and 'doesn't mutate me' so that we can instantly identify things that may or may not cause excitement?
Matthew

On 18 May 2011 18:35, Paul Phillips <paulp@improving.org> wrote:

On 5/18/11 10:31 AM, Chris Marshall wrote:
> Understood - but when one tries something, finds that it works
> (basically I wanted an Iterator.nextOption and found that this was
> possible via Iterator.toStream.headOption) and doesn't see anything in
> the documentation to *explicitly* state that it is unsafe, then it is
> not entirely unreasonable to assume that the behaviour is valid.

It is, in fact, entirely unreasonable.  Any part of you which feels
otherwise will only lead you to sorrow.

--
Matthew Pocockmailto: turingatemyhamster@gmail.comgchat: turingatemyhamster@gmail.com msn: matthew_pocock@yahoo.co.ukirc.freenode.net: drdozer(0191) 2566550

--
Kevin Wright

gtalk / msn : kev.lee.wright@gmail.comkev.lee.wright@gmail.commail: kevin.wright@scalatechnology.com
vibe / skype: kev.lee.wrightquora: http://www.quora.com/Kevin-Wright
twitter: @thecoda

"My point today is that, if we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger" ~ Dijkstra

On 18 May 2011 19:19, Kevin Wright <kev.lee.wright@gmail.com> wrote:

Possibly...
But more useful still would be a full effect system, so you can specify where you only expect pure functions an immutable behaviour - and the compiler then throws an error if you attempt to do something side-effecting where it's forbidden.

I agree. We have at least one post a week where the correct answer is wholly or in part, "effect system". However, that wouldn't stop us adding @pure and @impure either at the source or scaladoc levels now, so that our intent can be documented even in the absence of enforcement by the compiler.
Is there any appetite for this, or am I a lone voice?
Matthew
--
Matthew Pocockmailto: turingatemyhamster@gmail.com gchat: turingatemyhamster@gmail.commsn: matthew_pocock@yahoo.co.uk irc.freenode.net: drdozer(0191) 2566550

On 19 May 2011 11:35, Matthew Pocock <turingatemyhamster@gmail.com> wrote:

On 18 May 2011 19:19, Kevin Wright <kev.lee.wright@gmail.com> wrote:

Possibly...
But more useful still would be a full effect system, so you can specify where you only expect pure functions an immutable behaviour - and the compiler then throws an error if you attempt to do something side-effecting where it's forbidden.

I agree. We have at least one post a week where the correct answer is wholly or in part, "effect system". However, that wouldn't stop us adding @pure and @impure either at the source or scaladoc levels now, so that our intent can be documented even in the absence of enforcement by the compiler.
Is there any appetite for this, or am I a lone voice?
Matthew

Lots of appetite :)
But perhaps this should come in addition to any effect system, with the compiler then using the annotations to validate your stated assumptions, in much the same way that @tailrec works.