isDefinedAt introduction

On Mon, Nov 9, 2009 at 6:48 PM, Paul Phillips wrote:
> r19466:
>
>    Adds isDefinedAt to Function1. As a consequence,
>    code that mixes in PartialFunction now have to
>    define isDefinedAt as override.
>
> This is a big decision! A lot of people are interested in this issue.
> Was there a big discussion somewhere I missed or did this come
> completely out of nowhere (from the perspective of those of us not at
> LAMP) or...? I can't even see any basis for the distinction between
> Function1 and PartialFunction anymore.

It came pretty much out of nowhere. I was asked whether I'm OK with it
but we did not really discuss this. Can we get an assessment how much
code would break? For instance, how much code did break in the
collection libraries when the change was done?

Cheers

On Mon, Nov 9, 2009 at 10:48 AM, Paul Phillips wrote:
> r19466:
>
>    Adds isDefinedAt to Function1. As a consequence,
>    code that mixes in PartialFunction now have to
>    define isDefinedAt as override.
>
> This is a big decision! A lot of people are interested in this issue.
> Was there a big discussion somewhere I missed or did this come
> completely out of nowhere (from the perspective of those of us not at
> LAMP) or...? I can't even see any basis for the distinction between
> Function1 and PartialFunction anymore.
>
> I especially can't understand this change coming at this moment, after
> changes being dismissed more than once in the past.  Aren't we trying to
> stabilize here?
>
> Don't get me wrong, I would dearly love to see the arrangement between
> these classes altered, but I don't understand the time, the change, or
> the absence of any discussion.
>
> --
> Paul Phillips      | The most dangerous man to any government is the man who
> Everyman           | is able to think things out [...] Almost inevitably he
> Empiricist         | comes to the conclusion that the government he lives under
> pp: i haul pills   | is dishonest, insane, intolerable.   -- H. L. Mencken
>

Whoah, what? Adding isDefinedAt to Function1 seems to me to be the
worst of all possible choices with respect to the insanity that
already rears its grotesque head in PartialFunction.

I mean, hell, PartialFunction shouldn't even exist! The compiler can
statically determine whether a match is a total function based upon
the sealedness of the type being matched on, right? Why can't match
simply generate A => B for exhaustive matches, and A => Option[B] for
incomplete matches, and everybody go along their merry way?

If somebody really, really wants to have their code error out instead
of behaving in a sane fashion, they can very well go ahead and write
implicit def theTypeIsALie[A,B](pf: A => Option[B]): A => B. May the
pain be with them.

I really, really wish that somebody would at least update
http://lampsvn.epfl.ch/trac/scala/ticket/85 with a description that is
more enlightening than "We decided not to." Seriously, PartialFunction
<: Function1 seems to me to be the most dangerously bug-prone feature
of Scala. Please axe it for 3.0. Pretty please with sugar on top!

Kris

On Mon, Nov 09, 2009 at 06:54:41PM +0100, martin odersky wrote:
> It came pretty much out of nowhere. I was asked whether I'm OK with it
> but we did not really discuss this. Can we get an assessment how much
> code would break? For instance, how much code did break in the
> collection libraries when the change was done?

And just to supplement that thought a little, I have very little problem
with breaking code (too little, some might say) and I am much more
concerned that this is the wrong change to be making, and that in making
it we greatly reduce the likelihood of the right change coming later.
(Not saying it IS the wrong change, only that it's not clear it's the
right one.) Lack of sleep prevents me from getting into details at the
moment but I'll read anything anyone would like to share.

FWIW, PartialFunction as it currently exists is massively powerful for dynamically chaining patterns together.  The Scala Actors library is based on this.  Most of Lift is based on this.

I'm not advocating for isDefinedAt in Fuction1 or not, but the concept of a PartialFunction is so darned powerful and *should* exist and it's a powerful thing in Scala that it does exist.

On Mon, Nov 9, 2009 at 10:00 AM, Kris Nuttycombe <kris.nuttycombe@gmail.com> wrote:

On Mon, Nov 9, 2009 at 10:48 AM, Paul Phillips <paulp@improving.org> wrote:
> r19466:
>
>    Adds isDefinedAt to Function1. As a consequence,
>    code that mixes in PartialFunction now have to
>    define isDefinedAt as override.
>
> This is a big decision! A lot of people are interested in this issue.
> Was there a big discussion somewhere I missed or did this come
> completely out of nowhere (from the perspective of those of us not at
> LAMP) or...? I can't even see any basis for the distinction between
> Function1 and PartialFunction anymore.
>
> I especially can't understand this change coming at this moment, after
> changes being dismissed more than once in the past.  Aren't we trying to
> stabilize here?
>
> Don't get me wrong, I would dearly love to see the arrangement between
> these classes altered, but I don't understand the time, the change, or
> the absence of any discussion.
>
> --
> Paul Phillips      | The most dangerous man to any government is the man who
> Everyman           | is able to think things out [...] Almost inevitably he
> Empiricist         | comes to the conclusion that the government he lives under
> pp: i haul pills   | is dishonest, insane, intolerable.   -- H. L. Mencken
>

Whoah, what? Adding isDefinedAt to Function1 seems to me to be the
worst of all possible choices with respect to the insanity that
already rears its grotesque head in PartialFunction.

I mean, hell, PartialFunction shouldn't even exist! The compiler can
statically determine whether a match is a total function based upon
the sealedness of the type being matched on, right? Why can't match
simply generate A => B for exhaustive matches, and A => Option[B] for
incomplete matches, and everybody go along their merry way?

If somebody really, really wants to have their code error out instead
of behaving in a sane fashion, they can very well go ahead and write
implicit def theTypeIsALie[A,B](pf: A => Option[B]): A => B. May the
pain be with them.

I really, really wish that somebody would at least update
http://lampsvn.epfl.ch/trac/scala/ticket/85 with a description that is
more enlightening than "We decided not to." Seriously, PartialFunction
<: Function1 seems to me to be the most dangerously bug-prone feature
of Scala. Please axe it for 3.0. Pretty please with sugar on top!

</rant>

Kris

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

Come to think of it, I believe we have to revert the change to
Function1, and remove the isDefinedAt method. Why? The root cause is
that PartialFunction is a subtype of Function1. How can it be, and
still respect the LSP? Well, nothing in the contract of Function1 says
that it might not reject its argument due to a MatchError, or a failed
requires, or something else. So PartialFunction simply refines the
contract of Function1 in saying that it will tell via its isDefinedAt
method whether its apply method is prepared to give back a meaningful
result for the given argument. It's true that we have not specified
what this means exactly, but that's the fundamental idea of it.

Now, if we added an isDefinedAt to Function1, we'd need to let it
return "undefined", or "don't know", but these are not Boolean values.
In the proposed implementation we always return true. This is clearly
false because then the LSP would force every subclass including
PartialFunction to always return true also.

So, in summary, PartialFunction being a subclass of Function1 is fine,
but isDefinedAt has no place in Function1.

Cheers

On Mon, Nov 9, 2009 at 11:19 AM, David Pollak
wrote:
> FWIW, PartialFunction as it currently exists is massively powerful for
> dynamically chaining patterns together.  The Scala Actors library is based
> on this.  Most of Lift is based on this.

scala> def a(i: Int) = if (i < 10) Some(i.toString) else None
a: (Int)Option[java.lang.String]

scala> def b(i: Int) = if (i >= 10) Some((i * 2).toString) else None
b: (Int)Option[java.lang.String]

scala> implicit def chain[A,B](f: A => Option[B]) = new {
| def orElse(f2: A => Option[B]) = (a: A) => f(a).orElse(f2(a))
| }
chain: [A,B]((A) => Option[B])java.lang.Object{def orElse((A) =>
Option[B]): (A) => Option[B]}

scala> ((a _) orElse (b _))(5)
res0: Option[java.lang.String] = Some(5)

scala> ((a _) orElse (b _))(15)
res1: Option[java.lang.String] = Some(30)

> I'm not advocating for isDefinedAt in Fuction1 or not, but the concept of a
> PartialFunction is so darned powerful and *should* exist and it's a powerful
> thing in Scala that it does exist.
>
> On Mon, Nov 9, 2009 at 10:00 AM, Kris Nuttycombe
> wrote:
>>
>> On Mon, Nov 9, 2009 at 10:48 AM, Paul Phillips
>> wrote:
>> > r19466:
>> >
>> >    Adds isDefinedAt to Function1. As a consequence,
>> >    code that mixes in PartialFunction now have to
>> >    define isDefinedAt as override.
>> >
>> > This is a big decision! A lot of people are interested in this issue.
>> > Was there a big discussion somewhere I missed or did this come
>> > completely out of nowhere (from the perspective of those of us not at
>> > LAMP) or...? I can't even see any basis for the distinction between
>> > Function1 and PartialFunction anymore.
>> >
>> > I especially can't understand this change coming at this moment, after
>> > changes being dismissed more than once in the past.  Aren't we trying to
>> > stabilize here?
>> >
>> > Don't get me wrong, I would dearly love to see the arrangement between
>> > these classes altered, but I don't understand the time, the change, or
>> > the absence of any discussion.
>> >
>> > --
>> > Paul Phillips      | The most dangerous man to any government is the man
>> > who
>> > Everyman           | is able to think things out [...] Almost inevitably
>> > he
>> > Empiricist         | comes to the conclusion that the government he
>> > lives under
>> > pp: i haul pills   | is dishonest, insane, intolerable.   -- H. L.
>> > Mencken
>> >
>>
>> Whoah, what? Adding isDefinedAt to Function1 seems to me to be the
>> worst of all possible choices with respect to the insanity that
>> already rears its grotesque head in PartialFunction.
>>
>> I mean, hell, PartialFunction shouldn't even exist! The compiler can
>> statically determine whether a match is a total function based upon
>> the sealedness of the type being matched on, right? Why can't match
>> simply generate A => B for exhaustive matches, and A => Option[B] for
>> incomplete matches, and everybody go along their merry way?
>>
>> If somebody really, really wants to have their code error out instead
>> of behaving in a sane fashion, they can very well go ahead and write
>> implicit def theTypeIsALie[A,B](pf: A => Option[B]): A => B. May the
>> pain be with them.
>>
>> I really, really wish that somebody would at least update
>> http://lampsvn.epfl.ch/trac/scala/ticket/85 with a description that is
>> more enlightening than "We decided not to." Seriously, PartialFunction
>> <: Function1 seems to me to be the most dangerously bug-prone feature
>> of Scala. Please axe it for 3.0. Pretty please with sugar on top!
>>
>>
>>
>> Kris
>
>
>
> --
> Lift, the simply functional web framework http://liftweb.net
> Beginning Scala http://www.apress.com/book/view/1430219890
> Follow me: http://twitter.com/dpp
> Surf the harmonics
>

On Mon, Nov 09, 2009 at 10:19:57AM -0800, David Pollak wrote:
> FWIW, PartialFunction as it currently exists is massively powerful for
> dynamically chaining patterns together. The Scala Actors library is
> based on this. Most of Lift is based on this.

Verily, I did not speak a word against PartialFunction, who I love like
a son; and it is because of that love that I must agitate against
partial (irony!) measures which could haunt him for the rest of his
existence. Powerful he may be, but as a wise man once said: "if you
strike me down, I will become more powerful than you can possibly
imagine." He was talking about something else but you take his meaning.

<20091109184424.GA83132@wsip-70-165-250-23.lv.lv.cox.net> MIME-Version: 1.0 --_04c6a7df-98c3-4964-9006-57e1d42a731c_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable > Date: Mon=2C 9 Nov 2009 10:44:24 -0800 > From: paulp@improving.org > To: feeder.of.the.bears@gmail.com > CC: kris.nuttycombe@gmail.com=3B scala-internals@listes.epfl.ch > Subject: Re: [scala-internals] isDefinedAt introduction >=20 > As a wise man once said: "if you strike me down=2C I will become more pow= erful than you can possibly=20 > imagine." He was talking about something else but you take his meaning. He was also wrong=3B he disappeared and became a slightly irritating disemb= odied voice. Far better if he was still alive=3B he could have texted Luke: On attacking the death star=3B "Use the force=2C lol" On Hoth=3B "go 2 dagobah. cu l8r" On Dagobah (texting Yoda): "I wuz tw@t too=2C if u rmbr." Whilst still being able to wave his light-stick about =20 _________________________________________________________________ New Windows 7: Find the right PC for you. Learn more. http://www.microsoft.com/uk/windows/buy/= --_04c6a7df-98c3-4964-9006-57e1d42a731c_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable &gt=3B Date: Mon=2C 9 Nov 2009 10:44:24 -0800
&gt=3B From: paulp@improvi= ng.org
&gt=3B To: feeder.of.the.bears@gmail.com
&gt=3B CC: kris.nutty= combe@gmail.com=3B scala-internals@listes.epfl.ch
&gt=3B Subject: Re: [s= cala-internals] isDefinedAt introduction
&gt=3B
&gt=3B As a wise man= once said: "if you strike me down=2C I will become more powerful than you = can possibly
&gt=3B imagine." He was talking about something else but y= ou take his meaning.

He was also wrong=3B he disappeared and became = a slightly irritating disembodied voice. Far better if he was still alive= =3B he could have texted Luke:

On attacking the death star=3B "Use t= he force=2C lol"
On Hoth=3B "go 2 dagobah. cu l8r"
On Dagobah (textin= g Yoda): "I wuz tw@t too=2C if u rmbr."

Whilst still being able to w= ave his light-stick about


New Windows 7: Fin= d the right PC for you. Learn more. = --_04c6a7df-98c3-4964-9006-57e1d42a731c_--

martin odersky wrote:
> Come to think of it, I believe we have to revert the change to
> Function1, and remove the isDefinedAt method. [...]
>
> So, in summary, PartialFunction being a subclass of Function1 is fine,
> but isDefinedAt has no place in Function1.
>

I did that change today. I have no opinion on the merit of the matter, I
just implemented the change as per ticket #2225.

http://lampsvn.epfl.ch/trac/scala/ticket/2225

The ticket was submitted about three months ago. Tiark assigned the ticket
to Philipp. During a subsequent Scala meeting, on 22/09/2009, the issue was
briefly discussed, a decision was taken to implement it, and the ticket was
assigned to me. I do remember the specific moment during the meeting.

If the decision is now that Function1 should not contain atDefinedAt, I
will simply revert the change and mark the ticket as "wontfix".
Toni

Perhaps a better factoring would be to have an abstract class Function
and subclasses TotalFunction and PartialFunction. Function would have
abstract apply() and isDefinedAt() methods. Function literals would
evaluate to TotalFunctions and case expressions would, as today,
evaluate to PartialFunctions. The benefit is that the contract for
TotalFunction could explicitly say that apply() is not expected to
throw a MatchError.

This is probably too big a change to effect now, though.

By the way, I'm not 100% convinced that an isDefinedAt() in Function
would violate the LSP. The fact that the best implementation is to
always return true doesn't necessarily imply the contract should
guarantee the method always return true.

My perspective as an user is that in the past I have felt the need for
an isDefinedAt in PartialFunction, but it's been a long time and I
don't remember why. Not a big deal, anyway.

--
Rafael de F. Ferreira.
http://www.rafaelferreira.net/

On Mon, Nov 9, 2009 at 4:30 PM, martin odersky wrote:
> Come to think of it, I believe we have to revert the change to
> Function1, and remove the isDefinedAt method. Why? The root cause is
> that PartialFunction is a subtype of Function1. How can it be, and
> still respect the LSP? Well, nothing in the contract of Function1 says
> that it might not reject its argument due to a MatchError, or a failed
> requires, or something else. So PartialFunction simply refines the
> contract of Function1 in saying that it will tell via its isDefinedAt
> method whether its apply method is prepared to give back a meaningful
> result for the given argument. It's true that we have not specified
> what this means exactly, but that's the fundamental idea of it.
>
> Now, if we added an isDefinedAt to Function1, we'd need to let it
> return "undefined", or "don't know", but these are not Boolean values.
> In the proposed implementation we always return true. This is clearly
> false because then the LSP would force every subclass including
> PartialFunction to always return true also.
>
> So, in summary, PartialFunction being a subclass of Function1 is fine,
> but isDefinedAt has no place in Function1.
>
> Cheers
>
>  -- Martin
>

On Mon, Nov 9, 2009 at 7:30 PM, martin odersky <martin.odersky@epfl.ch> wrote:

Come to think of it, I believe we have to revert the change to
Function1, and remove the isDefinedAt method. Why? The root cause is
that PartialFunction is a subtype of Function1. How can it be, and
still respect the LSP? Well, nothing in the contract of Function1 says
that it might not reject its argument due to a MatchError, or a failed
requires, or something else. So PartialFunction simply refines the
contract of Function1 in saying that it will tell via its isDefinedAt
method whether its apply method is prepared to give back a meaningful
result for the given argument. It's true that we have not specified
what this means exactly, but that's the fundamental idea of it.

Now, if we added an isDefinedAt to Function1, we'd need to let it
return "undefined", or "don't know", but these are not Boolean values.
In the proposed implementation we always return true. This is clearly
false because then the LSP would force every subclass including
PartialFunction to always return true also.

I read that as 'no concrete method should ever be overridden', which sounds a bit too much. However, after having 'isDefinedAt' in Function1, I don't see the use of a distinct PartialFunction. What if patterns were translated to Function1, with the right 'isDefinedAt', and remove PartialFunction? Lots of people seem to agree, and 2.8 will break a lot of code, so this seems a good time to tackle it.

iulian

 

So, in summary, PartialFunction being a subclass of Function1 is fine,
but isDefinedAt has no place in Function1.

Cheers

On Tue, Nov 10, 2009 at 9:46 AM, Iulian Dragos wrote:
>
>
> On Mon, Nov 9, 2009 at 7:30 PM, martin odersky
> wrote:
>>
>> Come to think of it, I believe we have to revert the change to
>> Function1, and remove the isDefinedAt method. Why? The root cause is
>> that PartialFunction is a subtype of Function1. How can it be, and
>> still respect the LSP? Well, nothing in the contract of Function1 says
>> that it might not reject its argument due to a MatchError, or a failed
>> requires, or something else. So PartialFunction simply refines the
>> contract of Function1 in saying that it will tell via its isDefinedAt
>> method whether its apply method is prepared to give back a meaningful
>> result for the given argument. It's true that we have not specified
>> what this means exactly, but that's the fundamental idea of it.
>>
>> Now, if we added an isDefinedAt to Function1, we'd need to let it
>> return "undefined", or "don't know", but these are not Boolean values.
>> In the proposed implementation we always return true. This is clearly
>> false because then the LSP would force every subclass including
>> PartialFunction to always return true also.
>
> I read that as 'no concrete method should ever be overridden', which sounds
> a bit too much. However, after having 'isDefinedAt' in Function1, I don't
> see the use of a distinct PartialFunction. What if patterns were translated
> to Function1, with the right 'isDefinedAt', and remove PartialFunction? Lots
> of people seem to agree, and 2.8 will break a lot of code, so this seems a
> good time to tackle it.

Sounds like good theory (all functions are potentially partial), but
how does this stack up regarding efficiency? If "def isDefinedAt(...)
= true" can be optimised away to nothing then it feels like a good
approach to take.

> iulian
>
>
>>
>> So, in summary, PartialFunction being a subclass of Function1 is fine,
>> but isDefinedAt has no place in Function1.
>>
>> Cheers
>>
>>  -- Martin
>
>
>
> --
> « Je déteste la montagne, ça cache le paysage »
> Alphonse Allais
>

On Tue, Nov 10, 2009 at 10:46:37AM +0100, Iulian Dragos wrote:
> However, after having 'isDefinedAt' in Function1, I don't see the use
> of a distinct PartialFunction. What if patterns were translated to
> Function1, with the right 'isDefinedAt', and remove PartialFunction?

That is what I was thinking too. This is maybe not optimal from a
theoretical standpoint in that it moves a distinction out of the type
system and into a value, but I do dream of being able to create, pass,
and compose partial functions without so frequently being reminded by
the compiler that it maintains that distinction and really prefers
Function1.

It reminds me of orderedness among the collections. Why is this not in
the type system? I don't know, but my first guess would be that it was
more trouble than it was worth. And that's how it feels with PF too.

On Tue, Nov 10, 2009 at 10:05:28AM +0000, Kevin Wright wrote:
> Sounds like good theory (all functions are potentially partial), but
> how does this stack up regarding efficiency? If "def isDefinedAt(...)
> = true" can be optimised away to nothing then it feels like a good
> approach to take.

Or, final val isTotal = true, which ought to get to the same place. I
think efficiency of compiled code need not suffer, but compilation speed
could.

On Tue, Nov 10, 2009 at 3:37 PM, Paul Phillips wrote:
> On Tue, Nov 10, 2009 at 10:46:37AM +0100, Iulian Dragos wrote:
>> However, after having 'isDefinedAt' in Function1, I don't see the use
>> of a distinct PartialFunction. What if patterns were translated to
>> Function1, with the right 'isDefinedAt', and remove PartialFunction?
>
> That is what I was thinking too.  This is maybe not optimal from a
> theoretical standpoint in that it moves a distinction out of the type
> system and into a value, but I do dream of being able to create, pass,
> and compose partial functions without so frequently being reminded by
> the compiler that it maintains that distinction and really prefers
> Function1.

It comes at a performance cost, though. Right now, every pattern
matching closure needs to be compiled twice if the expected type is a
partial function. The proposed change would make this happen for all
pattren matching closures. That could be a heavy price to pay. So we
decided against changing anything.
>
> It reminds me of orderedness among the collections.  Why is this not in
> the type system? I don't know, but my first guess would be that it was
> more trouble than it was worth.

Yes eactly.

Cheers

On Tue, Nov 10, 2009 at 2:44 PM, martin odersky wrote:
> On Tue, Nov 10, 2009 at 3:37 PM, Paul Phillips wrote:
>> On Tue, Nov 10, 2009 at 10:46:37AM +0100, Iulian Dragos wrote:
>>> However, after having 'isDefinedAt' in Function1, I don't see the use
>>> of a distinct PartialFunction. What if patterns were translated to
>>> Function1, with the right 'isDefinedAt', and remove PartialFunction?
>>
>> That is what I was thinking too.  This is maybe not optimal from a
>> theoretical standpoint in that it moves a distinction out of the type
>> system and into a value, but I do dream of being able to create, pass,
>> and compose partial functions without so frequently being reminded by
>> the compiler that it maintains that distinction and really prefers
>> Function1.
>
> It comes at a performance cost, though. Right now, every pattern
> matching closure needs to be compiled twice if the expected type is a
> partial function. The proposed change would make this happen for all
> pattren matching closures. That could be a heavy price to pay. So we
> decided against changing anything.

I was kind of hoping there might be a few shortcuts available in the
realm of sealed classes and suchlike, at least enough to make this
significantly cheaper in the majority of cases :)

>> It reminds me of orderedness among the collections.  Why is this not in
>> the type system? I don't know, but my first guess would be that it was
>> more trouble than it was worth.
>
> Yes eactly.
>
> Cheers
>
>  -- Martin
>

The compiler can already guess for many cases when a match is partial or complete. It could optimize isDefinedAt to just return true in such cases.   I'm mildly more concerned about the warnings about incomplete matches, but a single annotation silences that.
On Tue, Nov 10, 2009 at 12:44 PM, martin odersky <martin.odersky@epfl.ch> wrote:

On Tue, Nov 10, 2009 at 3:37 PM, Paul Phillips <paulp@improving.org> wrote:
> On Tue, Nov 10, 2009 at 10:46:37AM +0100, Iulian Dragos wrote:
>> However, after having 'isDefinedAt' in Function1, I don't see the use
>> of a distinct PartialFunction. What if patterns were translated to
>> Function1, with the right 'isDefinedAt', and remove PartialFunction?
>
> That is what I was thinking too.  This is maybe not optimal from a
> theoretical standpoint in that it moves a distinction out of the type
> system and into a value, but I do dream of being able to create, pass,
> and compose partial functions without so frequently being reminded by
> the compiler that it maintains that distinction and really prefers
> Function1.

It comes at a performance cost, though. Right now, every pattern
matching closure needs to be compiled twice if the expected type is a
partial function. The proposed change would make this happen for all
pattren matching closures. That could be a heavy price to pay. So we
decided against changing anything.
>
> It reminds me of orderedness among the collections.  Why is this not in
> the type system? I don't know, but my first guess would be that it was
> more trouble than it was worth.

Yes eactly.

Cheers

 -- Martin

--
Daniel C. Sobral

Veni, vidi, veterni.