This page is no longer maintained — Please continue to the home page at www.scala-lang.org

Re: Traversable and foldRight/reduceRight/dropRight/takeRight

16 replies
Jorge Ortiz
Joined: 2008-12-16,
User offline. Last seen 29 weeks 4 days ago.
On Thu, May 14, 2009 at 5:04 AM, martin odersky <martin.odersky@epfl.ch> wrote:
On Thu, May 14, 2009 at 1:10 PM, Jorge Ortiz <jorge.ortiz@gmail.com> wrote:
> While on the subject of consistency and other API issues, I'm going to air
> out all the dirty laundry I've found so far:
>
> 1) What's the deal with reduceLeftOpt/reduceRightOpt vs
> firstOption/tailOption? I'm not really happy with either naming convention,
> but we should pick one and stick to it.

Agreed. Which one should it be? I believe Option has been around for
2.7, so probably pick that (I also do not care for either operation).

> 2) I count 14 uses of breakable/break. Of those, 1 is strictly unnecessary,

Which one?

In TraversableTemplate.scala, look for def size.

> and the rest can be implemented with just "return". Now, I know both break
> and return compile down to throwing an exception, but at least return is
> statically checked, guaranteed to be caught, and much harder to throw by
> mistake.

Yes, but break should be more efficient, and that's what counts most here.

How so? Does return not suppress the stack trace? Or is it just that the exception isn't pre-allocated?

> If someone should, for example, call break (and forget to catch it)
> inside the function passed to "find", their program might be very subtly
> broken and they'd never realize it. It'd be much harder to accidentally call
> return than call break.
>
This could be solved by having a collection-private version of break.
But I am not sure it's worth it. It might be. But users that let break
escape in first-class functions are inviting disaster, no matter what.

All the more reason that library code shouldn't hide it.

> 3) What's the difference between Array.concat and TraversableFactory#concat?
> Can we get rid of the one on Array (object)?
>
Array.concat is more efficient because it can pre-allocate the array
after computing the right lengths. I think there's no harm leaving
both versions in. There are several other places where
we have overloaded specializations of a method which take specialized
parameter types.

> 4) What's this Iterable.toOld/fromOld business? I imagine it was there from
> transition purposes. Is it obsolete now?
>
Yes, I just removed them.

> 5) Will the new @specialized annotation be used? Everywhere or just
> judiciously? For example, there's a few methods on Array (object) that seem
> to be prime candidates.
>
That will take some more experimentation.

> 6) There's a few comments in which there's an autor/author typo.
>
OK, I'll go chase them

Thanks for the feedback!

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

>> This could be solved by having a collection-private version of break.
>> But I am not sure it's worth it. It might be. But users that let break
>> escape in first-class functions are inviting disaster, no matter what.
>
> All the more reason that library code shouldn't hide it.
>
I just thought we can actually provide the same functionality to the
user. Just make Breaks a class with an instance of itself as companion
object:

class Breaks {

private val breakException = new BreakException

/** A block from which one can exit with a `break''. */
def breakable(op: => Unit) {
try {
op
} catch {
case ex: BreakException =>
if (ex ne breakException) throw ex
}
}

/* Break from closest enclosing breakable block */
def break { throw breakException }
}

/** A singleton object providing the Break functionality */
object Breaks extends Breaks

That way, everyone can create new instance of Breaks, and breaks from
one Breaks instance can never be caught by breakables from another.
Neat, no? Now, I have deserved a break :-)

Cheers

DRMacIver
Joined: 2008-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

2009/5/14 Paul Phillips :
> On Thu, May 14, 2009 at 02:04:00PM +0200, martin odersky wrote:
>> > 1) What's the deal with reduceLeftOpt/reduceRightOpt vs
>> > firstOption/tailOption? I'm not really happy with either naming convention,
>> > but we should pick one and stick to it.
>>
>> Agreed. Which one should it be? I believe Option has been around for
>> 2.7, so probably pick that (I also do not care for either operation).
>
> I think the temptation to add Opt/Option methods should be resisted with
> great prejudice.  It has to be antipattern.

Very strongly agreed.

DRMacIver
Joined: 2008-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

2009/5/15 Miles Sabin :
> On Thu, May 14, 2009 at 7:50 PM, Paul Phillips wrote:
>>  I have def f(s: String) = s.toInt and it is
>>    Function1[String,Int]
>>
>>  Generalizing across the unexceptional Exception, it's actually
>>    Function1[String,Either[NumberFormatException,Int]]
>>
>>  Ungeneralizing to the way I almost always want to use it, it's
>>    Function1[String,Option[Int]]
>>
>> What I am hungry for is the lightest possible syntax that allows me all
>> those possibilities depending on what the occasion warrants.
>
> At the risk of stating the obvious and wandering off topic, the
> following _almost_ works,
>
>  def capture[X](implicit m : Manifest[X]) = new {
>    def apply[T](b : => T) : Either[X, T] = try {
>      Right(b)
>    } catch {
>      case e if m.erasure.isInstance(e) => Left(e.asInstanceOf[X])
>    }
>  }
>
> Bending reality for a moment this would enable,
>
>  capture[NumberFormatException]("23".toInt)
>
> for your second case and,
>
>  capture[NumberFormatException]("23".toInt).right
>
> for the third.
>
> Unfortunately reality doesn't bend so easily and what we actually get is,
>
>  scala> capture[NumberFormatException]("23".toInt)
>  :7: error: type mismatch;
>   found   : Int
>   required: scala.reflect.Manifest[NumberFormatException]
>         capture[NumberFormatException]("23".toInt)
>
> because the initial implicit argument list isn't discharged before the
> compiler runs into the second.
>
> Is there anything we could do about that for 2.8.0?

Really the problem here is that you're trying to use object creation
to mimic curried type parameters. Bending reality slightly further,
what you really want here is

def capture[Ex][T](t : =>T)(implicit mf : Manifest[Ex]) : Either[Ex, T] = ...

So you can write

capture[NumberFormatException]("23".toInt)

and have the second parameter inferred.

I'm not sure what to suggest in terms of things that would be
reasonable for making this work for 2.8 (There's already enough
important stuff that probably isn't making it into 2.8 that working on
very non essentials like this seems a dubious choice).

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

On Fri, May 15, 2009 at 11:01 AM, David MacIver wrote:
> Really the problem here is that you're trying to use object creation
> to mimic curried type parameters.

Oh, yes ... absolutely.

> Bending reality slightly further, what you really want here is
>
> def capture[Ex][T](t : =>T)(implicit mf : Manifest[Ex]) : Either[Ex, T] = ...

Actually, what I really want is,

def capture[X](implicit m : Manifest[X]) = [T](b : => T) { ... }

(which is what I'm encoding with the object and apply). But I agree,
that might be asking a bit much ...

Cheers,

Miles

milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

On Fri, May 15, 2009 at 11:25 AM, David MacIver wrote:
> 2009/5/15 Miles Sabin :
>> Actually, what I really want is,
>>
>>  def capture[X](implicit m : Manifest[X]) = [T](b : => T) { ... }
>>
>> (which is what I'm encoding with the object and apply). But I agree,
>> that might be asking a bit much ...
>
> The problem is one of disambiguation. I don't see how you can
> reasonably expect capture[Foo](bar) to know that you want to apply bar
> in the second argument list rather than the first. It's reasonably
> unambiguous here (what if you apply capture[T] to something that
> returns a Manifest[T] though?), but in general it can't be.

Sure, I understand that.

Nevertheless, implicit manifests are an important special case, I
think important enough to tweak things to disambiguate in this and
similar cases.

Perhaps we need something like a "non-greedy" implicit which never
consumes an argument list. Reusing "final" as a modifier,

def capture[X](final implicit m : Manifest[X]) = ...

Obviously this now makes it hard to supply a manifest _ex_plicitly,
but with manifests I think that's something you would very rarely want
to do.

Cheers,

Miles

Jorge Ortiz
Joined: 2008-12-16,
User offline. Last seen 29 weeks 4 days ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight
Or, to go crazy, interleave type arguments with parameters:

  def capture[T](b: => T)[X](implicit m: Manifest[X]): Either[X, T] = {
    try {
      Right(b)
    } catch {
      case e if m.erasure.isInstance(e) => Left(e.asInstanceOf[X])
    }
  }

Used like:

  caputre("23".toInt)[NumberFormatException]

Ok, I'm done with crazy ideas for the day.

--j

On Fri, May 15, 2009 at 10:19 AM, Jorge Ortiz <jorge.ortiz@gmail.com> wrote:
I might be suggesting an abomination, but... perhaps multiple type argument lists, like multiple parameter lists?

  def capture[X][T](b: => T)(implicit m: Manifest[X]): Either[X, T] = {
    try {
      Right(b)
    } catch {
      case e if m.erasure.isInstance(e) => Left(e.asInstanceOf[X])
    }
  }

Such that [X] can be manually specified but [T] can be inferred. Used like:

  capture[NumberFormatException]("23".toInt)

--j

On Fri, May 15, 2009 at 2:46 AM, Miles Sabin <miles@milessabin.com> wrote:
On Thu, May 14, 2009 at 7:50 PM, Paul Phillips <paulp@improving.org> wrote:
>  I have def f(s: String) = s.toInt and it is
>    Function1[String,Int]
>
>  Generalizing across the unexceptional Exception, it's actually
>    Function1[String,Either[NumberFormatException,Int]]
>
>  Ungeneralizing to the way I almost always want to use it, it's
>    Function1[String,Option[Int]]
>
> What I am hungry for is the lightest possible syntax that allows me all
> those possibilities depending on what the occasion warrants.

At the risk of stating the obvious and wandering off topic, the
following _almost_ works,

 def capture[X](implicit m : Manifest[X]) = new {
   def apply[T](b : => T) : Either[X, T] = try {
     Right(b)
   } catch {
     case e if m.erasure.isInstance(e) => Left(e.asInstanceOf[X])
   }
 }

Bending reality for a moment this would enable,

 capture[NumberFormatException]("23".toInt)

for your second case and,

 capture[NumberFormatException]("23".toInt).right

for the third.

Unfortunately reality doesn't bend so easily and what we actually get is,

 scala> capture[NumberFormatException]("23".toInt)
 <console>:7: error: type mismatch;
  found   : Int
  required: scala.reflect.Manifest[NumberFormatException]
        capture[NumberFormatException]("23".toInt)

because the initial implicit argument list isn't discharged before the
compiler runs into the second.

Is there anything we could do about that for 2.8.0?

Cheers,


Miles

--
Miles Sabin
tel: +44 (0)7813 944 528
skype:  milessabin
http://twitter.com/milessabin


Jorge Ortiz
Joined: 2008-12-16,
User offline. Last seen 29 weeks 4 days ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight
I've found myself wanting this feature in the past. I can't remember what I was trying to do, so I must have found a workaround or given up, but making type parameter inference an all-or-nothing proposition can be pretty limiting.

--j

On Fri, May 15, 2009 at 10:23 AM, martin odersky <martin.odersky@epfl.ch> wrote:
On Fri, May 15, 2009 at 7:19 PM, Jorge Ortiz <jorge.ortiz@gmail.com> wrote:
> I might be suggesting an abomination, but... perhaps multiple type argument
> lists, like multiple parameter lists?
>
>   def capture[X][T](b: => T)(implicit m: Manifest[X]): Either[X, T] = {
>     try {
>       Right(b)
>     } catch {
>       case e if m.erasure.isInstance(e) => Left(e.asInstanceOf[X])
>     }
>   }
>
> Such that [X] can be manually specified but [T] can be inferred. Used like:
>
>   capture[NumberFormatException]("23".toInt)
>
I believe the different between these two forms:

capture[NumberFormatException]("23".toInt)
try { Some(23.toInt) } catch { case _: NumberFormatException => None }

is not that great. Do we really want to put elaborate mechansims in
place to bridge it?

Cheers

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

On Fri, May 15, 2009 at 7:44 PM, Jorge Ortiz wrote:
> I've found myself wanting this feature in the past. I can't remember what I
> was trying to do, so I must have found a workaround or given up, but making
> type parameter inference an all-or-nothing proposition can be pretty
> limiting.

Yes, I agree that partial type parameter lists have many other uses.
That's not what I meant
in my examples.

Cheers

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

I think if we want to do this we should strive to make rather simple,
if necessary at the expense of generality. One possibility would be to
introduce an orElse operator
by implicit conversions:

expr orElse alt

gives expr, and if that throws an exception, alt.

optional(expr)

would then be the same as

Some(expr) orElse None

The existing orElse operator in Option would have to be adapted to
mask exceptions as well.

It's certainly very powerful. I am not sure whether it's a good idea
to sweep exceptions under the carpet like that, though. But once we
start with masking speciifc exceptions we can just fall back to
``try'', I think.

Cheers

DRMacIver
Joined: 2008-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

2009/5/15 Miles Sabin :
> On Fri, May 15, 2009 at 11:25 AM, David MacIver wrote:
>> 2009/5/15 Miles Sabin :
>>> Actually, what I really want is,
>>>
>>>  def capture[X](implicit m : Manifest[X]) = [T](b : => T) { ... }
>>>
>>> (which is what I'm encoding with the object and apply). But I agree,
>>> that might be asking a bit much ...
>>
>> The problem is one of disambiguation. I don't see how you can
>> reasonably expect capture[Foo](bar) to know that you want to apply bar
>> in the second argument list rather than the first. It's reasonably
>> unambiguous here (what if you apply capture[T] to something that
>> returns a Manifest[T] though?), but in general it can't be.
>
> Sure, I understand that.
>
> Nevertheless, implicit manifests are an important special case, I
> think important enough to tweak things to disambiguate in this and
> similar cases.
>
> Perhaps we need something like a "non-greedy" implicit which never
> consumes an argument list. Reusing "final" as a modifier,
>
>  def capture[X](final implicit m : Manifest[X]) = ...
>
> Obviously this now makes it hard to supply a manifest _ex_plicitly,
> but with manifests I think that's something you would very rarely want
> to do.

Please file a SIP. ;-)

Jorge Ortiz
Joined: 2008-12-16,
User offline. Last seen 29 weeks 4 days ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight
I might be suggesting an abomination, but... perhaps multiple type argument lists, like multiple parameter lists?

  def capture[X][T](b: => T)(implicit m: Manifest[X]): Either[X, T] = {
    try {
      Right(b)
    } catch {
      case e if m.erasure.isInstance(e) => Left(e.asInstanceOf[X])
    }
  }

Such that [X] can be manually specified but [T] can be inferred. Used like:

  capture[NumberFormatException]("23".toInt)

--j

On Fri, May 15, 2009 at 2:46 AM, Miles Sabin <miles@milessabin.com> wrote:
On Thu, May 14, 2009 at 7:50 PM, Paul Phillips <paulp@improving.org> wrote:
>  I have def f(s: String) = s.toInt and it is
>    Function1[String,Int]
>
>  Generalizing across the unexceptional Exception, it's actually
>    Function1[String,Either[NumberFormatException,Int]]
>
>  Ungeneralizing to the way I almost always want to use it, it's
>    Function1[String,Option[Int]]
>
> What I am hungry for is the lightest possible syntax that allows me all
> those possibilities depending on what the occasion warrants.

At the risk of stating the obvious and wandering off topic, the
following _almost_ works,

 def capture[X](implicit m : Manifest[X]) = new {
   def apply[T](b : => T) : Either[X, T] = try {
     Right(b)
   } catch {
     case e if m.erasure.isInstance(e) => Left(e.asInstanceOf[X])
   }
 }

Bending reality for a moment this would enable,

 capture[NumberFormatException]("23".toInt)

for your second case and,

 capture[NumberFormatException]("23".toInt).right

for the third.

Unfortunately reality doesn't bend so easily and what we actually get is,

 scala> capture[NumberFormatException]("23".toInt)
 <console>:7: error: type mismatch;
  found   : Int
  required: scala.reflect.Manifest[NumberFormatException]
        capture[NumberFormatException]("23".toInt)

because the initial implicit argument list isn't discharged before the
compiler runs into the second.

Is there anything we could do about that for 2.8.0?

Cheers,


Miles

--
Miles Sabin
tel: +44 (0)7813 944 528
skype:  milessabin
http://twitter.com/milessabin

extempore
Joined: 2008-12-17,
User offline. Last seen 35 weeks 3 days ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

I am going to focus on seeing how far I can go with exception
generalization and whether that introduces any performance or other
issues. I suspect that determining the approach for wrapping common
annoying exceptions is best left until that constraint is defined.

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

On Thu, May 14, 2009 at 5:20 PM, Jorge Ortiz wrote:
> On Thu, May 14, 2009 at 5:04 AM, martin odersky
> wrote:
>>
>> On Thu, May 14, 2009 at 1:10 PM, Jorge Ortiz
>> wrote:
>> > While on the subject of consistency and other API issues, I'm going to
>> > air
>> > out all the dirty laundry I've found so far:
>> >
>> > 1) What's the deal with reduceLeftOpt/reduceRightOpt vs
>> > firstOption/tailOption? I'm not really happy with either naming
>> > convention,
>> > but we should pick one and stick to it.
>>
>> Agreed. Which one should it be? I believe Option has been around for
>> 2.7, so probably pick that (I also do not care for either operation).
>>
>> > 2) I count 14 uses of breakable/break. Of those, 1 is strictly
>> > unnecessary,
>>
>> Which one?
>
> In TraversableTemplate.scala, look for def size.

Got it, thanks!
>
>> > and the rest can be implemented with just "return". Now, I know both
>> > break
>> > and return compile down to throwing an exception, but at least return is
>> > statically checked, guaranteed to be caught, and much harder to throw by
>> > mistake.
>>
>> Yes, but break should be more efficient, and that's what counts most here.
>
> How so? Does return not suppress the stack trace? Or is it just that the
> exception isn't pre-allocated?
>
The latter.

>> > If someone should, for example, call break (and forget to catch it)
>> > inside the function passed to "find", their program might be very subtly
>> > broken and they'd never realize it. It'd be much harder to accidentally
>> > call
>> > return than call break.
>> >
>> This could be solved by having a collection-private version of break.
>> But I am not sure it's worth it. It might be. But users that let break
>> escape in first-class functions are inviting disaster, no matter what.
>
> All the more reason that library code shouldn't hide it.

Yeah. I guess it can't hurt to provide our own breakables for collections.

Cheers

DRMacIver
Joined: 2008-09-02,
User offline. Last seen 42 years 45 weeks ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

2009/5/15 Miles Sabin :
> Actually, what I really want is,
>
>  def capture[X](implicit m : Manifest[X]) = [T](b : => T) { ... }
>
> (which is what I'm encoding with the object and apply). But I agree,
> that might be asking a bit much ...

The problem is one of disambiguation. I don't see how you can
reasonably expect capture[Foo](bar) to know that you want to apply bar
in the second argument list rather than the first. It's reasonably
unambiguous here (what if you apply capture[T] to something that
returns a Manifest[T] though?), but in general it can't be.

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

On Fri, May 15, 2009 at 7:19 PM, Jorge Ortiz wrote:
> I might be suggesting an abomination, but... perhaps multiple type argument
> lists, like multiple parameter lists?
>
>   def capture[X][T](b: => T)(implicit m: Manifest[X]): Either[X, T] = {
>     try {
>       Right(b)
>     } catch {
>       case e if m.erasure.isInstance(e) => Left(e.asInstanceOf[X])
>     }
>   }
>
> Such that [X] can be manually specified but [T] can be inferred. Used like:
>
>   capture[NumberFormatException]("23".toInt)
>
I believe the different between these two forms:

capture[NumberFormatException]("23".toInt)
try { Some(23.toInt) } catch { case _: NumberFormatException => None }

is not that great. Do we really want to put elaborate mechansims in
place to bridge it?

Cheers

odersky
Joined: 2008-07-29,
User offline. Last seen 45 weeks 6 days ago.
Re: Traversable and foldRight/reduceRight/dropRight/takeRight

On Sat, May 16, 2009 at 5:45 PM, Paul Phillips wrote:
> I am going to focus on seeing how far I can go with exception
> generalization and whether that introduces any performance or other
> issues.  I suspect that determining the approach for wrapping common
> annoying exceptions is best left until that constraint is defined.
>
Yes, good idea!

Cheers

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland