flatten vs. flatMap(x => x)

On Tue, Jun 30, 2009 at 5:44 PM, Paul Phillips wrote:
> I have these two lines in the pattern matcher.  The variables nrows and
> frows are lists of Options.
>
> -      val succRep = makeSuccRep(vs, tail, nrows.flatMap(x => x))
> -      val failRep = rep.make(scrut mkList rest.temp, frows.flatMap(x => x))
>
> If I replace them with these two lines, which should (?) be equivalent:
>
> +      val succRep = makeSuccRep(vs, tail, nrows.flatten)
> +      val failRep = rep.make(scrut mkList rest.temp, frows.flatten)
>
> Then scalac breaks trying to compile it.  (To be clear, the compiler
> which is crashing is standard trunk, not a compiler built with those two
> lines changed.)
>
It could be that the attribution of implicits is not correct here. Or
it might have to do with named arguments.

To find out more:
- have a look what implicit gets passed.
- try to pass the same implicit (i think it's just Predef.identity)
with the same type arguments manually -- does the problem persist?
- If yes, it could have to do with named arguments. Lukas has to do something
there to avoid renaming of names in method types. He should be able to help.
(Lukas, did you check the change already in which you showed me in
lambdalift?
If no, Paul should try again with the change in place).

Cheers

On Tue, Jun 30, 2009 at 06:49:26PM +0200, martin odersky wrote:
> - have a look what implicit gets passed.
> - try to pass the same implicit (i think it's just Predef.identity)
> with the same type arguments manually -- does the problem persist?

You are right. The crash goes away if I specify:

nrows.flatten[Row](Predef.identity)

It looks like it is instead passing Option.option2Iterable:

nrows.flatten[ParallelMatching.this.Row]({
((xo: Option[ParallelMatching.this.Row]) =>
scala.this.Option.option2Iterable[ParallelMatching.this.Row](xo)

That eventually expands to the implicit conversion and then the crash.

However if I pass option2Iterable explicitly instead of identity, it
doesn't crash.

On Tue, Jun 30, 2009 at 7:27 PM, Paul Phillips wrote:
> On Tue, Jun 30, 2009 at 06:49:26PM +0200, martin odersky wrote:
>>  - have a look what implicit gets passed.
>>  - try to pass the same implicit (i think it's just Predef.identity)
>> with the same type arguments manually -- does the problem persist?
>
> You are right.  The crash goes away if I specify:
>
>  nrows.flatten[Row](Predef.identity)
>
> It looks like it is instead passing Option.option2Iterable:
>
>  nrows.flatten[ParallelMatching.this.Row]({
>  ((xo: Option[ParallelMatching.this.Row]) =>
>  scala.this.Option.option2Iterable[ParallelMatching.this.Row](xo)
>
Is nrows then of type Option? We probably should add a flatten method
to option as well, but that's a different issue.

The tree you have shown misses some closing braces, so I was not
exactly sure what goes on. But anyway, one thing to do is print the
tree after typer with -Xprint-types on, once with implicit argument
once with explicit. To see more, also turn -uniqid on. There must be a
difference somewhere between the two trees, and that shpould tell us
what went wrong.

Cheers

On Tue, Jun 30, 2009 at 08:38:17PM +0200, martin odersky wrote:
> Is nrows then of type Option?

nrows is List[Option[Row]], with flatten/flatMap on it yielding List[Row].

Attempting to reduce the voluminous output to the relevant without
overshooting, here is what the crashing flatten version looks like after
typer and the noncrashing flatMap(x => x) version. Most obvious
difference is the BuilderFactory. If I'm going to spend much time
looking at this kind of output I had better write a pretty printer...

flatten{[B]
(implicit toTraversable: (Option[Row]) => Traversable[B])List[B]}
[Row]{(implicit toTraversable: (Option[Row]) => Traversable[Row])List[Row]}
({((xo: Option[Row]) => scala.this{type}.Option{object Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
[Row]{(xo: Option[Row])Iterable[Row]}
(xo{Option[Row]}){Iterable[Row]})
{(Option[Row]) => Iterable[Row]}}
{(Option[Row]) => Iterable[Row]}){List[Row]})
{(implicit theOwner: Symbol)Rep}
(theOwner{Symbol}){Rep};

flatMap{[B,That](f: (Option[Row]) => Traversable[B])
(implicit bf: BuilderFactory[B,That,List[Option[Row]]])That}
[Row, List[Row]]{(f: (Option[Row]) => Traversable[Row])(implicit bf: BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
(((x: Option[Row]) => scala.this{type}.Option{object Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
[Row]{(xo: Option[Row])Iterable[Row]}
(x{Option[Row]}){Iterable[Row]})
{(Option[Row]) => Iterable[Row]})
{(implicit bf: BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
(immutable.this{immutable.type}.List{object List}.builderFactory{[A]BuilderFactory[A,List[A],List#Coll]}[Row]
{BuilderFactory[Row,List[Row],List#Coll]}){List[Row]})
{(implicit theOwner: Symbol)Rep}
(theOwner{Symbol}){Rep};

On Tue, Jun 30, 2009 at 9:36 PM, Paul Phillips wrote:
> On Tue, Jun 30, 2009 at 08:38:17PM +0200, martin odersky wrote:
>> Is nrows then of type Option?
>
> nrows is List[Option[Row]], with flatten/flatMap on it yielding List[Row].
>
> Attempting to reduce the voluminous output to the relevant without
> overshooting, here is what the crashing flatten version looks like after
> typer and the noncrashing flatMap(x => x) version.  Most obvious
> difference is the BuilderFactory.  If I'm going to spend much time
> looking at this kind of output I had better write a pretty printer...
>
> flatten{[B]
>  (implicit toTraversable: (Option[Row]) => Traversable[B])List[B]}
>  [Row]{(implicit toTraversable: (Option[Row]) => Traversable[Row])List[Row]}
>  ({((xo: Option[Row]) => scala.this{type}.Option{object Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
>  [Row]{(xo: Option[Row])Iterable[Row]}
>  (xo{Option[Row]}){Iterable[Row]})
>  {(Option[Row]) => Iterable[Row]}}
>  {(Option[Row]) => Iterable[Row]}){List[Row]})
>  {(implicit theOwner: Symbol)Rep}
>  (theOwner{Symbol}){Rep};
>
> flatMap{[B,That](f: (Option[Row]) => Traversable[B])
>  (implicit bf: BuilderFactory[B,That,List[Option[Row]]])That}
>  [Row, List[Row]]{(f: (Option[Row]) => Traversable[Row])(implicit bf: BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
>  (((x: Option[Row]) => scala.this{type}.Option{object Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
>  [Row]{(xo: Option[Row])Iterable[Row]}
>  (x{Option[Row]}){Iterable[Row]})
>  {(Option[Row]) => Iterable[Row]})
>  {(implicit bf: BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
>  (immutable.this{immutable.type}.List{object List}.builderFactory{[A]BuilderFactory[A,List[A],List#Coll]}[Row]
>  {BuilderFactory[Row,List[Row],List#Coll]}){List[Row]})
>  {(implicit theOwner: Symbol)Rep}
>  (theOwner{Symbol}){Rep};
>
That's indeed hard to see anything here. But what I meant, rather,
was, can you take the same call, once with implicit arguments, once
with exactly the same arguments (including type arguments) supplied
explicitly. You wrote one of them failed the other didn't. If that's
the case we must see some difference in tree attribution at the end of
pahse typer. That would give us the clue what goes wrong.

Cheers

Can we just make Option formally an Iterable? There's implicit conversions in both directions, so I don't see why it isn't other than to cause implicit and typer headaches.

--j

On Tue, Jun 30, 2009 at 12:36 PM, Paul Phillips <paulp@improving.org> wrote:

On Tue, Jun 30, 2009 at 08:38:17PM +0200, martin odersky wrote:
> Is nrows then of type Option?

nrows is List[Option[Row]], with flatten/flatMap on it yielding List[Row].

Attempting to reduce the voluminous output to the relevant without
overshooting, here is what the crashing flatten version looks like after
typer and the noncrashing flatMap(x => x) version.  Most obvious
difference is the BuilderFactory.  If I'm going to spend much time
looking at this kind of output I had better write a pretty printer...

flatten{[B]
 (implicit toTraversable: (Option[Row]) => Traversable[B])List[B]}
 [Row]{(implicit toTraversable: (Option[Row]) => Traversable[Row])List[Row]}
 ({((xo: Option[Row]) => scala.this{type}.Option{object Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
 [Row]{(xo: Option[Row])Iterable[Row]}
 (xo{Option[Row]}){Iterable[Row]})
 {(Option[Row]) => Iterable[Row]}}
 {(Option[Row]) => Iterable[Row]}){List[Row]})
 {(implicit theOwner: Symbol)Rep}
 (theOwner{Symbol}){Rep};

flatMap{[B,That](f: (Option[Row]) => Traversable[B])
 (implicit bf: BuilderFactory[B,That,List[Option[Row]]])That}
 [Row, List[Row]]{(f: (Option[Row]) => Traversable[Row])(implicit bf: BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
 (((x: Option[Row]) => scala.this{type}.Option{object Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
 [Row]{(xo: Option[Row])Iterable[Row]}
 (x{Option[Row]}){Iterable[Row]})
 {(Option[Row]) => Iterable[Row]})
 {(implicit bf: BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
 (immutable.this{immutable.type}.List{object List}.builderFactory{[A]BuilderFactory[A,List[A],List#Coll]}[Row]
 {BuilderFactory[Row,List[Row],List#Coll]}){List[Row]})
 {(implicit theOwner: Symbol)Rep}
 (theOwner{Symbol}){Rep};

--
Paul Phillips      | Every election is a sort of advance auction sale
Stickler           | of stolen goods.
Empiricist         |     -- H. L. Mencken
slap pi uphill!    |----------* http://www.improving.org/paulp/ *----------

On Thu, Jul 2, 2009 at 1:32 AM, Jorge Ortiz <jorge.ortiz@gmail.com> wrote:

Can we just make Option formally an Iterable? There's implicit conversions in both directions, so I don't see why it isn't other than to cause implicit and typer headaches.

+1
 

--j

On Tue, Jun 30, 2009 at 12:36 PM, Paul Phillips <paulp@improving.org> wrote:

On Tue, Jun 30, 2009 at 08:38:17PM +0200, martin odersky wrote:
> Is nrows then of type Option?

nrows is List[Option[Row]], with flatten/flatMap on it yielding List[Row].

Attempting to reduce the voluminous output to the relevant without
overshooting, here is what the crashing flatten version looks like after
typer and the noncrashing flatMap(x => x) version.  Most obvious
difference is the BuilderFactory.  If I'm going to spend much time
looking at this kind of output I had better write a pretty printer...

flatten{[B]
 (implicit toTraversable: (Option[Row]) => Traversable[B])List[B]}
 [Row]{(implicit toTraversable: (Option[Row]) => Traversable[Row])List[Row]}
 ({((xo: Option[Row]) => scala.this{type}.Option{object Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
 [Row]{(xo: Option[Row])Iterable[Row]}
 (xo{Option[Row]}){Iterable[Row]})
 {(Option[Row]) => Iterable[Row]}}
 {(Option[Row]) => Iterable[Row]}){List[Row]})
 {(implicit theOwner: Symbol)Rep}
 (theOwner{Symbol}){Rep};

flatMap{[B,That](f: (Option[Row]) => Traversable[B])
 (implicit bf: BuilderFactory[B,That,List[Option[Row]]])That}
 [Row, List[Row]]{(f: (Option[Row]) => Traversable[Row])(implicit bf: BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
 (((x: Option[Row]) => scala.this{type}.Option{object Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
 [Row]{(xo: Option[Row])Iterable[Row]}
 (x{Option[Row]}){Iterable[Row]})
 {(Option[Row]) => Iterable[Row]})
 {(implicit bf: BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
 (immutable.this{immutable.type}.List{object List}.builderFactory{[A]BuilderFactory[A,List[A],List#Coll]}[Row]
 {BuilderFactory[Row,List[Row],List#Coll]}){List[Row]})
 {(implicit theOwner: Symbol)Rep}
 (theOwner{Symbol}){Rep};

--
Paul Phillips      | Every election is a sort of advance auction sale
Stickler           | of stolen goods.
Empiricist         |     -- H. L. Mencken
slap pi uphill!    |----------* http://www.improving.org/paulp/ *----------

--
Viktor Klang
Scala Loudmouth

Option can't be an Iterable. Option[T].flatMap takes a T => Option[S]
and returns an Option[S]. This is both signature and contract
incompatible with Iterable[T].flatMap.

2009/7/2 Jorge Ortiz :
> Can we just make Option formally an Iterable? There's implicit conversions
> in both directions, so I don't see why it isn't other than to cause implicit
> and typer headaches.
>
> --j
>
> On Tue, Jun 30, 2009 at 12:36 PM, Paul Phillips wrote:
>>
>> On Tue, Jun 30, 2009 at 08:38:17PM +0200, martin odersky wrote:
>> > Is nrows then of type Option?
>>
>> nrows is List[Option[Row]], with flatten/flatMap on it yielding List[Row].
>>
>> Attempting to reduce the voluminous output to the relevant without
>> overshooting, here is what the crashing flatten version looks like after
>> typer and the noncrashing flatMap(x => x) version.  Most obvious
>> difference is the BuilderFactory.  If I'm going to spend much time
>> looking at this kind of output I had better write a pretty printer...
>>
>> flatten{[B]
>>  (implicit toTraversable: (Option[Row]) => Traversable[B])List[B]}
>>  [Row]{(implicit toTraversable: (Option[Row]) =>
>> Traversable[Row])List[Row]}
>>  ({((xo: Option[Row]) => scala.this{type}.Option{object
>> Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
>>  [Row]{(xo: Option[Row])Iterable[Row]}
>>  (xo{Option[Row]}){Iterable[Row]})
>>  {(Option[Row]) => Iterable[Row]}}
>>  {(Option[Row]) => Iterable[Row]}){List[Row]})
>>  {(implicit theOwner: Symbol)Rep}
>>  (theOwner{Symbol}){Rep};
>>
>> flatMap{[B,That](f: (Option[Row]) => Traversable[B])
>>  (implicit bf: BuilderFactory[B,That,List[Option[Row]]])That}
>>  [Row, List[Row]]{(f: (Option[Row]) => Traversable[Row])(implicit bf:
>> BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
>>  (((x: Option[Row]) => scala.this{type}.Option{object
>> Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
>>  [Row]{(xo: Option[Row])Iterable[Row]}
>>  (x{Option[Row]}){Iterable[Row]})
>>  {(Option[Row]) => Iterable[Row]})
>>  {(implicit bf: BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
>>  (immutable.this{immutable.type}.List{object
>> List}.builderFactory{[A]BuilderFactory[A,List[A],List#Coll]}[Row]
>>  {BuilderFactory[Row,List[Row],List#Coll]}){List[Row]})
>>  {(implicit theOwner: Symbol)Rep}
>>  (theOwner{Symbol}){Rep};
>>
>> --
>> Paul Phillips      | Every election is a sort of advance auction sale
>> Stickler           | of stolen goods.
>> Empiricist         |     -- H. L. Mencken
>> slap pi uphill!    |----------* http://www.improving.org/paulp/
>> *----------
>
>

Hmm. I thought Scala had the necessary mechanics to make Iterable's
flatMap able to have a signature something like:

def flatMap[B, C[_]](f : (A) => C[B]) : C[B]

..possible.

2009/7/2 David MacIver :
> Option can't be an Iterable. Option[T].flatMap takes a T => Option[S]
> and returns an Option[S]. This is both signature and contract
> incompatible with Iterable[T].flatMap.
>
> 2009/7/2 Jorge Ortiz :
>> Can we just make Option formally an Iterable? There's implicit conversions
>> in both directions, so I don't see why it isn't other than to cause implicit
>> and typer headaches.
>>
>> --j
>>
>> On Tue, Jun 30, 2009 at 12:36 PM, Paul Phillips wrote:
>>>
>>> On Tue, Jun 30, 2009 at 08:38:17PM +0200, martin odersky wrote:
>>> > Is nrows then of type Option?
>>>
>>> nrows is List[Option[Row]], with flatten/flatMap on it yielding List[Row].
>>>
>>> Attempting to reduce the voluminous output to the relevant without
>>> overshooting, here is what the crashing flatten version looks like after
>>> typer and the noncrashing flatMap(x => x) version.  Most obvious
>>> difference is the BuilderFactory.  If I'm going to spend much time
>>> looking at this kind of output I had better write a pretty printer...
>>>
>>> flatten{[B]
>>>  (implicit toTraversable: (Option[Row]) => Traversable[B])List[B]}
>>>  [Row]{(implicit toTraversable: (Option[Row]) =>
>>> Traversable[Row])List[Row]}
>>>  ({((xo: Option[Row]) => scala.this{type}.Option{object
>>> Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
>>>  [Row]{(xo: Option[Row])Iterable[Row]}
>>>  (xo{Option[Row]}){Iterable[Row]})
>>>  {(Option[Row]) => Iterable[Row]}}
>>>  {(Option[Row]) => Iterable[Row]}){List[Row]})
>>>  {(implicit theOwner: Symbol)Rep}
>>>  (theOwner{Symbol}){Rep};
>>>
>>> flatMap{[B,That](f: (Option[Row]) => Traversable[B])
>>>  (implicit bf: BuilderFactory[B,That,List[Option[Row]]])That}
>>>  [Row, List[Row]]{(f: (Option[Row]) => Traversable[Row])(implicit bf:
>>> BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
>>>  (((x: Option[Row]) => scala.this{type}.Option{object
>>> Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
>>>  [Row]{(xo: Option[Row])Iterable[Row]}
>>>  (x{Option[Row]}){Iterable[Row]})
>>>  {(Option[Row]) => Iterable[Row]})
>>>  {(implicit bf: BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
>>>  (immutable.this{immutable.type}.List{object
>>> List}.builderFactory{[A]BuilderFactory[A,List[A],List#Coll]}[Row]
>>>  {BuilderFactory[Row,List[Row],List#Coll]}){List[Row]})
>>>  {(implicit theOwner: Symbol)Rep}
>>>  (theOwner{Symbol}){Rep};
>>>
>>> --
>>> Paul Phillips      | Every election is a sort of advance auction sale
>>> Stickler           | of stolen goods.
>>> Empiricist         |     -- H. L. Mencken
>>> slap pi uphill!    |----------* http://www.improving.org/paulp/
>>> *----------
>>
>>
>

Then you'd lose the ability to use flatMap with an Iterable.

This isn't a problem with the type system per se. The behaviour of
flatMap on Option and Iterable are inherently different.

2009/7/2 Ricky Clarkson :
> Hmm.  I thought Scala had the necessary mechanics to make Iterable's
> flatMap able to have a signature something like:
>
> def flatMap[B, C[_]](f : (A) => C[B]) : C[B]
>
> ..possible.
>
> 2009/7/2 David MacIver :
>> Option can't be an Iterable. Option[T].flatMap takes a T => Option[S]
>> and returns an Option[S]. This is both signature and contract
>> incompatible with Iterable[T].flatMap.
>>
>> 2009/7/2 Jorge Ortiz :
>>> Can we just make Option formally an Iterable? There's implicit conversions
>>> in both directions, so I don't see why it isn't other than to cause implicit
>>> and typer headaches.
>>>
>>> --j
>>>
>>> On Tue, Jun 30, 2009 at 12:36 PM, Paul Phillips wrote:
>>>>
>>>> On Tue, Jun 30, 2009 at 08:38:17PM +0200, martin odersky wrote:
>>>> > Is nrows then of type Option?
>>>>
>>>> nrows is List[Option[Row]], with flatten/flatMap on it yielding List[Row].
>>>>
>>>> Attempting to reduce the voluminous output to the relevant without
>>>> overshooting, here is what the crashing flatten version looks like after
>>>> typer and the noncrashing flatMap(x => x) version.  Most obvious
>>>> difference is the BuilderFactory.  If I'm going to spend much time
>>>> looking at this kind of output I had better write a pretty printer...
>>>>
>>>> flatten{[B]
>>>>  (implicit toTraversable: (Option[Row]) => Traversable[B])List[B]}
>>>>  [Row]{(implicit toTraversable: (Option[Row]) =>
>>>> Traversable[Row])List[Row]}
>>>>  ({((xo: Option[Row]) => scala.this{type}.Option{object
>>>> Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
>>>>  [Row]{(xo: Option[Row])Iterable[Row]}
>>>>  (xo{Option[Row]}){Iterable[Row]})
>>>>  {(Option[Row]) => Iterable[Row]}}
>>>>  {(Option[Row]) => Iterable[Row]}){List[Row]})
>>>>  {(implicit theOwner: Symbol)Rep}
>>>>  (theOwner{Symbol}){Rep};
>>>>
>>>> flatMap{[B,That](f: (Option[Row]) => Traversable[B])
>>>>  (implicit bf: BuilderFactory[B,That,List[Option[Row]]])That}
>>>>  [Row, List[Row]]{(f: (Option[Row]) => Traversable[Row])(implicit bf:
>>>> BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
>>>>  (((x: Option[Row]) => scala.this{type}.Option{object
>>>> Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
>>>>  [Row]{(xo: Option[Row])Iterable[Row]}
>>>>  (x{Option[Row]}){Iterable[Row]})
>>>>  {(Option[Row]) => Iterable[Row]})
>>>>  {(implicit bf: BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
>>>>  (immutable.this{immutable.type}.List{object
>>>> List}.builderFactory{[A]BuilderFactory[A,List[A],List#Coll]}[Row]
>>>>  {BuilderFactory[Row,List[Row],List#Coll]}){List[Row]})
>>>>  {(implicit theOwner: Symbol)Rep}
>>>>  (theOwner{Symbol}){Rep};
>>>>
>>>> --
>>>> Paul Phillips      | Every election is a sort of advance auction sale
>>>> Stickler           | of stolen goods.
>>>> Empiricist         |     -- H. L. Mencken
>>>> slap pi uphill!    |----------* http://www.improving.org/paulp/
>>>> *----------
>>>
>>>
>>
>
>
>
> --
> Ricky Clarkson
> Java Programmer, AD Holdings
> +44 1565 770804
> Skype: ricky_clarkson
> Google Talk: ricky.clarkson@gmail.com
>

I could have sworn there was an implicit iterable2Option, but seeing how I can't find it I guess I was mistaken.

There is already an implicit option2Iterable to deal with the following the case:

  List(3, 4).flatMap(x => Some(x*x)) // == List(9, 16)

If option2Iterable didn't exist then that would be a type error. This is already a concession that Option is "iterable-like". The only thing that stops Option from having Iterable's flatMap signature readily available is that it has its own, slightly more restrictive one. The only thing this signature prevents is something like the following:

  Some(3).flatMap(x => List(x*x, x*x*x)) // == error: type mismatch; found List expected Option

With an additional implicit:

  implicit def iterable2Option[T](i: Iterable[T]): Option[T] = i.headOption

Then:

  Some(3).flatMap(x => List(x*x, x*x*x)) // == Some(9)

Once you've got bidirectional implicits then Iterable and Option's flatMaps are fully compatible. Then it makes sense to unify Option into Iterable, because it simplifies the implemenation and Option acquires all sorts of useful functionality.

Except having Option[T] <: Iterable[T] is much nicer than having bidirectional implicits. Anywhere an Iterable[T] is expected, an Option[T] can be given (already true because of option2Iterable). But where an Option[T] is expected, an Iterable[T] can't be passed in (whereas with iterable2Option it could be). The only change is that Option#flatMap becomes a little more liberal; but not, I think, in a bad way.

Furthermore, I think it's beneficial to have some notion of "collections of 0 or 1 element" as a trait (not Option, but some intermediary between Option and Iterable). Then Weak and Soft References can be implemented as mutable collections of 0 or 1 element (i.e., you may or may not have an element, but if you do the garbage collector can snatch it out from under you at any moment). I've found this comes in really handy when building customizable HashMaps that can have strong, weak, or soft references for keys or values. The HashMap implementation simply uses the intermediary trait (of collections representing 0 or 1 element) and can be parametrized to use any kind of reference.

--j

On Thu, Jul 2, 2009 at 3:08 AM, David MacIver <david.maciver@gmail.com> wrote:

Option can't be an Iterable. Option[T].flatMap takes a T => Option[S]
and returns an Option[S]. This is both signature and contract
incompatible with Iterable[T].flatMap.

2009/7/2 Jorge Ortiz <jorge.ortiz@gmail.com>:
> Can we just make Option formally an Iterable? There's implicit conversions
> in both directions, so I don't see why it isn't other than to cause implicit
> and typer headaches.
>
> --j
>
> On Tue, Jun 30, 2009 at 12:36 PM, Paul Phillips <paulp@improving.org> wrote:
>>
>> On Tue, Jun 30, 2009 at 08:38:17PM +0200, martin odersky wrote:
>> > Is nrows then of type Option?
>>
>> nrows is List[Option[Row]], with flatten/flatMap on it yielding List[Row].
>>
>> Attempting to reduce the voluminous output to the relevant without
>> overshooting, here is what the crashing flatten version looks like after
>> typer and the noncrashing flatMap(x => x) version.  Most obvious
>> difference is the BuilderFactory.  If I'm going to spend much time
>> looking at this kind of output I had better write a pretty printer...
>>
>> flatten{[B]
>>  (implicit toTraversable: (Option[Row]) => Traversable[B])List[B]}
>>  [Row]{(implicit toTraversable: (Option[Row]) =>
>> Traversable[Row])List[Row]}
>>  ({((xo: Option[Row]) => scala.this{type}.Option{object
>> Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
>>  [Row]{(xo: Option[Row])Iterable[Row]}
>>  (xo{Option[Row]}){Iterable[Row]})
>>  {(Option[Row]) => Iterable[Row]}}
>>  {(Option[Row]) => Iterable[Row]}){List[Row]})
>>  {(implicit theOwner: Symbol)Rep}
>>  (theOwner{Symbol}){Rep};
>>
>> flatMap{[B,That](f: (Option[Row]) => Traversable[B])
>>  (implicit bf: BuilderFactory[B,That,List[Option[Row]]])That}
>>  [Row, List[Row]]{(f: (Option[Row]) => Traversable[Row])(implicit bf:
>> BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
>>  (((x: Option[Row]) => scala.this{type}.Option{object
>> Option}.option2Iterable{[A](xo: Option[A])Iterable[A]}
>>  [Row]{(xo: Option[Row])Iterable[Row]}
>>  (x{Option[Row]}){Iterable[Row]})
>>  {(Option[Row]) => Iterable[Row]})
>>  {(implicit bf: BuilderFactory[Row,List[Row],List[Option[Row]]])List[Row]}
>>  (immutable.this{immutable.type}.List{object
>> List}.builderFactory{[A]BuilderFactory[A,List[A],List#Coll]}[Row]
>>  {BuilderFactory[Row,List[Row],List#Coll]}){List[Row]})
>>  {(implicit theOwner: Symbol)Rep}
>>  (theOwner{Symbol}){Rep};
>>
>> --
>> Paul Phillips      | Every election is a sort of advance auction sale
>> Stickler           | of stolen goods.
>> Empiricist         |     -- H. L. Mencken
>> slap pi uphill!    |----------* http://www.improving.org/paulp/
>> *----------
>
>