generalization of TraversableOnce

Hi Paul,

This looks very good to me. But I think it's too late for 2.8. We want to push this thing out now, so only critical bugs will be fixed before 2.8 final. It could in principle go into trunk, because 2.8 already has its own branch. But I'd prefer we wait with it until 2.8 final is out. Once that's done we will anyway have some changes to the collection framework involving parallel collections.

Cheers

 -- Martin

On Thu, Jun 3, 2010 at 11:44 PM, Paul Phillips <paulp@improving.org> wrote:

I had emailed you a while back about generalizing Iterator.flatten to
work on TraversableOnce, and then I withdrew because I remembered only
foreach had made its way to TraversableOnce, and not map, flatMap, and
filter, so it didn't work right.

But! Today I realized the problem was that I was thinking too small.
Now I have implemented what I consider a massive and zero downside
improvement, and I think it's important it goes into 2.8 because it
changes the signature of flatMap and flatten.

Here is the patch:

 http://github.com/paulp/scala/commit/e1de701c419aa0f4784db426e7f20d34b87524fb

The gist of it is: everything which already worked continues to work as
it did before, but now you can mix and match iterators and traversables.
Perhaps the motivation is best illustrated with examples.  One could
achieve a portion of this with an implicit from traversable to iterator,
but a) that's a bad idea and b) pretty sure you can't achieve all of it
because everyone already has methods like map.


// BEFORE - BAD
scala> for (f <- scala.tools.nsc.io.Directory("/tmp").files ; s <- List("a", "b")) yield f+s
<console>:6: error: type mismatch;
 found   : List[java.lang.String]
 required: Iterator[?]
      for (f <- scala.tools.nsc.io.Directory("/tmp").files ; s <- List("a", "b")) yield f+s
                                                               ^

scala> for (x <- List(1,2,3) ; y <- List(1,2,3).iterator) yield x + y
<console>:6: error: type mismatch;
 found   : Iterator[Int]
 required: Traversable[?]
      for (x <- List(1,2,3) ; y <- List(1,2,3).iterator) yield x + y
                                ^

scala> Stream(1 to 10 toList, 1 to 10 iterator, Stream from 1).flatten
<console>:6: error: could not find implicit value for parameter asTraversable: (scala.collection.TraversableOnce[Int]) => Traversable[B]
      Stream(1 to 10 toList, 1 to 10 iterator, Stream from 1).flatten
                                                              ^


// AFTER - GOOD

scala> for (f <- scala.tools.nsc.io.Directory("/tmp").files ; s <- List("a", "b")) yield f+s
res0: Iterator[java.lang.String] = non-empty iterator

scala> res0.toList
res1: List[java.lang.String] = List(/tmp/bop.soa, ) [etc etc ]

scala> for (x <- List(1,2,3) ; y <- List(1,2,3).iterator) yield x + y
res2: List[Int] = List(2, 3, 4, 3, 4, 5, 4, 5, 6)

scala> for (x <- List(1,2,3).iterator ; y <- List(1,2,3)) yield x + y
res3: Iterator[Int] = non-empty iterator

scala> res3.toList
res4: List[Int] = List(2, 3, 4, 3, 4, 5, 4, 5, 6)

--
Paul Phillips      | It is hard to believe that a man is
In Theory          | telling the truth when you know that you
Empiricist         | would lie if you were in his place.
pp: i haul pills   |     -- H. L. Mencken

On Fri, Jun 04, 2010 at 12:00:19AM +0200, martin odersky wrote:
> This looks very good to me. But I think it's too late for 2.8. We want
> to push this thing out now, so only critical bugs will be fixed before
> 2.8 final. It could in principle go into trunk, because 2.8 already
> has its own branch. But I'd prefer we wait with it until 2.8 final is
> out. Once that's done we will anyway have some changes to the
> collection framework involving parallel collections.

That's fine then, it's 100% source compatible anyway. I agree, I'm not
committing anything to trunk unless it's intended for 2.8 final. I'll
give it a room to lie low in while the heat cools down.