- About Scala
- Documentation
- Code Examples
- Software
- Scala Developers
parallel collections architecture
Fri, 2011-03-18, 20:25
I was really glad to see r24496, as I had not until recently realized to
what extent the parallel collections have infiltrated the sequential
ones. On that note, can we discuss taking it further? For instance,
TraversableOnce is now free of any reference to the parallel package or
a *[Pp]ar* method. That's a very good thing and I suggest we keep it
that way. But I would say the same for Iterator, which still has:
trait Iterator[+A] extends TraversableOnce[A] with Parallelizable[A,
ParIterable[A]] {
protected[this] def parCombiner = ParIterable.newCombiner[A]
...
I feel strongly these belong in a separate trait so Iterator has no
direct dependency. I feel the same way (if less strongly) about
Traversable and friends, but I'm realistic that that's unlikely at this
juncture, so I would settle for freeing Iterator.
As a concrete illustration of why I would care about such a thing, it
was the impact of the parallel collections which made writing set and
map views go from unreasonably difficult to hopeless. And I don't mean
writing them for the parallel collections: I mean writing them for
anyone. The "virtual class" pattern takes the inheritance hierarchy and
clubs you with it until you cry.
I'm maybe not connecting all the pieces together at the moment, but I believe that Parallelizable doesn't have to be mixed into Iterator. So, I think it can be removed.
As for the other collection traits, the idea was to be able to convert from any collection type to a corresponding parallel collection. We had `toPar*` to do that before, and `par` is meant to simplify that game.
Implicit conversions to obtain `par` is one other possibility to get this functionality. Type classes are another.
Btw, there are two pending, somewhat related commits in my mirror:
https://github.com/axel22/scala
I will commit these once I test them. The story is that a parallel foreach doesn't play well with existing parts of the collection API, and, still, I'd say we want to have it to be able to write parallel for-comprehensions.
The problem is in the side-effecting operations used with for-loops. The changes above should ensure that side-effecting operations use a sequential version foreach, and avoid parallel.
________________________________________
From: Paul Phillips [paulp@improving.org]
Sent: 18 March 2011 20:25
To: Prokopec Aleksandar; scala-internals@googlegroups.com
Subject: parallel collections architecture
I was really glad to see r24496, as I had not until recently realized to
what extent the parallel collections have infiltrated the sequential
ones. On that note, can we discuss taking it further? For instance,
TraversableOnce is now free of any reference to the parallel package or
a *[Pp]ar* method. That's a very good thing and I suggest we keep it
that way. But I would say the same for Iterator, which still has:
trait Iterator[+A] extends TraversableOnce[A] with Parallelizable[A,
ParIterable[A]] {
protected[this] def parCombiner = ParIterable.newCombiner[A]
...
I feel strongly these belong in a separate trait so Iterator has no
direct dependency. I feel the same way (if less strongly) about
Traversable and friends, but I'm realistic that that's unlikely at this
juncture, so I would settle for freeing Iterator.
As a concrete illustration of why I would care about such a thing, it
was the impact of the parallel collections which made writing set and
map views go from unreasonably difficult to hopeless. And I don't mean
writing them for the parallel collections: I mean writing them for
anyone. The "virtual class" pattern takes the inheritance hierarchy and
clubs you with it until you cry.