betrayed by traits

On Sat, Dec 19, 2009 at 7:09 AM, Paul Phillips wrote:
> Today (day two of my awesome new bytecode project) I discovered that the
> *ViewLike* classes constitute 10% of the size of scala-library.jar.
> Here's a little 400 KB taste:
>
> -rw-r--r--  1 paulp  admin  38269 Dec 18 19:07 IndexedSeqViewLike$$anon$1.class
> -rw-r--r--  1 paulp  admin  37141 Dec 18 19:07 IndexedSeqViewLike$$anon$10.class
> -rw-r--r--  1 paulp  admin  37482 Dec 18 19:07 IndexedSeqViewLike$$anon$11.class
> -rw-r--r--  1 paulp  admin  37971 Dec 18 19:07 IndexedSeqViewLike$$anon$2.class
> -rw-r--r--  1 paulp  admin  38516 Dec 18 19:07 IndexedSeqViewLike$$anon$3.class
> -rw-r--r--  1 paulp  admin  38245 Dec 18 19:07 IndexedSeqViewLike$$anon$4.class
> -rw-r--r--  1 paulp  admin  37957 Dec 18 19:07 IndexedSeqViewLike$$anon$5.class
> -rw-r--r--  1 paulp  admin  38279 Dec 18 19:07 IndexedSeqViewLike$$anon$6.class
> -rw-r--r--  1 paulp  admin  38233 Dec 18 19:07 IndexedSeqViewLike$$anon$7.class
> -rw-r--r--  1 paulp  admin  37197 Dec 18 19:07 IndexedSeqViewLike$$anon$8.class
> -rw-r--r--  1 paulp  admin  37726 Dec 18 19:07 IndexedSeqViewLike$$anon$9.class
>
> Here is a sampling of the source which generates 38K a pop:
>
>  trait Mapped[B] extends Transformed[B] with super.Mapped[B] {
>    override def foreach[U](f: B =>  U) = super[Transformed].foreach(f)
>  }
>
>  trait FlatMapped[B] extends Transformed[B] with super.FlatMapped[B] {
>    override def foreach[U](f: B =>  U) = super[Transformed].foreach(f)
>  }
>
Can you tell me where the 38K is spent? Forwarding methods I assume,
but where are they added?

> That is a pretty steep price to pay for the thrill of overriding
> foreach.  We have a virtual classes problem.  Is this trait based design
> up for reconsideration?
>
What would be the alternatve?

Cheers

On Sat, Dec 19, 2009 at 11:25:47AM +0100, martin odersky wrote:
> Can you tell me where the 38K is spent? Forwarding methods I assume,
> but where are they added?

This is the Function1/AbstractFunction1 problem again, but concentrated
in a few titanic objects intead of thousands of little ones. The same
methods are duplicated into all those anonymous subclasses, though by
the time you get to IndexedSeqViewLike, the entire unique contribution
is a disambiguation about which foreach method to call.

> What would be the alternatve?

I'm not in a situation where I can test/debug right now so don't take
this as a full solution, but to a first approximation: I changed all the
Transformed traits into classes so there would be at least a little code
reuse, like:

- trait Transformed[+B] extends IndexedSeqView[B, Coll] with super.Transformed[B]
+ abstract class Transformed[+B] extends super.Transformed[B] with IndexedSeqView[B, Coll]

And that alone had this impact on the resulting jar:

// class
-rw-r--r-- 1 paulp admin 4582845 Dec 19 06:58 scala-library.jar
// trait
-rw-r--r-- 1 paulp admin 4801173 Dec 18 19:13 scala-library.jar

I expect I could do a lot better by examining the whole picture.

The classfiles mentioned in the previous email shrank 85% or so.

-rw-r--r-- 1 paulp admin 4243 Dec 19 06:51 IndexedSeqViewLike$$anon$1.class
-rw-r--r-- 1 paulp admin 2330 Dec 19 06:52 IndexedSeqViewLike$$anon$10.class
-rw-r--r-- 1 paulp admin 2813 Dec 19 06:52 IndexedSeqViewLike$$anon$11.class
-rw-r--r-- 1 paulp admin 3740 Dec 19 06:51 IndexedSeqViewLike$$anon$2.class
-rw-r--r-- 1 paulp admin 4404 Dec 19 06:52 IndexedSeqViewLike$$anon$3.class
-rw-r--r-- 1 paulp admin 4129 Dec 19 06:51 IndexedSeqViewLike$$anon$4.class
-rw-r--r-- 1 paulp admin 4060 Dec 19 06:51 IndexedSeqViewLike$$anon$5.class
-rw-r--r-- 1 paulp admin 4171 Dec 19 06:51 IndexedSeqViewLike$$anon$6.class
-rw-r--r-- 1 paulp admin 4119 Dec 19 06:52 IndexedSeqViewLike$$anon$7.class
-rw-r--r-- 1 paulp admin 2853 Dec 19 06:52 IndexedSeqViewLike$$anon$8.class
-rw-r--r-- 1 paulp admin 3562 Dec 19 06:51 IndexedSeqViewLike$$anon$9.class

On Sat, Dec 19, 2009 at 5:28 PM, Paul Phillips wrote:
> On Sat, Dec 19, 2009 at 11:25:47AM +0100, martin odersky wrote:
>> Can you tell me where the 38K is spent? Forwarding methods I assume,
>> but where are they added?
>
> This is the Function1/AbstractFunction1 problem again, but concentrated
> in a few titanic objects intead of thousands of little ones.  The same
> methods are duplicated into all those anonymous subclasses, though by
> the time you get to IndexedSeqViewLike, the entire unique contribution
> is a disambiguation about which foreach method to call.
>
>> What would be the alternatve?
>
> I'm not in a situation where I can test/debug right now so don't take
> this as a full solution, but to a first approximation: I changed all the
> Transformed traits into classes so there would be at least a little code
> reuse, like:
>
> -  trait Transformed[+B] extends IndexedSeqView[B, Coll] with super.Transformed[B]
> +  abstract class Transformed[+B] extends super.Transformed[B] with IndexedSeqView[B, Coll]
>
> And that alone had this impact on the resulting jar:
>
> // class
> -rw-r--r--  1 paulp  admin  4582845 Dec 19 06:58 scala-library.jar
> // trait
> -rw-r--r--  1 paulp  admin  4801173 Dec 18 19:13 scala-library.jar
>
> I expect I could do a lot better by examining the whole picture.
>
> The classfiles mentioned in the previous email shrank 85% or so.
>
> -rw-r--r--  1 paulp  admin   4243 Dec 19 06:51 IndexedSeqViewLike$$anon$1.class
> -rw-r--r--  1 paulp  admin   2330 Dec 19 06:52 IndexedSeqViewLike$$anon$10.class
> -rw-r--r--  1 paulp  admin   2813 Dec 19 06:52 IndexedSeqViewLike$$anon$11.class
> -rw-r--r--  1 paulp  admin   3740 Dec 19 06:51 IndexedSeqViewLike$$anon$2.class
> -rw-r--r--  1 paulp  admin   4404 Dec 19 06:52 IndexedSeqViewLike$$anon$3.class
> -rw-r--r--  1 paulp  admin   4129 Dec 19 06:51 IndexedSeqViewLike$$anon$4.class
> -rw-r--r--  1 paulp  admin   4060 Dec 19 06:51 IndexedSeqViewLike$$anon$5.class
> -rw-r--r--  1 paulp  admin   4171 Dec 19 06:51 IndexedSeqViewLike$$anon$6.class
> -rw-r--r--  1 paulp  admin   4119 Dec 19 06:52 IndexedSeqViewLike$$anon$7.class
> -rw-r--r--  1 paulp  admin   2853 Dec 19 06:52 IndexedSeqViewLike$$anon$8.class
> -rw-r--r--  1 paulp  admin   3562 Dec 19 06:51 IndexedSeqViewLike$$anon$9.class
>
That would be cool if we could do it that way, yes. We'd have to test
it carefully though because it would change the linearization of
Transformed. So it might change behavior. Nothing that could not be
fixed by a few judicious super[T]'s though, I would assume.

Cheers