case class specialization a bit indiscriminate

And from the dept. of slightly comic inverted curves:

case class Bob1(x: Int)
case class Bob2(x: Int, y: Int)
case class Bob3(x: Int, y: Int, z: Int)

// -rw-r--r-- 1 paulp wheel 5478 Apr 11 21:21 Bob1$.class
// -rw-r--r-- 1 paulp wheel 5680 Apr 11 21:21 Bob1.class
// -rw-r--r-- 1 paulp wheel 14577 Apr 11 21:21 Bob2$.class
// -rw-r--r-- 1 paulp wheel 13125 Apr 11 21:21 Bob2.class
// -rw-r--r-- 1 paulp wheel 1729 Apr 11 21:21 Bob3$.class
// -rw-r--r-- 1 paulp wheel 4148 Apr 11 21:21 Bob3.class

I guess this summarizes the situation. I apologize if there's some user
error on my part, to be honest I haven't kept tabs on specialization.

// class is 973 bytes
val f1 = (x: Int, y: Int) => x*y
// class is 14363 bytes
val f2 = new Function2[Int, Int, Int] { def apply(x: Int, y: Int) = x*y }

No, 140% increase in size isn't expected. I'll have a look at what can be done. Thanks for tracking it down, the Function2 example is really weird. I'll keep you up to date.
iulian

On Mon, Apr 12, 2010 at 6:32 AM, Paul Phillips <paulp@improving.org> wrote:

I guess this summarizes the situation.  I apologize if there's some user
error on my part, to be honest I haven't kept tabs on specialization.

 // class is 973 bytes
 val f1 = (x: Int, y: Int) => x*y
 // class is 14363 bytes
 val f2 = new Function2[Int, Int, Int] { def apply(x: Int, y: Int) = x*y }

--
Paul Phillips      | Every election is a sort of advance auction sale
Vivid              | of stolen goods.
Empiricist         |     -- H. L. Mencken
pp: i haul pills   |----------* http://www.improving.org/paulp/ *----------

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

The extra methods come from mixing in FunctionN traits in the companion object of a case class, plus the forwarder methods that are added in the mirror class. Those will go for sure, but I am looking into whether we can trim those mixed in methods.
The inversion comes from the fact that we specialize only up to Function2, so a case class of 3 parameters will not have any extra weight.
iulian

On Mon, Apr 12, 2010 at 6:23 AM, Paul Phillips <paulp@improving.org> wrote:

And from the dept. of slightly comic inverted curves:

case class Bob1(x: Int)
case class Bob2(x: Int, y: Int)
case class Bob3(x: Int, y: Int, z: Int)

// -rw-r--r--  1 paulp  wheel    5478 Apr 11 21:21 Bob1$.class
// -rw-r--r--  1 paulp  wheel    5680 Apr 11 21:21 Bob1.class
// -rw-r--r--  1 paulp  wheel   14577 Apr 11 21:21 Bob2$.class
// -rw-r--r--  1 paulp  wheel   13125 Apr 11 21:21 Bob2.class
// -rw-r--r--  1 paulp  wheel    1729 Apr 11 21:21 Bob3$.class
// -rw-r--r--  1 paulp  wheel    4148 Apr 11 21:21 Bob3.class

--
Paul Phillips      | Before a man speaks it is always safe to assume
Caged Spirit       | that he is a fool.  After he speaks, it is seldom
Empiricist         | necessary to assume it.
i pull his palp!   |     -- H. L. Mencken

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

On Mon, Apr 12, 2010 at 6:32 AM, Paul Phillips <paulp@improving.org> wrote:

I guess this summarizes the situation.  I apologize if there's some user
error on my part, to be honest I haven't kept tabs on specialization.

 // class is 973 bytes
 val f1 = (x: Int, y: Int) => x*y
 // class is 14363 bytes
 val f2 = new Function2[Int, Int, Int] { def apply(x: Int, y: Int) = x*y }

This is the result of an optimization for closures. Anonymous functions extend runtime.AbstractFunction, which is an abstract class mixing in the corresponding Function trait. This way we save some bytes resulting from re-mixing-in FunctionNconcrete methods, like curry/curried. Only that now Function0-2 have a lot more methods, so the effects are much more visible.
This also suggested the proper fix for case class objects, which should extend AbstractFunction instead of plain Function, bringing down the size of case classes to essentially the initial number.
Very good catch, by the way.
iulian  

--
Paul Phillips      | Every election is a sort of advance auction sale
Vivid              | of stolen goods.
Empiricist         |     -- H. L. Mencken
pp: i haul pills   |----------* http://www.improving.org/paulp/ *----------

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

That actually makes me wonder if I shouldn't be doing "new AbstractFunction" now, instead of "new Function". That's something to ponder upon.

On Mon, Apr 12, 2010 at 2:11 PM, Iulian Dragos <iulian.dragos@epfl.ch> wrote:

On Mon, Apr 12, 2010 at 6:32 AM, Paul Phillips <paulp@improving.org> wrote:

I guess this summarizes the situation.  I apologize if there's some user
error on my part, to be honest I haven't kept tabs on specialization.

 // class is 973 bytes
 val f1 = (x: Int, y: Int) => x*y
 // class is 14363 bytes
 val f2 = new Function2[Int, Int, Int] { def apply(x: Int, y: Int) = x*y }

This is the result of an optimization for closures. Anonymous functions extend runtime.AbstractFunction, which is an abstract class mixing in the corresponding Function trait. This way we save some bytes resulting from re-mixing-in FunctionNconcrete methods, like curry/curried. Only that now Function0-2 have a lot more methods, so the effects are much more visible.
This also suggested the proper fix for case class objects, which should extend AbstractFunction instead of plain Function, bringing down the size of case classes to essentially the initial number.
Very good catch, by the way.
iulian   

--
Paul Phillips      | Every election is a sort of advance auction sale
Vivid              | of stolen goods.
Empiricist         |     -- H. L. Mencken
pp: i haul pills   |----------* http://www.improving.org/paulp/ *----------

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

--
Daniel C. Sobral

I travel to the future all the time.

On Mon, Apr 12, 2010 at 6:30 PM, Daniel Sobral wrote:
> That actually makes me wonder if I shouldn't be doing "new AbstractFunction"
> now, instead of "new Function". That's something to ponder upon.

Well, you'd usually use the function literal syntax and get the right
behaviour automatically. Another aspect to consider is named classes
that extend/mix-in FunctionN. I was curious if AbstractFunctionN would
be used for "class Foo extends (Int => Boolean)", but it seems like
it's not (although I only tried it with a 2.8.0 build from last
month). It's not totally clear to me how this interacts with the
collections library since Seq and MapLike mix-in PartialFunction (and
possibly others). Seems like some bytecode analysis is in order.

Best,
Ismael

On Mon, Apr 12, 2010 at 07:11:17PM +0200, Iulian Dragos wrote:
> This also suggested the proper fix for case class objects, which
> should extend AbstractFunction instead of plain Function, bringing
> down the size of case classes to essentially the initial number.

I'm glad that one went down moderately easily, but I feel like a more
general solution is warranted. Every specialized trait has now taken on
significantly different performance characteristics. Poor partial
functions do not have an AbstractFunction helper, although if I remember
correctly from implementation time there's no obvious reason that
couldn't be fixed. But for now:

val pf1: PartialFunction[Int, Int] = { case 5 => 10 }

// before
-rw-r--r-- 1 paulp wheel 2327 Apr 12 11:41 A$$anonfun$1.class
// after
-rw-r--r-- 1 paulp wheel 7210 Apr 12 11:41 A$$anonfun$1.class

And then there's every map, set, or seq, all of which inherit from
Function1. The size boost is not so noticeable in collections because
they're already pretty heavy (25-30K) so another 5K doesn't trigger the
same alarms. But on the whole it's a bit worrisome.