[scala-language] Why are getters/setter methods generated for private val/var?

On Friday, January 28, 2011 8:34:47 AM UTC+1, Cay Horstmann wrote:

I can't figure out why private val/var fields need getter/setter
methods. For example, in this example,

class ChecksumAccumulator {
  private var sum = 0
  def add(b: Byte): Unit = {
    sum += b
  }
  def checksum(): Int = {
    return ~(sum & 0xFF) + 1
  }
}

the Scala 2.8 compiler generates methods sum() and sum_$eq(int), and
it calls them in add(Byte) and checksum(). Why doesn't the generated
code just access the field?

Cay,

I am not 100% sure anymore, but I believe it is because the translation is easier and more uniform that way. Note that there can be quite a lot of complications -- for instance, the private field might end up being accessed in inner classes. So the compiler needs to generate access methods.
In any case, simple accessor methods like that are more of less guaranteed to be inlined by the JIT
 

When I change the field to

  private[ChecksumAccumulator] var sum = 0

then I get public (!) getters and setters. Why is that? Is there any
difference between a private field and a field that is qualified
private to its own class?

Yes, there is. The spec makes quite a dance around that. Essentially, private[Q] is a static Scala compiler thing, it has no bearing on
the JVM's representation.

Finally, when I change the field to

  private[this] var sum = 0

then the methods go away. (I realize that the semantics has changed
slightly, but I don't understand the connection with the getter/setter
methods.

private[this] means that you'll never need a forwarder for the field, so you do not need a getter to start with.

Hope this clarifies things.

Good luck with your book project! I am very much looking forward to seeing the book come out, because it will fill an important need.

Cheers

 -- Martin

2011/1/29 martin odersky :
>
>
> On Friday, January 28, 2011 8:34:47 AM UTC+1, Cay Horstmann wrote:
>>
>> I can't figure out why private val/var fields need getter/setter
>> methods. For example, in this example,
>>
>> class ChecksumAccumulator {
>>   private var sum = 0
>>   def add(b: Byte): Unit = {
>>     sum += b
>>   }
>>   def checksum(): Int = {
>>     return ~(sum & 0xFF) + 1
>>   }
>> }
>>
>> the Scala 2.8 compiler generates methods sum() and sum_$eq(int), and
>> it calls them in add(Byte) and checksum(). Why doesn't the generated
>> code just access the field?

> I am not 100% sure anymore, but I believe it is because the translation is
> easier and more uniform that way. Note that there can be quite a lot of
> complications -- for instance, the private field might end up being accessed
> in inner classes. So the compiler needs to generate access methods.

Well, actually, if you add an inner class that accesses the field, you
get an entirely different set of methods, for just the purpose that
you describe. (I tried that and didn't mention it so as not to muddy
the waters :-)) So, let's say that these methods are an implementation
strategy.

>>
>> When I change the field to
>>
>>   private[ChecksumAccumulator] var sum = 0
>>
>> then I get public (!) getters and setters. Why is that? Is there any
>> difference between a private field and a field that is qualified
>> private to its own class?
>
> Yes, there is. The spec makes quite a dance around that. Essentially,
> private[Q] is a static Scala compiler thing, it has no bearing on
> the JVM's representation.

Really, my question is whether there is any difference between

class Q { private val x: T}

and

class Q { private[Q] val x: T } // same Q

in the Scala language.

I read that very long discussion in the spec. It seemed to me they
should be exactly the same, but why the difference in the
getters/setters? An implementation detail?

Now here is another odd thing that I noted from the compiler.

When I add methods

sum()
sum_=(Int)

to the class, I get error messages for

(a) private var sum
(b) private[ChecksumAccumulator] var sum

but not for

(c) private[this] var sum

The spec says that a non-private var is equivalent to a getter/setter
pair, but it seems murky about a private var. Of course, it makes
sense to outlaw the getters and setters that conflict with a private
var, even when they aren't actually generated. But why is
private[this] special?

Sorry to be such a pain.

Cheers,

Cay

On Sat, Jan 29, 2011 at 12:12 PM, Cay Horstmann <cay.horstmann@gmail.com> wrote:

2011/1/29 martin odersky <odersky@gmail.com>:
>
>
> On Friday, January 28, 2011 8:34:47 AM UTC+1, Cay Horstmann wrote:
>>
>> I can't figure out why private val/var fields need getter/setter
>> methods. For example, in this example,
>>
>> class ChecksumAccumulator {
>>   private var sum = 0
>>   def add(b: Byte): Unit = {
>>     sum += b
>>   }
>>   def checksum(): Int = {
>>     return ~(sum & 0xFF) + 1
>>   }
>> }
>>
>> the Scala 2.8 compiler generates methods sum() and sum_$eq(int), and
>> it calls them in add(Byte) and checksum(). Why doesn't the generated
>> code just access the field?

> I am not 100% sure anymore, but I believe it is because the translation is
> easier and more uniform that way. Note that there can be quite a lot of
> complications -- for instance, the private field might end up being accessed
> in inner classes. So the compiler needs to generate access methods.

Well, actually, if you add an inner class that accesses the field, you
get an entirely different set of methods, for just the purpose that
you describe. (I tried that and didn't mention it so as not to muddy
the waters :-)) So, let's say that these methods are an implementation
strategy.

>>
>> When I change the field to
>>
>>   private[ChecksumAccumulator] var sum = 0
>>
>> then I get public (!) getters and setters. Why is that? Is there any
>> difference between a private field and a field that is qualified
>> private to its own class?
>
> Yes, there is. The spec makes quite a dance around that. Essentially,
> private[Q] is a static Scala compiler thing, it has no bearing on
> the JVM's representation.

Really, my question is whether there is any difference between

class Q { private val x: T}

and

class Q { private[Q] val x: T } // same Q

in the Scala language.

I read that very long discussion in the spec. It seemed to me they
should be exactly the same, but why the difference in the
getters/setters? An implementation detail?

There is a difference. The spec is quite clear that qualified private and private is not the same.
I think it's when it introduces the term `private members' where it makes the difference.
 

Now here is another odd thing that I noted from the compiler.

When I add methods

sum()
sum_=(Int)

to the class, I get error messages for

 (a) private var sum
 (b) private[ChecksumAccumulator] var sum

but not for

 (c) private[this] var sum

The spec says that a non-private var is equivalent to a getter/setter
pair, but it seems murky about a private var. Of course, it makes
sense to outlaw the getters and setters that conflict with a private
var, even when they aren't actually generated. But why is
private[this] special?

Becuse that's the type of the actual fields that get getters and setters.

If you write

  private val x: String

you get

  private[this] val x: String   -- the field, which is hidden
  private def x: String = x    -- rhs refers to first x

At the rhs of the getter, if private[this] references also went through a getter, we'd get an infinite chain of references. (I should say here that expanded intermediate code is also typechecked and transformed later, so every transformation needs to be idempotent).

I'm not arguing that this is the only strategy that makes sense. In the end, it does not really matter because these are implementation details.

Cheers

 -- Martin

> On Sat, Jan 29, 2011 at 12:12 PM, Cay Horstmann
> wrote:
>> Really, my question is whether there is any difference between
>>
>> class Q { private val x: T}
>>
>> and
>>
>> class Q { private[Q] val x: T } // same Q
>>
>> in the Scala language.
>
2011/1/29 martin odersky :
> There is a difference. The spec is quite clear that qualified private and
> private is not the same.
> I think it's when it introduces the term `private members' where it makes
> the difference.

Sorry, I still don't get it. Let me put it a little differently.

In this class

class Q {
private var x = 0;
private[Q] var y = 0; // Note: Same Q
...
}

is there anything that anyone can do to x but not to y, or to y but not to x?

Thanks,

Cay

Saturday, January 29, 2011, 3:32:46 PM, martin odersky wrote:

> On Sat, Jan 29, 2011 at 12:12 PM, Cay Horstmann wrote:
> 2011/1/29 martin odersky :
>>
>>
>> On Friday, January 28, 2011 8:34:47 AM UTC+1, Cay Horstmann wrote:
[snip]
>> When I add methods
>>
>> sum()
>> sum_=(Int)
>>
>> to the class, I get error messages for
>>
>> (a) private var sum
>> (b) private[ChecksumAccumulator] var sum
>>
>> but not for
>>
>> (c) private[this] var sum
[snip]
> I'm not arguing that this is the only strategy that makes sense. In
> the end, it does not really matter because these are implementation
> details.

They aren't if these influence what is allowed in Scala. What I find
confusing here is that while this is an error:

class C {
var x: Int = 1 // not abstract
}

class D extends C {
override def x_=(v: Int) { ... }
}

with a C like this it isn't are error:

class C {
...
def x: Int = ...
def x_=(newX: Int) { ... }
}

nor with a C like this:

abstract class C {
var x: Int // abstract
}

Are these just implementation artifacts, or language design decisions?
If these are language design decisions, what's the philosophy behind
them?

On Sat, Jan 29, 2011 at 3:59 PM, Cay Horstmann <cay.horstmann@gmail.com> wrote:

> On Sat, Jan 29, 2011 at 12:12 PM, Cay Horstmann <cay.horstmann@gmail.com>
> wrote:
>> Really, my question is whether there is any difference between
>>
>> class Q { private val x: T}
>>
>> and
>>
>> class Q { private[Q] val x: T } // same Q
>>
>> in the Scala language.
>
2011/1/29 martin odersky <martin.odersky@epfl.ch>:
> There is a difference. The spec is quite clear that qualified private and
> private is not the same.
> I think it's when it introduces the term `private members' where it makes
> the difference.

Sorry, I still don't get it. Let me put it a little differently.

In this class

class Q {
 private var x = 0;
 private[Q] var y = 0; // Note: Same Q
 ...
}

is there anything that anyone can do to x but not to y, or to y but not to x?

Yes, you can redefine `x' in a subclass but not `y'.

  -- Martin

On Sat, Jan 29, 2011 at 11:16:53PM +0100, martin odersky wrote:
> > class Q {
> > private var x = 0;
> > private[Q] var y = 0; // Note: Same Q
> > ...
> > }
> >
> > is there anything that anyone can do to x but not to y, or to y but not to
> > x?
> >
> > Yes, you can redefine `x' in a subclass but not `y'.

I was hanging on a thread for the answer to this question, and I'm sorry
to say the implementation does not concur. If you mean that you cannot
redefine y with the same access, that's true because Q is not an
enclosing class. But if you pick any legal access it's fine with it. I
wondered if it had changed: this is true for all versions of scala.

The classes in question to make sure we're talking about the same thing:

class Q {
private var x = 0
private[Q] var y = 0
}

class QQ extends Q {
private var x = 0
private[QQ] var y = 0
// or private var y = 0 is also fine
}

On Sat, Jan 29, 2011 at 11:34 PM, Paul Phillips <paulp@improving.org> wrote:

On Sat, Jan 29, 2011 at 11:16:53PM +0100, martin odersky wrote:
> > class Q {
> >  private var x = 0;
> >  private[Q] var y = 0; // Note: Same Q
> >  ...
> > }
> >
> > is there anything that anyone can do to x but not to y, or to y but not to
> > x?
> >
> > Yes, you can redefine `x' in a subclass but not `y'.

I was hanging on a thread for the answer to this question, and I'm sorry
to say the implementation does not concur.  If you mean that you cannot
redefine y with the same access, that's true because Q is not an
enclosing class.  But if you pick any legal access it's fine with it.  I
wondered if it had changed: this is true for all versions of scala.

The classes in question to make sure we're talking about the same thing:

class Q {
 private var x = 0
 private[Q] var y = 0
}

class QQ extends Q {
 private var x = 0
 private[QQ] var y = 0
 // or private var y = 0 is also fine
}
 

You are right. What I should have said: You can redefine `x', and they are two different
fields, but for `y' it's an override. So `y' must be type and accessibility conformant to the
definition of the superclass, but the two definitions of `x' could have completely different types.

Cheers

 -- Martin
 

--
Paul Phillips      | On two occasions, I have been asked, 'Mr. Babbage, if you
Analgesic          | put into the machine wrong figures, will the right answers
Empiricist         | come out?' I am not able to rightly apprehend the kind of
pal, i pill push   | confusion of ideas that could provoke such a question.

--
----------------------------------------------
Martin Odersky
Prof., EPFL and CEO, Scala Solutions
PSED, 1015 Lausanne, Switzerland

On Sat, Jan 29, 2011 at 4:40 PM, martin odersky <martin.odersky@epfl.ch> wrote:

You are right. What I should have said: You can redefine `x', and they are two different
fields, but for `y' it's an override. So `y' must be type and accessibility conformant to the
definition of the superclass, but the two definitions of `x' could have completely different types.

So how are these two declarations different?
class Q {
  protected var x = 0  private[Q] var y = 0
}

On Sat, Jan 29, 2011 at 11:40:36PM +0100, martin odersky wrote:
> You are right. What I should have said: You can redefine `x', and they
> are two different fields, but for `y' it's an override. So `y' must be
> type and accessibility conformant to the definition of the superclass,
> but the two definitions of `x' could have completely different types.

I assume we're zeroing in on implementation bugs then. Again in all
versions of scala, this compiles.

class Q {
private var x: Int = 0
private[Q] var y: Int = 0
}
class QQ extends Q {
var x: String = "abc"
var y: String = "abc"
}

2011/1/30 Paul Phillips :
> On Sat, Jan 29, 2011 at 11:40:36PM +0100, martin odersky wrote:
>> You are right. What I should have said: You can redefine `x', and they
>> are two different fields, but for `y' it's an override. So `y' must be
>> type and accessibility conformant to the definition of the superclass,
>> but the two definitions of `x' could have completely different types.
>
> I assume we're zeroing in on implementation bugs then.  Again in all
> versions of scala, this compiles.
>
> class Q {
>  private var x: Int = 0
>  private[Q] var y: Int = 0
> }
> class QQ extends Q {
>  var x: String = "abc"
>  var y: String = "abc"
> }
>

In Java, a golden rule is "if you can't see it, you can't override it".

This always seemed like a good idea to me, and I seem to recall that
this behavior was specifically designed to avoid anomalies in C++
(where it is possible to redefine private virtual functions).

So, it seems entirely reasonable to me that QQ would not be able to
override either x or y.

However, Section 5.1.4. of the Scala reference, allows overriding an
invisible member that is declared private[Q]. (Note that section 5.2
defines "private" to mean "marked with private or private[this]", and
private[Q] is specifically excluded from "private".) So, Paul is right
that this an implementation bug.

But is there a good reason for the spec? It's not too late to change
since it's apparently never been properly implemented :-) One could
say "A member M of class C that matches (§5.1.3) a member M0 of a base
class of C, where M0 is accessible in C, ..."

Cheers,

Cay

On Sun, Jan 30, 2011 at 9:04 AM, Cay Horstmann <cay.horstmann@gmail.com> wrote:

2011/1/30 Paul Phillips <paulp@improving.org>:
> On Sat, Jan 29, 2011 at 11:40:36PM +0100, martin odersky wrote:
>> You are right. What I should have said: You can redefine `x', and they
>> are two different fields, but for `y' it's an override. So `y' must be
>> type and accessibility conformant to the definition of the superclass,
>> but the two definitions of `x' could have completely different types.
>
> I assume we're zeroing in on implementation bugs then.  Again in all
> versions of scala, this compiles.
>
> class Q {
>  private var x: Int = 0
>  private[Q] var y: Int = 0
> }
> class QQ extends Q {
>  var x: String = "abc"
>  var y: String = "abc"
> }
>

Yes, looks like it. We should go to the bottom of that with some urgency!

In Java, a golden rule is "if you can't see it, you can't override it".

That's a good rule, yes. Generally, if you can't see it, it should not affect what you can/cannot do in
a different module. It's also a rule that;'s surprisingly hard to implement. Gilad Bracha, former keeper of the Java spec argued that it was basically impossible to combine this rule with inheritance. Looking at the JVM's more limited model
it took a major language revision with JDK changes to make it sort-of work. So it might well be that without JDK changes we can never implement this rule completel for qualified privates. In any case let's not underestimate the effort it requires (my guess: a person year or two).

Cheers

 -- Martin
 

On Sun, Jan 30, 2011 at 10:28, martin odersky wrote:
>
>
> On Sun, Jan 30, 2011 at 9:04 AM, Cay Horstmann
> wrote:
>>
>> 2011/1/30 Paul Phillips :
>> > On Sat, Jan 29, 2011 at 11:40:36PM +0100, martin odersky wrote:
>> >> You are right. What I should have said: You can redefine `x', and they
>> >> are two different fields, but for `y' it's an override. So `y' must be
>> >> type and accessibility conformant to the definition of the superclass,
>> >> but the two definitions of `x' could have completely different types.
>> >
>> > I assume we're zeroing in on implementation bugs then.  Again in all
>> > versions of scala, this compiles.
>> >
>> > class Q {
>> >  private var x: Int = 0
>> >  private[Q] var y: Int = 0
>> > }
>> > class QQ extends Q {
>> >  var x: String = "abc"
>> >  var y: String = "abc"
>> > }
>> >
>
> Yes, looks like it. We should go to the bottom of that with some urgency!
>>
>> In Java, a golden rule is "if you can't see it, you can't override it".
>
> That's a good rule, yes. Generally, if you can't see it, it should not
> affect what you can/cannot do in
> a different module. It's also a rule that;'s surprisingly hard to implement.
> Gilad Bracha, former keeper of the Java spec argued that it was basically
> impossible to combine this rule with inheritance. Looking at the JVM's more
> limited model
> it took a major language revision with JDK changes to make it sort-of work.
> So it might well be that without JDK changes we can never implement this
> rule completel for qualified privates. In any case let's not underestimate
> the effort it requires (my guess: a person year or two).

That's really interesting. Could you give me some pointers to learn
more about the issue? Did Bracha argue this in a paper?

Cheers,
Olivier

On Sun, Jan 30, 2011 at 12:11 PM, Olivier Pernet <omp08@doc.ic.ac.uk> wrote:

On Sun, Jan 30, 2011 at 10:28, martin odersky <martin.odersky@epfl.ch> wrote:
>
>
> On Sun, Jan 30, 2011 at 9:04 AM, Cay Horstmann <cay.horstmann@gmail.com>
> wrote:
>>
>> 2011/1/30 Paul Phillips <paulp@improving.org>:
>> > On Sat, Jan 29, 2011 at 11:40:36PM +0100, martin odersky wrote:
>> >> You are right. What I should have said: You can redefine `x', and they
>> >> are two different fields, but for `y' it's an override. So `y' must be
>> >> type and accessibility conformant to the definition of the superclass,
>> >> but the two definitions of `x' could have completely different types.
>> >
>> > I assume we're zeroing in on implementation bugs then.  Again in all
>> > versions of scala, this compiles.
>> >
>> > class Q {
>> >  private var x: Int = 0
>> >  private[Q] var y: Int = 0
>> > }
>> > class QQ extends Q {
>> >  var x: String = "abc"
>> >  var y: String = "abc"
>> > }
>> >
>
> Yes, looks like it. We should go to the bottom of that with some urgency!
>>
>> In Java, a golden rule is "if you can't see it, you can't override it".
>
> That's a good rule, yes. Generally, if you can't see it, it should not
> affect what you can/cannot do in
> a different module. It's also a rule that;'s surprisingly hard to implement.
> Gilad Bracha, former keeper of the Java spec argued that it was basically
> impossible to combine this rule with inheritance. Looking at the JVM's more
> limited model
> it took a major language revision with JDK changes to make it sort-of work.
> So it might well be that without JDK changes we can never implement this
> rule completel for qualified privates. In any case let's not underestimate
> the effort it requires (my guess: a person year or two).

That's really interesting. Could you give me some pointers to learn
more about the issue? Did Bracha argue this in a paper?

I was trying to find one just a minute ago, but could not. Key terms I searched for were be abstract [data] types and inheritance. But I only see earlier work of his in the JigSaw context. So maybe he did not write it up..

 -- Martin

On Sat, Jan 29, 2011 at 5:57 PM, Paul Phillips <paulp@improving.org> wrote:

I assume we're zeroing in on implementation bugs then.  Again in all
versions of scala, this compiles.

class Q {
 private var x: Int = 0
 private[Q] var y: Int = 0
}
class QQ extends Q {
 var x: String = "abc"
 var y: String = "abc"
}

As far as I can tell here there is no problem.  Private entities should appear invisible to descendant classes, there's no override problems here at all:
class Q {  private var x: Int = 0 private[Q] var y: Int = 0 def hiddenX { println(x) } def hiddenY { println(x) }}class QQ extends Q { var x: String = "abc"  var y: String = "abc"}
val qq = new QQqq.hiddenX=> 0qq.hiddenY=> 0
class QQQ extends Q {  var x:Int = 1 var y:Int = 2}
val qqq = new QQQqqq.hiddenX=> 0qqq.hiddenY=> 0
class QQQQ extends Q {  override var x:Int = 1 override var y:Int = 2}=>error: method x overrides nothing        override var x:Int = 1error: method y overrides nothing         override var y:Int = 2
Unless I'm mistaken this all appears to be the correct behavior.
--
Jim Powers