Countdown to 2.8

Excellent news! With the signature changes, I hope the relevant tools
will catch up during the RC period.

On Saturday, March 20, 2010, martin odersky wrote:
> Hi all,
>
> I wanted to give you some update on the progress towards 2.8. We are
> essentially feature complete now. There have been two late changes
> which might affect builds.
>
> One change is the move of ScalaSignatures from classfile attributes to
> annotations. This should affect only tools that read the signatures
> (scalap, IntelliJ, maybe other IDEs?).
>
> Then there are some last minute changes to collections. In a number of
> cases, we wanted to change the meaning of an operation. Examples are
> Buffer.+ (is now copying) or Map.keys (now gives an Iterable).
> Previously, we planned to deprecate these methods in 2.8 (they are
> deprecated in the beta), remove them in some later version and then
> reintroduce them even later after that. With -Xmigration this process
> can now be sped up dramatically. We'll change the behavior of these
> methods now, and provide the -Xmigration option to tell you where
> meanings have changed.
>
> For users of collection classes this means that if your code compiled
> against 2.8 beta without deprecation warnings you should be OK. If you
> had deprecation warnings you should have a close look at the output of
> scalac -Xmigration. But it's a good idea anyway to compile everything
> with -Xmigration at least once.
>
> We are currently busy fixing bugs and plan to release a first RC
> either next week or the week after that. We'll discuss the release
> date on the next Scala meeting, Tuesday, March 23rd. If you need to
> commit some stuff that would delay an RC after Thursday, March 25th,
> we'd need your input by this Tuesday. Once we have the RC out, we'd
> like to get feedback from all the major tool maintainers whether their
> stuff runs with it. We'll not go to final without having received that
> feedback.
>
> Cheers
>
>  -- Martin
> Reply
>
> Forward
>
>
>
> Reply
>
> |
> Mail Delivery Subsystem
>  to me
>
> show details 12:27 PM (0 minutes ago)
>
> Delivery to the following recipient failed permanently:
>
>    scala-internals@epfl.ch
>
> Technical details of permanent failure:
> Google tried to deliver your message, but it was rejected by the
> recipient domain. We recommend contacting the other email provider for
> further information about the cause of this error. The error that the
> other server returned was: 550 550 unknown
> mailbox (state 14).
>
> ----- Original message -----
>
> MIME-Version: 1.0
> Sender: odersky@gmail.com
> Received: by 10.141.107.18 with SMTP id j18mr2427741rvm.283.1269084431143;
>       Sat, 20 Mar 2010 04:27:11 -0700 (PDT)
> Date: Sat, 20 Mar 2010 12:27:11 +0100
> X-Google-Sender-Auth: 8179dc545e45aefc
> Message-ID: <9461d7d01003200427t7b09570mbc2cd961eed8e10f@mail.gmail.com>
> Subject: Countdown for 2.8
> From: martin odersky
> To: scala-internals@epfl.ch
> Content-Type: text/plain; charset=ISO-8859-1
> - Show quoted text -
>
> Hi all,
>
> I wanted to give you some update on the progress towards 2.8. We are
> essentially feature complete now. There have been two late changes
> which might affect builds.
>
> One change is the move of ScalaSignatures from classfile attributes to
> annotations. This should affect only tools that read the signatures
> (scalap, IntelliJ, maybe other IDEs?).
>
> Then there are some last minute changes to collections. In a number of
> cases, we wanted to change the meaning of an operation. Examples are
> Buffer.+ (is now copying) or Map.keys (now gives an Iterable).
> Previously, we planned to deprecate these methods in 2.8 (they are
> deprecated in the beta), remove them in some later version and then
> reintroduce them even later after that. With -Xmigration this process
> can now be sped up dramatically. We'll change the behavior of these
> methods now, and provide the -Xmigration option to tell you where
> meanings have changed.
>
> For users of collection classes this means that if your code compiled
> against 2.8 beta without deprecation warnings you should be OK. If you
> had deprecation warnings you should have a close look at the output of
> scalac -Xmigration. But it's a good idea anyway to compile everything
> with -Xmigration at least once.
>
> We are currently busy fixing bugs and plan to release a first RC
> either next week or the week after that. We'll discuss the release
> date on the next Scala meeting, Tuesday, March 23rd. If you need to
> commit some stuff that would delay an RC after Thursday, March 25th,
> we'd need your input by this Tuesday. Once we have the RC out, we'd
> like to get feedback from all the major tool maintainers whether their
> stuff runs with it. We'll not go to final without having received that
> feedback.
>
> Cheers
>
>  -- Martin
>

Martin,
I'm looking forward to the first RC and I'm also looking forward to a final release of 2.8.
However, I urge caution in the release timing.
We have done a partial port of Lift to Scala 2.8.0-Beta1.  Once the first RC is released we will finish the port of Lift to 2.8.  Then I will test some of my apps in production and ask some other Lift users to do the same.
If we see stability, then we will spin a milestone build of Lift against 2.7.7 and swap SNAPSHOT (the main development branch of Lift) to 2.8-RCx  That will give many more in the Lift community to migrate to 2.8-RCx.  After a few weeks of testing, bug reporting, etc. we'll be able to give the Scala team reasonable feedback.
So, based on the above, the best case I could see (factoring in Lift community feedback), there will be a 6 week RC-release cycle.
Thanks,
David
On Sat, Mar 20, 2010 at 4:28 AM, martin odersky <martin.odersky@epfl.ch> wrote:

Hi all,

I wanted to give you some update on the progress towards 2.8. We are
essentially feature complete now. There have been two late changes
which might affect builds.

One change is the move of ScalaSignatures from classfile attributes to
annotations. This should affect only tools that read the signatures
(scalap, IntelliJ, maybe other IDEs?).

Then there are some last minute changes to collections. In a number of
cases, we wanted to change the meaning of an operation. Examples are
Buffer.+ (is now copying) or Map.keys (now gives an Iterable).
Previously, we planned to deprecate these methods in 2.8 (they are
deprecated in the beta), remove them in some later version and then
reintroduce them even later after that. With -Xmigration this process
can now be sped up dramatically. We'll change the behavior of these
methods now, and provide the -Xmigration option to tell you where
meanings have changed.

For users of collection classes this means that if your code compiled
against 2.8 beta without deprecation warnings you should be OK. If you
had deprecation warnings you should have a close look at the output of
scalac -Xmigration. But it's a good idea anyway to compile everything
with -Xmigration at least once.

We are currently busy fixing bugs and plan to release a first RC
either next week or the week after that. We'll discuss the release
date on the next Scala meeting, Tuesday, March 23rd. If you need to
commit some stuff that would delay an RC after Thursday, March 25th,
we'd need your input by this Tuesday. Once we have the RC out, we'd
like to get feedback from all the major tool maintainers whether their
stuff runs with it. We'll not go to final without having received that
feedback.

Cheers

On Sat, Mar 20, 2010 at 12:28:59PM +0100, martin odersky wrote:
> We are currently busy fixing bugs and plan to release a first RC
> either next week or the week after that. We'll discuss the release
> date on the next Scala meeting, Tuesday, March 23rd. If you need to
> commit some stuff that would delay an RC after Thursday, March 25th,
> we'd need your input by this Tuesday.

Although the list of things I might bring up is long, the one thing I
would really like to definitely have straight before another beta or RC
or whatever is stamped is classpaths. I know the work I've been doing
has made life (temporarily) harder for tool providers like mharrah, and
I'm close enough that the next release ought to have some clear
classpath defining semantics / cmd line options / properties which we
can expect to keep working.

The last ad-hoc usage of classpaths in trunk which I couldn't easily
tame was in partest, and rather than fiddling with it any longer I took
a longer-than-I-intended detour to rewrite it.

After getting 95% done and dropping 1000+ lines of code relative to the
original I became bogged down in ancillary issues like "why does the
build manager refuse to see that this file has changed" so I'm holding
at 95% for a second. I took yesterday off from that to recharge my
tedium batteries and distract myself with backlogged issues.

All of it as soon as I can manage it. My life situation doesn't give me
the luxury of choosing when I can give scala my undivided attention, and
it's not offering that right now.

One technical note:

> Buffer.+ (is now copying)

Actually that is the one thing which is not now copying. I would like
for it to copy, but I also thought it was slotted for total removal
because it's in the Seq hierarchy, and I couldn't both @deprecated and
@migration it (well I could, but they don't both get printed and it
seems a bad idea in light of that.) Here's the big source comment I have
in there right now.

// !!! Note that the two + methods below presently still modify in place. Arguably they
// should be changed to create new collections, and then have BOTH @migration and
// @deprecation annotations, because Seq-derived classes don't implement + since
// most of them are covariant and + interacts badly with +'s autostringification.
// However the compiler right now will not warn about both annotations (see the
// implementation comment) so I don't think that's an option without fixing that first.
//
// In addition, I don't think BufferLike should implement Addable since it
// deprecates all of it, and Addable has never shipped.
//
// Alternate implementations:
//
// clone() += elem
// clone() += elem1 += elem2 ++= elems

On Sat, Mar 20, 2010 at 7:53 PM, Paul Phillips wrote:
> On Sat, Mar 20, 2010 at 12:28:59PM +0100, martin odersky wrote:
>> We are currently busy fixing bugs and plan to release a first RC
>> either next week or the week after that. We'll discuss the release
>> date on the next Scala meeting, Tuesday, March 23rd. If you need to
>> commit some stuff that would delay an RC after Thursday, March 25th,
>> we'd need your input by this Tuesday.
>
> Although the list of things I might bring up is long, the one thing I
> would really like to definitely have straight before another beta or RC
> or whatever is stamped is classpaths.  I know the work I've been doing
> has made life (temporarily) harder for tool providers like mharrah, and
> I'm close enough that the next release ought to have some clear
> classpath defining semantics / cmd line options / properties which we
> can expect to keep working.
>
> The last ad-hoc usage of classpaths in trunk which I couldn't easily
> tame was in partest, and rather than fiddling with it any longer I took
> a longer-than-I-intended detour to rewrite it.
>
> After getting 95% done and dropping 1000+ lines of code relative to the
> original I became bogged down in ancillary issues like "why does the
> build manager refuse to see that this file has changed" so I'm holding
> at 95% for a second.  I took yesterday off from that to recharge my
> tedium batteries and distract myself with backlogged issues.
>
> All of it as soon as I can manage it.  My life situation doesn't give me
> the luxury of choosing when I can give scala my undivided attention, and
> it's not offering that right now.
>
OK, fair enough. Can you give us a status update by next tuesday? If
it's not netx to done by then, no problem we'll hold off another week
with the RC.

> One technical note:
>
>> Buffer.+ (is now copying)
>
> Actually that is the one thing which is not now copying.  I would like
> for it to copy, but I also thought it was slotted for total removal
> because it's in the Seq hierarchy, and I couldn't both @deprecated and
> @migration it (well I could, but they don't both get printed and it
> seems a bad idea in light of that.) Here's the big source comment I have
> in there right now.
>
Yes, you are right. Your comment makes sense.

Cheers

Hi, Martin.
Since IntelliJ fully relies to scalap functionality, appropriate changes have been made in the scalap decompilation logic. Now it supports both old and new ways of pickling scala signatures. I've sent the patch to Gilles and it will be applied hopefully with these changes on ScalaSignatures movements.
With best regards,Ilya

On 20 March 2010 12:28, martin odersky <martin.odersky@epfl.ch> wrote:

Hi all,

I wanted to give you some update on the progress towards 2.8. We are
essentially feature complete now. There have been two late changes
which might affect builds.

One change is the move of ScalaSignatures from classfile attributes to
annotations. This should affect only tools that read the signatures
(scalap, IntelliJ, maybe other IDEs?).

Then there are some last minute changes to collections. In a number of
cases, we wanted to change the meaning of an operation. Examples are
Buffer.+ (is now copying) or Map.keys (now gives an Iterable).
Previously, we planned to deprecate these methods in 2.8 (they are
deprecated in the beta), remove them in some later version and then
reintroduce them even later after that. With -Xmigration this process
can now be sped up dramatically. We'll change the behavior of these
methods now, and provide the -Xmigration option to tell you where
meanings have changed.

For users of collection classes this means that if your code compiled
against 2.8 beta without deprecation warnings you should be OK. If you
had deprecation warnings you should have a close look at the output of
scalac -Xmigration. But it's a good idea anyway to compile everything
with -Xmigration at least once.

We are currently busy fixing bugs and plan to release a first RC
either next week or the week after that. We'll discuss the release
date on the next Scala meeting, Tuesday, March 23rd. If you need to
commit some stuff that would delay an RC after Thursday, March 25th,
we'd need your input by this Tuesday. Once we have the RC out, we'd
like to get feedback from all the major tool maintainers whether their
stuff runs with it. We'll not go to final without having received that
feedback.

Cheers

Great! Thanks for the quick update, Ilya. Looking forward to see you
soon in Lausanne.

Cheers

On Sun, Mar 21, 2010 at 12:29:58PM +0100, martin odersky wrote:
> OK, fair enough. Can you give us a status update by next tuesday? If
> it's not netx to done by then, no problem we'll hold off another week
> with the RC.

I have about 8 hours today and hopefully 10+ tomorrow, so a lot can
happen if I avoid quicksand. I'll update later today.

> Yes, you are right. Your comment makes sense.

I like making sense, but to translate that into possible actions:

Is my understanding in the comment correct, that we expect to eventually
remove '+' from all Seq-derived classes, including buffers and others
which are invariant?

And, given these options:

* Buffer.+ mutates, making it inconsistent with every other +/++/-/--
method in collections
* Buffer.+ creates new collection, changing behavior from 2.7 and
requiring multiple warnings because it's both deprecated and has changed
its semantics for the short period before it's removed

Should we not perhaps consider just apologizing and removing the '+'
method completely right now?

And if we want to copy and issue both warnings, is someone available to
tweak annotation / error handling so that an element marked with both
@migration and @deprecation will reliably emit both messages if both
compiler options are on? That might be trivial, I haven't looked too
closely so as not to get sidetracked.

On Mon, Mar 22, 2010 at 4:36 PM, Paul Phillips wrote:
> On Sun, Mar 21, 2010 at 12:29:58PM +0100, martin odersky wrote:
>> OK, fair enough. Can you give us a status update by next tuesday? If
>> it's not netx to done by then, no problem we'll hold off another week
>> with the RC.
>
> I have about 8 hours today and hopefully 10+ tomorrow, so a lot can
> happen if I avoid quicksand.  I'll update later today.
>
>> Yes, you are right. Your comment makes sense.
>
> I like making sense, but to translate that into possible actions:
>
> Is my understanding in the comment correct, that we expect to eventually
> remove '+' from all Seq-derived classes, including buffers and others
> which are invariant?
>
yes.

> And, given these options:
>
>  * Buffer.+ mutates, making it inconsistent with every other +/++/-/--
> method in collections

I'd keep it that way, yes. Nobody cares about the semantics of
deprecated methods (at least I don't).

>  * Buffer.+ creates new collection, changing behavior from 2.7 and
> requiring multiple warnings because it's both deprecated and has changed
> its semantics for the short period before it's removed
>
> Should we not perhaps consider just apologizing and removing the '+'
> method completely right now?
>
No. In practice Buffer.+ wasn't a problem so far. I don't see how it
should become a problem now that we deprecate it.

Cheers

On Mon, Mar 22, 2010 at 05:26:58PM +0100, martin odersky wrote:
> > Is my understanding in the comment correct, that we expect to eventually
> > remove '+' from all Seq-derived classes, including buffers and others
> > which are invariant?
> >
> yes.

OK, that requires some followup then.

Currently BufferLike extends Addable. We should not have Seq classes
implementing Addable if we know it is not their destiny to have a +
method, especially since Addable is new to 2.8. There's a bit of a
domino dependency when I go after that:

AddingBuilder[ ... To <: Addable[Elem, To] ...]

object ListBuffer extends SeqFactory[ListBuffer] {
def newBuilder[A]: Builder[A, ListBuffer[A]] = new AddingBuilder(new ListBuffer[A])
}

As I see it we would need a "GrowableBuilder" for collections which
implement '+=' but not '+'.

On an individual collection basis, PriorityQueue derives from both Seq
and Addable, and implements its own + methods -- which create copies,
and are not deprecated.

def +(elem: A): PriorityQueue[A] = { this.clone() += elem }
override def +(elem1: A, elem2: A, elems: A*) = { this.clone().+=(elem1, elem2, elems : _*) }

And PriorityQueue is not fully integrated, it has no companion object.

So here is the path I see to consistency. I propose to:

* BufferLike and PriorityQueue no longer implement Addable
* deprecate the + methods on PriorityQueue
* Create GrowableBuilder with the obvious implementation

Or .

On Mon, Mar 22, 2010 at 5:57 PM, Paul Phillips wrote:
> On Mon, Mar 22, 2010 at 05:26:58PM +0100, martin odersky wrote:
>> > Is my understanding in the comment correct, that we expect to eventually
>> > remove '+' from all Seq-derived classes, including buffers and others
>> > which are invariant?
>> >
>> yes.
>
> OK, that requires some followup then.
>
> Currently BufferLike extends Addable.  We should not have Seq classes
> implementing Addable if we know it is not their destiny to have a +
> method, especially since Addable is new to 2.8.  There's a bit of a
> domino dependency when I go after that:
>
>  AddingBuilder[ ... To <: Addable[Elem, To] ...]
>
>  object ListBuffer extends SeqFactory[ListBuffer] {
>    def newBuilder[A]: Builder[A, ListBuffer[A]] = new AddingBuilder(new ListBuffer[A])
>  }
>
> As I see it we would need a "GrowableBuilder" for collections which
> implement '+=' but not '+'.
>
> On an individual collection basis, PriorityQueue derives from both Seq
> and Addable, and implements its own + methods -- which create copies,
> and are not deprecated.
>
>  def +(elem: A): PriorityQueue[A] = { this.clone() += elem }
>  override def +(elem1: A, elem2: A, elems: A*) = { this.clone().+=(elem1, elem2, elems : _*) }
>
> And PriorityQueue is not fully integrated, it has no companion object.
>
> So here is the path I see to consistency.  I propose to:
>
>  * BufferLike and PriorityQueue no longer implement Addable
>  * deprecate the + methods on PriorityQueue
>  * Create GrowableBuilder with the obvious implementation
>
Yes, I am happy with all of this.

Cheers

On Sat, Mar 20, 2010 at 6:28 AM, martin odersky <martin.odersky@epfl.ch> wrote:

We are currently busy fixing bugs and plan to release a first RC

Will the package objects feature be expected to work? Last I read (trac 2905), it appears to not be a priority.

On Mon, Mar 22, 2010 at 7:09 PM, Nils Kilden-Pedersen wrote:
> On Sat, Mar 20, 2010 at 6:28 AM, martin odersky
> wrote:
>>
>> We are currently busy fixing bugs and plan to release a first RC
>
> Will the package objects feature be expected to work? Last I read (trac
> 2905), it appears to not be a priority.
>
Define `work'? Package objects are work in progress. There will be
further developments after 2.8 is shipped.

Cheers

On Mon, Mar 22, 2010 at 1:15 PM, martin odersky <martin.odersky@epfl.ch> wrote:

Define `work'?

I've found them to be really unstable on a variety of issues. I've had to give up on aliasing my own type names. It just doesn't work reliably, at least not in Eclipse.
 

Package objects are work in progress. There will be
further developments after 2.8 is shipped.

Is there a fixed set of features that we can expect will work when 2.8 is released, or is it considered experimental?

On Mon, Mar 22, 2010 at 8:22 PM, Nils Kilden-Pedersen wrote:
> I've found them to be really unstable on a variety of issues. I've had to
> give up on aliasing my own type names. It just doesn't work reliably, at
> least not in Eclipse.

This could be a presentation compiler issue in which case it would
only affect Eclipse ... but in the absence of a detailed bug report
it's a bit hard to tell.

Cheers,

Miles

On Tue, Mar 23, 2010 at 4:22 AM, Nils Kilden-Pedersen wrote:
> On Mon, Mar 22, 2010 at 1:15 PM, martin odersky
> wrote:
>>
>> Define `work'?
>
> I've found them to be really unstable on a variety of issues. I've had to
> give up on aliasing my own type names. It just doesn't work reliably, at
> least not in Eclipse.
>
>>
>> Package objects are work in progress. There will be
>> further developments after 2.8 is shipped.
>
> Is there a fixed set of features that we can expect will work when 2.8 is
> released, or is it considered experimental?
>
package objects should support type aliases and value/method
definitions. Overlap with locally defined objects and classes is
currently not supported. This will most likely change in the future,
however.

Cheers