- About Scala
- Documentation
- Code Examples
- Software
- Scala Developers
Scala modularisation withdrawl
Tue, 2009-10-20, 00:32
The Scala modularisation proposal was put forward in the hope of avoiding some of Java's mistakes in the past, most notably the failure to split up the runtime into more modular components. In order to modularise, it's necessary for there to be binary backward compatibility between modules as otherwise this defeats the point of modularisation, since you just end up with a fragmented set of JARs that are coupled tightly together through their binary interactions.
In addition, it was hoped to avoid Java's mistake of using a constant 1.x version number, regardless of what happened with the underlying implementation. In part, the fault for this lay with the marketeers who coined the Java 2 moniker, which essentially prevented the existence of a Java 2.x release ever in the future. Learning from others mistakes, and in particular, the difference between the version of a package/module and a marketing release, is something that would be well heeded here.
Sadly, we are not in the position today to move Scala's modularisation forward. There is a lot of fragility in the binary output of the Scala compiler, and although it would be possible to encode modular constraints between the compiler and the runtime, the leaking of implementation details of both traits and implicits has been shown to be problematic (such as the Java5/Java6 builds being spun off at the moment for the same release). The last straw of this is the choice of version number for the next release of Scala, which not only causes backward compatibility breaks but also changes (and in some cases, completely removes) deprecated code.
Pretty much any other project, with the possible exception of Java itself, would recognise this as a fairly big hit to backward compatibility and bump it up to a 3.0 release. Instead, we're left with a stream of 2.x releases, each more incompatible than the last; and with each time a corporation gets burnt in the 2.x to 2.x+1 migration (whereby the have to recompile everything just to make sure it works) it will be one more reason not to trust Scala in the future.
What is far more telling, however, is the reasons against doing this. "too much has been said and printed about Scala 2.8" and "My personal quality standards for a '3' are a lot higher than for a '2.8'" are essentially the only arguments being put forward. When one basis policy on gut feelings and mentions on webpages (http://www.scala-lang.org/node/1564) rather than commonly accepted semantics for version numbers, it should not be a surprise when the release engineering process for Scala is called into question.
In any case, I have no further input of value on the modularisation work; I'll leave the requirements distilled at http://wiki.github.com/alblue/scala/scalamodularisation in case someone wants to clone them before I drop the repository at the end of the month.
Alex
In addition, it was hoped to avoid Java's mistake of using a constant 1.x version number, regardless of what happened with the underlying implementation. In part, the fault for this lay with the marketeers who coined the Java 2 moniker, which essentially prevented the existence of a Java 2.x release ever in the future. Learning from others mistakes, and in particular, the difference between the version of a package/module and a marketing release, is something that would be well heeded here.
Sadly, we are not in the position today to move Scala's modularisation forward. There is a lot of fragility in the binary output of the Scala compiler, and although it would be possible to encode modular constraints between the compiler and the runtime, the leaking of implementation details of both traits and implicits has been shown to be problematic (such as the Java5/Java6 builds being spun off at the moment for the same release). The last straw of this is the choice of version number for the next release of Scala, which not only causes backward compatibility breaks but also changes (and in some cases, completely removes) deprecated code.
Pretty much any other project, with the possible exception of Java itself, would recognise this as a fairly big hit to backward compatibility and bump it up to a 3.0 release. Instead, we're left with a stream of 2.x releases, each more incompatible than the last; and with each time a corporation gets burnt in the 2.x to 2.x+1 migration (whereby the have to recompile everything just to make sure it works) it will be one more reason not to trust Scala in the future.
What is far more telling, however, is the reasons against doing this. "too much has been said and printed about Scala 2.8" and "My personal quality standards for a '3' are a lot higher than for a '2.8'" are essentially the only arguments being put forward. When one basis policy on gut feelings and mentions on webpages (http://www.scala-lang.org/node/1564) rather than commonly accepted semantics for version numbers, it should not be a surprise when the release engineering process for Scala is called into question.
In any case, I have no further input of value on the modularisation work; I'll leave the requirements distilled at http://wiki.github.com/alblue/scala/scalamodularisation in case someone wants to clone them before I drop the repository at the end of the month.
Alex
Tue, 2009-10-20, 00:57
#2
Re: Scala modularisation withdrawl
Do you have a link to the "commonly accepted semantics for version numbers" you refer to?
--j
On Mon, Oct 19, 2009 at 4:31 PM, Alex Blewitt <alex.blewitt@gmail.com> wrote:
--j
On Mon, Oct 19, 2009 at 4:31 PM, Alex Blewitt <alex.blewitt@gmail.com> wrote:
The Scala modularisation proposal was put forward in the hope of avoiding some of Java's mistakes in the past, most notably the failure to split up the runtime into more modular components. In order to modularise, it's necessary for there to be binary backward compatibility between modules as otherwise this defeats the point of modularisation, since you just end up with a fragmented set of JARs that are coupled tightly together through their binary interactions.
In addition, it was hoped to avoid Java's mistake of using a constant 1.x version number, regardless of what happened with the underlying implementation. In part, the fault for this lay with the marketeers who coined the Java 2 moniker, which essentially prevented the existence of a Java 2.x release ever in the future. Learning from others mistakes, and in particular, the difference between the version of a package/module and a marketing release, is something that would be well heeded here.
Sadly, we are not in the position today to move Scala's modularisation forward. There is a lot of fragility in the binary output of the Scala compiler, and although it would be possible to encode modular constraints between the compiler and the runtime, the leaking of implementation details of both traits and implicits has been shown to be problematic (such as the Java5/Java6 builds being spun off at the moment for the same release). The last straw of this is the choice of version number for the next release of Scala, which not only causes backward compatibility breaks but also changes (and in some cases, completely removes) deprecated code.
Pretty much any other project, with the possible exception of Java itself, would recognise this as a fairly big hit to backward compatibility and bump it up to a 3.0 release. Instead, we're left with a stream of 2.x releases, each more incompatible than the last; and with each time a corporation gets burnt in the 2.x to 2.x+1 migration (whereby the have to recompile everything just to make sure it works) it will be one more reason not to trust Scala in the future.
What is far more telling, however, is the reasons against doing this. "too much has been said and printed about Scala 2.8" and "My personal quality standards for a '3' are a lot higher than for a '2.8'" are essentially the only arguments being put forward. When one basis policy on gut feelings and mentions on webpages (http://www.scala-lang.org/node/1564) rather than commonly accepted semantics for version numbers, it should not be a surprise when the release engineering process for Scala is called into question.
In any case, I have no further input of value on the modularisation work; I'll leave the requirements distilled at http://wiki.github.com/alblue/scala/scalamodularisation in case someone wants to clone them before I drop the repository at the end of the month.
Alex
Tue, 2009-10-20, 01:07
#3
Re: Scala modularisation withdrawl
On 20 Oct 2009, at 00:54, Jorge Ortiz wrote:
> Do you have a link to the "commonly accepted semantics for version
> numbers" you refer to?
http://alblue.blogspot.com/2008/05/version-numbers-and-jsr277.html
http://wiki.eclipse.org/Version_Numbering
http://www.osgi.org/blog/2008/05/is-9903520300447984150353281023-too.html
http://en.wikipedia.org/wiki/Software_versioning
http://apr.apache.org/versioning.html
These are just a selection but all cover the basic concept of the
major version being used to denote incompatible changes.
Alex
Tue, 2009-10-20, 01:27
#4
Re: Scala modularisation withdrawl
In this case, all major scala libraries should always use major version numbers, as traits are inherently difficult to keep backwards compatible.
I don't think most major libraries are in the habit of speaking in "abstract trait"'s (which BTW in scala 2.7.1 compiles to a "scala-less" interface). The problem here is that you'd have to promote a set of abstract traits for the standard library, so that you could pass around things like List, Set, etc. Then you have to redefine your APIs to speak with these pure abstract interfaces... Then you need to promote them to their own module such that you can have differing versions of scala communicate with one another.
It's no insurmountable, only a lot of work. Personally, I think it might be handy to wait until a method of "backwards compatable traits" is devised, as this will resolve a lot of the issues.
I also think it's too early to throw the version stick at Scala. We shouldn't expect 2.7 -> 2.8 change to be any worse than 2.6 -> 2.7. For the most part, it's source compatable (talk to Mark Harrah about the magic he does here).
Also, if you watch the scala communities, they tend to move as one large amoeba (with many pseudopods) towards the most recent Scala release. Usually if you're not running the latest release, you can't be guaranteed bug fixes or features. However, this is how things are done in a world without guaranteed binary compatability.
I think perhaps we need to see more SIDs on binary compatabiltiy. I'd love to see the community contribute some ideas here, as this is largely a pain point that we feel (but no the compiler devs, due to their workarounds). We're even planning a discussion around this at the upcoming Scala Lift Off in Reston, VA.
Anyway, I'm sorry to see you so dissillusioned, but please don't give up on Scala! It's an evolving thing, and there are still some peices that need to be improved.
- Josh
On Mon, Oct 19, 2009 at 8:02 PM, Alex Blewitt <alex.blewitt@gmail.com> wrote:
I don't think most major libraries are in the habit of speaking in "abstract trait"'s (which BTW in scala 2.7.1 compiles to a "scala-less" interface). The problem here is that you'd have to promote a set of abstract traits for the standard library, so that you could pass around things like List, Set, etc. Then you have to redefine your APIs to speak with these pure abstract interfaces... Then you need to promote them to their own module such that you can have differing versions of scala communicate with one another.
It's no insurmountable, only a lot of work. Personally, I think it might be handy to wait until a method of "backwards compatable traits" is devised, as this will resolve a lot of the issues.
I also think it's too early to throw the version stick at Scala. We shouldn't expect 2.7 -> 2.8 change to be any worse than 2.6 -> 2.7. For the most part, it's source compatable (talk to Mark Harrah about the magic he does here).
Also, if you watch the scala communities, they tend to move as one large amoeba (with many pseudopods) towards the most recent Scala release. Usually if you're not running the latest release, you can't be guaranteed bug fixes or features. However, this is how things are done in a world without guaranteed binary compatability.
I think perhaps we need to see more SIDs on binary compatabiltiy. I'd love to see the community contribute some ideas here, as this is largely a pain point that we feel (but no the compiler devs, due to their workarounds). We're even planning a discussion around this at the upcoming Scala Lift Off in Reston, VA.
Anyway, I'm sorry to see you so dissillusioned, but please don't give up on Scala! It's an evolving thing, and there are still some peices that need to be improved.
- Josh
On Mon, Oct 19, 2009 at 8:02 PM, Alex Blewitt <alex.blewitt@gmail.com> wrote:
On 20 Oct 2009, at 00:54, Jorge Ortiz wrote:
Do you have a link to the "commonly accepted semantics for version numbers" you refer to?
http://alblue.blogspot.com/2008/05/version-numbers-and-jsr277.html
http://wiki.eclipse.org/Version_Numbering
http://www.osgi.org/blog/2008/05/is-9903520300447984150353281023-too.html
http://en.wikipedia.org/wiki/Software_versioning
http://apr.apache.org/versioning.html
These are just a selection but all cover the basic concept of the major version being used to denote incompatible changes.
Alex
Tue, 2009-10-20, 01:37
#5
Re: Scala modularisation withdrawl
On 20 Oct 2009, at 01:25, Josh Suereth wrote:
> In this case, all major scala libraries should always use major
> version numbers, as traits are inherently difficult to keep
> backwards compatible.
I suspect that it should be possible to evolve traits in a compatible
way, in much the same way that Eclipse evolves APIs [1] (both for
abstract classes and interfaces). A set of guidelines might help with
that aspect.
But to your first part; yes, that's exactly the point I was making.
The next version of Scala should be called Scala 3.0, because it's
backwardly incompatible. If the next version after that is backwardly
incompatible, it should be Scala 4.0, and so on. If we ever get the
magic bullet of backward compatibility, then we could move to a Scala
4.1 etc.
In that way, dependent libraries (or modules) could express a
dependency on Scala [3.0,4.0) knowing that they'd be able to depend on
backward compatible behaviour of newer versions, whilst ruling out an
accidental dependency on a backward incompatible change. Right now, we
don't have that and so each version of a library is tied to a specific
version of the compliler and runtime, all to avoid changing the 2.x
prefix.
Tue, 2009-10-20, 01:47
#6
Re: Scala modularisation withdrawl
On Tue, Oct 20, 2009 at 1:31 AM, Alex Blewitt wrote:
> Right now, we don't have that and so each
> version of a library is tied to a specific version of the compliler and
> runtime, all to avoid changing the 2.x prefix.
This is true.
However, I see no reason for this to block a modularization effort.
Cheers,
Miles
Tue, 2009-10-20, 01:57
#7
Re: Scala modularisation withdrawl
Here's a proposal:
Scala is as-yet binary incompatible. Hence, every release (even the most minor), should have a major version bump.
So, instead of Scala 2.7.0, call it Scala 270. Likewise 2.7.1 becomes 271, etc.
The next major release of Scala is 280. A major version upgrade, because of binary incompatibility.
Problem solved.
--j
On Mon, Oct 19, 2009 at 5:31 PM, Alex Blewitt <alex.blewitt@gmail.com> wrote:
Scala is as-yet binary incompatible. Hence, every release (even the most minor), should have a major version bump.
So, instead of Scala 2.7.0, call it Scala 270. Likewise 2.7.1 becomes 271, etc.
The next major release of Scala is 280. A major version upgrade, because of binary incompatibility.
Problem solved.
--j
On Mon, Oct 19, 2009 at 5:31 PM, Alex Blewitt <alex.blewitt@gmail.com> wrote:
On 20 Oct 2009, at 01:25, Josh Suereth wrote:
In this case, all major scala libraries should always use major version numbers, as traits are inherently difficult to keep backwards compatible.
I suspect that it should be possible to evolve traits in a compatible way, in much the same way that Eclipse evolves APIs [1] (both for abstract classes and interfaces). A set of guidelines might help with that aspect.
But to your first part; yes, that's exactly the point I was making. The next version of Scala should be called Scala 3.0, because it's backwardly incompatible. If the next version after that is backwardly incompatible, it should be Scala 4.0, and so on. If we ever get the magic bullet of backward compatibility, then we could move to a Scala 4.1 etc.
In that way, dependent libraries (or modules) could express a dependency on Scala [3.0,4.0) knowing that they'd be able to depend on backward compatible behaviour of newer versions, whilst ruling out an accidental dependency on a backward incompatible change. Right now, we don't have that and so each version of a library is tied to a specific version of the compliler and runtime, all to avoid changing the 2.x prefix.
[1] http://wiki.eclipse.org/Evolving_Java-based_APIs
Tue, 2009-10-20, 02:07
#8
Re: Scala modularisation withdrawl
On Tue, Oct 20, 2009 at 1:48 AM, Jorge Ortiz wrote:
> Scala is as-yet binary incompatible. Hence, every release (even the most
> minor), should have a major version bump.
>
> So, instead of Scala 2.7.0, call it Scala 270. Likewise 2.7.1 becomes 271,
> etc.
>
> The next major release of Scala is 280. A major version upgrade, because of
> binary incompatibility.
>
> Problem solved.
If that's a solution, then the problem is obviously very shallow ;-)
Cheers,
Miles
Tue, 2009-10-20, 16:17
#9
Re: Scala modularisation withdrawl
Well, Alex, there's just a minor glitch with that. Libraries can't express dependency on Scala[2.0,3.0), because they are all backward-incompatible. So we gain nothing by turning Scala 2.8 into Scala 3.0.
On Mon, Oct 19, 2009 at 10:31 PM, Alex Blewitt <alex.blewitt@gmail.com> wrote:
--
Daniel C. Sobral
Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.
On Mon, Oct 19, 2009 at 10:31 PM, Alex Blewitt <alex.blewitt@gmail.com> wrote:
On 20 Oct 2009, at 01:25, Josh Suereth wrote:
In this case, all major scala libraries should always use major version numbers, as traits are inherently difficult to keep backwards compatible.
I suspect that it should be possible to evolve traits in a compatible way, in much the same way that Eclipse evolves APIs [1] (both for abstract classes and interfaces). A set of guidelines might help with that aspect.
But to your first part; yes, that's exactly the point I was making. The next version of Scala should be called Scala 3.0, because it's backwardly incompatible. If the next version after that is backwardly incompatible, it should be Scala 4.0, and so on. If we ever get the magic bullet of backward compatibility, then we could move to a Scala 4.1 etc.
In that way, dependent libraries (or modules) could express a dependency on Scala [3.0,4.0) knowing that they'd be able to depend on backward compatible behaviour of newer versions, whilst ruling out an accidental dependency on a backward incompatible change. Right now, we don't have that and so each version of a library is tied to a specific version of the compliler and runtime, all to avoid changing the 2.x prefix.
[1] http://wiki.eclipse.org/Evolving_Java-based_APIs
--
Daniel C. Sobral
Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.
Tue, 2009-10-20, 16:37
#10
Re: Scala modularisation withdrawl
Here's the thing, IMHO. The modularization effort has a laudable goal, but is has just started. As it stands right now, each individual version of Scala is, at most, source compatible with the previous one. And I very much doubt the NEXT version of Scala should be any different, as the necessary effort to achieve that goal isn't even on-going as yet.
So, I'm all for setting future targets as a result of the just started modularization effort. And I'm all against making changes to a decision that dates back one year, and has _practical_ consequences with regards to user confusion, given all the resources that have been making references to Scala 2.8 for so long. Consequences which are not as minor as you put them.
Furthermore, there is simply no immediate gain. Scala 2.8 (or 3.0) won't be modularized yet. The next version is as likely to break compatibility as this one. And, in fact, Scala 2.8 is being much of a stepping stone to really big changes in the future.
So fight for the decision to make all backward-incompatible versions of Scala major releases _from here on_.
Note: I have no connection to the EPFL people. I'm just a user.
On Mon, Oct 19, 2009 at 9:31 PM, Alex Blewitt <alex.blewitt@gmail.com> wrote:
--
Daniel C. Sobral
Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.
On Mon, Oct 19, 2009 at 9:31 PM, Alex Blewitt <alex.blewitt@gmail.com> wrote:
The Scala modularisation proposal was put forward in the hope of avoiding some of Java's mistakes in the past, most notably the failure to split up the runtime into more modular components. In order to modularise, it's necessary for there to be binary backward compatibility between modules as otherwise this defeats the point of modularisation, since you just end up with a fragmented set of JARs that are coupled tightly together through their binary interactions.
In addition, it was hoped to avoid Java's mistake of using a constant 1.x version number, regardless of what happened with the underlying implementation. In part, the fault for this lay with the marketeers who coined the Java 2 moniker, which essentially prevented the existence of a Java 2.x release ever in the future. Learning from others mistakes, and in particular, the difference between the version of a package/module and a marketing release, is something that would be well heeded here.
Sadly, we are not in the position today to move Scala's modularisation forward. There is a lot of fragility in the binary output of the Scala compiler, and although it would be possible to encode modular constraints between the compiler and the runtime, the leaking of implementation details of both traits and implicits has been shown to be problematic (such as the Java5/Java6 builds being spun off at the moment for the same release). The last straw of this is the choice of version number for the next release of Scala, which not only causes backward compatibility breaks but also changes (and in some cases, completely removes) deprecated code.
Pretty much any other project, with the possible exception of Java itself, would recognise this as a fairly big hit to backward compatibility and bump it up to a 3.0 release. Instead, we're left with a stream of 2.x releases, each more incompatible than the last; and with each time a corporation gets burnt in the 2.x to 2.x+1 migration (whereby the have to recompile everything just to make sure it works) it will be one more reason not to trust Scala in the future.
What is far more telling, however, is the reasons against doing this. "too much has been said and printed about Scala 2.8" and "My personal quality standards for a '3' are a lot higher than for a '2.8'" are essentially the only arguments being put forward. When one basis policy on gut feelings and mentions on webpages (http://www.scala-lang.org/node/1564) rather than commonly accepted semantics for version numbers, it should not be a surprise when the release engineering process for Scala is called into question.
In any case, I have no further input of value on the modularisation work; I'll leave the requirements distilled at http://wiki.github.com/alblue/scala/scalamodularisation in case someone wants to clone them before I drop the repository at the end of the month.
Alex
--
Daniel C. Sobral
Something I learned in academia: there are three kinds of academic reviews: review by name, review by reference and review by value.
On Tue, Oct 20, 2009 at 12:31 AM, Alex Blewitt wrote:
> I'll leave the requirements distilled
> at http://wiki.github.com/alblue/scala/scalamodularisation in case someone
> wants to clone them before I drop the repository at the end of the month.
Sure, I'll take them over ...
I have to say I found your post excessively pessimistic ... I hope
you'll reconsider and continue your involvement.
Cheers,
Miles