Re: jvm5 build should be default?

We just decided in the Scala meeting to keep Java 1.5 as not only the
default, but the only version required by Scala 2.8. We will make use
of dynamic class loading and reflection to load the real FJ framework
instead of its 1.5 backport or java.util.concurrent. This decision
holds unless some unexpected problems come up with this dynamic
loading scheme.

Cheers

how much of this is going to impact scala-based development on google
android? i don't know the extents of the changes but actors work
great when i use scala 2.7.x on android. will 2.8.0, when it is
released, work as well as 2.7.x on android?

On Tue, Dec 1, 2009 at 10:56 PM, martin odersky wrote:
> We just decided in the Scala meeting to keep Java 1.5 as not only the
> default, but the only version required by Scala 2.8. We will make use
> of dynamic class loading and reflection to load the real FJ  framework
> instead of its 1.5 backport or java.util.concurrent. This decision
> holds unless some unexpected problems come up with this dynamic
> loading scheme.
>
> Cheers
>
>  -- Martin
>

>>>>> "martin" == martin odersky writes:

martin> We just decided in the Scala meeting to keep Java 1.5 as not
martin> only the default, but the only version required by Scala
martin> 2.8. We will make use of dynamic class loading and reflection
martin> to load the real FJ framework instead of its 1.5 backport or
martin> java.util.concurrent. This decision holds unless some
martin> unexpected problems come up with this dynamic loading scheme.

Very good news. Thanks for reconsidering.

Excellent!
Markus

> -----Original Message-----
> From: odersky@gmail.com [mailto:odersky@gmail.com] On Behalf Of martin
> odersky
> Sent: Dienstag, 1. Dezember 2009 15:56
> To: Kohler, Markus
> Cc: scala@listes.epfl.ch
> Subject: Re: [scala] Re: jvm5 build should be default?
>
> We just decided in the Scala meeting to keep Java 1.5 as not only the
> default, but the only version required by Scala 2.8. We will make use
> of dynamic class loading and reflection to load the real FJ framework
> instead of its 1.5 backport or java.util.concurrent. This decision
> holds unless some unexpected problems come up with this dynamic
> loading scheme.
>
> Cheers
>

On Tue, Dec 1, 2009 at 6:56 AM, martin odersky <martin.odersky@epfl.ch> wrote:

We just decided in the Scala meeting to keep Java 1.5 as not only the
default, but the only version required by Scala 2.8. We will make use
of dynamic class loading and reflection to load the real FJ  framework
instead of its 1.5 backport or java.util.concurrent. This decision
holds unless some unexpected problems come up with this dynamic
loading scheme.

Excellent.   I think having a single distribution will make things easier for the large majority.

alex
 

Walter Chang wrote:
> how much of this is going to impact scala-based development on google
> android? i don't know the extents of the changes but actors work
> great when i use scala 2.7.x on android. will 2.8.0, when it is
> released, work as well as 2.7.x on android?

Yes, it should.

Philipp

> On Tue, Dec 1, 2009 at 10:56 PM, martin odersky wrote:
>> We just decided in the Scala meeting to keep Java 1.5 as not only the
>> default, but the only version required by Scala 2.8. We will make use
>> of dynamic class loading and reflection to load the real FJ framework
>> instead of its 1.5 backport or java.util.concurrent. This decision
>> holds unless some unexpected problems come up with this dynamic
>> loading scheme.
>>
>> Cheers
>>
>> -- Martin
>>
>
>
>

Yikes !!! I don't understand what just happened or the "whys" of it.
Frankly I thought the initial plan of jvm1.6 as the default along with a fully support 1.5jvm build was just fine.  The compromise to stop biasing on one as default and  going with two "peer" builds, one scala-jvm1.6 and one scala-jvm1.5 seemed a reasonable compromise.
Why the 180 degree change?  As pointed out, JDK 1.6 (Mustang) was released 12/12/2006, three years ago.  1.5 (Tiger) was released 9/29/2004 5 years ago.
Martin's original position on this subject made sense, the subsequent Scala teams compromise of having 2 co-equal builds was a nice comprise.  
Can someone summarize the rational in say 2-3 succinct reasons or less?   
Running the 1st Qtr 2010 upcoming major release of Scala, version 2.8, will not use Java 1.6, the current standard JVM released 3 years ago as the default and instead shall resort to a dynamic loading scheme because:  1) ???? 2) ????  3) ????
Obviously if I'm asking for 3 reasons justify the change in all fairness I should offer up 3 reasons as to why the original direction made perfect sense (to me).
The original approach made sense because:  1) If you are a 1.5 shop, use the 2.8-jvm1.5 build.  If you are a 1.6 shop (that's us), use the 2.8-jvm1.6 build.  2) A "dynamic loading scheme" with reflection is gratuitous and fraught with potential unintended consequences hanging out there.  e.g.,        - OSGi modules may now require special handling to deal with class for name loading.        - App servers, BEA or WebSphere can be particular with dynamic class loading, where allowed from etc.        - JVM security model restrictions on applets and devices. 3) As Martin pointed out initially (paraphrased)   - The main advantage of the jvm6 build is better concurrency support through the use of the more modern FJ library.   - Once the final 2.8 release is out, there might already be some fast parallel collection classes included, which would also rely on 1.6.   - I want to make fast concurrency and parallelism by default.  - Scala must be the obvious choice for harnessing concurrency and parallelism.   4) Martin again, "The main point is we want to leapfrog Java and we won't be able to do that by clinging onto versions that Sun/Oracle itself no longer supports."

With eager anticipation I await the 1,2,3 reasons as to why the latest decision from the Scala meeting is superior to the original direction.

Thanks, 

Ray

On Tue, Dec 1, 2009 at 9:56 AM, martin odersky <martin.odersky@epfl.ch> wrote:

We just decided in the Scala meeting to keep Java 1.5 as not only the
default, but the only version required by Scala 2.8. We will make use
of dynamic class loading and reflection to load the real FJ  framework
instead of its 1.5 backport or java.util.concurrent. This decision
holds unless some unexpected problems come up with this dynamic
loading scheme.

Cheers

Ray,

In short, the reasons are:

1) If Scala is distributed in two separate versions, the set of libraries
and tools built on top may easily turn into a jungle of versions that are
1.5-compatible and/or 1.6-specific. Cross-dependencies make the problem
worse: during our last Scala meeting (the notes will be published later
today) the example was made of a tool "X" that is only distributed in a
1.5 version, and a tool "Y" that needs "X", but also the features only
available with 1.6 (FJ, for instance). So, X needs the 1.5 Scala jars,
while "Y" needs the 1.6 Scala jars. Hilarity ensues.
2) We can obtain the same practical advantages of having a 1.6-specific
distribution by packaging 1.6-specific classfiles within the main jar, and
selecting those implementations automatically at runtime. The only thing
left out is the potential speed improvement deriving from the 1.6-specific
bytecode verifier, but that is something that we can live with for the
time being. In short: we get most of the advantages of a 1.6-specific
distribution, without causing a mess.
3) lots of folk in industrial environments are seemingly stuck with 1.5.
We shall postpone the switch to 1.6 as the recommended default with a 2.9
version; the switch in version number may take place at whichever moment
may seem a better time for the switch. It seems to be too soon right now.

These are the main reasons, analyzed during our latest meeting, that led
to the current decision. We cannot make everyone happy at the same time,
regrettably; the decision we have reached now seems to be the best
solution we can achieve at present.

Toni

On Wed, December 2, 2009 1:01 am, Ray Racine wrote:
> Yikes !!! I don't understand what just happened or the "whys" of it.
>
> Frankly I thought the initial plan of jvm1.6 as the default along with a
> fully support 1.5jvm build was just fine. The compromise to stop biasing
> on
> one as default and going with two "peer" builds, one scala-jvm1.6 and one
> scala-jvm1.5 seemed a reasonable compromise.
>
> Why the 180 degree change? As pointed out, JDK 1.6 (Mustang) was released
> 12/12/2006, three years ago. 1.5 (Tiger) was released 9/29/2004 5 years
> ago.
>
> Martin's original position on this subject made sense, the subsequent
> Scala
> teams compromise of having 2 co-equal builds was a nice comprise.
>
> Can someone summarize the rational in say 2-3 succinct reasons or less?
>
> Running the 1st Qtr 2010 upcoming major release of Scala, version 2.8,
> will
> not use Java 1.6, the current standard JVM released 3 years ago as the
> default and instead shall resort to a dynamic loading scheme because:
> 1) ????
> 2) ????
> 3) ????
>
> Obviously if I'm asking for 3 reasons justify the change in all fairness I
> should offer up 3 reasons as to why the original direction made perfect
> sense (to me).
>
> The original approach made sense because:
> 1) If you are a 1.5 shop, use the 2.8-jvm1.5 build. If you are a 1.6
> shop
> (that's us), use the 2.8-jvm1.6 build.
> 2) A "dynamic loading scheme" with reflection is gratuitous and fraught
> with potential unintended consequences hanging out there. e.g.,
> - OSGi modules may now require special handling to deal with class
> for name loading.
> - App servers, BEA or WebSphere can be particular with dynamic
> class
> loading, where allowed from etc.
> - JVM security model restrictions on applets and devices.
> 3) As Martin pointed out initially (paraphrased)
> - The main advantage of the jvm6 build is better concurrency
> support through the use of the more modern FJ library.
> - Once the final 2.8 release is out, there might already be some fast
> parallel collection classes included, which would also rely on 1.6.
> - I want to make fast concurrency and parallelism by default.
> - Scala must be the obvious choice for harnessing concurrency
> and parallelism.
> 4) Martin again, "The main point is we want to leapfrog Java and we won't
> be
> able to do that by clinging onto versions that Sun/Oracle itself no longer
> supports."
>
>
> With eager anticipation I await the 1,2,3 reasons as to why the latest
> decision from the Scala meeting is superior to the original direction.
>
>
> Thanks,
>
>
> Ray
>
>
> On Tue, Dec 1, 2009 at 9:56 AM, martin odersky
> wrote:
>
>> We just decided in the Scala meeting to keep Java 1.5 as not only the
>> default, but the only version required by Scala 2.8. We will make use
>> of dynamic class loading and reflection to load the real FJ framework
>> instead of its 1.5 backport or java.util.concurrent. This decision
>> holds unless some unexpected problems come up with this dynamic
>> loading scheme.
>>
>> Cheers
>>
>> -- Martin
>>
>
>
>
> --
> The object of life is not to be on the side of the majority, but to escape
> finding oneself in the ranks of the insane. - Marcus Aurelius
>

On Wed, Dec 2, 2009 at 1:01 AM, Ray Racine <ray.racine@gmail.com> wrote:

Martin's original position on this subject made sense, the subsequent Scala teams compromise of having 2 co-equal builds was a nice comprise.  

I think Toni explained well why the compromise was worse than shipping either one build. Given the choice, people would make different choices and we are at the bottom of the software stack. Everyone will have to ship 2 versions of their libraries, or if they don't users would be stuck with incompatible requirements.
 

Running the 1st Qtr 2010 upcoming major release of Scala, version 2.8, will not use Java 1.6, the current standard JVM released 3 years ago as the default and instead shall resort to a dynamic loading scheme because:

Scala will run just fine on 1.6, it just won't require it. The only real advantage in requiring 1.6 at this point was faster actors, but that can be achieved by having the library check on which version of the JVM it runs, and switch to the faster implementation on 1.6. That's the reflection bit that was mentioned earlier.

iulian

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

Toni,

I think the points Ray had raised could happen and they might just be
unsolvable unless other parties were willing to make the
modifications. It is quite unthinkable to fathom that IBM will make
code changes to accommodate us Scala coders, for instance. How will
this dynamic classloading behave if I need to run my jars through
obfuscators like ProGuard? I respect the issues you've highlighted
but I think they will be mere inconveniences. If we were to go ahead
with dynamic classloading, some problems might come up that might just
be show-stoppers.

Regards,

Walter

On Wed, Dec 2, 2009 at 3:25 PM, Antonio Cunei wrote:
> Ray,
>
> In short, the reasons are:
>
> 1) If Scala is distributed in two separate versions, the set of libraries
> and tools built on top may easily turn into a jungle of versions that are
> 1.5-compatible and/or 1.6-specific. Cross-dependencies make the problem
> worse: during our last Scala meeting (the notes will be published later
> today) the example was made of a tool "X" that is only distributed in a
> 1.5 version, and a tool "Y" that needs "X", but also the features only
> available with 1.6 (FJ, for instance). So, X needs the 1.5 Scala jars,
> while "Y" needs the 1.6 Scala jars. Hilarity ensues.
> 2) We can obtain the same practical advantages of having a 1.6-specific
> distribution by packaging 1.6-specific classfiles within the main jar, and
> selecting those implementations automatically at runtime. The only thing
> left out is the potential speed improvement deriving from the 1.6-specific
> bytecode verifier, but that is something that we can live with for the
> time being. In short: we get most of the advantages of a 1.6-specific
> distribution, without causing a mess.
> 3) lots of folk in industrial environments are seemingly stuck with 1.5.
> We shall postpone the switch to 1.6 as the recommended default with a 2.9
> version; the switch in version number may take place at whichever moment
> may seem a better time for the switch. It seems to be too soon right now.
>
> These are the main reasons, analyzed during our latest meeting, that led
> to the current decision. We cannot make everyone happy at the same time,
> regrettably; the decision we have reached now seems to be the best
> solution we can achieve at present.
>
> Toni
>
>
> On Wed, December 2, 2009 1:01 am, Ray Racine wrote:
>> Yikes !!! I don't understand what just happened or the "whys" of it.
>>
>> Frankly I thought the initial plan of jvm1.6 as the default along with a
>> fully support 1.5jvm build was just fine.  The compromise to stop biasing
>> on
>> one as default and  going with two "peer" builds, one scala-jvm1.6 and one
>> scala-jvm1.5 seemed a reasonable compromise.
>>
>> Why the 180 degree change?  As pointed out, JDK 1.6 (Mustang) was released
>> 12/12/2006, three years ago.  1.5 (Tiger) was released 9/29/2004 5 years
>> ago.
>>
>> Martin's original position on this subject made sense, the subsequent
>> Scala
>> teams compromise of having 2 co-equal builds was a nice comprise.
>>
>> Can someone summarize the rational in say 2-3 succinct reasons or less?
>>
>> Running the 1st Qtr 2010 upcoming major release of Scala, version 2.8,
>> will
>> not use Java 1.6, the current standard JVM released 3 years ago as the
>> default and instead shall resort to a dynamic loading scheme because:
>>  1) ????
>>  2) ????
>>  3) ????
>>
>> Obviously if I'm asking for 3 reasons justify the change in all fairness I
>> should offer up 3 reasons as to why the original direction made perfect
>> sense (to me).
>>
>> The original approach made sense because:
>>  1) If you are a 1.5 shop, use the 2.8-jvm1.5 build.  If you are a 1.6
>> shop
>> (that's us), use the 2.8-jvm1.6 build.
>>  2) A "dynamic loading scheme" with reflection is gratuitous and fraught
>> with potential unintended consequences hanging out there.  e.g.,
>>        - OSGi modules may now require special handling to deal with class
>> for name loading.
>>        - App servers, BEA or WebSphere can be particular with dynamic
>> class
>> loading, where allowed from etc.
>>        - JVM security model restrictions on applets and devices.
>>  3) As Martin pointed out initially (paraphrased)
>>   - The main advantage of the jvm6 build is better concurrency
>> support through the use of the more modern FJ library.
>>   - Once the final 2.8 release is out, there might already be some fast
>> parallel collection classes included, which would also rely on 1.6.
>>   - I want to make fast concurrency and parallelism by default.
>>   - Scala must be the obvious choice for harnessing concurrency
>> and parallelism.
>> 4) Martin again, "The main point is we want to leapfrog Java and we won't
>> be
>> able to do that by clinging onto versions that Sun/Oracle itself no longer
>> supports."
>>
>>
>> With eager anticipation I await the 1,2,3 reasons as to why the latest
>> decision from the Scala meeting is superior to the original direction.
>>
>>
>> Thanks,
>>
>>
>> Ray
>>
>>
>> On Tue, Dec 1, 2009 at 9:56 AM, martin odersky
>> wrote:
>>
>>> We just decided in the Scala meeting to keep Java 1.5 as not only the
>>> default, but the only version required by Scala 2.8. We will make use
>>> of dynamic class loading and reflection to load the real FJ  framework
>>> instead of its 1.5 backport or java.util.concurrent. This decision
>>> holds unless some unexpected problems come up with this dynamic
>>> loading scheme.
>>>
>>> Cheers
>>>
>>>  -- Martin
>>>
>>
>>
>>
>> --
>> The object of life is not to be on the side of the majority, but to escape
>> finding oneself in the ranks of the insane. - Marcus Aurelius
>>
>
>
>

Only a tangent to the original issue, but anyway: does the backported
concurrent stuff for the jvm5-version of Scala work only with the Sun
JVM or with others as well? I'm asking because it contains references
to internal Sun APIs (Unsafe). Is anyone using actors on another JVM,
e.g. on Android? Are other JVMs officially supported by Scala?

On Wed, Dec 2, 2009 at 9:43 AM, Walter Chang wrote:
> Toni,
>
> I think the points Ray had raised could happen and they might just be
> unsolvable unless other parties were willing to make the
> modifications.  It is quite unthinkable to fathom that IBM will make
> code changes to accommodate us Scala coders, for instance.  How will
> this dynamic classloading behave if I need to run my jars through
> obfuscators like ProGuard?  I respect the issues you've highlighted
> but I think they will be mere inconveniences.  If we were to go ahead
> with dynamic classloading, some problems might come up that might just
> be show-stoppers.
>
> Regards,
>
> Walter
>
> On Wed, Dec 2, 2009 at 3:25 PM, Antonio Cunei wrote:
>> Ray,
>>
>> In short, the reasons are:
>>
>> 1) If Scala is distributed in two separate versions, the set of libraries
>> and tools built on top may easily turn into a jungle of versions that are
>> 1.5-compatible and/or 1.6-specific. Cross-dependencies make the problem
>> worse: during our last Scala meeting (the notes will be published later
>> today) the example was made of a tool "X" that is only distributed in a
>> 1.5 version, and a tool "Y" that needs "X", but also the features only
>> available with 1.6 (FJ, for instance). So, X needs the 1.5 Scala jars,
>> while "Y" needs the 1.6 Scala jars. Hilarity ensues.
>> 2) We can obtain the same practical advantages of having a 1.6-specific
>> distribution by packaging 1.6-specific classfiles within the main jar, and
>> selecting those implementations automatically at runtime. The only thing
>> left out is the potential speed improvement deriving from the 1.6-specific
>> bytecode verifier, but that is something that we can live with for the
>> time being. In short: we get most of the advantages of a 1.6-specific
>> distribution, without causing a mess.
>> 3) lots of folk in industrial environments are seemingly stuck with 1.5.
>> We shall postpone the switch to 1.6 as the recommended default with a 2.9
>> version; the switch in version number may take place at whichever moment
>> may seem a better time for the switch. It seems to be too soon right now.
>>
>> These are the main reasons, analyzed during our latest meeting, that led
>> to the current decision. We cannot make everyone happy at the same time,
>> regrettably; the decision we have reached now seems to be the best
>> solution we can achieve at present.
>>
>> Toni
>>
>>
>> On Wed, December 2, 2009 1:01 am, Ray Racine wrote:
>>> Yikes !!! I don't understand what just happened or the "whys" of it.
>>>
>>> Frankly I thought the initial plan of jvm1.6 as the default along with a
>>> fully support 1.5jvm build was just fine.  The compromise to stop biasing
>>> on
>>> one as default and  going with two "peer" builds, one scala-jvm1.6 and one
>>> scala-jvm1.5 seemed a reasonable compromise.
>>>
>>> Why the 180 degree change?  As pointed out, JDK 1.6 (Mustang) was released
>>> 12/12/2006, three years ago.  1.5 (Tiger) was released 9/29/2004 5 years
>>> ago.
>>>
>>> Martin's original position on this subject made sense, the subsequent
>>> Scala
>>> teams compromise of having 2 co-equal builds was a nice comprise.
>>>
>>> Can someone summarize the rational in say 2-3 succinct reasons or less?
>>>
>>> Running the 1st Qtr 2010 upcoming major release of Scala, version 2.8,
>>> will
>>> not use Java 1.6, the current standard JVM released 3 years ago as the
>>> default and instead shall resort to a dynamic loading scheme because:
>>>  1) ????
>>>  2) ????
>>>  3) ????
>>>
>>> Obviously if I'm asking for 3 reasons justify the change in all fairness I
>>> should offer up 3 reasons as to why the original direction made perfect
>>> sense (to me).
>>>
>>> The original approach made sense because:
>>>  1) If you are a 1.5 shop, use the 2.8-jvm1.5 build.  If you are a 1.6
>>> shop
>>> (that's us), use the 2.8-jvm1.6 build.
>>>  2) A "dynamic loading scheme" with reflection is gratuitous and fraught
>>> with potential unintended consequences hanging out there.  e.g.,
>>>        - OSGi modules may now require special handling to deal with class
>>> for name loading.
>>>        - App servers, BEA or WebSphere can be particular with dynamic
>>> class
>>> loading, where allowed from etc.
>>>        - JVM security model restrictions on applets and devices.
>>>  3) As Martin pointed out initially (paraphrased)
>>>   - The main advantage of the jvm6 build is better concurrency
>>> support through the use of the more modern FJ library.
>>>   - Once the final 2.8 release is out, there might already be some fast
>>> parallel collection classes included, which would also rely on 1.6.
>>>   - I want to make fast concurrency and parallelism by default.
>>>   - Scala must be the obvious choice for harnessing concurrency
>>> and parallelism.
>>> 4) Martin again, "The main point is we want to leapfrog Java and we won't
>>> be
>>> able to do that by clinging onto versions that Sun/Oracle itself no longer
>>> supports."
>>>
>>>
>>> With eager anticipation I await the 1,2,3 reasons as to why the latest
>>> decision from the Scala meeting is superior to the original direction.
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Ray
>>>
>>>
>>> On Tue, Dec 1, 2009 at 9:56 AM, martin odersky
>>> wrote:
>>>
>>>> We just decided in the Scala meeting to keep Java 1.5 as not only the
>>>> default, but the only version required by Scala 2.8. We will make use
>>>> of dynamic class loading and reflection to load the real FJ  framework
>>>> instead of its 1.5 backport or java.util.concurrent. This decision
>>>> holds unless some unexpected problems come up with this dynamic
>>>> loading scheme.
>>>>
>>>> Cheers
>>>>
>>>>  -- Martin
>>>>
>>>
>>>
>>>
>>> --
>>> The object of life is not to be on the side of the majority, but to escape
>>> finding oneself in the ranks of the insane. - Marcus Aurelius
>>>
>>
>>
>>
>
>
>
> --
> .......__o
> .......\<,
> ....( )/ ( )...
>

I think there's no reflection involved, it's just a factory method. The
relevant code in scala/actors/Scheduler.scala:

  def makeNewScheduler: IScheduler = {
    // test on which JVM we are running
    val jvmVersion = System.getProperty("java.version")
    val jvmVendor =  System.getProperty("java.vm.vendor")
    val sched = if (jvmVersion.indexOf("1.5") != -1 ||
                    jvmVendor.indexOf("IBM") != -1) {
      // on IBM J9 1.6 do not use ForkJoinPool
      val s = new ResizableThreadPoolScheduler(false)
      s.start()
      s
    } else {
      val s = new ForkJoinScheduler
      s.start()
      s
    }
    Debug.info(this+": starting new "+sched+" ["+sched.getClass+"]")
    sched
  }


On Wed, Dec 2, 2009 at 09:43, Walter Chang <weihsiu@gmail.com> wrote:

Toni,

I think the points Ray had raised could happen and they might just be
unsolvable unless other parties were willing to make the
modifications.  It is quite unthinkable to fathom that IBM will make
code changes to accommodate us Scala coders, for instance.  How will
this dynamic classloading behave if I need to run my jars through
obfuscators like ProGuard?  I respect the issues you've highlighted
but I think they will be mere inconveniences.  If we were to go ahead
with dynamic classloading, some problems might come up that might just
be show-stoppers.

Regards,

Walter

On Wed, Dec 2, 2009 at 3:25 PM, Antonio Cunei <antonio.cunei@epfl.ch> wrote:
> Ray,
>
> In short, the reasons are:
>
> 1) If Scala is distributed in two separate versions, the set of libraries
> and tools built on top may easily turn into a jungle of versions that are
> 1.5-compatible and/or 1.6-specific. Cross-dependencies make the problem
> worse: during our last Scala meeting (the notes will be published later
> today) the example was made of a tool "X" that is only distributed in a
> 1.5 version, and a tool "Y" that needs "X", but also the features only
> available with 1.6 (FJ, for instance). So, X needs the 1.5 Scala jars,
> while "Y" needs the 1.6 Scala jars. Hilarity ensues.
> 2) We can obtain the same practical advantages of having a 1.6-specific
> distribution by packaging 1.6-specific classfiles within the main jar, and
> selecting those implementations automatically at runtime. The only thing
> left out is the potential speed improvement deriving from the 1.6-specific
> bytecode verifier, but that is something that we can live with for the
> time being. In short: we get most of the advantages of a 1.6-specific
> distribution, without causing a mess.
> 3) lots of folk in industrial environments are seemingly stuck with 1.5.
> We shall postpone the switch to 1.6 as the recommended default with a 2.9
> version; the switch in version number may take place at whichever moment
> may seem a better time for the switch. It seems to be too soon right now.
>
> These are the main reasons, analyzed during our latest meeting, that led
> to the current decision. We cannot make everyone happy at the same time,
> regrettably; the decision we have reached now seems to be the best
> solution we can achieve at present.
>
> Toni
>
>
> On Wed, December 2, 2009 1:01 am, Ray Racine wrote:
>> Yikes !!! I don't understand what just happened or the "whys" of it.
>>
>> Frankly I thought the initial plan of jvm1.6 as the default along with a
>> fully support 1.5jvm build was just fine.  The compromise to stop biasing
>> on
>> one as default and  going with two "peer" builds, one scala-jvm1.6 and one
>> scala-jvm1.5 seemed a reasonable compromise.
>>
>> Why the 180 degree change?  As pointed out, JDK 1.6 (Mustang) was released
>> 12/12/2006, three years ago.  1.5 (Tiger) was released 9/29/2004 5 years
>> ago.
>>
>> Martin's original position on this subject made sense, the subsequent
>> Scala
>> teams compromise of having 2 co-equal builds was a nice comprise.
>>
>> Can someone summarize the rational in say 2-3 succinct reasons or less?
>>
>> Running the 1st Qtr 2010 upcoming major release of Scala, version 2.8,
>> will
>> not use Java 1.6, the current standard JVM released 3 years ago as the
>> default and instead shall resort to a dynamic loading scheme because:
>>  1) ????
>>  2) ????
>>  3) ????
>>
>> Obviously if I'm asking for 3 reasons justify the change in all fairness I
>> should offer up 3 reasons as to why the original direction made perfect
>> sense (to me).
>>
>> The original approach made sense because:
>>  1) If you are a 1.5 shop, use the 2.8-jvm1.5 build.  If you are a 1.6
>> shop
>> (that's us), use the 2.8-jvm1.6 build.
>>  2) A "dynamic loading scheme" with reflection is gratuitous and fraught
>> with potential unintended consequences hanging out there.  e.g.,
>>        - OSGi modules may now require special handling to deal with class
>> for name loading.
>>        - App servers, BEA or WebSphere can be particular with dynamic
>> class
>> loading, where allowed from etc.
>>        - JVM security model restrictions on applets and devices.
>>  3) As Martin pointed out initially (paraphrased)
>>   - The main advantage of the jvm6 build is better concurrency
>> support through the use of the more modern FJ library.
>>   - Once the final 2.8 release is out, there might already be some fast
>> parallel collection classes included, which would also rely on 1.6.
>>   - I want to make fast concurrency and parallelism by default.
>>   - Scala must be the obvious choice for harnessing concurrency
>> and parallelism.
>> 4) Martin again, "The main point is we want to leapfrog Java and we won't
>> be
>> able to do that by clinging onto versions that Sun/Oracle itself no longer
>> supports."
>>
>>
>> With eager anticipation I await the 1,2,3 reasons as to why the latest
>> decision from the Scala meeting is superior to the original direction.
>>
>>
>> Thanks,
>>
>>
>> Ray
>>
>>
>> On Tue, Dec 1, 2009 at 9:56 AM, martin odersky
>> <martin.odersky@epfl.ch>wrote:
>>
>>> We just decided in the Scala meeting to keep Java 1.5 as not only the
>>> default, but the only version required by Scala 2.8. We will make use
>>> of dynamic class loading and reflection to load the real FJ  framework
>>> instead of its 1.5 backport or java.util.concurrent. This decision
>>> holds unless some unexpected problems come up with this dynamic
>>> loading scheme.
>>>
>>> Cheers
>>>
>>>  -- Martin
>>>
>>
>>
>>
>> --
>> The object of life is not to be on the side of the majority, but to escape
>> finding oneself in the ranks of the insane. - Marcus Aurelius
>>
>
>
>



--
.......__o
.......\<,
....( )/ ( )...

Philipp Haller wrote:
> Iulian Dragos wrote:
>> Scala will run just fine on 1.6, it just won't require it. The only real
>> advantage in requiring 1.6 at this point was faster actors, but that can
>> be achieved by having the library check on which version of the JVM it
>> runs, and switch to the faster implementation on 1.6. That's the
>> reflection bit that was mentioned earlier.
>
> Just to clarify: the current scheme does not require reflection.
>
> The 1.6 enhancements are achieved by including class files in
> scala-library.jar that
> - verify on 1.5,
> - throw `AbstractMethodError`s when used on 1.5, and that

This should have been `NoSuchMethodError`s and `NoClassDefFoundError`s.

Philipp

Iulian Dragos wrote:
> Scala will run just fine on 1.6, it just won't require it. The only real
> advantage in requiring 1.6 at this point was faster actors, but that can
> be achieved by having the library check on which version of the JVM it
> runs, and switch to the faster implementation on 1.6. That's the
> reflection bit that was mentioned earlier.

Just to clarify: the current scheme does not require reflection.

The 1.6 enhancements are achieved by including class files in
scala-library.jar that
- verify on 1.5,
- throw `AbstractMethodError`s when used on 1.5, and that
- are not loaded (we don't instantiate classes that refer to them) when
running on a JVM whose "java.vm.version" property is 1.5.

So, the switch to 1.6-specific classes is done by simply instantiating
different classes.

Philipp

Johannes Rudolph wrote:
> Only a tangent to the original issue, but anyway: does the backported
> concurrent stuff for the jvm5-version of Scala work only with the Sun
> JVM or with others as well? I'm asking because it contains references
> to internal Sun APIs (Unsafe). Is anyone using actors on another JVM,
> e.g. on Android? Are other JVMs officially supported by Scala?

The JVM 1.5 compatible classes do not use any internal Sun APIs.
`Unsafe` is only used in the 1.6-specific classes which are not loaded
on a 1.5 JVM (and neither on a 1.6 IBM JVM).

So, yes, other 1.5 JVMs as well as IBM's 1.6 JVM are also supported.

Cheers,
Philipp

> On Wed, Dec 2, 2009 at 9:43 AM, Walter Chang wrote:
>> Toni,
>>
>> I think the points Ray had raised could happen and they might just be
>> unsolvable unless other parties were willing to make the
>> modifications. It is quite unthinkable to fathom that IBM will make
>> code changes to accommodate us Scala coders, for instance. How will
>> this dynamic classloading behave if I need to run my jars through
>> obfuscators like ProGuard? I respect the issues you've highlighted
>> but I think they will be mere inconveniences. If we were to go ahead
>> with dynamic classloading, some problems might come up that might just
>> be show-stoppers.
>>
>> Regards,
>>
>> Walter
>>
>> On Wed, Dec 2, 2009 at 3:25 PM, Antonio Cunei wrote:
>>> Ray,
>>>
>>> In short, the reasons are:
>>>
>>> 1) If Scala is distributed in two separate versions, the set of libraries
>>> and tools built on top may easily turn into a jungle of versions that are
>>> 1.5-compatible and/or 1.6-specific. Cross-dependencies make the problem
>>> worse: during our last Scala meeting (the notes will be published later
>>> today) the example was made of a tool "X" that is only distributed in a
>>> 1.5 version, and a tool "Y" that needs "X", but also the features only
>>> available with 1.6 (FJ, for instance). So, X needs the 1.5 Scala jars,
>>> while "Y" needs the 1.6 Scala jars. Hilarity ensues.
>>> 2) We can obtain the same practical advantages of having a 1.6-specific
>>> distribution by packaging 1.6-specific classfiles within the main jar, and
>>> selecting those implementations automatically at runtime. The only thing
>>> left out is the potential speed improvement deriving from the 1.6-specific
>>> bytecode verifier, but that is something that we can live with for the
>>> time being. In short: we get most of the advantages of a 1.6-specific
>>> distribution, without causing a mess.
>>> 3) lots of folk in industrial environments are seemingly stuck with 1.5.
>>> We shall postpone the switch to 1.6 as the recommended default with a 2.9
>>> version; the switch in version number may take place at whichever moment
>>> may seem a better time for the switch. It seems to be too soon right now.
>>>
>>> These are the main reasons, analyzed during our latest meeting, that led
>>> to the current decision. We cannot make everyone happy at the same time,
>>> regrettably; the decision we have reached now seems to be the best
>>> solution we can achieve at present.
>>>
>>> Toni
>>>
>>>
>>> On Wed, December 2, 2009 1:01 am, Ray Racine wrote:
>>>> Yikes !!! I don't understand what just happened or the "whys" of it.
>>>>
>>>> Frankly I thought the initial plan of jvm1.6 as the default along with a
>>>> fully support 1.5jvm build was just fine. The compromise to stop biasing
>>>> on
>>>> one as default and going with two "peer" builds, one scala-jvm1.6 and one
>>>> scala-jvm1.5 seemed a reasonable compromise.
>>>>
>>>> Why the 180 degree change? As pointed out, JDK 1.6 (Mustang) was released
>>>> 12/12/2006, three years ago. 1.5 (Tiger) was released 9/29/2004 5 years
>>>> ago.
>>>>
>>>> Martin's original position on this subject made sense, the subsequent
>>>> Scala
>>>> teams compromise of having 2 co-equal builds was a nice comprise.
>>>>
>>>> Can someone summarize the rational in say 2-3 succinct reasons or less?
>>>>
>>>> Running the 1st Qtr 2010 upcoming major release of Scala, version 2.8,
>>>> will
>>>> not use Java 1.6, the current standard JVM released 3 years ago as the
>>>> default and instead shall resort to a dynamic loading scheme because:
>>>> 1) ????
>>>> 2) ????
>>>> 3) ????
>>>>
>>>> Obviously if I'm asking for 3 reasons justify the change in all fairness I
>>>> should offer up 3 reasons as to why the original direction made perfect
>>>> sense (to me).
>>>>
>>>> The original approach made sense because:
>>>> 1) If you are a 1.5 shop, use the 2.8-jvm1.5 build. If you are a 1.6
>>>> shop
>>>> (that's us), use the 2.8-jvm1.6 build.
>>>> 2) A "dynamic loading scheme" with reflection is gratuitous and fraught
>>>> with potential unintended consequences hanging out there. e.g.,
>>>> - OSGi modules may now require special handling to deal with class
>>>> for name loading.
>>>> - App servers, BEA or WebSphere can be particular with dynamic
>>>> class
>>>> loading, where allowed from etc.
>>>> - JVM security model restrictions on applets and devices.
>>>> 3) As Martin pointed out initially (paraphrased)
>>>> - The main advantage of the jvm6 build is better concurrency
>>>> support through the use of the more modern FJ library.
>>>> - Once the final 2.8 release is out, there might already be some fast
>>>> parallel collection classes included, which would also rely on 1.6.
>>>> - I want to make fast concurrency and parallelism by default.
>>>> - Scala must be the obvious choice for harnessing concurrency
>>>> and parallelism.
>>>> 4) Martin again, "The main point is we want to leapfrog Java and we won't
>>>> be
>>>> able to do that by clinging onto versions that Sun/Oracle itself no longer
>>>> supports."
>>>>
>>>>
>>>> With eager anticipation I await the 1,2,3 reasons as to why the latest
>>>> decision from the Scala meeting is superior to the original direction.
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>> Ray
>>>>
>>>>
>>>> On Tue, Dec 1, 2009 at 9:56 AM, martin odersky
>>>> wrote:
>>>>
>>>>> We just decided in the Scala meeting to keep Java 1.5 as not only the
>>>>> default, but the only version required by Scala 2.8. We will make use
>>>>> of dynamic class loading and reflection to load the real FJ framework
>>>>> instead of its 1.5 backport or java.util.concurrent. This decision
>>>>> holds unless some unexpected problems come up with this dynamic
>>>>> loading scheme.
>>>>>
>>>>> Cheers
>>>>>
>>>>> -- Martin
>>>>>
>>>>
>>>>
>>>> --
>>>> The object of life is not to be on the side of the majority, but to escape
>>>> finding oneself in the ranks of the insane. - Marcus Aurelius
>>>>
>>>
>>>
>>
>>
>> --
>> .......__o
>> .......\<,
>> ....( )/ ( )...
>>
>
>
>

On Wed, Dec 2, 2009 at 10:32 AM, Philipp Haller wrote:
> Johannes Rudolph wrote:
>> Only a tangent to the original issue, but anyway: does the backported
>> concurrent stuff for the jvm5-version of Scala work only with the Sun
>> JVM or with others as well? I'm asking because it contains references
>> to internal Sun APIs (Unsafe). Is anyone using actors on another JVM,
>> e.g. on Android? Are other JVMs officially supported by Scala?
>
> The JVM 1.5 compatible classes do not use any internal Sun APIs.
> `Unsafe` is only used in the 1.6-specific classes which are not loaded
> on a 1.5 JVM (and neither on a 1.6 IBM JVM).
>
> So, yes, other 1.5 JVMs as well as IBM's 1.6 JVM are also supported.
Ah, ok, so I mixed things up. Sorry, for spreading FUD. Thanks for the
clarification.

Shouldn't the condition Lukas posted before, then be written more
restrictive as "Use the ForkJoinScheduler only on the Sun JVM (because
of Unsafe) with version 1.6, else use the
ResizableThreadPoolScheduler"?

Johannes Rudolph wrote:
> On Wed, Dec 2, 2009 at 10:32 AM, Philipp Haller wrote:
>> Johannes Rudolph wrote:
>>> Only a tangent to the original issue, but anyway: does the backported
>>> concurrent stuff for the jvm5-version of Scala work only with the Sun
>>> JVM or with others as well? I'm asking because it contains references
>>> to internal Sun APIs (Unsafe). Is anyone using actors on another JVM,
>>> e.g. on Android? Are other JVMs officially supported by Scala?
>> The JVM 1.5 compatible classes do not use any internal Sun APIs.
>> `Unsafe` is only used in the 1.6-specific classes which are not loaded
>> on a 1.5 JVM (and neither on a 1.6 IBM JVM).
>>
>> So, yes, other 1.5 JVMs as well as IBM's 1.6 JVM are also supported.
> Ah, ok, so I mixed things up. Sorry, for spreading FUD. Thanks for the
> clarification.
>
> Shouldn't the condition Lukas posted before, then be written more
> restrictive as "Use the ForkJoinScheduler only on the Sun JVM (because
> of Unsafe) with version 1.6, else use the
> ResizableThreadPoolScheduler"?

Good point.

However, we also want to use FJ in at least the following cases:
- One of the next releases of IBM's J9 will have full support for Sun's
1.6 `Unsafe` mechanics.
- [Open]JDK 7 remains fully compatible with 1.6 `Unsafe`.

A more restrictive condition would have to be adjusted to those cases.

Cheers,
Philipp

How will this affect compilation?Currently, we're compiling the 1.5 release on a 1.5 vm and the 1.6 release on a 1.6 vmA single release would (presumably) require a single build, so what vm will that be done on?
- If it's a 1.5 vm, then we can't access features in the 1.5 libraries, some of which we use for performance(such as ForkJoin)
- If it's a 1.6 vm, then we could mistakenly use features not available under 1.5 (the String.isEmpty bug is an example of this)
How are we compiling so as to allow the use of 1.6 features whilst also restricting them to the runtime-loaded sections of code?

On Wed, Dec 2, 2009 at 9:21 AM, Philipp Haller <philipp.haller@epfl.ch> wrote:

Iulian Dragos wrote:
> Scala will run just fine on 1.6, it just won't require it. The only real
> advantage in requiring 1.6 at this point was faster actors, but that can
> be achieved by having the library check on which version of the JVM it
> runs, and switch to the faster implementation on 1.6. That's the
> reflection bit that was mentioned earlier.

Just to clarify: the current scheme does not require reflection.

The 1.6 enhancements are achieved by including class files in
scala-library.jar that
- verify on 1.5,
- throw `AbstractMethodError`s when used on 1.5, and that
- are not loaded (we don't instantiate classes that refer to them) when
 running on a JVM whose "java.vm.version" property is 1.5.

So, the switch to 1.6-specific classes is done by simply instantiating
different classes.

Philipp

If you only depend on the Scala Library, just compile against 1.5. Your jar will run on 1.5+, and take advantage of 1.6 features if you use the Actors and run on 1.6+.

If your application wants to target 1.6 features if the runtime VM supports them (as Scala does), compile on 1.6. You may like to integrate Animal Sniffer [1] into your build to ensure you only only use 1.6 features from methods annotated with @IgnoreJRERequirement.

-jason

[1] http://weblogs.java.net/blog/2008/11/14/compiling-jdk6-and-running-jdk5

On Wed, Dec 2, 2009 at 9:42 PM, Kevin Wright <kev.lee.wright@googlemail.com> wrote:

How are we compiling so as to allow the use of 1.6 features whilst also restricting them to the runtime-loaded sections of code?

Kevin Wright wrote:
> How will this affect compilation?
> Currently, we're compiling the 1.5 release on a 1.5 vm and the 1.6
> release on a 1.6 vm
> A single release would (presumably) require a single build, so what vm
> will that be done on?
>
> - If it's a 1.5 vm, then we can't access features in the 1.5 libraries,
> some of which we use for performance
> (such as ForkJoin)
>
> - If it's a 1.6 vm, then we could mistakenly use features not available
> under 1.5
> (the String.isEmpty bug is an example of this)
>
> How are we compiling so as to allow the use of 1.6 features whilst also
> restricting them to the runtime-loaded sections of code?

The fork-join Java sources have to be built on 1.6 (can still use target
1.5). They go into a forkjoin.jar to be put into `lib` in the Scala root
dir (like `fjbg.jar`). The Scala distribution itself can be built on
1.5. The `forkjoin.jar` can be integrated into `scala-library.jar` just
like `fjbg.jar` is currently integrated into `scala-compiler.jar`.

Cheers,
Philipp

> On Wed, Dec 2, 2009 at 9:21 AM, Philipp Haller > wrote:
>
> Iulian Dragos wrote:
> > Scala will run just fine on 1.6, it just won't require it. The
> only real
> > advantage in requiring 1.6 at this point was faster actors, but
> that can
> > be achieved by having the library check on which version of the
> JVM it
> > runs, and switch to the faster implementation on 1.6. That's the
> > reflection bit that was mentioned earlier.
>
> Just to clarify: the current scheme does not require reflection.
>
> The 1.6 enhancements are achieved by including class files in
> scala-library.jar that
> - verify on 1.5,
> - throw `AbstractMethodError`s when used on 1.5, and that
> - are not loaded (we don't instantiate classes that refer to them) when
> running on a JVM whose "java.vm.version" property is 1.5.
>
> So, the switch to 1.6-specific classes is done by simply instantiating
> different classes.
>
> Philipp
>
>

Interesting stuff...
I was just thinking to pull my jvmtarget compiler plugin out of the mothballs, see if I can retrofit some of these techniques, when it struck me that it can be handled with a bit of DSL magic:
def myMethod(arg: Int) = {  if (jvm("version <1.6 || vendor = IBM"))  {    ...legacy method body...  } else {    ...bleeding edge method body...   }}
jvm() is just a standard String -> Boolean function that looks at the configuration at runtime.
But... a plugin could easily use the same notation to remove code at compile time, allowing for a cross compile against a 1.5 vm, or other static analysis techniques that would catch problems such as the String.isEmpty bug.
What do you think of this technique?

On Wed, Dec 2, 2009 at 11:58 AM, Jason Zaugg <jzaugg@gmail.com> wrote:

If you only depend on the Scala Library, just compile against 1.5. Your jar will run on 1.5+, and take advantage of 1.6 features if you use the Actors and run on 1.6+.

If your application wants to target 1.6 features if the runtime VM supports them (as Scala does), compile on 1.6. You may like to integrate Animal Sniffer [1] into your build to ensure you only only use 1.6 features from methods annotated with @IgnoreJRERequirement.

-jason

[1] http://weblogs.java.net/blog/2008/11/14/compiling-jdk6-and-running-jdk5

On Wed, Dec 2, 2009 at 9:42 PM, Kevin Wright <kev.lee.wright@googlemail.com> wrote:

How are we compiling so as to allow the use of 1.6 features whilst also restricting them to the runtime-loaded sections of code?

I would like to recommend allowing an explicit override Java property to allow users who "know what they are doing" to force one version over the other.  Embedded decision logic may not always pick the version the user wants, as is shown by this more vs. less restrictive discussion.  Intelligent defaults work well in the majority of cases, but when you need to deviate from them, it should be possible.  A master switch (1.6 vs. 1.5) may be useful, but if this pattern becomes used more than once, a feature by feature switch would also be prudent.

Regards,
James

On Wed, Dec 2, 2009 at 6:29 AM, Philipp Haller <philipp.haller@epfl.ch> wrote:

Johannes Rudolph wrote:
> On Wed, Dec 2, 2009 at 10:32 AM, Philipp Haller <philipp.haller@epfl.ch> wrote:
>> Johannes Rudolph wrote:
>>> Only a tangent to the original issue, but anyway: does the backported
>>> concurrent stuff for the jvm5-version of Scala work only with the Sun
>>> JVM or with others as well? I'm asking because it contains references
>>> to internal Sun APIs (Unsafe). Is anyone using actors on another JVM,
>>> e.g. on Android? Are other JVMs officially supported by Scala?
>> The JVM 1.5 compatible classes do not use any internal Sun APIs.
>> `Unsafe` is only used in the 1.6-specific classes which are not loaded
>> on a 1.5 JVM (and neither on a 1.6 IBM JVM).
>>
>> So, yes, other 1.5 JVMs as well as IBM's 1.6 JVM are also supported.
> Ah, ok, so I mixed things up. Sorry, for spreading FUD. Thanks for the
> clarification.
>
> Shouldn't the condition Lukas posted before, then be written more
> restrictive as "Use the ForkJoinScheduler only on the Sun JVM (because
> of Unsafe) with version 1.6, else use the
> ResizableThreadPoolScheduler"?

Good point.

However, we also want to use FJ in at least the following cases:
- One of the next releases of IBM's J9 will have full support for Sun's
 1.6 `Unsafe` mechanics.
- [Open]JDK 7 remains fully compatible with 1.6 `Unsafe`.

A more restrictive condition would have to be adjusted to those cases.

Cheers,
Philipp

See Below.  And my sincere appreciation for taking the time to respond.  I am far less concerned about the relative impact of the change, my concerns are more oriented around the lateness of the change in the release cycle, the means in which it was done along with, IMHO, the rather weak foundation for doing it.  I truly do not want to pain the Scala Core Team who go way beyond only to suffer emails such as mine.  But I do want to make a point.  Caution: I can get a little "over the top" so my apologies ahead of time. 
On Wed, Dec 2, 2009 at 2:25 AM, Antonio Cunei <antonio.cunei@epfl.ch> wrote:

Ray,

In short, the reasons are:

1) If Scala is distributed in two separate versions, the set of libraries
and tools built on top may easily turn into a jungle of versions that are
1.5-compatible and/or 1.6-specific. Cross-dependencies make the problem
worse: during our last Scala meeting (the notes will be published later
today) the example was made of a tool "X" that is only distributed in a
1.5 version, and a tool "Y" that needs "X", but also the features only
available with 1.6 (FJ, for instance). So, X needs the 1.5 Scala jars,
while "Y" needs the 1.6 Scala jars. Hilarity ensues.

 Agree 100%, IMHO 2.8 should be "centered" on single target JVM release.  It is certainly unfair to expect the Scala team to support 2.8 on two JVMS.   Frankly if I were you I'd keep 2.7.x for 1.5 and moved forward with 2.8 on 1.6.  End of story.  In general, I feel the Scala Core Team cannot and should not be whipsawed by a tool "X" decision.  

2) We can obtain the same practical advantages of having a 1.6-specific
distribution by packaging 1.6-specific classfiles within the main jar, and
selecting those implementations automatically at runtime. The only thing
left out is the potential speed improvement deriving from the 1.6-specific
bytecode verifier, but that is something that we can live with for the
time being. In short: we get most of the advantages of a 1.6-specific
distribution, without causing a mess.

Practical advantage is subjective.  There is a reasonable chance that some unintended consequence shall ensue for someone from this last minute shift in direction.  Someone is always placed at a practical disadvantage.  There will always be someone who wants all the advantages of and not "most of the advantages".  There will always someone who does not want to surrender "a potential xxx improvement."  Someone like me for example. :)  

3) lots of folk in industrial environments are seemingly stuck with 1.5.
We shall postpone the switch to 1.6 as the recommended default with a 2.9
version; the switch in version number may take place at whichever moment
may seem a better time for the switch. It seems to be too soon right now.

I am very much in an industrial environment myself.  If an organization has not managed to switch to 1.6 in 3 years, what are the odds they will do so in 3.5 or 4?  A couple of more points on this.  Despite  protestations to the contrary they are "stuck" by choice and shall remain so by choice.  If a company is "fine" with staying with the 1.5 JVM; they are fine with staying with Scala 2.7.x.  I will tell you I am far more sympathetic to the argument that some individual Mac user's may not be able to use Scala 1.8 than some fortune 500 company "can't" get from 1.5 to 1.6.   A false premise.
Imagine we were all watching this conversation from a distance on the Haskell, Perl, Python, etc list and late in the cycle a subset of Python 2.x users requested certain compromises be made that in some degree impair the new Python 3.x, e.g. "Please don't add those features in 3.x as that requires the new async I/O API and my shop only runs the 2.4 Kernel."  Who would be surprised if the consensus response to that subset of Python users was anything other than "we believe the asynch I/O improvements are valuable and necessary to move Python forward for many of our users.  The 2.6 kernel has been the standard kernel for all Linux distributions for over 3 years.  If you are not ready we suggest you remain on the 2.4 kernel with 2.x Python for the time being."  

These are the main reasons, analyzed during our latest meeting, that led
to the current decision. We cannot make everyone happy at the same time,
regrettably; the decision we have reached now seems to be the best
solution we can achieve at present.

Truly the crux, you cannot and never will make everyone happy at the same time.  Last minute changes on weak foundation will only compound that situation.  
I am taking the time to write this response NOT in an attempt to reverse the current decision, though I do believe it is wrong and not accomplished in the most transparent manner that it could have been, I am writing this in the hopes of convincing the community that as Scala moves to the next level what was fine back in the Scala 2.2 days will not work in the 2.9 and 3.x days.  It's worthwhile to detail a bit more of my perspective as to why I'm being a bit harsh here with the above.
My small group, in a largish company is currently using Scala right now in parallel with and plans to completely replace early next year the first of several critical components.  Let just touch on a few aspects of said components.  
  - The absolute keystone of the whole architecture is Actor based.  I really want Scala and the JVM to offer nothing less than the best they can deliver in this regard. - Directly involved with revenue generation of several billion dollars annually, millions $$ every day.  - Performance is CRITICAL.  Reliability is CRITICAL.  Hell there is nothing about any aspect about of it that is not CRITICAL.   * 10's of millions of API invocations per day.   * Client / Server times below 100ms and designed to achieve 25-30ms responses. 24/7/365 HA, no outage window,  no single point of failure.    * Gigabyte of in-memory data leveraging large 64-bit heaps,  _must_ be processed in minutes and every single constituent second  of those minutes are reviewed.   * Every bit of advantage that 1.6 JVM gives me, whether its faster byte code verification, large heap GC performance, JITing, escape analysis, task switching, I/O, thread scheduling I'll take.  I do not have the time or inclination to empirically compare and contrast the details of what I'm losing with 1.5 over 1.6.  I am sure there are some here qualified to offer a heuristic argument as to why I'm really not really losing anything, but who has hard numbers...?  And I will tell you right now I'm in a sweat trying to recall the details of the Actor memory leak issue that I seem to recall being solved with 2.8+1.6,  is that now a problem for me back on 1.5 or was that a 1.4 issue?  Should I stop, take the time to reset the whole enchilada to 1.5?   As I sit typing this I see further clarifications on the impact appearing on the list and I'm feeling a bit more "OK" with what I'm reading than I did yesterday.  Remember, my little huff here is not on the current particular of the change, but how the whole process transpired.  - This particular project from conception was to take calculated risks and push the envelope in many facets of the architecture.  Constantly I reconsider each piece of the architecture including whether Scala was the correct choice.  Not just me, others as well, and I have to defend those decisions to them as well as to myself.  As part of that defense of Scala I often resort to the heavy concurrency, multicore capabilities of Scala and those to come,  over Java.  I can't tell you how many times I ask myself or am asked by others, "you said you needed Scala for X, but can't you get that with Java and library Y, or wouldn't  Java 7 give you that capability if you'd just waited or this JavaWorld article says Java and Kilim is superior to Scala, why did you chose Scala again?   Scala _needs_ to be clearly ahead of Java, nuanced differences are not enough to overcome adoption costs.  And I personally think that the winning area for Scala is clear and unambiguous advantages over Java in Concurrency, Parallelism and Distributed capability as well as code/ing advantages.
Ok, enough of my ranting on my "problems".   Hopefully it does emphasis the point, the Scala Team is correct, you will never, ever satisfy everyone.  The key issue here is the "surprise" aspect.  A simple "Yo, do you think the new release's long announced target deployment architecture could change ..." posting on the list and 72 hours later the motion is adopted on what appear to me to be solely "convenience" motivations for a subset of stakeholders.  Certainly a little more detail with the announcement would have reduced the acid churning.   
Roadmaps cannot appear to be whimsical.  The number of Scala community users will continue to expand.   An expanding number of those will have substantive stakes in the game.  You can't make them all happy.  Don't try.   You can demonstrate a steady hand on the tiller, a clear X on the chart to steer towards and dock into port with Swiss precision to the dot of the scheduled arrival time.   Please no course changes for anything less than hurricane avoidance.  Decisions that put one out on a swaying twig of branch are made in context, based on the best information at hand at the time, weighed against risk vs reward calibrated by a wet finger in the air.  Let's state the obvious, people with skin on the line can not and will not make decisions in Scala's favor beyond certain levels of uncertainties.  
To wrap up.  Once more my intent here is NOT to rescind the recent decision.  Given the clarifications now coming out on the list, along with a couple of days to mull over what we need to do and we'll see it through on our end.   What I really like to see from this teapot tempest going forward is some degree of confidence that when the Scala Team promulgates 2.9 _will_ or 3.0 will not, I and others can plan for exactly that unless a damn good reason rears its head.  My definition of a damn good reason?  I think the current movie 2012 captures the concept quite succinctly.
Sorry to get all soap boxish and all.  The Scala Core Team are genius academics, not politicians and without question are doing a fabulous job.  For me, a dominant part of the Scala decision, to this day a correct one at that, was and is driven by who they are, their tremendous capabilities, vision and demonstrated competency of execution.

Thanks for your time.

Ray

On Wed, December 2, 2009 1:01 am, Ray Racine wrote:
> Yikes !!! I don't understand what just happened or the "whys" of it.
>
> Frankly I thought the initial plan of jvm1.6 as the default along with a
> fully support 1.5jvm build was just fine.  The compromise to stop biasing
> on
> one as default and  going with two "peer" builds, one scala-jvm1.6 and one
> scala-jvm1.5 seemed a reasonable compromise.
>
> Why the 180 degree change?  As pointed out, JDK 1.6 (Mustang) was released
> 12/12/2006, three years ago.  1.5 (Tiger) was released 9/29/2004 5 years
> ago.
>
> Martin's original position on this subject made sense, the subsequent
> Scala
> teams compromise of having 2 co-equal builds was a nice comprise.
>
> Can someone summarize the rational in say 2-3 succinct reasons or less?
>
> Running the 1st Qtr 2010 upcoming major release of Scala, version 2.8,
> will
> not use Java 1.6, the current standard JVM released 3 years ago as the
> default and instead shall resort to a dynamic loading scheme because:
>  1) ????
>  2) ????
>  3) ????
>
> Obviously if I'm asking for 3 reasons justify the change in all fairness I
> should offer up 3 reasons as to why the original direction made perfect
> sense (to me).
>
> The original approach made sense because:
>  1) If you are a 1.5 shop, use the 2.8-jvm1.5 build.  If you are a 1.6
> shop
> (that's us), use the 2.8-jvm1.6 build.
>  2) A "dynamic loading scheme" with reflection is gratuitous and fraught
> with potential unintended consequences hanging out there.  e.g.,
>        - OSGi modules may now require special handling to deal with class
> for name loading.
>        - App servers, BEA or WebSphere can be particular with dynamic
> class
> loading, where allowed from etc.
>        - JVM security model restrictions on applets and devices.
>  3) As Martin pointed out initially (paraphrased)
>   - The main advantage of the jvm6 build is better concurrency
> support through the use of the more modern FJ library.
>   - Once the final 2.8 release is out, there might already be some fast
> parallel collection classes included, which would also rely on 1.6.
>   - I want to make fast concurrency and parallelism by default.
>   - Scala must be the obvious choice for harnessing concurrency
> and parallelism.
> 4) Martin again, "The main point is we want to leapfrog Java and we won't
> be
> able to do that by clinging onto versions that Sun/Oracle itself no longer
> supports."
>
>
> With eager anticipation I await the 1,2,3 reasons as to why the latest
> decision from the Scala meeting is superior to the original direction.
>
>
> Thanks,
>
>
> Ray
>
>
> On Tue, Dec 1, 2009 at 9:56 AM, martin odersky
> <martin.odersky@epfl.ch>wrote:
>
>> We just decided in the Scala meeting to keep Java 1.5 as not only the
>> default, but the only version required by Scala 2.8. We will make use
>> of dynamic class loading and reflection to load the real FJ  framework
>> instead of its 1.5 backport or java.util.concurrent. This decision
>> holds unless some unexpected problems come up with this dynamic
>> loading scheme.
>>
>> Cheers
>>
>>  -- Martin
>>
>
>
>
> --
> The object of life is not to be on the side of the majority, but to escape
> finding oneself in the ranks of the insane. - Marcus Aurelius
>

--
The object of life is not to be on the side of the majority, but to escape finding oneself in the ranks of the insane. - Marcus Aurelius

Hi Ray,

As far as I know, the only communication was that 2.8 would still support 1.5.
The split distributions showed up fairly late in the cycle. It was not a
sudden request by some of us to have only one distribution either- the issue
was discussed twice before this latest one.

At this point, the gain from building Scala against 1.6 would be faster class
loading. Actors will still get a better core on a 1.6 jvm.

The issue with separate distributions and tooling/libraries is real. sbt,
ScalaTest, ScalaCheck, specs, whatever tools you use are "tool X" and "tool
Y". Issues like this add to the complexity of developing these tools and
libraries. Time spent on these issues is better spent elsewhere. I was
happy to see the Scala team's analysis:

- major downside of having two builds: DLL HELL
     - we're at the bottom, if we split, everyone will have to split

because this is exactly the problem.

As for resistance to moving to 1.6 in the future, the notes from yesterday's
Scala meeting indicate that whatever the next version of Scala is (2.9, 3.0
or whatever), it will be targeted at 1.6. I think the proper advance notice
this time means you won't see the same resistance (I'll support it, at
least).

Personally, I am grateful for the Scala team reconsidering the decision for
split distributions. It is something that likely would have taken up time
that I could spend on more interesting things.

Thanks,
Mark

On Wednesday 02 December 2009 14:16, Ray Racine wrote:
> See Below. And my sincere appreciation for taking the time to respond. I
> am far less concerned about the relative impact of the change, my concerns
> are more oriented around the lateness of the change in the release cycle,
> the means in which it was done along with, IMHO, the rather weak foundation
> for doing it. I truly do not want to pain the Scala Core Team who go way
> beyond only to suffer emails such as mine. But I do want to make a point.
> Caution: I can get a little "over the top" so my apologies ahead of time.
>
> On Wed, Dec 2, 2009 at 2:25 AM, Antonio Cunei wrote:
>
> > Ray,
> >
> > In short, the reasons are:
> >
> > 1) If Scala is distributed in two separate versions, the set of libraries
> > and tools built on top may easily turn into a jungle of versions that are
> > 1.5-compatible and/or 1.6-specific. Cross-dependencies make the problem
> > worse: during our last Scala meeting (the notes will be published later
> > today) the example was made of a tool "X" that is only distributed in a
> > 1.5 version, and a tool "Y" that needs "X", but also the features only
> > available with 1.6 (FJ, for instance). So, X needs the 1.5 Scala jars,
> > while "Y" needs the 1.6 Scala jars. Hilarity ensues.
> >
>
> Agree 100%, IMHO 2.8 should be "centered" on single target JVM release. It
> is certainly unfair to expect the Scala team to support 2.8 on two JVMS.
> Frankly if I were you I'd keep 2.7.x for 1.5 and moved forward with 2.8 on
> 1.6. End of story. In general, I feel the Scala Core Team cannot and
> should not be whipsawed by a tool "X" decision.
>
> 2) We can obtain the same practical advantages of having a 1.6-specific
> > distribution by packaging 1.6-specific classfiles within the main jar, and
> > selecting those implementations automatically at runtime. The only thing
> > left out is the potential speed improvement deriving from the 1.6-specific
> > bytecode verifier, but that is something that we can live with for the
> > time being. In short: we get most of the advantages of a 1.6-specific
> > distribution, without causing a mess.
> >
>
> Practical advantage is subjective. There is a reasonable chance that some
> unintended consequence shall ensue for someone from this last minute shift
> in direction. Someone is always placed at a practical disadvantage. There
> will always be someone who wants all the advantages of and not "most of the
> advantages". There will always someone who does not want to surrender "a
> potential xxx improvement." Someone like me for example. :)
>
>
> > 3) lots of folk in industrial environments are seemingly stuck with 1.5.
> > We shall postpone the switch to 1.6 as the recommended default with a 2.9
> > version; the switch in version number may take place at whichever moment
> > may seem a better time for the switch. It seems to be too soon right now.
> >
>
> I am very much in an industrial environment myself. If an organization has
> not managed to switch to 1.6 in 3 years, what are the odds they will do so
> in 3.5 or 4? A couple of more points on this. Despite protestations to
> the contrary they are "stuck" by choice and shall remain so by choice. If a
> company is "fine" with staying with the 1.5 JVM; they are fine with staying
> with Scala 2.7.x. I will tell you I am far more sympathetic to the argument
> that some individual Mac user's may not be able to use Scala 1.8 than some
> fortune 500 company "can't" get from 1.5 to 1.6. A false premise.
>
> Imagine we were all watching this conversation from a distance on the
> Haskell, Perl, Python, etc list and late in the cycle a subset of Python 2.x
> users requested certain compromises be made that in some degree impair the
> new Python 3.x, e.g. "Please don't add those features in 3.x as that
> requires the new async I/O API and my shop only runs the 2.4 Kernel." Who
> would be surprised if the consensus response to that subset of Python users
> was anything other than "we believe the asynch I/O improvements are valuable
> and necessary to move Python forward for many of our users. The 2.6 kernel
> has been the standard kernel for all Linux distributions for over 3 years.
> If you are not ready we suggest you remain on the 2.4 kernel with 2.x
> Python for the time being."
>
>
> > These are the main reasons, analyzed during our latest meeting, that led
> > to the current decision. We cannot make everyone happy at the same time,
> > regrettably; the decision we have reached now seems to be the best
> > solution we can achieve at present.
> >
>
> Truly the crux, you cannot and never will make everyone happy at the same
> time. Last minute changes on weak foundation will only compound that
> situation.
>
> I am taking the time to write this response NOT in an attempt to reverse the
> current decision, though I do believe it is wrong and not accomplished in
> the most transparent manner that it could have been, I am writing this in
> the hopes of convincing the community that as Scala moves to the next level
> what was fine back in the Scala 2.2 days will not work in the 2.9 and 3.x
> days. It's worthwhile to detail a bit more of my perspective as to why I'm
> being a bit harsh here with the above.
>
> My small group, in a largish company is currently using Scala right now in
> parallel with and plans to completely replace early next year the first of
> several critical components. Let just touch on a few aspects of said
> components.
>
> - The absolute keystone of the whole architecture is Actor based. I
> really want Scala and the JVM to offer nothing less than the best they can
> deliver in this regard.
> - Directly involved with revenue generation of several billion dollars
> annually, millions $$ every day.
> - Performance is CRITICAL. Reliability is CRITICAL. Hell there is nothing
> about any aspect about of it that is not CRITICAL.
> * 10's of millions of API invocations per day.
> * Client / Server times below 100ms and designed to achieve 25-30ms
> responses. 24/7/365 HA, no outage window, no single point of failure.
> * Gigabyte of in-memory data leveraging large 64-bit heaps, _must_ be
> processed in minutes and every single constituent second of those minutes
> are reviewed.
> * Every bit of advantage that 1.6 JVM gives me, whether its faster byte
> code verification, large heap GC performance, JITing, escape analysis, task
> switching, I/O, thread scheduling I'll take. I do not have the time or
> inclination to empirically compare and contrast the details of what I'm
> losing with 1.5 over 1.6. I am sure there are some here qualified to offer
> a heuristic argument as to why I'm really not really losing anything, but
> who has hard numbers...? And I will tell you right now I'm in a sweat
> trying to recall the details of the Actor memory leak issue that I seem to
> recall being solved with 2.8+1.6, is that now a problem for me back on 1.5
> or was that a 1.4 issue? Should I stop, take the time to reset the whole
> enchilada to 1.5? As I sit typing this I see further clarifications on the
> impact appearing on the list and I'm feeling a bit more "OK" with what I'm
> reading than I did yesterday. Remember, my little huff here is not on the
> current particular of the change, but how the whole process transpired.
>
> - This particular project from conception was to take calculated risks and
> push the envelope in many facets of the architecture. Constantly I
> reconsider each piece of the architecture including whether Scala was the
> correct choice. Not just me, others as well, and I have to defend those
> decisions to them as well as to myself. As part of that defense of Scala I
> often resort to the heavy concurrency, multicore capabilities of Scala and
> those to come, over Java. I can't tell you how many times I ask myself or
> am asked by others, "you said you needed Scala for X, but can't you get that
> with Java and library Y, or wouldn't Java 7 give you that capability if
> you'd just waited or this JavaWorld article says Java and Kilim is superior
> to Scala, why did you chose Scala again? Scala _needs_ to be clearly ahead
> of Java, nuanced differences are not enough to overcome adoption costs. And
> I personally think that the winning area for Scala is clear
> and unambiguous advantages over Java in Concurrency, Parallelism and
> Distributed capability as well as code/ing advantages.
>
> Ok, enough of my ranting on my "problems". Hopefully it does emphasis the
> point, the Scala Team is correct, you will never, ever satisfy everyone.
> The key issue here is the "surprise" aspect. A simple "Yo, do you think
> the new release's long announced target deployment architecture could change
> ..." posting on the list and 72 hours later the motion is adopted on what
> appear to me to be solely "convenience" motivations for a subset of
> stakeholders. Certainly a little more detail with the announcement would
> have reduced the acid churning.
>
> Roadmaps cannot appear to be whimsical. The number of Scala community users
> will continue to expand. An expanding number of those will have
> substantive stakes in the game. You can't make them all happy. Don't try.
> You can demonstrate a steady hand on the tiller, a clear X on the chart to
> steer towards and dock into port with Swiss precision to the dot of the
> scheduled arrival time. Please no course changes for anything less
> than hurricane avoidance. Decisions that put one out on a swaying twig of
> branch are made in context, based on the best information at hand at the
> time, weighed against risk vs reward calibrated by a wet finger in the air.
> Let's state the obvious, people with skin on the line can not and will not
> make decisions in Scala's favor beyond certain levels of uncertainties.
>
> To wrap up. Once more my intent here is NOT to rescind the recent decision.
> Given the clarifications now coming out on the list, along with a couple of
> days to mull over what we need to do and we'll see it through on our end.
> What I really like to see from this teapot tempest going forward is some
> degree of confidence that when the Scala Team promulgates 2.9 _will_ or 3.0
> will not, I and others can plan for exactly that unless a damn good reason
> rears its head. My definition of a damn good reason? I think the current
> movie 2012 captures the concept quite succinctly.
>
> Sorry to get all soap boxish and all. The Scala Core Team
> are genius academics, not politicians and without question are doing a
> fabulous job. For me, a dominant part of the Scala decision, to this day
> a correct one at that, was and is driven by who they are,
> their tremendous capabilities, vision and demonstrated competency of
> execution.
>
>
> Thanks for your time.
>
>
> Ray
>
>
> > On Wed, December 2, 2009 1:01 am, Ray Racine wrote:
> > > Yikes !!! I don't understand what just happened or the "whys" of it.
> > >
> > > Frankly I thought the initial plan of jvm1.6 as the default along with a
> > > fully support 1.5jvm build was just fine. The compromise to stop
biasing
> > > on
> > > one as default and going with two "peer" builds, one scala-jvm1.6 and
> > one
> > > scala-jvm1.5 seemed a reasonable compromise.
> > >
> > > Why the 180 degree change? As pointed out, JDK 1.6 (Mustang) was
> > released
> > > 12/12/2006, three years ago. 1.5 (Tiger) was released 9/29/2004 5 years
> > > ago.
> > >
> > > Martin's original position on this subject made sense, the subsequent
> > > Scala
> > > teams compromise of having 2 co-equal builds was a nice comprise.
> > >
> > > Can someone summarize the rational in say 2-3 succinct reasons or less?
> > >
> > > Running the 1st Qtr 2010 upcoming major release of Scala, version 2.8,
> > > will
> > > not use Java 1.6, the current standard JVM released 3 years ago as the
> > > default and instead shall resort to a dynamic loading scheme because:
> > > 1) ????
> > > 2) ????
> > > 3) ????
> > >
> > > Obviously if I'm asking for 3 reasons justify the change in all fairness
> > I
> > > should offer up 3 reasons as to why the original direction made perfect
> > > sense (to me).
> > >
> > > The original approach made sense because:
> > > 1) If you are a 1.5 shop, use the 2.8-jvm1.5 build. If you are a 1.6
> > > shop
> > > (that's us), use the 2.8-jvm1.6 build.
> > > 2) A "dynamic loading scheme" with reflection is gratuitous and fraught
> > > with potential unintended consequences hanging out there. e.g.,
> > > - OSGi modules may now require special handling to deal with
class
> > > for name loading.
> > > - App servers, BEA or WebSphere can be particular with dynamic
> > > class
> > > loading, where allowed from etc.
> > > - JVM security model restrictions on applets and devices.
> > > 3) As Martin pointed out initially (paraphrased)
> > > - The main advantage of the jvm6 build is better concurrency
> > > support through the use of the more modern FJ library.
> > > - Once the final 2.8 release is out, there might already be some fast
> > > parallel collection classes included, which would also rely on 1.6.
> > > - I want to make fast concurrency and parallelism by default.
> > > - Scala must be the obvious choice for harnessing concurrency
> > > and parallelism.
> > > 4) Martin again, "The main point is we want to leapfrog Java and we
won't
> > > be
> > > able to do that by clinging onto versions that Sun/Oracle itself no
> > longer
> > > supports."
> > >
> > >
> > > With eager anticipation I await the 1,2,3 reasons as to why the latest
> > > decision from the Scala meeting is superior to the original direction.
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Ray
> > >
> > >
> > > On Tue, Dec 1, 2009 at 9:56 AM, martin odersky
> > > wrote:
> > >
> > >> We just decided in the Scala meeting to keep Java 1.5 as not only the
> > >> default, but the only version required by Scala 2.8. We will make use
> > >> of dynamic class loading and reflection to load the real FJ framework
> > >> instead of its 1.5 backport or java.util.concurrent. This decision
> > >> holds unless some unexpected problems come up with this dynamic
> > >> loading scheme.
> > >>
> > >> Cheers
> > >>
> > >> -- Martin
> > >>
> > >
> > >
> > >
> > > --
> > > The object of life is not to be on the side of the majority, but to
> > escape
> > > finding oneself in the ranks of the insane. - Marcus Aurelius
> > >
> >
> >
> >
>
>

Ray,
To be fair, having separate distribution for 1.5 and 1.6 was really the initial surprise.  This is really a return to the original road map for 2.8.  So I think your arguments about road map stability really support the decision.  Separate distributions would have created significant issues with down-stream libraries and tools.  Supporting 1.6 only would represent a significant departure from the published Java version support plans.  I think they've reached the right decision.

-Erik

On Wed, Dec 2, 2009 at 2:16 PM, Ray Racine <ray.racine@gmail.com> wrote:

See Below.  And my sincere appreciation for taking the time to respond.  I am far less concerned about the relative impact of the change, my concerns are more oriented around the lateness of the change in the release cycle, the means in which it was done along with, IMHO, the rather weak foundation for doing it.  I truly do not want to pain the Scala Core Team who go way beyond only to suffer emails such as mine.  But I do want to make a point.  Caution: I can get a little "over the top" so my apologies ahead of time. 
On Wed, Dec 2, 2009 at 2:25 AM, Antonio Cunei <antonio.cunei@epfl.ch> wrote:

Ray,

In short, the reasons are:

1) If Scala is distributed in two separate versions, the set of libraries
and tools built on top may easily turn into a jungle of versions that are
1.5-compatible and/or 1.6-specific. Cross-dependencies make the problem
worse: during our last Scala meeting (the notes will be published later
today) the example was made of a tool "X" that is only distributed in a
1.5 version, and a tool "Y" that needs "X", but also the features only
available with 1.6 (FJ, for instance). So, X needs the 1.5 Scala jars,
while "Y" needs the 1.6 Scala jars. Hilarity ensues.

 Agree 100%, IMHO 2.8 should be "centered" on single target JVM release.  It is certainly unfair to expect the Scala team to support 2.8 on two JVMS.   Frankly if I were you I'd keep 2.7.x for 1.5 and moved forward with 2.8 on 1.6.  End of story.  In general, I feel the Scala Core Team cannot and should not be whipsawed by a tool "X" decision.  

2) We can obtain the same practical advantages of having a 1.6-specific
distribution by packaging 1.6-specific classfiles within the main jar, and
selecting those implementations automatically at runtime. The only thing
left out is the potential speed improvement deriving from the 1.6-specific
bytecode verifier, but that is something that we can live with for the
time being. In short: we get most of the advantages of a 1.6-specific
distribution, without causing a mess.

Practical advantage is subjective.  There is a reasonable chance that some unintended consequence shall ensue for someone from this last minute shift in direction.  Someone is always placed at a practical disadvantage.  There will always be someone who wants all the advantages of and not "most of the advantages".  There will always someone who does not want to surrender "a potential xxx improvement."  Someone like me for example. :)  

3) lots of folk in industrial environments are seemingly stuck with 1.5.
We shall postpone the switch to 1.6 as the recommended default with a 2.9
version; the switch in version number may take place at whichever moment
may seem a better time for the switch. It seems to be too soon right now.

I am very much in an industrial environment myself.  If an organization has not managed to switch to 1.6 in 3 years, what are the odds they will do so in 3.5 or 4?  A couple of more points on this.  Despite  protestations to the contrary they are "stuck" by choice and shall remain so by choice.  If a company is "fine" with staying with the 1.5 JVM; they are fine with staying with Scala 2.7.x.  I will tell you I am far more sympathetic to the argument that some individual Mac user's may not be able to use Scala 1.8 than some fortune 500 company "can't" get from 1.5 to 1.6.   A false premise.
Imagine we were all watching this conversation from a distance on the Haskell, Perl, Python, etc list and late in the cycle a subset of Python 2.x users requested certain compromises be made that in some degree impair the new Python 3.x, e.g. "Please don't add those features in 3.x as that requires the new async I/O API and my shop only runs the 2.4 Kernel."  Who would be surprised if the consensus response to that subset of Python users was anything other than "we believe the asynch I/O improvements are valuable and necessary to move Python forward for many of our users.  The 2.6 kernel has been the standard kernel for all Linux distributions for over 3 years.  If you are not ready we suggest you remain on the 2.4 kernel with 2.x Python for the time being."  

These are the main reasons, analyzed during our latest meeting, that led
to the current decision. We cannot make everyone happy at the same time,
regrettably; the decision we have reached now seems to be the best
solution we can achieve at present.

Truly the crux, you cannot and never will make everyone happy at the same time.  Last minute changes on weak foundation will only compound that situation.  
I am taking the time to write this response NOT in an attempt to reverse the current decision, though I do believe it is wrong and not accomplished in the most transparent manner that it could have been, I am writing this in the hopes of convincing the community that as Scala moves to the next level what was fine back in the Scala 2.2 days will not work in the 2.9 and 3.x days.  It's worthwhile to detail a bit more of my perspective as to why I'm being a bit harsh here with the above.
My small group, in a largish company is currently using Scala right now in parallel with and plans to completely replace early next year the first of several critical components.  Let just touch on a few aspects of said components.  
  - The absolute keystone of the whole architecture is Actor based.  I really want Scala and the JVM to offer nothing less than the best they can deliver in this regard. - Directly involved with revenue generation of several billion dollars annually, millions $$ every day.  - Performance is CRITICAL.  Reliability is CRITICAL.  Hell there is nothing about any aspect about of it that is not CRITICAL.   * 10's of millions of API invocations per day.   * Client / Server times below 100ms and designed to achieve 25-30ms responses. 24/7/365 HA, no outage window,  no single point of failure.    * Gigabyte of in-memory data leveraging large 64-bit heaps,  _must_ be processed in minutes and every single constituent second  of those minutes are reviewed.   * Every bit of advantage that 1.6 JVM gives me, whether its faster byte code verification, large heap GC performance, JITing, escape analysis, task switching, I/O, thread scheduling I'll take.  I do not have the time or inclination to empirically compare and contrast the details of what I'm losing with 1.5 over 1.6.  I am sure there are some here qualified to offer a heuristic argument as to why I'm really not really losing anything, but who has hard numbers...?  And I will tell you right now I'm in a sweat trying to recall the details of the Actor memory leak issue that I seem to recall being solved with 2.8+1.6,  is that now a problem for me back on 1.5 or was that a 1.4 issue?  Should I stop, take the time to reset the whole enchilada to 1.5?   As I sit typing this I see further clarifications on the impact appearing on the list and I'm feeling a bit more "OK" with what I'm reading than I did yesterday.  Remember, my little huff here is not on the current particular of the change, but how the whole process transpired.  - This particular project from conception was to take calculated risks and push the envelope in many facets of the architecture.  Constantly I reconsider each piece of the architecture including whether Scala was the correct choice.  Not just me, others as well, and I have to defend those decisions to them as well as to myself.  As part of that defense of Scala I often resort to the heavy concurrency, multicore capabilities of Scala and those to come,  over Java.  I can't tell you how many times I ask myself or am asked by others, "you said you needed Scala for X, but can't you get that with Java and library Y, or wouldn't  Java 7 give you that capability if you'd just waited or this JavaWorld article says Java and Kilim is superior to Scala, why did you chose Scala again?   Scala _needs_ to be clearly ahead of Java, nuanced differences are not enough to overcome adoption costs.  And I personally think that the winning area for Scala is clear and unambiguous advantages over Java in Concurrency, Parallelism and Distributed capability as well as code/ing advantages.
Ok, enough of my ranting on my "problems".   Hopefully it does emphasis the point, the Scala Team is correct, you will never, ever satisfy everyone.  The key issue here is the "surprise" aspect.  A simple "Yo, do you think the new release's long announced target deployment architecture could change ..." posting on the list and 72 hours later the motion is adopted on what appear to me to be solely "convenience" motivations for a subset of stakeholders.  Certainly a little more detail with the announcement would have reduced the acid churning.   
Roadmaps cannot appear to be whimsical.  The number of Scala community users will continue to expand.   An expanding number of those will have substantive stakes in the game.  You can't make them all happy.  Don't try.   You can demonstrate a steady hand on the tiller, a clear X on the chart to steer towards and dock into port with Swiss precision to the dot of the scheduled arrival time.   Please no course changes for anything less than hurricane avoidance.  Decisions that put one out on a swaying twig of branch are made in context, based on the best information at hand at the time, weighed against risk vs reward calibrated by a wet finger in the air.  Let's state the obvious, people with skin on the line can not and will not make decisions in Scala's favor beyond certain levels of uncertainties.  
To wrap up.  Once more my intent here is NOT to rescind the recent decision.  Given the clarifications now coming out on the list, along with a couple of days to mull over what we need to do and we'll see it through on our end.   What I really like to see from this teapot tempest going forward is some degree of confidence that when the Scala Team promulgates 2.9 _will_ or 3.0 will not, I and others can plan for exactly that unless a damn good reason rears its head.  My definition of a damn good reason?  I think the current movie 2012 captures the concept quite succinctly.
Sorry to get all soap boxish and all.  The Scala Core Team are genius academics, not politicians and without question are doing a fabulous job.  For me, a dominant part of the Scala decision, to this day a correct one at that, was and is driven by who they are, their tremendous capabilities, vision and demonstrated competency of execution.

Thanks for your time.

Ray

On Wed, December 2, 2009 1:01 am, Ray Racine wrote:
> Yikes !!! I don't understand what just happened or the "whys" of it.
>
> Frankly I thought the initial plan of jvm1.6 as the default along with a
> fully support 1.5jvm build was just fine.  The compromise to stop biasing
> on
> one as default and  going with two "peer" builds, one scala-jvm1.6 and one
> scala-jvm1.5 seemed a reasonable compromise.
>
> Why the 180 degree change?  As pointed out, JDK 1.6 (Mustang) was released
> 12/12/2006, three years ago.  1.5 (Tiger) was released 9/29/2004 5 years
> ago.
>
> Martin's original position on this subject made sense, the subsequent
> Scala
> teams compromise of having 2 co-equal builds was a nice comprise.
>
> Can someone summarize the rational in say 2-3 succinct reasons or less?
>
> Running the 1st Qtr 2010 upcoming major release of Scala, version 2.8,
> will
> not use Java 1.6, the current standard JVM released 3 years ago as the
> default and instead shall resort to a dynamic loading scheme because:
>  1) ????
>  2) ????
>  3) ????
>
> Obviously if I'm asking for 3 reasons justify the change in all fairness I
> should offer up 3 reasons as to why the original direction made perfect
> sense (to me).
>
> The original approach made sense because:
>  1) If you are a 1.5 shop, use the 2.8-jvm1.5 build.  If you are a 1.6
> shop
> (that's us), use the 2.8-jvm1.6 build.
>  2) A "dynamic loading scheme" with reflection is gratuitous and fraught
> with potential unintended consequences hanging out there.  e.g.,
>        - OSGi modules may now require special handling to deal with class
> for name loading.
>        - App servers, BEA or WebSphere can be particular with dynamic
> class
> loading, where allowed from etc.
>        - JVM security model restrictions on applets and devices.
>  3) As Martin pointed out initially (paraphrased)
>   - The main advantage of the jvm6 build is better concurrency
> support through the use of the more modern FJ library.
>   - Once the final 2.8 release is out, there might already be some fast
> parallel collection classes included, which would also rely on 1.6.
>   - I want to make fast concurrency and parallelism by default.
>   - Scala must be the obvious choice for harnessing concurrency
> and parallelism.
> 4) Martin again, "The main point is we want to leapfrog Java and we won't
> be
> able to do that by clinging onto versions that Sun/Oracle itself no longer
> supports."
>
>
> With eager anticipation I await the 1,2,3 reasons as to why the latest
> decision from the Scala meeting is superior to the original direction.
>
>
> Thanks,
>
>
> Ray
>
>
> On Tue, Dec 1, 2009 at 9:56 AM, martin odersky
> <martin.odersky@epfl.ch>wrote:
>
>> We just decided in the Scala meeting to keep Java 1.5 as not only the
>> default, but the only version required by Scala 2.8. We will make use
>> of dynamic class loading and reflection to load the real FJ  framework
>> instead of its 1.5 backport or java.util.concurrent. This decision
>> holds unless some unexpected problems come up with this dynamic
>> loading scheme.
>>
>> Cheers
>>
>>  -- Martin
>>
>
>
>
> --
> The object of life is not to be on the side of the majority, but to escape
> finding oneself in the ranks of the insane. - Marcus Aurelius
>

--
The object of life is not to be on the side of the majority, but to escape finding oneself in the ranks of the insane. - Marcus Aurelius

--
http://erikengbrecht.blogspot.com/

On Wed, Dec 02, 2009 at 02:16:02PM -0500, Ray Racine wrote:
> See Below. And my sincere appreciation for taking the time to
> respond. I am far less concerned about the relative impact of the
> change, my concerns are more oriented around the lateness of the
> change in the release cycle, the means in which it was done along
> with, IMHO, the rather weak foundation for doing it.

OK, I didn't read the rest of your email because it's too long for me,
but the late change was from 1.5 to 1.6, not the other way around. The
entire last year of development took place with the presumption that 1.5
was the baseline. (In fact it took many months before the idea of
supporting 1.4 was finally abandoned.)

> Practical advantage is subjective. There is a reasonable chance that
> some unintended consequence shall ensue for someone from this last
> minute shift in direction.

You have a seriously tilted idea of the order of events. The last
minute shift is what is (thankfully) now being avoided.

Erik,
The whole stability thing was supposed to be the point of the email.
1) My primary concerns are with the shifting itself, timings of, the inputs that I perceived to be driving the stay or shift back decision in the current discussion chain, and _not_ necessarily on the final outcome of the shifting.  Of course, I readily admit learning the 2.8+1.6 Actors are now scheduled for a bit of open heart surgery on the cusp of 2.8 Beta is not what I prefer  to hear right now, but the doctors are top notch.
2) It was _not_ supposed to be a plea for 2.8 to be on 1.6.  It was an attempt to show the impact these cycling decisions have.
3) Ideally the next release plan should be clear and unambiguous up front, firm to a freeze at the appropriate time, and should not change without compelling cause.

On Wed, Dec 2, 2009 at 3:06 PM, Erik Engbrecht <erik.engbrecht@gmail.com> wrote:

Ray,
To be fair, having separate distribution for 1.5 and 1.6 was really the initial surprise.  This is really a return to the original road map for 2.8.  So I think your arguments about road map stability really support the decision.  Separate distributions would have created significant issues with down-stream libraries and tools.  Supporting 1.6 only would represent a significant departure from the published Java version support plans.  I think they've reached the right decision.

-Erik

On Wed, Dec 2, 2009 at 2:16 PM, Ray Racine <ray.racine@gmail.com> wrote:

See Below.  And my sincere appreciation for taking the time to respond.  I am far less concerned about the relative impact of the change, my concerns are more oriented around the lateness of the change in the release cycle, the means in which it was done along with, IMHO, the rather weak foundation for doing it.  I truly do not want to pain the Scala Core Team who go way beyond only to suffer emails such as mine.  But I do want to make a point.  Caution: I can get a little "over the top" so my apologies ahead of time. 
On Wed, Dec 2, 2009 at 2:25 AM, Antonio Cunei <antonio.cunei@epfl.ch> wrote:

Ray,

In short, the reasons are:

1) If Scala is distributed in two separate versions, the set of libraries
and tools built on top may easily turn into a jungle of versions that are
1.5-compatible and/or 1.6-specific. Cross-dependencies make the problem
worse: during our last Scala meeting (the notes will be published later
today) the example was made of a tool "X" that is only distributed in a
1.5 version, and a tool "Y" that needs "X", but also the features only
available with 1.6 (FJ, for instance). So, X needs the 1.5 Scala jars,
while "Y" needs the 1.6 Scala jars. Hilarity ensues.

 Agree 100%, IMHO 2.8 should be "centered" on single target JVM release.  It is certainly unfair to expect the Scala team to support 2.8 on two JVMS.   Frankly if I were you I'd keep 2.7.x for 1.5 and moved forward with 2.8 on 1.6.  End of story.  In general, I feel the Scala Core Team cannot and should not be whipsawed by a tool "X" decision.  

2) We can obtain the same practical advantages of having a 1.6-specific
distribution by packaging 1.6-specific classfiles within the main jar, and
selecting those implementations automatically at runtime. The only thing
left out is the potential speed improvement deriving from the 1.6-specific
bytecode verifier, but that is something that we can live with for the
time being. In short: we get most of the advantages of a 1.6-specific
distribution, without causing a mess.

Practical advantage is subjective.  There is a reasonable chance that some unintended consequence shall ensue for someone from this last minute shift in direction.  Someone is always placed at a practical disadvantage.  There will always be someone who wants all the advantages of and not "most of the advantages".  There will always someone who does not want to surrender "a potential xxx improvement."  Someone like me for example. :)  

3) lots of folk in industrial environments are seemingly stuck with 1.5.
We shall postpone the switch to 1.6 as the recommended default with a 2.9
version; the switch in version number may take place at whichever moment
may seem a better time for the switch. It seems to be too soon right now.

I am very much in an industrial environment myself.  If an organization has not managed to switch to 1.6 in 3 years, what are the odds they will do so in 3.5 or 4?  A couple of more points on this.  Despite  protestations to the contrary they are "stuck" by choice and shall remain so by choice.  If a company is "fine" with staying with the 1.5 JVM; they are fine with staying with Scala 2.7.x.  I will tell you I am far more sympathetic to the argument that some individual Mac user's may not be able to use Scala 1.8 than some fortune 500 company "can't" get from 1.5 to 1.6.   A false premise.
Imagine we were all watching this conversation from a distance on the Haskell, Perl, Python, etc list and late in the cycle a subset of Python 2.x users requested certain compromises be made that in some degree impair the new Python 3.x, e.g. "Please don't add those features in 3.x as that requires the new async I/O API and my shop only runs the 2.4 Kernel."  Who would be surprised if the consensus response to that subset of Python users was anything other than "we believe the asynch I/O improvements are valuable and necessary to move Python forward for many of our users.  The 2.6 kernel has been the standard kernel for all Linux distributions for over 3 years.  If you are not ready we suggest you remain on the 2.4 kernel with 2.x Python for the time being."  

These are the main reasons, analyzed during our latest meeting, that led
to the current decision. We cannot make everyone happy at the same time,
regrettably; the decision we have reached now seems to be the best
solution we can achieve at present.

Truly the crux, you cannot and never will make everyone happy at the same time.  Last minute changes on weak foundation will only compound that situation.  
I am taking the time to write this response NOT in an attempt to reverse the current decision, though I do believe it is wrong and not accomplished in the most transparent manner that it could have been, I am writing this in the hopes of convincing the community that as Scala moves to the next level what was fine back in the Scala 2.2 days will not work in the 2.9 and 3.x days.  It's worthwhile to detail a bit more of my perspective as to why I'm being a bit harsh here with the above.
My small group, in a largish company is currently using Scala right now in parallel with and plans to completely replace early next year the first of several critical components.  Let just touch on a few aspects of said components.  
  - The absolute keystone of the whole architecture is Actor based.  I really want Scala and the JVM to offer nothing less than the best they can deliver in this regard. - Directly involved with revenue generation of several billion dollars annually, millions $$ every day.  - Performance is CRITICAL.  Reliability is CRITICAL.  Hell there is nothing about any aspect about of it that is not CRITICAL.   * 10's of millions of API invocations per day.   * Client / Server times below 100ms and designed to achieve 25-30ms responses. 24/7/365 HA, no outage window,  no single point of failure.    * Gigabyte of in-memory data leveraging large 64-bit heaps,  _must_ be processed in minutes and every single constituent second  of those minutes are reviewed.   * Every bit of advantage that 1.6 JVM gives me, whether its faster byte code verification, large heap GC performance, JITing, escape analysis, task switching, I/O, thread scheduling I'll take.  I do not have the time or inclination to empirically compare and contrast the details of what I'm losing with 1.5 over 1.6.  I am sure there are some here qualified to offer a heuristic argument as to why I'm really not really losing anything, but who has hard numbers...?  And I will tell you right now I'm in a sweat trying to recall the details of the Actor memory leak issue that I seem to recall being solved with 2.8+1.6,  is that now a problem for me back on 1.5 or was that a 1.4 issue?  Should I stop, take the time to reset the whole enchilada to 1.5?   As I sit typing this I see further clarifications on the impact appearing on the list and I'm feeling a bit more "OK" with what I'm reading than I did yesterday.  Remember, my little huff here is not on the current particular of the change, but how the whole process transpired.  - This particular project from conception was to take calculated risks and push the envelope in many facets of the architecture.  Constantly I reconsider each piece of the architecture including whether Scala was the correct choice.  Not just me, others as well, and I have to defend those decisions to them as well as to myself.  As part of that defense of Scala I often resort to the heavy concurrency, multicore capabilities of Scala and those to come,  over Java.  I can't tell you how many times I ask myself or am asked by others, "you said you needed Scala for X, but can't you get that with Java and library Y, or wouldn't  Java 7 give you that capability if you'd just waited or this JavaWorld article says Java and Kilim is superior to Scala, why did you chose Scala again?   Scala _needs_ to be clearly ahead of Java, nuanced differences are not enough to overcome adoption costs.  And I personally think that the winning area for Scala is clear and unambiguous advantages over Java in Concurrency, Parallelism and Distributed capability as well as code/ing advantages.
Ok, enough of my ranting on my "problems".   Hopefully it does emphasis the point, the Scala Team is correct, you will never, ever satisfy everyone.  The key issue here is the "surprise" aspect.  A simple "Yo, do you think the new release's long announced target deployment architecture could change ..." posting on the list and 72 hours later the motion is adopted on what appear to me to be solely "convenience" motivations for a subset of stakeholders.  Certainly a little more detail with the announcement would have reduced the acid churning.   
Roadmaps cannot appear to be whimsical.  The number of Scala community users will continue to expand.   An expanding number of those will have substantive stakes in the game.  You can't make them all happy.  Don't try.   You can demonstrate a steady hand on the tiller, a clear X on the chart to steer towards and dock into port with Swiss precision to the dot of the scheduled arrival time.   Please no course changes for anything less than hurricane avoidance.  Decisions that put one out on a swaying twig of branch are made in context, based on the best information at hand at the time, weighed against risk vs reward calibrated by a wet finger in the air.  Let's state the obvious, people with skin on the line can not and will not make decisions in Scala's favor beyond certain levels of uncertainties.  
To wrap up.  Once more my intent here is NOT to rescind the recent decision.  Given the clarifications now coming out on the list, along with a couple of days to mull over what we need to do and we'll see it through on our end.   What I really like to see from this teapot tempest going forward is some degree of confidence that when the Scala Team promulgates 2.9 _will_ or 3.0 will not, I and others can plan for exactly that unless a damn good reason rears its head.  My definition of a damn good reason?  I think the current movie 2012 captures the concept quite succinctly.
Sorry to get all soap boxish and all.  The Scala Core Team are genius academics, not politicians and without question are doing a fabulous job.  For me, a dominant part of the Scala decision, to this day a correct one at that, was and is driven by who they are, their tremendous capabilities, vision and demonstrated competency of execution.

Thanks for your time.

Ray

On Wed, December 2, 2009 1:01 am, Ray Racine wrote:
> Yikes !!! I don't understand what just happened or the "whys" of it.
>
> Frankly I thought the initial plan of jvm1.6 as the default along with a
> fully support 1.5jvm build was just fine.  The compromise to stop biasing
> on
> one as default and  going with two "peer" builds, one scala-jvm1.6 and one
> scala-jvm1.5 seemed a reasonable compromise.
>
> Why the 180 degree change?  As pointed out, JDK 1.6 (Mustang) was released
> 12/12/2006, three years ago.  1.5 (Tiger) was released 9/29/2004 5 years
> ago.
>
> Martin's original position on this subject made sense, the subsequent
> Scala
> teams compromise of having 2 co-equal builds was a nice comprise.
>
> Can someone summarize the rational in say 2-3 succinct reasons or less?
>
> Running the 1st Qtr 2010 upcoming major release of Scala, version 2.8,
> will
> not use Java 1.6, the current standard JVM released 3 years ago as the
> default and instead shall resort to a dynamic loading scheme because:
>  1) ????
>  2) ????
>  3) ????
>
> Obviously if I'm asking for 3 reasons justify the change in all fairness I
> should offer up 3 reasons as to why the original direction made perfect
> sense (to me).
>
> The original approach made sense because:
>  1) If you are a 1.5 shop, use the 2.8-jvm1.5 build.  If you are a 1.6
> shop
> (that's us), use the 2.8-jvm1.6 build.
>  2) A "dynamic loading scheme" with reflection is gratuitous and fraught
> with potential unintended consequences hanging out there.  e.g.,
>        - OSGi modules may now require special handling to deal with class
> for name loading.
>        - App servers, BEA or WebSphere can be particular with dynamic
> class
> loading, where allowed from etc.
>        - JVM security model restrictions on applets and devices.
>  3) As Martin pointed out initially (paraphrased)
>   - The main advantage of the jvm6 build is better concurrency
> support through the use of the more modern FJ library.
>   - Once the final 2.8 release is out, there might already be some fast
> parallel collection classes included, which would also rely on 1.6.
>   - I want to make fast concurrency and parallelism by default.
>   - Scala must be the obvious choice for harnessing concurrency
> and parallelism.
> 4) Martin again, "The main point is we want to leapfrog Java and we won't
> be
> able to do that by clinging onto versions that Sun/Oracle itself no longer
> supports."
>
>
> With eager anticipation I await the 1,2,3 reasons as to why the latest
> decision from the Scala meeting is superior to the original direction.
>
>
> Thanks,
>
>
> Ray
>
>
> On Tue, Dec 1, 2009 at 9:56 AM, martin odersky
> <martin.odersky@epfl.ch>wrote:
>
>> We just decided in the Scala meeting to keep Java 1.5 as not only the
>> default, but the only version required by Scala 2.8. We will make use
>> of dynamic class loading and reflection to load the real FJ  framework
>> instead of its 1.5 backport or java.util.concurrent. This decision
>> holds unless some unexpected problems come up with this dynamic
>> loading scheme.
>>
>> Cheers
>>
>>  -- Martin
>>
>
>
>
> --
> The object of life is not to be on the side of the majority, but to escape
> finding oneself in the ranks of the insane. - Marcus Aurelius
>

--
The object of life is not to be on the side of the majority, but to escape finding oneself in the ranks of the insane. - Marcus Aurelius

--
http://erikengbrecht.blogspot.com/

--
The object of life is not to be on the side of the majority, but to escape finding oneself in the ranks of the insane. - Marcus Aurelius

Ray,
  I think you're expectations are higher than can be realized without a more substantial base of active contributors to Scala.  I don't think the active contributor base grow that much without the development process being far more open than it has been in the past.  The team is actively working to open it up, and has made substantial progress.

  Given the current resources available I think the best outcome that can be expected has been achieved.  I don't think the flip-flopping could have been avoided.  Only the communication of the flip-flopping could have been.  From a tactical perspective avoiding the communication would have lead to less community feedback, and quite likely a half-baked decision actually ending up in a release as opposed to a pre-beta.  Strategically, avoiding the communication would have perpetuated the perception that Scala development is a rather closed, black box and thus reduced the likelihood of attracting more active contributors.

  It's unfortunate that it took a pre-beta release in order to attract sufficient community attention to get the issue solidly resolved.  But I think that's just the current state of affairs regarding Scala.  In the past it has taken doing a full-up release in order to get people engaged (i.e. screaming).  Pre-beta may not be ideal, but it is substantial progress.

  My point is I think issues like this and the somewhat ad-hoc way they are resolved are just growing pains for Scala development, and I think too agressively pushing to make the process appear more controlled will cause some of the progress that has been recently made to be lost.

-Erik

On Wed, Dec 2, 2009 at 6:36 PM, Ray Racine <ray.racine@gmail.com> wrote:

Erik,
The whole stability thing was supposed to be the point of the email.
1) My primary concerns are with the shifting itself, timings of, the inputs that I perceived to be driving the stay or shift back decision in the current discussion chain, and _not_ necessarily on the final outcome of the shifting.  Of course, I readily admit learning the 2.8+1.6 Actors are now scheduled for a bit of open heart surgery on the cusp of 2.8 Beta is not what I prefer  to hear right now, but the doctors are top notch.
2) It was _not_ supposed to be a plea for 2.8 to be on 1.6.  It was an attempt to show the impact these cycling decisions have.
3) Ideally the next release plan should be clear and unambiguous up front, firm to a freeze at the appropriate time, and should not change without compelling cause.

On Wed, Dec 2, 2009 at 3:06 PM, Erik Engbrecht <erik.engbrecht@gmail.com> wrote:

Ray,
To be fair, having separate distribution for 1.5 and 1.6 was really the initial surprise.  This is really a return to the original road map for 2.8.  So I think your arguments about road map stability really support the decision.  Separate distributions would have created significant issues with down-stream libraries and tools.  Supporting 1.6 only would represent a significant departure from the published Java version support plans.  I think they've reached the right decision.

-Erik

On Wed, Dec 2, 2009 at 2:16 PM, Ray Racine <ray.racine@gmail.com> wrote:

See Below.  And my sincere appreciation for taking the time to respond.  I am far less concerned about the relative impact of the change, my concerns are more oriented around the lateness of the change in the release cycle, the means in which it was done along with, IMHO, the rather weak foundation for doing it.  I truly do not want to pain the Scala Core Team who go way beyond only to suffer emails such as mine.  But I do want to make a point.  Caution: I can get a little "over the top" so my apologies ahead of time. 
On Wed, Dec 2, 2009 at 2:25 AM, Antonio Cunei <antonio.cunei@epfl.ch> wrote:

Ray,

In short, the reasons are:

1) If Scala is distributed in two separate versions, the set of libraries
and tools built on top may easily turn into a jungle of versions that are
1.5-compatible and/or 1.6-specific. Cross-dependencies make the problem
worse: during our last Scala meeting (the notes will be published later
today) the example was made of a tool "X" that is only distributed in a
1.5 version, and a tool "Y" that needs "X", but also the features only
available with 1.6 (FJ, for instance). So, X needs the 1.5 Scala jars,
while "Y" needs the 1.6 Scala jars. Hilarity ensues.

 Agree 100%, IMHO 2.8 should be "centered" on single target JVM release.  It is certainly unfair to expect the Scala team to support 2.8 on two JVMS.   Frankly if I were you I'd keep 2.7.x for 1.5 and moved forward with 2.8 on 1.6.  End of story.  In general, I feel the Scala Core Team cannot and should not be whipsawed by a tool "X" decision.  

2) We can obtain the same practical advantages of having a 1.6-specific
distribution by packaging 1.6-specific classfiles within the main jar, and
selecting those implementations automatically at runtime. The only thing
left out is the potential speed improvement deriving from the 1.6-specific
bytecode verifier, but that is something that we can live with for the
time being. In short: we get most of the advantages of a 1.6-specific
distribution, without causing a mess.

Practical advantage is subjective.  There is a reasonable chance that some unintended consequence shall ensue for someone from this last minute shift in direction.  Someone is always placed at a practical disadvantage.  There will always be someone who wants all the advantages of and not "most of the advantages".  There will always someone who does not want to surrender "a potential xxx improvement."  Someone like me for example. :)  

3) lots of folk in industrial environments are seemingly stuck with 1.5.
We shall postpone the switch to 1.6 as the recommended default with a 2.9
version; the switch in version number may take place at whichever moment
may seem a better time for the switch. It seems to be too soon right now.

I am very much in an industrial environment myself.  If an organization has not managed to switch to 1.6 in 3 years, what are the odds they will do so in 3.5 or 4?  A couple of more points on this.  Despite  protestations to the contrary they are "stuck" by choice and shall remain so by choice.  If a company is "fine" with staying with the 1.5 JVM; they are fine with staying with Scala 2.7.x.  I will tell you I am far more sympathetic to the argument that some individual Mac user's may not be able to use Scala 1.8 than some fortune 500 company "can't" get from 1.5 to 1.6.   A false premise.
Imagine we were all watching this conversation from a distance on the Haskell, Perl, Python, etc list and late in the cycle a subset of Python 2.x users requested certain compromises be made that in some degree impair the new Python 3.x, e.g. "Please don't add those features in 3.x as that requires the new async I/O API and my shop only runs the 2.4 Kernel."  Who would be surprised if the consensus response to that subset of Python users was anything other than "we believe the asynch I/O improvements are valuable and necessary to move Python forward for many of our users.  The 2.6 kernel has been the standard kernel for all Linux distributions for over 3 years.  If you are not ready we suggest you remain on the 2.4 kernel with 2.x Python for the time being."  

These are the main reasons, analyzed during our latest meeting, that led
to the current decision. We cannot make everyone happy at the same time,
regrettably; the decision we have reached now seems to be the best
solution we can achieve at present.

Truly the crux, you cannot and never will make everyone happy at the same time.  Last minute changes on weak foundation will only compound that situation.  
I am taking the time to write this response NOT in an attempt to reverse the current decision, though I do believe it is wrong and not accomplished in the most transparent manner that it could have been, I am writing this in the hopes of convincing the community that as Scala moves to the next level what was fine back in the Scala 2.2 days will not work in the 2.9 and 3.x days.  It's worthwhile to detail a bit more of my perspective as to why I'm being a bit harsh here with the above.
My small group, in a largish company is currently using Scala right now in parallel with and plans to completely replace early next year the first of several critical components.  Let just touch on a few aspects of said components.  
  - The absolute keystone of the whole architecture is Actor based.  I really want Scala and the JVM to offer nothing less than the best they can deliver in this regard. - Directly involved with revenue generation of several billion dollars annually, millions $$ every day.  - Performance is CRITICAL.  Reliability is CRITICAL.  Hell there is nothing about any aspect about of it that is not CRITICAL.   * 10's of millions of API invocations per day.   * Client / Server times below 100ms and designed to achieve 25-30ms responses. 24/7/365 HA, no outage window,  no single point of failure.    * Gigabyte of in-memory data leveraging large 64-bit heaps,  _must_ be processed in minutes and every single constituent second  of those minutes are reviewed.   * Every bit of advantage that 1.6 JVM gives me, whether its faster byte code verification, large heap GC performance, JITing, escape analysis, task switching, I/O, thread scheduling I'll take.  I do not have the time or inclination to empirically compare and contrast the details of what I'm losing with 1.5 over 1.6.  I am sure there are some here qualified to offer a heuristic argument as to why I'm really not really losing anything, but who has hard numbers...?  And I will tell you right now I'm in a sweat trying to recall the details of the Actor memory leak issue that I seem to recall being solved with 2.8+1.6,  is that now a problem for me back on 1.5 or was that a 1.4 issue?  Should I stop, take the time to reset the whole enchilada to 1.5?   As I sit typing this I see further clarifications on the impact appearing on the list and I'm feeling a bit more "OK" with what I'm reading than I did yesterday.  Remember, my little huff here is not on the current particular of the change, but how the whole process transpired.  - This particular project from conception was to take calculated risks and push the envelope in many facets of the architecture.  Constantly I reconsider each piece of the architecture including whether Scala was the correct choice.  Not just me, others as well, and I have to defend those decisions to them as well as to myself.  As part of that defense of Scala I often resort to the heavy concurrency, multicore capabilities of Scala and those to come,  over Java.  I can't tell you how many times I ask myself or am asked by others, "you said you needed Scala for X, but can't you get that with Java and library Y, or wouldn't  Java 7 give you that capability if you'd just waited or this JavaWorld article says Java and Kilim is superior to Scala, why did you chose Scala again?   Scala _needs_ to be clearly ahead of Java, nuanced differences are not enough to overcome adoption costs.  And I personally think that the winning area for Scala is clear and unambiguous advantages over Java in Concurrency, Parallelism and Distributed capability as well as code/ing advantages.
Ok, enough of my ranting on my "problems".   Hopefully it does emphasis the point, the Scala Team is correct, you will never, ever satisfy everyone.  The key issue here is the "surprise" aspect.  A simple "Yo, do you think the new release's long announced target deployment architecture could change ..." posting on the list and 72 hours later the motion is adopted on what appear to me to be solely "convenience" motivations for a subset of stakeholders.  Certainly a little more detail with the announcement would have reduced the acid churning.   
Roadmaps cannot appear to be whimsical.  The number of Scala community users will continue to expand.   An expanding number of those will have substantive stakes in the game.  You can't make them all happy.  Don't try.   You can demonstrate a steady hand on the tiller, a clear X on the chart to steer towards and dock into port with Swiss precision to the dot of the scheduled arrival time.   Please no course changes for anything less than hurricane avoidance.  Decisions that put one out on a swaying twig of branch are made in context, based on the best information at hand at the time, weighed against risk vs reward calibrated by a wet finger in the air.  Let's state the obvious, people with skin on the line can not and will not make decisions in Scala's favor beyond certain levels of uncertainties.  
To wrap up.  Once more my intent here is NOT to rescind the recent decision.  Given the clarifications now coming out on the list, along with a couple of days to mull over what we need to do and we'll see it through on our end.   What I really like to see from this teapot tempest going forward is some degree of confidence that when the Scala Team promulgates 2.9 _will_ or 3.0 will not, I and others can plan for exactly that unless a damn good reason rears its head.  My definition of a damn good reason?  I think the current movie 2012 captures the concept quite succinctly.
Sorry to get all soap boxish and all.  The Scala Core Team are genius academics, not politicians and without question are doing a fabulous job.  For me, a dominant part of the Scala decision, to this day a correct one at that, was and is driven by who they are, their tremendous capabilities, vision and demonstrated competency of execution.

Thanks for your time.

Ray

On Wed, December 2, 2009 1:01 am, Ray Racine wrote:
> Yikes !!! I don't understand what just happened or the "whys" of it.
>
> Frankly I thought the initial plan of jvm1.6 as the default along with a
> fully support 1.5jvm build was just fine.  The compromise to stop biasing
> on
> one as default and  going with two "peer" builds, one scala-jvm1.6 and one
> scala-jvm1.5 seemed a reasonable compromise.
>
> Why the 180 degree change?  As pointed out, JDK 1.6 (Mustang) was released
> 12/12/2006, three years ago.  1.5 (Tiger) was released 9/29/2004 5 years
> ago.
>
> Martin's original position on this subject made sense, the subsequent
> Scala
> teams compromise of having 2 co-equal builds was a nice comprise.
>
> Can someone summarize the rational in say 2-3 succinct reasons or less?
>
> Running the 1st Qtr 2010 upcoming major release of Scala, version 2.8,
> will
> not use Java 1.6, the current standard JVM released 3 years ago as the
> default and instead shall resort to a dynamic loading scheme because:
>  1) ????
>  2) ????
>  3) ????
>
> Obviously if I'm asking for 3 reasons justify the change in all fairness I
> should offer up 3 reasons as to why the original direction made perfect
> sense (to me).
>
> The original approach made sense because:
>  1) If you are a 1.5 shop, use the 2.8-jvm1.5 build.  If you are a 1.6
> shop
> (that's us), use the 2.8-jvm1.6 build.
>  2) A "dynamic loading scheme" with reflection is gratuitous and fraught
> with potential unintended consequences hanging out there.  e.g.,
>        - OSGi modules may now require special handling to deal with class
> for name loading.
>        - App servers, BEA or WebSphere can be particular with dynamic
> class
> loading, where allowed from etc.
>        - JVM security model restrictions on applets and devices.
>  3) As Martin pointed out initially (paraphrased)
>   - The main advantage of the jvm6 build is better concurrency
> support through the use of the more modern FJ library.
>   - Once the final 2.8 release is out, there might already be some fast
> parallel collection classes included, which would also rely on 1.6.
>   - I want to make fast concurrency and parallelism by default.
>   - Scala must be the obvious choice for harnessing concurrency
> and parallelism.
> 4) Martin again, "The main point is we want to leapfrog Java and we won't
> be
> able to do that by clinging onto versions that Sun/Oracle itself no longer
> supports."
>
>
> With eager anticipation I await the 1,2,3 reasons as to why the latest
> decision from the Scala meeting is superior to the original direction.
>
>
> Thanks,
>
>
> Ray
>
>
> On Tue, Dec 1, 2009 at 9:56 AM, martin odersky
> <martin.odersky@epfl.ch>wrote:
>
>> We just decided in the Scala meeting to keep Java 1.5 as not only the
>> default, but the only version required by Scala 2.8. We will make use
>> of dynamic class loading and reflection to load the real FJ  framework
>> instead of its 1.5 backport or java.util.concurrent. This decision
>> holds unless some unexpected problems come up with this dynamic
>> loading scheme.
>>
>> Cheers
>>
>>  -- Martin
>>
>
>
>
> --
> The object of life is not to be on the side of the majority, but to escape
> finding oneself in the ranks of the insane. - Marcus Aurelius
>

--
The object of life is not to be on the side of the majority, but to escape finding oneself in the ranks of the insane. - Marcus Aurelius

--
http://erikengbrecht.blogspot.com/

--
The object of life is not to be on the side of the majority, but to escape finding oneself in the ranks of the insane. - Marcus Aurelius

--
http://erikengbrecht.blogspot.com/

James Matlik wrote:
> I would like to recommend allowing an explicit override Java property to
> allow users who "know what they are doing" to force one version over the
> other. Embedded decision logic may not always pick the version the user
> wants, as is shown by this more vs. less restrictive discussion.
> Intelligent defaults work well in the majority of cases, but when you
> need to deviate from them, it should be possible. A master switch (1.6
> vs. 1.5) may be useful, but if this pattern becomes used more than once,
> a feature by feature switch would also be prudent.

I certainly agree.

Cheers,
Philipp

> On Wed, Dec 2, 2009 at 6:29 AM, Philipp Haller > wrote:
>
> Johannes Rudolph wrote:
> > On Wed, Dec 2, 2009 at 10:32 AM, Philipp Haller
> > wrote:
> >> Johannes Rudolph wrote:
> >>> Only a tangent to the original issue, but anyway: does the
> backported
> >>> concurrent stuff for the jvm5-version of Scala work only with
> the Sun
> >>> JVM or with others as well? I'm asking because it contains
> references
> >>> to internal Sun APIs (Unsafe). Is anyone using actors on
> another JVM,
> >>> e.g. on Android? Are other JVMs officially supported by Scala?
> >> The JVM 1.5 compatible classes do not use any internal Sun APIs.
> >> `Unsafe` is only used in the 1.6-specific classes which are not
> loaded
> >> on a 1.5 JVM (and neither on a 1.6 IBM JVM).
> >>
> >> So, yes, other 1.5 JVMs as well as IBM's 1.6 JVM are also supported.
> > Ah, ok, so I mixed things up. Sorry, for spreading FUD. Thanks
> for the
> > clarification.
> >
> > Shouldn't the condition Lukas posted before, then be written more
> > restrictive as "Use the ForkJoinScheduler only on the Sun JVM
> (because
> > of Unsafe) with version 1.6, else use the
> > ResizableThreadPoolScheduler"?
>
> Good point.
>
> However, we also want to use FJ in at least the following cases:
> - One of the next releases of IBM's J9 will have full support for Sun's
> 1.6 `Unsafe` mechanics.
> - [Open]JDK 7 remains fully compatible with 1.6 `Unsafe`.
>
> A more restrictive condition would have to be adjusted to those cases.
>
> Cheers,
> Philipp
>
>